Flux.1 Control Net

Testing Setup:

• Base Model: black-forest-labs/FLUX.1-dev

• LoRA Model: prithivMLmods/Retro-Pixel-Flux-LoRA (scale=0.8)

• ControlNet: Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro

• Prompt: 8 by 8 Retro Pixel Art Character Sheet of NPC characters for pixel art game called Machi.

Tile Control Net Image:

Without control net:

With controlnet_conditioning_scale of 0.25:

With controlnet_conditioning_scale of 0.30:

From what I can tell, the control net is able to help with the layout.

With controlnet_conditioning_scale of 0.50:

Somewhere between 0.3 and 0.5, the control net becomes too strong. At that level, the image just becomes a mere copy with some dittering.

With controlnet_conditioning_scale of 0.35:

0.35 is still strong.

Now, let's keep the controlnet_conditioning_scale at 0.30, and variate the template image resolution. Above ones were done with 1024×1024 templates.

128×128 Template Image:

64×64 Template Image:

64×64 still works on providing the overall layout. There doesn’t seem to be much difference from 128×128.

32×32 Template Image:

At this resolution, the layout is still preserved.

16×16 Template Image:

At 16×16, the grid collapses and does not give much meaning.

---

Now, let’s fix the template resolution to 1024×1024 and variate zoom with nearest neighbor sampling.

• lora_scale still fixed at 0.8.

• controlnet_scale still fixed at 0.3.

2x Zoom Template:

4 by 4 Retro Pixel Art Character Sheet of NPC characters for pixel art game called Machi.

Generated:

4x Zoom Template:

2 by 2 Retro Pixel Art Character Sheet of NPC characters for pixel art game called Machi.

Generated:

When using nearest neighbor downsampling, we get the following:

It does become blurry. We probably can find a better downsampling algorithm…

But, so far from the past generations, 4x zoom seems to be most stable.

---

To be continued…