Based on our recent pivot, I’m working toward the SpriteDX prototype.
During my vacation, I’ve been doing some preliminary research on using Flux pipelines to build an image-to-image system for generating consistent retro-style pixel sprite sheets.
Here’s what I’ve explored so far:
TL;DR: The Flux1-Pro/Dev model, when combined with Retro Pixel LoRA and Tile ControlNet, can generate consistent character sheets through image-to-image workflows.
But… it falls short when it comes to animation-ready sprite-sheets. I think the root issue is this: the models were trained on image types you’d find on Instagram or Pinterest—images optimized for human aesthetics, not machine-readable character animation. These reference styles just aren’t structured enough for animating frame-by-frame sprites.
So, I’m experimenting with using walk cycle templates sprite sheets as ControlNet input to guide the layout better. I’m not overly optimistic, but I want to test it out before ruling it out.
Today, I installed ComfyUI and started playing around with the Flux1-Dev models inside it. Mostly just prompt tweaking for now—nothing groundbreaking to share yet.
Tomorrow, I plan to plug in IP Adapters and LoRAs (as described here) to try generating sprite sheets with walk cycles that follow a provided template. Fingers crossed.
— Sprited Dev
