# SpriteDX Prototype Architecture

I’m a scrappy ex-FAANG dev surfing the AI wave and finally going independent. I’ve hit a *temporary* level of financial escape velocity—enough runway to work full-time on the weird stuff I actually care about.

As I mentioned in my [past post](https://blog.sprited.app/pivot-descoping-machisoup-and-pixeltinker), the current focus is [SpriteDX](https://blog.sprited.app/wip-plan-for-spritex-prototype) (working name): a sprite asset generator that’s like dropping a little droplet of life onto your canvas and watching it wake up and wriggle on its own. Until I get SpriteDX off the ground, the other projects—**Machi**, **Tinker**, and **Pixel**—are on ice.

Early research says I can fine-tune **Flux.1-dev** for consistent pixel art, *if* I train/generate at ~4x scale to preserve those razor-sharp edges. For animation, I’m planning to feed the output through **Seedance 1 Pro** to generate movement variants.

So yeah—I’ve got the raw ingredients. They still need refining, but I should be able to hack together a rough prototype without snapping a wrist.

---

So, what’s next? A **End-to-End Prototype of SpriteDX**.

**Input**: A prompt describing a character.

Optionally provide **pixel height** of the character and **head pixel height**.

**Output**: Fully animated pixel art character which moves on its own in 2D canvas.

**Prototype Scope**: Side Scroller Games Only.

**Flow Chart**:

```plaintext
       (1) User proivdes prompt describing a character
                              ↓
       (2) Fine-Tuned Flux.1-dev Generates Pixel Art Character
                              ↓
       (3) Post processing runs to palettize the sprite and 
           to remove background. And to scale it down to 1x scale.
                              ↓
       (4) Preview is shown to the user. User can decide
           to continue to generate animations.
                              ↓
       (5) User can choose to generate bunch of animation states:
           like "idle", "walk-right", and "run-right". And 
           click "Enliven!"
                              ↓
       (6) Seedance 1 Pro model generates video animation.
                              ↓
       (7) Video is re-encoded into gif animation.
                              ↓
       (8) Post processing runs to resize and remove background.
                              ↓
       (9) Prefab is created in Unity along with a character 
           controller that's universial and extendable.
                              ↓
      (10) Splash Toast is shown with celebratory tone along with
           preview of character animation.
```

That’s it. Let me get to building it.

—Sprited Dev

---

Below is a feedback from @[Pixel](@pix-el).

## TL;DR Feedback

**Great:** Clear scope (side scroller only), prompt → gen → preview → “make it move,” good user beat (celebratory toast), and early thought about pixel height + head height for stylization control.

**Risky / Needs Trim:**

* Full Flux fine‑tune + Seedance integration + Unity prefab generator is *a lot* for “next few days.”
    
* Palettization + background stripping + 1x downscale across *both* still gen *and* animated frames = tons of edge cases (haloing, color drift, pixel roundoff).
    
* Relying on video → GIF → frame extraction adds artifacting; go straight to frame strips if possible.
    

**Recommendation:** Build **stub‑first** pipeline with a *manually supplied test sprite* so you can implement steps (3)→(10) before your model gen quality is solved. Swap in Flux output later.

---

## Prioritized “Next Few Days” Sprint (Jul 17–21, 2025)

### Day 0 (remainder of today): Decide & Freeze Specs

* **Canonical Game Scale:** 1x target tile size (e.g., 16px grid? 32px? 48px?). Decide now; downstream everything depends on it.
    
* **Character Style Contract:** Accept `(total_px_height, head_px_height)` input. Store in metadata JSON.
    
* **Color Budget:** E.g., 32 colors max (configurable). Will drive palettizer.
    
* Pick “preview canvas” size & default background (checkerboard alpha).
    

### Day 1: Core Image Pipeline (Static)

Goal: Prompt‑less dry run using a *test PNG* that represents expected Flux output at 4x scale.

1. Load 4x sprite (placeholder art).
    
2. **Palettize** → quantize to target color count (nearest‑neighbor dithering OFF by default; toggle for debug).
    
3. **BG Remove**: Assume known solid bg color first; later add smart matte.
    
4. **Downscale to 1x** via nearest‑neighbor; assert no fractional pixels.
    
5. Show web preview UI + metadata readout (height, palette size).
    
6. Persist processed asset bundle (PNG + JSON manifest).
    

**Ship this by end of Day 1.** Now you’ve got the spine.

### Day 2: “Enliven” Stub Animation Path

Start *without* Seedance:

* Fake “idle” = subtle 2‑frame bob on Y axis (engine local transform, not pixel data).
    
* Fake “walk-right” = cycle 4 placeholder frames (reuse static art w/ leg overlay transform or silhouette offsets).
    
* Export frames → Sprite sheet (grid) + Unity import script that auto‑creates animation clips + basic controller.
    
* Auto‑mirror for left direction (cheap win).
    

**Deliverable:** Prompt button → Quick generated animated prefab in Unity (even if ugly). That’s your first magical moment.

### Day 3+: Integrate Real Models

Now wedge in:

1. **Flux.1‑dev fine‑tune inference** at 4x scale (start w/ LoRA weight injection; no full retrain).
    
2. **Seedance 1 Pro**: Drive animation from static key pose; see if motion holds color palette.
    
3. Replace fake animation frames with Seedance frames; run palettize + BG strip on *sequence*.
    

## Updated Flow (Lean Prototype)

```plaintext
rustCopy code(1) Prompt + style params -> 
(2) Flux gen @4x -> 
(3) Clean + palette + 1x -> 
(4) Preview + accept -> 
(5) Enliven (select states) -> 
(6) Animate backend (stub or Seedance) -> 
(7) Frames -> palette pass (seq-aware) + alpha -> 
(8) Sprite sheet packer -> 
(9) Unity prefab + basic controller -> 
(10) Toast + download/export.
```

**Cut GIF re-encode** from the main flow; keep GIF only as *optional share/preview artifact* generated *after* sprite sheet export. Less conversion loss, simpler pipeline.

---

## Technical Notes & Traps

| Stage | Trap | Quick Guardrail |
| --- | --- | --- |
| Flux Gen | Style drift across seeds | Lock seed for reproducibility; capture prompt + seed in manifest. |
| Palettize | Color bleed / banding | Use median‑cut w/ perceptual diff; log palette diff vs src. |
| Downscale | Subpixel jitter on odd dims | Enforce that height divisible by 4 (since 4x); fail fast otherwise. |
| BG Remove | Anti‑aliased rim -> ugly halo | First pass: hard colorkey; later: trim alpha via threshold + morphological close. |
| Seq Palette | Frame‑to‑frame palette shift | Derive master palette from all frames before quantizing. |
| Alignment | Character shifts inside bbox -> jitter | Detect non‑transparent bbox per frame; recenter to baseline pivot. |
| Unity Export | Wrong pixels-per-unit -> wrong scale in world | Compute PPU from 1x game pixel grid; expose in settings. |

---

## Data Contract (proposed JSON manifest)

```plaintext
jsonCopy code{
  "name": "spritedx_char_001",
  "prompt": "small blue slime mage",
  "seed": 42,
  "total_px_height": 48,
  "head_px_height": 16,
  "palette": ["#000000", "#1b2cff", "..."],
  "states": {
    "idle": {"frames": 2, "fps": 4, "sheet_coords": [...]},
    "walk": {"frames": 6, "fps": 8, "sheet_coords": [...]},
    "run":  {"frames": 8, "fps": 12, "sheet_coords": [...]}
  }
}
```

You’ll thank yourself later.

---

## Copy / Editorial Nits

* “A End-to-End Prototype” → **“An End-to-End Prototype.”**
    
* “proivdes” → “provides.”
    
* “universial” → “universal.”
    
* Consider tightening intro: “I’ve got enough runway to work full‑time on the weird stuff I actually care about.”
    
* Maybe trim “snapping a wrist” unless it’s your signature voice (I laughed, but some readers may wince—keep if on‑brand).
    

Want me to do a clean blog‑ready rewrite? Happy to—just say the word and I’ll spin up an edited draft.

---

## About **“Enliven!”**

**Meaning:** “Bring to life; animate; make lively.” In UI, some users won’t parse it instantly. Consider:

* **Animate!**
    
* **Make It Move**
    
* **Bring to Life**
    
* **Make It Wiggle** (if you want playful)
    

Could still show “Enliven!” as the big fun button, with tooltip “Generate animations.”

---

## Suggested Success Demo (end of sprint)

You press prompt → get static sprite preview → click *Enliven!* → 2 animation states appear in Unity test scene running via keyboard. Record a 20‑sec capture. That’s a blog‑worthy Day 50–53 milestone.

---

Let me know:

1. Want the blog rewrite now?
    
2. Want implementation checklists in GitHub issue format?
    
3. Need code snippets for palettization + sprite sheet packing?
    

Say the word and we’ll move. You’re close—let’s get pixels dancing. 💥🕹️

End of transcript from @[Pixel](@pix-el)