SpriteDX - Designing the Playground
We’v successfully integrated Stage 1 through 4 into SpriteDX WebUI — a meaningful milestone. Starting from simple text prompt, we can generate a character and guide it through a pipeline that produces consistent style, scale, and animation.
This is only the beginning. Our long-term vision is larger: to see these characters come alive inside a shared virtual space — the Machi project. To prepare for that, we’re designing a Playground: an environment where generated characters can be dropped in, moved around, and tested in motion.
The Playground will serve two purposes: it will be a practical tool for previewing and refining characters, and it will also be the first glimpse of what a living world build from AI-generated characters might feel like. Clicking “Play in Playground“ in SpriteDX won’t just preview an animation — it will put your character in to a simple platformer scene, ready to move and interact.
It’s a small step towards Machi, but an important one. The Playground is where generated sprites begin to feel less like files, and more like inhabitants of a world.
Design
Okay, all that said, I’m just building a fancy preview of animated character with a very simple I/O allowing users to control the character. Here is the ASCII design mockup from before.
┌─────────────────────┬─────────┬─────────┬────────────────────────────────────┐
│[//] File Edit │Gen 1 │Gen 2 │ + [Playground] [Donwload] [Share] │
├────┬────────────────┴─┬───────┴─────────┴────────────────────────────────────┤
│┌──┐│⌄ Template │ Gen 1 > Animation > Idle ⌄ │
││Cr││┌────────────────┐│ │
│└──┘││Machi/Character ││ ┌────────┐ │
│┌──┐│└────────────────┘│ < │ Idle │ > │
││Hi││┌────────────────┐│ └────────┘ │
│└──┘││┌─┐ ┌─┐ ┌─┐ ┌─┐ ││ │
│ ││├─┤ ├─┤ ├─┤ ├─┤ │├─┐ ┌──────────────────────────────────────────────┐ ┌─┤
│ ││└─┘ └─┘ └─┘ └─┘ ││ │ │ │ │ │
│ │└────────────────┘│ │ │ │ │ │
│ ├──────────────────┤ │ │ ┌ ─ ─ ─ ─ ┐ │ │ │
│ │⌄ Prompt │ │ │ .─. │ │ │
│ │┌────────────────┐│ │ │ │ ( ) │ │ │ │
│ ││Boy scout kid ││ │ │ `┬' │ │ │
│ ││doing yoyo ││ │ │ │ ───┼─── │ │ │ │
│ │└────────────────┘│ │ │ │ │ │ │
│ │Seed [42]│ │ │ │ ┌┴┐ │ │ │ │
│ │Guidance [60]│ │ │ │ │ │ │ │
│ ├──────────────────┤ │ │ │ ─┘ └─ │ Hit │ │ │
│ │⌄ Animation [+]│ │ │ ─ ─ ─ ─ ─ box │ │ │
│ │ ┌────┐┌───────┐│ │ │ │ │ │
│ │[-]│Idle││... ││ │ │ │ │ │
│ │ ├────┤├───────┤├─┘ └──────────────────────────────────────────────┘ └─┤
│ │[-]│Run ││... ││ │
│ │ └────┘└───────┘│ ┌━━━━━━━━━━━━━━━━━━━━━━━━━━┐ │
│ ├──────────────────┤ │ ┌────┐ ┌──┐ │ │
│ │Post Processing │ │ │ |> │ │Re│ │ │
│┌──┐│BG Cleanup [v]│ │ └─▲──┘ └─▲┘ │ │
││Ac│├━━━━━━━━━━━━━━━━━━┤ └────────────┼──────────┼──┘ │
│└▲─┘│[Run] ▲ eta 2m │ │ │ │
└─┼──┴───────┼──────────┴──────────────────────────┼──────────┼────────────────┘
│ │ │ │
Acct, Progress Bar Play Regenerate
Settings
The “Playground“ button is in the Top Bar. However, after building out the UI, I feel we need the button to be somewhere else. First, user will click into a character and preview them. The call to action should be right on that Preview pane. “We should meet the user where they are.” So, I want to build “Play in Playground“ button into the Content Preview pane itself.
Let’s try this out. First iteration, we just have a ghost button with the text.
This won’t work because the content panel’s header is a floating panel, so I will need some button with a background. Let’s use little gamepad icon.
Once the button is clicked, we need to transition from the Content Preview to Playground.
Option 1: Playground as a Mode
We can have "playground" as a mode in SpriteDX which you can switch On. Just a universal switch in the UI that you can simply toggle on to see the character animated living in the screen with various states--idle, greet, and run.
This design is simple but it assumes that there is a "unified playground" for all of the generations.
Right now, SpriteDX only has 2D character generation for side scrolling platformer, thus our Playground is a side scroller simulation as well.
Now, what happens if we introduce Isometric 2D character generation, then the Playground will need to be different.
Pros:
Unified System
Low Conceptual Load
Focus on platformers
Precursor to "Machi"
Cons:
Only support Platformer
Not customizable by pipeline creator
Option 2: Playground as part of pipeline
Alternatively we can generate the playground as part of the pipeline.
So, pipeline engineer will be able to "define" what a playground is.
If you generate an isometric character, isometric playground will be bundled with your system.
This however has a problem of fragmentation. The playground no longer becomes universial unified system, it just becomes glorified sprite viewer.
Pros:
- Pipeline designers can customize their own playground.
Cons:
No unified sense of belonging
"They spread, they die"
No interoperability between playgrounds
Just Glorified Preview Engine
What to do...
Option 1 has critical flaw of inability to generalize.
Option 2 has the risk of fragmentation.
Inevitably we will probably have to do both.
Ability to control the character or have AI control the character in a confined environment.
Ability to put the character in a unified system where characters, props and building can interact.
So, I'll do both. First obvious strategy is to have a playground as a generated file that user can click on to "see it in action."
In order to not get mixed up with the true concept of "Playground" which is a shared "living" space.
I propose to call this playground-as-a-file something else.
Perhaps "play.dx" which will bring up a character in a confined environment that user cna control.
Then, later down the line, we will introduce unified simulation sandbox which is more like a 2D minecraft or terrarium that users can drop their sprites in. That, I can call it "Playground."
Option 3: Lightweight Control system.
Right now, the workflow is that user previews animated WEBP files. There is no magic moment there. The initial playground should not be a new iframe load but just a simple click that makes it feel "alive" instantly in context.
The character in the view should still stay at the same place and be a living character.
I want the character to simply "walk" out of its confinement and essentially move around, and even talk to the user.
What does this mean?
Thinking is that we still generate a file "character.json" file that describes the character. It should have bounding box information and different animation states it can exhibit. It will have paths to all animation sequences.
Then, when user clicks on "Play" or "Control" button, the content preview will seemlessly transition into play-mode where character is controllable and also moves on and about own its own using simple AI. It will also have LLM component if user desires.
This "character.json"'s schema will be SpriteDX specific schema. In my mind, I think we should name it like "character.spritedx.json" or "spritedx.json" or even "char.spritedx" which marks the branding.
Should *.spritedx file be packaged zip file?
I think that may be a good idea. We generate "char.spritedx" file which people can just drop into any game and the loader load it. It will provide a compressed and portable file that contains it all.
This however limits "editability." If you just use folder, everything will be editable.
Let's ask Pixel about this. I'm having decision paralysis.
What about generating playground.html?
It won't be called "playground." It will just be something more like "test.html" that is optional.
Pipeline Step that will generate *.spritedx file
We will call it "Packaging" step, and by default it will be turned On.
At a high level it can look like this:
┌──────────────────┐
│⌄ Packaging │
│ │
│ [v] SpriteDX │
│ [ ] Unity │
│ [ ] Gadot │
│ │
└──────────────────┘
And for the time being we will only offer SpriteDX.
What does Web UI do?
If web UI sees "*.spritedx" file, it will show a "Play" button in preview header which will bind that character to keyboard and have it move around. That's the MVP.
Sorry for unpolished post. I’m traveling to Japan right now, and have a spotty connection!
— Sprited Dev 🌱
