SpriteDX

We are introducing custom-trained Image-to-Image model for pixel art restoration called Anti-Corruption (AC) Model. I hear you. The name is menacing. It sounds like super intelligent AI trying to pluck out corrupt politicians. No, it’s not. It’s just Anti-pixel-art-Corruption Model. We named the process of converting real pixel arts into pseudo pixel art “corruption.“ And this “anti-corruption” model just does the opposite.

From blurry pseudo pixel art animation (corrupted):

To clean pixel art animations with transparency (anti-corrupted):

What Does It Do?

1. Artifact Correction: SpriteDX generates what we call “pseudo pixel art“ animations. The animation frames are often upscaled 4-5x with blurry edges. And simply using nearest neighbors leads to poor quality animations which are blurry.

2. Background Matting: Because the video generation models output RGB frames, we do not have any transparent alpha channel. This model recovers the transparent alpha looking at the image.

What’s Next?

Okay, the architecture is super simple and we have whole slew of ideas to make it better (especially around sub-pixel animations), but it works and Pixel says I should integrate this model ASAP instead of working on improvements. Let’s work on this today, and perhaps I can post Reddit post with this “update.”

First off we are replacing the template image’s background color from white to magenta:

https://github.com/kndlt/sprite-dx/pull/13 (private)

Uploaded the model to Hugging Face:
https://huggingface.co/sprited/sprite-dx-anti-corruption-v1

Then, we write up a Custom node for this anti-corruption model:
https://github.com/sprited-ai/sprited-comfyui-nodes/pull/1

Then updated the worker-comfyui to include this custom node:
https://hub.docker.com/layers/sprited/worker-comfyui/0.2.8/images/sha256-c296a636f522ed9098e0a80392db0d09caac507ecfaf8e4ecd71877cf9891411

Then updated the Stage 2 to do anti-corruption as part of its steps.

Then tested locally that the Stage 2 generates correct clean_animation.webp file:

Appears to be the case! 😀

One thing I notice is that when using magenta bg, the colors of characters get extra bright too. So this may be something we should note down and fix.

Resulting clean_animation.webp:

Gotta go pick up my kid. Next step is to change Stage 3 and Stage 4. Basically though, I can get rid of Stage 4 and make Stage 3 use the clean_animation.webp instead of animation.webp.

— Sprited Dev 🐛