SpriteDX - Pixel Alignment - Ideation

After the last stage of SpriteDX pipeline, we get pseudo pixel art animation that are around 2x the sizes of pixel perfect versions.

From this upscaled blurry version of the image u, we want to find a function f(u) which generates a downscaled version p that best maps the upscaled image.

Can we solve this without using ML?

I was thinking of a practical approach which only looks at local pixel groups

[ ][ ] [ ][ ] [ ][ ]
[ ][ ] [ ][ ] [ ][ ]

[ ][ ] [ ][ ] [ ][ ]
[ ][ ] [ ][ ] [ ][ ]

[ ][ ] [ ][ ] [ ][ ]
[ ][ ] [ ][ ] [ ][ ]

For each pixel, there are 9×9 subpixels to choose from. The idea is that at least one of the subpixels in this group will have the correct subpixel.

Tricky part is that since a choice that a neighboring pixel makes should impact the choice I am making.

So, I’m thinking stochastic solution over timestep to jitter the pixels and have it settle at equilibrium.

• Each pixels in 3×3 pixel grid and try to pick distinctive feature and place it at the center.

• However, there will be pushing-away force which restricts each pixels from veering too close to each other.

• After the pixel centers are located, we will use the subpixel’s color.

Sample Sprite:

Next Steps:

1. Try to come up with a sample web UI that takes in a upscaled image and tries to ovelay super pixels on top of it.

2. Then try to jitter it and see the preview results.

3. Then try to steer it towards a direction

   1. If a super pixel perfectly aligns with a consistent pixel, great.

   2. If there are some gradient, or overlap, we try to move towards less of those with small step size.