# Pixelating a Sprute hero, and what the game renderer did to it

Sprute renders our heroes as 256 px, 8-direction sprite sheets. This week we tried making one of them, 0018, into real 128 px pixel art and dropping her into Materia, our voxel sandbox. The sheet part took an afternoon. Most of what we learned came from looking at her in the game, where she doesn't look like the sheet.

## Two new pixelation tools, tried on our sprites

Two open-source tools came across our feed the same day:

- **xprite-pixelate** (TypeScript + WASM) recovers the hidden pixel grid in AI-made pixel art and returns one clean pixel per cell.
- **raster_to_pixel** (Rust) turns any image into pixel art, with lots of options for how each cell picks its colour, and supports a shared palette.

Our sprites are a different problem from what either was built for. A Sprute frame is a smooth render with no hidden grid (xprite's grid detector says `reliable: false` on every frame), and it's an animation, so whatever we do to one frame has to hold still across the next. We ran both on 0018's run and idle (81 frames, all 8 directions) and on two baked Materia heroes, next to our own downscale (Lanczos + medoid + shared palette + hysteresis).

On their own, both flicker more than ours: each frame gets a fresh palette, and nothing ties a pixel to the same pixel one frame later. We added two things from our own pipeline:

1. **One palette for every motion**, built once from a montage of frames, so colours don't jump between idle and run.
2. **Hysteresis along optical flow**: follow each pixel to where it moved in the next frame, and keep its old palette colour unless a new one is better by more than RGB 24.

| method (0018, 8 clips avg) | flicker ↓ | colour error vs source ↓ |
| --- | --- | --- |
| Lanczos (no palette) | 8.9 | 4.0 |
| ours, 64 colours | 6.8 | 9.0 |
| raster_to_pixel, 64 colours | 9.0 | 8.5 |
| raster_to_pixel 128 + shared palette + hysteresis | 6.8 | 9.2 |

*Flicker* is the colour error between a frame and the next one warped back onto it with optical flow. It favours anything that uses the same flow, so we checked every result by eye too. Going from 64 to 128 colours lowers colour error a little and doesn't change flicker.

raster_to_pixel kept outlines best by eye, so that's the one we carried forward.

![Walk cycle: source, nearest, lanczos, raster_to_pixel with hysteresis and ink fix](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/9e8f321c-7288-4273-b995-c14b470f95aa.gif)

## The game changes everything

Materia draws heroes as camera-facing billboards with **bilinear filtering and no mipmaps**. On a retina screen a 128 px cell is shown at about 2.1x at every zoom level. Under bilinear magnification the differences between all those methods mostly vanish: every 128 version just looked like a blurrier 0018.

Switch the sprite to **nearest** sampling and the pixel art comes back:

![In game: bilinear vs nearest for the 256 original, nearest 128 and r2p 128](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/37d27797-11b5-4122-b3bb-5928f19fff1b.png)

That's the same lesson as Ragnarok Online: the sprite sheet and the sprite in the game are two different pictures, and the renderer decides which one players see.

## The white bleed

In the bilinear version, 0018-pixel had a pale rim around her arms. The sheet had no white pixels in it at all. The source draws its dark ink line in **semi-transparent** pixels, a downscale with binary alpha drops them, and the outermost opaque pixels end up as bright skin. Bilinear then smears that skin into the ground.

raster_to_pixel's own outline options didn't change it. What fixed it was growing the silhouette by one ring: a 128 px pixel is opaque if any of its four source pixels is, and the new and edge pixels take the darkest opaque source colour of their block.

![Edge zoom: source, no fix, edge recolour, ink fix (bilinear and nearest)](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/3694ed7c-bf8f-4358-ba72-2f14f43b57f6.png)

| | no fix | ink fix | source 256 |
| --- | --- | --- | --- |
| rim pixels >40 brighter than the source | 21.3% | **0.4%** | - |
| mean brightness of the rim | 129 | 84 | 113 |
| silhouette size | 17.6k px | 18.6k px (+6%) | - |

Almost all the effect is the alpha growth; recolouring the edge alone barely helps. The cost is an outline a little darker than the source and a silhouette about one pixel fatter.

## Three ways to draw a pixel sprite

With binary-alpha sheets, we could try three renderers on the same sprite:

![Four render modes in game](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/9d388800-a553-4029-9d14-4f3eb6f0d86c.png)

- **Bilinear + blend** (what Materia already does): soft, sits in the painted world.
- **Nearest**: the pixel art itself, but jagged at the uneven 2.1x scale.
- **Ragnarok-style** (bilinear inside, a hard alpha cut at the edge): soft colour with a crisp dark silhouette.

Jin picked bilinear + blend. That meant no renderer change at all. 0018-pixel shipped in Materia as a second variant of 0018, next to the original, as 1.2 MB of sheets against 10 MB:

![0018 and 0018-pixel in play and in the hero picker](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/36bb61c6-f24e-41b3-889d-3810166459bc.png)

## Bonus: tinting her by the ground

Next idea: should the sprite pick up the colour of the ground she stands on? Tinting the whole sprite by the background would make her skin go green on grass, and she'd change colour every time she crossed from dirt to grass. So we only tinted the lower body, strongest at the feet, with the colour of the ground under her.

A physically shaped **multiplicative** bounce (her own colour times the ground's hue) turned out to be invisible on 0018: even at 35% her navy shoes moved by 3 to 5 RGB, because dark colours barely change under a multiply. An **additive** bounce shows: at 10% her shoes pick up the grass, dirt or stone and sit in the scene, and from 20% up it starts to read as fog on her legs.

![Additive ground bounce at 0, 10, 20, 30 percent on grass, dirt and stone](https://cdn.hashnode.com/uploads/covers/682665f051e3d254b7cd5062/f5bc3850-745e-4d20-8477-b22255e09b2f.png)

That one is still a prototype.

## What's next

- A gentler ink fix, so the outline matches the source's weight instead of overshooting it.
- The sword for 0018-pixel (it needs sockets at the new cell size).
- The other heroes, if 0018-pixel holds up in play.