Our hero is now a live 3D model that plays like a sprite

Materia's heroes are 2D sprite sheets drawn by Sprute, our pipeline that turns one drawing into an eight-direction animated character. Sprute is powerful, but every new motion is a roll of the dice: a seed can flip a clip, a held pose like sitting can come back standing, and a small tweak means rendering again.

Earlier today we showed a monster modelled in 3D and baked into a sprite sheet. So we asked the same question of the heroes: what if a hero were a 3D model too?

The model: Tripo, then a rig

Hand-building a chibi in code never got close to the drawn look. Tripo changed that. One image in, a textured 3D model out in about two minutes, then Tripo's auto-rig puts a standard humanoid skeleton in it.

We tried a few models. One lesson came fast: the input image matters. A reference shot under studio lights gets its highlights painted into the texture, and they show up as blotches. A flat-lit model is much cleaner.

The motion: Sprute's own clips

The skeleton is a common one, so we didn't animate anything by hand. A small script retargets the same motion clips Sprute uses to drive its heroes, with Sprute's own retarget code, onto our model. The hero walks, runs, idles, jumps and lands exactly like Sprute's mannequin does.

The first walk went sideways. The idle clip stands turned about 20°, and we had measured "front" in that pose, so every motion was off by that much. Measuring front in the rig's rest pose fixed it.

The look: mostly Jin pointing at the screen

• "Nothing from the left." Tripo paints soft light into the texture from the left, which no Sprute hero has. Each colour region is now one flat colour, with one soft light from just above the camera, and no shine on skin.
• "The head is too big. Smaller. A bit smaller." We scale the head bone and keep the body as it is, so a chart shows exactly what each size does. Feet and hands got the same treatment.

• A face. Our test kid had none. Claude and Codex each tried one. Jin judged them by laying 0018's face over ours, as if hair sat on the bald head. Codex's eyes matched best, and our kid wears them, a little smaller and further apart. There are no outlines around the eyes or mouth, which was a rule from day one.

• A little Flattice. Flattice is a Blender trick for anime rigs: it presses the face flat toward the camera, so it reads drawn rather than sculpted. Our camera is already orthographic, so the outline doesn't move. What changes is the shading. We kept a light touch.

Live, but still a sprite

We started out baking the model into sprite sheets, like the monster. Every tweak meant a rebake, so we stopped and drew it live instead, keeping the sheets' rules.

The model renders offscreen into one 256-pixel cell through an orthographic camera at 22°. That cell becomes the hero's billboard texture, so size, feet, depth and shadow work exactly as they do for a sheet. It only re-renders when the picture would change: when the motion steps to its next frame at 12 fps, or when the hero turns into the next of eight directions. It moves like our sprites, and costs about as little.

What's next

It's a hidden feature for now. Hair comes next: a Tripo hair mesh on the head bone, with a few locks that swing. After that come lines drawn live, and the rest of the moves.