SpriteDX

For background removal, strong worry-free alternative is to use PhotoRoom API.

Let’s do some testing.

Given:

Following sprite sheet without transparency (1024×1024):

we can run create a transparent version(1024×1024):

Let’s replace the background to see things better:

Alpha Channel:

There are some alpha bleeds.

Let’s threshold it at half way point.

Result

Let’s zoom in on them

I do see some artifacts around the edges.

Thresholding at 0.75 instead of half way point (0.5) does the trick to make it better.

Zoomed in:

Full result:

It’s not perfect.

but very good most of the time.

Using 0.8128 as thresholding criteria seems to give better result in black BG

But, now, it loses continuity of lines against white bgs.

I think it may behave little differently because I am passing it the whole spritesheet at once.

Just doing it on one frame:

seems to give much better result.

To do this per frame, cost is 0.02 per frame.

120 frames would cost 2.4 dollars.

even if I reduce it to 50 frames, it would cost a dollar.

I could run 2×2 grid at a time

Quality is slightly (can’t tell visually) reduced.

and cut the cost by4. that would be around 0.25 cents which would be competitive against our old Flux1.Kontext approach but nowhere close enough to argue against open source or custom model alternatives.

4×4 grid gives me 0.0625 cents which is rather competitive.

So what……..

I’m not sure.

I want to go back to not having to use magenta color backgrounds when inferencing at least the stage 1 character gen because it impacts the types of characters being generated.

It is subtle but:

The colors used in the magenta backgrounds are more vibrant (less pastel-ly).

With PhotoRoom API, we could go back to generating characters on white. Then apply a purple background later.

Alternatively, we could improve the anti-corruption model to be able to perform well on white backgrounds.

We also tried ReCraft (through comfy) on character sheet, quality is not the best…… There may be a way to improve it through color bleeding where for each partial transparency, we look neighboring pixels to less-transparent-pixel (or pixel group) then use their color. and dilute it further. We should try that.

I will… not change status quo now, but keep this in mind.

I want to rather improve the Anti-corruption model because it is more cost effective.

So two items:

I want to revert the template BG back to white.
I want to get more cost effective inference for matting.
After inferencing on Whites, we should change the background dynamically to a color that isn’t used much in the character. Then we animate on that color, then perform RMBG.

— Sprited Dev 🐛