SpriteDX - Background Removal Issues
We’ve so far worked on better shot splitter and loop detector.
The shot splitter now guarantees that we get 3 shots as outputs.
The loop detector hasn’t improved that much but we’ve verified that the current approach provides reasonable loop segments.
As a next step, I want to review the background removal logic. We’ve been seeing some issues here.
A. Picture Quality Degradation: The picture quality degrades.
The face is significantly blurrier than before.
B. Temporal Blur: There is also some artifact where we see temporally amplified blurs. No idea where it is coming from.
C. Pixel Shifts: There is sometimes pixel ghosting.
D. Cost Efficiency: Generating character reference and animation each cost 10-12 cents but the background removal takes 10 cents and we need to run it at least 3 times for the three states. So the overall this post processing step is the most expensive step. It also limits how long the sequence can be since we want to typically fit it into the context window which is about 1024 pixels by 1024 pixels.
Just listing out the issues. I will look at those this week.
—Sprited Dev 🌱
