Seedance 2.0 — 8-Directional Run

In previous post, we used SCAIL2 to do motion transfer on 8 directional chibi character.

This round, we will try the same with Seedance 2.0.

Configuration:

Provider: Replicate
Model: bytedance/seedance-2.0
Resolution setting: 480p, square (actual output: 640×640)
Requested duration: 5 seconds
Output: 24fps, 121 decoded frames (5.042 seconds)
Seed: 42
Audio: disabled
Inputs: one reference image and one driving video; no mask
Provider inference time: 176.68 seconds
Estimated request cost: $0.50; actual billed amount not verified
Prompt: 8 directional character sprite animation. Use [Image1] for the character appearance and pixel-art style. Use [Video1] for the running motion, timing, eight viewing directions, and 3x3 layout. Replace each mannequin with the corresponding character view from [Image1]. Keep the center cell empty, the camera fixed, and each character in its own cell facing the same direction throughout. Continue the same running cycle for the full duration. Plain gray background.

Driving Video

Kimodo-generated run retargeted to the chibi rig. The driving video is 768×768, 65 frames at 24fps, using a 25-frame source cycle at 30fps. Camera elevation is 15°. White body, gray background (#808080); the existing face texture retains a warm tint.

Had to change the color of mannequin to white in order to get past safety filter on Seedance 2.0.

Reference Image:

Result (seeds: 42, 0, 1)

It is performing pretty well. Compared to SCAIL 2 approach, it looks more natural and more sprite-like.

Replicate Links:

Comparison with SCAIL 2:

Model: wan2.1_14B_SCAIL_2_fp8_scaled.safetensors
LoRA: lightx2v_I2V_14B_480p_cfg_step_distill_rank128_bf16.safetensors
LoRA strength: 1.0

Positive prompt: "8 directional character sprite animation"
Negative prompt: ""

Steps: 8
CFG: 1
Shift: 1
Sampler: uni_pc
Scheduler: simple
Seed: 42

Resolution: 768×768
Frame rate: 24fps
Frames: 65
Duration: approximately 2.71 seconds

Driving motion: Kimodo run, retargeted to a 3D chibi rig
Camera elevation: 15°
Layout: 3×3 grid, eight directions, center empty
Inputs: reference image, driving video, and mask video

Inference time: 110.39 seconds

The SCAIL 2 is more accurate in terms of joint movements but Seedance 2 Pro better "imagines" more natural movements only referring to the driving video as reference than something set in stone.

Aspect	SCAIL 2	Seedance 2.0
Motion fidelity	Follows the driving motion more closely	Reinterprets the motion more freely
Visual impression	Looks more like the underlying 3D animation	Looks more natural and sprite-like in these examples
Arm movements	Closer to the driver’s joint movements	Sometimes invents gestures or changes the arm swing
Direction consistency	Front/back flips occurred in earlier tests	Sampled 480p results maintained the intended front/back views
Anatomical errors	Close motion transfer does not eliminate generation errors	Can draw hands over the back instead of hiding them behind the torso
Background consistency	No comparable grid artifact noted in this comparison	Some seeds produced a central square or different shades across cells
Control trade-off	Better suited to preserving a specific motion	Better suited to allowing creative interpretation
Remaining work	Validate anatomy and loop seams	Validate anatomy, motion fidelity, backgrounds, and loop seams

Discussions:

Seedance 2 gives more natural looking sprites so far.
SCAIL 2 gives more control.
It would be great if we can match the animation quality of Seedance 2 using SCAIL2 using various methods like fine-tuning or better driving animation.
Both Seedance and SCAIL 2 has shortcomings that sometimes it flips faces but they can be remedied by trying with different configurations.
Seedance 2 is around 10 cent per second of animation.
For SCAIL 2, we only pay the compute cost. I'd imagine it could be around 2-3 cents per second of animation. So may be 3-5 times cheaper.
SCAIL 2 is not widely available so it requires infrastructure setup like offering SCAIL2 in Replicate and what not.

Conclusion:

I think for the first MAGIC moment, Seedance 2.0 wins. We can cut the cost by packing multiple animations in one inference or shortening duration.
Good thing is that we can offer multiple different model options depending on what people like.
My goto still is Seedance 2.0. I think Seedance 2.0 Fast with reduced duration would be the best for this time and keep the SCAIL2 option as a local-only inference option.
Good hybrid may be that we can use Seedance 2.0 to convert 3D rendered driving motion into more sprite-y feel. Then use that as driving videos for SCAIL2 for cost reduction.
Use of closed source models like Seedance 2 goes against the sprute's open source nature though. Sprute's main idea is to have open source sprite animation generator on your desktop that you can just batch run at night and create 1000s of moving characters over night.
In that sense, we shall have Seedance 2 quality as the goal but still experiment with using the likes of SCAIL2 which can run entirely on premise.

Next Steps: