# SpriteDX - Stage 2 - Prompt Engineering

We did some trials and error on the Stage 2 Prompts in [previous post](https://blog.sprited.app/spritedx-animation-generation-error-rate). I want to formalize a unified format for defining multi-shot prompt that works with Seedance 1 Pro.

## XML Format

Here is the multi-shot prompt format, I’m using:

```xml
<shot 
  num="1"
  id="UNIQUE_ID"
  src="filename_that_best_describes_the_shot.gif"
  tags="keywords for the shot and overall animation"
  …other attributes… 
>
  Description of the shot
</shot>
…
```

The benefit is that (1) this works for Seedance 1 Pro multishot prompting and is in XML format and we can easily add other key value pairs.

Here is the example prompt.

```xml
<shot 
  num="1"
  id="HI" 
  src="eliana-sprite-greet-loop.gif"
  character="eliana"
  costume="1"
  camera="fixed"
  tags="white-bg sprite-anim 角色帧动画"
>
  Pixel art 2D game sprite character for the game “Machi.” 
  Her name is Eliana (brown hair, ponytail, blue dress, red shoes, red hair tie)
  Eliana (from the reference image) waves or says “hi.” 
  full-body view. Smooth looping motion.
</shot>
<shot
  num="2" 
  id="IDLE" 
  character="eliana"
  costume="1"
  camera="fixed"
  src="eliana-sprite-idle-loop.gif"
  tags="white-bg sprite-anim idle-loop 角色帧动画"
>
  Eliana stands still facing right (+x). 
  Shows an idle animation with gentle breathing — at least two full cycles. 
  Character stays fixed in place, no translation. Loop seamlessly. 
</shot>
<shot
  num="3"
  id="RUN"
  character="eliana"
  costume="1"
  camera="fixed"
  src="eliana-sprite-run-loop.gif"
  dir="right"
  tags="white-bg sprite-anim run-cycle 角色帧动画"
>
  Eliana (runs:1.6) in-place facing (right:1.6) (+x direction). 
  Feet move while her body stays fixed in the frame. 
  No scrolling, zooming, or panning. Loop smoothly. Facing right.
</shot>
```

### Why XML?

It is just a clean way to define an animation sequence. We could choose JSON or YAML but this seems to work fine and is quite readable.

And there is clear separation between “metadata“ like `camera="fixed"` and descriptions of the content.

### Example Renders

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1759980791812/84151744-1617-41ad-87b0-297e47c58406.webp align="center")

---

## Alternate Idea - XML Format + 1

Instead of defining the content in natural language can we describe them in a more structured way?

Right now, the innerText of shots contain description of what is happening in the scene.

But if we were to really mimic what HTML does with the content, what is happening should really go into `alt` attribute, and the HTML structure and hierarchy should define what goes inside the shots.

### Format

```xml
<shot 
  ...
  alt="description of the shot"
>
  <character
    name="Name of the character"
    state="character animation state"
    src="character_animation_state_gif_file_name.gif"
    style="style: attributes;"
    alt="Description of what this object/character is doing"
  >
</shot>
… 
```

### Example

```xml
<shot 
  num="1"
  id="greet" 
  camera="fixed"
  zoom="1"
  loop="true"
  style="background: white;"
  alt="Pixel art 2D game sprite character for the game 'Machi'. Character waves or says 'hi.'"
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="greet"
    dir="front"
    src="eliana-sprite-greet-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none;"
    alt="pixel art character says HI."
  />
</shot>
<shot
  num="2"
  id="idle"
  camera="fixed"
  zoom="1"
  loop="true"
  style="background: white;"
  alt="Eliana stands still facing right (+x). Shows an idle animation with gentle breathing — at least two full cycles. "
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="idle"
    dir="right"
    src="eliana-sprite-idle-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; direction: right;"
    alt="Pixel art character facing right breathes in and out."
  />
</shot>
<shot
  num="3"
  id="run"
  camera="fixed"
  zoom="1"
  loop="true"
  style="background: white;"
  alt="Eliana run in-place facing right. Shows an run loop animation — at least two full cycles."
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character
    class="ELI-93Q"
    name="Eliana"
    state="run"
    dir="right"
    src="eliana-sprite-run-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none;"
    alt="Pixel art character runs in-place for at least 3 full cycles."
  />
</shot>
```

### Example Generations

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1759982628461/6d771777-d83b-40a6-b37f-c57d3514e4a3.webp align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1759982945055/779551e4-c5fc-4d77-afb1-d96aca5a65fd.webp align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1759983011240/64ad00e1-79a4-4f7b-941d-85285e7536f8.webp align="center")

This wing thingy always comes. I guess it is just the issue with Seedance’s data distribution.

---

## Prompt Issues

Let’s iterate on above method. So far we’ve seen (sample size=38):

| **Issue Type** | **Count** | Percentage | Notes |
| --- | --- | --- | --- |
| No issue | 20 | 52.6% |  |
| Wing issue | 5 | 13.2% | Perhaps we can add a style attribute `decorations: none;` |
| Costume Change Issue | 2 | 5.3% |  |
| Missing Shot Issue | 3 | 7.9% | Perhaps we can add `duration:"1s"` field on each. |
| Run Direction Issue | 3 | 7.9% | Perhaps `dir="right"` is not explicit enough. Perhaps we make it explicit by `direction="right"`. |
| Character Dances | 2 | 5.3% | Umm… |
| Character Offset Issue | 1 | 2.6% |  |
| Zoom Issue | 1 | 2.6% |  |
| Background Issue | 1 | 2.6% |  |

### Updated Prompt

Here is the updated prompts

```xml
<shot 
  num="1"
  id="greet" 
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Pixel art 2D game sprite character for the game 'Machi'. Character waves or says 'hi.'"
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="greet"
    dir="front"
    src="eliana-sprite-greet-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none;"
    alt="pixel art character says HI."
  />
</shot>
<shot
  num="2"
  id="idle"
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Eliana stands still facing right (+x). Shows an idle animation with gentle breathing — at least two full cycles. "
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="idle"
    direction="right"
    src="eliana-sprite-idle-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none; direction: right;"
    alt="Pixel art character facing right breathes in and out."
  />
</shot>
<shot
  num="3"
  id="run"
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Eliana run in-place facing right. Shows an run loop animation — at least two full cycles."
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character
    class="ELI-93Q"
    name="Eliana"
    state="run"
    dir="right"
    src="eliana-sprite-run-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none;"
    alt="Pixel art character runs in-place for at least two full cycles."
  />
</shot>
```

### Results

Sample size: 28

| **Issue Type** | **Counts** | **Percentage** | **Notes** |
| --- | --- | --- | --- |
| No issue | 18 | 64.3% | 22% increase |
| Wing issue | 1 | 3.5% | I guess `decoration: none` is not really enough. Let’s add `wing: none;` to be more explicit. |
| Costume Change Issue | 0 | 0.0% |  |
| Missing Shot Issue | 1 | 3.5% | This was reduced significantly. 👍 |
| Run Direction Issue | 5 | 17.9% | This is still rather high. In above `direction=”right”` was only in idle character, lets add to `run`. |
| Character Dances | 0 | 0.0% |  |
| Character Offset Issue | 1 | 3.5% |  |
| Zoom Issue | 2 | 7.1% |  |
| Background Issue | 0 | 0.0% |  |

### Updated Prompt 2

```xml
<shot 
  num="1"
  id="greet" 
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Pixel art 2D game sprite character for the game 'Machi'. Character waves or says 'hi.'"
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="greet"
    direction="front"
    src="eliana-sprite-greet-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none; direction: front; wing: none;"
    alt="pixel art character says HI."
  />
</shot>
<shot
  num="2"
  id="idle"
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Eliana stands still facing right (+x). Shows an idle animation with gentle breathing — at least two full cycles. "
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character 
    class="ELI-93Q"
    name="Eliana"
    state="idle"
    direction="right"
    src="eliana-sprite-idle-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none; direction: right; wing: none;"
    alt="Pixel art character facing right breathes in and out."
  />
</shot>
<shot
  num="3"
  id="run"
  camera="fixed"
  zoom="1"
  duration="1s"
  loop="true"
  style="background: white;"
  alt="Eliana run in-place facing right. Shows an run loop animation — at least two full cycles."
  tags="pixelart spriteanim fullbody loopanim 角色帧动画"
>
  <character
    class="ELI-93Q"
    name="Eliana"
    state="run"
    direction="right"
    src="eliana-sprite-run-loop.gif"
    style="rendering: pixelart; shadow: none; effects: none; decoration: none; direction: right; wing: none; "
    alt="Pixel art character runs in-place for at least two full cycles."
  />
</shot>
```

### Results

Sample size: 21

| **Issue Type** | **Counts** | **Percentage** | **Notes** |
| --- | --- | --- | --- |
| No issue | 17 | 81.0% | Additional 26% increase, 54% increase from original base line. |
| Wing issue | 0 | 0.0% | Wing issue was reduced significantly |
| Costume Change Issue | 0 | 0.0% |  |
| Missing Shot Issue | 0 | 0.0% |  |
| Run Direction Issue | 3 | 14.3% | This is still an issue but slightly reduced. |
| Character Dances | 0 | 0.0% |  |
| Character Offset Issue | 1 | 4.8% |  |
| Zoom Issue | 0 | 0.0% |  |
| Background Issue | 0 | 0.0% |  |

---

## Conclusion

We defined a generic XML based prompt format that allows for key-value type information specification and negative prompting through `style="shadow: none"` attributes. We also defined a way to deliver a scene graph via prompt.

The benefit is that it is (1) flexible (2) extendable and can work in various scenarios and give the prompter ability to prompt composition in a scene graph format rather than natural language.

We also increased the prompt accuracy significantly via tuning those attributes resulting in 81% success rate.

---

This was rather succesful research. I will update the code with this next.

—Sprited Dev