SpriteDX - Representing Stage 3 in Pipeline.json

Yesterday, right before sleep, I played with SpriteDX with my daughter a bit and it was kinda rewarding to see things coming along. The generation of reference character was being done consistently and animation sequences were somewhat consistent as well. It was the kind of the magical experience I was looking for.

Today’s task is to move on to representing Stage 3 in the pipeline.json. Stage 3 is rather simple, it just receives an animated WEBP clip and outputs N shorter clips.

Requirements

Requirement is that we need to split the clips into shots defined in the animate.shots array.

[
  { "id": "greet", "prompt": "Intro: Camera static full body shot and pixel art character says “hi.”", "loop": false },
  { "id": "idle", "prompt": "Idle Loop: Camera static full body shot and the pixel art character is facing slightly right (+x) looking straight right (+x) and shows sprite animation loop for “idle” state, breathing in and breathing out, on a pure white background.", "loop": true },
  { "id": "run", "prompt": "Run Cycle (in-place, +x): Camera LOCKED on the pixel art character running in positive x direction, and character starts showing sprite animation loop for “run” state on a pure white background.", "loop": true }
]

We then assign each shot an id, so in the future we can reference the clip with a stable identifier. We will use clip_${id}.webp as file name. For example, for above clips:

• clip_greet.webp

• clip_idle.webp

• clip_run.webp

ComfyUI Workflow

Here is what we have currently.

Right now it cuts the shots and creates three shots all with the same file name prefix.

Goal is to save each of these clips with the proper file names.

To feed the the shot ids into the Split String node, we create a computed property:

"shotIds": {
  "type": "string",
  "computed": {
    "sandbox": "quickjs",
    "with": { "shots": "..animate.shots" },
    "script": "shots.map((shot) => `clip_${shot.id}`).join(',')"
  },
  "mapTo": "75.inputs.STRING"
}

---

Collecting Outputs

ComfyUI now generates 3 files in following format:

• greet_00001_.webp

• idle_00001_.webp

• run_00001_.webp

We need a way to rewrite this to more friendly url. To support a rewrite rule, let’s implement a rewrite rule structure in stage outputs.

"outputs": {
  "clips": {
    "type": "array",
    "items": { "type": "image" },
    "mapToFile": "*.webp",
    "saveAs": {
      "match": "^(.*)_(\\d+)_\\.webp$",
      "replace": "$1.webp",
      "conflict": "skip"
    }
  }
}

• mapToFile is preexisting and is a glob pattern to collect the output image files.

• Then, we have saveAs which used to be simple string value. We have expanded it to be an object defining following properties:

  • match: Regex pattern to match part of string.

  • replace: replacement string.

  • conflict: Conflict resolution options. It is either skip, version or overwrite.

---

With that Stage 3 is completed. It is generating the right clips with proper file names.

---

Next Steps

This will have to do for this week. I’m volunteering for a conference and I also need to catch up to the course work this week. So, probably best that I only start on the Stage 4 once I have a handle on other deliverables.

— Sprited Dev 🌱