SpriteDX - Entity.json - Implementation Plan

For Playground to come to fruition, we need to have entity.json be automatically generated. The format looks like:

{
  /** Schema */
  "$schema": "schema/entity.schema.json",
  /** For character controller */
  "kind": "humanoid",
  /** World constrains character style and animation states */
  "world": "machi",
  /** Textures and animations */
  "textures": [{
    "meta": { "image": "char-idle-sheet.png", "size": { "w": 1024, "h": 768 }, "scale": "1" },
    "frames": { "idle_0": { "frame": { "x": 0,   "y": 0,   "w": 256, "h": 256 }, "duration": 2000 }, … },
    "animations": { "idle": ["idle_0", …] }
  }, …]
}

Components

• kind defines what character controller to use.

• world defines what scale/style of the world we are in. For instance, if you have a Ragnarok character next to MapleStory character in a scene, the scene would lose visual coherence.

• textures define TexturePacker compatible definition of animation states and its frames.

In the last post, we tried implementing it using computed properties and saveAs semantics. However, we noticed some blockers.

Blockers

Blocker #1: Gathering Stage

Our plan is to add Stage 5 which processes the results from Stage 4 which uses forEach semantics. So, each output from the Stage 4 is array.

After explaining this, it doesn’t appear to be a problem. Although, this level of cognitive overhead to understand this smell of design issues. That said, let’s do the one rational thing and ignore this. We have to proceed.

Result: Not a Blocker ✅

Blocker #2: Mapping Primitive Outputs

Right now, stage 4 only outputs spritesheet png files. However, we need to be able to output metadata like number of frames in each sprite sheet.

I did some research, and in our own version of working-comfy-runner implementation, the number of frames information does come back from the request.

// Response JSON -> output -> nodes
{
    "53": { "value": [0] },
    "55": { "value": [-512] },
    "98": { "text": ["7"] },
    "107": { "value": [1] },
    "121": { "value": [4] },
    "123": { "animated": [true] },
    "126": { "text": ["processed_greet"] },
    "132": { "text": ["sheet_greet"] },
    "134": { "text": ["21"] },
    "135": { "value": [3] },
    "143": { "text": ["7"] }
}

In this nodes payload, each node prints out some information. Node 98 in particular prints out string version of number of frames (i.e. “7”). So, we can parse this value to get number of frames.

However, we do not yet have any semantics for reading these values. So, we implemented following semantics

mapFrom Semantic

The mapForm semantics will look like this.

{
  "outputs": {
    "frameCount": {
      "type": "number",
      "mapFrom": "98.text[0]"
    }
  }
}

Where we are able to map primitive values from response to get integer, string values.

Blocker #3: Script that generates entity.json

Our goal is to generate a json file. To do this, we will need to be able to run arbitrary script in sandboxed environment. Currently, SpriteDX supports Computed Properties in inputs. So, technically it would be possible to support generation of JSON definition using that. Something like:

    "entityJson": {
      "type": "object",
      "computed": {
        "sandbox": "quickjs",
        "with": { ... },
        "src": "pipelines/character/v1/scripts/generate-entity-json.js",
        "script": "main(...)"
      },
      "saveAs": "entity.json"
    }

While this is probably possible, the purpose of Computed Property is to lightly transform the data to be fed into next parts.

It would be too much to overload computed properties for manifest file generation.

Alternative I’m thinking is that we should introduce a new stage runner type for quickjs. We can introduce something like:

{
  "id": "manifest",
  "name": "Generate Manifest",
  "type": "runner",
  "runner": "quickjs",
  "src": "pipelines/character/v1/scripts/generate-manifest.js",
  "inputs": { … },
  "outputs": { … }
}

I’m envisioning generate-manifest.js to look like:

export function run(context) {
  const { inputs } = context;
  const { shots, frameCounts, animationIds } = inputs;
  // …pack textures… 
  return {
    entityJson: {
      "$schema": "schema/entity.schema.json",
      "kind": "humanoid",
      "world": "machi",
      "textures": textures
    }
  };
}

Result: This has been implemented ✅

Conclusion

We implemented entity.json generation logic as Stage 5 by implementing quickjs runner type and saveAs semantics.

Now, given that we have a way to generate entity.json, next I want to focus on:

• Private playground using opfs urls.

• Very compelling simple elegant dim lit dark stage room. A private stage.

• It should be called “stage“ or something.

— Sprited Dev