SpriteDX - Designing Computed Properties

What is a Computed Property?

It’s a transform on the stage inputs to provide computed version of the input.

When designing a pipeline, there are times we have to on do some transforms on set of inputs on demand.

We could transform the inputs in the ComfyUI workflow, but it becomes rather tedious. For those who can do light coding, having a scriptability is really going to expand the horizon of SpriteDX.

First Draft

Here is the first draft I wrote when designing the first SpriteDX pipeline.

        "fullPrompt": {
          "type": "string",
          "computed": [
            {
              "type": "map",
              "args": ["shots"],
              "as": "shot",
              "return": "`[SHOT ${idx}] ${shot.prompt} \n\n`"
            },
            {
              "type": "reduce",
              "args": ["computed"],
              "as": ["acc", "curr"],
              "initialValue": "`${sceneDescription}\n\n`",
              "return": "`${acc}${curr}`"
            }
          ],
          "mapTo": "2.inputs.prompt"
        }

This basically introduces a structure where we can define computed inputs. In this particular case, we are showing map, reduce and template string functionality.

However, this structure is quite limiting unless we can support lots of different commands. And supporting lots of different commands will take lots of effort.

So, I want to introduce a more generic scripting language to power computed properties.

---

QuickJS

In SpriteDX most of the data transformation will go from json into another json. Obvious scripting language is JavaScript. However, the traditional eval functionality exposes the client to very high security risk.

Warning: Executing JavaScript from a string is an enormous security risk. It is far too easy for a bad actor to run arbitrary code when you use eval(). See Never use direct eval()!, below.

The eval has access to various browser APIs that are too powerful for the SpriteDX’s script needs and we want to make sure we don’t accidentally introduce security risk to our system.

If we want SpriteDX to be enterprise level product, having safe-eval for script becomes a must-have.

So, after looking around, QuickJS-emscripten project seems promising. It effectively runs a JS VM inside wasm container.

It doesn’t have access to the caller or window object. There is no networking APIs either. It is pure scripting and run inside its own container. I think it fits our needs perfectly.

JSON Schema

Let’s then define the schema.

"inputs": {
  "fullName": {
    "type": "string",
    "computed": {
      "props": {
        "firstName": "..userInfo.firstName",
        "lastName": "..userInfo.lastName"
      },
      "script": "`${firstName} ${lastName}`"
    }
  }
}

In here, we are creating a fullName input based on two parameters firstName and lastName. The structure seems simple and clean. It seems rather clean.

---

Trust Building

However, I think this is not good enough. The structure seems good but it still does not make me feel safe. It feels like the we may be using normal eval here. How do we make the JSON scream that we are not using eval or any of the unsafe alternatives. Perhaps using names like “safeEval“ is an option.

"inputs": {
  "fullName": {
    "type": "string",
    "computed": {
      "props": {
        "firstName": "..userInfo.firstName",
        "lastName": "..userInfo.lastName"
      },
      "safeEval": "`${firstName} ${lastName}`"
    }
  }
}

Or we can mention “quick-js“ branding as an assurance.

"inputs": {
  "fullName": {
    "type": "string",
    "computed": {
      "props": {
        "firstName": "..userInfo.firstName",
        "lastName": "..userInfo.lastName"
      },
      "runner": "quick-js",
      "script": "`${firstName} ${lastName}`"
    }
  }
}

After some back and forth, we settled with:

"inputs": {
  "fullName": {
    "type": "string",
    "computed": {
      "sandbox": "quick-js", // default, but required to be specified
      "with": {
        "firstName": "..userInfo.firstName",
        "lastName": "..userInfo.lastName"
      },
      "script": "`${firstName} ${lastName}`"
    }
  }
}

Guess, this is the plan then!

— Sprited Dev 🌱