SpriteDX - Pipeline Editor Mockup
Let’s work on mocking up Pipeline Editor. I’m still jet-lagged and my brain’s not 100%, so may have lots of mistakes but please bear with me.
Initial Draft
The Pipeline Editor will have 2 panes. Input Editor Panel and the Stage Editor Panel.
On the left hand side, user puts set of input boxes.
Then on the right is the actual “process“ information with multiple stages each containing mapping information from Inputs to comfy workflow fields.
The mappings are done though visual programming of dragging an arrow from a input node to comfy workflow’s node inputs.
┌─────────────────────┬─────────┬─────────┬────────────────────────────────────┐
│[//] File Edit │Gen 1 │Pipe 1 │ + │
├────┬────────────────┴─┬───────┴─────────┴────────────────────────────────────┤
│┌──┐│Inputs │ ┌─────────────────────────┐ │
││Cr│├──────────────────┤ │ Cat Generator Pipeline │ │
│└──┘│⌄ Template │ └─────────────────────────┘ │
│┌──┐│┌────────────────┐│ ┌──────────────────────────────────────────┐ │
││Hi│││Machi/Character ││ │ 1. Comfy Workflow │ │
│└──┘│└────────────────┘│ │┌─────────────────────────┐ │ │
│ │┌────────────────┐│ ││ stage-1.workflow.json │ │ │
│ ││┌─┐ ┌─┐ ┌─┐ ┌─┐ ││ │└─────────────────────────┘ │ │
│ ││├─┤ ├─┤ ├─┤ ├─┤ ││ │ Inputs │ │
│ ││└─┘ └─┘ └─┘ └─┘ ││ │┌────────────────────────────────┐ │ │
│ │└────────────────┘│ ┌───▶│workflow["39"].inputs["prompt"] │ │ │
│ ├──────────────────┤ │ │├────────────────────────────────┤ │ │
│ │⌄ Prompt │ │┌──┼▶ workflow["39"].inputs["seed"] │ │ │
│ │┌────────────────┐│ ││ │└────────────────────────────────┘ │ │
│ ││Boy scout kid ││ ││ │ Outputs │ │
│ ││doing yoyo ├┼─┘│ │┌────────────────────────────────┐ │ │
│ │└────────────────┘│ │┌─┤│workflow["59"] │ │ │
│ │Seed [42─┼──┘│ │└────────────────────────────────┘ │ │
│ │Guidance [60]│ │ └──────────────────────────────────────────┘ │
│ ├──────────────────┤ │ ┌──────────────────────────────────────────┐ │
│ │⌄ Animation [+]│ │ │ 2. Comfy Workflow │ │
│ │ ┌────┐┌───────┐│ │ │┌─────────────────────────┐ │ │
│ │[-]│Idle││... ││ │ ││ stage-2.workflow.json │ │ │
│ │ ├────┤├───────┤│ │ │└─────────────────────────┘ │ │
│ │[-]│Run ││... ││ │ │ Inputs │ │
│ │ └────┘└───────┘│ │ │┌────────────────────────────────┐ │ │
│┌──┐├──────────────────┤ └─▶│ workflow["39"].inputs["image"] │ │ │
││Ac││Post Processing │ │└────────────────────────────────┘ │ │
│└──┘│BG Cleanup [v]│ │ │ │
└────┴──────────────────┴─────┴──────────────────────────────────────────┴─────┘
Why Two Pane Layout?
We decided to “decouple“ the UI with the actual stage information because stage may not map exactly to each sections in the Input pane.
Will we have Run button?
Instead of run button, we will have lightweight Preview mode which will be a toggle. Once switched on, left hand panel becomes the final input preview and the right side shows content preview and File panel.
Where will we have the Preview button?
Probably top right corner of the stage editor as a primary button.
How will users drag an arrow from input box to the stages?
Similar to how ComfyUI does it. Same for deletion?
How to add new input boxes on the left pane?
at the end of each section will be + button which will open a search modal with all the input types and user can create the input boxes.
Also, user will be able to create a new node by dragging from comfyUI workflows input to + sign which would automatically create the compatible typed input box with guessed names.
How to add new input/outputs in right panel?
There will be similar + button there to add these in similar ways.
What types of stages would you support?
We will only support comfy ui workflow.
What’s going to be the schema for the pipeline?
We will define that in next blog post.
— Sprited Dev
