WORKFLOW_GUIDE.md

Workflow Guide

How the catalog is organised

workflows/ holds two trees, and which one a file is in says what it is for.

workflows/templates/ teaches a pattern. One file per capability - a shape (image to video, a multi-shot cut sequence), a mechanism (shared components, sub-workflows, pipeline_reference, typed references), or a reference convention (prompt:, previous_result:). These are what to read and copy. Where several checkpoints run the same pattern through the same pipeline class, one template carries them all and its description spells out the per-checkpoint argument sets, so the variations travel with the file rather than in a document that drifts from it. The templates/ltx2/ and templates/minimax/ subfolders each hold a family whose members build on one baseline.

workflows/models/ records a hardware fact: the quantization, offloading and component placement that make one checkpoint fit a real card. That is knowledge you cannot re-derive from a template, so it is kept runnable - but nobody learns a pattern from the fifth one, so these stay out of the way. Each carries a configures naming the template it is an instance of:

{
    "id": "flux-dev",
    "description": "Text-to-image with FLUX.1 dev - the reference FLUX workflow.",
    "configures": "templates/text-to-image",
    "steps": [ ... ]
}

The distinction exists because a catalog entry that cannot say which of the two it is leaves every reader - and every agent - to guess from the filename.

Structure

Every workflow is a JSON file with an id, optional variables, and a list of steps:

{
    "id": "my_workflow",
    "variables": {
        "prompt": "default prompt text",
        "steps": 25
    },
    "steps": [ ... ]
}

Variables define defaults that can be overridden from the command line:

python -m dw.run my_workflow.json prompt="a cat" steps=50

Variable names must be alphanumeric with underscores or hyphens.

Step Types

Each step has a name and exactly one of four types:

Pipeline Steps

Run a HuggingFace Diffusers model:

{
    "name": "generate",
    "pipeline": {
        "configuration": { "component_type": "FluxPipeline" },
        "from_pretrained_arguments": {
            "model_name": "black-forest-labs/FLUX.1-dev",
            "torch_dtype": "torch.bfloat16"
        },
        "arguments": {
            "prompt": "variable:prompt",
            "num_inference_steps": 25
        }
    },
    "result": { "content_type": "image/jpeg" }
}

Pipeline Reference Steps

Re-run an already-loaded pipeline from an earlier step with a fresh set of arguments, instead of loading the model again. This is how a two-pass technique like RF-Inversion works: an invert step loads the pipeline, and a main step reuses it with the inverted latents:

{
    "name": "main",
    "pipeline_reference": {
        "reference_name": "invert",
        "arguments": {
            "prompt": "variable:prompt",
            "inverted_latents": "previous_result:invert.inverted_latents",
            "image_latents": "previous_result:invert.image_latents"
        }
    },
    "result": { "content_type": "image/jpeg" }
}

reference_name must name a step earlier in the same workflow that has a pipeline. See workflows/templates/community-pipeline.json for a full example.

Task Steps

Run utility operations (image processing, QR codes, data gathering):

{
    "name": "preprocess",
    "task": {
        "command": "canny",
        "arguments": {
            "image": { "location": "https://example.com/photo.jpg" }
        }
    },
    "result": { "content_type": "image/jpeg" }
}

A task can take inputs (a plain array) instead of arguments. Each array item becomes its own iteration, the same way multiple previous_result values do:

{
    "name": "prompts",
    "task": {
        "command": "gather_inputs",
        "inputs": ["a marmot on a bicycle", "a bug driving a cycle"]
    }
}

Workflow Steps

Invoke another workflow file:

{
    "name": "expand",
    "workflow": {
        "path": "builtin:h3_context_ir.json",
        "arguments": { "prompt": "variable:prompt" }
    },
    "result": { "content_type": "text/plain" }
}

path is read the way run_workflow's workflow_path is: a catalog name as list_workflows reports it (templates/minimax/reference-to-video), with or without .json; a path relative to the file that names it (../models/x.json); or builtin:name.json for the packaged fragments in dw/workflows/. A name resolves beside the referencing file first, then against the run's own workflows/ directory, then against each read-only source the server lists - so a stored template can be composed without copying it into the workspace. A path that lands outside every source is refused, and one that resolves nowhere is a validation error rather than a run that fails on its first step.

When the composing step declares a result, that is where the composed output is written, once: the child's own last step does not save it a second time under its own name. A composing step that declares no result (or one with no content_type) leaves the saving to the child, as before. The child's other steps write into the same run directory, with the composing step's name leading their file names.

Cross-Step Data Flow

Variable References

Reference workflow variables with variable:name:

"prompt": "variable:prompt"

A variable's declared value is both its default and its type — a value passed in is converted to the type of the default, so declaring 25 and "25" are different things (see the schema note under Variables). Declaring null opts out of that: the variable becomes optional and untyped, taking whatever it is given and staying null when it is given nothing.

"variables": { "image": null }

A value the library already declares does not need a variable at all - see Constant References.

This is how a workflow exposes an argument a caller may pass without inventing a sentinel for its absence — a sub-workflow that behaves differently when handed an image, say. A caller can only set variables the workflow declares, so an optional argument still has to be declared to be passable.

Previous Result References

Pass output from one step to another with previous_result:step_name:

{
    "steps": [
        {
            "name": "preprocess",
            "task": { "command": "canny", "arguments": { "image": { "location": "photo.jpg" } } }
        },
        {
            "name": "generate",
            "pipeline": {
                "arguments": {
                    "control_image": "previous_result:preprocess",
                    "prompt": "a painting"
                }
            }
        }
    ]
}

A reference is resolved wherever it appears in the arguments, not only at the top of them - an argument holding a list or a nested object can reference a step too, which is what lets a constructed object be built from an earlier step.

Multiple previous_result references create a cartesian product: if step A produces 4 images and step B produces 3 masks, a step referencing both will run 12 times.

A step whose result is a dict (a task returning several named outputs, or a pipeline step that returns something like inverted_latents) can be referenced property by property with previous_result:step_name.property_name:

"inverted_latents": "previous_result:invert.inverted_latents"

Media Arguments

Images and videos load automatically for arguments named image/*_image and video/*_video. Any other argument - mask, depth_map, a controlnet's second conditioning image - can load the same way with an explicit form that says what the media is instead of relying on its argument name:

"mask": { "media_type": "image", "location": "mask.png" }

media_type is "image" or "video". location is a path relative to the workflow file, or a URL, exactly like the plain image/video forms.

Authoring a workflow from an agent

For an agent that has read the catalog (list_workflows), found nothing that produces the shape it needs, and is about to write JSON. get_schema says what is well-formed; this section says what the engine does with a well-formed document, which is where a draft that validates still fails.

References

An argument value is a reference when it begins with one of these prefixes. Each resolves before the step runs. variable: and previous_result: names are checked statically, so a bad one is a validation error at the path it sits at; a constant:, asset:, prompt: or output: name in the definition body resolves only when the step runs, and one that is missing fails the run - unless it arrives in the arguments passed to validate_workflow, which checks an asset:, prompt: or output: there for existence.

• variable: — variable:name is the workflow's own variables entry, overridden by the caller's arguments. A variable declared null is optional and untyped. Schema validation runs before substitution, so a default must already be the JSON type the field expects: 25, not "25".
• previous_result: — previous_result:step_name is the outputs of an earlier step, named by that step's name. It iterates; see the cartesian rule below. Validation checks it: a literal reference naming no earlier step is an error with the JSON path it sits at, rather than a run-time failure reached after everything before it has generated. A .field suffix (previous_result:invert.inverted_latents) picks one field of a result that is a dict, or a data attribute of a result object.
• constant: — constant:module.path.NAME is a value declared in Python, read by import rather than copied into JSON. Anything callable is refused.
• asset: — asset:name is a file in the asset library. Rooted at the library and confined to it, never resolved relative to the workflow file; a path that escapes the library is rejected. upload_asset returns one of these names.
• output: — output:<workflow identity>/<run id>/<file> is a file an earlier run wrote, under the output root and confined to it. latest in the run-id position picks the newest run that holds that file. A run id is not stable against pruning: to depend on a generated file, promote it with keep_output and reference the asset: name instead.
• prompt: — prompt:name or prompt:folder/name is a stored prompt's text, rooted at the prompt library. That text may not itself begin with any of these prefixes; the engine rejects such a prompt rather than resolving twice.
• item: — only inside a step that carries for_each: item: is the entry the member was made for, item:field one field of an object entry, spliced in whole whatever its type — a string, a number, a list of references. See "One step per entry" below.
• gather: — gather:shot is the result of every member of the for_each step shot, in list order, as one list. Inside a list it splices into it. It is how a step downstream of a fan-out reads the whole group; previous_result:shot naming a for_each step is an error that says so.

After a long inline run that is worth keeping, get_job_workflow(job_id) returns the realized workflow — the definition with the arguments, seed and prompts of that run pinned into it — and save_workflow gives it a name, so the next run is by name rather than by pasting JSON again. export_job(job_id) bundles the whole run (workflow, manifest, job row, the media on both sides) into a directory on the server plus a zip URL, for a run worth committing or handing to someone else.

A reference is resolved wherever it appears in the arguments, including inside a nested object or list — not only at the top level. It is always the whole value: "variable:base_prompt" resolves, "variable:base_prompt, in fog" asks for a variable named base_prompt, in fog and fails the run. Nothing is interpolated around a reference. To vary a fixed prompt across steps, write each full prompt out, or put the shared text in a variable and let a step's argument override it whole. A variable: reference that names nothing the workflow declares is a validation error, not a warning: once a variables block exists the engine refuses an undeclared reference, so it is a run that cannot start.

When several steps share a block of text — a character's description and voice repeated in every shot of a dialogue short — the answer is composition rather than interpolation: write the shared text once as a variable and assemble each step's prompt with a compose_text task, whose parts are whole references joined in order. The shot then references the composed result ("prompt": "previous_result:shot_1_prompt"), so changing the voice changes it in every shot instead of in however many copies were made by hand.

Types and escaping

Any key ending in _type or _dtype, or named dtype, has its string value loaded as a Python object: "FluxPipeline" from diffusers, a dotted name ("torch.bfloat16", "sdnq.SDNQConfig") by full module path. Wrapping a value in braces keeps it a plain string — "{nf4}" is the string nf4. Getting this wrong fails at load time, after validation has already passed, so a value that is meant as text under one of those keys must be braced.

What a variable is allowed to be

A model's own rule about a value belongs in the workflow, not in engine code (CLAUDE.md) and not in a consumer's head. variable_constraints declares it per variable, in the same field names a chain step's frame_snap uses:

"variable_constraints": {
    "num_frames": {
        "modulus": 17,
        "remainder": 5,
        "min_frames": 124,
        "max_frames": 345,
        "snap": "up",
        "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps"
    }
}

The value has to be modulus * n + remainder within min_frames to max_frames. With snap: "up" an off-grid value is rounded to the next one the rule accepts and the run says so - 130 becomes 141, reported as a warning at validation time and again in the job's warnings; without snap it is refused. The bounds are checked against the value the run will use, so they hold for the rounded number: on the rule above 108 is accepted (it becomes 124) and 346 is refused (it would become 362).

Checked three times, for the reasons the task-argument domains are: in validation_errors, so POST /api/validate, validate_workflow and the pre-queue check all refuse a bad value at arguments.<name> or variables.<name> for free; at run time before anything loads, which is the backstop for a value the static pass cannot see (an inline workflow, a value a parent passed down); and in the catalog, where list_workflows and get_workflow(variables_only=true) report the rule beside the default - the half that stops the next caller picking a number the model refuses.

State the rule once. Where a template both declares a constraint and snaps a chain, the chain's frame_snap names it rather than repeating the numbers:

"frame_snap": "constraint:num_frames"

Two limits, both accepted. A constraint cannot express a bound that depends on another variable (a maximum that is fps * seconds where a template exposes fps), and it reaches a top-level variable only - not a field inside a list entry, so a for_each template whose entries each carry their own num_frames is unconstrained and relies on the run-time check.

A workflow takes only the keys the engine reads

The workflow object itself, step, task, workflow, pipeline_reference and result are closed: a property the engine does not read is a validation error naming the object and the key, not a warning. There is no when, no retry, no select - if a draft reaches for one, the shape it wants is a different arrangement of steps, not a flag. The error exists because a plausible invented key used to validate cleanly and then do nothing, so the expensive work ran with the input silently having had no effect - a mistyped sedd left the run unseeded while validation advised setting a seed, and a mistyped subfoldr put the deliverable at the run root rather than in final/.

pipeline is closed to the same rule with one opening: any key whose value is a component definition - an object carrying from_pretrained_arguments - names a component to load, because diffusers grows component names faster than the schema does (latent_upsampler, prompt_enhancer and processor all appear that way in shipped templates). A pipeline key that is not one of those is refused, which is what catches pipeline_type or model_name written a level too high. from_pretrained_arguments stays open - it passes its keys through to from_pretrained.

Where a workflow may read and reach

An argument that names a location is confined, untrusted (the default):

• a path must resolve inside the workflow's own directory, the asset libraries, or the output root. An absolute path elsewhere is refused whether or not it exists. The remedy is upload_asset (or keep_output) and an asset: reference - which is what those exist for.
• a glob is confined the same way, and each match re-checked.
• an http(s) URL may not resolve to an address inside the deployment - loopback, link-local, private ranges.
• remote_text_encoder.url is https-only, and only a HuggingFace host is sent this machine's token.
• model_name must be a Hub repo id, or a path inside one of those roots.

All of it is reported by validate_workflow, before anything is queued, so a draft that names a file the server may not read costs nothing to find out. get_server_info's trust_workflows says which posture is in force.

Remote code is refused by default

A server started without --trust-workflows refuses any from_pretrained_arguments that sets trust_remote_code or custom_pipeline, at load time, after validation has passed. Use a pipeline diffusers ships: no bundled catalog entry carries either key, and a workflow that does runs only on a server whose operator turned trust on, which get_server_info reports as trust_workflows.

Several previous_result references multiply

When one step carries two or more previous_result references, the engine runs that step once for every combination — a cartesian product. Four images and three masks is twelve iterations, not three pairs. Past 10000 combinations the run is refused outright.

This is deliberate: it is how one prompt fans out over a set. The consequence is that a pairing — shot i with speaker i, prompt i with portrait i — cannot be expressed with two references on one step. Write it as one step per pair, each referencing exactly the two things it pairs, or gather the pairs upstream so each is a single result. A step that seems to need a "zip" is the signal to restructure the workflow, not to add another reference.

One step per entry: for_each

A step that carries for_each runs once per entry of a list — a shot per entry of shots — and the list is a variable the caller supplies, so a six-shot episode is an argument rather than a different file.

{
  "name": "shot",
  "for_each": "variable:shots",
  "pipeline": {
    "arguments": {
      "prompt": "item:prompt",
      "references": "item:references"
    }
  }
}

with

"shots": [
  { "name": "wide_open", "prompt": "the band walks on, wide",
    "references": [{ "reference_type": "…", "from_previous_result": "draw_singer" }] },
  { "name": "closeup", "prompt": "closeup on the singer",
    "references": [{ "reference_type": "…", "from_previous_result": "draw_singer" }] }
]

and downstream

{ "name": "edit",
  "task": { "command": "concat_videos", "arguments": { "videos": "gather:shot" } } }

Before the run starts, the engine replaces the for_each step with one ordinary step per entry, named shot@wide_open, shot@closeup — the entry's name, or its index for an entry without one. Those are the names the manifest, the job's events and the gallery show, and @ is reserved for them: a hand-written step name may not contain it. An entry's name must be unique in its list and match ^[a-zA-Z_][a-zA-Z0-9_-]*$. Give entries names: the step cache keys on the member name, so a shot inserted in the middle of a named list leaves every other shot cached, while an indexed list shifts every later shot onto a different entry and regenerates it.

item:field is the whole value of that field, so an entry can carry anything a step argument can — including a references list whose length differs by shot, with from_previous_result and asset: strings inside it. Nothing is interpolated: "item:prompt" is the field, "shot: item:prompt" is a literal string.

An entry may name another variable: "from_file": "variable:character_a_voice" inside a references entry is that variable's value by the time the member exists, so one variable sets a voice in every shot the character speaks in and a caller who supplies the list still writes variable: for the parts the template fixes. Those references are resolved before anything in the entry is loaded, and an undeclared one is a validation error at the entry's path (arguments.shots[2].references[1].from_file when the list is yours, variables.shots[...] when it is the template's). A value may not reference itself, directly or through another variable.

Two for_each steps over the same list are paired by key — the entry's name, or its index for an entry without one: inside shot@closeup, a reference to another for_each step slice over the same shots list resolves to slice@closeup. That is how a shot reads the audio slice cut for it when slicing and generating are two steps. It is the one pairing the engine has; for_each runs over exactly one list, and there is no zip and no loop index.

Limits: a list has at most 32 entries, and an empty list is a validation error — the step would run nothing. Validation realizes a constant: default before checking it, so a list defaulted to a constant validates the same way it will run. release_pipeline on a for_each step releases after the last member. Each entry is a full generation, so quote the cost before running a list-driven workflow: the listing's lists block names the fields an entry takes and the steps over it, and its cost carries per_entry once one entry has been measured. validate_workflow with your arguments answers with a plan whose estimate already does that arithmetic (basis: per_entry); without per_entry it extrapolates the stored total linearly over your list (basis: derived - an estimate rather than a measurement) and reports the stored total unchanged only when your list is the one it was measured with (basis: catalog). Ahead of all of those it quotes this box's own finished runs of the shape you are about to run when it has any (basis: observed, with runs saying how many) - quote the plan's figure and say which basis it has. An entry key no step reads is a validation warning at the entry's path, so a misspelt field is caught before the run. Then validate_workflow with the arguments you will run with: it expands your list, not the template's default, resolves the variables your entries name, and reports a duplicate name or a missing field at the entry's path.

Every error carries a path in the file you wrote, not in the expanded step list: a bad reference inside a member is reported at the for_each step's own path, with the member it failed in named in the message.

templates/minimax/dialogue-short and templates/minimax/music-video are this shape: each takes one shots list, and get_workflow on either shows the entry an item needs.

The loop

1. validate_workflow — free and instant. It reports every schema error at once, each with the JSON path it sits at, plus warnings for argument names that do not appear in the real pipeline signature. It also catches a previous_result: (or from_previous_result) that names no earlier step, which is what renaming a step half way through leaves behind. Pass the arguments you are going to run with as well: a name the workflow no longer declares, a value that will not coerce to the declared type, and an asset:, prompt: or output: reference that names nothing in this workspace each come back at arguments.<name>, for free, instead of after the model has loaded. Without them the verdict is about the stored definition and its stock defaults - checked_arguments in the answer says which it was.

2. Fix everything reported, including the warnings: a passing validation does not mean the pipeline accepts the arguments, and a typo against a real __call__ shows up only as one of those warnings. The server only computes signature warnings for a schema-valid draft — while schema errors remain it returns warnings: [], so validate again after fixing them to see the warnings.

3. save_workflow — validates again on the way in and returns the catalog metadata the saved draft will carry.

4. run_workflow with acknowledged_cost=true, after telling the user what it costs. Without the acknowledgement the call is refused. The figure to tell them is the plan on the validate answer - estimate.minutes with its basis, and every downloads_required entry named as its own line item, since weights not on this box are minutes and gigabytes the cost block never counted. Then pass that plan back: acknowledged_cost={"fingerprint": plan.fingerprint, "minutes": plan.estimate.minutes, "downloads": [...]} - the server refuses with 409 if the run's shape changed since the quote, and the refusal carries the new plan to quote from. true is for a plan that was null. When basis is unknown: a workflow you wrote or copied carries no cost of its own, but the pipeline inside it usually does: list_workflows(include_models=true) finds the models/ entry that loads the same checkpoint, and its per-image figure times the number of images is the number to quote. Say "a few minutes" only when no entry with that pipeline has been measured.

5. wait_for_job rather than a polling loop; call it again if it returns still_running: true. One call blocks for at most 55 seconds whatever timeout_seconds says, so a minutes-long render takes several - the reply's timeout_capped and waited_seconds say which happened.

6. get_output_image to look at what was actually made, and say whether it answers the request. Nothing before this step establishes that it does.

7. Getting the files to the user's machine. download_output and export_job write on the machine running dw.serve, which over a remote --mcp endpoint is the GPU box. The last mile of every deliverable is the url each list_gallery entry carries (or export_job's zip_url), fetched with the same bearer token the MCP connection uses:

   curl -H "Authorization: Bearer $DW_API_TOKEN" \
        -o exports/still.png "http://<box>:8765/outputs/ltx2/Gyre/20260910-.../still.png"
   

   Put the result under exports/ in the session's working directory - it is the user's deliverable, not a temporary file.

Keeping a set consistent

"Four pictures of the same thing" is the commonest shape a request takes that the catalog does not name directly, and what "the same" means decides the workflow.

• The same style, different subjects or scenes. One prompt per picture, the same seed on the workflow, and a shared style phrase in every prompt. A shared seed does not make the pictures alike; it makes the run reproducible. Consistency here comes from the prompts.
• The same object, differing in one stated way - four spoons identical but for colour, one mug in four glazes, a product in each colourway. Generate the object once, then run an image-edit pass per variant with the base step's result as its image and an instruction that names only the change ("make the mug red"). Separate generations, seeded or not, draw a different object every time; an edit holds everything the instruction does not mention. templates/consistent-set.json is this shape.
• The same character in different situations. A reference rather than an edit: an identity-referencing pipeline or IP-Adapter conditioned on one portrait, used by every picture (templates/ip-adapter.json, templates/multi-image-reference.json, and for video the MiniMax reference-to-video and dialogue-short templates). The identity-referenced trait in the listing marks the workflows that take one.

Saying which output is the deliverable

A run writes everything into one directory, so a finished episode sits beside the twenty scratch files that went into it. A step's result block can name a subfolder of the run directory for its files:

"result": { "content_type": "video/mp4", "subfolder": "final" }

The convention is two names: final for a step whose output the user will be shown, intermediate for everything else. The engine treats no name specially and applies no default - a step that says nothing writes to the run's root as it always has - but the gallery, get_job and list_gallery all carry the value, so a consumer that follows the convention can tell the deliverable from the scratch without knowing the workflow. Mark every saving step of a multi-step workflow; a one-step workflow needs nothing.

The shipped templates follow it: every template with two or more saving steps marks each one, so a workflow copied from a template starts with the roles in place.

The value is a relative path of any depth (shots/act-1), may be a variable: or, inside a for_each step, an item: reference, and follows the output: segment rule - each segment starts with a letter, digit or underscore; .., a backslash and a leading . are refused - so every subfolder written is one a later workflow can name: output:dialogue-short/latest/final/episode.mp4. A bad value is a validation error at its JSON path. file_base_name is a name, not a path: a separator there is refused, and subfolder is the way to place a file. It replaces the name the engine would derive from the workflow and step rather than prefixing it, so "file_base_name": "episode" in a final subfolder writes final/episode-0.0.mp4 - name each step that sets one differently, or the second collides and picks up a -2.

A step that saves nothing and which no later step reads does not run at all: the engine drops it before the first step executes and warns once per dropped step. That is how a template whose portraits can be supplied as asset: files stops paying for the steps that would have drawn them. It follows from what the definition says, never from a value produced during the run, so it is decided at validate time too - the plan a validate call answers with counts only the steps that will run and lists the rest under elided_steps. Four things keep a step: a result with a content_type and save not false, being the last step, being read by a later step (previous_result:, gather:, a pipeline_reference, a shared component), or being read by a step that is itself kept - elision is transitive. If a step you meant to run is named in the warnings, a reference to it is misspelled somewhere later or it needs a result.

Composing a stored workflow

A step with a workflow block runs another workflow as one step of this one, with arguments handed down as that workflow's variables. Its path is read the way run_workflow's workflow_path is - a catalog name from list_workflows, with or without .json, a path relative to the file that names it, or builtin:name.json - and resolves beside the referencing file first, then in this workspace's workflows/, then in each read-only source the server lists. A stored template is composed by its catalog name; copying it into the workspace to reach it is no longer necessary, and a copy silently stops tracking the original.

Declare a result on the composing step and the composed output is saved there, once, under that step's name and subfolder - the composed workflow's own last step does not write a second copy. Its other steps write into the same run directory, prefixed with the composing step's name.

validate_workflow resolves the path, so a name that reaches nothing is an error at steps[N].workflow.path before anything is queued; it also validates the workflow named, refuses a composition cycle, and warns about an argument the composed workflow declares no variable for.

Being found next time

The catalog derives each entry's shape — one of image, image-set, image-edit, shot, sequence, audio, text, utility — and its traits (has-audio, chained, image-conditioned, identity-referenced, needs-input-media, composes-workflows) from the structure of the definition, and its summary from the first sentence of description.

So write that first sentence to say what the workflow makes and what it needs supplied, in under 120 characters — "H3 video with audio between two supplied stills" — rather than what technique it demonstrates. A first sentence longer than that is truncated with an ellipsis in every listing.

Declare shape, traits or summary at the top level only when derivation gets it wrong; a declaration that merely repeats the derivation is noise that rots when the rules change, and the repo's catalog tests refuse it. cost is never derived — leave it absent until a run has been measured.

cost_drivers is the other half of saying what a workflow costs, and it is for derivation: the variables that move the wall clock — a frame count, a step count, a segment count, the list a for_each runs over — never a prompt or a seed. The server buckets its own finished runs by those values and reports the result as observed beside the curated cost, so a 345-frame run never informs a 124-frame figure and a list driver buckets on its length. Declaring none is not neutral: the figure then falls back to runs that overrode nothing at all, which most real runs do, so a measured workflow with no drivers keeps answering "unknown". Each name must be a variable the workflow declares — tests/test_observed_cost.py sweeps the catalog for one that is not, since a driver bucketing on nothing looks exactly like a driver that works.

Result Configuration

"result": {
    "content_type": "image/jpeg",
    "save": true,
    "file_base_name": "episode",
    "subfolder": "final"
}

Supported content types: image/jpeg, image/png, image/webp, image/gif, video/mp4, audio/wav, audio/flac, audio/mpeg (mp3), audio/ogg, audio/opus, audio/aiff, application/json, text/plain (plus the common aliases audio/x-wav, audio/mp3, audio/vorbis).

subfolder places the step's files in a subfolder of the run directory - see Saying which output is the deliverable above. file_base_name is the base name the step's files are written under, replacing the name derived from the workflow and step; it may not contain a path separator.

For video, "fps" is the rate the file is written at. It is rarely needed: frames that know their own rate carry it - a video read from a file or an asset:, a concat_videos/dissolve_videos join, an interpolation - and the engine writes them at it. Frames that bring no rate (most generations) fall back to 8, so a workflow that assembles from bare frames should say what they run at. A declared fps always wins over the carried one and warns when the two differ, which is how a deliberate slow motion is written. For audio, add "sample_rate": 44100 when the waveform doesn't already carry a rate of its own (a declared rate always wins). Setting embed_metadata: true on an image result embeds the step's model name and arguments as generation metadata - PNG info chunks for image/png, EXIF UserComment (via piexif) for image/jpeg and image/webp.

A pipeline that generates a video with its own audio track (LTX-2, or a modular pipeline whose output asks for both videos and audio) is muxed into one video/mp4 file with PyAV. audio_sample_rate overrides the rate the pipeline itself reports, for the rare case it needs correcting.

Audio Encoding

Audio is written through soundfile, so both lossless and compressed containers work:

"result": {
    "content_type": "audio/mpeg",
    "sample_rate": 44100,
    "compression_level": 0.3
}

• subtype — encoding subtype, such as "PCM_24" for wav and flac. Defaults to the container's own default, which is "PCM_16" for wav and flac.
• compression_level — 0.0 to 1.0 for flac, mp3 and ogg. Higher means smaller files.
• bitrate_mode — "CONSTANT", "AVERAGE" or "VARIABLE" for compressed formats.

audio/opus writes an Opus stream in an ogg container, and only encodes at sample rates of 8000, 12000, 16000, 24000 or 48000.

Output files are saved as {output_dir}/{base_name}-{result_index}.{artifact_index}.{ext}, where base_name is {workflow_id}-{step_name}.{step_index} unless the step's result sets file_base_name, which replaces it entirely. step_index is the step's position in the workflow, result_index counts the argument-combination iterations the step ran (see cartesian product, above), and artifact_index counts multiple artifacts within one result (num_images_per_prompt > 1, or a dict result saved key by key). The derived name is what makes two steps' files distinct, so when you replace it on more than one step in the same subfolder, give each a different name - otherwise the second one gets a -2 counter.

Pipeline Configuration

A step's configuration is dw's own vocabulary rather than the model's — each key drives a different call — so it is a closed set: a name the schema does not declare fails validation instead of being ignored. That matters most for the keys it would otherwise be quietest about. A misspelled offload used to validate, load, and run with no offloading at all, surfacing as an out-of-memory error with nothing pointing at the spelling; it now fails before the first model loads. Model-side values that are not part of this vocabulary have blocks of their own: from_pretrained_arguments for the constructor, arguments for the call, and configs for a modular pipeline's block configs.

Memory Offloading

Control how models use memory:

"configuration": {
    "component_type": "FluxPipeline",
    "offload": "model"
}

• "model" — Moves entire models between CPU and GPU. Good balance of speed and memory.
• "sequential" — Moves individual layers. Slowest but uses least GPU memory. On MPS it is downgraded to "model" with a warning: with unified memory there is no separate pool to keep small, so the per-layer copies cost speed and save nothing. exclude_from_cpu_offload names components the sweep should leave alone.
• Omit for no offloading (fastest, requires enough VRAM).

For components the pipeline loads itself — which is all of a modular pipeline's — use components, applied once the pipeline is loaded:

"configuration": {
    "component_type": "ModularPipeline",
    "components": {
        "transformer": {
            "group_offload": {
                "offload_type": "block_level",
                "num_blocks_per_group": 1,
                "use_stream": true
            }
        },
        "text_encoder.model": {
            "group_offload": { "offload_type": "leaf_level", "use_stream": true }
        },
        "vae": { "device": "cuda", "residency": "on_demand" },
        "audio_vae": { "device": "cuda" }
    }
}

• group_offload — streams the component between system memory and the accelerator a block or a leaf module at a time, which is what fits a component larger than the device. offload_type is required ("block_level" or "leaf_level"); onload_device defaults to the pipeline's device and offload_device to the CPU. Anything else in the block is passed through to apply_group_offloading, so use_stream, num_blocks_per_group, low_cpu_mem_usage and offload_to_disk_path work as diffusers documents them.
• device — moves a component that is small enough to stay resident.
• residency — "resident" (the default) leaves the component on its device for the whole run; "on_demand" rests it in system memory and moves it to the device only while one of its own calls runs. See On-demand components.
• enable_tiling — tiled decode for a decoder not named vae (LTX-2.5's diffusion_decoder, for example).
• attention_backend — a persistent set_attention_backend on one component, which a compiled component needs (the pipeline-level attention_backend applies per call).
• attn_processor_type — the attention processor the component runs, constructed with no arguments and handed to set_attn_processor. The unet and transformer blocks cover those two; this covers any other component that carries attention (LTX-2.5's diffusion_decoder, whose default processor is a portable fallback rather than the NATTEN path the decoder was built around).
• compile, truncate_layers, remove_modules — see ACCELERATION.md.
• A dotted key reaches a module inside a component, for a component that holds the model rather than being one.
• A components block that group offloads anything, or marks anything on_demand, already keeps the pipeline itself off the device - the components are placed individually, so moving the whole pipeline would load it in full before the hooks and wrappers exist. Nothing extra is needed for that.

preserve_device_placement covers the case that is left: a component loaded already placed, which must not be moved afterwards. A device_map load or a quantization that pins its tensors to one device is the usual reason.

"transformer": {
    "configuration": {
        "component_type": "FluxTransformer2DModel",
        "preserve_device_placement": true
    },
    "from_pretrained_arguments": {
        "model_name": "black-forest-labs/FLUX.1-dev",
        "subfolder": "transformer",
        "device_map": "cuda"
    }
}

Renamed: this setting was do_not_send_to_device. The old name is no longer recognized - a workflow still using it will load the component and then move it to the device anyway, since an unknown key is ignored rather than rejected. Rename the key.

On-demand components

"residency": "on_demand" sits between the two placements above. A device component holds VRAM for the whole run, wasted on a component used twice; group offloading streams per submodule forward, so it restreams the model once per call of every leaf - ruinous for a VAE, whose tiled decode calls its blocks once per tile. On-demand moves the model as a whole around each call, so a tiling loop sits inside a single pair of transfers.

"components": {
    "vae": { "device": "cuda", "residency": "on_demand" },
    "audio_vae": { "device": "cuda", "residency": "on_demand" }
}

The component rests on the CPU and is moved to device around whichever of forward, encode and decode it defines, then moved back and the freed VRAM released to the driver. Nested calls are counted, so a decode that calls forward internally is moved once, not twice.

• Use it for a component that is large but called a handful of times - a VAE that encodes references at the start and decodes the result at the end. Freeing it for the denoise loop is the whole point.
• Not for a component called every step. A denoising transformer would pay per-call transfers 20-50 times; group offloading is the tool for those.
• Cannot be combined with group_offload on the same component - a group offloaded module holds one group at a time and ignores whole-model moves, so the two cannot both own its placement. Configuring both is rejected at load.
• Ignored when the component's device is the CPU, where there is nothing to move it off of.

On a 24GB card, templates/minimax/reference-to-video.json peaks at 18.9GiB of reserved VRAM with on-demand VAEs against 23.2GiB resident, and the tighter resident fit costs 40 allocator retries - cache flushes forced by a failed allocation - where the on-demand run has none. The headroom is also what lets the chained variant run: its later segments carry an extra reference and need ~1.9GiB more than the first.

The same holds for the frame-conditioned workflows. Generating 124 frames at 960x544 from a keyframe, with everything else held equal:

VAE placement	peak reserved	allocator retries
resident	22.71GiB	22
on-demand	18.03GiB	0

The resident run also logs a memory mapping failed with OOM warning per retry, with as little as 3MB free while it tries to map 20MB. It completes - the allocator flushes its cache and succeeds on the retry - but each one is a synchronising stall, and a run that close to the limit fails outright on any workload that needs slightly more. Every MiniMax H3 example uses on-demand VAEs for this reason.

Example: reference-to-video.json, image-to-video.json

Releasing a pipeline mid-workflow

Pipelines stay loaded for the whole run (and across REPL runs) so repeated steps reuse them. When a workflow chains two large models that cannot both fit - generate with one, upscale with another - release the first once its step completes instead of configuring offload on everything:

{
    "name": "generate",
    "release_pipeline": true,
    "pipeline": { ... }
}

The step-level release_pipeline flag unloads the step's pipeline after its results are saved. A later pipeline_reference to a released step is an error, and the REPL's cross-run cache will not retain it.

Releasing task models mid-workflow

Task models - the checkpoints behind text_generation, segment, depth_estimator and the rest - are cached separately from pipelines, so that a step running its task once per result does not reload the same weights on every iteration. Nothing evicts that cache during a run, which matters when a task loads a large model on the device ahead of a generation step: a prompt-expanding language model would hold its weights for the whole run. release_models clears it once the step completes:

{
    "name": "expand_prompt",
    "release_models": true,
    "workflow": { "path": "builtin:h3_context_ir.json", "arguments": { ... } }
}

The flag applies to any step type, and on a workflow step it fires once the whole sub-workflow has finished. It clears every cached task model, not only this step's, and a later step needing one of them reloads it.

Example: enhance-prompt.json

A step nothing reads does not run

Before the first step executes, the engine drops any step whose result no later step reads and which writes no file, and warns once per dropped step saying which and why. dialogue-short cast from portraits that already exist used to run its two Z-Image steps anyway and throw the pictures away - about a minute of GPU per episode on something nothing looked at (#122).

Four things keep a step:

• it saves - a result with a content_type, and save not false. A workflow whose whole point is writing three images references nothing, so this is the rule that keeps elision from being destructive. "save": false is how a step says it is scaffolding.
• it is the last step - it is the run's answer, whatever it declares.
• something reads it - previous_result:/from_previous_result (including previous_result:step.property), a gather: (which is a list of those by the time this runs), a pipeline_reference naming it, or a reused_components entry naming a component it shares.
• Elision is transitive, so dropping a step can drop the step it read in turn.

release_pipeline on an elided step moves onto the last surviving step before it when that step loaded the same pipeline, and release_models moves unconditionally - a release that vanished with its step would leak the memory it existed to free. The plan a validate call answers with is computed after elision, so steps, downloads_required and the cost it quotes are the work that will actually happen, and it lists what was dropped under elided_steps; the run manifest records the same list.

If a step you expected to run is named in the warnings, the usual cause is a reference to it spelled wrong somewhere later, or a step that was meant to declare a result.

VAE Options

"configuration": {
    "vae": {
        "enable_slicing": true,
        "enable_tiling": true
    }
}

• enable_slicing — Process VAE in slices to reduce memory
• enable_tiling — Tile large images through the VAE

LoRAs

Attach one or more LoRAs to a pipeline with loras, a sibling of configuration:

"loras": [
    { "model_name": "XLabs-AI/flux-RealismLora", "adapter_name": "realism", "scale": 0.8 },
    { "model_name": "user/other-lora", "weight_name": "lora.safetensors", "subfolder": "loras" }
]

• model_name — the LoRA's hub repo, required.
• weight_name / subfolder — pick a specific weights file within the repo.
• adapter_name — name passed to set_adapters(). Defaults to the LoRA's index in the list.
• scale — the adapter's weight, passed to set_adapters(). Defaults to 1.0.

See workflows/templates/lora.json for a full example.

IP-Adapter

"ip_adapter": {
    "model_name": "h94/IP-Adapter",
    "weight_name": "ip-adapter_sdxl.bin",
    "scale": 0.6
}

model_name is required; weight_name, subfolder and scale are optional. The adapter image itself is passed as a normal ip_adapter_image pipeline argument. See workflows/templates/ip-adapter.json.

Sharing Components Across Steps

Two pipeline steps that load the same underlying component (a shared text encoder, for instance) can avoid loading it twice:

"configuration": { "component_type": "FluxPipeline", "shared_components": ["text_encoder"] }

"configuration": { "component_type": "FluxPipeline", "reused_components": ["text_encoder"] }

The step naming shared_components stores those components after it loads; a later step naming the same names in reused_components gets them instead of loading its own copy. The names must match exactly between the two steps. Either list can sit in the step's configuration or beside it on the pipeline itself.

How the component reaches the second pipeline depends on what kind it is. A standard pipeline takes it as a from_pretrained argument. A modular pipeline cannot — it is built from the component specs in its own index — so it is registered with update_components() before load_components() runs, which is also what keeps load_components() from pulling a second copy: it only loads what is not already there. That is what lets two MiniMax-H3 steps of different tasks (t2va and ref2va load different transformer partitions) share the 14GB text encoder and the VAEs between them.

A reused component keeps the device placement the step that shared it gave it. Any components entry naming one is skipped with a log line rather than applied a second time — offloading hooks do not survive being installed twice, and the step that loaded the component is the one that decided how it is placed.

Sharing outlives the pipeline that did it: a step can share a component and still set release_pipeline, which frees everything else it loaded while the shared component stays alive for the steps that reuse it.

Attention and Performance

"configuration": {
    "component_type": "FluxPipeline",
    "attention_backend": "flash_hub",
    "enable_attention_slicing": true,
    "no_generator": false
}

• enable_attention_slicing — process attention in slices to reduce memory. Enabled automatically on MPS unless disable_attention_slicing is set.
• attention_backend — selects a diffusers attention backend (e.g. "flash_hub") for the duration of each pipeline call.
• prompt_weighting — enables A1111-style prompt weighting ((word:1.5), [word], ((word))) and prompts over 77 tokens. Currently supports Flux pipelines. Mutually exclusive with remote_text_encoder.
• no_generator — set true to skip creating a torch.Generator for pipelines that don't accept one.
• inversion — run the pipeline's invert() instead of the pipeline itself; the step returns the inverted/image latents for a later step to consume (see community-pipeline.json).
• generate — run the pipeline's generate() instead, for components with a generation head (the step returns generated_ids).

Cache Acceleration

Two mutually exclusive ways to speed up inference by skipping redundant computation:

"configuration": {
    "cache": { "type": "first_block", "threshold": 0.05 }
}

cache wraps diffusers' own cache hooks - type is one of first_block, faster, mag, taylorseer or text_kv, each with its own tuning fields (threshold, num_inference_steps, max_skip_steps, retention_ratio, cache_interval, max_order, mag_ratios, calibrate — see dw/workflow_schema.json for which fields apply to which type). See workflows/templates/step-caching.json.

"configuration": {
    "teacache": { "rel_l1_thresh": 0.4 }
}

teacache enables TeaCache, currently for Flux transformers, and requires num_inference_steps among the pipeline's arguments.

Device and Dtype

Device is auto-detected (CUDA > MPS > CPU). Dtype is set per-component:

"from_pretrained_arguments": {
    "model_name": "black-forest-labs/FLUX.1-dev",
    "torch_dtype": "torch.bfloat16"
}

A step can name a device instead, in a pipeline configuration (which becomes the default for that pipeline's components), in a component configuration, or in a task's arguments. A device naming a backend the machine running the workflow does not have is translated to the one it does, with a warning, so a workflow written on a CUDA box runs on a Mac and back again:

"configuration": {
    "component_type": "FluxPipeline",
    "device": "cuda"
}

Only the backend is translated. A device index survives when the backend matches, so cuda:1 on a single-GPU CUDA box remains the error it always was; when the backend does not match, the index is dropped and the warning says so — a workflow that meant to spread work across two accelerators will not on a machine that has one. "device": "cpu" is never translated, since pinning a step to the CPU is how a GPU-specific problem gets ruled out.

Modular Pipelines

Modular pipelines (ModularPipeline and its subclasses) load their configuration and their component weights separately, so from_pretrained_arguments only names the model and load_components pulls the weights:

"configuration": {
    "component_type": "MiniMaxMusic3ModularPipeline",
    "load_components": { "dtype": "torch.bfloat16" },
    "components_manager": { "enable_auto_cpu_offload": true }
}

• load_components — arguments for load_components(). Use dtype for the component dtype and names to load only some of the components. quantization_config is keyed by component name, since a modular pipeline loads each component itself:

  "load_components": {
      "dtype": "torch.bfloat16",
      "quantization_config": {
          "transformer": {
              "configuration": { "config_type": "TorchAoConfig" },
              "arguments": {
                  "quant_type": "torchao.quantization.Int8WeightOnlyConfig",
                  "modules_to_not_convert": ["proj_in", "proj_out"]
              }
          },
          "language_model": {
              "configuration": { "config_type": "transformers.TorchAoConfig" },
              "arguments": { "quant_type": "torchao.quantization.Int8WeightOnlyConfig" }
          }
      }
  }

  A component the map does not name loads unquantized. Note which TorchAoConfig each component takes: the diffusers one for its own models, the transformers one for a transformers model such as a conditioner.

• configs — values the pipeline's blocks declare and read while they run. They are neither components nor call arguments, which is why they have a block of their own:

  "configs": {
      "canvas_short_edge": 768,
      "reference_image_short_edge": 1024
  }

  The names are whatever the pipeline itself declares, so they differ per model rather than being a fixed list here — MiniMax-H3 declares canvas_short_edge (768), canvas_max_pixels (1032192) and reference_image_short_edge (2048), the last being the resolution its image references are encoded at. A name the pipeline does not declare raises rather than passing quietly, since a dropped config reads as a setting that did nothing.

• components_manager — attaches a ComponentsManager, which tracks the pipeline's components. With enable_auto_cpu_offload it keeps only the running components on the device and moves the rest to system memory, reserving memory_reserve_margin (default "3GB") of free device memory. It requires a device that reports free memory (CUDA) and replaces offload, which modular pipelines do not support.

A modular pipeline returns whatever its output argument asks for — one output by name, or several of them together:

"arguments": {
    "prompt": "variable:prompt",
    "output": ["videos", "audio", "sampling_rate"]
}

Asked for several, the outputs come back keyed by name. Video generated with its own soundtrack is muxed into a single video/mp4 file, the same way a video pipeline's own output is, and a later step can still reference any of the outputs by name.

Some repositories hold more than one task's weights. workflow names the task, which prunes the pipeline to the blocks that task runs, so only the components it needs are downloaded and loaded:

"from_pretrained_arguments": {
    "model_name": "MiniMaxAI/MiniMax-H3",
    "workflow": "t2va"
}

A task is chosen by the arguments the step passes, so one workflow name can cover more than one of them: MiniMax-H3's fl2va takes an image, a last_image, or both. Given only a last_image it generates up to that frame, inventing everything that leads to it — see workflows/templates/minimax/last-frame-only.json beside workflows/templates/minimax/first-and-last-frame.json.

See workflows/templates/minimax/music.json and workflows/templates/minimax/video-with-audio.json for full examples.

Chained Video Generation

Video pipelines generate short clips - a chain block on a pipeline step runs the pipeline once per segment and stitches the segments into one long video. The model loads once; each segment's last frame is carried into the next segment as its keyframe, the duplicated boundary frames are trimmed, and frames and audio are joined into a single file:

"pipeline": {
    "configuration": { "component_type": "LTX2ImageToVideoPipeline" },
    "from_pretrained_arguments": { "model_name": "Lightricks/LTX-2.5-Diffusers" },
    "chain": {
        "segments": 3,
        "trim_frames": 2,
        "crossfade_ms": 80
    },
    "arguments": { "prompt": "variable:prompt", "image": "variable:image" }
}

• segments — how many times the pipeline runs. Total length is roughly segments * num_frames, minus trim_frames per seam.
• match_audio — instead of a count, derive the length from the audio reference in the step's arguments. The audio is sliced into frame-aligned per-segment chunks, each segment is generated against its slice, and the final video is muxed with the original, unsliced track - so the soundtrack has no seams at all. Requires num_frames (the per-segment length) and a frame rate. Exactly one of segments or match_audio must be given.
• continuity — how continuity carries across segments. last_frame (the default) extracts each segment's last frame and passes it to the next segment - single-frame conditioning, which carries pose and colour. last_segment carries the previous segment itself (frames and its generated soundtrack) into the next as a video reference, which also carries motion, camera, and voice across the seam; it requires a segment_argument that takes a references list.
• carry_frames — with last_segment, bound the carry to the last N frames of the segment (the audio is cut to the same span). Unset carries the whole segment.
• carry_audio — with last_segment, whether the carried reference includes its soundtrack (default true).
• segment_argument — where the carried frame or reference lands: image (default) for image-to-video pipelines, or references for reference-conditioned modular pipelines, where it is appended alongside the workflow's own.
• trim_frames — image-to-video pipelines reproduce their keyframe as frame 0, so this many frames are dropped from the head of every segment after the first (default 1). The matching audio is used as crossfade material, so video and audio stay exactly in sync. It also bounds the crossfade window: trim_frames / fps seconds (at 24 fps, trim_frames: 2 allows the full default 75 ms fade).
• crossfade_ms — equal-power crossfade applied to generated audio at each seam (default 75). Not used with match_audio, which keeps the original track.
• fps — frame rate for the chain's audio math. Defaults to the pipeline's frame_rate argument; pipelines with a fixed rate need it set (MiniMax H3: 24).
• frame_snap — the constraint the pipeline puts on num_frames, used to snap the final match_audio segment to a valid length. MiniMax H3 accepts 17n+5 frames between 124 and 345: { "modulus": 17, "remainder": 5, "min_frames": 124, "max_frames": 345 }. Where the workflow already declares that rule as a variable_constraints entry, write "frame_snap": "constraint:num_frames" instead, so the numbers live in one place (What a variable is allowed to be).
• prompts — optional per-segment prompt list for narrative progression; segment i uses prompts[min(i, len - 1)].
• save_segments — write each completed segment to the output directory as a playable mp4 and free its frames, bounding memory to roughly one segment regardless of chain length. The final video is streamed from the segment files at save time, and they are removed once it is written (keep_segments: true retains them). A crashed chain leaves the finished segments behind - stitch them by hand by listing their paths in a concat_videos step (trim_frames: 0, the trim was already applied). Requires PyAV and a frame rate. The trade-off is one extra encode/decode cycle through h264 for the segment files.

The chain runs inside one iteration of the step, so it composes with previous_result fan-out (three keyframes in, three chained videos out), and a pipeline_reference step can carry its own chain. Seeds behave like a normal run: the step's generator advances across segments, so one seed reproduces the whole chain. Expect some visual drift across many segments with last_frame continuity - it is single-frame conditioning; last_segment continuity exists for exactly that, where the pipeline can take a video reference.

See workflows/templates/ltx2/chained-segments.json, workflows/templates/minimax/chained-segments.json, and workflows/templates/minimax/chain-matched-to-audio.json (audio-matched lip-sync of arbitrary length).

Schedulers

Override the default scheduler:

"scheduler": {
    "configuration": {
        "scheduler_type": "DPMSolverMultistepScheduler"
    },
    "from_config_args": {
        "use_karras_sigmas": true
    }
}

A scheduler block may also carry shift, the exponential sigma shift for schedulers that take one (MiniMax H3's released checkpoint: 12.0 for video, 3.0 for audio). A pipeline that carries a second scheduler takes an audio_scheduler block with the same shape - MiniMax H3 steps video and audio latents down two schedules whose shifts are set independently.

Seeds

Set a seed for reproducibility at workflow, step, or pipeline level - most specific wins: a pipeline's own seed overrides its step's, which overrides the workflow's:

{
    "id": "my_workflow",
    "seed": 42,
    "steps": [
        { "name": "step1", "seed": 123, "pipeline": { "seed": 7, ... } }
    ]
}

Omit seed entirely to let the workflow draw a random one at run time. The seed a run actually used - drawn or named - is recorded in its manifest.json, so a run you liked can be reproduced after the fact.

Beside that manifest the run also writes workflow.json — the realized workflow, meaning the one that actually ran. Every mutable input is pinned into it: the caller's arguments folded into the variables defaults, the seed the run used, each prompt: reference replaced by the stored text, and each output:<identity>/latest/<file> rewritten to the run id it resolved to. asset:, constant:, previous_result: and builtin: are kept as written — each already names something pinned by the asset library or by the manifest's dw_version — and a sub-workflow named by local path is kept with its file's SHA-256 recorded in the manifest. The manifest also lists which stored prompts were inlined, since inlining loses the name.

The file is a valid workflow, and running it again is python -m dw.run workflow.json or handing its contents to run_workflow as inline_workflow — but either way the asset: and output: names in it resolve against the server's or CLI's own libraries, not against the run directory, so doing this from inside that directory reproduces the run only when its libraries are the ones the original run used too. Writing the file is best effort, exactly like the manifest — a run that produced its files has succeeded either way — and --output-layout flat writes no run directory, so it writes neither file.

Any of the three levels accepts a variable: reference, which is how a seed becomes settable per run without editing the file:

{
    "variables": { "seed": 42 },
    "seed": "variable:seed",
    "steps": [ ... ]
}

python -m dw.run workflows/models/z-image.json seed=1234

Declare the variable with an integer default, as above: the value from the command line arrives as a string and is converted to the declared type. A string that is not a variable: reference is rejected by the schema.

The seed also reaches sub-workflows: a delegated workflow step runs the child under the parent's seed unless the child names its own. Without that a child draws its own random seed, and a workflow whose real generation happens inside a sub-workflow would not reproduce from the seed it was given.

Type System

Dynamic type conversion applies to certain values:

• Keys ending in _type or _dtype, or named dtype: "torch.bfloat16" becomes torch.bfloat16
• Dotted names: "sdnq.SDNQConfig" loads the class via importlib
• Escape with braces to keep as string: "{nf4}" stays as "nf4"
• content_type and offload_type are exempt even though they end in _type - they name a category, not a Python type, so their value always stays a plain string (the {} escape is accepted but not required for these two keys)
• Values prefixed with constant: are read from python rather than copied into the workflow: "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES"

Constant References

Some arguments have a value the library already declares: the sigma schedule a distilled model was trained on, the negative prompt a model family ships. Reference it with constant: and its dotted python name instead of copying it into the workflow:

"sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES",
"negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT"

The leading part of the name that imports is the module, and the rest is read from it - so a constant held in a config object is reachable too:

"prompt_max_new_tokens": "constant:diffusers.pipelines.ltx2.utils.GEMMA4_PROMPT_ENHANCEMENT_CONFIG.max_new_tokens"

A reference resolves anywhere in a workflow's arguments, including in a variables default, where it becomes the value a caller overrides - and its type, since a variable is declared by its default:

"variables": { "negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT" }

A constant is data. The name has to resolve to a value - anything callable is refused, because a type is named with a *_type argument and constructed there, and reaching a function this way would be evaluating python rather than referencing it. Mutable values are copied, so a pipeline that consumes its schedule in place cannot edit the library's constant for the rest of the session.

The value the library declares is the value the workflow gets, which is the point: a constant that changes upstream changes here, and one that is renamed or moved fails loudly rather than leaving a stale copy behind.

Prompt References

A prompt worth keeping is worth keeping once. Stored prompts live as JSON files in a prompt library - the prompts/ folder by default - and a workflow argument written as prompt: plus the file's name (without .json, optionally one folder deep) loads its text at run time:

"prompt": "prompt:scenic_landscape",
"prompt": "prompt:minimax/fox_dawn_t2va"

A prompt file holds the text plus the metadata the server's Prompts page shows:

{
  "text": "A sweeping alpine valley at golden hour...",
  "description": "General-purpose scenic landscape",
  "intended_model": "z-image",
  "negative_prompt": "blurry, low quality",
  "tags": ["landscape", "golden-hour"]
}

Only text is required, and it is what the reference resolves to. intended_model is informational - the engine ignores it, but the library badges and filters by it, and the server's prompt enhancer uses it to preselect a preset.

The library's location is resolved in order: the DW_PROMPT_DIR environment variable (which --prompt-dir on both dw.run and dw.serve sets), then ./prompts in the working directory when it exists, then the first prompts/ folder found walking up from the workflow file's own directory - which is how a repo workflow run from any working directory still reaches the library beside it. dw.serve resolves the directory once at startup with this same order (anchored at its workflow directory) and pins it for every job, so the Prompts page and prompt: resolution always agree on one library. References are rooted at that one directory - not at the workflow file - so the same reference means the same text from every workflow. Like constant:, a reference resolves anywhere in a workflow's arguments, including a variables default, and it always resolves to exactly one string: it never multiplies a step's iterations the way previous_result: references do. A prompt's text may not itself begin with a reference prefix such as variable: - the engine refuses it rather than resolving text as syntax.

Asset References

A workflow's plain media paths resolve against the workflow file's own directory, which means a workflow that reads anything has to keep that thing beside it. An asset: reference is rooted at the asset library instead - the workspace's assets/ folder - so a workflow and the media it reads do not have to live in the same place:

"image": "asset:iris.png",
"video": "asset:gyre/frames/web.mp4",
"references": [
    {
        "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3ImageReference",
        "from_file": "asset:subject.png"
    }
]

A reference names a file with its extension, at most four folders deep, and resolves to that file's path - so it works under any argument that accepts a path: image, video, a from_file, a list of any of them, or a task argument that names a file. What loads the path is unchanged; only where the path comes from is.

The library's location is resolved in order: the DW_ASSET_DIR environment variable (which --asset-dir on both dw.run and dw.serve sets), then the workspace's assets/ when a workspace was named explicitly, then ./assets in the working directory when it exists, then the first assets/ folder found walking up from the workflow file's own directory. See Workspaces.

A reference can only name a file inside the library: .., an absolute path, or a symlink pointing out of it are all refused. Browser uploads land in the library's uploads/ folder and come back as asset:uploads/<name>, so a workflow saved after an upload still resolves on the next run.

Output References

Multi-stage work — generate stills, then animate them; generate a score, then mux it — used to mean copying files out of the output directory and back in beside the next workflow. An output: reference names what an earlier run wrote, directly:

"image": "output:ltx2/Gyre/latest/Gyre-still.0-0.0.png",
"audio": "output:ltx2/GyreScore/20260905-181530-a1b2c3d4/Gyre-score.10-0.0.wav"

The name is a path under the output directory — the workflow's identity, the run, and the file (see Runs). Writing latest where the run id goes resolves to the newest run of that workflow that holds the file, which is what lets a second-stage workflow name the first stage's product without being edited after every run - and keeps working when the newest run failed part way, or reused every step from the cache and so wrote nothing of its own but a manifest. Runs sort by their id, which starts with a UTC timestamp, so "newest" needs no file timestamps and survives a directory being copied. latest only selects a run where run directories are; a workflow or file that happens to be called latest is still named as itself.

Like asset:, a reference resolves to a path and then whatever loads paths loads it, so it works under image, video, a from_file, or a list of them. The audio tasks take a video file's path too and use the soundtrack muxed into it, which is how a finished cut is scored in a later run without re-cutting it. It resolves against the output directory the run was told to write to, and cannot leave it: .., an absolute path, and a symlink pointing out are all refused.

To name an earlier step of the same run, use previous_result: instead — that passes the value in memory rather than through the filesystem.

A generated file worth reusing repeatedly is better kept than referenced by the run that made it: POST /api/assets/keep (the gallery's Keep as asset, or MCP's keep_output) copies it into the workspace's asset library under a name you choose, and from then on it is an asset: reference like any other — stable whatever happens to the run directory it came from.

Objects Built From a File

Some pipelines take arguments that are objects rather than plain media. An argument that names a type and a from_file is constructed by that type's own from_file():

"references": [
    {
        "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3ImageReference",
        "from_file": "subject.png"
    },
    {
        "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3AudioReference",
        "from_file": "voice.wav"
    }
]

Loading the media this way rather than as a plain image or video argument is what brings its frame rate or sample rate along with it, which MiniMax-H3 resamples a reference from. The file may be a path — relative to the workflow file, like all media a workflow names — or a URL, and is validated like any other media. variable: references work as the file location; previous_result: does not, since the object is built when the workflow loads — use from_previous_result for that. A dict that merely contains a from_file key without a *_type key is not an object description and is passed through untouched.

An entry in a list whose source is null is left out of that list. That is what makes a reference optional: write it as an ordinary entry whose from_file (or from_previous_result) is a variable, declare the variable null, and a run that is given nothing for it generates exactly as it did before the reference existed — one workflow serving both, instead of two spellings of the same steps. It applies to from_file, from_previous_result and from_arguments alike. On its own rather than in a list there is nothing to leave it out of, so a null source there is an error.

Any other key goes wherever the type can take it: to from_file() where its signature names it, and onto the object it returns where it does not. That is what corrects a decoded file, which is the only thing that knows what the container claimed:

{
    "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3VideoReference",
    "from_file": "motion.mp4",
    "fps": 30.0,
    "audio": null
}

fps overrides a rate the container got wrong — MiniMax-H3 resamples a reference onto its own 24 fps, so a wrong rate is a request conditioned at the wrong speed — and audio: null drops the decoded soundtrack, leaving a reference that conditions on motion and camera alone. A name that is neither an argument of from_file() nor a field of the object raises, with the fields it does have.

See workflows/templates/minimax/reference-to-video.json for a full example.

Objects Built From an Earlier Step

The same object can be built from what an earlier step generated, by naming the step instead of a file:

"references": [
    {
        "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3ImageReference",
        "from_previous_result": "draw_subject"
    }
]

from_file cannot do this — it names a file, and the object is built when the workflow loads, before any step has run. from_previous_result waits: the description is checked at load time and constructed once the step it names has produced its media, which is what lets one workflow generate a subject and then condition on it without writing it out and reading it back.

The media never touches the disk, so it arrives as the step produced it. Which field it lands in comes from the type's own kind:

kind	Built from
image	The generated image
video	The generated frames, and the soundtrack generated with them if there was one
audio	The generated soundtrack - or, for a step that produced audio alone (a music pipeline, a slice_audio task), the waveform itself. The rate travels with the waveform when the pipeline or task reports one (an AudioTrack - AudioLDM2, StableAudio, generate_speech); declare sample_rate beside from_previous_result only for a waveform from a task or file that carries none, and a declared rate always wins

Any other key is a field of the object and wins over what the media carried — "fps": 30.0 where the producing pipeline generated at a rate the consuming one does not share, for instance. A step that produced several artifacts fans out the same way every previous_result reference does: four images in, four videos out.

See workflows/templates/minimax/generated-subject-reference.json for a full example.

Objects Built From Named Arguments

Not every type a pipeline takes knows how to open a file. LTX-2's keyframe conditions and IC-LoRA references are plain dataclasses holding frames the caller already loaded, plus the numbers that say what to do with them. Those are written as the arguments to construct the object with:

"conditions": [
    {
        "condition_type": "diffusers.pipelines.ltx2.pipeline_ltx2_condition.LTX2VideoCondition",
        "from_arguments": {
            "frames": { "media_type": "image", "location": "first.png" },
            "index": 0,
            "strength": 1.0
        }
    },
    {
        "condition_type": "diffusers.pipelines.ltx2.pipeline_ltx2_condition.LTX2VideoCondition",
        "from_arguments": {
            "frames": { "media_type": "image", "location": "last.png" },
            "index": -1,
            "strength": 1.0
        }
    }
]

from_arguments holds every argument the type is constructed with - a key beside it raises rather than being silently dropped, and so does an argument the type does not take, naming the ones it does. The arguments inside are ordinary arguments: a media reference loads there, a variable: reference resolves there, and a previous_result: reference waits the way from_previous_result does - the object is constructed once the step it names has run.

Which of the three forms a type wants is decided by the type, not by preference:

Form	For a type that
from_file	opens the media itself, bringing its frame or sample rate along (MiniMax-H3's references)
from_previous_result	declares a media kind, so a step's output lands in the right field on its own
from_arguments	is a plain record of fields - no from_file(), no kind (LTX-2's conditions and references)

See workflows/templates/ltx2/keyframes.json for the file form and workflows/templates/ltx2/extend-clip.json for the one built from an earlier step.

Frames Across a Step Boundary

A pipeline that generates video with a soundtrack returns the two paired, and the result muxes them into one file. A step that works on the frames alone - a latent upsampler, an interpolator - returns frames without it. Two tasks carry the pieces across:

• video_frames takes a generated video and returns its frames as one (frames, height, width, channels) uint8 array - the 0-255 shape LTX-2's conditions want, and one artifact rather than one per frame.
• pair_audio puts a soundtrack back beside frames that lost it, so the step that saves them writes a single muxed mp4.

{
    "name": "film",
    "task": {
        "command": "pair_audio",
        "arguments": {
            "video": "previous_result:edit",
            "audio": "previous_result:balanced",
            "sample_rate": "variable:sample_rate"
        }
    },
    "result": { "content_type": "video/mp4", "fps": 24 }
}

audio takes either a waveform or the earlier step whose video carried the soundtrack, which brings its sample rate along; here it is an earlier step's waveform, so sample_rate is given explicitly. The frames keep the rate they arrived with - video given a file or an asset: carries that file's fps through to the saved mp4 - so result.fps is only needed for frames that bring no rate of their own. A mono track needs no preparation: an mp4 audio stream takes stereo and nothing else, so saving duplicates the single channel into two and emits a warning saying it did.

The track and the frames are two lengths a workflow used to have to keep equal by hand. "fit": "video" derives one from the other instead: the track is cut to exactly the frames it is laid over, or padded with silence and warned about when it is shorter than they are. That is what a soundtrack over a cut whose length is an argument needs - nothing in a workflow can multiply a list's length by a frame count, so music-video.json sliced a fixed 496 frames of song while its cut followed a shots list, and a two-shot run wrote 10.3 s of picture into a 20.7 s container and reported succeeded with no warnings (#142). Left unset the track is used as it is and a disagreement is warned about rather than passing in silence.

Which shape a pipeline argument wants is the pipeline's business, and the two LTX-2 paths differ: a keyframe condition is mapped from 0-255, so it takes the video_frames array, while an IC-LoRA reference goes through the video processor, which expects the [0, 1] frames the pipeline returned - previous_result:step.frames hands those over untouched.

Example: workflows/templates/assemble-and-score.json