Job record and export: the realized workflow beside every run, and exporting a job

CLAUDE.md

@@ -147,6 +147,12 @@ docs/WORKSPACES.md, and docs/proposals/server-workspaces.md for the later stages
   rooted at the library rather than the workflow file. The library is `DW_PROMPT_DIR` /
   `--prompt-dir`, else `./prompts` if it exists, else found by walking up from the
   workflow file's directory
+- Every run directory holds `workflow.json` beside its manifest: the *realized*
+  workflow, with the run's arguments folded into the variable defaults, the seed
+  it used, stored prompt text inlined and `output:.../latest/...` pinned to the
+  run it resolved to. Written by `realize_workflow` (`dw/realize.py`) at run
+  start, best effort. Over MCP, `get_job_workflow` reads it back and
+  `save_workflow` names it; `export_job` bundles the run
 
 The same conventions, written for an agent composing a workflow over MCP, are
 the `Authoring a workflow from an agent` section of docs/WORKFLOW_GUIDE.md;
@@ -202,7 +208,14 @@ All entry points use `dw/security.py`. When adding features:
   A sub-workflow inherits the parent's run directory and writes no manifest of its own.
   `--output-layout flat` / `DW_OUTPUT_LAYOUT` / the `output_layout` setting restores the
   old layout. The gallery groups a workflow's runs under one folder by stripping the run
-  id (`strip_run_id`)
+  id (`strip_run_id`). The realized workflow is written into the same directory as
+  `workflow.json` (`dw/realize.py`, `write_realized_workflow`), and the manifest's
+  `workflow` block carries `realized`, `prompts` (the stored prompts inlined) and
+  `sub_workflows` (path -> SHA-256). A job records the run it was
+  (`run_id`/`run_dir` on `Job` and in `jobs.sqlite`), which is how
+  `JobManager.realized` finds the file. `exports` is a reserved workspace name:
+  `POST /api/jobs/{id}/export` gathers one finished job into
+  `<workspace>/exports/<job id>/` and `GET /exports/<job id>.zip` streams it.
 - **Step cache**: a process-wide singleton (`dw/step_cache.py`) consulted by every `Workflow.run`, including server jobs; entries are keyed by `(workflow id, step name)` and validated against the output
   *root*, never the per-run directory - a run directory is new every execution and would
   defeat the cache; disabled entirely when the workflow sets no `seed`; a hit reports the earlier run's files with `reused: true` and writes nothing new; `memory clear` drops it

docs/MCP.md

@@ -280,6 +280,8 @@ references written in the same session.
 | --- | --- | --- |
 | `run_workflow(workflow_path=None, inline_workflow=None, arguments=None, acknowledged_cost=False)` | exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path to a workflow file on the server) or `inline_workflow`, optional `arguments`, `acknowledged_cost` | Queue a workflow for generation. Returns as soon as the job is queued |
 | `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback |
+| `get_job_workflow(job_id)` | `job_id` | The workflow the job actually ran. `realized: true` means every mutable input is pinned (arguments, seed, prompts, `output:latest`); `false` means the job predates run tracking and this is the definition as submitted. Pass it to `save_workflow` to keep it under a name |
+| `export_job(job_id, overwrite=False)` | `job_id`, `overwrite` | Gather one finished job into `<workspace>/exports/<job id>/` on the server: the realized workflow, the run's manifest, the job row, a README, and copies of the assets, earlier-run inputs and outputs. Returns the directory, a zip URL, the file list with sizes and the three JSON files inline. **The directory is on the machine running the server**, like `download_output`'s destination - fetch the zip URL and unpack it into `exports/` under the session's working directory (a deliverable, not a temp file); the archive already unpacks into one folder named after the job id |
 | `get_job_events(job_id, after=-1, limit=200)` | `job_id`, `after`, `limit` | Get a page of a job's progress events |
 | `wait_for_job(job_id, timeout_seconds=20)` | `job_id`, `timeout_seconds` | Block until a job reaches a terminal status, or `timeout_seconds` elapses (capped well under a generation's real runtime). Use instead of hand-polling `get_job`/`get_job_events` in a loop; if it returns `still_running: true`, call it again |
 | `cancel_job(job_id)` | `job_id` | Ask a queued or running job to stop |

docs/SERVER.md

@@ -135,7 +135,9 @@ from another machine:
 | `POST /api/jobs` | Queue a run: `{"workflow_path": ...}` or an inline `{"workflow": {...}, "base_dir": ...}`, plus `arguments` for variable overrides. `workflow_path` accepts a stored workflow name as listed by `/api/workflows` (with or without `.json`, nested names included), or a relative/absolute path that still resolves under `--workflow-dir` - confined the same way the `/api/workflows` CRUD routes are; a path that names a real file outside that directory is rejected with 400, not opened. Answers with argument warnings from signature checking. |
 | `GET /api/jobs` | Queue + history summaries |
 | `GET /api/jobs/{id}` | Full detail: spec, events, manifest, error. A manifest entry for a step served from the step cache carries `reused: true` |
-| `GET /api/jobs/{id}/workflow` | The definition the job ran, for the job page's read-only flow graph: `{id, definition}`. An inline definition comes from the job's own spec; a job launched from a path is re-read from the root it was confined to, so 404 means the file has since moved or changed - the job itself is still readable |
+| `GET /api/jobs/{id}/workflow` | The workflow the job ran: `{id, definition, realized}`. `realized: true` is the copy the run itself wrote (`workflow.json` in its run directory), with arguments, seed, prompts and `output:latest` pinned; `false` falls back to the submitted definition, which is what a job from before run tracking has. 404 means neither is readable - the job itself still is |
+| `POST /api/jobs/{id}/export?workspace=&overwrite=` | Gather one finished job into `<workspace>/exports/<job id>/`: `workflow.json`, `manifest.json`, `job.json`, `README.md`, `assets/`, `inputs/`, `outputs/`. 201 with the file list, total bytes, anything it could not find, a `zip_url`, and the three JSON files inline. 404 unknown job, 409 for a job still running or an existing export without `overwrite` |
+| `GET /exports/{id}.zip?workspace=` | The same tree as one archive, built on request rather than kept as a second copy. Entries are named `<job id>/<relative path>`. Ungated exactly as `/outputs` is |
 | `GET /api/jobs/{id}/events` | Server-sent events stream; `?after=N` / `Last-Event-ID` replay missed events, so reconnects are lossless |
 | `GET /api/jobs/{id}/event-log?after=-1&limit=200` | The same events as the SSE stream, as one JSON page: `{id, status, events, last_seq, truncated, note}`. `after` is exclusive; page by passing back the previous `last_seq`. A job restored from history serves the bounded event tail persisted with it; a job that finished before events were retained returns an empty list and a `note` saying so. |
 | `POST /api/jobs/{id}/cancel` | Cooperative cancel (takes effect at the next step boundary or denoise step) |
@@ -154,6 +156,7 @@ Every event in the stream carries a `seq` and an `event` name:
 | `job_status` | queued/running/terminal transitions | `status` |
 | `log` | worker output lines | `message` |
 | `memory` | device memory after a run | `info` |
+| `run_start` | the run directory is chosen, before the first step | `run_id`, `identity`, `run_dir` |
 | `workflow_start` | the run begins | `workflow`, `total_steps`, `steps`, `seed` |
 | `step_start` / `step_end` | each step | `step`, `index`, `total_steps`; `files` at the end. A step served from the step cache adds `reused: true` to `step_end`, and its `files` are the earlier run's files rather than newly written ones |
 | `iteration_start` | each argument combination in a step | `step`, `iteration`, `total_iterations` |
@@ -259,6 +262,10 @@ The editor's forms come from these; they are just as usable from scripts:
   refuses until acknowledged, refuses the default, and refuses a workspace
   with jobs still queued. A workspace is a namespace, **not** a security
   boundary: the API token is all-or-nothing
+
+  `exports/` sits beside the workspace's own folders, holding one directory per
+  exported job. It is a reserved name: no workspace can be called `exports`, and
+  the folder is never listed as one.
 - `GET /api/assets` — the asset library: input media, each with the
   `asset:` reference a workflow carries rather than a path, since a path
   only means something on the server's own machine. Empty rather than an

docs/WORKFLOW_GUIDE.md

@@ -270,6 +270,14 @@ not validation.
   `text`, rooted at the prompt library. That text may not itself begin with any of these
   prefixes; the engine rejects such a prompt rather than resolving twice.
 
+After a long inline run that is worth keeping, `get_job_workflow(job_id)`
+returns the realized workflow — the definition with the arguments, seed and
+prompts of that run pinned into it — and `save_workflow` gives it a name, so
+the next run is by name rather than by pasting JSON again. `export_job(job_id)`
+bundles the whole run (workflow, manifest, job row, the media on both sides)
+into a directory on the server plus a zip URL, for a run worth committing or
+handing to someone else.
+
 A reference is resolved wherever it appears in the arguments, including inside
 a nested object or list — not only at the top level. It is always the *whole*
 value: `"variable:base_prompt"` resolves, `"variable:base_prompt, in fog"` asks
@@ -999,6 +1007,26 @@ Omit `seed` entirely to let the workflow draw a random one at run time. The seed
 actually used - drawn or named - is recorded in its `manifest.json`, so a run you liked
 can be reproduced after the fact.
 
+Beside that manifest the run also writes `workflow.json` — the *realized*
+workflow, meaning the one that actually ran. Every mutable input is pinned into
+it: the caller's `arguments` folded into the `variables` defaults, the seed the
+run used, each `prompt:` reference replaced by the stored text, and each
+`output:<identity>/latest/<file>` rewritten to the run id it resolved to.
+`asset:`, `constant:`, `previous_result:` and `builtin:` are kept as written —
+each already names something pinned by the asset library or by the manifest's
+`dw_version` — and a sub-workflow named by local path is kept with its file's
+SHA-256 recorded in the manifest. The manifest also lists which stored prompts
+were inlined, since inlining loses the name.
+
+The file is a valid workflow, and running it again is `python -m dw.run
+workflow.json` or handing its contents to `run_workflow` as `inline_workflow`
+— but either way the `asset:` and `output:` names in it resolve against the
+server's or CLI's own libraries, not against the run directory, so doing this
+from inside that directory reproduces the run only when its libraries are the
+ones the original run used too. Writing the file is best effort, exactly like
+the manifest — a run that produced its files has succeeded either way — and
+`--output-layout flat` writes no run directory, so it writes neither file.
+
 Any of the three levels accepts a `variable:` reference, which is how a seed becomes
 settable per run without editing the file:
 

docs/WORKSPACES.md

@@ -176,6 +176,7 @@ outputs/
       Gyre-still.0-0.0.png
       Gyre-video.1-0.0.mp4
       manifest.json
+      workflow.json
 ```
 
 The folder is the workflow's identity — its path under a `workflows/` tree
@@ -185,6 +186,10 @@ runs of the same workflow sort by time and a rerun of an edited workflow is
 visibly different; a second run of the same spec in the same second takes a
 counter rather than sharing a directory.
 
+`workflow.json` is the realized workflow — the definition with this run's
+arguments, seed and stored prompts pinned into it, so the directory reproduces
+itself. `manifest.json` points at it and lists which prompts were inlined.
+
 `manifest.json` records the run beside what it made — status, seed, arguments,
 device, dw version, and each step's files, named relative to the directory so
 it keeps describing itself if you move or copy it. It is written even when a
@@ -229,6 +234,10 @@ reference, and a prompt duplicated per workspace would resolve to different
 text depending on where a workflow happened to be saved. `workflows`,
 `prompts`, `assets` and `outputs` are reserved names for that reason.
 
+A fifth name is reserved beside `workflows`, `prompts`, `assets` and `outputs`:
+`exports`. `POST /api/jobs/{id}/export` gathers one finished job into
+`<root>/exports/<job id>/`, and that folder is never mistaken for a workspace.
+
 This is what lets two agents share one GPU without sharing a namespace: each
 takes a workspace, and neither can save over the other's workflows or delete
 the other's renders.

docs/proposals/codeql-sanitizer-model.md

@@ -0,0 +1,114 @@
+# Proposal: making CodeQL see the path validators
+
+Status: draft, 2026-09-08. No implementation.
+
+## Problem
+
+Every release adds a few `py/path-injection` alerts from code scanning, and
+every one so far has had the same shape: a path is joined and then confined
+by `validate_path` in `dw/security.py` (or one of the wrappers around it),
+and CodeQL flags the `os.*` or `shutil.*` call downstream because it does
+not treat the validator as a sanitizer. The count on 2026-09-08 is 28 open
+and 14 dismissed as "mitigated", all of that rule, plus the ten on #56 that
+were dismissed the same way after a review found one real gap among them
+(an `export_directory` job id of `.` named the exports root itself; fixed
+in 7ead046 with a shape check).
+
+Dismissing by hand works but scales badly, and it teaches the reviewer to
+skim: the one real finding on #56 was flagged in exactly the same words as
+the nine that were not.
+
+## What already exists, and why it is not working
+
+`.github/codeql/extensions/dw-models/` is a CodeQL model pack, added with
+the workspaces PR (#37, 2026-09-05), that declares the validators' return
+values as `path-injection` barriers with the `barrierModel` extensible
+predicate:
+
+```yaml
+- ["dw.security", "Member[validate_path].ReturnValue", "path-injection"]
+```
+
+The approach is right. `barrierModel` and `barrierGuardModel` were added
+to models-as-data for Python in CodeQL 2.25.2 (April 2026), the path
+injection query's `SanitizerFromModel` reads barriers of kind
+`path-injection`, code scanning default setup picks up model packs from
+`.github/codeql/extensions/` without a workflow, and the repository is
+scanned with CodeQL 2.26.4. Yet every alert listed above was created
+*after* the pack landed: 26 to 43 on 2026-09-06 and 07, 44 to 53 on the
+08th. The pack is not taking effect, and nothing in the scanning UI says
+why.
+
+Hypotheses, most likely first:
+
+1. **The type string does not resolve.** Every caller imports the
+   validators relatively (`from ..security import validate_path`,
+   `from .security import ...`). The `type` column of a Python model is
+   matched through the API graph's module-import resolution, which is
+   built for absolute imports of library modules; whether a relative
+   import inside the analysed package resolves to `dw.security` is not
+   documented and is the first thing to test.
+2. **The pack is not being loaded at all.** A malformed
+   `codeql-pack.yml`, a name collision, or a default-setup restriction
+   the docs do not spell out. Default setup exposes no log for this.
+3. **The barrier is on the wrong node.** A barrier on the return value
+   stops flow that *passes through* the call. The flow CodeQL reports may
+   instead go around it: `validate_path` returns a value computed from
+   its argument, and if the analysed body contributes a second path
+   (the argument itself flowing to a sink inside `validate_path`, then
+   summarised) the return-value barrier would not cut it. Unlikely, but
+   the alert's path view would show it.
+4. **`base_dir=None` is a real hole in the model.** `validate_path(p,
+   None)` resolves and pattern-checks but confines to nothing, so a
+   barrier on its return value claims more than the function guarantees.
+   That does not explain the alerts, but it is why the model should sit
+   on the wrappers that always pass a root (`validate_output_path`,
+   `validate_workflow_path`, `validate_prompt_path`) and on
+   `validate_path` only when the root argument is present, which the
+   model language cannot express. Decide whether to accept the
+   over-claim or narrow the model.
+
+## Proposed shape
+
+**Reproduce locally, then fix the model, then let the scanner confirm.**
+Default setup gives no feedback loop; the CodeQL CLI does.
+
+1. Install the CodeQL CLI (`gh extension install github/gh-codeql`, which
+   also fetches the bundle) on the Mac.
+2. `codeql database create --language=python dw-db` at the repo root,
+   then `codeql database analyze dw-db codeql/python-queries:codeql-suites/python-code-scanning.qls --model-packs dkackman/dw-models --format=sarif-latest --output=out.sarif`
+   with the pack path supplied through `--additional-packs
+   .github/codeql/extensions`. Count the `py/path-injection` results.
+3. Iterate on `dw-security.model.yml` until the count drops to the
+   findings that are real: try an absolute-import form of the type, try
+   the wrappers rather than `validate_path`, try `barrierGuardModel` on
+   the `SecurityError` branch. Each try is one edit and one analyze.
+4. Commit the working model. Code scanning re-runs on the next push and
+   the open alerts close as "fixed" on their own, which is the only
+   confirmation that matters.
+5. Record in `dw/security.py`'s module docstring that the validators are
+   modelled, and where, so the next person adding a validator adds a row.
+
+A `scripts/codeql-local.sh` that does steps 2 and 3 in one command is
+worth keeping, since the model will need the same loop each time a
+validator is added.
+
+## What this is not
+
+- Not a workflow migration. Default setup honours repository model packs,
+  and advanced setup is only needed if step 3 shows the fix requires a
+  custom query, which nothing so far suggests.
+- Not a reason to stop reading the alerts. Once the model holds, an alert
+  of this rule means a path that did not go through a validator, which is
+  exactly the signal the rule is for.
+
+## Open questions
+
+- Whether the fourteen "mitigated" dismissals and the ten from #56 should
+  be reopened once the model works, so the scanner re-evaluates them. They
+  would close as fixed if the model is right and stay open if it is not,
+  which is a useful check; but reopening is a manual click each.
+- Whether `validate_path` with no root should keep returning a path at
+  all, or whether every caller should be made to pass one. That is a
+  security question independent of CodeQL, and the model's over-claim in
+  hypothesis 4 is the argument for looking at it.