@@ -11,11 +11,16 @@ re-runs each one and byte-compares it against the output block underneath. No
1111live model is called anywhere in this file; the one snippet that would need a
1212key says so and was not run.
1313
14- The ` console ` transcripts are the exception, and it is worth naming rather than
15- leaving to be discovered: they were produced by running the commands shown, but
16- the test only pairs ```python blocks, so nothing re-checks them on every commit.
17- Treat a console block as a record of one real run rather than as a guarantee
18- about the current tree.
14+ The ` console ` transcripts are held to the same standard, marked the way
15+ ` 02-models.md ` marks its own. ` <!-- verified: cli --> ` above a transcript means
16+ every command in it is re-run on every commit and byte-compared — run ids are
17+ random and durations are wall-clock, so the test maps the former and masks the
18+ latter, and every other character has to match. ` <!-- verified: cli varies --> `
19+ means every command is re-run and must succeed, but the output describes one
20+ machine — ` serve ` binds a port, ` models --check ` probes the host — so its bytes
21+ are the record of one real run rather than a guarantee about yours. The one
22+ ` agent ` invocation always needs a live model; it is marked
23+ ` <!-- needs-credentials --> ` and nothing here claims to have run it.
1924
2025Three things to know before you start:
2126
@@ -1241,28 +1246,31 @@ REGISTRY = GraphRegistry({"qa": build_qa})
12411246```
12421247
12431248Then, in a shell. This transcript is a real one: the ids and timestamps differ
1244- per run, and the only editing is the ` ... ` marking where a long JSON body was
1245- cut — nothing was reworded.
1249+ per run, and the only editing is the ` ... ` marking where a timestamp or a long
1250+ JSON body was cut, plus the session view re-indented for reading — nothing was
1251+ reworded. The test re-runs every command here against a fresh server and
1252+ requires each to succeed; the bytes below are one run's.
12461253
1254+ <!-- verified: cli varies -->
12471255``` console
12481256$ PYTHONPATH=. grapharc serve --registry mygraphs:REGISTRY --port 8124
12491257serving grapharc.server on http://127.0.0.1:8124
12501258graphs : qa
12511259ctrl-c to stop
12521260
12531261$ curl -s localhost:8124/healthz
1254- {"status":"ok","version":"0.1.0 ","graphs":["qa"]}
1262+ {"status":"ok","version":"0.1.1 ","graphs":["qa"]}
12551263
12561264$ curl -s -X POST localhost:8124/sessions -H ' content-type: application/json' \
12571265 -d '{"graph":"qa","input":{"question":"how do budgets work?"}}'
1258- {"id":"38231fa41b8b4dad ","graph":"qa","thread_id":"38231fa41b8b4dad ","status":"queued", ...}
1266+ {"id":"bf5ca55bff7b480f ","graph":"qa","thread_id":"bf5ca55bff7b480f ","status":"queued", ...}
12591267
1260- $ curl -s localhost:8124/sessions/38231fa41b8b4dad
1268+ $ curl -s localhost:8124/sessions/bf5ca55bff7b480f
12611269{
1262- "id": "38231fa41b8b4dad ",
1270+ "id": "bf5ca55bff7b480f ",
12631271 "graph": "qa",
12641272 "status": "succeeded",
1265- "run_id": "f3f2554b3959 ",
1273+ "run_id": "0d9dce7f61c4 ",
12661274 "result": {
12671275 "question": "how do budgets work?",
12681276 "answer": "Budgets cap iterations, tokens and time."
@@ -1271,9 +1279,9 @@ $ curl -s localhost:8124/sessions/38231fa41b8b4dad
12711279 ...
12721280}
12731281
1274- $ curl -s localhost:8124/sessions/38231fa41b8b4dad /trace
1275- {"ts": "...", "run_id": "f3f2554b3959" , "graph": "qa", "node": "answer", "phase": "start", "step": 1}
1276- {"ts": "...", "run_id": "f3f2554b3959 ", "graph": "qa", "node": "answer", "phase": "end", "step": 1, "state_delta": {"answer": "Budgets cap iterations, tokens and time."}, "duration_ms": 0.91 , "tokens": 15}
1282+ $ curl -s localhost:8124/sessions/bf5ca55bff7b480f /trace
1283+ {"ts": "...", "run_id": "0d9dce7f61c4", "thread_id": "bf5ca55bff7b480f", "attempt": 1 , "graph": "qa", "node": "answer", "phase": "start", "step": 1}
1284+ {"ts": "...", "run_id": "0d9dce7f61c4 ", "thread_id": "bf5ca55bff7b480f", "attempt": 1, " graph": "qa", "node": "answer", "phase": "end", "step": 1, "state_delta": {"answer": "Budgets cap iterations, tokens and time."}, "duration_ms": 1.0288769999533542 , "tokens": 15}
12771285```
12781286
12791287The banner is printed * before* the server blocks, so a script watching stdout
@@ -1579,8 +1587,11 @@ Exit codes are part of the interface: `0` did the job, `1` ran and the answer
15791587was negative (two runs differed, a run id had no events, no backend was usable),
15801588` 2 ` could not run at all (missing file, missing component, unknown model spec).
15811589
1582- A whole session, verbatim:
1590+ A whole session, verbatim (run ids are random per run and durations are
1591+ wall-clock; the test maps the former, masks the latter, and byte-compares every
1592+ other character):
15831593
1594+ <!-- verified: cli -->
15841595``` console
15851596$ grapharc demo stage1 --trace trace.jsonl
15861597...
@@ -1612,6 +1623,8 @@ duration_ms: 0.68
16121623attempts: 1
16131624termination_reason: target_met
16141625per_node: {'start': 1, 'plan': 2, 'act': 2, 'verify': 2, 'finish_target_met': 1}
1626+ events: 16
1627+ per_phase: {'start': 8, 'end': 8}
16151628
16161629$ grapharc viz trace.jsonl 2a47f18064b7
16171630flowchart TD
@@ -1625,6 +1638,7 @@ flowchart TD
16251638 n6["verify"] --> n7["finish_target_met"]
16261639
16271640$ grapharc replay trace.jsonl 2a47f18064b7 | tail -4
1641+ pending = []
16281642 8 ok finish_target_met (0.0ms)
16291643 termination_reason = 'target_met'
16301644 8 nodes · 81 tokens · stopped: target_met
@@ -1642,6 +1656,7 @@ Everything downstream (`metrics`, `viz`, `replay`, `diff`) wants that id.
16421656
16431657` --json ` on any of them, and on failures too:
16441658
1659+ <!-- verified: cli -->
16451660``` console
16461661$ grapharc metrics trace.jsonl 2a47f18064b7 --json
16471662{
@@ -1655,7 +1670,18 @@ $ grapharc metrics trace.jsonl 2a47f18064b7 --json
16551670 "duration_ms": 0.75,
16561671 "attempts": 1,
16571672 "termination_reason": "target_met",
1658- "per_node": {"start": 1, "plan": 2, "act": 2, "verify": 2, "finish_target_met": 1}
1673+ "per_node": {
1674+ "start": 1,
1675+ "plan": 2,
1676+ "act": 2,
1677+ "verify": 2,
1678+ "finish_target_met": 1
1679+ },
1680+ "events": 16,
1681+ "per_phase": {
1682+ "start": 8,
1683+ "end": 8
1684+ }
16591685}
16601686
16611687$ grapharc metrics nope.jsonl abc --json; echo " exit $? "
@@ -1669,12 +1695,18 @@ exit 2
16691695
16701696` grapharc models ` needs no credentials to answer what a spec * resolves* to:
16711697
1698+ <!-- verified: cli -->
16721699``` console
16731700$ grapharc models openrouter/anthropic/claude-haiku-4.5
16741701spec: openrouter/anthropic/claude-haiku-4.5
16751702backend: openrouter
16761703model: anthropic/claude-haiku-4.5
1704+ ```
1705+
1706+ ` --check ` reports the machine it is run on:
16771707
1708+ <!-- verified: cli varies -->
1709+ ``` console
16781710$ grapharc models --check
16791711claude-cli usable 'claude' on PATH at /home/you/.local/bin/claude
16801712 credential: claude subscription login (no API key)
@@ -1699,6 +1731,7 @@ count.
16991731` grapharc agent ` is the one command in this list that always needs a model, so
17001732there is no scripted form of it and nothing here claims to have run it:
17011733
1734+ <!-- needs-credentials -->
17021735``` console
17031736$ grapharc agent " summarise README.md" --workspace ./work --max-turns 6 --json
17041737```
0 commit comments