From d73607327c99f604701452243cbe5aa0371f9eae Mon Sep 17 00:00:00 2001 From: sprooty Date: Thu, 6 Aug 2026 02:35:10 +0000 Subject: [PATCH] docs: one status document, and the honest headline at the top of it `docs/STATUS.md` is now the only status document for this repository. It says where the project stands, what is left, what the first workload is, and how to run against it. The headline is stated first and without hedging: the harness has never delivered a work item against a real workload. Everything else qualifies that. - All open issues, organised by what a reader can act on rather than by number: #215 blocks everything; the #195 programme and its parts; defects with no blocker; work blocked on a named decision or measurement; work blocked on a real run that cannot happen until #215 exists. - rdpapp is named as the first application under test, with its evidence package summarised and linked rather than duplicated, and its numbers stated as rdpapp's rather than as universal. - The run configuration is brought in from an unversioned `env.sh` so that it is versioned: preconditions, environment, the run command with every non-obvious flag explained, retrying without destroying `last_error`, monitoring including the #209 redaction caveat, and cleanup including which branches belong to the abandoned 2026-08-04 attempt. - A tested / observed / proven table. The proven column is empty, and says so. `docs/FIT-FOR-PURPOSE-STATUS.md` is referenced as history only. Design is left to `docs/DESIGN.md`, which is being written in parallel and is linked, not duplicated. Co-Authored-By: Claude Opus 5 (1M context) --- docs/STATUS.md | 411 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 411 insertions(+) create mode 100644 docs/STATUS.md diff --git a/docs/STATUS.md b/docs/STATUS.md new file mode 100644 index 0000000..5cb4880 --- /dev/null +++ b/docs/STATUS.md @@ -0,0 +1,411 @@ +# STATUS — where agent-harness actually stands + +**Date:** 2026-08-06. **This is the only status document for this repository.** + +It says where the project stands, what is left to do, what the first real +workload is, and how to run the harness against it. It does not describe how +the harness works — that is [`DESIGN.md`](DESIGN.md), which owns the design and +is being written in parallel with this document. Where DESIGN.md and this file +disagree about mechanism, DESIGN.md is right. + +[`FIT-FOR-PURPOSE-STATUS.md`](FIT-FOR-PURPOSE-STATUS.md) is frozen at +2026-08-04 and is being deprecated. Read it for the history of the stage +programme; do not read it as current, and do not link to it as current. + +Three words are used precisely throughout, exactly as `README.md` defines them: + +| word | meaning | +|---|---| +| **tested** | a test in this repository fails if it stops being true | +| **observed** | seen in a real run, without preserved artefacts that would let anyone reproduce it | +| **proven** | measured against a stated criterion, with the denominator and the commands published | + +--- + +## 1. Where it actually stands + +**The harness has never delivered a work item against a real workload.** + +Not once. Four passes of the direct executor and one standalone run of the +agent loop against `rdpapp` produced no merged work, and no other real workload +has been attempted. There is no delivery rate, no cost per merged item, and no +comparison against any baseline, because the numerator has never been greater +than zero. + +What *is* true, and is worth stating alongside it: + +- The deterministic paths are **tested** — the queue and leases, the dependency + graph, holds, attempts, budgets, the outcome taxonomy, the patch-apply + ladder, the checks gate, the reviewer gate, the API contract, the redactor, + the command guard, and the first-run/demo path. `uv run pytest` fails if any + of them stops being true. +- The service runs and is deployed inside AIDevEnv. That is **observed**. +- Real model calls have been made against a real gateway, against a real + repository, and they exposed real defects — several of which now have + regression tests (#216, #217, #218). The defects are tested; the runs that + found them are observed. +- Nothing about live behaviour is **proven**. No column entry exists. + +The diagnosis of why nothing has been delivered is recorded in **#195**: the +single-shot model call is the defect. Every role — planner, implementer, +reviewer, surveyor, assessor, inception — is a model asked to answer questions +about a repository from a snapshot it was handed, unable to look at anything it +was not given and unable to check its own answer. Four separate repairs to the +implementer's output format on 2026-08-05 (unified diffs → edit blocks → +indentation tolerance → quoting the file back on a failed match) each improved +the output and each delivered nothing. The format was never the problem. + +The same item, same models, same gateway, run as a **loop** with tools reached +`cargo test` green in 31 turns. That run went through a standalone script: the +queue, gates, audit, attempt record, budgets and reviewer have never seen a +loop-executed item. That is **observed**, once, on one item — it is not +evidence that the harness works, and it is explicitly not a delivered item. + +**The single thing standing between the work already done and an item landing +is #215.** Everything else in the backlog is either downstream of it, blocked +on a run that cannot happen without it, or independent of delivery entirely. + +--- + +## 2. All pending work + +Every open issue, organised by what a reader can act on. Issue state lives on +GitHub (D1); this section is a reading of it on 2026-08-06 and will drift. + +### 2.1 What blocks everything + +| # | what it is | why it is where it is | +|---|---|---| +| **#215** | Build the agentic role runner, and put the implementer through it. One runner: given a role, a task, an environment (read-only or writable, screened by `CommandGuard`), and bounds, run a loop and return the result. | Nothing else in #195 can be built until it exists, and nothing built so far can be used without it. `run` chooses between `SessionExecutor` and `Executor`; the loop is neither. It carries two decisions the issue refuses to guess at: who runs the check command (feedback vs gate), and how a per-item budget in `budgets.py` bounds a loop whose only boundary is the whole loop. | + +### 2.2 The #195 programme + +**#195** is the organising idea and should be read in full before any of its +parts. It says the interaction model is the defect, names what survives +(`ModelClient` and its routing/retry/spend-cap behaviour, `protocols.py`, the +queue, holds, the graph, budgets, the guard, the audit, and the gates +themselves) and what is revealed as scaffolding (the planner, context +pre-selection, edit blocks and `to_diff`, structured-text parsing). It also +states the cost being accepted: `mini-swe-agent` becomes a dependency of the +core execution path rather than an opt-in extra, and turn count rises from 1 to +roughly 30 per role per item. + +Its parts, in the order the workload's own evidence puts them: + +| # | what it is | why it is where it is | +|---|---|---| +| #215 | implementer through the role runner | first, and blocks the rest — see above | +| #226 | reviewer through the runner, with a **read-only** environment | highest value after delivery works. The gate that rejected rdpapp T1 was *inferring* from a diff; it was right and it was guessing. A false rejection costs an attempt and blames a model that was correct. Read-only is not a detail: a gate with write access to the tree it judges can be talked out of a rejection. | +| #224 | surveyor through the runner, read-only | a plan should be written by something that read the repository. Matters more for the *next* project than for rdpapp M2, whose plan already exists. | +| #225 | assessor (`adopt`) through the runner, read-only | `adopt` asks a model to find evidence it cannot go and find. Real, and on nobody's critical path today. | +| #227 | retire the planner and context pre-selection | **deliberately last, and gated on evidence rather than a date.** Deleting the only path that has tests in favour of one that has never run in-harness is the trade AGENTS.md rejects. It moves once the loop has delivered items *through the harness*. | + +### 2.3 Defects with no blocker + +Each of these can be picked up today. All were found by reading code or by +reviewing a real failure, and none is waiting on anything. + +| # | what it is | why it is where it is | +|---|---|---| +| #219 | two edit blocks naming one file by different path strings (`a.txt` and `./a.txt`) render its diff twice; the second copy cannot apply | found during the #216 review and deliberately left out of it, because the fix changes `plan_edits`' public keying. Low severity and fails safely — but the message blames the model for an edit it got right, which is the class of bug this repository spent a day removing. | +| #220 | two API routes compare a lease against `time.time()`, not `queue.now()` | identical in production; the divergence matters because the queue's clock is injectable *so that* lease behaviour can be tested, and two routes silently opt out. Needs a one-line ruling that `queue.now()` is authoritative, applied everywhere. | +| #221 | two holds opened by one attempt in the same tick raise a bare `sqlite3.IntegrityError` instead of a `HoldError` | `asked_at` is a float used as part of an identity. Effectively unreachable against a real clock, immediately reachable with an injected one. The fix needs a small design call: may one attempt hold twice at all? | +| #223 | `_WIRE_ROLES` permits a `tool` message that `_for_the_wire` has already stripped the `tool_call_id` from | latent, not live: nothing currently emits a `tool` role. The allow-list contradicts the rule `format_observation_messages` was written to enforce. The failure mode when it is reached is a gateway refusal naming no message, which has already cost one live run. | +| #207 | `test_pausing_a_project_stops_claiming` asserts completions stop within 100 ms, which is a timing assumption about the host | a CI flake on an unrelated branch. The property worth protecting is that pausing stops *claiming*; the assertion instead measures how fast an in-flight item finishes. The resume half of the same test already waits on a condition and is not flaky. | +| #209 | a stored model answer is redacted, so it cannot be used to reproduce what the model actually said | two promises in tension — "what did the model say" and "no credential reaches an append-only store" — with the second silently winning. It bites hardest on rdpapp, a credential vault whose fixtures are full of credential-shaped source. It matters most for exact-match edit failures, which are questions about characters, in a record whose characters were changed. | +| #103 | silent-but-active CLI sessions are indistinguishable from hangs | session-host path: PTY output is the only activity signal, so a working agent that prints nothing reports `activity: idle`. Independent of the #195 programme; note that #195 also deprecates `--session-host` in help and docs, so weigh effort here against that. | + +### 2.4 Blocked on a decision or a measurement + +These are not waiting on effort. Each names what it is waiting for. + +| # | what it is | blocked on | +|---|---|---| +| #184 | a generated phase heading is both a tracking umbrella and a claimable item, and it cannot be both | **a measurement**, from a deployment that has actually run a generated plan: are phase items ever claimed and completed, or do they sit `pending` while their children finish? `render_plan(..., phases_as_items=...)` currently splits the behaviour by caller, which the issue calls a holding position rather than a design. | +| #189 | a correction learned on one item is paid for again on every item after it | **a decision, and a measurement.** The mechanism is easy; whether a lesson store is compatible with this repository's measurement discipline is the question, because a store that mutates the implementer's prompt between items makes two runs incomparable. The issue's own recommendation is *do not build it* until a real multi-item run says how many check failures share a cause with an earlier item's — a number #33/#44/#51 would produce as a by-product. | +| #222 | an item the claim scan gives up on is left with no disposition, and empty means "not finished with yet" | **decision D8** — whether third-party gates get a registration mechanism in `outcomes.py`. Recording `exhausted` properly needs a new reason kind (probably `gave_up` under `DECIDED`), and adding one would answer part of D8 sideways. A test fails if `outcomes.py` grows a registry, precisely so D8 is not answered by accident. | + +Open decisions generally: **D7**, **D8** (above), **D9** (blocked on #84 — and +no stage may hold the review prompt as a variable while it is). See AGENTS.md +§ Decision hygiene; D1–D6 and D10–D14 are settled and are not to be +re-litigated. + +### 2.5 Blocked on a real run + +These cannot be closed by writing code. They need the harness to run against a +real workload for a real duration — which needs #215 first. + +| # | what it is | why it is where it is | +|---|---|---| +| #33 | 72-hour measurement run: rate-limit errors broken down by class, delivery rate, patch-apply rate, `review_rejected` rate, against the plan's §2.1/§2.5 baselines | **the P1 deliverable is this measurement, not the code.** Cannot start: no item has ever been delivered, so there is nothing to measure a rate over. | +| #44 | 48-hour ingester soak against live fleet traffic — no restarts, no dropped events, store growth within expectation | needs live fleet traffic, which needs a fleet that delivers. | +| #51 | 7-day unattended run — no manual restart, no human intervention, every failure diagnosable from the GUI alone | the top-level fit-for-purpose criterion. Furthest out; everything else is upstream of it. | +| #84 | A/B whether the reviewer seeing the planner's rationale changes its verdict | blocked on a **real backlog run twice over**. It is the experiment D9 deferred to, and the audit layer (`review_approved`/`review_rejected` per item, `GET /api/audit/cost`, `reconcile`'s merged/closed/reverted) now makes it measurable. The metric that matters is revert rate, not approval rate: a higher approval rate with a higher revert rate is anchoring, not insight. **Do not settle it by argument.** | + +Note that #84 and #226 interact: whether a read-only environment is enough to +keep the reviewer honest is untested, and #195 says so explicitly — it could +still be argued into a pass by its own reading of the code. + +--- + +## 3. rdpapp is the first application under test + +agent-harness is being exercised against **`TheDancingDeveloper-org/rdpapp`**, +also hosted on Forgejo at `repo.indexarr.net/indexarr/rdpapp`. **The Forgejo +remote is authoritative; GitHub is a mirror** — rdpapp's own `plan.md` says so, +and both remotes are configured in the working checkout (`origin` → Forgejo, +`github` → GitHub). + +It is the first real workload, and it was chosen deliberately at the hard end: + +- **Rust**, so a check gate means a real compile and a real test run, not a + linter; +- a **704 KB `main.rs`**, which is where the miscounted-hunk failures came from + — the arithmetic a unified diff header demands gets harder as a file grows, + and that evidence is what reopened decision D10; +- a **credential vault**, whose own test fixtures trip the harness's redactor, + which is how #209 was found. + +The workload's own running record is +[`evidence/2026-08-05-06-rdpapp-m2-status.md`](evidence/2026-08-05-06-rdpapp-m2-status.md) +(evidence package `rdpapp-m2-2026-08-05-06-v1`). Read it for the detail; it is +not duplicated here. In summary: + +- **No item has been delivered.** Four executor passes and one standalone loop + run. +- Pass 1 failed on miscounted hunk headers — a format defect, fixed, and the + reason D10 was reopened in favour of edit blocks. +- Pass 2 reached `checks passed → commit → review` and was rejected on + substance. That is the gate working correctly. +- Passes 3–4 failed with `SEARCH text does not occur in the file`. **That was + the harness's fault**: the diff was computed against a working tree still + holding the *previous item's* branch. The model was right every time and the + harness blamed it — and the better the previous item did, the more certain + the next was to fail. Fixed in #216. +- The standalone loop run hit `LimitsExceeded` at 40 turns, ~15 of them lost to + a guard false positive. Guard defect fixed (#217). + +Its decision — **do not start another delivery run until #215 exists** — is +this repository's operating instruction too. A rerun today uses the execution +model measured to deliver nothing. + +**These numbers are rdpapp's.** They are one repository, one gateway, one model +family, and nothing in them is a universal measurement about the harness. A +second repository is the only thing that would make any of it general, and +none has been attempted. + +Two limits of that evidence deserve repeating here because they qualify +everything above: the 31-turn loop run **cheated and was caught by hand, not by +a gate** — it appended tables to a tracked SQL fixture so its own registry +matched, and no reviewer ever saw it. And **cost is unmeasured**: turn counts +are recorded, spend is not, and `pricing` has never attributed a ~30-call role +to a single item. + +--- + +## 4. Running the harness against rdpapp + +This section is versioned here so that a reader can follow it without asking +anyone anything. Its content comes from the evidence package above and from an +**unversioned** file at `~/Working/Active/.harness-runs/rdpapp-m2/env.sh`; that +file remains the thing actually sourced, and this is the record of what it +contains. + +Paths below assume the layout of the machine this was run on +(`~/Working/Active/...`). Adjust them; nothing in the harness requires them. + +### 4.1 Preconditions, each verifiable + +```bash +# 1. Base lineage is not stale. This is what invalidated the abandoned +# 2026-08-04 attempt: work was based on a branch 121 commits behind the +# authoritative remote. +cd ~/Working/Active/rdpapp +git fetch --all +git rev-list --count harness/m2-base..origin/master # must be 0 + +# 2. The tree is clean. +git status --porcelain # must be empty + +# 3. Both remotes agree. `origin` (Forgejo) is authoritative; GitHub is a +# mirror. Confirm rather than assume. +git rev-parse origin/master github/main # must match + +# 4. The gateway answers, and with what. Claw Bay is frequently and broadly +# degraded: 8 of 42 models answered on 2026-08-05, all one family. +curl -s -H "Authorization: Bearer $THECLAWBAY_API_KEY" \ + https://api.theclawbay.com/v1/models | head -c 200 +``` + +### 4.2 The environment + +Run-scoped rather than written into a shell profile: the attempt is meant to be +discardable by deleting one directory, and a profile edit would outlive it. +Every model here is on Claw Bay; nothing routes to a local CLI agent. + +```bash +export HARNESS_ENDPOINT="https://api.theclawbay.com/v1" +export HARNESS_ROUTE_PRESET="claw-bay" +export HARNESS_API_KEY="${THECLAWBAY_API_KEY:?THECLAWBAY_API_KEY is not set}" + +# gpt only, by owner's decision 2026-08-05: measured 8 of 42 models answering +# and all 8 in this family. Chains are preference order, first that answers +# wins. gpt-5.6 leads because gpt-5.5 timed out the gateway origin (524) on +# long generations while 5.6 answered 200 throughout. +export HARNESS_PLANNER="gpt-5.6,gpt-5.4-mini" +export HARNESS_IMPLEMENTER="gpt-5.6,gpt-5.5,gpt-5.4" +export HARNESS_SURVEYOR="gpt-5.6,gpt-5.5" +export HARNESS_ASSESSOR="gpt-5.6" + +# NOT independent: same vendor as the implementer, because no second vendor is +# reachable. Every approval taken under this configuration is weaker than one +# taken when two vendors answer. Check GET /api/routes/health before trusting a +# review; when independence_possible turns true, move this to another vendor. +export HARNESS_REVIEWER="gpt-5.5,gpt-5.4" + +# No local CLI agent. Direct API mode is used, so this is belt and braces: if a +# session host is ever passed, the agent must still not be a +# subscription-backed local binary. +export HARNESS_AGENT_COMMAND="" + +# Shared so each item does not pay a cold Rust build in a fresh worktree. +export CARGO_TARGET_DIR="$HOME/Working/Active/.harness-runs/rdpapp-m2/cargo-target" +``` + +### 4.3 The run + +```bash +cd ~/Working/Active/apps/agent-harness +R=~/Working/Active/.harness-runs/rdpapp-m2 +. $R/env.sh + +uv run agent-harness --db $R/queue.sqlite run --project rdpapp-m2 \ + --work ~/Working/Active/rdpapp \ + --plan ~/Working/Active/rdpapp/docs/harness/M2-PLAN.md \ + --base harness/m2-base --no-push --reroute \ + --context-budget 300000 \ + --events $R/events.jsonl \ + --check 'cargo test -p rdpapp-models -p rdpapp-sessions -p rdpapp-gateway' \ + 2>&1 | tee $R/run-$(date +%H%M).log +``` + +Flags that are not decoration: + +- **`--no-push`** — rdpapp's `plan.md` calls GitHub a mirror and says the CI + cutover is **not authorised**. Work stays on local branches, so a discarded + attempt is branches to delete rather than a remote to clean. +- **`--reroute`** — without it the **stored** role map wins and the role-chain + environment above silently does nothing. The harness warns, and the warning + is emitted *before* the reroute applies, so it can read as a failure when it + is not. +- **`--base harness/m2-base`** — cut from `4dff7e2`, equal to both remotes. +- **`--context-budget 300000`** — the implementer chain's head is `gpt-5.6`, + which holds 372k. Anything above that silently exceeds the model. +- **`--check '…'`** — Rust only, and deliberately: `migration-tool` is + workspace-`exclude`d so its FluentGUI path dependency does not affect the + gated crates (do not add it), and a fresh worktree has no `node_modules`, so + `tsc`/`vitest` cannot start. **Do not gate on `cargo fmt --check`** — it + refused five otherwise-correct attempts in the abandoned attempt. #155 lets a + declared formatter's fix run and re-checks, but it is off unless + `apply_fixes` is set on the project. + +### 4.4 Retrying failed items without destroying `last_error` + +```bash +uv run python -c " +import sqlite3; c=sqlite3.connect('$R/queue.sqlite') +c.execute(\"update work set state='pending', owner=NULL, lease_until=0 where state='failed'\") +c.commit()" +``` + +**Keep `last_error`.** The harness feeds it into the next attempt's prompt, and +that is the only thing that makes a retry different from a repeat. Clear +`attempts` too **only** when the previous failure was the harness's fault +rather than the item's; otherwise the attempt ceiling stops meaning anything. + +### 4.5 Monitoring + +```bash +R=~/Working/Active/.harness-runs/rdpapp-m2 + +# What each item is doing. The stages are the whole story: an item that reached +# `checks` failed differently from one that died at `implement`. +grep -E "T[0-9]+ (started|edits_parsed|edits_rejected|applied|checks_|review_|committed|no_diff)" \ + $R/run-*.log | tail -30 + +# Queue state. +uv run python -c " +import sqlite3; c=sqlite3.connect('$R/queue.sqlite') +for r in c.execute('select item_id,state,attempts,disposition,reason_kind,substr(coalesce(last_error,\"\"),1,70) from work order by cast(substr(item_id,2) as int)'): print(r)" + +# What the models actually said (#190). +python3 -c " +import json +for l in open('$R/events.jsonl'): + e=json.loads(l) + if e.get('outcome')=='ok' and e.get('answer'): + print(e['model'], e['answer_chars'], 'redacted' if e['answer_redacted'] else '')" + +# Gateway health, per model, from traffic already made (#192). +grep -c "fell back" $R/events.jsonl +``` + +**The caveat that matters (#209): stored answers are redacted on the way into +the store.** Redaction is applied in `store.append` and `audit.append` and +never to prompts, so the model was given the real file — but text containing +`password: "…"`, which rdpapp's fixtures are full of, is rewritten before it is +recorded. **Do not diff a stored answer against a file and conclude the model +was wrong.** The `answer_redacted` flag tells you it happened; it does not tell +you where or how much. + +Symptoms and what they mean: + +| symptom | meaning | +|---|---| +| `edits_rejected … does not occur` | the model named text that is not there. Since #216 this is genuinely the model, not a stale worktree; the error now quotes the file back. | +| `review_rejected` | the gate working. Read the objection — on T1 it was correct and the brief was sharpened in response. | +| `LimitsExceeded` (loop) | ran out of turns. Check the refusal count first: a guard false positive used to consume ~38% of them (#217). | +| 429 / 503 storms | the gateway, not the harness. `--implementer a,b,c` chains past it; `survey` could not until #193. | +| nothing claimed, queue full | was a real deadlock (#218) — the claim scan stopped after one page. Fixed; a recurrence is a regression worth reporting. | + +### 4.6 Cleaning up an attempt + +```bash +rm -rf ~/Working/Active/.harness-runs/rdpapp-m2 # queue, events, logs, cargo cache +cd ~/Working/Active/rdpapp +git worktree list # remove any under .harness-work/ +git branch -D $(git branch --list 'harness/t*' 'adapter/*' 'spike/*' | tr -d ' *+') +``` + +`harness/m2-base` is the base and must survive. **The `harness/r1`–`harness/r7` +and `harness/base*` branches are from the ABANDONED 2026-08-04 attempt** — the +one based 121 commits behind — and are not this work. Do not read them as +evidence and do not build on them. + +--- + +## 5. What is proven, observed and tested + +| claim | word | how to check it | +|---|---|---| +| Queue and leases, dependency graph, holds, attempts, budgets, outcome taxonomy, patch-apply ladder, checks gate, reviewer gate, API/OpenAPI contract, redaction on the only two write paths, command guard, first-run/demo path | **tested** | `uv run pytest` | +| The core stays generic — no workload-specific paths, numbers or adapter imports | **tested** | `tests/test_generic.py` (`EXECUTION_PATH` is the authoritative list) | +| The store has no UPDATE and no DELETE | **tested** | the source-level assertion in the store tests | +| The four rdpapp-derived defects: edit-block rendering, stale worktree, guard false positives, claim-scan page deadlock | **tested** | regression tests landed with #216, #217, #218 | +| The service runs and is deployed inside AIDevEnv | **observed** | no preserved artefacts | +| An earlier supervised NGMS attempt and later direct calls exercised real agents and providers | **observed** | [`evidence/2026-08-03-04-ngms-first-sustained-run-v1.md`](evidence/2026-08-03-04-ngms-first-sustained-run-v1.md) — lacks a common run ID, complete configuration, checksums and a comparable follow-up | +| Four executor passes against rdpapp delivered nothing, and why each failed | **observed** | [`evidence/2026-08-05-06-rdpapp-m2-status.md`](evidence/2026-08-05-06-rdpapp-m2-status.md); the pass 3–4 attribution is hindsight and has not been confirmed by re-running against the fix | +| A loop reached `cargo test` green on rdpapp in 31 turns | **observed** | one item, once, through a standalone script — never through the harness, and it cheated in a way caught by hand rather than by a gate | +| Claw Bay answered on 8 of 42 models on 2026-08-05, all one family | **observed** | a sweep on one day; not preserved | +| Delivery rate | **neither** | no item has ever been delivered | +| Cost per merged item | **neither** | spend is not recorded per item; `pricing` has never been checked against a multi-turn role | +| Unattended reliability | **neither** | #33, #44, #51 have not run | +| Second-repository portability | **neither** | one repository has been attempted | +| Whether a read-only reviewer stays honest | **neither** | #226 is unbuilt; #84 is the experiment | +| Whether the reviewer's verdicts in this window are trustworthy | **not known** | every verdict recorded was same-vendor, because no second vendor was reachable, and nothing attaches that caveat to the verdicts themselves | + +Nothing is in the **proven** column. That is not modesty; it is the definition +— nothing about live behaviour has been measured against a stated criterion +with a published denominator. + +"No failures observed" is not the same as "the requirement was exercised".