Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c744a66
docs(testing): retire five dead SMTP-posture assertions in the alerti…
wshallwshall Aug 15, 2026
6012781
docs(testing): retire two stale gap rows in the pipeline and connecto…
wshallwshall Aug 15, 2026
5faf551
docs(testing): two P0 HA/DR rows say "runs nowhere" about suites that…
wshallwshall Aug 15, 2026
6111dfe
docs(testing): the HTTP listener does authenticate -- correct a P1 ro…
wshallwshall Aug 15, 2026
375e543
docs(testing): the lagging-ADR-status count is four, not five (BACKLO…
wshallwshall Aug 15, 2026
dae4d1f
docs(testing): mark the HA catalog-drift row verified-live and fix it…
wshallwshall Aug 15, 2026
04ddf1a
docs(testing): verify the leaderless-cluster P0 row and re-anchor its…
wshallwshall Aug 15, 2026
4352494
docs(testing): close a P0 that describes a failure the engine refuses…
wshallwshall Aug 15, 2026
1568fac
docs(testing): verify the cross-store divergence P0 and sharpen what …
wshallwshall Aug 15, 2026
a72e2be
docs(testing): a P0 about unrun suites was wrong, and the real defect…
wshallwshall Aug 15, 2026
e07018a
docs(testing): verify the poison-crash P0 and locate exactly where it…
wshallwshall Aug 15, 2026
039d731
docs(testing): narrow the leak-gate floor P0 to the one claim that su…
wshallwshall Aug 15, 2026
c2cdfc1
docs(testing): close a security P0 whose guard already exists (BACKLO…
wshallwshall Aug 15, 2026
b5a28ce
docs(testing): re-verify the security-wave coverage P0 with a positiv…
wshallwshall Aug 15, 2026
cc5faca
docs(testing): the suspected Kerberos SPN defect is CONFIRMED, and wo…
wshallwshall Aug 15, 2026
715df2a
docs(testing): confirm the AD account-state P0 and re-point all four …
wshallwshall Aug 15, 2026
95f48e2
docs(testing): grade the ADR 0142 row only as far as this repository …
wshallwshall Aug 15, 2026
3c4acd8
docs(testing): confirm the mock-seam P0, and correct the one clause a…
wshallwshall Aug 15, 2026
480869d
docs(testing): confirm the PHI retention-gate bypass and re-point it …
wshallwshall Aug 15, 2026
b16e5c1
docs(testing): narrow the at-rest AAD row from "barely exercised" to …
wshallwshall Aug 15, 2026
d027fba
fix(testing): repair the R2 row I split into six columns in eb32618f …
wshallwshall Aug 15, 2026
d2de228
fix(testing): restore the G4 row's missing column, and sweep all 172 …
wshallwshall Aug 15, 2026
5a13b30
docs(testing): confirm the per-connection purge coverage gap, and sho…
wshallwshall Aug 15, 2026
f70b20e
docs(testing): confirm the API wire-contract P0, and flag two tokens …
wshallwshall Aug 15, 2026
d47da76
docs(testing): confirm no API route runs on a server DB, with both po…
wshallwshall Aug 15, 2026
4935424
docs(testing): a control exists for the false-promote P0, and it comp…
wshallwshall Aug 15, 2026
98b8ae1
docs(testing): confirm the fingerprint's environments/ blind spot aga…
wshallwshall Aug 15, 2026
d6b5487
docs(testing): confirm the dual-control promote renders as success (B…
wshallwshall Aug 15, 2026
7b1ddae
docs(testing): confirm the restart-publish attribution gap, re-pointi…
wshallwshall Aug 15, 2026
03aa8f2
docs(testing): confirm the split-config row, and record that three P0…
wshallwshall Aug 15, 2026
da3b587
docs(testing): correct a false claim I committed in f21d0e33 -- the C…
wshallwshall Aug 15, 2026
ec6aa85
docs(testing): audit my own committed counts, and correct the second …
wshallwshall Aug 15, 2026
b01551d
docs(testing): confirm the composite-pipeline row, and name why it ha…
wshallwshall Aug 15, 2026
1164f3c
docs(testing): the security warning IS tested, and the real config ga…
wshallwshall Aug 15, 2026
90edf4f
docs(testing): confirm the unreachable-settings-sections P0, exactly …
wshallwshall Aug 15, 2026
ef16a7b
docs(testing): confirm the shipped gate runs in no CI leg, past a col…
wshallwshall Aug 15, 2026
2969dc1
docs(testing): the check SCHEMA is pinned, the ROSTER is not -- sharp…
wshallwshall Aug 15, 2026
807a5e9
docs(testing): the lens fixtures are unguarded, and the guard next do…
wshallwshall Aug 15, 2026
5b14cd9
docs(test-plan): re-establish the ch06 blank-segment P0 by AST, not l…
wshallwshall Aug 15, 2026
2b7450d
docs(test-plan): confirm the ch06 escaped-NUL P0 by reproduction, and…
wshallwshall Aug 15, 2026
5636852
docs(test-plan): ch14 -- an end-line-number sold as a length, and a c…
wshallwshall Aug 15, 2026
0d19b1f
docs(test-plan): ch14 lying-tray confirmed end to end, and named as a…
wshallwshall Aug 15, 2026
b6029d7
docs(test-plan): WITHDRAW the ch17 'engine sharding has zero CI execu…
wshallwshall Aug 15, 2026
e703c85
docs(test-plan): ch17 sizing P0 withdrawn as stale; the benchmark-top…
wshallwshall Aug 15, 2026
aef47f9
docs(test-plan): ch18 -- two P0s confirmed, and a composition neither…
wshallwshall Aug 15, 2026
f5c99fe
docs(test-plan): ch18 PHI row traced to the shared helper; a crash-on…
wshallwshall Aug 15, 2026
055ff04
docs(test-plan): ch11 -- both P0s confirmed, and the app.js one is wo…
wshallwshall Aug 15, 2026
23df07a
docs(test-plan): ch15 G2 and G3 confirmed; G3 anchor off by one, and …
wshallwshall Aug 15, 2026
88b852f
docs(test-plan): ch15 G5 -- one write site, not two, and all three an…
wshallwshall Aug 15, 2026
e05cb94
docs(test-plan): ch15 G1 -- no live drift, and a naive-guard trap I w…
wshallwshall Aug 15, 2026
bde94fa
docs(test-plan): ch05 accounting verified exact, and the two-table P0…
wshallwshall Aug 15, 2026
c572fda
docs(test-plan): audit my OWN corrections -- two welded claims I carr…
wshallwshall Aug 15, 2026
a09b094
docs(test-plan): finish the self-audit -- two withdrawn rows were con…
wshallwshall Aug 15, 2026
ffcc2be
docs(test-plan): ch12 advisory-ide-leg P0 confirmed, both anchors re-…
wshallwshall Aug 15, 2026
0613055
docs(test-plan): my ch17 r112 WITHDRAWAL REASON was itself unverified…
wshallwshall Aug 15, 2026
cd0965f
docs(test-plan): ch12 exec-gate and full-replace P0s confirmed (BACKL…
wshallwshall Aug 15, 2026
4594e2b
docs(test-plan): ch12 -- the auth gate is TESTED, its CALL SITE is no…
wshallwshall Aug 15, 2026
dd4f959
docs(test-plan): ch12 rows 124/125/129/130 confirmed, every anchor ex…
wshallwshall Aug 15, 2026
ee76b66
docs(test-plan): ch13 webview-mirror anchors re-pointed; the drift is…
wshallwshall Aug 15, 2026
8988816
docs(test-plan): ch13 rows 65/67 confirmed; fixture staleness is WIDE…
wshallwshall Aug 15, 2026
bda30f5
fix(test-plan): repair two ch13 rows I broke with unescaped pipes in …
wshallwshall Aug 15, 2026
b11e805
docs(test-plan): sweep ch01, the chapter I nearly wrote off on a P0 c…
wshallwshall Aug 15, 2026
388ec09
docs(test-plan): re-point the first anchor off the blank-line worklis…
wshallwshall Aug 15, 2026
17f5aa9
docs(test-plan): second anchor off the blank-line worklist (BACKLOG #…
wshallwshall Aug 15, 2026
627b8dd
docs(test-plan): re-point six decayed ci.yml anchors in the ch17 shar…
Aug 26, 2026
0467580
backlog: record #1100's progress without claiming it is finished (BAC…
Aug 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6588,6 +6588,18 @@ contention; recorded in the #1095 handoff note rather than lost.

## 1100. The master test plan asserts document contradictions that were resolved before it was written

> **PROGRESS 2026-08-26 (lander), NOT A CLOSURE -- this item stays OPEN.** Landed the documentation
> half of stranded PR #433 on a fresh branch (the original ref carried an owner-armed auto-merge that
> would have fused it to six unrelated engine commits; disarmed and split per the owner's 2026-08-26
> ruling, ADR 0165 authorship note below). Measured directly against this landed diff, not carried
> forward from the branch's own claim: 18 chapter files, +90/-75, matching the branch's own count
> exactly. **WHAT IS NOT ESTABLISHED, and it is why this stays open: nobody has verified that the NINE
> SITES THIS ITEM NAMES are among those revised.** Closing needs a pass that walks those nine sites
> and checks each against the now-landed chapters. **Author's note, ADR 0165:** this progress note is
> Lander-authored on the builder's behalf -- the 64 underlying commits are the builder's own verified
> work, cherry-picked unmodified; only this ledger paragraph is mine, because ledger authorship on an
> open item is reserved to the Dispatcher or Lander.
>
> 🔢 **Re-scored 2026-08-20 -> P2.** Value **5/10** · Difficulty **3/10** · _fill-in_. The stale claims are not confined to narrative: :593 is item 4 of the chapter's numbered release sign-off list (the three contradicting documents agree with the code) and :606 declares Blocks: ALERT-08, 09, 58, 67 as the chapter's only P0 with a real security consequence, so a plan executor is held on a question the code answered on 2026-08-02. Difficulty is nine content edits, each requiring the claim re-checked against code first, at least one possibly warranting deletion, with the item forbidding a scripted pass. _(was 6/10 · 3/10.)_
>
> **Filed 2026-08-07 - not started.** Nine sites in the master test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ is never redirected to a committed file, a ticket, or a CI log.
message families the engine handles: `adt`, `oru`, `orm`, `oml`, `orl`, `mdm`, `mfn`, `dft`, `bar`,
`ras`, `rde`, `siu`, `vxu`, `documents`, with `all_types.py` as the registry. Two entry points:

- `messagefoundry generate` — the CLI subcommand ([`__main__.py:349`](../../../messagefoundry/__main__.py)).
- `messagefoundry generate` — the CLI subcommand ([`__main__.py:350`](../../../messagefoundry/__main__.py) — **RE-POINTED 2026-08-15 (BACKLOG #1100), was `:349`, which is now blank**; the subparser is `generate = sub.add_parser(` at `:350` with the name `"generate"` on `:351`, and its flags follow at `:353-361`).
- `python -m messagefoundry.generators.adt [--triggers A01,A04] [--count N] [--out DIR]` — the ADT
corpus builder: **57 triggers across 25 message structures** (A01–A62 excluding the A19 query event
and reserved A56–A59), with segment order and the allowed segment set driven by **hl7apy's own
Expand Down
13 changes: 9 additions & 4 deletions docs/testing/master-test-plan/02-pipeline-reliability.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,13 @@ chapter or any downstream one:
run `test_load_failover_{sqlserver,postgres}` with the harness setting **no** `claim_mode`, i.e.
under the pooled default, and hard-gate zero acknowledged loss + `lane_inversions == 0`.
4. **Postgres 2-engine crash-and-restart recovery.** `FCP:STORE-10`'s "not built" is stale —
`tests/test_shard_recovery_postgres.py` exists (4 tests). Its problem is that it **runs nowhere**
(PIPE-01), not that it is missing.
`tests/test_shard_recovery_postgres.py` exists (4 tests). **CORRECTED (BACKLOG #1100): it does not
"run nowhere".** It runs in `ci.yml`'s `postgres-store` job, step *"Run the failover +
engine-shard recovery suites on real Postgres"* — but that job is gated on
`schedule || workflow_dispatch || changes.outputs.serverdb == 'true'`, so it does **not** run on a
PR that touches no server-DB path. The accurate statement is *runs only on the server-DB
path-gated leg*, which is what PIPE-01 is actually for. "Runs nowhere" understated the coverage and
"runs in CI" would overstate it; only the gated form supports a decision about PIPE-01.
5. **`accepts=` static fail-closed validation** — `FCP:PIPE-9` is closed by
`test_accepts_seam.py` (the three static-validation negatives).
6. **Purity replay-equality harness** — `FCP:PIPE-14`'s replay half is closed by
Expand All @@ -137,8 +142,8 @@ chapter or any downstream one:

| Risk | Failure mode | Blast radius | Detected today? | Priority |
|---|---|---|---|---|
| Six `MEFOR_TEST_*`-gated pipeline suites (~36 tests) are named in **no** workflow step | A live regression in `batch_handoff_statements` (DEFAULT-ON, SQL-Server-only, restructures the route/transform handoff DML) or in ownership-scoped **engine-shard** recovery ships green | Silent data loss, wrong disposition, or **duplicate PHI deliveries across engine shards** on the production-scale backend | **No.** `ci.yml:424-428`'s own comment says the path gate "MUST list every file the sqlserver/postgres steps run"; these are in neither the steps nor the gate regex | **P0** |
| No poison-crash attempts ceiling on the **default split** ingress/routed path | A hard abort with no Python exception (C-extension segfault, OOM kill) inside `route_only`/`transform_one`/handoff is caught by neither the internal-error policy nor the ADR 0070 T17 handler; `reset_stale_inflight` re-pends the head, the lane re-runs, the process dies again | Lane head-of-line blocked **forever** across NSSM/supervisor restarts; nothing dead-letters; every message behind it stops flowing. The G6 ceiling exists only inside `if inline:` (`wiring_runner.py:4475-4496`), and its own comment at `:4479-4480` states no ingress/routed path enforces `max_attempts` on the split path. `supervisor.py:22` lists "restart backoff / crash-loop breaker" as deferred | **No** — the ADR 0087 sandbox that would contain it is default OFF, and no test drives a hard abort on the split path | **P0** |
| Gated suites the path gate PULLS but no step RUNS — **the inverse of the invariant `ci.yml:983` states** | **CORRECTED (BACKLOG #1100, re-measured 2026-08-15): the original evidence was false in both parts.** It said `batch_handoff_statements` and engine-shard recovery "are in neither the steps nor the gate regex". **Both are in both**: `tests/test_adr0075_batch_sqlserver.py` (`ci.yml:1375`) and `tests/test_shard_recovery_sqlserver.py` (`:1371`) run in the `sqlserver-store` step *"Run the engine-shard + statement-dispatch suites on real SQL Server"*, and `adr0075`/`shard_recovery` are both in the gate alternation at `:993`. The cited anchor `:424-428` had drifted onto a comment about CI run timings; the real comment is `:983-988`. **THE RESIDUAL GAP IS REAL AND SHARPER:** of **54** `MEFOR_TEST_*`-gated suites, **19 are named in no workflow step**, and every server-DB step invokes **explicit files** (never a directory), so an unnamed suite runs nowhere on a real backend. **Three of the 19 are MATCHED BY THE GATE REGEX** — `test_adr0157_fence_scope` (8 tests), `test_sqlserver_sync_handoff_offline` (6), `test_adr0071_fusion_wiring` — each named **0** times anywhere in `.github/` (positive control: `shard_recovery_sqlserver` = 1) | A change to those three **pulls the expensive server-DB legs and then never executes them** — the legs go green having not run the suite the gate fired for. `test_adr0157_fence_scope` is the sharpest: ADR 0157 fence scope, 8 tests, no real-Postgres execution anywhere | **Partly.** `ci.yml:983-988` states only ONE direction — the alternation must list every file the steps run. **The inverse is unstated and violated three times**: every file the alternation lists should be run by some step | **P0** |
| No poison-crash attempts ceiling on the **default split** ingress/routed path | A hard abort with no Python exception (C-extension segfault, OOM kill) inside `route_only`/`transform_one`/handoff is caught by neither the internal-error policy nor the ADR 0070 T17 handler; `reset_stale_inflight` re-pends the head, the lane re-runs, the process dies again. **RE-VERIFIED LIVE 2026-08-15 (BACKLOG #1100), and the mechanism is sharper than "no ceiling exists":** the `queue` table DOES carry an `attempts` column (`store.py:1319`) for **every** stage, and a `max_attempts` ceiling IS enforced — but **only inside `mark_failed()` (`:5756`) and `mark_batch_failed()` (`:5816`)**, which are the *caught-Python-exception* paths. A hard abort calls neither, and recovery instead runs `reset_stale_inflight` (`:5948`, 69 lines) which references **neither `attempts` nor `dead_letter`** — verified by reading the whole function, not a window. So the row is right, and the fix has a precise location: **the increment must happen at CLAIM time or in `reset_stale_inflight`, not on the failure path**, because the failure path is exactly what a hard abort skips. Note also that G6 is an **outbox-stage** mechanism throughout (every reference reads `OutboxItem.attempts`), which is why it does not cover this | Lane head-of-line blocked **forever** across NSSM/supervisor restarts; nothing dead-letters; every message behind it stops flowing. The G6 ceiling exists only inside `if inline:` (`wiring_runner.py:4475-4496`), and its own comment at `:4479-4480` states no ingress/routed path enforces `max_attempts` on the split path. `supervisor.py:22` lists "restart backoff / crash-loop breaker" as deferred | **No** — the ADR 0087 sandbox that would contain it is default OFF, and no test drives a hard abort on the split path | **P0** |
| `W25:S3.4` / `W25:S2.7` and `harness/config/coverage.py:17` assert an **AE NAK** for a post-ACK Handler raise | Under ACK-on-receipt the AA fires at the ingress commit (`wiring_runner.py:3726-3745`) before the Router or Handler runs; a Handler raise **cannot** NAK | A human running `W25:S3.4` either fails a correct system or records a NAK that never happened. `harness/scenarios.py:63` already expects only disposition `error`, so the docs contradict both the code and the harness they instruct the tester to run — on the single most partner-visible behaviour change in ADR 0001 | **No** — the docs *are* the detector, and they are wrong | **P0** |
| No live end-to-end committed-transactions-per-message ceiling in CI | An accidental extra handoff commit doubles `committed_txns/msg` | Passes every test; surfaces only as a production capacity shortfall. ADR 0051 sizes capacity on `3 + 2H + 2N`; the counters already exist (`store/base.py:220-234`, surfaced at `api/app.py:4142-4143`) | Partly — `test_txn_per_message_cost_model.py` pins the **model** over a recording connection, not the **live** counter through a real runner | P1 |
| A refactor hoists the lookup-runner `ExitStack` to wrap `route_only` | Routers silently gain live `db_lookup` access; the at-least-once re-run invariant breaks and an unbudgeted live DB read lands on the routing hot path for every message | Non-pure Routers ⇒ duplicate/divergent downstream side effects on every crash re-run | **No** for `db_lookup` — the FHIR twin exists (`test_fhir_lookup.py:545`); the db_lookup side has only "no active runner" (`test_db_lookup.py:104`) and "dry-run raises" (`:310`). The guarantee is *positional* (`wiring_runner.py:5027-5031`), not structural | P1 |
Expand Down
Loading
Loading