A test failure's cause was read off its name, not its traceback — twice, the second time inside the fix for the first

[topij]

Two occurrences in one session (2026-09-07, PR #701 and its three review rounds).
Severity M. Filed rather than logged because it carries a reproduction, a named
mechanism and a proposed rule; the recurrence — the second instance landing inside the
commit written to fix the first — is the finding.

The shape

A test failure was attributed to a cause read off the test's name, when the traceback
said something else. The name is right there in the FAILED line, it reads like a
statement of what the test asserts, and grouping by it produces a clean, plausible
classification. Getting the real cause costs another step — parsing the FAILURES section
and pulling each block's terminal exception — and the cheap answer looks exactly like the
expensive one.

Occurrence 1

Classifying the 79 failures in a retained installed-suite log, test_portability.py's
seven were recorded as "runtime-parity-contract assertions about the kit's own workflows
and adapters" and filed under #534 cause 1. That reading came entirely from names like
test_runtime_parity_contract_covers_workflows_and_adapters.

The tracebacks say otherwise. All seven carry
AssertionError: assert 'docs/agentic-dev-kit/workflows/wrap-up.md' in '\nPreserve this adapter.\n' — the fixture's deliberately preserved fixture-owned wrap-up adapter, the
same mechanism behind most of test_kit_doctor.py's failures in the same log, and not a
parity-contract failure at all.

It also mattered for routing: #534's body lists test_portability.py under cause 3,
so the file sits under two causes and the name-derived label picked neither on evidence.

Occurrence 2

The same session's correction commit — written specifically because a review lens had
just caught an unearned completeness claim — attributed the same seven failures the same
way, because the correction re-checked the count and not the cause. It took a second
lens round to catch it.

Why this is worth a rule rather than a reminder

Both were caught by a review lens pulling the tracebacks, never by the classifying agent.
The cheap signal is also the more legible one — a test name is prose written to be read,
while a traceback is noise you have to work through — so the incentive points the wrong way
under time pressure, which is exactly when a large log gets classified.

This is #205's family (a verification that fails into the reassuring answer) with the
harness removed: no instrument malfunctioned, the agent simply sampled a proxy. #205's
remedies are all about making harnesses assert harder; none of them reaches this, because
there is no harness.

Proposed

• A failure's cause is its traceback's terminal exception, never its test name. A
  classification that groups by name is an index, not a finding, and should say so.
• Where a document assigns causes to a set of failures, it should state how many blocks
  it actually opened — the gap between "failures counted" and "tracebacks read" is the
  thing that goes unstated in both occurrences above.
• Worth considering for fallback-review-panel.md: the correctness lens caught this twice
  by doing exactly this, so a line in Verified clean asking a lens to say whether it
  read tracebacks or names would make the distinction routine rather than incidental.

Related: #205 (the harness-side version), #54 (name the command and its actual
result), #534 (where both occurrences landed).