Skip to content

fix(trace): make the gap rows visible that a vacuous capability was hiding (#1268) - #1281

Merged
dc0sk merged 1 commit into
mainfrom
fix/1268a-requirements-data
Sep 5, 2026
Merged

fix(trace): make the gap rows visible that a vacuous capability was hiding (#1268)#1281
dc0sk merged 1 commit into
mainfrom
fix/1268a-requirements-data

Conversation

@dc0sk

@dc0sk dc0sk commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Part A of #1268 — the data half. Mechanical and maintainer-specified; the checker change is PR B.

What was wrong

trace.py clears REQ-GAP for any requirement whose covered_by is non-empty. It never asks
whether the capability implements anything — so a capability with no code and no tests cleared
the gap exactly as well as a real one. Five were in that state, and between them "covered" 16
requirements that had no other coverage.

What this changes

  • REQ-SEC-09 → gap row, per the maintainer's ruling. Done on both sides of the join in one
    edit
    (covered_by: [] and out of CAP-47.satisfies): removing only one side trades REQ-GAP
    for BIDIR-DRIFT and hides the same fact under a different name. relay_integration.rs tests
    deny-lists and route scoring; the only per-hop trust-level admission in the workspace is mesh's
    trust_filter_allows, which after Mesh imports a peer's self-asserted trust_state=0x00 as TrustLevel::Verified into the cache its relay policy enforces against #1253 admits nobody.
  • CAP-72/73/74/75 filled from the acceptance table — front-end conditioning, device resolution and
    multi-mode receive, PTT backends, MFSK16. Every cited file was verified to exist before citing it.
  • CAP-69 deleted. Genuinely unimplemented — REQ-BW-* are 1.x wide-channel work — so its seven
    requirements become honest gap rows. This one row is the join defect in miniature: the matrix
    column already said "📝 planned" while the yaml asserted coverage.

The proposal for this issue was rejected before implementation

docs/dev/reviews/2026-09-05-1268-traceability-join.md. I proposed a new ratchet baselining the 120
membership-only rows. Three findings killed it:

  1. The ratchet already exists. trace-grandfathered-ids.txt (221 ids, shrink-only) plus
    NOT-GRANDFATHERED as a hard error already means a new requirement cannot be baseline, so it
    must be enforced, which already requires a VERIFIES marker whose test passed in the last real
    gate. I proposed an imitation of a mechanism I had not looked for.
  2. The premise that a ratchet pays down is false here, measured from git:
    reachability-baseline.txt went 456 → 511 in four weeks — grew 12 %, with exactly one shrink
    (fix(gate): the reachability ratchet counted compound-cfg test modules as production (#1270) #1272, the day before). A second 120-row file would be the same ids under a second lifecycle.
  3. It catches 0 of the 3 rows that motivate the issue. For REQ-MAC-02 there is
    csma_blocks_broadcast_when_dcd_busy; for REQ-SEC-09 there is literally
    fn trust_policy_rejects_at_hop() — each named for its requirement, passing, and depending on the
    cited code, so a marker lands on them cleanly while the actual property stays untested. And the
    QSY row is not the same defect: REQ-REG-13/14 never mention signing, so no join was wrong —
    the false claim lived in a capability description, which no requirement-side check can reach.

The sweep took three rounds, and the checker caught every one

Deleting the capability block left seven covered_by references, a matrix column, a summary line and
an entry in the grandfathered-id list. Then my first fix re-created the dangling reference in
prose
— I wrote the deleted id into the explanatory cell text, and the id scanner reads prose. That
is the #1192 lesson met from the other side. The note is now hoisted above the table and omits the id
deliberately, saying why.

Results

TRACE: PASS
GATE: PASS 8fb3f6f788931dcb1c34a2b7f4a8634c4c7381d0 clean 20260905T162656Z
        suites=324 tests_passed=2456 tests_failed=0

Gap rows 25 → 33. Capabilities with empty code AND tests: 5 → 0.

Left for PR B, with the boundary stated rather than assumed

Five capabilities still satisfy a requirement while listing no tests — CAP-58, 67, 68, 70, 71. The
EMPTY-CAP rule must say which it means: no code and no tests (0 today) or satisfies something
and lists no test
(5 today). My original item 3 was fitted to the five I happened to count, and
saying so is the point.

Also split out: #1279scripts/req-mutation.sh is the project's only tier-2 instrument (does a
bound test depend on the bound code?) and runs in no workflow at all.

Review: docs/dev/reviews/2026-09-05-1268-traceability-join.md

🤖 Generated with Claude Code

https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6

…iding (#1268)

The checker clears REQ-GAP for any requirement with a non-empty covered_by, so a capability with no
code and no tests cleared it exactly as well as a real one. Five capabilities were in that state and
between them "covered" 16 requirements that had no other coverage.

Data half only, mechanical:
- REQ-SEC-09 becomes a gap row per the maintainer's ruling, on BOTH sides of the join in one edit
  (covered_by: [] and out of CAP-47.satisfies) — removing one side only trades REQ-GAP for
  BIDIR-DRIFT and hides the same fact under another name.
- CAP-72/73/74/75 filled from the acceptance table, each cited file verified to exist first.
- CAP-69 deleted: genuinely unimplemented (REQ-BW-* are 1.x wide-channel work). Its seven
  requirements are now honest gap rows. The matrix column already read "planned" while the yaml
  asserted coverage — the join defect in a single row.

The proposal for this issue was rejected before implementation (see the review artifact): the
ratchet I wanted to build already exists as trace-grandfathered-ids.txt + NOT-GRANDFATHERED, the
premise that a ratchet pays down is false here (reachability-baseline grew 456 -> 511 in four weeks,
one shrink), and it would have caught none of the three rows that motivate the issue.

The sweep took three rounds and the checker caught each. Deleting the capability left seven
covered_by references, a summary line, a matrix column and a grandfathered-id entry — and my first
fix re-created the dangling reference IN PROSE, because the id scanner reads prose. That is the
#1192 lesson met from the other side; the note is now hoisted above the table and omits the id
deliberately, saying why.

Results: TRACE: PASS. Gap rows 25 -> 33. Capabilities with empty code AND tests: 5 -> 0.

Left for PR B with its boundary stated rather than assumed: five capabilities still satisfy a
requirement while listing no tests (CAP-58, 67, 68, 70, 71).

Refactors: CAP-77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant