Skip to content

Half the spec's conformance requirements are missing from the page that mirrors them #158

Description

@macanderson

The problem

GOVERNANCE.md names docs/protocol-surface.md's
§ Conformance requirements
as the mirror of SPEC.md's requirement tables. The mirror is missing more than
half of them, and there is nothing that would notice.

SPEC.md states 57 numbered requirements. docs/protocol-surface.md restates
23. The 34 it omits include every requirement the conformance suite grew a
check for after the mirror was written:

$ rg -o '^\| \*\*([A-Z][0-9]+)\*\*' -r '$1' SPEC.md | sort -u > /tmp/spec.txt
$ rg -o '^\| ([A-Z][0-9]+) \|'      -r '$1' docs/protocol-surface.md | sort -u > /tmp/surface.txt
$ comm -23 /tmp/spec.txt /tmp/surface.txt | tr '\n' ' '
A1 A2 A3 B3 B4 C7 C8 D3 D4 E1 F10 F4 F5 F6 F7 F8 F9 G1 G2 G3 G4 H4 P1 P2 P3 P4 P5 Q1 U1 U2 U3 U4 X1 X2

A provider author who reads that page and stops has not been told about the
kinds filter (Q1), the embedding-fingerprint rejection (E1), the canonical
token count (B3), the frame cap (B4), the correlation echo (H4), graph edges
(G1–G4), representation invariants (P1–P5), provenance attestation (F6–F9), or
the forward-compatibility rules (U1–U4) — and every one of those is enforced by
a conformance check that will fail them.

The page's own header already says SPEC.md wins where the two disagree, so the
contract is clear. What is missing is the second half: nothing keeps the mirror
from silently falling behind.

Files

  • docs/protocol-surface.md — the mirror, in its ## Conformance requirements
    section.
  • SPEC.md — the normative source.
  • GOVERNANCE.md § "What counts as a normative change" — where the mirror
    relationship is asserted.

Reproduce

Run the two rg invocations above and diff the id lists.

Constraints found

  • A count in prose is a promise nobody keeps, so a guard should compare the two
    id sets rather than a number.
  • The mirror is deliberately a summary, not a copy: several of its rows fold
    a SPEC row into plainer wording. So the guard should check that every SPEC id
    has a row, not that the two texts match.
  • schema/validate-examples.py already parses SPEC.md for fenced blocks, so
    there is a precedent for a script that reads the spec structurally.

Definition of done

  • docs/protocol-surface.md's conformance-requirement tables carry a row
    for every numbered requirement in SPEC.md, each naming what enforces or
    verifies it.
  • A gate script fails when a SPEC.md requirement id has no row in
    docs/protocol-surface.md, and is wired into CI on the paths that reach
    either file.
  • The guard has a self-test that proves it can fail (add a requirement to a
    fixture spec, see it caught), in the shape the repo's other Python gates
    use.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Next cycle

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions