Skip to content

The examples README's annotated session quotes lines the transcript no longer contains #144

Description

@macanderson

Problem

examples/README.md's "A complete stdio session (annotated)" walks through the transcript step by step and quotes a JSON line for each step. Those quoted lines are hand-copied, nothing checks them, and they have drifted from examples/full-stdio-session.ndjson.

Concretely, as of PR #138:

  • The section used to open with "contains exactly these five lines"; the file had seven lines before that PR and has nine after. (PR feat(wire): carry provenance attestations on the frames envelope #138 replaced the count with a pointer to the file, so the sentence is no longer wrong — but the quoted bodies below it still are.)
  • The quoted query and frames lines carry no id, while the file's do (q1), so the walkthrough silently fails to demonstrate correlation, which is the one thing H4 asks a provider to get right.
  • The quoted frames line omits content_digest, recorded_at and relations, all of which the file's frames carry, and its token_cost values (41, 23) do not match the file's (32, 18) — so the prose claiming they "sum to 64" describes neither.
  • The verify/verified exchange in the file has no step in the walkthrough at all.

The rest of the README is fine; this is one section.

Why it matters

The README says these are "the exact JSON shapes a host and provider exchange — useful when implementing a provider in any language, because you can diff your own output against them." A provider author who diffs against the README rather than the file gets an answer that is wrong in a way schema/validate-examples.py cannot see, because the validator reads the .ndjson and the .json, never the fenced blocks in the README.

Reproduce

python3 -c "
import json
for i, l in enumerate(open('examples/full-stdio-session.ndjson'), 1):
    if l.strip(): print(i, json.loads(l)['type'], json.loads(l).get('id'))
"

then compare against the fenced blocks in examples/README.md.

Constraints

  • schema/validate-examples.py already extracts and validates fenced jsonc blocks from `SPEC.md` (section 4 of that script), including a `//`-comment stripper and a placeholder substitution table. The same machinery can read `examples/README.md`'s json blocks — but validating them against the schema only proves they are well-formed envelopes, not that they match the file. The stronger check is equality against the corresponding transcript line.
  • The walkthrough deliberately elides some fields for readability, so a byte-equality check would force the prose to carry full frames. Decide whether the section should quote the file verbatim (simplest to enforce) or stay abridged with a weaker check.

Done looks like

  • The quoted blocks agree with examples/full-stdio-session.ndjson, including the correlation ids, and the walkthrough covers every message the file contains.
  • Whatever agreement is chosen is enforced — a check that fails when the file and the README part company, not a promise to remember. Extending schema/validate-examples.py is the obvious home; it already runs in CI as "schema validates the examples".
  • Any number stated in the prose ("sum to 64") is either correct or replaced by something that cannot go stale.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Next cycle

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions