You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The conformance suite runs 13 checks. Six documents say it runs five or seven,
one document names a check that does not exist, and one page's whole structure
is built around the wrong number.
contextgraph-conformance/src/lib.rs declares 13 CHECK_* constants and run_conformance emits all 13. This repository's own committed report agrees:
$ python3 -c "import json;d=json.load(open('registry/contextgraph-example-docs.report.json'));print(len(d['checks']),[c['name'] for c in d['checks']])"
13 ['handshake', 'consent-scope', 'frame-validity', 'verify-honesty', 'budget-honesty',
'as-of-temporal', 'kinds-filter', 'anchor-relevance', 'provenance-fixture-consistency',
'shutdown-clean', 'malformed-input-tolerance', 'embedding-fingerprint', 'correlation']
Where it says otherwise
$ rg -n "five checks|5 checks|seven checks|7 checks" --glob '!CHANGELOG.md' .
sdk/README.md:15:| TypeScript | … | ✅ conformant — passes all 7 checks in CI |
sdk/README.md:16:| Python | … | ✅ conformant — passes all 7 checks in CI |
sdk/README.md:17:| Go | … | ✅ conformant — passes all 7 checks in CI |
sdk/README.md:26:`conformance-external.sh` asserts the provider is **green** (all seven checks).
README.md:76:| **Conformance** | … | `contextgraph-conformance`, 5 checks |
docs/overview.md:65:| **Conformance** | … | `contextgraph-conformance`, 5 checks |
docs/running-conformance.md:8:## The five checks
docs/protocol-advantages.md:94:| **Conformance verification** | … | `contextgraph-conformance` — 5 checks that deliberately trip each failure mode |
Three other files already say 13 (docs/registry.md, docs/reference-providers.md, docs/composition-walkthrough.md), so the tree contradicts itself.
docs/running-conformance.md is the worst of them
It is not one stale number — the page is organised around it:
The heading is ## The five checks, followed by a five-row table listing handshake, frame-validity, budget-honesty, shutdown-clean, malformed-input-tolerance. Eight checks are missing, including consent-scope, verify-honesty, kinds-filter, anchor-relevance and correlation.
Its sample transcript ends CONFORMANT — 5 passed, 0 skipped. A real run of
the bundled fixture prints 13.
Its ProviderTarget section says a stdio target runs "all five checks".
Its own closing paragraph discusses provenance-fixture-consistency and the malformed-digest/stale-digest misbehaviour modes — none of which appear in
its own table. The page contradicts itself within one screen.
Its sample handshake evidence line prints query kinds=["doc"], upsert=false, graph=false. describe_handshake in contextgraph-conformance/src/lib.rs formats "… query kinds={:?}, graph={}" — there is no upsert= segment, because
ADR 0004 deleted Capabilities::upsert.
docs/protocol-surface.md names a check that does not exist
FanOut::usage_report exists. There is no check named usage-report — not
among the 13 provider CHECK_* constants, the 9 HCHECK_* host constants, or
the 4 CCHECK_* composition constants. rg 'usage-report' over the tree
returns only prose.
This is the same shape #22 already recorded for §G2 ("Verified by frame-validity" with no G2 implementation) — a "Verified by" cell naming a
check nobody wrote. That one has since been fixed in the code; this one has not
been noticed. Small correction in the same table: row D1 cites "verify
conformance check"; the check is called verify-honesty.
The crate's own module header is also short
contextgraph-conformance/src/lib.rs's //! block introduces "The checks (all
against the frozen contextgraph-types contracts):" and then lists ten.
Missing: correlation, kinds-filter, anchor-relevance — each of which has a CHECK_* constant and a dedicated --misbehave witness in the same crate.
docs/protocol-advantages.md also names the wrong check for budget honesty
Its guarantees table says budget honesty is enforced by
"Host::query_one_isolated budget audit; frame-validity conformance
check". It is budget-honesty; frame-validity checks score, title, citation
label, representation invariants, timestamps and digest grammar, and never looks
at the budget. The same document gets it right further down ("Conformance
(budget-honesty check)").
README.md and docs/overview.md misattribute --misbehave
Both say "The suite ships a --misbehave mode that trips every check on
purpose." --misbehave is a flag on the fixture provider contextgraph-example-docs, not on the suite and not on contextgraph-inspect,
which has no such flag. And it does not cover every check: shutdown-clean has
no misbehaviour mode among the Misbehave variants.
What I verified
By reading both sides on origin/main at a01ca64: the 13 CHECK_* constants
and their emission in run_conformance; the 13 names in registry/contextgraph-example-docs.report.json; each quoted document line; the
absence of any usage-report check name in the source; describe_handshake's
format string; the Misbehave variant list and the absence of a shutdown mode;
and contextgraph-inspect's clap definition having no --misbehave.
Nothing here is inferred.
Why it matters more than a stale number
This project's pitch is that "conformant" is a checked claim. The number of
checks behind that claim is the first thing an evaluator looks for, and the
front page understates it by more than half. Worse, docs/running-conformance.md
is the page someone reads to decide whether the suite covers their concern —
consent scoping, correlation, anchor relevance and verify honesty are all
covered and all absent from its list, so a reader concludes the suite is weaker
than it is.
What "done" looks like
Every count in the tree matches the suite. Better than recounting: stop
writing the number. "the conformance suite" reads the same and never goes
stale. Where a count genuinely helps, generate it.
docs/running-conformance.md's table lists all 13 checks, its sample output
matches a real run, its ProviderTarget prose is corrected, and the upsert=false segment is removed from the sample evidence line.
docs/protocol-surface.md's UR1 row names a check that exists — or says
plainly that UR1 is host-side and has no wire check, which is the honest
answer if none is planned. D1's citation says verify-honesty.
contextgraph-conformance/src/lib.rs's module header lists every check it
emits.
README.md and docs/overview.md attribute --misbehave to the fixture
provider and drop "every check", or name the one it does not cover.
Worth considering in the same pass: a check that the documented names are a
subset of the CHECK_*/HCHECK_*/CCHECK_* constants. Two of the errors
above are invented check names, which a mechanical check catches for free and
a reader does not.
Constraints
registry/contextgraph-example-docs.report.json is the generated artifact and
is correct; use it as the source of truth rather than recounting by hand.
P1: Roadmap: from contextgraph/1.0-draft to a frozen 1.0 with an ecosystem #22 tracks the same class of defect at the SPEC level ("Verified by" cells
pointing at nothing). This issue is about the prose counts and the two
invented check names in the docs; the SPEC's own tables were separately
audited and their named checks all exist.
The problem
The conformance suite runs 13 checks. Six documents say it runs five or seven,
one document names a check that does not exist, and one page's whole structure
is built around the wrong number.
contextgraph-conformance/src/lib.rsdeclares 13CHECK_*constants andrun_conformanceemits all 13. This repository's own committed report agrees:Where it says otherwise
Three other files already say 13 (
docs/registry.md,docs/reference-providers.md,docs/composition-walkthrough.md), so the tree contradicts itself.docs/running-conformance.mdis the worst of themIt is not one stale number — the page is organised around it:
## The five checks, followed by a five-row table listinghandshake,frame-validity,budget-honesty,shutdown-clean,malformed-input-tolerance. Eight checks are missing, includingconsent-scope,verify-honesty,kinds-filter,anchor-relevanceandcorrelation.CONFORMANT — 5 passed, 0 skipped. A real run ofthe bundled fixture prints 13.
ProviderTargetsection says a stdio target runs "all five checks".provenance-fixture-consistencyand themalformed-digest/stale-digestmisbehaviour modes — none of which appear inits own table. The page contradicts itself within one screen.
query kinds=["doc"], upsert=false, graph=false.describe_handshakeincontextgraph-conformance/src/lib.rsformats"… query kinds={:?}, graph={}"— there is noupsert=segment, becauseADR 0004 deleted
Capabilities::upsert.docs/protocol-surface.mdnames a check that does not existIts UR1 row cites:
FanOut::usage_reportexists. There is no check namedusage-report— notamong the 13 provider
CHECK_*constants, the 9HCHECK_*host constants, orthe 4
CCHECK_*composition constants.rg 'usage-report'over the treereturns only prose.
This is the same shape #22 already recorded for §G2 ("Verified by
frame-validity" with no G2 implementation) — a "Verified by" cell naming acheck nobody wrote. That one has since been fixed in the code; this one has not
been noticed. Small correction in the same table: row D1 cites "
verifyconformance check"; the check is called
verify-honesty.The crate's own module header is also short
contextgraph-conformance/src/lib.rs's//!block introduces "The checks (allagainst the frozen
contextgraph-typescontracts):" and then lists ten.Missing:
correlation,kinds-filter,anchor-relevance— each of which has aCHECK_*constant and a dedicated--misbehavewitness in the same crate.docs/protocol-advantages.mdalso names the wrong check for budget honestyIts guarantees table says budget honesty is enforced by
"
Host::query_one_isolatedbudget audit;frame-validityconformancecheck". It is
budget-honesty;frame-validitychecks score, title, citationlabel, representation invariants, timestamps and digest grammar, and never looks
at the budget. The same document gets it right further down ("Conformance
(
budget-honestycheck)").README.mdanddocs/overview.mdmisattribute--misbehaveBoth say "The suite ships a
--misbehavemode that trips every check onpurpose."
--misbehaveis a flag on the fixture providercontextgraph-example-docs, not on the suite and not oncontextgraph-inspect,which has no such flag. And it does not cover every check:
shutdown-cleanhasno misbehaviour mode among the
Misbehavevariants.What I verified
By reading both sides on
origin/mainat a01ca64: the 13CHECK_*constantsand their emission in
run_conformance; the 13 names inregistry/contextgraph-example-docs.report.json; each quoted document line; theabsence of any
usage-reportcheck name in the source;describe_handshake'sformat string; the
Misbehavevariant list and the absence of a shutdown mode;and
contextgraph-inspect's clap definition having no--misbehave.Nothing here is inferred.
Why it matters more than a stale number
This project's pitch is that "conformant" is a checked claim. The number of
checks behind that claim is the first thing an evaluator looks for, and the
front page understates it by more than half. Worse,
docs/running-conformance.mdis the page someone reads to decide whether the suite covers their concern —
consent scoping, correlation, anchor relevance and verify honesty are all
covered and all absent from its list, so a reader concludes the suite is weaker
than it is.
What "done" looks like
writing the number. "the conformance suite" reads the same and never goes
stale. Where a count genuinely helps, generate it.
docs/running-conformance.md's table lists all 13 checks, its sample outputmatches a real run, its
ProviderTargetprose is corrected, and theupsert=falsesegment is removed from the sample evidence line.docs/protocol-surface.md's UR1 row names a check that exists — or saysplainly that UR1 is host-side and has no wire check, which is the honest
answer if none is planned. D1's citation says
verify-honesty.docs/protocol-advantages.md's budget-honesty row namesbudget-honesty.contextgraph-conformance/src/lib.rs's module header lists every check itemits.
README.mdanddocs/overview.mdattribute--misbehaveto the fixtureprovider and drop "every check", or name the one it does not cover.
subset of the
CHECK_*/HCHECK_*/CCHECK_*constants. Two of the errorsabove are invented check names, which a mechanical check catches for free and
a reader does not.
Constraints
registry/contextgraph-example-docs.report.jsonis the generated artifact andis correct; use it as the source of truth rather than recounting by hand.
pointing at nothing). This issue is about the prose counts and the two
invented check names in the docs; the SPEC's own tables were separately
audited and their named checks all exist.