Add a second implementation, and fix what it found - #16
Merged
Conversation
A standard with one implementation is a design. Nothing had tested whether a second implementer, reading only the specification, arrives at the same bytes. conformance/second_impl is the Core profile (§2.1.1) in Python, written from the specification text rather than from this source tree: normalization, the concrete syntax, import resolution and merge, profile enforcement, and canonical rendering with policy hashing. cross_check.py runs both implementations over the example corpus and compares document_hash and messages, excluding `profile` and `mode`, which are properties of the implementation rather than of the document. A new divergence fails; a documented one does not. It found four things, none of which were visible from inside the reference — an implementation cannot notice the questions it silently answered. The serious one is fixed here. §7.2 said imported content is "expanded in-place" and no more, which leaves the byte-level result open. Since `document_hash` is the SHA-256 of the Resolved Source Form (§18.3), the two implementations produced *different identities for the same contract*: sha256:3bd041ee… against sha256:4eb3df04…. §7.2 now states the splice exactly — the `@import` line is replaced by the imported document's Resolved Source Form with exactly one trailing newline and no other separator. The trailing-newline clause is not decoration: without it an imported file that does not end in a newline joins its last line to what follows. Three remain open, recorded in DIVERGENCES.md rather than papered over: Core meets a `$` reference in a message with no phase able to resolve it and the spec does not say whether to refuse, pass through, or resolve; Core reports `budget_units` while having no Token Box Model to enforce it; and Appendix B's grammar cannot derive the multi-line inline literals that the specification's own example 09 uses. Everything else agreed, including document_hash across imports once §7.2 was pinned. That is the useful half of the result: the parts written tightly enough produced identical bytes from two implementations sharing no code. This second implementation shares an author with the reference, so it tests whether the text is sufficient, not whether it is unambiguous to a stranger. ROADMAP.md keeps that distinction open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ROADMAP item 4.
A standard with one implementation is a design. Nothing had tested whether a second implementer, reading only the specification, arrives at the same bytes.
What this is
conformance/second_impl— the Core profile (§2.1.1) in Python, written from the specification text rather than from this source tree.cross_check.pyruns both implementations over the example corpus and comparesdocument_hashand messages, excludingprofileandmode(properties of the implementation, not the document). Gated in CI; a new divergence fails, a documented one does not.What it found
An implementation cannot notice the questions it silently answered. This one asked four.
Fixed here: §7.2 left a contract's identity undefined
§7.2 said imported content is "expanded in-place" and no more.
document_hashis the SHA-256 of the Resolved Source Form (§18.3) — so the two implementations produced different identities for the same contract:Both readings of "expanded in-place" are defensible; that is the problem. §7.2 now states the splice exactly: the
@importline is replaced by the imported document's Resolved Source Form with exactly one trailing newline and no other separator. The trailing-newline clause is not decoration — without it, an imported file that does not end in a newline joins its last line to whatever follows.Open, and written down in
DIVERGENCES.md$promptin a message with no phase able to resolve it. Refuse (F801), pass through as text, or resolve when the variable is a literal? All three are defensible and produce different canonical output.budget_unitswhile having no Token Box Model to enforce it, so its payload can exceed the budget it declares.09_multimodal_content.facetbut cannot be derived from Appendix B's grammar, whereSPis a space, not a newline. Informative annex, so not a violation — but an implementer working from the grammar writes a parser that rejects the specification's examples. This one did.What agreed
Everything else: normalization, concrete syntax, facet cardinality and merge order,
@metarestrictions, canonical message ordering,policy_hashover the effective policy object, anddocument_hashbyte for byte including across imports once §7.2 was pinned.Honest limit
This second implementation shares an author with the reference. It tests whether the text is sufficient, not whether it is unambiguous to a stranger — that item stays open in
ROADMAP.md.498 tests, coverage 60 → 62, all gates green.
🤖 Generated with Claude Code