Decode embedded-encoding strings by one shared policy in both body-diff engines - #94
Open
iemyashasvi wants to merge 2 commits into
Open
iemyashasvi wants to merge 2 commits into
iemyashasvi wants to merge 2 commits into
Conversation
iemyashasvi
force-pushed
the
work/embedded-string-canon
branch
from
September 2, 2026 13:47
ae4b4cd to
f084853
Compare
…ff engines The scorer's order-canonical body recompute (the engine whose rows decide BodyMismatch) read strings as bytes while the kernel's diff_json decoded them — so prism's rawConnectorRequest echo, an embedded JSON string whose headers map serializes in per-process HashMap order, produced zero kernel rows and 17 blocking recompute leaves on the same replay, and the report said '0 response fields differ' beside a body-mismatch verdict. WHAT a string carries is now the kernel's decoding policy, stated once and imported: embedded_json_documents (both sides parse to an object or array) and the strict form-pair qualifier become pub. HOW a decoded document is compared stays each engine's own recursion — inside an embedded string the recompute's array canon applies exactly as outside one, and the echo's form-encoded body string re-enters the arm on recursion. Seeing through the encoding is not tolerance: a genuinely changed inner value still blocks, at its inner path. Verified against the measured run: all 18 recorded/replayed body pairs from rp-sbx-97ed6d68b9-1788270827951761795-0901144939416 are equivalent under the shared policy; the fixture in the tests is that shape with synthesized credentials.
…and pin the incident it exists for Two things changed under this branch in the thirteen days it sat unmerged. rustfmt wants the `else if let (Some(bp), Some(cp))` tuple broken across lines; that was the whole of the failing `rust` check. Main now absorbs a pure permutation by the default rule (`canon_absorbed`, `ClauseSource::Default`) instead of counting it as one blocking ordering fact, so `an_array_inside_an_embedded_document_uses_the_array_canon` asserted the old rule. Its own doc string already said an array inside a document must behave exactly as one outside it, so the test now asserts what the outside does: not blocking, absorbed at `$.doc.tags`, no `order_only_paths`. And the incident that motivates the arm is pinned as a test of its own: a system echoes the connector request it built as a serialized document whose `headers` object comes from a hash map, so its key order is seeded per process. Byte-different, the same JSON value, and therefore not a difference at all — no rows, nothing to absorb, nothing blocking. Without this arm the deployed recompute compares the two strings as scalars and blocks three of three correlations on a body the kernel scores as identical.
iemyashasvi
force-pushed
the
work/embedded-string-canon
branch
from
September 15, 2026 13:50
f084853 to
4115fb9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
Replay
rp-sbx-97ed6d68b9-1788270827951761795-0901144939416failed with 17BodyMismatches while the report rendered "0 response fields differ" beside them. Main has two body-diff engines since6a0e2dc: the kernel'sdiff_json(writesbody_diff, decodes embedded-JSON strings and form-pair strings) and the scorer's order-canonical recompute (decides the verdict, read strings as bytes). Prism'srawConnectorRequestecho — an embedded JSON string whose headers map serializes in per-process HashMap order, carrying a form-encoded body whose pair order is random too — passed one engine and blocked the other, on every response, for identical behavior.The fix
WHAT a string carries is now the kernel's decoding policy, stated once and imported by both engines:
embedded_json_documents(pub, hoisted fromdiff_json's own arm): both sides must parse to an object or array — scalars keep byte semantics, prose stays opaque, a document on one side only is a real difference.parse_form_pairs/diff_form_pairs(now pub): the strict form qualifier and the pair-order policy (distinct keys order-insensitive, repeated keys keep relative order, values stay percent-encoded).HOW a decoded document is compared stays each engine's own recursion: inside an embedded string the recompute's array canon applies exactly as outside one, and nested encodings (the echo's form body) re-enter the arm on recursion. Seeing through the encoding is not tolerance — a genuinely changed inner value still blocks, at its inner path. The tape is never touched: this normalizes judgment, not evidence.
Verification