Stop one ledger name from making a whole book unreadable by lamemustafa · Pull Request #400 · lamemustafa/bridge

lamemustafa · 2026-09-15T17:35:48Z

A ledger in a real book is named across two lines. Tally sends the break as 
 in the NAME attribute — as a character reference, which XML attribute-value normalization preserves where a literal CR LF would have been folded to a space. observed_standard_ledger_name refuses control characters, and because that refusal fails the whole parse, that one master made all 864 unreadable.

Every read that needs a catalogue returns ledger_export_invalid on that book today: presence, a ledger-scoped voucher window, and import validation alike.

How it was found

The live presence replay on #294 refused with ledger_export_invalid. Four layers each discarded the cause on the way out — connector.rs and agent_import.rs both map_err(|_| "ledger_export_invalid"), parse_standard_ledger_catalog_rows does map_err(|_| MalformedResponse), and the replay asserted on isError without printing the refusal — so each had to be instrumented in turn. The guard itself, printing codepoints only, gave offending=["U+000D", "U+000A"] at catalogue row 73.

An earlier hypothesis that an alias's nested LANGUAGENAME.LIST <NAME> was to blame does not hold: both parsers already skip that subtree, and that exact shape is in tests/fixtures/agent/native-ledger-catalogue.utf16le.xml, which a passing test parses.

The decision

A catalogue name is an identity value, not a display string — matched by exact codepoint, and echoed back as exact_live_spelling, which has to round-trip to Tally. So it is kept verbatim: never rewritten, because a cleaned-up spelling addresses a ledger that does not exist, and never dropped, because a malformed name must not remove a real master.

What stays refused is the set that makes a name lie about itself — bidi overrides and zero-width characters, which render one spelling as another. A newline is untidy and belongs to whatever draws it; a right-to-left override is a forged name and belongs here.

My first cut removed the control-character check outright and two existing contract tests caught it, one pinning U+061C. Both pass unchanged under the narrowed fix.

MasterCatalog::new carried the same policy and gets the same split — between a name observed from Tally and one supplied by a caller. A proposal carrying a control character is still refused; that assertion moved to the source side rather than being dropped, and the test says so. Such a ledger stays unbindable by name, since no admissible proposal can spell it, but it is present, counted, and cannot be mistaken for absent.

Family

The fourth this week after #378, #387 and #388 — Bridge failing closed on data Tally legitimately produces, losing a whole window rather than one row. The same guard did the same thing on 2026-08-20, with C1 bytes from an old double-encoding import instead of a newline.

Verification

Both new tests confirmed failing before the fix and passing after.
The catalogue test injects a synthetic two-line name into an existing lab fixture — no client data.
Full workspace suite 1361 passed, 0 failed, zero panics.
clippy exit 0 on both workspaces, status read directly rather than through a pipe.
reseal.sh --verify exit 0; pin set compared as paths, unchanged at 212.

Scope

Catalogue-side only, and deliberately separate from #294. The presence adapter half — an absent party arriving as "" rather than None — stays on #294, because agent_presence.rs does not exist on master.

With both halves in place the live replay replay_the_twenty_invoice_engagement passes against the real book for the first time.

🤖 Generated with Claude Code

A ledger in a real book is named across two lines. Tally sends the break as `
` in the NAME attribute -- as a character reference, which XML attribute-value normalization preserves where a literal CR LF would have been folded to a space. `observed_standard_ledger_name` refuses control characters, and because that refusal fails the whole parse, that one master made all 864 unreadable: `ledger_export_invalid` from every read that needs a catalogue, which is presence, a ledger-scoped voucher window, and import validation alike. This is the fourth of its family this week, after #378, #387 and #388: Bridge failing closed on data Tally legitimately produces, and losing a whole window rather than one row. The vault recorded the same guard doing the same thing on 2026-08-20, where the characters were C1 bytes baked in by an old double-encoding import rather than a newline. A catalogue name is an identity value, not a display string: it is matched by exact codepoint and echoed back as `exact_live_spelling`, which has to round-trip to Tally. So it is kept verbatim -- never rewritten, since a cleaned-up spelling addresses a ledger that does not exist, and never dropped, since a malformed name must not remove a real master. What stays refused is the set that makes a name lie about itself: bidi overrides and zero-width characters, which render one spelling as another. A newline is untidy and belongs to whatever draws it; a right-to-left override is a forged name and belongs here. My first cut removed the whole check and two existing contract tests caught it, one of them pinning U+061C; both pass unchanged now. `MasterCatalog::new` carried the same policy and gets the same split, between a name **observed** from Tally and one **supplied** by a caller. A proposal with a control character is still refused -- that assertion moved to the source side rather than being dropped, and is noted in the test. Such a ledger stays unbindable by name, because no admissible proposal can spell it, but it is present and counted and cannot be mistaken for absent. Both tests were confirmed failing before the fix and passing after. The catalogue test injects a synthetic two-line name into an existing lab fixture, so no client data is involved. Gate: 1361 passed, 0 failed; clippy silent on both workspaces; reseal --verify exits 0 with the pin set unchanged at 212. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

chatgpt-codex-connector · 2026-09-15T17:35:52Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

The window JSON spells an absent string as `""` rather than omitting the key, so `row["party"].as_str()` yields `Some("")` -- which claims the voucher *has* a party whose name is blank. Two guards then fired on a book that merely held a voucher naming nobody: the catalogue check reported all 864 masters incomplete, and the presence engine refused the whole window with `presence_text_blank`. `present_text` translates that back at the boundary where it is introduced, so neither guard needs a special case. Only the **observed** side uses it. A blank on a caller's proposal is input the schema already refuses, and mapping it to "absent" here would quietly accept what the schema rejects -- so the two proposal sites keep reading the field literally. An empty `PARTYLEDGERNAME` is a real Tally shape rather than something we invent: 277 of 2,829 party fields across 53 captured windows of the reference book are genuinely empty, none absent. That is the opposite of an entry's ledger name, which is populated 7,633 times and blank zero times -- measured by the read-path session, and the reason this fix reads a blank party as "no party" instead of teaching the engine to carry blanks generally. The replay assertion now prints the refusal it failed on. Asserting `isError` alone reported that the read failed while withholding the one thing that says why, which cost a full diagnosis round the first time it fired. With the catalogue half in #400, the live replay `replay_the_twenty_invoice_engagement` passes against the real book for the first time: 20 vouchers, presence verdicts, no refusal. Gate: 1498 passed, 0 failed; clippy silent on both workspaces; reseal --verify exits 0 with the pin set unchanged at 216. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ls-whole-catalogue # Conflicts: # docs/tally/compatibility/compatibility-matrix.json # docs/tally/compatibility/compatibility-surface.json

The merge resolution staged the compatibility pair and *then* ran the reseal, so the regenerated bytes never entered the index and the merge commit carried master's pair against this branch's changed pinned sources. CI read the commit and refused with `surface_file_changed`; my local run read the working tree, where the reseal had landed, and passed. A gate that reads the tree cannot see an unstaged regeneration. Regenerate, verify, and stage last -- in that order. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ls-whole-catalogue # Conflicts: # docs/tally/compatibility/compatibility-matrix.json # docs/tally/compatibility/compatibility-surface.json

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Master gained d09bf9b (#400, one ledger name making a whole book unreadable), which resealed master_binding.rs, bridge-tally-protocol's lib.rs and simulator_corpus.rs. This branch had resealed the workflow, so both sides moved the same two manifests and they conflicted. The pin lists were identical on both sides (212, no path only on either), so the conflict was hash content alone. Resolved by taking master's manifests and running scripts/reseal.sh, which recomputes every hash from the merged tree rather than hand-merging. Exactly one hash now differs from master -- dependency-security.yml, this branch's own change -- and no path is missing versus either side. reseal.sh --verify exit 0, read directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

#400 let a catalogue carry a ledger name holding a control character -- a ledger genuinely named across two lines -- because a catalogue name is a fact about the book. The proposal side still refuses one, because a caller's name is input. Both rules are right, and the gap between them is a spelling the tool can report but the caller cannot send back. `master_recovery_guidance` told them to send it anyway: "For identifier-bound entries, copy exact_live_spelling from this fresh result." Following that does not fail one row. `source_entities` collects into a single Result and refuses on the first bad name, so one such ledger fails the **entire** batch with `master_name_unsafe` -- every voucher in the request, including those naming no unusual ledger at all. The report now carries `importable`, and the guidance offers the copy only for entries where copying can work. Where it cannot, it says so and names the one remedy that exists: have an operator rename the ledger in Tally. `importable` is answered by calling `SourceEntity::new` rather than restating what it accepts, so the report and the import path cannot drift apart. No corruption was possible -- the failure was closed, not silent. What it cost was a caller following the tool's own instructions into a dead end it could not diagnose. Found by an independent review of #294 that was asked to trace this exact path. Both tests confirmed failing before the fix and passing after. Both workspaces green; reseal --verify exits 0 with the pin set unchanged at 216. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: t <dev@example.invalid>

`U+200B..=U+200F` is not a set of deceptive characters. It contains ZWNJ (U+200C) and ZWJ (U+200D), which Devanagari and other Indic scripts use to prevent or force a conjunct. They are ordinary spelling, and this repository's own fixtures are full of Indic ledger names. Refusing them failed the **whole** catalog on a legitimately spelled Hindi or Marathi ledger -- the exact failure the newline fix existed to remove, on a book type Indian CA practice is full of. **This was a regression I introduced.** Before #400, `MasterCatalog::new` checked only `char::is_control`, which is category Cc and admits ZWNJ. #400 added the deceptive-set check to the catalog side and took the whole `U+200B..=U+200F` span with it, so a Devanagari name that worked before stopped working. The protocol crate refused it before and after; only the core is a regression. The set is narrowed rather than abandoned: `U+200B`, `U+200E` and `U+200F` carry no orthographic role and stay refused, alongside the override, isolate and BOM ranges. The test pins both directions -- ZWNJ and ZWJ admissible on both sides, their neighbours in the same span still refused -- so this cannot be widened back by accident. Prompted by asking an independent reviewer to check exactly this before merging, rather than after. Gate: both workspaces green, clippy silent on both, reseal --verify exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Refuse a proposed name that reads as a different master `char::is_control` is Unicode category Cc. Bidi overrides and zero-width characters are Cf, so every one of them walked straight through the proposal side's control-character check: `U+202E`, `U+200B`, `U+061C` and `U+FEFF` all answer `false`. The catalog side refuses them and this side did not, which is the wrong way round. A name read from a book is a fact about the book. A name a caller supplies is the one that can be *chosen* so that what a reviewer sees is not the master it binds. #400 added the catalog-side refusal and its doc comment claimed these characters "stay refused on both sides". That was false when I wrote it -- I asserted a property of a function I had not changed. This makes the claim true rather than deleting it. The test asserts the premise before the conclusion: for each character it first checks `char::is_control` is false, so the test fails loudly if a future Rust release reclassifies one and the coverage silently becomes vacuous. It also pins what must keep working -- an ordinary name, and the two-line catalog name #400 deliberately admits. Found by an independent review of #405, raised as suspected and unverified; confirmed by running `char::is_control` over the set. Gate: both workspaces green, clippy silent on both, reseal --verify exits 0 with the pin set unchanged at 216. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Stop treating Indic orthography as deception `U+200B..=U+200F` is not a set of deceptive characters. It contains ZWNJ (U+200C) and ZWJ (U+200D), which Devanagari and other Indic scripts use to prevent or force a conjunct. They are ordinary spelling, and this repository's own fixtures are full of Indic ledger names. Refusing them failed the **whole** catalog on a legitimately spelled Hindi or Marathi ledger -- the exact failure the newline fix existed to remove, on a book type Indian CA practice is full of. **This was a regression I introduced.** Before #400, `MasterCatalog::new` checked only `char::is_control`, which is category Cc and admits ZWNJ. #400 added the deceptive-set check to the catalog side and took the whole `U+200B..=U+200F` span with it, so a Devanagari name that worked before stopped working. The protocol crate refused it before and after; only the core is a regression. The set is narrowed rather than abandoned: `U+200B`, `U+200E` and `U+200F` carry no orthographic role and stay refused, alongside the override, isolate and BOM ranges. The test pins both directions -- ZWNJ and ZWJ admissible on both sides, their neighbours in the same span still refused -- so this cannot be widened back by accident. Prompted by asking an independent reviewer to check exactly this before merging, rather than after. Gate: both workspaces green, clippy silent on both, reseal --verify exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Reseal after merging master Took master's pin list, which is a superset of this branch's at 217 to 216 -- it adds the scheduled audit workflow and this branch adds nothing. Sets compared as paths against both parents; none dropped from either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Name the trade-off, and cover the crate that actually reads books Two gaps an independent review found, both fair. The protocol crate carries its own copy of the deceptive set and it is the one on the real read path -- `observed_standard_ledger_name` runs on every name a book returns -- yet the narrowing landed there with no test at all. It has one now, driven to failure by restoring the wide range in that crate alone. And the predicates now say what admitting ZWNJ costs. A codepoint filter cannot tell a joiner between two Devanagari consonants from one injected into ASCII, so `Alpha<ZWJ> Traders` is byte-distinct from `Alpha Traders` and renders the same. The guarantee is bounded rather than closed: the fold and the token index keep the joiner verbatim, so such a name tends to fail exact and token matching rather than quietly aliasing a real master. Worse than refusal, far better than breaking every Indic book. One claim withdrawn: the `char::is_control` premise assertion does not guard against the refusal going vacuous -- `unwrap_err()` already panics on an unexpected `Ok`. It guards against Unicode reclassifying a codepoint out from under the comment, which is narrower than I said. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Reseal after merging master Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: t <dev@example.invalid>

Rebases the narration-marker work onto current master. The branch had diverged badly: #294 squash-merged onto master, so git chose a merge base from before the presence engine existed on either side and reported the whole of book_presence.rs, book_presence_tests.rs, agent_presence.rs and agent_presence_tests.rs as add/add conflicts. Master's copies of the first two are byte-identical to the branch at 64b8666, so the real resolution was six files, not ten. Two of master's fixes had to be carried into code the branch restructured: - present_text() from #400 replaces row["party"].as_str() and row["voucher_number"].as_str() at all four sites inside book_window's marker match arms. A voucher with no party carries "" rather than a missing key, and the raw read made a blank name look like a real one -- which is what made a whole 864-master book report as incomplete. - The agent_catalog.rs doc comment on the admission-contract test keeps master's corrected text from #409. The branch still carried the claim that the file is not in the compatibility surface; it is pinned. MAX_SURFACE_FILES goes to 218. Master raised it to 217 for dependency-security-scheduled.yml and this branch raised it to 217 for agent_import_identity.rs -- the same number for different files, so the merged pin set is their union. Both reasons are recorded; the seals are regenerated with scripts/reseal.sh --pins-changed, never by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Rebases the narration-marker work onto current master. The branch had diverged badly: #294 squash-merged onto master, so git chose a merge base from before the presence engine existed on either side and reported the whole of book_presence.rs, book_presence_tests.rs, agent_presence.rs and agent_presence_tests.rs as add/add conflicts. Master's copies of the first two are byte-identical to the branch at 64b8666, so the real resolution was six files, not ten. Two of master's fixes had to be carried into code the branch restructured: - present_text() from #400 replaces row["party"].as_str() and row["voucher_number"].as_str() at all four sites inside book_window's marker match arms. A voucher with no party carries "" rather than a missing key, and the raw read made a blank name look like a real one -- which is what made a whole 864-master book report as incomplete. - The agent_catalog.rs doc comment on the admission-contract test keeps master's corrected text from #409. The branch still carried the claim that the file is not in the compatibility surface; it is pinned. MAX_SURFACE_FILES goes to 218. Master raised it to 217 for dependency-security-scheduled.yml and this branch raised it to 217 for agent_import_identity.rs -- the same number for different files, so the merged pin set is their union. Both reasons are recorded; the seals are regenerated with scripts/reseal.sh --pins-changed, never by hand. The cap's own history comment is corrected while it is being edited. It said 212 to 215 and then 216 to 217, leaving a step nothing accounted for; #294 in fact raised the constant 212 to 216 in one commit, pinning four files at once. The enumeration now says so, rather than being reconcilable only against a later paragraph. docs/module-decomposition.md and docs/proposed-rust-module-conventions.md quote the cap and the pin count, and were correct at 217; they are updated here because this change is what makes them stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Rebases the narration-marker work onto current master. The branch had diverged badly: #294 squash-merged onto master, so git chose a merge base from before the presence engine existed on either side and reported the whole of book_presence.rs, book_presence_tests.rs, agent_presence.rs and agent_presence_tests.rs as add/add conflicts. Master's copies of the first two are byte-identical to the branch at 64b8666, so the real resolution was six files, not ten. Two of master's fixes had to be carried into code the branch restructured: - present_text() from #400 replaces row["party"].as_str() and row["voucher_number"].as_str() at all four sites inside book_window's marker match arms. A voucher with no party carries "" rather than a missing key, and the raw read made a blank name look like a real one -- which is what made a whole 864-master book report as incomplete. - The agent_catalog.rs doc comment on the admission-contract test keeps master's corrected text from #409. The branch still carried the claim that the file is not in the compatibility surface; it is pinned. MAX_SURFACE_FILES goes to 218. Master raised it to 217 for dependency-security-scheduled.yml and this branch raised it to 217 for agent_import_identity.rs -- the same number for different files, so the merged pin set is their union. Both reasons are recorded; the seals are regenerated with scripts/reseal.sh --pins-changed, never by hand. The cap's own history comment is corrected while it is being edited. It said 212 to 215 and then 216 to 217, leaving a step nothing accounted for; #294 in fact raised the constant 212 to 216 in one commit, pinning four files at once. The enumeration now says so, rather than being reconcilable only against a later paragraph. docs/module-decomposition.md and docs/proposed-rust-module-conventions.md quote the cap and the pin count, and were correct at 217; they are updated here because this change is what makes them stale. Co-authored-by: t <dev@example.invalid> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

lamemustafa and others added 4 commits September 15, 2026 23:18

Merge remote-tracking branch 'origin/master' into fix/ledger-name-fai…

e4f1373

…ls-whole-catalogue # Conflicts: # docs/tally/compatibility/compatibility-matrix.json # docs/tally/compatibility/compatibility-surface.json

Merge remote-tracking branch 'origin/master' into fix/ledger-name-fai…

636f3c4

…ls-whole-catalogue # Conflicts: # docs/tally/compatibility/compatibility-matrix.json # docs/tally/compatibility/compatibility-surface.json

Reseal after merging master

aa20cc1

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

lamemustafa merged commit d09bf9b into master Sep 15, 2026
13 checks passed

lamemustafa deleted the fix/ledger-name-fails-whole-catalogue branch September 15, 2026 18:11

This was referenced Sep 15, 2026

Decide which proposed vouchers are already in the book #294

Merged

Stop offering a live spelling imports will refuse #405

Merged

lamemustafa mentioned this pull request Sep 15, 2026

Refuse a proposed name that reads as a different master #408

Merged

lamemustafa mentioned this pull request Sep 15, 2026

Bug: ledger_movement refuses a whole window when it contains a voucher with no accounting entries #418

Closed

Navigation Menu

Stop one ledger name from making a whole book unreadable - #400

Stop one ledger name from making a whole book unreadable#400
lamemustafa merged 5 commits into
masterfrom
fix/ledger-name-fails-whole-catalogue

lamemustafa commented Sep 15, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

lamemustafa commented Sep 15, 2026

How it was found

The decision

Family

Verification

Scope

Uh oh!

chatgpt-codex-connector Bot commented Sep 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant