Parse password-protected bank statements into voucher proposals, through PDFium

[lamemustafa]

What this adds

parse_bank_statement is a local, read-only MCP tool that turns a password-protected SBI or HDFC bank-statement PDF into Payment / Receipt / Contra proposals, in exactly the voucher shape build_import_xml accepts. It ports the parsing half of scripts/bank_statement_import.py into a new crate, src-tauri/crates/bridge-bank-statement, and reads PDFs through PDFium (pdfium-render 0.9.4, dynamic binding).

The ported half covers word geometry, _lines and _dewrap, the SBI and HDFC profiles, account binding, the exact-decimal balance replay and control totals, the mapping with its suspense fallback, and the self-check. Every stage refuses the whole run.

It mints no REMOTEID and renders no XML. Proposals carry bridge_txn_id labels only. The label is derived from the statement row (account, date, amounts, balance, narration), so re-parsing with a corrected mapping yields the same labels. An amends_batch_id amendment has to name exactly those labels. A label is never a Tally key: Bridge's writer derives the REMOTEID from its own batch.

build_import_xml builds from the proposals file (third commit, owner's go-ahead). Pass proposals_id and proposals_sha256, the proposals_id and sha256 values parse_bank_statement returned, in place of vouchers.

The resolver (agent_bank_statement.rs::resolve_import_arguments) refuses in these cases:

• vouchers is also given, or either proposals field is missing;
• the id is malformed;
• the file is not a regular, single-link, user-owned file under bank-statements/;
• its schema or id is wrong;
• its bytes no longer hash to the digest (proposals_changed).

It then passes the file's vouchers to the unchanged admission path, which checks the schema, masters, cash/bank groups, and the amendment compare-and-swap. The only edit in agent_import.rs is that one call. Nothing under agent_import*.rs decides anything new.

Found while doing it: amendments were unreachable through tools/call on master. The catalog validator refuses any schema pattern outside its vocabulary, and amends_batch_id's pattern was not in it. So every MCP call naming amends_batch_id returned argument_invalid:amends_batch_id before the handler ran. #439's tests call the handler directly, which is why they pass. Measured with validate_tool_arguments on this branch before the fix. The validator now knows that exact shape, and the schema and validator share one constant.

Decisions (owner choices marked)

1. Egress, owner's choice: summary plus local file. Every row's date, amount, reference and narration goes to a 0600 file under <data_dir>/bank-statements/, written staged then renamed. The tool returns only what is needed to write the mapping: counterparty spellings grouped by mapping key, row counts and totals, disposition, suspense flag, and the ledger names to check with validate_masters. Every name is marked as a party, so mask_parties masks it.
2. Password, owner's choice: a password file path. The tool takes password_file. It must be a regular file owned by this user, with a single link and no group or other permission bits on Unix. It is read into a zeroizing buffer, handed to PDFium once and dropped. It is never an argument, never echoed, and never written anywhere.
3. PDFium CI, owner's choice: a pinned download with SHA-256. packaging/pdfium/pdfium.lock.json pins bblanchon/pdfium-binaries chromium/7881 (V8 and XFA disabled). scripts/fetch-pdfium.py checks byte count and SHA-256 before extracting. PDF tests are #[ignore]d by default and fail rather than skip when BRIDGE_PDFIUM_LIBRARY is unset. The native CI job fetches the library and runs them.
4. Where it lives. The crate holds parsing and a pipeline::prepare that fixes the fail-closed order. src-tauri/src/agent_bank_statement.rs holds the tool: file admission, persistence and the summary. The tool is listed and dispatched only when imports are enabled, because it only prepares an import and its summary carries names.
5. Licences. PDFium is BSD-3-Clause. The binary package bundles 14 third-party licences, including FreeType (FTL) and libjpeg-turbo (IJG), both of which require attribution in shipped documentation. pdfium-render is MIT/Apache-2.0. THIRD_PARTY_LICENSES_RUST.txt is regenerated with cargo-about 0.9.1 and the inventory check passes. PDFium itself is not shipped by this PR; see below.

Packaging: PDFium is bundled in the MCPB (second commit, owner's go-ahead)

The MCPB manifest enables imports by default, so every archive lists parse_bank_statement. The archive therefore now carries the pinned library.

• scripts/fetch-pdfium.py checks three digests pinned per platform in packaging/pdfium/pdfium.lock.json: the release archive (before extraction), the extracted library, and THIRD_PARTY_LICENSES_PDFIUM.txt. That notice is generated from every licence file the archive ships, byte for byte. It is generated rather than committed because the Windows and macOS archives ship different licence bytes: CRLF line endings, and a //-prefixed pdfium.txt on macOS.
• scripts/package-mcpb.mjs requires --pdfium. It stages the library at bin/<triple>/libpdfium.dylib or pdfium.dll, beside bridge_mcp where the tool loads it, and the notice at the archive root.
• scripts/check-mcpb-bundle.py requires exactly 8 members and matches the library and notice to the host's pins. It then parses the synthetic encrypted statement through the unpacked bundle and requires 6 vouchers, with the password absent from stdout and stderr.
• The CI bundle smoke and release-mcpb-preview.yml fetch the pinned build before staging.
• NOTICE carries the PDFium, FreeType and IJG attributions. The MCPB README, preview release notes and release process describe the unsigned third-party library.

Archive size grows by roughly the library: 7,732,336 bytes on macOS arm64 and 7,211,520 bytes on Windows x64, before compression. The library ships unsigned and un-notarized, like bridge_mcp itself. Whether macOS Gatekeeper lets a quarantined, installed MCPB load it has not been tested on a real client install.

Measured

• Parity with the Python reference on real geometry. The Rust parser reproduces the Python suite's pinned party, reference and narration digests over both sanitised captures (scripts/fixtures/{hdfc,sbi}-bbox-capture.xml).
• PDFium against poppler on the synthetic PDFs. For both synthetic statements, PDFium's words produce exactly the rows pdftotext -bbox-layout words produce (poppler 26.04.0). Those rows match four digests the Python reference computes over the poppler capture, including a whole-row JSON digest. Horizontal word edges agree within 0.01pt. Vertical edges are shifted by a constant per font: y0 by 1.43pt and y1 by 0.99pt on 7pt Courier, with spread under 0.01pt. The cause is that PDFium takes ascent from a substituted Courier New face, while poppler uses the AFM metrics.
• Two real divergences found by that comparison, and fixed.
  • PDFium reports a hyphen that ends a line as U+0002. Narrations wrap at hyphens and every HDFC party boundary is a hyphen, so it is read back as -. A test failed before the fix.
  • pdfium-render panics on a password containing NUL, and its panic message prints the password's bytes. Such a password is now refused first (unusable_password).
• Owner-password-only statement. pdftotext -upw rejects the synthetic SBI file and -opw opens it. PDFium opens it with the one password supplied, and refuses wrong or empty passwords.
• Controls proven by mutation. 53 crate controls: each disabled in turn, and the suite fails every time. For 52, a named test fails. The NUL-password case was checked by hand and fails malformed_input_is_refused_not_parsed via the pdfium-render panic. 6 tool-surface controls were checked the same way: import gating in catalog and dispatch, password-file permissions, party marking, rows staying out of the result, and nested mapping validation. Two tests were missing and were added: the running balance in the label, and rotated-page refusal.
• Password. Driven through the real MCP framing, the password appears in neither the wire output nor any file under the data directory, and an egress receipt is written.
• Suites. cargo test --workspace in src-tauri: 1,619 passed, 0 failed, 7 ignored (6 are these PDFium tests; they pass with the library). tools/: 55 passed. Clippy with -D warnings -A clippy::pedantic is clean on the workspace. scripts/reseal.sh --verify is current. The fixture byte-integrity and provenance gate tests pass, and the new fixtures' declared hashes are machine-checked.

Inferred or not measured

• Real statements through PDFium. Not run: they are private, and none were used. The synthetic fixtures use the non-embedded base-14 Courier font and RC4-128 encryption. Real statements commonly use embedded proportional fonts and AES. The balance replay and control totals are the backstop for a misread, but a misattributed party is not caught by arithmetic. A differential run (PDFium rows against pdftotext rows on real statements, on the operator's machine, reporting counts only) is the next measurement.
• Windows. Nothing ran on Windows. The CI job will be the first run of the PDF tests there.
• Mixed fonts on one line. A line mixing two non-embedded fonts could group differently from pdftotext, because the vertical shift is per font. Not observed.

Deliberate divergences from the Python reference

• Amounts and dates accept ASCII digits only. Python's \d also accepts other scripts' digits; here they refuse (malformed_amount, unparseable_date).
• A zero-amount row is refused (zero_amount_row), because build_import_xml refuses zero amounts.
• A narration or ledger build_import_xml would refuse (control characters, [BRIDGE:, length) is refused with its row number.
• Python's second sentinel lock in build is omitted: a Rust Mapping can only be built by from_rows, which refuses sentinel keys, so the lock could never fire.
• The CSV loader accepts a UTF-8 BOM, as the csv crate strips it; Python refused such a file as mapping_headers_missing. The tool itself takes the mapping inline.
• Refusal messages do not echo the offending cell, account number or tail, where Python's messages did. The tool forwards only the category and row number (statement_<category>:row_<n>), so an operator gets less detail than the script gave.
• _looks_like_utr and SBI's digit test use regex \d (Unicode Nd), while Python's str.isdigit() also accepts characters like superscript digits. Not reachable on the upper-case ASCII narrations seen so far; not measured.
• Output plumbing is not ported: output claiming, Windows ACLs, manifest CSV, XML self-check.

Compatibility surface

MAX_SURFACE_FILES goes from 239 to 251, with twelve pins, each named with its reason in the constant's comment: agent_bank_statement.rs, the crate Cargo.toml, and pdf, pipeline, money, parse, bank, geometry, text, mapping, date and proposals from the crate. Resealed with --pins-changed, then verified. No open PR changes the cap as of this push.

Review

A Sonnet subagent reviewed the full diff against the Python reference, function by function. It ran both crates' tests, including the PDFium-gated ones, and the exact CI commands, and reproduced the csv BOM behaviour standalone. It found no P1s. Its three narrower findings:

• The UPI mask test is case-sensitive while the IMPS mask test is not. This is a faithful port of the Python, so any bug predates this PR. Left as is.
• Refusal messages no longer echo raw values. Deliberate, but it was not listed; now listed above.
• The \d versus isdigit() gap. Not reachable in practice; now listed above.

It did not reproduce the mutation counts or the full-workspace total; those rest on the runs described above.

Second review (2026-09-17), after rebasing onto ad13d7b0. A fresh Sonnet subagent reviewed the bundling commit and the proposals-build commit. It found no P1 or P2. It ran:

• fetch-pdfium.test.py: 5 tests.
• check-mcpb-bundle.test.py: 13 tests.
• package-mcpb.test.mjs: 7 tests.
• The agent_import suite (194 tests) and the agent::catalog suite (7 tests).
• The traversal, symlink, digest-change and schema-mismatch proposals tests.
• The compatibility gate.
  It also confirmed, with --list, that the CI --ignored filter selects 4 tests rather than none.

After the rebase I ran, locally:

• cargo test --workspace: 1,622 passed, 0 failed, 8 ignored.
• Clippy: clean.
• The PDFium-gated crate and tool tests: pass.
• tools/ tests: pass.
• reseal.sh --verify: current.
  The conflict was only in the surface JSON. I took master's copy, re-added the same 12 pins, and resealed.

Proposals build: measured

• Through tools/call against the simulator. A proposals build consumes the same 44 requests as the inline build of the same vouchers. The two import XML files are identical once REMOTEIDs and batch markers are blanked.
• End to end (ignored; needs PDFium). The synthetic encrypted HDFC PDF goes through parse_bank_statement, then build_import_xml by proposals_id. Both calls go through tool dispatch against the simulator, with Cash as the bank and a captured debtor as suspense. The file has 6 vouchers (4 Payment, 2 Receipt), and the narration carries UPI 612345678901 from NORTHWIND TRADERS with the wrapped reference whole. CI runs it with the pinned PDFium.
• Refusals, before any Tally read: proposals_changed after a one-amount edit, proposals_sha256_required, proposals_id_required, vouchers_required, proposals_id_with_vouchers, proposals_not_found, a 46-character traversal id refused as argument_invalid:proposals_id, an uppercase digest, proposals_file_invalid, and a symlinked file (proposals_file_unreadable).
• Mutation. Each of the six controls was disabled in turn, and a named test failed each time: the validator arm, digest compare, schema and id check, vouchers/proposals exclusivity, the id format check, and the resolver call itself. The id format check first survived: my traversal id was 47 characters, so the schema length bound refused it before the resolver ran. With a 46-character id, it is caught.
• Suites. Workspace: 1,622 passed, 0 failed, 8 ignored. tools/: 55 passed. Clippy is clean, and the reseal verifies. One run of reseal-merge-driver.test.mjs failed 2 cases straight after the commit, and a rerun passed all 3; not investigated further.

Test plan

• cargo test --locked --workspace in src-tauri, and in tools/
• BRIDGE_PDFIUM_LIBRARY=… cargo test -p bridge-bank-statement -- --ignored and cargo test -p bridge --lib bank_statement -- --ignored (macOS arm64)
• cargo clippy --locked --workspace --all-targets -- -D warnings -A clippy::pedantic
• scripts/reseal.sh --verify; node scripts/check-fixture-byte-integrity.mjs; fixture provenance and byte-integrity gate tests; check-dependency-inventory.mjs --rust; check-mcpb-bundle.test.py
• python3 scripts/generate-bank-statement-fixtures.py <fixtures> --check
• CI native job on Windows and macOS, including the PDFium fetch. Windows ran the 6 PDFium tests, and PDFium's rows equal poppler's there too.
• Local MCPB on macOS arm64: package-mcpb.mjs --pdfium, a zip of the stage, and check-mcpb-bundle.py. 8 members, library pin matched, statement_vouchers: 6. Control: removing the dylib fails the smoke with statement_pdf_engine_unavailable.
• fetch-pdfium.test.py (archive, library and notice refusals), check-mcpb-bundle.test.py (13), package-mcpb.test.mjs (7)
• CI bundle smoke on Windows and macOS with the bundled library, archives built by mcpb pack: 8 archive members, the library digest matches its pin on each OS (1bc45b15… / 79d4676b…), and statement_vouchers: 6 parsed through the bundle on both.
• Install the preview MCPB in Claude Desktop on each OS and call the tool
• build_import_xml builds from a proposals file (bridge-aa was not running; no open PR touched agent_import*.rs)
• Live end to end on a real Tally book and real statement (bridge-aa's lane)

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[lamemustafa]

Coordination note from #446, which touches the same file.

#446 moves the inline tests module out of tools/bridge-tally-compatibility/src/lib.rs into lib_tests.rs. It is in review.

• Source text: no overlap. This PR edits the doc comment and MAX_SURFACE_FILES near the top of that file. Move the compatibility tool's inline tests out of its pinned lib.rs #446 replaces only the test block starting at line 1,684. Git should merge the two cleanly in either order.
• Generated files: must be merged, not picked. Both PRs regenerate docs/tally/compatibility/compatibility-surface.json and the matrix, and this PR adds 12 pins. Whichever lands second must merge the pin list, keeping this PR's 12 entries, and then run scripts/reseal.sh --pins-changed. Taking either side of the generated file silently drops pins, and the gate still passes (docs/release-process.md, "When the surface itself conflicts").
• Tests moving files. After Move the compatibility tool's inline tests out of its pinned lib.rs #446, the reserve-bound assertion MAX_SURFACE_FILES - files.len() <= RESERVED_SURFACE_FILES lives in lib_tests.rs (around line 914). If this PR changes any of the tool's tests, rebase and apply those changes there.

[lamemustafa]

Update: #446 merged as eb5e3e9c. Rebasing this PR onto master will: (1) merge lib.rs cleanly (#446's review confirmed this with git merge-tree in both orders), and (2) conflict in compatibility-surface.json and the matrix. Resolve those by keeping this PR's 12 new pin entries plus master's list, then run scripts/reseal.sh --pins-changed. The reserve-bound test now lives in tools/bridge-tally-compatibility/src/lib_tests.rs.