perf(cache): warm no-op checks stop re-proving the cache (draft — review before merge) by hellovai · Pull Request #4101 · BoundaryML/baml

hellovai · 2026-07-21T07:47:36Z

Draft on purpose — this touches cache-soundness invariants; walk through the risk section below before merging. Stacked on #4100.

The problem

A warm no-op baml check at 295k LOC cost 7.2s — identical to a full recompute — because the warm path re-proved its own cache validity from scratch every run: the serve-time throws gate re-parsed every file (via package_items) just to convert types for a comparison, and plan_reuse read+hashed+decoded 78MB of unit payloads just to extract the tiny throws fragments it seeds from. Profiled: 70% gate, 20% unit loads.

The fix (adversarially reviewed design)

The naive fix (compare throws in "fact space") was killed in adversarial review — it was either circular or unsound (alias-retarget hole). What's implemented is the reviewed survivor:

No-delta gate skip. When the dirty partition finds zero changed/added/removed files, the gate is provably a tautology: the values it would compare are byte-for-byte what the manifest was stored with (verbatim persist; content-addressed units written before the manifest), and the solve+conversion are deterministic. Comparing a value to itself cannot demote anything. Every real edit still takes the full runtime-space gate — the corner the adversary showed the gate genuinely protects (body-only alias retargets) is untouched.
Seeds from manifest, not units. ManifestFile carries a verbatim copy of each unit's throws-fragment blob; seeding reads that. A no-op invocation opens zero unit files. Units load (in parallel, order preserved) only when a delta means relink/re-emit will need them.
Prime wave in the gate path so the package_items derivation (still needed on edits and for package-level diagnostics) is a parallel fold, not a serial parse.
bex_cache: unit content-key is derived from the integrity digest entry-validation already computed — one hash pass per unit instead of two.

Measured

warm scenario	before	after
no-op check, 295k LOC	7.2 s	2.0 s (zero unit reads)
no-op check, 59k LOC	0.54 s	0.30 s
leaf-edit check	~unchanged	emit-bound (it seeds the cache; the follow-up `run`/`pack` then hit in ~0.7 s at 295k LOC)

Risk analysis (what to scrutinize)

Where a bug would live: the no-delta detection (dirty == added == removed == ∅) reuses the same dirty-partition logic every warm compile already trusts; a bug there was already a soundness bug before this PR. The new failure surface is manifest-fragment drift from the unit copy — mitigated by copying from the same in-memory bytes at store time, and by re-pointing both verify oracles (full BAML_CACHE_VERIFY compare + always-on 1-in-32 sampled re-derive) at the manifest copy, i.e. exactly what is served.
Deliberate behavior change: a unit file deleted out from under a warm cache is not detected on a no-op run (nothing reads it); the next compile that needs it falls back to a byte-identical full compile. Slower once, never wrong.
Wire format: old manifests fail borsh decode → treated as absent → one full recompile re-seeds. No mixed-version reads (manifest key includes compiler fingerprint).

Verification run

66 cache unit tests green (incl. the poison-tripwire tests, re-pointed at the manifest copy; the missing-unit degradation test now exercises the delta path where detection still lives).
BAML_CACHE_VERIFY=1 honest-recompile byte-compare: PASS on no-op and after-edit.
BAML_CACHE_SAMPLED_VERIFY=1 forced on 7 consecutive warm invocations across edits: PASS.
Diagnostics byte-identical to the pre-fix binary, cold and warm.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V4oee6wrCHzXH6mM2h8eJo

vercel · 2026-07-21T07:47:43Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
beps	Ready	Preview, Comment	Jul 22, 2026 3:39am
promptfiddle	Ready	Preview, Comment	Jul 22, 2026 3:39am
promptfiddle2	Ready	Preview, Comment	Jul 22, 2026 3:39am

coderabbitai · 2026-07-21T07:47:44Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 31e2bbf2-764c-4317-9616-ddd8ae755348

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch hellovai/warm-floor-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-07-21T07:48:15Z

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

Add RUN_CODSPEED=1 to the PR description, or
Include run-perf or /perf in the PR title or any commit message.

github-actions · 2026-07-21T07:55:22Z

Binary size checks passed

✅ 7 passed

	Artifact	Platform	File	Gzip	Gated on	Baseline	Delta	Status
✅	`baml-cli`	Linux	🔒 25.6 MB	10.9 MB	file	25.3 MB	+347.1 KB (+1.4%)	OK
✅	`packed-program`	Linux	🔒 17.3 MB	7.1 MB	file	17.0 MB	+218.6 KB (+1.3%)	OK
✅	`baml-cli`	macOS	🔒 19.9 MB	9.5 MB	file	19.6 MB	+281.3 KB (+1.4%)	OK
✅	`packed-program`	macOS	🔒 13.5 MB	6.2 MB	file	13.2 MB	+231.5 KB (+1.8%)	OK
✅	`baml-cli`	Windows	🔒 21.4 MB	9.7 MB	file	21.1 MB	+285.7 KB (+1.4%)	OK
✅	`packed-program`	Windows	🔒 14.4 MB	6.3 MB	file	14.2 MB	+200.7 KB (+1.4%)	OK
✅	`bridge_wasm`	WASM	16.3 MB	🔒 4.4 MB	gzip	4.4 MB	+23.6 KB (+0.5%)	OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.

Generated by cargo size-gate · workflow run

…nifest fragments, not unit payloads The warm path spent most of a no-op invocation re-proving its own cache validity: the serve-time throws gate re-derived package_items (re-parsing every file) to convert types for a comparison, and plan_reuse read + hashed + decoded every clean unit payload just to extract the tiny callable-throws fragments it seeds from. At a 295k LOC project that was ~7.2s to conclude "nothing changed" — the same wall time as recompiling from scratch. - No-delta gate skip: when the dirty partition finds no changed, added, or removed files, the gate is provably a tautology — the seeds it would validate are byte-for-byte the values the manifest was stored with (store persists file_throw_facts verbatim; units are content-addressed and written before the manifest), and the solve + runtime conversion are deterministic pure functions of those inputs. Any real delta still takes the full gate. - Manifest-resident fragments: ManifestFile now carries a verbatim copy of each unit's CallableThrowsFragment blob, and the callable_throws seeds project from that copy — so a no-delta invocation opens zero unit payloads. Units are loaded (now in parallel, manifest order preserved) only when a delta means something will be re-emitted or relinked. Both verify oracles (the BAML_CACHE_VERIFY full compare and the 1-in-32 sampled re-derive) now check the manifest copy — exactly the artifact that is served. - The gate path (and every warm collection) primes per-file semantic indexes across workers first, so the package_items derivation the gate and package-level diagnostics demand is a parallel-fed fold instead of a serial parse of the project. - bex_cache: unit_key is derived from the payload's integrity digest, which entry validation already computes — load_unit_shared now reuses that digest instead of hashing the payload a second time. Behavior change, deliberate: a unit file deleted out from under a warm cache is no longer detected on a no-op invocation (nothing reads it); the next compile that needs it falls back to a byte-identical full compile. Slower once, never wrong. Old manifests (without the fragment field) fail borsh decode and are treated as absent — one full recompile re-seeds them. Measured (Apple Silicon): warm no-op check at 295k LOC 7.2s -> 2.0s (zero unit reads, gate skipped); at 59k LOC 0.54s -> 0.30s. Warm leaf-edit check is emit-bound (it seeds the cache for later run/pack hits) and unchanged. Verified: 66 cache tests, BAML_CACHE_VERIFY honest-recompile compare (no-op + after-edit), forced sampled verify across 7 warm invocations, diagnostics byte-identity cold + warm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V4oee6wrCHzXH6mM2h8eJo

vercel Bot deployed to Preview – beps July 21, 2026 07:48 View deployment

vercel Bot deployed to Preview – promptfiddle2 July 21, 2026 07:55 View deployment

vercel Bot deployed to Preview – promptfiddle July 21, 2026 08:09 View deployment

hellovai mentioned this pull request Jul 21, 2026

refactor(cli): shared ProjectSession for all project-loading commands #4105

Draft

hellovai force-pushed the hellovai/compiler-parallel-load-lsp branch from c2da018 to f214619 Compare July 22, 2026 03:17

hellovai force-pushed the hellovai/warm-floor-cache branch from dc5828e to 5573885 Compare July 22, 2026 03:17

vercel Bot deployed to Preview – beps July 22, 2026 03:18 View deployment

vercel Bot deployed to Preview – promptfiddle2 July 22, 2026 03:26 View deployment

vercel Bot deployed to Preview – promptfiddle July 22, 2026 03:39 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(cache): warm no-op checks stop re-proving the cache (draft — review before merge)#4101

perf(cache): warm no-op checks stop re-proving the cache (draft — review before merge)#4101
hellovai wants to merge 1 commit into
hellovai/compiler-parallel-load-lspfrom
hellovai/warm-floor-cache

hellovai commented Jul 21, 2026

Uh oh!

vercel Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 21, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

hellovai commented Jul 21, 2026

The problem

The fix (adversarially reviewed design)

Measured

Risk analysis (what to scrutinize)

Verification run

Uh oh!

vercel Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Jul 21, 2026

⏭️ Performance benchmarks were skipped

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary size checks passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 21, 2026 •

edited

Loading

coderabbitai Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading