Skip to content

triedb/pathdb: persist address-biased preload cache across restarts - #2396

Draft
lucca30 wants to merge 6 commits into
developfrom
lmartins/preload-persist-repair
Draft

lucca30 wants to merge 6 commits into
developfrom
lmartins/preload-persist-repair

Conversation

@lucca30

@lucca30 lucca30 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

AddressBiasedCache (triedb/pathdb) warms dedicated in-RAM caches for a handful of large,
rarely-mutated contracts on mainnet block producers via cache.addresscachesizes, but does
a full rate-limited BFS walk from scratch on every startup and unconditionally wipes the
cache on every graceful shutdown. On the real BP config (4 addresses, up to 8196MB, at the
deployed 512KB/s preload rate limit) this costs up to ~3h of degraded-cache operation on
every planned restart.

This PR persists each address's cache to <TrieJournalDirectory>/addresscache/<accountHash>.cache
on genuine graceful Database.Close() (not on Disable() or Journal() — see below), and
reloads it via fastcache.LoadFromFileOrNew on startup, skipping the preload walk when the
reload reaches the same 2/3-fill target the preload itself targets. Correctness does not
depend on freshness: reader.Node() already hash-verifies every cache hit and evicts+refetches
on mismatch regardless of why an entry is stale, so a stale persisted snapshot can only cost
efficiency, never correctness. commonCache is untouched — only the configured per-address
caches are persisted. Full design rationale: agent-zero repo,
docs/superpowers/specs/2026-09-03-bor-preload-persist-repair-design.md.

One correction made during implementation: diskLayer.terminate() has 3 callers
(Database.Close, Database.Disable, Journal), not 1. Persisting on all 3 would
double-save on every real shutdown and add a redundant multi-GB write on Disable()
(snap-sync entry, not a restart). Fixed by threading a persist bool through
terminate()/AddressBiasedCache.Close(persist bool) — only Database.Close() passes true.

Executed tests

  • Full unit suite (go test ./triedb/...): passing, race-clean.

  • Live validation on a disposable GCP disk-clone of posdevnodes-mainnet-bor-1 (a synced
    mainnet full node), configured with the real BP addresscachesizes/preloadratelimit
    from anonymous-91..94:

    Cold (today) Warm (this PR)
    Preload wall-clock (4 addresses, bound by the 8196MB one) 2h53m20s ~4.5s
    Cache-hit rate post-restart (pathdb_biased_address_{hit,miss}) n/a 99.58%
    Block-processing insert latency (p50, steady state) 74.9ms 69.0ms
    Shutdown wall-clock (systemctl stop) 18-20s (pre-existing trie journal + peer teardown) +~4.8s for the persist write (8.1GB total)

    Also validated: two consecutive persist→reload cycles (no staleness compounding); a
    deliberately-deleted single-address snapshot correctly falls back to cold preload for
    that address only, while the other 3 stay warm; zero panics/hash-mismatch/corruption
    signals across the whole run.

Rollout notes

Not consensus-affecting — this is a read-cache optimization only, and correctness is
backstopped by the pre-existing, unmodified reader.Node() hash-verify-and-evict path.
No coordinated upgrade needed. Backwards-compatible: no new required config (the
persistence directory derives from the existing JournalDirectory); a node with no prior
snapshot, a mismatched cache size, or a corrupted file all fall back to today's cold-walk
behavior unchanged. Operator-visible additions: a new <datadir>/.../triedb/addresscache/
directory on disk (sized to the configured addresscachesizes, see table above) and a new
"Reloaded address cache snapshot" info log line on warm startup. Orphaned snapshot files
for an address later removed from config are not auto-cleaned (documented, deliberate v1
scope decision — not a leak, just a manual-cleanup wart).

terminate() has three callers (Close, Journal, Disable) but only Close
represents the node actually restarting; Journal and Disable now pass
persist=false so the address-biased cache save no longer runs redundantly
or ahead of the trie journal write. Also: a reload is only considered warm
once it reaches the same 2/3 fill target the preload itself targets (a
snapshot saved mid-preload no longer permanently skips the top-up), the
size-mismatch regression test now uses sizes that are actually rejected by
fastcache instead of silently accepted ones, and a warm reload now logs.
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.45614% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.55%. Comparing base (39ec8c4) to head (3554d31).
⚠️ Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
triedb/pathdb/biased_fastcache.go 86.27% 5 Missing and 2 partials ⚠️
triedb/pathdb/database.go 0.00% 0 Missing and 2 partials ⚠️
triedb/pathdb/journal.go 0.00% 0 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (82.45%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2396      +/-   ##
===========================================
- Coverage    55.57%   55.55%   -0.02%     
===========================================
  Files          918      918              
  Lines       167133   167177      +44     
===========================================
- Hits         92877    92870       -7     
- Misses       68775    68830      +55     
+ Partials      5481     5477       -4     
Files with missing lines Coverage Δ
triedb/pathdb/disklayer.go 81.65% <100.00%> (ø)
triedb/pathdb/journal.go 46.84% <0.00%> (ø)
triedb/pathdb/database.go 59.07% <0.00%> (ø)
triedb/pathdb/biased_fastcache.go 88.49% <86.27%> (-0.11%) ⬇️

... and 19 files with indirect coverage changes

Files with missing lines Coverage Δ
triedb/pathdb/disklayer.go 81.65% <100.00%> (ø)
triedb/pathdb/journal.go 46.84% <0.00%> (ø)
triedb/pathdb/database.go 59.07% <0.00%> (ø)
triedb/pathdb/biased_fastcache.go 88.49% <86.27%> (-0.11%) ⬇️

... and 19 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant