feat(client): cache vtxos and sync them incrementally - #264
Conversation
Instead of crawling the entire (ever-growing) vtxo history on every query, the client now caches vtxos and only fetches the delta since the last sync. The server filters GetVtxos by the vtxo's updated_at timestamp (milliseconds), which is bumped on creation, spend, settlement and unroll, so one delta request catches new vtxos and state changes to known ones. Sweeps and expiry updates do not bump updated_at server-side, so the sync additionally refreshes the cached unspent vtxos by outpoint. Spent vtxos are terminal and are never refetched. The watermark is taken from the local clock with a five minute margin to absorb clock skew. The storage is pluggable via the VtxoCacheStore trait so integrators can back the cache with e.g. Redis; InMemoryVtxoCache is the default. Client::clear_vtxo_cache forces a full refetch.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughAdds a pluggable VTXO cache with configurable storage, incremental synchronization, serialized concurrent syncing, script-based requests, timestamp tracking, reusable balance calculation, larger pagination, and cache reset support. ChangesVTXO cache integration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant VtxoCacheStore
participant ArkServer
Client->>VtxoCacheStore: read sync state
Client->>ArkServer: request full or incremental VTXOs
ArkServer-->>Client: return VTXOs
Client->>VtxoCacheStore: upsert and mark synced
Client->>VtxoCacheStore: return cached VTXOs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ark-client/src/lib.rs`:
- Around line 1794-1797: Update clear_vtxo_cache to acquire
self.inner.vtxo_sync_lock before clearing self.inner.vtxo_cache, ensuring cache
invalidation is serialized with in-flight VTXO synchronization. Preserve the
existing async Result return behavior and clear operation.
In `@ark-client/src/vtxo_cache.rs`:
- Around line 38-41: Add store-scoped synchronization to VtxoCacheStore so the
complete sync sequence, including synced_scripts and watermark reads/updates,
executes atomically across all Client instances sharing a store. Expose and use
a lease/transaction mechanism, or maintain per-script watermarks atomically, and
update the client sync flow to hold that protection through delta fetching and
state advancement.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b828c536-c403-4068-88ac-a15c9613acf8
📒 Files selected for processing (4)
ark-client/src/lib.rsark-client/src/utils.rsark-client/src/vtxo_cache.rsark-core/src/server.rs
| /// Clear the VTXO cache, forcing the next VTXO query to fetch everything from the server | ||
| /// again. | ||
| pub async fn clear_vtxo_cache(&self) -> Result<(), Error> { | ||
| self.inner.vtxo_cache.clear().await |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Serialize cache clearing with an in-flight sync.
clear_vtxo_cache can run after upsert but before mark_synced; the sync then records the script and watermark over an empty cache, so later calls return no VTXOs instead of doing a full fetch. Acquire vtxo_sync_lock before clearing.
Proposed fix
pub async fn clear_vtxo_cache(&self) -> Result<(), Error> {
+ let _sync_guard = self.vtxo_sync_lock.lock().await;
self.inner.vtxo_cache.clear().await
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Clear the VTXO cache, forcing the next VTXO query to fetch everything from the server | |
| /// again. | |
| pub async fn clear_vtxo_cache(&self) -> Result<(), Error> { | |
| self.inner.vtxo_cache.clear().await | |
| /// Clear the VTXO cache, forcing the next VTXO query to fetch everything from the server | |
| /// again. | |
| pub async fn clear_vtxo_cache(&self) -> Result<(), Error> { | |
| let _sync_guard = self.vtxo_sync_lock.lock().await; | |
| self.inner.vtxo_cache.clear().await |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ark-client/src/lib.rs` around lines 1794 - 1797, Update clear_vtxo_cache to
acquire self.inner.vtxo_sync_lock before clearing self.inner.vtxo_cache,
ensuring cache invalidation is serialized with in-flight VTXO synchronization.
Preserve the existing async Result return behavior and clear operation.
| /// Implementations must be safe for concurrent use. The client serializes cache _syncs_ | ||
| /// internally, but reads may happen concurrently. | ||
| #[async_trait] | ||
| pub trait VtxoCacheStore: Send + Sync { |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Add cache-scoped synchronization for shared stores.
Send + Sync only protects individual calls. Two Client instances sharing this store can interleave synced_scripts/watermark updates: one can mark a script synced after the other captured its script set, then the other can advance the global watermark without fetching that script’s delta. After the five-minute overlap, its updates can be skipped permanently. Add a store-owned sync lease/transaction, or use atomically maintained per-script watermarks.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ark-client/src/vtxo_cache.rs` around lines 38 - 41, Add store-scoped
synchronization to VtxoCacheStore so the complete sync sequence, including
synced_scripts and watermark reads/updates, executes atomically across all
Client instances sharing a store. Expose and use a lease/transaction mechanism,
or maintain per-script watermarks atomically, and update the client sync flow to
hold that protection through delta fetching and state advancement.
arkana-ai-bot
left a comment
arkana-ai-bot
left a comment
There was a problem hiding this comment.
PROTOCOL-CRITICAL: human review required.
This PR reshapes how the client learns about VTXO state (spent / settled / unrolled / swept / expired) — i.e. the wallet's view of protocol-level truth. The cache logic looks reasonable but has correctness holes worth resolving before merge. Please get a maintainer familiar with arkd's updated_at semantics to sign off.
Server-side claims I verified in arkd (internal/infrastructure/db/sqlite/sqlc/query.sql): upsert (create), UpdateVtxoSpent, UpdateVtxoSettled, UpdateVtxoUnrolled all bump updated_at; UpdateVtxoExpiration and sweep markers do not. So the PR's cache invariants match server behavior today. SelectVtxosWithPubkeys uses updated_at >= :after (inclusive), so passing back an after we've already seen is dupe-safe via upsert.
Findings:
-
clear_vtxo_cachedoesn't holdvtxo_sync_lock(ark-client/src/lib.rs newclear_vtxo_cache, ~L1795). Interleaved with an in-flightget_virtual_tx_outpoints, this sequence is possible:- sync fetches +
upserts a batch, clear()wipes state,- sync completes and calls
mark_synced(unknown_scripts)+set_last_sync_ms(now_ms).
Result:
synced_scriptsis populated andlast_sync_msis set, but the actual VTXOs are gone. The very nextget_virtual_tx_outpointscall sees a non-emptysynced_scripts, skips the full fetch, only issues a delta withafter = now_ms - 5min, and permanently loses every VTXO whoseupdated_atis older than that watermark. On a wallet with any historical activity that means silent balance loss until the caller thinks to clear again. Either acquire the sync lock inclear_vtxo_cache, or document that clearing must not race with sync (with a hard debug-assertion behind the trait). - sync fetches +
-
Clock skew is only compensated in one direction (ark-client/src/vtxo_cache.rs L26–29 and lib.rs delta computation
after = (last_sync_ms - SYNC_MARGIN_MS).max(1)). The watermark is written from the local clock but compared against server-sideupdated_at. If the client is more thanSYNC_MARGIN_MSahead of the server, server updates whose realupdated_atis<= client_watermark - 5minnever come back in future deltas. In practice this hides spent/settled/unrolled transitions — a protocol-critical VTXO state — from the wallet indefinitely, with no error surfaced. Options: keep the max server-observedupdated_at(would need the server response to expose it) and use that as the watermark; or clampSYNC_MARGIN_MSsymmetrically and document the drift budget; or fall back to a periodic full re-sync. This is the failure mode most likely to bite mobile / IoT wallets whose clocks drift. -
No coverage on the sync algorithm itself. The tests in ark-client/src/vtxo_cache.rs exercise only
CacheState(upsert, sort, filter, clear). None of the interesting integration behavior inget_virtual_tx_outpointsis tested: mixed unknown+synced scripts, delta-only path, refresh-outpoints ordering, partial-failure mid-sync, concurrentclearvs sync (#1), first-sync race where a VTXO is upserted beforemark_syncedruns, empty inputs. Danger already flagged this: "Source changed with no test changes — protocol code needs tests." A minimal fakeVtxoCacheStore+ark_grpcmock would let you cover the interesting cases. -
Cross-script staleness (lib.rs new sync flow).
refresh_outpoints = cache.unspent_outpoints_for(&scripts)filters by current request scripts, but the delta covers allsynced_scripts. So if I ever callget_virtual_tx_outpoints({A, B, C}), then later only{A}, the cached unspent VTXOs on B and C never get their swept/expiry state refreshed until someone requests them again.list_vtxos()(which passes all active addresses) sidesteps this; direct callers oflist_vtxos_for_addresseswon't. Either refresh all synced-script unspent outpoints, or document that unspent state on non-requested scripts may lag. -
Unbounded growth / large request bodies.
synced_scriptsonly ever grows — long-lived wallets that churn through addresses will accumulate every historical script forever. And on a mature cache, both the delta request (every synced script) and the refresh request (every unspent outpoint) go out as a single un-chunked gRPC message, whilefetch_all_vtxosonly paginates the response. This will eventually hit the gRPC message-size ceiling. Not a launch blocker, but worth an eviction / chunking follow-up. -
Nit: the module doc says the server "filters
GetVtxosby the VTXO'supdated_attimestamp"; the actual query isupdated_at >= :after(inclusive). Behavior is fine (upsert makes duplicates harmless), but mentioning the inclusive semantic prevents a future contributor from tightening the client to strict>and breaking recovery. -
Cross-repo:
GetVtxosRequest::new_for_scriptsis additive; no existing public API changes. Consumers of ark-core in the other SDKs are unaffected.
Please block on (1) and (2) before merging. (3) is strongly recommended given the protocol surface. (4)–(6) can be follow-ups.
offchain_balance discarded the vtxo list it fetched, so callers needing both the balance and the list paid for two full syncs back-to-back. Move the classification fold into OffChainBalance::from_vtxo_list and add Client::offchain_balance_with_vtxos, which returns the balance together with the list it was computed from. offchain_balance is now a thin wrapper and fetches the server info only once.
The unknown-scripts fetch, the delta fetch and the unspent outpoint refresh are independent, so run them via try_join instead of sequentially. Warm sync latency drops from the sum of the three round trips to the slowest one. The watermark still only advances when all three succeed.
The server honors the requested page size (its per-endpoint max is only the default for requests that don't set one) and re-runs the full vtxo query for every page before paginating in memory, so fewer, larger pages are strictly cheaper on both ends. 2000 vtxos stay comfortably within tonic's 4 MiB response limit.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
ark-client/src/lib.rs (1)
1871-1873: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
clear_vtxo_cachestill doesn't serialize with an in-flight sync.This was flagged in a previous review round on this stack and remains unfixed. If
clear_vtxo_cacheruns concurrently withget_virtual_tx_outpoints, it can execute after that call'supserts but before itsmark_synced/set_last_sync_ms. The in-flight sync then finishes by recording scripts as synced and advancing the watermark over a cache that was just wiped, so subsequent calls treat those scripts as fully synced and never do a full refetch — silently losing all historical VTXOs for them.🔒 Proposed fix
pub async fn clear_vtxo_cache(&self) -> Result<(), Error> { + let _sync_guard = self.vtxo_sync_lock.lock().await; self.inner.vtxo_cache.clear().await }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ark-client/src/lib.rs` around lines 1871 - 1873, Update clear_vtxo_cache to synchronize with the in-flight get_virtual_tx_outpoints sync, using the same lock or coordination mechanism that protects its upsert and mark_synced/set_last_sync_ms sequence. Ensure cache clearing cannot occur between those operations, so a clear either completes before the sync or after the entire sync has finished.
🧹 Nitpick comments (1)
ark-client/src/lib.rs (1)
1834-1845: 🚀 Performance & Scalability | 🔵 TrivialGlobal delta scope will grow with the wallet's known-script set.
delta_futintentionally re-fetches the delta for every script ever marked synced (not just the ones requested), which is necessary for watermark correctness as the comment explains. Butget_offchain_addresses_with_server_info/persist_watch_boarding_outputsenumerate a script per (server key × exit-delay candidate × delegator), sosynced_scriptscan grow substantially for wallets with several deprecated signers/delegators/legacy exit delays. Every futureget_virtual_tx_outpointscall then pays a delta query over that entire, ever-growing set regardless of what was actually requested.Worth keeping an eye on server-side query cost as this list grows; a per-script (or per-batch) watermark could avoid the global re-scan if this becomes a bottleneck in practice.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ark-client/src/lib.rs` around lines 1834 - 1845, The global delta query in delta_fut scans the ever-growing synced_scripts set on every get_virtual_tx_outpoints call; assess and, if appropriate, redesign this flow to use per-script or batched watermarks while preserving complete delta coverage and preventing skipped updates. Focus changes around delta_fut and the associated synced_scripts watermark handling, and retain the existing behavior unless the new scope-aware approach is implemented safely.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@ark-client/src/lib.rs`:
- Around line 1871-1873: Update clear_vtxo_cache to synchronize with the
in-flight get_virtual_tx_outpoints sync, using the same lock or coordination
mechanism that protects its upsert and mark_synced/set_last_sync_ms sequence.
Ensure cache clearing cannot occur between those operations, so a clear either
completes before the sync or after the entire sync has finished.
---
Nitpick comments:
In `@ark-client/src/lib.rs`:
- Around line 1834-1845: The global delta query in delta_fut scans the
ever-growing synced_scripts set on every get_virtual_tx_outpoints call; assess
and, if appropriate, redesign this flow to use per-script or batched watermarks
while preserving complete delta coverage and preventing skipped updates. Focus
changes around delta_fut and the associated synced_scripts watermark handling,
and retain the existing behavior unless the new scope-aware approach is
implemented safely.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8c0493c3-8979-42d7-ab9f-8bcd2424cdf1
📒 Files selected for processing (1)
ark-client/src/lib.rs
arkana-ai-bot
left a comment
arkana-ai-bot
left a comment
There was a problem hiding this comment.
PROTOCOL-CRITICAL: human review required.
Follow-up pass on 8d9f4a0 (3 commits since 625ae15, all in ark-client/src/lib.rs, +113/-61). Zero prior findings fixed; six of seven still open. One new regression from the parallelization commit.
Status of prior findings
clear_vtxo_cachedoesn't holdvtxo_sync_lock— Still open. L1871-1873 untouched. Same silent-balance-loss race as before. CodeRabbit is flagging it independently too.- Unidirectional clock-skew compensation (
SYNC_MARGIN_MSprotects against server-ahead-of-client, not client-ahead-of-server) — Still open. Thedelta_aftercomputation was extracted to L1822 but the semantics are identical. - No test coverage on the sync algorithm — Still open. Danger flags it again. The refactor to
try_join!and the newfrom_vtxo_listsplit make this more urgent, not less — both are new joinpoints that unit tests would have exercised trivially. - Cross-script staleness in
refresh_outpoints(only refreshes unspent outpoints on requested scripts, not all synced ones) — Still open. L1820 unchanged. - Unbounded
synced_scriptsgrowth and un-chunked delta/refresh request bodies — Still open. Commit 8d9f4a0 bumps response pagination (PAGE_SIZE 100→2000) which is fine — I verified inarkd internal/core/application/indexer.goL1397 that the server treatsmaxSizeas a default only whenPageSize <= 0, so the claim in the commit message is accurate, and 2000 fits comfortably under tonic's 4 MiB default. But that is orthogonal to the request-body concern — every synced script and every unspent outpoint still ships in a single un-chunked gRPC message. Follow-up eviction/chunking still warranted. - Doc nit on inclusive
updated_at >= after— Still open.vtxo_cache.rsnot touched. - Cross-repo additive OK — Still applies; no new public API breakage in this delta.
New findings on the incremental diff
8. Parallelizing the three sync fetches introduces a delta-vs-refresh ordering regression (commit 72e67f2, ark-client/src/lib.rs L1856-1861).
The old sequential code guaranteed refresh was queried strictly after delta returned, so refresh always observed at-least-as-fresh state and last-write-wins was correct. In the new try_join! version the three server queries execute in undefined order, but the upserts still run in fixed order: unknown → delta → refresh. That leaves this window:
- VTXO X is cached unspent (so it's in
refresh_outpoints), X's script is insynced_scripts, X'supdated_atis inside the delta window (recent activity). t_refresh_queryexecutes before the spend → returns X as unspent.- Spend of X hits the server →
updated_atbumped. t_delta_queryexecutes after the spend → returns X as spent.- Upsert order: delta (spent) first, then refresh (unspent). Final cache state: unspent.
Because the spend bumps updated_at and the value stays inside the next delta window for SYNC_MARGIN_MS, this self-heals on the next sync — so it's transient staleness, not silent permanent loss like finding #1. Still a real regression from the ordering guarantee the sequential code provided, and a wallet displaying "unspent" for something the server considers spent is protocol-visible.
Simplest fix: swap the upsert order so delta is applied last (delta covers everything with a bumped updated_at, which by definition includes any spend that raced the sync). Cleaner fix: give upsert a spent-wins-over-unspent merge rule, since spent is terminal — that also removes the ordering coupling entirely and would be the more robust invariant for the cache trait to advertise.
9. Nit (commit 8d9f4a0, PAGE_SIZE=2000). No safety net if a VTXO ever carries an unusually long commitment_txids chain or assets list — the response is uncapped, so a pathological account could still exceed tonic's 4 MiB default. Not a launch blocker; consider either bumping max_decoding_message_size explicitly or halving-and-retrying on ResourceExhausted.
10. Refactor commit 60f11e2 looks correct. OffChainBalance::from_vtxo_list is a straight extraction of the existing computation. offchain_balance_with_vtxos fetches server_info once, calls list_vtxos_with_server_info (which takes the sync lock), then classifies — one sync, no double-fetch. offchain_balance is now a thin wrapper. Public API surface grows (new from_vtxo_list, new offchain_balance_with_vtxos) but nothing breaks. No issues.
Recommendation
Please still block on prior findings #1 and #2, and either reorder the upserts or add a spent-wins merge to close #8 (all three are simple, contained changes). Then get a maintainer to sign off on the arkd updated_at semantics assumption before merge.
Inlining three copies of the fetch future into the sync future made it large enough to overflow the thread stack in debug builds (caught by the e2e suite). Box each fetch future so the sync future stays small.
arkana-ai-bot
left a comment
arkana-ai-bot
left a comment
There was a problem hiding this comment.
PROTOCOL-CRITICAL: human review required.
Third pass on 361160a (1 commit, 1 file, +9/-7 in ark-client/src/lib.rs). Pure debug-build stack-overflow fix: each of unknown_fut / delta_fut / refresh_fut is now Box::pin(async { ... }) so the outer sync future stays small. Semantics of the concurrent fetch are unchanged. 0 of 8 prior findings fixed in this delta.
Delta review
The Box::pin wrapping is correct — it heap-allocates each subfuture's state instead of embedding all three into the parent generator, which is the standard fix for try_join!-of-large-async-blocks blowing the debug stack. try_join! still runs them concurrently with the same all-or-nothing cancellation semantics, and the surrounding lock / upsert / mark_synced / set_last_sync_ms sequence is byte-identical (L1805, L1858-1866). No new correctness concerns from this hunk itself.
Worth noting for the author: this incident is a soft signal that this function is doing too much for one async body — the fact that inlining three fetches overflowed the debug stack means the outer future is already carrying a lot of state (guard + cache handle + three script sets + refresh outpoints + delta watermark + three request builders + join output). Boxing works, but a future refactor that extracts the "fetch three deltas concurrently" step into a helper async fn would give the compiler natural boxing at the fn boundary and shrink the outer frame without hand-boxing.
Status of prior findings (all against unchanged code)
clear_vtxo_cachedoesn't holdvtxo_sync_lock(lib.rs L1873-1875) — Still open. Untouched. Silent-balance-loss race persists. CodeRabbit is flagging it independently as well.- Unidirectional clock-skew compensation (lib.rs L1822, vtxo_cache.rs
SYNC_MARGIN_MS) — Still open.delta_aftercomputation unchanged. - No test coverage on the sync algorithm — Still open. Danger flags again ("Source changed with no test changes"). The Box::pin change is exactly the kind of thing a unit test around
get_virtual_tx_outpointswould have caught before e2e — even a smoke test that constructs aClientwith a fakeVtxoCacheStoreand asserts the sync path runs to completion in a debug build would have prevented this round-trip. - Cross-script staleness in
refresh_outpoints(lib.rs L1820) — Still open. Filter scope unchanged. - Unbounded
synced_scriptsgrowth / un-chunked delta+refresh request bodies — Still open. - Doc nit on inclusive
updated_at >= afterin vtxo_cache.rs — Still open. File not touched in this delta. - Cross-repo additive OK — Still applies; no public API change in this hunk.
- Delta-vs-refresh ordering regression from parallelization (lib.rs L1858-1863, upsert order
unknown → delta → refresh) — Still open. The Box::pin change did not touch the upsert order and did not add a spent-wins merge rule. A spent-then-delta-races-refresh window still allows the cache to end in the staleunspentstate until the next sync.
Recommendation
Unchanged from last pass: block on (1) and (2), fix (8) with either an upsert reorder (delta last) or a spent-wins merge rule in the VtxoCacheStore trait, and please add at least a minimal fake-store unit test for get_virtual_tx_outpoints before merge — the pattern of "e2e catches what unit tests would have" is now recurring on this PR. Human protocol review on the arkd updated_at assumptions still recommended.
luckysori
left a comment
luckysori
left a comment
There was a problem hiding this comment.
I did not look at what any agent said, but LGTM. I will run my own AI-assisted review too.
| @@ -0,0 +1,281 @@ | |||
| //! # VTXO cache | |||
| //! | |||
| //! The Ark client caches VTXOs and syncs them incrementally, instead of crawling the entire | |||
There was a problem hiding this comment.
We should use Arkade instead of Ark.
There was a problem hiding this comment.
Just curious, in which specific cases should we use Arkade instead of Ark? In comments only? I am asking this because this is the second time I am seeing this review comment and do not fully understand where specifically Arkade should be used in place of Ark.
There was a problem hiding this comment.
We should mostly use Arkade here because Arkade is an Ark implementation. It also slightly differs to the original Ark protocol as proposed in 2023.
So yes, we should use Arkade here.
4 participants
Instead of crawling the entire (ever-growing) vtxo history on every query, the client now caches vtxos and only fetches the delta since the last sync. The server filters GetVtxos by the vtxo's updated_at timestamp (milliseconds), which is bumped on creation, spend, settlement and unroll, so one delta request catches new vtxos and state changes to known ones.
Sweeps and expiry updates do not bump updated_at server-side, so the sync additionally refreshes the cached unspent vtxos by outpoint. Spent vtxos are terminal and are never refetched. The watermark is taken from the local clock with a five minute margin to absorb clock skew.
The storage is pluggable via the VtxoCacheStore trait so integrators can back the cache with e.g. Redis; InMemoryVtxoCache is the default. Client::clear_vtxo_cache forces a full refetch.
Summary by CodeRabbit