fix(oracle-api): bound the publisher and resolver caches (GH#2416) - #2437
fix(oracle-api): bound the publisher and resolver caches (GH#2416)#2437dcccrypto wants to merge 1 commit into
Conversation
Both oracle routes used module-level Maps as TTL caches with no cap, no
eviction, and no deletion of stale entries — the TTL was only a freshness
check, so an expired entry was ignored but stayed strongly referenced and
unreclaimable.
app/api/oracle/publishers derived its cache key from request-controlled
query params:
const cacheKey = `${mode}:${feedId || authority || ""}`;
and `authority` had no validation at all — any string, any length. So an
unauthenticated caller could mint unlimited unique keys via
mode=admin&authority=<arbitrary> and grow the Map until the process died.
app/api/oracle/resolve/[ca] does validate `ca` as a base58 pubkey, but
any random 32 bytes is a syntactically valid pubkey, so validation alone
does not bound cardinality. Each miss also fans out to Jupiter and
DexScreener, making it upstream request amplification as well.
Add app/lib/bounded-ttl-cache.ts and use it in both routes. It deletes
expired entries on access, sweeps expired entries when the map reaches
its cap, and evicts oldest-first if still at cap, so an insert can never
grow it past maxEntries (500). Eviction is insertion-order rather than
strict LRU — that bounds memory, which is the security property, without
paying a delete+insert on every read; documented as such so nobody
mistakes it for a hit-rate optimisation.
Also on the publishers route:
- validate `authority` as a base58 Solana pubkey (32-44 chars) and
reject with 400 instead of caching it or echoing it back
- serve `admin` mode uncached. getAdminPublishers is a pure local
function over `authority` with no network or decoding, so caching it
bought nothing and was exactly the unbounded-cardinality vector.
Mirrors the eviction approach already used by createMemoryRateLimiter.
Closes #2416
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 6 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
I self-directed to #2416 from the open issue list and targeted
So this change fixes Recommendation: close this PR unless What I got wrong: I picked work from the issue tracker without first checking which branch each issue targeted or which branch was active. I flagged a symptom of this on PR #2439 last cycle (the issue named Full audit of the remaining launch HIGH backlog is in #2440. |
|
Correction to my previous comment — I over-corrected, and I was wrong about #2414's status. Two errors to fix on the record. 1. #2414 IS already fixed on
That is strictly better than my My verification was faulty. I grepped playground for 2. But "redundant → close" was too strong. So Revised status of my three launch PRs:
So the decision is purely about release intent, and I cannot make it:
I am not closing them. Flagging clearly instead so nobody reviews them under a false assumption in either direction. |
|
@Bayyan16 — thanks for the confirmation on #2416. Since you've already validated the approach there, would you be willing to review this one? It's the only thing standing between the live site and that fix. Context to make review quick: Scope: 3 files. New Three changes:
Testing: 18 tests. Verified non-vacuous — with the route reverted to One deliberate design call worth your eye: eviction is insertion-order, not strict LRU — Note: If deploying |
|
Thanks for asking, yes, I’m happy to review #2437. I agree that insertion-order eviction is sufficient for the security invariant here; strict LRU is not required as long as the cache remains hard-bounded. I’ll review the |
Bayyan16
left a comment
There was a problem hiding this comment.
Approved.
I reviewed the targeted main patch against the core #2416 security invariant and validated it locally at commit 0244e3c67536613e9b3f7505672a0b706fe522d5.
Confirmed:
- the process-local publisher and resolver caches are hard-bounded;
- expired entries are removed rather than remaining permanently retained;
- insertion cannot grow either cache beyond
maxEntries; adminrequests no longer populate the shared in-memory cache;- insertion-order eviction is sufficient for the required memory-bound security property; strict LRU is not necessary.
Local validation:
Focused regression tests: 18/18 passed
TypeScript validation: passed
Production build: passed
git diff --check: passed
Working tree: clean
A few non-blocking follow-ups could further strengthen the implementation:
- return
adminresponses withCache-Control: no-store; - prefer canonical Solana
PublicKeyparsing over regex-only authority validation; - strengthen the route regression test so its authority inputs are genuinely unique;
- consider resolver in-flight deduplication and a concurrency cap as additional defense-in-depth.
These follow-ups do not block the targeted production remediation for the persistent unbounded-cache condition reported in #2416.
|
Thanks @Bayyan16 — appreciate the local validation, and specifically for confirming the insertion-order eviction call rather than just waving it through. Status: merge-ready. I'm not merging it myself, and I want to be explicit about why, because last cycle I said this "waits for a reviewer" and that framing was incomplete.
So this needs one of:
Flagging the deploy implication rather than quietly merging, since "approved" and "authorized to ship to users" are different things and I'd rather surface that than assume. |
|
Thanks @dcccrypto for clarifying the deployment boundary. My approval is limited to the code and security review of commit I will not request or perform the merge. I’ll leave that decision to the maintainer or production deployment owner. I’ll also keep #2416 open until either #2437 is merged and confirmed live on |
|
Verification of this one turned up something worth fixing before merge — the fix is good, but the regression test named after the issue doesn't actually catch the issue. Method: disabled only the eviction loop in while (map.size >= maxEntries) { → while (false) {Result:
The But every one of the 10 tests in the gh2416 DoS file passes against an unbounded cache. That file is the one a future reader will trust as "the #2416 regression test", and it would stay green through exactly the regression it's named for — e.g. someone swapping the route back to a plain Suggestion: add one assertion there that drives the route with many unique keys and asserts the cache stops growing, so the protection is pinned at the layer the issue is about. The unit-level guarantee is already proven; what's missing is that the route still uses it. Not blocking on my account — the fix itself is real and I verified it. Flagging because a regression test that can't fail is worse than no test: it makes the next person confident without cause. |
|
Thanks for the mutation check. Agreed: the bounded-cache unit suite correctly catches the eviction regression, but the GH #2416 route suite does not currently pin the route-level memory-bound invariant. The production fix remains valid, but strengthening the route regression before merge would prevent a future route implementation from bypassing the bounded cache while leaving the issue-named test green. My existing approval remains scoped to commit |
Closes #2416 —
[HIGH]Unbounded oracle API caches allow persistent memory-exhaustion DoS.The bug
Both oracle routes used module-level
Maps as TTL caches with no cap, no eviction, and no deletion of stale entries. The TTL was only a freshness check — an expired entry was ignored but stayed strongly referenced, so it could never be garbage collected.app/api/oracle/publishersbuilt its cache key from request-controlled query params:and
authorityhad no validation at all — any string, any length. An unauthenticated caller could mint unlimited unique keys viamode=admin&authority=<arbitrary>and grow the Map until the process died.app/api/oracle/resolve/[ca]does validatecaas a base58 pubkey — but any random 32 bytes is a syntactically valid pubkey, so validation alone does not bound cardinality. Each miss also fans out to Jupiter and DexScreener, so it is upstream request amplification as well as a leak.The fix
New
app/lib/bounded-ttl-cache.ts, used by both routes. It closes the leak three ways:maxEntries(500)Eviction is insertion-order, not strict LRU:
set()on an existing key re-inserts it at the end, but a plainget()does not promote. That bounds memory — the actual security property — without paying a delete+insert on every read. It's documented as such so nobody later mistakes it for a hit-rate optimisation. This mirrors the eviction approach already used bycreateMemoryRateLimiter.Two additional changes on the publishers route:
authorityas a base58 Solana pubkey (32–44 chars), returning 400 rather than caching it or echoing it back in the response body.adminmode uncached.getAdminPublishersis a pure local function overauthority— no network, no decoding. Caching it bought nothing and was precisely the unbounded-cardinality vector, so removing it from the cache eliminates the primary attack path outright; the bound and validation are then defence in depth.Testing
18 new tests.
__tests__/lib/bounded-ttl-cache.test.ts(8) — 5,000 unique keys never exceedmaxEntries; expired entries are deleted on access (assertingsize()drops, which is the leak itself); TTL boundary treated as expired; expired entries swept before live ones are evicted; oldest-first eviction; re-insert refreshes rather than duplicates; invalid construction throws.__tests__/api/gh2416-oracle-publishers-cache-dos.test.ts(10) — drives the real route handler: oversized/short/non-base58/path-traversal/whitespace authorities all rejected 400; a valid authority still resolves; 300 unique unauthenticated requests do not retain proportional heap;modeandfeedIdvalidation unchanged.Verified non-vacuous: with the route reverted to its
origin/mainversion, 6 of the 10 route tests fail. The 4 that still pass are the no-regression guards (valid authority, mode validation, feedId validation), which is correct.Note on typecheck
npx tsc --noEmitreports 19 errors, all in.next/dev/types/validator.ts— stale Next.js build artifacts referencing routes not present in the current dev build. Identical count (19) on cleanorigin/main, and none reference the files touched here, so they are pre-existing and unrelated.🤖 Generated with Claude Code