clock for test - #2
Merged
Merged
Conversation
Waiters in fetchOnce blocked on wg.Wait() with no way to honor their own context, and the shared fetch ran on the initiating caller's context, so one caller giving up canceled the fetch and poisoned the result for every waiter. The shared fetch now runs in its own goroutine on a cancel-free copy of the initiator's context (values kept for tracing, cancellation dropped), completion is signaled by closing a channel, and every caller selects on that against its own ctx.Done(). A caller that gives up gets ctx.Err(); the fetch still completes and its result is cached for the others. context.WithoutCancel requires go 1.21, so the module floor moves up from 1.18. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Previously a failed refresh re-put the stale value with a fresh StaleTTL, so every failure pushed the expiry out again: under a long outage staleness was unbounded, and the re-put could overwrite a value a concurrent successful fetch had just stored. Items now record fetchedAt instead of a mutable expiry. A value is fresh until fetchedAt+TTL and may be served stale until fetchedAt+TTL+StaleTTL; past that it is dead and the next lookup fetches synchronously, GetAsync included. Failed refreshes no longer touch the stored item, so the cap never moves and the overwrite race is gone. "Serve stale for at most StaleTTL" is now a guarantee an operator can state. StaleTTL == 0 keeps its existing meaning: Get never serves stale, and GetAsync's serve-while-refreshing staleness stays unbounded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The store was an unbounded sync.Map, which rules the cache out for attacker-influenced key spaces (caching by API key or request path lets a client grow memory without limit). Config.Capacity (<= 0 keeps today's unbounded behavior) now bounds the entry count; on overflow dead entries — past the staleness cap — are evicted first, then arbitrary ones. Enforcing a size atomically needs a size-aware store, so the value store moves from sync.Map to an RWMutex-guarded map; the read path stays a single RLock'd map lookup. Range now iterates a snapshot, so its callback may safely mutate the cache. The inflight/refreshing bookkeeping is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fetch error was always treated as transient, which made stale-if-error dangerous for lookups where the upstream can authoritatively say a key is gone: a deleted model or revoked API key kept being served stale for up to StaleTTL. Config gains two opt-in fields. NotFound classifies a fetch error as an authoritative "does not exist"; such a miss evicts any cached value (authoritative-wins — it is never served stale, in GetStale's error path and in background refreshes alike). NegativeTTL then caches the miss itself: lookups within the TTL are answered from the negative entry without touching the upstream, and misses are never served past it (no stale negatives). NegativeTTL of 0 keeps evicting without caching. OnError no longer fires for authoritative background misses — they are cache state, not failures. With NotFound unset every error stays transient and nothing changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The go.mod floor moved to 1.21 for context.WithoutCancel, so the 1.18 matrix leg fails to build (and fail-fast cancels the stable legs with it). Test 1.21 as the minimum plus stable. Also bump checkout/setup-go past their Node 20 deprecation warnings and disable the setup-go module cache — a zero-dependency module has no go.sum to key it on, which logged a restore warning every run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The clock was already injectable internally; expose it so consumers can pin time in their own tests instead of sleeping through TTLs. Nil keeps time.Now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-flight-stale-cap-capacity
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.