fix(sessions): pin a session per weekly bucket, not once overall - #189
Open
ak2k wants to merge 1 commit into
Open
fix(sessions): pin a session per weekly bucket, not once overall#189ak2k wants to merge 1 commit into
ak2k wants to merge 1 commit into
Conversation
A session held one pin, so a routing decision taken for one model family relocated the whole session. An account whose Fable weekly is spent still serves Opus, so a Fable request diverts to another account, re-pins the session there, and the next Opus request follows it onto an account nothing evaluated for Opus and whose Opus cache is cold. Whichever family routed last owned the session. Key each pin by the weekly bucket that governs the request, using the key selection already applies for eligibility, so the families keep independent affinities. recordSession takes the request's model to name the bucket it spent; session selection looks up the executor's. The load metric follows the pin: a session counts on every account it is currently spending, once each, and a pin stops counting once its bucket has been quiet for the active window. Finishing a request ages the pin it was spending, so an account does not drop out of the metric the moment it completes a long stream. removeAccount renumbers session pins the way it already renumbers route pins.
This was referenced Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A session holds one pin, so whichever model family routed last owns the whole session.
The bug
Quota and eligibility are already decided per weekly bucket: an account whose Fable weekly is spent still serves Opus, and
_isAvailablesays so. Session affinity is the one place that reads a single answer out of a per-bucket world. A Fable request diverted off the pinned account re-pins the session there, and the next Opus request follows it onto an account nothing evaluated for Opus, whose Opus cache is cold.Measured with a fixed schedule (one session interleaving Opus and Fable across four accounts, account
i's Fable weekly going spent after phasei, both trees seeing identical fleet state at every boundary): before, the Opus traffic re-homed twice, walking accounts it was never evaluated for; after, it holds its account for all nine of its Opus turns while the Fable half diverts independently.The fix
Key each pin by the bucket that governs the request, using
_weeklyBucketFor, the same key selection already uses for eligibility.recordSessionpins the bucket the request spent;_selectForSessionlooks up the executor's.removeAccountremaps session pins beside the route pins it already remaps, so both survive account removal symmetrically.Scope
distributeSessionsgates the whole routing effect, and it defaults off. Flag off, 300+ interleaved requests with realistic durations across four families: account sequences byte-identical to master, per-session attribution 0 split / 0 unattributed on both trees.What the flag does not gate is the readout: pins are recorded either way, and
perAccountcounts a split session on each account it is live on, once each, so it can sum to more thanactive. The invariant that holds: a routed, currently active session is counted on at least one account. One narrow case is disclosed in the body of the change: with two concurrent requests on different buckets, the account whose request ends last can read idle for up to the activity window.Tests
525 before, 541 after, 0 fail;
npx eslint .clean. Reverting the pin keying with the new tests in place: 532 pass, 9 fail. Dropping only the end-of-request refresh: 538 pass, 3 fail. Dropping only the remap: 539 pass, 2 fail. Each red test names the behavior it pins.Notes
This is the base of a small stack: the expiry-routing PR and the accounting PR build on it. Merging out of order will conflict.