feat: ordered: 'eager' — concurrent dispatch, in-order delivery - #53
Open
voxpelli wants to merge 6 commits into
Open
feat: ordered: 'eager' — concurrent dispatch, in-order delivery#53voxpelli wants to merge 6 commits into
ordered: 'eager' — concurrent dispatch, in-order delivery#53voxpelli wants to merge 6 commits into
Conversation
Design spike for the requested "out-of-order callbacks, in-order yields" mode: dispatch callbacks (including async generators) concurrently while delivering in strict source order — distinct from, and opt-in alongside, `ordered: true`. This commit lands the design + a typed skeleton for review; the mode is NOT yet functional (passing `ordered: 'eager'` throws "ordered: 'eager' is not yet implemented" at construction). - DESIGN-eager-mode.md: the lane architecture (one lane per in-flight source item, placeholder lanes for source-order + head-wakeup, per-lane K-bounded look-ahead for concurrency without unbounded buffering), abort/cleanup and error-ordering semantics, the drafted ADVANCED.md contract, and the follow-up implementation checklist. Not shipped in the npm package (not in package.json "files"). - index.js: widen `ordered` to `boolean | 'eager'`, validate it (it was previously unvalidated), normalize to a single `mode` and rewrite the four `ordered`-truthiness sites to explicit `mode === 'ordered'` checks so the truthy string can't misroute through the ordered path. Add the typed `Lane<R>` shape and the isolated `admitLanes` / `pumpLane` / `nextValueEager` stubs, gated by one `mode === 'eager'` check at construction and in `next()`. The existing ordered/unordered hot paths are otherwise untouched. - test/eager.spec.js: active specs pinning the not-implemented throw and the new option validation; a skipped block pinning the intended behaviour (concurrency, in-order delivery, bounded look-ahead, cleanup, and both error modes) for the implementation PR to un-skip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
…very Turns the design spike into a working mode. `ordered: 'eager'` dispatches callbacks (async generators included) concurrently up to bufferSize while still delivering in strict source order — the capability `ordered: true` lacks for generator callbacks (which it runs strictly serial). Implementation (all isolated behind `mode === 'eager'`, the ordered/ unordered hot paths untouched): - Lane model: one lane per in-flight source item, in source order, each with a K-bounded private buffer (LANE_LOOKAHEAD = 1). admitLanes admits placeholder lanes (the source's FIFO .next() binds lane order == source order); pumpLane steps non-head lanes one value ahead concurrently then parks them (bounded buffering even for an unbounded non-head generator); nextValueEager only ever waits on the head lane, reusing the same fresh-per-pull park / ABORT_SENTINEL machinery as nextValue (so the exactly-once rejection and per-pull retention contracts hold). - Errors surface in source order, so observable order — including fail-fast picking the source-order-earliest error and a generator's buffered values arriving before its own later error — matches ordered: true. - doCleanup .return()s live lane iterators (deduped against pendingCloses); classifyStep factors the shared result-classification ladder for the two eager helpers without touching the two envelope factories. A subtle bug fixed along the way: admitOneLane must keep lane.pending set across the callback dispatch's await and clear it only once the lane settles — clearing it early left a parked consumer racing an instantly-resolving undefined and busy-looping (OOM on a fast source). Tests (test/eager.spec.js, 17 specs): option handling, real concurrency (maxInFlight === bufferSize vs 1 for ordered:true), in-order delivery, faster-than-ordered timing, bounded look-ahead under an unbounded generator, cleanup of every live lane, both error modes, and malformed-result/hostile-getter handling. Plus an eager retention case in test/memory.spec.js and eager benchmark rows in benchmark/nested.js. Docs (ADVANCED.md "Ordered mode", README, DESIGN-eager-mode.md) updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
ordered: 'eager' — concurrent dispatch, in-order delivery (design spike)ordered: 'eager' — concurrent dispatch, in-order delivery
voxpelli
marked this pull request as ready for review
July 11, 2026 10:07
The per-lane look-ahead in eager mode (how many values a not-yet-at-head input may buffer ahead of delivery) was the module constant LANE_LOOKAHEAD = 1. Expose it as the `lookahead` option so a deep per-item pipeline (e.g. paginated fetches whose cost is spread across yields) can let non-head lanes run further ahead. - index.js: rename LANE_LOOKAHEAD → DEFAULT_LANE_LOOKAHEAD (still 1); add `lookahead` to the options (no default so an explicit value is detectable); validate it is a positive integer AND reject it outright with any non-eager mode (it would be a no-op there) — matching the strict validation of the other options; thread it into pumpLane. Total buffering stays bounded at bufferSize × lookahead. JSDoc unions updated; mergeIterables inherits it by forwarding options. - Flat per-lane allocation is deliberate: buffers and step-slots are per-lane and independent, so the head is never internally starved, and the head (a serial, consumer-paced generator) can't use a bigger share anyway. The only real contention is external, for which bufferSize is the lever. Documented in ADVANCED.md + DESIGN-eager-mode.md. Tests: validation specs (positive-integer; eager-only mismatch throws) and a lookahead:3 behaviour spec (an unbounded non-head generator is stepped exactly 3 times, then parks). This also fixes the prior bounded-look-ahead spec, which used an async *arrow* returning a generator — delivered as a plain value, never fanned out — so it passed its `at.most(1)` assertion vacuously (0 steps). It now uses an async generator *function* and asserts the exact step count. Docs: ADVANCED.md, README (both signatures + options), and DESIGN-eager-mode.md updated, including the memory↔critical-path trade-off and why the allocation is flat rather than head-prioritising. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
The design file was a review artifact for the spike. With the mode
implemented and documented, the README and ADVANCED.md ("Ordered mode")
are the source of truth. Repoint the few in-code/test comments that
referenced the design file at ADVANCED.md's Ordered mode section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
At the default lookahead of 1, a generator whose cost is spread across its yields runs each non-head lane one value ahead and then parks, so bufferSize has no effect and the mode performs like ordered: true. Document the magnitude (not just the axis), frame bufferSize × lookahead as a memory budget in values, and name the hard limit: for a very-long or unbounded spread-cost generator no lookahead both bounds memory and recovers full concurrency — prefer ordered: false there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
Close two gaps in ordered: 'eager' coverage: an external abort now has an eager case asserting the exactly-once-rejection-then-done contract in source order (pinning ADVANCED.md's "abort delivery observably identical to ordered: true"), and early close now covers a non-head lane still holding lookahead > 1 buffered values, asserting its iterator is still returned. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A new opt-in mode that fills the one gap in ordered delivery: out-of-order callback dispatch with in-order yields.
Under
ordered: true, async-generator callbacks run strictly serial — the buffer always feeds from the current sub-iterator, sobufferSizeonly buffers the undispatched generator objects and never steps them concurrently.ordered: 'eager'delivers in strict source order likeordered: true, but dispatches callbacks — generators included — concurrently up tobufferSize.Implementation — the lane model
Full contract in ADVANCED.md, "Ordered mode". Fully isolated behind one
mode === 'eager'check at construction and innext(); the ordered/unordered hot paths only get the equivalentordered→mode === 'ordered'renames, so their V8 shapes are untouched.Laneper in-flight source item, in source order (lanes[0]= delivery head), each with a private buffer of ≤lookaheadvalues.admitLanesadmits placeholder lanes; the source's own FIFO.next()binds lane order == source order, and a placeholder always exists as a head to race (no long-lived promise — the memory invariant holds).pumpLanesteps non-head lanes up tolookaheadvalues ahead concurrently, then parks them. Total buffering ≤bufferSize × lookahead, independent of an unbounded non-head generator.nextValueEageronly ever waits on the head lane, reusing the same fresh-per-pull park /ABORT_SENTINELmachinery asnextValue(exactly-once rejection + retention contracts preserved).ordered: true.doCleanup.return()s live lane iterators (deduped vspendingCloses); a sharedclassifyStepfactors the result-classification ladder for the two eager helpers without touching the two envelope factories.lookaheadoptionThe per-lane buffer depth is exposed as
lookahead(positive integer, default1,ordered: 'eager'only — throws with any other mode). It trades memory (and, under shared-resource contention, the head's critical-path priority) for pipeline depth on deep per-item generators. The allocation is deliberately flat (onelookaheadfor every lane): buffers and step-slots are per-lane and independent, so the head is never internally starved, and a serial/consumer-paced head can't use a bigger share anyway. External contention is abufferSizeconcern the library can't schedule. (Rationale in ADVANCED.md.)Magnitude worth knowing (documented in ADVANCED.md): at the default
lookahead: 1, a generator whose cost is spread across its yields (rather than before the first) gets no benefit frombufferSize— each non-head lane runs one value ahead then parks, so the mode performs likeordered: trueand the shortfall grows with thebufferSizeyou asked for. TreatbufferSize × lookaheadas a memory budget in values. There is a genuine limit underneath: for a very-long or unbounded spread-cost generator, nolookaheadboth bounds memory and recovers full concurrency —ordered: falseis the better fit there if out-of-order delivery is acceptable.Tests & verification
npm testgreen — 166 passing, type-coverage 99.25% strict, coverage 99.92%, lint clean.test/eager.spec.js: option handling +lookaheadvalidation; real concurrency (maxInFlight === bufferSizevs1forordered: true); in-order delivery under reordering; faster-than-ordered:truetiming; bounded look-ahead (an unbounded non-head generator is stepped exactlylookaheadtimes, then parks — default 1 and a custom 3); cleanup of every live lane; external-abort delivery (exactly-once rejection then done, in source order — identical toordered: true); early close withlookahead > 1buffered values (the non-head lane is still.return()ed); both error modes; and malformed-result / hostile-getter handling.test/memory.spec.js: an eager retention case alongside the unordered one.benchmark/nested.js:ordered: trueandordered: 'eager'fan-out rows.Notable bugs caught & fixed
admitOneLanemust keeplane.pendingset across the callback dispatch'sawaitand clear it only once the lane settles — clearing it early left a parked consumer racing an instantly-resolvingundefined→ busy-loop. Fixed and pinned by the array-sourced error specs.at.most(1)with 0 steps. Rewritten to use an async generator function and assert the exact step count.Breaking changes (v2)
orderednow throws on a non-boolean, non-'eager'value.{ ordered: 1 },{ ordered: 'yes' },{ ordered: null }all worked in v1.0.1 (coerced by truthiness) and now throw aTypeError. Correct — a typo shouldn't silently pick a mode — but a genuine break for anyone writingordered: flag ? 1 : 0, and the strongest reason this can't ship as a minor.Release-please note (maintainer decision, not made here): this branch is
feat:-driven, which release-please cuts as a minor. For it to cut a major, a commit in the release needs aBREAKING CHANGE:footer. I have deliberately not added one — whether to force the major now or cut v2 by hand is a release-timing call for you.Decisions (settled)
ordered: 'eager'(widenedorderedtoboolean | 'eager').lookahead: exposed as an option, eager-only, default 1, flat per-lane allocation.ordered: trueexactly. The only cost is not short-circuiting quite as early — a latency detail, not a semantic one.🤖 Generated with Claude Code
https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu