Skip to content

feat: ordered: 'eager' — concurrent dispatch, in-order delivery - #53

Open
voxpelli wants to merge 6 commits into
claude/merge-branches-to-master-nixCBfrom
claude/eager-ordered-mode-spike
Open

feat: ordered: 'eager' — concurrent dispatch, in-order delivery#53
voxpelli wants to merge 6 commits into
claude/merge-branches-to-master-nixCBfrom
claude/eager-ordered-mode-spike

Conversation

@voxpelli

@voxpelli voxpelli commented Jul 11, 2026

Copy link
Copy Markdown
Owner

What this adds

A new opt-in mode that fills the one gap in ordered delivery: out-of-order callback dispatch with in-order yields.

Under ordered: true, async-generator callbacks run strictly serial — the buffer always feeds from the current sub-iterator, so bufferSize only buffers the undispatched generator objects and never steps them concurrently. ordered: 'eager' delivers in strict source order like ordered: true, but dispatches callbacks — generators included — concurrently up to bufferSize.

Stacked on #50 (which introduces ordered: true). Review/merge that first; this PR's diff resolves once #50 lands.

Implementation — the lane model

Full contract in ADVANCED.md, "Ordered mode". Fully isolated behind one mode === 'eager' check at construction and in next(); the ordered/unordered hot paths only get the equivalent orderedmode === 'ordered' renames, so their V8 shapes are untouched.

  • Lanes — one Lane per in-flight source item, in source order (lanes[0] = delivery head), each with a private buffer of ≤ lookahead values.
  • admitLanes admits placeholder lanes; the source's own FIFO .next() binds lane order == source order, and a placeholder always exists as a head to race (no long-lived promise — the memory invariant holds).
  • pumpLane steps non-head lanes up to lookahead values ahead concurrently, then parks them. Total buffering ≤ bufferSize × lookahead, independent of an unbounded non-head generator.
  • nextValueEager only ever waits on the head lane, reusing the same fresh-per-pull park / ABORT_SENTINEL machinery as nextValue (exactly-once rejection + retention contracts preserved).
  • Errors surface in source order, so observable order — fail-fast picking the source-order-earliest error, and a generator's buffered values arriving before its own later error — matches ordered: true.
  • doCleanup .return()s live lane iterators (deduped vs pendingCloses); a shared classifyStep factors the result-classification ladder for the two eager helpers without touching the two envelope factories.

lookahead option

The per-lane buffer depth is exposed as lookahead (positive integer, default 1, ordered: 'eager' only — throws with any other mode). It trades memory (and, under shared-resource contention, the head's critical-path priority) for pipeline depth on deep per-item generators. The allocation is deliberately flat (one lookahead for every lane): buffers and step-slots are per-lane and independent, so the head is never internally starved, and a serial/consumer-paced head can't use a bigger share anyway. External contention is a bufferSize concern the library can't schedule. (Rationale in ADVANCED.md.)

Magnitude worth knowing (documented in ADVANCED.md): at the default lookahead: 1, a generator whose cost is spread across its yields (rather than before the first) gets no benefit from bufferSize — each non-head lane runs one value ahead then parks, so the mode performs like ordered: true and the shortfall grows with the bufferSize you asked for. Treat bufferSize × lookahead as a memory budget in values. There is a genuine limit underneath: for a very-long or unbounded spread-cost generator, no lookahead both bounds memory and recovers full concurrency — ordered: false is the better fit there if out-of-order delivery is acceptable.

Tests & verification

npm test green — 166 passing, type-coverage 99.25% strict, coverage 99.92%, lint clean.

  • test/eager.spec.js: option handling + lookahead validation; real concurrency (maxInFlight === bufferSize vs 1 for ordered: true); in-order delivery under reordering; faster-than-ordered:true timing; bounded look-ahead (an unbounded non-head generator is stepped exactly lookahead times, then parks — default 1 and a custom 3); cleanup of every live lane; external-abort delivery (exactly-once rejection then done, in source order — identical to ordered: true); early close with lookahead > 1 buffered values (the non-head lane is still .return()ed); both error modes; and malformed-result / hostile-getter handling.
  • test/memory.spec.js: an eager retention case alongside the unordered one.
  • benchmark/nested.js: ordered: true and ordered: 'eager' fan-out rows.

Notable bugs caught & fixed

  • OOM on a fast source: admitOneLane must keep lane.pending set across the callback dispatch's await and clear it only once the lane settles — clearing it early left a parked consumer racing an instantly-resolving undefined → busy-loop. Fixed and pinned by the array-sourced error specs.
  • Vacuous look-ahead test: the original bounded-look-ahead spec used an async arrow returning a generator (delivered as a plain value, never fanned out), so it passed at.most(1) with 0 steps. Rewritten to use an async generator function and assert the exact step count.

Breaking changes (v2)

  • ordered now throws on a non-boolean, non-'eager' value. { ordered: 1 }, { ordered: 'yes' }, { ordered: null } all worked in v1.0.1 (coerced by truthiness) and now throw a TypeError. Correct — a typo shouldn't silently pick a mode — but a genuine break for anyone writing ordered: flag ? 1 : 0, and the strongest reason this can't ship as a minor.

Release-please note (maintainer decision, not made here): this branch is feat:-driven, which release-please cuts as a minor. For it to cut a major, a commit in the release needs a BREAKING CHANGE: footer. I have deliberately not added one — whether to force the major now or cut v2 by hand is a release-timing call for you.

Decisions (settled)

  • API: ordered: 'eager' (widened ordered to boolean | 'eager').
  • lookahead: exposed as an option, eager-only, default 1, flat per-lane allocation.
  • Fail-fast ordering: surfaces the source-order-earliest error (not the chronologically-first). Not a concession — if delivery is in source order, an error is just another thing delivered in that order; the surprising mode would deliver values in source order but errors in wall-clock order. ADVANCED.md already states this as a flat property, and it matches ordered: true exactly. The only cost is not short-circuiting quite as early — a latency detail, not a semantic one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu

claude added 2 commits July 11, 2026 09:18
Design spike for the requested "out-of-order callbacks, in-order yields"
mode: dispatch callbacks (including async generators) concurrently while
delivering in strict source order — distinct from, and opt-in alongside,
`ordered: true`. This commit lands the design + a typed skeleton for
review; the mode is NOT yet functional (passing `ordered: 'eager'`
throws "ordered: 'eager' is not yet implemented" at construction).

- DESIGN-eager-mode.md: the lane architecture (one lane per in-flight
  source item, placeholder lanes for source-order + head-wakeup,
  per-lane K-bounded look-ahead for concurrency without unbounded
  buffering), abort/cleanup and error-ordering semantics, the drafted
  ADVANCED.md contract, and the follow-up implementation checklist. Not
  shipped in the npm package (not in package.json "files").
- index.js: widen `ordered` to `boolean | 'eager'`, validate it (it was
  previously unvalidated), normalize to a single `mode` and rewrite the
  four `ordered`-truthiness sites to explicit `mode === 'ordered'` checks
  so the truthy string can't misroute through the ordered path. Add the
  typed `Lane<R>` shape and the isolated `admitLanes` / `pumpLane` /
  `nextValueEager` stubs, gated by one `mode === 'eager'` check at
  construction and in `next()`. The existing ordered/unordered hot paths
  are otherwise untouched.
- test/eager.spec.js: active specs pinning the not-implemented throw and
  the new option validation; a skipped block pinning the intended
  behaviour (concurrency, in-order delivery, bounded look-ahead,
  cleanup, and both error modes) for the implementation PR to un-skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
…very

Turns the design spike into a working mode. `ordered: 'eager'` dispatches
callbacks (async generators included) concurrently up to bufferSize while
still delivering in strict source order — the capability `ordered: true`
lacks for generator callbacks (which it runs strictly serial).

Implementation (all isolated behind `mode === 'eager'`, the ordered/
unordered hot paths untouched):

- Lane model: one lane per in-flight source item, in source order, each
  with a K-bounded private buffer (LANE_LOOKAHEAD = 1). admitLanes admits
  placeholder lanes (the source's FIFO .next() binds lane order == source
  order); pumpLane steps non-head lanes one value ahead concurrently then
  parks them (bounded buffering even for an unbounded non-head generator);
  nextValueEager only ever waits on the head lane, reusing the same
  fresh-per-pull park / ABORT_SENTINEL machinery as nextValue (so the
  exactly-once rejection and per-pull retention contracts hold).
- Errors surface in source order, so observable order — including fail-fast
  picking the source-order-earliest error and a generator's buffered values
  arriving before its own later error — matches ordered: true.
- doCleanup .return()s live lane iterators (deduped against pendingCloses);
  classifyStep factors the shared result-classification ladder for the two
  eager helpers without touching the two envelope factories.

A subtle bug fixed along the way: admitOneLane must keep lane.pending set
across the callback dispatch's await and clear it only once the lane
settles — clearing it early left a parked consumer racing an
instantly-resolving undefined and busy-looping (OOM on a fast source).

Tests (test/eager.spec.js, 17 specs): option handling, real concurrency
(maxInFlight === bufferSize vs 1 for ordered:true), in-order delivery,
faster-than-ordered timing, bounded look-ahead under an unbounded
generator, cleanup of every live lane, both error modes, and
malformed-result/hostile-getter handling. Plus an eager retention case in
test/memory.spec.js and eager benchmark rows in benchmark/nested.js.
Docs (ADVANCED.md "Ordered mode", README, DESIGN-eager-mode.md) updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
@voxpelli voxpelli changed the title feat: scaffold ordered: 'eager' — concurrent dispatch, in-order delivery (design spike) feat: ordered: 'eager' — concurrent dispatch, in-order delivery Jul 11, 2026
@voxpelli
voxpelli marked this pull request as ready for review July 11, 2026 10:07
claude added 4 commits July 11, 2026 14:30
The per-lane look-ahead in eager mode (how many values a not-yet-at-head
input may buffer ahead of delivery) was the module constant
LANE_LOOKAHEAD = 1. Expose it as the `lookahead` option so a deep
per-item pipeline (e.g. paginated fetches whose cost is spread across
yields) can let non-head lanes run further ahead.

- index.js: rename LANE_LOOKAHEAD → DEFAULT_LANE_LOOKAHEAD (still 1); add
  `lookahead` to the options (no default so an explicit value is
  detectable); validate it is a positive integer AND reject it outright
  with any non-eager mode (it would be a no-op there) — matching the
  strict validation of the other options; thread it into pumpLane. Total
  buffering stays bounded at bufferSize × lookahead. JSDoc unions updated;
  mergeIterables inherits it by forwarding options.
- Flat per-lane allocation is deliberate: buffers and step-slots are
  per-lane and independent, so the head is never internally starved, and
  the head (a serial, consumer-paced generator) can't use a bigger share
  anyway. The only real contention is external, for which bufferSize is
  the lever. Documented in ADVANCED.md + DESIGN-eager-mode.md.

Tests: validation specs (positive-integer; eager-only mismatch throws)
and a lookahead:3 behaviour spec (an unbounded non-head generator is
stepped exactly 3 times, then parks). This also fixes the prior
bounded-look-ahead spec, which used an async *arrow* returning a
generator — delivered as a plain value, never fanned out — so it passed
its `at.most(1)` assertion vacuously (0 steps). It now uses an async
generator *function* and asserts the exact step count.

Docs: ADVANCED.md, README (both signatures + options), and
DESIGN-eager-mode.md updated, including the memory↔critical-path
trade-off and why the allocation is flat rather than head-prioritising.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
The design file was a review artifact for the spike. With the mode
implemented and documented, the README and ADVANCED.md ("Ordered mode")
are the source of truth. Repoint the few in-code/test comments that
referenced the design file at ADVANCED.md's Ordered mode section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
At the default lookahead of 1, a generator whose cost is spread across its
yields runs each non-head lane one value ahead and then parks, so bufferSize
has no effect and the mode performs like ordered: true. Document the
magnitude (not just the axis), frame bufferSize × lookahead as a memory
budget in values, and name the hard limit: for a very-long or unbounded
spread-cost generator no lookahead both bounds memory and recovers full
concurrency — prefer ordered: false there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
Close two gaps in ordered: 'eager' coverage: an external abort now has an
eager case asserting the exactly-once-rejection-then-done contract in source
order (pinning ADVANCED.md's "abort delivery observably identical to
ordered: true"), and early close now covers a non-head lane still holding
lookahead > 1 buffered values, asserting its iterator is still returned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSyTJjbP823pjYfueuSsdu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants