[pull] main from vercel:main - #512
Merged
Merged
Conversation
* Add world-sim: a deterministic simulation World for concurrency scenarios `@workflow/world-sim` is an in-process World implementation whose point is that nothing in it races. Every world call is a stoppable point with `before`/`positioned`/`after` phases, every call is attributed to a writer (orchestrator, a named step body, or an out-of-band external client), and the clock is virtual — a thirty-day sleep costs no wall time. A scenario script says "stop this writer here, do that, let it go", so an interleaving that a real deployment leaves to chance becomes something you can name. The model follows workflow-server where it matters: - Event ids are minted at the handler boundary, not at storage append. DynamoDB does not generate ids, so the server does, in the request handler — and that id is the log's sort key. This is what makes "earlier log position, later commit" expressible, and it is the hazard the fence scenarios are about. - The out-of-band write marker is keyed on the event's own ULID time and is forward-only, over `hook_received`, `step_completed` and `step_failed`. - Both halves of the staleness fence are modelled behind flags: the watermark (`preconditionGuard`) that clients send today, and the count (`countGuard`) that they do not. The count is synthesized on the caller's behalf and keyed by run rather than by writer, since an orchestrator and its inline step bodies are one process sharing one loaded log. `workbench/sim-world` is the scenario book — 39 of them, each a workflow plus a script. Several are pinned corruptions rather than passing assertions: they record what the runtime does today, so that a change in behaviour shows up as a diff. The doc-29/30/31 trio is the argument for the count guard, one flag apart: (B, A, C) corrupts under the watermark alone, is fenced once the count is on, and (B, C, A) corrupts with both on. That last one suspends mid-run on purpose. The fence is a conditional append evaluated inside the storage write, so there is no checked-but-uncommitted moment to slip past; the window nothing can close is the quiescent gap between deliveries, where the run makes no writes and so meets no checks. Both packages are private and unpublished, so no changeset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Add the new sim-world workspaces to the lockfile Kept separate from the source commit because it is not a clean diff. The two new importers are the part that belongs to this branch; the rest is re-resolution churn from running `pnpm install` at a later date than whoever last touched the file — `latest` specifiers like docs' `radix-ui` move on their own. Drop or regenerate this commit if the churn is unwelcome; the source commit stands on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Make a consistency violation fail the scenario that tripped it `expect.violations` let a scenario declare the corruption it reproduced and pass for reproducing it. That is a green suite describing a broken system, and it goes red on the day someone *fixes* the bug — backwards, and the opposite of what a test is for. The field is gone. A scenario now states the outcome the run should have reached, which for a corruption means the branch its own durable log implies, and stays red until the runtime gets there. Any violation fails the scenario. The six reproductions read as ordinary failures now: expected output "afterSlow:doc-26", got "afterFast:doc-26" Six scenarios are therefore red, and `run.ts` exits non-zero. The count is the signal: seven is a regression, five means something was fixed and a scenario is ready to retire. Five of the six have known fixes — four predate the count guard, and doc-29 goes green the moment a client sends `stateEventCount`. doc-31 has none, which is the point of it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Document the world-sim implementation in DESIGN.md Folds the working notes into a checked-in design doc: module map, the runtime model the simulator has to match, the interception model and its three call phases, the determinism machinery, the store's guards and fault injection, the writer vocabulary, the termination budgets, both consistency checkers, and current test status. Links it from both READMEs, and updates the workbench's doc-31 note now that the append-tail fence it needs exists as a proposal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Say which fix closes each red scenario, and which are demonstrated The fix column read "predates the count guard" on three rows, which scans as "the count guard is involved" when it meant the opposite. It was also wrong on one: `two racing STEPS, no hook anywhere` cannot be fixed by anything predating the count guard, since the row below it is that same fault with `preconditionGuard` on and failing identically. The column now names the specific change, and a second column separates a fix that is argued for from one that is shown — a passing scenario that is the red one with the fix armed, same tempo, one flag apart. Two of the six have that; three name a fix with no paired scenario yet; doc-31 has none. Also states the thing the column could imply but does not mean: `countGuard` requires `stateEventCount`, which no client sends, so three of the five identified fixes are dark in production. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Give scenarios stable ids, refer to events by log position, add colour Four changes to make a rendered scenario something you can cite and read. Scenario ids. Every scenario gets a hyphenated `id` next to its prose name — `in-flight-after-decision`, `stale-read-step-count-fork`. The name is a sentence and will be reworded; the id is what a commit message, a bug report or `pnpm sim <id>` refers to. The runner matches the id first and falls back to the name, so existing invocations keep working, and the table of the six red scenarios in DESIGN.md now cites ids. One reference scheme. Events were referred to three ways at once: the trace printed raw correlation ULIDs, violation messages carried `evnt_<ulid>` event ids, and the docs spoke of "position 7". Now there is only log position. `#12` is the twelfth event in the durable log sorted the way `events.list` sorts it, `@7` is the resource created at position 7, and ids inside violation messages are rewritten on the way out. That numbering is deliberately in *log* order while the trace prints in *commit* order, which makes the subject of the red scenarios visible on the page: a run whose log disagrees with the order its writers committed in shows positions counting backwards. Colour, by event family, and only when the destination is a terminal. Off under `NO_COLOR` or `--no-color`, forced on with `--color`. With colour off the output is the same plain ASCII, so it stays usable as a golden file. The workflows under test are now one file. Nothing imported across them and scenarios read them alongside the tempo that steers them, so five modules were a two-file hop for no benefit. Unit tests 61/61. Scenario book unchanged at 33 passed, 6 failed. Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Add an append-only log world, and split the book into one file per scenario Three things the book could not do before. `appendOnlyLog` gives an event its log position at *commit* instead of when its handler minted one. That is the single change that makes a stale read impossible — the log can be behind, never wrong — so playing the same 39 scenarios with and without it is how you tell which of the six reds the change would actually close. All six: 33 pass / 6 violations mint-ordered, 39 pass / 0 violations append-only. It bundles two effects that have to move together: an overtaken write re-mints at the tail, and a withheld read returns a prefix rather than a hole, which is why `onStaleRead` now carries `{eventId, hidden, truncated}` and the trace distinguishes a lagging read from a stale one. `preconditionGuard` on `RunScenarioOptions` forces the fence off across the book, asking whether anything relies on it. Violations 6 -> 8 mint-ordered, so it is load-bearing there; 0 -> 0 append-only, so it is dead weight once positions are assigned at commit. Both flags are tri-state: `undefined` leaves it to the spec, which is not the same as `false`. The scenario book was a 1020-line file; it is now 39 files and an index that only decides reading order. Same 39 ids. This is the whole answer to "how do I add a scenario" — copy the file next door — and it is why the README can be short. Three `in-flight-*` scenarios are reworded so that no expectation is restated per world: a scenario is one sequence of movements, the only thing a world changes is what a read returns, and what catches the fault in both is the invariant that a run's log must replay back into that run. Also: `loadFlowHandler` moves out of `build.ts` into `load.ts`, so playing scenarios no longer drags SWC and esbuild into the module graph; the CLI gains `--report-only`, `--summary-file` and `--detail-file` for CI, with the default still exiting non-zero; and the workbench's `test` script points at `--report-only` so a recursive `pnpm -r test` does not go red for the six. Docs are split by task: the workbench README is how to add a scenario, and the package README plus DESIGN.md are how to change the simulator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Publish the scenario book from CI, without blocking a merge Plays the book on every pull request, once per log world, and posts both summaries as one sticky comment. It never gates. Six scenarios are red on purpose — each is a reproduction of a corruption the runtime can still produce, stating the outcome its own durable log implies and staying red until the runtime gets there — so a lane that failed on them would be red on every PR and read as broken rather than as informative. What it publishes is the pair of counts, and the thing to look at is whether they still say 33/6 and 39/0. A seventh red is a regression; five means something got fixed and a scenario is ready to retire. The append-only column is the measurement the pair exists for: it says which of the six would close if positions were assigned at commit. Non-blocking at the job level rather than only on the two sim steps, so that a failed install or a PR comment the token cannot write does not turn this into a red X either. The steps themselves keep their real exit codes, so the run still says which world was clean. `--title` is new on the CLI: two summaries land in one comment, and two headings reading "world-sim scenario book" would leave the chips line as the only way to tell them apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * Make the Sim World comment four lines until you open it Two collapsed folds, one per world, each with its count and a green or orange dot on the visible line; the table is behind them. Nothing else above the fold. The failures list is gone. Six scenarios are red on purpose, so a comment that led with them led with the part that was not news, and grew a wall of text on exactly the PRs that changed nothing. The count is the signal — and anyone who wants the names can open the table, which has always had them. `renderMarkdownSummary` now renders no heading of its own, since it is built to be stacked under one; the workflow supplies the heading and the one-line description. `mdCell`, `clip` and `dedupe` existed only for the failures table and go with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(sim-world): drop the six-red list, add an API reference The "six red scenarios" section was a second copy of something `pnpm sim` already prints exactly, and the copy that goes stale. What replaces it says how to read a red — an open bug stating the outcome its own durable log implies, green when fixed rather than when seen again — and points at DESIGN.md for the part that is not re-derivable from a run. The reference has three tables. Writers: what a concurrent thread of execution is here, and the four ids one can have. Movements: the instruction that moves a writer to a place and holds it, described against the world boundary, the position in the event log, and the commit to storage — three moments, which is why there are three stops rather than one. Withholdings: the two ways to change what a reader sees without holding anybody. Terminology unified on hold/held (the word `Held` and the trace already use) and on "assigned a position in the event log" / "committed to storage", which also fixes the `runToEventProduced` docstring: the event has crossed the world boundary there, so "submitted" was the misleading half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(sim-world): move the API reference into the package, uniform vocabulary The reference documents `@workflow/world-sim`, so it belongs beside the API and not in the workbench that happens to be its first caller. It replaces the old `## Writers` section, which said most of the same things in a different order and a different vocabulary. Three nouns, one meaning each. A *writer* is a thread of execution — the table lists all three kinds with the handle that names each and the events each one writes. An *advance* moves one writer to a named place and holds it; the three `runTo*` stops are the three moments a write has (crossed the world boundary, assigned a position in the event log, committed to storage), which is why there are three rather than one, and the table says which way a concurrent commit sorts at each. A *withholding* hides something from readers without holding anybody, which is what `withholdNextEvent` and `beginHookDelivery` have in common and why the latter is not an advance. "Movements" is gone in favour of "advances", which the code and the older prose already used. "Held" replaces "stopped"/"paused" throughout, matching `Held` and `isHeld()`. The workbench README loses the duplicate tables and keeps the four rules that bite on a first scenario, so it is a guide to adding one and links out for the rest. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(world-sim): rename "Two logs" to "World behaviors", one-line motivations The section described two logs, but there are four world behaviors a scenario can pick between: the two log orderings and the two guards. Renamed and regrouped so each one is described by what it does rather than by which of the current reds it closes. Measured counts and "production" / "what happens today" framing are out of this README throughout — they belong to a run of the book, and a doc that carries them is stale the first time a scenario changes colour. The workbench README and the CI lane still print them, where they read as measurements. Every section's motivation is one line; the file-top motivation is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(world-sim): Usage shows both halves — the workflow and the script The example was a script with no workflow beside it, which hid the fact that you write both. Replaced with the smallest pair that has something to control: two steps in flight at once, and a script that decides which of them reaches the log first. The scenario in it was run before it was written down — `prepare` held before it takes a position, `finalize` committed into the earlier slot, #6 then #7 — so the positions the prose cites are the ones the trace prints. Link to the workbench moved to the end of the section, where it reads as "where to go next" rather than as an aside mid-example. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(world-sim): say what "arm" meant instead of using the word The READMEs told a reader to arm a wait without ever saying that calling an advance and awaiting it are two separate things. That is the whole mechanism — `runTo` registers its watch synchronously when called, and the returned promise only reports arrival — so it is stated plainly once in Advances and the jargon is dropped everywhere it stood in for the explanation. "Armed" survives only where it means a guard is switched on, which is a different word doing a different job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(world-sim): two symmetric watches in the Usage example The example started one watch and awaited the other inline, which is exactly the shape a reader cannot generalise from — it looks like the two calls do different things. Both are now started, then both awaited, so the sentence above the block and the code below it say the same thing. Re-run before committing: same log, finalize at #6 and prepare at #7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * docs(world-sim): neutral names in the Usage example `prepare` / `finalize` / `held` / `committed` made a reader decode four domain-ish names to see a mechanism that has nothing to do with any of them. stepA and stepB, watchA and watchB: the only thing left to notice is that the two watches stop at different points, which is the whole lesson. This detaches the example from the workbench's `parallelStepsWorkflow`, so it was verified against a throwaway copy of the workflow rather than assumed — same shape, stepB at #6 and stepA at #7, replay ok. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * refactor(world-sim): one event fold, two call phases, a smaller entry Three reductions to the same end — less surface to keep consistent — with the scenario book as the check that none of them changed behaviour. **One fold.** `create` validated *and* applied an event in a 17-case switch; `foldSeededEvent` applied it again in 15 cases without validating. Two copies of the event -> entity state machine that only a replay compares, and any divergence between them makes a replay disagree with the run it is checking for a reason that is not the runtime's fault. Both paths now end in one total `applyEvent`: the write path validates first and calls it, `seedFromLog` calls it with no validation at all, because those events were accepted once already and re-litigating them would reject legitimate history. store.ts loses 175 lines. Characterization tests for the seeded path went in first and were confirmed to bite before the refactor started. **Two phases.** `CallPhase` had a `positioned` phase between `before` and `after`, and `runToPositionMinted` / `runToCall` to park on it. Nothing in the book used any of the three: the mint/commit gap they existed for is reached through `beginHookDelivery`, which owns both halves explicitly and does not block the writer that made the write. Removing the phase also removes an `await` from the interception path, so this was measured rather than reasoned about — all four book runs are unchanged. **A smaller entry.** `index.ts` re-exported the construction kit — `createSimWorld`, `createSimStore`, `driveQueue`, `verifyReplay`, `checkInvariants`, the clock — which nothing outside the package imports and which made every one of their signatures a compatibility promise. The entry is now the scenario surface; extenders import from the module. `InFlightWrite` joins it, having been missing though `beginHookDelivery` returns it. Unchanged, and the point of saying so: 33/6/6 mint-ordered, 39/0/0 append-only, 8 violations with `--no-fence`, 0 with both. 70 unit tests green (up from 67), tsc and biome clean in both workspaces. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * fix(world-sim): follow the new event-result contract Rebase fixups for four core replay commits, no behaviour change of our own. `EventResult` is now a union — a populated page (`events` + `cursor` + `hasMore`) or all three absent — so the three fields cannot be assembled from three independent variables the type has no way to see agree. They travel as one `deltaPage` object, spread as a whole or not at all. The spread has to be conditional rather than optional: `...maybe` widens each field to `T | undefined`, which is neither arm. `processExitTriggersQueueRedelivery` is gone from the `World` interface — #3385 stopped exiting on an exhausted replay budget, so there is nothing left to tell it not to. The book is unmoved across the rebase, which is the thing worth reporting: 33/6/6 mint-ordered, 39/0/0 append-only, 8 violations with `--no-fence`, 0 with both — and the same six ids red, not a swap that nets to the same count. 70 unit tests green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * feat(world-sim): a scenario for the unclaimed-payload delivery race Reproduces #3406 as a pair of scenarios. A hook payload nobody reads registers an unarmed delivery barrier, and a step result is allowed to skip it — otherwise the workflow stalls until the barrier registry idles. The skip is transitive, so the step also skips an armed `wait_completed` merely parked behind that payload. Both branches then draw their next `step_created` id in the order the log does not record. The replay invariant cannot see this: live and replay run the same code, make the same mistake, and agree. What is checkable is the log disagreeing with itself — `wait_completed` is committed first, yet the step branch draws the earlier id. Verified red on the current runtime and green with #3406's diff applied. `claimed-payload-under-fork` is the control: same steps, same tempo, same log order through the step result, one branch awaiting the hook so the payload lands claimed. It passes in both cases. Getting there needed one new primitive. The delivery loop is serial, so a held inline step body stops the loop inside its own delivery and no timer can fire — every interleaving where a `wait_completed` lands while a step result is outstanding was unreachable. `sim.deliverQueued(select?)` takes a message out of the pending set and delivers it from the script, concurrently with the hold; `takeById` removes it first, so the loop can never pick up the same message. The book is now 41 scenarios: 34/7 with 6 violations mint-ordered, 40/1 with 0 violations append-only. The six violation reds and the --no-fence 6 -> 8 / 0 -> 0 measurements are unchanged. The new red is the first that stays red in both worlds, because no log position is wrong in it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * fix(world-sim): address review on the simulation world Fifteen findings from review, and one measurement that moved because of them. Correctness: - tempo: `abort()` stored the raw `reject` rather than the wrapper that also disposes the watch, so a point reached after teardown blocked a world call on a promise nobody could release. The deadline is one-shot and already spent by then, so that is a hang. Route abort through the disposing path and make the action a no-op once aborted. - world: `externalDepth` was a plain counter, so a step body writing concurrently with a scripted `deliverHook` was attributed to the script — invisible to any armed `runTo`, and `ext` in the trace. Measured: one misattribution book-wide. Scope it with `AsyncLocalStorage` instead. - writers: `runToEventCommitted` matched a *rejected* create, so in fence scenarios a script could wake believing a write was durable when it had 412'd. Record `failed` on the observed point and require `failed: false`. - clock/ids: reject fractional `advanceBy`, and floor in `ulid()` — a fractional millisecond silently minted `undefinedundefined…` ids that failed much later at `z.ulid()`. - streams: literal NUL bytes as key separators, `limit: 0` paging forever, and a garbled cursor slicing the array away as `NaN`. - scenario: register the handler under the exported `WORKFLOW_QUEUE_PREFIX`. The count guard, which is the substantive one. Since #3145 `@workflow/core` sends `stateEventCount` on every replay-context create and the server's count guard defaults on, so the sim's "no client sends the count" claim was stale in six places. `countGuard` now follows the fence, the sim prefers the runtime's own count over its reconstruction, and the two scenarios whose subject is isolating the watermark half say `countGuard: false` explicitly. The scoreboard does not move under the production-shaped default, which is itself the answer to the review's question. `log.monotonic-order` could never fire: its only caller fed it the sorted array. It now takes commit order, supplied only by a world that promises the two agree — under a mint-ordered log an out-of-order commit is the premise the scenario injected, not a defect. Docs: DESIGN §5 gains the two ways the sim's guards are stronger than production's (FIFO-vs-mint-order pruning, and a fence that is exact in-process where production's is region-local and fails open); §9's scoreboard is regenerated and its "no fix armed anywhere real" claim corrected; §10 gains the parallel hook-resume path, which no sim delivery takes. `step-vs-step-fork` and its twin now say who the withheld reader is in production. `unclaimed-payload-under-fork` is green after the rebase — #3406 fixed the delivery-barrier ordering — so the book is back to six reds and stays there. Counts refreshed everywhere they appear: 35/6/6 mint-ordered, 41/0/0 append-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> * fix(world-sim): make the README's code samples type-check `packages/docs-typecheck` globs `packages/*/README.md`, so the new README went into the Docs Code Samples job and five of its `ts` blocks failed. Four were genuine excerpts with an unbound `sim` or `spec`; one was a match- object *sketch* containing a `…`, so not TypeScript at all — that one loses its `ts` tag, joining the eleven other untagged blocks in the file. The rest name what they use. That needed two `paths` entries in the checker: `@workflow/world-sim` was unmapped, and an unresolved import is deliberately tolerated there (`isExpectedMissingModule`), so every identifier in those samples was `any` — they would have gone green while checking nothing. Mapped, they are checked against the real declarations: seeding `id: 12345`, `readyAtMsTYPO`, `dirsTYPO` and `runToEventCommittedTYPO` is caught against `ScenarioSpec`, `PendingMessageView`, `SimBuildOptions` and `Writer`. The lead teaser keeps its four unadorned lines and takes a `@skip-typecheck` marker instead; the same calls appear in full, checked form further down. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com> --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )