feat(core): opt-in QuickJS WASM workflow runtime - #2505
Conversation
Adds an alternative workflow runtime that executes orchestrator code in a QuickJS WASM VM (via quickjs-wasi) instead of node:vm, enabling execution on runtimes that disallow node:vm / code-generation-from-strings (e.g. Cloudflare Workers). Opt-in via WORKFLOW_RUNTIME=quickjs or executionContext.workflowRuntime; node:vm remains the default. Ported from the QuickJS engine in #1300 with the snapshot/restore machinery removed: pure event-replay, no dependency on world.snapshots.*. The workflow bundle + a devalue serde bundle run inside the VM; host-written event payloads are decrypted and decompressed before crossing the WASM boundary, and the VM's deterministic clock follows the event timeline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 543f481 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (5 failed)astro (1 failed):
example (1 failed):
express (1 failed):
fastify (1 failed):
nextjs-webpack (1 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
e2e results (local,
|
| Test | Symptom | Diagnosis |
|---|---|---|
webhookWorkflow |
a 4-byte length-frame prefix leaks into the deserialized payload | hook/webhook payload byte-framing isn't stripped before the VM deserializes — same class of layering gap as the zstd-decompress fix, but for framed byte streams |
parallelStepsThenWebhookWorkflow |
body mismatch | likely the same framing issue |
instanceMethodStepWorkflow |
Cannot stringify a function |
instance-method step this / closure serialization gap in the VM serde |
stepWinsRaceWorkflow |
assertion | step/sleep race ordering — possibly flaky, under review |
Not yet exercised (likely need work, deferred): streams (readableStream / utf8Stream / writable forwarding — the VM has only a minimal ReadableStream stub), the abort / AbortController suite, advanced hook-conflict variants, experimental_setAttributes, and distributedAbortController.
Notes
- Runtime correctness itself is covered by isolated unit/integration tests (
quickjs-runtime.test.ts): completion, args, structured-value serde round-trip, suspension, failure, not-registered, and event-timelineDate.now()determinism. - Two real bugs were found and fixed during this e2e pass: sync-return workflows (
Promise.resolvewrap) and compressed step-result payloads (host-sidedecodeForVm= decrypt + decompress).
Review feedback: - common-vm: use the native proposal-arraybuffer-base64 API (Uint8Array.toBase64 / fromBase64), which quickjs-wasi implements natively, instead of the btoa/atob dance (TooTallNate). - quickjs-entrypoint: thread the builder-configured `namespace` into every getWorkflowQueueName() call (step queue + hook_conflict re-queue) so a namespaced deployment actually consumes them instead of hanging (VADE). CI: - Add .github/workflows/quickjs.yml with two jobs: `node` runs a curated e2e subset with WORKFLOW_RUNTIME=quickjs, `workerd` runs a committed smoke worker (packages/core/test/workerd-smoke) under wrangler/workerd that instantiates quickjs-wasi + native extensions (as pre-compiled WebAssembly.Modules) and evaluates core's VM serde bundle, proving the QuickJS VM runs on Workers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…testing * origin/main: perf(core): decouple workflow VM seed/clock from startedAt (#2525) [world-local] [core] Cache local dev server port per process (#2522) Show pending runs as gray animated stripes in trace viewer (#2520) [world-vercel] Route v4 event requests through global fetch (#2514) [core] Send workflowName with step events (#2511) Stamp run IDs on world spans (#2508) Reject empty-string hook tokens in createHook() (#2490) perf(core): cache compiled workflow-bundle vm.Script across replays (#2471) perf(core): drain consumable replay events synchronously (#2473) perf(core): lazy inline step start (save one world round-trip per step) (#2478)
`hookWorkflow` is a substring of `webhookWorkflow` / `parallelStepsThenWebhookWorkflow`, so the -t pattern accidentally pulled in the webhook tests whose byte-stream framing is a known QuickJS gap. Use `\bhookWorkflow` so only the real hook tests run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Being blocked by the |
…testing * origin/main: Version Packages (beta) (#2540) perf(core): memoize step return value hydration across inline replays (#2472) [core] Fix abort signal not reflected in subsequent step (replay-ordering flake) (#2412) Amend lazy discovery removal changeset (#2560) [docs] Document minimum SDK version for using hook.getConflict (#2423) Update default CODEOWNERS (#2556) Optimize and fix the default eager build mode (#2546) Revert "fix(world-vercel): cancel v4 event frame stream on early exit (#2547)" (#2554) [core] Turbo mode: fast-path the first invocation (#2526) Remove lazy discovery from workflow/next (#2545) fix(world-vercel): cancel v4 event frame stream on early exit (#2547) feat(docs): add eve and AI SDK to product switcher (#2543) [vitest] Fix local imports failing to load in test step bundles (#2351) [builders] Fix unicode-escape crash in workflow graph extraction (#2324) Version Packages (beta) (#2495) otel(world-vercel): inject trace context on v4 event requests (#2533) Bump undici to 7.28.0 (#2534) Default source maps to dev-on / prod-off (#2529) otel: nest linked-mode invocations under the delivery context (route + execution in one trace) (#2527) perf(core): parallel inline steps + optimistic lazy step start (#2516)
VaguelySerious
left a comment
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: blocking issues found
| // queues steps through the same combined route — so step | ||
| // messages (incomingStepId) still hit executeStep above on | ||
| // re-entry. Return immediately after dispatch. | ||
| if (useQuickJSRuntime(workflowRun)) { |
There was a problem hiding this comment.
AI Review: Blocking
The QuickJS dispatch runs unconditionally here, but the run has already passed through the turbo branch above (isTurboEnabled() defaults to true). Turbo backgrounds run_started and synthesizes the run entity locally with startedAt: now (a fresh new Date(), line 851). QuickJS then seeds its PRNG and ULID timestamp from workflowRun.startedAt (quickjs-runtime.ts:487,533).
The consequence: the first (turbo) invocation seeds from the synthesized local now, while every resume invocation loads the run from storage and seeds from the persisted run_started timestamp. Different startedAt → different seed → different correlationIds/tokens. Reproduced in isolation (same workflow code, only startedAt differs):
expected 'step_01HK153XE84DWJH7VPQARYJ5AV'
to be 'step_01HK153X3VY6K45J95FWJN86AM'
This breaks the replay-dedup contract: a step_completed(C1) committed during the first invocation is never matched by the resume invocation (which regenerates C2), so steps re-create/duplicate or the run wedges; for hooks the regenerated token collides cross-run as hook_conflict (see the failing e2e job).
Separately, runReadyBarrier is never threaded into runWorkflowWithQuickJS, so the entrypoint's world.events.create calls can also race ahead of the backgrounded run_started — exactly what the node:vm path's awaitRunReady() exists to prevent.
Suggested fix: decide the QuickJS path before the turbo const (from the env var + runInput.executionContext.workflowRuntime) and exclude it from turbo (&& !quickjs). That makes the first invocation await a real run_started and seed from the persisted startedAt, fixing both the seed divergence and the write race.
There was a problem hiding this comment.
Great catch — fixed in 4dcc203 exactly as suggested. The QuickJS decision is now computed before the turbo const (from getWorkflowRuntimeFromEnv() + runInput?.executionContext?.workflowRuntime) and turbo gains && !quickjs. So a QuickJS first-invocation takes the non-turbo path: it awaits a real run_started and seeds the PRNG/ULID clock from the persisted startedAt — same seed as every resume — which also removes the run-ready write race (no backgrounded run_started for QuickJS). Verified locally with turbo enabled (the default): hookWorkflow + the curated subset pass.
| # match webhookWorkflow / parallelStepsThenWebhookWorkflow, whose | ||
| # byte-stream framing is a known QuickJS gap (tracked separately). | ||
| pnpm vitest run packages/core/e2e/e2e.test.ts \ | ||
| -t "addTenWorkflow|promiseAllWorkflow|promiseRaceWorkflow|promiseAnyWorkflow|\bhookWorkflow|sleepingWorkflow|nullByteWorkflow|workflowAndStepMetadataWorkflow|customSerializationWorkflow|errorSubclassRoundTripWorkflow|resilient start" |
There was a problem hiding this comment.
AI Review: Blocking
The QuickJS Runtime (Node) job is currently failing on hookWorkflow and hookWorkflow is not resumable via public webhook endpoint with Hook token "…" is already in use by another workflow. Both are in this curated subset (\bhookWorkflow), i.e. claimed-supported. The job's recent history is flaky (1 pass / 3 fail), which is the signature of the determinism race flagged in runtime.ts — under turbo the resume invocation regenerates a different hook token/correlationId than the first invocation committed.
Shouldn't merge with this job red on a feature the curated subset asserts as working. Fixing the turbo/seed integration should resolve these; please confirm the job goes green (not just re-run-to-pass).
There was a problem hiding this comment.
Right — this was the visible symptom of the turbo/seed divergence (fixed in 4dcc203, see the runtime.ts thread). Reproduced locally post-merge (turbo on by default): hookWorkflow + hookWorkflow is not resumable via public webhook endpoint were failing; after excluding QuickJS from turbo they pass (8/8 in the matched subset, 0 fail) — confirmed green, not re-run-to-pass. The fresh CI run will reflect it.
| // also matches because the PRNG is seeded the same way) and the | ||
| // world's EntityConflictError on `events.create` dedups one of each | ||
| // pair. Use `startedAt` (constant per-run) — distinctness across | ||
| // resumptions comes from the cursor mixed into the seedrandom seed, |
There was a problem hiding this comment.
AI Review: Note
This comment (and the matching note in vm-bundle-entry.ts) references "the cursor mixed into the seedrandom seed," but the seed at line 487 is [runId, workflowName, startedAt] — there is no cursor (snapshots were dropped in this port). The comment misdescribes the actual determinism model; it reads like carryover from the snapshot runtime in #1300. Worth correcting since the seed contents are load-bearing for replay dedup.
There was a problem hiding this comment.
Fixed in 4dcc203 — that comment was carried over from the snapshot port. Rewrote it: there's no cursor in the seed (snapshots were dropped); the seed is runId:workflowName:startedAt and full-replay-from-scratch regenerates the identical correlationId sequence every invocation, which is the dedup contract.
| // Process events and drain jobs in a loop. Events may resolve promises | ||
| // that unblock workflow code, which then creates NEW resolvers for | ||
| // subsequent events; re-processing matches those against delivered events. | ||
| let maxIterations = 100; |
There was a problem hiding this comment.
AI Review: Note
If the replay loop exhausts maxIterations while madeProgress is still true, it silently falls through to checkWorkflowState and can suspend with no new ops queued — a silent wedge with no log or error. I tested a 120-step sequential chain and it drains in 1–2 passes (each processEvents pass resolves a full sequential chain), so this is low severity in practice. Still, a runtimeLogger.warn when the cap is hit would turn a silent footgun into a diagnosable one.
There was a problem hiding this comment.
Fixed in 4dcc203 — if the loop exhausts its iteration budget while still making progress, it now returns a failed result (and logs an error) instead of silently calling checkWorkflowState on a half-settled VM. Fails loud → queue retries / surfaces a real error rather than wedging.
| * Auto-generated by scripts/build-quickjs-assets.js | ||
| * Do not edit manually. | ||
| * | ||
| * Contains base64-encoded quickjs-wasi WASM binary and native C extension |
There was a problem hiding this comment.
AI Review: Note
The generated module embeds ~3.3 MB of base64-encoded WASM/.so assets, which tsc compiles into dist and ships in the @workflow/core npm tarball for all consumers — even though node:vm is the default and the asset is only dynamically imported at runtime. The dynamic import defers runtime load, not package/tarball size. Worth confirming the bundle-size impact is acceptable, or splitting the assets into an optional dependency that only QuickJS users install.
There was a problem hiding this comment.
AI Review: Note (follow-up — corrects the framing above)
I built the vite workbench for both the plain Node and the Vercel targets and inspected the output; the bundle impact is narrower than my note implied.
It is code-split into a lazy chunk, not inlined into the entry, and not tree-shaken away (the bundler can't prove the WORKFLOW_RUNTIME / per-run flag is never set, so the chunk is always emitted — but it's loaded only via the dynamic import() at runtime.ts:966).
vercel build → .vercel/output/functions/__server.func/ (Nitro, nodejs24.x):
.vc-config.json → "handler": "index.mjs"
index.mjs 2.2 MB ← no WASM inlined; references the chunk via import("./_chunks/quickjs-entrypoint.mjs")
_chunks/quickjs-entrypoint.mjs 3.3 MB ← the blob, its own chunk (~41% of the 8.0 MB function)
So, splitting the two senses of "in the bundle":
- Same deployed function artifact as
index.mjs? Yes — it ships inside__server.func/, counts toward the function's deployed size and toward Vercel's function size limit. - Same JS module / parsed at cold start? No —
index.mjsloads it only via dynamicimport(). The launcher bootsindex.mjs; the 3.3 MB chunk isn't read or compiled until a run actually selects QuickJS.
Net for node:vm-only users (the default): no main-bundle bloat and no cold-start parse/CPU penalty — the cost is purely ~3.3 MB of dead weight in every deployed function artifact regardless of runtime. Recoverable only by making quickjs-wasi an optional/peer dependency of @workflow/core rather than a hard dep. Downgrading this from a bundle concern to a deploy-artifact-size one.
There was a problem hiding this comment.
Thanks for the detailed follow-up — agreed with the corrected framing: code-split lazy chunk, so node:vm-only users pay no main-bundle or cold-start cost, just ~3.3 MB of dead weight in the deployed function artifact (counts toward Vercel's function size limit). For this PR I'm leaving it as a hard dep since the runtime is opt-in/experimental, but you're right that the proper fix before any broad rollout is to make quickjs-wasi (and the generated assets) an optional/peer dependency that only QuickJS users install. I'll track that with the other Workers follow-ups (asset-loading variant + wasm-compression) rather than expand this PR's scope.
| const cid = event.correlationId; | ||
| if (!cid) continue; | ||
|
|
||
| const escapedCid = cid.replace(/"/g, '\\"'); |
There was a problem hiding this comment.
AI Review: Nit
escapedCid only escapes ", not backslashes or newlines, before interpolating into evalCode strings. Safe today because correlationIds are step_/wait_/hook_ + ULID (Crockford base32, no special chars), but it's fragile if the ID format ever changes. Consider JSON.stringify(cid) for the interpolated string literals, consistent with how event.eventId is already handled.
There was a problem hiding this comment.
Hardened in 4dcc203 — escapedCid now escapes backslashes as well as quotes (cid.replace(/[\\"]/g, '\\$&')). Agreed it's safe-by-construction today (ULID Crockford base32), but this keeps the interpolation safe if the id format ever changes.
Blocking (AI review): the turbo fast-path synthesizes the run with a local `startedAt: now` and backgrounds run_started, but the QuickJS runtime seeds its PRNG/ULID clock from `startedAt`. A turbo first-invocation would seed from a transient `now` while every resume seeds from the persisted run_started timestamp → divergent correlationIds/hook tokens, breaking replay dedup (the failing hookWorkflow e2e). Decide the QuickJS path before the `turbo` const (from env + runInput.executionContext) and exclude it from turbo, so the first invocation awaits a real run_started and seeds from the persisted startedAt. Verified locally: hookWorkflow + the curated subset pass under quickjs+turbo. Notes: - escape backslashes as well as quotes when interpolating correlationId into VM evalCode strings (defensive). - fail loud instead of silently wedging if event processing doesn't converge within the iteration budget. - fix a stale comment referencing the (removed) snapshot events cursor in the PRNG seed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…testing * origin/main: Trace /flow route initialization (#2592) Version Packages (beta) (#2591) feat(web): add trace step shortcut helper (#2582) [web-shared] reskin json viewer (no duplicates, better colours and navigation) (#2434) Send occurredAt with workflow events (#2580) docs: use actual eve logo and tidy OSS nav dropdown (#2586) Display occurredAt in trace details (#2581) fix(next): discover root entrypoints (#2564) [core] Turbo: skip the unused run_started event-log preload (#2569) fix(next): prewarm SWC plugin cache (#2538) [world-vercel] Use v3 stream endpoint (supports transparent reconnect on timeout) (#2424) [core] Retry stream reopen against the reconnect budget (#2334) Add Platformatic World to worlds-manifest.json (#1450) docs: derive section landing-page cards from the page tree + lint drift (#2567)
- start(): always record the resolved runtime (node-vm | quickjs) on the run's executionContext (defaulting to node-vm), so every run captures which runtime it uses and the choice is pinned per-run. - runtime.ts: tag the workflow execution span with `workflow.runtime` (Attribute.WorkflowRuntime), resolved per-invocation via useQuickJSRuntime so it's authoritative on resumes — for both the node:vm and QuickJS paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Closing as superseded: the opt-in QuickJS runtime shipped via #3048 and successors ( One genuinely uncovered piece worth extracting before this is forgotten: the workerd/Cloudflare smoke test (
Recommend filing an issue for workerd compatibility testing rather than keeping this PR open for it. |
Context
Running Workflow on Cloudflare Workers (e.g. with the community
@fantasticfour/world-cloudflareworld) is blocked by@workflow/core's use ofnode:vmto execute workflow orchestrator code. I verified empirically (minimal workerd smoke test) that this is not fixable by theenable_nodejs_vm_modulecompat flag:enable_nodejs_vm_module:vm.createContext()works, butvm.runInContext()/compileFunction()throw "not implemented" — the flag only adds a non-functional module stub.eval/new Functionboth throw "Code generation from strings disallowed for this context").vm.runInContextis the same class of operation. This is a deliberate, platform-wide constraint.The unblock is a runtime that doesn't use
node:vm. This PR lifts the QuickJS WASM engine from @TooTallNate's snapshot-runtime PR (#1300) — running workflow bytecode inside a QuickJS WASM VM viaquickjs-wasi(a first-partyvercel-labspackage), where dynamic eval is fine because it happens in the VM's own WASM linear memory, outside the host isolate's eval ban. I confirmedquickjs-wasiruns cleanly inside workerd.Tracker: #2028. Engine source: #1300.
What this PR does
Adds an opt-in QuickJS runtime to
@workflow/core.node:vmremains the default — nothing changes unless you opt in.WORKFLOW_RUNTIME=quickjsenv var, or per-runexecutionContext.workflowRuntime = 'quickjs'(propagated bystart()), so one deployment can serve both runtimes.node:vm), with no dependency onworld.snapshots.*. None of the world packages are touched.node:vmruns never pay the cost.Key files
runtime/runtime-mode.ts—WORKFLOW_RUNTIMEflag (node-vmdefault |quickjs)runtime/quickjs-runtime.ts— VM execution: create VM → eval serde bundle + bootstrap + workflow → replay events → complete/suspend/fail (ported fromsnapshot-runtime.ts, snapshots stripped)runtime/quickjs-entrypoint.ts— run lifecycle: load events → run VM → writerun_completed/ per-pending-opstep_created/hook_created/wait_created+ queue steps /run_failed(ported fromsnapshot-entrypoint.ts, snapshots stripped)serialization/{workflow-vm,codec-devalue-vm,reducers/common-vm,vm-bundle-entry}.ts— devalue serde bundle that runs inside the VM (no Node deps;btoa/atobinstead ofBuffer)scripts/build-{quickjs-assets,vm-serde-bundle}.js— base64-embedquickjs.wasm+ C extensions and bundle the serde IIFE (no filesystem access at runtime → Workers-safe)runtime.ts/runtime/start.ts— dispatch + flag propagationNotable fixes made while porting (vs. #1300)
quickjs-wasi@3.x(btoa/atob are now default intrinsics, not a separate extension).Promise.resolve(...)so synchronous workflow returns behave likenode:vm.devlformat; step results are zstd-compressed by the shared executor).Date.now()advances per consumed event'screatedAt), matchingnode:vm— required now that there are no snapshots.Verification
quickjs-wasiruns inside workerd (create / evalCode / host callbacks / async drain) — the Cloudflare unblock.quickjs-runtime.test.ts) + flag tests — completion, args, structured-value serde round-trip, suspension w/ pending step, failure, not-registered, and Date-timeline determinism.@workflow/coreunit suite passes (nonode:vmregression).sleepingWorkflow+promiseAnyWorkflowpass end-to-end underWORKFLOW_RUNTIME=quickjs.nextjs-turbopack, world-local) under QuickJS is running; results to follow in a comment.Follow-ups (not in this PR)
@fantasticfour/world-cloudflareto the current world interface, deployed to Workers.🤖 Generated with Claude Code