Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/date-subclass-vm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

Fix `Date` subclassing inside workflow functions. The deterministic `Date` override in the workflow VM now forwards `new.target` via `Reflect.construct`, so subclasses like `TZDate` from `@date-fns/tz` keep their identity, methods, and fields. Calling `Date()` without `new` now returns the (fixed) time string per spec, instead of a `Date` object.
4 changes: 4 additions & 0 deletions .changeset/olive-pugs-repeat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
---

Fix the `onAfterTransform` sample in the builders README so it type-checks on its own.
5 changes: 5 additions & 0 deletions .changeset/quickjs-default-engine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': minor
---

The QuickJS WASM VM is now the default workflow engine. Set `WORKFLOW_VM=node` to opt back into the `node:vm` engine. Existing runs keep executing on the engine stamped in their `executionContext` at start.
5 changes: 5 additions & 0 deletions .changeset/quickjs-divergence-parity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

QuickJS engine: divergence-detection and write-fencing parity with the node:vm engine. Replays now arbitrate the event log at each fixed point (orphaned events, stepName/token/resumeAt mismatches) and escalate `ReplayDivergenceError` through the existing recovery machinery instead of silently delivering wrong payloads or surfacing corruption as `USER_ERROR`; all replay-context event writes now carry the optimistic-concurrency precondition snapshot (closing the documented KNOWN GAP), with `run_failed` left unfenced to match the node engine.
5 changes: 5 additions & 0 deletions .changeset/quiet-donkeys-repeat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

Fix replay divergence when a step result overtook an earlier sleep or hook delivery that was parked behind an unread hook's payload
5 changes: 5 additions & 0 deletions .changeset/tidy-buttons-swim.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

Stop reporting replay divergence for an event the workflow is still on its way to consuming, by waiting for in-flight step and hook deliveries instead of a fixed delay
5 changes: 5 additions & 0 deletions .changeset/tidy-dodos-observe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/builders': minor
---

Add an optional observer for accepted workflow SWC transform results.
44 changes: 26 additions & 18 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine (WORKFLOW_VM=quickjs).
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset) and the explicit node:vm opt-in (WORKFLOW_VM=node).
vm: [quickjs, node]
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
Expand All @@ -278,7 +278,8 @@ jobs:
run: pnpm test
working-directory: workbench/vitest
env:
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it.
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}

e2e-package-build:
name: Build Shared E2E Packages
Expand Down Expand Up @@ -331,12 +332,14 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine. The env var is set on the e2e test runner, which is
# the client that starts runs against the deployed app — start()
# stamps executionContext.workflowVm so the deployed handler
# executes each run on the requested engine.
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset so the default-selection path is exercised end to end) and
# the explicit node:vm opt-in (WORKFLOW_VM=node). The env var is set
# on the e2e test runner, which is the client that starts runs
# against the deployed app — start() stamps
# executionContext.workflowVm so the deployed handler executes each
# run on the requested engine.
vm: [quickjs, node]
app:
- name: "example"
project-id: "prj_xWq20Dd860HHAfzMjK2Mb6TPVxMa"
Expand Down Expand Up @@ -438,13 +441,16 @@ jobs:
run: echo "ms=$(($(date +%s) * 1000))" >> "$GITHUB_OUTPUT"

- name: Run E2E Tests
run: pnpm run test:e2e --reporter=verbose --reporter=json --reporter=./packages/core/e2e/github-reporter.ts "--outputFile=e2e-vercel-prod-$APP_NAME-$WORKFLOW_VM.json"
run: pnpm run test:e2e --reporter=verbose --reporter=json --reporter=./packages/core/e2e/github-reporter.ts "--outputFile=e2e-vercel-prod-$APP_NAME-$MATRIX_VM.json"
env:
NODE_OPTIONS: "--enable-source-maps"
DEPLOYMENT_URL: ${{ steps.waitForDeployment.outputs.deployment-url || steps.prodDeployment.outputs.deployment-url }}
VERCEL_DEPLOYMENT_ID: ${{ steps.waitForDeployment.outputs.deployment-id || steps.prodDeployment.outputs.deployment-id }}
APP_NAME: ${{ matrix.app.name }}
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it
# (MATRIX_VM carries the label for file/job naming).
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}
MATRIX_VM: ${{ matrix.vm }}
# changeset-release PRs test main's production deployment, so they
# must be treated as a production run everywhere downstream.
WORKFLOW_VERCEL_ENV: ${{ (github.ref == 'refs/heads/main' || startsWith(github.head_ref, 'changeset-release/')) && 'production' || 'preview' }}
Expand Down Expand Up @@ -483,8 +489,8 @@ jobs:
if: always()
env:
APP_NAME: ${{ matrix.app.name }}
WORKFLOW_VM: ${{ matrix.vm }}
run: node .github/scripts/aggregate-e2e-results.js . --job-name "E2E Vercel Prod ($APP_NAME - $WORKFLOW_VM)" >> $GITHUB_STEP_SUMMARY || true
MATRIX_VM: ${{ matrix.vm }}
run: node .github/scripts/aggregate-e2e-results.js . --job-name "E2E Vercel Prod ($APP_NAME - $MATRIX_VM)" >> $GITHUB_STEP_SUMMARY || true

- name: Upload E2E results
if: always()
Expand Down Expand Up @@ -1106,9 +1112,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine (WORKFLOW_VM=quickjs).
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset) and the explicit node:vm opt-in (WORKFLOW_VM=node).
vm: [quickjs, node]
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
Expand Down Expand Up @@ -1219,7 +1225,9 @@ jobs:
DEV_TEST_CONFIG: '{"generatedStepRegistrationPath":"app/.well-known/workflow/v1/flow/__step_registrations.js","generatedWorkflowPath":"app/.well-known/workflow/v1/flow/route.js","apiFilePath":"app/api/chat/route.ts","apiFileImportPath":"../../..","port":3000,"testWorkflowFile":"96_many_steps.ts"}'
DEV_SERVER_LOG_PATH: "${{ github.workspace }}/nextjs-server.log"
WORKFLOW_DEV_HMR_LOGS: "1"
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it
# (MATRIX_VM carries the label for reporting).
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}
MATRIX_VM: ${{ matrix.vm }}

- name: Print Next.js server logs
Expand Down
1 change: 1 addition & 0 deletions .vercel.approvers
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@vercel/workflow
10 changes: 5 additions & 5 deletions docs/content/docs/v5/configuration/runtime-tuning.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,12 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL

### `WORKFLOW_VM`

- Default: `node`
- Default: `quickjs`
- Values: `node` or `quickjs`
- Selects the sandboxed VM engine that executes workflow functions (`"use workflow"`). Step functions are unaffected — they always run with full Node.js access.
- `node` (default) runs workflow code in a [`node:vm`](https://nodejs.org/api/vm.html) context.
- `quickjs` (experimental) runs workflow code in a [QuickJS](https://github.com/quickjs-ng/quickjs) VM compiled to WebAssembly (via [`quickjs-wasi`](https://github.com/vercel-labs/quickjs-wasi)). Both engines implement the same event-replay execution model (seeded PRNG, deterministic clock, and correlation-ID sequences are identical), but the **global surface is not identical** — see the differences below before switching an existing deployment. The QuickJS engine is intended for platforms that do not implement `node:vm`, and is the foundation for future VM-memory snapshotting.
- Global-surface differences under `quickjs` (workflow functions only — step functions always have full Node.js):
- `quickjs` (default) runs workflow code in a [QuickJS](https://github.com/quickjs-ng/quickjs) VM compiled to WebAssembly (via [`quickjs-wasi`](https://github.com/vercel-labs/quickjs-wasi)). It works on platforms that do not implement `node:vm` (e.g. edge runtimes), and is the foundation for VM-memory snapshotting.
Comment thread
TooTallNate marked this conversation as resolved.
- `node` runs workflow code in a [`node:vm`](https://nodejs.org/api/vm.html) context instead. Both engines implement the same event-replay execution model (seeded PRNG, deterministic clock, and correlation-ID sequences are identical), but the **global surface is not identical** — review the differences below before flipping an existing deployment either direction.
- Global-surface differences under `quickjs` relative to `node` (workflow functions only — step functions always have full Node.js):
- `crypto.getRandomValues()` and `crypto.randomUUID()` are provided and deterministic (seeded like the node engine's). All `crypto.subtle.*` methods throw with guidance to move to a step function — including `digest`, which the node engine supports.
- `Intl` is not available (QuickJS has no ICU). The `Intl.*` constructors throw, and `toLocaleString`-family methods (including `localeCompare`) throw when called **with an explicit locale** — calling them without arguments keeps the engine default. Perform locale-sensitive formatting in a step function.
- `WebAssembly` and `Atomics` are not available.
Expand All @@ -182,7 +182,7 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL

- Default: `0` (disabled)
- Values: non-negative integer
- Only used by the QuickJS engine (`WORKFLOW_VM=quickjs`).
- Only used by the QuickJS engine (the default; see `WORKFLOW_VM`).
- When set above `0`, the runtime persists a **VM-memory snapshot** at a suspension once at least this many events have been processed since the last snapshot. Subsequent invocations restore the VM from the snapshot and replay only the events recorded since — instead of re-executing the workflow from the top against the full event log.
- Short-lived runs below the threshold never pay the snapshot cost; long-running or unbounded runs stop scaling their resume cost with total event-log length. `1` snapshots at every qualifying suspension.
- Snapshots are an optimization, not a source of truth: the event log remains authoritative, and a missing, corrupt, or incompatible snapshot automatically falls back to a full replay.
Expand Down
26 changes: 26 additions & 0 deletions packages/builders/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,32 @@ class MyBuilder extends BaseBuilder {
}
```

### Observing transforms

Builder configurations can provide an optional `onAfterTransform` observer for
tooling that derives metadata from the exact SWC output used by a build:

```typescript
import type { WorkflowAfterTransformHook } from '@workflow/builders';

// Pass as `onAfterTransform` in the builder configuration.
const onAfterTransform: WorkflowAfterTransformHook = async ({
mode,
filename,
absolutePath,
source,
code,
workflowManifest,
}) => {
// Observe the accepted transform result.
};
```

The observer is awaited after the transform's manifest entries have been
accepted. It cannot replace the generated code, and throwing aborts the build.
A source file may be observed multiple times across transform modes, bundles,
and watch rebuilds, so consumers should deduplicate results when necessary.

## Architecture

The builder system uses:
Expand Down
3 changes: 3 additions & 0 deletions packages/builders/src/base-builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1159,6 +1159,7 @@ export const __steps_registered = true;
projectRoot: this.transformProjectRoot,
moduleSpecifierRoot: this.moduleSpecifierRoot,
workflowManifest,
onAfterTransform: this.config.onAfterTransform,
bundleTransitiveLocalStepDependencies,
rewriteTsExtensions,
sideEffectEntries: normalizedSideEffectEntries,
Expand Down Expand Up @@ -1392,6 +1393,7 @@ export const __steps_registered = true;
projectRoot: this.transformProjectRoot,
moduleSpecifierRoot: this.moduleSpecifierRoot,
workflowManifest,
onAfterTransform: this.config.onAfterTransform,
sideEffectEntries: normalizedWorkflowSideEffectEntries,
}),
// This plugin must run after the swc plugin to ensure dead code elimination
Expand Down Expand Up @@ -1940,6 +1942,7 @@ ${createWorkflowRouteHandlersCode(`workflowEntrypoint(workflowCode${workflowEntr
mode: 'step',
projectRoot: this.transformProjectRoot,
moduleSpecifierRoot: this.moduleSpecifierRoot,
onAfterTransform: this.config.onAfterTransform,
sideEffectEntries: normalizedClientSideEffectEntries,
}),
],
Expand Down
6 changes: 5 additions & 1 deletion packages/builders/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,11 @@ export {
type SerdeClassCheckResult,
} from './serde-checker.js';
export { StandaloneBuilder } from './standalone.js';
export { createSwcPlugin } from './swc-esbuild-plugin.js';
export {
createSwcPlugin,
type WorkflowAfterTransformHook,
type WorkflowTransformResult,
} from './swc-esbuild-plugin.js';
export {
detectWorkflowPatterns,
generatedWorkflowPathPattern,
Expand Down
119 changes: 119 additions & 0 deletions packages/builders/src/swc-esbuild-plugin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,127 @@ describe('createSwcPlugin externalizeNonSteps', () => {
rmSync(testRoot, { recursive: true, force: true });
});

it('reports authoritative transform results to an optional observer', async () => {
const srcDir = join(testRoot, 'src');
const stepFile = join(srcDir, 'step.ts');
const source = 'export const value = 42;';
const workflowManifest = {
steps: {
'src/step.ts': {
value: {
stepId: 'step//src/step//value',
},
},
},
};
const onAfterTransform = vi.fn();

writeFile(stepFile, source);
applySwcTransformMock.mockResolvedValue({
code: `${source}\n/* transformed */`,
workflowManifest,
});

await esbuild.build({
entryPoints: [stepFile],
absWorkingDir: testRoot,
outdir: join(testRoot, 'out'),
bundle: true,
write: false,
plugins: [
createSwcPlugin({
mode: 'step',
entriesToBundle: [stepFile],
onAfterTransform,
}),
],
});

expect(onAfterTransform).toHaveBeenCalledOnce();
expect(onAfterTransform).toHaveBeenCalledWith({
mode: 'step',
filename: 'src/step.ts',
absolutePath: stepFile,
source,
code: `${source}\n/* transformed */`,
workflowManifest,
});
});

it('awaits asynchronous transform observers', async () => {
const stepFile = join(testRoot, 'src', 'step.ts');
let markObserverStarted: () => void = () => {};
let releaseObserver: () => void = () => {};
const observerStarted = new Promise<void>((resolve) => {
markObserverStarted = resolve;
});
const observerBlocked = new Promise<void>((resolve) => {
releaseObserver = resolve;
});
let buildCompleted = false;

writeFile(stepFile, 'export const value = 42;');

const build = esbuild.build({
entryPoints: [stepFile],
absWorkingDir: testRoot,
outdir: join(testRoot, 'out'),
bundle: true,
write: false,
plugins: [
createSwcPlugin({
mode: 'step',
entriesToBundle: [stepFile],
onAfterTransform: async () => {
markObserverStarted();
await observerBlocked;
},
}),
],
});
void build.then(() => {
buildCompleted = true;
});

await observerStarted;
await Promise.resolve();
expect(buildCompleted).toBe(false);

releaseObserver();
await build;
expect(buildCompleted).toBe(true);
});

it('fails the build when a transform observer throws', async () => {
const stepFile = join(testRoot, 'src', 'step.ts');

writeFile(stepFile, 'export const value = 42;');

await expect(
esbuild.build({
entryPoints: [stepFile],
absWorkingDir: testRoot,
outdir: join(testRoot, 'out'),
bundle: true,
write: false,
plugins: [
createSwcPlugin({
mode: 'step',
entriesToBundle: [stepFile],
onAfterTransform: () => {
throw new Error('transform observer failed');
},
}),
],
})
).rejects.toThrow(/transform observer failed/);
});

it('fails the build when two files emit the same step id', async () => {
const srcDir = join(testRoot, 'src');
const firstStepFile = join(srcDir, 'confirmation.ts');
const secondStepFile = join(srcDir, 'reschedule.ts');
const onAfterTransform = vi.fn();

writeFile(firstStepFile, `export const first = true;`);
writeFile(secondStepFile, `export const second = true;`);
Expand Down Expand Up @@ -86,10 +203,12 @@ describe('createSwcPlugin externalizeNonSteps', () => {
plugins: [
createSwcPlugin({
mode: 'step',
onAfterTransform,
}),
],
})
).rejects.toThrow(/Duplicate workflow step ID/);
expect(onAfterTransform).toHaveBeenCalledOnce();
});

it('fails the build when two files emit the same workflow id', async () => {
Expand Down
Loading
Loading