[e2e] Port more fixtures to the Python workbench app

workbench/python/README.md

@@ -269,30 +269,85 @@ files in one process while the queue delivery takes an HTTP round trip, so
 ## Conformance baseline
 
 What the suite runs here is declared in `e2e-conformance.json`: the ported
-fixtures, and — when there is one — an `unsupported` map naming individual tests
-whose failure is a runtime gap rather than a missing fixture. There is none right
-now. Both axes are ratchets: a claim that stops being true fails the run instead
-of quietly skipping, so growing the file is the only way to move.
-`ConformanceConfig` in `packages/core/e2e/utils.ts` spells out each direction.
-
-Current baseline: **9 passing, 128 skipped, of 137** on `world-local`, and
-**8 of 156** on Vercel. It is one baseline, not two — the extra 19 collected on
-Vercel are `e2e-agent.test.ts`, which that lane also picks up and skips whole,
-and the ninth pass is `deploymentId: 'latest' is a no-op in non-Vercel worlds`,
-which is local by definition.
+fixtures, and an `unsupported` map naming individual tests whose failure is a
+runtime gap rather than a missing fixture. Both axes are ratchets: a claim that
+stops being true fails the run instead of quietly skipping, so growing the file is
+the only way to move. `ConformanceConfig` in `packages/core/e2e/utils.ts` spells
+out each direction.
+
+Current baseline: **32 passing, 105 skipped, of 137** on `world-local` — 27
+fixtures and 3 exempted tests. The Vercel lane collects 19 more tests
+(`e2e-agent.test.ts`, which it also picks up and skips whole) and passes one
+fewer, because `deploymentId: 'latest' is a no-op in non-Vercel worlds` is local
+by definition. It is one baseline, not two.
+
+The three exemptions are two upstream causes, and both are ones this lane found
+rather than predicted:
+
+- **`hookTokenReuseLoopWorkflow`** — `hook_created` and `hook_disposed` are
+  flushed with no ordering between them, so a run conflicts against its own
+  disposed hook. The event log says it outright: `hook_created` (round 0),
+  `hook_received` (round 0), `hook_conflict` (round 1), `hook_disposed`
+  (round 0). That is the shape upstream fixed as #2777 on the TypeScript side.
+- **The two `FatalError` tests** — the step *lifecycle* is right, one attempt and
+  the run fails on it, but a thrown error does not keep its identity across the
+  event log. `step_failed.error` is written as text and comes back as
+  `RuntimeError`, so the workflow's `except` sees no `FatalError` and the failed
+  run's `errorCode` is `RuntimeError` where the driver wants `USER_ERROR`.
+
+Two things that used to be here are worth noting as gone, because both were
+costing more than their own tests. A hook payload followed by a step used to
+never create the step, which took out two fixtures and — since those were the
+only fixtures producing an `encp` payload — also stopped this lane from checking
+`encp` at all; both are fixed upstream and in the baseline now. And `encp`
+itself, which is not the niche format its name suggests: on Vercel *every* hook
+payload arrives sealed to the run's public key, because the driver resumes from
+outside the run with no symmetric key, so a runtime that cannot read it fails
+every hook fixture outright.
 
 ## What is missing
 
 This app is honest about being early. In rough order of how much it costs:
 
-- **Most fixtures are simply not ported yet** — 66 tests across 52 fixtures.
-  They are not blocked on one thing anymore: the largest blocks are hooks (19
-  tests, where vercel-py's `BaseHook.wait()` has a different shape than the
-  async-iterable hook the fixtures use), streams (11, where vercel-py now has
-  `read_stream` / `get_writable` and nothing here uses them yet),
-  `setAttributes` (9, no Python equivalent), and `FatalError` /
-  `RetryableError` (7 — `FatalError` is exported now, `RetryableError` has no
-  Python counterpart at all).
+- **Most fixtures are still not ported** — 40 tests across 28 fixtures. What
+  changed is not the size but the shape: every one now names a missing API, where
+  the list used to include "ordinary porting". Two entries that used to be here
+  are gone for opposite reasons — streams, because `get_writable()` landed and
+  five fixtures use it now, including two that forward a writable into a child
+  run; and hooks-as-a-design-problem, because that premise was wrong (`HookEvent`
+  is both awaitable and async-iterable, so `for await (const p of hook)` ports
+  unchanged, and five hook fixtures are in).
+
+  What is left, largest first:
+
+  | tests | blocked on |
+  | --- | --- |
+  | 18 | hooks — `metadata` on `BaseHook.wait()` (6 alone), `getConflict()` (6 alone), both (4), other (2) |
+  | 9 | `setAttributes` — no Python equivalent at all, and spec version 4, which Python does not claim |
+  | 3 | distributed abort |
+  | 3 | `start()` or `fetch` from a workflow body — the sandbox denies both by design, so these are not gaps |
+  | 2 | error identity across the event log — the same cause as two of the three exemptions |
+  | 2 | invoking a step id that was never registered — see below, it is a gap of its own |
+  | 3 | one name each: `RetryableError` + `StepInfo.step_started_at`, `getWorkflowMetadata`, a step returning a `ReadableStream` |
+  | 1 | the webhook route |
+
+  **`metadata` is the best-value item on that table by a distance**: 6 tests for
+  one keyword argument. `HookCreatedEventData.metadata` already exists on the
+  wire, the `Hook` entity carries it, and `HookCreatedEvent.payloads()` already
+  puts it in the key-resolution path — the only thing missing is somewhere to
+  *pass* it, since `BaseHook.wait()` takes `token` and nothing else and the flush
+  hard-codes `HookCreatedEventData(token=s.token)`.
+- **An unregistered step id is a `KeyError`, not a failed step.** Found while
+  checking whether `stepNotRegistered{Catchable,Uncaught}` were portable: they are
+  not, and the reason is a gap rather than a missing API.
+  `Workflows._get_step` is `return self._steps[step_name]`, called *before* the
+  `try` that writes `step_failed`, so an unknown name raises out of the queue
+  handler and the delivery 500s. TypeScript raises a `StepNotRegisteredError` that
+  fails the step with "is not registered", which is what lets the workflow catch
+  it and the run complete — which is exactly what those two tests assert. Worth
+  more than its two tests: in a real app this is what a bundling mistake looks
+  like, and the difference between the two behaviours is a clear failed run versus
+  an endless redelivery loop.
 - **The `.well-known/workflow/v1` surface lives in `app.py`, not the SDK**, and
   reaching it needs three `vercel.workflow._internal` imports
   (`workflow_entrypoint`, `FLOW_ROUTE`, and the `HTTPRequest` base), none of

workbench/python/e2e-conformance.json

@@ -10,10 +10,36 @@
   "language": "python",
   "fixtures": [
     "addTenWorkflow",
+    "errorFatalCatchable",
+    "errorRetryDisabled",
+    "errorRetryFatal",
+    "errorRetrySuccess",
+    "hookTokenReuseLoopWorkflow",
+    "hookWithSleepFinalStepWorkflow",
+    "hookWithSleepWorkflow",
+    "metadataFromHelperWorkflow",
     "nullByteWorkflow",
+    "outputStreamInsideStepWorkflow",
+    "outputStreamWorkflow",
     "parallelSleepWorkflow",
     "promiseAllWorkflow",
+    "promiseAnyWorkflow",
+    "promiseRaceStressTestWorkflow",
+    "promiseRaceWorkflow",
+    "retainedInterleavingWorkflow",
     "sleepInLoopWorkflow",
-    "sleepingWorkflow"
-  ]
+    "sleepWinsRaceWorkflow",
+    "sleepWithSequentialStepsWorkflow",
+    "sleepingWorkflow",
+    "spawnWorkflowFromStepWorkflow",
+    "stepWinsRaceWorkflow",
+    "utf8StreamWorkflow",
+    "writableForwardedFromStepWorkflow",
+    "writableForwardedFromWorkflowWorkflow"
+  ],
+  "unsupported": {
+    "hookTokenReuseLoopWorkflow - same run recreates a hook with the same token after dispose()": "vercel-py flushes `hook_created` and `hook_disposed` from one task group with no ordering between them, so a run conflicts against its own disposed hook \u2014 the shape upstream fixed as vercel/workflow#2777. Confirmed by the event log rather than read off the code: `hook_created` (round 0), `hook_received` (round 0), `hook_conflict` (round 1), `hook_disposed` (round 0) \u2014 round 1's registration was validated a beat before round 0's disposal landed. The failure the driver reports is downstream of that: the fixture returns early on the conflict, so the hook it was waiting for is gone and `waitForHook` 404s with `HookNotFoundError`.",
+    "FatalError fails immediately without retries": "The step lifecycle is right \u2014 `FatalError` burns exactly one attempt \u2014 but the run's `errorCode` is not. vercel-py writes `run_failed.code` as `type(e).__name__`, and by then the step's `FatalError` has been rehydrated into a `RuntimeError`, so the run carries `RuntimeError` where the test expects `USER_ERROR`.",
+    "FatalError can be caught and detected with FatalError.is()": "A thrown error does not keep its identity across the event log: vercel-py writes `step_failed.error` as text and the workflow side raises `RuntimeError(<text>)`, so the `FatalError` the step raised is no longer one by the time the body catches it. Upstream needs the serialized-error pipeline (`SerializedData` in place of the pre-#1851 `StructuredError` shape) before this and the two `RoundTrip` fixtures can pass."
+  }
 }

workbench/python/pyproject.toml

@@ -24,6 +24,15 @@ dependencies = [
   # `@workflow/world-vercel` encrypts it whenever it can resolve a per-run key,
   # which on a deployment it always can, with no opt-out.
   "vercel",
+  # And `vercel-workflow` as well, which the note above says to add "if they
+  # differ" — they do, as of this rev. A `[tool.uv.sources]` entry only applies
+  # to a direct requirement, so without this line the git pin below reaches
+  # `vercel` and every sibling still comes from PyPI. That was harmless while
+  # `4814d61f` and the 0.9.0 release were byte-identical; the rev pinned now is
+  # one commit past it and that commit is *entirely* inside
+  # `src/vercel-workflow`, so pinning `vercel` alone would have installed the
+  # release and silently tested the code without the fix.
+  "vercel-workflow",
   # vercel-py carries uvicorn as a dev-only dependency, so declare it here.
   "uvicorn>=0.30",
 ]
@@ -65,7 +74,8 @@ dependencies = [
 #     builder then reject under `--locked`. `uv run` re-locks too, which is why
 #     the `dev` script passes it as well.
 [tool.uv.sources]
-vercel = { git = "https://github.com/vercel/vercel-py", rev = "4814d61fa2ac1074ea5d67aede8f159ba1daae9b", subdirectory = "src/vercel" }
+vercel = { git = "https://github.com/vercel/vercel-py", rev = "6a10dd4c03457efce22c28986bafecbb925624a7", subdirectory = "src/vercel" }
+vercel-workflow = { git = "https://github.com/vercel/vercel-py", rev = "6a10dd4c03457efce22c28986bafecbb925624a7", subdirectory = "src/vercel-workflow" }
 
 # How `@vercel/python` builds this app. Reached only because `vercel.json`
 # declares `pyproject.toml` as the build src, which puts the builder in