Harden Redis, Workflow, and Runtime hot paths - #19
Conversation
Bound Workflow, Durable Object alarm, and Runtime KV protocol outcomes before durable or destructive state transitions. Reduce Redis head-of-line blocking and consolidate Base64, request-completion, and key-formatting work across JS and Rust owners. Document the schema-3 dedicated-DB2 maintenance boundary and align cross-language fixtures and integration coverage. Signed-off-by: Lu Zhang <lu@wdl.dev>
There was a problem hiding this comment.
Pull request overview
Hardens Redis, Runtime KV, Workflow replay, Durable Object alarms, Base64 handling, and observability across JavaScript and Rust services.
Changes:
- Adds bounded, invocation-scoped KV and alarm response handling.
- Introduces Workflow schema 3 with operation-kind validation and strict replay variants.
- Enables shared Redis connection settings, pooled managers, and consolidated observability.
Reviewed changes
Copilot reviewed 92 out of 92 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
tests/unit/style-contracts.test.js |
Adds cross-language contract checks. |
tests/unit/runtime-wrapper-generate.test.js |
Tests Workflow KV attribution. |
tests/unit/runtime-pool-boundary.test.js |
Stubs KV capacity metrics. |
tests/unit/runtime-load.test.js |
Verifies generated wrapper metadata. |
tests/unit/runtime-lib.test.js |
Tests canonical Base64 behavior. |
tests/unit/runtime-internal-auth.test.js |
Tests KV metrics preparation. |
tests/unit/runtime-binding-surface.test.js |
Updates KV binding test graph. |
tests/unit/request-scope.test.js |
Tests lazy completion extras. |
tests/unit/observability.test.js |
Tests suppressed-log optimization. |
tests/unit/integration-test-plan.test.js |
Updates slow-test ordering. |
tests/unit/do-runtime-index.test.js |
Tests bounded alarm dispatch. |
tests/unit/do-alarm-client.test.js |
Tests strict alarm responses. |
tests/unit/control-shared.test.js |
Tests alarm cleanup envelopes. |
tests/unit/control-lib.test.js |
Expands strict Base64 tests. |
tests/unit/bounded-body.test.js |
Tests exact-length stream reads. |
tests/integration/workflows-runtime-core.test.js |
Covers cached Workflow KV facades. |
tests/integration/worker-modules.test.js |
Checks Base64 runtime parity. |
tests/integration/helpers/stack.js |
Initializes Workflow schema 3. |
tests/helpers/runtime-dispatch-fixtures.js |
Supports entrypoint props in stubs. |
tests/helpers/load-runtime-dispatch.js |
Loads new Runtime dependencies. |
tests/helpers/load-control-shared.js |
Loads alarm response parser. |
tests/helpers/load-control-lib.js |
Loads shared Base64 module. |
tests/fixtures/workflow-step-response.json |
Defines Workflow step variants. |
tests/fixtures/secret-envelope-parity.json |
Adds canonical Base64 vectors. |
tests/fixtures/do-alarm-response.json |
Defines alarm response variants. |
test-workers/workflows-demo/src/index.js |
Exercises cached Workflow KV. |
terraform/modules/compute/workflows_service.tf |
Exposes data Redis identity. |
shared/secret-envelope.js |
Uses canonical Base64 decoder. |
shared/request-scope.js |
Defers extras evaluation. |
shared/observability.js |
Adds lazy log-level gating. |
shared/do-alarm-response.js |
Adds strict bounded alarm parsing. |
shared/bounded-body.js |
Supports exact-length allocations. |
shared/base64.js |
Consolidates Base64 validation. |
scripts/integration-test-plan.js |
Recalibrates test ordering. |
rust/workflows/src/tests.rs |
Expands Redis configuration tests. |
rust/workflows/src/server.rs |
Uses shared Redis and observability. |
rust/workflows/src/schema.rs |
Advances schema to version 3. |
rust/workflows/src/config.rs |
Enforces dedicated Redis DB 2. |
rust/workflows/src/api/execution/sleep.rs |
Persists sleep operation kinds. |
rust/workflows/src/api/execution/model.rs |
Adds step-kind identity. |
rust/workflows/src/api/execution/history.rs |
Validates replay payload variants. |
rust/workflows/src/api/execution/events.rs |
Tags wait-for-event records. |
rust/workflows/src/api/execution.rs |
Tags and validates step.do. |
rust/workflows/src/api/do_alarms/model.rs |
Tests mutation wire contract. |
rust/workflows/src/api/do_alarms/dispatch.rs |
Bounds and validates dispatch responses. |
rust/scheduler/src/server.rs |
Uses shared Redis construction. |
rust/redis-proxy/src/secrets.rs |
Streamlines canonical secret decoding. |
rust/redis-proxy/src/observability.rs |
Reports Redis pool size. |
rust/redis-proxy/src/lib.rs |
Adds round-robin Redis pools. |
rust/redis-proxy/src/kv.rs |
Simplifies KV hash keys. |
rust/common/src/request_completion.rs |
Centralizes request completion telemetry. |
rust/common/src/redis_conn.rs |
Enables shared TCP_NODELAY settings. |
rust/common/src/lib.rs |
Exports request completion support. |
runtime/load/module-rewrite.js |
Defines private Workflow invocation prop. |
runtime/load/code-budget.js |
Plans KV and Workflow wrapper inputs. |
runtime/load.js |
Updates Runtime plan types. |
runtime/internal.js |
Publishes KV capacity gauges. |
runtime/infrastructure-error.js |
Tracks invocation-scoped diagnostics. |
runtime/dispatch/workflow-replay-cache.js |
Caches operation kinds. |
runtime/dispatch.js |
Attributes infrastructure failures. |
runtime/config-user.capnp |
Embeds new Runtime modules. |
runtime/config-system.capnp |
Embeds Runtime and Control modules. |
runtime/bindings/kv-capacity.js |
Implements KV read admission. |
docs/source-map.zh.md |
Updates Chinese ownership map. |
docs/source-map.md |
Updates source ownership map. |
docs/redis-key-layout.zh.md |
Documents Chinese Redis ownership. |
docs/redis-key-layout.md |
Documents dedicated Workflow DB 2. |
docs/protocol-contracts.zh.md |
Adds Chinese wire contracts. |
docs/protocol-contracts.md |
Adds Workflow and alarm contracts. |
docs/modules/workflows.zh.md |
Documents Chinese schema-3 migration. |
docs/modules/workflows.md |
Documents schema-3 migration. |
docs/modules/runtime.zh.md |
Documents Chinese Runtime boundaries. |
docs/modules/runtime.md |
Documents Runtime resource boundaries. |
docs/modules/log-tail-observability.zh.md |
Updates Chinese telemetry contract. |
docs/modules/log-tail-observability.md |
Documents lazy completion logging. |
docs/modules/durable-objects.zh.md |
Documents Chinese alarm variants. |
docs/modules/durable-objects.md |
Documents bounded alarm responses. |
docker-compose.yml |
Exposes data Redis URL. |
do-runtime/index.js |
Publishes KV capacity metrics. |
do-runtime/config.capnp |
Embeds alarm and KV modules. |
do-runtime/alarm.js |
Validates mutation responses. |
do-runtime/alarm-dispatch.js |
Validates actor alarm responses. |
deploy/kubernetes/overlays/local/kustomization.yaml |
Removes obsolete Workflow DB override. |
control/workflows-client.js |
Supports custom response readers. |
control/shared.js |
Strictly parses alarm cleanup results. |
control/bundle.js |
Uses shared Base64 validation. |
CHANGELOG.md |
Records behavior and deployment requirements. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf7f5a904d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "Runtime KV response exceeded its wire byte limit", | ||
| infrastructureInvocationId | ||
| ), | ||
| aborter.signal, |
There was a problem hiding this comment.
When redis-proxy sends headers and then stalls or disconnects mid-body on current workerd, aborting this signal only leads readBoundedStreamBytes() to call reader.cancel() while it remains blocked in reader.read(). Since response-body cancellation does not reliably settle there, the five-second timer releases the capacity accounting but the KV promise can hang indefinitely, allowing repeated stalls to accumulate stuck requests; race the read against an independent rejecting timeout rather than relying on cancellation. CLAUDE.mdL197-L200
Useful? React with 👍 / 👎.
| * @param {string | null} [infrastructureInvocationId] | ||
| */ | ||
| async get(key, typeOrOpts) { | ||
| async get(key, typeOrOpts, infrastructureInvocationId = null) { |
There was a problem hiding this comment.
The new third parameter on get()—and the equivalent parameters on getWithMetadata() and list()—crosses the public WorkerEntrypoint JSRPC surface and is trusted to select the active workflow diagnostic record. Although the generated facade currently replaces a tenant-supplied value, attribution correctness now depends on every call path retaining that wrapper; any direct or subsequently added raw-stub path can provide caller-controlled identity and mark another live invocation as infrastructure-failed. Carry this identity in the KV entrypoint's host-side ctx.props and remove it from method arguments. CLAUDE.mdL191-L194
Useful? React with 👍 / 👎.
Summary
TCP_NODELAY, two managers per control/data logical pool, parallel manager initialization, and one-operation connection affinity.Correctness and resource boundaries
register-waitpreserve explicit JSONnullwhile malformed terminal variants become result-unknown. Replay pages validate atomically before cache admission and shared cursors only advance.Measured performance
TCP_NODELAYreduced mean small-command operation latency by 18.7% and mean Worker request latency by 18.6% in matched Tokyo ECS rounds.Deployment
Workflow operation kinds require the documented schema-3 maintenance reset:
user-runtimeandsystem-runtimereaders first.wf:schema_version.Legacy non-DB2 configurations must remove the override and select an endpoint with an empty dedicated DB 2. Never clear a shared database.
Validation
npm test: lint, ordinary/strict/Node type checks, and 2,735 unit testsnpm run lint:unusedgit diff main --checkThe commit is GPG-signed and carries
Signed-off-by.