Allow service workers to reopen recorded conditions inside parallel and selection groups

[rmcdaniel]

User-visible defect

A service worker cannot safely reopen a condition inside an existing parallel or durable selection group after a signal leaves its predicate false. The worker retains the group's original identity and logical occurrence, but its new physical open_condition_wait command is rejected as invalid_commands.

This was found while qualifying cooperative cancellation for durable-workflow/.github#136. It occurs before the cancellation request. The actual Server run reports nine passing cases and two failures. Raw retained evidence. The old connected pipeline masked Cargo's exit through tee; that separate gate is corrected in Rust #55, and the failure remains authoritative.

Exact observed tuple: source Rust c8abb391ed86ea6a5a6c4c86435ab621f89d17d8, source Server 3c15bfb0f98e038febb657412f1ca97bbdb23ad8, published Workflow 2.3.1 (fb3f3e59a4342fdebf8ced6160798906c3ee4387). Published SDK/Server reproduction is still required before claiming a released-tuple outcome.

Reproduction

1. Start a workflow with a timer and an authored condition inside a parallel or keyed selection group. The predicate requires two vote signals.
2. Run a real Worker until the first ConditionWaitOpened is recorded.
3. Send one selected-run vote signal.
4. The same logical condition remains false and the Worker emits a physical reopen with the original occurrence and group path.
5. Completion returns HTTP 409, invalid_commands. The workflow does not reach a second recorded open.

DefaultWorkflowTaskBridge::parallelCommandsMatchSequences compares original group member identity to the new physical sequence and requires all initial group members in the current command batch. Both requirements are appropriate for a new group, but do not admit a proven reopen of an already recorded member.

Acceptance

• Reproduce with exact published Server and SDK artifacts and retain the failing counterfactual.
• Admit a grouped condition reopen only with durable proof of its original run, occurrence, predicate, timeout and full group/member path. Preserve strict validation of new or changed groups.
• Repeated false wakes must retain one logical condition, append valid physical waits, and avoid prematurely recording a durable selection winner.
• Verify cold Worker replay, final satisfaction/timeout, adjacent calls and complete histories. Reject changed identities, reordered/nested group paths, malformed partial new groups and expired/stale claims without recording work.
• Run affected Native/Server tests and exact published PHP/Python/Rust group-condition conformance before closing the released-product outcome.
• Keep the cancellation draft's reopened-group cases and qualify their original request/delivery ranges after the prerequisite fix. Cooperative capability publication still belongs to the complete shared gate in .github#136.

This owns a concrete product prerequisite, not a new architecture or benchmark project.

[rmcdaniel]

Published counterfactual confirmed, 2026-10-01 15:59 UTC

The defect reproduces with untouched published artifacts and ordinary protocol 1.19, without cancellation support or a cancellation request:

• Server 2.4.36, index digest sha256:74cdf7feae85052151de905d877a2e1ba5fcbfabf9ef50f3a8aab7c285685204.
• Native Workflow 2.3.1, fb3f3e59a4342fdebf8ced6160798906c3ee4387, verified from the image's package provenance.
• Rust SDK 2.1.2, resolved from crates.io with an exact =2.1.2 constraint in a fresh consumer.
• Rust 1.86, isolated MySQL 8/Redis 7 Server stack, actual Worker registration/poll/completion and selected-run signal APIs. No source package override.

Authored wait	After one of two required votes
Scalar condition	PASS, second physical open at sequence 2 with one logical occurrence.
Nested parallel group	FAIL, HTTP 409 invalid_commands, only initial condition open at sequence 3.
Keyed selection group	FAIL, HTTP 409 invalid_commands, only initial condition open at sequence 2.

Bounded raw outcome summary:

{"mode":"condition","opens":2,"result":{"outcome":"pass"}}
{"mode":"parallel","opens":1,"result":{"outcome":"product-fail","reason":"invalid_commands","status":409}}
{"mode":"selection","opens":1,"result":{"outcome":"product-fail","reason":"invalid_commands","status":409}}

The scalar history records ConditionWaitOpened(1), the vote signal, ConditionWaitSatisfied(1) and ConditionWaitOpened(2). The group histories retain their original timer/signal/condition opens and the vote signal, with no second condition open or premature selection winner. The consumer exits 1 because of these two product failures.

Reproduction source will be retained with the prerequisite PR. Correct the Native validation only for a proven reopen of an existing condition/member. Preserve rejection of malformed new group batches and changed occurrence/predicate/timeout/group paths. Repeat this unchanged published consumer after the fixed image is published, then verify the cancellation draft's original group ranges separately.

[rmcdaniel]

Follow-through after admitted reopens

The engine's full source matrix 36891688925 passes at fc4ef6d5c29d219e595ad4d4f37f004b55b60585, including MySQL/PostgreSQL feature shards, MariaDB, Laravel embedded upgrades, full unit coverage and the coverage ratchet. The current Native head adds only the locked dependencies for the published reproduction.

Actual Worker follow-through exposed the necessary Rust prerequisite, now in Rust PR #56. The decoder correction passes the official replay corpus and rejects seven identity changes, an unsettled predecessor, a timed-out predecessor and a resolution recorded after its reopen. Current condition tests: 17 pass.

With corrected decoder source, scalar and nested parallel waits accept their reopened history and record final satisfaction on the second vote. Keyed selection also needs the SDK to stop reopening when that recorded predicate becomes true. That correction now records the durable SelectionResolved marker, but the workflow remains pending.

The isolated Server task diagnostic shows the signal task completed with waiting_for_history_acknowledged: true. Its canonical selection marker was committed during that completion, and no ready workflow successor drove the next replay during the bounded check. The candidate stack is being removed after retaining the fixture evidence.

Next concrete work: ensure committing a condition selection winner schedules the replay needed to observe it, then verify the winner's correlation to the original authored member across its later physical condition sequence. Run timeout, repeated false-wake and real cold-process checks before publishing the exact Native/Server/Rust tuple. Both fix PRs remain draft. No stable artifact or cooperative capability has been published from this work.

[rmcdaniel]

Stable prerequisite publication and downstream plan

• Rust SDK 2.1.3, reviewed PR fix: replay grouped condition physical reopens sdk-rust#56, source e948cab74406f8e245bd8f761ddb65e4e662ce20. Protected publication and registry verification passes. Registry, built and downloaded crate SHA-256 all equal cdaef7b493bf2876726412d7c436299578f6fd6726e3b1d4b6dd0848795f445b. Downloaded VCS provenance matches the clean release commit. A fresh exact-version consumer builds on Rust 1.86.
• Native 2.3.2, reviewed PR fix: reopen recorded service conditions within groups #602, source d8594a67661f63f17d76cd5a1a1349c2d28f4a6d. Packagist is available and Server's ordinary Composer update resolves this exact source and dist commit. Published package and Laravel upgrade verification is running, with its published package contract step already passing.
• Stable Server 2.4.37 follow-through is draft Server PR 295. Its lockfile changes only the Native package, and release consumers are generated from the source release record. Normal Server checks are running.

Affected acceptance checks after the exact Server digest publishes: the official deterministic replay runner across PHP/Python/Rust, the published workflow lifecycle matrix, and the retained five-case actual Worker consumer built against crates.io Rust 2.1.3. Use exact PHP 2.1.6, Python 2.3.7, CLI 2.1.3 and Waterline 2.0.9 pins. The five grouped cases verify final typed results, durable completion, false wakes and both delayed deadline paths with a fresh process for each claim.

Consumer review: Server needs a rebuild because it pins Native. Sample App pins Native in its lockfile and Rust in its consumer lockfile, so its baked artifact tuple and locks need an update after the new Server tuple qualifies. Waterline's Native dependency is development-only and its embedded observation surface does not execute these group commands, so no unchanged Waterline release is needed. CLI and the PHP/Python SDKs retain their existing published versions and participate in the affected qualification. Cloud deployment decisions remain in its private owning record.

Merged Native and Rust PR head branches have been deleted and verified absent from GitHub. The completed Native and temporary Server source qualification worktrees, stack and local candidate image have been removed. Keep #601 open until the stable digest and downstream acceptance checks pass. Cooperative feature drafts remain separately gated.

[rmcdaniel]

Exact published five-case Worker qualification passes

All five cases pass through typed results and durable WorkflowCompleted, with a new actual Rust Worker process for each claim. The invocation exits 0 and stderr is empty.

Case	Physical opens	Satisfied wakes	Final result
Scalar insufficient then sufficient signal	2	2	Satisfied, completed
Nested parallel, other members finish while condition remains false	3	3	ConditionSatisfied, completed
Keyed selection after insufficient signal	2	2	ConditionSatisfied, selected, completed
Keyed selection delayed condition deadline	2	1	ConditionTimedOut, selected, completed
Nested parallel deadline after another false wake	3	2	ConditionTimedOut, completed

Published artifacts consumed:

• Server 2.4.37, docker.io/durableworkflow/server@sha256:b813d2ffdac6a3ddf861f0aab8fb224217633f1083a9983bed43ec087b09444d, source aca1bc0e8012c3719eb2193e38ca53e94f9a5505.
• Native 2.3.2, source/dist d8594a67661f63f17d76cd5a1a1349c2d28f4a6d, verified from the actual image's OCI labels and package provenance.
• Rust SDK 2.1.3 from crates.io, exact registry checksum cdaef7b493bf2876726412d7c436299578f6fd6726e3b1d4b6dd0848795f445b. No path dependency or source patch. The consumer builds with cargo build --locked on Rust 1.86.
• Ordinary protocol 1.19, isolated MySQL 8 and Redis 7 stack. The stack and data were removed after the successful run.

The raw results, exact consumer source/lockfile, Compose configuration and build/start logs are retained as a Server release artifact. Archive SHA-256: 2726b14c3c07aebde77c2c519fbadb660c56065dd0c690ccb9fe6c404ab8277d.

The protected Server image publication job passes exact amd64/arm64 identity, bare first-run readiness, published Compose and protocol catalog checks. Matching chart publication is still running. Broader published lifecycle qualification and the official PHP/Python/Rust deterministic replay check are running on this same seven-artifact tuple, with PHP 2.1.6, Python 2.3.7, CLI 2.1.3 and Waterline 2.0.9.

Sample App PR 133 updates its baked tuple and Native/Rust locks. Keep #601 open until those remaining acceptance checks and consumer follow-through pass.

[rmcdaniel]

All affected published qualification now passes on the exact tuple in the prior receipt: 31/31 official deterministic replay scenarios across PHP/Python/Rust, 12/12 published lifecycle cells in https://github.com/durable-workflow/server/actions/runs/36904404955, and all five actual grouped-condition Worker cases. Server image and matching Helm chart publication pass in https://github.com/durable-workflow/server/actions/runs/36903934262. Replay distribution identity validation covers all seven consumed public artifacts. Raw replay results and identities: https://github.com/durable-workflow/server/releases/download/2.4.37/workflow-601-published-replay-2.4.37.tar.gz. Remaining before closure: Sample App PR durable-workflow/sample-app#133 checks and merge. Its application/microservice tests pass after updating both Rust locks; its baked-image, Compose and polyglot checks are running. No local qualification containers remain.

[rmcdaniel]

Sample App PR 133 is merged as 7456408ae9beef74ff5269075b77f34606dfd142. Reviewed candidate bf076e59710e5de7aaea34925e746a560ab46fd6 passed all PHP 8.4/8.5 application and microservice checks, PHP/Python/Rust smoke, Compose workflows, public-boundary checks, and candidate development-image startup qualification. Both Composer locks and both Rust worker/template locks now use the published patch tuple. The merged remote branch is deleted and verified absent. The completed local worktree is removed.

The final consumer step is the main development-image publication. Both architectures must qualify before its main channel is promoted. Keeping this issue open until that customer-facing artifact finishes.

The raw published replay evidence is retained as a Server 2.4.37 release asset, SHA-256 9ca26c5a36e52a6fa227ed29574c26b21c32883ea28842db89a7c16df2830c0e. The five-case Worker evidence is also retained on that release. All disposable qualification stacks, volumes, replay dependencies and completed source worktrees have been removed.

[rmcdaniel]

Delivered and verified the released grouped-condition fix.

Artifact	Published version / exact identity
Native	2.3.2, source d8594a67661f63f17d76cd5a1a1349c2d28f4a6d
Server	2.4.37, index sha256:b813d2ffdac6a3ddf861f0aab8fb224217633f1083a9983bed43ec087b09444d
Rust SDK	2.1.3, crates.io checksum cdaef7b493bf2876726412d7c436299578f6fd6726e3b1d4b6dd0848795f445b
Sample App	PR 133, source 7456408ae9beef74ff5269075b77f34606dfd142, published development-image index sha256:1843b1e6d507f16f4064c745581d20802bd11cc127e18f87a2904268227b5fee

Sample App publication 36906963414 passed both architecture builds, anonymous published-image qualification and startup checks on amd64/arm64, and promotion to both registries' main channels. Fresh manifest reads confirm both main channels contain the qualified amd64 sha256:d6df73e66b55897678b5393279ed21707cc711ca4eef06f708e5f75890af6ece and arm64 sha256:d1a6395bf3e25d0eb3a13a478890019fec80624c424ea0532b9adde309125a46 images and matching attestations. The merge tree equals the reviewed and tested candidate tree.

The exact published Server/SDK tuple passed 31/31 deterministic replay scenarios across PHP/Python/Rust, 12/12 lifecycle scenarios, and all five actual Worker false-wake/satisfaction/deadline cases. Qualification receipt. Raw five-case and replay evidence is retained on Server 2.4.37. Native's full supported database/Laravel matrix, strict invalid-group guards and regression corpus, Server release verification and Rust registry/fresh-consumer checks pass.

Implementation PRs: Native 602, Rust 56, Server 295, and Sample App 133. Their completed remote branches were deleted and verified absent. Completed worktrees, temporary dependency files, qualification stacks and volumes were removed after retaining evidence.

The separate cooperative cancellation draft's scalar/nested/selection cases now pass at their original request and delivery ranges. 11-case confirmation. The remaining feature qualification stays owned by durable-workflow/.github#136. This prerequisite's released-product acceptance is complete.