fix(core): retry omitted-revision mutations under contention - #164
Merged
Merged
Conversation
A call that omits expectedRevision and expectedWorkspaceRevision is not asking for compare-and-swap, yet the engine resolved revisions before taking the store lock, so losing a race surfaced revision_conflict. The engine now re-runs such a call (fresh read, policy, requirement and candidate checks, then commit) when the store rejects it with a CAS conflict, up to 8 attempts with jittered backoff, then returns retryable busy. Explicit revisions keep strict CAS. Recovery and native-receipt decisions never retry. The bootstrap guidance now says revision_conflict means an explicit revision is stale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A revision the caller omitted is now retried even when the other one was passed explicitly; the store reports workspace conflicts with workspace detail keys so the engine can tell which revision lost the race. Decisions retry only their commit, re-reading the revision and repeating the record-dependent checks, so a host receipt retired before the commit is not burned by contention; the in-lock receipt check still prevents a double consume. Retries also stop once the lock budget has elapsed, which bounds blocking to about twice that budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g on retry A decision now refuses a closed task inside the mutateTask update as well as in the retry re-check, and a standing decision records the provenance of the re-verification it passed on retry rather than the first one. The retry deadline starts after the first attempt, so a slow first lock wait still leaves at least one retry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
🎉 This PR is included in version 2.2.1 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice S2b (
docs/workit-next/plan.md, design §0 item 14).Problem
Mutations that omit
expectedRevision/expectedWorkspaceRevisionread the revision before taking the store lock, so under contention they returnedrevision_conflicteven though the caller never asked for CAS.Change
task,policy,evidence,finding,worker,writerandstateimport re-run the whole operation when the store rejects the commit as a revision clash: fresh read, policy, requirement and candidate checks, then commit. Nothing is resolved silently under the lock.expectedRevisionand omits the workspace revision survives an unrelatedtask.start. A clash without actual-revision details (for example an import whose source export changed) is never retried.mutateWorkspace/mutateTaskAndWorkspacenow reportexpectedWorkspaceRevision/actualWorkspaceRevision. The recovery paths are unchanged, to avoid overlapping fix(core): bound recovery copies and add workit gc #154.decision,observeDecisionandobserveStandingDecisionverify and retire the native receipt once. On a revision clash they re-read the task, re-check that it is not closed, re-verify the standing approval, and retry justmutateTask.configDigestmatches the rule that is live at commit.observeDecision.defaultLockTimeout, 250 ms in-process) has elapsed, measured from the end of the first attempt, so a slow first lock wait still leaves at least one retry. The result is retryablebusy.state.recovernever retries: it is an operator CAS over snapshot bytes.methods.ts): an omitted revision absorbs a concurrent write (re-read, re-check, reapply), and persistent contention returnsbusy.revision_conflictmeans a revision you passed is stale.Notes
observeStandingDecisionretry rarely fires in practice, because auto-approval passes explicit revisions.writer.acquiresees a lead-held writer and transfers it, as a sequential call would. Worker-held writers still returnwriter_conflict.Tests (
test/workit-core/revision-retry.test.ts, 20)revision_conflictafter 1 attempt.requirements_unsatisfied.busyafter exactly 8 attempts.revision_conflictwith workspace detail keys, after 1 attempt.revision_conflictwith 1importTaskcall.state.recoverwith a concurrent workspace write is not retried.invalid_transition, and nothing lands on the closed task.permission_denied, nothing recorded.configDigest).ok/busy; the recorded findings exactly match the reported successes and the task count matches.Mutation checks:
state.recoverwrapped: 1 fails.Measured (finding.record, 50 calls/process, Linux)
Findings always equal
ok.Verified on Node 24: lint, format:check, typecheck, knip,
bun run test(1258 pass),bun run test:packaging(247 pass).🤖 Generated with Claude Code