Skip to content

fix(state-integrity): salvage durable state fences + review quality harness (#757 #771 #773 #777) - #786

Merged
unohee merged 3 commits into
mainfrom
salvage/state-review
Sep 28, 2026
Merged

unohee merged 3 commits into
mainfrom
salvage/state-review

Conversation

@unohee

@unohee unohee commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

What this is

Salvage of four abandoned draft PRs onto current main, squashed into one reviewable change:

draft title verdict
#757 fix(automation-state): concurrency-safe workflow/bridge mutations SALVAGE
#771 feat(review): CodeQL-style quality harness SALVAGE
#773 fix(state-integrity): transactional, outcome-aware state updates SALVAGE
#777 fix(automation): constrain issue and delivery workflow mutations SALVAGE

Each draft sat 172–247 commits behind, and main uses squash merges, so ancestry says nothing. Method per PR: git diff $(merge-base origin/main <head>) <head>, then keep only the hunks still absent from main; hunks main had already implemented independently were left at main's version.

Kept

#757 — automation-state

  • taskState/store.ts — storeFileStamp(): the cache stamp now includes the inode. mtimeMs:size alone misses a same-size cross-process replacement (atomic rename) inside one mtime tick, and the cache then serves the stale snapshot indefinitely.
  • orchestration/workflow.ts — validateExecution(): rejects incomplete step results (completed without completedAt, failed without error) and DAG-illegal lifecycle states (a step advancing past a pending/running/failed dependency) before persist. Wired into saveExecution alongside main's assertExecutionPersistable.
  • support/dev.ts — close handler's reporting wrapped in try/catch/finally, so onComplete + activeTasks.delete always run.
  • issues/memoryBridge.ts — dropped the duplicate memory_linked event (linkMemory already emits one).
  • issues/linearBridge.ts — a failed SDK load no longer pins the rejected init promise; the next call can retry.

#771 — review quality harness

  • New src/verify/qualityHarness.ts: full-tree fail-closed static scan of every tracked source file (oversize/binary/unreadable/symlink each become explicit error findings — never "passed with gaps"), plus isolated verify commands. Wired into review --max (every run), --harness-only (no LLM), and the markdown audit report via mergeQualityHarnessResult / QUALITY_HARNESS_AREA in src/cli/reviewAudit.ts.
  • Fixed the draft's failing test: the src/bad.ts fixture was 'try { doWork(); } catch {\n}\n', splitting the braces across lines. scanFileContent matches /catch\s*(?:\([^)]*\))?\s*\{\s*\}/ per line (content.split('\n')), so the regex could never fire and the assertion was vacuous. The fixture is single-line now and asserts a genuine openswarm/quality-bs/exception_hiding finding.

#773 — transactional state updates

  • issues/sqliteStore.ts — changeStatus() and updateIssue() re-read the effective status inside the write transaction, so a concurrent transition can no longer stamp a stale oldValue onto the event log.
  • orchestration/workflow.ts — WorkflowExecution.definitionStamp fence: saveExecution refuses to persist a snapshot whose workflow definition was replaced underneath it.
  • automation/dailyReporter.ts — bounded per-project retry (one extra attempt for failures only); the watermark advances only when every project succeeded after that retry.
  • linear/projectUpdater.ts — buildBoundedProjectDescription() reserves room for the summary before truncating, so the 255-char Linear limit can no longer chop the summary off.
  • cli/projectHandler.ts — a failed quarantine surfaces as its own error instead of being reported as a successful preserve.

#777 — issue/delivery mutation scope

  • issues/linearBridge.ts — pendingLinearMappings recovery + idempotencyKey, so a failed local mapping persist cannot orphan-recreate a Linear issue.
  • issues/sqliteStore.ts — createIssue() honors the documented idempotent-id contract (see the reconciled contract below).
  • automation/backlogGrooming.ts — validIssueIds is mandatory at parse time and apply time, so a hallucinated id can never reach a mutation path.
  • agents/workerValidationEvidence.ts — EXECUTABLE_SOURCE_FILE_RE keeps shell/sql/etc. source under data dirs in scope for validation evidence.
  • linear/projectUpdater.ts + linear/index.ts — fetchProjectOverviewIssues surfaces missing/repeated cursors instead of silently truncating the page walk.

Dropped

Junk / scratch: _patch_pr_processor.py, .restore-from-main.mjs, _tester_probe.txt, .tmp_inflate_git.py, scripts/run-harness-tests.sh, scripts/run-harness-vitest.mjs, .cursor-run-tests.sh, .cursor/hooks/*, AGT3489-STATUS.md, hooks.json probe, ls, ls-bash-env.sh, scripts/agt3489-verify.sh, tmp-write-probe-a598.txt.

Already in main (kept main's version): isDependencyTerminal / dependencyCheckedAt / dependencyLookupFailed / reconcileDependencyBlockers (757's store.ts hunks), runnerState stageError/workerFailure/failedStage fallbacks, prProcessor PR-state withFileLock, github.ts CI-timeout sleep clamp, resolvers.ts auto-link semaphore, dailyReporter's reportInFlight try/finally and "Disabled by config" wording, vitest.config.ts cacheDir churn.

Also dropped: package.json/package-lock.json version churn (no deps changed), and #757's storeClaimProcess.fixture.ts/store.test.ts edits — main's multi-process fixture assertions are a superset, so the inode fix ships as its own src/taskState/storeFileStamp.test.ts with its own fixture instead.

Conflicts resolved

  • orchestration/workflow.ts (757 ∩ 773): kept both. saveExecution now runs workflowDefinitionStamp fence → schema parse → assertExecutionPersistable → validateExecution, against a single loadWorkflow read (the definition is loaded once and passed to assertExecutionPersistable). saveWorkflow keeps main's WorkflowConfigSchema.safeParse + validateWorkflow.
  • issues/sqliteStore.ts (773 ∩ 777): the idempotent-id guard runs before main's UNIQUE/linear-index reclaim — see "Reconciled contract" below; it is not an unconditional reuse.
  • linear/projectUpdater.ts (773 ∩ 777): both changes are in disjoint regions (description builder vs. pagination); merged.
  • automation/dailyReporter.ts (773 ∩ main): main's watermark gate + wording kept; the retry pass was integrated before the watermark decision, so a retried-success advances the watermark and a persistent failure does not.

Draft-supplied tests were corrected against main's actual behavior, not the reverse:

  • qualityHarness.test.ts:50 — single-line fixture (above).
  • dailyReporter.watermark.test.ts — "partial failure" now fails both attempts in a run, so the assertion still probes the watermark rule under the new bounded retry (3 calls total).
  • __tests__/issueStore.test.ts — getEvents returns newest-first (created_at DESC, rowid DESC), so the expected pairs are reversed.
  • dailyReporter.retry.test.ts — added a homedir mock so the test is not skipped by the developer's real ~/.openswarm/daily-reporter-watermark.json.

Reconciled contract: idempotent createIssue vs. collision rejection

#777's guard returned the existing row for any caller-supplied id. That masked a contract main already depends on: SqliteTaskSource.createSubIssue wraps createIssue in try/catch and uses the UNIQUE(id) collision as its signal — identical title+description means "the same decomposition retried, reuse the child", while a changed title/description means "the retried plan changed", which must surface as existing artifact does not match the requested plan (AGT-2908's deterministic duplicate-sibling guard).

Returning the stored row unconditionally erased that distinction (CI run 36385323866: src/automation/taskSource.test.ts:174 — the caller got the old child's title back instead of { error }). Reconciled instead of reverted, via a new sameIssueContent(existing, input) helper comparing only the identity-bearing fields (title, description, parentId, projectId; optional metadata such as priority/estimate is deliberately not compared, so a retry that re-derives it is still the same artifact):

stored row for the caller-supplied id behavior
absent insert, as before
present and identical return the existing row (keeps #777's intent)
present but materially different throw Issue <id> already exists with different content: existing artifact does not match the requested create — the stored row is not overwritten, so the first plan's artifact survives

The same comparison guards the concurrent-create catch branch, which also used to return the winner's row unconditionally. Inbound syncFromLinear passes no caller id, so it is unaffected.

The salvaged sqliteStore.test.ts case asserted the over-permissive behavior (a changed title returning second.title === 'first'); it now asserts the real contract, plus a new case pinning the rejection, rather than being deleted.

Verification

$ npx tsc --noEmit
(no output, exit 0)

$ npx vitest run <19 affected test files>
 Test Files  19 passed (19)
      Tests  245 passed (245)

Affected suites: taskState/storeFileStamp, orchestration/workflow, orchestration/workflow.coverage, verify/qualityHarness, cli/reviewAudit, cli/reviewMaxCommand, cli/reviewMaxHarness.smoke, automation/dailyReporter.retry, automation/dailyReporter.watermark, linear/projectUpdater.boundedDesc, linear/projectUpdater.pagination, issues/sqliteStore, __tests__/issueStore, issues/linearBridge.recovery, agents/workerValidationEvidence, automation/backlogGrooming, automation/backlogGrooming.coverage, cli/projectHandler, cli/projectHandler.coverage.

Broader sweep including src/automation/** (the module that caught the regression): 157 files, 2511 passed. src/automation/ alone: 69 files, 978 passed.

The taskSource.test.ts:174 regression is reproduced on a throwaway copy with the idempotent guard reverted to unconditional reuse (Tests 1 failed | 15 passed) and passes with the fix.

Regression proof for the inode fix (src/taskState/storeFileStamp.test.ts), reverting ${...ino} on a throwaway copy:

 ❯ src/taskState/storeFileStamp.test.ts:67:44
     Expected: "SWAP-DST"
     Received: "SWAP-SRC"
  Test Files  1 failed (1)

With the fix: 2 passed. The second case pins that an unchanged file still serves the cached object (no gratuitous re-parse).

Live CLI check of the new harness:

$ npx tsx src/cli.ts review --max --harness-only --yes --no-linear --path <fixture>
◐ Quality harness — static scan + isolated verify commands (no LLM)
  Quality harness: passed, scanned 1/1, 0 finding(s), 0 command(s).
Verdict: APPROVE
Report saved: <fixture>/report.md

and the fail-closed path (tracked source over the 512 KiB ceiling) exits 1 with openswarm/quality-truncated in the report (cli/reviewMaxHarness.smoke.test.ts).

Notes

  • No merge performed; opened ready for review.
  • package.json/package-lock.json are byte-identical to main — no dependency changed, so no lockfile regeneration was needed.

…lity harness

Salvages the still-unique work from four abandoned draft PRs (#757, #771,
#773, #777) onto current main. Each PR's merge base is 172-247 commits
behind, so only hunks absent from main were taken; work main already
implemented independently was kept as-is.

#757 (automation-state concurrency/recovery)
- storeFileStamp(): the task-state cache stamp now includes the inode, so a
  same-size cross-process replacement (atomic rename) inside one mtime tick
  invalidates the cache instead of serving a stale snapshot forever.
- validateExecution(): rejects incomplete step results and DAG-illegal
  lifecycle states before a workflow execution is persisted.
- dev.ts: the close handler's reporting is wrapped in try/catch/finally so
  onComplete and activeTasks.delete always run.
- memoryBridge.ts: one memory_linked event, not two (linkMemory already
  emits it).
- linearBridge.ts: a failed SDK load no longer pins the rejected init
  promise, so the next call can retry.

#771 (review quality harness)
- New src/verify/qualityHarness.ts: full-tree, fail-closed static scan over
  every tracked source file plus isolated verify commands, wired into
  `review --max`, `--harness-only`, and the markdown audit report.
- The harness test fixture put the empty catch body on its own line, which
  the per-line bs detector can never match — the detection assertion was
  vacuous. The fixture is single-line now and asserts a real finding.

#773 (transactional state updates)
- changeStatus()/updateIssue() read the effective status inside the write
  transaction so the event log's oldValue cannot be stamped stale.
- saveExecution() gains a definitionStamp fence that refuses a snapshot
  whose workflow definition was replaced underneath it.
- dailyReporter: bounded per-project retry; the watermark advances only
  when every project succeeded after that retry.
- projectUpdater: buildBoundedProjectDescription() reserves room for the
  summary before truncating, so the 255-char limit cannot chop it off.
- projectHandler: a failed quarantine surfaces as its own error instead of
  being reported as a successful preserve.

#777 (issue/delivery mutation scope)
- linearBridge: pendingLinearMappings recovery + idempotencyKey, so a
  failed local mapping persist cannot orphan-recreate a Linear issue.
- sqliteStore.createIssue() honors the documented idempotent-id contract.
- backlogGrooming: validIssueIds is mandatory at parse time and apply time,
  so a hallucinated id can never reach a mutation path.
- workerValidationEvidence: EXECUTABLE_SOURCE_FILE_RE keeps shell/sql/etc.
  source under data dirs in scope for validation evidence.
- projectUpdater: fetchProjectOverviewIssues surfaces missing/repeated
  cursors instead of silently truncating the page walk.
…ent artifact

CI caught a regression from the salvaged idempotent-createIssue guard:
src/automation/taskSource.test.ts:174 "rejects an idempotent child collision
when a retried plan changed" (6799 passed, 1 failed).

The guard returned the existing row for any caller-supplied id, which erased
SqliteTaskSource's deterministic duplicate-sibling guard (AGT-2908): its
createSubIssue wraps createIssue in try/catch and relies on the UNIQUE(id)
collision to distinguish "the same decomposition retried" (identical title+
description → reuse the child) from "the retried plan changed" (→ report
`existing artifact does not match the requested plan`). Returning the stored
row unconditionally made every changed-plan retry look like a successful
reuse, so the caller got the OLD child's title back instead of `{ error }`.

Reconciled rather than reverted: a caller-supplied id found in the store is
only treated as an idempotent retry when the identity-bearing fields agree
(title, description, parentId, projectId — via sameIssueContent). Otherwise
createIssue throws an explicit collision error naming the id, so:
  - an identical retry returns the existing row  (keeps #777's intent), and
  - a materially different artifact under the same id is rejected and NOT
    overwritten (keeps AGT-2908's invariant and its test green).
The same comparison is applied to the concurrent-create catch branch, which
previously also returned the winner's row unconditionally.

sqliteStore.test.ts's salvaged case asserted the over-permissive behavior
("second.title === 'first'" for a changed title); it now asserts the real
contract instead of being deleted, plus a new case pinning the rejection.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant