Repository navigation
Description
Implementation progress — 27 October 2026
Implementation candidate; acceptance not yet qualified. #259 adds durable operations, exact receipt reconciliation and API/CLI/MCP/dashboard state.
Current-head hosted checks and the full public response-loss/restart acceptance remain to be qualified. Later lifecycle and pause endpoints must compose their own operation kinds without treating unknown effects as completed.
The referenced candidates are open and unmerged in stack #253. This progress map is non-closing; issue acceptance remains authoritative. Historical text below calling #238 open is superseded: #238 merged as c43cf0b.
Problem and evidence
A diagnostic/recovery preflight can exceed its caller's timeout. The caller then cannot distinguish rejection, still running, successful completion or a lost response. Retrying can collide with a one-use acknowledgement, reservation or approval even though the user has already authorized that same operation.
Existing command IDs and typed durable CI inventory commands in the locally installed recovery branch are useful foundations. Generalize the supported contract rather than adding another special retry command.
Required behavior
- Persist an operation ID and stable request digest before starting long-running work; acknowledge promptly and expose queryable pending/running/succeeded/refused/failed/unknown states with timestamps and progress.
- Repeating the identical command returns the same operation and result. Reusing its ID with different contents refuses. A client timeout is not cancellation or permission to launch a second operation.
- Persist result and acknowledgement state across API/worker restart and lost responses. Reconcile uncertain external effects before any retry.
- Keep observational preflight results, reservations, one-use execution grants and actual execution as distinct recorded stages. Pure observation must not accidentally consume execution authorization.
- Expose the same behavior in API, CLI, MCP and dashboard, with bounded status reads and an explicit next action.
- Preserve the existing user's scoped authorization while checking current revision/ownership before effectful execution; changed scope still requires a new decision.
Acceptance
Use real public operations with response loss both before and after durable completion, concurrent duplicate requests, service restart and an expired client timeout. Verify one operation/effect, no duplicate reservation or repair debit, recoverable results and deterministic conflict handling. Include a slow diagnostic/preflight that outlives the client's connection.
Do not implement timeout recovery by automatically reissuing an effectful command with a new ID.
Existing implementation to reuse
PR #238 is open at bfc918a545ce820f47c814e99a6fe5e92b1c3fc1, matching the inspected installed runtime. Reuse its typed diagnostic and CI preflight/acknowledgement operations. This issue adds a general durable operation/status contract so caller timeout does not strand an otherwise authorized operation.
Dependencies
No blocking prerequisite in this issue set. This work can proceed in parallel with the other independent foundations; integrated qualification will verify their composition.
Blocks:
- feat(runtime): support durable pause and resume at every delivery stage #245 — support durable pause and resume at every delivery stage.
Activity
ebarti commentedon Oct 26, 2026
Reassessment finding, 2026-10-26 — P2: a transient diagnostic dispatch read failure consumes its only wake hint and strands accepted queued work. Reusing #236's durable accepted-operation progress scope.
Public reproduction: owned HTTP preflight and authenticated ACK return 200,
state=queued, wake hint true. Inject one localOSErrorduring the first dispatch'seffective_specread, before any body is scheduled. That pass leaves the hint false. Three healthy subsequent scheduler passes perform no further spec read, schedule zero tasks, and public GET still reports queued.Working manual control: replaying the exact same ACK returns 200 and restores the hint. The next pump schedules one no-effect tripwire. This is not lost authorization or a claim that manual recovery is unavailable; it is a durable queue becoming invisible to normal pumping after a temporary observation failure.
Code/cause: delivery_native_diagnostic.py:720–758 clears the optimization hint before reading/validating queued rows. An initial queue-query exception restores it, but a per-row spec/ownership exception simply continues. The next pump returns before reading durable queue membership. Exact ACK replay already reawakens the identity correctly.
Invariant / fix owner: persisted queue membership owns accepted work; an in-memory performance hint cannot silently remove its scheduling visibility. Retain bounded retry/backoff or equivalent durable reconsideration for unresolved queued rows. Preserve empty-queue idle optimization, ownership validation and refusal to relaunch running/unknown/retired identities.
Acceptance: public ACK → one metadata/ownership-observation failure → restored observation must schedule exactly one same-identity diagnostic through normal pumping without another ACK/restart. Include first startup pass with queued work, idle empty queue, foreign/current owner changes and unknown-body no-relaunch controls. Exact ACK replay must remain idempotent.
Seams/limits: historical provider/GitHub/closed Temporal records and actor observations use the existing controlled diagnostic fixture. Public proof readers, HTTP ACK/status, SQLite and dispatcher run unchanged. The transient read failure is injected; final scheduling uses a no-effect tripwire, not a diagnostic/native/provider body. No real gate, external effect or production/runtime/settings/active-delivery mutation ran.
All-state issue/PR searches diagnostic/queued/wake/dispatch found #236 as owner, no exact repair. PR #157's ordinary-outbox retry does not cover this separate API-owned task scheduler; PR #248 leaves it unchanged. #243/#246 related. No new blocking dependency.
fix(api): preserve delivery operations across caller loss and restart (…
fix(recovery): share typed lifecycle assessments and custody checks (#…