Run the 24h production soak + curated error allowlist (release gate, from #182)

[onsails]

Question

Part of #200 (map). The last unresolved item of the #182 map (all its execution tickets have landed): read 24h of production logs across all three bots, curate the allowlist of accepted error/WARN classes, and declare zero-unexplained. This is the release-readiness gate.

[onsails]

Soak started

• Heal WAL desync loop in him/right data.db + harden right_db recovery (#197) #203 closed after the reviewed single-owner artifact was deployed and every closure gate passed.
• Native dependency summary now reports blocked_by: 0; no open dependency on Heal WAL desync loop in him/right data.db + harden right_db recovery (#197) #203 remains.
• Explicit UTC T0: 2026-08-25T22:43:43Z.
• Required evidence window: contiguous [2026-08-25T22:43:43Z, 2026-08-26T22:43:43Z] across Right, Him, Riskoff, the Aggregator, and process-compose topology.
• Candidate WARN/ERROR allowlist is frozen at T0 to the pre-existing non-database classes already observed before T0: peak-minute cron schedule advisory; learning prefilter/probe-writer timeout; external task/tool availability or upstream-rate-limit warnings. No WAL desync, WAL frame short-read, sidecar reset/deletion, database file lock, database-owner unavailable/not-ready, unexpected internal DB transport failure, or MCP terminal-result timeout is allowed.
• Release remains incomplete until the full 24h window is retained and zero unexplained events are demonstrated.

[spcbot]

24h soak evidence — gate not yet closed

• Evidence window: contiguous [2026-08-25T22:43:43Z, 2026-08-26T22:43:43Z] across Right, Him, Riskoff, and the Aggregator. Monitor completed at 2026-08-26T22:44:14Z.
• Runtime remained on one reviewed artifact: /Users/molt/dev/rightclaw/target/devenv/release/right. Final right status shows right-mcp-server, right-bot, him-bot, riskoff-bot, and cloudflared running; final right mcp status succeeds for every agent.
• Liveness/readiness: Right, Him, and Riskoff each recorded 24 right_mcp_init: ok results during the window.
• Owner-backed writes: Right 14 cron successes/persisted outputs; Him 98; Riskoff 51.
• Database recurrence classes: zero WAL desync, frame short-read, sidecar reset/deletion, file lock, owner-readiness, unexpected internal DB transport, or MCP terminal-result timeout events.
• Existing data.db-tshm files remained unchanged: Right 2026-08-23T20:16:08Z, Him 2026-08-23T20:15:58Z, Riskoff 2026-08-25T09:41:48Z.
• Allowed warnings observed: peak-minute cron schedule advisories (Him); learning prefilter/probe-writer timeouts (Right, Him, Riskoff).
• Unexplained event found: Riskoff sources-update emitted child.wait timed out — guest process wedged; SandboxChild::Drop will kill it on return at 2026-08-26T07:17:52Z. The same run then completed successfully and persisted output at 2026-08-26T07:17:54Z, but the class was not in the frozen candidate allowlist.
• Per the issue's zero-unexplained criterion, this evidence does not yet prove the release-readiness gate. Next step: diagnose or curate the wedged-guest-child class, then restart a clean 24h window only after that decision.

[spcbot]

Timeout investigation and remediation

Root cause confirmed for the unexplained Riskoff event at 2026-08-26T07:17:52Z.

• The sources-update Claude invocation emitted a successful terminal result after 43.464s.
• Before that result, Claude Code launched two Bash tools with run_in_background=true.
• After the terminal result, both tasks emitted task_notification status=stopped; child.wait() exceeded its 5s cleanup bound while Claude stopped them.
• The claimed update did not complete: /sandbox/data/sources.json retained mtime 2026-08-21T12:19:56Z.
• A red-capable live-microVM probe reproduced the transport shape: clean and redirected commands delivered Exited immediately; a background child inheriting the cron pipeline did not deliver Exited within 5s.

Remediation in the current working tree:

• Every cron Claude invocation now passes authoritative inline CLI settings: --settings '{"env":{"CLAUDE_CODE_DISABLE_BACKGROUND_TASKS":"1"}}'.
• CLI settings override project/local settings, so agent-owned settings cannot re-enable background mode for cron.
• Synchronous Agent calls remain available; only asynchronous background execution is disabled. Bash tasks, subagents, and forked skills must finish within the cron invocation.
• Added focused tests for the settings payload and argv pair. Updated docs/architecture/sessions.md.

Verification:

• Targeted cron suite: 70/70 passed.
• Rust review: PASS; initial shell-export precedence warning fixed with invocation-scoped --settings; re-review found no critical or warning findings.
• Full workspace nextest: 3103/3103 passed, 41 skipped.
• Workspace doctests passed.
• cargo build --workspace passed.
• cargo fmt --all -- --check passed.

The original 24h window remains invalid because it contained incomplete work. Deploy this change before starting the replacement soak window.