Improve agent loop recovery and tool failure handling

[Open-Squilla]

Scope

Scope boundary: agent-loop recovery, tool error delivery, execution-log retention and client outcome display.

Keep real tool errors in model context so an unsuccessful call can be corrected and the task can continue. Remove repeat-call blocking, progress/source-loop interventions, final-diff salvage, identical-request guards, and deadline wrap-up instructions.

Provider recovery retries the current physical provider leg: explicit connection failures use cancellable 5–60 second backoff; temporary HTTP 429 responses use each provider's configured finite retry budget and Retry-After before fallback. Failed-attempt tool frames are discarded and committed output is not replayed. Connection errors from a stream-to-non-stream compatibility retry use the existing finite timeout/request-error recovery: a later connection failure cannot turn the preceding possibly accepted stream into an indefinitely replayed connection-only failure. Initial connection failures still use persistent recovery, and successful compatibility requests still complete. When a 429 wait cannot fit the remaining task deadline, the existing typed error lets Agent select an immediately eligible, different-authority fallback. This covers both primary and previously selected fallback legs while retaining capability, health, capacity, and replay constraints. There is no foreground-only eligibility flag, cross-fallback cooldown ledger, or additional 30-minute recovery cap.

Tool execution uses each tool's declared deadline. The mixed iteration timeout and generic tool timeout are inert compatibility fields. Task deadlines, cancellation, tool-specific execution limits, explicit budgets, mutation receipts, and accepted user messages remain intact. MCP and code execution declare their actual time budgets.

Execution logs preserve the complete original bytes instead of discarding the middle after 8 MiB. The existing result store keeps execution logs outside ordinary snapshot capacity eviction, while retaining the existing expiration policy (7 days by default) and protecting active writers. Execution logs can therefore consume more than the ordinary 256 MiB snapshot budget. Code execution returns at most 50,000 original characters per stdout/stderr stream, plus the existing omission marker and result metadata; the separate 1 MiB collection preview and full disk log are unchanged. Shortened host output includes the existing log handle even when collection itself did not truncate. Missing or failed storage never claims a complete log, and retrieval instructions are only emitted when that tool is available. Memory previews and model response budgets remain bounded; finalization and queries read chunks instead of loading an entire execution log. Disk failures retain available evidence and explicitly mark incomplete output without replaying commands or turning a completed command into a task failure. After process exit, the existing one-second grace applies only to idle pipe reads; healthy disk writes are allowed to finish. Reader errors and cancellation still finalize the log and release its active-writer lease. Windows stdin execution uses nonblocking output reads while retaining its existing stdin worker.

The model can retrieve retained execution logs through the existing tool, including after transcript reload. process.log reads actual log character ranges. The WebUI uses a separate, session-authorized page reader in the existing result dialog, with a 12,000-character page size, current-page copy, incomplete-log disclosure, and a retryable pending state while a background log is still being saved. The RPC follows the repository's generated v4 contract and authorization conventions.

The static completion prompt retains three short reference-backed rules: continue toward the requested outcome after tool failures, verify and report results accurately, and ask only for missing necessary information or authorization. No additional static prompt was added for log retrieval. Conditional tool feedback supplies the log reference when output is shortened.

The WebUI retains failed tool rows while displaying the actual task outcome. Recovered tasks display Completed; unlimited retries display the attempt number. Actual failed, timed-out, interrupted, and stopped tasks retain their terminal status.

Non-goals: sandbox or approval changes, MetaSkill changes, partial-stream replay, new job APIs, permanent log retention, or a project-wide retry architecture rewrite. The internal, nondefault max_turn_llm_calls counter remains unchanged; aligning its treatment of direct and selector-internal retries is deferred. It is not a universal HTTP-request cap.

Branch

Base branch: main

Target exception: N/A

Issue

Linked issue: None

If None, reason: agent recovery and reliability maintenance identified through source review and deterministic regressions.

Release Note

Release note: Tool failures can be corrected without heuristic blocking. Provider recovery and client task status are more reliable. Large execution logs retain their middle content and can be queried by the model or viewed by page. Legacy iteration and generic tool timeout options are ignored; task deadlines and tool-specific timeouts remain effective.

Tests

Current merge resolution: d79b60df2, based on origin/main = 2bd0912e8 after #1652 merged.

• Engine and image integration: 183 passed across seven complete targeted files. The combined regression uses a deterministic provider with real Agent dispatch, subprocess output and log storage: load an image, rebind to a vision model, receive a failed command, retrieve its missing middle diagnostic, run successful verification, and emit one final completion. It also asserts that the system prompt is unchanged. Both image and plain-text variants failed with the residual deleted workspace-gate call before the merge fix and pass afterward.
• Provider recovery, compatibility streaming, bounded output, HTTP request projection and Goal settlement: 318 passed, 1 Windows-only skipped across seven complete targeted files. These include the two previously committed compatibility/preview fixes and the included Windows LF/CRLF and expected-error-rendering fixture fixes.
• ToolContext positional compatibility, identity prompts, execution-log RPC authorization and queries: 126 passed across four complete targeted files. The first 105 ToolContext fields match published main; log fields are appended after them.
• Windows sharding governance: 46 passed, 1 platform skip. The combined branch initially exceeded the existing unweighted-test allowance (six versus four). Only this PR's two previously unweighted log test files were registered, in their existing fallback shards with the repository's documented 0.01-second provisional weights. Existing assignments, measured durations, override policy and thresholds were preserved.
• Frontend: 410 unit tests passed across eight targeted files, plus 3 Chromium browser scenarios passed for unlimited retry labels, tool failure followed by Completed, and independent complete-log pagination. The browser scenarios use the merged Vite development server and synthetic WebSocket fixtures, not a live model API.
• The earlier Linux PowerShell timeout was not reproduced locally. The unchanged signing-preflight file passed 78 tests with 1 Windows-only skip using real PowerShell 7.6.6 and ordinary repository fixtures. The original failing case also passed 12 separate runs with four concurrent workers. No signing behavior or timeout was changed; these macOS results do not replace final Linux CI.
• ruff check src tests passed. Mypy passed for 1586 source files. npm run build passed, including architecture/type checks, bundle validation and staging of the verified WebUI artifact. Gateway contract files were regenerated from the merged schemas and independently compared with generator output. Independent review covered engine ordering, UI task outcomes, prompt changes, published positional fields and Windows registration.
• Final-head CI passed: run 34887438492 on d79b60df2a59524b6ed123429daa9565fd6daeab, 38 successful jobs and 2 expected conditional skips. All four Linux offline shards, all four Windows high-risk shards, macOS native checks, frontend/browser and desktop E2E jobs passed. The previously failing Linux signing-preflight and Windows shards passed in this complete run. No full suite was run in the protected local worktree.
• Merge-queue CI passed: run 34892592852 on 0ae76d56baa8b7229ed28994939134f55d70e388, 39 successful jobs and 1 conditional skip. This verifies the final combination with main at 5da1dc117, including fix(windows): retry legacy NSIS uninstall with a shorter temporary path #1648. GitHub merged that exact commit into main through the normal merge queue on 2026-09-14 at 21:16:46 UTC. No required checks were bypassed.
• Latest-main integration review: 26d03a424 adds fix: match sessions list --channel against the projected source fields #1600 CLI channel filtering and its registered tests. The temporary combined tree merges cleanly, preserves that CLI implementation, and passes Windows assignment and unweighted-test governance validation. No additional production changes were needed.

Regression tests: added

Notes: default tests remain offline, deterministic, credential-free and safe for forks. Earlier red/green evidence for the compatibility-error and output-preview fixes remains applicable to their unchanged production code; current targeted files were rerun after merging main. This is not a live-provider success-rate or billing measurement. The internal, nondefault call-count accounting difference remains deferred. Provider retries may incur unknown fees. Original logs remain subject to retention expiration and physical storage failures; bytes discarded by older versions cannot be recovered.

Maintainer Live Check

Maintainer live check: yes

Surface: provider, browser, gateway

Earlier PR validation included TokenRhythm API smoke checks and real WebUI recovery tasks. Those predate this follow-up and are not presented as current-commit live API validation. The existing Chromium scenarios rerun for this follow-up use a real browser with synthetic responses; real storage and subprocess behavior are covered separately by deterministic integration tests.

Safety

No credentials, private transcripts, local runtime artifacts, or user-derived fixtures are committed. Log references and reads remain scoped to the authenticated session; guest and cross-session cases are tested. Commands with unknown effects are never automatically replayed. Process cleanup and existing task cancellation remain in place. Linux, macOS, and Windows acceptance uses the existing CI suites. Final-head CI and the required merge-queue checks passed before the normal queue merge.

Third-Party Origin

Third-party origin: inspired-by

Recovery behavior and minimal completion guidance were compared with DeepSeek harness (MIT), Codex CLI (Apache-2.0), and Pi (MIT). Complete execution-log preservation with bounded model output follows Pi's output accumulator and Bash tool behavior; post-exit read-idle handling follows its child-process helper. The latest compatibility-error fix follows Codex's separation of connection and response-stream failures. The independent code-result preview and conditional full-log reference follow Pi's bounded tool output; 50,000 characters is OpenSquilla's previous limit, not Pi's byte threshold. No new static guidance is introduced by this follow-up. Codex and DeepSeek do not provide an identical complete-log retention policy; session authorization and expiration remain OpenSquilla-specific. No third-party implementation source was copied.

[github-actions]

Merge queue CI: cancelled — run 34890990891, attempt 1.
Tested queue candidate: 5848541405dd3c44ee24c61d6de304b640c3a999. This result belongs to that combined tree, not necessarily the PR's current head.

• Windows high-risk (desktop-installer-contracts): failure
• Full offline tests (ubuntu-latest, desktop-installer-contracts): failure
• CI result: failure

The run summary lists reused suites and executed checks. A green PR check alone does not mean queue validation has passed.

[github-actions]

Merge queue CI: success — run 34892592852, attempt 1.
Tested queue candidate: 0ae76d56baa8b7229ed28994939134f55d70e388. This result belongs to that combined tree, not necessarily the PR's current head.

The run summary lists reused suites and executed checks. A green PR check alone does not mean queue validation has passed.