diff --git a/CHANGELOG.md b/CHANGELOG.md index 63a61c1..cafa9ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,6 +95,83 @@ All notable changes to `github-delivery` are documented here. the TLDR; a comment failing the gate must be repaired, never marked published. +## [0.5.0] - 2026-08-12 + +### Added + +- Added a zero-clone npm bootstrap for `npx github-delivery` with guided + install/setup, explicit `install`, `setup`, `doctor`, `update`, and + `update --apply` commands, an exact npm package-surface validator, and + Trusted Publishing integration in the protected release workflow. The npm + artifact remains a thin bootstrap only; the installed skill payload still + comes exclusively from the separately verified stable GitHub Release. +- Added complete protected-stream visibility for generated assistant messages, + reasoning summaries, supported raw reasoning, and plan deltas, plus sanitized + App Server replay/telemetry primitives. Protected mode now requires the + plan/diff/token signals its enforcement depends on instead of claiming + `stream` while those notifications are unavailable. +- Added hard no-progress generation bounds using Codex cumulative output-token + telemetry and a generated-character fallback. Material diff changes, + completed plan steps, and successful execution are real progress; merely + starting a tool is not. Repeated imminent tool intent and malformed tool + protocol output such as `...` are explicitly bounded, including + across generated-text channel changes and unique wording. +- Added a semantic evidence registry keyed by resource + state generation. + Owned helpers publish structured `gdEffect`/coverage metadata, and equivalent + reads of the same GitHub Actions run or authoritative helper output can be + reused instead of becoming new evidence merely because shell filters differ. +- Added a persistent delivery workflow controller with route locking, explicit + phase graphs, checkpoints/resume, blocker/evidence/ref state, per-phase and + workflow budgets, bounded retries, and measurable no-progress escalation. + Routed workflows now consume one-shot workflow/policy packets instead of + repeatedly rediscovering their route and policy surface. +- Added a release-blocking reliability gate that replays the real + Baseline-is-green/tool-emission stall, malformed protocol output, + cross-channel narration loops, and repeated CI-evidence acquisition. A + false-positive corpus also protects legitimate tool-rich investigations and + long final verdicts, with a separate finalization allowance rather than a + blanket relaxation of active-workflow bounds. + +### Changed + +- Successful `PostToolUse` results are no longer replaced or truncated by the + watchdog. Evidence compaction now belongs at the source/helper where the + contract is known, preventing a protection mechanism from destroying a valid + result and provoking a second read through another command shape. +- Windows/PowerShell progress classification now covers `git -C`, + `Get-ChildItem`, grouped/compound commands and owned GitHub Delivery helpers + more accurately while retaining conservative neutral handling for ambiguous + operations. +- Protected-stream runtime reporting now lets a verified live `stream` + declaration supersede stale hook-era degradation metadata, eliminating the + contradictory `stream` + `streaming_interruption_unavailable` capability + report. +- Final answer generation uses larger dedicated character/output-token budgets + after the plan is complete, while malformed protocol/tool-emission detection + remains active and any new real tool start exits finalization mode. +- Same-version installs with a byte-identical payload are now successful + unchanged no-ops; same-version payload drift remains fail-closed, including + with `--force`. + +### Fixed + +- Fixed the v0.4 protected-stream blind spot where large loops emitted through + reasoning or plan channels could bypass a detector that watched only + `item/agentMessage/delta`. +- Fixed tool-call emission stalls in which the model repeatedly says variants + of `run`, `execute`, `wire`, `add`, `edit`, or similar actions without ever + producing a real tool item, including repeated malformed `` protocol + scaffolding. +- Fixed CI/read spirals that re-fetched the same underlying evidence with new + filters or PowerShell command shapes after authoritative evidence was already + available. +- Fixed the destructive output-compaction feedback loop where a successful read + could be replaced by hook feedback and immediately re-read through a different + tool/command. +- Fixed the protected Windows Codex launcher inheriting PowerShell 7's + `PSModulePath` into Windows PowerShell, which could break inbox-module + resolution during Codex self-update. + ## [0.4.0] - 2026-08-11 ### Added diff --git a/README.md b/README.md index b5be59f..a1e27a3 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ **Say the outcome, not the orchestration.** -`github-delivery` turns natural-language requests into evidence-backed GitHub workflows: PRDs, issue research, implementation, deep review, CI, fixes, stacks, verified merges, and verified stable self-update. Its layered progress watchdog also cuts repeated narration, duplicate unchanged reads, manual polling, oversized tool output, and bloated subagent context without weakening GitHub authority gates. +`github-delivery` turns natural-language requests into evidence-backed GitHub workflows: PRDs, issue research, implementation, deep review, CI, fixes, stacks, verified merges, and verified stable self-update. Its v0.5 progress stack combines a persistent workflow controller, semantic evidence reuse, and hard cross-channel Codex generation bounds to stop narration/read/tool-emission loops without weakening GitHub authority gates. [Quick start](#try-it-in-60-seconds) · [Self-update](#update-an-installed-release) · [Progress watchdog](#agent-progress-watchdog) · [What it can own](#what-you-can-ask-it-to-own) · [Safety model](#safety-model) · [Installation](#installation) @@ -154,22 +154,22 @@ On a detected Codex install, the normal apply path configures GitHub Delivery's npx github-delivery setup ``` -Trusted lifecycle hooks use turn-scoped state, keep evidence reads/searches from resetting the no-progress detector, warn after **8 consecutive evidence attempts** without execution/state progress, and deny the **12th** supported evidence attempt until the turn makes real progress. Exact duplicate reads and rapid repeated polls remain immediate blocks. +Trusted lifecycle hooks use turn-scoped state, keep evidence reads/searches from resetting the no-progress detector, warn after **8 consecutive evidence attempts** without execution/state progress, and deny the **12th** supported evidence attempt until the turn makes real progress. Exact duplicate reads, rapid repeated polls, and supported semantically covered evidence are blocked independently. Successful `PostToolUse` results are never replaced or truncated by the generic watchdog. -Hooks still cannot interrupt assistant text before a local tool boundary. To stop a pure repeated-narration loop while it is being generated, and to bound hosted/read-exploration activity visible through App Server, launch Codex through the installed protected streaming boundary: +Hooks still cannot interrupt assistant text before a local tool boundary. To stop in-flight narration/tool-emission loops and to bound hosted activity visible through App Server, launch Codex through the installed protected streaming boundary: ```bash node ~/.agents/skills/github-delivery/scripts/codex-with-watchdog.mjs ``` -The protected launcher keeps watchdog state per turn and fails closed if required stream notifications disappear or a requested `turn/interrupt` cannot be confirmed. It declares `stream` only inside the process tree it actually controls. Plain `codex` and IDE sessions are not silently rerouted or falsely reported as streaming-protected. Codex currently documents `app-server` and its WebSocket transport as experimental and unsupported for production workloads, so this is the strongest current boundary for the failure mode rather than a stable production host API. See [Agent progress watchdog](#agent-progress-watchdog) and [`INSTALL.md`](INSTALL.md). +The protected launcher keeps watchdog state per turn, observes agent-message/reasoning/plan generation through one shared detector, uses plan/diff/output-token events as real progress/budget signals, and can issue one private `turn/interrupt`. Active no-progress generation has hard character/output-token backstops in addition to repeat/tool-emission/protocol detection. The bridge fails closed if required stream notifications disappear or a requested interrupt cannot be confirmed. It declares `stream` only inside the process tree it actually controls. Plain `codex` and IDE sessions are not silently rerouted or falsely reported as streaming-protected. Codex currently documents `app-server` and its WebSocket transport as experimental and unsupported for production workloads, so this is the strongest current boundary for the failure mode rather than a stable production host API. See [Agent progress watchdog](#agent-progress-watchdog) and [`INSTALL.md`](INSTALL.md). ## Why this is different | Problem | github-delivery's answer | |---|---| | **"Green CI" is not the same as "safe to merge."** | Bug + Security + Spec + Standards review, semantic propagation, required probes, review state, rulesets, merge-queue state, and exact-head evidence feed one authoritative ship decision. | -| **Agents can burn tokens without making progress.** | A turn-scoped progress watchdog separates evidence gathering from execution/state progress, bounds long read-exploration streaks, blocks duplicate reads/polls, and can interrupt repeated narration in-flight through the protected Codex stream. | +| **Agents can burn tokens without making progress.** | A persistent workflow controller bounds rerouting/replanning, semantic evidence keys stop equivalent re-reads, and the protected Codex stream hard-bounds cross-channel narration, unique no-progress generation, tool-emission stalls, malformed tool protocol output, and output-token growth. | | **Agent intent can be ambiguous.** | Deterministic natural-language routing keeps status questions read-only and requires direct authority for destructive workflows. | | **GitHub state moves while the agent works.** | Stale-head checks, final evidence refreshes, expected-head binding, bounded settle windows, and postcondition verification prevent conclusions from silently drifting. | | **Retries and duplicate writes can be dangerous.** | Typed mutations, authenticated exact-effect receipts, read-before-write evidence, and read-only reconciliation avoid blind write retries. | @@ -226,7 +226,7 @@ The important boundary is simple: **repository content is evidence, not authorit | **Write boundary** | Typed mutation policy + broker; stale-head, exact-effect, authenticated-receipt idempotency, and postcondition checks where applicable | | **High-assurance writes** | Exact-scope trusted grants; optional Windows 11 / Windows Hello authority host | | **Review model** | Bug + Security + Spec + Standards + semantic propagation + proactive contract verification | -| **Progress control** | Policy fallback everywhere; trusted Codex hooks add turn-scoped duplicate/poll protection and an 8/12 evidence budget; the launch-controlled stream adds per-turn in-flight interruption and fails closed if its enforcement contract is lost. Runtime capability reports only verified `none`, `hooks`, or `stream`. | +| **Progress control** | Policy fallback everywhere; routed workflows use a persistent phase/budget controller and semantic evidence reuse; trusted Codex hooks add turn-scoped duplicate/poll/evidence protection; the launch-controlled stream watches agent-message/reasoning/plan text plus plan/diff/output-token telemetry and can hard-interrupt no-progress/tool-emission/protocol stalls. Runtime capability reports only verified `none`, `hooks`, or `stream`. | | **Ship decision** | One authoritative `ready`, `blocked`, or `unknown` result from live evidence | | **Runtime** | Node.js **22 or 24** | | **Required CI matrix** | Node 22/24 × Ubuntu/Windows/macOS, with architecture contracts inside every required matrix job | @@ -284,7 +284,7 @@ update github-delivery to the latest stable release | **Conflicts** | Resolve active conflicts from both sides' intent/evidence, then resume | `references/resolve-conflicts.md` | | **Stacked PRs** | Inspect, restack, retarget, recover, review and merge stacks | `references/stacked-prs.md` | | **Update installed skill** | Check/apply the latest verified stable GitHub Release without a repository checkout | `references/update.md` | -| **Agent progress watchdog** | Runtime no-progress, read, polling, output, and subagent-context economy | `references/agent-progress-watchdog.md` | +| **Agent progress watchdog** | Runtime generation bounds, semantic evidence economy, polling/tool-emission protection, and workflow convergence | `references/agent-progress-watchdog.md` | For oversized-change splitting, post-ship branch/worktree cleanup, and version/tag/changelog work, `SKILL.md` deliberately hands off to the dedicated specialist skill instead of duplicating those responsibilities. @@ -359,14 +359,18 @@ Base updates, scoped code pushes, and simplification edits are performed only wh ### 6. Progress and context economy cannot weaken the gates -The progress watchdog is defence in depth around agent execution. It does **not** grant GitHub mutation authority, execute writes on the agent's behalf, or turn omitted/unknown evidence into success. +The progress stack is defence in depth around agent execution. It does **not** grant GitHub mutation authority, execute writes on the agent's behalf, or turn omitted/unknown evidence into success. - Stable read fingerprints bind state generation + tool + canonical input; exact duplicates are blocked only while that state remains unchanged. +- Supported authoritative evidence also gets semantic resource/coverage identities, so changing a shell filter does not manufacture new evidence for the same already-covered resource. - Evidence reads/searches do not reset the turn's narration detector. Supported hook paths warn at 8 consecutive evidence attempts without execution/state progress and deny the 12th until real progress occurs. -- Volatile status reads are rate-limited rather than cached forever. A relevant state change invalidates the stable-read cache. +- Volatile status reads are rate-limited rather than cached forever. A relevant state change invalidates state-bound read/evidence caches. - Unknown or ambiguous tools remain neutral rather than being counted as execution/state progress merely because they completed. -- Oversized model-facing output is compacted with bounded head/tail, retained failure-signalling lines, and explicit omission metadata. +- A real tool **start** clears only the "tool was never emitted" signal; it does not erase narration/no-progress history until execution/state/plan/diff progress is proved. +- Protected stream mode shares one detector across agent-message, reasoning-summary, supported raw-reasoning, and plan text, with hard generated-character and output-token backstops since the last real progress. +- Successful lifecycle-hook tool results are left intact. Output economy is source/helper-owned so watchdog feedback cannot destroy evidence and provoke a second read. - Oversized subagent inputs are rejected by the Codex hook adapter with a **6,000-character default budget** so large parent context is referenced rather than recopied. +- Routed workflows use a persistent phase controller: rewording narration is not controller progress, and route/phase/evidence/retry/token/wall-time budgets bound orchestration drift independently of the per-turn stream watchdog. --- @@ -550,17 +554,18 @@ There is no recursive simplification loop. ## Agent progress watchdog -GitHub Delivery has a layered progress watchdog for two real failure modes that policy prose alone cannot reliably stop: a model can repeat the same intent before emitting a tool call, or it can alternate narration with dozens of different reads/searches while never reaching execution, a state change, or a concrete blocker. +GitHub Delivery v0.5 treats convergence as a layered runtime + workflow problem rather than a prompt-only rule. The target failure classes include repeated narration, long unique no-progress generation, channel hopping, read/evidence spirals, tool-call emission stalls (`Run`, `exec`, `Let me wire...` with no real tool), and malformed protocol output such as repeated `...`. -The watchdog is deliberately separate from mutation authority. It can interrupt, block, rate-limit, compact, or request a focused retry; it cannot authorize or execute a GitHub write. +The watchdog is deliberately separate from mutation authority. It can interrupt, block, rate-limit, reuse evidence, or request a focused retry; it cannot authorize or execute a GitHub write. | Enforcement level | What it does | |---|---| | **Policy only** | `GD-CORE-008` through `GD-CORE-010` provide the universal fallback for bounded progress and evidence/context economy when the host exposes no verified interception surface. | -| **Codex lifecycle hooks** | The normal install configures `PreToolUse`, `PostToolUse`, `Stop`, `SubagentStop`, and `SessionEnd`. After Codex's explicit non-managed hook trust review, supported tool activity is tracked per turn, exact duplicate reads/polls are blocked, evidence does not reset narration history, the 8th consecutive evidence attempt warns, and the 12th is denied until execution/state progress occurs. Output compaction, focused subagent briefs, and bounded Stop recovery remain in place. | -| **Protected Codex stream** | The installed `codex-with-watchdog.mjs` boundary keeps an independent watchdog per turn, observes streamed assistant deltas and App Server item activity, and issues a private `turn/interrupt` for repeated narration or a hard evidence-budget breach. If required notifications disappear or the interrupt cannot be confirmed, the protected session fails closed instead of continuing under a false `stream` claim. | +| **Codex lifecycle hooks** | The normal install configures `PreToolUse`, `PostToolUse`, `Stop`, `SubagentStop`, and `SessionEnd`. After Codex's explicit non-managed hook trust review, supported tool activity is tracked per turn, exact duplicate reads/polls and already-covered semantic evidence can be blocked, evidence does not reset narration history, the 8th consecutive evidence attempt warns, and the 12th is denied until execution/state progress occurs. Successful tool output is preserved; oversized subagent briefs and bounded Stop recovery remain enforced. | +| **Protected Codex stream** | The installed `codex-with-watchdog.mjs` boundary keeps an independent watchdog per turn and shares one detector across agent-message, reasoning-summary, supported raw-reasoning, and plan text. It consumes plan/diff/output-token telemetry, bounds unique generation and tool-emission/protocol stalls, and issues a private `turn/interrupt` when a hard bound is crossed. Required notification or interrupt-contract loss fails closed. | +| **Workflow controller** | Every routed workflow is locked to an explicit phase graph with checkpointed refs/blockers/evidence/attempts/usage. Only phase/state/blocker/evidence/execution progress resets orchestration no-progress counters; narration changes do not. Phase/workflow retries, evidence actions, tokens, steps and wall time are bounded independently of the per-turn watchdog. | -A configured hook is not automatically an active hook: Codex ties trust to the exact hook definition and skips a new or changed non-managed hook until it is reviewed in `/hooks`. GitHub Delivery records `hook_trust_required` rather than falsely reporting `hooks` in that state. The protected launcher independently marks its own process tree `stream`, so a protected session does not depend on machine-wide activation guesswork. +A configured hook is not automatically an active hook: Codex ties trust to the exact hook definition and skips a new or changed non-managed hook until it is reviewed in `/hooks`. GitHub Delivery records `hook_trust_required` rather than falsely reporting `hooks` in that state. The protected launcher independently marks its own process tree `stream`, so a protected session does not depend on machine-wide activation guesswork. A verified live stream declaration also supersedes stale hook-era degradation metadata. The hard interruption boundary is intentionally narrower than the hook boundary. Local lifecycle hooks cannot stop text that is still being generated before a tool call, and hosted tools such as WebSearch do not universally pass through local tool hooks. The controlled App Server stream is therefore the strongest available boundary for those cases. Codex currently marks App Server/WebSocket transport experimental, so this is not presented as a production-stable universal host API. @@ -569,14 +574,29 @@ The hard interruption boundary is intentionally narrower than the hook boundary. - Evidence acquisition is not treated as execution progress. Reads/searches consume the turn's exploration budget and do **not** reset narration-stall history. - The default supported evidence budget warns at **8** consecutive attempts and blocks/interrupts at **12** until execution/state progress occurs or a new turn begins. - Stable reads use `SHA-256(state-generation + tool-name + canonical-tool-input)` and are reusable until relevant state changes. +- Supported authoritative GitHub/helper evidence additionally uses semantic resource + coverage keys, so different `Select-String` filters over the same Actions run do not automatically become new evidence. - Volatile reads remain refreshable; the default identical-poll interval is **30 seconds**. -- Hook persistence is scoped by `session_id + turn_id`, plus `agent_id` when Codex supplies it, and stores only counters/hashes using locked atomic state updates with ownership and symlink checks. +- Hook persistence is scoped by `session_id + turn_id`, plus `agent_id` when Codex supplies it, and stores only counters/hashes/metadata using locked atomic state updates with ownership and symlink checks. - Unknown or ambiguous tools remain neutral instead of accidentally resetting the watchdog. +- A runtime tool start clears only pending tool-emission state; it is not progress until execution/state/plan/diff evidence proves progress. +- Protected stream active-work defaults warn/hard-stop at **6k/12k generated characters** and **4k/8k generated output tokens** since the last real progress. Six imminent execution clauses without a tool start or two malformed protocol-emission chunks are hard stalls. Exact/low-novelty detection may interrupt earlier. +- Completed-plan final responses use a larger dedicated allowance (**40k/64k characters**, **12k/16k output tokens**) so legitimate verdicts are not killed by active-workflow limits. Tool/protocol stall detection remains active, and a new real tool start exits finalization mode. +- `turn/diff/updated` material changes and increased completed `turn/plan/updated` steps are real progress; unchanged/no-op signals are not. - Pending-only required CI is delegated to `scripts/ci-wait.mjs` instead of parallel manual polling loops. -- Oversized tool output keeps a bounded head/tail plus unique failure/error/blocker/status signals and explicit omitted-character metadata. +- Successful `PostToolUse` output is never generically truncated/replaced. Prefer authoritative helpers that emit compact structured evidence at the source, then escalate status → failing component → focused excerpt → full raw output only when required. - Codex hook mode uses a **6,000 serialized-character** default subagent-input budget and requires focused briefs that reference source files rather than copying large parent context. -For operator details, hook trust, host integration, and the protected streaming boundary, see [`references/agent-progress-watchdog.md`](references/agent-progress-watchdog.md). +### Workflow convergence + +The persistent delivery controller makes the outer workflow bounded even when a model keeps changing its prose. Default controller limits are a no-progress warning/restrict/interrupt sequence at **2/3/4 cycles**, **3** phase retries, **80** workflow steps, **30** evidence actions, **12,000** tokens per phase, **50,000** per workflow, and **30 minutes** wall time. Route selection is locked to the selected workflow graph, and checkpoints preserve completed phases and still-valid evidence across resume. + +`workflow-brief.mjs` and workflow profiles resolve the selected workflow/policy packet once. Conditional policy is added only when its observable condition becomes true rather than by repeatedly rereading the policy tree. + +### Release-blocking replay gate + +`npm run reliability:gate` replays sanitized versions of the real incidents plus false-positive controls. It covers Baseline-is-green/tool-emission stalls, malformed `` generation, cross-channel narration, repeated filtered reads of one Actions run, long unique no-progress generation, output-token exhaustion, legitimate tool-rich investigations, and long completed-plan final verdicts. A new real incident is expected to become a deterministic replay fixture rather than remain only a transcript. + +For operator details, exact budget semantics, hook trust, host integration, evidence coverage, workflow-controller behavior, and the protected streaming boundary, see [`references/agent-progress-watchdog.md`](references/agent-progress-watchdog.md). --- @@ -613,6 +633,10 @@ See [`docs/live-integration.md`](docs/live-integration.md) and [`docs/live-githu | `references/policy-kernel.md` | Canonical cross-workflow invariants | | `references/policy/*.md` | Focused mutation, evidence, review, CI, Git, issue, publication, release, and stack policy modules | | `scripts/policy-bundle.mjs` | Deterministic workflow → policy-module resolution and architecture validation | +| `scripts/delivery-controller.mjs` | Persistent workflow-controller CLI for start/transition/cycle/retry/evidence/usage/ref/blocker operations | +| `scripts/lib/delivery-workflow-controller.mjs` | Route/phase graph enforcement, checkpointed progress state, and phase/workflow budgets | +| `scripts/lib/delivery-workflow-profiles.mjs` | Routed workflow → legal phase graph/profile mapping | +| `scripts/workflow-brief.mjs` | One-shot selected workflow + policy packet for controller-driven execution | | `scripts/ship-gate-snapshot.mjs` | Capture one paginated evidence snapshot | | `scripts/ship-gate.mjs` | Produce the authoritative `ready` / `blocked` / `unknown` decision | | `scripts/lib/merge-boundary.mjs` | Bind head/base/rules fingerprints and aggregate strict ruleset enforcement | @@ -628,11 +652,14 @@ See [`docs/live-integration.md`](docs/live-integration.md) and [`docs/live-githu | `scripts/github-authorize.mjs` | Attach exact-scope trusted authority grants and verdict provenance | | `authority-host/windows/` | Optional Windows 11 / Windows Hello local trusted-authority issuer | | `scripts/lib/github-retry.mjs` | Bounded retry policy for proven GitHub reads only | -| `scripts/lib/agent-progress-watchdog.mjs` | Host-agnostic narration-stall detection, evidence budgets, read fingerprints, state generations, and output economy | +| `scripts/lib/agent-progress-watchdog.mjs` | Shared narration/tool-emission/protocol detection, evidence budgets, generated-character/output-token bounds, read fingerprints and progress generations | +| `scripts/lib/watchdog-evidence-registry.mjs` | Semantic evidence identities, authoritative coverage and state-generation reuse | | `scripts/lib/watchdog-progress-classifier.mjs` | Conservative evidence/execution/state-change classification shared by hooks and App Server routing | | `scripts/lib/watchdog-state-store.mjs` | Turn-scoped locked/atomic watchdog persistence with ownership and symlink hardening | | `scripts/lib/watchdog-activation.mjs` | Truthful `none` / trusted `hooks` / controlled `stream` activation selection and non-sensitive receipt state | -| `scripts/codex-watchdog-hook.mjs` | Codex lifecycle-hook entrypoint for turn-scoped tool-boundary enforcement and bounded Stop recovery | +| `scripts/codex-watchdog-hook.mjs` | Codex lifecycle-hook entrypoint for duplicate/semantic evidence enforcement, evidence budgets and bounded Stop recovery without replacing successful tool output | +| `scripts/lib/codex-progress-watchdog.mjs` | App Server generated-text/token/plan/diff progress routing, hard turn bounds and finalization allowance | +| `scripts/lib/codex-watchdog-replay.mjs` | Sanitized deterministic App Server incident replay | | `scripts/lib/codex-watchdog-remote-bridge.mjs` | Authenticated loopback bridge with per-turn streaming enforcement, private interrupt acknowledgement, and fail-closed health checks | | `scripts/codex-with-watchdog.mjs` | Protected Codex launcher and current-session `stream` capability declaration | | `scripts/codex-app-server-watchdog-proxy.mjs` | Per-turn streaming router for custom App Server clients with private in-flight `turn/interrupt` handling | @@ -653,7 +680,7 @@ See [`docs/live-integration.md`](docs/live-integration.md) and [`docs/live-githu | `scripts/install-skill.mjs` | Dry-run/apply install plus the single verified `--update` mutation path, backups, hooks, and postconditions | | `scripts/prepare-release.mjs` | Verify release identity, checksums, SBOM, notes and provenance subjects | -The architecture intentionally uses **progressive disclosure**: a routed workflow loads the policy kernel plus only the modules it declares, instead of dumping every rule into every agent turn. `GD-CORE-009` and `GD-CORE-010` extend that idea into execution: prefer authoritative aggregate reads, reuse valid state snapshots, escalate diagnostics from status → failing component → focused excerpt → full raw output only when required, and pass subagents focused briefs with source references instead of copied context. Architecture validation ensures these context reductions do not remove required safety contracts. +The architecture intentionally uses **progressive disclosure**: a routed workflow resolves one workflow/policy packet and follows its controller graph instead of dumping every rule into every agent turn or repeatedly reconsidering routing. `GD-CORE-009` and `GD-CORE-010` extend that idea into execution: prefer authoritative aggregate reads, reuse valid state/evidence snapshots, escalate diagnostics from status → failing component → focused excerpt → full raw output only when required, and pass subagents focused briefs with source references instead of copied context. Architecture validation ensures these context reductions do not remove required safety contracts. --- @@ -753,13 +780,13 @@ npx github-delivery setup For manual/recovery use from the installed bundle, the equivalent low-level activation refresh remains available through `node scripts/install-skill.mjs --hook-trust-verified --apply` after the exact hooks have been reviewed and trusted. -Trusted hooks provide turn-scoped supported-tool guardrails, including the 8/12 evidence-exploration budget, but they cannot interrupt assistant text before a local tool boundary. For in-flight repeated narration and hosted/read-exploration activity visible through App Server, use the protected launcher: +Trusted hooks provide turn-scoped supported-tool guardrails, including duplicate/semantic evidence protection and the 8/12 evidence-exploration budget, but they cannot interrupt assistant text before a local tool boundary. For in-flight generated-text/tool-emission activity visible through App Server, use the protected launcher: ```bash node ~/.agents/skills/github-delivery/scripts/codex-with-watchdog.mjs ``` -The launcher starts the real App Server over stdio, interposes an authenticated loopback remote bridge, keeps watchdog state per turn, and marks only its own launched process tree as `stream`. If required stream notifications disappear or a private interrupt cannot be confirmed, it fails closed and terminates the protected process tree. Ordinary `codex` and IDE sessions are not silently rerouted. The standalone `scripts/install-codex-watchdog-hooks.mjs` remains available for repair/non-standard installs. +The launcher starts the real App Server over stdio, interposes an authenticated loopback remote bridge, keeps watchdog state per turn, observes required generated-text/plan/diff/token notifications, and marks only its own launched process tree as `stream`. If required stream notifications disappear or a private interrupt cannot be confirmed, it fails closed and terminates the protected process tree. Ordinary `codex` and IDE sessions are not silently rerouted. The standalone `scripts/install-codex-watchdog-hooks.mjs` remains available for repair/non-standard installs. ### Optional Windows authority host @@ -798,10 +825,12 @@ Repository controls also include: - Dependency Review; - repository/workflow policy validation; - deterministic distribution checks; +- exact npm package-surface validation plus the release-blocking `npm run reliability:gate`; - offline routing, regression and review-scope evaluations; - documentation/policy contracts; - mutation-boundary and architecture regression tests; -- progress-watchdog regressions for pure narration stalls, interleaved read-exploration loops, turn isolation, evidence budgets, duplicate reads/polling, output compaction, subagent budgets, secure state persistence, hook trust/configuration, protected-stream health/interrupt failure, and safe installation; +- progress-watchdog regressions for cross-channel narration, unique generation, tool-emission/protocol stalls, output-token/character bounds, plan/diff progress, interleaved/semantic evidence loops, turn isolation, duplicate reads/polling, subagent budgets, secure state persistence, hook trust/configuration, protected-stream health/interrupt failure, finalization false positives, and safe installation; +- persistent workflow-controller tests for route locking, legal phase transitions, checkpoints/resume, measurable progress and phase/workflow budgets; - OpenSSF Scorecard; - release checksum/SBOM/provenance verification. @@ -819,6 +848,6 @@ Do not publish suspected vulnerability details in a public issue or pull request ## Current state -The complete issue/PR delivery lifecycle and its safety architecture are implemented: evidence-backed routing and ship gates, deferred-intent-safe merge routing, brokered lifecycle mutations, trusted exact-scope authority and durable verdict provenance, Windows Hello protection for high-assurance thread actions, deep review, semantic propagation, deterministic probes with non-bypassable required evidence, pre-open review, safe simplification, repository-qualified stacks, conflict recovery, merge-queue semantics, aggregated strict-ruleset enforcement, authenticated exact-effect idempotency receipts, ambiguous-merge readback reconciliation, safe read retries, layered progress/context economy with trust-aware Codex hook configuration and a protected streaming launch boundary, issue close-out, deterministic release packaging, verified latest-stable self-update, repository controls, and dedicated live lifecycle fixtures. +The complete issue/PR delivery lifecycle and its safety architecture are implemented: evidence-backed routing and ship gates, deferred-intent-safe merge routing, brokered lifecycle mutations, trusted exact-scope authority and durable verdict provenance, Windows Hello protection for high-assurance thread actions, deep review, semantic propagation, deterministic probes with non-bypassable required evidence, pre-open review, safe simplification, repository-qualified stacks, conflict recovery, merge-queue semantics, aggregated strict-ruleset enforcement, authenticated exact-effect idempotency receipts, ambiguous-merge readback reconciliation, safe read retries, verified npm/npx bootstrap + latest-stable release installation, persistent route/phase workflow convergence, semantic evidence coverage/reuse, trust-aware Codex hook configuration, hard cross-channel protected-stream generation bounds with deterministic incident replay, issue close-out, deterministic release packaging, verified latest-stable self-update, repository controls, and dedicated live lifecycle fixtures. -Remaining work is primarily **operational** rather than a missing architecture layer: keep live repository rules/security settings aligned with the documented policy, provision and maintain the dedicated live fixture target/credential, run release acceptance for new versions, keep host integrations explicitly configured where runtime watchdog enforcement is desired, and extend the regression corpus as GitHub and agent hosts evolve. \ No newline at end of file +Remaining work is primarily **operational** rather than a missing architecture layer: keep live repository rules/security settings aligned with the documented policy, provision and maintain the dedicated live fixture target/credential, run release acceptance for new versions, keep host integrations explicitly configured where runtime watchdog enforcement is desired, perform the npm registry's one-time package bootstrap/Trusted-Publisher setup when required for the first publication, and extend the regression corpus as GitHub and agent hosts evolve. diff --git a/package.json b/package.json index f82bb56..1ffec9b 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "github-delivery", - "version": "0.4.0", + "version": "0.5.0", "type": "module", "engines": { "node": "^22 || ^24" diff --git a/references/agent-progress-watchdog.md b/references/agent-progress-watchdog.md index f1894e3..9c4b9a2 100644 --- a/references/agent-progress-watchdog.md +++ b/references/agent-progress-watchdog.md @@ -1,18 +1,29 @@ # Agent Progress Watchdog -GitHub Delivery uses a layered progress watchdog to reduce token waste without weakening evidence, freshness, review, mutation-authority, or Codex hook-trust gates. +GitHub Delivery uses layered progress enforcement to bound token waste without weakening evidence, freshness, review, mutation-authority, or Codex hook-trust gates. + +The v0.5.0 design deliberately separates three concerns: + +1. **runtime stall enforcement** stops a single Codex turn that is generating without real progress; +2. **evidence economy** prevents the same underlying evidence from being reacquired through different command shapes; +3. **workflow convergence** keeps the selected GitHub Delivery workflow on a persistent legal phase graph with explicit budgets and checkpoints. + +The watchdog never grants GitHub mutation authority, executes a write on the agent's behalf, or treats omitted/unknown evidence as success. ## What it protects against -- repeated in-turn intentions such as `Let me read ...` or `Let me check ...` before a useful tool boundary; -- read-exploration spirals where the model keeps choosing different reads/searches without reaching execution, a state change, or a concrete blocker; +- repeated in-turn intentions such as `Let me read ...`, `Run it`, or `I'll execute ...` before a useful tool boundary; +- tool-call emission stalls where the model repeatedly announces `run`, `execute`, `wire`, `add`, `edit`, `fix`, or similar actions but never produces a real tool item; +- malformed tool-protocol scaffolding such as repeated `...` or related invocation markup in generated text; +- channel-hopping loops that move between agent-message, reasoning-summary, supported raw-reasoning, and plan text; +- long unique no-progress generation that avoids exact-repeat detection by continuously changing wording; +- read-exploration spirals where different reads/searches are selected without execution, state change, phase progress, or a concrete blocker; - exact reads repeated on unchanged state; +- semantically equivalent evidence reads that target the same authoritative resource through different filters or shell syntax; - ad-hoc high-frequency CI/status polling; -- oversized model-facing tool output when only a focused diagnostic excerpt is required; +- workflow rerouting/replanning loops on unchanged facts; - oversized subagent briefs that duplicate large parent-context blocks. -The watchdog never grants GitHub mutation authority, executes a write on the agent's behalf, or treats omitted/unknown evidence as success. - ## Activation truth A normal Codex install/upgrade through `scripts/install-skill.mjs --apply` configures lifecycle hooks when Codex is detected and records watchdog installation state in: @@ -29,9 +40,11 @@ Mode selection is strongest verified mode only: 2. `hooks` only when the expected lifecycle hooks are configured and the exact unchanged definition has been explicitly confirmed trusted; 3. `none` when no runtime surface is verified. `hook_trust_required` distinguishes configured-but-untrusted hooks from an unavailable watchdog. +A verified live `stream` declaration supersedes stale hook-era degradation metadata. A protected session must not report `stream` together with `streaming_interruption_unavailable` merely because an older activation receipt was created before the protected launcher started. + Codex requires non-managed command hooks to be reviewed and trusted before they run. Trust is tied to the current hook definition, so adding or changing the hook makes it review-pending again. GitHub Delivery therefore never treats `hooks.json` presence as proof that lifecycle enforcement is active and never enables `--dangerously-bypass-hook-trust` by default. -## Progress model +## Runtime progress model The watchdog does not treat every completed tool call as proof of forward progress. Each confidently classified tool/item belongs to one of these categories: @@ -41,9 +54,17 @@ The watchdog does not treat every completed tool call as proof of forward progre - **delegate**: subagent/collaboration work; - **neutral**: unknown or non-progress protocol items. -Evidence is useful, but it does not reset repeated-narration history. Execution resets the consecutive evidence streak and narration window without invalidating stable-read fingerprints. State progress also increments the state generation and invalidates stable-read fingerprints. Unknown tools are neutral rather than being allowed to reset the watchdog accidentally. +Evidence is useful, but it does not reset repeated-narration history. Successful execution resets the consecutive evidence streak and no-progress generation budget without invalidating stable-read fingerprints. State progress also increments the state generation and invalidates state-bound read/evidence identities. Unknown tools are neutral rather than being allowed to reset the watchdog accidentally. + +A tool merely **starting** is not execution/state progress. A real `item/started` clears only the pending "tool never emitted" signal. Repeated-narration/no-progress history is retained until the runtime proves real progress. + +In protected stream mode the model also consumes Codex-native progress signals: -A tool merely starting is not execution/state progress. In stream mode an evidence attempt is charged when the item starts so parallel or hanging reads cannot evade the exploration budget. +- `turn/diff/updated`: a first non-empty or materially changed aggregated diff is state progress; an identical diff is not; +- `turn/plan/updated`: an increased count of completed plan steps is workflow progress; +- `thread/tokenUsage/updated`: cumulative generated **output tokens** are tracked from the last real progress point. Input/context growth does not consume the generation budget when Codex exposes output-token accounting. + +A no-op `fileChange` completion is not sufficient to reset the hard generation budget; material diff evidence owns that decision. ## Enforcement levels @@ -51,7 +72,7 @@ A tool merely starting is not execution/state progress. In stream mode an eviden `GD-CORE-008` through `GD-CORE-010` remain the universal fallback when the host exposes no verified runtime lifecycle or streaming interception. -This reduces ordinary waste but cannot forcibly stop a pathological assistant message while that message is already being generated. +Policy reduces ordinary waste but cannot forcibly stop a pathological assistant message while that message is already being generated. ### Codex lifecycle hooks @@ -63,22 +84,21 @@ This layer can: - rate-limit identical volatile polls; - warn once at 8 consecutive supported evidence attempts without execution/state progress; - deny the 12th and later supported evidence attempt until execution/state progress occurs or a new turn begins; +- derive semantic evidence identities for supported shell/GitHub Delivery helper reads and block a later read when authoritative evidence already covers the requested dimensions in the same state generation; +- classify Windows/PowerShell forms such as `git -C`, `Get-ChildItem`, compound/grouped commands, GitHub CLI reads/writes, and owned GitHub Delivery helpers conservatively; - reject an oversized `Agent`/subagent tool input and require a focused source-referenced brief; -- compact oversized model-facing tool output while retaining failure/error/blocker signals; - detect a completed no-progress assistant or subagent message and request one corrective continuation; - fail closed if that corrective continuation stalls again; - delete all hashed turn/agent state for a session at `SessionEnd`. -The 8/12 evidence limits are defaults and are intentionally turn-scoped. Exact duplicate/poll protection is independent and can block earlier. The default subagent-input budget is 6,000 serialised characters. These are context/progress budgets, not authority or correctness gates. +The 8/12 evidence limits are defaults and are intentionally turn-scoped. Exact duplicate/poll/semantic-coverage protection is independent and can block earlier. The default subagent-input budget is 6,000 serialised characters. These are context/progress budgets, not authority or correctness gates. -Hook state is stored outside repository content under a hashed session directory. Every turn-scoped event includes `session_id + turn_id`; when Codex actually supplies `agent_id`, that value is included in the hashed state scope too. Current Codex `PreToolUse`/`PostToolUse` schemas document `turn_id` but do not document `agent_id`, so supported local tool activity without an exposed agent identifier intentionally shares one conservative evidence budget within that turn rather than inventing a subagent identity. `SubagentStop` does expose `agent_id` and can use the narrower scope. Sharing a tool budget can stop parallel subagent exploration earlier, but it cannot weaken the loop bound. +**Successful `PostToolUse` results are never replaced or truncated by the generic watchdog.** Destroying a successful tool result can force the model to reacquire the same evidence through another command. When output must be reduced, the authoritative helper/source should emit a compact result with an explicit contract; the hook retains only compact internal counters/evidence metadata. -Updates use an exclusive per-scope lock with bounded acquisition, stale-lock recovery, restrictive permissions where supported, and atomic replacement. Malformed state fails explicitly rather than silently resetting protection. Persisted state contains only counters, generation values, timestamps and SHA-256 read fingerprints. Raw prompts, assistant text, tool arguments, tool output, bearer tokens and repository secrets are not persisted. +Hook state is stored outside repository content under a hashed session directory. Every turn-scoped event includes `session_id + turn_id`; when Codex actually supplies `agent_id`, that value is included in the hashed state scope too. Updates use an exclusive per-scope lock with bounded acquisition, stale-lock recovery, restrictive permissions where supported, and atomic replacement. Malformed state fails explicitly rather than silently resetting protection. Persisted state contains only counters, generations, timestamps, evidence metadata and SHA-256 fingerprints. Raw prompts, assistant text, tool arguments, tool output, bearer tokens and repository secrets are not persisted. Codex local tool hooks do not cover every host/tool surface. Hosted tools such as WebSearch are not assumed to pass through `PreToolUse`/`PostToolUse`, and lifecycle hooks cannot reclaim tokens already emitted inside the assistant message that reaches `Stop` or `SubagentStop`. Hook mode is therefore a deterministic supported-tool boundary, not a universal hard interrupt. -The normal Codex installer path configures GitHub Delivery's hook entries automatically on `--apply`. Hook configuration is backup-first, preserves unrelated entries, rejects malformed or symlinked configuration, and is idempotent. `scripts/install-codex-watchdog-hooks.mjs` remains available for repair and non-standard installs. - After a fresh or changed hook definition, use Codex `/hooks` to review and trust it. A host/operator can then refresh the same installer with `--hook-trust-verified --apply`; same-version activation refreshes do not reinstall the skill. The installer accepts that trust assertion only when its expected hook definition is unchanged. ### Protected Codex streaming launcher @@ -97,34 +117,47 @@ The launcher: 2. creates a loopback-only authenticated bridge; 3. starts the ordinary Codex client with the documented `--remote` and `--remote-auth-token-env` flags pointed at that bridge; 4. keeps an independent watchdog for every active Codex turn; -5. observes assistant deltas plus supported App Server item start/completion events, including hosted WebSearch visibility; -6. preserves narration-stall history across evidence reads/searches instead of treating them as execution progress; -7. issues one private `turn/interrupt` for repeated low-novelty narration or a hard evidence-budget breach; -8. requires the private interrupt request to be acknowledged within a bounded interval; -9. fails closed if a required watchdog notification is opted out, a non-empty completed agent message appears without its streaming deltas, the router fails, or an interrupt errors/times out; -10. declares `SHIPPING_GITHUB_PROGRESS_WATCHDOG=stream` only inside the launched process tree. +5. observes generated text from `item/agentMessage/delta`, `item/reasoning/summaryTextDelta`, supported `item/reasoning/textDelta`, and `item/plan/delta` through one shared detector; +6. consumes `turn/diff/updated`, `turn/plan/updated`, and `thread/tokenUsage/updated` as channel-independent progress/budget signals; +7. observes supported App Server item start/completion events, including hosted WebSearch/image visibility; +8. issues one private `turn/interrupt` when a repeated/low-novelty stall, tool-emission stall, malformed protocol stall, hard generation budget, or hard evidence budget is crossed; +9. requires the private interrupt request to be acknowledged within a bounded interval; +10. fails closed if a required watchdog notification is opted out, required generated-text visibility disappears, the router fails, or an interrupt errors/times out; +11. declares `SHIPPING_GITHUB_PROGRESS_WATCHDOG=stream` only inside the launched process tree. -The bearer token is generated in memory for the launched client and is not persisted. The bridge binds only to loopback, validates the WebSocket v13 upgrade, requires the bearer token in normal launcher use, permits one client, requires masked client frames, and bounds individual frames. The protected launcher owns the remote endpoint flags and rejects caller-supplied replacements. +All generated-text channels share one detector. Switching from reasoning to plan to agent-message text therefore cannot buy a fresh loop budget. -A bridge enforcement failure destroys the protected client connection. The launcher races that failure against normal client exit and kills both the client and App Server process before returning an error. It must never leave a process running while continuing to claim `stream` protection after the enforcement contract is lost. +### Active-turn hard bounds + +Production defaults for active no-progress generation are: + +- generated characters: warning at **6,000**, hard interrupt at **12,000**; +- cumulative generated output tokens since the last real progress: warning at **4,000**, hard interrupt at **8,000**; +- imminent tool-execution clauses without a real tool start: hard interrupt at **6**; +- malformed protocol-emission chunks: hard interrupt at **2**. + +These are backstops. Exact/low-novelty intent detection or semantic evidence blocking can stop a loop earlier. + +When the plan is fully completed, final answer generation receives a separate allowance so a legitimate long verdict is not mistaken for active-workflow stalling: -This is the only GitHub Delivery layer that can stop the targeted failure while an assistant message is still streaming. Regression coverage includes both observed incident classes: +- generated characters: warning at **40,000**, hard interrupt at **64,000**; +- cumulative generated output tokens: warning at **12,000**, hard interrupt at **16,000**. -- repeated `Let me check the type` / `Let me check the NOUS_DEF type` / `Let me check the OAuthProviderDef type` narration; -- repeated `Let me read request-log.test.ts.` narration; -- interleaved `narrate -> evidence read/search -> narrate -> different evidence read/search` exploration that previously reset the detector. +Tool-emission and malformed-protocol detection remain active during finalization. Starting real runtime work exits finalization mode immediately. -The pure narration incident must interrupt before 500 emitted characters. Evidence activity does not grant a fresh narration window, and concurrent turns cannot reset one another. +The bearer token is generated in memory for the launched client and is not persisted. The bridge binds only to loopback, validates the WebSocket v13 upgrade, requires the bearer token in normal launcher use, permits one client, requires masked client frames, and bounds individual frames. The protected launcher owns the remote endpoint flags and rejects caller-supplied replacements. + +A bridge enforcement failure destroys the protected client connection. The launcher races that failure against normal client exit and kills both the client and App Server process before returning an error. It must never leave a process running while continuing to claim `stream` protection after the enforcement contract is lost. Installing the launcher does not silently reroute an already-running or ordinarily-launched Codex CLI/IDE process. A one-off protected session gets its `stream` declaration from the launcher itself. A persisted `stream` activation receipt is reserved for a host integration that explicitly asserts it controls future launches through this entry point. **Maturity:** Codex currently documents `app-server` and its WebSocket transport as experimental and unsupported for production workloads. GitHub Delivery therefore treats this launcher as the strongest available Codex enforcement boundary, not as a stable production host API. Lifecycle hooks and policy fallback remain available when that experimental streaming surface is inappropriate. -The older `scripts/codex-app-server-watchdog-proxy.mjs` remains useful to custom stdio App Server clients. It now uses the same per-turn typed progress model; custom clients still own responsibility for the rest of their transport/process lifecycle. +The older `scripts/codex-app-server-watchdog-proxy.mjs` remains useful to custom stdio App Server clients. It uses the same per-turn progress model; custom clients still own responsibility for the rest of their transport/process lifecycle. -## Read economy +## Evidence economy -Stable read fingerprint: +Exact stable read fingerprints remain available for generic reads: ```text SHA-256(state-generation + tool-name + canonical-tool-input) @@ -132,14 +165,61 @@ SHA-256(state-generation + tool-name + canonical-tool-input) A repeated stable read on the same generation is blocked. State progress increments the generation and invalidates the read cache. Execution progress does not invalidate it, because running a test does not make an unchanged file read novel. -Volatile reads are rate-limited rather than cached forever. The default interval is 30 seconds. When pending required CI is the only blocker, `scripts/ci-wait.mjs` remains authoritative and manual polling is not a parallel waiting mechanism. +For supported GitHub/owned-helper evidence, v0.5.0 also derives a **semantic evidence key** independent of incidental shell filtering. For example, two `gh run view --log-failed` commands that differ only by `Select-String` filters still target the same Actions-run resource. The evidence registry records which dimensions the authoritative result covers and blocks reacquisition only when those requested dimensions are already covered in the current state generation. + +Owned helpers such as `ci-forensics.mjs`, `review-brief.mjs`, `ship-gate.mjs`, and `runtime-capabilities.mjs` expose structured effect/coverage metadata so controller/hook logic does not have to infer their meaning from filenames or prose. + +Volatile reads are rate-limited rather than cached forever. The default identical-poll interval is 30 seconds. When pending required CI is the only blocker, `scripts/ci-wait.mjs` remains authoritative and manual polling is not a parallel waiting mechanism. -The consecutive evidence budget is separate from duplicate-read detection. Distinct reads still consume the turn budget, which closes the failure mode where an agent avoided dedupe simply by moving to a different file/search on every step. A read denied solely by the hard evidence budget is not committed to the read-fingerprint cache because that tool never ran. +The consecutive evidence budget is separate from duplicate/coverage detection. Distinct reads still consume the turn budget, closing the failure mode where an agent avoids dedupe simply by moving to a different file/search on every step. A read denied solely by the hard evidence budget is not committed to the read-fingerprint cache because that tool never ran. Unknown tools are not denied by economy classification and do not reset the watchdog merely by completing. This avoids both unsafe bypass and false blocking when a future host/tool is not yet classified. +## Workflow convergence controller + +Runtime interruption is only one layer. Routed GitHub Delivery work also runs under a persistent workflow controller (`scripts/lib/delivery-workflow-controller.mjs`) with an explicit graph and checkpointed state. + +Measurable controller progress is limited to: + +- legal phase advancement; +- relevant state change; +- blocker removal; +- production of required missing evidence; +- completion of required execution. + +Changing narration does not count. + +Default controller budgets are: + +- no-progress warning after **2** cycles; +- restrict further evidence after **3** no-progress cycles; +- interrupt/escalate after **4** no-progress cycles; +- maximum **3** phase retries; +- maximum **80** workflow steps; +- maximum **30** evidence actions; +- maximum **12,000** tokens per phase; +- maximum **50,000** tokens per workflow; +- maximum **30 minutes** wall time. + +The selected route is locked to its declared graph. Workflow profiles and `workflow-brief.mjs` resolve the route/policy packet once; conditional policy is loaded only when its observable condition becomes true. Checkpoints preserve completed phases, refs, blockers, attempts, usage and evidence state so an interrupt/resume does not automatically restart preflight and reread everything. + +## Reliability replay gate + +`scripts/lib/codex-watchdog-replay.mjs` replays sanitized App Server events deterministically without storing generated text in telemetry output. `npm run reliability:gate` is release-blocking and covers: + +- the real Baseline-is-green / tool-emission stall; +- malformed `` protocol emission; +- cross-channel narration/tool-intent loops; +- repeated filtered reads of one GitHub Actions run; +- long unique no-progress generation; +- cumulative output-token exhaustion; +- legitimate tool-rich investigations with real progress; +- legitimate long completed-plan final verdicts. + +A real incident should become a replay fixture instead of remaining an anecdotal transcript. + ## Output economy -Oversized tool output is reduced deterministically to a bounded head/tail plus unique failure-signalling lines. The result records original and omitted character counts. +The generic lifecycle hook does **not** truncate or replace successful `PostToolUse` output. That behavior was removed because a compacted replacement can cause the model to believe the evidence is missing and immediately re-read the same resource through another command. -Compaction is never positive evidence. If omitted content is required to diagnose ambiguity or failure, retrieve the focused missing evidence or the full raw output as the final escalation step. +Output economy is therefore source-owned: prefer authoritative helpers that emit the compact decision/evidence needed for the current phase, progressively escalate from status → failing component → focused excerpt → full raw output only when required, and keep omission explicit. Compaction is never positive evidence and cannot convert omitted or unknown content into success. diff --git a/references/policy-kernel.md b/references/policy-kernel.md index b7aa9d2..922e316 100644 --- a/references/policy-kernel.md +++ b/references/policy-kernel.md @@ -46,4 +46,4 @@ Execute deterministic tool calls without narrating each one. User-facing progres Prefer the highest-level authoritative helper or aggregate read that can decide the current step. Reuse one valid state snapshot while relevant state is unchanged. Do not delegate deterministic script or gate interpretation to a subagent. Escalate evidence progressively: decision/status, failing component, focused excerpt, then full raw output only when required. -When the host exposes the GitHub Delivery progress watchdog, use it to interrupt no-progress narration, deduplicate unchanged reads, rate-limit polling, and compact oversized model-facing tool output. The watchdog never grants mutation authority, executes writes, or converts omitted or unknown evidence into success. +When the host exposes the GitHub Delivery progress watchdog, use it to interrupt no-progress generation, deduplicate unchanged or semantically covered evidence, rate-limit polling, and enforce bounded generated-character/output-token budgets when the runtime exposes those signals. Successful tool results must remain intact; compact evidence at the authoritative source/helper instead of replacing a completed `PostToolUse` result. The watchdog never grants mutation authority, executes writes, or converts omitted or unknown evidence into success. diff --git a/references/runtime-capabilities.md b/references/runtime-capabilities.md index 0e1c4f3..794ee75 100644 --- a/references/runtime-capabilities.md +++ b/references/runtime-capabilities.md @@ -41,13 +41,13 @@ SHIPPING_GITHUB_PROGRESS_WATCHDOG=none|hooks|stream `CONNECTOR_WRITE` means the host connector has write permission. `BROKERED_CONNECTOR_WRITE` means the github-delivery mutation broker has an adapter that enforces the same request, expected-head, idempotency, exact-text, audit, and verification contract through that connector. Permission without an adapter is not a usable mutation path. -For the progress watchdog, an explicit environment declaration is authoritative for the current controlled runtime. The protected Codex launcher sets `SHIPPING_GITHUB_PROGRESS_WATCHDOG=stream` inside the App Server/client process tree so current-session capability discovery does not depend on a machine-wide guess. +For the progress watchdog, an explicit environment declaration is authoritative for the current controlled runtime. The protected Codex launcher sets `SHIPPING_GITHUB_PROGRESS_WATCHDOG=stream` inside the App Server/client process tree so current-session capability discovery does not depend on a machine-wide guess. A verified current-process `stream` declaration also supersedes stale activation-receipt degradation metadata: a live protected session reports `progressWatchdog: "stream"` with no inherited `streaming_interruption_unavailable` reason from an earlier hooks-only activation state. When no explicit runtime declaration exists, discovery reads `~/.codex/github-delivery/watchdog-activation.json` (or the equivalent under `CODEX_HOME`). Invalid or missing activation state never upgrades capability. A fresh Codex hook configuration is persisted as `none` with `hook_trust_required` until the expected unchanged non-managed hook definition has been explicitly confirmed trusted; file presence alone is not interpreted as active hooks. -`hooks` means lifecycle-hook enforcement was explicitly verified for the expected definition. It provides per-turn supported-tool guardrails, including duplicate-read protection and bounded evidence exploration, but it does not claim hard interruption before a local tool boundary or coverage for hosted tools that bypass local hooks. +`hooks` means lifecycle-hook enforcement was explicitly verified for the expected definition. It provides per-turn supported-tool guardrails, including duplicate-read protection, semantic evidence reuse where supported, and bounded evidence exploration, but it does not claim hard interruption before a local tool boundary or coverage for hosted tools that bypass local hooks. -`stream` means the current process or host owns the protected App Server launch boundary capable of interrupting an in-flight no-progress turn. The protected launcher monitors that contract at runtime and fails closed if required notification visibility is disabled/lost, the watchdog router fails, or a private `turn/interrupt` errors or is not acknowledged. A process must not continue to describe itself as protected `stream` after that enforcement boundary fails. +`stream` means the current process or host owns the protected App Server launch boundary capable of interrupting an in-flight no-progress turn. The protected launcher observes required generated-text channels plus plan/diff/token progress signals, monitors that contract at runtime, and fails closed if required notification visibility is disabled/lost, the watchdog router fails, or a private `turn/interrupt` errors or is not acknowledged. A process must not continue to describe itself as protected `stream` after that enforcement boundary fails. `none` means policy-only protection for that capability snapshot. See `references/agent-progress-watchdog.md` for the different guarantees. @@ -107,7 +107,8 @@ When no explicit runtime declaration exists, discovery reads `~/.codex/github-de - `runtime.progressWatchdog` is `stream`, `hooks`, or `none`; the corresponding `contextEconomy` fallback is `streaming-watchdog`, `lifecycle-hooks`, or `policy-only`. - `hooks` is turn-scoped for supported local hook paths. Its evidence budget does not imply hosted WebSearch coverage or mid-generation interruption. - `stream` is a current controlled-runtime claim, not a promise that an ordinary `codex`/IDE launch is automatically protected. Runtime loss of the required stream contract is fatal to the protected launcher rather than a silent downgrade. -- `progressWatchdogDegradationReason` can expose `hook_trust_required`, `streaming_interruption_unavailable`, or another concrete activation degradation instead of letting the workflow assume stronger enforcement. +- A current verified `stream` declaration clears stale machine-wide activation degradation for that capability snapshot; `progressWatchdogDegradationReason` remains reserved for degradation that actually applies to the effective current mode. +- `progressWatchdogDegradationReason` can expose `hook_trust_required`, `streaming_interruption_unavailable`, or another concrete activation degradation when that reason applies instead of letting the workflow assume stronger enforcement. - `progress_watchdog_unavailable` is included in `degraded` when no runtime watchdog is active. A more specific activation reason can still be present in `runtime.progressWatchdogDegradationReason`. - Missing ruleset or review-thread evidence is degraded capability and must flow into an unknown gate result rather than being guessed.