Skip to content

feat(coding): inspect and retry Depot CI through the edge - #2413

Merged
justinhelmer merged 1 commit into
mainfrom
plan/fix-issue-2250-so-coding-678140/u1
Sep 26, 2026
Merged

justinhelmer merged 1 commit into
mainfrom
plan/fix-issue-2250-so-coding-678140/u1

Conversation

@coreplane-switchboard

@coreplane-switchboard coreplane-switchboard Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Coding children can inspect failed Depot workflows, read their logs, and request a failed-only retry for their bound repository. The Depot organization token stays in the Cloudflare Worker edge.

Why: Issue #2250 records repeated CI failures that children could not diagnose because Depot logs needed separate authentication. People had to relay failures by hand. The new tools give the coding run the evidence and one narrow retry operation.

Where to look

  1. Worker authorization The edge consumes a one-use run and repository permit before fixed read or failed-only retry calls.
  2. Log handling Job and attempt membership are checked; whole-capture redaction withholds ambiguous split secrets before logs reach the model.
  3. Bot bridge The bot binds the run and repository, rechecks access per call, and issues a one-use permit while holding only an internal bearer.
  4. Coding tools Inspection, logs, and failed-only retry accept typed inputs; repository, token, upstream URL, and arbitrary RPC are not model inputs.
  5. Secret boundary The organization token is Worker-only; only the bridge bearer is forwarded to the bot.
  6. Behavior and live proof The spec defines repo binding, retry semantics, bounded logs, and deployment-gated acceptance.

Feedback wanted: Check one-use permit consumption, credential placement, cross-page redaction, malformed retry acknowledgements, and failed-only retry. Confirm the small shared tool/context edits stay compatible with draft #1625.

Risk: Depot CI API and new optional secrets add a deployment dependency. Without both secrets, tools fail closed. A live child diagnosis and retry cannot be credited until a reviewed release is deployed and configured.

Verified: The new upstream-acknowledgement regression failed before the fix and passes now. The Cloudflare Worker gate passes (75 tests and typecheck); root consistency, typecheck, lint, and format passed. The full local suite had one unrelated macOS chown justin:justin failure (12,640 passed). Current-head CI is green. Independent review run bfa746ad approved exact head e774944 with no code findings. Live acceptance remains open.

Decisions (3)
  • Hold the Depot token at the Worker edge. Depot CI needs an organization credential with broad access; the model shell and bot container receive none of it.
  • Use one-use run permits. A shared bridge bearer cannot choose a repository or operation; the edge obtains a consumed, exact-run permit from its fixed container binding.
  • Fence retry to the observed head. A failed or cancelled workflow must belong to the bound repository and match the full expected head before the edge calls Depot's failed-only operation.
Validation (4 criteria)
Criterion Proof
The edge requires a consumed run and repository permit before returning metadata, logs, or retrying deploy/cloudflare/depotCi.test.ts::Depot CI edge::*
A coding child receives repo-bound evidence through its tool context src/core/dispatch/runLoop.test.ts::runLoop — the model turn and everything that rides on it::a coding child reads Depot failure evidence through its repo-bound Worker bridge
The organization token is not forwarded to the bot container src/core/secretsManifest.test.ts::deploy/secrets.manifest.json::Depot credentials stay at the edge and only the bridge bearer reaches the bot
A real child diagnoses a failed Depot check and retries only failed work Deployment-gated live procedure in docs/reference/specs/depot-ci.md; no live receipt yet
For agents

Requested in the Ship thread. The source is one #2250 unit. Draft #1625 has limited overlap in src/tools/toolsets.ts and src/tools/runnableTool.ts; do not take over that draft. Merge, release, secret provisioning, and live proof remain separate gates.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: The edge lacks independent run/repository authorization, page-boundary redaction leaks secret fragments, and the required live acceptance receipt is outstanding.

Warning

Changes requested · head ba7620e · 3 findings: 1 major, 2 minor

Severity Finding Where
major F1 Bind the edge request to an authenticated run and repository deploy/cloudflare/depotCi.ts:69
minor F2 Spec contradiction — depot-ci.md item 3: page-local redaction exposes split credentials deploy/cloudflare/depotCi.ts:148
minor F3 Provide the live failed-check diagnosis and failed-only retry receipt docs/reference/specs/depot-ci.md:32
Full review

F1 (major, high confidence): The edge authenticates a shared bearer, then trusts the caller-supplied repo; it never verifies which run or repository the caller represents. A holder of that bearer can name any repository accessible to the organization token, inspect its workflows and request failed-only retries. Bind each request to an edge-verifiable run and repository before using the Depot credential.

F2 (minor, medium confidence): Log redaction runs separately on each page. If a credential is split across two pages, neither fragment matches the configured credential, and both can reach the returned tail. Redact across page boundaries before returning evidence, as depot-ci.md item 3 promises.

F3 (minor, high confidence): The required live acceptance receipt is still missing. Fixture tests cannot show that a coding child can diagnose a real failed Depot check and retry only failed jobs. This remains credential- and deployment-gated; record the child’s actual check, log diagnosis and retry outcome before claiming live proof.

@coreplane-switchboard
coreplane-switchboard Bot force-pushed the plan/fix-issue-2250-so-coding-678140/u1 branch from ba7620e to 45e98a5 Compare September 26, 2026 04:02

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: Run/repo binding and log-redaction fixes landed, but live proof is still pending and malformed successful retry responses are accepted.

Warning

Changes requested · head 45e98a5 · 2 findings: 2 minor

Severity Finding Where
minor F3 Provide the live failed-check diagnosis and failed-only retry receipt docs/reference/specs/depot-ci.md:35
minor F4 Treat malformed 200 retry responses as unknown outcomes src/execution/depotCi.ts:102
Full review

F3 (minor, high confidence): The live acceptance criterion remains unproven, as the spec acknowledges. Before claiming this capability works for coding children, record a deployed child diagnosing a real failed Depot check from returned logs and observe its failed-only retry. This requires edge provisioning and a human-run procedure; fixture tests cannot supply the receipt.

F4 (minor, high confidence): WorkerDepotCi.call accepts any non-null object from a 200 bridge response as a successful retry. A malformed response such as {} is therefore reported as success even though the mutation outcome is unknown, contrary to depot-ci.md item 4. Validate the retry response shape and workflow ID before returning it; otherwise report an unknown outcome.

@coreplane-switchboard
coreplane-switchboard Bot force-pushed the plan/fix-issue-2250-so-coding-678140/u1 branch from 45e98a5 to c4c8085 Compare September 26, 2026 04:13
@coreplane-switchboard

Copy link
Copy Markdown
Contributor Author

Please re-review exact head c4c8085d64211abaf0ce784e972fb2cf7c38db0c for the existing #2250 Ship unit.

  • F4 fixed: malformed HTTP 200 retry acknowledgements now report unknown outcome; the bot validates the complete response, workflow identity and job count/IDs, without replay. Fifteen new cases failed before the fix; all 151 focused tests now pass after rebase.
  • F3 remains explicitly human-gated and unproven, as requested. The deployed-child diagnosis and observed failed-only retry receipt cannot be replaced by fixture tests; no credential provisioning or deployment was performed.

Dispositions and the current-head PR description have been submitted. Current-head CI and independent review remain gates. No merge, approval, new Ship task or changes to the Nominal PRs.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: Malformed bridge acknowledgements are now handled safely, but live proof remains unproduced and its spec row still claims an agent proof.

Warning

Changes requested · head c4c8085 · 2 findings: 2 minor

Severity Finding Where
minor F3 Provide the live failed-check diagnosis and failed-only retry receipt (human-gated) docs/reference/specs/depot-ci.md:36
minor F5 Mark unproven live acceptance as a [gap] rather than an [agent] proof docs/reference/specs/depot-ci.md:36
Full review

F3 (minor, high confidence) — The live receipt is still missing. Fixture tests do not show a coding child diagnosing a real failed Depot check and observing a failed-only retry. Keep issue #2250 and the release acceptance gate open until a person can run the documented procedure after deployment.

F5 (minor, high confidence) — The same unproven criterion is still labeled [agent] at docs/reference/specs/depot-ci.md:36. Change it to [gap] with the existing live procedure so the spec does not present an outstanding, human-gated proof as satisfied.

F4 is fixed at this head: a malformed HTTP 200 bridge retry acknowledgement now produces an unknown outcome without replay.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: Retry response normalization can turn a malformed upstream acknowledgement into apparent success.

Warning

Changes requested · head 749f320 · 1 finding: 1 major

Severity Finding Where
major F1 Reject incomplete upstream RetryFailedJobs acknowledgements before returning success deploy/cloudflare/depotCi.ts:49
Full review

F1 (major, high confidence) — deploy/cloudflare/depotCi.ts:49: If Depot returns HTTP 200 with a workflowId but omits jobIds and jobCount, the defaults turn that incomplete mutation response into {jobIds: [], jobCount: 0}. The bot then accepts it as a successful retry, despite the response not establishing what happened. Require those fields in the upstream acknowledgement and report an unknown outcome when either is missing; do not replay the retry.

Co-Authored-By: coreplane-switchboard[bot] <318072483+coreplane-switchboard[bot]@users.noreply.github.com>
@justinhelmer
justinhelmer force-pushed the plan/fix-issue-2250-so-coding-678140/u1 branch from 749f320 to e774944 Compare September 26, 2026 14:59

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: Prior retry-acknowledgement defect is fixed; deployed-child acceptance remains unproven and human-gated.

Warning

Changes requested · head e774944 · 1 finding: 1 major

Severity Finding Where
major F3 Real-child Depot CI diagnosis and failed-only retry remain unproven docs/reference/specs/depot-ci.md:37
Full review

F3 (major, high confidence): The spec and current PR body correctly leave live acceptance open. The fixture tests do not prove that a deployed coding child can diagnose a real failed Depot check or that a retry leaves successful jobs untouched. Keep this human-gated until the post-deploy procedure records the child’s log diagnosis and observed failed-only retry. The prior F1 is resolved: the Worker now requires both jobIds and jobCount, and the regression covers missing fields without replay.

@justinhelmer

Copy link
Copy Markdown
Contributor

F3 disposition: declined as a pre-merge code finding; retained as a human-gated release and live-acceptance requirement. The spec labels the real-child check [gap], and issue #2250 remains open. After human merge, release, Worker-only secret provisioning, and deployment, the documented procedure must record the child's exact failed check and head, diagnosis from returned logs, and observed failed-only retry with successful jobs unchanged. Fixture tests are not live proof. This PR cannot produce that deployed-child receipt before those gates. Please judge code merge readiness at head e77494436fccbf475afa8dfda905fcb9c8138ba1 while keeping F3 open as post-deploy acceptance.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM: Code is merge-ready; the real-child Depot receipt remains an explicit post-deploy acceptance gap in #2250, not a pre-merge code finding.

Note

Approved · head e774944 · no findings

Full review

The current code is reviewable and safe to merge. F3 remains an explicitly unproven [gap] and a post-deploy acceptance requirement in #2250; this review does not claim a real-child Depot receipt.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: coreplane-switchboard[bot] reviewed this PR and posted an LGTM verdict (see its review). This repository opted in through its REVIEW_BOT_LOGIN and REVIEW_BOT_ID variables.

@justinhelmer
justinhelmer merged commit a941941 into main Sep 26, 2026
31 checks passed
@justinhelmer
justinhelmer deleted the plan/fix-issue-2250-so-coding-678140/u1 branch September 26, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant