Skip to content

fix(ci): give the opencode-review required check a workflow_run second chance - #1494

Closed
seonghobae wants to merge 1 commit into
mainfrom
fix/opencode-review-workflow-run-second-chance
Closed

fix(ci): give the opencode-review required check a workflow_run second chance#1494
seonghobae wants to merge 1 commit into
mainfrom
fix/opencode-review-workflow-run-second-chance

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Summary

Closes the gap documented (not fixed) in #1485: opencode-review.yml's opencode-review-target job — the opencode-review required check — evaluates synchronously inside its own pull_request_target run, seconds after push, by querying whether an opencode-agent review already exists for the current head. The real review is posted much later by the separate opencode-review-dispatch.yml repository_dispatch path (an LLM review, sometimes over an hour per docs/product-goal-directive.md), and nothing re-ran the already-failed check once it landed. Confirmed live and 100% reproducible this session on 6+ independent PRs across .github and contextual-orchestrator.

Direction chosen, and why it isn't a literal copy of noema-review.yml

#1485 recorded two candidate directions. I chose direction 1 (workflow_run re-entry) — but traced the actual execution model before wiring it, rather than mirroring noema-review.yml's exact source-workflow list, because a literal copy would silently do nothing for most of the affected PRs:

  • opencode-review.yml is distributed by the org's required-workflow ruleset (18156473) into every target repository — it runs as contextual-orchestrator, naruon, etc., not only as .github.
  • opencode-review-dispatch.yml (the workflow that actually posts the review) is deliberately default-branch-only and always runs inside ContextualWisdomLab/.github — it is not part of the required-workflow ruleset's distributed set (docs/org-required-workflow-rollout.md's active path list has 7 entries; the dispatch workflow isn't one of them). workflow_run cannot cross repositories, so a sibling repository's copy of opencode-review.yml could never observe that completion. Listening to "OpenCode Review Dispatch" — the literal analog of noema's own list — would fix .github's own PRs only and leave every sibling repo (the majority of the affected PRs, including contextual-orchestrator#955/#956) exactly as broken as today.
  • "Required PR Review Merge Scheduler" (pr-review-merge-scheduler.yml) is ruleset-distributed into every target repository, and already reacts to pull_request_review: [submitted, dismissed] — the exact event GitHub fires the instant the real opencode-agent review posts — plus its own periodic sweep (*/15/*/30 * * * *). Listening to its completion gives opencode-review a same-repository second chance, without needing pull_request_review as a direct trigger (not in the required-workflow ruleset's supported trigger set — pull_request, pull_request_target, push, workflow_run only) and without any cross-repository credential.

I also checked whether re-running the existing failed job via the Actions Jobs API (direction 2, POST .../actions/jobs/{id}/rerun) could work from opencode-review-dispatch.yml directly. It can't: that workflow always runs as .github, and this codebase's own scheduler_dispatch_env() already documents that the OpenCode app credential has no cross-repository Actions permission, so a cross-repository job-control call has no working credential today. workflow_run re-entry sidesteps that entirely — it needs no privileged credential, just the same read-only github.token the check already used.

Change

Kept as close to the existing pull_request_target shape as possible:

  • on.workflow_run listens for "Required PR Review Merge Scheduler" completions (excluding cancelled, matching noema-review.yml's own exclusion).
  • required-workflow-bootstrap gains if: github.event_name == 'pull_request_target' — it has no meaningful inputs (PR number, head SHA, event action) for a workflow_run event; coverage-source-tree/coverage-evidence cascade-skip with it, no changes needed there.
  • opencode-review-target now runs standalone on a workflow_run event (needs.coverage-evidence.result == 'success' is only required on the pull_request_target path, preserving that path's exact prior behavior byte-for-byte), resolving PR_NUMBER/HEAD_SHA from github.event.workflow_run.pull_requests[0] (GitHub-computed server-side, not attacker-controlled) when github.event.pull_request is absent, with a new early exit for workflow_run events that carry no associated PR (the periodic-sweep case — nothing to verify, not an error).
  • concurrency.group and run-name gain the same PR-number/head-SHA fallback chain for workflow_run events.

Security posture (unchanged)

No checkout, no secrets, no new permissions anywhere in this diff. The verdict step still only performs a read via gh api .../pulls/{PR_NUMBER}/reviews with the same read-only github.token (contents: read, pull-requests: read, unchanged), gated by the exact same opencode-agent/current-head-SHA/non-fallback-body checks as before. This only changes when that exact check gets a chance to re-run — never what it accepts as passing, never what credential it uses, and it never touches PR content.

Trust-boundary analysis specifically for spoofing risk: PR_NUMBER/HEAD_SHA for the workflow_run path come from github.event.workflow_run.pull_requests[0], a field GitHub computes server-side from genuine branch/repo correlation — not something a PR author's branch content, title, or commits can forge. workflow_run cannot cross repositories, so a fork/PR in one repository can never cause this re-entry to fire in a different repository's copy of the workflow. The chain pr-review-merge-scheduler → opencode-review → pr-review-merge-scheduler is bounded by GitHub's own 3-level workflow_run chain-depth cap (no runaway loop possible).

Contract tests

tests/ pins exact strings/structure of this file (test_opencode_required_verdict_regression.py, test_pingora_edge_workflow_contract.py, test_required_workflow_queue_contract.py, test_opencode_agent_contract.py, test_opencode_security_boundaries.py, test_central_required_workflow_ruleset_audit.py) — read all of them before editing; none needed weakening. Extended test_opencode_required_verdict_regression.py with:

  • Static assertions locking in the workflow_run trigger source, the #1485 rationale comment, the bootstrap job's pull_request_target-only gate, and the target job's workflow_run re-entry condition.
  • Three executable bash runs of the actual verdict step (not a Python mirror, matching this file's existing "one executable owner" discipline): a workflow_run event with no associated PR exits 0 cleanly without ever invoking gh; a workflow_run re-entry with a real review now present passes and prints the verdict; the original pull_request_target fail-closed behavior (empty reviews → exit 1) is unchanged.

Verification

  • python3 -c "import yaml; yaml.safe_load(open('.github/workflows/opencode-review.yml'))" — OK
  • bash -n on the extracted, GitHub-expression-rendered verdict step — OK
  • PYTHONPATH=. coverage run -m pytest tests -q && coverage report --show-missing2123 passed, 1 skipped, 21 subtests, coverage 100% (scripts/ci)
  • interrogate100% (this fix is workflow/test-only; confirms the docstring gate wasn't disturbed)

Expected first-run behavior on this PR itself

Because the fix isn't live on main yet when this PR's own opencode-review check first evaluates, this PR is very likely to hit the exact same race on its own first run — that's expected, not a sign the fix is wrong (the base branch's current opencode-review.yml, not this PR's, is what actually executes for a pull_request_target-triggered required check). Once merged, opencode-review-dispatch.yml posting the real review will trigger pr-review-merge-scheduler.yml (via pull_request_review: submitted), which will in turn re-trigger opencode-review.yml here on the next PR that needs it.

Other PRs

#1492 (.github) and contextual-orchestrator#953, #955, #956 (already documented in #1485, plus #1479/#1482/#1437/#1478) hit this exact same race this session and already carry "not this PR's fault, see #1485" comments. Once this fix is merged and the base branch picks it up, those PRs' opencode-review check should self-resolve on its next natural re-evaluation (their own next pull_request_review/scheduler-completion workflow_run, or a manual re-run) — I have not pushed anything to those PRs myself; they're out of scope for this change.

Refs #1485.


Generated by Claude Code

…d chance

opencode-review.yml's opencode-review-target job (the "opencode-review"
required check) evaluates synchronously inside its own pull_request_target
run, seconds after push, by querying whether an opencode-agent review
already exists for the current head. The real review is posted much later
by the separate opencode-review-dispatch.yml repository_dispatch path, and
nothing re-ran the already-failed check once it landed -- documented and
left unfixed in #1485.

Direction chosen: workflow_run re-entry (candidate 1 from #1485), but NOT a
literal copy of noema-review.yml's own workflow_run source list. Tracing the
actual execution model first:

- opencode-review.yml is distributed by the org's required-workflow ruleset
  into every target repository (it runs "as" contextual-orchestrator, etc.,
  not only as .github), so workflow_run re-entry has to work per repository.
- opencode-review-dispatch.yml (the workflow that actually posts the review)
  is deliberately default-branch-only and always runs inside
  ContextualWisdomLab/.github -- it is NOT part of the required-workflow
  ruleset's distributed set. workflow_run cannot cross repositories, so a
  sibling repository's copy of opencode-review.yml could never observe its
  completion. Listening to "OpenCode Review Dispatch" (a literal mirror of
  noema's source list swapped in) would silently do nothing for every
  sibling-repo PR -- the majority of the affected PRs.
- "Required PR Review Merge Scheduler" (pr-review-merge-scheduler.yml) IS
  ruleset-distributed into every target repository, and already reacts to
  pull_request_review: [submitted, dismissed] -- the exact event GitHub
  fires the instant the real opencode-agent review posts -- plus its own
  periodic sweep. Listening to its completion gives this required check a
  same-repository second chance without needing pull_request_review as a
  direct trigger (not supported by the required-workflow ruleset's trigger
  set) and without any cross-repository credential.

Change, kept as close to the existing pull_request_target shape as
possible:
- on.workflow_run listens for "Required PR Review Merge Scheduler"
  completions (excluding cancelled runs, matching noema-review.yml's own
  exclusion).
- required-workflow-bootstrap gains
  `if: github.event_name == 'pull_request_target'`; it has no meaningful
  inputs for a workflow_run event, and coverage-source-tree /
  coverage-evidence cascade-skip with it (no changes needed there).
- opencode-review-target now runs standalone on a workflow_run event
  (needs.coverage-evidence.result == 'success' is only required on the
  pull_request_target path, preserving that path's exact prior behavior),
  resolving PR_NUMBER/HEAD_SHA from
  github.event.workflow_run.pull_requests[0] (GitHub-computed, not
  attacker-controlled) when github.event.pull_request is absent, with a new
  early exit for workflow_run events that carry no associated PR (the
  periodic-sweep case).
- concurrency.group and run-name gain the same PR-number/head-SHA fallback
  chain for workflow_run events.

Security posture, unchanged: no checkout, no secrets, no new permissions.
The verdict step still only performs a read via
`gh api .../pulls/{PR_NUMBER}/reviews` with the existing read-only
github.token, gated by the same opencode-agent/current-head/non-fallback
checks as before -- this only changes when that exact check gets a chance
to re-run, never what it accepts as passing.

New tests in test_opencode_required_verdict_regression.py lock in: the
workflow_run trigger source and its #1485 rationale, the bootstrap job's
pull_request_target-only gate, the target job's workflow_run re-entry
condition, and three executable bash runs of the actual verdict step (no
associated PR exits cleanly without calling gh; a workflow_run re-entry
with a real review now present passes; the original pull_request_target
fail-closed behavior is unchanged).

Full suite: 2123 passed, 1 skipped, 21 subtests. Coverage 100%
(scripts/ci), interrogate 100%.

Refs #1485. Once merged, #1492 and
ContextualWisdomLab/contextual-orchestrator#955/#956 (this session's PRs
that hit this exact race) are expected to self-resolve on their next
opencode-review re-run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

opencode-review failed on this PR's own first run — exactly as predicted in this PR's own body ("this PR is very likely to hit the exact same race on its own first run... that's expected, not a sign the fix is wrong"). The base branch's current opencode-review.yml (not this PR's fixed version) is what actually executes for the pull_request_target trigger, so the fix isn't live yet for its own evaluation. Zero reviews exist yet (get_reviews[]), consistent with the race. Queued one re-run in the meantime; this will self-resolve once merged and the base branch's workflow_run re-entry is live for future PRs (this one included, on a later natural re-evaluation).


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

@opencode-agent please review this draft PR.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Closing as superseded: while resolving this PR's merge conflict against current main to land it (it looked like the single highest-leverage fix for the spurious opencode-review failures on #1500/#1415/etc.), I found that main's 4a5dfd82 (#1497, "require substantive agent verdicts") already solved this exact race — the one #1485 documented and this PR set out to fix — via a completely different, already-merged mechanism: the opencode-review-target job now has a "Request current-head OpenCode review execution" step that actively dispatches a merge-scheduler repository_dispatch event itself, then the verdict step polls synchronously for up to 180 × 30s = 90 minutes waiting for that review to land (with timeout-minutes: 100 bounding the job). That's a fundamentally different, more self-contained design than this PR's passive workflow_run re-entry off pr-review-merge-scheduler.yml's completion.

Concrete proof this PR's premise is now stale: after merging main into this branch, tests/test_verdict_step_pull_request_target_still_fails_closed_without_a_review (this PR's own new test, which fakes gh to return empty reviews and expects the verdict step to fail fast) now hangs for the full 90 minutes, because the script it extracts and executes is main's current active-dispatch-and-poll version, not the simple synchronous check this PR was written against at its a6fc45e9 base. Merging this PR as-is on top of current main would add a workflow_run re-entry path that's functionally near-dead code — the in-job poll almost always resolves the verdict (or exhausts its own 90-minute budget) before any external re-entry could matter.

Not pushing the merge I did locally. The real, still-open question this surfaces — why the actively dispatched review didn't land within 90 minutes on #1500's actual CI run — is a new, different problem from what this PR was fixing, and I'll track that separately.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants