Skip to content

fix(ci): exempt draft PRs from the opencode-review current-head verdict gate - #1443

Open
seonghobae wants to merge 4 commits into
mainfrom
claude/opencode-review-draft-gate-fix
Open

fix(ci): exempt draft PRs from the opencode-review current-head verdict gate#1443
seonghobae wants to merge 4 commits into
mainfrom
claude/opencode-review-draft-gate-fix

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

The bug

.github/workflows/opencode-review.yml's required check job (opencode-review-target, displayed as the opencode-review check) triggers on pull_request_target: types: [opened, synchronize, reopened, ready_for_review, closed]. Its one step, "Fail closed without a current-head OpenCode verdict", unconditionally demands an APPROVED/CHANGES_REQUESTED review from opencode-agent on the current head, with no draft handling:

if [ "${{ github.event.action }}" = "closed" ]; then
  echo "PR closed; a current-head OpenCode verdict is not required."
  exit 0
fi
if [ -z "${PR_NUMBER:-}" ] || [ -z "${HEAD_SHA:-}" ]; then
  echo "::error::Missing PR number or head SHA; cannot verify a current-head OpenCode verdict."
  exit 1
fi
reviews="$(gh api --paginate "repos/${TARGET_REPOSITORY}/pulls/${PR_NUMBER}/reviews")"
verdict="$(... jq query for an APPROVED/CHANGES_REQUESTED review from opencode-agent on the exact HEAD_SHA ...)"
if [ -z "$verdict" ]; then
  echo "::error::No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. ..."
  exit 1
fi

Meanwhile scripts/ci/pr_review_merge_scheduler.py — the thing that actually issues the repository_dispatch event that would eventually cause opencode-agent to post a verdict — deliberately never requests a review for a draft PR (line 2401-2402):

if pr.get("isDraft"):
    return Decision(number, "skip", "draft PR")

Net effect: every draft PR in the org showed this required check as a hard exit 1 failure on every single push, forever, until marked ready for review — not a transient/pending state, an active failure with a scary-looking error message every time. This was independently observed recurring identically on two different draft PRs across two different repos (ContextualWisdomLab/.github#1437 and ContextualWisdomLab/contextual-orchestrator#922).

The fix

Add a draft-aware early-exit to the same step, mirroring the existing closed early-exit exactly in style and placement (right after it, before the PR_NUMBER/HEAD_SHA check):

if [ "${{ github.event.pull_request.draft }}" = "true" ]; then
  echo "PR is a draft; a current-head OpenCode verdict is not required until it is marked ready for review."
  exit 0
fi

This preserves the org's own stated design principle (9550c18: "a required-workflow check must never depend on event payload fields to materialize") — that principle governs the required-workflow-bootstrap job's materialization (confirmed via scripts/ci/test_strix_quick_gate.sh's job-scoped if: scan), which is unaffected here. The opencode-review-target job itself still always runs unconditionally and always reports a status; only its internal bash logic decides pass/fail, exactly like the pre-existing closed branch.

Why this is safe / doesn't weaken the real gate:

  • Once a PR is marked ready for review, ready_for_review and subsequent synchronize events carry draft: false, so the check goes back to genuinely requiring a current-head verdict, exactly as today.
  • A draft PR still cannot be merged via GitHub's own draft mechanism regardless of this check's status, so this closes a false-alarm gap without opening a real merge-bypass gap.
  • Searched for duplicates of this pattern (same jq query text / same "No APPROVED or CHANGES_REQUESTED from opencode-agent" string) across every workflow in the repo: this is the only occurrence. opencode-review-dispatch.yml has an unrelated job also named opencode-review-target (the privileged reviewer itself, repository_dispatch-triggered) with no equivalent unconditional-demand pattern. noema-review.yml is a different shape entirely — it performs the review itself rather than demanding a pre-existing verdict — so it isn't affected by this bug.

Tests

Added shell-level regression coverage in tests/test_opencode_required_verdict_regression.py that extracts the production step's literal bash body from the YAML (mirroring the existing _extract_run_block pattern already used in tests/test_opencode_workflow_shell_syntax.py), substitutes the two inline ${{ github.* }} expressions the way GitHub Actions would, and executes it directly against fake gh binaries:

  • draft PR short-circuits before any Reviews API call is ever attempted (a "refuse to be invoked" fake gh proves this), across opened/synchronize/reopened
  • closed still takes precedence over draft (ordering regression guard)
  • a non-draft, ready_for_review PR with a matching current-head APPROVED review still passes through the real gate unchanged
  • a non-draft PR with no matching review still fails closed exactly as before

Full local check suite:

coverage run -m pytest tests -q   # 1889 passed, 1 skipped (pre-existing, unrelated: missing LLVM 19 toolchain), 21 subtests passed
interrogate                        # 100.0% (pass)

Note: coverage report --show-missing shows 99% (one line in scripts/ci/pingora_edge_policy.py's changed-file pagination fallback, untouched by this PR) — confirmed byte-for-byte identical and pre-existing on a clean main checkout before any of this PR's changes, via git stash/re-run. Not introduced or worsened by this PR; flagging for visibility rather than silently masking it.

Scope

This is a narrow, self-contained fix to one required-check step. No changes to docs/pr-review-and-merge-procedure.md or PR_GOVERNANCE_AUDIT.md — neither currently describes this check's draft behavior, so neither was rendered inaccurate by this change.

Opening as draft per repo governance — OpenCode-approval + scheduler review/merge applies here same as any other PR.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4


Generated by Claude Code

…ct gate

opencode-review.yml's required opencode-review-target check ("Fail closed
without a current-head OpenCode verdict") unconditionally demanded an
APPROVED/CHANGES_REQUESTED review from opencode-agent on the current head for
every opened/synchronize/reopened event, with no draft handling. Meanwhile
scripts/ci/pr_review_merge_scheduler.py deliberately never dispatches an
OpenCode review request for a draft PR:

    if pr.get("isDraft"):
        return Decision(number, "skip", "draft PR")

Net effect: every draft PR showed this required check as a hard exit-1
failure on every push, forever, until marked ready for review -- a permanent
false alarm, not a transient/pending state.

Add a github.event.pull_request.draft early-exit mirroring the existing
closed early-exit exactly in style and placement (right after it, before the
PR_NUMBER/HEAD_SHA check). The job still always runs and always reports a
status -- it just reports success instead of a misleading failure for a
state where a verdict was never going to be requested. Once the PR is marked
ready for review, ready_for_review and subsequent synchronize events carry
draft: false, so the real gate applies unchanged.

Adds shell-level regression coverage in
tests/test_opencode_required_verdict_regression.py that executes the
production step body directly (draft short-circuits before any Reviews API
call, closed still takes precedence over draft, and non-draft PRs still
genuinely require a verdict).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 31 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7ad4b5b7-622a-402f-ab88-32053c211130

📥 Commits

Reviewing files that changed from the base of the PR and between 1d8e872 and f4bf13a.

📒 Files selected for processing (5)
  • .github/workflows/opencode-review.yml
  • CHANGELOG.md
  • scripts/ci/test_strix_quick_gate.sh
  • tests/test_opencode_required_verdict_regression.py
  • tests/test_required_workflow_queue_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 30, 2026 12:04
devin-ai-integration[bot]

This comment was marked as resolved.

claude and others added 2 commits August 30, 2026 12:22
A ready PR converted back to draft with no new commit never fired the
required-workflow gate again (converted_to_draft wasn't in its trigger
list), so a previously failed opencode-review check stayed failed
forever even though the existing draft exemption would have passed it.

Add converted_to_draft to opencode-review.yml's pull_request_target
types, and update the matching contract/regression tests.

Found by Devin's automated review on PR #1443.

Copy link
Copy Markdown
Contributor Author

Status on the failing opencode-review check (run 33310503217): it failed because no opencode-agent review existed yet for this head SHA — this is a pipeline-timing gap, not a bug in this PR's diff. This PR was just converted from draft to ready-for-review; on a cross-repo target ready_for_review triggers the required-workflow gate check immediately, but the actual OpenCode review dispatch is scheduled by pr_review_merge_scheduler.py (immediately for same-repo .github PRs like this one, otherwise via its 15-minute org-wide sweep), so there's a normal window where the gate has nothing to check yet.

Separately, pushed 644255a to fix the real bug Devin found on this PR (draft-reconversion trigger gap) — see the resolved review thread. That push will itself trigger a fresh required-workflow run and review dispatch for the new head.


Generated by Claude Code

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Devin Review

Copy link
Copy Markdown
Contributor Author

Fresh operational contradiction: explicit draft review-only request is discarded

Protected main@1d8e872487838e16a003e96e76df9300c388e258 still violates the draft review-only contract at the scheduler boundary.

On Draft PR #1450 at unchanged exact head 7779b20fd0f525c87baa6ccc156ee79c607fa9a1, a targeted @opencode-agent review request explicitly required review-only execution and prohibited branch/merge-state mutation.

Fresh exact evidence:

Current #1443 source makes the required status green for a Draft, but leaves pr_review_merge_scheduler.py's unconditional draft skip intact. That converts an explicit review-only request into no review at all. It is not acceptance for the required contract that Draft review dispatch may run while lifecycle/branch mutations remain disabled.

Please extend the TDD boundary so:

  1. an explicit targeted/mention review request for an open Draft dispatches the exact-current-head OpenCode review;
  2. the same Draft path is structurally review-only and cannot update refs, merge, enable auto-merge, or otherwise change lifecycle state;
  3. ordinary merge-queue sweeps may still skip Drafts;
  4. non-Draft verdict gating remains fail-closed; and
  5. a live post-integration Draft canary produces a formal exact-head review without leaving Draft.

The addressed converted_to_draft thread is separately resolved; this is a distinct current operational defect.

Copy link
Copy Markdown
Contributor Author

Confirmed — this is a distinct defect from the one this PR fixes. Filed and fixed in #1456: inspect_pr() returned skip: draft PR unconditionally, before ever reaching the dispatch logic, so agent-mention-opencode-dispatch.yml's already-structurally-review-only forward (it already hardcodes enable_auto_merge=false, update_branches=false, merge_mode=disabled) was silently discarded for drafts.

New opt-in --allow-draft-review-dispatch (requires --pr-number, so it can never apply to the multi-PR queue sweep) routes a draft PR through a new dispatch_draft_review_only() helper that runs the same Strix-then-OpenCode gate the ready-PR pipeline uses, then returns before any merge/branch-update/auto-merge logic — gated in the workflow by a new ALLOW_DRAFT_REVIEW_DISPATCH env var keyed on client_payload.agent_invocation_key, a field only the mention-dispatch workflow ever sets. Full test suite, 100% coverage/docstrings on the changed script, and a dedicated regression test proving the review-only path never calls any merge/branch/auto-merge function. See #1456 for the itemized response to all 5 requirements.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants