Skip to content

fix: ignore CodeRabbit approval pending notices - #1443

Open
seonghobae wants to merge 22 commits into
developfrom
fix/coderabbit-approval-pending-gate-develop
Open

fix: ignore CodeRabbit approval pending notices#1443
seonghobae wants to merge 22 commits into
developfrom
fix/coderabbit-approval-pending-gate-develop

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • ignore CodeRabbit approval_notice_start comments when classifying substantive blocking issue comments
  • preserve the current-head CodeRabbit evidence wait state
  • add a regression fixture for the approval-pending notice shape so all develop-based PRs share the fix

Validation

  • bash scripts/ci/test_pr_governance_gate.sh

This is the develop-targeted form of the same fix needed by feature PR #1440; the gate previously matched the notice phrase “blocking issues” as a source blocker.


Open in Devin Review

Summary by CodeRabbit

  • Bug Fixes

    • Strengthened merge-gate validation for repository identities and pull request numbers.
    • Prevented malformed or unsafe repository values from triggering GitHub API requests.
    • Improved approval-notice evaluation, including multiline, malformed, whitespace-only, and potential-issue notices.
    • Reduced false positives from generic blocking language.
  • Documentation

    • Updated the security changelog and merge-gate policy.
  • Tests

    • Expanded coverage for approval-pending scenarios, invalid identities, and shell-metacharacter inputs.

@seonghobae
seonghobae enabled auto-merge (squash) August 21, 2026 06:16
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Approval pending

CodeRabbit has no unresolved comments, but it has not reviewed the latest commit.

Use the checkbox below to review the latest commit. CodeRabbit will approve the changes if it finds no blocking issues.

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The governance gate now uses a dedicated blocking pattern for approval notices and an anchored current-head match. Tests cover valid, stale, malformed, mixed-content, warning, failure, and GitHub Code Quality notices.

Changes

Approval notice filtering

Layer / File(s) Summary
Gate filter and blocking rules
scripts/ci/pr_governance_gate.sh
The gate adds a dedicated approval-notice blocking pattern. It matches headCommitId on the same line with optional quotes and counts current approval notices with blocking language as blockers.
Approval notice regression coverage
scripts/ci/test_pr_governance_gate.sh
Fixtures and tests cover multiline, stale-head, mixed-content, current-head warning and failure, malformed, and GitHub Code Quality approval notices.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🟡 Moderate · up to 06327

The gate can still misclassify CodeRabbit approval-pending notices as blocking, while a separate regex can suppress a genuine blocking comment when an unrelated sentence contains the current commit SHA. These concrete classification errors should be fixed or explicitly accepted before merge.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: excluding non-blocking CodeRabbit approval-pending notices from governance checks.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/coderabbit-approval-pending-gate-develop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@OpenCode review current head e5759fd8c1d7c4484159dc99bc5969226a93c0dd. Validate the develop-targeted CodeRabbit approval-pending gate regression.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

The approval-notice exclusion is intentionally scoped to issue comments: CodeRabbit emits approval_notice_start as an issue comment, while review-thread comments are authoritative review findings and must remain counted. Regression fixture coderabbit_approval_pending_comment covers the exact false-positive body; bash scripts/ci/test_pr_governance_gate.sh passes.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

PR governance metadata gate is not ready for 2c227f1924978b1592f72151d529664730145a34:

  • Review decision is CHANGES_REQUESTED; address requested changes before merge.
  • 1 unresolved current review thread(s) remain.
  • Required check strix is FAILURE on the current head.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head disposition for 98a9dae: the github-code-quality observation is a conditional integration concern, not an evidenced source defect. This gate intentionally exempts only a complete current-head CodeRabbit approval notice; github-code-quality[bot] is a separate identity and its approval-shaped marker remains blocking. The exact-head contract test github_code_quality_approval_pending_notice_blocks preserves that fail-closed boundary. No source change is warranted.

@opencode-agent
opencode-agent Bot disabled auto-merge August 21, 2026 10:49
@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head remediation for PR #1443\n\nExact head: \nExact base: \n\nResolved the valid review finding in : structured notices with a current are now ignored for every allowlisted review-bot family, including , while malformed, stale, and substantive blocking comments remain blocking. Added a regression case proving the quality-bot notice waits for review evidence instead of publishing a false blocker.\n\nExact-head verification:\n- test_pr_governance_gate: PASS — PASS\n- — PASS\n- — PASS\n\nPlease regenerate the required OpenCode/Noema and repository checks for this exact head. This is a normal protected-path update; no bypass or force operation is requested.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head remediation for PR #1443

Exact head: d61d316f67e130f951ef8d769c6d148b9bf7b9d0
Exact base: 81c105645ca6e680f5f8c15ba9c33b67eb63c48b

Resolved the valid review finding in scripts/ci/pr_governance_gate.sh: structured approval_notice_start/end notices with a current headCommitId are now ignored for every allowlisted review-bot family, including github-code-quality, while malformed, stale, and substantive blocking comments remain blocking. Added a regression case proving the quality-bot notice waits for review evidence instead of publishing a false blocker.

Exact-head verification:

  • bash scripts/ci/test_pr_governance_gate.sh — PASS
  • shellcheck scripts/ci/pr_governance_gate.sh scripts/ci/test_pr_governance_gate.sh — PASS
  • git diff --check — PASS

Please regenerate the required OpenCode/Noema and repository checks for this exact head. This is a normal protected-path update; no bypass or force operation is requested.

@seonghobae
seonghobae enabled auto-merge (squash) August 21, 2026 11:07
devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head remediation for PR #1443.

Exact head: 049a443aa7010441ab0fe6514cdf2e8ff75420ae
Exact base: 81c105645ca6e680f5f8c15ba9c33b67eb63c48b

Addressed Devin finding: approval notices now recognize headCommitId, the separator, and the current SHA across line breaks, while still requiring both notice boundary markers. Added a regression fixture for a multiline notice; malformed notices remain blocking.

Exact-head verification:

  • bash scripts/ci/test_pr_governance_gate.sh — PASS
  • shellcheck scripts/ci/pr_governance_gate.sh scripts/ci/test_pr_governance_gate.sh — PASS
  • git diff --check — PASS

Please regenerate the required checks and independent reviews for this exact head. Normal protected-path update only; no bypass or force operation is requested.

@OpenCode review current head 049a443. @coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '== checked-out HEAD =='
git rev-parse HEAD
echo '== requested commit =='
git show -s --format='%H%n%P%n%s' 049a443aa7010441ab0fe6514cdf2e8ff75420ae

echo '== exact commit diff =='
git diff --no-ext-diff --unified=80 \
  81c105645ca6e680f5f8c15ba9c33b67eb63c48b \
  049a443aa7010441ab0fe6514cdf2e8ff75420ae -- \
  scripts/ci/pr_governance_gate.sh \
  scripts/ci/test_pr_governance_gate.sh

echo '== governance gate structure =='
ast-grep outline scripts/ci/pr_governance_gate.sh --items all

echo '== relevant approval-notice logic =='
rg -n -C 12 \
  'approval_notice|headCommitId|github-code-quality|allowlist|issue comment|issue_comment' \
  scripts/ci/pr_governance_gate.sh scripts/ci/test_pr_governance_gate.sh

echo '== diff whitespace check =='
git diff --check \
  81c105645ca6e680f5f8c15ba9c33b67eb63c48b \
  049a443aa7010441ab0fe6514cdf2e8ff75420ae

Length of output: 69240


🏁 Script executed:

#!/bin/bash
set -euo pipefail

head_sha='0123456789abcdef0123456789abcdef01234567'
predicate='
  test("approval_notice_start"; "i")
  and test("approval_notice_end"; "i")
  and test("headCommitId[\\s\\S]*:[\\s\\S]*" + $head_sha; "i")
'

valid_multiline=$'<!-- approval_notice_start -->\nheadCommitId:\n0123456789abcdef0123456789abcdef01234567\n<!-- approval_notice_end -->'
stale_with_later_current_sha=$'<!-- approval_notice_start -->\nheadCommitId: old-head\nPotential issue for 0123456789abcdef0123456789abcdef01234567\n<!-- approval_notice_end -->'

printf '%s' "$valid_multiline" | jq -R --arg head_sha "$head_sha" "$predicate"
printf '%s' "$stale_with_later_current_sha" | jq -R --arg head_sha "$head_sha" "$predicate"

Length of output: 911


🏁 Script executed:

#!/bin/bash
set -euo pipefail

head_sha='0123456789abcdef0123456789abcdef01234567'
predicate='
  test("approval_notice_start"; "i")
  and test("approval_notice_end"; "i")
  and test("headCommitId[\\s\\S]*:[\\s\\S]*" + $head_sha; "i")
'

valid_multiline=$'<!-- approval_notice_start -->\nheadCommitId:\n0123456789abcdef0123456789abcdef01234567\n<!-- approval_notice_end -->'
stale_with_later_current_sha=$'<!-- approval_notice_start -->\nheadCommitId: old-head\nPotential issue for 0123456789abcdef0123456789abcdef01234567\n<!-- approval_notice_end -->'

echo '== valid multiline notice =='
printf '%s' "$valid_multiline" | jq -Rs --arg head_sha "$head_sha" "$predicate"
echo '== stale headCommitId with later current SHA =='
printf '%s' "$stale_with_later_current_sha" | jq -Rs --arg head_sha "$head_sha" "$predicate"

Length of output: 1043


@seonghobae I found one blocking issue.

  • Line 444 — The approval-notice match accepts stale headCommitId values. The expression uses unbounded [\s\S]* on both sides of :. It matches a body such as:
    <!-- approval_notice_start -->
    headCommitId: old-head
    Potential issue for 0123456789abcdef0123456789abcdef01234567
    <!-- approval_notice_end -->
    
    The controlled jq -Rs evaluation returned true for this body. The gate then excludes the comment before it evaluates the blocking content. This contradicts the stated fail-closed behavior for stale notices. Bind the value immediately following the headCommitId separator to $head_sha, while allowing only whitespace and optional JSON quoting across line breaks. Add a stale-headCommitId fixture that includes a later current-head SHA and verify that it blocks.

I confirmed the reviewed checkout is 049a443aa7010441ab0fe6514cdf2e8ff75420ae. I did not run repository scripts in this sandbox.

You are interacting with an AI system.

coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head remediation evidence for commit a54a7e0fc06f4ebb5e0ee4ee62089f059c9253a3 against base 81c105645ca6e680f5f8c15ba9c33b67eb63c48b:

  • Narrowed the approval-notice exemption to an immediate headCommitId field value for the exact current SHA; stale identifiers followed by unrelated prose remain blocking.
  • Added regression coverage for multiline exact-head notices and stale-head/unrelated-current-SHA notices.
  • bash scripts/ci/test_pr_governance_gate.sh: PASS
  • shellcheck scripts/ci/pr_governance_gate.sh scripts/ci/test_pr_governance_gate.sh: PASS
  • git diff --check: PASS

The prior exact-head findings are addressed. Please regenerate independent review for the current head; no bypass or force push is being used.

@OpenCode review current head a54a7e0
@coderabbitai review

@seonghobae
seonghobae dismissed coderabbitai[bot]’s stale review August 22, 2026 07:47

Stale review: all review-thread comments on this PR are resolved and the reviewer's cited commit predates the current head, which passes all non-metadata-gate required checks (verified via gh pr checks and the reviewThreads GraphQL query — 0 unresolved threads). Dismissing as superseded per AGENTS.md stale-review guidance.

@opencode-agent opencode-agent Bot added priority: medium Normal-priority or P2 work status: blocked Blocked by conflict, dependency, or required prerequisite type: bug Defect or incorrect behavior labels Aug 22, 2026
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The governance gate now validates GITHUB_REPOSITORY before API evaluation. It rejects malformed and dot-segment identities. It validates approval notices and applies notice-specific blocking rules to well-formed current-head notices. Malformed notices block evaluation.

Changes

PR governance gate

Layer / File(s) Summary
Repository identity validation
scripts/ci/pr_governance_gate.sh, docs/development/merge-gate-policy.md, scripts/ci/test_pr_governance_gate.sh, CHANGELOG.md
The gate rejects malformed repository identities before GitHub API calls. Tests verify fail-closed behavior and no gh invocations. Policy and changelog entries document the validation.
Approval notice detection and tests
scripts/ci/pr_governance_gate.sh, scripts/ci/test_pr_governance_gate.sh
The gate validates approval markers, ordering, content, and current-head metadata. Well-formed current-head notices use notice-specific blocking rules. Tests cover walkthrough, multiline JSON, potential-issue, mixed-content, malformed, whitespace-only, and stale-head scenarios.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🟡 Moderate · up to be923

The merge gate can treat a stale approval notice as current and ignore a real blocking comment because it accepts the commit SHA from the wrong field; the field matching and regression coverage should be corrected before merging.

Sequence Diagram(s)

sequenceDiagram
  participant GitHub as GitHub comments
  participant Gate as pr_governance_gate.sh
  participant Tests as test_pr_governance_gate.sh
  GitHub->>Gate: Issue comments and head SHA
  Gate->>Gate: Validate approval markers and notice content
  Gate->>Gate: Apply notice-specific blocking detection
  Gate->>Tests: Return blocking or non-blocking result
Loading
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary change: excluding CodeRabbit approval-pending notices from blocking issue classification. It is concise and directly related to the pull request objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/coderabbit-approval-pending-gate-develop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current exact-head check 97705801302 is annotated STRIX_PROVIDER_UNAVAILABLE: Strix could not complete authoritative vulnerability analysis because its provider/backend was unavailable. Source/security/image checks pass; no source change can correct this external provider outage. The gate repair remains otherwise ready for fresh protected checks and review.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head review note for 55a84f9: the existing approval-pending gate fix is the owning root change for the live Naruon metadata failures. Local bash scripts/ci/test_pr_governance_gate.sh passed, including exact-head and multiline approval-notice cases; git diff --check passed. The hosted Strix failure is provider evidence, not a source finding. Normal protected merge still requires current hosted gates and qualifying review.

* fix(governance): accept multiline approval notice heads

* test(governance): exercise multiline approval notice

* test(governance): exercise multiline pending blocker text

* fix(governance): validate repository identity before API calls

* docs(governance): record repository identity boundary

* fix(governance): keep mixed approval blockers visible
coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae
seonghobae enabled auto-merge (squash) August 25, 2026 17:03
devin-ai-integration[bot]

This comment was marked as resolved.

@opencode-agent
opencode-agent Bot disabled auto-merge August 25, 2026 17:07
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

and $ordered_approval_pair
and (($approval_notice | gsub("[[:space:]]"; "")) | length > 0)) as $approval_notice_well_formed
| ($approval_notice_well_formed
and ($approval_notice | test("(^|[^A-Za-z0-9_-])\"?headCommitId\"?[[:space:]]*:[[:space:]]*\"?" + $head_sha + "(\"|[[:space:]]|$)"; "i"))) as $current_approval_notice

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Prefixed head field correctly rejected

The current-head regex at pr_governance_gate.sh requires a non-[A-Za-z0-9_-] character before headCommitId. In previous-headCommitId the preceding - is excluded, so a prefixed field cannot be read as current-head evidence, matching the stale-prefixed fixture.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: medium Normal-priority or P2 work status: blocked Blocked by conflict, dependency, or required prerequisite type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant