Skip to content

ci: automated Claude PR review + human merge gate - #2

Merged
travisccook merged 8 commits into
mainfrom
feat/claude-pr-review
Jul 6, 2026
Merged

ci: automated Claude PR review + human merge gate#2
travisccook merged 8 commits into
mainfrom
feat/claude-pr-review

Conversation

@travisccook

Copy link
Copy Markdown
Owner

Adds two workflows (auto review for in-repo PRs; @claude static review for fork PRs) and a design/plan under docs/superpowers/. Branch protection applied separately. This PR is reviewed by the new setup itself.

travisccook and others added 8 commits July 6, 2026 16:07
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…needed)

The first run failed with 'Could not fetch an OIDC token'. claude-code-action@v1
defaults to authenticating GitHub via the Claude GitHub App over OIDC, which needs
both id-token: write and the App installed. Providing github_token makes it use the
built-in Actions token instead and skip OIDC entirely. Docs updated to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…vert next)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ckout

The review authenticated but hit max-turns (20) after thrashing on denied
git/gh exploration commands that weren't in the npm-only allowlist. Allow
read-only git + read utils (no git fetch/push, no gh — stays review-only),
raise to 40 turns, and fetch-depth: 0 so git diff against base works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nostic

Root cause of the empty PRs: passing --allowedTools REPLACES the action's default
allowlist, and in github_token mode there is no MCP comment tool — the action
expects posting via gh. The review completed fine but its 'gh pr comment' call was
denied. Allow gh pr comment (only — not gh api/merge, keeping review-only), tell the
prompt to post that way, and remove the now-done show_full_output diagnostic. Mirror
the tool set on the mention path (read-only + gh pr comment, still no npm).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Automated review — PR #2 (feat/claude-pr-review)

Scope note: this PR does not touch libraries/boards/*.json, libraries/manifest.json, or src/*.js — it only adds two GitHub Actions workflows (.github/workflows/claude-review.yml, .github/workflows/claude-mention.yml) and two planning docs under docs/superpowers/. So checklist items 2–4 (board-contribution rules, safety-class mislabeling, engine/UI invariants) are N/A for this diff — noted for completeness, not a defect.

1. Checks — all green

  • npm ci — installed clean (272 packages; 1 moderate npm audit advisory, pre-existing/unrelated).
  • npm run validate — passes for manifest + all 8 board files + cross-file checks.
  • npm test — 7 suites / 132 tests passed.

No functional code or board data changed, so this is expected, but confirms the PR doesn't regress the pipeline.

2–4. Board rules / safety labeling / engine invariants

Not applicable — no files under libraries/ or src/ are touched by this PR.

Findings on the actual diff (the two workflows)

Fork-safety design looks correct. claude-review.yml:19-21 gates on head.repo.full_name == github.repository and uses the plain pull_request event (never pull_request_target), so fork PRs can't get secrets/execution — they fall through to claude-mention.yml, which is explicitly read-only/no-npm (claude-mention.yml:47-49) and cites the existing validate-and-test CI run for pass/fail instead of re-executing fork code. That split is the right way to avoid a secrets-exfiltration path via a malicious fork PR.

Medium — PR number/repo aren't resolvable with the allowed toolset (claude-review.yml:41-49,87). The prompt tells Claude to run gh pr comment <PR_NUMBER> --repo <OWNER/REPO> --body "…", but neither value is interpolated via GitHub Actions expressions (e.g. ${{ github.event.pull_request.number }} / ${{ github.repository }}), and the job's --allowedTools only grants Bash(gh pr comment:*) — there's no gh pr view, gh api, or generic env/printenv allowlisted to let Claude discover those values itself. This works only if gh pr comment (invoked with no explicit number/repo) auto-detects the current PR from the checked-out branch, which the gh CLI can do — but that's implicit and undocumented here. Recommend either explicitly interpolating ${{ github.event.pull_request.number }}/${{ github.repository }} into the prompt text, or rewording step to gh pr comment --body "…" and noting that gh auto-detects the PR from the checkout, so a future edit to the allowedTools list doesn't silently break comment-posting.

Low — no concurrency guard. claude-review.yml triggers on synchronize as well as opened/reopened (line 9), so rapid pushes to a PR will queue/overlap multiple full Claude review runs. Consider adding a concurrency: { group: claude-review-${{ github.event.pull_request.number }}, cancel-in-progress: true } block to avoid stacked runs and wasted quota.

Low — unused permission. claude-review.yml:87 allowlists Bash(npm run lint), but the prompt's instructions (steps 1–4) never ask Claude to run it. Harmless (lint is || true per CLAUDE.md) but dead weight — either add a lint step to the prompt or drop it from --allowedTools.

Nit — trade-off worth double-checking. claude-review.yml:36-39 notes reviews will post as github-actions[bot] rather than claude[bot] because github_token bypasses the OIDC exchange. That's a reasonable, clearly-documented trade-off, just flagging it's intentional so it isn't "fixed" by a later PR without discussion.

Docs-only additions (docs/superpowers/plans/2026-07-06-automated-pr-review.md, docs/superpowers/specs/2026-07-06-automated-pr-review-design.md) follow the existing plan/spec convention in this repo and don't affect runtime behavior.

Overall: workflow design is sound and the fork/trusted-PR split correctly avoids the dangerous pull_request_target + secrets pattern. The main actionable item is the PR-number/repo resolution gap in claude-review.yml's prompt vs. its allowedTools — worth confirming intent before merge. Advisory only; no changes made.

@travisccook
travisccook merged commit d03afe2 into main Jul 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant