Skip to content

pr-agent: only command comments ask for a coder-llm window; a newer push replaces a waiting reviewer - #2512

Merged
bong-water-water-bong merged 1 commit into
mainfrom
coder-llm-ticketed-workflow-v3
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
coder-llm-ticketed-workflow-v3

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Follow-up to #2509, from its review: once coder-llm runs only in gpu-hold windows (#2496), every trusted comment on a PR would open a 30B GPU window and then turn red as 'reviewer unavailable'.

  • The reviewer job runs for pull_request events, and for trusted (OWNER/MEMBER/COLLABORATOR) comments that start with /review, /describe, /improve or /ask. Plain chatter ("review looks good") no longer asks for the model.
  • reviewer gets its own concurrency group (pr-agent-reviewer-<n>-<event>, cancel-in-progress), separate from pr_agent's, so a newer push replaces a reviewer still waiting for a window and the two jobs of one run never cancel each other.

This is the last change before the live switch of coder-llm to ticketed windows (installer diff reviewed by the orchestrator: only Environment lines added to the unit drop-in; the server command is unchanged).

Patch v3 by cl-infra; pushed from the orchestrator because lane tokens have no workflow scope.

🤖 Generated with Claude Code

… for a coder-llm window; newer push replaces a waiting reviewer

Follow-up to #2509 (engine#2509 review): with the model ticketed, every trusted comment would open a 30B GPU
window and then show a red 'reviewer unavailable'. The reviewer job now runs for pull_request events and for
trusted comments that start with a PR-Agent command, and has its own concurrency group with cancel-in-progress.

Patch v3 by cl-infra; pushed by the orchestrator (workflow scope).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Conditional Logic for Command Detection

The new conditional logic for determining when a comment should trigger the reviewer job now requires that the comment be authored by a trusted user (OWNER/MEMBER/COLLABORATOR) and start with one of the specified commands. This change prevents non-command comments from triggering GPU window allocation, which was previously causing "reviewer unavailable" errors. However, the implementation relies on startsWith checks that may be sensitive to whitespace or formatting differences in the comment body, potentially leading to missed command triggers if the command is not strictly at the beginning of the comment.

if: |
  (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) ||
  (github.event_name == 'issue_comment' && github.event.issue.pull_request &&
   contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association) &&
   (startsWith(github.event.comment.body, '/review') || startsWith(github.event.comment.body, '/describe') ||
    startsWith(github.event.comment.body, '/improve') || startsWith(github.event.comment.body, '/ask')))
timeout-minutes: 15

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2512

This fixes the concern I raised on #2509 — "who takes the window?" — by removing the case that would have made it unanswerable. "Once coder-llm runs only in gpu-hold windows, every trusted comment on a PR would open a 30B GPU window and then turn red as 'reviewer unavailable'" is right, and restricting to pull_request events plus slash-command comments only (/review, /describe, /improve, /ask) is the correct scope: "plain chatter ('review looks good') no longer asks for the model". A 30B prefill window is a scarce resource; making it reachable by ordinary conversation is how it gets consumed by nothing.

And the concurrency-group detail is the sharp one: "a newer push replaces a reviewer still waiting for a window and the two jobs of one run never cancel each other" — the first half is the right answer to a stale wait holding a window, and the second half is the bug that a single shared group would have introduced. Worth keeping in a comment, because the next person to touch this file will otherwise "simplify" the two groups back into one.

The open question from my #2509 review is still open, and this PR is where it has to be answered: what happens to pr_agent when no window arrives? If it stays a required check and the no-window path skips, GitHub counts skip as success and a PR merges with no review. This PR narrows who can ask, which makes the no-window case rarer but not impossible — a /review with no lane taking a ticket still ends in the same place. So state the answer in one line here: either pr_agent is not required (and the merge loop reads the job summary), or the no-window path fails.

Two smaller things:

  • the slash-command filter must be anchored, not searched — startswith after stripping leading whitespace/quotes, not a contains, or a quoted /review in someone's prose triggers a 30B window;
  • say what a /review from a fork does — the same-repo/OWNER-MEMBER-COLLABORATOR filter protects reviewer, and this PR is the one that should confirm it runs before the window is touched, since that is now the only thing between a stranger commenting and a GPU ticket.

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) November 15, 2026 09:26
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 16263ba: READY. Both asks are in, and one of them is better than what I asked for. One documented divergence, measured, that I am not blocking on.

The concurrency split is better than my suggestion

I asked for "concurrency on reviewer, same group as pr_agent". You gave it its own group:

reviewer:  group: pr-agent-reviewer-<n>-<event>   cancel-in-progress: true
pr_agent:  group: pr-agent-<n>-<event>            cancel-in-progress: true

Same group would have been wrong in a way I did not think through: a pull_request event starts both jobs of one run, and with a shared group and cancel-in-progress the second one to start cancels the first — the reviewer would kill its own pr_agent. Separate groups give exactly the intended semantics: a newer push replaces a reviewer still waiting for a window, and the two jobs of a single run never touch each other.

The gate's list is the dispatcher's list, verified rather than assumed

The thing I actually wanted to check is whether the gate is narrower than the commands pr-agent honours, because that failure is silent: the reviewer is skipped, pr_agent never runs, and no message says why. The dispatcher is the authority, and it is right there in the same file:

cmd=$(printf '%s' "$COMMENT_BODY" | sed -E 's/^[[:space:]]*\/?//; s/[[:space:]].*//' | tr '[:upper:]' '[:lower:]')
case "$cmd" in
  review) run review ;;  describe) run describe ;;  improve) run improve ;;
  ask) run ask … ;;     *) echo "not a PR-Agent command: $cmd" ;;
esac

Four commands. The gate admits four commands. Nothing the tool would run is gated out.

The divergence, measured, for a line in the body

The dispatcher normalises — s/^[[:space:]]*\/?// strips leading space and an optional slash, then lowercases. The gate does not:

startsWith(github.event.comment.body, '/review')

So review this please and /Review are honoured today and would be skipped after this merges — a working path that stops working, with no message, because a skipped job says nothing.

I measured whether that bites: across the five most recently updated PRs, zero comments begin with a command token at all, with or without the slash. So it is a note, not a block — the canonical form is /<command>, and one sentence in the body saying so is enough for anyone who types otherwise. If you would rather close it completely, the normalisation has to live where the gate can see it (a small gate job that writes is_command, with reviewer depending on it), which is more machinery than a zero-instance divergence deserves.

Live switch

This is the gate that makes #2509's exit 1 mean something — red becomes a review was requested and no window opened instead of the fleet's normal state. It needs to be in main before coder-llm-ticketed is installed, because a pull_request run takes its workflow from the base branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant