Repository navigation
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
A pull request force-pushed back to its base has no commit, and reading its check suites ended the PR sweep in a jq error. On a branch that still merges, a current hand-written rebase request was offered for hiding while the older fixed-text copy it superseded stayed. The issue sweep read only the last body edit, so a reporter who answered by editing stayed listed as waiting once a maintainer tidied the body; it also reported idle days for an assignee none of whose acts was read, missed a heading followed by a higher-level one or ending the body, took comments in a pasted shell session for headings, and ignored an uppercase hash. Test both sweeps against a fake gh in a new suite, and shellcheck the skills' scripts with the rest.
The skills said how to read a review but not how to fetch every inline thread, a long one included, or what a push leaves unproven. They said which lane to rerun but not how to tell a current failure from one on an older head. They named the UI rules reviewers raise but gave no walk through a changed flow, no Playwright method that reuses the existing browser checks, and no rules for applying a requested label, assignee, sub-issue or dependency. Add each where its owner already sits, linking the rules other sections hold rather than restating them, and state the existing rule on asking about unclear review items once.
The reference checker was awk over the skills, outside the gate, and guessed wrong in both directions. It gave a repeated heading, closing hashes, a link in a heading and indented or underlined headings anchors GitHub does not, read headings inside tilde or longer fences, checked links in inline code, and cut make targets at an underscore or a digit, while a link title, a path:line reference or a flag that is a prefix of another raised false alarms. Rewrite it as scripts/check-skill-refs.py, which follows GitHub's slug rule and parses fences and code spans once, also checks code blocks and root files, and runs as a gate with tests against a scratch repository.
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".claude/skills/codetrial-verify/SKILL.md">
<violation number="1" location=".claude/skills/codetrial-verify/SKILL.md:243">
P2: `headSha` from `gh run view` identifies the PR head, not the merge commit these `pull_request` jobs test. Record and compare the run's `GITHUB_SHA` with the current PR merge ref; matching `headSha` alone can accept results from an outdated base.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| | fixed | the original reproduction now behaves | the code changed | | ||
| | tests pass | `./scripts/test.sh` at this head, its skips read | an earlier run | | ||
| | CI is green | `gh pr checks` naming this head | a local pass | | ||
| | CI is green | `gh pr checks` passing, and each run's `headSha` from `gh run view` at this head | a local pass, or a run on an older head | |
There was a problem hiding this comment.
P2: headSha from gh run view identifies the PR head, not the merge commit these pull_request jobs test. Record and compare the run's GITHUB_SHA with the current PR merge ref; matching headSha alone can accept results from an outdated base.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/skills/codetrial-verify/SKILL.md, line 243:
<comment>`headSha` from `gh run view` identifies the PR head, not the merge commit these `pull_request` jobs test. Record and compare the run's `GITHUB_SHA` with the current PR merge ref; matching `headSha` alone can accept results from an outdated base.</comment>
<file context>
@@ -240,7 +240,7 @@ rebase or amend:
| fixed | the original reproduction now behaves | the code changed |
| tests pass | `./scripts/test.sh` at this head, its skips read | an earlier run |
-| CI is green | `gh pr checks` with each run's `headSha` at this head | a local pass, or a run on an older head |
+| CI is green | `gh pr checks` passing, and each run's `headSha` from `gh run view` at this head | a local pass, or a run on an older head |
| review addressed | each open thread reread at this head | a pushed commit |
| the agent did it | `git diff` shows the change | the agent's summary |
</file context>
Contributor
Author
There was a problem hiding this comment.
Not adopting this. The row's claim is that CI passed at the head being pushed, which headSha settles: for a pull_request run it is the PR head, as run 39626571224 showed. A base that moved after the run is the stale-base case in collaboration.md's "Make CI green before asking for review", and it is settled by rebasing and rerunning. Comparing against the merge ref cannot replace that, because the ref moves with every push to main and no stored run would ever match it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The skills told an agent how to read a review but not how to fetch every inline thread or what a push leaves unproven, which lane to rerun but not how to tell a current CI failure from one on an older head, and which UI rules reviewers raise but not how to walk a changed flow or investigate it with the existing browser checks. This adds that guidance where each topic already lives, plus the rules for applying a requested label, assignee, sub-issue or dependency, linking the sections that already hold a rule rather than restating it. It also fixes edge cases in both triage sweeps (a PR with no commits ended the PR sweep; a reporter's body edit hidden by a later maintainer edit, an assignee with no recorded act, and several empty-section and revision cases were misjudged), and replaces the awk reference checker with scripts/check-skill-refs.py, which follows GitHub's anchor rule and parses fences and code spans properly. Both now run in scripts/test.sh: a skill-refs gate over the real skills, and tests/test_skill_scripts.py, which drives the sweeps against a fake gh and the checker against a scratch repository.
Verified with the full ./scripts/test.sh at the tip (all three lanes, nothing skipped), plus shellcheck, ruff, the new suite and the reference check at each of the three commits. The new suite fails all ten bug cases against the previous scripts and passes its baseline cases on both. Both sweeps were also run read-only against this repository; their lists differed from an earlier snapshot only where issues and PRs had changed on GitHub since.
Summary by cubic
Extends the agent skills with the mechanics of closing out review feedback (fetching every inline thread, confirming a push addressed it), of telling a current CI failure from one on an older head, and of walking a changed UI flow with the existing browser lanes, plus the rules for applying a requested label, assignee, sub-issue or dependency. Fixes edge cases in both triage sweeps and replaces the awk reference checker with
scripts/check-skill-refs.py, which follows GitHub's anchor rule and parses fences and code spans correctly; both now run inscripts/test.shvia the newtests/test_skill_scripts.py.Bug Fixes
Refactors
skill-refs.shbecomesscripts/check-skill-refs.py, which follows GitHub's anchor rule and parses inline code and fences properly.scripts/test.sh, the sweeps against a fakeghand the checker against a scratch repository, and the skills' shell scripts join the shellcheck pass.Written for commit 6a251df. Summary will update on new commits.