Skip to content

feat(dev-kit): add plugin with ci-fix, pr-shepherd, dep-upgrade-check and pr-open skills - #15

Merged
grixu merged 12 commits into
mainfrom
feat/dev-kit
Sep 25, 2026
Merged

grixu merged 12 commits into
mainfrom
feat/dev-kit

Conversation

@grixu

@grixu grixu commented Sep 25, 2026

Copy link
Copy Markdown
Owner

Adds dev-kit, a plugin with four everyday skills for CI, pull requests and dependency upgrades.

Why it matters: fixing red CI, answering review comments, checking a dependency bump and opening a PR are routine work that now runs as one skill each, invocable by the user or picked up by Claude.

What changed:

  • ci-fix: takes a failed run, job, PR or commit URL (or the current branch), finds the root cause, fixes and verifies it locally, commits once per cause. Pushes on consent, or unattended with push-authorized. Transient and flaky failures get one rerun; environment problems are reported.
  • pr-shepherd: self-paced /loop over the current branch's PR. Red CI goes to ci-fix; justified review comments are fixed, pushed and resolved; unjustified ones get an in-thread reply mentioning the author. Stops when checks are green and no thread is open, or hands over when a human is needed.
  • dep-upgrade-check: read-only check of an upgrade (package + version, or a Renovate/Dependabot PR). Builds a ledger of every change in the range and maps it to real usage. Verdict: safe / safe with changes / breaking, with a file:line table.
  • pr-open: pushes the branch and opens a PR that keeps the repository's template structure and fills it in Smart Brevity style.
  • Plugin registered in .claude-plugin/marketplace.json at 0.1.0; README and [Unreleased] changelog entries added.

How tested:

  • claude plugin validate plugins/dev-kit and claude plugin validate . pass.
  • No skill has been run end to end yet; this PR is the first pr-open + pr-shepherd run.

@grixu grixu self-assigned this Sep 25, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Two of the four skills have steps that cannot execute as written: pr-shepherd ends its loop with a ScheduleWakeup parameter that does not exist, and dep-upgrade-check's allowed-tools omits most of the tools its own steps invoke. The debut version baseline also needs to be 0.0.0 for release.sh to cut v0.1.0.

Reviewed changes — full read of all 5 commits (0aef892..58e78ad) adding plugins/dev-kit/, a prompt-only plugin with no scripts, evals, or CI hook.

  • Plugin scaffold — plugin.json, CHANGELOG.md ([Unreleased] only, correct for a debut), README.md, and a marketplace.json entry. Author name/email and description are byte-identical across both manifests.
  • ci-fix — resolves a failed Actions run from a URL/ID/PR or the current branch, traces a root cause, fixes and verifies locally, one commit per cause. A push-authorized token switches it to unattended mode where every "ask the user" becomes status blocked. Emits a fixed-key result block.
  • pr-shepherd — one iteration of a self-paced /loop over the current branch's PR. Delegates red CI to ci-fix … push-authorized and reads its result block; classifies GraphQL reviewThreads as open/awaiting and replies or resolves accordingly.
  • dep-upgrade-check — read-only upgrade impact analysis intersecting a change ledger with a usage surface, plus bot-prs.md (Renovate/Dependabot parsing) and ecosystems.md (npm/yarn/Python/Go/PHP/Rust lookups).
  • pr-open — pushes the branch and opens a PR filled from the repository's template, falling back to a built-in Smart Brevity skeleton.

I verified four things that looked wrong and are in fact correct, so they are deliberately not raised below: disallowed-tools is a documented SKILL.md field whose own doc example is AskUserQuestion for a background loop; mcp__context7__query-docs is the current Context7 tool name (get-library-docs was removed in @upstash/context7-mcp@2.0.0); the space form Bash(gh pr view *) is the canonical permission syntax, not a malformed :*; and find -ipath './.github/pull_request_template*' does match PULL_REQUEST_TEMPLATE/bug.md, since -ipath wildcards cross /.

ℹ️ Four skills that commit, push, rerun CI and post review replies ship with no automated verification

Nothing in CI touches plugins/dev-kit/**: fd3-evals.yml is scoped to plugins/fd3/**, and lint.yml is shellcheck over *.sh only. The PR body states no skill has been run end to end. That is a reasonable call for a prompt-only debut, but these four skills push commits, rerun workflows and post review replies under push-authorized — the blast radius of a prose defect is higher here than for a read-only skill, and the three findings above are all the kind a single dry run would have surfaced.

Technical details
# No verification surface for plugins/dev-kit

## Affected sites
- `.github/workflows/fd3-evals.yml` — the only eval CI, scoped to `plugins/fd3/**`
- `.github/workflows/lint.yml` — shellcheck over `*.sh`; dev-kit ships no shell scripts

## Open questions for the human
- Is an `evals/` suite intended for dev-kit later, or are these skills deliberately
  manual-verification-only?
- `pr-shepherd` and `ci-fix` both act unattended with push rights. Is a dry run against a
  throwaway PR worth doing before the debut release tag, given no eval gates them?

ℹ️ Nitpicks

  • plugins/dev-kit/skills/ci-fix/SKILL.md:48 — "when attempt is already above 1 with the same failure, it is not transient" removes the job from the Transient bucket but does not route it to any of the other four, leaving the classification undefined for exactly the case that matters most (a repeat failure). Naming the fallback bucket would close it.
  • plugins/dev-kit/skills/pr-open/SKILL.md:19 — "push (step 5)" points at a step whose four substeps include gh pr create, which would fail when a PR already exists. Referring to git push -u origin HEAD directly avoids the literal reading.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using Claude Opus | 𝕏

Comment thread plugins/dev-kit/.claude-plugin/plugin.json Outdated
Comment thread plugins/dev-kit/skills/pr-shepherd/SKILL.md
Comment thread plugins/dev-kit/skills/dep-upgrade-check/SKILL.md
Comment thread plugins/dev-kit/skills/pr-shepherd/SKILL.md

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the 2 commits since the previous review at 58e78ad, a 3-line delta across 3 files, both of them fixes for that review.

  • Set the debut version baseline to 0.0.0 — plugins/dev-kit/.claude-plugin/plugin.json:4 and .claude-plugin/marketplace.json:195 moved off 0.1.0 and stay in parity. bump_version in scripts/release.sh:60-69 turns 0.0.0 into 0.1.0 on a minor, so the first ./scripts/release.sh dev-kit minor now cuts dev-kit/v0.1.0 instead of skipping it.
  • Gave pr-shepherd's loop an exit for threads it has decided not to handle — skills/pr-shepherd/SKILL.md:97 widened the CI-green stop to cover handed over threads (two rebuttals already posted, or viewerCanReply/viewerCanResolve false) alongside the 24-hour quiet-reviewer case.

I re-traced both leak paths the previous review described against the new wording. A permission-blocked thread parked by :89 and a twice-rebutted thread parked by :72 are each still classified open by :49, so :93's Done condition still does not fire for them — but both now match the handed over clause at :97, so a loop holding only those threads stops on the iteration it reaches them rather than spinning to the iteration-48 cap. The two remaining stops (:98, :99) are unchanged and the awaiting-thread wake cadence at :101-104 is untouched.

The ScheduleWakeup and allowed-tools threads were declined with reasoned, sourced rebuttals and resolved by the author; both rest on lines these commits did not touch, so they are not reopened here. The no-automated-verification observation from the previous review still stands as written and is not restated.

Pullfrog  | View workflow run | Using Claude Opus | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues — one suggestion inline, plus one scope question about how long the new grant lasts.

Reviewed changes — the 2 commits since the previous review at 45404e2, a 3-line delta across 3 files that gives the two unattended skills an allowed-tools grant.

  • Pre-approved ci-fix's git and gh commands — skills/ci-fix/SKILL.md:5 now grants gh run view|list|rerun, gh pr checks|checkout, gh api repos/*, and git status|fetch|switch|log|diff|show|add|commit|push. Every git/gh invocation in the skill body is covered.
  • Pre-approved pr-shepherd's git, gh and delegation calls — skills/pr-shepherd/SKILL.md:5 adds the same core plus git rev-list, git pull --ff-only, gh pr view|checks|comment, gh api graphql *, and Skill(dev-kit:ci-fix *) for the hand-off to ci-fix. Also fully covered.
  • Documented the limit of the pre-approval — README.md:30 states that the reproduce/verify commands (tests, lint, build) are repository-specific, cannot be shipped in the plugin, and must go in the project's own .claude/settings.json or the loop pauses at the first such prompt.

Three things I checked that are correct and are deliberately not raised below. Skill(dev-kit:ci-fix *) is the documented rule shape — the skills doc gives "Skill(name) for exact match, Skill(name *) for prefix match with any arguments", and plugin skills are namespaced plugin-name:skill-name. The absence of ScheduleWakeup and CronDelete from the list is harmless: the tool reference marks both as requiring no approval, so pr-shepherd:19 and :102 never prompt. And Bash(gh api graphql *) genuinely needs write scope, since :78 and :82 are the addPullRequestReviewThreadReply and resolveReviewThread mutations.

ℹ️ The grant is scoped to one turn, but pr-shepherd is designed to span many

The skills documentation is explicit that allowed-tools "grants permission for the listed tools during the turn that invokes the skill" and that "the grant clears when you send your next message, even though the skill content stays in context; invoking the skill again re-applies it for that turn." pr-shepherd is a self-paced /loop: every iteration after the first arrives as a separate turn woken by ScheduleWakeup. Whether the pre-approval survives therefore depends on each wake-up re-entering through the Skill tool rather than running the body from context — and ## Entry (:15) explicitly sanctions the second shape with "or this turn is already a /loop fire, run the iteration". If that branch ever runs without a fresh invocation, iteration 2 onward executes with the grant already cleared and stalls on the first git push, which is the exact failure these commits set out to remove.

Technical details
# `allowed-tools` is a per-turn grant; `pr-shepherd` iterates across turns

## Affected sites
- `plugins/dev-kit/skills/pr-shepherd/SKILL.md:5` — the grant that the loop depends on
- `plugins/dev-kit/skills/pr-shepherd/SKILL.md:15` — `## Entry` admits two ways an
  iteration can begin: the arguments carry `iteration`, or "this turn is already a
  `/loop` fire". Only a path that re-invokes the skill re-applies the grant.
- `plugins/dev-kit/skills/pr-shepherd/SKILL.md:102` — `ScheduleWakeup` for the next
  iteration; each fire is a new turn
- `plugins/dev-kit/README.md:30` — names the repo-specific test/lint commands as the one
  limit of the pre-approval, and does not mention turn scoping

## Required outcome
- An iteration woken by `ScheduleWakeup` must run with the same pre-approval as the first
  one, or the README should say that it does not.

## Open questions for the human
- Does the `/loop` skill re-fire `/dev-kit:pr-shepherd iteration` as a skill invocation on
  every wake-up (re-applying the grant), or does the iteration continue from the skill
  content already in context?
- If it is the latter, should `## Entry` require re-invoking the skill on every fire rather
  than offering the "already a `/loop` fire" shortcut?

ℹ️ Nitpicks

  • plugins/dev-kit/skills/ci-fix/SKILL.md:5 and plugins/dev-kit/skills/pr-shepherd/SKILL.md:5 — Bash(git fetch) and Bash(git push) are redundant next to their * twins. The permissions doc: "A * at the end, with a space before it, also matches the bare command. Bash(ls *) matches ls, and Bash(git log *) matches git log." So Bash(git fetch *) and Bash(git push *) already cover the argument-less invocations, and the four paired rules can collapse to two.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using Claude Opus | 𝕏

Comment thread plugins/dev-kit/skills/ci-fix/SKILL.md Outdated

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues — one suggestion inline on the new --assignee flag.

Reviewed changes — the single commit since the previous review at d000668, a 2-line delta across 2 files that self-assigns the pull requests pr-open opens.

  • Self-assigned newly created pull requests — skills/pr-open/SKILL.md:79 appends --assignee @me to step 5.3's gh pr create. It affects the creation path only; the existing-PR branch at :19 still just refreshes the body with gh pr edit --body-file and does not assign.
  • Documented it in the skill list — README.md:43 gains "Assigns the PR to you." to the pr-open paragraph.

Two things I checked that are correct and are deliberately not raised below. --assignee @me is valid gh pr create syntax, and pr-open is the one skill declaring no allowed-tools, so the new flag needs no permission-rule change. The [Unreleased] changelog correctly gains nothing: its pr-open bullet is the plugin's debut entry and describes the skill at a level that enumerates neither assignment nor the existing-PR refresh.

The Bash(gh api repos/*) thread from the previous review sits on a line this commit did not touch, so it stays open and is not restated here.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using Claude Opus | 𝕏

Comment thread plugins/dev-kit/skills/pr-open/SKILL.md Outdated
@grixu

grixu commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

@pullfrog On the per-turn allowed-tools grant (#15 (review)): no change. Every /loop fire re-enters through the Skill tool, since the loop replays its prompt /dev-kit:pr-shepherd iteration and the /loop skill invokes a slash-command prompt via Skill on each iteration. The "already a /loop fire" branch in Entry covers /loop /dev-kit:pr-shepherd typed by the user, which reaches the skill the same way, so the grant is re-applied on every iteration.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues — both open threads are properly fixed; one README line went stale with the gh api grant removal.

Reviewed changes — the 2 commits since the previous review at b449db9, a 3-line delta across 2 files, both of them fixes for the open inline threads.

  • Dropped ci-fix's gh api pre-approval — skills/ci-fix/SKILL.md:5 no longer grants Bash(gh api repos/*). The two check-run reads at :27 are untouched and now go through the normal prompt, and no replacement grant was introduced anywhere in the skill, so ci-fix now matches the deliberate "gh api can write, keep it behind consent" stance already taken for dep-upgrade-check.
  • Split PR assignment out of gh pr create — skills/pr-open/SKILL.md:79 reverts to a plain gh pr create, and a new step 5.5 at :81 assigns with gh pr edit <url> --add-assignee @me, stating why it runs apart from the create and that on failure the PR stays opened and is reported unassigned.

I re-traced the assignment failure path against the new ordering. gh pr create is now the only mutation whose failure means no pull request, :80's temp-file cleanup sits ahead of the assignment rather than behind it, and :83's Done condition ("the PR exists and its URL is printed") is no longer satisfiable by a non-zero exit — which was the exact shape of the previous finding. The syntax holds too: gh pr edit accepts a URL selector and @me is a documented special value of --add-assignee.

pr-shepherd:5 deliberately keeps Bash(gh api graphql *), which is load-bearing there rather than incidental — :78 and :82 are the addPullRequestReviewThreadReply and resolveReviewThread mutations — so it is not raised as a parallel to the ci-fix removal. The turn-scoping question on that grant and the absence of any CI covering plugins/dev-kit/** both rest on lines these commits did not touch and are not restated.

ℹ️ Nitpicks

  • plugins/dev-kit/README.md:30 — "pr-shepherd and ci-fix pre-approve the git and gh commands they run" stopped being true in 07254c6: ci-fix:27's two gh api repos/.../check-runs reads are now outside the grant on purpose. The paragraph then names the repository-specific test/lint/build commands as the one exception, so someone setting up an unattended loop would not expect the third-party-check branch to prompt. A clause naming gh api as the second, deliberate exception would close it.

Pullfrog  | Fix it ➔ | View workflow run | Using Claude Opus | 𝕏

@pullfrog

pullfrog Bot commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

No change to pr-shepherd's allowed-tools grant, and no README note about turn scoping.

For the record, the grant-scope question is in review 5315803467 — the linked 5317717122 only covers pr-open's --assignee. It was raised in the review body rather than as an inline thread, so this is a standalone reply.

The rationale holds on the re-entry path, traced as far as this environment allows:

  • ScheduleWakeup in dynamic /loop mode carries the /loop input as its prompt, and its own contract is explicit that passing it verbatim is what makes "the next firing re-enter the skill". So each fire arrives as /loop /dev-kit:pr-shepherd iteration, not as a bare continuation of the previous turn.
  • A slash-command prompt is invoked through the Skill tool, so that fire re-enters dev-kit:pr-shepherd as a fresh skill invocation and the allowed-tools grant is re-applied for that turn.
  • The "already a /loop fire" branch at skills/pr-shepherd/SKILL.md:15 does not open a grant-less path. It is a recursion guard for /loop /dev-kit:pr-shepherd typed by the user, where the loop skill invokes pr-shepherd via Skill with no iteration argument — without that branch the skill would hand straight back to loop and nest. Every turn the branch covers therefore also began with a Skill invocation.

One limit on the verification worth stating plainly: the loop skill is not on disk in this ephemeral runner, so the final hop rests on the documented ScheduleWakeup and Skill contracts rather than on the loop skill's own text. If a future loop ever replays an iteration from context instead of re-invoking the skill, SKILL.md:15 and README.md:30 are the two places that would need revisiting.

No files changed; the working tree is clean.

Pullfrog  | View workflow run | via Pullfrog | Using Claude Opus | 𝕏

@grixu
grixu merged commit ff2b553 into main Sep 25, 2026
3 checks passed
@grixu
grixu deleted the feat/dev-kit branch September 25, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant