Repository navigation
feat(ci): add declarative bug fix playbook pipeline #1820
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
skevetter
wants to merge
21
commits into
main
Choose a base branch
from
chore/daily-ci-triage-job
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
5e8d59b
chore(ci): establish daily triage agent and playbook pipeline
skevetter 79d4bf6
fix(ci): configure full-access mode and clean gate text in playbook
skevetter 406c576
chore(ci): update schedule sync tool and daemon schema parsing
skevetter ab0bf0f
feat(ci): add already_resolved outcome transitions to fix playbook
skevetter da757f5
feat(ci): add open pr deduplication to daily triage and playbook roles
skevetter 1a4c59e
fix(ci): set already_resolved outcome transition to end for clean run…
skevetter 7461c68
feat(ci): add wait_ci polling script step before human pr approval gate
skevetter 9bbc4b1
fix(ci): format wait_ci script and resolve code review findings
skevetter d5b26a5
fix(playbooks): update step definitions to block syntax
skevetter 78b235b
feat(agents): register and assign gemini 3.8 flash to investigator role
skevetter a9cb879
feat(tasks): decompose daily triage into isolated tasks
skevetter 1f9f2c7
fix(tasks): resolve review findings and generalize paths and project …
skevetter 58d6e2b
refactor(agents): consolidate subagent metadata into agent.md and rem…
skevetter 4e2385a
refactor(agents): remove redundant authorized models sections from ag…
skevetter d611b34
fix(playbooks): switch ci-investigator role to 6.1 terra via oh-my-pi
skevetter 445f12f
chore(ci): scope PR to .paseo bug fix playbook pipeline
skevetter 2e3282c
fix(playbooks): default agent to omp claude haiku 5.5
skevetter b8e33c6
fix(playbooks): switch fix-worker role to omp gpt-6.1-terra
skevetter e370431
chore(ci): move wait_ci polling logic into mise task
skevetter 10dc99b
fix(playbooks): manager to omp claude haiku 5.5
skevetter 075f708
fix(playbooks): allow implement retries to update the open pull reque…
skevetter File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| agent: | ||
| provider: omp | ||
| model: anthropic/claude-haiku-5-5 | ||
| mode: full | ||
| manager: | ||
| provider: omp | ||
| model: anthropic/claude-haiku-5-5 | ||
| mode: full | ||
| timeout: 2h | ||
| role: | ||
| silence: 30m | ||
| archive_agents: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Playbook Manager Instructions | ||
|
|
||
| You are the Playbook Manager supervising execution of Devsy playbooks. | ||
| Follow the playbook protocol strictly. | ||
|
|
||
| ## Manager Responsibilities | ||
|
|
||
| 1. Read inputs and start ready steps using start_step. | ||
| 2. Provide concise and focused briefs to step agents. | ||
| 3. Validate step outputs upon step completion. | ||
| 4. Open gates for human decisions using request_gate. | ||
| 5. Relay feedback to target steps when loops trigger. | ||
| 6. Enforce zero tracker references across all commits and pull requests. | ||
| 7. Ensure all subagents run in standard normal mode with fast mode disabled. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| title: Devsy Fix and PR Pipeline | ||
| description: Investigate, implement, validate, review, and merge a verified bug fix in Devsy. | ||
| concurrency: 1 | ||
|
|
||
| inputs: | ||
| fingerprint: | ||
| type: string | ||
| required: true | ||
| description: Normalized failure fingerprint | ||
| description: | ||
| type: string | ||
| required: true | ||
| description: Description of the failure or issue | ||
| failing_step: | ||
| type: string | ||
| default: unspecified | ||
| description: Failing CI step or component | ||
|
|
||
| outputs: | ||
| pr_url: | ||
| type: string | ||
| value: "{{ steps.implement.outputs.pr_url }}" | ||
| commit_sha: | ||
| type: string | ||
| value: "{{ steps.implement.outputs.commit_sha }}" | ||
|
|
||
| steps: | ||
| - id: investigate | ||
| role: | ||
| name: ci-investigator | ||
| prompt: "Investigate failure: {{ inputs.description }}.\nFingerprint: {{ inputs.fingerprint }}.\nFailing step: {{ inputs.failing_step }}." | ||
| inputs: [fingerprint, description, failing_step] | ||
| outputs: | ||
| root_cause: { type: string } | ||
| repro_command: { type: string } | ||
| fix_hypothesis: { type: string } | ||
| outcomes: [success, failure, already_resolved] | ||
| on: | ||
| already_resolved: end | ||
| - id: approve_plan | ||
| needs: [investigate] | ||
| gate: | ||
| question: "Root cause identified. Approve fix hypothesis to begin implementation, or request changes." | ||
| choices: | ||
| approved: { label: Approve and Implement, color: success } | ||
| rejected: { label: Request Changes, comment: required, color: danger } | ||
| on: | ||
| rejected: { goto: investigate } | ||
|
|
||
| - id: implement | ||
| needs: [approve_plan] | ||
| role: | ||
| name: fix-worker | ||
| prompt: "Implement approved fix for {{ inputs.fingerprint }}. Loop {{ loop }}.\nHypothesis: {{ inputs.fix_hypothesis }}.\nRoot cause: {{ inputs.root_cause }}.\nRepro: {{ inputs.repro_command }}.\nIf loop is greater than 0, apply changes to the existing open pull request branch and push with force-with-lease instead of opening a duplicate PR." | ||
| inputs: | ||
| fingerprint: "{{ playbook.inputs.fingerprint }}" | ||
| fix_hypothesis: "{{ steps.investigate.outputs.fix_hypothesis }}" | ||
| root_cause: "{{ steps.investigate.outputs.root_cause }}" | ||
| repro_command: "{{ steps.investigate.outputs.repro_command }}" | ||
| outputs: | ||
| pr_url: { type: string } | ||
| commit_sha: { type: string } | ||
| outcomes: [success, failure, already_resolved] | ||
| on: | ||
| already_resolved: end | ||
| - id: wait_ci | ||
| needs: [implement] | ||
| script: | ||
| run: wait_ci.sh | ||
| poll: 30s | ||
| timeout: 45m | ||
| inputs: | ||
| pr_url: "{{ steps.implement.outputs.pr_url }}" | ||
| outcomes: [success, failure] | ||
| on: | ||
| failure: { goto: implement } | ||
|
|
||
| - id: approve_pr | ||
| needs: [wait_ci] | ||
| gate: | ||
| question: "All CI status checks have passed and PR is green. Review and approve to merge, or request changes." | ||
| choices: | ||
| approved: { label: Approve and Merge, color: success } | ||
| changes_requested: { label: Request Changes, comment: required, color: danger } | ||
| on: | ||
| changes_requested: { goto: implement } | ||
| - id: merge | ||
| needs: [approve_pr] | ||
| role: | ||
| name: fix-worker | ||
| prompt: "Run gh pr ready {{ inputs.pr_url }} to mark the draft pull request ready for review. Then merge pull request {{ inputs.pr_url }}. Retrieve PR title and number using gh pr view {{ inputs.pr_url }} --json title,number. Run squash merge with one-line subject and empty body: gh pr merge {{ inputs.pr_url }} --squash --delete-branch --subject \"<title> (#<number>)\" --body \"\". Verify commit message is single line." | ||
| inputs: | ||
| pr_url: "{{ steps.implement.outputs.pr_url }}" | ||
| outputs: | ||
| merge_status: { type: string } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| You are a read-only investigation agent for devsy-org/devsy. | ||
| Your task is performing root cause analysis on a reported bug or CI failure. | ||
|
|
||
| ## Guidelines | ||
|
|
||
| 1. Check for existing open pull requests targeting this failure: | ||
| gh pr list -R devsy-org/devsy --state open --search "<fingerprint>" | ||
| If an open pull request already exists, report outcome: already_resolved with the active PR URL in summary. | ||
| 2. Locate the failing signature and inspect relevant logs, workflows, and source files. | ||
| 3. Identify the root cause with exact file paths and line numbers. | ||
| 4. Formulate a minimal reproduction command or test case. | ||
| 5. Formulate a minimal fix hypothesis adhering to devsy-pr-ready. | ||
| 6. Never edit files, create commits, push branches, or open pull requests. | ||
| 7. Populate required outputs: root_cause, repro_command, and fix_hypothesis. | ||
| 8. If the failure has already been resolved on origin/main, report outcome: already_resolved. | ||
|
|
||
| ## Output Template | ||
|
|
||
| Structure your final summary using this template: | ||
|
|
||
| ### Root Cause | ||
| Exact file paths and line numbers causing the failure. | ||
|
|
||
| ### Reproduction Command | ||
| Deterministic command or test demonstrating the issue. | ||
|
|
||
| ### Proposed Fix Hypothesis | ||
| Minimal behavior-preserving change adhering to devsy-pr-ready. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| title: CI Investigator | ||
| readonly: true | ||
| agent: | ||
| provider: omp | ||
| model: openai-codex/gpt-6.1-terra | ||
| mode: full |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| You are the fix implementation worker for devsy-org/devsy. | ||
| Your task is implementing verified bug fixes and opening pull requests adhering strictly to devsy-pr-ready. | ||
| When the prompt asks you to merge a pull request, perform only the merge instructions in the prompt and skip the guidelines below. | ||
|
|
||
| ## Implementation Guidelines | ||
|
|
||
| 1. When the prompt reports loop 0, check for open pull requests targeting this failure: | ||
| gh pr list -R devsy-org/devsy --state open --search "<fingerprint>" | ||
| If an open pull request already exists, report outcome: already_resolved and exit without opening a duplicate pull request. | ||
| When the prompt reports a loop greater than 0, skip this check. Apply the changes to the existing open pull request branch, push with force-with-lease, and report that pull request as pr_url. | ||
| 2. Minimal behavior-preserving changes and targeted regression tests. | ||
| 3. Self-documenting code with plain ASCII text. Do not add redundant comments. | ||
| 4. Run local validation: | ||
| mkdir -p dist | ||
| mise run cli:format | ||
| mise run cli:lint:ci | ||
| mise run cli:test | ||
| 5. Create signed conventional commit using the host configured git identity: | ||
| git commit -S -m "fix(scope): concise subject" | ||
| 6. Run CodeRabbit review on committed changes: | ||
| coderabbit review --agent --committed --base-commit $(git merge-base HEAD origin/main) | ||
| If CodeRabbit reports actionable findings, fix them, revalidate, and amend: | ||
| git commit -S --amend --no-edit | ||
| 7. Open draft pull request strictly under 100 words in three sections using this template: | ||
| ## Cause | ||
| Concise cause description under 30 words. | ||
|
|
||
| ## Fix | ||
| Concise fix description under 30 words. | ||
|
|
||
| ## Test evidence | ||
| Validation commands run and test output summary. | ||
| devsy-fingerprint: <fingerprint> | ||
| 8. Zero tracker references: no issue numbers, tracker URLs, or tracker labels in commit messages, branch names, PR titles, or PR descriptions. | ||
| 9. Populate required step outputs: pr_url and commit_sha. | ||
| 10. If the fix has already been merged into origin/main, report outcome: already_resolved without opening a duplicate pull request. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| title: Fix Worker | ||
| readonly: false | ||
| agent: | ||
| provider: omp | ||
| model: openai-codex/gpt-6.1-terra | ||
| mode: full |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| #!/bin/sh | ||
| set -eu | ||
| exec mise run playbooks:wait-ci -- "$@" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Skip PR creation on retries.
On a retry, this instruction says to update the existing PR, but guideline 7 still says to open a draft PR. If the worker follows both, PR creation can fail because the branch already has an open PR. Make guideline 7 apply only when no PR exists.