Skip to content

docs(runbooks): add a pull-request runbook (#668) - #670

Merged
ss-o merged 7 commits into
mainfrom
feature-668-pull-requests
Sep 26, 2026
Merged

ss-o merged 7 commits into
mainfrom
feature-668-pull-requests

Conversation

@ss-o

@ss-o ss-o commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

Summary

Refs #668

Instruction impact review

  1. Kind: shared policy procedure, a canonical-detail runbook. It adds no enforcement.
  2. Recipients: every runtime that follows AGENTS.md routing, in every z-shell repository, when the task is pull-request, code-review or github-operations (file_patterns: ["**"], required: true).
  3. Owner: runbooks/ in z-shell/.github, as ADR-0032 places organization procedures. Still correct.
  4. Duplication or contradiction: it restates the AGENTS.md done gate, ADR-0003, ADR-0022, ADR-0026 and branch-protection.md. It says the ADRs and commit-lint.yml win over it; it sets no precedence against AGENTS.md or branch-protection.md, and none of its rules contradicts them except as below. Interim rules go beyond ADR-0026 and the AGENTS.md done gate, all recorded on decision: amend ADR-0026 for maintainer-elected fallback reviews #664: the up-front fallback election, a push voiding a fallback review (so a fallback must be on the merged head), and the review of record where the repository or the base branch has no Copilot review configured. The ADR-0026 amendment and a matching AGENTS.md change reconcile them.
  5. Manifest routes: one route added, runbook-pull-requests in .github/instruction-surfaces.json. No route is changed or removed.
  6. Delivery without optional mechanics: yes. The runbook is reached through the manifest routing that AGENTS.md makes mandatory for every runtime, not through a hook or skill.
  7. Generated output and limits: this pull request changes no generated file. validate-agent-policy.py, org-routing.py validate and the validator's unit tests pass on a clean export of the head.

Verification

  • python3 -m unittest scripts/test_validate_agent_policy.py (OK) and scripts/validate-agent-policy.py (passed), run on a copy containing only tracked files. The local checkout carries the workspace's generated .claude/rules/ws-workspace.md, which the validator rejects and CI never sees.
  • trunk check --no-fix on the two changed files: no issues.
  • Every relative link resolves (10 targets: 6 ADRs and 4 runbooks). The cited limits were checked against commit-lint.yml: the 72-character description, the issue reference, and the branch pattern.
  • Prose is one paragraph per line; no U+2014.

Agent handoff

Branch naming, opening, the ADR-0026 review gate including the
maintainer-elected fallback decided on #664, merge, and post-merge steps,
in one place for humans and agents, with each rule pointing at the ADR
or workflow that owns it. Declared in the instruction manifest.

Refs #668
@ss-o
ss-o requested a review from a team as a code owner September 26, 2026 18:46
Require a maintainer decision before the not-registered fallback, limit
push voiding to fallback reviews and restore the two-round cap, keep merge
commits for zi hotfix synchronization, name #664 as an interim exception to
ADR precedence, cover zi hotfix bases and issue closing at promotion, and
add the automation-only diff exemption from ADR-0026 decision 4.

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: Copilot request not registered on f0a25d5

The Copilot request on this head returned an empty requested_reviewers list and no review_requested event followed; the maintainer elected this fallback. Executed under .github/skills/code-review/SKILL.md against .github/instructions/code-review-generic.instructions.md, plus documentation.instructions.md and markdown-accessibility.instructions.md, on runbooks/pull-requests.md and .github/instruction-surfaces.json.

Result: 2 IMPORTANT, 9 SUGGESTION (10 inline threads; one thread covers two points), nothing CRITICAL.

Checklist:

  • Repository contract (AGENTS.md, skill, scoped instructions) read: pass.
  • Relative links resolve (9 targets): pass.
  • Claims checked against ADR-0003, 0013, 0019, 0022, 0026, branch-protection.md, project-tracker.md, commit-lint.yml, lib/repository-classes.yml, the #664 decision, zi and F-Sy-H AGENTS.md, and the live zi rulesets (read-only): findings, see threads.
  • BRANCH_PATTERN examples against the regex: pass.
  • Internal consistency between sections: finding (line 39).
  • JSON valid and manifest entry shaped like sibling runbook entries: pass.
  • Repository validators (validate-agent-policy.py, decision-records.py --check, org-routing.py validate, validate-zsh-standard-policy.py) and unit tests: pass.
  • markdownlint and prettier: pass.
  • Heading hierarchy, descriptive links, one paragraph per line: pass.
  • Zsh, Go and workflow checks: not applicable (docs and JSON only).

Residual: the reviewer is an agent in the same session as the author, so this is a second pass under the written checklist, not an independent reader.

Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Require merge commits for every zi pull request into main, exempt declared
automation-only diffs from the merge-gate review, add the meta:no-issue
path, cite commit-lint.yml for the 72-character limit, match the branch
regex (lowercase slugs, feature/bug/hotfix slash form), point stricter
repositories at the branch-pattern input, restore the ADR-0026 decision 4
condition, name the Copilot reviewer and scope it to where configured,
describe Project 28 as automatic, and link cross-repository references.

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: maintainer elected, no Copilot request on 081938a

Executed under .github/skills/code-review/SKILL.md against .github/instructions/code-review-generic.instructions.md, plus documentation.instructions.md and markdown-accessibility.instructions.md, on runbooks/pull-requests.md and .github/instruction-surfaces.json (scope from the PR base 2a5498d).

Result: 2 IMPORTANT, 5 SUGGESTION (6 inline threads; the line 30 thread carries two points, and one point is below), nothing CRITICAL. All 11 points from review 5327237421 are fixed correctly; the Copilot scoping fix opened the line 39 gap.

Not a line finding: the pull-request body is stale. It says this can merge independently of ADR-0032 (#669), which is already merged and is this branch's base, and Verification counts 6 link targets where the file has 9. The See also list could link ADR-0032, which this runbook follows up.

Checklist:

  • Repository contract (AGENTS.md, skill, three instruction files) read: pass.
  • Claims against ADR-0003, 0013, 0019, 0022, 0026, branch-protection.md, project-tracker.md, labels.md, commit-lint.yml, lib/repository-classes.yml, the #664 decision, zi AGENTS.md, the zi rulesets and the F-Sy-H commit-lint caller (read-only): findings, see threads.
  • Live Project 28 workflows (GraphQL, read-only): finding (line 47).
  • Relative links (9 targets): pass.
  • JSON valid and manifest entry shape against sibling runbook entries: pass.
  • validate-agent-policy.py on a clean export: pass.
  • PR checks on this head: all 12 pass.
  • Heading hierarchy, descriptive links, one paragraph per line, no em dash: pass.
  • markdownlint: not run in this pass (CI and the previous pass report it clean).
  • Zsh, Go and workflow checks: not applicable.

Correction to review 5327237421: it opened with the not-registered marker, which was accurate (a Copilot request was made on f0a25d5 and did not register), but its body said the maintainer elected the fallback. The maintainer chose the fallback after the failed request, which is the not-registered case; no up-front election was made on that head.

Residual: the reviewer is an agent reviewing an agent-authored change, so this is a second pass under the written checklist, not an independent reader.

Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
Count only a review_requested event for Copilot as confirmation, demote
the empty requested_reviewers list to an early sign, and name the review
of record where Copilot is not configured: a human review or, in classes
2 to 4, a maintainer-elected fallback. Keep thread handling for exempt
automation-only diffs, allow the instructions file from z-shell/.github,
check Project 28 only when the merge closed the issue, require lowercase
slash-form slugs, and link ADR-0032.

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: maintainer elected, no Copilot request on 6ce8aaf

Executed under .github/skills/code-review/SKILL.md against .github/instructions/code-review-generic.instructions.md, plus documentation.instructions.md and markdown-accessibility.instructions.md, on runbooks/pull-requests.md and .github/instruction-surfaces.json (scope git diff 2a5498dd HEAD) and the pull-request body.

Result: 2 IMPORTANT (2 inline threads), 1 SUGGESTION (below), nothing CRITICAL. All 18 points from reviews 5327237421 and 5327289986 are fixed as their replies say, and 6ce8aaf adds no contradiction between sections.

SUGGESTION, pull-request body: runbooks/instruction-update.md asks every material instruction change to answer its impact questions in the issue or pull-request body. This pull request adds a required: true surface with file_patterns: ["**"] on the pull-request, code-review and github-operations tasks, so every review and GitHub-operations session now reads it, and neither the body nor #668 answers those questions. The body's "Each rule points at the ADR or workflow that owns it" is also untrue for the step 1 rule above.

Checklist:

  • Repository contract (AGENTS.md, skill, three instruction files) read: pass.
  • Earlier fixes (git show 081938a 6ce8aaf, all inline comments): pass.
  • Copilot timeline login (Copilot, type Bot; team events show as tsc) over recent merged pull requests: pass; repeated Copilot events per pull request: finding (line 29).
  • #664 decision, ADR-0026 decisions 1 to 4, ADR-0013, lib/repository-classes.yml: finding (line 3).
  • ADR-0003, ADR-0022, labels.md, commit-lint.yml patterns, F-Sy-H caller, zi rulesets and AGENTS.md, branch-protection.md, project-tracker.md: pass.
  • Relative links (10 targets): pass.
  • JSON, manifest entry shape, validate-agent-policy.py, org-routing.py validate, unit tests (clean export): pass.
  • PR checks on this head: all 12 pass.
  • Heading hierarchy, descriptive links, one paragraph per line, no em dash: pass.
  • Zsh, Go and workflow checks: not applicable.

Residual: the reviewer is an agent reviewing an agent-authored change, so this is a second pass under the written checklist, not an independent reader.

Comment thread runbooks/pull-requests.md Outdated
Comment thread runbooks/pull-requests.md Outdated
…gs (#668)

Count only a Copilot review_requested event created after the current
request, so an earlier round's event cannot stand in for one that did not
register. Cite the 2026-09-26 maintainer decision on #664 for the review
of record where Copilot review is not configured, and widen the interim
exception in the introduction to cover both #664 decisions.

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: maintainer elected, no Copilot request on dc9d3fa

Executed under .github/skills/code-review/SKILL.md against .github/instructions/code-review-generic.instructions.md, plus documentation.instructions.md, markdown-accessibility.instructions.md and runbooks/instruction-update.md for the impact review, on runbooks/pull-requests.md, .github/instruction-surfaces.json (scope git diff 2a5498dd HEAD) and the pull-request body.

Result: 1 IMPORTANT (inline), 1 SUGGESTION (below), nothing CRITICAL. The dc9d3fa fixes are applied as replied; the citation it added to step 1 exposed the scope gap in the inline thread.

SUGGESTION, pull-request body: two traceability claims are untrue. The Summary says each rule points at its owner or at #664, but lines 19, 20, 42 and 49 (maintainer agreement for a departure, checking claims, re-checking Closes after merge, deleting branches only with authorization) are new rules from the #668 findings with no cited owner. Question 4 of the impact review says the runbook yields to the AGENTS.md done gate and branch-protection.md, but line 3 gives precedence only to the ADRs and the workflow. Remedy: say the runbook adds new rules from #668, and state the precedence line 3 actually sets (or widen line 3).

Checklist:

  • Repository contract, skill, instruction files and instruction-update.md read: pass.
  • Base 2a5498d identical to main (API compare): pass.
  • dc9d3fa fixes (git show dc9d3fa, all earlier threads and replies): pass.
  • #664 decisions against steps 1 and 3: finding (line 29).
  • Live zi and F-Sy-H rulesets (read-only): zi main has copilot_code_review and merge-only; zi next and F-Sy-H main have no Copilot rule.
  • commit-lint.yml branch pattern and 72-character limit, meta:no-issue, PR template, ADR-0013, ADR-0026 decisions 1 to 4, branch-protection.md, zi AGENTS.md, lib/repository-classes.yml: pass.
  • Relative links (10 targets): pass.
  • JSON, manifest entry shape, validate-agent-policy.py, org-routing.py validate, both unit-test suites (clean export): pass.
  • PR checks on this head: all 12 pass.
  • Headings, descriptive links, no em dash: pass.
  • Impact review answers all 7 questions: finding (above).
  • Zsh, Go and workflow checks: not applicable.

Residual: the reviewer is an agent reviewing an agent-authored change, so this is a second pass under the written checklist, not an independent reader.

Comment thread runbooks/pull-requests.md Outdated
Decide whether Copilot review is configured from the ruleset of the pull
request's base branch, as the #664 ruling does, so a zi pull request into
next, which has no copilot_code_review rule, takes the no-Copilot path.
Link the #664 comment that records the ruling.

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: maintainer elected, no Copilot request on 1d7e3e9

Executed under .github/skills/code-review/SKILL.md against .github/instructions/code-review-generic.instructions.md, plus documentation.instructions.md, markdown-accessibility.instructions.md and runbooks/instruction-update.md for the impact review, on runbooks/pull-requests.md, .github/instruction-surfaces.json (scope git diff 2a5498dd HEAD) and the pull-request body.

Result: no findings in runbooks/pull-requests.md or .github/instruction-surfaces.json. One SUGGESTION on the pull-request body: the Agent handoff says the runbook was "revised after three fallback reviews", but four were on record before this one (5327237421, 5327289986, 5327329923, 5327367265). Remedy: correct the count.

The 1d7e3e9 change scopes step 1 to the base branch's ruleset, matching the #664 ruling of 2026-09-26; live rules confirm zi next and F-Sy-H main have no copilot_code_review rule while zi main, .github main and zsh-lint main do, each governed by a single repository ruleset.

Checklist:

  • Repository contract, skill and instruction files read: pass.
  • Diff scope (2 files) and git show 1d7e3e9: pass.
  • Relative links (10 targets): pass.
  • JSON validity, manifest entry shape, unique id and canonical_for: pass.
  • Branch pattern, 72-character limit, F-Sy-H stricter pattern, zi merge methods, hotfix and synchronization, zi issue closure, Project 28 wording: pass.
  • ADR-0026 decisions 1 to 4 and both #664 decisions against steps 1 to 7, line 3 and the merge gate: pass.
  • Copilot timeline login (Copilot, type Bot; team events separate): pass.
  • Headings, links, one paragraph per line, no em dash: pass.
  • Impact review in the body: pass, apart from the count above.
  • PR checks on this head: all 12 pass. validate-agent-policy.py and org-routing.py validate pass on a clean export (run by the author on this head).
  • Zsh, Go and workflow checks: not applicable.

Residual: the reviewer is an agent reviewing an agent-authored change, so this is a second pass under the written checklist, not an independent reader.

@ss-o
ss-o merged commit ca7f6b2 into main Sep 26, 2026
16 of 20 checks passed
@ss-o
ss-o deleted the feature-668-pull-requests branch September 26, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant