Skip to content

feat(review): add MCP context profiles and evidence guidance - #611

Merged
ss-o merged 2 commits into
mainfrom
feature-610-mcp-review-context
Sep 11, 2026
Merged

feat(review): add MCP context profiles and evidence guidance#611
ss-o merged 2 commits into
mainfrom
feature-610-mcp-review-context

Conversation

@ss-o

@ss-o ss-o commented Sep 11, 2026

Copy link
Copy Markdown
Member

Reviews need linked requirements, CI evidence, and relevant documentation without assuming an installed MCP server was used. This change adds optional GitHub and GitHub-plus-documentation profiles and separates configuration, discovery, and invocation evidence in the existing health procedure.

The portable review skill now selects external context only when relevant. Hosted compatibility guidance requires explicit tool allowlists and readOnlyHint annotations, distinguishes interactive OAuth from supported hosted authentication, and preserves repository access boundaries. No custom server, dependency, credential, or settings change is included.

Closes #610. Refs #606 for the coordinated rollout. Wiki and zsh-fancy-completions will receive native installations pinned to this published source revision. Hosted settings and actual use remain unverified pending configuration and session evidence.

Instruction-impact review

  1. Existing scoped integration guidance owns hosted compatibility; the advisory skill selects context; org-review owns health evidence.
  2. All supported runtimes retain existing routes. Hosted limits apply specifically to Copilot code review.
  3. The canonical organization owners are unchanged.
  4. Interactive OAuth guidance is separated from hosted restrictions; shared review criteria are not duplicated.
  5. Existing manifest routes cover these three files; no new surface or route is required.
  6. Required health guidance carries evidence rules independently of optional skill selection. Third-party providers remain optional.
  7. Public validation and downstream generated-delivery checks passed. Hosted discovery remains unverified.

Validation

88 public agent-policy tests, the public policy and Zsh-standard validators, skill validation, and relevant formatting/whitespace checks passed. Independent review verified the conditional context and authority boundaries.

A public Cloudflare documentation-server initialization preflight returned HTTP 403. Its upstream source declares a read-only search tool, but source inspection does not establish hosted discovery or invocation. Pilot activation therefore remains pending.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Hosted-profile composition and OAuth-evidence wording corrections remain unresolved.

Pull request overview

Adds optional MCP context profiles and evidence guidance for repository reviews.

Changes:

  • Defines GitHub and GitHub-plus-documentation context profiles.
  • Adds conditional external-context retrieval.
  • Documents hosted compatibility boundaries and MCP evidence requirements.
File summaries
File Summary
runbooks/org-review.md Adds MCP health-evidence guidance.
.github/skills/code-review/SKILL.md Adds conditional context retrieval rules.
.github/instructions/mcp-plugins.instructions.md Documents hosted MCP profiles and constraints.
Review details

Suppressed comments (2)

.github/instructions/mcp-plugins.instructions.md:60

  • The phrase “interactive OAuth availability proves no hosted compatibility” reverses the evidence rule: an interactive OAuth setup says nothing about whether the hosted review supports that server. Please state that it “does not prove hosted compatibility”; otherwise reviewers may incorrectly record a supported hosted integration as incompatible.
- **Linear:** optional addition only when a linked requirement is unavailable
  in GitHub. Evaluate a restricted read-only API-key configuration if supported
  by the current server; interactive OAuth availability proves no hosted
  compatibility. Do not mirror all tracker context into every review.

.github/instructions/mcp-plugins.instructions.md:55

  • github-docs is described as documentation lookup only, so a consumer can select it without the github baseline and skip linked requirements, PR context, and CI evidence. The issue and PR scope call for a GitHub-plus-documentation profile; define this profile as the github baseline plus documentation lookup (or explicitly document the composition in the runbook).
- **`github-docs`:** add documentation lookup only for framework, API, or
  platform changes that need it, such as wiki Docusaurus or Cloudflare changes.
  Prefer official version-matched documentation; Context7 or Cloudflare's
  documentation server is optional. Check actual tools and annotations before
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI review requested due to automatic review settings September 11, 2026 14:04
@ss-o

ss-o commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Agent handoff

Status: Ready for review
Repository: z-shell/.github
Branch/PR: #611 (feature-610-mcp-review-context)
Tracker/Issue: #610, native child of #606; Project 28 In Review

Current state

  • Canonical head: 1cb4ec70b8f2755c9c5ada9195908c544c10f055.
  • Optional GitHub and GitHub-plus-documentation profiles, hosted compatibility rules, and configuration/discovery/invocation evidence are published.
  • Copilot's profile-composition and OAuth-evidence wording findings are fixed. A fresh review is requested.
  • Native wiki #915 and zsh-fancy-completions build(deps): bump dessant/lock-threads from 3.0.0 to 4.0.0 #65 installations retain source pin 0258173641bffb1c0fe736de1a5d10b68cc7dacf. The follow-up changes do not alter the canonical skill body. Repeat native installation plus repository formatting matched both delivered files byte-for-byte.

Verification

  • 88 public policy tests and public policy/Zsh-standard validators passed after the correction.
  • All 11 hosted checks pass at the current canonical head.
  • Copilot reviews provide no explicit skill or MCP invocation attribution.

Blockers

Next steps

  1. Complete current-head review and obtain authorization to merge feat(review): add MCP context profiles and evidence guidance #611, then verify the hosted-guidance anchor on main.
  2. Complete pilot review and merge wiki #915, then plugin build(deps): bump dessant/lock-threads from 3.0.0 to 4.0.0 #65 under the existing pilot merge authority, subject to branch policy.
  3. With explicit settings authority, apply the selected read-only profile in repository settings and record actual discovery and invocation evidence. Continue the wider rollout under feat(review): require repository review-skill readiness #606 only after the pilot evidence is satisfactory.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Hosted MCP discovery and invocation remain unverified, requiring final human review.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The documentation changes consistently satisfy the linked requirements and current hosted MCP constraints.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ss-o
ss-o merged commit e1f8f6c into main Sep 11, 2026
13 checks passed
@ss-o
ss-o deleted the feature-610-mcp-review-context branch September 11, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(review): add tailored MCP review context profiles

2 participants