Skip to content

test(pr): cover draft readiness with existing human feedback - #300

Merged
jongio merged 1 commit into
mainfrom
jongio-draft-pr-human-feedback
Nov 18, 2026
Merged

jongio merged 1 commit into
mainfrom
jongio-draft-pr-human-feedback

Conversation

@jongio

@jongio jongio commented Nov 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add exactly one synthetic regression stimulus for making an approved draft PR ready while known human review feedback remains and bot feedback is settled.
  • Require handling known human findings, real fixing-commit replies and resolution of addressed threads, preservation of genuine open human questions, and accurate outstanding-work reporting without inferring merge consent or waiting for reviews that start only after readiness.
  • Synchronize the directly coupled expected-case array with one entry. Existing cases, assertions, guards, weights, SKILL.md, and watcher code are unchanged.

Validation

  • Passed: node --test --test-concurrency=1 --test-name-pattern="eval spec covers private repo reviewer rules" skills/pr/test/skill.test.mjs (1 test; case inventory, mandatory guards, decisive weights).
  • Passed: node --test --test-concurrency=1 skills/pr/test/eval-gates.test.mjs (10 tests).
  • Passed: .github/tools/vally/node_modules/.bin/vally lint --eval-spec skills/pr/evals/pr/eval.yaml --strict.
  • Passed: git diff --check.
  • Direct diff, documentation, and light safety review found no issues. No runtime source changed, so build and runtime coverage are not applicable.
  • No model-backed, paid, or live eval was run; semantic behavior remains unmeasured by this static/deterministic verification.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the added PR-readiness regression case.

No open issues 🎉

What changed in this PR

Adds a synthetic regression case to the PR skill’s evaluation suite for making an approved draft PR ready while existing human feedback remains unresolved.

Changes:

  • Checks that plans address known human findings, preserve open questions, and report outstanding work without assuming merge consent or waiting for post-readiness reviews.
  • Adds the case to the expected inventory; existing cases, shared guards, scoring weights, and runtime code remain unchanged.
File Description
skills/​pr/​test/​skill.test.mjs Adds the new scenario to the expected evaluation-case list.
skills/​pr/​evals/​pr/​eval.yaml Defines the scenario, behavioral rubric, and execution guards.

Review effort: Balanced

@jongio
jongio merged commit 8e53a67 into main Nov 18, 2026
8 checks passed
@jongio
jongio deleted the jongio-draft-pr-human-feedback branch November 18, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants