Skip to content

feat(pr-triage): add optional Jev pre-filter wave before parallel review - #9

Merged
griffinwork40 merged 1 commit into
mainfrom
feat/pr-triage-jev-prefilter
Sep 27, 2026
Merged

griffinwork40 merged 1 commit into
mainfrom
feat/pr-triage-jev-prefilter

Conversation

@griffinwork40

Copy link
Copy Markdown
Owner

What

Adds an optional Wave 0.5 Jev pre-filter to /pr-triage. Before the parallel review wave, Jev (TypeSafe AI's judgment engine, via the jev MCP server) reads each PR's diff server-side and answers three narrow questions:

Question Type Used for
trivial: docs, comments, formatting, lockfiles or dep versions only? check, yes at >= 0.85 Light route
ready: finished work with no WIP markers or placeholder TODOs? check, no at <= 0.2 Deferred route
risk: how sensitive is the touched code (0-3)? score Light gate (< 1.0) and review order

Routing:

  • Light: trivial and risk < 1.0, so /review --light on sonnet.
  • Deferred: GitHub draft or ready = no (never for PRs named in --prs), so not reviewed and listed as SKIP. The operator can reply include <N> at the triage gate.
  • Full: everything else, unchanged, dispatched in descending risk order.

Safety properties

  • Jev never assigns GREEN or BLOCKED. Buckets still come only from /review, and every merge keeps its gate.
  • --auto-merge does not cover Light-routed PRs; each needs individual confirmation.
  • Fail open: tool missing, path refused, call error, or per-item error (for example an oversized diff) all fall back to the Full route. The skill never retries by pasting diffs as text, since that would pull them into context.
  • Data egress: on by default only for public repos. Private repos need --prefilter, because diffs go to TypeSafe's API. --no-prefilter restores the old behavior exactly.
  • Diff files are written to .afk/tmp/pr-triage/ under the working directory (the Jev server only reads below its start directory) and deleted at the end.

Verification

Ran Wave 0.5 step by step against the 3 open PRs on the public griffinwork40/agent-afk repo (jev-1.13.0, one jev_triage call, ~0.36s, 9,149 input tokens):

PR Content P(trivial) P(ready) Risk Route
#2280 docs-only fix 0.96 0.95 0.01 Light
#2279 code change 0.01 0.95 0.88 Full
#2217 GitHub draft, CI workflow using provider secrets 0.01 0.23 2.94 Deferred (draft)

#2217 only reached P(ready) = 0.23, just above the 0.2 defer cutoff. That is why the isDraft flag was added as a deterministic defer signal, alongside Jev's judgment. The frontmatter parses as YAML. This is a prompt-only change with no code or tests in this repo.

Not verified

  • A full end-to-end /pr-triage session using the new wave (the reviews themselves were not re-run).
  • Jev's scores vary between runs by roughly 0.13 near 0.65-0.75. The thresholds were set well away from that band (0.85 and 0.2), and a wrong call only changes review depth, never a merge.

Adds Wave 0.5: when the jev MCP server is connected, Jev reads each PR's
diff server-side (never entering agent context) and answers three narrow
questions (trivial? ready? risk 0-3). Trivial low-risk PRs get a light
review on sonnet, drafts and unfinished PRs are deferred to SKIP (the
operator can pull them back with "include <N>"), and the rest get the
unchanged full review, dispatched riskiest first.

Jev never assigns GREEN or BLOCKED and any Jev error falls back to a full
review. On by default for public repos, off for private repos (diffs are
sent to TypeSafe's API); --prefilter / --no-prefilter override.
--auto-merge does not cover light-reviewed PRs.
@griffinwork40

Copy link
Copy Markdown
Owner Author

Mirrored in the standalone repo: griffinwork40/pr-triage#2

griffinwork40 added a commit to griffinwork40/pr-triage that referenced this pull request Sep 27, 2026
Adds Wave 0.5: when the jev MCP server is connected, Jev reads each PR's
diff server-side (never entering agent context) and answers three narrow
questions (trivial? ready? risk 0-3). Trivial low-risk PRs get a light
review on sonnet, drafts and unfinished PRs are deferred to SKIP (the
operator can pull them back with "include <N>"), and the rest get the
unchanged full review, dispatched riskiest first.

Jev never assigns GREEN or BLOCKED and any Jev error falls back to a full
review. On by default for public repos, off for private repos (diffs are
sent to TypeSafe's API); --prefilter / --no-prefilter override.
--auto-merge does not cover light-reviewed PRs.

Also refreshes README: documents the new wave and flags, and drops stale
FOLLOW-UP bucket and compose mentions removed in earlier commits.
Mirrors griffinwork40/agent-framework#9.
@griffinwork40
griffinwork40 merged commit bbcb865 into main Sep 27, 2026
1 check failed
@griffinwork40
griffinwork40 deleted the feat/pr-triage-jev-prefilter branch September 27, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant