Skip to content

feat: pluggable triage Verifier seam (false-positive reduction, engine / Plan A)#7

Merged
velzepooz merged 8 commits into
mainfrom
feat/triage-verifier-engine
May 29, 2026
Merged

feat: pluggable triage Verifier seam (false-positive reduction, engine / Plan A)#7
velzepooz merged 8 commits into
mainfrom
feat/triage-verifier-engine

Conversation

@velzepooz

Copy link
Copy Markdown
Collaborator

Summary

Adds a pluggable Verifier seam to the detection engine so findings can be triaged — confidently-benign findings (e.g. a curl | bash inside a fenced doc example) are demoted out of axis grading to cut false positives — without breaking the deterministic engine. Engine-only (Plan A); the LLM-backed verifier, verdict cache, tier gating, UI, and SP-7 benchmark live in skilltrust (Plan B).

  • New pkg/triage: Verifier interface, Verdict/VerdictKey, inert NoopVerifier (default), ScriptedVerifier test double.
  • model.Finding gains optional Triage *TriageVerdict (+ IsSuppressed(), TriageDemoteThreshold = 0.85); nil-pointer omitempty ⇒ un-triaged scans marshal byte-identically.
  • grade.Grade excludes suppressed findings from the worst-severity pool; notes (N suppressed by triage) when a pool empties. Caps table / CanonicalMetadata() untouched — registry checksum unchanged (f1dcffd63faabeb3 on both branches).
  • scanner.Options gains Verifier + TriageTimeout; applyTriage runs after the deterministic sort and before grading. Demote-only + fail-safe: on verifier error / deadline / ctx-cancel / unmatched verdict, findings are stamped unavailable and never suppressed (grade stays ≥ deterministic floor). Deterministic file iteration (sorted paths).
  • SchemaVersion bumped 1.21.3.

Design: docs/specs/2026-05-29-llm-triage-verifier-design.md · Plan: docs/plans/2026-05-29-llm-triage-verifier-engine.md

Invariants (verified in final review)

  1. No verifier ⇒ byte-identical to before (checksum + omitempty JSON + green e2e). 2. Checksum stable. 3. Demote-only, fail-safe. 4. Deterministic with a verifier. 5. No model→triage import cycle. 6. Seam sufficient for the Plan-B LLM verifier with no engine change. 7. Schema 1.3 consistent.

Test Plan

  • go test ./... -p 1 — 660 passed, 12 packages
  • go vet ./... clean; gofmt clean
  • Registry checksum identical pre/post (DefaultRegistry().Checksum())
  • Fail-safe paths covered (error, cancelled ctx, unmatched verdict → not suppressed)
  • Re-pin expectedChecksum ldflag at next release (schema/checksum unchanged here, but confirm at tag)

@velzepooz
velzepooz merged commit 2a0320a into main May 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant