Skip to content

feat(verification): independently reproduce and rescore findings - #1182

Draft
kusonooyasumi wants to merge 2 commits into
usestrix:mainfrom
kusonooyasumi:feat/vulnerability-verification-loop
Draft

feat(verification): independently reproduce and rescore findings#1182
kusonooyasumi wants to merge 2 commits into
usestrix:mainfrom
kusonooyasumi:feat/vulnerability-verification-loop

Conversation

@kusonooyasumi

Copy link
Copy Markdown
Contributor

Summary

  • add opt-in, pre-persistence verification that runs a fresh restricted agent for every candidate finding
  • require the verifier to create and execute an independent PoC before confirming dynamic findings and reachable dependency CVEs
  • preserve non-exercisable dependency advisories, and perform evidence-based confidence/CVSS review when reproduction fails
  • persist verification attempts, evidence, and original/final scoring across Markdown, JSON, SARIF, PDF, and the local viewer
  • add --verify-findings for one run, alongside persistent STRIX_VERIFY_FINDINGS configuration

Safety and lifecycle

  • use an explicit verifier tool allowlist with no reporting, graph, threat-model, coverage, or MCP mutation tools
  • accept verdicts only through the validated terminal tool after a qualifying shell or HTTP replay action
  • bound attempts, turns, time, and shared budget usage; keep verifier credentials request-local
  • serialize final dedupe/persistence so concurrent verification cannot file duplicate reports
  • settle background imports before loading report/agent packages to avoid partially initialized SDK modules

The verifier intentionally reuses the authorized scan sandbox and proxy so it can reproduce the original behavior. A separate disposable verifier container is not part of this draft.

Testing

  • uv run pytest (1191 passed)
  • Ruff and mypy pass
  • isolated pyright passes for the verifier modules
  • viewer production build passes
  • exact startup import sequence passes in 50 fresh processes
  • strix --help exposes --verify-findings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant