FEAT-066: scry-mcp — analyze + query as agent tools, verify structurally absent - #190
Merged
Conversation
…y structurally absent New publishable crate crates/scry-mcp (scry-sai-mcp): an MCP server over newline-delimited JSON-RPC 2.0 on stdio (initialize / tools/list / tools/call), hand-rolled on serde_json — deliberately no MCP SDK dependency (nothing new to clear cargo-deny). Modeled on scry-viz: a plain std host tool whose only analyzer dependency is scry-sai-core. Tools (the full v3.3.0 surface): - analyze: module path (.wasm/.wat) -> compact structured summary — advisory counts by class and code, trap verdicts (proven-safe / potential-trap), gap counts. Never HTML, never a multi-MB dump (AC#1). - query: AnalysisResult::query (FEAT-067) over MCP — class / code / func_index / op / gap_kind filters, ANDed; matches carry the REQ-020 stable obligation identity + honesty flags; limit-capped with an exact total_matches. AC#2 is structural (DD-022 family): `verify` is ABSENT from the tools/list payload itself, and the test asserts against that actual payload — not documentation. The deferral is measured, not cautious: REQ-021 found FEAT-065's verify_against yields discharged=0 with every verdict degrading to `uncertain` on real (stripped) inputs; exposing that over MCP would put an always-uncertain verdict directly into an agent's tool loop. The tool follows FEAT-065 into v3.4.0. Verification: 10 tests, each written RED-first against a stub (all 10 observed failing), then GREEN; 5 mutants (verify added to the tool list, by_class count inverted, query filters ignored, limit cap dropped, notifications answered) each applied at exactly 1 site, each compiled with 0 errors, each killed by exactly the targeted test. End-to-end stdio smoke of the binary passed. check-gate-coverage.py negative control observed red with the crate missing from a gate. Wiring (all four places): workspace members + default-members; BOTH cargo test and cargo clippy lists in ci.yml (check-gate-coverage.py: 13 crates in both gates); scripts/publish.rs after scry-sai-core (leaf-before-core); no new CI job so required-checks.txt untouched. README + claims.yaml crate count moved 12 -> 13 in lockstep (CRATES-13). Known mid-cycle condition (pre-existing pattern, not introduced here): cargo package verify of scry-sai-mcp fails against crates.io scry-sai-core 3.2.7, which predates the Query API — identical to scry-sai-viz on main (uses FEAT-065 types absent from 3.2.7). Resolved at release by the version bump + leaf-first publish order. Refs: FEAT-066 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc
Reviewed the delegated implementation rather than accepting its report, and independently re-ran the two load-bearing claims. AC2's structural guard WORKS -- but my first mutant did not prove it. Appending a duplicate "name" line inside the query object APPLIED (count asserted) and COMPILED (0 errors) and changed nothing, because JSON keeps the last key, so no third tool ever appeared and the test passed. Mutating properly -- a whole third tool object appended to the json! array -- turns it red with left: ["analyze", "query", "verify"] So the mutation ladder has a third rung: APPLIED, then COMPILED, then ACTUALLY CREATES THE CONDITION. Only the third proves anything, and all three look alike in the output. Stopping at rung two would have had me report a correct test as ineffective. RELEASE-MACHINERY NOTE, verified and not introduced here: `cargo package -p scry-sai-mcp` fails verify against crates.io scry-sai-core 3.2.7, which predates the Query API. Confirmed `scry-sai-viz` ON MAIN fails identically (it uses FEAT-065's VerifyReport, also absent from 3.2.7). Pre-existing mid-cycle condition affecting any crate consuming an unreleased core API; it resolves at release via the version bump plus leaf-before-core publish order. The consequence worth writing down: `cargo package` is NOT a valid pre-release check mid-cycle. It fails for a correct tree, so a release runner reaching for it as a smoke test gets a red that means nothing. tests=0 clippy=0 fmt=0 rivet=0 claim-check=0 gate-coverage=0 drift-gate=0. Refs: FEAT-066 Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
📐 rivet artifact deltaPR: #190 Base SHA: Validationhead — `rivet validate` resultbase — `rivet validate` result (for comparison)Artifact stats
full stats — headDiff (base → head)AADL model — headPosted by the |
avrabe
added a commit
that referenced
this pull request
Aug 28, 2026
…191) FEAT-066 -> accepted. Merged in #190 with 13/13 CI green; its 10 tests re-run on main, gate-coverage reports 13 publishable crates in BOTH gates. Both ACs met, and AC2's structural guarantee was mutation-verified independently: appending a third tool object turns it red with ["analyze", "query", "verify"]. BUT PROMOTING IT ALONE WOULD HAVE DELETED WORK FROM THE PLAN. FEAT-066's title promises `analyze / query / verify`; its description defers `verify` to v3.4.0; and NOTHING carried the deferred half -- grepping every artifact for MCP returns FEAT-066 and nothing else. Mark it accepted and the verify tool stops existing anywhere in the plan. That is exactly scry#157/#160: an implementation with no artifact, invisible to the release plan, resurfacing later as a surprise. Here it would have been the inverse -- a plan item quietly evaporating on promotion. So FEAT-094 is filed first, in v3.4.0, `depends-on REQ-021`. Its deferral is measured, not cautious: REQ-021 measured `discharged` = 0 on every real commit pair, gate failing on all of them, ~47% uncertain, and `discharged` unreachable by construction on a stripped module. An always- uncertain verdict in an agent's tool loop is worse than an absent tool -- an absent tool is a fact an agent plans around; a useless one is noise it must learn to ignore. FEAT-094's AC3 is the part worth keeping: when `verify` is eventually added, FEAT-066's AC2 test must be UPDATED, not deleted. The structural guarantee moves from "verify is absent" to "verify is present and its verdict set is complete". A guard removed is a guard that stops being checked. v3.3.0 accepted 7/proposed 3; v3.4.0 accepted 9/proposed 3. rivet=0 claim-check=0 fmt=0 drift-gate=0 gate-coverage=0. Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New publishable crate
crates/scry-mcp(scry-sai-mcp), modelled onscry-viz: lib +[[bin]]stdio loop. Hand-rolled JSON-RPC 2.0 on the existingserde_jsonworkspacedep — no MCP SDK, no new external dependency to clear
cargo-deny.analyzereturns counts only (by class, by code, trap verdicts, gaps by kind) — neverHTML, never a dump.
querywraps FEAT-067's filter withtotal_matchesand a limit cap.AC2 is the point of this PR
Same family as DD-022: a deferral a consumer can rely on must be structural. And the
reason is now measured, not cautious — REQ-021 found
dischargedis 0 on real inputs andevery verdict degrades to
uncertain. Exposing that over MCP puts an always-uncertainverdict directly into an agent's tool loop. That rationale lives in the
tool_definitions()doc comment, so nobody "helpfully" adds it later.I nearly reported a good test as weak
My first independent mutant — a duplicate
"name"line inside the query object —applied (count asserted) and compiled (0 errors) and the test still passed. It
changed nothing: JSON keeps the last key, so no third tool ever appeared.
Mutating properly (a whole third tool object appended to the
json!array) turns it red:The mutation ladder has three rungs: applied → compiled → actually creates the
condition. Only the third proves anything, and all three look identical in the output.
The agent's own five mutants were each applied-asserted, compiled, and killed exactly one
test — including the AC2 one.
Release-machinery note — verified, not introduced here
cargo package -p scry-sai-mcpfails verify against crates.ioscry-sai-core3.2.7,which predates the
QueryAPI.scry-sai-vizonmainfails identically (it usesFEAT-065's
VerifyReport, also absent from 3.2.7).So: pre-existing mid-cycle condition for any crate consuming an unreleased core API,
resolved at release by the version bump + leaf-before-core publish order.
cargo packageis not a valid pre-release check mid-cycle — it fails for a correcttree, so a release runner reaching for it gets a red that means nothing.
Wiring — all four places
Cargo.tomlmembers and default-members ·ci.ymlboth test and clippy blocks(
check-gate-coverage.py→ "OK — 13 publishable crates in BOTH gates", negative controlrun) ·
scripts/publish.rsleaf-before-core ·required-checks.txtuntouched (no new job).publish = trueforced a claim coupling: README "12 pure" → "13 pure" andclaims.yamlCRATES-12 → CRATES-13, updated in lockstep — claim-check was observed red between the
publish.rs and README edits, then 7/7.
tests=0 clippy=0 fmt=0 rivet=0 claim-check=0 gate-coverage=0 drift-gate=0🤖 Generated with Claude Code
https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc