Skip to content

[Feature]: Make review-critical evidence discoverable by value and source path #160

Description

@xeonvs

Problem

The built-in evidence MCP exposes safe structured facts, but a reviewer must already know the exact evidence kind and component before it can discover a relevant value. Current list filters are limited to exact kind, delta kind, component, and ref, followed by stable-ID get.

That works for deterministic integrations, but it is weak for model-led verification. When a changed query depends on a runtime dialect, the evidence store may already contain the decisive container image and dependency records while the compact bootstrap intentionally contains only counts. A model asking "which database and version does this test use?" cannot search record values or source paths; it must guess the kind and page through broad inventories.

Native repository search remains available, but it does not replace the toolkit-owned immutable evidence channel or its provenance, completeness, and redaction contracts.

Synthetic scenario

A public synthetic repository contains:

# compose.yaml
services:
  database:
    image: mysql:8.0
// tests/database.go
const databaseImage = "mysql:8.0"

A changed Go helper adds a dialect-specific expression such as:

SELECT COALESCE(metadata, CAST('{}' AS JSON))

The evidence store correctly contains a container.image fact for mysql:8.0, but the bootstrap says only that container-image records exist. To retrieve it through MCP, the model must infer kind=container.image, potentially infer the component, and page through results. It cannot query for mysql, 8.0, compose.yaml, or database.

Proposed behavior

Add a bounded read-only discovery action or filters to ocr_toolkit_evidence that search only already-admitted, already-redacted evidence. The interface should support exact or normalized matching over safe indexed fields such as:

  • source path;
  • fact identity;
  • selected scalar value tokens such as name, version, runtime, and provider;
  • record kind/component/ref.

A possible closed-schema request:

{
  "action": "search",
  "query": "mysql 8.0",
  "fields": ["source_path", "identity", "value"],
  "ref": "head",
  "page_size": 20
}

The exact API is open to design. It must remain deterministic and bounded rather than becoming arbitrary repository grep.

Also evaluate a narrow review-critical projection for statically discoverable test/runtime facts not represented today, such as literal testcontainer image references and embedded test DDL column types. Add such extraction only where a conservative grammar, provenance, scope, bounds, and completeness semantics can be defined; do not index arbitrary source text by default.

Safety and compatibility constraints

  • Search only records already accepted by the evidence store; do not read new files, execute code, or use network access.
  • Apply redaction before indexing and again before response rendering.
  • Do not return snippets from unstructured private source.
  • Bind cursors to normalized filters/query and enforce current request/response/page budgets.
  • Treat no match as absence only when the applicable indexed scope is complete.
  • Keep existing summary, list, and get behavior compatible.
  • Do not expand the compact bootstrap with complete manifests or private values.

Acceptance criteria

  • A model can find a synthetic mysql:8.0 container fact without knowing its exact kind/component in advance.
  • Source-path lookup can find the admitted fact behind a synthetic compose or CI file.
  • Search results retain kind, ref, source path, stable ID, trust, provenance, sensitivity, and completeness context.
  • Matching and pagination are deterministic and query-bound.
  • Secrets, URL userinfo, credentials, oversized values, control characters, and redacted fields are not recoverable through the index.
  • Unsupported fields/operators, broad empty queries, malformed cursors, and response-budget overflow fail closed.
  • Existing list/get clients continue to work unchanged.
  • Installed-wheel MCP integration proves the result reaches a real OCR request without exposing raw tool arguments or responses in public summaries.

Relationship to existing work

This complements #111 rather than replacing it. #111 made at least one evidence call mandatory; this issue makes relevant admitted evidence practically discoverable after that call. It also preserves the strategy that detailed values stay in MCP instead of expanding the bootstrap.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions