Problem
A code review can make a falsifiable runtime or compatibility claim even though an earlier mandatory CI job already executed the exact changed path successfully on the same source revision. When review runs in a separate pipeline or job, the toolkit currently supplies immutable repository evidence but no trusted, revision-bound outcome from sibling or earlier pipelines.
The reviewer must then infer behavior from source and model knowledge alone. This is particularly weak for SQL dialect, generated-code, compiler, integration, and platform claims where a successful targeted job is materially stronger than a static guess.
This is not a request to treat "pipeline green" as proof that all code is correct. It is a request to expose narrowly scoped execution evidence with explicit provenance and limitations.
Synthetic scenario
- Revision H changes a functional-test helper to execute a database expression.
- Mandatory job functional-tests for revision H runs the complete test package and succeeds.
- A later review-only pipeline for the same revision H has no dependency edge to that job and receives no result metadata.
- OCR reports that the expression is syntactically invalid and would make the same functional suite fail.
The already-observed execution does not prove every semantic property, but it directly contradicts the claimed unconditional syntax failure on the executed path.
Proposed behavior
Introduce an optional provider-neutral evidence record for verified CI outcomes, populated by a forge adapter before OCR starts. A candidate contract could include:
{
"kind": "review.ci_outcome",
"revision": "<exact immutable head>",
"check": "functional-tests",
"status": "passed",
"required": true,
"scope": {
"mode": "declared",
"paths": ["tests/**"]
},
"source": "forge-api",
"completed_at": "<bounded timestamp>"
}
The exact schema should distinguish:
- exact revision identity;
- required/advisory/skipped/canceled/failed/unknown status;
- provider-declared check identity;
- trustworthy scope versus unknown scope;
- pipeline/job origin and freshness without exposing private URLs or logs;
- whether the result came from the current pipeline or another pipeline for the same revision.
Expose records through the existing read-only MCP and include only compact counts/status hints in the bootstrap.
Trust and safety constraints
- Accept only forge API data authenticated for the selected project and bound to the exact immutable head revision.
- Never accept repository-authored status text as authoritative execution evidence.
- Do not download or send job logs by default.
- Do not expose project names, private URLs, runner names, user identities, raw IDs, variables, artifacts, or failure traces to the model/public summary.
- Normalize provider-specific statuses into a closed schema and retain unknown values as unknown.
- A passing job proves only that its declared/known scope completed; it does not prove absence of unrelated defects.
- A skipped, canceled, allowed-to-fail, stale, different-revision, or unknown-scope job must not be presented as successful validation.
- Do not make CI data an authorization channel for posting, approval, merge, or suppression.
Acceptance criteria
Non-goals
- Arbitrary CI log ingestion.
- Executing repository tests from the evidence collector.
- Treating all green checks as proof of correctness.
- Provider-specific policy embedded in review prompts.
- Changing pipeline gating or allow-failure behavior.
Problem
A code review can make a falsifiable runtime or compatibility claim even though an earlier mandatory CI job already executed the exact changed path successfully on the same source revision. When review runs in a separate pipeline or job, the toolkit currently supplies immutable repository evidence but no trusted, revision-bound outcome from sibling or earlier pipelines.
The reviewer must then infer behavior from source and model knowledge alone. This is particularly weak for SQL dialect, generated-code, compiler, integration, and platform claims where a successful targeted job is materially stronger than a static guess.
This is not a request to treat "pipeline green" as proof that all code is correct. It is a request to expose narrowly scoped execution evidence with explicit provenance and limitations.
Synthetic scenario
The already-observed execution does not prove every semantic property, but it directly contradicts the claimed unconditional syntax failure on the executed path.
Proposed behavior
Introduce an optional provider-neutral evidence record for verified CI outcomes, populated by a forge adapter before OCR starts. A candidate contract could include:
{ "kind": "review.ci_outcome", "revision": "<exact immutable head>", "check": "functional-tests", "status": "passed", "required": true, "scope": { "mode": "declared", "paths": ["tests/**"] }, "source": "forge-api", "completed_at": "<bounded timestamp>" }The exact schema should distinguish:
Expose records through the existing read-only MCP and include only compact counts/status hints in the bootstrap.
Trust and safety constraints
Acceptance criteria
Non-goals