Skip to content

Take citation from a required field, not from prose - #39

Merged
cabibbz merged 1 commit into
mainfrom
claude/pwa-chatgpt-model-selection-o38216
Jul 29, 2026
Merged

cabibbz merged 1 commit into
mainfrom
claude/pwa-chatgpt-model-selection-o38216

Conversation

@cabibbz

@cabibbz cabibbz commented Jul 29, 2026

Copy link
Copy Markdown
Owner

You're right — I never fixed it. The last change asked the reviewer politely to cite paths, and this run is proof either the ask didn't deploy or the reviewer ignored it. Prompts are hopeful; schema fields are gates. This is the gate version.

Structural change: filesReferenced is required

The output schema (both JSON and Zod) now requires a filesReferenced: string[] field on every review. The reviewer commits to which attached paths it relied on — empty is a commitment that none bore on the decision, not something inferred from what the prose happened not to name.

The count comes from that field, not from prose

fileEvidence no longer text-searches. It reads the reviewer's own list, ignores duplicates, normalises leading ./, and shows exactly what the reviewer committed to. The old heuristic could have missed a citation the reviewer wrote as overlay.cpp in a code fence, and it couldn't distinguish "partial citation" from "none".

The gate rejects fabrication

A path in filesReferenced that was not among the packet's attachments is rejected as GATE_FABRICATED_PATH before release. The reviewer cannot claim to have read files it wasn't given. Backwards-safe: older packets without a recorded attachment list are not subjected to the check retroactively.

What every layer now does

Layer What changed
sandbox/review-schema.json filesReferenced required
lib/types.ts (Zod) Same, both review and opaque variants
sandbox/*-policy.md Explicit paragraph: the field is required, the phone counts from it, a fabrication rejects the review
lib/gate.ts Renders under ATTACHED FILES USED:, merges with attribution, rejects fabricated paths
lib/sandbox-runtime.ts Passes attached paths to the gate; the mock references real attachments so the whole pipe is exercised in tests
lib/research.ts fileEvidence takes the reviewer's list, not review text
components/dashboard/review-parsing.ts Parses the new section
components/dashboard.tsx Card fed from the structured field, with raw-JSON fallback for the Bob-Regress case

Tests

  • Gate refuses an unattached path (with the leading-./ allowance verified)
  • Gate skips the check when no attachment list was recorded, so old packets still release
  • fileEvidence takes only paths that were actually attached, handles duplicates, normalises ./
  • Every policy states the field is required and warns fabrication rejects the review
  • e2e: the mock references its real attachments, the review renders ATTACHED FILES USED:, that section survives onto the answered job
  • Schema field-order test updated
  • Browser at 390×844: FILES READ | 2 of 2 (green), card lists the referenced files

122 tests (121 pass / 1 Windows-only skip), lint, build, e2e all clean.

Honest limits

Older reviews in your history keep their existing text — the card will read Bob-Regress as no field found. Any reviewer that returns a review missing filesReferenced will now fail schema parse and be blocked with GATE_INVALID_SCHEMA — that's stricter than before, and intentional: it's what makes this a gate rather than an ask.


Generated by Claude Code

53 files were attached and none appeared in the review. The last
change asked the reviewer politely to cite the paths, and that ask
either did not deploy or did not stick. Prompts are hopeful; schema
fields are gates.

filesReferenced is now a required array in every review. The reviewer
commits to which attached paths it relied on. An empty array is an
explicit commitment that none bore on the decision; it is no longer
inferred from what the review happened not to name. The phone counts
from that field, not from a text search over prose.

A path in filesReferenced that was not among the packet's attachments
is a fabrication and rejects the review with GATE_FABRICATED_PATH.
The reviewer cannot claim to have read files it was not given, and
belt and suspenders, the phone ignores any path that was not attached.

The dashboard reads only the structured field. The previous heuristic
missed a citation the reviewer wrote as `overlay.cpp` in a code fence
and could not distinguish a partial citation from none, so the count
it printed was not the count the reviewer had made.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxxEQsJEXq9hZmjpNH5PxY
@vercel

vercel Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
sol-review-gate Ready Ready Preview Jul 29, 2026 7:57pm

Request Review

@cabibbz
cabibbz merged commit 5c34c77 into main Jul 29, 2026
3 checks passed

This branch was successfully deployed

1 active deployment
Preview — 4d62a4a9 Deployed Jul 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants