Skip to content

Stop requiring an exact item type nobody has verified - #32

Merged
cabibbz merged 1 commit into
mainfrom
claude/pwa-chatgpt-model-selection-o38216
Jul 27, 2026
Merged

cabibbz merged 1 commit into
mainfrom
claude/pwa-chatgpt-model-selection-o38216

Conversation

@cabibbz

@cabibbz cabibbz commented Jul 27, 2026

Copy link
Copy Markdown
Owner

A contradiction inside this repository

sandbox/worker.mjs permitted a search only when the item type matched ^(web_search|web_search_request|web_fetch|browser_search)$.

tests/core/research.test.ts asserts Codex emits web_search_call.

Anchored, web_search_call does not match. So under the guard on main right now, a real search does not merely go uncounted — it terminates the run as a tool attempt. The allowance was written so narrowly that it excluded the thing it exists to permit. I wrote both files and never compared them.

The deeper fault

The design required knowing Codex's exact event vocabulary, and I don't have it. Four consecutive root causes for this problem were inferred from silence and at least two were wrong. Continuing to guess is the actual bug.

So the run now reports rather than being assumed:

Recorded Answers
Every distinct item and event kind emitted What does a real run actually look like?
What stopped the run, named Was this a genuine isolation breach or an item type I failed to anticipate?
Codex's stderr Did Codex say why?

All three are retained when a researched run records no search, and the phone prints them under the research trace:

Research — No search ran
The run was stopped by item type "web_search_call".

This run emitted: event:thread.started, item:agent_reasoning, item:web_search_call.

That is a diagnosis. Every previous answer was a guess.

Matching is now on a substring, so both spellings work, while command_execution, apply_patch, mcp_tool_call, file_change, list_dir, and an unknown future tool all still terminate the run — verified by replay across 15 event classes.

Tests

  • Both web_search and web_search_call are recognised; command_execution and apply_patch are not
  • A stopped run names its blocker; a quiet run lists its vocabulary; all three signals travel together; nothing to report stays undefined rather than an empty box; output is bounded

107 unit tests (106 pass / 1 Windows-only skip), mocked e2e, lint, build clean.

What I am claiming

Not that this fixes it. Only this: after deploying, one researched review will print what the run emitted and what stopped it, in Codex's own vocabulary. That is the evidence I have been missing for five rounds, and it ends the guessing either way.


Generated by Claude Code

The guard matched the search item type against an anchored list
containing `web_search`. This repository's own test asserts Codex
emits `web_search_call`. Under that guard a real search does not
merely go uncounted, it terminates the run as a tool attempt: the
allowance was written so narrowly that it excluded the thing it
exists to permit.

The deeper fault is that the design needed the exact vocabulary and
nobody had it. Matching is now on a substring, and more importantly
the run reports what it saw rather than being assumed:

- every distinct item and event kind the run emitted
- what stopped the run, named, when the guard stopped it
- what Codex wrote on stderr

All three are kept for a researched run that recorded no search, and
the phone prints them under the research trace. Four consecutive root
causes for this were inferred from silence and at least two were
wrong. One real run now answers the question instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxxEQsJEXq9hZmjpNH5PxY
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
sol-review-gate Building Building Preview, Comment Jul 27, 2026 2:16pm

Request Review

@cabibbz
cabibbz merged commit 9ac15df into main Jul 27, 2026
2 of 3 checks passed

This branch was successfully deployed

1 active deployment
Preview — af7c2ff6 Deployed Jul 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants