fix(ai): a read is bounded once, and the sentence saying how to page survives it - #413
Merged
Conversation
…survives it `read_file` was capped twice. `MAX_READ_CHARS` cut a long file at 40,000 and appended the sentence naming `offset` and `limit`; `InvokePolicy.max_tool_result_chars` then cut every tool result at 20,000 and appended a bare `…[truncated]`. The second cut always landed on the first one's notice, so a model reading anything over 20,000 characters was told a file had been cut and never told how to reach the rest — the exact behaviour #402 added the range parameters to end. Measured on this tree, `strategy.py` reached a model as 38% of itself and `.lockstep/lockstep.py` as 50%. The band between the two caps was worse: the tool reported no truncation at all and the invoker cut mid-word, so nothing said the file continued. GATE-READ-1 has stated the notice clause since #402 and was green throughout, because every test discharging it called the tool runner — and the tool runner is not what hands a result to a model. A property asserted one layer below the one that delivers it is a property nobody is testing, so the tests added here run through `AiInvoker`. Two mechanisms, because either alone can drift. `_window` returns at most `MAX_READ_CHARS` INCLUDING the header and notice it appends, so nothing downstream has to cut it again; and the invoker's cap IS `MAX_READ_CHARS` rather than a number of its own, so the two cannot be edited apart. The cap rises to 64,000 — about sixteen thousand tokens, and 97% of this repository's Python files whole — because turns are the expensive axis rather than bytes: the same file read in three calls puts the same bytes in the prompt and pays two extra turns, each re-sending everything accumulated so far. `search_code`'s `skeleton` and `map` remain the answer for the six files above that, which is a choice a model can only make if the notice reaches it. What the invoker does still cut now says how much it dropped. `…[truncated]` alone cannot tell a result that was nearly whole from one that was a tenth, and gives no number to narrow the next query with. Closes #412 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
in-lockstep review — intentsucceeded · $0.0606 · 3 in / 575 out tokens
|
Contributor
in-lockstep review — performancesucceeded · $0.0486 · 3 in / 117 out tokens No findings. |
Contributor
in-lockstep review — securitysucceeded · $0.0538 · 3 in / 99 out tokens
|
Contributor
in-lockstep review — testssucceeded · $0.0611 · 3 in / 609 out tokens
|
tpouyer
added a commit
that referenced
this pull request
Sep 9, 2026
…ys which caller it means (#415) `MAX_READ_CHARS` was 64,000 because I measured this repository — 97% of its Python files fit whole. That is a fine way to pick a default and no way to pick a number nobody else can change. A repository of short modules pays for a window it never fills, on every read, on every later turn; one carrying generated clients an order of magnitude larger cannot read its own code and has no way to say so. `Workshop(max_read_chars=...)` states it, a contributed policy layer may lower it, and `ls` prints what the stack contributed. The value travels the road `max_test_runs` already walks — `InvokePolicy` to the binding site to `ToolRunnerImpl` — with `_window` at the end instead of the test counter. All three session constructors take it, not only the one that executes: a review reads files exactly as an implement does, and a cap honoured in one and not the other is two answers to one question. `max_tool_result_chars` stops being a field and becomes a property returning the read window. #413 set the two equal so they could not be edited apart, and making both settable would put that defect back in through the front door: a repository raising its window and leaving the result bound at the shipped default would be cut in half again, with nothing saying so. One number an adopter states. GATE-READ-1 has said "the cap is not negotiable by the caller" for a year without saying which caller, and the two it could mean are not alike. The caller is the MODEL, which reaches the reader through `read_file`'s `offset` and `limit` — neither of which is a byte budget — and the row now says so, because leaving it ambiguous is how a reader resolves it the wrong way: either weakening the rule that stops a model widening its own prompt, or abandoning the configuration because "the gate says no". `test_gate_read_1_a_model_cannot_widen_the_window_it_reads_through` now runs against a non-default cap, so it proves the rule rather than the constant. Closes #414 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #412.
read_filewas capped twice and the smaller cap was half the declared one, so the number in thecode said 40,000 and a model received 20,000 — and what the second cut removed was the sentence
telling it how to read the rest.
Measured on this tree before the change:
And after:
The part worth reading twice
GATE-READ-1has promised since #402 that "the truncation says how to continue — how many linesthe file has, and that
offsetandlimitreach the rest". It was green the whole time. Everytest discharging that clause called the tool runner:
The tool runner is not what hands a result to a model.
AiInvoker._dispatchsits between them, andthat is where the second cut lived. The property was asserted one layer below the one that keeps
it, so the gate could not see the defect — which is the failure mode this repository's whole gate
apparatus exists to refuse, arriving from the inside.
The new tests in
tests/in_lockstep/test_read_delivery.pydriveread_filethroughAiInvokerwith a stub provider and read the
tool_resultmessage the model is actually sent.The fix
Two mechanisms, because either alone can drift:
_windowbounds a read including what it appends to itself. The notice and the[path lines a-b of N]header are insideMAX_READ_CHARS, not on top of it, so a read resultis never longer than the one cap it has and nothing downstream needs to cut it again.
MAX_READ_CHARS, imported rather than restated, so the two cannot beedited apart the way they were.
InvokePolicyalready takesDEFAULT_TEST_RUNSfrombuiltinsthis way.
test_gate_read_1_a_read_is_never_cut_twiceasserts both halves, so neither can regress silently.The number
64,000 characters, about sixteen thousand tokens. Measured over this repository's 225 Python files:
The six above it —
cli.pyat 407,760 leading — are files nothing should read end to end;search_code'sskeletonandmapare the answer there, and a working notice is what leaves amodel able to choose them.
Turns are the expensive axis, not bytes: reading a 52,000-character file in one call or in three
puts the same bytes in the prompt, and the three-call version also pays two extra turns, each
re-sending everything accumulated so far. In the run that prompted #412, 1,566,345 cache-read
tokens were $2.35 of the $3.83.
Also
What the invoker does still cut — a delegated child's answer, an adopter's own tool — now says
how much it dropped.
…[truncated]alone cannot tell a result that was nearly whole from one thatwas a tenth, and gives a model no number to narrow the next query with.
test_gate_read_1_a_staged_file_is_readable_past_the_caphad a literal 5,000-line fixture thatstopped being over the cap when the cap rose; it now derives its size from
MAX_READ_CHARS, so thenext change to the number cannot leave it asserting truncation of a file that fits.
Objectives
O13 — a run is bounded before it starts, and two bounds that disagree mean the real one was
never declared. O7 — determinism first: what a tool returns is now decided in one place by one
rule, rather than emerging from two caps applied in sequence by layers that could not see each
other.
Verification
make checkgreen from the committed tree: 2867 passed, 6 skipped; ruff format reports 268 filesunchanged, so what is pushed is what passed.
make covgreen at 92% (floor 90%).landed first:
between-the-caps test all go red;
test_a_huge_read_is_truncated_and_says_how_to_read_the_restgo red;test_a_tool_result_the_invoker_cuts_says_how_much_it_droppedgoes red.