Skip to content

fix(ai): a read is bounded once, and the sentence saying how to page survives it - #413

Merged
tpouyer merged 1 commit into
mainfrom
fix/read-bounded-once
Sep 9, 2026
Merged

fix(ai): a read is bounded once, and the sentence saying how to page survives it#413
tpouyer merged 1 commit into
mainfrom
fix/read-bounded-once

Conversation

@tpouyer

@tpouyer tpouyer commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Closes #412.

read_file was capped twice and the smaller cap was half the declared one, so the number in the
code said 40,000 and a model received 20,000 — and what the second cut removed was the sentence
telling it how to read the rest.

Measured on this tree before the change:

MAX_READ_CHARS=40000  max_tool_result_chars=20000

.lockstep/lockstep.py: file=40080  model_saw=20013  (50%)   paging hint survived: False
strategy.py:           file=52507  model_saw=20013  (38%)   paging hint survived: False
a 30,000-char file:    the tool reported no truncation at all; the invoker cut it mid-word

And after:

MAX_READ_CHARS=64000  max_tool_result_chars=64000

.lockstep/lockstep.py: 100%   strategy.py: 100%
cli.py (407,201):      truncated, and the notice survives:
  "…[truncated; src/in_lockstep/cli.py has 8280 line(s) and 407201 chars.
    Pass `offset` (1-based) and `limit` to read any part of it]"

The part worth reading twice

GATE-READ-1 has promised since #402 that "the truncation says how to continue — how many lines
the file has, and that offset and limit reach the rest". It was green the whole time. Every
test discharging that clause called the tool runner:

answer = asyncio.run(run("builtin", "read_file", {"path": "big.txt"}))
assert "`offset`" in answer and "`limit`" in answer

The tool runner is not what hands a result to a model. AiInvoker._dispatch sits between them, and
that is where the second cut lived. The property was asserted one layer below the one that keeps
it, so the gate could not see the defect — which is the failure mode this repository's whole gate
apparatus exists to refuse, arriving from the inside.

The new tests in tests/in_lockstep/test_read_delivery.py drive read_file through AiInvoker
with a stub provider and read the tool_result message the model is actually sent.

The fix

Two mechanisms, because either alone can drift:

  1. _window bounds a read including what it appends to itself. The notice and the
    [path lines a-b of N] header are inside MAX_READ_CHARS, not on top of it, so a read result
    is never longer than the one cap it has and nothing downstream needs to cut it again.
  2. The invoker's cap IS MAX_READ_CHARS, imported rather than restated, so the two cannot be
    edited apart the way they were. InvokePolicy already takes DEFAULT_TEST_RUNS from builtins
    this way.

test_gate_read_1_a_read_is_never_cut_twice asserts both halves, so neither can regress silently.

The number

64,000 characters, about sixteen thousand tokens. Measured over this repository's 225 Python files:

cap files delivered whole
20,000 (the old effective) 77.3%
40,000 (the old declared) 91.6%
64,000 97.3%
80,000 98.7%

The six above it — cli.py at 407,760 leading — are files nothing should read end to end;
search_code's skeleton and map are the answer there, and a working notice is what leaves a
model able to choose them.

Turns are the expensive axis, not bytes: reading a 52,000-character file in one call or in three
puts the same bytes in the prompt, and the three-call version also pays two extra turns, each
re-sending everything accumulated so far. In the run that prompted #412, 1,566,345 cache-read
tokens were $2.35 of the $3.83.

Also

What the invoker does still cut — a delegated child's answer, an adopter's own tool — now says
how much it dropped. …[truncated] alone cannot tell a result that was nearly whole from one that
was a tenth, and gives a model no number to narrow the next query with.

test_gate_read_1_a_staged_file_is_readable_past_the_cap had a literal 5,000-line fixture that
stopped being over the cap when the cap rose; it now derives its size from MAX_READ_CHARS, so the
next change to the number cannot leave it asserting truncation of a file that fits.

Objectives

O13 — a run is bounded before it starts, and two bounds that disagree mean the real one was
never declared. O7 — determinism first: what a tool returns is now decided in one place by one
rule, rather than emerging from two caps applied in sequence by layers that could not see each
other.

Verification

  • make check green from the committed tree: 2867 passed, 6 skipped; ruff format reports 268 files
    unchanged, so what is pushed is what passed.
  • make cov green at 92% (floor 90%).
  • Three falsification controls fired and restored from the commit, each asserting its own edit
    landed first:
    • the invoker's cap back to 20,000 → the delivery test, the never-cut-twice test and the
      between-the-caps test all go red;
    • the read notice appended outside the cap again → the never-cut-twice test and
      test_a_huge_read_is_truncated_and_says_how_to_read_the_rest go red;
    • the invoker's cut back to a bare marker → test_a_tool_result_the_invoker_cuts_says_how_much_it_dropped
      goes red.

…survives it

`read_file` was capped twice. `MAX_READ_CHARS` cut a long file at 40,000 and appended the sentence
naming `offset` and `limit`; `InvokePolicy.max_tool_result_chars` then cut every tool result at
20,000 and appended a bare `…[truncated]`. The second cut always landed on the first one's notice,
so a model reading anything over 20,000 characters was told a file had been cut and never told how
to reach the rest — the exact behaviour #402 added the range parameters to end. Measured on this
tree, `strategy.py` reached a model as 38% of itself and `.lockstep/lockstep.py` as 50%. The band
between the two caps was worse: the tool reported no truncation at all and the invoker cut
mid-word, so nothing said the file continued.

GATE-READ-1 has stated the notice clause since #402 and was green throughout, because every test
discharging it called the tool runner — and the tool runner is not what hands a result to a model.
A property asserted one layer below the one that delivers it is a property nobody is testing, so
the tests added here run through `AiInvoker`.

Two mechanisms, because either alone can drift. `_window` returns at most `MAX_READ_CHARS`
INCLUDING the header and notice it appends, so nothing downstream has to cut it again; and the
invoker's cap IS `MAX_READ_CHARS` rather than a number of its own, so the two cannot be edited
apart. The cap rises to 64,000 — about sixteen thousand tokens, and 97% of this repository's Python
files whole — because turns are the expensive axis rather than bytes: the same file read in three
calls puts the same bytes in the prompt and pays two extra turns, each re-sending everything
accumulated so far. `search_code`'s `skeleton` and `map` remain the answer for the six files above
that, which is a choice a model can only make if the notice reaches it.

What the invoker does still cut now says how much it dropped. `…[truncated]` alone cannot tell a
result that was nearly whole from one that was a tenth, and gives no number to narrow the next
query with.

Closes #412

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

in-lockstep review — intent

succeeded · $0.0606 · 3 in / 575 out tokens

location finding
⚠️ tests/in_lockstep/test_read_delivery.py:110 test_gate_read_1_a_read_is_never_cut_twice checks _window output length, not what the model receives.
⚠️ src/in_lockstep/ai/invoker.py:235 _bounded is not called when text is exactly cap characters; off-by-one is benign but asymmetric with _window.
ℹ️ the lens was given this change's blast radius (3509 chars)

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

in-lockstep review — performance

succeeded · $0.0486 · 3 in / 117 out tokens

No findings.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

in-lockstep review — security

succeeded · $0.0538 · 3 in / 99 out tokens

location finding
ℹ️ the lens was given this change's blast radius (3509 chars)

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

in-lockstep review — tests

succeeded · $0.0611 · 3 in / 609 out tokens

location finding
⚠️ tests/in_lockstep/test_read_delivery.py:110 Invoker-delivery guarantee is only asserted for whole-file reads, not for ranged reads.
⚠️ tests/in_lockstep/test_read_delivery.py:139 test_a_file_between_the_two_old_caps_arrives_whole asserts equality to raw body, but _delivered strips redaction and may reformat content.
ℹ️ the lens was given this change's blast radius (3509 chars)

@tpouyer
tpouyer merged commit 81d52a0 into main Sep 9, 2026
4 checks passed
@tpouyer
tpouyer deleted the fix/read-bounded-once branch September 9, 2026 20:35
tpouyer added a commit that referenced this pull request Sep 9, 2026
…ys which caller it means (#415)

`MAX_READ_CHARS` was 64,000 because I measured this repository — 97% of its Python files fit whole.
That is a fine way to pick a default and no way to pick a number nobody else can change. A
repository of short modules pays for a window it never fills, on every read, on every later turn;
one carrying generated clients an order of magnitude larger cannot read its own code and has no way
to say so. `Workshop(max_read_chars=...)` states it, a contributed policy layer may lower it, and
`ls` prints what the stack contributed.

The value travels the road `max_test_runs` already walks — `InvokePolicy` to the binding site to
`ToolRunnerImpl` — with `_window` at the end instead of the test counter. All three session
constructors take it, not only the one that executes: a review reads files exactly as an implement
does, and a cap honoured in one and not the other is two answers to one question.

`max_tool_result_chars` stops being a field and becomes a property returning the read window.
#413 set the two equal so they could not be edited apart, and making both settable would put that
defect back in through the front door: a repository raising its window and leaving the result bound
at the shipped default would be cut in half again, with nothing saying so. One number an adopter
states.

GATE-READ-1 has said "the cap is not negotiable by the caller" for a year without saying which
caller, and the two it could mean are not alike. The caller is the MODEL, which reaches the reader
through `read_file`'s `offset` and `limit` — neither of which is a byte budget — and the row now
says so, because leaving it ambiguous is how a reader resolves it the wrong way: either weakening
the rule that stops a model widening its own prompt, or abandoning the configuration because "the
gate says no". `test_gate_read_1_a_model_cannot_widen_the_window_it_reads_through` now runs against
a non-default cap, so it proves the rule rather than the constant.

Closes #414

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ai): a read is capped twice, the smaller cap wins, and it cuts off the sentence GATE-READ-1 promises

1 participant