Repository navigation
fix(ptc): teach the session scope every PTC program shares - #869
Open
WeiLinCool wants to merge 1 commit into
Open
WeiLinCool wants to merge 1 commit into
WeiLinCool wants to merge 1 commit into
Conversation
WeiLinCool
force-pushed
the
fix/ptc-scope-guidance
branch
from
November 13, 2026 15:59
17a7c43 to
658b72f
Compare
A state-retaining PTC runtime keeps a program's top-level `const`, `let`, `class` and `function`, so a second declaration of that name is a SyntaxError at compile time and the whole program is discarded before its first line runs, tool calls included. A program that throws keeps its names too, which is why re-sending a failed program verbatim dies on a name rather than on its arguments, and why the name is never reclaimable. Measured on one 127-program ops-ptc session: 11 of 17 failed programs were this exact error, two of them retries of a program that had already failed, with names declared on turn 1 still colliding on turn 6. The guidance now states one scope per session, makes each program own its top-level names, separates a retained name from a re-declared one, and lists both traps in the mistake table. The examples keep their `return`: batching-examples executes them and asserts the returned summary, so a printed example would silently drop out of that coverage. A model-driven A/B over one audit task in a retained REPL discards 7 of 56 programs on the 0.5.5 text and 3 of 56 after (4 runs per arm, 14-program budget, only the guidance file differs).
WeiLinCool
force-pushed
the
fix/ptc-scope-guidance
branch
from
November 13, 2026 16:01
658b72f to
828d42b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #868
What was wrong
PTC can evaluate a program in a runtime that retains state, so a top-level
const/let/class/functionoutlives the program that declared it. A second declaration of that name is aSyntaxErrorat compile time and the whole program is discarded before its first line runs, tool calls included. A program that throws keeps its names as well — bindings are instantiated before evaluation — so a failed program cannot be retried verbatim: the retry dies on a name, not on its arguments. The name is never reclaimable: there is no way to delete a lexical binding, andconstis not acontextproperty, soObject.keyscannot even enumerate it.In the session that filed #868 (127
run_codeprograms) this was 11 of 17 failures (65% of failures, 8.7% of programs). Two were retries of a program that had already failed. Names declared on turn 1 (q1,e1) still collided on turn 6, and failed programs left ~70 permanently unusable names.The shipped guidance taught the pattern: its Step 3 example is
const [load, head] = await Promise.all([...]), and the model copiedloadinto the next program, dying on program two.The change
release/0.5.5rewrote the same file. This branch is rebased onto the currentmaster(which contains 0.5.5), so the diff against the 0.5.5 text is 15 lines:## One session, one scope— a top-level name can outlive its program, a second one is a compile-time error that discards the whole program, a failed program keeps its names, so each program owns its names and a retry renames.BATCHING_GUIDANCE_REVISION→'5'(0.5.5 already used 4).What this deliberately does not do: wrap the examples in
{ }. That was the first shape of this PR and it is incompatible with 0.5.5:test/batching-examples.test.mjscompiles and executes every shipped example through anAsyncFunctionand asserts the returned summary, so an example has toreturn, andreturncannot live inside a bare block. Switching the examples toconsole.logsilently removed them from that test's coverage — I reverted it for that reason.The stronger alternative, if maintainers prefer 0% over 5%: wrap each example body in
return await (async () => { ... })(). It keeps the returned value, keepsbatching-examplesgreen, and makes the names local, at the cost of turning every example into an IIFE. Say the word and I will send it as a follow-up.Validation
A. Model-driven A/B over one audit task in a retained REPL — same model, same task, same
run_codecontract, only--guidancediffers, 4 runs per arm, 14-program budget:B. Why the examples are not the vehicle —
bench/probes/probe-scope-guidance.mjsruns every fenced example twice in one retained REPL. Both texts score 6 of 7 redeclared, because the examples are unchanged; that is the measurement that says this fix has to be the rule, not the shape.C. Suite —
npm run verifyon this branch (darwin-arm64): 247 of 248 suites pass, 7 skipped. The one failure ispreset-fragment, which compares the shippedpresets/ops-ptc.patch.ymlchecksum against the locally installed host checkout; that checkout is at a newer commit (838665e0…vs the recordedce02148b…), and this branch does not touchpresets/. It passes in CI, where the host is installed from the pinned range.D.
CI required— green on the rebased head: https://github.com/T-Auto/dsh-ops/actions/runs/43458558329An earlier run of the same content was red twice for infrastructure reasons, not for this change:
validate (darwin-arm64)was cancelled with zero steps and no runner assigned, andvalidate (darwin-x64)failed once ontest/native-guidance.test.mjs:59, which waits for the Native reader to be published and then asserts the officialreadis already hidden — publication and hiding are two steps, so a loaded runner can lose that race. Both legs pass on the current head.