Skip to content

fix(core): size background turns for hidden reasoning - #2276

Merged
justinhelmer merged 1 commit into
mainfrom
plan/fix-issue-2267-it-is-the-2fb5e3/u1
Sep 23, 2026
Merged

justinhelmer merged 1 commit into
mainfrom
plan/fix-issue-2267-it-is-the-2fb5e3/u1

Conversation

@justinhelmer

@justinhelmer justinhelmer commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Intake and memory reflection now reserve room for hidden reasoning without misclassifying capped replies. Wrapped intake cuts retain their structured-attempt history, and production composition tests prove both background paths receive their model card’s cap field.

Why: #2267 blocks first-party OpenAI defaults because Responses counts hidden reasoning inside the output cap. The original fix missed a wrapped retry failure, production wiring proof, and the behavior specs.

Where to look

  1. Wrapped cap classification preserves structured attempts while classifying the underlying cap cut before generic provider failure. ⚠ A mistake here silently changes outage handling.
  2. The regression test reproduces a malformed first answer followed by a capped retry.
  3. Intake production composition carries the resolved card’s cap field into the live decision dependencies.
  4. Reflection production composition proof exercises scheduling through model-card resolution and observes the expanded completion request.
  5. Memory contract defines the dynamic reasoning-aware cap and rejects every capped response before parsing.
  6. Intake validation contract binds cap sizing, wrapped-cut behavior, and production wiring to named tests.

Feedback wanted: Check that unwrapping only StructuredAskError’s direct preserved cause is the right boundary, and that the two composition tests cover the live bridges without over-coupling startup.

Risk: Incorrect classification could turn an output-cap cut into a permanent provider failure or omit the reasoning allowance in production. Revert this single commit to restore the prior behavior.

Verified: 97 focused tests, four exact regression/wiring tests, root TypeScript, specs/docs/hygiene/coverage/test-guard, Prettier, and npm run fix passed; CI remains the full-suite gate.

Decisions (3)
  • Unwrap only the structured seam wrapper. askStructured is the boundary that wraps a later model throw after collecting violations, so intake examines that known wrapper instead of recursively interpreting arbitrary error causes.
  • Test the production bridge, not only each endpoint. intakeDecisionDeps is used by src/index.ts, while reflection is exercised through scheduleReflection; both tests observe the completion request after model-card resolution.
  • Reject every capped reflection. A syntactically complete object can still be semantically incomplete when the provider reports the cap, so no max_tokens response reaches parsing or storage.
Validation (7 criteria)
Criterion Proof
A capped retry after malformed intake output remains an output-cap error and keeps the first violation npx vitest run src/core/intake.test.ts -t "an output-cap cut after a malformed answer stays an output-cap error and keeps the structured attempt" — passed
Intake production composition carries the resolved cap field into the verdict request npx vitest run src/intakeModel.test.ts -t "production intake deps carry the completion's cap field into the verdict call" — passed
Reflection production composition carries the resolved cap field into the completion request npx vitest run src/core/memory/index.test.ts -t "carries the reflection model card's cap field into the completion request" — passed
Complete capped reflection output is rejected without a write npx vitest run src/core/memory/reflection.test.ts -t "every capped response is rejected even when the object is complete; another unusual stop is named" — passed
All focused intake/reflection behavior remains green Four exact test files — 97 tests passed
The behavioral specs resolve and cover every changed source path npm run specs:check; npm run specs:coverage -- --changed origin/main...HEAD --test-guard; npm run docs:check — passed
Changed code is typed, formatted, hygienic, and generated artifacts are current root tsc --noEmit; changed-file Prettier; npm run hygiene:check; npm run fix — passed
For agents

Rebased onto current origin/main, resolving the intake test conflict by retaining both the typed provider-failure cases from the base and this PR’s cap tests. The branch is one coherent commit above main.

Red step: the new malformed-then-capped intake test failed because the decision reason was The model provider refused the call… rather than containing answer cut at the output cap; it passed after unwrapping the preserved OutputCapError.

The first npm run fix attempt used a symlinked dependency tree and rewrote generated Nuxt import paths; those unrelated changes were discarded. A local copy-on-write dependency tree then let npm run fix complete successfully without generated path drift.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: Two minor issues remain: wrapped output-cap errors are misclassified, and the memory spec contradicts the new cap behavior.

Warning

Changes requested · head 6f0b663 · 2 findings: 2 minor

Severity Finding Where
minor F1 Wrapped output-cap errors become provider failures src/core/intake.ts:274
minor F2 Spec contradiction — docs/reference/specs/memory.md item 12: reflection no longer uses or reports a fixed 1024-token cap docs/reference/specs/memory.md:27
Full review

F1: If intake first receives a malformed answer and the re-ask hits the output cap, askStructured wraps OutputCapError in StructuredAskError. The instanceof OutputCapError check then misses it and records a generic permanent provider failure, potentially emitting the wrong failure sentence. Inspect the preserved cause when classifying truncation and retain the structured attempts.

F2: Memory spec item 12 and validation rows still require a fixed 1,024-token cap, report truncation at 1,024, and allow a complete object that survived truncation to be written. The implementation now uses a reasoning-aware dynamic cap and rejects every max_tokens response without writing. Update the behavior and validation rows to describe the new contract.

Co-Authored-By: coreplane-switchboard[bot] <318072483+coreplane-switchboard[bot]@users.noreply.github.com>
@justinhelmer
justinhelmer force-pushed the plan/fix-issue-2267-it-is-the-2fb5e3/u1 branch from 6f0b663 to ad5cba5 Compare September 23, 2026 01:45
@justinhelmer

Copy link
Copy Markdown
Contributor Author

Re-review requested at ad5cba5 — rebased onto current main; fixed wrapped output-cap classification, aligned memory and intake specs, added production cap-propagation proofs, and preserved the typed provider-failure work. Independent OpenCode review is in progress.

@justinhelmer

Copy link
Copy Markdown
Contributor Author

OpenCode re-review at ad5cba5: ready to merge. All four prior findings are resolved, the #2232 typed-provider behavior remains intact, no additional findings were found, and all CI checks are green.

@coreplane-switchboard coreplane-switchboard Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM: The prior findings are resolved and the reasoning-aware cap behavior is now wired, specified, and covered end to end.

Note

Approved · head ad5cba5 · no findings

Full review

The prior findings are fully resolved; no remaining issues found.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: coreplane-switchboard[bot] reviewed this PR and posted an LGTM verdict (see its review). This repository opted in through its REVIEW_BOT_LOGIN and REVIEW_BOT_ID variables.

@justinhelmer
justinhelmer merged commit 44df09d into main Sep 23, 2026
30 checks passed
@justinhelmer
justinhelmer deleted the plan/fix-issue-2267-it-is-the-2fb5e3/u1 branch September 23, 2026 02:00
@justinhelmer

Copy link
Copy Markdown
Contributor Author

Production inclusion receipt — 1.260.6. v1.260.6 is full commit 903ae2855833637fbcc0c0a22d54e7268be75965; GitHub compare reports this PR's merge commit as the merge base (release ahead 2, behind 0). The release workflow succeeded, bot/memory/resident /healthz report the full tag SHA, and the authenticated sandbox gate passed on the tag. This is deployment evidence only; it does not claim intake or reflection hit an output-cap cut in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant