fix(lab): read per-model overrides in the report the way the runtime reads them - #2077
fix(lab): read per-model overrides in the report the way the runtime reads them#2077ntdatt812 wants to merge 1 commit into
Conversation
…reads them lidge-jun#2059 fixed the list-shaped half of the behavior report. The per-model maps are the other half, and they had the same shape of bug plus one more. `modelValue` was a bare index, `map?.[modelId]`. The runtime reads these maps through `modelRecordValue`, which checks own properties, then the pre-colon family, then a case-folded key. So the report disagreed three ways: - ollama-cloud serves `gpt-oss:120b`. With `modelMaxOutputTokens: {"gpt-oss": 1234}` the adapter puts `max_tokens: 1234` on the wire, while the report said `limits.maxOutputTokens: null`. - a differently-cased key resolved at runtime and not in the report. - the index walked the prototype chain. Model ids are operator-controlled, so one can be `constructor` or `toString`, and the row then held an Object.prototype *function*. That last one is not merely wrong data. `jcsStringify` rejects a function, so `buildBehaviorFingerprintV1` threw `unsupported value type function`, and `resolvePassiveRouteSubjectId` swallows the throw -- the subject silently never links, and Lab loses that traffic with no diagnostic. The linker's own contract says a registered implementation is "synchronous, free of side effects with respect to the request, and non-throwing"; the try/catch is the backstop, not a licence. openai-responses.ts already guards `modelPreferHostedTools` against exactly this, with a comment saying why. Nine of the ten maps the report reads are read through `modelRecordValue` at runtime; delegating to it makes the report agree with all of them at once. Fingerprints, same config, before and after: gpt-oss:120b 54154e19bd2c8ee4 -> 5c992edff35bd7e9 (was missing 1234) gpt-oss 5c992edff35bd7e9 -> 180a84b2d837619d (was missing 55555) glm-5.3 54154e19bd2c8ee4 -> 54154e19bd2c8ee4 (unchanged) constructor THROW -> 54154e19bd2c8ee4 (now computable) Only subjects whose overrides were being missed move; `resolverVersion` stays at 2 for the reason given in lidge-jun#2059. Tests extend the file lidge-jun#2059 added: the adapter's wire is asserted first, then the report is held to it, and the prototype-shaped ids get their own cases including one that the fingerprint stays computable. Seven of the nine fail on current dev. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
✅ Deterministic PR hygiene checks passed. |
✅ READY
Review readiness checklist
✅ 4/4 boxes ticked. This pull request is already Ready for Review. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 WalkthroughWalkthroughThe change makes behavior reporting use shared per-model override resolution. It adds coverage for family matching, case-insensitive keys, unrelated models, prototype-shaped IDs, and equivalent behavior fingerprints. ChangesModel override resolution
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to The change aligns report lookup behavior with runtime overrides and adds focused regression coverage; no actionable merge-blocking risk remains beyond normal checks and review. Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
#2059 fixed the list-shaped half of the behavior report. The per-model override maps are the other half, and they had the same shape of bug plus one more.
modelValuewas a bare index:The runtime reads these maps through
modelRecordValue(src/reasoning-effort.ts:73), which checks own properties, then the pre-colon family, then a case-folded key. Nine of the ten maps the report reads go through it at runtime —modelContextWindows,modelMaxInputTokens,modelMaxOutputTokens,modelInputModalities,modelReasoningEfforts,modelDefaultReasoningEfforts,modelReasoningEffortMap,modelSupportsReasoningSummaries,modelReasoningSummaryDelivery. The tenth,modelPreferHostedTools, is read inopenai-responses.tswith its own explicithasOwnPropertyguard.The three disagreements
1. Pre-colon family. ollama-cloud serves
gpt-oss:120b. WithmodelMaxOutputTokens: {"gpt-oss": 1234}:max_tokens: 1234limits.maxOutputTokensnull2. Case folding. A differently-cased key resolves at runtime and did not in the report.
3. Prototype chain — this one is not just wrong data.
Model ids are operator-controlled, so one can be
constructorortoString. The bare index then returned anObject.prototypefunction:jcsStringifyrejects a function, sobuildBehaviorFingerprintV1threwunsupported value type function, andresolvePassiveRouteSubjectIdswallows the throw — the subject silently never links and Lab loses that traffic with no diagnostic. The linker's contract states an implementation is "synchronous, free of side effects with respect to the request, and non-throwing"; the try/catch is described there as belonging to the mechanism so the guarantee is not restated by callers — a backstop, not a licence.openai-responses.ts:996-1001already guardsmodelPreferHostedToolsagainst exactly this, and says why in a comment.Change
modelValuedelegates tomodelRecordValue. One helper, so the report cannot disagree with the runtime on any of the nine at once.Blast radius, measured
Same config, before and after, first 16 hex of the behavior fingerprint:
gpt-oss:120b54154e19bd2c8ee45c992edff35bd7e9gpt-oss5c992edff35bd7e9180a84b2d837619dglm-5.354154e19bd2c8ee454154e19bd2c8ee4constructor54154e19bd2c8ee4Only subjects whose overrides were being missed move, so
resolverVersionstays at2for the same reason as #2059 — say the word if you would rather draw a generation boundary.Tests
Extends
tests/routing-compatibility-model-matching.test.tsfrom #2059, same pattern: assert the wire the adapter really builds, then hold the report to it. Prototype-shaped ids get their own cases, including one asserting the fingerprint stays computable and that two such ids hash alike.Seven of the nine new cases fail on current
dev(9 pass / 7 fail); the two that pass there are the ground-truth wire assertion and the control.Verification
On the exact head of this branch:
Not the full suite: this is a Windows machine and
bun run testpanics partway through on Bun 1.3.14 (index out of bounds: index 0, len 0), so a result from it would be a truncated log. The batch ran throughscripts/test.tsso each file keeps its isolated home.Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.
Summary by CodeRabbit
Bug Fixes
Tests