Skip to content

Make the Posit AI Pass "Off" thinking option follow the capability table - #194

Merged
wch merged 1 commit into
mainfrom
derive-thinking-off-none-from-table
Nov 19, 2026
Merged

wch merged 1 commit into
mainfrom
derive-thinking-off-none-from-table

Conversation

@wch

@wch wch commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

This PR makes Posit AI Pass send thinking "Off" as reasoning_effort: "none" based on the Model APIs capability table, instead of a separate list of model IDs. Behavior for every current model is unchanged.

The table decides whether the picker offers "Off"; the list decided whether it was sent. A model in the table but not the list showed an "Off" that sent nothing, so Baseten applied its default effort. This happened three times (#88, #176, #192). The capability flag proposed in #191 would touch about 8 files in two repos, and a host that forgot to forward it would bring the bug back.

positAiThinkingRequestFields (packages/ai-provider-bridge/src/utils.ts) now sends "none" when the model's table entry offers "off" next to a named effort level. The charter says clients never infer capabilities from model IDs; the old list already did, so memory-bank/architecture.md now names this as an exception. Models added through providers.json are not in the table and can still show an "Off" that sends nothing, as before.

After merge, bump the packages/ai-lib pin in the monorepo.

…he capability table

positAiThinkingRequestFields kept its own list of model IDs that get
reasoning_effort "none" for thinking "off", separate from the Model APIs
capability table that decides whether the picker offers "off". A model in
the table but not the list showed an "Off" that sent nothing, so Baseten
applied its default effort. This happened for DeepSeek V4 Flash, Kimi K3.1
Pro, and Kimi K3.

The function now reads the model's thinkingEffortLevels from the table and
sends "none" when "off" sits next to at least one named effort level. The
ID list is gone, so for built-in models the picker and the wire cannot
disagree. Behavior for every current model is unchanged.

- Export getModelApisModelCapabilities from ai-config.
- Document in the table that offering "off" next to named levels promises
  Baseten accepts "none".
- Record the bridge reading the table as a named charter exception.
@wch
wch merged commit e3f8502 into main Nov 19, 2026
4 checks passed
@wch
wch deleted the derive-thinking-off-none-from-table branch November 19, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant