sdk: keep mid-conversation system messages positional - #399
Merged
Conversation
Hoisting them into the top-level system field invalidated the cached prefix and could leave messages ending in an assistant turn. Leading items still merge; positional ones emit as role:system (now accepted by Anthropic) or degrade to a round-tripping <system> user turn.
… default Canonical follows OpenAI semantics: positional system items stay native on OpenAI shapes; anthropic/gemini deliver them in place as <system>- wrapped user turns (works on every model, prefix stays cacheable). hoist:true merges an item into the start-of-conversation system form.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anthropic accepts
role: "system"insidemessageson current models, and some harnesses (Claude Code) now send it. The anthropic and gemini translators previously hoisted every canonical system/developer item into the top-level system field /systemInstruction, whichNow only items leading the transcript merge into the start-of-conversation system form. Mid-conversation items stay positional, with canonical semantics following the OpenAI shape:
<system>markers — works on every model (no capability matrix), keeps position, and leaves the cached prefix untouched. On anthropic they are deferred past an open tool round so atool_use/tool_resultpair is never split.hoist: true(new field on canonical message items): explicit opt-in to full system authority — the item merges into the top-levelsystem/systemInstruction/ leading system message on every shape, at the documented cost of invalidating the cached prefix.sdk/v1owns the marker wrap/unwrap pair and the hoist splitter so the forms are identical across adapters.role:systemmessages to canonical system items positionally (previously they fell through to user).Behavior documented in
docs/reference/inference-api.mdx.