Align OpenAI-compatible streaming reasoning_content with non-streaming responses#6373
Open
jewoodev wants to merge 1 commit into
Open
Align OpenAI-compatible streaming reasoning_content with non-streaming responses#6373jewoodev wants to merge 1 commit into
reasoning_content with non-streaming responses#6373jewoodev wants to merge 1 commit into
Conversation
This was referenced Jun 10, 2026
0b1fa9d to
293bfd9
Compare
reasoning_content with non-streaming responses
ericbottard
reviewed
Jun 11, 2026
|
|
||
| private static String stringProperty(Delta delta, String key) { | ||
| JsonValue value = delta._additionalProperties().get(key); | ||
| return value == null ? "" : (String) value.asString().orElse(""); |
Member
There was a problem hiding this comment.
I'm not very keen on transforming a null into "". If a value isn't there, it isn't there. Or does the downstream code rely on a value being present in all cases?
Contributor
Author
There was a problem hiding this comment.
Good point. The downstream replay path only needs the value when actual reasoning content is present, so missing provider fields do not need to be modeled as empty strings here. I updated the merge logic to leave missing fields absent and only concatenate/propagate reasoning fields when the incoming chunk carries a non-empty fragment.
In the OpenAI-compatible streaming path, ChunkMerger.chunkToChatCompletion rebuilt the message from content/refusal/toolCalls only, so provider-specific delta fields such as reasoning_content (DeepSeek) or reasoning (OpenRouter) were never copied onto the message that getReasoningContent reads. The reasoningContent metadata was therefore always empty in streaming, and the next-turn replay in createRequest had nothing to re-attach, so providers that require reasoning on assistant tool-call messages reject the replayed request. - Copy delta additional properties onto the rebuilt message, matching the non-streaming path - Concatenate reasoning fragments across merged tool-call chunks - Accumulate reasoning across the stream so the final response carries the full reasoning content through last-wins metadata aggregation See spring-projects#5898 Signed-off-by: jewoodev <jewoos15@naver.com>
293bfd9 to
aceb6d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
OpenAiChatModelalready preserves provider reasoning metadata such as DeepSeek'sreasoning_contenton the non-streamingcall()path, but the OpenAI-compatible streaming path drops the same fields before they can be captured inreasoningContent, leaving that metadata empty on streamed responses.ChunkMerger.chunkToChatCompletionrebuilds each streamed chunk's message fromcontent/refusal/toolCallsonly, so provider-specific delta fields such as DeepSeek'sreasoning_content(or OpenRouter'sreasoning) never reach the message thatgetReasoningContentreads. The next-turn replay increateRequestthen has nothing to re-attach, so providers that require reasoning on assistant tool-call messages reject the request with"thinking is enabled but reasoning_content is missing in assistant tool call message".How
The fix stays inside
OpenAiChatModel:chunkToChatCompletionnow carries the delta's additional properties onto the rebuilt message (matching the non-streaming path),mergeDeltasconcatenates reasoning fragments across merged tool-call chunks, and the streaming pipeline accumulates reasoning per choice so the final emitted response holds the full reasoning content — surviving the last-wins metadata aggregation thatMessageAggregator-based advisors use to build the assistant message they re-inject on the next turn. As a consequence, intermediate streamed responses carry the reasoning accumulated so far rather than the per-chunk fragment (previously this metadata was always empty on the streaming path).Test
Unit tests reproduce the failure with mocked streaming chunks: reasoning followed by a split tool call (parameterized over DeepSeek's
reasoning_contentand OpenRouter'sreasoning) and reasoning without tools. The new regression cases fail onmainwith emptyreasoningContentand pass with this change. The tool-call test also replays the aggregated assistant message throughcreateRequestand asserts thereasoning_contentwire field is re-attached:The native DeepSeek starter loses reasoning at a different point (
MessageAggregatorflattening ofDeepSeekAssistantMessage, #6026), which this PR intentionally does not touch.See #5898