fix(gemini): emit reasoning chunks and fix thought parts extraction - #56
Conversation
…ransport='provider'
Gemini API returns thinking content as parts with `thought: true`. Previously
these were pushed as regular text blocks regardless of reasoningTransport,
so think_chunk SSE events were never emitted for Gemini models.
When reasoningTransport is 'provider', parts with thought===true are now
pushed as { type: 'reasoning', reasoning } blocks. The agent emits these
as think_chunk_start / think_chunk / think_chunk_end progress events,
allowing clients to render thinking separately from the response text.
Default behavior (reasoningTransport: 'text') is unchanged — thought parts
continue to arrive as text blocks and get normalized by normalizeThinkBlocks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without includeThoughts, the model thinks internally but does not return thought parts in the response. This adds it automatically whenever thinkingBudget or thinkingLevel is set. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parseGeminiChunk now separates parts with thought===true into thoughtChunks when reasoningTransport is 'provider'. The streaming loop yields them as content_block_start(reasoning) + content_block_delta(reasoning_delta) before text chunks, so the agent emits think_chunk_start/think_chunk/think_chunk_end SSE events that clients can render separately from the response text. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thank you @kirillreutski for identifying the Gemini thought-loss paths and putting together the original fix. I opened #58 to carry this through the repository's full merge gate. Rather than cherry-picking or rewriting your work, #58 merges all four commits from this PR unchanged: The follow-up adds the protocol hardening and deterministic coverage found during pre-merge review: complete block lifecycles, ordered interleaved parts, all history transports, thought signatures (including signature-only empty parts), function-call IDs, robust SSE/JSON fallback parsing, Agent persistence/privacy behavior, and Node 20/full-suite checks. I have requested your review on #58 as a courtesy; no action is required for attribution. Once #58 passes the full CI chain and is merged with a merge commit, this PR can be closed as incorporated/superseded without losing its history. |
Problem
When using
GeminiProviderwith thinking-capable models (e.g.gemini-2.5-pro), reasoning content is silently lost in three separate places:includeThoughtsmissing from request — Gemini API requiresincludeThoughts: trueinthinkingConfig, otherwise the model thinks internally but returns no thought parts in the response. The SDK was settingthinkingBudget/thinkingLevelwithout this flag.stream()drops thought parts —parseGeminiChunklumped all text parts intotextChunkswithout distinguishingpart.thought === true. Thought content was emitted as regular text instead ofthink_chunk_*SSE events.collectAllpath drops thought parts — The fallback path (when Gemini returns a JSON array instead of line-by-line SSE) had the same problem:thoughtChunkswere never extracted, so the agent never emitted reasoning events on this path either.A fourth issue: empty
part.textstrings ("") were being pushed intotextChunks, causing spurious emptytext_chunkSSE events.Changes
src/infra/providers/gemini.ts_buildGenerationConfig: addincludeThoughts: truewheneverthinkingBudgetorthinkingLevelis set.parseGeminiChunk: split parts bypart.thought === trueinto a newthoughtChunks: string[]return field (only whenreasoningTransport === 'provider'); skip empty strings.stream()— SSE path: consumethoughtChunksand yieldcontent_block_start(reasoning)+content_block_delta(reasoning_delta)events before text chunks.stream()—collectAllpath: same reasoning-chunk handling for the JSON-array fallback path.parseBlocks(used bycomplete()): mappart.thought === trueparts to{ type: 'reasoning', reasoning }blocks instead of text blocks.Testing
Verified end-to-end with
gemini-2.5-prousingreasoningTransport: 'provider':think_chunk_start/think_chunk/think_chunk_endSSE events now arrive correctly in streaming modecomplete()returns reasoning blocks in the message contentthoughtChunksstays empty)