Skip to content

fix(gemini): emit reasoning chunks and fix thought parts extraction - #56

Merged
CrazyBoyM merged 4 commits into
shareAI-lab:mainfrom
kirillreutski:fix/gemini-thinking-upstream
Jul 10, 2026
Merged

fix(gemini): emit reasoning chunks and fix thought parts extraction#56
CrazyBoyM merged 4 commits into
shareAI-lab:mainfrom
kirillreutski:fix/gemini-thinking-upstream

Conversation

@kirillreutski

Copy link
Copy Markdown
Contributor

Problem

When using GeminiProvider with thinking-capable models (e.g. gemini-2.5-pro), reasoning content is silently lost in three separate places:

  1. includeThoughts missing from request — Gemini API requires includeThoughts: true in thinkingConfig, otherwise the model thinks internally but returns no thought parts in the response. The SDK was setting thinkingBudget/thinkingLevel without this flag.

  2. stream() drops thought partsparseGeminiChunk lumped all text parts into textChunks without distinguishing part.thought === true. Thought content was emitted as regular text instead of think_chunk_* SSE events.

  3. collectAll path drops thought parts — The fallback path (when Gemini returns a JSON array instead of line-by-line SSE) had the same problem: thoughtChunks were never extracted, so the agent never emitted reasoning events on this path either.

A fourth issue: empty part.text strings ("") were being pushed into textChunks, causing spurious empty text_chunk SSE events.

Changes

src/infra/providers/gemini.ts

  • _buildGenerationConfig: add includeThoughts: true whenever thinkingBudget or thinkingLevel is set.

  • parseGeminiChunk: split parts by part.thought === true into a new thoughtChunks: string[] return field (only when reasoningTransport === 'provider'); skip empty strings.

  • stream() — SSE path: consume thoughtChunks and yield content_block_start(reasoning) + content_block_delta(reasoning_delta) events before text chunks.

  • stream()collectAll path: same reasoning-chunk handling for the JSON-array fallback path.

  • parseBlocks (used by complete()): map part.thought === true parts to { type: 'reasoning', reasoning } blocks instead of text blocks.

Testing

Verified end-to-end with gemini-2.5-pro using reasoningTransport: 'provider':

  • think_chunk_start / think_chunk / think_chunk_end SSE events now arrive correctly in streaming mode
  • complete() returns reasoning blocks in the message content
  • No regressions on non-thinking models (thought parts absent → thoughtChunks stays empty)

kirillreutski and others added 4 commits May 17, 2026 14:15
…ransport='provider'

Gemini API returns thinking content as parts with `thought: true`. Previously
these were pushed as regular text blocks regardless of reasoningTransport,
so think_chunk SSE events were never emitted for Gemini models.

When reasoningTransport is 'provider', parts with thought===true are now
pushed as { type: 'reasoning', reasoning } blocks. The agent emits these
as think_chunk_start / think_chunk / think_chunk_end progress events,
allowing clients to render thinking separately from the response text.

Default behavior (reasoningTransport: 'text') is unchanged — thought parts
continue to arrive as text blocks and get normalized by normalizeThinkBlocks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without includeThoughts, the model thinks internally but does not return
thought parts in the response. This adds it automatically whenever
thinkingBudget or thinkingLevel is set.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parseGeminiChunk now separates parts with thought===true into thoughtChunks
when reasoningTransport is 'provider'. The streaming loop yields them as
content_block_start(reasoning) + content_block_delta(reasoning_delta) before
text chunks, so the agent emits think_chunk_start/think_chunk/think_chunk_end
SSE events that clients can render separately from the response text.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@CrazyBoyM

Copy link
Copy Markdown
Contributor

Thank you @kirillreutski for identifying the Gemini thought-loss paths and putting together the original fix.

I opened #58 to carry this through the repository's full merge gate. Rather than cherry-picking or rewriting your work, #58 merges all four commits from this PR unchanged: 51acf76, 4ddd074, 5f1f665, and ff5cf2a remain in its ancestry with your original authorship.

The follow-up adds the protocol hardening and deterministic coverage found during pre-merge review: complete block lifecycles, ordered interleaved parts, all history transports, thought signatures (including signature-only empty parts), function-call IDs, robust SSE/JSON fallback parsing, Agent persistence/privacy behavior, and Node 20/full-suite checks.

I have requested your review on #58 as a courtesy; no action is required for attribution. Once #58 passes the full CI chain and is merged with a merge commit, this PR can be closed as incorporated/superseded without losing its history.

@CrazyBoyM
CrazyBoyM merged commit 8be6c1d into shareAI-lab:main Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants