Skip to content

fix(provider): normalize 1-based streaming tool_call indexes and fallback missing ids (#9590) - #9593

Open
x1051445024 wants to merge 1 commit into
AstrBotDevs:masterfrom
x1051445024:fix/toolcall-index-offset-9590
Open

fix(provider): normalize 1-based streaming tool_call indexes and fallback missing ids (#9590)#9593
x1051445024 wants to merge 1 commit into
AstrBotDevs:masterfrom
x1051445024:fix/toolcall-index-offset-9590

Conversation

@x1051445024

@x1051445024 x1051445024 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

问题概述

部分 OpenAI 兼容网关(如 new-api 风格的代理)在流式返回时把 tool_call.index1 开始编号(OpenAI 规范要求从 0 开始)。openai SDK 的流式快照累加器直接把 index 当作列表下标使用:

  • _accumulate_chunktool_calls[tool_call_chunk.index]
  • _build_eventstool_calls[tool_call_delta.index]

于是 index=1 落到空列表上会抛 IndexError: list index out of range。AstrBot 在 _query_stream 中吞掉该异常(仅记录 Saving chunk state error),导致快照被写坏:同一个 tool_call 的 namearguments 被拆到两个错位的槽位,中间多出 id=None / name="" 的幽灵条目。下游 ToolCall(id=None) 随即抛 pydantic ValidationError,整轮工具调用(如子代理委托)失败。

只在流式 + 多个并行 tool_call 时触发,因此表现为偶发,单工具调用复现不出来。

与已有 issue 的关系

日志签名(按序出现)

ERRO sources.openai_source  Saving chunk state error: list index out of range
ERRO sources.openai_source  解析参数失败: Expecting value: line 1 column 1 (char 0)
INFO tool_loop_agent_runner  Agent 使用工具: ['real_tool', '__malformed_tool_name__']
ERRO pydantic ValidationError: ToolCall id Input should be a valid string, input_value=None
ERRO subagent_worktogether  Failed to delegate task to main agent

修复内容(两处)

  1. openai_source.py — index 重映射:在 _query_stream 内维护每次请求独立的 tool_call_index_map(请求级局部变量,不能挂 self,否则并发请求互相污染下标映射),把上游 index 重映射为从 0 开始的连续序号。现有 Streaming tool_call arguments lost when OpenAI-compatible proxy omits index field (e.g. Gemini) #6661 补丁只处理 index 缺失,本补丁在其 else 分支处理偏移。

  2. entities.py — id/name 兜底to_openai_tool_calls_model() 中,当上游返回的 id 缺失/非法时回退为稳定的 call_{idx},name 缺失时回退为 __malformed_tool_name__(与 tool_loop_agent_runner 的占位符一致),并做列表长度边界判断。extra_content 仍按上游原始 id 查询(兜底 id 不会存在于该字典中)。

验证

用真实上游形态的 SSE 流(index 从 1 开始、多个并行 tool_call)做 SDK 层对照测试:

  • 修复前handle_chunk 失败 2 次(IndexError),最终快照 3 条 tool_call,其中 1 条 id=None, name=None 幽灵条目,且 arguments 错位(es_search 的 query 参数跑到了 web_search 上)。
  • 修复后handle_chunk 失败 0 次,2 条 tool_call 的 id/name/arguments 全部正确合并。

entities 兜底层用例(id=None、name 空、列表比 args 短)均通过,deprecated 别名 to_openai_to_calls_model() 保持可用。

影响范围

ProviderOpenAIOfficial 及其 7 个子类(groq / longcat / oai_aihubmix / openrouter / xai / xiaomi / zhipu)共用 openai_source.py,且只有该文件使用 ChatCompletionStreamState,因此改一处即覆盖全部 8 个 provider。

关联 issue:#9590

Summary by Sourcery

Normalize tool_call indexes from OpenAI-compatible streaming providers and harden ToolCall model construction against malformed upstream data.

Bug Fixes:

  • Normalize non-zero-based tool_call.index values from streaming responses to contiguous 0-based indices to prevent IndexError and misaligned tool_call snapshots.
  • Provide stable fallback ids and names for tool_calls missing or carrying invalid id/name fields to avoid pydantic ValidationError and failed tool-calling turns.

Enhancements:

  • Guard access to tool_call id/name lists with bounds checks and use raw upstream ids only for extra_content lookup to preserve original metadata when available.

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. labels Aug 7, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

…back missing ids (AstrBotDevs#9590)

## Problem

Some OpenAI-compatible gateways (e.g. new-api style proxies) start
streaming `tool_call.index` at 1 instead of 0 (OpenAI spec requires
0-based). The openai SDK streaming snapshot accumulator uses the index
directly as a list subscript:

- `_accumulate_chunk`: `tool_calls[tool_call_chunk.index]`
- `_build_events`: `tool_calls[tool_call_delta.index]`

So `index=1` on an empty list raises `IndexError: list index out of
range`. AstrBot swallows this error in `_query_stream` (only logs
`Saving chunk state error`), which corrupts the snapshot: a tool_call's
`name` and `arguments` get split into misaligned slots, producing a
ghost entry with `id=None` / `name=""`. Downstream,
`ToolCall(id=None)` then raises a pydantic `ValidationError` and the
whole tool-calling turn (e.g. subagent delegation) fails.

This only triggers with streaming + multiple parallel tool_calls,
which is why it appears intermittent.

## Fix

1. `openai_source.py`: remap upstream tool_call indexes to a
   contiguous 0-based sequence per request (request-local dict, not
   `self`, to avoid cross-request pollution under concurrency). The
   existing AstrBotDevs#6661 patch only handled a *missing* index, not an offset.

2. `entities.py`: in `to_openai_tool_calls_model()`, fall back to a
   stable `call_{idx}` id and `__malformed_tool_name__` when the
   upstream id/name is missing or invalid, so a single malformed
   tool_call can no longer fail the entire turn. `extra_content` is
   still looked up by the raw upstream id.

## Impact

Affects `ProviderOpenAIOfficial` and its 7 subclasses (groq, longcat,
oai_aihubmix, openrouter, xai, xiaomi, zhipu) — all share
`openai_source.py`; only that file uses `ChatCompletionStreamState`,
so the patch covers all of them.

Closes AstrBotDevs#9590
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant