fix(provider): normalize 1-based streaming tool_call indexes and fallback missing ids (#9590) - #9593
Open
x1051445024 wants to merge 1 commit into
Open
Conversation
…back missing ids (AstrBotDevs#9590) ## Problem Some OpenAI-compatible gateways (e.g. new-api style proxies) start streaming `tool_call.index` at 1 instead of 0 (OpenAI spec requires 0-based). The openai SDK streaming snapshot accumulator uses the index directly as a list subscript: - `_accumulate_chunk`: `tool_calls[tool_call_chunk.index]` - `_build_events`: `tool_calls[tool_call_delta.index]` So `index=1` on an empty list raises `IndexError: list index out of range`. AstrBot swallows this error in `_query_stream` (only logs `Saving chunk state error`), which corrupts the snapshot: a tool_call's `name` and `arguments` get split into misaligned slots, producing a ghost entry with `id=None` / `name=""`. Downstream, `ToolCall(id=None)` then raises a pydantic `ValidationError` and the whole tool-calling turn (e.g. subagent delegation) fails. This only triggers with streaming + multiple parallel tool_calls, which is why it appears intermittent. ## Fix 1. `openai_source.py`: remap upstream tool_call indexes to a contiguous 0-based sequence per request (request-local dict, not `self`, to avoid cross-request pollution under concurrency). The existing AstrBotDevs#6661 patch only handled a *missing* index, not an offset. 2. `entities.py`: in `to_openai_tool_calls_model()`, fall back to a stable `call_{idx}` id and `__malformed_tool_name__` when the upstream id/name is missing or invalid, so a single malformed tool_call can no longer fail the entire turn. `extra_content` is still looked up by the raw upstream id. ## Impact Affects `ProviderOpenAIOfficial` and its 7 subclasses (groq, longcat, oai_aihubmix, openrouter, xai, xiaomi, zhipu) — all share `openai_source.py`; only that file uses `ChatCompletionStreamState`, so the patch covers all of them. Closes AstrBotDevs#9590
x1051445024
force-pushed
the
fix/toolcall-index-offset-9590
branch
from
August 7, 2026 14:29
856930e to
718d4bf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题概述
部分 OpenAI 兼容网关(如 new-api 风格的代理)在流式返回时把
tool_call.index从 1 开始编号(OpenAI 规范要求从 0 开始)。openai SDK 的流式快照累加器直接把 index 当作列表下标使用:_accumulate_chunk:tool_calls[tool_call_chunk.index]_build_events:tool_calls[tool_call_delta.index]于是
index=1落到空列表上会抛IndexError: list index out of range。AstrBot 在_query_stream中吞掉该异常(仅记录Saving chunk state error),导致快照被写坏:同一个 tool_call 的name与arguments被拆到两个错位的槽位,中间多出id=None/name=""的幽灵条目。下游ToolCall(id=None)随即抛 pydanticValidationError,整轮工具调用(如子代理委托)失败。只在流式 + 多个并行 tool_call 时触发,因此表现为偶发,单工具调用复现不出来。
与已有 issue 的关系
indexfield (e.g. Gemini) #6661 的补丁只处理 index 缺失(if not hasattr(tc, "index") or tc.index is None: tc.index = idx),不处理 index 偏移(从 1 开始)—— 本 PR 正是在该补丁基础上补上偏移分支。list index out of range+解析参数失败组合,同样经 new-api 网关),但当时只在下游tool_loop_agent_runner过滤了None名称 —— 治症状没治源头,所以换个入口又复现。tool_loop_agent_runner.py已为畸形 name 兜底(MALFORMED_TOOL_NAME_PLACEHOLDER),openai_source.py已为非法 arguments 兜底(置{}),唯独 id 从未兜底 —— 这正是本故障仍会崩的原因。日志签名(按序出现)
修复内容(两处)
openai_source.py— index 重映射:在_query_stream内维护每次请求独立的tool_call_index_map(请求级局部变量,不能挂self,否则并发请求互相污染下标映射),把上游 index 重映射为从 0 开始的连续序号。现有 Streaming tool_call arguments lost when OpenAI-compatible proxy omitsindexfield (e.g. Gemini) #6661 补丁只处理 index 缺失,本补丁在其else分支处理偏移。entities.py— id/name 兜底:to_openai_tool_calls_model()中,当上游返回的 id 缺失/非法时回退为稳定的call_{idx},name 缺失时回退为__malformed_tool_name__(与 tool_loop_agent_runner 的占位符一致),并做列表长度边界判断。extra_content仍按上游原始 id 查询(兜底 id 不会存在于该字典中)。验证
用真实上游形态的 SSE 流(index 从 1 开始、多个并行 tool_call)做 SDK 层对照测试:
handle_chunk失败 2 次(IndexError),最终快照 3 条 tool_call,其中 1 条id=None, name=None幽灵条目,且 arguments 错位(es_search 的 query 参数跑到了 web_search 上)。handle_chunk失败 0 次,2 条 tool_call 的 id/name/arguments 全部正确合并。entities 兜底层用例(id=None、name 空、列表比 args 短)均通过,deprecated 别名
to_openai_to_calls_model()保持可用。影响范围
ProviderOpenAIOfficial及其 7 个子类(groq / longcat / oai_aihubmix / openrouter / xai / xiaomi / zhipu)共用openai_source.py,且只有该文件使用ChatCompletionStreamState,因此改一处即覆盖全部 8 个 provider。关联 issue:#9590
Summary by Sourcery
Normalize tool_call indexes from OpenAI-compatible streaming providers and harden ToolCall model construction against malformed upstream data.
Bug Fixes:
Enhancements: