System Info
- CPU architecture: N/A (reproducible with a pure-Python snippet, no GPU required)
- GPU: N/A
- TensorRT-LLM branch:
main
- TensorRT-LLM commit:
f7f596b94e
- OS: N/A
The report concerns tensorrt_llm/serve/tool_parser/qwen3_coder_parser.py and affects
streaming /v1/chat/completions responses for tool_parser: qwen3_coder, which
tool_parser_factory.py also selects for nemotron_h_omni.
Who can help?
No response
Information
Tasks
Reproduction
A tool call whose function takes no parameters is written by the model as a
<function=...> block with no <parameter> children. In streaming,
Qwen3CoderToolParser.parse_streaming_increment emits the name, then on </tool_call> it
only appends a closing brace when some argument text was already streamed
(qwen3_coder_parser.py:156-172):
current_streamed = self.streamed_args_for_tool[self.current_tool_id]
if current_streamed:
...
if open_braces > close_braces:
calls.append(ToolCallItem(..., parameters="}"))
With no parameter block nothing was streamed, so the branch is skipped and the call
completes with an empty arguments string. detect_and_parse on the same text returns
{}.
from tensorrt_llm.serve.openai_protocol import ChatCompletionToolsParam, FunctionDefinition
from tensorrt_llm.serve.tool_parser.qwen3_coder_parser import Qwen3CoderToolParser
tools = [ChatCompletionToolsParam(
type="function",
function=FunctionDefinition(
name="get_time", description="Get current time",
parameters={"type": "object", "properties": {}}))]
text = "<tool_call>\n<function=get_time>\n</function>\n</tool_call>"
streamed = Qwen3CoderToolParser().parse_streaming_increment(text, tools)
print([(c.name, c.parameters) for c in streamed.calls])
oneshot = Qwen3CoderToolParser().detect_and_parse(text, tools)
print([(c.name, c.parameters) for c in oneshot.calls])
Output on main:
[('get_time', '')]
[('get_time', '{}')]
The same happens when the call arrives across several deltas: the name is streamed, and
no argument delta ever follows.
Expected behavior
Concatenating the streamed argument deltas of a call yields the same JSON that
detect_and_parse returns for it. A zero-argument call streams {}, as the other
parsers in the package already do (Glm47ToolParser._finalize_tool_call sends "{}"
when no parameter was streamed, and BaseToolParser.parse_streaming_increment was fixed
to do the same in #17575).
actual behavior
The streaming client receives function.arguments == "", which is not valid JSON, so a
client that parses the arguments before invoking the tool fails on every zero-argument
call, while the non-streaming request for the same output gets {}.
additional notes
nemotron_h_omni is mapped to this parser in tool_parser_factory.py, so it is affected
as well.
- I have a fix ready (stream
"{}" on completion when nothing was streamed, mirroring
Glm47ToolParser) and will open a PR referencing this issue.
System Info
mainf7f596b94eThe report concerns
tensorrt_llm/serve/tool_parser/qwen3_coder_parser.pyand affectsstreaming
/v1/chat/completionsresponses fortool_parser: qwen3_coder, whichtool_parser_factory.pyalso selects fornemotron_h_omni.Who can help?
No response
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
A tool call whose function takes no parameters is written by the model as a
<function=...>block with no<parameter>children. In streaming,Qwen3CoderToolParser.parse_streaming_incrementemits the name, then on</tool_call>itonly appends a closing brace when some argument text was already streamed
(
qwen3_coder_parser.py:156-172):With no parameter block nothing was streamed, so the branch is skipped and the call
completes with an empty
argumentsstring.detect_and_parseon the same text returns{}.Output on
main:The same happens when the call arrives across several deltas: the name is streamed, and
no argument delta ever follows.
Expected behavior
Concatenating the streamed argument deltas of a call yields the same JSON that
detect_and_parsereturns for it. A zero-argument call streams{}, as the otherparsers in the package already do (
Glm47ToolParser._finalize_tool_callsends"{}"when no parameter was streamed, and
BaseToolParser.parse_streaming_incrementwas fixedto do the same in #17575).
actual behavior
The streaming client receives
function.arguments == "", which is not valid JSON, so aclient that parses the arguments before invoking the tool fails on every zero-argument
call, while the non-streaming request for the same output gets
{}.additional notes
nemotron_h_omniis mapped to this parser intool_parser_factory.py, so it is affectedas well.
"{}"on completion when nothing was streamed, mirroringGlm47ToolParser) and will open a PR referencing this issue.