Skip to content

[Bug]: Qwen3-Coder streaming tool parser never completes a zero-argument tool call #19227

Description

@Yigtwxx

System Info

  • CPU architecture: N/A (reproducible with a pure-Python snippet, no GPU required)
  • GPU: N/A
  • TensorRT-LLM branch: main
  • TensorRT-LLM commit: f7f596b94e
  • OS: N/A

The report concerns tensorrt_llm/serve/tool_parser/qwen3_coder_parser.py and affects
streaming /v1/chat/completions responses for tool_parser: qwen3_coder, which
tool_parser_factory.py also selects for nemotron_h_omni.

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

A tool call whose function takes no parameters is written by the model as a
<function=...> block with no <parameter> children. In streaming,
Qwen3CoderToolParser.parse_streaming_increment emits the name, then on </tool_call> it
only appends a closing brace when some argument text was already streamed
(qwen3_coder_parser.py:156-172):

current_streamed = self.streamed_args_for_tool[self.current_tool_id]
if current_streamed:
    ...
    if open_braces > close_braces:
        calls.append(ToolCallItem(..., parameters="}"))

With no parameter block nothing was streamed, so the branch is skipped and the call
completes with an empty arguments string. detect_and_parse on the same text returns
{}.

from tensorrt_llm.serve.openai_protocol import ChatCompletionToolsParam, FunctionDefinition
from tensorrt_llm.serve.tool_parser.qwen3_coder_parser import Qwen3CoderToolParser

tools = [ChatCompletionToolsParam(
    type="function",
    function=FunctionDefinition(
        name="get_time", description="Get current time",
        parameters={"type": "object", "properties": {}}))]

text = "<tool_call>\n<function=get_time>\n</function>\n</tool_call>"

streamed = Qwen3CoderToolParser().parse_streaming_increment(text, tools)
print([(c.name, c.parameters) for c in streamed.calls])
oneshot = Qwen3CoderToolParser().detect_and_parse(text, tools)
print([(c.name, c.parameters) for c in oneshot.calls])

Output on main:

[('get_time', '')]
[('get_time', '{}')]

The same happens when the call arrives across several deltas: the name is streamed, and
no argument delta ever follows.

Expected behavior

Concatenating the streamed argument deltas of a call yields the same JSON that
detect_and_parse returns for it. A zero-argument call streams {}, as the other
parsers in the package already do (Glm47ToolParser._finalize_tool_call sends "{}"
when no parameter was streamed, and BaseToolParser.parse_streaming_increment was fixed
to do the same in #17575).

actual behavior

The streaming client receives function.arguments == "", which is not valid JSON, so a
client that parses the arguments before invoking the tool fails on every zero-argument
call, while the non-streaming request for the same output gets {}.

additional notes

  • nemotron_h_omni is mapped to this parser in tool_parser_factory.py, so it is affected
    as well.
  • I have a fix ready (stream "{}" on completion when nothing was streamed, mirroring
    Glm47ToolParser) and will open a PR referencing this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions