Skip to content

[None][fix] validate reasoning item content in harmony responses input #17738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lonexreb wants to merge 3 commits into
base: main
Choose a base branch
from
+17 −3
Open

[None][fix] validate reasoning item content in harmony responses input #17738

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e212130
[None][fix] validate reasoning item content in harmony responses input
lonexreb Aug 15, 2026
f989376
[None][fix] ignore reasoning input items without content per review
lonexreb Aug 26, 2026
e47a848
[None][chore] apply yapf formatting to reasoning fallback debug log
lonexreb Sep 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions tensorrt_llm/serve/responses_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,23 @@ def _parse_response_input(
Author.new(Role.TOOL, f"functions.{call_response.name}"),
input_msg["output"])
elif input_msg["type"] == "reasoning":
content = input_msg["content"]
assert len(content) == 1
msg = Message.from_role_and_content(Role.ASSISTANT, content[0]["text"])
content = input_msg.get("content") or []
if not content:
# The Responses API allows reasoning items without 'content'
# (e.g. when a prior turn is replayed and the reasoning text
# would have to be inferred from 'id' or recovered from
# 'encrypted_content', which is not implemented). Follow the
# standard fallback behavior and ignore such items.
logger.debug("Responses API: ignoring reasoning input item without "
"'content'")
msg = None
elif len(content) > 1:
raise ValueError(

@tongyuantongyu tongyuantongyu Aug 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you have stated, OpenAI Responses API allows reasoning "content" to be missing, and infers it from "id" or recovered from "encrypted_content".

Can you either

  • Make it clear in the error message that the support is unimplemented instead of the request being invalid
  • Follow the fallback standard behavior to ignore an empty reasoning message

@lonexreb lonexreb Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tongyuantongyu — went with the second option in f989376: a reasoning item with missing or empty content is now skipped (with a debug log noting that recovery from id/encrypted_content is unimplemented), matching the standard fallback behavior. The caller already filters None messages, so replayed turns parse cleanly. The descriptive error is kept only for the >1 content elements case.

"Input item of type 'reasoning' must have at most one "
f"'content' element, got {len(content)}")
else:
msg = Message.from_role_and_content(Role.ASSISTANT,
content[0]["text"])
elif input_msg["type"] == "function_call":
msg = Message.from_role_and_content(Role.ASSISTANT,
input_msg["arguments"])
Expand Down
Loading