Skip to content

Enable multi-turn messages#618

Open
rebel-jinhwan wants to merge 6 commits into
vllm-project:mainfrom
rebel-jinhwan:feat/support-multi-turn
Open

Enable multi-turn messages#618
rebel-jinhwan wants to merge 6 commits into
vllm-project:mainfrom
rebel-jinhwan:feat/support-multi-turn

Conversation

@rebel-jinhwan

@rebel-jinhwan rebel-jinhwan commented Mar 3, 2026

Copy link
Copy Markdown

Summary

This PR adds support for multi-turn chat messages in GuideLLM. When the dataset provides a messages (or conversations / conversation) column, the OpenAI-style request handler uses that column as the full conversation for each request instead of building a single-turn message from prefix_column + text_column + media columns

Details

  • Added "messages_column" to the literal type for dataset column types.
  • Column mapping (src/guidellm/data/preprocessors/mappers.py): messages_column maps from names ["messages", "conversations", "conversation"] so datasets with those column names are recognized.
  • Chat completions formatting (src/guidellm/backends/openai/request_handlers.py): In format(), when building the request body:
    1. Read raw_messages = data.columns.get("messages_column", []).
    2. If raw_messages is a non-empty list and the first element is a list of dicts with role and content, set arguments.body["messages"] = conversation and return.
    3. Otherwise, keep existing behavior: build messages from prefix_column and text_column (and image/video/audio columns) as a single system + user turn.

No changes to scheduler or backend interfaces are required for this; the backend still receives a single request per row, with that request’s body["messages"] already containing the full conversation when messages_column is used.

Test Plan

Related Issues

  • Resolves #

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@rebel-jinhwan rebel-jinhwan changed the title enable multi-turn messages Enable multi-turn messages Mar 3, 2026
@rebel-jinhwan rebel-jinhwan force-pushed the feat/support-multi-turn branch from 6a57908 to b6bd9e0 Compare March 3, 2026 16:13
@sjmonson

sjmonson commented Mar 5, 2026

Copy link
Copy Markdown
Collaborator

Sorry, I'm still mulling over how to best handle this. Ideally we would support some kind of "raw" column which gets merged with the other columns in the formatter. The problem here is that each jsonl row is a raw request so making each row a single key under a new dict level would necessitate changing the dataset format.

Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
Signed-off-by: rebel-jinhwan <jinhwan.suk@rebellions.ai>
@sjmonson sjmonson force-pushed the feat/support-multi-turn branch from 070558d to 1c2016e Compare March 13, 2026 17:44
@mergify

mergify Bot commented May 30, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @rebel-jinhwan.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label May 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants