Skip to content

chat : preserve object alternatives in Qwen XML tool schemas - #28651

Open
arc-uri-el wants to merge 1 commit into
ggml-org:masterfrom
arc-uri-el:fix/qwen-xml-object-unions-upstream
Open

chat : preserve object alternatives in Qwen XML tool schemas#28651
arc-uri-el wants to merge 1 commit into
ggml-org:masterfrom
arc-uri-el:fix/qwen-xml-object-unions-upstream

Conversation

@arc-uri-el

@arc-uri-el arc-uri-el commented Sep 9, 2026

Copy link
Copy Markdown

Overview

The Qwen XML handler reads only root parameters.properties. For a tool whose arguments are an object union, it therefore builds a grammar that permits an empty call and excludes every required argument. This can force repeated {} calls even though the full schema is present in the model prompt.

A minimal example is a records tool with these parameters:

{
  "oneOf": [
    {
      "type": "object",
      "properties": {"mode": {"type": "string", "const": "list"}},
      "required": ["mode"],
      "additionalProperties": false
    },
    {
      "type": "object",
      "properties": {
        "mode": {"type": "string", "const": "load"},
        "paths": {"type": "array", "items": {"type": "string"}, "minItems": 1}
      },
      "required": ["mode", "paths"],
      "additionalProperties": false
    }
  ]
}

Before this change, the generated function rule has no parameter productions. Afterward, the list branch requires mode=list; the load branch requires mode=load and paths. No client-side flattening or particular discriminator name is needed.

The patch compiles direct object alternatives separately and enforces string const/enum values in XML. oneOf is accepted only when every pair is provably disjoint through a required finite string property. Unsupported root compositions return an explicit error instead of an empty or weakened argument grammar. Nested values retain the existing JSON converter's limitations.

Union arguments are committed only once the argument branch completes. Otherwise a late discriminator can reinterpret an already-streamed value, for example from "42" to 42, which cannot be repaired with append-only deltas. Reasoning and tool names still stream. Ordinary object schemas keep their existing path except for string constant/enum enforcement.

Additional information

Validation

Base: 4850c7727fa73bbe3098e10ee369fbc3467c445f.

The same correction was proposed first in halo-box/llama.cpp#25. This port follows the current common/parsers/ file layout and contains no other downstream changes.

Built the clean base and candidate separately on Linux x86_64 with GCC 15.2, Release, shared libraries, native CPU and OpenMP. Both builds produced zero compiler warnings. All seven selected suites passed on each build:

  • test-chat
  • test-json-schema-to-grammar
  • test-chat-peg-parser
  • test-chat-auto-parser
  • test-chat-template
  • test-grammar-parser
  • test-grammar-integration

The candidate's exact test-chat binary fails its first valid root-union grammar assertion when linked against the baseline libraries and passes against the candidate libraries. Added cases exercise Qwen3-Coder and Qwen3.5 templates, required arguments in both orders, missing fields, branch-specific integer limits, array contents, unknown discriminators, overlapping alternatives, conflicting root constraints, parallel calls and byte-by-byte streaming with a late discriminator.

Separately, the identical parser delta passed downstream Vulkan tests on a Ryzen AI MAX+ 395 / Radeon 8060S with a custom 27B Qwen model. It completed tool calls, nested JSON edits, vision, daemon restart and cached return at 121K-143K occupied context with the original schemas. This is downstream evidence, not an inference result from the clean mainline CPU build. Two initial short-answer trials failed despite correct calls; a fresh matched original/patched replay produced identical reasoning and tool arguments. No broad model-quality improvement or throughput gain is claimed.

Not run: full CI, Windows or other-platform builds, other-model inference, perplexity, backend-op or broad performance sweeps. No tensor kernels, model graphs, weights or public APIs changed. Full JSON Schema conformance is outside this patch.

Requirements

  • I have read and agree with the contributing guidelines
  • This is a human-directed contribution, submitted at my explicit request. I own its review follow-up.
  • AI usage disclosure: YES. GPT-6 Astra assisted with investigation, code and regression-test development, execution, evidence analysis and preparation of this submission. No independent human code-review attestation is claimed.
  • The repository guidelines and existing submissions were checked. The change is confined to the Qwen XML parser, its property-enumeration helper, existing chat tests and function-calling documentation.

Compile direct object alternatives with their required fields and string discriminator constraints. Hold ambiguous argument deltas until the selected branch is complete. Add grammar and streaming regressions to the existing chat tests.

Assisted-by: GPT-6 Astra
@arc-uri-el
arc-uri-el requested review from a team and pwilkin as code owners September 9, 2026 12:47
@github-actions github-actions Bot added documentation Improvements or additions to documentation testing Everything test related labels Sep 9, 2026
@ggml-gh-bot

ggml-gh-bot Bot commented Sep 9, 2026

Copy link
Copy Markdown

Hi @arc-uri-el, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@ggml-gh-bot ggml-gh-bot Bot added the draft PR will be changed to draft by github-actions bot label Sep 9, 2026
@github-actions
github-actions Bot marked this pull request as draft September 9, 2026 12:53
@github-actions github-actions Bot removed the draft PR will be changed to draft by github-actions bot label Sep 9, 2026
@arc-uri-el
arc-uri-el marked this pull request as ready for review September 9, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant