Skip to content

feat: strip request params models declare unsupported - #397

Merged
aaliboyev merged 1 commit into
mainfrom
feat/unsupported-param-gating
Jul 28, 2026
Merged

feat: strip request params models declare unsupported#397
aaliboyev merged 1 commit into
mainfrom
feat/unsupported-param-gating

Conversation

@aaliboyev

@aaliboyev aaliboyev commented Jul 27, 2026

Copy link
Copy Markdown
Member

Some models reject sampling params the adapters forward unconditionally (#394, #395) — customer requests fail with upstream 400s the relay could prevent. This makes param support capability data, gated once in code:

  • model.Capabilities gains unsupportedParams (inverted: zero value = everything supported; unknown names inert). Mirrored in the SDK catalog structs; cmd/modelsdev-import now maps models.dev's explicit temperature: false into it (*bool so an absent field is never misread).
  • Dispatch strips flagged params pre-upstream: canonical path via the sampling opts, byte-pass path via Spec.ParamPaths (wire locations declared on the Spec literals; same json.RawMessage technique as the model-field rewrite — bodies with nothing to strip pass through byte-identical). The batch runner reuses the strip via inference.StripUnsupportedParams; WS already flows through Dispatch. Proxy mode is deliberately untouched.
  • Never silent: every drop sets X-WR-Warnings: dropped_params=… and increments relay_dropped_params_total{param,model}.

Fixing a newly drifted model then needs only a catalog data change — no adapter or relay release.

Fixes #394. Fixes #395 (data side: wyolet/relay-catalog#27).

Models can declare rejected sampling params
(capabilities.unsupportedParams, imported from models.dev). Dispatch
strips them pre-upstream on the canonical and byte-pass paths (batch
included), surfacing every drop via X-WR-Warnings and a Prometheus
counter. Refs #394, #395.
@aaliboyev
aaliboyev merged commit df3e11a into main Jul 28, 2026
1 check passed
@aaliboyev
aaliboyev deleted the feat/unsupported-param-gating branch July 28, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

api drift: openai rejects top_p api drift: openai restricts temperature to its default on gpt-5

1 participant