Skip to content

OpenRouter provider hardcodes reasoning:{enabled:true} — wedges non-thinking models into token-repetition garbage on some backends #319

Description

@Sunster1970

providers/openrouter.py unconditionally sends:

body = {"reasoning": {"enabled": True, "max_tokens": max_tokens, "exclude": True}}

for every model. Reasoning-token mode is an endpoint-specific capability; for non-thinking models (e.g. deepseek-chat-v3-0324) support depends on which upstream provider OpenRouter routes to.

Measured effect (A/B, frozen identical prompt, temperature 0.1, 400 max_tokens):

Request shape Backend A (DeepInfra) Backend B (SiliconFlow)
WITHOUT reasoning field 22/22 clean all clean
WITH reasoning field 25/26 garbage (~96% fail, token-repetition, finish_reason: length) all clean

The garbage output is decoder collapse: repeated token ladders, foreign-script tokens not present anywhere in the prompt, whole output budget burned. Combined with the session_id sticky routing (which pins a session to one backend), an agent can be glued to a failing backend for hours — the operator sees intermittent multi-hour "the model went insane" episodes that are actually request-shape × backend interactions.

Fix we run locally: make the reasoning block opt-in per model/config (send it only when explicitly enabled; log the decision each call). One screen of code; happy to PR it. Also worth noting for operators: provider: {"order": [...], "allow_fallbacks": false} gives determinism when a specific backend is misbehaving.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions