Problem
Since #74, llm:choose always sends its options as a strict json_schema response format on OpenAI-compatible providers (OpenAICompatibleProvider.applyResponseFormat). Models that do not support structured outputs, such as gpt-3.5-turbo, or providers that validate the format server-side, reject the request with HTTP 400. The bare-text fallback in extractEnumChoice only helps when a provider ignores the format; it cannot recover from an HTTP rejection, so a choice that worked before #74 now errors.
Found by a Codex review on 2026-09-08. The Groq-specific prompt/schema mismatch was fixed in #77; this is the remaining unsupported-format case.
Fix
Either gate native constraints on a per-model capability in models.yaml, or catch a format-rejection 400 (unsupported/response_format in the error body) and retry the same request unconstrained through the prompted path. The retry is smaller and self-healing.
Test
A stub backend that returns 400 for any request carrying response_format, then 200 for the retry. Current degradation tests only simulate a provider ignoring the format.
Problem
Since #74,
llm:choosealways sends its options as a strictjson_schemaresponse format on OpenAI-compatible providers (OpenAICompatibleProvider.applyResponseFormat). Models that do not support structured outputs, such asgpt-3.5-turbo, or providers that validate the format server-side, reject the request with HTTP 400. The bare-text fallback inextractEnumChoiceonly helps when a provider ignores the format; it cannot recover from an HTTP rejection, so a choice that worked before #74 now errors.Found by a Codex review on 2026-09-08. The Groq-specific prompt/schema mismatch was fixed in #77; this is the remaining unsupported-format case.
Fix
Either gate native constraints on a per-model capability in
models.yaml, or catch a format-rejection 400 (unsupported/response_formatin the error body) and retry the same request unconstrained through the prompted path. The retry is smaller and self-healing.Test
A stub backend that returns 400 for any request carrying
response_format, then 200 for the retry. Current degradation tests only simulate a provider ignoring the format.