fix(providers): return a clean error on empty choices in localai and OpenAI completions - #10437
Conversation
…OpenAI completions
There was a problem hiding this comment.
👍 All Clear
I reviewed the provider changes for LocalAI and OpenAI completions. The PR adds defensive checks to return clean errors when upstream returns 200 responses with empty choices, improving robustness. No new LLM capabilities, prompt flows, or execution sinks were introduced. I did not find any LLM security vulnerabilities related to the six defined classes.
Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c80d52fb9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // upstream hiccup) must surface as a clean malformed-response error, not | ||
| // an opaque TypeError from choices[0].message. Mirrors the sibling fix in | ||
| // openrouter.ts / snowflake.ts (#10418). | ||
| if (!data.choices?.[0]?.message) { |
There was a problem hiding this comment.
Guard null response bodies before reading choices
When LocalAI returns a valid JSON null response body, data.choices throws before this condition can return the intended normalized error. Previously, that dereference occurred inside the following try block and produced a ProviderResponse, so this change turns that malformed-response case into a rejected callApi() promise; use data?.choices or keep the validation inside the error-handling block. The new guard in LocalAiCompletionProvider has the identical regression.
AGENTS.md reference: src/providers/AGENTS.md:L7-L12
Useful? React with 👍 / 👎.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10437 +/- ##
=======================================
Coverage 82.11% 82.12%
=======================================
Files 943 943
Lines 79176 79182 +6
Branches 26253 26256 +3
=======================================
+ Hits 65016 65027 +11
+ Misses 14160 14155 -5
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
The Smoke Tests red was mine: the new completion test's mock carried a severity property that FetchWithCacheResult does not have (TS2353). Dropped it in 6d5d769, both touched suites pass locally (32/32). Style Check showed a prettier drift that is also covered by the same push. The Redteam (Production API) lane is the known production-API-dependent one, not this change. |
There was a problem hiding this comment.
👍 All Clear
I reviewed provider error-handling changes for LocalAI and OpenAI completions that add clean checks for empty or missing choices and return structured errors. The modifications reduce opaque exceptions and do not introduce new LLM tool capabilities, execution sinks, or prompt changes. Based on the diff, there are no clear paths to prompt injection, data exfiltration, insecure output handling, excessive agency, or jailbreak risks.
Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more
What
Same empty-
choicestreatment that just landed for openrouter/snowflake in #10418, applied to the two remaining unguarded OpenAI-compatible readers:localai.ts(chat + legacy completions paths) andopenai/completion.ts.A 200 response with an empty or missing
choicesarray currently falls into these files' catch-alltry/catchand comes back as an opaqueAPI response error: TypeError: Cannot read properties of undefined. With this change it surfaces as a cleanMalformed response data: …error, same as the siblings.The guards check for the choice object's presence (not text truthiness), so a legitimate empty-string completion still passes through.
Tests
test/providers/localai.test.ts: both LocalAI providers return the clean error onchoices: [].test/providers/openai/completion.test.ts: new empty-choices case plus the existing missing-choices parsing test updated to the new message (its old expectation pinned the opaque TypeError text).