Skip to content

fix(providers): return a clean error on empty choices in localai and OpenAI completions - #10437

Open
he-yufeng wants to merge 3 commits into
promptfoo:mainfrom
he-yufeng:fix/providers-empty-choices-localai-openai-completion
Open

fix(providers): return a clean error on empty choices in localai and OpenAI completions#10437
he-yufeng wants to merge 3 commits into
promptfoo:mainfrom
he-yufeng:fix/providers-empty-choices-localai-openai-completion

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

What

Same empty-choices treatment that just landed for openrouter/snowflake in #10418, applied to the two remaining unguarded OpenAI-compatible readers: localai.ts (chat + legacy completions paths) and openai/completion.ts.

A 200 response with an empty or missing choices array currently falls into these files' catch-all try/catch and comes back as an opaque API response error: TypeError: Cannot read properties of undefined. With this change it surfaces as a clean Malformed response data: … error, same as the siblings.

The guards check for the choice object's presence (not text truthiness), so a legitimate empty-string completion still passes through.

Tests

  • test/providers/localai.test.ts: both LocalAI providers return the clean error on choices: [].
  • test/providers/openai/completion.test.ts: new empty-choices case plus the existing missing-choices parsing test updated to the new message (its old expectation pinned the opaque TypeError text).
  • 32/32 pass across both files; biome/eslint/prettier clean.

@promptfoo-scanner promptfoo-scanner Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 All Clear

I reviewed the provider changes for LocalAI and OpenAI completions. The PR adds defensive checks to return clean errors when upstream returns 200 responses with empty choices, improving robustness. No new LLM capabilities, prompt flows, or execution sinks were introduced. I did not find any LLM security vulnerabilities related to the six defined classes.

Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more


Was this helpful?  👍 Yes  |  👎 No 

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c80d52fb9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/providers/localai.ts
// upstream hiccup) must surface as a clean malformed-response error, not
// an opaque TypeError from choices[0].message. Mirrors the sibling fix in
// openrouter.ts / snowflake.ts (#10418).
if (!data.choices?.[0]?.message) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard null response bodies before reading choices

When LocalAI returns a valid JSON null response body, data.choices throws before this condition can return the intended normalized error. Previously, that dereference occurred inside the following try block and produced a ProviderResponse, so this change turns that malformed-response case into a rejected callApi() promise; use data?.choices or keep the validation inside the error-handling block. The new guard in LocalAiCompletionProvider has the identical regression.

AGENTS.md reference: src/providers/AGENTS.md:L7-L12

Useful? React with 👍 / 👎.

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.12%. Comparing base (c149fcf) to head (382bc27).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10437   +/-   ##
=======================================
  Coverage   82.11%   82.12%           
=======================================
  Files         943      943           
  Lines       79176    79182    +6     
  Branches    26253    26256    +3     
=======================================
+ Hits        65016    65027   +11     
+ Misses      14160    14155    -5     
Flag Coverage Δ
backend 83.55% <100.00%> (+<0.01%) ⬆️
site 21.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@he-yufeng

Copy link
Copy Markdown
Contributor Author

The Smoke Tests red was mine: the new completion test's mock carried a severity property that FetchWithCacheResult does not have (TS2353). Dropped it in 6d5d769, both touched suites pass locally (32/32). Style Check showed a prettier drift that is also covered by the same push. The Redteam (Production API) lane is the known production-API-dependent one, not this change.

@promptfoo-scanner promptfoo-scanner Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 All Clear

I reviewed provider error-handling changes for LocalAI and OpenAI completions that add clean checks for empty or missing choices and return structured errors. The modifications reduce opaque exceptions and do not introduce new LLM tool capabilities, execution sinks, or prompt changes. Based on the diff, there are no clear paths to prompt injection, data exfiltration, insecure output handling, excessive agency, or jailbreak risks.

Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more


Was this helpful?  👍 Yes  |  👎 No 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant