fix(llm): drop falsy logprobs from payloads for strict providers

[paultranvan]

Falsy logprobs were making 400 when using /chat/completions with external LLM providers such as Gemini

Summary by CodeRabbit

• Bug Fixes

  • Prevented disabled log-probability settings from being sent in inference requests.
  • Ensured related top-log-probability settings are also omitted when log probabilities are disabled.
  • Preserved both settings when log probabilities are explicitly enabled.

• Tests

  • Added coverage for log-probability behavior across chat and text generation requests.

[coderabbitai]

📝 Walkthrough

Walkthrough

VLLM request construction now removes falsy logprobs and top_logprobs fields for chat and completion requests. Tests cover omitted fields when disabled and preserved fields when enabled.

Changes

VLLM logprob payload handling

Layer / File(s)	Summary
Conditional logprob filtering openrag/services/inference/vllm_client.py	Adds _strip_falsy_logprobs and applies it to chat-completion and legacy completion payloads.
Logprob payload coverage tests/unit/services/inference/test_vllm_client.py	Tests omission of disabled logprob fields and forwarding of enabled logprobs and top_logprobs values.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: hedhoud

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: dropping falsy logprobs from LLM payloads to support strict providers.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/llm-drop-falsy-logprobs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/unit/services/inference/test_vllm_client.py (1)

279-291: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover truthy logprobs on generate.

The new completion-path normalization is only tested with logprobs=False; add a generate(..., logprobs=True, top_logprobs=5) assertion to catch future divergence from the chat path.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/services/inference/test_vllm_client.py` around lines 279 - 291,
Add coverage for the completion path by extending the relevant inference client
tests around the existing falsey logprobs normalization test. Invoke generate
with logprobs=True and top_logprobs=5, capture the request payload, and assert
those values remain unchanged, matching test_truthy_logprobs_forwarded without
altering the chat-path test.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unit/services/inference/test_vllm_client.py`:
- Around line 279-291: Add coverage for the completion path by extending the
relevant inference client tests around the existing falsey logprobs
normalization test. Invoke generate with logprobs=True and top_logprobs=5,
capture the request payload, and assert those values remain unchanged, matching
test_truthy_logprobs_forwarded without altering the chat-path test.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 88770b89-fe8e-4beb-a52a-4d0bb6ad438c

📥 Commits

Reviewing files that changed from the base of the PR and between db3482c and bbf42d1.

📒 Files selected for processing (2)

• openrag/services/inference/vllm_client.py
• tests/unit/services/inference/test_vllm_client.py