feat: pluggable LLM provider interface for judge

[andrew-stelmach-fleet]

Summary

• Adds an LLMProvider abstraction layer (fleet/llm_provider.py) that decouples judge LLM calls from the Fleet orchestrator, enabling on-prem deployments
• Ships three provider implementations: FleetProvider (default/existing behavior), ExternalProvider (OpenRouter, Anthropic, local models via OpenAI-compatible API), and the abstract LLMProvider base for custom implementations
• SyncJudge, AsyncJudge, SyncEnv, and AsyncEnv accept an optional llm_provider kwarg — fully backward compatible, no changes needed for existing callers

Usage

# Default (no change needed) — routes through Fleet orchestrator
result = env.judge.grade(rubric, submission)

# External provider — direct LLM API calls (on-prem / no orchestrator)
from fleet.llm_provider import ExternalProvider

provider = ExternalProvider(
    api_key="sk-or-...",
    base_url="https://openrouter.ai/api/v1",
    model="anthropic/claude-sonnet-4",
)
judge = SyncJudge(client=None, instance_id="local", llm_provider=provider)
result = judge.grade(rubric, submission)

# Custom provider — implement your own
class MyProvider(LLMProvider):
    def grade(self, request: GradeRequest) -> GradeResponse:
        ...  # your logic here

Architecture

                        ┌─────────────────────┐
                        │   SyncJudge.grade()  │
                        └────────┬────────────┘
                                 │
                   ┌─────────────┴─────────────┐
                   │  llm_provider is None?     │
                   └──────┬──────────────┬──────┘
                     Yes  │              │  No
                          ▼              ▼
              ┌───────────────┐  ┌──────────────────┐
              │ FleetProvider │  │  LLMProvider      │
              │ (orchestrator │  │  .grade(request)  │
              │  POST /v1/    │  │                   │
              │  judge/grade) │  │  ExternalProvider │
              └───────────────┘  │  CustomProvider   │
                                 └──────────────────┘

Test plan

• 21 new unit tests in tests/test_llm_provider.py covering:
  • GradeResponse serialization
  • User message construction from rubrics (string + structured)
  • LLM JSON response parsing (valid, markdown-fenced, invalid)
  • FleetProvider delegation to orchestrator client
  • ExternalProvider request building + mocked HTTP calls
  • Custom LLMProvider implementation
  • SyncJudge routing (with provider / without provider)
  • End-to-end: ExternalProvider → SyncJudge → JudgeResult
• All 9 existing test_judge_criteria_markers.py tests still pass (backward compatibility)

# Run tests
cd fleet-sdk
uv run pytest tests/test_llm_provider.py tests/test_judge_criteria_markers.py -v

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}