Skip to content

feat(provider): add Yutori provider support for Navigator model#355

Open
dhruvbatra wants to merge 1 commit into
groq:mainfrom
dhruvbatra:feat/add-yutori-provider
Open

feat(provider): add Yutori provider support for Navigator model#355
dhruvbatra wants to merge 1 commit into
groq:mainfrom
dhruvbatra:feat/add-yutori-provider

Conversation

@dhruvbatra
Copy link
Copy Markdown

@dhruvbatra dhruvbatra commented May 21, 2026

Summary

Adds Yutori as a model provider so openbench can target Yutori's Navigator computer-use model family. The Navigator API is OpenAI-compatible Chat Completions, so the new YutoriAPI plugs into Inspect AI's standard OpenAICompatibleAPI machinery — same pattern as Baseten, SiliconFlow, etc.; closely modeled on #269.

Navigator is a screenshot-driven vision/computer-use model, not a text reasoner. This PR is just the provider plumbing — the natural pairing is a browser-use or vision benchmark (bring your own with bench eval path/to/eval.py, or wait for a future built-in).

What are you adding?

  • New model provider

Changes Made

  • New YutoriAPI(OpenAICompatibleAPI) in src/openbench/model/_providers/yutori.py.
  • @modelapi(name="yutori") registration in _registry.py; ProviderType.YUTORI + entry in PROVIDER_CONFIGS.
  • Provider row in README.md; provider row + "Yutori-Specific Configuration Options" subsection in docs/providers.mdx.
  • 21 unit tests in tests/test_yutori_provider.py.

Navigator-specific behavior in the provider

  • Navigator's Chat Completions API uses max_completion_tokens (the modern OpenAI Chat Completions field) and rejects max_tokens. The provider's completion_params() silently remaps openbench's --max-tokens flag onto max_completion_tokens so callers don't need to know about the rename. An explicit -M max_completion_tokens=N wins over the remapped value.
  • A few Navigator-only request fields are exposed as -M kwargs and forwarded as extra_body: tool_set, disable_tools, json_schema (matching the public Yutori SDK surface).

Testing

  • pytest tests/test_yutori_provider.py — 21 passed
  • pytest — full non-integration suite passing
  • pre-commit run --all-files — ruff, mypy, registry-imports all green
  • Smoke-tested: inspect_ai.get_model("yutori/n1.5-latest") resolves to YutoriAPI, picks up YUTORI_API_KEY and YUTORI_BASE_URL overrides correctly

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my feature works
  • New and existing unit tests pass locally with my changes

@dhruvbatra dhruvbatra requested a review from nmayorga7 as a code owner May 21, 2026 19:00
Adds Yutori as a model provider via the standard OpenAICompatibleAPI
pattern. Navigator's Chat Completions API uses `max_completion_tokens`
and rejects `max_tokens`, so the provider silently remaps the openbench
`--max-tokens` flag. Navigator-only request fields (`tool_set`,
`disable_tools`, `json_schema`) are exposed via `-M` and forwarded as
`extra_body`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@dhruvbatra dhruvbatra force-pushed the feat/add-yutori-provider branch from 8096d5a to e4fd32f Compare May 21, 2026 19:03
@dhruvbatra dhruvbatra changed the title feat(provider): add Yutori Navigator provider support feat(provider): add Yutori provider support for Navigator model May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant