Skip to content

fix(benchmark): share benchmark model manifest across web and API#723

Draft
madara88645 wants to merge 1 commit into
mainfrom
fix/benchmark-model-drift-guard
Draft

fix(benchmark): share benchmark model manifest across web and API#723
madara88645 wants to merge 1 commit into
mainfrom
fix/benchmark-model-drift-guard

Conversation

@madara88645
Copy link
Copy Markdown
Owner

What changed

  • moved benchmark model definitions into a shared web/app/benchmark/models.json file
  • made the API load supported benchmark models and the default model from that shared manifest
  • kept the frontend catalog wired to the same manifest so both sides stay aligned
  • added a backend regression test that locks the API model list to the shared manifest

What this does for the product

This keeps the benchmark model dropdown and the backend validator reading the same source of truth, so we avoid the situation where users can pick a model in the UI that the server later rejects.

Risk

Low. Scope is limited to benchmark model configuration and related tests.

Verification

  • py -3.12 -m pytest tests/test_benchmark_api.py -q
  • npm run test -- app/benchmark/page.test.tsx
  • node --test web/app/benchmark/modelCatalog.test.mts

@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
compiler Ready Ready Preview, Comment Jun 1, 2026 7:51pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant