From d20cc3063effa4e701a93493154a31b92bd0fca1 Mon Sep 17 00:00:00 2001 From: Guillaume Gay Date: Fri, 21 Aug 2026 00:06:18 +0200 Subject: [PATCH] =?UTF-8?q?feat(mcp):=20list=5Fprovider=5Fmodels=20?= =?UTF-8?q?=E2=80=94=20a=20provider's=20whole=20surface=20in=20one=20call?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces find_models_supporting. Picking a model on Bedrock or Vertex is not the question the MCP tools answered. The provider is fixed — that is what the AI credits are tied to — and the open questions are which models it serves, what each one is called on the wire, and which parameters each one takes. Answering that cost one list_models call plus one get_model_params call per model: 57 round trips for Bedrock. list_provider_models answers it in one. Models that share a parameter surface share a profile, because a provider's models are far less varied than their count suggests: Converse normalises Bedrock's 56 models down to four distinct surfaces. That grouping is what makes the whole provider fit in a tool result at all — flat, Bedrock is 25.8k tokens against 4.4k as profiles. Each row carries the model's wireId and, when tracked, its lifecycle status, so an agent choosing from this list sees a retirement on the row it is choosing from rather than only on a separate lookup. The reply says so when a wireId carries the {scope} placeholder, since a Bedrock caller has to substitute a routing geography into it before sending. find_models_supporting goes because the inverse index it offered — which models expose parameter X — is subsumed once a provider's whole surface arrives in one call, and four tools is the right budget for an agent's context. Exporting PROVIDER_ENDPOINTS from modelparams is what lets the reply name the endpoint the models are served from. --- README.md | 2 +- api/mcp.ts | 2 +- packages/modelparams-mcp/README.md | 40 ++++- packages/modelparams-mcp/src/server.ts | 27 ++-- packages/modelparams-mcp/src/tools.ts | 131 +++++++++++----- packages/modelparams-mcp/tests/server.test.ts | 145 +++++++++++++++--- packages/modelparams/src/index.ts | 1 + skills/llm-model-parameters/SKILL.md | 3 +- src/data/llms.ts | 2 +- src/views/api.ejs | 2 +- tests/mcp-endpoint-protocol.test.ts | 2 +- tests/mcp-endpoint.test.ts | 2 +- 12 files changed, 281 insertions(+), 78 deletions(-) diff --git a/README.md b/README.md index a818c9f..e4d40f0 100644 --- a/README.md +++ b/README.md @@ -125,7 +125,7 @@ claude mcp add --transport http modelparams https://modelparams.dev/mcp codex mcp add modelparams --url https://modelparams.dev/mcp ``` -Four tools: `validate_model_params` to check a params object before you send it, `get_model_params` for one model's parameter surface and lifecycle metadata, `list_models` and `find_models_supporting` to search the catalog. +Four tools: `validate_model_params` to check a params object before you send it, `get_model_params` for one model's parameter surface and lifecycle metadata, `list_provider_models` for everything one provider serves, and `list_models` to search the catalog. **Without MCP** — `npx skills add mnfst/modelparams.dev` installs the companion agent skill, and [llms.txt](https://modelparams.dev/llms.txt) points an agent at a URL. diff --git a/api/mcp.ts b/api/mcp.ts index d55928b..b9939cc 100644 --- a/api/mcp.ts +++ b/api/mcp.ts @@ -34,7 +34,7 @@ const USAGE = { "parameters a model accepts, and validates a params object before you send it.", transport: "streamable-http", stateless: true, - tools: ["validate_model_params", "get_model_params", "list_models", "find_models_supporting"], + tools: ["validate_model_params", "get_model_params", "list_models", "list_provider_models"], install: { "claude-code": "claude mcp add --transport http modelparams https://modelparams.dev/mcp", codex: "codex mcp add modelparams --url https://modelparams.dev/mcp", diff --git a/packages/modelparams-mcp/README.md b/packages/modelparams-mcp/README.md index ab5fe80..2f6e09f 100644 --- a/packages/modelparams-mcp/README.md +++ b/packages/modelparams-mcp/README.md @@ -83,9 +83,45 @@ Every parameter for one model — type, range, enum values, default, and the con Model ids, filtered by provider or substring. Use it to resolve the exact catalog id. -### `find_models_supporting` +### `list_provider_models` -Which models expose a given parameter — `reasoning_effort`, `thinking.budget_tokens`, `top_k`. Answers "which models let me set X". Returns near-miss suggestions when nothing matches, since callers usually have the provider's spelling rather than the catalog's. +Everything one provider serves, in one call: its models, the `wireId` each one needs, their lifecycle status, and the parameter surfaces they expose. For picking a model *and* configuring it in the same step — the question you have when a provider is fixed and the model isn't, which is the normal state of affairs on Bedrock and Vertex. + +Models are grouped by parameter surface rather than listed one by one, because a provider's models are far less varied than their count suggests — Bedrock's 56 models are four distinct surfaces, since Converse normalises most of them: + +```json +{ "provider": "bedrock" } +``` + +```json +{ + "provider": "bedrock", + "baseUrls": ["https://bedrock-runtime.*.amazonaws.com"], + "total": 56, + "returned": 56, + "truncated": false, + "paramProfiles": [ + { + "id": "p1", + "modelCount": 41, + "parameterCount": 4, + "params": [{ "path": "inferenceConfig.maxTokens", "type": "integer", "range": { "min": 1 } }], + "defaults": {} + } + ], + "models": [ + { + "model": "bedrock/claude-opus-4-6", + "authType": "api_key", + "wireId": "{scope}.anthropic.claude-opus-4-6-v1", + "profile": "p2" + } + ], + "wireIdNote": "A wireId containing {scope} needs one substitution before you send it…" +} +``` + +That grouping is what makes the whole provider fit in a tool result: flat, Bedrock's parameters are roughly an order of magnitude larger than the profile form. ## Model ids diff --git a/packages/modelparams-mcp/src/server.ts b/packages/modelparams-mcp/src/server.ts index 9041a50..5cf82d7 100644 --- a/packages/modelparams-mcp/src/server.ts +++ b/packages/modelparams-mcp/src/server.ts @@ -2,9 +2,9 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; import { CATALOG } from "modelparams"; import { z } from "zod"; import { - findModelsSupporting, getModelParams, listCatalogModels, + listProviderModels, validateModelParams, type ToolPayload, } from "./tools.js"; @@ -116,21 +116,28 @@ export function createServer(version = "0.0.0"): McpServer { ); server.registerTool( - "find_models_supporting", + "list_provider_models", { - title: "Find models supporting a parameter", + title: "List a provider's models and their parameters", description: - 'Find which models accept a given parameter, e.g. "reasoning_effort", ' + - '"thinking.budget_tokens", or "top_k". Use to answer "which models support X" or to ' + - "pick a model that exposes a knob you need.", + "Everything one provider serves: its models, the wire id each one needs, their " + + "lifecycle status, and the parameter surfaces they expose — types, ranges, enum values, " + + "defaults, and conditional rules — in a single call. Models that share a parameter " + + "surface share a profile, so a provider with dozens of models stays small enough to " + + "read. Use this to pick a model and configure it in one step, especially on Bedrock and " + + "Vertex where the parameter paths differ from the underlying model's native API.", inputSchema: { - parameter: z.string().describe('Exact parameter path, e.g. "top_k" or "thinking.type".'), - provider: z.string().optional().describe("Restrict to one provider slug."), - limit: z.number().int().positive().max(1000).optional().describe("Default 100."), + provider: z + .string() + .describe( + 'Provider slug, e.g. "bedrock", "vertex", "anthropic". Call list_models with no arguments to see them all.', + ), + query: z.string().optional().describe("Case-insensitive substring match on the model id."), + limit: z.number().int().positive().max(1000).optional().describe("Default 200."), }, annotations: READ_ONLY, }, - async (args) => reply(findModelsSupporting(args)), + async (args) => reply(listProviderModels(args)), ); return server; diff --git a/packages/modelparams-mcp/src/tools.ts b/packages/modelparams-mcp/src/tools.ts index 8b57ab8..5201111 100644 --- a/packages/modelparams-mcp/src/tools.ts +++ b/packages/modelparams-mcp/src/tools.ts @@ -4,6 +4,7 @@ import { getDefaults, getModel, listModels, + PROVIDER_ENDPOINTS, PROVIDERS, resolveByBaseUrl, resolveModelId, @@ -181,54 +182,104 @@ export function listCatalogModels(input: { }; } -/** Which models expose a given knob — the "can I use X anywhere?" question. */ -export function findModelsSupporting(input: { - parameter: string; - provider?: string; +/** + * Every parameter surface a provider serves, and which models share each one. + * + * A provider's models are far less varied than their count suggests: Bedrock's + * 56 models are four distinct parameter surfaces, because Converse normalises + * most of them. Returning one profile per surface, with the models pointing at + * it, is what makes "give me everything you have on this provider" fit in a + * tool result at all — flat, Bedrock is an order of magnitude larger. + */ +export function listProviderModels(input: { + provider: string; + query?: string; limit?: number; }): ToolPayload { - const { parameter, provider, limit = 100 } = input; - const needle = parameter.toLowerCase(); - - const matches: { model: ModelId; param: Param }[] = []; - for (const entry of CATALOG) { - if (provider && entry.provider !== provider) continue; - const param = (entry.params as readonly Param[]).find((p) => p.path.toLowerCase() === needle); - if (param) matches.push({ model: modelIdOf(entry), param }); - } + const { provider, query, limit = 200 } = input; - if (matches.length === 0) { - // A near-miss list beats an empty result — the caller usually has the - // provider's spelling of the parameter, not the catalog's. - const known = new Set(); - for (const entry of CATALOG) { - for (const p of entry.params as readonly Param[]) { - if (p.path.toLowerCase().includes(needle) || needle.includes(p.path.toLowerCase())) { - known.add(p.path); - } - } - } + if (!PROVIDERS.includes(provider as Provider)) { return { - parameter, - total: 0, - message: `No model in the catalog accepts "${parameter}".`, - similarParameters: [...known].sort().slice(0, 10), + error: "unknown_provider", + message: `"${provider}" is not a provider in the catalog.`, + providers: PROVIDERS, }; } + let entries = CATALOG.filter((e) => e.provider === provider); + if (query) { + const needle = query.toLowerCase(); + entries = entries.filter((e) => modelIdOf(e).toLowerCase().includes(needle)); + } + + const total = entries.length; + const shown = entries.slice(0, limit); + + // Group by parameter surface. Codegen emits params in a stable order, so the + // serialised array is a sound identity for "same surface". + const byShape = new Map(); + for (const entry of shown) { + const params = entry.params as readonly Param[]; + const shape = JSON.stringify(params); + const bucket = byShape.get(shape); + if (bucket) bucket.entries.push(entry); + else byShape.set(shape, { params, entries: [entry] }); + } + + // Commonest surface first, then alphabetically — so `p1` is the provider's + // house style and the ids stay stable across identical calls. + const profiles = [...byShape.values()].sort( + (a, b) => + b.entries.length - a.entries.length || + modelIdOf(a.entries[0]!).localeCompare(modelIdOf(b.entries[0]!)), + ); + const profileOf = new Map(); + profiles.forEach((profile, i) => { + for (const entry of profile.entries) profileOf.set(modelIdOf(entry), `p${i + 1}`); + }); + + const models = shown.map((entry) => { + const id = modelIdOf(entry); + return { + model: id, + authType: entry.authType, + ...("wireId" in entry ? { wireId: entry.wireId } : {}), + // Lifecycle travels with the row: a tool whose whole job is helping an + // agent choose a model must not hand back a retired one unmarked. + ...(entry.status !== undefined ? { status: entry.status } : {}), + ...(entry.replacement !== undefined ? { replacement: entry.replacement } : {}), + ...(entry.shutdownOn !== undefined ? { shutdownOn: entry.shutdownOn } : {}), + profile: profileOf.get(id), + }; + }); + + // Bedrock reaches most models through a cross-region inference profile, so the + // wire id carries a placeholder the caller has to fill in. Say so, in the + // response that hands over those ids, rather than leaving it to the docs. + const templated = shown.some((e) => "wireId" in e && e.wireId.includes("{scope}")); + return { - parameter, - total: matches.length, - returned: Math.min(matches.length, limit), - truncated: matches.length > limit, - models: matches.slice(0, limit).map(({ model, param }) => ({ - model, - type: param.type, - ...(param.default !== undefined ? { default: param.default } : {}), - ...(param.range ? { range: param.range } : {}), - ...(param.values ? { values: param.values } : {}), - ...(param.applicability ? { appliesOnlyWhen: param.applicability } : {}), + provider, + baseUrls: PROVIDER_ENDPOINTS[provider as keyof typeof PROVIDER_ENDPOINTS], + total, + returned: shown.length, + truncated: total > shown.length, + paramProfiles: profiles.map((profile, i) => ({ + id: `p${i + 1}`, + modelCount: profile.entries.length, + parameterCount: profile.params.length, + params: profile.params.map(describeParam), + defaults: getDefaults(modelIdOf(profile.entries[0]!) as ModelId), })), - docs: `https://modelparams.dev/parameters/${parameter.replace(/\./g, "-")}`, + models, + ...(templated + ? { + wireIdNote: + "A wireId containing {scope} needs one substitution before you send it: the " + + "routing geography of the inference profile — us, eu, apac, jp, au, ca, sa, or " + + "global. A wireId without the placeholder is already complete.", + } + : {}), + docs: `https://modelparams.dev/providers/${provider}`, }; } diff --git a/packages/modelparams-mcp/tests/server.test.ts b/packages/modelparams-mcp/tests/server.test.ts index 5b794f9..e1cc6c4 100644 --- a/packages/modelparams-mcp/tests/server.test.ts +++ b/packages/modelparams-mcp/tests/server.test.ts @@ -41,11 +41,31 @@ interface ListResult { error?: string; } -interface FindResult { - parameter: string; +interface ProviderModelsResult { + provider: string; + baseUrls: string[]; total: number; - models: { model: string; type: string }[]; - similarParameters?: string[]; + returned: number; + truncated: boolean; + paramProfiles: { + id: string; + modelCount: number; + parameterCount: number; + params: ParamInfo[]; + defaults: Record; + }[]; + models: { + model: string; + authType: string; + wireId?: string; + profile: string; + status?: string; + replacement?: string; + shutdownOn?: string; + }[]; + wireIdNote?: string; + error?: string; + providers?: string[]; } /** Parse the JSON payload a tool replies with. */ @@ -70,9 +90,9 @@ describe("tool discovery", () => { it("advertises the four catalog tools", async () => { const { tools } = await client.listTools(); expect(tools.map((t) => t.name).sort()).toEqual([ - "find_models_supporting", "get_model_params", "list_models", + "list_provider_models", "validate_model_params", ]); }); @@ -236,24 +256,111 @@ describe("list_models", () => { }); }); -describe("find_models_supporting", () => { - it("finds models exposing a parameter", async () => { - const out = await call("find_models_supporting", { parameter: "top_k" }); - expect(out.total).toBeGreaterThan(0); - expect(out.models[0]!.model).toBeTruthy(); +describe("list_provider_models", () => { + it("returns a provider's whole surface in one call", async () => { + const out = await call("list_provider_models", { provider: "bedrock" }); + expect(out.provider).toBe("bedrock"); + expect(out.total).toBeGreaterThan(10); + expect(out.truncated).toBe(false); + expect(out.models).toHaveLength(out.total); + expect(out.baseUrls.length).toBeGreaterThan(0); }); - it("scopes to a provider", async () => { - const out = await call("find_models_supporting", { - parameter: "top_k", - provider: "anthropic", + it("collapses shared parameter surfaces into far fewer profiles than models", async () => { + const out = await call("list_provider_models", { provider: "bedrock" }); + expect(out.paramProfiles.length).toBeLessThan(out.models.length / 4); + // Commonest surface first, so p1 is the provider's house style. + expect(out.paramProfiles[0]!.id).toBe("p1"); + expect(out.paramProfiles[0]!.modelCount).toBeGreaterThanOrEqual( + out.paramProfiles[out.paramProfiles.length - 1]!.modelCount, + ); + }); + + it("points every model at a profile that exists", async () => { + const out = await call("list_provider_models", { provider: "vertex" }); + const ids = new Set(out.paramProfiles.map((p) => p.id)); + expect(out.models.every((m) => ids.has(m.profile))).toBe(true); + }); + + it("groups models by surface rather than by name", async () => { + const out = await call("list_provider_models", { provider: "bedrock" }); + const byProfile = new Map(); + for (const profile of out.paramProfiles) byProfile.set(profile.id, profile.params); + // Two models on the same profile must agree parameter-for-parameter. + const grouped = out.models.filter((m) => m.profile === "p1"); + expect(grouped.length).toBeGreaterThan(1); + const paths = byProfile.get("p1")!.map((p) => p.path); + expect(paths).toContain("inferenceConfig.maxTokens"); + }); + + it("hands over the wire id, and flags the ones needing substitution", async () => { + const out = await call("list_provider_models", { provider: "bedrock" }); + const templated = out.models.find((m) => m.wireId?.includes("{scope}")); + expect(templated).toBeDefined(); + expect(out.wireIdNote).toContain("{scope}"); + }); + + it("omits the wire id note for a provider that has no placeholders", async () => { + const out = await call("list_provider_models", { provider: "openai" }); + expect(out.wireIdNote).toBeUndefined(); + }); + + it("filters by substring", async () => { + const out = await call("list_provider_models", { + provider: "bedrock", + query: "claude", + }); + expect(out.models.length).toBeGreaterThan(0); + expect(out.models.every((m) => m.model.includes("claude"))).toBe(true); + }); + + it("flags truncation rather than silently cutting off", async () => { + const out = await call("list_provider_models", { + provider: "bedrock", + limit: 3, }); - expect(out.models.every((m) => m.model.startsWith("anthropic/"))).toBe(true); + expect(out.models).toHaveLength(3); + expect(out.truncated).toBe(true); + expect(out.total).toBeGreaterThan(3); }); - it("suggests near misses when nothing matches", async () => { - const out = await call("find_models_supporting", { parameter: "temperatur" }); - expect(out.total).toBe(0); - expect(out.similarParameters).toContain("temperature"); + it("marks a model whose lifecycle is tracked", async () => { + const entry = getModel("openai/gpt-5.5") as unknown as Record; + const previous = { + status: entry.status, + replacement: entry.replacement, + shutdownOn: entry.shutdownOn, + }; + + Object.assign(entry, { + status: "deprecated", + replacement: "openai/gpt-5.6-sol", + shutdownOn: "2026-10-23", + }); + + try { + const out = await call("list_provider_models", { + provider: "openai", + query: "gpt-5.5", + }); + // An agent choosing from this list must see the retirement on the row it + // is choosing from, not only if it looks the model up separately. + expect(out.models.find((m) => m.model === "openai/gpt-5.5")).toMatchObject({ + status: "deprecated", + replacement: "openai/gpt-5.6-sol", + shutdownOn: "2026-10-23", + }); + } finally { + for (const [key, value] of Object.entries(previous)) { + if (value === undefined) delete entry[key]; + else entry[key] = value; + } + } + }); + + it("reports an unknown provider with the valid set", async () => { + const out = await call("list_provider_models", { provider: "aws" }); + expect(out.error).toBe("unknown_provider"); + expect(out.providers).toContain("bedrock"); }); }); diff --git a/packages/modelparams/src/index.ts b/packages/modelparams/src/index.ts index ce16bc3..cc2502b 100644 --- a/packages/modelparams/src/index.ts +++ b/packages/modelparams/src/index.ts @@ -23,6 +23,7 @@ export type { } from "./standard-schema.js"; export { MODEL_IDS, PROVIDERS } from "./generated/model-ids.js"; +export { PROVIDER_ENDPOINTS } from "./generated/provider-endpoints.js"; export { DEFAULTS } from "./generated/defaults.js"; export { CATALOG, BY_ID } from "./generated/data.js"; diff --git a/skills/llm-model-parameters/SKILL.md b/skills/llm-model-parameters/SKILL.md index 6a77087..1e92ec3 100644 --- a/skills/llm-model-parameters/SKILL.md +++ b/skills/llm-model-parameters/SKILL.md @@ -35,7 +35,8 @@ If the `modelparams` MCP server is connected, prefer these — they need no netw - `get_model_params` — every parameter for one model, with types, ranges, defaults, and conditional rules. - `list_models` — find the exact catalog id, filtered by provider or substring. -- `find_models_supporting` — which models expose a given parameter. +- `list_provider_models` — every model one provider serves, with its wire id and + parameter surface. Reach for it when the provider is fixed and the model is not. Not connected? Install it: diff --git a/src/data/llms.ts b/src/data/llms.ts index 2baafde..c5b4edb 100644 --- a/src/data/llms.ts +++ b/src/data/llms.ts @@ -109,7 +109,7 @@ function guideApi(siteUrl: string): string[] { "```", "", "Tools: `validate_model_params`, `get_model_params`, `list_models`,", - "`find_models_supporting`.", + "`list_provider_models`.", "", "## JSON Schema", "", diff --git a/src/views/api.ejs b/src/views/api.ejs index 9d3c297..2a53b80 100644 --- a/src/views/api.ejs +++ b/src/views/api.ejs @@ -126,7 +126,7 @@ codex mcp add modelparams --url https://modelparams.dev/mcp validate_model_params, get_model_params, list_models, - find_models_supporting. + list_provider_models.

Using a coding agent that supports skills? Install the companion skill with diff --git a/tests/mcp-endpoint-protocol.test.ts b/tests/mcp-endpoint-protocol.test.ts index 9682a01..b6e0968 100644 --- a/tests/mcp-endpoint-protocol.test.ts +++ b/tests/mcp-endpoint-protocol.test.ts @@ -50,9 +50,9 @@ describe("the MCP endpoint over real HTTP", () => { it("advertises the four tools", async () => { const { tools } = await client.listTools(); expect(tools.map((tool) => tool.name).sort()).toEqual([ - "find_models_supporting", "get_model_params", "list_models", + "list_provider_models", "validate_model_params", ]); }); diff --git a/tests/mcp-endpoint.test.ts b/tests/mcp-endpoint.test.ts index a3cb4b5..a495096 100644 --- a/tests/mcp-endpoint.test.ts +++ b/tests/mcp-endpoint.test.ts @@ -60,9 +60,9 @@ describe("POST /mcp", () => { const body = (await res.json()) as RpcReply; const names = (body.result as { tools: { name: string }[] }).tools.map((t) => t.name).sort(); expect(names).toEqual([ - "find_models_supporting", "get_model_params", "list_models", + "list_provider_models", "validate_model_params", ]); });