diff --git a/CHANGELOG.md b/CHANGELOG.md index 1952be5..db535cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,45 @@ All notable changes to this project are documented here. The format follows ## [Unreleased] +### Changed — BREAKING, and it breaks a host contract rather than the wire + +- **`BeamMCP.ToolCatalog` is replaced by `BeamMCP.Catalog`, and `all/0` by `capabilities/0`.** + Every host implementing a catalog must change. While this package is `0.x` a break lands at + the **minor** position, so this is `0.4.0` and `~> 0.3.0` — the requirement the README + recommends — already excludes it. No consumer is carried across by a routine + `mix deps.update`; that is what the tight pin is for. + + `capabilities/0` returns `%{tools: [ToolSpec.t()], resources: [], prompts: []}`. `resources` + and `prompts` are **required and may be empty**. Nothing reads them yet. They exist so that + serving resources and prompts later adds a reader rather than changing this contract a second + time — the break is taken once, now, before those slices exist. + + **The callback is renamed, not just re-typed.** Keeping `all/0` while changing its return from + a list to a map would compile against every existing host and fail at the first request with a + `BadMapError`. Renaming makes the break arrive at compile time as an unimplemented callback. + + The option is `:catalog`, not `:tool_catalog` — a catalog carrying resources and prompts is + not a tool catalog, and renaming it in the same break costs less than a second one later. + + **Migration:** + + ```elixir + # before + @behaviour BeamMCP.ToolCatalog + def all, do: [%BeamMCP.ToolSpec{...}] + + # after + @behaviour BeamMCP.Catalog + def capabilities, do: %{tools: [%BeamMCP.ToolSpec{...}], resources: [], prompts: []} + ``` + +- **A malformed catalog is refused by `BeamMCP.Server.new/1`**, at startup, with a message naming + what is wrong — an absent key, a non-list `:tools`, an entry that is not a `%ToolSpec{}`, a + `capabilities/0` that does not return a map, or a module that does not export it. + `Transport.HTTP.init/1` checks only that the callback is exported, deliberately: under Plug's + default initialisation it runs at the host's **compile** time, where calling a catalog that + reads config would fail for a correct host. + ### Added - **`:authorize_body`, an optional post-read authorization hook.** `authorize/1` runs before the diff --git a/README.md b/README.md index 8484e0a..b095974 100644 --- a/README.md +++ b/README.md @@ -29,15 +29,23 @@ not. The tighter form is deliberate and is not an over-pin to be tidied away. Injection without a specification is a claim with nothing behind it, so both are declared. -**`BeamMCP.ToolCatalog`** — the host names the tools. +**`BeamMCP.Catalog`** — the host names what it offers. + +`capabilities/0` returns a map with three required keys. `resources` and `prompts` may be empty +and nothing reads them yet; they are required so that serving them later adds a reader rather +than changing this contract a second time. **An absent key is a malformed catalog, not an empty +one**, and `BeamMCP.Server.new/1` refuses it at startup rather than at the first request. ```elixir defmodule MyApp.Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do - [ + def capabilities do + %{ + resources: [], + prompts: [], + tools: [ %BeamMCP.ToolSpec{ name: :get_weather, command_class: :observe, @@ -50,7 +58,8 @@ defmodule MyApp.Catalog do "additionalProperties" => false } } - ] + ] + } end end ``` @@ -65,13 +74,13 @@ end ```elixir BeamMCP.Transport.Stdio.run( - tool_catalog: MyApp.Catalog, + catalog: MyApp.Catalog, dispatch: &MyApp.Dispatch.call/3, server_name: "my-app" ) ``` -`:tool_catalog` is required. `:dispatch` is required for `tools/call`. `:server_name` defaults +`:catalog` is required. `:dispatch` is required for `tools/call`. `:server_name` defaults to `beam_mcp`, and a host that wants its own name in `initialize` says so. ## One schema, one source @@ -110,7 +119,7 @@ no such step. ```elixir Bandit.child_spec( plug: {BeamMCP.Transport.HTTP, - tool_catalog: MyApp.Catalog, + catalog: MyApp.Catalog, dispatch: &MyApp.Dispatch.call/3, authorize: &MyApp.Auth.check/1, allowed_origins: ["https://app.example.com"]}, @@ -131,7 +140,7 @@ defmodule MyApp.Router do forward "/mcp", to: BeamMCP.Transport.HTTP, init_opts: [ - tool_catalog: MyApp.Catalog, + catalog: MyApp.Catalog, dispatch: &MyApp.Dispatch.call/3, authorize: &MyApp.Auth.check/1, allowed_origins: ["https://app.example.com"] diff --git a/lib/beam_mcp/catalog.ex b/lib/beam_mcp/catalog.ex new file mode 100644 index 0000000..b2b9288 --- /dev/null +++ b/lib/beam_mcp/catalog.ex @@ -0,0 +1,154 @@ +# SPDX-FileCopyrightText: 2026 Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 + +defmodule BeamMCP.Catalog do + @moduledoc """ + The contract a host implements to tell a `BeamMCP.Server` what it offers. + + The server holds no catalog of its own. It advertises what `capabilities/0` returns and + accepts a `tools/call` only for a tool `capabilities/0` names, so one implementation governs + both — a tool advertised by `tools/list` and refused by `tools/call` is the defect this + behaviour exists to make impossible. + + ## The shape carries keys this package does not yet serve + + %{tools: [BeamMCP.ToolSpec.t()], resources: [], prompts: []} + + `resources` and `prompts` are **required and may be empty**. Nothing reads them today. They + are here so that serving them later adds a reader rather than changing this contract a second + time, and every host that has already written `capabilities/0` keeps working when they do. + + A key that is absent is a malformed catalog, not an empty one — the two are different claims + and only one of them is checkable. + + ## Why `capabilities/0` and not `all/0` + + This behaviour replaces `BeamMCP.ToolCatalog`, whose callback was `all/0` returning a list. + Keeping the name while changing the return from a list to a map would compile against every + existing host and fail at the first request with a `BadMapError` — a silent shape change, + which is the defect class this repository keeps finding. Renaming makes the break arrive at + compile time as an unimplemented callback, which is the loudest place it can arrive. + """ + + @typedoc """ + What a host offers. Every key is required; `resources` and `prompts` are typed as generic + lists because nothing in this package reads them yet, and typing them precisely now would be + a claim about a shape no code enforces. + """ + @type t :: %{ + required(:tools) => [BeamMCP.ToolSpec.t()], + required(:resources) => list(), + required(:prompts) => list() + } + + @callback capabilities() :: t() + + @required_keys [:tools, :resources, :prompts] + + @doc """ + The tools a catalog offers. + + **This is the single reader, and that is the point rather than a convenience.** Both paths go + through it: `tools/list` advertises what it returns, and `fetch/2` decides callability from + the same call. Slice 002 fixed a real defect where advertising honoured an injected catalog + and calling ignored it — two readers, two answers. One function is how that stays fixed, and + a mutant that gives the two paths different sources is scored in + `slices/008-catalog-generalization/`. + """ + @spec tools(module()) :: [BeamMCP.ToolSpec.t()] + def tools(catalog), do: catalog.capabilities().tools + + @doc """ + Finds the spec a tool name refers to, or `:error`. + + One lookup, used by every caller that has to answer "which tool does this name mean". The + core uses it to decide whether a `tools/call` is callable at all; the HTTP transport uses it + to read the `x-mcp-header` annotations it must validate against. Two implementations of this + question would be two answers, which is precisely the header-versus-body disagreement the + transport's validation exists to prevent. + + ## This function raises on a malformed catalog, and the `@spec` does not say so + + Stated rather than caught, and the reason is the guarantee above. A host bug turned into + `:error` is indistinguishable from "no such tool" — so a catalog that is broken would present + exactly as a catalog that is working and simply does not have that tool. That is the + advertise-versus-call disagreement this behaviour exists to prevent, reintroduced by the + error handling meant to be defensive. + + Measured on this tree rather than inherited from a prior note + (`slices/008-catalog-generalization/logs/probe-fetch-spec.txt`): + + host spec.name is a binary, not an atom ArgumentError + capabilities/0 returns a non-list map BadMapError + capabilities/0 returns nil Protocol.UndefinedError + module is not loaded / does not exist UndefinedFunctionError + + Use `validate/1` to refuse these at startup instead, which is what `BeamMCP.Server.new/1` + does. + """ + @spec fetch(module(), String.t() | atom()) :: {:ok, BeamMCP.ToolSpec.t()} | :error + def fetch(catalog, name) when is_atom(name), do: fetch(catalog, Atom.to_string(name)) + + def fetch(catalog, name) when is_atom(catalog) and is_binary(name) do + catalog + |> tools() + |> Enum.find_value(:error, fn spec -> + if Atom.to_string(spec.name) == name, do: {:ok, spec} + end) + end + + def fetch(_catalog, _name), do: :error + + @doc """ + Checks that a module is a usable catalog, returning `:ok` or `{:error, reason}`. + + Calls `capabilities/0`, so it belongs where host code may safely run. `BeamMCP.Server.new/1` + calls it; the HTTP transport's `Plug` callback `init/1` deliberately does not, because Plug's + default initialisation is the host's **compile** time and a catalog reading config or ETS + there would fail for a correct host. That transport checks the export and leaves the shape to + `new/1`. (`init/1` is not linked here: it is a hidden callback, and a doc reference to it is a + broken link rather than a useful one.) + """ + @spec validate(module()) :: :ok | {:error, String.t()} + def validate(catalog) when is_atom(catalog) and not is_nil(catalog) do + with {:module, _} <- Code.ensure_compiled(catalog), + true <- function_exported?(catalog, :capabilities, 0) do + validate_shape(catalog) + else + _ -> {:error, "#{inspect(catalog)} does not export capabilities/0"} + end + end + + def validate(other), do: {:error, "expected a module, got: #{inspect(other)}"} + + defp validate_shape(catalog) do + case catalog.capabilities() do + %{} = caps -> + validate_keys(catalog, caps) + + other -> + {:error, "#{inspect(catalog)}.capabilities/0 must return a map, got: #{inspect(other)}"} + end + rescue + e -> {:error, "#{inspect(catalog)}.capabilities/0 raised #{inspect(e.__struct__)}"} + end + + defp validate_keys(catalog, caps) do + missing = Enum.reject(@required_keys, &Map.has_key?(caps, &1)) + + cond do + missing != [] -> + {:error, + "#{inspect(catalog)}.capabilities/0 is missing required key(s): #{inspect(missing)}"} + + not is_list(caps.tools) -> + {:error, "#{inspect(catalog)}.capabilities/0's :tools must be a list"} + + not Enum.all?(caps.tools, &match?(%BeamMCP.ToolSpec{}, &1)) -> + {:error, "#{inspect(catalog)}.capabilities/0's :tools must all be %BeamMCP.ToolSpec{}"} + + true -> + :ok + end + end +end diff --git a/lib/beam_mcp/server.ex b/lib/beam_mcp/server.ex index a2b80c8..7c84bb3 100644 --- a/lib/beam_mcp/server.ex +++ b/lib/beam_mcp/server.ex @@ -2,8 +2,8 @@ # SPDX-License-Identifier: Apache-2.0 defmodule BeamMCP.Server do + alias BeamMCP.Catalog alias BeamMCP.Schema - alias BeamMCP.ToolCatalog @moduledoc """ The protocol core: one message in, one response out, no process and no state of its own. @@ -30,7 +30,7 @@ defmodule BeamMCP.Server do ## What the host supplies BeamMCP.Server.new( - tool_catalog: MyApp.Catalog, # required, a BeamMCP.ToolCatalog + catalog: MyApp.Catalog, # required, a BeamMCP.Catalog dispatch: &MyApp.Dispatch.call/3, # required for tools/call server_name: "my-app" # optional, defaults to "beam_mcp" ) @@ -70,7 +70,7 @@ defmodule BeamMCP.Server do initialized?: boolean(), server_name: String.t(), shutdown?: boolean(), - tool_catalog: module(), + catalog: module(), tools_cache_scope: String.t(), tools_ttl_ms: non_neg_integer() } @@ -83,7 +83,7 @@ defmodule BeamMCP.Server do initialized?: false, server_name: Keyword.get(opts, :server_name, @default_server_name), shutdown?: false, - tool_catalog: Keyword.fetch!(opts, :tool_catalog), + catalog: fetch_catalog!(opts), # 2026-07-28 requires ttlMs and cacheScope on tools/list results. Neither is the # package's to invent: ttlMs is a freshness hint about a catalog the host owns, and # cacheScope is a disclosure decision -- "public" lets shared intermediaries cache a @@ -183,7 +183,7 @@ defmodule BeamMCP.Server do end def handle_message(state, %{"jsonrpc" => "2.0", "id" => id, "method" => "tools/list"}) do - tools = Enum.map(state.tool_catalog.all(), &tool_definition/1) + tools = state.catalog |> Catalog.tools() |> Enum.map(&tool_definition/1) # CacheableResult: 2026-07-28 requires both fields on tools/list. They are carried at # both eras rather than only the modern one -- 2025-11-25 permits any result structure, @@ -299,8 +299,35 @@ defmodule BeamMCP.Server do defp stringify(value), do: Jason.encode!(value) # One lookup governs both paths: a tool is callable exactly when the injected catalog names - # it, and the spec it returns carries the schema that will be enforced. - defp find_tool(state, name), do: ToolCatalog.fetch(state.tool_catalog, name) + # it, and the spec it returns carries the schema that will be enforced. Both this and the + # advertisement above go through `Catalog.tools/1`, so there is one reader rather than two + # call sites that could drift apart. + defp find_tool(state, name), do: Catalog.fetch(state.catalog, name) + + # THE SHAPE IS REFUSED HERE, not at the first request. `new/1` is runtime, so calling the + # host's `capabilities/0` is safe -- unlike `Transport.HTTP.init/1`, which under Plug's + # default init_mode is the host's COMPILE time. Same pattern as `:authorize`: a host that + # mis-wires this learns when it starts, not from a BadMapError in a request path. + defp fetch_catalog!(opts) do + catalog = Keyword.fetch!(opts, :catalog) + + case Catalog.validate(catalog) do + :ok -> + catalog + + {:error, reason} -> + raise ArgumentError, """ + BeamMCP.Server requires a :catalog implementing the BeamMCP.Catalog behaviour. + + #{reason} + + capabilities/0 must return a map with all three keys; resources and prompts may be + empty, but an absent key is a malformed catalog rather than an empty one: + + %{tools: [%BeamMCP.ToolSpec{}], resources: [], prompts: []} + """ + end + end # The advertised schema is the contract. Validate the wire form -- string keys, as the # client sent them -- before normalising, so `required` and `additionalProperties` diff --git a/lib/beam_mcp/tool_catalog.ex b/lib/beam_mcp/tool_catalog.ex deleted file mode 100644 index 1ea0f8b..0000000 --- a/lib/beam_mcp/tool_catalog.ex +++ /dev/null @@ -1,35 +0,0 @@ -# SPDX-FileCopyrightText: 2026 Sudo Apt Holdings LLC -# SPDX-License-Identifier: Apache-2.0 - -defmodule BeamMCP.ToolCatalog do - @moduledoc """ - The contract a host implements to tell a `BeamMCP.Server` which tools exist. - - The server holds no catalog of its own. It advertises what `all/0` returns and accepts a - `tools/call` only for a tool `all/0` names, so one implementation governs both — a tool - advertised by `tools/list` and refused by `tools/call` is the defect this behaviour exists - to make impossible. - """ - - @callback all() :: [BeamMCP.ToolSpec.t()] - - @doc """ - Finds the spec a tool name refers to, or `:error`. - - One lookup, used by every caller that has to answer "which tool does this name mean". The - core uses it to decide whether a `tools/call` is callable at all; the HTTP transport uses it - to read the `x-mcp-header` annotations it must validate against. Two implementations of this - question would be two answers, which is precisely the header-versus-body disagreement the - transport's validation exists to prevent. - """ - @spec fetch(module(), String.t() | atom()) :: {:ok, BeamMCP.ToolSpec.t()} | :error - def fetch(catalog, name) when is_atom(name), do: fetch(catalog, Atom.to_string(name)) - - def fetch(catalog, name) when is_atom(catalog) and is_binary(name) do - Enum.find_value(catalog.all(), :error, fn spec -> - if Atom.to_string(spec.name) == name, do: {:ok, spec} - end) - end - - def fetch(_catalog, _name), do: :error -end diff --git a/lib/beam_mcp/transport/http.ex b/lib/beam_mcp/transport/http.ex index 4c9db0f..181f739 100644 --- a/lib/beam_mcp/transport/http.ex +++ b/lib/beam_mcp/transport/http.ex @@ -92,7 +92,7 @@ if Code.ensure_loaded?(Plug) do Bandit.child_spec( plug: {BeamMCP.Transport.HTTP, - tool_catalog: MyApp.Catalog, + catalog: MyApp.Catalog, dispatch: &MyApp.Dispatch.call/3, authorize: &MyApp.Auth.check/1, allowed_origins: ["https://app.example.com"]}, @@ -110,8 +110,8 @@ if Code.ensure_loaded?(Plug) do require Logger + alias BeamMCP.Catalog alias BeamMCP.Server - alias BeamMCP.ToolCatalog @modern_version "2026-07-28" @version_meta_key "io.modelcontextprotocol/protocolVersion" @@ -202,7 +202,7 @@ if Code.ensure_loaded?(Plug) do """ end - catalog = Keyword.get(opts, :tool_catalog) + catalog = Keyword.get(opts, :catalog) # Checked for the behaviour, not for truthiness: `tool_catalog: true` passed the old check # and failed later, at the first tools/list, as an UndefinedFunctionError from inside the @@ -213,12 +213,19 @@ if Code.ensure_loaded?(Plug) do # `ensure_loaded?` raised ArgumentError at build time naming a perfectly valid catalog. # An option contract that rejects correct configurations is worse than the truthiness # check it replaced. + # STRUCTURAL ONLY, DELIBERATELY. This checks that the module exports `capabilities/0` + # and does NOT call it. Under Plug's default initialisation this runs at the host's + # COMPILE time, and a correct catalog that reads config or ETS would fail there -- the + # same class as the `ensure_loaded?` defect this check already carries a comment about. + # The map's SHAPE is refused by `BeamMCP.Server.new/1`, which is runtime. Recorded in + # slices/008-catalog-generalization/FINDINGS.md as the one place "refuse at init" is + # answered with "as much as is safe here". unless is_atom(catalog) and catalog != nil and match?({:module, _}, Code.ensure_compiled(catalog)) and - function_exported?(catalog, :all, 0) do + function_exported?(catalog, :capabilities, 0) do raise ArgumentError, """ - BeamMCP.Transport.HTTP requires a :tool_catalog option: a module implementing the - BeamMCP.ToolCatalog behaviour, that is, exporting all/0. + BeamMCP.Transport.HTTP requires a :catalog option: a module implementing the + BeamMCP.Catalog behaviour, that is, exporting capabilities/0. Got: #{inspect(catalog)} """ @@ -769,13 +776,13 @@ if Code.ensure_loaded?(Plug) do end) end - # One lookup, `BeamMCP.ToolCatalog.fetch/2`, is what the core uses to decide whether a tool + # One lookup, `BeamMCP.Catalog.fetch/2`, is what the core uses to decide whether a tool # is callable. The transport asks the same question of the same function: two lookups would # be two answers to "which tool does this name mean", which is the disagreement this whole # header mechanism exists to prevent. defp mirrored_params(%{"method" => "tools/call"} = message, opts) do with name when is_binary(name) <- param(message, "name"), - catalog when not is_nil(catalog) <- opts.server_opts[:tool_catalog], + catalog when not is_nil(catalog) <- opts.server_opts[:catalog], {:ok, entries} <- host_call(fn -> tool_annotations(catalog, name) end) do entries else @@ -798,14 +805,14 @@ if Code.ensure_loaded?(Plug) do # EVERYTHING THE HOST SUPPLIES IS READ INSIDE `host_call/1`, and the boundary is the whole # point of this function existing rather than the three steps sitting in the `with` above. # - # `ToolCatalog.fetch/2` was already wrapped; `spec.input_schema` and the schema walk were + # `Catalog.fetch/2` was already wrapped; `spec.input_schema` and the schema walk were # not. So a host catalog that RAISED kept the request's id, and the same host catalog # returning MALFORMED DATA -- a map where a `%ToolSpec{}` was promised -- raised `KeyError` # one line later, escaped to `call/2`'s rescue and answered `id: null`. Measured in # `slices/002-streamable-http/logs/probe-fault-ids.txt`: # - # host tool_catalog RAISES (header validation) 500 -32603 id=4242 - # host tool_catalog returns a malformed spec 500 -32603 id=nil + # host catalog RAISES (header validation) 500 -32603 id=4242 + # host catalog returns a malformed spec 500 -32603 id=nil # # One host bug, two envelopes, decided by which line it landed on. The line is not the # boundary; the host is. @@ -822,11 +829,11 @@ if Code.ensure_loaded?(Plug) do # `annotation_detail/1` interpolate it. No JSON-derived schema can produce that. It is # recorded as a survivor with this argument rather than pinned by a contrived test. # - # `ToolCatalog.fetch/2` returning anything but `{:ok, spec}` still falls through this + # `Catalog.fetch/2` returning anything but `{:ok, spec}` still falls through this # `with` unchanged, to the caller's `_ -> []`: a tool this catalog does not have mirrors # no parameters, which is not a fault. defp tool_annotations(catalog, name) do - with {:ok, spec} <- ToolCatalog.fetch(catalog, name), + with {:ok, spec} <- Catalog.fetch(catalog, name), entries = annotations(spec.input_schema), :ok <- check_annotations(name, entries) do {:ok, entries} @@ -1103,10 +1110,10 @@ if Code.ensure_loaded?(Plug) do # "every exception, throw and exit out of the host". Two round-5 lanes independently # derived the real population with one command -- # - # grep -n 'authorize_fun\.(\|ToolCatalog.fetch\|Server.handle_message' http.ex + # grep -n 'authorize_fun\.(\|Catalog.fetch\|Server.handle_message' http.ex # # -- and found three sites, of which one was inside that rescue. A host `authorize/1` or - # `tool_catalog` raising `Plug.BadRequestError` still handed its HTTP status to the adapter + # `catalog` raising `Plug.BadRequestError` still handed its HTTP status to the adapter # and dropped the envelope, on a path the docs above call possibly unauthenticated. One host # function raising one exception had two behaviours depending on which catalog lookup fired # first, which is the disagreement the single-lookup discipline exists to prevent. diff --git a/lib/beam_mcp/transport/stdio.ex b/lib/beam_mcp/transport/stdio.ex index 24b1909..24df847 100644 --- a/lib/beam_mcp/transport/stdio.ex +++ b/lib/beam_mcp/transport/stdio.ex @@ -19,12 +19,12 @@ defmodule BeamMCP.Transport.Stdio do option with no default. BeamMCP.Transport.Stdio.run( - tool_catalog: MyApp.Catalog, + catalog: MyApp.Catalog, dispatch: &MyApp.Dispatch.call/3, server_name: "my-app" ) - Options are `BeamMCP.Server.new/1`'s; `:tool_catalog` is required. + Options are `BeamMCP.Server.new/1`'s; `:catalog` is required. """ alias BeamMCP.Server diff --git a/test/beam_mcp/catalog_test.exs b/test/beam_mcp/catalog_test.exs new file mode 100644 index 0000000..9503bf9 --- /dev/null +++ b/test/beam_mcp/catalog_test.exs @@ -0,0 +1,198 @@ +# SPDX-FileCopyrightText: 2026 Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 + +defmodule BeamMCP.CatalogTest do + use ExUnit.Case, async: true + + alias BeamMCP.{Catalog, Server, ToolSpec} + + defp spec(name) do + %ToolSpec{name: name, command_class: :observe, mode: :read_only, description: "d"} + end + + defmodule OnlyHere do + @behaviour BeamMCP.Catalog + @impl true + def capabilities do + %{ + tools: [ + %BeamMCP.ToolSpec{ + name: :only_in_this_catalog, + command_class: :observe, + mode: :read_only, + description: "Exists in no other catalog." + } + ], + resources: [], + prompts: [] + } + end + end + + describe "the single-lookup guarantee" do + # Slice 002 fixed a real defect: `tools/list` honoured the injected catalog and + # `tools/call` consulted a different source, so a tool could be advertised and refused. + # `Catalog.tools/1` is now the one reader and both paths go through it. This pins that by + # EFFECT -- the two answers are compared against each other, not against a constant -- and + # the catalog's tool exists in no other catalog in this suite, so neither path can pass by + # coincidence the way slice 002's original test did. + test "what tools/list advertises is exactly what tools/call will accept" do + state = Server.new(catalog: OnlyHere, dispatch: fn _n, a, _o -> {:ok, a} end) + + {_state, listed} = + Server.handle_message(state, %{ + "jsonrpc" => "2.0", + "id" => 1, + "method" => "tools/list" + }) + + advertised = listed["result"]["tools"] |> Enum.map(& &1["name"]) |> Enum.sort() + + callable = + for name <- advertised do + {_s, r} = + Server.handle_message(state, %{ + "jsonrpc" => "2.0", + "id" => 2, + "method" => "tools/call", + "params" => %{"name" => name, "arguments" => %{}} + }) + + if Map.has_key?(r, "result"), do: name + end + |> Enum.reject(&is_nil/1) + |> Enum.sort() + + assert advertised == ["only_in_this_catalog"] + assert callable == advertised + end + + test "a name the catalog does not advertise is not callable either" do + state = Server.new(catalog: OnlyHere, dispatch: fn _n, a, _o -> {:ok, a} end) + + {_s, r} = + Server.handle_message(state, %{ + "jsonrpc" => "2.0", + "id" => 3, + "method" => "tools/call", + "params" => %{"name" => "not_advertised", "arguments" => %{}} + }) + + assert r["error"]["code"] == -32_601 + end + + test "Catalog.tools/1 and Catalog.fetch/2 read the same source" do + # The structural half of the same guarantee: fetch resolves exactly the specs tools/1 + # returns, so a mutant giving them different sources has something to break. + names = OnlyHere |> Catalog.tools() |> Enum.map(& &1.name) + + for n <- names do + assert {:ok, %ToolSpec{name: ^n}} = Catalog.fetch(OnlyHere, n) + end + + assert Catalog.fetch(OnlyHere, "absent") == :error + end + end + + describe "a malformed catalog is refused at new/1, not at the first request" do + defmodule MissingPrompts do + def capabilities, do: %{tools: [], resources: []} + end + + defmodule ToolsNotAList do + def capabilities, do: %{tools: %{}, resources: [], prompts: []} + end + + defmodule NotSpecs do + def capabilities, do: %{tools: [%{name: :echo}], resources: [], prompts: []} + end + + defmodule NotAMap do + def capabilities, do: [:tools] + end + + defmodule NoCallback do + def unrelated, do: :ok + end + + test "an absent key is a malformed catalog, not an empty one" do + assert_raise ArgumentError, ~r/missing required key\(s\): \[:prompts\]/, fn -> + Server.new(catalog: MissingPrompts) + end + end + + test ":tools must be a list" do + assert_raise ArgumentError, ~r/:tools must be a list/, fn -> + Server.new(catalog: ToolsNotAList) + end + end + + test ":tools entries must be %ToolSpec{}" do + # This one is the silent failure the @spec probe found: fetch/2 RETURNS a bare map + # happily, so without this check a malformed entry reaches dispatch as a struct-shaped + # thing that is not a struct. + assert_raise ArgumentError, ~r/must all be %BeamMCP.ToolSpec\{\}/, fn -> + Server.new(catalog: NotSpecs) + end + end + + test "capabilities/0 must return a map" do + assert_raise ArgumentError, ~r/must return a map/, fn -> + Server.new(catalog: NotAMap) + end + end + + test "a module that does not export capabilities/0 is refused" do + assert_raise ArgumentError, ~r/does not export capabilities\/0/, fn -> + Server.new(catalog: NoCallback) + end + end + end + + describe "Catalog.validate/1 reports rather than raises" do + test "a good catalog is :ok" do + assert Catalog.validate(OnlyHere) == :ok + end + + test "a non-module is refused with the value it got" do + assert {:error, msg} = Catalog.validate("not a module") + assert msg =~ "expected a module" + end + end + + describe "fetch/2's documented raise conditions, held to what was measured" do + defmodule BinaryName do + def capabilities do + %{ + tools: [ + %BeamMCP.ToolSpec{ + name: "echo", + command_class: :observe, + mode: :read_only, + description: "d" + } + ], + resources: [], + prompts: [] + } + end + end + + test "a binary spec.name raises rather than returning :error" do + # The @spec says {:ok, t} | :error and this raises. That is DOCUMENTED rather than + # caught, because catching would make a host bug indistinguishable from "no such tool" -- + # the advertise-versus-call confusion this behaviour exists to prevent. The moduledoc + # lists the measured conditions; this pins one of them so the list cannot go stale + # silently. + assert_raise ArgumentError, fn -> Catalog.fetch(BinaryName, "echo") end + end + + test "an unloaded module raises" do + assert_raise UndefinedFunctionError, fn -> Catalog.fetch(NoSuchCatalogAnywhere, "echo") end + end + end + + test "the spec/1 helper is unused elsewhere and exists only to keep this file readable" do + assert %ToolSpec{name: :x} = spec(:x) + end +end diff --git a/test/beam_mcp/error_payload_test.exs b/test/beam_mcp/error_payload_test.exs index 9cecece..013fc9d 100644 --- a/test/beam_mcp/error_payload_test.exs +++ b/test/beam_mcp/error_payload_test.exs @@ -14,10 +14,12 @@ defmodule BeamMCP.ErrorPayloadTest do alias BeamMCP.Server defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :widget, @@ -35,7 +37,7 @@ defmodule BeamMCP.ErrorPayloadTest do end defp call(dispatch, args) do - Server.new(tool_catalog: Catalog, dispatch: dispatch) + Server.new(catalog: Catalog, dispatch: dispatch) |> Server.handle_message(%{ "jsonrpc" => "2.0", "id" => 1, diff --git a/test/beam_mcp/injection_test.exs b/test/beam_mcp/injection_test.exs index d54b6fe..9d4f2e0 100644 --- a/test/beam_mcp/injection_test.exs +++ b/test/beam_mcp/injection_test.exs @@ -14,10 +14,12 @@ defmodule BeamMCP.InjectionTest do alias BeamMCP.Server defmodule CatalogWithNovelTool do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :novel_tool, @@ -30,7 +32,7 @@ defmodule BeamMCP.InjectionTest do end test "an injected catalog's tool is advertised by tools/list" do - state = Server.new(tool_catalog: CatalogWithNovelTool) + state = Server.new(catalog: CatalogWithNovelTool) {_s, resp} = Server.handle_message(state, %{"jsonrpc" => "2.0", "id" => 1, "method" => "tools/list"}) @@ -46,7 +48,7 @@ defmodule BeamMCP.InjectionTest do {:ok, %{}} end - state = Server.new(dispatch: dispatch, tool_catalog: CatalogWithNovelTool) + state = Server.new(dispatch: dispatch, catalog: CatalogWithNovelTool) {_s, resp} = Server.handle_message(state, %{ diff --git a/test/beam_mcp/negotiation_test.exs b/test/beam_mcp/negotiation_test.exs index 5aa6992..c035d8f 100644 --- a/test/beam_mcp/negotiation_test.exs +++ b/test/beam_mcp/negotiation_test.exs @@ -23,10 +23,12 @@ defmodule BeamMCP.NegotiationTest do @legacy "2025-11-25" defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -38,7 +40,7 @@ defmodule BeamMCP.NegotiationTest do end end - defp state, do: Server.new(tool_catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end) + defp state, do: Server.new(catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end) defp send_msg(msg), do: state() |> Server.handle_message(msg) |> elem(1) diff --git a/test/beam_mcp/readme_claims_test.exs b/test/beam_mcp/readme_claims_test.exs index 81c3419..ef1a63b 100644 --- a/test/beam_mcp/readme_claims_test.exs +++ b/test/beam_mcp/readme_claims_test.exs @@ -42,10 +42,12 @@ defmodule BeamMCP.ReadmeClaimsTest do @vkey "io.modelcontextprotocol/protocolVersion" defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -62,7 +64,7 @@ defmodule BeamMCP.ReadmeClaimsTest do me = self() Server.new( - tool_catalog: Catalog, + catalog: Catalog, dispatch: fn name, args, _opts -> send(me, {:dispatched, name}) {:ok, args} @@ -342,6 +344,28 @@ defmodule BeamMCP.ReadmeClaimsTest do end end + describe "the BeamMCP.Catalog README claims" do + test "the catalog contract's behavioural sentences are pinned to the sentences that state them" do + # Fragments chosen to sit within one wrapped line, checked with grep against the file + # first -- slice 007 lost three fragments to the README's hard wrap. + claims("the host names what it offers") + claims("An absent key is a malformed catalog, not an empty") + claims("refuses it at startup rather than at the first request") + end + + test "the refusal the README promises is the refusal the code performs" do + # Held to the code, not only to itself. The README says an absent key is refused at + # startup; this is that startup, and it is the sentence's only evidence. + defmodule ReadmeMissingKey do + def capabilities, do: %{tools: [], resources: []} + end + + assert_raise ArgumentError, ~r/missing required key\(s\)/, fn -> + BeamMCP.Server.new(catalog: ReadmeMissingKey) + end + end + end + describe "the :authorize_body README claims" do test "every behavioural sentence about the hook is pinned to the sentence that states it" do # FRAGMENTS ARE CHOSEN TO SIT WITHIN ONE WRAPPED LINE. The README is hard-wrapped, and @@ -449,9 +473,11 @@ defmodule BeamMCP.ReadmeClaimsTest do end defmodule WeatherCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do Process.get(:catalog_fun).() end end @@ -460,7 +486,7 @@ defmodule BeamMCP.ReadmeClaimsTest do state = BeamMCP.Server.new( - tool_catalog: WeatherCatalog, + catalog: WeatherCatalog, dispatch: fn _name, args, _opts -> send(me, {:dispatched, args}) {:ok, args} @@ -536,7 +562,7 @@ defmodule BeamMCP.ReadmeClaimsTest do # The core is dual-era and still serves these on stdio -- that is the README's point, and # the reason the refusal lives in the transport. Asserted so the sentence explaining the # split is pinned to a core that actually still answers them. - state = Server.new(tool_catalog: Catalog, dispatch: fn _, a, _ -> {:ok, a} end) + state = Server.new(catalog: Catalog, dispatch: fn _, a, _ -> {:ok, a} end) {_state, initialize} = Server.handle_message(state, %{ @@ -572,10 +598,12 @@ defmodule BeamMCP.ReadmeClaimsTest do claims("until the schema is corrected") defmodule ForbiddenAnnotationCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -594,7 +622,7 @@ defmodule BeamMCP.ReadmeClaimsTest do end end - conn = http_request(tool_catalog: ForbiddenAnnotationCatalog) + conn = http_request(catalog: ForbiddenAnnotationCatalog) assert conn.status == 500 assert Jason.decode!(conn.resp_body)["error"]["code"] == -32_603 @@ -653,7 +681,7 @@ defmodule BeamMCP.ReadmeClaimsTest do |> Plug.Conn.put_req_header("mcp-name", "echo") |> HTTP.call( HTTP.init( - tool_catalog: Catalog, + catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> :ok end, allowed_origins: :any @@ -681,7 +709,7 @@ defmodule BeamMCP.ReadmeClaimsTest do opts = Keyword.merge( [ - tool_catalog: Catalog, + catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> :ok end, allowed_origins: :any diff --git a/test/beam_mcp/server_test.exs b/test/beam_mcp/server_test.exs index cb8e940..aa0ff9d 100644 --- a/test/beam_mcp/server_test.exs +++ b/test/beam_mcp/server_test.exs @@ -6,8 +6,10 @@ defmodule BeamMCP.ServerTest do alias BeamMCP.Server - defmodule FakeToolCatalog do - def all do + defmodule FakeCatalog do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :get_latest_alerts, @@ -36,7 +38,7 @@ defmodule BeamMCP.ServerTest do end test "initialize advertises MCP tool capability" do - state = Server.new(tool_catalog: FakeToolCatalog) + state = Server.new(catalog: FakeCatalog) {next_state, response} = Server.handle_message(state, %{"jsonrpc" => "2.0", "id" => 1, "method" => "initialize"}) @@ -50,7 +52,7 @@ defmodule BeamMCP.ServerTest do end test "tools/list exposes MCP-compatible tool metadata" do - state = Server.new(tool_catalog: FakeToolCatalog) + state = Server.new(catalog: FakeCatalog) {_next_state, response} = Server.handle_message(state, %{"jsonrpc" => "2.0", "id" => 2, "method" => "tools/list"}) @@ -71,7 +73,7 @@ defmodule BeamMCP.ServerTest do {:ok, %{received: args[:action_class], case_id: args[:case_id], target: args[:target]}} end - state = Server.new(dispatch: dispatch, tool_catalog: FakeToolCatalog) + state = Server.new(dispatch: dispatch, catalog: FakeCatalog) {_next_state, response} = Server.handle_message(state, %{ @@ -105,7 +107,7 @@ defmodule BeamMCP.ServerTest do end test "unknown tools return a JSON-RPC error" do - state = Server.new(tool_catalog: FakeToolCatalog) + state = Server.new(catalog: FakeCatalog) {_next_state, response} = Server.handle_message(state, %{ diff --git a/test/beam_mcp/tool_spec_schema_test.exs b/test/beam_mcp/tool_spec_schema_test.exs index 309ee9a..da140a3 100644 --- a/test/beam_mcp/tool_spec_schema_test.exs +++ b/test/beam_mcp/tool_spec_schema_test.exs @@ -26,10 +26,12 @@ defmodule BeamMCP.ToolSpecSchemaTest do } defmodule CatalogWithSchema do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :inspect_widget, @@ -48,7 +50,7 @@ defmodule BeamMCP.ToolSpecSchemaTest do end defp state(dispatch \\ fn _n, _a, _o -> {:ok, %{}} end) do - Server.new(dispatch: dispatch, tool_catalog: CatalogWithSchema) + Server.new(dispatch: dispatch, catalog: CatalogWithSchema) end defp call(state, args) do diff --git a/test/beam_mcp/transport/http_bandit_test.exs b/test/beam_mcp/transport/http_bandit_test.exs index 89d271c..79a93a4 100644 --- a/test/beam_mcp/transport/http_bandit_test.exs +++ b/test/beam_mcp/transport/http_bandit_test.exs @@ -59,9 +59,11 @@ defmodule BeamMCP.Transport.HTTPBanditTest do @declared_but_unsent 5_000_000 defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -80,7 +82,7 @@ defmodule BeamMCP.Transport.HTTPBanditTest do plug_opts = Keyword.merge( [ - tool_catalog: Catalog, + catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> :ok end, allowed_origins: :any diff --git a/test/beam_mcp/transport/http_test.exs b/test/beam_mcp/transport/http_test.exs index 2acf70c..9a2e4a0 100644 --- a/test/beam_mcp/transport/http_test.exs +++ b/test/beam_mcp/transport/http_test.exs @@ -18,9 +18,11 @@ defmodule BeamMCP.Transport.HTTPTest do @hdr "mcp-protocol-version" defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -57,7 +59,7 @@ defmodule BeamMCP.Transport.HTTPTest do defp opts(extra \\ []) do Keyword.merge( [ - tool_catalog: Catalog, + catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> :ok end, allowed_origins: :any @@ -106,20 +108,20 @@ defmodule BeamMCP.Transport.HTTPTest do describe "the two options the package refuses to default" do test "init/1 raises without :authorize, naming what the host must decide" do assert_raise ArgumentError, ~r/requires an :authorize option, and it has no default/, fn -> - HTTP.init(tool_catalog: Catalog, allowed_origins: :any) + HTTP.init(catalog: Catalog, allowed_origins: :any) end end test "init/1 raises without :allowed_origins" do assert_raise ArgumentError, ~r/requires an :allowed_origins option/, fn -> - HTTP.init(tool_catalog: Catalog, authorize: fn _ -> :ok end) + HTTP.init(catalog: Catalog, authorize: fn _ -> :ok end) end end test "the failure is at init, not at request time" do # The distinction is the whole contract: a host that forgets cannot start, rather than # serving unauthorised requests until someone notices. - assert_raise ArgumentError, fn -> HTTP.init(tool_catalog: Catalog) end + assert_raise ArgumentError, fn -> HTTP.init(catalog: Catalog) end end test "an authorize that refuses stops the request before any message is handled" do @@ -697,7 +699,7 @@ defmodule BeamMCP.Transport.HTTPTest do # The transport matches the core's `Method not found:` prefix. That coupling is real, so # it is asserted: a reword fails this test instead of silently turning every 404 into a # 200 with no test noticing. - state = BeamMCP.Server.new(tool_catalog: Catalog, dispatch: fn _, a, _ -> {:ok, a} end) + state = BeamMCP.Server.new(catalog: Catalog, dispatch: fn _, a, _ -> {:ok, a} end) {_, unimplemented} = BeamMCP.Server.handle_message(state, %{ @@ -747,12 +749,12 @@ defmodule BeamMCP.Transport.HTTPTest do end end - describe ":tool_catalog is checked for the behaviour, not for truthiness" do - test "a value that is not a module exporting all/0 raises at init, not at the first request" do + describe ":catalog is checked for the behaviour, not for truthiness" do + test "a value that is not a module exporting capabilities/0 raises at init, not at the first request" do for bad <- [true, "MyApp.Catalog", Enum, :not_a_module] do - assert_raise ArgumentError, ~r/BeamMCP.ToolCatalog behaviour/, fn -> + assert_raise ArgumentError, ~r/BeamMCP.Catalog behaviour/, fn -> HTTP.init( - tool_catalog: bad, + catalog: bad, dispatch: fn _, a, _ -> {:ok, a} end, authorize: fn _ -> :ok end, allowed_origins: :any @@ -1010,7 +1012,7 @@ defmodule BeamMCP.Transport.HTTPTest do # ROUND 5, FOUND BY TWO LANES INDEPENDENTLY. The rule was applied to dispatch/3 and the # comment beside it claimed "every exception, throw and exit out of the host". The # population was LISTED, not derived. Derived by command, host-supplied code runs at three - # sites in the request path -- authorize/1, ToolCatalog.fetch/2 and Server.handle_message/2 + # sites in the request path -- authorize/1, Catalog.fetch/2 and Server.handle_message/2 # -- and only the third was inside the rescue that was fixed. # # authorize/1 is the branch this module's own docs call possibly unauthenticated. @@ -1023,7 +1025,7 @@ defmodule BeamMCP.Transport.HTTPTest do assert body!(conn)["error"]["code"] == -32_603 end - test "a host tool_catalog's exception does not choose the HTTP status either" do + test "a host catalog's exception does not choose the HTTP status either" do # The catalog raises ONCE, and that is the whole design of this test. # # The first version raised on every call, so `Server.handle_message/2`'s own lookup raised @@ -1037,9 +1039,11 @@ defmodule BeamMCP.Transport.HTTPTest do # succeeds. So if the fault were swallowed, this request would be served rather than # refused -- which is exactly the mutant. defmodule FlakyCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do case Process.put(:flaky_called, true) do nil -> raise Plug.BadRequestError @@ -1065,7 +1069,7 @@ defmodule BeamMCP.Transport.HTTPTest do end Process.delete(:flaky_called) - o = opts(tool_catalog: FlakyCatalog) + o = opts(catalog: FlakyCatalog) # A header that LIES about the body value: 99 against a body of 42. If mirroring is # silently disabled by the swallowed fault, this is dispatched. @@ -1081,14 +1085,16 @@ defmodule BeamMCP.Transport.HTTPTest do assert body!(conn)["id"] == 92 end - test "a host tool_catalog that THROWS is answered with the id, not just rescued" do + test "a host catalog that THROWS is answered with the id, not just rescued" do # host_call/1 has a `catch` as well as a `rescue`, and a mutant dropping the `catch` # survived the suite: the throw propagated to call/2's own catch, which answers with # `id: null`. Status and code were identical, so only the id moves under that mutation. defmodule ThrowingCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do case Process.put(:throwing_called, true) do nil -> throw(:catalog_unavailable) @@ -1120,7 +1126,7 @@ defmodule BeamMCP.Transport.HTTPTest do post( body, call_headers([{"mcp-param-maxrows", "99"}]), - opts(tool_catalog: ThrowingCatalog) + opts(catalog: ThrowingCatalog) ) assert conn.status == 500 @@ -1159,28 +1165,29 @@ defmodule BeamMCP.Transport.HTTPTest do end test "a host catalog returning a malformed spec answers with the request's id" do - # ONE HOST BUG, ONE ENVELOPE. A host `tool_catalog` that RAISES is answered by + # ONE HOST BUG, ONE ENVELOPE. A host `catalog` that RAISES is answered by # `check_param_headers/4`'s fault branch and keeps the request's id; the same host # catalog returning MALFORMED DATA -- a spec-shaped map that is not a `%ToolSpec{}` -- # used to raise `KeyError` on the `spec.input_schema` read one line later, escape to # `call/2`'s rescue and answer `id: null`. Measured before the fix, in # `slices/002-streamable-http/logs/probe-fault-ids.txt`: # - # host tool_catalog RAISES (header validation) 500 -32603 id=4242 - # host tool_catalog returns a malformed spec 500 -32603 id=nil + # host catalog RAISES (header validation) 500 -32603 id=4242 + # host catalog returns a malformed spec 500 -32603 id=nil # # Which envelope a host bug got was decided by which line it landed on. The id is the # ONLY thing that distinguishes the two routes, which is why it is what this asserts: # status and code are identical on both, so asserting those alone is an anchor that # cannot move. defmodule BadSpecCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all, do: [%{name: :echo}] + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + defp all_tools, do: [%{name: :echo}] end body = call_body(%{}) |> Map.put("id", 4242) - conn = post(body, call_headers([]), opts(tool_catalog: BadSpecCatalog)) + conn = post(body, call_headers([]), opts(catalog: BadSpecCatalog)) assert conn.status == 500 assert conn.resp_body != "" @@ -1195,9 +1202,11 @@ defmodule BeamMCP.Transport.HTTPTest do end defmodule InvalidUtf8HeaderNameCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -1243,7 +1252,7 @@ defmodule BeamMCP.Transport.HTTPTest do # A property with an unencodable annotation name is still a recorded gap -- the tool is # advertised and uncallable -- and it is filed in this slice's FINDINGS.md rather than # described here as filed. - o = opts(tool_catalog: InvalidUtf8HeaderNameCatalog) + o = opts(catalog: InvalidUtf8HeaderNameCatalog) body = call_body(%{"value" => "x"}) |> Map.put("id", 4243) conn = post(body, call_headers([]), o) @@ -1623,9 +1632,11 @@ defmodule BeamMCP.Transport.HTTPTest do # and the 400 blames the caller for the host's schema. defmodule FloatAnnotationCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -1642,9 +1653,11 @@ defmodule BeamMCP.Transport.HTTPTest do end defmodule ObjectAnnotationCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -1671,7 +1684,7 @@ defmodule BeamMCP.Transport.HTTPTest do o = opts( - tool_catalog: FloatAnnotationCatalog, + catalog: FloatAnnotationCatalog, dispatch: fn n, a, _ -> send(me, {:dispatched, n, a}) && {:ok, a} end ) @@ -1706,7 +1719,7 @@ defmodule BeamMCP.Transport.HTTPTest do test "the refusal does not depend on the caller sending the header" do # Both directions were closed before, and both must now land on the same answer: the # verdict is a property of the SCHEMA, so it cannot depend on what the caller sent. - o = opts(tool_catalog: FloatAnnotationCatalog) + o = opts(catalog: FloatAnnotationCatalog) ExUnit.CaptureLog.capture_log(fn -> assert post(call_body(%{"ratio" => 1.5}), call_headers([]), o).status == 500 @@ -1714,7 +1727,7 @@ defmodule BeamMCP.Transport.HTTPTest do end test "an annotated `object` property is the same fault" do - o = opts(tool_catalog: ObjectAnnotationCatalog) + o = opts(catalog: ObjectAnnotationCatalog) ExUnit.CaptureLog.capture_log(fn -> conn = post(call_body(%{"obj" => %{"k" => "v"}}), call_headers([]), o) @@ -1730,9 +1743,11 @@ defmodule BeamMCP.Transport.HTTPTest do # would turn a caller sending the wrong shape into a host fault -- the wrong side of the # boundary, and the mistake this fix exists to stop making in the other direction. defmodule UntypedAnnotationCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -1748,7 +1763,7 @@ defmodule BeamMCP.Transport.HTTPTest do end end - o = opts(tool_catalog: UntypedAnnotationCatalog) + o = opts(catalog: UntypedAnnotationCatalog) body = call_body(%{"loose" => "v"}) assert post(body, call_headers([{"mcp-param-loose", "v"}]), o).status == 200 @@ -1773,9 +1788,11 @@ defmodule BeamMCP.Transport.HTTPTest do # schema. defmodule CollidingCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :collide, @@ -1795,9 +1812,11 @@ defmodule BeamMCP.Transport.HTTPTest do end defmodule NestedCollisionCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :collide, @@ -1834,7 +1853,7 @@ defmodule BeamMCP.Transport.HTTPTest do o = opts( - tool_catalog: CollidingCatalog, + catalog: CollidingCatalog, dispatch: fn n, a, _ -> send(me, {:dispatched, n, a}) && {:ok, a} end ) @@ -1869,7 +1888,7 @@ defmodule BeamMCP.Transport.HTTPTest do test "the collision is refused however the caller writes the header, or omits it" do # The verdict is a property of the schema. A caller who supplies both spellings, or # neither, gets the same answer -- otherwise the check is being decided by the request. - o = opts(tool_catalog: CollidingCatalog) + o = opts(catalog: CollidingCatalog) body = collide_body(%{"alpha" => "A", "beta" => "B"}) ExUnit.CaptureLog.capture_log(fn -> @@ -1884,7 +1903,7 @@ defmodule BeamMCP.Transport.HTTPTest do test "a nested annotation colliding with an outer one is the same fault" do # This is the `Map.merge` half rather than the `Map.put` half: the nested walk's result # was merged over the accumulator, so a nested annotation silently won. - o = opts(tool_catalog: NestedCollisionCatalog) + o = opts(catalog: NestedCollisionCatalog) ExUnit.CaptureLog.capture_log(fn -> conn = diff --git a/test/beam_mcp/transport/stdio_test.exs b/test/beam_mcp/transport/stdio_test.exs index b6ae272..6f97520 100644 --- a/test/beam_mcp/transport/stdio_test.exs +++ b/test/beam_mcp/transport/stdio_test.exs @@ -19,10 +19,12 @@ defmodule BeamMCP.Transport.StdioTest do alias BeamMCP.Transport.Stdio defmodule Catalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do + def capabilities, do: %{tools: all_tools(), resources: [], prompts: []} + + defp all_tools do [ %BeamMCP.ToolSpec{ name: :echo, @@ -34,7 +36,7 @@ defmodule BeamMCP.Transport.StdioTest do end end - defp opts, do: [tool_catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end] + defp opts, do: [catalog: Catalog, dispatch: fn _n, a, _o -> {:ok, a} end] # Drive the real loop over a StringIO standing in for stdio. The group leader is what # `IO.binread(:stdio, _)` and `IO.binwrite(:stdio, _)` resolve to, so this exercises the diff --git a/tools/mutants/Mcat1.py b/tools/mutants/Mcat1.py new file mode 100644 index 0000000..d5a742b --- /dev/null +++ b/tools/mutants/Mcat1.py @@ -0,0 +1,33 @@ +# SPDX-FileCopyrightText: 2026 Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +# +# Run by tools/mutate.sh, which passes the file to mutate as argv[1]. Never hard-code +# a path here: a mutant that names one machine's worktree is the unresolvable +# `$S/mut.sh` that slice 003's record cites, in another file. +# +# TARGET=lib/beam_mcp/catalog.ex tools/mutate.sh score Mcat1 +# +# Mcat1 -- DROP A REQUIRED KEY FROM THE SHAPE. `@required_keys` loses `:prompts`, so a host +# whose capabilities/0 omits it is accepted as well formed. +# +# This is the mutation the whole slice is about. The reason `resources` and `prompts` are +# required-and-may-be-empty is that an absent key and an empty one are different claims: absent +# means the host never wrote the key, empty means it wrote it and has none. Relaxing the check +# by one key makes the two indistinguishable again, and the only place that difference is +# visible is startup -- by the first request the map has already been read. +# +# It is deliberately the smallest possible relaxation: one key out of three, no message change, +# no behaviour change for a correct host. A checker that only rejects the malformed catalogs +# nobody writes is not a checker. +import sys + +p = sys.argv[1] +s = open(p).read() + +old = " @required_keys [:tools, :resources, :prompts]" +new = " @required_keys [:tools, :resources]" + +if s.count(old) != 1: + sys.exit("Mcat1: anchor found %d times" % s.count(old)) + +open(p, "w").write(s.replace(old, new, 1)) diff --git a/tools/mutants/Mcat2.py b/tools/mutants/Mcat2.py new file mode 100644 index 0000000..1002963 --- /dev/null +++ b/tools/mutants/Mcat2.py @@ -0,0 +1,53 @@ +# SPDX-FileCopyrightText: 2026 Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +# +# Run by tools/mutate.sh, which passes the file to mutate as argv[1]. Never hard-code +# a path here. +# +# TARGET=lib/beam_mcp/server.ex tools/mutate.sh score Mcat2 +# +# Mcat2 -- GIVE ADVERTISE AND CALL DIFFERENT SOURCES. `tools/list` keeps reading the injected +# catalog through `Catalog.tools/1`; `tools/call` stops reading it and answers "which tool does +# this name mean" from the request itself, synthesising a spec for whatever was asked for. +# +# This is slice 002's defect restored in the direction it actually occurred: one path honoured +# the injected catalog and the other consulted something else, so the set a server advertises +# and the set it accepts are no longer the same set. The single-lookup guarantee in +# `BeamMCP.Catalog.tools/1` exists to make that unrepresentable, and a guarantee is only worth +# the mutation that has to break to violate it. +# +# It is also the shape CONVENTIONS.md names directly: deriving a population from +# attacker-controlled input is not deriving a population. Here the population of callable tools +# becomes the name in the request. +# +# WHY THE EMPTY-NAME BRANCH IS HERE. The first version of this mutant returned {:ok, spec} +# unconditionally. Elixir's type checker then narrowed find_tool/2's return to that one shape, +# declared the caller's `:error ->` clause unreachable, and --warnings-as-errors failed the +# build: the score line came back with no test count at all. That is a compiler kill, which +# CONVENTIONS.md says records a kill that never happened -- no test ever ran. The branch below +# keeps the return a union so the mutation is scored by the suite instead. It costs the mutant +# nothing: every name a client can actually send is still callable. +import sys + +p = sys.argv[1] +s = open(p).read() + +old = " defp find_tool(state, name), do: Catalog.fetch(state.catalog, name)" +new = """ defp find_tool(_state, name) do + if name == "" do + :error + else + {:ok, + %BeamMCP.ToolSpec{ + name: String.to_atom(name), + command_class: :observe, + mode: :read_only, + description: "synthesised from the request rather than read from the catalog" + }} + end + end""" + +if s.count(old) != 1: + sys.exit("Mcat2: anchor found %d times" % s.count(old)) + +open(p, "w").write(s.replace(old, new, 1)) diff --git a/tools/probe_ping.exs b/tools/probe_ping.exs index 89b0497..8b517f3 100644 --- a/tools/probe_ping.exs +++ b/tools/probe_ping.exs @@ -7,10 +7,10 @@ # mix run tools/probe_ping.exs > slices/001b-ping-guard/logs/probe-after.txt 2>&1 defmodule ProbeCatalog do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do - [ + def capabilities do + tools = [ %BeamMCP.ToolSpec{ name: :echo, command_class: :observe, @@ -18,10 +18,12 @@ defmodule ProbeCatalog do description: "Echo." } ] + + %{tools: tools, resources: [], prompts: []} end end -state = BeamMCP.Server.new(tool_catalog: ProbeCatalog, dispatch: fn _n, a, _o -> {:ok, a} end) +state = BeamMCP.Server.new(catalog: ProbeCatalog, dispatch: fn _n, a, _o -> {:ok, a} end) key = "io.modelcontextprotocol/protocolVersion" send_one = fn label, msg -> diff --git a/tools/probes/drain_mechanism.exs b/tools/probes/drain_mechanism.exs index c5369a2..e6044e0 100644 --- a/tools/probes/drain_mechanism.exs +++ b/tools/probes/drain_mechanism.exs @@ -17,10 +17,10 @@ alias BeamMCP.Transport.HTTP defmodule Cat do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do - [ + def capabilities do + tools = [ %BeamMCP.ToolSpec{ name: :echo, command_class: :observe, @@ -29,6 +29,8 @@ defmodule Cat do input_schema: %{"type" => "object", "properties" => %{}, "additionalProperties" => true} } ] + + %{tools: tools, resources: [], prompts: []} end end @@ -38,7 +40,7 @@ modern = "2026-07-28" vkey = "io.modelcontextprotocol/protocolVersion" opts = [ - tool_catalog: Cat, + catalog: Cat, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> {:error, :nope} end, allowed_origins: :any diff --git a/tools/probes/loss_site.exs b/tools/probes/loss_site.exs index 67117af..9724f31 100644 --- a/tools/probes/loss_site.exs +++ b/tools/probes/loss_site.exs @@ -12,10 +12,10 @@ alias BeamMCP.Transport.HTTP defmodule Cat do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do - [ + def capabilities do + tools = [ %BeamMCP.ToolSpec{ name: :echo, command_class: :observe, @@ -24,6 +24,8 @@ defmodule Cat do input_schema: %{"type" => "object", "properties" => %{}, "additionalProperties" => true} } ] + + %{tools: tools, resources: [], prompts: []} end end @@ -34,7 +36,7 @@ vkey = "io.modelcontextprotocol/protocolVersion" owner = self() opts = [ - tool_catalog: Cat, + catalog: Cat, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> send(owner, :authorize_ran) diff --git a/tools/probes/write_shape.exs b/tools/probes/write_shape.exs index 516aab3..cba01f2 100644 --- a/tools/probes/write_shape.exs +++ b/tools/probes/write_shape.exs @@ -12,10 +12,10 @@ alias BeamMCP.Transport.HTTP defmodule Cat do - @behaviour BeamMCP.ToolCatalog + @behaviour BeamMCP.Catalog @impl true - def all do - [ + def capabilities do + tools = [ %BeamMCP.ToolSpec{ name: :echo, command_class: :observe, @@ -24,6 +24,8 @@ defmodule Cat do input_schema: %{"type" => "object", "properties" => %{}, "additionalProperties" => true} } ] + + %{tools: tools, resources: [], prompts: []} end end @@ -34,7 +36,7 @@ vkey = "io.modelcontextprotocol/protocolVersion" owner = self() opts = [ - tool_catalog: Cat, + catalog: Cat, dispatch: fn _n, a, _o -> {:ok, a} end, authorize: fn _conn -> send(owner, :authorize_ran)