Slice 011: LLM provider layer

ROADMAP.md

@@ -34,7 +34,7 @@ standards register names the rows that ask for them.
 | 002 | Supply chain, early: SBOM, build provenance, the TLS floor | 0 Foundation | S | 000 | planned |
 | 003 | FIPS build leg in CI, from source | 0 Foundation | M | 000 | planned |
 | 010 | Core domain + persistence (Ecto/SQLite, schemas, Repo owner) | 1 Core loop | M | 000 | approved |
-| 011 | LLM provider layer (req_llm behind `Trinity.LLM` behaviour) | 1 Core loop | M | 010 | planned |
+| 011 | LLM provider layer (req_llm behind `Trinity.LLM` behaviour) | 1 Core loop | M | 010 | done |
 | 012 | Session process + agent loop (gen_statem, DynamicSupervisor, rehydration) | 1 Core loop | L | 010, 011 | planned |
 | 013 | LiveView chat UI with streaming | 1 Core loop | M | 012 | planned |
 | 020 | Tool protocol + registry | 2 Tools | M | 012 | planned |

VERSIONS.md

@@ -89,7 +89,7 @@ never pin a version hex marks as retired or vulnerable.
 | `pgvector` | optional, ~> 0.3 | 🔍 not yet a dependency | Vectors on the Postgres path. Not yet a dependency; Slice 032 decides. Split from the postgrex row at Slice 010. |
 | `oban` | ~> 2.24 | 🔍 not yet a dependency | Uses `Oban.Engines.Lite` on SQLite. ⚠️ Oban Pro Workflows/Smart engine are Postgres-only. Added at Slice 050. |
 | `req` | ~> 0.5 | ✅ in `mix.lock` | HTTP client. |
-| `req_llm` | ~> 1.22 | 🔍 not yet a dependency | Provider layer (streaming, tools, structured output, usage). ⚠️ The pin was `~> 1.10` against a recorded latest of 1.10.0; the real latest was twelve minors ahead. Check event shapes against the current version at Slice 011, not against this file's prose. Added at Slice 011. |
+| `req_llm` | ~> 1.22 | ✅ in `mix.lock` | Provider layer (streaming, tools, structured output, usage). ⚠️ The pin was `~> 1.10` against a recorded latest of 1.10.0; the real latest was twelve minors ahead. Check event shapes against the current version at Slice 011, not against this file's prose. Added at Slice 011. |
 | `beam_mcp` | ~> 0.8 | 🔍 not yet a dependency | MCP server core, Apache-2.0, ADR-0007 decision 5 (owner decision 2026-09-08, recorded 2026-09-20). 0.8.0 on hex.pm, standing before 1.0.0. Server side only: the client, MRTR and OAuth are Trinity's, above it. Added at Slice 059. The earlier candidate list (anubis_mcp, fastest_mcp, gen_mcp) is history. |
 | `jido` | ~> 2.3 (pending ADR-0009) | 🔍 not yet a dependency | Actions, directives and the effect boundary, if the Slice 012 checkpoint adopts it. |
 | `jason` | ~> 1.2 | ✅ in `mix.lock` |  |

config/config.exs

@@ -36,6 +36,10 @@ config :trinity,
           other -> raise "TRINITY_DB must be sqlite or postgres, got #{inspect(other)}"
         end)
 
+# Slice 011: the model registry lives in its own file so the live test suite can read it
+# without evaluating the environment-specific imports below.
+import_config "llm.exs"
+
 # Slice 010, every environment, SQLite only (the Postgres adapter ignores keys it does not
 # know, and the CI matrix proves that). One writer: the pool has exactly one connection, so the
 # single-writer rule SQLite imposes is the pool's shape rather than a hope. Each pragma is named

config/llm.exs

@@ -0,0 +1,44 @@
+# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
+# SPDX-License-Identifier: Apache-2.0
+import Config
+
+# Slice 011: the model registry. Ids are Trinity's; `model` is req_llm's "provider:model";
+# keys are read through Trinity.Config.secret/1 from the named variable at call time. Prices
+# are US dollars per million tokens and are the source of every recorded cost. Free-tier
+# models carry 0.0. The default model is the OpenRouter one the owner picked; the NVIDIA
+# endpoint is reached as an OpenAI-compatible base_url. Test config replaces all of this.
+config :trinity, :llm,
+  default_model: "openrouter:ling",
+  providers: %{req_llm: Trinity.LLM.Providers.ReqLLM},
+  retry: [attempts: 3, base_ms: 200],
+  models: [
+    %{
+      id: "openrouter:ling",
+      provider: :req_llm,
+      model:
+        "openrouter:" <>
+          System.get_env("TRINITY_LIVE_MODEL", "inclusionai/ling-3.0-flash-vl:free"),
+      api_key_env: "OPENROUTER_API_KEY",
+      caps: [:stream, :tools, :json],
+      price: %{input: 0.0, output: 0.0}
+    },
+    %{
+      id: "nvidia:nemotron",
+      provider: :req_llm,
+      model:
+        "openai:" <> System.get_env("NEMOTRON_MODEL", "nvidia/nemotron-3.5-lightning-30b-a3b"),
+      base_url: System.get_env("NEMOTRON_BASE_URL", "https://integrate.api.nvidia.com/v1"),
+      api_key_env: "NEMOTRON_API_KEY",
+      caps: [:stream, :tools, :json],
+      price: %{input: 0.0, output: 0.0}
+    },
+    %{
+      id: "nvidia:embed",
+      provider: :req_llm,
+      model: "openai:nvidia/nemotron-3-embed-1b",
+      base_url: System.get_env("NEMOTRON_BASE_URL", "https://integrate.api.nvidia.com/v1"),
+      api_key_env: "NEMOTRON_API_KEY",
+      caps: [:embed],
+      price: %{input: 0.0, output: 0.0}
+    }
+  ]

config/test.exs

@@ -7,6 +7,36 @@ import Config
 # The MIX_TEST_PARTITION environment variable can be used
 # to provide built-in test partitioning in CI environment.
 # Run `mix help test` for more information.
+# Slice 011: the registry in tests is the scripted fake plus a Mox mock; the live tests set
+# their own entries from the environment at runtime.
+config :trinity, :llm,
+  default_model: "fake:chat",
+  providers: %{fake: Trinity.LLM.Providers.Fake, mock: Trinity.LLM.ProviderMock},
+  retry: [attempts: 3, base_ms: 1],
+  models: [
+    %{
+      id: "fake:chat",
+      provider: :fake,
+      model: "chat",
+      caps: [:stream, :tools, :json],
+      price: %{input: 1.0, output: 2.0}
+    },
+    %{
+      id: "fake:embed",
+      provider: :fake,
+      model: "embed",
+      caps: [:embed, {:embed_dim, 8}],
+      price: %{input: 0.5, output: 0.0}
+    },
+    %{
+      id: "mock:chat",
+      provider: :mock,
+      model: "chat",
+      caps: [:stream, :tools],
+      price: %{input: 0.0, output: 0.0}
+    }
+  ]
+
 # Slice 010: the data-dir lock takes a temporary directory in tests, so a test run never
 # contends with a running Trinity on the same machine, and two test runs at once do contend,
 # which is the property under test.

coverage.tsv

@@ -2,3 +2,4 @@ slice_id	percent	sha	date
 000	27.01	e935c7b	2026-09-06
 001	30.37	5a9c8f7	2026-09-06
 010	44.88	45ba4f0	2026-09-20
+011	51.57	ec5334a	2026-09-20

docs/05-data-model.md

@@ -101,8 +101,20 @@ Pending/decided approval requests: `session_id`, `tool`, `args`, `risk`, `status
 Execution history is in `oban_jobs` + a `task_runs` table (status, session_id, summary).
 
 ### usage_events (Slice 011; the ledger and budgets that read it are Slice 090)
-Per LLM call: `session_id`, `provider`, `model`, `prompt_tokens`, `completion_tokens`, `cached_tokens`,
-`cost_usd`, `latency_ms`. Cost ledger and budgets derive from this.
+One row per completed call, as built at slice 011:
+
+| column | type | notes |
+|---|---|---|
+| model_id | string | the registry id (`"openrouter:ling"`), not the provider's model name |
+| provider | string | the registry entry's provider atom as text |
+| kind | string | "chat" \| "object" \| "embed" |
+| input_tokens, output_tokens, cached_tokens, reasoning_tokens | integer | the names `Trinity.LLM.Event`'s usage map uses; `prompt_tokens` and `completion_tokens` in the first draft of this table are these two |
+| cost_usd | float | computed from the registry's price in dollars per million tokens; the only cost Trinity reports |
+| session_id | fk sessions, nullable | set when the call belongs to a session |
+| provider_meta | map | `provider_cost`: the provider's own figure when it reports one, kept for comparison and never used |
+
+Append-only; `inserted_at` only. Latency is not a column: it is a Telemetry measurement at slice 090, where the
+call is timed at the one place every call passes.
 
 ### gateway_identities (Slice 070)
 `adapter`, `external_user_id`, `display`, `paired_at`, `allowed`: DM pairing and allowlists.

lib/trinity.ex

@@ -10,7 +10,7 @@ defmodule Trinity do
   use Boundary,
     deps: [],
     exports:
-      [Paths, Repo, UUID, Sessions] ++
+      [Paths, Repo, UUID, Config, Sessions, LLM] ++
         if(Mix.env() == :test, do: [DataCase, NetworkGuard, Factory], else: [])
 
   @moduledoc """

lib/trinity/application.ex

@@ -33,6 +33,8 @@ defmodule Trinity.Application do
            repos: Application.fetch_env!(:trinity, :ecto_repos), skip: skip_migrations?()},
           {DNSCluster, query: Application.get_env(:trinity, :dns_cluster_query) || :ignore},
           {Phoenix.PubSub, name: Trinity.PubSub},
+          # Slice 011: streams to a pid run under this supervisor, never as bare tasks.
+          {Task.Supervisor, name: Trinity.LLM.TaskSupervisor},
           # Start to serve requests, typically the last entry
           TrinityWeb.Endpoint
         ] ++ Trinity.Smoke.children(Trinity.Smoke.argv())

lib/trinity/config.ex

@@ -0,0 +1,17 @@
+# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
+# SPDX-License-Identifier: Apache-2.0
+defmodule Trinity.Config do
+  @moduledoc """
+  Secrets, read from one place. Slice 011: the environment. Slice 100 adds the OS keychain
+  behind the same function, and nothing else in the tree reads a key any other way.
+  """
+
+  @doc "The secret named by `env_var`, or a named error; never nil handed to a provider."
+  @spec secret(String.t()) :: {:ok, String.t()} | {:error, {:missing_secret, String.t()}}
+  def secret(env_var) when is_binary(env_var) do
+    case System.get_env(env_var) do
+      value when is_binary(value) and value != "" -> {:ok, value}
+      _ -> {:error, {:missing_secret, env_var}}
+    end
+  end
+end

lib/trinity/llm.ex

@@ -0,0 +1,134 @@
+# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
+# SPDX-License-Identifier: Apache-2.0
+defmodule Trinity.LLM do
+  @moduledoc """
+  The one door to a language model. Slice 011.
+
+  Sessions call this and never a provider. A request names a registry model id (or nothing, for
+  the default); the registry names the provider module; the call is retried on transient errors;
+  a completed call writes a `usage_events` row with cost from the registry price. Events reach
+  the caller either through a function (`stream/3`) or as messages to a pid (`stream_to/3`),
+  which is what slice 012's Session wants.
+  """
+  use Boundary, deps: [Trinity], exports: [Error, Event, Request, Provider, Registry]
+
+  alias Trinity.LLM.{Error, Registry, Request, Retry, Usage}
+
+  @type opts :: keyword()
+
+  @doc "Streams events through `emit`, in the calling process. Returns the usage."
+  @spec stream(Request.t(), opts(), Trinity.LLM.Provider.emit()) ::
+          {:ok, map()} | {:error, Error.t() | term()}
+  def stream(%Request{} = request, opts \\ [], emit) when is_function(emit, 1) do
+    with {:ok, entry, module} <- resolve(request) do
+      call(entry, "chat", opts, fn ->
+        module.stream(request, provider_opts(entry, opts), emit)
+      end)
+    end
+  end
+
+  @doc """
+  Streams events as messages `{:llm_event, ref, event}` to `pid`, then `{:llm_done, ref, result}`.
+  Runs in a supervised task so the caller is never blocked; `ref` is returned at once.
+  """
+  @spec stream_to(Request.t(), opts(), pid()) :: {:ok, reference()} | {:error, term()}
+  def stream_to(%Request{} = request, opts \\ [], pid) when is_pid(pid) do
+    ref = make_ref()
+
+    with {:ok, _entry, _module} <- resolve(request),
+         {:ok, _task} <-
+           Task.Supervisor.start_child(Trinity.LLM.TaskSupervisor, fn ->
+             result = stream(request, opts, &send(pid, {:llm_event, ref, &1}))
+             send(pid, {:llm_done, ref, result})
+           end) do
+      {:ok, ref}
+    end
+  end
+
+  @doc "One complete response."
+  @spec generate(Request.t(), opts()) :: {:ok, Trinity.LLM.Provider.result()} | {:error, term()}
+  def generate(%Request{} = request, opts \\ []) do
+    with {:ok, entry, module} <- resolve(request) do
+      call(entry, "chat", opts, fn -> module.generate(request, provider_opts(entry, opts)) end)
+    end
+  end
+
+  @doc "A map validated against a JSON Schema."
+  @spec generate_object(Request.t(), map(), opts()) :: {:ok, map()} | {:error, term()}
+  def generate_object(%Request{} = request, schema, opts \\ []) when is_map(schema) do
+    with {:ok, entry, module} <- resolve(request),
+         {:ok, object, _usage} <-
+           call(entry, "object", opts, fn ->
+             module.generate_object(request, schema, provider_opts(entry, opts))
+           end) do
+      {:ok, object}
+    end
+  end
+
+  @doc "One vector per text. `opts[:model]` names the embedding model's registry id."
+  @spec embed([String.t()], opts()) :: {:ok, [[float()]]} | {:error, term()}
+  def embed(texts, opts \\ []) when is_list(texts) do
+    with {:ok, entry} <- Registry.lookup(Keyword.get(opts, :model)),
+         {:ok, module} <- Registry.provider_module(entry),
+         {:ok, vectors, _usage} <-
+           call(entry, "embed", opts, fn -> module.embed(texts, provider_opts(entry, opts)) end) do
+      {:ok, vectors}
+    end
+  end
+
+  @doc "The registry's models."
+  @spec models() :: [Registry.entry()]
+  def models, do: Registry.models()
+
+  @doc "The registry's default model id."
+  @spec default_model() :: String.t() | nil
+  def default_model, do: Registry.default_model()
+
+  @doc "A model's capabilities, from its registry entry."
+  @spec capabilities(String.t()) :: {:ok, [atom() | {atom(), term()}]} | {:error, term()}
+  def capabilities(model_id) do
+    with {:ok, entry} <- Registry.lookup(model_id), do: {:ok, entry.caps}
+  end
+
+  defp resolve(%Request{model: model}) do
+    with {:ok, entry} <- Registry.lookup(model),
+         {:ok, module} <- Registry.provider_module(entry) do
+      {:ok, entry, module}
+    end
+  end
+
+  # The entry's keys win: a caller's `:model` is a registry id, the entry's is the provider's
+  # own name, and the provider must see the latter. Found by the live suite, where an embed
+  # call asked req_llm for a provider named after the registry id.
+  defp provider_opts(entry, opts) do
+    entry_opts =
+      entry |> Map.take([:model, :base_url, :api_key_env, :price, :caps]) |> Map.to_list()
+
+    Keyword.merge(opts, entry_opts)
+  end
+
+  # Retry around the provider; on success the usage is recorded. The usage row is written for
+  # the call that completed, once, whatever the number of attempts it took.
+  defp call(entry, kind, opts, fun) do
+    case Retry.run(fun, opts) do
+      {:ok, %{usage: usage}} = ok ->
+        record(entry, kind, usage, opts)
+        ok
+
+      {:ok, usage} = ok when is_map(usage) ->
+        record(entry, kind, usage, opts)
+        ok
+
+      {:ok, _value, usage} = ok ->
+        record(entry, kind, usage, opts)
+        ok
+
+      other ->
+        other
+    end
+  end
+
+  defp record(entry, kind, usage, opts) do
+    {:ok, _} = Usage.record(entry, kind, usage, Keyword.take(opts, [:session_id]))
+  end
+end

lib/trinity/llm/error.ex

@@ -0,0 +1,36 @@
+# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
+# SPDX-License-Identifier: Apache-2.0
+defmodule Trinity.LLM.Error do
+  @moduledoc """
+  A provider failure with one bit that matters to the caller: whether trying again could help.
+  Timeouts, rate limits, server errors and refused connections are transient; everything else
+  (a bad key, an unknown model, a malformed request) is not and is returned at once.
+  """
+
+  @type t :: %__MODULE__{transient?: boolean(), reason: term(), status: non_neg_integer() | nil}
+  defexception [:reason, :status, transient?: false]
+
+  @impl true
+  def message(%__MODULE__{transient?: t, status: status, reason: reason}) do
+    kind = if t, do: "transient", else: "permanent"
+    "#{kind} LLM error#{if status, do: " (HTTP #{status})", else: ""}: #{inspect(reason)}"
+  end
+
+  @doc "Classifies an HTTP status: 408, 425, 429 and 5xx are transient."
+  @spec from_status(non_neg_integer(), term()) :: t()
+  def from_status(status, reason) do
+    %__MODULE__{
+      status: status,
+      reason: reason,
+      transient?: status in [408, 425, 429] or status >= 500
+    }
+  end
+
+  @doc "A transient error with no status: a timeout, a closed or refused connection."
+  @spec transient(term()) :: t()
+  def transient(reason), do: %__MODULE__{reason: reason, transient?: true}
+
+  @doc "A permanent error with no status."
+  @spec permanent(term()) :: t()
+  def permanent(reason), do: %__MODULE__{reason: reason, transient?: false}
+end

lib/trinity/llm/event.ex

@@ -0,0 +1,32 @@
+# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
+# SPDX-License-Identifier: Apache-2.0
+defmodule Trinity.LLM.Event do
+  @moduledoc """
+  The event stream shape every provider emits and every consumer reads. Slice 011.
+
+  Seven shapes, from SLICE.md, and nothing else. Slice 012's Session and slice 013's UI consume
+  these and never a provider's own chunks; `valid?/1` is the guard a provider's output is
+  held to in tests.
+  """
+
+  @type tool_call_id :: String.t()
+  @type t ::
+          {:text_delta, String.t()}
+          | {:tool_call_start, tool_call_id(), name :: String.t()}
+          | {:tool_call_delta, tool_call_id(), json_chunk :: String.t()}
+          | {:tool_call_end, tool_call_id(), args :: map()}
+          | {:usage, map()}
+          | {:done, reason :: :stop | :length | :tool_calls | :content_filter | atom()}
+          | {:error, term()}
+
+  @doc "True for exactly the seven shapes above."
+  @spec valid?(term()) :: boolean()
+  def valid?({:text_delta, s}) when is_binary(s), do: true
+  def valid?({:tool_call_start, id, name}) when is_binary(id) and is_binary(name), do: true
+  def valid?({:tool_call_delta, id, chunk}) when is_binary(id) and is_binary(chunk), do: true
+  def valid?({:tool_call_end, id, args}) when is_binary(id) and is_map(args), do: true
+  def valid?({:usage, usage}) when is_map(usage), do: true
+  def valid?({:done, reason}) when is_atom(reason), do: true
+  def valid?({:error, _}), do: true
+  def valid?(_), do: false
+end