refactor(server): the model access and the tool access are composed when first used

capabilities/reasoning/README.md

@@ -74,4 +74,4 @@ Two tests wait on real time on purpose, where a slow machine only lengthens the
 
 ## Source
 
-`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries: it imports `google-auth-library`, with the AI SDK's Vertex providers, only when model access is built with Vertex configured (`src/adapter/vertex-providers.ts`), the AWS credential chain only with Bedrock configured, through `awsCredentialChainOf` of `@on.auto/outbound`, and `@azure/identity` only for Azure without a key, and no entry of the capability reaches any of them (`src/adapter/credential-libraries.test.ts`). `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model.
+`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries: it imports `google-auth-library`, with the AI SDK's Vertex providers, only when model access is built with Vertex configured (`src/adapter/vertex-providers.ts`), the AWS credential chain only with Bedrock configured, through `awsCredentialChainOf` of `@on.auto/outbound`, and `@azure/identity` only for Azure without a key, and no entry of the capability reaches any of them (`src/adapter/credential-libraries.test.ts`). `installedPackages` (`src/adapter/entra-id.ts`) says whether that optional package is installed by resolving it, without loading it, so a server can read from its settings alone, with `providerStatus` and `offeredModels`, which providers are configured and which models it offers, before it builds the model access, as the server does at start. `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model.

capabilities/reasoning/src/adapter/entra-id.test.ts

@@ -5,6 +5,7 @@ import {
   entraIdentityLoader,
   entraTokens,
   importUnlessMissing,
+  installedPackages,
   loadEntraIdentity,
   type AccessToken,
 } from './entra-id.ts';
@@ -15,6 +16,10 @@ function missingModule(): Promise<never> {
   );
 }
 
+function unresolved(specifier: string): never {
+  throw Object.assign(new Error(`Cannot find package '${specifier}'`), { code: 'ERR_MODULE_NOT_FOUND' });
+}
+
 function tokenFor(token: string): Promise<AccessToken> {
   return Promise.resolve({ token, expiresOnTimestamp: Date.now() + 3_600_000 });
 }
@@ -71,3 +76,23 @@ describe('entraTokens', () => {
     await expect(tokens()).resolves.toBe(`token for ${cognitiveServicesScope}`);
   });
 });
+
+describe('installedPackages', () => {
+  it('finds the identity library installed where it resolves', () => {
+    expect(installedPackages()).toEqual({ entraId: true });
+  });
+
+  it('finds it missing when the package does not resolve', () => {
+    expect(installedPackages(unresolved)).toEqual({ entraId: false });
+  });
+
+  it('passes any other failure on', () => {
+    const broken = new TypeError('Invalid module specifier');
+
+    expect(() =>
+      installedPackages(() => {
+        throw broken;
+      }),
+    ).toThrow(broken);
+  });
+});

capabilities/reasoning/src/adapter/entra-id.ts

@@ -1,6 +1,7 @@
 import { Effect, Predicate } from 'effect';
 
 import type { AzureSettings } from '../settings/provider-settings.ts';
+import type { OptionalPackages } from '../settings/provider-status.ts';
 import type { Availability } from '../settings/setting-values.ts';
 import type { AccessTokenSource } from './credential-sources.ts';
 
@@ -50,6 +51,24 @@ export async function importUnlessMissing<Module>(load: () => Promise<Module>):
   }
 }
 
+export type Resolve = (specifier: string) => string;
+
+function resolvedInPlace(specifier: string): string {
+  return import.meta.resolve(specifier);
+}
+
+export function installedPackages(resolve: Resolve = resolvedInPlace): OptionalPackages {
+  try {
+    resolve('@azure/identity');
+    return { entraId: true };
+  } catch (error) {
+    if (isMissingModule(error)) {
+      return { entraId: false };
+    }
+    throw error;
+  }
+}
+
 export interface IdentityLibrary {
   readonly DefaultAzureCredential: new () => TokenCredential;
   readonly getBearerTokenProvider: (credential: TokenCredential, scope: string) => AccessTokenSource;

capabilities/reasoning/src/index.ts

@@ -1,4 +1,5 @@
 export { languageModelLayer } from './adapter/language-model-layer.ts';
+export { installedPackages } from './adapter/entra-id.ts';
 export { makeModelAccess, type ModelAccess } from './adapter/model-access.ts';
 export type {
   ModelAccessOptions,
@@ -31,6 +32,7 @@ export { makeReasoningFunctionAdapter, type ReasoningFunctionAdapterOptions } fr
 export { defaultMostInputTokens, type ModelReading } from './capability/definition-request.ts';
 export type { ReasoningFunctionDefinitionDocument } from './definition/reasoning-function-definition.ts';
 export { parseModelReference, type ModelReference } from './model/model-reference.ts';
+export { offeredModels, type OfferedModels } from './model/offered-models.ts';
 export type {
   ContentPart,
   GenerationSettings,

docs/self-host/configuration.md

@@ -223,7 +223,7 @@ The folder of `PREDICTION_CACHE_DIR` follows these rules:
 
 - At start, the server creates the folder when it is missing, readable only by its own user, and does the same for each folder within it.
 - It refuses to start, naming the setting, when the folder cannot be made, belongs to another user, or other users may write to it. A folder another user may write to would let that user change a model before it is loaded.
-- The files found in the folder at start are read again, and kept only when their bytes match the digest in their name.
+- The files found in the folder are read again when a prediction first uses them, and kept only when their bytes match the digest in their name.
 - Each server keeps at most 1 GiB of model files and 1 GiB of rows, by its own count. Give two servers on one machine a folder each.
 
 ## Usage ping {#usage-ping}
@@ -276,11 +276,11 @@ A tool server gives functions tools to call. It comes from one of two places: an
 
 Each entry is one of three types. When `type` is left out, it is taken from the field that names the server:
 
-| Type      | Named by   | What it is                                                                                                       |
-| --------- | ---------- | ---------------------------------------------------------------------------------------------------------------- |
-| `mcp`     | `url`      | A remote MCP server, spoken to over Streamable HTTP                                                              |
-| `stdio`   | `command`  | An MCP server the runtime runs as a process                                                                      |
-| `openapi` | `document` | An HTTP API described by an OpenAPI document, 3.0 or 3.1, read once at start. Each of its operations is one tool |
+| Type      | Named by   | What it is                                                                                                                                           |
+| --------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `mcp`     | `url`      | A remote MCP server, spoken to over Streamable HTTP                                                                                                  |
+| `stdio`   | `command`  | An MCP server the runtime runs as a process                                                                                                          |
+| `openapi` | `document` | An HTTP API described by an OpenAPI document, 3.0 or 3.1, read once, when a request first needs the tool servers. Each of its operations is one tool |
 
 ```yaml
 tool_servers:
@@ -396,10 +396,10 @@ Which credentials a call sends:
 - An operation is offered as a tool when the entry gives one of its alternatives.
 - A call sends the credentials of the first alternative the entry gives, in the document's order, and no other. Where the operation needs a header that the entry's `headers` do not name, it sends the first alternative the entry gives whose `apiKey` schemes place that header instead.
 - An operation whose requirement is empty, or of a document that declares none, sends none.
-- An operation for which the entry gives none of the alternatives is not offered as a tool. At start, the server logs a warning that names the schemes it needs.
+- An operation for which the entry gives none of the alternatives is not offered as a tool. When it reads the document, the server logs a warning that names the schemes it needs.
 - The scopes an `oauth2` requirement names are the operation's. The `auth` block asks for its own `scope`, so set it to cover what the operations require.
 
-The server stops at start, naming the entry, when:
+The server refuses the documents as it reads them, naming the entry, when:
 
 - a name under `security` is not a scheme the document declares;
 - a name under `security` is an `apiKey` the document places in a cookie;
@@ -412,7 +412,7 @@ The server stops at start, naming the entry, when:
 
 ### How an HTTP API becomes tools
 
-At start the server reads each `openapi` entry's document: at most 4 MiB and 500 operations, within 10 seconds. It stops, naming the entry, when:
+The server reads each `openapi` entry's document the first time a request needs the tool servers, such as a `search`, a call made with `execute`, a run of a function that calls tools, or a request of an interaction function that comes due: at most 4 MiB and 500 operations, within 10 seconds. It refuses the documents, naming the entry, when:
 
 - it cannot read the document;
 - the document's URL responds with a redirect, which the server does not follow;
@@ -425,6 +425,8 @@ At start the server reads each `openapi` entry's document: at most 4 MiB and 500
 - a reference does not resolve inside the document. This includes a reference to an http, https or file address, which the server never fetches;
 - its calls would have nowhere to go: no `base_url` and no server, or a server that is neither https nor on a loopback address.
 
+A refusal fails the request that needed the tool servers, with the incident of an unexpected error, and the server logs the incident with the refusal, naming the entry and the place. The next request reads the documents again, so a document put right while the server runs is read without a restart.
+
 Trailing slashes on `base_url` or on the server's URL are dropped.
 
 Each operation becomes one tool:
@@ -434,7 +436,7 @@ Each operation becomes one tool:
 - Its input is one object: each path and query parameter by its name, and `body` for a JSON request body, each required when the document says so. The schemas of `components` they refer to are held inside it, so a function's model reads one schema whole.
 - `GET` and `HEAD` are read-only, `DELETE` destructive, and `GET`, `HEAD`, `PUT` and `DELETE` idempotent. So `allowed`, `search` and a run's read-only calls read an API's tools as they read an MCP server's.
 
-Some operations are not offered as tools. At start, the server logs a warning for each operation it leaves out, naming the entry and the reason. It leaves out an operation that:
+Some operations are not offered as tools. When it reads the document, the server logs a warning for each operation it leaves out, naming the entry and the reason. It leaves out an operation that:
 
 - needs a cookie;
 - needs a header that the entry's `headers` do not name and its security scheme does not place. The warning says that `headers` can give it, or, for a header the server sets itself such as `Host`, that it does. A header parameter named `Accept`, `Content-Type` or `Authorization` is ignored, as the OpenAPI specification says;

docs/self-host/docker.md

@@ -264,7 +264,7 @@ Then run `docker compose up --detach`, which replaces Auto's container because i
 Rules for the file:
 
 - Secrets stay in `auto-runtime.env`. The file refers to them by name, such as `${GATEWAY_API_KEY}`.
-- An `openapi` tool server whose entry in `tool_servers` names its OpenAPI document by a path reads that document at start, from the directory of the file. Mount the document there too, as `--volume "$PWD/billing.json:/etc/auto-runtime/billing.json:ro"`, or name it by an https URL the container can reach.
+- An `openapi` tool server whose entry in `tool_servers` names its OpenAPI document by a path reads that document from the directory of the file, the first time a request needs the tool servers. Mount the document there too, as `--volume "$PWD/billing.json:/etc/auto-runtime/billing.json:ro"`, or name it by an https URL the container can reach.
 - A document that Auto cannot read stops the start, and the error names the entry.
 
 The [configuration reference](configuration.md) describes the file and every setting.