diff --git a/capabilities/reasoning/README.md b/capabilities/reasoning/README.md index 5abb16f35..a0c157e38 100644 --- a/capabilities/reasoning/README.md +++ b/capabilities/reasoning/README.md @@ -74,4 +74,4 @@ Two tests wait on real time on purpose, where a slow machine only lengthens the ## Source -`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries: it imports `google-auth-library`, with the AI SDK's Vertex providers, only when model access is built with Vertex configured (`src/adapter/vertex-providers.ts`), the AWS credential chain only with Bedrock configured, through `awsCredentialChainOf` of `@on.auto/outbound`, and `@azure/identity` only for Azure without a key, and no entry of the capability reaches any of them (`src/adapter/credential-libraries.test.ts`). `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model. +`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries: it imports `google-auth-library`, with the AI SDK's Vertex providers, only when model access is built with Vertex configured (`src/adapter/vertex-providers.ts`), the AWS credential chain only with Bedrock configured, through `awsCredentialChainOf` of `@on.auto/outbound`, and `@azure/identity` only for Azure without a key, and no entry of the capability reaches any of them (`src/adapter/credential-libraries.test.ts`). `installedPackages` (`src/adapter/entra-id.ts`) says whether that optional package is installed by resolving it, without loading it, so a server can read from its settings alone, with `providerStatus` and `offeredModels`, which providers are configured and which models it offers, before it builds the model access, as the server does at start. `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model. diff --git a/capabilities/reasoning/src/adapter/entra-id.test.ts b/capabilities/reasoning/src/adapter/entra-id.test.ts index 590157c6d..0f5888e6f 100644 --- a/capabilities/reasoning/src/adapter/entra-id.test.ts +++ b/capabilities/reasoning/src/adapter/entra-id.test.ts @@ -5,6 +5,7 @@ import { entraIdentityLoader, entraTokens, importUnlessMissing, + installedPackages, loadEntraIdentity, type AccessToken, } from './entra-id.ts'; @@ -15,6 +16,10 @@ function missingModule(): Promise { ); } +function unresolved(specifier: string): never { + throw Object.assign(new Error(`Cannot find package '${specifier}'`), { code: 'ERR_MODULE_NOT_FOUND' }); +} + function tokenFor(token: string): Promise { return Promise.resolve({ token, expiresOnTimestamp: Date.now() + 3_600_000 }); } @@ -71,3 +76,23 @@ describe('entraTokens', () => { await expect(tokens()).resolves.toBe(`token for ${cognitiveServicesScope}`); }); }); + +describe('installedPackages', () => { + it('finds the identity library installed where it resolves', () => { + expect(installedPackages()).toEqual({ entraId: true }); + }); + + it('finds it missing when the package does not resolve', () => { + expect(installedPackages(unresolved)).toEqual({ entraId: false }); + }); + + it('passes any other failure on', () => { + const broken = new TypeError('Invalid module specifier'); + + expect(() => + installedPackages(() => { + throw broken; + }), + ).toThrow(broken); + }); +}); diff --git a/capabilities/reasoning/src/adapter/entra-id.ts b/capabilities/reasoning/src/adapter/entra-id.ts index 51c879359..21c1b7f3d 100644 --- a/capabilities/reasoning/src/adapter/entra-id.ts +++ b/capabilities/reasoning/src/adapter/entra-id.ts @@ -1,6 +1,7 @@ import { Effect, Predicate } from 'effect'; import type { AzureSettings } from '../settings/provider-settings.ts'; +import type { OptionalPackages } from '../settings/provider-status.ts'; import type { Availability } from '../settings/setting-values.ts'; import type { AccessTokenSource } from './credential-sources.ts'; @@ -50,6 +51,24 @@ export async function importUnlessMissing(load: () => Promise): } } +export type Resolve = (specifier: string) => string; + +function resolvedInPlace(specifier: string): string { + return import.meta.resolve(specifier); +} + +export function installedPackages(resolve: Resolve = resolvedInPlace): OptionalPackages { + try { + resolve('@azure/identity'); + return { entraId: true }; + } catch (error) { + if (isMissingModule(error)) { + return { entraId: false }; + } + throw error; + } +} + export interface IdentityLibrary { readonly DefaultAzureCredential: new () => TokenCredential; readonly getBearerTokenProvider: (credential: TokenCredential, scope: string) => AccessTokenSource; diff --git a/capabilities/reasoning/src/index.ts b/capabilities/reasoning/src/index.ts index c951d4903..0b1f6a7ef 100644 --- a/capabilities/reasoning/src/index.ts +++ b/capabilities/reasoning/src/index.ts @@ -1,4 +1,5 @@ export { languageModelLayer } from './adapter/language-model-layer.ts'; +export { installedPackages } from './adapter/entra-id.ts'; export { makeModelAccess, type ModelAccess } from './adapter/model-access.ts'; export type { ModelAccessOptions, @@ -31,6 +32,7 @@ export { makeReasoningFunctionAdapter, type ReasoningFunctionAdapterOptions } fr export { defaultMostInputTokens, type ModelReading } from './capability/definition-request.ts'; export type { ReasoningFunctionDefinitionDocument } from './definition/reasoning-function-definition.ts'; export { parseModelReference, type ModelReference } from './model/model-reference.ts'; +export { offeredModels, type OfferedModels } from './model/offered-models.ts'; export type { ContentPart, GenerationSettings, diff --git a/docs/self-host/configuration.md b/docs/self-host/configuration.md index 4218abf47..f4dae5662 100644 --- a/docs/self-host/configuration.md +++ b/docs/self-host/configuration.md @@ -223,7 +223,7 @@ The folder of `PREDICTION_CACHE_DIR` follows these rules: - At start, the server creates the folder when it is missing, readable only by its own user, and does the same for each folder within it. - It refuses to start, naming the setting, when the folder cannot be made, belongs to another user, or other users may write to it. A folder another user may write to would let that user change a model before it is loaded. -- The files found in the folder at start are read again, and kept only when their bytes match the digest in their name. +- The files found in the folder are read again when a prediction first uses them, and kept only when their bytes match the digest in their name. - Each server keeps at most 1 GiB of model files and 1 GiB of rows, by its own count. Give two servers on one machine a folder each. ## Usage ping {#usage-ping} @@ -276,11 +276,11 @@ A tool server gives functions tools to call. It comes from one of two places: an Each entry is one of three types. When `type` is left out, it is taken from the field that names the server: -| Type | Named by | What it is | -| --------- | ---------- | ---------------------------------------------------------------------------------------------------------------- | -| `mcp` | `url` | A remote MCP server, spoken to over Streamable HTTP | -| `stdio` | `command` | An MCP server the runtime runs as a process | -| `openapi` | `document` | An HTTP API described by an OpenAPI document, 3.0 or 3.1, read once at start. Each of its operations is one tool | +| Type | Named by | What it is | +| --------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| `mcp` | `url` | A remote MCP server, spoken to over Streamable HTTP | +| `stdio` | `command` | An MCP server the runtime runs as a process | +| `openapi` | `document` | An HTTP API described by an OpenAPI document, 3.0 or 3.1, read once, when a request first needs the tool servers. Each of its operations is one tool | ```yaml tool_servers: @@ -396,10 +396,10 @@ Which credentials a call sends: - An operation is offered as a tool when the entry gives one of its alternatives. - A call sends the credentials of the first alternative the entry gives, in the document's order, and no other. Where the operation needs a header that the entry's `headers` do not name, it sends the first alternative the entry gives whose `apiKey` schemes place that header instead. - An operation whose requirement is empty, or of a document that declares none, sends none. -- An operation for which the entry gives none of the alternatives is not offered as a tool. At start, the server logs a warning that names the schemes it needs. +- An operation for which the entry gives none of the alternatives is not offered as a tool. When it reads the document, the server logs a warning that names the schemes it needs. - The scopes an `oauth2` requirement names are the operation's. The `auth` block asks for its own `scope`, so set it to cover what the operations require. -The server stops at start, naming the entry, when: +The server refuses the documents as it reads them, naming the entry, when: - a name under `security` is not a scheme the document declares; - a name under `security` is an `apiKey` the document places in a cookie; @@ -412,7 +412,7 @@ The server stops at start, naming the entry, when: ### How an HTTP API becomes tools -At start the server reads each `openapi` entry's document: at most 4 MiB and 500 operations, within 10 seconds. It stops, naming the entry, when: +The server reads each `openapi` entry's document the first time a request needs the tool servers, such as a `search`, a call made with `execute`, a run of a function that calls tools, or a request of an interaction function that comes due: at most 4 MiB and 500 operations, within 10 seconds. It refuses the documents, naming the entry, when: - it cannot read the document; - the document's URL responds with a redirect, which the server does not follow; @@ -425,6 +425,8 @@ At start the server reads each `openapi` entry's document: at most 4 MiB and 500 - a reference does not resolve inside the document. This includes a reference to an http, https or file address, which the server never fetches; - its calls would have nowhere to go: no `base_url` and no server, or a server that is neither https nor on a loopback address. +A refusal fails the request that needed the tool servers, with the incident of an unexpected error, and the server logs the incident with the refusal, naming the entry and the place. The next request reads the documents again, so a document put right while the server runs is read without a restart. + Trailing slashes on `base_url` or on the server's URL are dropped. Each operation becomes one tool: @@ -434,7 +436,7 @@ Each operation becomes one tool: - Its input is one object: each path and query parameter by its name, and `body` for a JSON request body, each required when the document says so. The schemas of `components` they refer to are held inside it, so a function's model reads one schema whole. - `GET` and `HEAD` are read-only, `DELETE` destructive, and `GET`, `HEAD`, `PUT` and `DELETE` idempotent. So `allowed`, `search` and a run's read-only calls read an API's tools as they read an MCP server's. -Some operations are not offered as tools. At start, the server logs a warning for each operation it leaves out, naming the entry and the reason. It leaves out an operation that: +Some operations are not offered as tools. When it reads the document, the server logs a warning for each operation it leaves out, naming the entry and the reason. It leaves out an operation that: - needs a cookie; - needs a header that the entry's `headers` do not name and its security scheme does not place. The warning says that `headers` can give it, or, for a header the server sets itself such as `Host`, that it does. A header parameter named `Accept`, `Content-Type` or `Authorization` is ignored, as the OpenAPI specification says; diff --git a/docs/self-host/docker.md b/docs/self-host/docker.md index 892be6121..a7baaf00b 100644 --- a/docs/self-host/docker.md +++ b/docs/self-host/docker.md @@ -264,7 +264,7 @@ Then run `docker compose up --detach`, which replaces Auto's container because i Rules for the file: - Secrets stay in `auto-runtime.env`. The file refers to them by name, such as `${GATEWAY_API_KEY}`. -- An `openapi` tool server whose entry in `tool_servers` names its OpenAPI document by a path reads that document at start, from the directory of the file. Mount the document there too, as `--volume "$PWD/billing.json:/etc/auto-runtime/billing.json:ro"`, or name it by an https URL the container can reach. +- An `openapi` tool server whose entry in `tool_servers` names its OpenAPI document by a path reads that document from the directory of the file, the first time a request needs the tool servers. Mount the document there too, as `--volume "$PWD/billing.json:/etc/auto-runtime/billing.json:ro"`, or name it by an https URL the container can reach. - A document that Auto cannot read stops the start, and the error names the entry. The [configuration reference](configuration.md) describes the file and every setting. diff --git a/packages/server/README.md b/packages/server/README.md index e63eccc1b..e5bc8af4d 100644 --- a/packages/server/README.md +++ b/packages/server/README.md @@ -2,6 +2,10 @@ The Node.js server of Auto Runtime: `src/main.ts` starts it on the composition root of `src/composition/composition-root.ts`, `dev.ts` restarts it on every save, and `auto-runtime.ts` is the command of the npm package `@on.auto/runtime`, which `npm/pack.ts` packs with the `README.md` in `npm/`; the `Dockerfile` builds its container image. How the files the packages ship are read is composed once, `shippedFilesOnDisk` of `@on.auto/operations/shipped-files-on-disk` in the options of `src/server-options/composed-options.ts`, and handed on as `ServerOptions.shippedFiles`: at the start, to the providers' entries the settings are read against (`readShippedSettings`, `src/settings/served-settings.ts`) and to the style of the API's pages, `readPageStyle` of `@on.auto/api`, each read once and handed on as content, so a file that cannot be read stops the start (`src/lifecycle/lifecycle.ts`, `src/lifecycle/page-style-startup.test.ts`); then to `composed`, and from there to the files the pool of workers starts with, read where the one pool is made (`src/composition/served-computation.ts`), and to the guides and recipes the server serves (`src/guides`). `src/server-options/server-options.ts` holds what a server is started with and answers, `ServerOptions`, `Composed`, `Served` and `RunningServer` among them, which the start in `src/lifecycle` and every part the composition serves import, so nothing imports the start for them. The composition imports the entry of the ledger of the store its settings name, and no other: `ledgerDatabaseOf` of `src/database/ledger-database.ts` imports `src/database/sqlite-ledger-database.ts`, over `@on.auto/ledger/sqlite3`, or `src/database/postgresql-ledger-database.ts`, over `@on.auto/ledger/postgresql` and `pg`, through a dynamic import, and answers the database the workflow host opens with the layer of the ledger and the rows of the secrets on it, which `composed` awaits before it builds anything, so a server on PostgreSQL never loads the native addon of `sqlite3` and one on SQLite never loads `pg` (`src/database/ledger-entries.test.ts`); `composed` may therefore answer a promise, which the start awaits. +## What is composed at start, and what on first use + +The composition root composes at start what every request and the host's own work need, in `serve` of `src/composition/composition-root.ts`: the application layer, with the ledger, the playbook registry, the incident reporter and telemetry; the store of secrets and the connections over it; the one pool of workers, which starts no worker before its first job, and whose evaluations decide the steps of workflows and filter the events reactions take; the store of the workflow host, opened with the tables of recall's views and of the datasets of prediction functions; the pool of model processes and the ports over it, which start no process before their first job; the adapters, operations and due work of every capability; the workflow host; and the table of routes, so `list_operations`, the OpenAPI document and the guides are whole from the start. What costs a composition is an entry of the capability registry (`src/capability-registry/capability-registry.ts`), which the root holds, each a `LayerRef` of Effect over the layer that composes it, built the first time it is asked for and kept for the server's life (`composedOnFirstUse`, `src/capability-registry/first-use.ts`): `models`, the model access of reasoning functions, with a client for each configured provider and gateway and the credential libraries some of them load, and `tools`, the access to the tool servers, with the OpenAPI document of each `openapi` entry read and described. The reasoning and interaction adapters, `list_models`, the five operations over the tool servers and the due work of interaction functions are made at start over a language model, a model catalog and a tool access that ask the registry (`src/composition/served-reasoning.ts`, `src/composition/served-tool-access.ts`), so the first request that calls a model, lists the models or uses a tool server, and the first request of an interaction function that comes due, pays the composition inside its own request or due item and its bounds, and every later one finds it composed (`src/reasoning/models-on-first-use.test.ts`). A composition that fails is not kept: the model access cannot fail, and documents the tool access cannot read fail the request that needed them with an incident, whose log line carries the words the start used to stop with, `tool_server_document_refused: The documents of the tool servers could not be read. TOOL_SERVERS: …`, and the next request reads them again (`src/tool-servers/documents-on-first-use.test.ts`). The registry is held in a scope the runtime closes as it is disposed, once the API has closed and before the services built ahead of it (`scopeOfTheRuntime`), since an entry closed answers no ask, and a request still in flight while the server stops its work keeps the models and the tools it uses. The start log still names where the ledger, content and secrets are kept and which model providers are configured, read from the settings with `providerStatus` and `installedPackages` of [`@on.auto/reasoning`](../../capabilities/reasoning/README.md#source), and the folder of the model files is made and checked at start, while the caches read and check the files in it when a prediction first uses them (`src/prediction/model-cache-folder.test.ts`). `Auto Runtime listening on port ` therefore means the server listens: no composition waits behind it. The start and the first request of each kind, before and after, are measured in [`@on.auto/tool-servers`](../tool-servers/README.md#entry-points), beside its start measure. + ## The secrets key `src/settings/keeper-settings.ts` reads where the value of each secret rests: by default, or where `SECRETS_KEEPER` is `ledger`, sealed in the ledger's database, and where it is `aws-parameter-store:`, in AWS Systems Manager Parameter Store under that prefix, an absolute parameter path of at most 14 segments of letters, digits, `.`, `-` and `_`, the first beginning with neither `aws` nor `ssm`, and no slash at its end, in the region `AWS_REGION` names, or `AWS_DEFAULT_REGION` where that is not set, kept with the name of the variable it was read from; a keeper named with neither stops the start, naming `SECRETS_KEEPER` and `AWS_REGION`. `SECRETS_KEEPER_PREVIOUS` names the keeper being replaced, only beside `SECRETS_KEEPER` and the other of the two. diff --git a/packages/server/src/capability-registry/capability-registry.ts b/packages/server/src/capability-registry/capability-registry.ts new file mode 100644 index 000000000..09822e4dc --- /dev/null +++ b/packages/server/src/capability-registry/capability-registry.ts @@ -0,0 +1,56 @@ +import type { AppRuntime } from '@on.auto/api'; +import type { DispatcherServices } from '@on.auto/operations'; +import type { ModelAccess, ModelTelemetry } from '@on.auto/reasoning'; +import type { ToolAccess, ToolServerDocumentRefused } from '@on.auto/tool-servers'; +import { Effect, type Layer, type Scope } from 'effect'; + +import { ComposedModels, modelsComposedBy, type ModelAccessOf } from '../composition/served-reasoning.ts'; +import { + ComposedTools, + toolsComposedBy, + type ServedConnections, + type ToolAccessWiring, +} from '../composition/served-tool-access.ts'; +import type { Settings } from '../settings/settings.ts'; +import { composedOnFirstUse, scopeOfTheRuntime } from './first-use.ts'; + +export interface CapabilityRegistry { + readonly models: Effect.Effect; + readonly tools: Effect.Effect; +} + +export interface HeldRegistry { + readonly closedFirstOf: (services: Layer.Layer) => Layer.Layer; + readonly openedIn: ( + runtime: AppRuntime, + settings: Settings, + wiring: RegistryWiring, + ) => Promise; +} + +interface RegistryWiring extends ToolAccessWiring, ServedConnections { + readonly modelAccessOf: ModelAccessOf; + readonly telemetry: NonNullable & { readonly models: ModelTelemetry | undefined }; +} + +function capabilityRegistryOf( + runtime: AppRuntime, + settings: Settings, + wiring: RegistryWiring, +): Effect.Effect { + return Effect.all({ + models: composedOnFirstUse( + ComposedModels, + modelsComposedBy(settings, wiring.modelAccessOf, wiring.telemetry.models), + ), + tools: composedOnFirstUse(ComposedTools, toolsComposedBy(runtime, settings, wiring)), + }); +} + +export function capabilityRegistryHeld(): HeldRegistry { + const { closedFirstOf, openedIn } = scopeOfTheRuntime(); + return { + closedFirstOf, + openedIn: (runtime, settings, wiring) => openedIn(runtime, capabilityRegistryOf(runtime, settings, wiring)), + }; +} diff --git a/packages/server/src/capability-registry/first-use.test.ts b/packages/server/src/capability-registry/first-use.test.ts new file mode 100644 index 000000000..20f059a2b --- /dev/null +++ b/packages/server/src/capability-registry/first-use.test.ts @@ -0,0 +1,140 @@ +import { makeAppRuntime } from '@on.auto/api'; +import { memoryLedger, memoryPlaybookRegistry, recordingReporter } from '@on.auto/operations/testing'; +import { Context, Effect, Exit, Layer, Scope } from 'effect'; +import { describe, expect, it } from 'vitest'; + +import { composedOnFirstUse, scopeOfTheRuntime } from './first-use.ts'; + +interface Composition { + readonly number: number; +} + +class Composed extends Context.Service()('test/Composed') {} + +interface Counted { + readonly composed: () => number; + readonly ended: () => number; + readonly layer: Layer.Layer; +} + +function counted(failing: (attempt: number) => boolean = () => false): Counted { + const counts = { composed: 0, ended: 0 }; + const composing = Effect.suspend(() => { + counts.composed += 1; + const attempt = counts.composed; + return failing(attempt) ? Effect.fail(new Error(`attempt ${attempt} failed`)) : Effect.succeed({ number: attempt }); + }); + const ending = Effect.sync(() => { + counts.ended += 1; + }); + return { + composed: () => counts.composed, + ended: () => counts.ended, + layer: Layer.effect( + Composed, + Effect.acquireRelease(composing, () => ending), + ), + }; +} + +async function openedOver(layer: Layer.Layer) { + const scope = Scope.makeUnsafe(); + const firstUse = await Effect.runPromise(Scope.provide(composedOnFirstUse(Composed, layer), scope)); + return { firstUse, close: () => Effect.runPromise(Scope.close(scope, Exit.void)) }; +} + +describe('a composition on first use', () => { + it('composes nothing until it is asked for', async () => { + const counts = counted(); + const { close } = await openedOver(counts.layer); + + await close(); + + expect([counts.composed(), counts.ended()]).toEqual([0, 0]); + }); + + it('composes the first time it is asked for, and answers that same composition ever after', async () => { + const counts = counted(); + const { firstUse, close } = await openedOver(counts.layer); + + const first = await Effect.runPromise(firstUse); + const second = await Effect.runPromise(firstUse); + await close(); + + expect({ same: first === second, composed: counts.composed(), first }).toEqual({ + same: true, + composed: 1, + first: { number: 1 }, + }); + }); + + it('composes once when it is asked for many times at once', async () => { + const counts = counted(); + const { firstUse, close } = await openedOver(counts.layer); + + const answers = await Effect.runPromise(Effect.all([firstUse, firstUse, firstUse], { concurrency: 'unbounded' })); + await close(); + + expect({ numbers: answers.map(({ number }) => number), composed: counts.composed() }).toEqual({ + numbers: [1, 1, 1], + composed: 1, + }); + }); +}); + +describe('a composition on first use that fails or ends', () => { + it('answers a failed composition with its failure, and composes again the next time it is asked for', async () => { + const counts = counted((attempt) => attempt === 1); + const { firstUse, close } = await openedOver(counts.layer); + + const failed = await Effect.runPromise(Effect.flip(firstUse)); + const composed = await Effect.runPromise(firstUse); + const kept = await Effect.runPromise(firstUse); + await close(); + + expect({ failed: failed.message, composed, kept, attempts: counts.composed() }).toEqual({ + failed: 'attempt 1 failed', + composed: { number: 2 }, + kept: { number: 2 }, + attempts: 2, + }); + }); + + it('ends the composition it holds once, when it is closed', async () => { + const counts = counted(); + const { firstUse, close } = await openedOver(counts.layer); + await Effect.runPromise(firstUse); + await Effect.runPromise(firstUse); + + const before = counts.ended(); + await close(); + + expect([before, counts.ended()]).toEqual([0, 1]); + }); +}); + +describe('the scope of the runtime that holds the compositions', () => { + it('closes what it holds when the runtime is disposed, before the services the runtime built first', async () => { + const ended: string[] = []; + const endingAs = (name: string) => + Effect.addFinalizer(() => + Effect.sync(() => { + ended.push(name); + }), + ); + const services = Layer.mergeAll( + memoryLedger().layer, + memoryPlaybookRegistry([]), + recordingReporter().layer, + Layer.effectDiscard(endingAs('services')), + ); + const { closedFirstOf, openedIn } = scopeOfTheRuntime(); + const runtime = await makeAppRuntime(closedFirstOf(services)); + await openedIn(runtime, endingAs('compositions')); + + const beforeDisposal = [...ended]; + await runtime.dispose(); + + expect({ beforeDisposal, ended }).toEqual({ beforeDisposal: [], ended: ['compositions', 'services'] }); + }); +}); diff --git a/packages/server/src/capability-registry/first-use.ts b/packages/server/src/capability-registry/first-use.ts new file mode 100644 index 000000000..4c36160a3 --- /dev/null +++ b/packages/server/src/capability-registry/first-use.ts @@ -0,0 +1,31 @@ +import type { AppRuntime } from '@on.auto/api'; +import type { DispatcherServices } from '@on.auto/operations'; +import { Duration, Effect, Exit, Layer, LayerRef, Scope } from 'effect'; + +import { inRuntime } from '../workflows/in-runtime.ts'; + +export interface ScopeOfTheRuntime { + readonly closedFirstOf: (services: Layer.Layer) => Layer.Layer; + readonly openedIn: ( + runtime: AppRuntime, + opening: Effect.Effect, + ) => Promise; +} + +export function composedOnFirstUse( + service: Effect.Effect, + layer: Layer.Layer, +): Effect.Effect, never, Scope.Scope> { + return Effect.map(LayerRef.make(layer, { idleTimeToLive: Duration.infinity, preload: false }), (composed) => + Effect.provide(service, composed.get), + ); +} + +export function scopeOfTheRuntime(): ScopeOfTheRuntime { + const scope = Scope.makeUnsafe(); + const closing = Layer.effectDiscard(Effect.addFinalizer(() => Scope.close(scope, Exit.void))); + return { + closedFirstOf: (services) => Layer.provideMerge(closing, services), + openedIn: (runtime, opening) => Effect.runPromise(inRuntime(runtime, Scope.provide(opening, scope))), + }; +} diff --git a/packages/server/src/composition/composition-root.ts b/packages/server/src/composition/composition-root.ts index 5e0105fe1..70bd045ef 100644 --- a/packages/server/src/composition/composition-root.ts +++ b/packages/server/src/composition/composition-root.ts @@ -1,3 +1,4 @@ +import { capabilityRegistryHeld } from '../capability-registry/capability-registry.ts'; import { storesAndConnectionsOf } from '../connections/served-connections.ts'; import { ledgerDatabaseOf, withARefusedLayoutExplained } from '../database/ledger-database.ts'; import { serverOptionsComposing, type ComposedServerOptions } from '../server-options/composed-options.ts'; @@ -6,31 +7,39 @@ import { applicationLayer } from './application-layer.ts'; import { functionWiringOf, type FunctionWiring } from './function-wiring.ts'; import { ledgerLayerFor } from './ledger-store.ts'; import { functionsServedBy } from './served-functions.ts'; -import { loggedModelAccess } from './served-reasoning.ts'; +import { loggedModelAccess, type ModelAccessOf } from './served-reasoning.ts'; export function compositionRootWith( - modelAccessOf: FunctionWiring['modelAccessOf'], + modelAccessOf: ModelAccessOf, programPoolOf?: FunctionWiring['programPoolOf'], clock?: WorkflowParts['clock'], toolTiming?: FunctionWiring['toolTiming'], ): ComposedServerOptions { - const wiring = functionWiringOf(modelAccessOf, programPoolOf, toolTiming); + const wiring = functionWiringOf(programPoolOf, toolTiming, clock?.now); return serverOptionsComposing(async (settings, telemetry, shippedFiles) => { const database = await ledgerDatabaseOf(settings, telemetry.appended); + const capabilities = capabilityRegistryHeld(); return { - services: applicationLayer( - withARefusedLayoutExplained(ledgerLayerFor(settings, database, wiring.projections.appends), settings), + services: capabilities.closedFirstOf( + applicationLayer( + withARefusedLayoutExplained(ledgerLayerFor(settings, database, wiring.projections.appends), settings), + ), ), serve: async (runtime, ownOrigin) => { const { ledger, workflows, identityIssuer } = settings; const { stores, retryStateKeys, consent, connections, source, offering, clientMetadata } = await storesAndConnectionsOf(runtime, settings, database, ownOrigin); - const functions = await functionsServedBy(runtime, settings, database, { + const registry = await capabilities.openedIn(runtime, settings, { ...wiring, - ...(clock === undefined ? {} : { now: clock.now }), + modelAccessOf, telemetry, consent, connections: source, + }); + const functions = await functionsServedBy(runtime, settings, database, { + ...wiring, + registry, + connections: source, offering, shippedFiles, }); diff --git a/packages/server/src/composition/function-wiring.ts b/packages/server/src/composition/function-wiring.ts index c5490605b..1626041ab 100644 --- a/packages/server/src/composition/function-wiring.ts +++ b/packages/server/src/composition/function-wiring.ts @@ -1,22 +1,21 @@ import { workerPool, type ProgramPoolOf } from './served-computation.ts'; import { projectionWiring, type ProjectionWiring } from './served-projections.ts'; -import type { ModelAccessOf } from './served-reasoning.ts'; -import type { ToolWiring } from './served-tools.ts'; +import type { ToolAccessWiring } from './served-tool-access.ts'; -export interface FunctionWiring extends ToolWiring { +export interface FunctionWiring extends ToolAccessWiring { readonly programPoolOf: ProgramPoolOf; readonly projections: ProjectionWiring; } export function functionWiringOf( - modelAccessOf: ModelAccessOf, programPoolOf: ProgramPoolOf = workerPool, toolTiming?: FunctionWiring['toolTiming'], + now?: FunctionWiring['now'], ): FunctionWiring { return { - modelAccessOf, programPoolOf, projections: projectionWiring(), ...(toolTiming === undefined ? {} : { toolTiming }), + ...(now === undefined ? {} : { now }), }; } diff --git a/packages/server/src/composition/served-functions.ts b/packages/server/src/composition/served-functions.ts index 1b4f25bdd..afe683423 100644 --- a/packages/server/src/composition/served-functions.ts +++ b/packages/server/src/composition/served-functions.ts @@ -30,10 +30,9 @@ export async function functionsServedBy( projections: wiring, shippedFiles, ...tools - }: FunctionWiring & - Pick & { readonly shippedFiles: ShippedFiles }, + }: FunctionWiring & ToolUsersWiring & { readonly shippedFiles: ShippedFiles }, ): Promise { - const { reasoning, interaction, toolOperations, withToolsClosed } = await toolUsersServedBy(runtime, settings, tools); + const { reasoning, interaction, toolOperations } = await toolUsersServedBy(runtime, settings, tools); const computation = await computationServedBy(settings.computation, programPoolOf, shippedFiles); const projections = await wiring.served(runtime, { recall: settings.recall, database }, computation.pool, [ datasetsFolderOf(computation.pool), @@ -64,6 +63,6 @@ export async function functionsServedBy( evaluations: computation.pool.evaluations, check: computation.check, }, - closing: (served) => prediction.withProcessesClosed(computation.withPoolClosed(withToolsClosed(served))), + closing: (served) => prediction.withProcessesClosed(computation.withPoolClosed(served)), }; } diff --git a/packages/server/src/composition/served-models.ts b/packages/server/src/composition/served-models.ts index 261e67593..b59a737c2 100644 --- a/packages/server/src/composition/served-models.ts +++ b/packages/server/src/composition/served-models.ts @@ -1,7 +1,6 @@ import { cacheFolderProblem } from '@on.auto/operations'; import { modelPorts, type ModelPorts } from '@on.auto/prediction'; import { processPool, type ProcessPool } from '@on.auto/workflow-engine/dsl'; -import { Effect } from 'effect'; import type { PredictionSettings } from '../function-settings/prediction-settings.ts'; import { InvalidSettingsError } from '../settings/invalid-settings-error.ts'; @@ -26,7 +25,5 @@ export async function servedModels(settings: PredictionSettings): Promise; } +type ReasoningSettings = Pick; + +export class ComposedModels extends Context.Service()('@on.auto/server/ComposedModels') {} + export const loggedModelAccess: ModelAccessOf = (settings, proxy, telemetry) => makeModelAccess(settings, { proxy, @@ -35,25 +44,45 @@ export const loggedModelAccess: ModelAccessOf = (settings, proxy, telemetry) => telemetry, }); +export function modelsComposedBy( + { models, proxy, providerSetup }: ReasoningSettings, + modelAccessOf: ModelAccessOf, + telemetry?: ModelTelemetry, +): Layer.Layer { + return Layer.effect(ComposedModels, modelAccessOf({ ...models, providerSetup }, proxy, telemetry)); +} + +function languageModelOf(models: Effect.Effect): LanguageModel['Service'] { + return { + admit: (request) => Effect.flatMap(models, ({ languageModel }) => languageModel.admit(request)), + generate: (request) => Effect.flatMap(models, ({ languageModel }) => languageModel.generate(request)), + }; +} + +function catalogOf(models: Effect.Effect): ModelCatalog { + return { + list: (provider) => Effect.flatMap(models, ({ catalog }) => catalog.list(provider)), + contextWindowOf: (model) => Effect.flatMap(models, ({ catalog }) => catalog.contextWindowOf(model)), + }; +} + export async function reasoningServedBy( runtime: AppRuntime, - settings: Pick, - modelAccessOf: ModelAccessOf, + settings: ReasoningSettings, + models: Effect.Effect, tools: ToolAccess, ): Promise { - const { languageModel, status, offered, catalog } = await Effect.runPromise( - modelAccessOf({ ...settings.models, providerSetup: settings.providerSetup }, settings.proxy), - ); + const status = providerStatus(settings.models, installedPackages()); await runtime.run(logModelProviders(status)); const { mostInputTokens } = settings.reasoning; await runtime.run(logReasoning(mostInputTokens)); - const reading = { mostInputTokens, contextWindowOf: catalog.contextWindowOf }; + const catalog = catalogOf(models); return { capability: makeReasoningFunctionAdapter({ - languageModel, - offered, + languageModel: languageModelOf(models), + offered: offeredModels(status.configured, settings.models.aliases, settings.models.allowed), tools, - reading, + reading: { mostInputTokens, contextWindowOf: catalog.contextWindowOf }, providerSetup: settings.providerSetup, }), listModels: defineListModels(catalog, settings.providerSetup), diff --git a/packages/server/src/composition/served-tool-access.test.ts b/packages/server/src/composition/served-tool-access.test.ts new file mode 100644 index 000000000..ae0717c06 --- /dev/null +++ b/packages/server/src/composition/served-tool-access.test.ts @@ -0,0 +1,60 @@ +import { ToolServerDocumentRefused, type ToolAccess } from '@on.auto/tool-servers'; +import { noToolServers } from '@on.auto/tool-servers/testing'; +import { Cause, Effect } from 'effect'; +import { describe, expect, it } from 'vitest'; + +import { toolAccessOf } from './served-tool-access.ts'; + +const noServers = { toolServers: { servers: [], referencedVariables: [] } }; + +const scope = { org: 'acme', playbook: 'alpha' }; + +function countedComposing(): { readonly asked: () => number; readonly composing: Effect.Effect } { + const counts = { asked: 0 }; + return { + asked: () => counts.asked, + composing: Effect.sync(() => { + counts.asked += 1; + return noToolServers; + }), + }; +} + +describe('the tool access a server makes at start', () => { + it('asks for the composed access only when one of its ways is used, and closes nothing of it', async () => { + const { asked, composing } = countedComposing(); + const tools = toolAccessOf(composing, noServers, { takesConnections: false }); + const beforeUse = asked(); + + const listed = await Effect.runPromise(tools.listServers(scope)); + await tools.close(); + + expect({ beforeUse, listed, afterUse: asked() }).toEqual({ beforeUse: 0, listed: [], afterUse: 1 }); + }); + + it('says it is configured as the composed access will, from the servers it names and the connections it takes', () => { + const { composing } = countedComposing(); + + expect( + [ + toolAccessOf(composing, noServers, { takesConnections: false }), + toolAccessOf(composing, noServers, { takesConnections: true }), + ].map(({ configured, takesConnections }) => ({ configured, takesConnections })), + ).toEqual([ + { configured: false, takesConnections: false }, + { configured: true, takesConnections: true }, + ]); + }); + + it('fails a way used with the refusal of a composition that fails, as a defect', async () => { + const refusal = new ToolServerDocumentRefused({ + message: 'The documents of the tool servers could not be read.', + problems: [], + }); + const tools = toolAccessOf(Effect.fail(refusal), noServers, { takesConnections: false }); + + const cause = await Effect.runPromise(Effect.flip(Effect.sandbox(tools.listServers(scope)))); + + expect([Cause.hasDies(cause), Cause.squash(cause)]).toEqual([true, refusal]); + }); +}); diff --git a/packages/server/src/composition/served-tool-access.ts b/packages/server/src/composition/served-tool-access.ts index 240560820..e09a9d94e 100644 --- a/packages/server/src/composition/served-tool-access.ts +++ b/packages/server/src/composition/served-tool-access.ts @@ -10,8 +10,9 @@ import { type Timing, type ToolAccess, type ToolAccessOptions, + type ToolServerDocumentRefused, } from '@on.auto/tool-servers'; -import { Effect } from 'effect'; +import { Context, Effect, Layer } from 'effect'; import { logLeftOutOperation, logServerMessage } from '../logging/tool-server-logging.ts'; import type { Settings } from '../settings/settings.ts'; @@ -32,29 +33,58 @@ export interface ServedConnections { readonly connections: ConnectionSource; } -export async function toolAccessOf( +type ToolSettings = Pick; + +export class ComposedTools extends Context.Service()('@on.auto/server/ComposedTools') {} + +export function toolsComposedBy( runtime: AppRuntime, - { toolServers, registeredClients }: Pick, + { toolServers, registeredClients }: ToolSettings, { toolTiming, now, consent, connections, telemetry }: ToolAccessWiring & ServedConnections, -): Promise { +): Layer.Layer { const traceMeta = telemetry?.traceMeta; const reportLeftOut: Describing['reportLeftOut'] = (server, leftOut) => { void runtime.run(logLeftOutOperation(server, leftOut)); }; - const servers = await Effect.runPromise(describedToolServers(toolServers, { fetch: outboundFetch, reportLeftOut })); - return makeToolAccess(servers, { - clientInfo: release, - consent, - fetch: outboundFetch, - connections, - reportLeftOut, - ...(toolTiming === undefined ? {} : { timing: toolTiming }), - ...(now === undefined ? {} : { now }), - ...(traceMeta === undefined ? {} : { traceMeta }), - content: runtimeLedger(runtime).content, - runtimeSecrets: registeredClientSecrets(registeredClients), - reportServerMessage: (report) => { - void runtime.run(logServerMessage(report)); - }, - }); + const composing = Effect.map(describedToolServers(toolServers, { fetch: outboundFetch, reportLeftOut }), (servers) => + makeToolAccess(servers, { + clientInfo: release, + consent, + fetch: outboundFetch, + connections, + reportLeftOut, + ...(toolTiming === undefined ? {} : { timing: toolTiming }), + ...(now === undefined ? {} : { now }), + ...(traceMeta === undefined ? {} : { traceMeta }), + content: runtimeLedger(runtime).content, + runtimeSecrets: registeredClientSecrets(registeredClients), + reportServerMessage: (report) => { + void runtime.run(logServerMessage(report)); + }, + }), + ); + return Layer.effect( + ComposedTools, + Effect.acquireRelease(composing, (tools) => Effect.promise(() => tools.close())), + ); +} + +export function toolAccessOf( + composed: Effect.Effect, + { toolServers }: Pick, + { takesConnections }: Pick, +): ToolAccess { + const tools = Effect.orDie(composed); + return { + configured: toolServers.servers.length > 0 || takesConnections, + takesConnections, + callOnce: (call, runCall) => Effect.flatMap(tools, (access) => access.callOnce(call, runCall)), + preparedCall: (call) => Effect.flatMap(tools, (access) => access.preparedCall(call)), + open: (context, references) => Effect.flatMap(tools, (access) => access.open(context, references)), + named: (address, references) => Effect.flatMap(tools, (access) => access.named(address, references)), + listServers: (scope, named) => Effect.flatMap(tools, (access) => access.listServers(scope, named)), + listServersInOrg: (scope, named) => Effect.flatMap(tools, (access) => access.listServersInOrg(scope, named)), + lookUp: (address, reference) => Effect.flatMap(tools, (access) => access.lookUp(address, reference)), + close: () => Promise.resolve(), + }; } diff --git a/packages/server/src/composition/served-tools.ts b/packages/server/src/composition/served-tools.ts index 4882998e3..0690a6b16 100644 --- a/packages/server/src/composition/served-tools.ts +++ b/packages/server/src/composition/served-tools.ts @@ -1,49 +1,34 @@ import type { AppRuntime } from '@on.auto/api'; import type { DispatcherServices } from '@on.auto/operations'; -import type { ToolAccessOptions } from '@on.auto/tool-servers'; +import type { ConnectionSource } from '@on.auto/tool-servers'; -import type { Served } from '../server-options/server-options.ts'; +import type { CapabilityRegistry } from '../capability-registry/capability-registry.ts'; import type { Settings } from '../settings/settings.ts'; import { interactionServedBy, type ServedInteraction } from './served-interaction.ts'; -import { reasoningServedBy, type ModelAccessOf, type ServedReasoning } from './served-reasoning.ts'; -import { toolAccessOf, type ServedConnections, type ToolAccessWiring } from './served-tool-access.ts'; +import { reasoningServedBy, type ServedReasoning } from './served-reasoning.ts'; +import { toolAccessOf } from './served-tool-access.ts'; import { toolOperationsOf, type ProvidersOffered, type ServedToolOperations } from './served-tool-operations.ts'; -export interface ToolWiring extends ToolAccessWiring { - readonly modelAccessOf: ModelAccessOf; - readonly telemetry?: { - readonly models: Parameters[2]; - readonly traceMeta: ToolAccessOptions['traceMeta']; - }; +export interface ToolUsersWiring extends ProvidersOffered { + readonly registry: CapabilityRegistry; + readonly connections: Pick; } -export type ToolUsersWiring = ToolWiring & ServedConnections & ProvidersOffered; - export interface ServedToolUsers { readonly reasoning: ServedReasoning; readonly interaction: ServedInteraction; readonly toolOperations: ServedToolOperations; - readonly withToolsClosed: (served: Served) => Served; } export async function toolUsersServedBy( runtime: AppRuntime, settings: Settings, - { modelAccessOf, ...wiring }: ToolUsersWiring, + { registry, connections, offering }: ToolUsersWiring, ): Promise { - const tools = await toolAccessOf(runtime, settings, wiring); - const models = wiring.telemetry?.models; - const access: ModelAccessOf = (modelSettings, proxy) => modelAccessOf(modelSettings, proxy, models); + const tools = toolAccessOf(registry.tools, settings, connections); return { - reasoning: await reasoningServedBy(runtime, settings, access, tools), + reasoning: await reasoningServedBy(runtime, settings, registry.models, tools), interaction: interactionServedBy(runtime, settings, tools), - toolOperations: toolOperationsOf(tools, wiring), - withToolsClosed: ({ routes, stopWork }) => ({ - routes, - stopWork: async () => { - await stopWork(); - await tools.close(); - }, - }), + toolOperations: toolOperationsOf(tools, { offering }), }; } diff --git a/packages/server/src/config-file/config-file-errors.test.ts b/packages/server/src/config-file/config-file-errors.test.ts index f363a866a..4d7445099 100644 --- a/packages/server/src/config-file/config-file-errors.test.ts +++ b/packages/server/src/config-file/config-file-errors.test.ts @@ -92,16 +92,3 @@ allowed_models: [] ); }); }); - -describe('an openapi server whose document cannot be read', () => { - it('stops the start, naming its entry', async () => { - const lines = (await startupLine('tool_servers:\n billing:\n document: billing.json\n org: acme\n')).split( - '\n', - ); - - expect(lines.at(-2)).toBe( - 'Auto Runtime could not start: tool_server_document_refused: The documents of the tool servers could not be read. ' + - 'TOOL_SERVERS: /billing/document: The document does not exist', - ); - }); -}); diff --git a/packages/server/src/prediction/model-cache-folder.test.ts b/packages/server/src/prediction/model-cache-folder.test.ts index 259920af1..40dbe83ea 100644 --- a/packages/server/src/prediction/model-cache-folder.test.ts +++ b/packages/server/src/prediction/model-cache-folder.test.ts @@ -1,4 +1,5 @@ -import { chmodSync, mkdirSync, statSync } from 'node:fs'; +import { chmodSync, existsSync, mkdirSync, statSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { describe, expect, it, onTestFinished } from 'vitest'; @@ -49,4 +50,19 @@ describe('the folder of the model files', () => { expect([made.isDirectory(), made.mode & 0o777]).toEqual([true, 0o700]); }); + + it('is left unread at start, so the files in it are checked against their digests when a prediction first needs them', async () => { + const ledger = aLedger(); + const entry = join(ledger.cacheDirectory, 'models', 'a'.repeat(64)); + mkdirSync(entry, { recursive: true, mode: 0o700 }); + chmodSync(ledger.cacheDirectory, 0o700); + writeFileSync(join(entry, 'model.onnx'), 'not the bytes of its digest'); + + const child = serverOn(ledger); + await child.port; + child.signal('SIGTERM'); + await child.exited; + + expect(existsSync(join(entry, 'model.onnx'))).toBe(true); + }); }); diff --git a/packages/server/src/reasoning/models-on-first-use.test.ts b/packages/server/src/reasoning/models-on-first-use.test.ts new file mode 100644 index 000000000..06bc5440c --- /dev/null +++ b/packages/server/src/reasoning/models-on-first-use.test.ts @@ -0,0 +1,57 @@ +import { makeModelAccess } from '@on.auto/reasoning'; +import { jsonResponse } from '@on.auto/reasoning/testing'; +import { Effect } from 'effect'; +import { describe, expect, it, onTestFinished } from 'vitest'; + +import { compositionRootWith } from '../composition/composition-root.ts'; +import { startServer } from '../lifecycle/lifecycle.ts'; +import type { RunningServer } from '../server-options/server-options.ts'; +import { temporaryLedger } from '../testing/records/temporary-ledger.ts'; +import { request } from '../testing/servers/http-client.ts'; +import { alpha, loopbackSettingsOver } from '../testing/servers/reasoning-server.ts'; + +const greeting = ['---', 'model: anthropic/claude-sonnet-4-5', '---', 'Say hello.'].join('\n'); + +function listingNothing(): Promise { + return Promise.resolve(jsonResponse({ data: [], has_more: false })); +} + +function servingCounted(composing: () => void): Promise { + const ledger = temporaryLedger(); + onTestFinished(ledger.remove); + return startServer( + { ...loopbackSettingsOver(ledger), ANTHROPIC_API_KEY: 'sk-ant-test-key' }, + compositionRootWith((settings) => + Effect.suspend(() => { + composing(); + return makeModelAccess(settings, { fetch: listingNothing }); + }), + ), + ); +} + +describe('the models reasoning functions call', () => { + it('are composed when a request first needs them, not at the start nor for a definition saved, and kept', async () => { + const composings = { count: 0 }; + const server = await servingCounted(() => { + composings.count += 1; + }); + const atStart = composings.count; + await request(server.port, 'POST', '/v1/orgs/acme/playbooks', { body: { playbook: 'alpha', name: 'Alpha' } }); + await request(server.port, 'POST', `${alpha}/definitions/reasoning`, { body: { name: 'hello', source: greeting } }); + const afterTheSave = composings.count; + + const listed = [ + await request(server.port, 'GET', '/v1/orgs/acme/models'), + await request(server.port, 'GET', '/v1/orgs/acme/models'), + ]; + await server.stop(); + + expect({ atStart, afterTheSave, listed: listed.map(({ status }) => status), composed: composings.count }).toEqual({ + atStart: 0, + afterTheSave: 0, + listed: [200, 200], + composed: 1, + }); + }); +}); diff --git a/packages/server/src/tool-servers/documents-on-first-use.test.ts b/packages/server/src/tool-servers/documents-on-first-use.test.ts new file mode 100644 index 000000000..aab706325 --- /dev/null +++ b/packages/server/src/tool-servers/documents-on-first-use.test.ts @@ -0,0 +1,55 @@ +import { rmSync, writeFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; + +import { describe, expect, it } from 'vitest'; + +import { configuredServer } from '../testing/processes/configured-server.ts'; +import { request } from '../testing/servers/http-client.ts'; +import { alpha } from '../testing/servers/reasoning-server.ts'; + +const billingDocument = JSON.stringify({ + openapi: '3.1.0', + info: { title: 'Billing', version: '1' }, + servers: [{ url: 'https://billing.example.com' }], + paths: { + '/invoices': { + get: { operationId: 'list_invoices', summary: 'Lists the invoices', responses: { '200': { description: 'ok' } } }, + }, + }, +}); + +const refused = + 'tool_server_document_refused: The documents of the tool servers could not be read. TOOL_SERVERS: /billing/document: The document does not exist'; + +const search = `${alpha}/tool-servers/search?terms=invoices`; + +describe('the document of an openapi server', () => { + it('is read when a request first needs the tool servers; a document that cannot be read fails that request, naming its entry, and is read again on the next', async () => { + const { child, configFile } = configuredServer( + 'tool_servers:\n billing:\n document: billing.json\n org: acme\n', + {}, + ); + const document = join(dirname(configFile), 'billing.json'); + const port = await child.port; + await request(port, 'POST', '/v1/orgs/acme/playbooks', { body: { playbook: 'alpha', name: 'Alpha' } }); + + const missing = await request(port, 'GET', search); + writeFileSync(document, billingDocument); + const read = await request(port, 'GET', search); + rmSync(document); + const kept = await request(port, 'GET', search); + child.signal('SIGTERM'); + await child.exited; + const incident = /urn:uuid:(?[\da-f-]+)/u.exec(missing.text)?.groups?.['id']; + const logged = child + .output() + .stderr.split('\n') + .filter((line) => line.includes(refused)); + + expect({ + statuses: [missing.status, read.status, kept.status], + logged: logged.map((line) => line.includes(`"incident":"${String(incident)}"`)), + found: [read.text.includes('"tool":"list_invoices"'), kept.text === read.text], + }).toEqual({ statuses: [500, 200, 200], logged: [true], found: [true, true] }); + }); +}); diff --git a/packages/tool-servers/README.md b/packages/tool-servers/README.md index 6eb4c8423..18c966100 100644 --- a/packages/tool-servers/README.md +++ b/packages/tool-servers/README.md @@ -4,7 +4,7 @@ How a playbook reaches the outside world: the tool servers of its org, MCP serve ## Entry points -- `@on.auto/tool-servers`: the settings, with `ServerEntrySchema`, the schema of one entry, `readEntry`, the reading of one entry of the file, with which `@on.auto/connections` reads a connection's entry, its credential placed, into the connection's `ServerSettings`, and whose context's `placedValues`, the values the runtime places by the name of their placeholder, stand in the entry's header and query values once the entry as written is judged, and whose context's `providers`, a `ProviderResolver`, resolves the provider an entry of the file names (see [An entry that names a provider](#an-entry-that-names-a-provider)), `noProviders` where it is left out; `givenByTheProvider(field)`, the words of a field of a server written beside a provider, and `secretOrKeyExpected`, the words of client credentials given neither a client secret nor a private key with its algorithm; `makeConsentClient`, the runtime acting as a client at the authorization server of a provider's way `authorization`, and the types of its contract, `ConsentClient`, `ClientRegistrations` and `ConnectionTokens`, the store of the tokens of a connection connected by consent, with `KeptRow`, a row as a store reads it, and `HeldTokens`, that row decoded, among them, and `keptTokensIn`, which decodes the value of a row, which `@on.auto/connections` calls and implements, and `ConsentCredential`, the third kind of the credential of `AuthSettings`, the tokens of a consent, which it composes, with `ClientSourceSchema`, `clientSources` and `isSameIssuer`, the client sources an entry may list, in the runtime's order, and the comparison of two issuers, `clientMetadataDocumentOf` and its `ClientMetadataDocument`, the document the runtime names its client by where `PUBLIC_URL` is https, which the server gives `@on.auto/api` to answer, `ClientIdSchema`, the rule of a client id the record and a registration's answer are held to, and `isScope`, the rule of a scope a provider's way and a grant are held to (see [The runtime as a client](#the-runtime-as-a-client)); `serverNameProblems`, the check of a server's name, with which it checks a connection's name, and `servedFieldsOf`, the playbooks, tools, `record_content` and `request_id` an entry is served by, as it writes them or by default where it leaves them out, with which it builds the serving of every connection it lists, and `isPinnableUrl` and `headerNamePattern`; `redactedValuesOf`, the values of a record each redacted under its key, which the reasoning capability takes for the headers and the query parameters of a gateway; `@on.auto/providers` takes the server part of a provider's entry from the fields of `ServerEntrySchema`, as a connection's entry does, and checks a provider's name with `serverNameProblems` and its addresses with `isPinnableUrl`; `namesEveryTool`, whether a tool reference names every tool of its server with `*`, as a reasoning function's `tools` may; the contract of a source of connections, `ConnectionSource` and its types, which `@on.auto/connections` implements, and `noConnections`; `describedToolServers`, `makeToolAccess`, the five operations, `defineSearch` and its org twin `defineSearchInOrg`, with `ProviderOffering` and `OfferedProvider`, the contract of the providers offered to a caller, which `@on.auto/connections` implements, `defineIntrospect`, `defineValidate`, `defineDryRun` and `defineExecute`, `handCallPresenter`, which shows a call made by hand, and the facts of a call that `@on.auto/definitions` builds the run's events from. The access loads the MCP client, its OAuth providers and `node:child_process` (`src/access/linked-access.ts`) through a dynamic import when a run first opens its tools, a search or a read first asks a server or `execute` first calls a tool, so a server that never does any of them never loads them: the server's start loaded 55 files and 0.94 MiB more with them (measured 2026-10-06 with a module load hook, 1,241 files against main's 1,186). Without them it loads 84 files and 236 KiB of this package, the five operations, the reading of the settings of the three types and of an entry that names a provider, `describedToolServers` and the naming of the servers of an org and of its connections among them (measured 2026-10-20 with a module load hook over both entries, the sizes of the files as they are on disk, against 82 files and 219 KiB before the switch of Auto's consent service and the listing of its clients, `src/consent-through-auto/auto-clients.ts`, and the contract they call, `src/consent/consent-service.ts`, came to load with the consent client, against 82 files and 217 KiB before the metadata of the runtime's client, which its document and a dynamic registration share, was written in `src/consent/client-sources.ts`, against 81 files and 211 KiB before the tokens of a consent were served, which added `src/consent-tokens/kept-tokens.ts`, the value a token source of a consent reads, and against 77 files and 192 KiB before an entry could name a provider); the reading resolves a provider through the resolver its context gives, so it loads no code of a provider's; the reading of an OpenAPI document, its decoding and the tools derived from it (`src/openapi`, `described-servers.ts` and the types aside) load through a dynamic import too, when an entry of type `openapi` is described at start. The server's start loads neither the TypeScript compiler, which the check worker of `@on.auto/definitions` loads with the first check, nor the driver of a store its settings do not name: measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, by the same hook on the thread that serves requests, from the start of the process to its ready line, at the median of 11 starts interleaved with main's, it loaded on SQLite 2,504 files and 21.0 MiB in 802 ms, against main's 2,573 files and 30.4 MiB in 1,143 ms, at a load average of 20 to 13, and on PostgreSQL 2,551 files and 19.5 MiB in 810 ms, against 2,573 files and 30.4 MiB in 1,134 ms, at 11 to 7. The compiler was 12 files, 8.8 MiB and 318 to 328 ms of it, the warming of the check worker among them, which the first save of a program now pays: 245 ms against 17 ms at the median of 9 servers interleaved with main's, and 6.7 ms against 5.9 ms for the second, at a load average of about 5. The driver of the other store, `pg` with the PostgreSQL entry on SQLite and `sqlite3` with its native addon on PostgreSQL, was 57 files and 0.6 MiB, and 10 files and 2.1 MiB, with a time, 13 and 6 ms, within the spread of the starts. Nor does it load the SDK of the workflow specification and the validator of its schema, which the coordination capability imports with the first document it reads, the S3 client, which the server imports when its settings name a bucket, `google-auth-library` with the AI SDK's Vertex providers, which model access imports when Vertex is configured, or the credential chain of AWS, which the first client that asks for it imports: measured on 2026-10-23 on the same machine, with Node 26.10.0, by the same hook, at the median of 11 starts on SQLite interleaved with main's, at a load average of 5 to 4, it loaded 2,307 files and 18.5 MiB in 654 ms, against main's 2,636 files and 21.4 MiB in 821 ms. The SDK and the validator were 222 files, 1.4 MiB and 130 ms of it, the SDK's 171 validators and Ajv among them, which the first save of a workflow now pays, 175 ms against 34 ms at the median of 9 servers; the S3 client was 26 files, 0.7 MiB and 13 ms, within the spread of the starts, and the Google library and the credential chain 81 files, 0.9 MiB and 24 ms. A CPU profile of 7 starts of each, sampled every 100 µs, gave the SDK with Ajv 115 ms of main's start, the validator 14 ms, the S3 client 11 ms, the Google library 10 ms and the credential chain 6 ms. +- `@on.auto/tool-servers`: the settings, with `ServerEntrySchema`, the schema of one entry, `readEntry`, the reading of one entry of the file, with which `@on.auto/connections` reads a connection's entry, its credential placed, into the connection's `ServerSettings`, and whose context's `placedValues`, the values the runtime places by the name of their placeholder, stand in the entry's header and query values once the entry as written is judged, and whose context's `providers`, a `ProviderResolver`, resolves the provider an entry of the file names (see [An entry that names a provider](#an-entry-that-names-a-provider)), `noProviders` where it is left out; `givenByTheProvider(field)`, the words of a field of a server written beside a provider, and `secretOrKeyExpected`, the words of client credentials given neither a client secret nor a private key with its algorithm; `makeConsentClient`, the runtime acting as a client at the authorization server of a provider's way `authorization`, and the types of its contract, `ConsentClient`, `ClientRegistrations` and `ConnectionTokens`, the store of the tokens of a connection connected by consent, with `KeptRow`, a row as a store reads it, and `HeldTokens`, that row decoded, among them, and `keptTokensIn`, which decodes the value of a row, which `@on.auto/connections` calls and implements, and `ConsentCredential`, the third kind of the credential of `AuthSettings`, the tokens of a consent, which it composes, with `ClientSourceSchema`, `clientSources` and `isSameIssuer`, the client sources an entry may list, in the runtime's order, and the comparison of two issuers, `clientMetadataDocumentOf` and its `ClientMetadataDocument`, the document the runtime names its client by where `PUBLIC_URL` is https, which the server gives `@on.auto/api` to answer, `ClientIdSchema`, the rule of a client id the record and a registration's answer are held to, and `isScope`, the rule of a scope a provider's way and a grant are held to (see [The runtime as a client](#the-runtime-as-a-client)); `serverNameProblems`, the check of a server's name, with which it checks a connection's name, and `servedFieldsOf`, the playbooks, tools, `record_content` and `request_id` an entry is served by, as it writes them or by default where it leaves them out, with which it builds the serving of every connection it lists, and `isPinnableUrl` and `headerNamePattern`; `redactedValuesOf`, the values of a record each redacted under its key, which the reasoning capability takes for the headers and the query parameters of a gateway; `@on.auto/providers` takes the server part of a provider's entry from the fields of `ServerEntrySchema`, as a connection's entry does, and checks a provider's name with `serverNameProblems` and its addresses with `isPinnableUrl`; `namesEveryTool`, whether a tool reference names every tool of its server with `*`, as a reasoning function's `tools` may; the contract of a source of connections, `ConnectionSource` and its types, which `@on.auto/connections` implements, and `noConnections`; `describedToolServers`, `makeToolAccess`, the five operations, `defineSearch` and its org twin `defineSearchInOrg`, with `ProviderOffering` and `OfferedProvider`, the contract of the providers offered to a caller, which `@on.auto/connections` implements, `defineIntrospect`, `defineValidate`, `defineDryRun` and `defineExecute`, `handCallPresenter`, which shows a call made by hand, and the facts of a call that `@on.auto/definitions` builds the run's events from. The access loads the MCP client, its OAuth providers and `node:child_process` (`src/access/linked-access.ts`) through a dynamic import when a run first opens its tools, a search or a read first asks a server or `execute` first calls a tool, so a server that never does any of them never loads them: the server's start loaded 55 files and 0.94 MiB more with them (measured 2026-10-06 with a module load hook, 1,241 files against main's 1,186). Without them it loads 84 files and 236 KiB of this package, the five operations, the reading of the settings of the three types and of an entry that names a provider, `describedToolServers` and the naming of the servers of an org and of its connections among them (measured 2026-10-20 with a module load hook over both entries, the sizes of the files as they are on disk, against 82 files and 219 KiB before the switch of Auto's consent service and the listing of its clients, `src/consent-through-auto/auto-clients.ts`, and the contract they call, `src/consent/consent-service.ts`, came to load with the consent client, against 82 files and 217 KiB before the metadata of the runtime's client, which its document and a dynamic registration share, was written in `src/consent/client-sources.ts`, against 81 files and 211 KiB before the tokens of a consent were served, which added `src/consent-tokens/kept-tokens.ts`, the value a token source of a consent reads, and against 77 files and 192 KiB before an entry could name a provider); the reading resolves a provider through the resolver its context gives, so it loads no code of a provider's; the reading of an OpenAPI document, its decoding and the tools derived from it (`src/openapi`, `described-servers.ts` and the types aside) load through a dynamic import too, when an entry of type `openapi` is described at start. The server's start loads neither the TypeScript compiler, which the check worker of `@on.auto/definitions` loads with the first check, nor the driver of a store its settings do not name: measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, by the same hook on the thread that serves requests, from the start of the process to its ready line, at the median of 11 starts interleaved with main's, it loaded on SQLite 2,504 files and 21.0 MiB in 802 ms, against main's 2,573 files and 30.4 MiB in 1,143 ms, at a load average of 20 to 13, and on PostgreSQL 2,551 files and 19.5 MiB in 810 ms, against 2,573 files and 30.4 MiB in 1,134 ms, at 11 to 7. The compiler was 12 files, 8.8 MiB and 318 to 328 ms of it, the warming of the check worker among them, which the first save of a program now pays: 245 ms against 17 ms at the median of 9 servers interleaved with main's, and 6.7 ms against 5.9 ms for the second, at a load average of about 5. The driver of the other store, `pg` with the PostgreSQL entry on SQLite and `sqlite3` with its native addon on PostgreSQL, was 57 files and 0.6 MiB, and 10 files and 2.1 MiB, with a time, 13 and 6 ms, within the spread of the starts. Nor does it load the SDK of the workflow specification and the validator of its schema, which the coordination capability imports with the first document it reads, the S3 client, which the server imports when its settings name a bucket, `google-auth-library` with the AI SDK's Vertex providers, which model access imports when Vertex is configured, or the credential chain of AWS, which the first client that asks for it imports: measured on 2026-10-23 on the same machine, with Node 26.10.0, by the same hook, at the median of 11 starts on SQLite interleaved with main's, at a load average of 5 to 4, it loaded 2,307 files and 18.5 MiB in 654 ms, against main's 2,636 files and 21.4 MiB in 821 ms. The SDK and the validator were 222 files, 1.4 MiB and 130 ms of it, the SDK's 171 validators and Ajv among them, which the first save of a workflow now pays, 175 ms against 34 ms at the median of 9 servers; the S3 client was 26 files, 0.7 MiB and 13 ms, within the spread of the starts, and the Google library and the credential chain 81 files, 0.9 MiB and 24 ms. A CPU profile of 7 starts of each, sampled every 100 µs, gave the SDK with Ajv 115 ms of main's start, the validator 14 ms, the S3 client 11 ms, the Google library 10 ms and the credential chain 6 ms. Nor does the start compose what a server builds only for a request of a kind, which the server's capability registry composes on first use (see [the server](../server/README.md#what-is-composed-at-start-and-what-on-first-use)): the model access of reasoning functions, with a client for each provider and the credential libraries of Vertex, Bedrock and Azure without a key, and the access to the tool servers, with the document of each `openapi` entry read and described; and the prediction caches check the files they hold when a prediction first reads them. Measured on 2026-10-23 on the same machine, with Node 26.10.0, by the same hook, at the median of 11 starts on SQLite interleaved with main's: with nothing configured, 2,312 files and 18.5 MiB in 727 ms, against main's 2,310 files and 18.5 MiB in 727 ms, the two files more being the registry's, at a load average of 16 to 13; with OpenAI, Anthropic, Vertex, Bedrock, Azure without a key and two `openapi` entries over two of the documents the providers ship, of 1.4 MiB and 484 KiB, served on loopback, 2,312 files and 18.5 MiB in 732 ms, against 2,757 files and 20.8 MiB in 833 ms, at 13 to 12; and with 320 MiB of model files and rows in the folder of the caches, 723 ms against 871 ms, at 12 to 10. The first request of each kind pays its composition instead, at the median of 9 servers of that second setting interleaved with main's, at a load average of 10 to 9: the first `list_models` took 65.6 ms against 1.0 ms and the first `search` 51.0 ms against 17.8 ms, the second 2.0 ms against 0.8 ms and 1.6 ms against 2.4 ms, converging by the eighth, and the first listing of the playbooks, which composes nothing, 1.8 ms against 2.1 ms. Readying both caches over those 320 MiB, which the first prediction to read them pays, took 146 ms at the median of 9, and 0.04 ms once ready; one ask of the registry for what it composed takes about 10 µs. - `@on.auto/tool-servers/policy`: the pure helpers a caller needs without connecting to anything (`toolReferenceOf`, `toolReferenceShape`, `writtenOf`, `namesEveryTool`, `runBoundMs` and `runIdKey`). It loads no transport code, so the reasoning function adapter, which parses a function's `tools` and bounds its run, does not load the MCP client wherever it is bundled; it takes `ToolAccess` as a type only. - `@on.auto/tool-servers/testing`: the fake server and the helpers of the tests (see Testing). @@ -42,7 +42,7 @@ A secret is a value resolved from a `${NAME}` reference to the environment when An entry may name a provider in place of its server: `provider: tracker`, with `placeholders` where the provider has them and `credential`, and `org`, `playbooks`, `allowed` and `record_content`, which are the entry's as for any other (`src/settings/provider-entries.ts`). This package knows no provider: the reading's context gives it a `ProviderResolver`, which `makeProviderResolver` of `@on.auto/connections` implements over a list of providers and the server gives over the one list it reads at start, and `readEntry` resolves the provider at its top, before the references are substituted; without one, `noProviders` refuses every provider at `/provider`. Before it asks, it refuses, each at its pointer, a field of the server written beside `provider`, in the words of `givenByTheProvider`, `Expected no url: the provider gives the server`; `placeholders` or `credential` without `provider`; a value of `credential` that holds anything but a reference alone where a reference goes, the credential as a string, `value`, `client_secret` and `private_key`, as `Bearer ${TRACKER_TOKEN}` or a value written out; a `$` in `kind`, `client_id` or `algorithm`; and a `client_id` in the shape of a credential, in the words of `clientIdWrittenOut` of `@on.auto/config`, which the settings file's own check gives such a client id. It asks with `ProviderNamed`, the provider's name, the placeholders and the credential, each of its references read as the name of its variable, and the resolver answers `ProviderResolved`: the server part and the credential placed as an entry of the file writes them, its references kept, the values the reading places by the name of their placeholder, the folder a document of the provider is read from, and `writtenAt`, which names the key the operator wrote for a place of what it resolved, or the problems of the resolution, each at the key the operator wrote, such as `/placeholders/site` or `/credential/kind`. Every check after it runs unchanged over the resolved entry, with the entry's own `org`, `playbooks`, `allowed` and `record_content`: the reference checks, the checks of the credentials, the derivation of the entry's secrets and those of its type; a problem one of them finds in what the provider resolved is placed by `writtenAt`, and a credential written as a string is placed at `credential` itself. The server so read carries `provider`, so `search` lists it with its provider and leaves the provider out of the providers it offers in the orgs the entry serves. -Nothing is learned from a server when the settings are read: they are read without the network and without spawning anything. `describedToolServers(settings, { fetch?, readMs?, reportLeftOut })` then reads, at start, the document of each `openapi` entry and derives its tools (see [An HTTP API as a tool server](#an-http-api-as-a-tool-server)), answering `ToolServers`, the settings with each API described, which `makeToolAccess` takes, or failing with `ToolServerDocumentRefused`, whose problems name each entry and the place in its document; it leaves the other types as they are and, with none of type `openapi`, runs without suspending, so a test may run it synchronously. The server calls it as it composes its services and stops when it fails, and logs at WARN each operation `reportLeftOut` reports. +Nothing is learned from a server when the settings are read: they are read without the network and without spawning anything. `describedToolServers(settings, { fetch?, readMs?, reportLeftOut })` then reads the document of each `openapi` entry, which the server does the first time a request or a due item needs its tool servers, and derives its tools (see [An HTTP API as a tool server](#an-http-api-as-a-tool-server)), answering `ToolServers`, the settings with each API described, which `makeToolAccess` takes, or failing with `ToolServerDocumentRefused`, whose problems name each entry and the place in its document; it leaves the other types as they are and, with none of type `openapi`, runs without suspending, so a test may run it synchronously. The server calls it as it composes its services and stops when it fails, and logs at WARN each operation `reportLeftOut` reports. ## A run's tools @@ -120,13 +120,13 @@ A call made by hand is the same happening as a call a run makes, so it records t ## The credentials of an HTTP API -The document's `components.securitySchemes` and the `security` of the document and of each operation decide what a call sends (`src/openapi/security-schemes.ts`). An entry gives a credential by the scheme's name under `security`, a map read at start like `headers`, whose every value holds a reference, one written out refused with the advice a credential written out gets, and is a secret whole as well as its reference's value, so a Basic credential built of a user and a referenced password is scrubbed in its base64. `requirementOf` reads an operation's alternatives, its own `security` or else the document's: the first alternative every scheme of which the entry gives is the one a call sends, as `credentials` of its `ApiOperation`, and no other, unless the operation needs a header the entry's `headers` do not name, when the first alternative the entry gives whose `apiKey` schemes place every such header is the one sent; an `apiKey` in a header or the query goes in its header or query parameter, an `http` scheme of `bearer` as `Authorization: Bearer`, of `basic`, `user:password`, as `Authorization: Basic` and its base64, and an `oauth2` or `openIdConnect` scheme is given by the `auth` block alone, whose token a call sends as a bearer only when the alternative it sends names such a scheme. An empty requirement, an empty alternative, or a document that requires nothing sends no credential; an operation the entry gives no alternative of is left out, its reason naming the alternatives. `securityProblemsOf` refuses the entry at start, at `//security/`, `//headers/`, `//query_params/` or `//auth`, for a scheme the document does not declare, an `apiKey` in a cookie, an `http` scheme other than `bearer` and `basic`, `mutualTLS`, an `oauth2` or `openIdConnect` scheme given a value, a `basic` value without a colon, a header or query parameter of the entry that a scheme it gives also places, and an `auth` block whose document declares no `oauth2` or `openIdConnect` scheme. The scopes of an `oauth2` requirement are the operation's; the token source asks for `auth.scope` as for an MCP server, so the operator sets it to cover what the operations require. +The document's `components.securitySchemes` and the `security` of the document and of each operation decide what a call sends (`src/openapi/security-schemes.ts`). An entry gives a credential by the scheme's name under `security`, a map read at start like `headers`, whose every value holds a reference, one written out refused with the advice a credential written out gets, and is a secret whole as well as its reference's value, so a Basic credential built of a user and a referenced password is scrubbed in its base64. `requirementOf` reads an operation's alternatives, its own `security` or else the document's: the first alternative every scheme of which the entry gives is the one a call sends, as `credentials` of its `ApiOperation`, and no other, unless the operation needs a header the entry's `headers` do not name, when the first alternative the entry gives whose `apiKey` schemes place every such header is the one sent; an `apiKey` in a header or the query goes in its header or query parameter, an `http` scheme of `bearer` as `Authorization: Bearer`, of `basic`, `user:password`, as `Authorization: Basic` and its base64, and an `oauth2` or `openIdConnect` scheme is given by the `auth` block alone, whose token a call sends as a bearer only when the alternative it sends names such a scheme. An empty requirement, an empty alternative, or a document that requires nothing sends no credential; an operation the entry gives no alternative of is left out, its reason naming the alternatives. `securityProblemsOf` refuses the entry as its document is read, at `//security/`, `//headers/`, `//query_params/` or `//auth`, for a scheme the document does not declare, an `apiKey` in a cookie, an `http` scheme other than `bearer` and `basic`, `mutualTLS`, an `oauth2` or `openIdConnect` scheme given a value, a `basic` value without a colon, a header or query parameter of the entry that a scheme it gives also places, and an `auth` block whose document declares no `oauth2` or `openIdConnect` scheme. The scopes of an `oauth2` requirement are the operation's; the token source asks for `auth.scope` as for an MCP server, so the operator sets it to cover what the operations require. ## An HTTP API as a tool server An `openapi` entry is a `ServerLink` like the others (`src/clients/server-links.ts`), whose client is the bridge (`src/openapi/api-client.ts`): it answers `listTools` and `call` of the client contract (`src/clients/server-client.ts`), so the rest of `ToolAccess`, the bounds of a run, the scrubbing, the recording, a call's facts and the failures counted against a run are the same code for an MCP server and for an HTTP API, which the shared behaviour suite of `src/access-testing` proves by running the same tests over the fake of each. -At start `describedToolServers` reads the document, from its URL through the server's outbound fetch, without the entry's headers and following no redirect, which is refused naming the entry and its status, since the document chooses where the calls go, within 10 s and 4 MiB, or from its file, or from the text the providers ship, whose bytes they measured as they read it, each within 4 MiB, refuses one that is neither JSON nor YAML or nests deeper than 128 levels, and decodes it with a schema of the parts the bridge reads alone (`src/openapi/api-document.ts`): the version, the servers, the paths with their operations, parameters and request bodies, and the `components` they refer to. Swagger 2.0 and any version but 3.0 and 3.1 are refused at `/swagger` or `/openapi`, and a part of another shape at its place. A path that does not begin with `/` is refused at `/paths/`, since joined to `base_url` it would extend its host or its port, and so is one with a `.` or `..` segment, written plainly, percent-encoded or split by a tab or a line break, which an address drops, or with a `?` or a `#`. Each operation becomes one tool (`src/openapi/api-tools.ts`): +When the server first needs its tool servers, `describedToolServers` reads the document, from its URL through the server's outbound fetch, without the entry's headers and following no redirect, which is refused naming the entry and its status, since the document chooses where the calls go, within 10 s and 4 MiB, or from its file, or from the text the providers ship, whose bytes they measured as they read it, each within 4 MiB, refuses one that is neither JSON nor YAML or nests deeper than 128 levels, and decodes it with a schema of the parts the bridge reads alone (`src/openapi/api-document.ts`): the version, the servers, the paths with their operations, parameters and request bodies, and the `components` they refer to. Swagger 2.0 and any version but 3.0 and 3.1 are refused at `/swagger` or `/openapi`, and a part of another shape at its place. A path that does not begin with `/` is refused at `/paths/`, since joined to `base_url` it would extend its host or its port, and so is one with a `.` or `..` segment, written plainly, percent-encoded or split by a tab or a line break, which an address drops, or with a `?` or a `#`. Each operation becomes one tool (`src/openapi/api-tools.ts`): - its name is the `operationId`, each character outside the grammar of a tool name an underscore and cut at 128 characters, or, without one, the method and the path with the leading `/` dropped, `{x}` as `x` and `/` as `_`, in lower case, read in one pass however many braces the path holds, as the variables of a server's address are, as `get_invoices_id` for `GET /invoices/{id}`; two operations of one name refuse the document, naming both places, as do more than 500 operations; - its description is the operation's `summary` and `description`, cut at 4 KiB as descriptions are; @@ -209,7 +209,7 @@ The MCP client has no logging option. The errors it meets, such as a line from a | Tool calls in one run | 25 | | One answer, as the model reads it | whole, or cut to its room | | One answer of an `mcp` server or of an HTTP API (`httpAnswerBytes`) | 16 MiB, past it `too_large` | -| The document of an HTTP API, read at start | 4 MiB, 500 operations, within 10 s | +| The document of an HTTP API, read on first use | 4 MiB, 500 operations, within 10 s | | One answer of a `stdio` server | 4 MiB of output at once, past it `too_large` | | One call's arguments | 16 KiB | | The same tool with the same arguments | twice |