From 862d6f7ae9d4861640c603bff8eb673b13de69f7 Mon Sep 17 00:00:00 2001 From: Rami Date: Fri, 23 Oct 2026 13:54:54 +0100 Subject: [PATCH 1/3] refactor(coordination): the SDK and the schema validator load with the first document read The capability imported @openworkflowspec/sdk and @hyperjump/json-schema with its entry: dsl-validation.ts took buildGraph and Classes from the SDK, and specification-schema.ts its workflowSchema, the validator, its formats and the setting that asserts them, at module load. Importing the SDK compiles its 171 validators, the heaviest module the server loaded, so every server paid for both at start, though a server that reads no workflow document needs neither. - dsl-libraries.ts is the one place that loads each, a dynamic import kept while the server runs: specificationSdk the SDK, and schemaValidator the validator, its formats registered and asserted once, as it loads. The schema check awaits both when it compiles, with the first document it checks; the graph check awaits the SDK where it builds the graph, which becomes a promise inside dslProblems, which already answered one. The JSON of the SDK's package, which names the versions of the DSL it reads, still loads with the capability. - library-entries.test.ts holds that no module the capability's entry imports reaches either. Measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, by a module load hook on the thread that serves requests, from the start of the process to its ready line, at the median of 11 starts on SQLite interleaved with main's, at a load average of 5 to 4: 691 ms against 821 ms, 2,414 files and 20.1 MiB against 2,636 and 21.4 MiB. A CPU profile of 7 starts of each, sampled every 100 us, at a load average of 3, gave the SDK's module 88 ms of main's start, Ajv, which it imports, 28 ms more, and the validator 14 ms; after, none of them loads. The first save of a workflow without expressions pays the loads: 175 ms against 34 ms at the median of 9 servers of each, and 5.2 ms against 4.9 ms for the second. Co-Authored-By: Claude Fable 5.1 --- capabilities/coordination/README.md | 2 +- .../coordination/src/dsl/dsl-libraries.ts | 49 +++++++++++++++ .../coordination/src/dsl/dsl-validation.ts | 23 ++++--- .../src/dsl/library-entries.test.ts | 24 +++++++ .../src/dsl/specification-schema.ts | 63 +++++++++---------- 5 files changed, 116 insertions(+), 45 deletions(-) create mode 100644 capabilities/coordination/src/dsl/dsl-libraries.ts create mode 100644 capabilities/coordination/src/dsl/library-entries.test.ts diff --git a/capabilities/coordination/README.md b/capabilities/coordination/README.md index 2a30fece1..a690f5435 100644 --- a/capabilities/coordination/README.md +++ b/capabilities/coordination/README.md @@ -31,7 +31,7 @@ A run a caller starts, one whose context names no `trigger`, whether `run_defini `dslProblems` (`src/dsl/dsl-validation.ts`) checks a document against the DSL, before the policy's rejections join its problems, and answers a promise, which the document's reader awaits: the version it names, its names, the schema of the specification, the names of its tasks and the graph of its steps, in that order, each only when those before it found nothing. It reads the document without the runtime's own `output.shape` and without the `kind` and `because` of an error, the runtime's extensions to the DSL's errors. -The schema is `workflowSchema` of `@openworkflowspec/sdk`, the specification's JSON Schema of draft 2020-12, and `@hyperjump/json-schema`, its `draft-2020-12` entry, checks a document against it by interpreting it (`src/dsl/specification-schema.ts`). The SDK's own validation, which `Classes.Workflow.deserialize` runs unless it is given `{ validate: false }`, is Ajv, which compiles each of the SDK's 171 validators into code generated from strings: it costs start-up time wherever it loads and is refused wherever no code may be generated. The interpreting validator reads the same schema as data, generates no code, and compiles the schema into the tree it interprets only when the first document is checked, in about 35 ms; its validation is asynchronous only for that, so the capability's check is a promise. It asserts the formats the schema names, `uri`, `uri-template`, `json-pointer` and `date-time`, registered with the checks of `@hyperjump/json-schema-formats` the library's own formats use, and asserted with `setShouldValidateFormat(true)`, a setting of the library for the whole process, since Ajv asserted them too and draft 2020-12 otherwise reads a format as an annotation. It gives the same answers: over the 499 workflow documents the runtime's tests check, its corpus and its input logs hold, and the 13,340 made from them by changing one value each, it accepts and refuses exactly the documents Ajv does. Its `date-time` is stricter than Ajv's: it refuses a space or a tab between the date and the time, which the grammar of RFC 3339 does not allow, and a leap second on a day that had none. Nothing calls the SDK's validation; the SDK stays for its classes and its graph, and importing it still compiles its validators, so this check alone does not take that cost out of loading the capability. +The schema is `workflowSchema` of `@openworkflowspec/sdk`, the specification's JSON Schema of draft 2020-12, and `@hyperjump/json-schema`, its `draft-2020-12` entry, checks a document against it by interpreting it (`src/dsl/specification-schema.ts`). The SDK's own validation, which `Classes.Workflow.deserialize` runs unless it is given `{ validate: false }`, is Ajv, which compiles each of the SDK's 171 validators into code generated from strings: it costs start-up time wherever it loads and is refused wherever no code may be generated. The interpreting validator reads the same schema as data, generates no code, and compiles the schema into the tree it interprets only when the first document is checked, in about 35 ms; its validation is asynchronous only for that, so the capability's check is a promise. It asserts the formats the schema names, `uri`, `uri-template`, `json-pointer` and `date-time`, registered with the checks of `@hyperjump/json-schema-formats` the library's own formats use, and asserted with `setShouldValidateFormat(true)`, a setting of the library for the whole process, both made once, as the library is loaded (`src/dsl/dsl-libraries.ts`), since Ajv asserted them too and draft 2020-12 otherwise reads a format as an annotation. It gives the same answers: over the 499 workflow documents the runtime's tests check, its corpus and its input logs hold, and the 13,340 made from them by changing one value each, it accepts and refuses exactly the documents Ajv does. Its `date-time` is stricter than Ajv's: it refuses a space or a tab between the date and the time, which the grammar of RFC 3339 does not allow, and a leap second on a day that had none. Nothing calls the SDK's validation; the SDK stays for its schema, its classes and its graph, and importing it still compiles its validators. So the capability imports the SDK, and the validator with the formats it asserts, with the first document it reads, through `specificationSdk` and `schemaValidator` (`src/dsl/dsl-libraries.ts`), the one place each is loaded, and keeps them while the server runs; no module the capability's entry imports reaches either (`src/dsl/library-entries.test.ts`). A server that reads no workflow document never loads them, which took 130 ms, 222 files and 1.4 MiB of a server's start, and the first document read pays for them beside the compiling of the schema: the first save of a workflow without expressions took 175 ms against 34 ms when the two loaded with the capability, and the second 5.2 ms against 4.9 ms (measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, at the median of 11 starts and of 9 servers). The schema is read with two changes that keep what it accepts. A `false` subschema is read as `{ not: {} }`, which refuses the same values, so that every refusal is a keyword of a schema at its place and a property the schema refuses outright is a `not` that allows nothing, wherever the schema refuses it. A task is checked on its own: each reference to `#/$defs/task` is read as any object, and each task `allTaskEntries` of `@on.auto/workflow-engine` walks is checked against `#/$defs/task`, so a document is checked once a task. Checked whole, a list of tasks is checked again for each type of task that may hold it, `do` and `for` among them, and the time doubles with each level the tasks nest, on the one thread every workflow shares: Ajv took 55 ms for 16 levels and about a second for 20; task by task, 30 levels take about a millisecond (`src/dsl/dsl-validation.test.ts`). A task under `use.functions`, which the policy refuses, is not checked further. diff --git a/capabilities/coordination/src/dsl/dsl-libraries.ts b/capabilities/coordination/src/dsl/dsl-libraries.ts new file mode 100644 index 000000000..8a01d4c2e --- /dev/null +++ b/capabilities/coordination/src/dsl/dsl-libraries.ts @@ -0,0 +1,49 @@ +import type { registerSchema, validate } from '@hyperjump/json-schema/draft-2020-12'; +import type { buildGraph, Classes } from '@openworkflowspec/sdk'; + +export interface SpecificationSdk { + readonly workflowSchema: unknown; + readonly buildGraph: typeof buildGraph; + readonly Classes: typeof Classes; +} + +export interface SchemaValidator { + readonly registerSchema: typeof registerSchema; + readonly validate: typeof validate; +} + +let loadingSdk: Promise | undefined; + +let loadingValidator: Promise | undefined; + +export function specificationSdk(): Promise { + loadingSdk ??= import('@openworkflowspec/sdk'); + return loadingSdk; +} + +export function schemaValidator(): Promise { + loadingValidator ??= validatorAssertingFormats(); + return loadingValidator; +} + +async function validatorAssertingFormats(): Promise { + const [validator, { addFormat }, formats] = await Promise.all([ + import('@hyperjump/json-schema/draft-2020-12'), + import('@hyperjump/json-schema/experimental'), + import('@hyperjump/json-schema-formats'), + ]); + const formatChecks: ReadonlyMap boolean> = new Map([ + ['date-time', formats.isDateTime], + ['json-pointer', formats.isJsonPointer], + ['uri', formats.isUri], + ['uri-template', formats.isUriTemplate], + ]); + validator.setShouldValidateFormat(true); + for (const [name, isFormat] of formatChecks) { + addFormat({ + id: `https://json-schema.org/format/${name}`, + handler: (value: unknown) => typeof value !== 'string' || isFormat(value), + }); + } + return validator; +} diff --git a/capabilities/coordination/src/dsl/dsl-validation.ts b/capabilities/coordination/src/dsl/dsl-validation.ts index 7e9dae963..d0df4c252 100644 --- a/capabilities/coordination/src/dsl/dsl-validation.ts +++ b/capabilities/coordination/src/dsl/dsl-validation.ts @@ -14,16 +14,16 @@ import { type JsonObject, type TaskList, } from '@on.auto/workflow-engine'; -import { buildGraph, Classes } from '@openworkflowspec/sdk'; import specificationPackage from '@openworkflowspec/sdk/package.json' with { type: 'json' }; import { Predicate, Schema } from 'effect'; import satisfies from 'semver/functions/satisfies.js'; import { withoutOutputShape } from '../document/workflow-output.ts'; +import { specificationSdk, type SpecificationSdk } from './dsl-libraries.ts'; import { problemsOf, type Problem } from './schema-problems.ts'; import { schemaChecksOf } from './specification-schema.ts'; -type Workflow = ReturnType; +type Workflow = ReturnType; type Reading = { readonly text: string; readonly document: JsonObject } | { readonly problems: readonly Problem[] }; @@ -44,9 +44,11 @@ export async function dslProblems( return unread; } const refused = (await schemaChecksOf(reading.document)).flatMap((check) => problemsOf(check)); - return refused.length > 0 - ? refused - : firstFound([() => duplicatedNames(reading.document), () => (connecting ? connectionProblems(reading.text) : [])]); + if (refused.length > 0) { + return refused; + } + const duplicated = duplicatedNames(reading.document); + return duplicated.length > 0 || !connecting ? duplicated : connectionProblems(reading.text); } function firstFound(checks: readonly (() => readonly Problem[])[]): readonly Problem[] { @@ -130,8 +132,9 @@ function duplicatedNames(document: JsonObject): readonly Problem[] { ); } -function connectionProblems(text: string): readonly Problem[] { - const hydration = hydrated(text); +async function connectionProblems(text: string): Promise { + const { buildGraph, Classes } = await specificationSdk(); + const hydration = hydrated(() => Classes.Workflow.deserialize(text, { validate: false })); if ('problems' in hydration) { return hydration.problems; } @@ -143,9 +146,11 @@ function connectionProblems(text: string): readonly Problem[] { } } -function hydrated(text: string): { readonly workflow: Workflow } | { readonly problems: readonly Problem[] } { +function hydrated( + deserialized: () => Workflow, +): { readonly workflow: Workflow } | { readonly problems: readonly Problem[] } { try { - return { workflow: Classes.Workflow.deserialize(text, { validate: false }) }; + return { workflow: deserialized() }; } catch (error) { return { problems: [{ pointer: '', detail: messageOf(error) }] }; } diff --git a/capabilities/coordination/src/dsl/library-entries.test.ts b/capabilities/coordination/src/dsl/library-entries.test.ts new file mode 100644 index 000000000..08612f6a4 --- /dev/null +++ b/capabilities/coordination/src/dsl/library-entries.test.ts @@ -0,0 +1,24 @@ +import { fileURLToPath } from 'node:url'; + +import { modulesReachedFrom } from '@on.auto/operations/testing'; +import { describe, expect, it } from 'vitest'; + +const sources = fileURLToPath(new URL('..', import.meta.url)); + +const libraries: ReadonlySet = new Set([ + '@hyperjump/json-schema-formats', + '@hyperjump/json-schema/draft-2020-12', + '@hyperjump/json-schema/experimental', + '@openworkflowspec/sdk', +]); + +function librariesReachedFrom(module: string): readonly string[] { + return modulesReachedFrom(sources, module).packages.filter((name) => libraries.has(name)); +} + +describe('the SDK of the specification and the validator of its schema', () => { + it('are reached by no module the entry of the capability imports, and load with the first document it reads', () => { + expect(modulesReachedFrom(sources, 'index.ts').files).toContain('dsl/dsl-libraries.ts'); + expect(librariesReachedFrom('index.ts')).toEqual([]); + }); +}); diff --git a/capabilities/coordination/src/dsl/specification-schema.ts b/capabilities/coordination/src/dsl/specification-schema.ts index 2a65bd051..aaf28a8bc 100644 --- a/capabilities/coordination/src/dsl/specification-schema.ts +++ b/capabilities/coordination/src/dsl/specification-schema.ts @@ -1,12 +1,4 @@ -import { isDateTime, isJsonPointer, isUri, isUriTemplate } from '@hyperjump/json-schema-formats'; -import { - registerSchema, - setShouldValidateFormat, - validate, - type SchemaObject, - type Validator, -} from '@hyperjump/json-schema/draft-2020-12'; -import { addFormat } from '@hyperjump/json-schema/experimental'; +import type { SchemaObject, Validator } from '@hyperjump/json-schema/draft-2020-12'; import { allTaskEntries, entriesOf, @@ -18,9 +10,10 @@ import { type JsonEntry, type JsonObject, } from '@on.auto/workflow-engine'; -import { workflowSchema } from '@openworkflowspec/sdk'; import { Schema } from 'effect'; +import { schemaValidator, specificationSdk, type SchemaValidator } from './dsl-libraries.ts'; + export interface SchemaRefusal { readonly keyword: string; readonly at: string; @@ -55,14 +48,12 @@ interface Place { readonly location: string; } -const taskDefinition = '#/$defs/task'; +interface Specification { + readonly id: string; + readonly schema: JsonObject; +} -const formatChecks: ReadonlyMap boolean> = new Map([ - ['date-time', isDateTime], - ['json-pointer', isJsonPointer], - ['uri', isUri], - ['uri-template', isUriTemplate], -]); +const taskDefinition = '#/$defs/task'; const schemaKeywords: ReadonlySet = new Set([ 'additionalItems', @@ -114,11 +105,9 @@ function readMembers(members: JsonObject): JsonObject { ); } -const published = Schema.decodeUnknownSync( +const publishedOf = Schema.decodeUnknownSync( Schema.StructWithRest(Schema.Struct({ $id: Schema.String }), [Schema.Record(Schema.String, Schema.Json)]), -)(workflowSchema); - -const specification = readObject(published); +); let compiling: Promise | undefined; @@ -136,24 +125,27 @@ export async function schemaChecksOf(document: JsonObject): Promise { - setShouldValidateFormat(true); - for (const [name, isFormat] of formatChecks) { - addFormat({ - id: `https://json-schema.org/format/${name}`, - handler: (value: unknown) => typeof value !== 'string' || isFormat(value), - }); - } - registerSchema(schemaObjectOf(specification)); - const [document, task] = await Promise.all([checkAt('#'), checkAt(taskDefinition)]); + const [{ workflowSchema }, { registerSchema, validate }] = await Promise.all([specificationSdk(), schemaValidator()]); + const published = publishedOf(workflowSchema); + const specification: Specification = { id: published.$id, schema: readObject(published) }; + registerSchema(schemaObjectOf(specification.schema)); + const [document, task] = await Promise.all([ + checkAt(validate, specification, '#'), + checkAt(validate, specification, taskDefinition), + ]); return { document, task }; } -async function checkAt(definition: string): Promise { - const root: Place = { at: '#', location: `${published.$id}${definition}` }; +async function checkAt( + validate: SchemaValidator['validate'], + specification: Specification, + definition: string, +): Promise { + const root: Place = { at: '#', location: `${specification.id}${definition}` }; const validator = await validate(root.location); return (instance) => { const output = validator(instanceOf(instance), 'DETAILED'); - return output.valid ? [] : refusalsIn(output.errors, root, instance); + return output.valid ? [] : refusalsIn(output.errors, root, instance, specification.schema); }; } @@ -161,6 +153,7 @@ function refusalsIn( units: readonly RefusedUnit[] | undefined, parent: Place, instance: Json, + schema: JsonObject, ): readonly SchemaRefusal[] { return (units ?? []).flatMap((unit) => { const place = placeOf(unit.absoluteKeywordLocation, parent); @@ -169,10 +162,10 @@ function refusalsIn( keyword: place.at.slice(place.at.lastIndexOf('/') + 1), at: place.at, pointer, - rule: valueAtPointer(specification, decodeURI(fragmentOf(place.location))), + rule: valueAtPointer(schema, decodeURI(fragmentOf(place.location))), value: valueAtPointer(instance, pointer), }; - return [refusal].concat(refusalsIn(unit.errors, place, instance)); + return [refusal].concat(refusalsIn(unit.errors, place, instance, schema)); }); } From 1a8b2ed9a6c81d8f06004757cdad0b061dad08dd Mon Sep 17 00:00:00 2001 From: Rami Date: Fri, 23 Oct 2026 13:55:02 +0100 Subject: [PATCH 2/3] refactor(object-storage): the S3 client loads only when the settings name a bucket The server's settings read CONTENT_STORE_URL with contentLocationOf from the package's entry, which also exports the client and the store and so reaches @aws-sdk/client-s3 and @aws-sdk/lib-storage; content-store.ts imported the same entry. Every server loaded the S3 client at start, though one keeps its content in a bucket only when CONTENT_STORE_URL is set. - contentLocationOf moves to a subpath of its own, @on.auto/object-storage/location, the module it is written in, which reaches nothing of the AWS SDK; the entry of the adapter keeps the client, the store and the check of the bucket. - contentStoreOf, the one place that picks the store, answers nothing when the content is kept in the ledger, as before, and otherwise an effect that imports the adapter and composes it; the ledger's layer awaits it before the bucket is checked, as it awaited the check. The path of the ledger's own content stays synchronous. - location-entry.test.ts and content-entries.test.ts hold that the location's entry reaches no S3 client and that the server's start reaches only that entry. Measured on 2026-10-23 on the same machine, by the same hook, at the median of 11 starts on SQLite interleaved with the tree of the commit before, at a load average of 5 to 4: 678 ms against 691 ms, within the spread of the starts, 2,388 files and 19.4 MiB against 2,414 and 20.1 MiB. The CPU profile gave the two clients 11 ms of the start before and none after, and the AWS SDK with Smithy 7 ms against 17 ms. The AWS credential chain still loads at start, through the reasoning capability and the server's other clients, until the commit after. Co-Authored-By: Claude Fable 5.1 --- packages/object-storage/README.md | 6 ++-- packages/object-storage/package.json | 1 + packages/object-storage/src/index.ts | 1 - .../src/location/location-entry.test.ts | 19 ++++++++++++ .../src/composition/content-entries.test.ts | 17 +++++++++++ .../server/src/composition/content-store.ts | 30 ++++++++++++------- .../server/src/composition/ledger-store.ts | 8 +++-- .../server/src/settings/content-settings.ts | 2 +- .../src/telemetry/object-commands.test.ts | 10 +++---- 9 files changed, 70 insertions(+), 24 deletions(-) create mode 100644 packages/object-storage/src/location/location-entry.test.ts create mode 100644 packages/server/src/composition/content-entries.test.ts diff --git a/packages/object-storage/README.md b/packages/object-storage/README.md index bf37d1dfb..352a0c323 100644 --- a/packages/object-storage/README.md +++ b/packages/object-storage/README.md @@ -4,14 +4,14 @@ The content port of [`@on.auto/operations`](../operations/README.md#recorded-con ## Entry -`@on.auto/object-storage` (`src/index.ts`) exports four functions and a failure: +`@on.auto/object-storage` (`src/index.ts`) exports three functions and a failure, and `@on.auto/object-storage/location` (`src/location/content-location.ts`) a fourth, a subpath of its own so that the settings of the server read `CONTENT_STORE_URL` without loading the S3 client, which the entry of the adapter reaches (`src/location/location-entry.test.ts`): -- `contentLocationOf(url)` reads `CONTENT_STORE_URL`, an `s3://bucket/prefix` URL, into `{ bucket, prefix }`, or nothing for a URL that is not one: the scheme is `s3:`, the bucket follows the S3 naming rule, 3 to 63 lowercase letters, digits, dots and hyphens, and the URL names no user, password, port, query or fragment. The prefix is the path without its leading, trailing and doubled slashes, and may be empty. +- `contentLocationOf(url)`, of `@on.auto/object-storage/location`, reads `CONTENT_STORE_URL`, an `s3://bucket/prefix` URL, into `{ bucket, prefix }`, or nothing for a URL that is not one: the scheme is `s3:`, the bucket follows the S3 naming rule, 3 to 63 lowercase letters, digits, dots and hyphens, and the URL names no user, password, port, query or fragment. The prefix is the path without its leading, trailing and doubled slashes, and may be empty. - `storageClientOf({ endpoint, region, proxy }, credentials, waits?)` makes the `S3Client`. A custom endpoint is addressed by path, `http://host:9000/bucket/key`, as S3-compatible servers expect; without one the client addresses the S3 endpoint of its region by host. Without a region the SDK finds one by its own chain, and fails the first request when it finds none. With the environment proxy on, `NODE_USE_ENV_PROXY=1`, the client's requests go through the proxy as the credential chain's do, through `awsClientSettingsOf` of [`@on.auto/outbound/aws`](../outbound/README.md#aws-clients). A connection waits at most 5 s to open and a request at most 60 s for its answer, after which it fails as a `TimeoutError`, so a store that stops answering never holds a check, a put, a get or the cleanup of a failed put; `waits` sets other bounds, which the tests use. The client computes a request checksum and checks a response checksum only where the S3 API requires one, `WHEN_REQUIRED`, since the SDK's default of a checksum on every request is one many S3-compatible servers refuse; the content's own SHA-256 proves every byte read back. - `objectStoreReached(client, location)` asks the store for the bucket with `HeadBucket` and fails with `ObjectStoreUnreachable`, `{ bucket, answer }`, when it cannot: the answer is the store's, its name and status, such as `NotFound (404)` or `AccessDenied (403)`, or the failure of the request itself, such as `Error: connect ECONNREFUSED 127.0.0.1:9000` or `Error: Region is missing`; it carries no credential. - `objectContentOn(client, location, waits?)` answers a `ContentStore` of `@on.auto/operations`, `{ content, sweepContent }`: `content`, the port, and `sweepContent(before)`, the sweep of uploads below. `client` is a function that answers the client, so that nothing passes the client itself, whose fields are not read-only, as a parameter. -The server composes them in `packages/server/src/composition/content-store.ts`: the location and the endpoint from the content store settings, the region from `AWS_REGION`, the environment proxy as the server reads it once for every client it makes, and the credentials from `awsCredentialChain` of `@on.auto/outbound/aws`, the default credential chain of the AWS SDK that the Bedrock provider of the reasoning capability uses too. Before the ledger opens, `packages/server/src/composition/ledger-store.ts` checks the bucket with `objectStoreReached`, and a bucket it cannot reach stops the server at start, naming `CONTENT_STORE_URL`, the bucket, the endpoint's host and the store's answer. +The server composes them in `packages/server/src/composition/content-store.ts`: the location and the endpoint from the content store settings, the region from `AWS_REGION`, the environment proxy as the server reads it once for every client it makes, and the credentials from `awsCredentialChain` of `@on.auto/outbound/aws`, the default credential chain of the AWS SDK that the Bedrock provider of the reasoning capability uses too. It imports the entry of the adapter only when the settings name a bucket, so a server whose content is kept in the ledger never loads `@aws-sdk/client-s3` and `@aws-sdk/lib-storage`, which were 26 files and 0.7 MiB of its start, and 11 ms of it in a CPU profile (`packages/server/src/composition/content-entries.test.ts`; measured on 2026-10-23, at the median of 11 starts and of 7 profiles). Before the ledger opens, `packages/server/src/composition/ledger-store.ts` checks the bucket with `objectStoreReached`, and a bucket it cannot reach stops the server at start, naming `CONTENT_STORE_URL`, the bucket, the endpoint's host and the store's answer. `@on.auto/object-storage/testing` (`src/testing/index.ts`) is the entry the tests of this package and of the server take a real S3-compatible server through, so the reading of its URL and the making of its bucket are written once: `testObjectStoreOf(url)` reads `http://access:secret@host:port/bucket` into the bucket, with a client of its own, the endpoint and the credentials; `bucketReady(bucket)` makes the bucket when it is missing, and finds it when another test made it at the same moment; `keysIn(bucket, prefix)` and `openUploadsIn(bucket, prefix)` list what a prefix holds, and `emptied(bucket, prefix)` aborts and deletes it. diff --git a/packages/object-storage/package.json b/packages/object-storage/package.json index a7cb744af..1cbfa0c1a 100644 --- a/packages/object-storage/package.json +++ b/packages/object-storage/package.json @@ -6,6 +6,7 @@ "type": "module", "exports": { ".": "./src/index.ts", + "./location": "./src/location/content-location.ts", "./testing": "./src/testing/index.ts" }, "scripts": { diff --git a/packages/object-storage/src/index.ts b/packages/object-storage/src/index.ts index 48b00aeca..b4b162fa0 100644 --- a/packages/object-storage/src/index.ts +++ b/packages/object-storage/src/index.ts @@ -5,4 +5,3 @@ export { type StorageWaits, } from './client/storage-client.ts'; export { objectContentOn, objectStoreReached, ObjectStoreUnreachable } from './content/object-content.ts'; -export { contentLocationOf, type ContentLocation } from './location/content-location.ts'; diff --git a/packages/object-storage/src/location/location-entry.test.ts b/packages/object-storage/src/location/location-entry.test.ts new file mode 100644 index 000000000..9330e78a8 --- /dev/null +++ b/packages/object-storage/src/location/location-entry.test.ts @@ -0,0 +1,19 @@ +import { fileURLToPath } from 'node:url'; + +import { modulesReachedFrom } from '@on.auto/operations/testing'; +import { describe, expect, it } from 'vitest'; + +const sources = fileURLToPath(new URL('..', import.meta.url)); + +const clients: ReadonlySet = new Set(['@aws-sdk/client-s3', '@aws-sdk/lib-storage']); + +function clientsReachedFrom(entry: string): readonly string[] { + return modulesReachedFrom(sources, entry).packages.filter((name) => clients.has(name)); +} + +describe('the S3 client', () => { + it('is reached from the entry of the adapter, and not from the entry of the location, which the settings read', () => { + expect(clientsReachedFrom('location/content-location.ts')).toEqual([]); + expect(clientsReachedFrom('index.ts')).toEqual(['@aws-sdk/client-s3', '@aws-sdk/lib-storage']); + }); +}); diff --git a/packages/server/src/composition/content-entries.test.ts b/packages/server/src/composition/content-entries.test.ts new file mode 100644 index 000000000..05fef9630 --- /dev/null +++ b/packages/server/src/composition/content-entries.test.ts @@ -0,0 +1,17 @@ +import { fileURLToPath } from 'node:url'; + +import { modulesReachedFrom } from '@on.auto/operations/testing'; +import { describe, expect, it } from 'vitest'; + +const sources = fileURLToPath(new URL('..', import.meta.url)); + +function objectStorageReachedFrom(module: string): readonly string[] { + return modulesReachedFrom(sources, module).packages.filter((name) => name.startsWith('@on.auto/object-storage')); +} + +describe('the entries of the object store the server imports as it starts', () => { + it('are the entry of the location alone, which the settings read, and the adapter only when they name a bucket', () => { + expect(objectStorageReachedFrom('lifecycle/run-server.ts')).toEqual(['@on.auto/object-storage/location']); + expect(objectStorageReachedFrom('composition/composition-root.ts')).toEqual([]); + }); +}); diff --git a/packages/server/src/composition/content-store.ts b/packages/server/src/composition/content-store.ts index aab40cd30..89ffa421a 100644 --- a/packages/server/src/composition/content-store.ts +++ b/packages/server/src/composition/content-store.ts @@ -1,4 +1,3 @@ -import { objectContentOn, objectStoreReached, storageClientOf } from '@on.auto/object-storage'; import type { ContentStore } from '@on.auto/operations'; import { awsCredentialChain } from '@on.auto/outbound/aws'; import { Effect } from 'effect'; @@ -17,23 +16,32 @@ function hostOf(endpoint: string | undefined): string { return endpoint === undefined ? 'the S3 endpoint of its region' : new URL(endpoint).host; } -export function contentStoreOf({ content, proxy }: ContentStoreSettings): ComposedContentStore | undefined { +export function contentStoreOf({ + content, + proxy, +}: ContentStoreSettings): Effect.Effect | undefined { if (content.store === 'ledger') { return undefined; } const { endpoint, region, location } = content; - const client = storageClientOf({ endpoint, region, proxy }, awsCredentialChain(proxy)); - client.middlewareStack.add(commandSpansIn(location.bucket), commandSpanStep); const unreachable = (answer: string) => new Error( `CONTENT_STORE_URL names the bucket ${location.bucket} on ${hostOf(endpoint)}, which could not be reached: ${answer}`, ); - return { - store: objectContentOn(() => client, location), - reached: objectStoreReached(() => client, location).pipe( - Effect.catchTag('object_store_unreachable', ({ answer }: { readonly answer: string }) => - Effect.die(unreachable(answer)), + return Effect.gen(function* () { + const { objectContentOn, objectStoreReached, storageClientOf } = yield* Effect.promise( + () => import('@on.auto/object-storage'), + ); + const client = storageClientOf({ endpoint, region, proxy }, awsCredentialChain(proxy)); + client.middlewareStack.add(commandSpansIn(location.bucket), commandSpanStep); + const composed: ComposedContentStore = { + store: objectContentOn(() => client, location), + reached: objectStoreReached(() => client, location).pipe( + Effect.catchTag('object_store_unreachable', ({ answer }: { readonly answer: string }) => + Effect.die(unreachable(answer)), + ), ), - ), - }; + }; + return composed; + }); } diff --git a/packages/server/src/composition/ledger-store.ts b/packages/server/src/composition/ledger-store.ts index 95539a7fc..c8c4d533c 100644 --- a/packages/server/src/composition/ledger-store.ts +++ b/packages/server/src/composition/ledger-store.ts @@ -28,8 +28,10 @@ export function ledgerLayerFor( database: LedgerDatabase, appends?: AppendSignal, ): Layer.Layer { - const content = contentStoreOf(settings); - return content === undefined + const composing = contentStoreOf(settings); + return composing === undefined ? ledgerLayerOf(database, appends) - : Layer.unwrap(Effect.as(content.reached, ledgerLayerOf(database, appends, content.store))); + : Layer.unwrap( + Effect.flatMap(composing, ({ reached, store }) => Effect.as(reached, ledgerLayerOf(database, appends, store))), + ); } diff --git a/packages/server/src/settings/content-settings.ts b/packages/server/src/settings/content-settings.ts index 10c469ce7..c36099727 100644 --- a/packages/server/src/settings/content-settings.ts +++ b/packages/server/src/settings/content-settings.ts @@ -1,5 +1,5 @@ import type { Environment } from '@on.auto/config'; -import { contentLocationOf, type ContentLocation } from '@on.auto/object-storage'; +import { contentLocationOf, type ContentLocation } from '@on.auto/object-storage/location'; import { Config, ConfigProvider, Effect, Option } from 'effect'; import { InvalidSettingsError } from './invalid-settings-error.ts'; diff --git a/packages/server/src/telemetry/object-commands.test.ts b/packages/server/src/telemetry/object-commands.test.ts index 8825d71b1..bc04144bb 100644 --- a/packages/server/src/telemetry/object-commands.test.ts +++ b/packages/server/src/telemetry/object-commands.test.ts @@ -39,17 +39,17 @@ async function aStoreHoldingNothing(): Promise { return `http://127.0.0.1:${String(decodeAddress(server.address()).port)}`; } -function composedOn(environment: Readonly>): ComposedContentStore { +function composedOn(environment: Readonly>): Promise { vi.stubEnv('AWS_ACCESS_KEY_ID', 'an-access-key'); vi.stubEnv('AWS_SECRET_ACCESS_KEY', 'a-secret-access-key'); onTestFinished(() => { vi.unstubAllEnvs(); }); - const composed = contentStoreOf(readSettingsOf(environment)); - if (composed === undefined) { + const composing = contentStoreOf(readSettingsOf(environment)); + if (composing === undefined) { throw new TypeError('CONTENT_STORE_URL is set, so the content is kept in an object store'); } - return composed; + return Effect.runPromise(composing); } async function spansOfAReadOfNothing(): Promise { @@ -63,7 +63,7 @@ async function spansOfAReadOfNothing(): Promise { OTEL_EXPORTER_OTLP_PROTOCOL: 'http/json', }; const telemetry = startTelemetry(readSettingsOf(environment).telemetry); - const { reached, store } = composedOn(environment); + const { reached, store } = await composedOn(environment); await Effect.runPromise(reached); await Effect.runPromise(store.content.get(alpha, sha256Of('{"kept":false}'))); await telemetry.shutdown(); From 93ec46157f8ee7e9f7fe663dbde22b5fac72299a Mon Sep 17 00:00:00 2001 From: Rami Date: Fri, 23 Oct 2026 13:55:12 +0100 Subject: [PATCH 3/3] refactor(reasoning): the credential libraries load only when their provider is configured Every server loaded at start two credential libraries that only some use. credential-sources.ts imported google-auth-library for GoogleAuth and OAuth2Client, and cloud-providers.ts the AI SDK's two Vertex providers, which import it too, though only a server with Vertex configured uses it. model-factories.ts imported awsCredentialChain from @on.auto/outbound/aws, whose module imports @aws-sdk/credential-providers and @aws-sdk/rds-signer, and so did the server's content store, its KMS custodian, its Parameter Store keeper and its list of the variables of the libraries, which took awsCredentialVariables from it, though only Bedrock, a bucket, KMS and Parameter Store use the chain. The Entra ID path already loads @azure/identity only for Azure without a key. - vertex-providers.ts holds what needs google-auth-library: the default credential chain, the refreshing client and the two Vertex model factories, moved from credential-sources.ts and cloud-providers.ts. modelFactories, where the providers are picked, imports it once when Vertex is configured, and the AWS chain once when Bedrock is configured with no credential source given, and so answers a promise, which makeModelAccess, already an effect, awaits. - awsCredentialChainOf, in the main entry of @on.auto/outbound, is the one place that loads the AWS chain: it imports the aws subpath when it is first asked. The content store, the KMS custodian and the Parameter Store keeper ask for it beside the adapter each already loads, the Bedrock entries as model access is built, and awsCredentialVariables moves to the main entry, beside the variables of the proxy. A server whose ledger is in PostgreSQL still loads the subpath with the module of its store, for the signer of the tokens of IAM. - credential-libraries.test.ts holds that neither entry of the capability reaches google-auth-library, the Vertex providers, @on.auto/outbound/aws or @azure/identity; aws-credentials.test.ts that the main entry of outbound loads the credential providers of the AWS SDK only when the chain is first asked for; and aws-entries.test.ts that neither module the server starts with reaches @on.auto/outbound/aws. - The tool-servers README records the start of the three commits beside its measure. Measured on 2026-10-23 on the same machine, by the same hook, at the median of 11 starts on SQLite interleaved with the tree of the commit before, at a load average of 5 to 4: 654 ms against 678 ms, 2,307 files and 18.5 MiB against 2,388 and 19.4 MiB. The CPU profile gave the Google library and the packages it alone loads 10 ms of the start before, the credential providers 4.5 ms and the RDS signer 1.7 ms, and none of them anything after. Against main, with the two commits before, 654 ms against 821 ms, and 2,307 files against 2,636. Co-Authored-By: Claude Fable 5.1 --- capabilities/reasoning/README.md | 2 +- .../reasoning/src/adapter/cloud-providers.ts | 27 +----- .../src/adapter/credential-libraries.test.ts | 29 +++++++ .../src/adapter/credential-sources.test.ts | 8 +- .../src/adapter/credential-sources.ts | 14 --- .../reasoning/src/adapter/model-access.ts | 2 +- .../reasoning/src/adapter/model-factories.ts | 85 +++++++++++-------- .../src/adapter/vertex-providers.test.ts | 9 ++ .../reasoning/src/adapter/vertex-providers.ts | 40 +++++++++ docs/engineering/reference/models.md | 4 +- packages/object-storage/README.md | 2 +- packages/outbound/README.md | 2 +- packages/outbound/src/aws/aws-clients.ts | 8 -- .../outbound/src/aws/aws-credentials.test.ts | 24 ++++++ packages/outbound/src/aws/aws-credentials.ts | 15 ++++ packages/outbound/src/index.ts | 1 + packages/server/README.md | 2 +- .../src/composition/aws-entries.test.ts | 19 +++++ .../server/src/composition/content-store.ts | 8 +- .../connections/variables-of-the-libraries.ts | 3 +- packages/server/src/secrets/kms-custodians.ts | 11 +-- packages/server/src/secrets/secret-keepers.ts | 9 +- packages/tool-servers/README.md | 2 +- 23 files changed, 211 insertions(+), 115 deletions(-) create mode 100644 capabilities/reasoning/src/adapter/credential-libraries.test.ts create mode 100644 capabilities/reasoning/src/adapter/vertex-providers.test.ts create mode 100644 capabilities/reasoning/src/adapter/vertex-providers.ts create mode 100644 packages/outbound/src/aws/aws-credentials.test.ts create mode 100644 packages/outbound/src/aws/aws-credentials.ts create mode 100644 packages/server/src/composition/aws-entries.test.ts diff --git a/capabilities/reasoning/README.md b/capabilities/reasoning/README.md index ac77822ed..5abb16f35 100644 --- a/capabilities/reasoning/README.md +++ b/capabilities/reasoning/README.md @@ -74,4 +74,4 @@ Two tests wait on real time on purpose, where a slow machine only lengthens the ## Source -`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries. `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model. +`src/index.ts` is the entry point and `src/testing/index.ts` the entry point of the test support. `src/model` holds the interface: the request, the result, the `LanguageModel` service and the request checks. `src/failure` holds one class per failure. `src/schema` holds answer schemas: their limits and their compilation, made with the JSON Schema compiler of [`@on.auto/json-schema`](../../packages/json-schema/README.md), and `checkAnswerSchema`, the report of what in a schema is unsupported, not portable across providers or not checked. `src/settings` reads the settings from the environment with Effect `Config`. `src/adapter` and `src/tools` are the only production code that imports the AI SDK, and `src/adapter` the only code that imports the cloud credential libraries: it imports `google-auth-library`, with the AI SDK's Vertex providers, only when model access is built with Vertex configured (`src/adapter/vertex-providers.ts`), the AWS credential chain only with Bedrock configured, through `awsCredentialChainOf` of `@on.auto/outbound`, and `@azure/identity` only for Azure without a key, and no entry of the capability reaches any of them (`src/adapter/credential-libraries.test.ts`). `src/listing` reads the list of models of each provider, and `src/catalog` keeps the lists, combines them with the declared models and the aliases, applies the allow list, and defines `list_models`. `src/template` is the only code that imports liquidjs: its instance of the shared engine of [`@on.auto/definitions/template`](../../packages/definitions/README.md#templates-of-a-capability), with the registrations that stay this capability's own, its prompt filters and its system block, and compiling and rendering a prompt over the shared parse and render, behind readonly types of its own; `rendering-corpus.json` holds what each template of the rendering tests rendered before the engine moved into the shared factory, and `rendering-corpus.test.ts` holds the engine to it. `src/definition` parses and validates a definition document with the reader of `@on.auto/definitions/document`, which splits it and reads its YAML: the keys of its front matter, the settings, the schemas and the variables a template reads, and gives the capability its `parse` and `summarize`. `src/tools` gives the model a run's tools: the tools as the AI SDK takes them, the last step without them, opening a run's tools and its endings. `src/capability` is the capability: its guide, the public reference page, served to agents as `reasoning-function` and ending with the sentence `onThisServer` makes of what this server offers, preparing the input, rendering the prompt, the request, the answer, the rejections and the record. `src/testing` holds the fake and what the tests share, including the SDK's mock model. diff --git a/capabilities/reasoning/src/adapter/cloud-providers.ts b/capabilities/reasoning/src/adapter/cloud-providers.ts index 0262efe45..8a28602cc 100644 --- a/capabilities/reasoning/src/adapter/cloud-providers.ts +++ b/capabilities/reasoning/src/adapter/cloud-providers.ts @@ -1,12 +1,10 @@ import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock'; import { createAmazonBedrockAnthropic } from '@ai-sdk/amazon-bedrock/anthropic'; import { createAzure } from '@ai-sdk/azure'; -import { createGoogleVertex } from '@ai-sdk/google-vertex'; -import { createGoogleVertexAnthropic } from '@ai-sdk/google-vertex/anthropic'; import { Redacted } from 'effect'; -import type { AzureSettings, BedrockSettings, VertexSettings } from '../settings/provider-settings.ts'; -import { refreshingGoogleClient, type AccessTokenSource, type AwsCredentialSource } from './credential-sources.ts'; +import type { AzureSettings, BedrockSettings } from '../settings/provider-settings.ts'; +import type { AccessTokenSource, AwsCredentialSource } from './credential-sources.ts'; import type { Fetch, ModelFactory } from './sdk-model.ts'; function bedrockOptions(settings: BedrockSettings, credentials: AwsCredentialSource, fetch: Fetch) { @@ -47,24 +45,3 @@ export function azureModels(settings: AzureSettings, credential: AzureCredential }); return (modelId) => provider.languageModel(modelId); } - -export function vertexModels(settings: VertexSettings, tokens: AccessTokenSource, fetch: Fetch): ModelFactory { - const provider = createGoogleVertex({ - project: settings.project, - location: settings.location, - apiKey: '', - googleAuthOptions: { authClient: refreshingGoogleClient(tokens) }, - fetch, - }); - return (modelId) => provider.languageModel(modelId); -} - -export function vertexAnthropicModels(settings: VertexSettings, tokens: AccessTokenSource, fetch: Fetch): ModelFactory { - const provider = createGoogleVertexAnthropic({ - project: settings.project, - location: settings.location, - generateAuthToken: tokens, - fetch, - }); - return (modelId) => provider.languageModel(modelId); -} diff --git a/capabilities/reasoning/src/adapter/credential-libraries.test.ts b/capabilities/reasoning/src/adapter/credential-libraries.test.ts new file mode 100644 index 000000000..63f5285b7 --- /dev/null +++ b/capabilities/reasoning/src/adapter/credential-libraries.test.ts @@ -0,0 +1,29 @@ +import { fileURLToPath } from 'node:url'; + +import { modulesReachedFrom } from '@on.auto/operations/testing'; +import { describe, expect, it } from 'vitest'; + +const sources = fileURLToPath(new URL('..', import.meta.url)); + +const libraries: ReadonlySet = new Set([ + '@ai-sdk/google-vertex', + '@ai-sdk/google-vertex/anthropic', + '@azure/identity', + '@on.auto/outbound/aws', + 'google-auth-library', +]); + +function librariesReachedFrom(module: string): readonly string[] { + return modulesReachedFrom(sources, module).packages.filter((name) => libraries.has(name)); +} + +describe('the credential libraries of the providers', () => { + it('are reached by no entry of the capability, and load when model access is built with their provider configured', () => { + expect([...librariesReachedFrom('index.ts'), ...librariesReachedFrom('testing/index.ts')]).toEqual([]); + expect(librariesReachedFrom('adapter/vertex-providers.ts')).toEqual([ + '@ai-sdk/google-vertex', + '@ai-sdk/google-vertex/anthropic', + 'google-auth-library', + ]); + }); +}); diff --git a/capabilities/reasoning/src/adapter/credential-sources.test.ts b/capabilities/reasoning/src/adapter/credential-sources.test.ts index d1540e953..c8cc0b228 100644 --- a/capabilities/reasoning/src/adapter/credential-sources.test.ts +++ b/capabilities/reasoning/src/adapter/credential-sources.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest'; -import { googleAccessTokens, googleCredentialChain, guarded } from './credential-sources.ts'; +import { googleAccessTokens, guarded } from './credential-sources.ts'; import { OutboundFailure } from './outbound-failure.ts'; describe('guarded', () => { @@ -28,9 +28,3 @@ describe('googleAccessTokens', () => { ); }); }); - -describe('the default credential chains', () => { - it('are built without looking anything up', () => { - expect(typeof googleCredentialChain('acme-ai')).toBe('function'); - }); -}); diff --git a/capabilities/reasoning/src/adapter/credential-sources.ts b/capabilities/reasoning/src/adapter/credential-sources.ts index 55ed78852..f0a2d892f 100644 --- a/capabilities/reasoning/src/adapter/credential-sources.ts +++ b/capabilities/reasoning/src/adapter/credential-sources.ts @@ -1,5 +1,3 @@ -import { GoogleAuth, OAuth2Client } from 'google-auth-library'; - import { OutboundFailure } from './outbound-failure.ts'; export const googleCredentialVariables: readonly string[] = ['GOOGLE_APPLICATION_CREDENTIALS']; @@ -24,8 +22,6 @@ export interface CredentialSources { readonly azure?: AccessTokenSource; } -const googleCloudScope = 'https://www.googleapis.com/auth/cloud-platform'; - export function guarded(source: () => Promise): () => Promise { return async () => { try { @@ -45,13 +41,3 @@ export function googleAccessTokens(client: GoogleTokenClient): AccessTokenSource return token; }; } - -export function googleCredentialChain(project: string): AccessTokenSource { - return googleAccessTokens(new GoogleAuth({ scopes: [googleCloudScope], projectId: project })); -} - -export function refreshingGoogleClient(tokens: AccessTokenSource): OAuth2Client { - const client = new OAuth2Client(); - client.refreshHandler = async () => ({ access_token: await tokens(), expiry_date: Date.now() }); - return client; -} diff --git a/capabilities/reasoning/src/adapter/model-access.ts b/capabilities/reasoning/src/adapter/model-access.ts index b419941a0..74d81a336 100644 --- a/capabilities/reasoning/src/adapter/model-access.ts +++ b/capabilities/reasoning/src/adapter/model-access.ts @@ -27,7 +27,7 @@ export const makeModelAccess = Effect.fnUntraced(function* (settings: ModelSetti const status = providerStatus(settings, { entraId: azureTokens !== undefined }); const fetch = options.fetch ?? globalThis.fetch; const proxy = options.proxy ?? { enabled: false, environment: {} }; - const models = modelFactories(settings, { fetch, credentials, azureTokens, proxy }); + const models = yield* Effect.promise(() => modelFactories(settings, { fetch, credentials, azureTokens, proxy })); const access: ModelAccess = { languageModel: resolvedLanguageModel(models, settings, status, options), status, diff --git a/capabilities/reasoning/src/adapter/model-factories.ts b/capabilities/reasoning/src/adapter/model-factories.ts index 61b936198..30953d80f 100644 --- a/capabilities/reasoning/src/adapter/model-factories.ts +++ b/capabilities/reasoning/src/adapter/model-factories.ts @@ -1,21 +1,14 @@ -import { awsCredentialChain } from '@on.auto/outbound/aws'; +import { awsCredentialChainOf, type ProxySettings } from '@on.auto/outbound'; import type { ModelSettings } from '../settings/model-settings.ts'; -import type { AzureSettings } from '../settings/provider-settings.ts'; +import type { AzureSettings, BedrockSettings, VertexSettings } from '../settings/provider-settings.ts'; import type { Availability } from '../settings/setting-values.ts'; +import { azureModels, bedrockAnthropicModels, bedrockModels, type AzureCredential } from './cloud-providers.ts'; import { - azureModels, - bedrockAnthropicModels, - bedrockModels, - vertexAnthropicModels, - vertexModels, - type AzureCredential, -} from './cloud-providers.ts'; -import { - googleCredentialChain, guarded, type AccessTokenSource, type AwsCredentials, + type AwsCredentialSource, type CredentialSources, } from './credential-sources.ts'; import { anthropicModels, gatewayModels, googleModels, openAiModels } from './direct-providers.ts'; @@ -27,7 +20,7 @@ export interface FactorySources { readonly fetch: Fetch; readonly credentials: CredentialSources; readonly azureTokens: AccessTokenSource | undefined; - readonly proxy: Parameters[0]; + readonly proxy: ProxySettings; } type Entry = readonly [string, ModelFactory]; @@ -46,40 +39,60 @@ function azureCredentialOf( return tokens === undefined ? undefined : { tokens: guarded(tokens) }; } -function cloudEntries( - settings: ModelSettings, - { fetch, credentials, azureTokens, proxy }: FactorySources, -): readonly Entry[] { +function azureEntries(azure: AzureSettings, tokens: AccessTokenSource | undefined, fetch: Fetch): readonly Entry[] { + const credential = azureCredentialOf(azure, tokens); + return credential === undefined ? [] : [['azure', azureModels(azure, credential, fetch)]]; +} + +async function bedrockEntries( + bedrock: Availability, + given: AwsCredentialSource | undefined, + proxy: ProxySettings, + fetch: Fetch, +): Promise { + if (!bedrock.configured) { + return []; + } + const aws = guarded(given ?? (await awsCredentialChainOf(proxy))); + return [ + ['bedrock', bedrockModels(bedrock.settings, aws, fetch)], + ['bedrock-anthropic', bedrockAnthropicModels(bedrock.settings, aws, fetch)], + ]; +} + +async function vertexEntries( + vertex: Availability, + google: AccessTokenSource | undefined, + fetch: Fetch, +): Promise { + if (!vertex.configured) { + return []; + } + const { googleCredentialChain, vertexAnthropicModels, vertexModels } = await import('./vertex-providers.ts'); + const tokens = guarded(google ?? googleCredentialChain(vertex.settings.project)); return [ - ...entriesFor(settings.bedrock, (bedrock) => { - const aws = guarded(credentials.aws ?? awsCredentialChain(proxy)); - return [ - ['bedrock', bedrockModels(bedrock, aws, fetch)], - ['bedrock-anthropic', bedrockAnthropicModels(bedrock, aws, fetch)], - ]; - }), - ...entriesFor(settings.azure, (azure) => { - const credential = azureCredentialOf(azure, azureTokens); - return credential === undefined ? [] : [['azure', azureModels(azure, credential, fetch)]]; - }), - ...entriesFor(settings.vertex, (vertex) => { - const google = guarded(credentials.google ?? googleCredentialChain(vertex.project)); - return [ - ['vertex', vertexModels(vertex, google, fetch)], - ['vertex-anthropic', vertexAnthropicModels(vertex, google, fetch)], - ]; - }), + ['vertex', vertexModels(vertex.settings, tokens, fetch)], + ['vertex-anthropic', vertexAnthropicModels(vertex.settings, tokens, fetch)], ]; } -export function modelFactories(settings: ModelSettings, sources: FactorySources): ReadonlyMap { +export async function modelFactories( + settings: ModelSettings, + sources: FactorySources, +): Promise> { installSdkGlobals(); const fetch = withCertificateFailures(sources.fetch); + const [bedrock, vertex] = await Promise.all([ + bedrockEntries(settings.bedrock, sources.credentials.aws, sources.proxy, fetch), + vertexEntries(settings.vertex, sources.credentials.google, fetch), + ]); return new Map([ ...entriesFor(settings.anthropic, (anthropic) => [['anthropic', anthropicModels(anthropic, fetch)]]), ...entriesFor(settings.openai, (openai) => [['openai', openAiModels(openai, fetch)]]), ...entriesFor(settings.google, (google) => [['google', googleModels(google, fetch)]]), - ...cloudEntries(settings, { ...sources, fetch }), + ...bedrock, + ...entriesFor(settings.azure, (azure) => azureEntries(azure, sources.azureTokens, fetch)), + ...vertex, ...settings.gateways.map((gateway): Entry => [gateway.name, gatewayModels(gateway, fetch)]), ]); } diff --git a/capabilities/reasoning/src/adapter/vertex-providers.test.ts b/capabilities/reasoning/src/adapter/vertex-providers.test.ts new file mode 100644 index 000000000..680047306 --- /dev/null +++ b/capabilities/reasoning/src/adapter/vertex-providers.test.ts @@ -0,0 +1,9 @@ +import { describe, expect, it } from 'vitest'; + +import { googleCredentialChain } from './vertex-providers.ts'; + +describe('the default credential chains', () => { + it('are built without looking anything up', () => { + expect(typeof googleCredentialChain('acme-ai')).toBe('function'); + }); +}); diff --git a/capabilities/reasoning/src/adapter/vertex-providers.ts b/capabilities/reasoning/src/adapter/vertex-providers.ts new file mode 100644 index 000000000..4337c29b5 --- /dev/null +++ b/capabilities/reasoning/src/adapter/vertex-providers.ts @@ -0,0 +1,40 @@ +import { createGoogleVertex } from '@ai-sdk/google-vertex'; +import { createGoogleVertexAnthropic } from '@ai-sdk/google-vertex/anthropic'; +import { GoogleAuth, OAuth2Client } from 'google-auth-library'; + +import type { VertexSettings } from '../settings/provider-settings.ts'; +import { googleAccessTokens, type AccessTokenSource } from './credential-sources.ts'; +import type { Fetch, ModelFactory } from './sdk-model.ts'; + +const googleCloudScope = 'https://www.googleapis.com/auth/cloud-platform'; + +export function googleCredentialChain(project: string): AccessTokenSource { + return googleAccessTokens(new GoogleAuth({ scopes: [googleCloudScope], projectId: project })); +} + +function refreshingGoogleClient(tokens: AccessTokenSource): OAuth2Client { + const client = new OAuth2Client(); + client.refreshHandler = async () => ({ access_token: await tokens(), expiry_date: Date.now() }); + return client; +} + +export function vertexModels(settings: VertexSettings, tokens: AccessTokenSource, fetch: Fetch): ModelFactory { + const provider = createGoogleVertex({ + project: settings.project, + location: settings.location, + apiKey: '', + googleAuthOptions: { authClient: refreshingGoogleClient(tokens) }, + fetch, + }); + return (modelId) => provider.languageModel(modelId); +} + +export function vertexAnthropicModels(settings: VertexSettings, tokens: AccessTokenSource, fetch: Fetch): ModelFactory { + const provider = createGoogleVertexAnthropic({ + project: settings.project, + location: settings.location, + generateAuthToken: tokens, + fetch, + }); + return (modelId) => provider.languageModel(modelId); +} diff --git a/docs/engineering/reference/models.md b/docs/engineering/reference/models.md index 0510c533e..4ec0214f4 100644 --- a/docs/engineering/reference/models.md +++ b/docs/engineering/reference/models.md @@ -130,8 +130,8 @@ No failure or defect carries the request or response bodies the SDK keeps on its ## Credentials -- Bedrock takes `credentialProvider` from the AWS default chain (`awsCredentialChain(proxy)` of `@on.auto/outbound/aws`). `AWS_BEARER_TOKEN_BEDROCK`, when set, is passed as `apiKey`, which the provider sends instead of signing. The chain's own STS and SSO clients make agents of their own, which `NODE_USE_ENV_PROXY` does not reach, so `awsClientSettingsOf(proxy)` gives them the proxy's variables when the switch is on. -- Vertex takes tokens from `GoogleAuth` with the scope `https://www.googleapis.com/auth/cloud-platform` (`googleCredentialChain`), wrapped in an `OAuth2Client` for `vertex` and passed as `generateAuthToken` for `vertex-anthropic`. A `vertex-anthropic` call reads the access token twice, because the AI SDK's Anthropic model resolves its headers twice per request; the token client caches it, so this costs no extra exchange. +- Bedrock takes `credentialProvider` from the AWS default chain (`awsCredentialChainOf(proxy)` of `@on.auto/outbound`), which `modelFactories` loads once, when model access is built with Bedrock configured and no `aws` credential source given (`bedrockEntries` in `src/adapter/model-factories.ts`), so a server without Bedrock loads no part of it for the capability. `AWS_BEARER_TOKEN_BEDROCK`, when set, is passed as `apiKey`, which the provider sends instead of signing. The chain's own STS and SSO clients make agents of their own, which `NODE_USE_ENV_PROXY` does not reach, so `awsClientSettingsOf(proxy)` gives them the proxy's variables when the switch is on. +- Vertex takes tokens from `GoogleAuth` with the scope `https://www.googleapis.com/auth/cloud-platform` (`googleCredentialChain`), wrapped in an `OAuth2Client` for `vertex` and passed as `generateAuthToken` for `vertex-anthropic`. A `vertex-anthropic` call reads the access token twice, because the AI SDK's Anthropic model resolves its headers twice per request; the token client caches it, so this costs no extra exchange. `GoogleAuth` and `OAuth2Client` are `google-auth-library`'s, which the AI SDK's two Vertex providers import too, and all three are in `src/adapter/vertex-providers.ts`, which `modelFactories` imports once, when model access is built with Vertex configured (`vertexEntries` in `src/adapter/model-factories.ts`), so a server without Vertex never loads them, which were 75 files and 0.8 MiB of its start, and 10 ms of it in a CPU profile (`src/adapter/credential-libraries.test.ts`; measured on 2026-10-23, at the median of 11 starts and of 7 profiles). - Azure without a key loads `@azure/identity` once, when model access is built (`azureTokensFor` in `src/adapter/entra-id.ts`). A missing module (`ERR_MODULE_NOT_FOUND`) leaves Azure unconfigured; any other import error is not swallowed. With the module, tokens come from `getBearerTokenProvider(new DefaultAzureCredential(), 'https://cognitiveservices.azure.com/.default')`, looked up again when they expire. ## Answer schemas diff --git a/packages/object-storage/README.md b/packages/object-storage/README.md index 352a0c323..c26f33a36 100644 --- a/packages/object-storage/README.md +++ b/packages/object-storage/README.md @@ -11,7 +11,7 @@ The content port of [`@on.auto/operations`](../operations/README.md#recorded-con - `objectStoreReached(client, location)` asks the store for the bucket with `HeadBucket` and fails with `ObjectStoreUnreachable`, `{ bucket, answer }`, when it cannot: the answer is the store's, its name and status, such as `NotFound (404)` or `AccessDenied (403)`, or the failure of the request itself, such as `Error: connect ECONNREFUSED 127.0.0.1:9000` or `Error: Region is missing`; it carries no credential. - `objectContentOn(client, location, waits?)` answers a `ContentStore` of `@on.auto/operations`, `{ content, sweepContent }`: `content`, the port, and `sweepContent(before)`, the sweep of uploads below. `client` is a function that answers the client, so that nothing passes the client itself, whose fields are not read-only, as a parameter. -The server composes them in `packages/server/src/composition/content-store.ts`: the location and the endpoint from the content store settings, the region from `AWS_REGION`, the environment proxy as the server reads it once for every client it makes, and the credentials from `awsCredentialChain` of `@on.auto/outbound/aws`, the default credential chain of the AWS SDK that the Bedrock provider of the reasoning capability uses too. It imports the entry of the adapter only when the settings name a bucket, so a server whose content is kept in the ledger never loads `@aws-sdk/client-s3` and `@aws-sdk/lib-storage`, which were 26 files and 0.7 MiB of its start, and 11 ms of it in a CPU profile (`packages/server/src/composition/content-entries.test.ts`; measured on 2026-10-23, at the median of 11 starts and of 7 profiles). Before the ledger opens, `packages/server/src/composition/ledger-store.ts` checks the bucket with `objectStoreReached`, and a bucket it cannot reach stops the server at start, naming `CONTENT_STORE_URL`, the bucket, the endpoint's host and the store's answer. +The server composes them in `packages/server/src/composition/content-store.ts`: the location and the endpoint from the content store settings, the region from `AWS_REGION`, the environment proxy as the server reads it once for every client it makes, and the credentials from `awsCredentialChainOf` of `@on.auto/outbound`, the default credential chain of the AWS SDK that the Bedrock provider of the reasoning capability uses too, loaded with the adapter. It imports the entry of the adapter only when the settings name a bucket, so a server whose content is kept in the ledger never loads `@aws-sdk/client-s3` and `@aws-sdk/lib-storage`, which were 26 files and 0.7 MiB of its start, and 11 ms of it in a CPU profile (`packages/server/src/composition/content-entries.test.ts`; measured on 2026-10-23, at the median of 11 starts and of 7 profiles). Before the ledger opens, `packages/server/src/composition/ledger-store.ts` checks the bucket with `objectStoreReached`, and a bucket it cannot reach stops the server at start, naming `CONTENT_STORE_URL`, the bucket, the endpoint's host and the store's answer. `@on.auto/object-storage/testing` (`src/testing/index.ts`) is the entry the tests of this package and of the server take a real S3-compatible server through, so the reading of its URL and the making of its bucket are written once: `testObjectStoreOf(url)` reads `http://access:secret@host:port/bucket` into the bucket, with a client of its own, the endpoint and the credentials; `bucketReady(bucket)` makes the bucket when it is missing, and finds it when another test made it at the same moment; `keysIn(bucket, prefix)` and `openUploadsIn(bucket, prefix)` list what a prefix holds, and `emptied(bucket, prefix)` aborts and deletes it. diff --git a/packages/outbound/README.md b/packages/outbound/README.md index 56e3b496d..31ce60296 100644 --- a/packages/outbound/README.md +++ b/packages/outbound/README.md @@ -22,7 +22,7 @@ How the server calls systems on the internet. It walks the causes of a failed ca `proxySettingsOf(environment)`, in the main entry, reads `ProxySettings`, `{ enabled, environment }`: the switch, `NODE_USE_ENV_PROXY=1`, and the proxy's variables that are set, `HTTPS_PROXY`, `HTTP_PROXY` and `NO_PROXY` in either case. The server reads them once and hands them to every client it makes, the model providers', the object store's, the KMS custodian's and the Parameter Store keeper's. -`@on.auto/outbound/aws` (`src/aws/aws-clients.ts`) is a subpath of its own, so that the packages that take the classification of certificates alone, the tool-servers package among them, load no part of the AWS SDK. `awsCredentialChain(proxy)` is the default credential chain of the AWS SDK, `fromNodeProviderChain` of `@aws-sdk/credential-providers`, which resolves credentials from the environment, the shared files, single sign-on, a process, a web identity, the container or the instance, looking nothing up until a call needs them. `awsClientSettingsOf(proxy)` is the part of a client's settings that sends its requests through the environment proxy, `HTTPS_PROXY`, `HTTP_PROXY` and `NO_PROXY`, when Node is told to with `NODE_USE_ENV_PROXY=1`, by the `proxyEnv` of the agents of its request handler, since a client of the AWS SDK makes agents of its own that the switch alone does not reach; without the switch it is empty. The chain's own clients take the same settings, by HTTP as well as by HTTPS: the agent of each carries the proxy's variables, so a credential endpoint reached by HTTP, such as a container's, goes through the proxy too unless `NO_PROXY` names it, where the reasoning capability's chain once set the HTTPS agent alone. +`@on.auto/outbound/aws` (`src/aws/aws-clients.ts`) is a subpath of its own, so that the packages that take the classification of certificates alone, the tool-servers package among them, load no part of the AWS SDK. The main entry gives the chain to the rest: `awsCredentialChainOf(proxy)` (`src/aws/aws-credentials.ts`) imports the subpath when it is first asked and answers its `awsCredentialChain(proxy)`, so a server loads `@aws-sdk/credential-providers` and `@aws-sdk/rds-signer` only with the first client its settings name that asks for it, the Bedrock provider, the object store, the KMS custodian or the Parameter Store keeper, and with the pools of a ledger in PostgreSQL, which import the subpath for the signer below; `awsCredentialVariables`, the variables of the chain the server reads with its settings, is in the main entry too. Without them a server on SQLite loads 6 files fewer at start, and spends 6 ms less on them in a CPU profile (`src/aws/aws-credentials.test.ts`; measured on 2026-10-23, at the median of 11 starts and of 7 profiles). `awsCredentialChain(proxy)` is the default credential chain of the AWS SDK, `fromNodeProviderChain` of `@aws-sdk/credential-providers`, which resolves credentials from the environment, the shared files, single sign-on, a process, a web identity, the container or the instance, looking nothing up until a call needs them. `awsClientSettingsOf(proxy)` is the part of a client's settings that sends its requests through the environment proxy, `HTTPS_PROXY`, `HTTP_PROXY` and `NO_PROXY`, when Node is told to with `NODE_USE_ENV_PROXY=1`, by the `proxyEnv` of the agents of its request handler, since a client of the AWS SDK makes agents of its own that the switch alone does not reach; without the switch it is empty. The chain's own clients take the same settings, by HTTP as well as by HTTPS: the agent of each carries the proxy's variables, so a credential endpoint reached by HTTP, such as a container's, goes through the proxy too unless `NO_PROXY` names it, where the reasoning capability's chain once set the HTTPS agent alone. `awsDatabaseTokenSigner(proxy, { hostname, port, username, region })` signs the password of a connection to a database of Amazon RDS that authenticates its users with IAM: each call signs a token of its own with the `Signer` of `@aws-sdk/rds-signer`, pinned to the version of `@aws-sdk/credential-providers`, a request to connect as that user to that host and port, presigned for the region and valid for 15 minutes, from the credentials of `awsCredentialChain(proxy)`, which it looks up when it is first asked for a token and not when it is made. The signer itself sends no request; the credential chain does, through the proxy. The server's pools of the ledger's PostgreSQL database ask it for a token at each connection they open, so each connection signs in with a token of its own. diff --git a/packages/outbound/src/aws/aws-clients.ts b/packages/outbound/src/aws/aws-clients.ts index 375780080..dcb65a3a4 100644 --- a/packages/outbound/src/aws/aws-clients.ts +++ b/packages/outbound/src/aws/aws-clients.ts @@ -22,14 +22,6 @@ export interface DatabaseTokenRequest { export type DatabaseTokenSigner = () => Promise; -export const awsCredentialVariables: readonly string[] = [ - 'AWS_ACCESS_KEY_ID', - 'AWS_SECRET_ACCESS_KEY', - 'AWS_SESSION_TOKEN', - 'AWS_CONTAINER_AUTHORIZATION_TOKEN', - 'AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE', -]; - export function awsClientSettingsOf(proxy: ProxySettings): AwsClientSettings { if (!proxy.enabled) { return {}; diff --git a/packages/outbound/src/aws/aws-credentials.test.ts b/packages/outbound/src/aws/aws-credentials.test.ts new file mode 100644 index 000000000..18823378d --- /dev/null +++ b/packages/outbound/src/aws/aws-credentials.test.ts @@ -0,0 +1,24 @@ +import { createRequire } from 'node:module'; + +import { describe, expect, it } from 'vitest'; + +import { awsCredentialChainOf } from '../index.ts'; + +const modulesLoaded = createRequire(import.meta.url).cache; + +function credentialProvidersLoaded(): readonly string[] { + return Object.keys(modulesLoaded).filter((path) => path.includes('/@aws-sdk/credential-providers/')); +} + +describe('the default credential chain of AWS, from the main entry', () => { + it('loads the credential providers of the AWS SDK when it is first asked for, and not with the entry', async () => { + expect(credentialProvidersLoaded()).toEqual([]); + + const chain = await awsCredentialChainOf({ enabled: false, environment: {} }); + + expect(typeof chain).toBe('function'); + expect(credentialProvidersLoaded()).toContainEqual( + expect.stringContaining('/@aws-sdk/credential-providers/dist-cjs/'), + ); + }); +}); diff --git a/packages/outbound/src/aws/aws-credentials.ts b/packages/outbound/src/aws/aws-credentials.ts new file mode 100644 index 000000000..d4b3aa6f3 --- /dev/null +++ b/packages/outbound/src/aws/aws-credentials.ts @@ -0,0 +1,15 @@ +import type { ProxySettings } from '../proxy/proxy-settings.ts'; +import type { AwsCredentialChain } from './aws-clients.ts'; + +export const awsCredentialVariables: readonly string[] = [ + 'AWS_ACCESS_KEY_ID', + 'AWS_SECRET_ACCESS_KEY', + 'AWS_SESSION_TOKEN', + 'AWS_CONTAINER_AUTHORIZATION_TOKEN', + 'AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE', +]; + +export async function awsCredentialChainOf(proxy: ProxySettings): Promise { + const { awsCredentialChain } = await import('./aws-clients.ts'); + return awsCredentialChain(proxy); +} diff --git a/packages/outbound/src/index.ts b/packages/outbound/src/index.ts index d2ecf4c25..db608968a 100644 --- a/packages/outbound/src/index.ts +++ b/packages/outbound/src/index.ts @@ -1,3 +1,4 @@ +export { awsCredentialChainOf, awsCredentialVariables } from './aws/aws-credentials.ts'; export { causesOf, codesOf } from './causes/error-causes.ts'; export { isUntrustedCertificate } from './certificates/untrusted-certificates.ts'; export { proxySettingsOf, proxySettingsVariables, type ProxySettings } from './proxy/proxy-settings.ts'; diff --git a/packages/server/README.md b/packages/server/README.md index b6c082ed5..e63eccc1b 100644 --- a/packages/server/README.md +++ b/packages/server/README.md @@ -6,7 +6,7 @@ The Node.js server of Auto Runtime: `src/main.ts` starts it on the composition r `src/settings/keeper-settings.ts` reads where the value of each secret rests: by default, or where `SECRETS_KEEPER` is `ledger`, sealed in the ledger's database, and where it is `aws-parameter-store:`, in AWS Systems Manager Parameter Store under that prefix, an absolute parameter path of at most 14 segments of letters, digits, `.`, `-` and `_`, the first beginning with neither `aws` nor `ssm`, and no slash at its end, in the region `AWS_REGION` names, or `AWS_DEFAULT_REGION` where that is not set, kept with the name of the variable it was read from; a keeper named with neither stops the start, naming `SECRETS_KEEPER` and `AWS_REGION`. `SECRETS_KEEPER_PREVIOUS` names the keeper being replaced, only beside `SECRETS_KEEPER` and the other of the two. -`src/settings/secrets-settings.ts` reads where the key that wraps the data key of each secret is held, while the ledger keeps secrets, as the current keeper or the previous one; with Parameter Store as the only keeper there is no custodian, a key setting given stops the start in words that say so, and no key file is made: in `SECRETS_KEY`, in the file `SECRETS_KEY_FILE` names, by default `secrets.key` beside a SQLite ledger, or in AWS KMS, under the ARN of a key or of an alias that `SECRETS_KEY_KMS` names, whose region is the ARN's; one of the three at most, and at most one key being replaced beside it, `SECRETS_KEY_PREVIOUS`, `SECRETS_KEY_FILE_PREVIOUS` or `SECRETS_KEY_KMS_PREVIOUS`. `src/secrets/secret-store.ts` makes a custodian of each, `localCustodian` of [`@on.auto/secrets`](../secrets/README.md#the-custodian) for a key in hand and, through `src/secrets/kms-custodians.ts`, its `kmsCustodian` for a key in KMS, and `src/secrets/secret-keepers.ts` a keeper of each keeper named, the `sealedKeeper` of the custodians for the ledger and its `parameterStoreKeeper` for Parameter Store, both from AWS with the proxy settings and `awsCredentialChain(proxy)` of `@on.auto/outbound/aws`, loaded through `import('@on.auto/secrets/aws')` (`src/secrets/aws-adapters.ts`) only where a setting names a key in KMS or a keeper in Parameter Store, so a server without either loads no part of `@aws-sdk/client-kms` or `@aws-sdk/client-ssm`, as `src/secrets/aws-loading.test.ts` holds by the modules its process has loaded. The custodian of an alias resolves it to the key it names with one `DescribeKey` and is pinned to that key until the next start, and a resolution that fails stops the start, `The secrets settings are invalid. SECRETS_KEY_KMS: The KMS alias alias/ it names could not be resolved to its key at start: `. `src/database/ledger-stores.ts` makes them and starts the store through `src/secrets/secret-store-start.ts`, which probes a current key in KMS before it opens the rows, one data key wrapped, unwrapped and filled with zeros, and a probe that fails stops the start, `The secrets settings are invalid. SECRETS_KEY_KMS: The KMS key it names could not wrap and unwrap a data key at start: `; probes a current keeper in Parameter Store with `keeperProbed`, one parameter, `/probe-`, written, read and deleted, and a probe that fails stops the start, `The secrets settings are invalid. SECRETS_KEEPER: AWS Parameter Store under could not write, read and delete a parameter at start: `; then moves the secrets the previous keeper keeps to the current one, a move that fails stopping the start, `… SECRETS_KEEPER_PREVIOUS: The secrets its keeper keeps could not be moved to the keeper SECRETS_KEEPER names: `, rotates, and logs where secrets are kept: `… sealed under the key that the KMS key holds, and sealed under the previous key were sealed again under it`, with `, and kept by the previous keeper were moved there` after it where a previous keeper is named, or `Secrets are kept in AWS Parameter Store under in , and kept by the previous keeper were moved there`. The line names a prefix, a region and a key's id, never a value, a key or a credential. +`src/settings/secrets-settings.ts` reads where the key that wraps the data key of each secret is held, while the ledger keeps secrets, as the current keeper or the previous one; with Parameter Store as the only keeper there is no custodian, a key setting given stops the start in words that say so, and no key file is made: in `SECRETS_KEY`, in the file `SECRETS_KEY_FILE` names, by default `secrets.key` beside a SQLite ledger, or in AWS KMS, under the ARN of a key or of an alias that `SECRETS_KEY_KMS` names, whose region is the ARN's; one of the three at most, and at most one key being replaced beside it, `SECRETS_KEY_PREVIOUS`, `SECRETS_KEY_FILE_PREVIOUS` or `SECRETS_KEY_KMS_PREVIOUS`. `src/secrets/secret-store.ts` makes a custodian of each, `localCustodian` of [`@on.auto/secrets`](../secrets/README.md#the-custodian) for a key in hand and, through `src/secrets/kms-custodians.ts`, its `kmsCustodian` for a key in KMS, and `src/secrets/secret-keepers.ts` a keeper of each keeper named, the `sealedKeeper` of the custodians for the ledger and its `parameterStoreKeeper` for Parameter Store, both from AWS with the proxy settings and the chain of `awsCredentialChainOf(proxy)` of `@on.auto/outbound`, loaded with them, through `import('@on.auto/secrets/aws')` (`src/secrets/aws-adapters.ts`), only where a setting names a key in KMS or a keeper in Parameter Store, so a server without either loads no part of `@aws-sdk/client-kms` or `@aws-sdk/client-ssm`, as `src/secrets/aws-loading.test.ts` holds by the modules its process has loaded. The custodian of an alias resolves it to the key it names with one `DescribeKey` and is pinned to that key until the next start, and a resolution that fails stops the start, `The secrets settings are invalid. SECRETS_KEY_KMS: The KMS alias alias/ it names could not be resolved to its key at start: `. `src/database/ledger-stores.ts` makes them and starts the store through `src/secrets/secret-store-start.ts`, which probes a current key in KMS before it opens the rows, one data key wrapped, unwrapped and filled with zeros, and a probe that fails stops the start, `The secrets settings are invalid. SECRETS_KEY_KMS: The KMS key it names could not wrap and unwrap a data key at start: `; probes a current keeper in Parameter Store with `keeperProbed`, one parameter, `/probe-`, written, read and deleted, and a probe that fails stops the start, `The secrets settings are invalid. SECRETS_KEEPER: AWS Parameter Store under could not write, read and delete a parameter at start: `; then moves the secrets the previous keeper keeps to the current one, a move that fails stopping the start, `… SECRETS_KEEPER_PREVIOUS: The secrets its keeper keeps could not be moved to the keeper SECRETS_KEEPER names: `, rotates, and logs where secrets are kept: `… sealed under the key that the KMS key holds, and sealed under the previous key were sealed again under it`, with `, and kept by the previous keeper were moved there` after it where a previous keeper is named, or `Secrets are kept in AWS Parameter Store under in , and kept by the previous keeper were moved there`. The line names a prefix, a region and a key's id, never a value, a key or a credential. ## The runtime's own keys diff --git a/packages/server/src/composition/aws-entries.test.ts b/packages/server/src/composition/aws-entries.test.ts new file mode 100644 index 000000000..97e453f58 --- /dev/null +++ b/packages/server/src/composition/aws-entries.test.ts @@ -0,0 +1,19 @@ +import { fileURLToPath } from 'node:url'; + +import { modulesReachedFrom } from '@on.auto/operations/testing'; +import { describe, expect, it } from 'vitest'; + +const sources = fileURLToPath(new URL('..', import.meta.url)); + +function awsClientsReachedFrom(module: string): readonly string[] { + return modulesReachedFrom(sources, module).packages.filter((name) => name === '@on.auto/outbound/aws'); +} + +describe('the AWS clients of the outbound package', () => { + it('are reached from neither module the server starts with, and load with the first client its settings name', () => { + expect([ + ...awsClientsReachedFrom('lifecycle/run-server.ts'), + ...awsClientsReachedFrom('composition/composition-root.ts'), + ]).toEqual([]); + }); +}); diff --git a/packages/server/src/composition/content-store.ts b/packages/server/src/composition/content-store.ts index 89ffa421a..0c61353f8 100644 --- a/packages/server/src/composition/content-store.ts +++ b/packages/server/src/composition/content-store.ts @@ -1,5 +1,5 @@ import type { ContentStore } from '@on.auto/operations'; -import { awsCredentialChain } from '@on.auto/outbound/aws'; +import { awsCredentialChainOf } from '@on.auto/outbound'; import { Effect } from 'effect'; import type { Settings } from '../settings/settings.ts'; @@ -29,10 +29,10 @@ export function contentStoreOf({ `CONTENT_STORE_URL names the bucket ${location.bucket} on ${hostOf(endpoint)}, which could not be reached: ${answer}`, ); return Effect.gen(function* () { - const { objectContentOn, objectStoreReached, storageClientOf } = yield* Effect.promise( - () => import('@on.auto/object-storage'), + const [{ objectContentOn, objectStoreReached, storageClientOf }, credentials] = yield* Effect.promise(() => + Promise.all([import('@on.auto/object-storage'), awsCredentialChainOf(proxy)]), ); - const client = storageClientOf({ endpoint, region, proxy }, awsCredentialChain(proxy)); + const client = storageClientOf({ endpoint, region, proxy }, credentials); client.middlewareStack.add(commandSpansIn(location.bucket), commandSpanStep); const composed: ComposedContentStore = { store: objectContentOn(() => client, location), diff --git a/packages/server/src/connections/variables-of-the-libraries.ts b/packages/server/src/connections/variables-of-the-libraries.ts index 72b21d112..3c92c0d44 100644 --- a/packages/server/src/connections/variables-of-the-libraries.ts +++ b/packages/server/src/connections/variables-of-the-libraries.ts @@ -1,8 +1,7 @@ import { configFileVariable, serverConfigVariables } from '@on.auto/config'; import { connectionsSetting } from '@on.auto/connections'; import { apiKeysVariable } from '@on.auto/identity'; -import { proxySettingsVariables } from '@on.auto/outbound'; -import { awsCredentialVariables } from '@on.auto/outbound/aws'; +import { awsCredentialVariables, proxySettingsVariables } from '@on.auto/outbound'; export const variablesOfTheLibraries: readonly string[] = [ ...serverConfigVariables, diff --git a/packages/server/src/secrets/kms-custodians.ts b/packages/server/src/secrets/kms-custodians.ts index eb328dd21..8ff406b89 100644 --- a/packages/server/src/secrets/kms-custodians.ts +++ b/packages/server/src/secrets/kms-custodians.ts @@ -1,4 +1,4 @@ -import { awsCredentialChain } from '@on.auto/outbound/aws'; +import { awsCredentialChainOf } from '@on.auto/outbound'; import type { Custodian } from '@on.auto/secrets'; import { Effect } from 'effect'; @@ -16,13 +16,8 @@ function unresolved({ setting, arn }: KeyInKms, detail: string): InvalidSettings } export async function kmsCustodianOf(source: KeyInKms, { proxy, loadAws }: AwsMaking): Promise { - const { kmsCustodian } = await loadAws(); - const made = kmsCustodian({ - keyId: source.arn, - region: source.region, - proxy, - credentials: awsCredentialChain(proxy), - }); + const [{ kmsCustodian }, credentials] = await Promise.all([loadAws(), awsCredentialChainOf(proxy)]); + const made = kmsCustodian({ keyId: source.arn, region: source.region, proxy, credentials }); return Effect.runPromise( Effect.mapError(made, ({ detail }: { readonly detail: string }) => unresolved(source, detail)), ); diff --git a/packages/server/src/secrets/secret-keepers.ts b/packages/server/src/secrets/secret-keepers.ts index abfc4de6c..573157328 100644 --- a/packages/server/src/secrets/secret-keepers.ts +++ b/packages/server/src/secrets/secret-keepers.ts @@ -1,4 +1,4 @@ -import { awsCredentialChain } from '@on.auto/outbound/aws'; +import { awsCredentialChainOf } from '@on.auto/outbound'; import { sealedKeeper, type Custodians, type Keeper, type Keepers } from '@on.auto/secrets'; import { Option } from 'effect'; @@ -9,9 +9,12 @@ async function keeperOf(given: KeeperGiven, custodians: Custodians | undefined, if (given.kind === 'ledger') { return sealedKeeper(Option.getOrThrow(Option.fromNullishOr(custodians))); } - const { parameterStoreKeeper } = await making.loadAws(); + const [{ parameterStoreKeeper }, credentials] = await Promise.all([ + making.loadAws(), + awsCredentialChainOf(making.proxy), + ]); const { prefix, region } = given; - return parameterStoreKeeper({ prefix, region, proxy: making.proxy, credentials: awsCredentialChain(making.proxy) }); + return parameterStoreKeeper({ prefix, region, proxy: making.proxy, credentials }); } export async function keepersOf( diff --git a/packages/tool-servers/README.md b/packages/tool-servers/README.md index ceee2b163..6eb4c8423 100644 --- a/packages/tool-servers/README.md +++ b/packages/tool-servers/README.md @@ -4,7 +4,7 @@ How a playbook reaches the outside world: the tool servers of its org, MCP serve ## Entry points -- `@on.auto/tool-servers`: the settings, with `ServerEntrySchema`, the schema of one entry, `readEntry`, the reading of one entry of the file, with which `@on.auto/connections` reads a connection's entry, its credential placed, into the connection's `ServerSettings`, and whose context's `placedValues`, the values the runtime places by the name of their placeholder, stand in the entry's header and query values once the entry as written is judged, and whose context's `providers`, a `ProviderResolver`, resolves the provider an entry of the file names (see [An entry that names a provider](#an-entry-that-names-a-provider)), `noProviders` where it is left out; `givenByTheProvider(field)`, the words of a field of a server written beside a provider, and `secretOrKeyExpected`, the words of client credentials given neither a client secret nor a private key with its algorithm; `makeConsentClient`, the runtime acting as a client at the authorization server of a provider's way `authorization`, and the types of its contract, `ConsentClient`, `ClientRegistrations` and `ConnectionTokens`, the store of the tokens of a connection connected by consent, with `KeptRow`, a row as a store reads it, and `HeldTokens`, that row decoded, among them, and `keptTokensIn`, which decodes the value of a row, which `@on.auto/connections` calls and implements, and `ConsentCredential`, the third kind of the credential of `AuthSettings`, the tokens of a consent, which it composes, with `ClientSourceSchema`, `clientSources` and `isSameIssuer`, the client sources an entry may list, in the runtime's order, and the comparison of two issuers, `clientMetadataDocumentOf` and its `ClientMetadataDocument`, the document the runtime names its client by where `PUBLIC_URL` is https, which the server gives `@on.auto/api` to answer, `ClientIdSchema`, the rule of a client id the record and a registration's answer are held to, and `isScope`, the rule of a scope a provider's way and a grant are held to (see [The runtime as a client](#the-runtime-as-a-client)); `serverNameProblems`, the check of a server's name, with which it checks a connection's name, and `servedFieldsOf`, the playbooks, tools, `record_content` and `request_id` an entry is served by, as it writes them or by default where it leaves them out, with which it builds the serving of every connection it lists, and `isPinnableUrl` and `headerNamePattern`; `redactedValuesOf`, the values of a record each redacted under its key, which the reasoning capability takes for the headers and the query parameters of a gateway; `@on.auto/providers` takes the server part of a provider's entry from the fields of `ServerEntrySchema`, as a connection's entry does, and checks a provider's name with `serverNameProblems` and its addresses with `isPinnableUrl`; `namesEveryTool`, whether a tool reference names every tool of its server with `*`, as a reasoning function's `tools` may; the contract of a source of connections, `ConnectionSource` and its types, which `@on.auto/connections` implements, and `noConnections`; `describedToolServers`, `makeToolAccess`, the five operations, `defineSearch` and its org twin `defineSearchInOrg`, with `ProviderOffering` and `OfferedProvider`, the contract of the providers offered to a caller, which `@on.auto/connections` implements, `defineIntrospect`, `defineValidate`, `defineDryRun` and `defineExecute`, `handCallPresenter`, which shows a call made by hand, and the facts of a call that `@on.auto/definitions` builds the run's events from. The access loads the MCP client, its OAuth providers and `node:child_process` (`src/access/linked-access.ts`) through a dynamic import when a run first opens its tools, a search or a read first asks a server or `execute` first calls a tool, so a server that never does any of them never loads them: the server's start loaded 55 files and 0.94 MiB more with them (measured 2026-10-06 with a module load hook, 1,241 files against main's 1,186). Without them it loads 84 files and 236 KiB of this package, the five operations, the reading of the settings of the three types and of an entry that names a provider, `describedToolServers` and the naming of the servers of an org and of its connections among them (measured 2026-10-20 with a module load hook over both entries, the sizes of the files as they are on disk, against 82 files and 219 KiB before the switch of Auto's consent service and the listing of its clients, `src/consent-through-auto/auto-clients.ts`, and the contract they call, `src/consent/consent-service.ts`, came to load with the consent client, against 82 files and 217 KiB before the metadata of the runtime's client, which its document and a dynamic registration share, was written in `src/consent/client-sources.ts`, against 81 files and 211 KiB before the tokens of a consent were served, which added `src/consent-tokens/kept-tokens.ts`, the value a token source of a consent reads, and against 77 files and 192 KiB before an entry could name a provider); the reading resolves a provider through the resolver its context gives, so it loads no code of a provider's; the reading of an OpenAPI document, its decoding and the tools derived from it (`src/openapi`, `described-servers.ts` and the types aside) load through a dynamic import too, when an entry of type `openapi` is described at start. The server's start loads neither the TypeScript compiler, which the check worker of `@on.auto/definitions` loads with the first check, nor the driver of a store its settings do not name: measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, by the same hook on the thread that serves requests, from the start of the process to its ready line, at the median of 11 starts interleaved with main's, it loaded on SQLite 2,504 files and 21.0 MiB in 802 ms, against main's 2,573 files and 30.4 MiB in 1,143 ms, at a load average of 20 to 13, and on PostgreSQL 2,551 files and 19.5 MiB in 810 ms, against 2,573 files and 30.4 MiB in 1,134 ms, at 11 to 7. The compiler was 12 files, 8.8 MiB and 318 to 328 ms of it, the warming of the check worker among them, which the first save of a program now pays: 245 ms against 17 ms at the median of 9 servers interleaved with main's, and 6.7 ms against 5.9 ms for the second, at a load average of about 5. The driver of the other store, `pg` with the PostgreSQL entry on SQLite and `sqlite3` with its native addon on PostgreSQL, was 57 files and 0.6 MiB, and 10 files and 2.1 MiB, with a time, 13 and 6 ms, within the spread of the starts. +- `@on.auto/tool-servers`: the settings, with `ServerEntrySchema`, the schema of one entry, `readEntry`, the reading of one entry of the file, with which `@on.auto/connections` reads a connection's entry, its credential placed, into the connection's `ServerSettings`, and whose context's `placedValues`, the values the runtime places by the name of their placeholder, stand in the entry's header and query values once the entry as written is judged, and whose context's `providers`, a `ProviderResolver`, resolves the provider an entry of the file names (see [An entry that names a provider](#an-entry-that-names-a-provider)), `noProviders` where it is left out; `givenByTheProvider(field)`, the words of a field of a server written beside a provider, and `secretOrKeyExpected`, the words of client credentials given neither a client secret nor a private key with its algorithm; `makeConsentClient`, the runtime acting as a client at the authorization server of a provider's way `authorization`, and the types of its contract, `ConsentClient`, `ClientRegistrations` and `ConnectionTokens`, the store of the tokens of a connection connected by consent, with `KeptRow`, a row as a store reads it, and `HeldTokens`, that row decoded, among them, and `keptTokensIn`, which decodes the value of a row, which `@on.auto/connections` calls and implements, and `ConsentCredential`, the third kind of the credential of `AuthSettings`, the tokens of a consent, which it composes, with `ClientSourceSchema`, `clientSources` and `isSameIssuer`, the client sources an entry may list, in the runtime's order, and the comparison of two issuers, `clientMetadataDocumentOf` and its `ClientMetadataDocument`, the document the runtime names its client by where `PUBLIC_URL` is https, which the server gives `@on.auto/api` to answer, `ClientIdSchema`, the rule of a client id the record and a registration's answer are held to, and `isScope`, the rule of a scope a provider's way and a grant are held to (see [The runtime as a client](#the-runtime-as-a-client)); `serverNameProblems`, the check of a server's name, with which it checks a connection's name, and `servedFieldsOf`, the playbooks, tools, `record_content` and `request_id` an entry is served by, as it writes them or by default where it leaves them out, with which it builds the serving of every connection it lists, and `isPinnableUrl` and `headerNamePattern`; `redactedValuesOf`, the values of a record each redacted under its key, which the reasoning capability takes for the headers and the query parameters of a gateway; `@on.auto/providers` takes the server part of a provider's entry from the fields of `ServerEntrySchema`, as a connection's entry does, and checks a provider's name with `serverNameProblems` and its addresses with `isPinnableUrl`; `namesEveryTool`, whether a tool reference names every tool of its server with `*`, as a reasoning function's `tools` may; the contract of a source of connections, `ConnectionSource` and its types, which `@on.auto/connections` implements, and `noConnections`; `describedToolServers`, `makeToolAccess`, the five operations, `defineSearch` and its org twin `defineSearchInOrg`, with `ProviderOffering` and `OfferedProvider`, the contract of the providers offered to a caller, which `@on.auto/connections` implements, `defineIntrospect`, `defineValidate`, `defineDryRun` and `defineExecute`, `handCallPresenter`, which shows a call made by hand, and the facts of a call that `@on.auto/definitions` builds the run's events from. The access loads the MCP client, its OAuth providers and `node:child_process` (`src/access/linked-access.ts`) through a dynamic import when a run first opens its tools, a search or a read first asks a server or `execute` first calls a tool, so a server that never does any of them never loads them: the server's start loaded 55 files and 0.94 MiB more with them (measured 2026-10-06 with a module load hook, 1,241 files against main's 1,186). Without them it loads 84 files and 236 KiB of this package, the five operations, the reading of the settings of the three types and of an entry that names a provider, `describedToolServers` and the naming of the servers of an org and of its connections among them (measured 2026-10-20 with a module load hook over both entries, the sizes of the files as they are on disk, against 82 files and 219 KiB before the switch of Auto's consent service and the listing of its clients, `src/consent-through-auto/auto-clients.ts`, and the contract they call, `src/consent/consent-service.ts`, came to load with the consent client, against 82 files and 217 KiB before the metadata of the runtime's client, which its document and a dynamic registration share, was written in `src/consent/client-sources.ts`, against 81 files and 211 KiB before the tokens of a consent were served, which added `src/consent-tokens/kept-tokens.ts`, the value a token source of a consent reads, and against 77 files and 192 KiB before an entry could name a provider); the reading resolves a provider through the resolver its context gives, so it loads no code of a provider's; the reading of an OpenAPI document, its decoding and the tools derived from it (`src/openapi`, `described-servers.ts` and the types aside) load through a dynamic import too, when an entry of type `openapi` is described at start. The server's start loads neither the TypeScript compiler, which the check worker of `@on.auto/definitions` loads with the first check, nor the driver of a store its settings do not name: measured on 2026-10-23 on one machine of 16 cores, with Node 26.10.0, by the same hook on the thread that serves requests, from the start of the process to its ready line, at the median of 11 starts interleaved with main's, it loaded on SQLite 2,504 files and 21.0 MiB in 802 ms, against main's 2,573 files and 30.4 MiB in 1,143 ms, at a load average of 20 to 13, and on PostgreSQL 2,551 files and 19.5 MiB in 810 ms, against 2,573 files and 30.4 MiB in 1,134 ms, at 11 to 7. The compiler was 12 files, 8.8 MiB and 318 to 328 ms of it, the warming of the check worker among them, which the first save of a program now pays: 245 ms against 17 ms at the median of 9 servers interleaved with main's, and 6.7 ms against 5.9 ms for the second, at a load average of about 5. The driver of the other store, `pg` with the PostgreSQL entry on SQLite and `sqlite3` with its native addon on PostgreSQL, was 57 files and 0.6 MiB, and 10 files and 2.1 MiB, with a time, 13 and 6 ms, within the spread of the starts. Nor does it load the SDK of the workflow specification and the validator of its schema, which the coordination capability imports with the first document it reads, the S3 client, which the server imports when its settings name a bucket, `google-auth-library` with the AI SDK's Vertex providers, which model access imports when Vertex is configured, or the credential chain of AWS, which the first client that asks for it imports: measured on 2026-10-23 on the same machine, with Node 26.10.0, by the same hook, at the median of 11 starts on SQLite interleaved with main's, at a load average of 5 to 4, it loaded 2,307 files and 18.5 MiB in 654 ms, against main's 2,636 files and 21.4 MiB in 821 ms. The SDK and the validator were 222 files, 1.4 MiB and 130 ms of it, the SDK's 171 validators and Ajv among them, which the first save of a workflow now pays, 175 ms against 34 ms at the median of 9 servers; the S3 client was 26 files, 0.7 MiB and 13 ms, within the spread of the starts, and the Google library and the credential chain 81 files, 0.9 MiB and 24 ms. A CPU profile of 7 starts of each, sampled every 100 µs, gave the SDK with Ajv 115 ms of main's start, the validator 14 ms, the S3 client 11 ms, the Google library 10 ms and the credential chain 6 ms. - `@on.auto/tool-servers/policy`: the pure helpers a caller needs without connecting to anything (`toolReferenceOf`, `toolReferenceShape`, `writtenOf`, `namesEveryTool`, `runBoundMs` and `runIdKey`). It loads no transport code, so the reasoning function adapter, which parses a function's `tools` and bounds its run, does not load the MCP client wherever it is bundled; it takes `ToolAccess` as a type only. - `@on.auto/tool-servers/testing`: the fake server and the helpers of the tests (see Testing).