Skip to content

refactor(global): shipped files by package and path, the sandbox as a module, exports a shell needs - #306

Merged
rami-hatoum merged 6 commits into
mainfrom
refactor/the-core-relies-on-standard-web-apis
Oct 23, 2026
Merged

rami-hatoum merged 6 commits into
mainfrom
refactor/the-core-relies-on-standard-web-apis

Conversation

@rami-hatoum

Copy link
Copy Markdown
Contributor

What this changes

Six small seams a program that composes the core outside the server needs, each on its own terms.

  • The sandbox's module is given as bytes or already compiled. The engine compiled its WebAssembly from bytes it read itself; an instance source now takes a SandboxModule, bytes to compile once or a module a host compiled, and a pool worker reads its handed bytes only when its first unit asks.
  • A stdio server's transport loads with the first one opened, as the MCP client already does, so a server that configures none never loads it.
  • A shipped file is named by its package and its path within it, { package, path }, rather than by a URL computed from a module's address at load, which collapses when modules are bundled together. Each package declares its roots once and exports them; a third-party file, the engine's WebAssembly and the compiler's lib, is named under its dependency at the root the dependent resolves; the served documentation is docs of the root package; the server builds the map of roots in one place and the disk reader resolves nothing itself.
  • Encoding and byte APIs are explicit where a choice is open, at ten sites: the decoder options both named, a worker started from a URL, a file handle opened on first read, and the rest the commit message names, each the smallest form two type sets accept.
  • Exports a shell needs: the secret rows on SQLite, the check handler on its own subpath so no other entry reaches the compiler, the ledger's behaviour suite with its entry type from the ledger's testing entry, and the SQLite parts of the suite's entry over a caller's executor, none of which loads a driver.

Tests

  • A pre-compiled module and a late read of the handed bytes; the stdio transport not reached from the linked access; every shipped file read through the port with no URL computed at load; the exported parts opened on sqlite3 by the ledger's own tests, with the sqlite3 entry still the only module that loads the driver.
  • pnpm check passes on each commit, cold.

rami-hatoum and others added 6 commits October 23, 2026 17:56
…ule as bytes or compiled

The instance sources compiled the sandbox's WebAssembly from the bytes a reader of shipped files
read, where a host may give the module only as one it compiled. A source now takes a
SandboxModule, { bytes } that it compiles once, or { compiled }, a WebAssembly.Module it
instantiates as it is. sandboxModuleIn(shippedFiles) of the dsl entry reads the bytes, and a
worker of the pool makes its source over the bytes it was handed when its first unit asks for an
instance, so the check worker, which is handed the kept lib alone, reads none.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e first one opened

The access loaded the MCP client on demand, and with it the transport of a stdio server, the one
module of the package that imports node:child_process. The opener of a stdio link now imports its
client when it first opens one, so an access whose settings name no stdio server never loads
child processes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dger's suite are exported

Code that composes the packages on a driver or a worker of its own needed parts no entry
exported. The ledger's main entry exports sqliteSecretRowsOn, the rows of kept secrets on any of
Emmett's SQLite drivers; its testing entry exports ledgerBehaviour with its LedgerEntry; and
@on.auto/definitions/check-handler exports handedChecks, the handler the check worker serves, an
entry of its own since it is the one that reaches the compiler.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…xplicit where a choice is open

Ten uses type-checked against Node's declarations alone and failed against a stricter set of the
platform's, which requires both options of a decoder and keeps its own URL and stream types. Each
now takes the smallest form both accept, and behaves as before:

- identity issuer-requests.ts and operations streamed-content.ts: the fatal UTF-8 decoders also
  say ignoreBOM: false, the standard's default.
- prediction upload/utf8-text.ts: the decoder of a valid prefix also says fatal: false.
- providers reading/shipped-providers.ts: fileURLToPath takes the href of an entry's URL.
- workflow-engine process-pool/process-thread.ts: fileURLToPath takes the href of the module.
- workflow-engine program-pool/pool-threads.ts and evaluation-thread.ts: a worker starts from a
  URL of node:url made of its module's href, workerModuleOf, so a data: module still starts.
- workflow-engine instances/compiled-module.ts: the bounded memory is typed as the standard's
  WebAssembly.Memory is, with buffer, grow and the two buffer conversions Node 26 has.
- prediction files/series-rows.ts: a web stream becomes a byte Readable through Readable.from
  with objectMode false rather than Readable.fromWeb.
- prediction trainer/candidate-staging.ts: the candidate is staged as a ReadableStream of the
  standard's, read through a file handle opened on the first read and closed at its end or on a
  cancel, rather than Readable.toWeb, whose declared stream type is Node's own.

The engine's import check lets the program pool import node:url, as the process pool does.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… within it

A shipped file was a URL each package computed from import.meta.url as it loaded, which no
longer names the file once the modules are bundled into one. A ShippedFile is now
{ package, path }, as { package: '@on.auto/providers', path: 'src/entries/slack.yaml' }, and
shippedFileName gives the name it goes by in an error and among the files a pool hands a worker.

Each package that ships files declares its root once and exports it from the entry that reads
them: providersShippedRoots, apiShippedRoots and checkShippedRoots of @on.auto/definitions/check.
A dependency's file is read by its dependent under the dependency's name, at the root the
dependent resolves with import.meta.resolve: sandboxShippedRoots of the engine's dsl entry for
the sandbox's WebAssembly, and compilerShippedRoots for the lib of typescript6. The server's
recipes are files of @on.auto/server; the documentation it serves is docs of @on.auto/runtime,
two folders above the server's root, which the server declares too.

shippedFilesOnDisk(roots) resolves no package itself: the server gives it the roots it composes
from those the packages declare (src/server-options/shipped-roots.ts), and it refuses a file of
a package it was given no root for. shippedFilesInMemory holds its contents by package and path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r the caller's executor

A store opened on another of Emmett's SQLite drivers could not fill a LedgerEntry: its layout,
its plans and the check of the definition streams' index were built on the sqlite3 driver the
tests open. sqlitePlanOf(execute, read) and sqliteDefinitionStreamsIndexed(execute) now take a
SQLExecutor of dumbo the caller opens (src/recorded/sqlite-plans.ts), and the main entry exports
them with sqliteLayout and the types the entry names, LedgerStore, RecordedStore and
StatementExecutor. None of them loads a driver; the tests' own helpers open them on sqlite3, and
the sqlite3 entry stays the one module of the package that loads it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@rami-hatoum
rami-hatoum force-pushed the refactor/the-core-relies-on-standard-web-apis branch from 4b419b5 to 719c33f Compare October 23, 2026 17:06
@rami-hatoum
rami-hatoum merged commit 6062272 into main Oct 23, 2026
1 check passed
@rami-hatoum
rami-hatoum deleted the refactor/the-core-relies-on-standard-web-apis branch October 23, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant