diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5823a25..fbba4a6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,9 +7,9 @@ lowest layer that can prove it. ## Setup Install Node.js 24.15 or newer, enable Corepack, and install the locked -dependencies. The package supports Node.js 24.4.0 or newer, and CI runs the -default suite, the build, the packaged artifact smoke test, and the recovery -demo on that floor. Node.js 24.15 is where `node:sqlite` stops printing an +dependencies. The package supports Node.js 24.4.0 or newer. On that floor, CI +runs the default suite, the build, the packaged artifact smoke test, and the +recovery demo. Node.js 24.15 is where `node:sqlite` stops printing an experimental warning: ```bash diff --git a/README.md b/README.md index 2d985d5..7b63f03 100644 --- a/README.md +++ b/README.md @@ -3,23 +3,32 @@ [![CI](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml) [![npm](https://img.shields.io/npm/v/solid-objects)](https://www.npmjs.com/package/solid-objects) -Self-hosted, distributed Durable Objects in Node without a daemon using your -existing SQL database. +Open Source Durable Objects for JavaScript, in the SQL database you already run. +No daemon, no broker, and no new datastore. -Build addressable TypeScript objects with serialized calls and durable state -using SQLite, PostgreSQL, or MySQL, without deploying to Cloudflare. +Build addressable TypeScript objects with serialized calls and durable state on +SQLite, PostgreSQL, or MySQL. You don't need Cloudflare for this. Concurrent calls for one identity cannot overwrite each other. Calls for different identities can run at the same time. Define ordinary TypeScript classes and run them in ordinary Node.js processes. -State, queued operations, retries, reminders, effects, and realtime -invalidations are stored in the database the application already operates. - -> **Early release:** the correctness core has automated coverage across the -> supported databases, the Chromium browser client, process recovery, and -> packaged artifacts, but the TypeScript implementation is new. Read the -> [delivery boundaries](#delivery-boundaries) before using it for important data. +Solid Objects keeps the state, the queued operations, the retries, the +reminders, the effects, and the realtime invalidations in the database the +application already operates. + +> **Early release:** the correctness core has automated coverage. That coverage +> includes the supported databases, the Chromium browser client, process +> recovery, and the packaged artifacts. The TypeScript implementation is still +> new. There is one deployed first-party reference application. There is no +> measured scale and no third-party production use yet. Read the +> [delivery boundaries](#delivery-boundaries) before you use it for important +> data. + +> **Not a replacement for SQL transactions:** when one row update inside one +> transaction solves the problem, use that. Solid Objects earns its cost when an +> entity needs ordered calls across requests, retries, reminders, effects, and +> realtime state. See [Good and poor fits](#good-and-poor-fits). ## The programming model @@ -61,29 +70,31 @@ processes submit them concurrently. ## Run it now with SQLite Node.js 24.4.0 or newer is required. Node.js 24.15 or newer is preferred, -because `node:sqlite` prints an experimental warning before it. The `0.13.3` -release includes a packaged quickstart: +because `node:sqlite` prints an experimental warning before it. The published +package includes a quickstart: ```bash -npm exec --yes --package=solid-objects@0.13.3 -- solid-objects quickstart +npm exec --yes --package=solid-objects@latest -- solid-objects quickstart ``` The command needs no repository checkout, database server, Redis, container, or application configuration. It uses Node's built-in SQLite module and removes its scoped temporary database before exiting. -The executable asserts rather than merely printing a plausible result. In one -local run, it verifies that: +It states its plan first, prints the `Counter` class it runs, and asks for +permission. It executes the work only after you answer, and then it explains +what each result proves. It asks nothing when stdin is not a terminal, so CI +never waits. Add `--yes` to skip the question in a terminal, or `--json` for a +machine-readable summary. -- 25 concurrent calls to one identity produce the exact committed state `25`; -- their return values are the complete sequence from `1` through `25`; -- operations for two different identities overlap in time; and -- the runtime closes and temporary state is removed. +The executable asserts rather than merely printing a plausible result. It exits +with a non-zero code when one of those checks fails. ## What Solid Objects is for Use Solid Objects when more than one request, job, or process can act on the -same logical thing and the next action must use its latest committed state. +same logical thing. The next action must then use the latest committed state of +that thing. These are the stateful coordination patterns for which people often reach for Durable Objects: @@ -102,11 +113,46 @@ limiter or another very hot identity is a poor fit because it becomes an intentional bottleneck. If one ordinary row transaction solves the problem, prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. +## Measured behavior + +One developer machine, not a capacity promise. Apple M5, Node.js 24.18.0, 250 +measured operations at client concurrency 16, on August 22, 2026. PostgreSQL +17.11 and MySQL 9.7.1 run natively, not in a container. + +| Measurement | Result | +| ------------------------------------------------------------- | ---------------: | +| Committed operations per second, one hot identity, SQLite | 286 to 323 ops/s | +| The same identity across four processes, SQLite | 507 to 519 ops/s | +| The same identity across four processes, PostgreSQL | 266 to 331 ops/s | +| The same identity across four processes, MySQL | 214 to 228 ops/s | +| Idle wake-up to committed result, one process | 2.66 ms p50 | +| Idle wake-up to committed result, two processes, polling only | 1,006 ms p50 | +| Idle CPU per process, 100 ms fast interval | 0.121% | +| Idle database passes per second, after backoff | 4.0 | + +The four idle rows come from a separate harness on August 16, 2026. + +Each range spans the synchronous and the asynchronous handler shape. Calls to +one identity are serialized on purpose, so the per-call latency in these runs +includes the wait behind the other fifteen concurrent callers. Throughput is +the honest number for that case. + +The same PostgreSQL and MySQL versions in Docker Desktop reached 1.8x to 4.9x +less throughput on those rows. Measure your own deployment shape before you +plan capacity. + +The polling-only row is the tradeoff to know before you deploy: use PostgreSQL +notifications or the optional Redis Pub/Sub when separate processes need +low-latency delivery. + +Conditions, sources of bias, and the complete matrix for all three databases +are in [Benchmarks](docs/benchmarks.md). + ## Running in a deployed application [Shuffle Up and Play](https://shuffleupandplay.com/) is a deployed reference -application where two players create a table, load decks, and move cards while -realtime updates reach both browsers. Its +application. Two players create a table, load decks, and move cards. Realtime +updates reach both browsers. Its [source](https://github.com/cardmagic/shuffleupandplay) uses Node 24, TypeScript, SQLite, `node:http`, and `ws`. Each table code addresses one `GameRoom` actor that owns both seats, so mutations share one durable mailbox @@ -125,23 +171,23 @@ The application and its tests exercise more than a counter-shaped happy path: | Time and schema changes | The actor defines [versioned state migrations](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/src/actors/game-room.ts#L47-L91) and a [durable reminder](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/src/actors/game-room.ts#L276-L310). Tests load [stored version-one state](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/operations.test.ts#L135-L201) and [run the reminder scheduler](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/durability.test.ts#L236-L257). | | Operations and CI | The [operations tests](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/operations.test.ts#L39-L202) exercise doctor, process, retention, and reconciliation APIs; server suites cover the [dashboard](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/server.test.ts#L297-L326), [rate limits](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/rate-limit.test.ts), and [shutdown](https://github.com/cardmagic/shuffleupandplay/blob/519a343e8db0bb6eed961a2ffd374dba80d67cd6/test/shutdown.test.ts). The [current main CI run](https://github.com/cardmagic/shuffleupandplay/actions/runs/31963000789) passed typechecking, 171 tests, the build, the doctor, and a Docker image build. | -**Scope:** the checked-in deployment configuration runs one Node process using -SQLite on one Docker host. It demonstrates a real deployed workload, not a -measured traffic level or every supported topology. Its deck-import effect -reads an external API; effects that write to an external system still need a -stable idempotency key because delivery is at least once. The application -restart tests close the runtime cleanly; abrupt termination, PostgreSQL, MySQL, -and multi-process lease fencing are verified separately by the library's +**Scope:** the checked-in deployment configuration runs one Node process with +SQLite on one Docker host. It shows a real deployed workload. It does not show a +measured traffic level or every supported topology. Its deck-import effect reads +an external API. An effect that writes to an external system still needs a +stable idempotency key, because delivery is at least once. The restart tests +close the runtime cleanly. The library verifies abrupt termination, PostgreSQL, +MySQL, and multi-process lease fencing separately in its [test matrix](docs/support.md), [failure-recovery demonstration](examples/failure-recovery/demo.ts), and -[correctness contract](docs/correctness.md). Evaluate those guarantees and -limits against your own workload. +[correctness contract](docs/correctness.md). Compare those guarantees and limits +with your own workload. ## How it works -An object is addressed by its TypeScript class and application-defined ID. -Public fields are JSON state, public methods are durable operations, and public -getters are ordered queries. +Solid Objects addresses an object by its TypeScript class and its +application-defined ID. Public fields are JSON state, public methods are durable +operations, and public getters are ordered queries. For each identity, Solid Objects: @@ -157,9 +203,9 @@ claimed message. A worker that finishes JavaScript after losing its lease cannot commit. See the executable [failure-recovery demonstration](examples/failure-recovery/demo.ts) and the full [architecture](docs/architecture.md). -Redis is optional wake-up infrastructure. It can reduce notification latency -for a multi-process MySQL deployment, but the relational database remains the -durable source of truth and polling remains the recovery path. +Redis is optional wake-up infrastructure. It can reduce notification latency in +a multi-process MySQL deployment. The relational database stays the durable +source of truth, and polling stays the recovery path. Idle roles back off from the configured 100 ms fast polling interval to one second. Processed work and wake-up notifications reset that interval @@ -230,7 +276,7 @@ class Room extends Actor { ``` `version` crosses the shared invalidation channel. `hands` contributes only its -name when its real value changes, allowing a reauthorized component endpoint to +name when its real value changes. A reauthorized component endpoint can then render subscriber-specific state without a manual revision counter. The browser package handles replay, reconnection, incarnation/revision fences, @@ -243,19 +289,31 @@ provide authentication, WebSocket transport, and rendering. See the These systems solve different coordination problems. The table describes their default unit and deployment model, not a quality ranking. -| Approach | Serialization and state unit | Durable substrate | Additional runtime | Recovery model | Placement | -| --------------------------- | ----------------------------------------------------- | ------------------------------------- | ------------------------------------------------------- | ---------------------------------------------- | ---------------------------- | -| SQL transaction or row lock | Selected rows in one transaction | Application database | None | Application retries the transaction | Application deployment | -| Traditional job queue | Job or queue; ordering depends on queue configuration | Broker or queue database | Queue workers and usually a broker | Retry the job | Application deployment | -| Solid Objects | TypeScript class plus object ID | Existing SQLite, PostgreSQL, or MySQL | Library in application processes | Retry the per-ID operation from durable state | Application deployment | -| Cloudflare Durable Objects | Object class plus globally unique ID | Per-object managed storage | Cloudflare Workers platform | Managed object activation | Cloudflare-selected location | -| Rivet Actors | Addressable actor | Actor state, KV, or per-actor SQLite | Rivet Engine or managed compute | Actor sleep, wake, and persistence | Configured Rivet deployment | -| DBOS | Workflow ID and checkpointed steps | PostgreSQL system database | Library; Conductor recommended for distributed recovery | Deterministic workflow replay from checkpoints | Application deployment | -| Restate | Service handler or keyed virtual object | Restate log and state store | Restate server or cloud service | Durable handler execution and journal replay | Restate deployment | - -The sourced, dimension-by-dimension comparison—including realtime projections, -edge placement, cross-identity transactions, and operational data access—is in -[docs/comparisons.md](docs/comparisons.md). +| Approach | Serialization and state unit | Durable substrate | Additional runtime | Recovery model | Placement | +| --------------------------- | ----------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------- | +| SQL transaction or row lock | Selected rows in one transaction | Application database | None | Application retries the transaction | Application deployment | +| Traditional job queue | Job or queue; ordering depends on queue configuration | Broker or queue database | Queue workers and usually a broker | Retry the job | Application deployment | +| Solid Objects | TypeScript class plus object ID | Existing SQLite, PostgreSQL, or MySQL | Library in application processes | Retry the per-ID operation from durable state | Application deployment | +| Cloudflare Durable Objects | Object class plus globally unique ID | Per-object managed storage | Cloudflare Workers platform | Managed object activation | Cloudflare-selected location | +| celld | Object class plus object name | Per-object SQLite replicated to a bucket you own | celld daemon that embeds V8 and runs Wrangler bundles | A new owner restores the object database from the bucket | Any node in your fleet, chosen by bucket compare-and-swap | +| Rivet Actors | Addressable actor | Actor state, KV, or per-actor SQLite | Rivet Engine or managed compute | Actor sleep, wake, and persistence | Configured Rivet deployment | +| DBOS | Workflow ID and checkpointed steps | PostgreSQL system database | Library; Conductor recommended for distributed recovery | Deterministic workflow replay from checkpoints | Application deployment | +| Restate | Service handler or keyed virtual object | Restate log and state store | Restate server or cloud service | Durable handler execution and journal replay | Restate deployment | + +celld and Solid Objects both self-host the Durable Objects model. The difference +is where the state lives and what you run. celld runs a daemon that embeds V8 +and executes Wrangler bundles. It gives each object its own SQLite database, +and it replicates that database to an object-storage bucket you own. Object +ownership moves between nodes through compare-and-swap on that bucket. Solid +Objects runs plain TypeScript classes inside your Node processes, adds no +daemon, and keeps object state in the SQL database the application already +operates. Choose celld to run Workers-format code across a fleet with +bucket-based placement. Choose Solid Objects to keep one database, no extra +process, and an ordinary Node deployment. + +[docs/comparisons.md](docs/comparisons.md) holds the sourced comparison for each +dimension: realtime projections, edge placement, cross-identity transactions, +and operational data access. ## Requirements and supported systems @@ -267,8 +325,8 @@ edge placement, cross-identity transactions, and operational data access—is in - optional `pg`, `mysql2`, or `redis` peer dependency only for the selected adapter -The exact CI matrix and boundaries are documented in -[Supported versions](docs/support.md). +[Supported versions](docs/support.md) records the exact CI matrix and the +boundaries. ## Operations @@ -277,10 +335,10 @@ and stale-process recovery roles. The database-backed operator dashboard is an optional `solid-objects/web` export with deny-by-default administration policy, session-backed CSRF protection, and Fetch or Node/Connect mounting. -The dashboard defaults to authorized read/write access. An authorized -read-only mode removes mutations, while an explicitly public read-only mode is -appropriate only for synthetic demo data because it exposes stored arguments, -results, errors, identifiers, and operational metadata. +The dashboard defaults to authorized read/write access. An authorized read-only +mode removes the mutations. Use the explicitly public read-only mode only for +synthetic demo data, because it exposes stored arguments, results, errors, +identifiers, and operational metadata. Administration remains available through the JSON CLI and typed runtime managers. See [Operations](docs/operations.md), the [dashboard guide](docs/dashboard.md), @@ -288,11 +346,11 @@ and [Configuration](docs/configuration.md). ## Design provenance -Solid Objects JS is a Node.js and TypeScript implementation informed by the -Ruby [`solid_objects`](https://github.com/cardmagic/solid_objects) design. It -began at the `0.12` capability generation because the initial implementation -targeted the Ruby `0.12` contract; the number does not represent twelve earlier -JavaScript release generations. +Solid Objects JS is a Node.js and TypeScript implementation. The Ruby +[`solid_objects`](https://github.com/cardmagic/solid_objects) design informed +it. It began at the `0.12` capability generation, because the first +implementation targeted the Ruby `0.12` contract. That number does not represent +twelve earlier JavaScript release generations. The TypeScript implementation is not a source translation. It redesigned the API around inferred TypeScript references, Node runtime supervision, @@ -304,9 +362,9 @@ runtime differences. The Ruby project first appeared publicly on August 6, 2026, and this TypeScript repository on August 13, 2026. Both remain early releases. The [`mtg-playmat`](https://github.com/cardmagic/mtg-playmat) application uses the -Ruby actor and realtime design, while +Ruby actor and realtime design. [Shuffle Up and Play](https://github.com/cardmagic/shuffleupandplay) uses the -TypeScript package in the deployed Node and SQLite topology documented above. +TypeScript package in the deployed Node and SQLite topology above. ## Documentation diff --git a/docs/api.md b/docs/api.md index 63d899a..ca32e0d 100644 --- a/docs/api.md +++ b/docs/api.md @@ -14,11 +14,10 @@ generic signatures; this index explains the supported role of every export. - `SolidObjectsRuntime`: installation, registration, supervision, and manager owner. The normal lifecycle is `install()`, `run(signal)`, then `close()`. `snapshotWithIncarnation(reference)` returns the same authorized fields as - `snapshot()` alongside the read instance's `instanceId`, `revision`, and - `createdAtMs`, computed from the identical read so a caller can fence a - derived write (for example a downstream projection) against a stale or - superseded actor incarnation. `createdAtMs` orders incarnations at - millisecond granularity; see + `snapshot()`. It adds the read instance's `instanceId`, `revision`, and + `createdAtMs` from that identical read. A caller can therefore fence a derived + write, such as a downstream projection, against a stale or superseded actor + incarnation. `createdAtMs` orders incarnations to the millisecond. See [Limitations and non-goals](correctness.md#limitations-and-non-goals) for the same-millisecond boundary. - `Actor`: base class providing `ref()`, `actorId`, `currentMessage`, @@ -38,8 +37,8 @@ createdAtMs }` shape returned by `SolidObjectsRuntime.snapshotWithIncarnation`. - `MessageReference`: immutable durable message identity with `id`, `requestId`, actor identity, `sequence`, `status()`, `result()`, and `wait()`. - `InvocationOptions`, `AsyncInvocationOptions`, `SnapshotOptions`, and - `DestroyOptions`: authorization, idempotency, timing, and scheduling options - used by reference methods. + `DestroyOptions`: the options for authorization, idempotency, time, and + schedule that the reference methods use. `ActorIntents`, `EffectIntent`, `CommitActionIntent`, `ReminderIntent`, `OutboundMessageIntent`, `ReminderOptions`, `OutboundMessageOptions`, @@ -59,11 +58,11 @@ override observables(): Record { } ``` -Both values must be JSON-compatible and are evaluated after each successful -turn. An invalidation-only value participates in change detection but is never -written to the broadcast outbox or invalidation envelope. The envelope carries -its name in `invalidations`, allowing component registries to refresh a -reauthorized endpoint without exposing the value. +Both values must be JSON-compatible. The runtime evaluates them after each +successful turn. An invalidation-only value takes part in change detection, but +the runtime never writes it to the broadcast outbox or the invalidation +envelope. The envelope carries its name in `invalidations`. A component registry +can then refresh a reauthorized endpoint, and the value stays private. `MessageReference` does not retain an invocation's authorization context. Supply `authorizationContext` to each `status()`, `result()`, and `wait()` call; @@ -91,12 +90,13 @@ function playerForSession(options: { ### Reminders -A reminder is one alarm per actor and name. Scheduling a name that is already -armed **moves the existing alarm** rather than adding a second one, which is -what makes a reminder safe to re-arm from a handler that may run more than once. +A reminder is one alarm per actor and name. If you schedule a name that is +already armed, the runtime **moves the existing alarm**. It does not add a +second one. A reminder is therefore safe to re-arm from a handler that can run +more than once. -Without a key that name is the operation, so one actor holds one alarm per -operation, and arming one per queued item keeps only the last: +Without a key, that name is the operation. One actor then holds one alarm per +operation. If you arm one alarm per queued item, only the last one remains: ```typescript // Wrong. Every entry overwrites the previous entry's alarm. @@ -126,10 +126,10 @@ alone, so a long operation with a short key is caught too. A key may hold colons of its own, because an actor member name cannot. An actor that only needs to know "what is next" can still keep one alarm and -drain everything due when it fires. That costs one row instead of one per item -and cannot strand an entry when an occurrence is coalesced, so prefer it for a -large queue of interchangeable items and prefer `key` when an item needs an -alarm that can be moved on its own. +drain everything that is due when it fires. That costs one row instead of one +row per item. It also cannot strand an entry when the runtime coalesces an +occurrence. Prefer it for a large queue of interchangeable items. Prefer `key` +when one item needs an alarm that you can move on its own. ### Runtime managers @@ -333,8 +333,8 @@ The wire format, trust boundary, revision rules, and component semantics are in `DashboardExtension` objects, and `DashboardMiddleware` functions. - `DashboardRequestContext` supplies the existing administration authorization context and an optional `DashboardSession`. Read/write access requires the - session so its `read()` and `write()` methods can hold the masked CSRF token - across requests; read-only modes do not create CSRF state. + session, because its `read()` and `write()` methods hold the masked CSRF token + across requests. Read-only modes create no CSRF state. - `DashboardRoute`, `DashboardRouteContext`, `DashboardPolicy`, `DashboardPage`, and `DashboardTab` define extension pages. Every route requires a policy. - `DashboardRenderer`, `DashboardRenderInput`, and `DashboardMiddlewareInput` diff --git a/docs/architecture.md b/docs/architecture.md index a9c8af6..24e51c6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -14,8 +14,9 @@ same operation, delivery mode, and arguments. The correctness contract is: -> Messages for one actor are durably enqueued and processed sequentially, at -> least once, by at most one valid activation lease holder at a time. +> Solid Objects durably enqueues the messages for one actor. At most one valid +> activation lease holder processes them at a time, in sequence, and at least +> once. Actor code runs outside the database transaction. A short transaction guarded by the activation owner, token, generation, expiration, and claimed-message @@ -36,10 +37,10 @@ at the shutdown boundary; database leases and fencing remain the correctness mechanism if a failed role was still executing actor code. SQLite serializes access through one process-local connection and begins write -transactions immediately. PostgreSQL and MySQL use bounded pools, keep each -transaction on one checked-out client, store timestamps and sequences as -64-bit integers, and lock an actor's instance row while allocating mailbox -sequences. MySQL creates InnoDB tables and retries only the side-effect-free +transactions immediately. PostgreSQL and MySQL use bounded pools. They keep each +transaction on one checked-out client. They store timestamps and sequences as +64-bit integers. They lock an actor's instance row during mailbox sequence +allocation. MySQL creates InnoDB tables and retries only the side-effect-free enqueue transaction when InnoDB chooses it as a deadlock victim. Every adapter uses database time and the same fencing predicates. @@ -51,19 +52,21 @@ statement and lock timeouts. MySQL bounds pool checkout and client queries and installs transaction execution and lock-wait limits. A deadline before enqueue commit produces no durable message. After commit, timeout diagnostics retain the message reference for later recovery. -Already-running JavaScript actor code is cooperative rather than forcefully -preempted; leases and fenced commits remain authoritative if it outlives the -caller's wait. +JavaScript actor code that already runs is cooperative. The runtime does not +preempt it by force. If it outlives the caller's wait, leases and fenced commits +stay authoritative. Each database adapter also tracks its active transaction through Node's async -context. A committed call or message wait fails before enqueue or polling when -the same logical call stack already owns a Solid Objects transaction, rather -than waiting for a connection or serialized SQLite slot it cannot release. +context. A committed call or message wait fails early when the same logical call +stack already owns a Solid Objects transaction. It fails before enqueue or +polling. It does not wait for a connection or a serialized SQLite slot that it +cannot release. PostgreSQL notifications are an opt-in latency layer. One event-driven client -per runtime listens on role-specific channels before the worker checks durable -state, which closes the listener-startup race without holding a polling -connection per worker. A notification advances a process-local role generation +per runtime listens on role-specific channels. It listens before the worker +checks durable state. This closes the listener-startup race, and no worker holds +a polling connection of its own. A notification advances a process-local role +generation and wakes every matching waiter. Reconnection and notification loss fall back to adaptive polling, whose current wait can be as long as the configured idle ceiling. @@ -81,8 +84,8 @@ keeps its original future availability. The global claim reads at most `claimScanLimit` ordered candidates. If another worker acquires the first candidate's lease, the transaction continues through -that bounded set instead of returning idle and sacrificing parallelism across -independent actor identities. +that bounded set. It does not return idle, because an idle return loses +parallelism across independent actor identities. When a pass becomes idle, a long-running worker keeps the hydrated actor and continues renewing the same fenced lease until its idle timeout. A later turn diff --git a/docs/authorization.md b/docs/authorization.md index 433528e..068ff2d 100644 --- a/docs/authorization.md +++ b/docs/authorization.md @@ -15,12 +15,13 @@ WebSocket or stream connection, passes that fresh server-side subject as the session's `authorizationContext`, and forwards incoming protocol messages to `session.receive()`. Every subscribe request calls `authorizeSubscription` before actor type lookup, so denied callers cannot probe the registry. A new -connection must use a newly resolved authorization context; do not copy a user -object from an earlier request or trust an actor ID supplied by the browser. +connection must use a newly resolved authorization context. Do not copy a user +object from an earlier request. Do not trust an actor ID that the browser +supplies. -Successful subscription authorization allows the `broadcastValue()` portion -of the explicit `observables()` projection for that actor, including the -immediate committed replay. Unwrapped observables and values marked with +Successful subscription authorization allows the `broadcastValue()` portion of +the explicit `observables()` projection for that actor. The immediate committed +replay is part of that portion. Unwrapped observables and values marked with `broadcastInvalidation()` disclose only their names, not their values. Subscription authorization does not authorize actor state, operations, queries, destruction, or administration. diff --git a/docs/benchmarks.md b/docs/benchmarks.md index c0dcfbf..b68d3a1 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -83,72 +83,93 @@ dataset. Redirect stdout to retain the JSON result. ## Observed results -Measured on August 15, 2026 with the prepared `0.13.0` source tree: - -- Apple M5, 10 logical CPUs, 24 GiB memory -- macOS 26.6 (`darwin 25.6.0`) -- Node.js 26.7.0 -- SQLite 3.53.4 on the internal SSD, PostgreSQL 18.4 and MySQL 8.4.11 in - Docker Desktop +Measured on August 22, 2026 with the `0.14.0` source tree: + +- Apple M5 (Mac17,2), 10 logical CPUs +- macOS 26.6 +- Node.js 24.18.0 +- SQLite 3.53.1 through `node:sqlite` on the internal SSD +- PostgreSQL 17.11 and MySQL 9.7.1, both installed natively and started on a + scoped temporary data directory - 25 warmup operations, 250 measured operations, concurrency 16 -### SQLite 3.53.4 - -| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | -| -------------- | --------- | ------------ | -----: | -----: | ------: | ------: | -| one process | warm hot | synchronous | 95.51 | 35.34 | 1159.07 | 1711.76 | -| one process | warm hot | asynchronous | 448.63 | 35.96 | 38.95 | 41.47 | -| one process | warm many | synchronous | 44.35 | 141.55 | 1886.35 | 2543.15 | -| one process | warm many | asynchronous | 240.64 | 48.62 | 75.10 | 671.64 | -| one process | cold many | synchronous | 30.91 | 436.98 | 1502.26 | 1729.10 | -| one process | cold many | asynchronous | 57.15 | 167.36 | 967.74 | 1066.49 | -| four processes | warm hot | synchronous | 453.10 | 27.24 | 72.09 | 77.88 | -| four processes | warm hot | asynchronous | 487.39 | 29.27 | 60.66 | 68.28 | -| four processes | warm many | synchronous | 78.88 | 86.33 | 890.29 | 1381.65 | -| four processes | warm many | asynchronous | 47.56 | 220.37 | 1130.53 | 1341.86 | -| four processes | cold many | synchronous | 70.45 | 174.67 | 643.53 | 668.46 | -| four processes | cold many | asynchronous | 39.92 | 221.01 | 1309.13 | 1442.30 | - -### PostgreSQL 18.4 - -| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | -| -------------- | --------- | ------------ | ----: | ------: | ------: | ------: | -| one process | warm hot | synchronous | 66.99 | 232.89 | 317.94 | 375.68 | -| one process | warm hot | asynchronous | 72.27 | 211.57 | 280.89 | 316.61 | -| one process | warm many | synchronous | 76.69 | 129.88 | 552.01 | 1963.27 | -| one process | warm many | asynchronous | 25.78 | 340.96 | 2326.38 | 6917.26 | -| one process | cold many | synchronous | 12.70 | 1262.08 | 1734.77 | 2199.20 | -| one process | cold many | asynchronous | 11.23 | 1254.46 | 2796.81 | 2905.99 | -| four processes | warm hot | synchronous | 83.71 | 191.83 | 220.87 | 231.84 | -| four processes | warm hot | asynchronous | 86.42 | 184.56 | 210.11 | 215.48 | -| four processes | warm many | synchronous | 95.47 | 100.31 | 330.27 | 1663.84 | -| four processes | warm many | asynchronous | 37.04 | 232.71 | 1601.29 | 4447.85 | -| four processes | cold many | synchronous | 14.79 | 1114.18 | 1263.33 | 1313.70 | -| four processes | cold many | asynchronous | 11.68 | 1231.94 | 2555.34 | 2848.80 | - -### MySQL 8.4.11 - -| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | -| -------------- | --------- | ------------ | ----: | ------: | ------: | ------: | -| one process | warm hot | synchronous | 28.11 | 504.36 | 1648.10 | 2088.05 | -| one process | warm hot | asynchronous | 25.29 | 508.26 | 1750.17 | 2222.72 | -| one process | warm many | synchronous | 61.79 | 165.62 | 463.59 | 2036.31 | -| one process | warm many | asynchronous | 22.45 | 446.17 | 2420.92 | 7818.65 | -| one process | cold many | synchronous | 10.23 | 1353.37 | 3038.76 | 3185.15 | -| one process | cold many | asynchronous | 10.04 | 1338.31 | 3344.18 | 3477.06 | -| four processes | warm hot | synchronous | 29.89 | 427.18 | 1483.88 | 1997.56 | -| four processes | warm hot | asynchronous | 27.59 | 448.16 | 1518.00 | 1589.57 | -| four processes | warm many | synchronous | 69.77 | 157.79 | 410.83 | 2263.13 | -| four processes | warm many | asynchronous | 36.21 | 265.24 | 1579.43 | 4594.33 | -| four processes | cold many | synchronous | 13.68 | 1088.93 | 2186.30 | 2519.92 | -| four processes | cold many | asynchronous | 11.38 | 1209.84 | 2532.38 | 2730.74 | - -The poor throughput and tail latency in cold and asynchronous cases are -observed limitations, not capacity recommendations. The small asynchronous -yield changed scheduling enough to improve some cases and worsen others; -repeat runs on application-shaped payloads are required before drawing a -general conclusion. PostgreSQL 14, MySQL 8.0, and other database versions are -covered by integration tests but were not benchmarked. +### SQLite 3.53.1 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | -----: | -----: | -----: | -----: | +| one process | warm hot | synchronous | 286.09 | 50.09 | 135.15 | 176.55 | +| one process | warm hot | asynchronous | 322.53 | 48.66 | 70.59 | 75.87 | +| one process | warm many | synchronous | 119.05 | 132.25 | 237.52 | 247.05 | +| one process | warm many | asynchronous | 288.26 | 32.64 | 70.33 | 562.42 | +| one process | cold many | synchronous | 47.67 | 326.04 | 435.46 | 507.03 | +| one process | cold many | asynchronous | 99.39 | 148 | 236.03 | 295.26 | +| four processes | warm hot | synchronous | 518.66 | 30.15 | 52.73 | 78.57 | +| four processes | warm hot | asynchronous | 506.8 | 26.39 | 92.26 | 102.18 | +| four processes | warm many | synchronous | 190.36 | 76.65 | 145.7 | 158.94 | +| four processes | warm many | asynchronous | 77.12 | 194.6 | 399.43 | 527.1 | +| four processes | cold many | synchronous | 54.74 | 273.33 | 646.68 | 650.09 | +| four processes | cold many | asynchronous | 93.85 | 145.27 | 333.78 | 426.6 | + +### PostgreSQL 17.11 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | -----: | -----: | -----: | ------: | +| one process | warm hot | synchronous | 206.31 | 69.33 | 132.69 | 153.03 | +| one process | warm hot | asynchronous | 212.07 | 72.38 | 96.75 | 106.12 | +| one process | warm many | synchronous | 191.4 | 55.71 | 147.99 | 722.6 | +| one process | warm many | asynchronous | 66.6 | 121.61 | 920.97 | 2441.28 | +| one process | cold many | synchronous | 31.38 | 500.73 | 705.15 | 722.54 | +| one process | cold many | asynchronous | 17.24 | 909.02 | 1192.9 | 1211.35 | +| four processes | warm hot | synchronous | 265.63 | 57.94 | 82.52 | 109.46 | +| four processes | warm hot | asynchronous | 330.94 | 46.81 | 63.76 | 70.25 | +| four processes | warm many | synchronous | 161.73 | 58.67 | 237.04 | 718.55 | +| four processes | warm many | asynchronous | 119.63 | 68.71 | 531.12 | 1372.5 | +| four processes | cold many | synchronous | 50.84 | 314.51 | 487.09 | 506.45 | +| four processes | cold many | asynchronous | 29.93 | 546.7 | 615.36 | 638.1 | + +### MySQL 9.7.1 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | -----: | ------: | ------: | ------: | +| one process | warm hot | synchronous | 79.59 | 208.66 | 222.86 | 225.19 | +| one process | warm hot | asynchronous | 79.98 | 206.78 | 232.74 | 237.94 | +| one process | warm many | synchronous | 79.7 | 129.25 | 395.52 | 1987.46 | +| one process | warm many | asynchronous | 28.06 | 338.85 | 1992.54 | 6004.98 | +| one process | cold many | synchronous | 13.78 | 1215.91 | 1445.61 | 1470.91 | +| one process | cold many | asynchronous | 11.63 | 1347.74 | 1733.01 | 1820.26 | +| four processes | warm hot | synchronous | 213.75 | 72.85 | 94.48 | 106.73 | +| four processes | warm hot | asynchronous | 228.32 | 70.45 | 84.96 | 90.39 | +| four processes | warm many | synchronous | 105.31 | 90.36 | 297.88 | 1064.14 | +| four processes | warm many | asynchronous | 80.03 | 149.92 | 324.98 | 1090.95 | +| four processes | cold many | synchronous | 51.82 | 306.1 | 407.22 | 424.45 | +| four processes | cold many | asynchronous | 27.35 | 554.03 | 807.18 | 868.48 | + +The cold and asynchronous cases keep the poorest throughput and the longest +tail. These are observed limitations. They are not capacity recommendations. +Repeat the runs on application-shaped payloads before you draw a general +conclusion. Integration tests cover PostgreSQL 14, MySQL 8.0, and other database +versions, but this harness did not measure them. + +### Virtualization cost + +The same server version ran natively and in Docker Desktop on the same machine, +on the same day, through the same harness. Only the container boundary changes. + +| Database | Topology and shape | Native ops/s | Docker ops/s | Native gain | +| ---------------- | ------------------------------- | -----------: | -----------: | ----------: | +| PostgreSQL 17.11 | four processes, warm hot, async | 330.94 | 67.69 | 4.9x | +| PostgreSQL 17.11 | one process, warm hot, sync | 206.31 | 56.44 | 3.7x | +| MySQL 9.7.1 | four processes, warm hot, async | 228.32 | 60.27 | 3.8x | +| MySQL 9.7.1 | one process, warm hot, sync | 79.59 | 44.56 | 1.8x | + +Across the full matrix, Docker Desktop cost between 1.0x and 7.8x of the native +throughput. The multi-process rows lose the most, because more connections and +more commits cross the container boundary. Measure a database on the deployment +shape you intend to run, and state the boundary with any number you publish. + +Earlier releases of this document reported PostgreSQL 18.4 and MySQL 8.4.11 in +Docker Desktop on the `0.13.0` tree. Those numbers measured the container as +much as the database, so the tables above replace them. ## Sources of bias @@ -156,7 +177,8 @@ covered by integration tests but were not benchmarked. - Loopback database connections exclude production network latency. - Filesystem cache, SQLite WAL state, Node JIT warmup, and garbage collection affect short runs. -- Docker Desktop adds virtualization overhead to containerized databases. +- Docker Desktop costs between 1.0x and 7.8x of the native throughput. The + tables above use native servers. See [Virtualization cost](#virtualization-cost). - The payload is a small counter, not a representative application state size. - The harness measures default durability settings and one client concurrency. - Hot-identity results deliberately include serialization and cannot be scaled diff --git a/docs/browser-protocol.md b/docs/browser-protocol.md index 19d2d42..0546049 100644 --- a/docs/browser-protocol.md +++ b/docs/browser-protocol.md @@ -79,9 +79,9 @@ stale revisions within an incarnation. `SolidObjectsComponentRegistry` maps changed observable names to keyed UI registrations. The browser supplies an asynchronous `refresh` function and a -synchronous `apply` function, so HTML, virtual DOM, and framework-native render -results use the same coordination contract without assuming a rendering -framework. +synchronous `apply` function. HTML, virtual DOM, and framework-native render +results therefore use the same coordination contract. The registry assumes no +render framework. Components may share a batch name. A microtask unions affected components in the same actor, batch, incarnation, and revision into one refresh request. diff --git a/docs/comparisons.md b/docs/comparisons.md index bbe9ce8..628cb36 100644 --- a/docs/comparisons.md +++ b/docs/comparisons.md @@ -3,15 +3,16 @@ This guide compares coordination models so an application can choose the smallest mechanism that meets its requirements. It does not rank the projects. -| Approach | State and serialization unit | Deployment and durable substrate | Separate service | Replay versus state | Realtime and edge placement | Cross-identity transaction | Data access | -| --------------------------- | ---------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------ | --------------------------------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------- | -| SQL transaction or row lock | Rows selected by one transaction | Application process and SQL database | No | The application retries a failed transaction | Application-owned | Yes, for rows in the same database transaction | Ordinary application tables and SQL tools | -| Traditional job queue | A job, queue, or configured grouping key | Workers plus broker or queue database | Usually | The job is retried; mutable entity state remains application-owned | Application-owned | Not supplied by the queue | Queue administration plus application data stores | -| Solid Objects | Actor class and application-defined ID | Node processes plus existing SQLite, PostgreSQL, or MySQL | No; Redis wake-up is optional | The operation retries against durable actor state | Committed projections; application-owned transport; no edge placement | No | Relational tables, typed administration, CLI, and dashboard | -| Cloudflare Durable Objects | Object class and globally unique ID | Cloudflare Workers plus per-object managed storage | Cloudflare platform | Object activation with durable state, not workflow-step replay | WebSockets and Cloudflare-selected object location | Storage transactions are scoped to one object | Object storage APIs and platform tooling | -| Rivet Actors | Addressable actor | Rivet Engine or managed compute with actor state, KV, or per-actor SQLite | Rivet Engine | Actor persistence and lifecycle; workflows add recorded steps | Actor events and deployment-dependent placement | No general transaction across actors | Actor APIs and selected persistence model | -| DBOS | Workflow ID and checkpointed steps | Application processes plus PostgreSQL system database | No orchestration server for the library; Conductor is recommended for distributed recovery | Deterministic workflow replay skips checkpointed steps | Workflow events; application placement | PostgreSQL transactions remain separate from workflow identity | PostgreSQL system database, client, CLI, and optional Conductor | -| Restate | Service handler or keyed virtual object | Application services plus Restate's durable log and state store | Yes | Durable execution journals handler progress and object state | Service protocol and Restate deployment | No shared SQL transaction across object keys | Restate APIs, state tools, snapshots, and backups | +| Approach | State and serialization unit | Deployment and durable substrate | Separate service | Replay versus state | Realtime and edge placement | Cross-identity transaction | Data access | +| --------------------------- | ---------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------------------- | ------------------------------------------------------------------- | +| SQL transaction or row lock | Rows selected by one transaction | Application process and SQL database | No | The application retries a failed transaction | Application-owned | Yes, for rows in the same database transaction | Ordinary application tables and SQL tools | +| Traditional job queue | A job, queue, or configured grouping key | Workers plus broker or queue database | Usually | The job is retried; mutable entity state remains application-owned | Application-owned | Not supplied by the queue | Queue administration plus application data stores | +| Solid Objects | Actor class and application-defined ID | Node processes plus existing SQLite, PostgreSQL, or MySQL | No; Redis wake-up is optional | The operation retries against durable actor state | Committed projections; application-owned transport; no edge placement | No | Relational tables, typed administration, CLI, and dashboard | +| Cloudflare Durable Objects | Object class and globally unique ID | Cloudflare Workers plus per-object managed storage | Cloudflare platform | Object activation with durable state, not workflow-step replay | WebSockets and Cloudflare-selected object location | Storage transactions are scoped to one object | Object storage APIs and platform tooling | +| celld | Object class and object name | celld nodes plus one object-storage bucket; each object is its own SQLite database | Yes, the celld daemon on every node | The new owner restores the object's SQLite database from the bucket and resumes | Cloudflare Workers APIs; an object runs on one node of your fleet, not at an edge location | No | Per-object SQLite through the Workers storage APIs, plus the bucket | +| Rivet Actors | Addressable actor | Rivet Engine or managed compute with actor state, KV, or per-actor SQLite | Rivet Engine | Actor persistence and lifecycle; workflows add recorded steps | Actor events and deployment-dependent placement | No general transaction across actors | Actor APIs and selected persistence model | +| DBOS | Workflow ID and checkpointed steps | Application processes plus PostgreSQL system database | No orchestration server for the library; Conductor is recommended for distributed recovery | Deterministic workflow replay skips checkpointed steps | Workflow events; application placement | PostgreSQL transactions remain separate from workflow identity | PostgreSQL system database, client, CLI, and optional Conductor | +| Restate | Service handler or keyed virtual object | Application services plus Restate's durable log and state store | Yes | Durable execution journals handler progress and object state | Service protocol and Restate deployment | No shared SQL transaction across object keys | Restate APIs, state tools, snapshots, and backups | ## Primary references @@ -22,6 +23,10 @@ smallest mechanism that meets its requirements. It does not rank the projects. distinguishes local concurrency from multiple worker processes. - Cloudflare documents global uniqueness, per-object storage, single-threaded execution, and placement in [What are Durable Objects?](https://developers.cloudflare.com/durable-objects/concepts/what-are-durable-objects/). +- celld documents per-object SQLite databases, bucket replication, and + object-storage compare-and-swap ownership in its + [repository](https://github.com/denoland/celld) and its + [documentation](https://celld.dev/docs). - Rivet documents addressable actors and persistence in [Actors](https://rivet.dev/docs/actors/) and [Persistence](https://rivet.dev/docs/actors/persistence). diff --git a/docs/configuration.md b/docs/configuration.md index 44df7d2..245e38a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -59,8 +59,8 @@ runtime role enabled. Broadcast workers are started only when `broadcast` or Consecutive empty passes double it up to `idlePollingIntervalMilliseconds`. Actor workers never wait longer than `leaseRenewalIntervalMilliseconds`. Set the fast and idle values equal for a -fixed cadence. A custom wake-up adapter should return `true` for a notification -and `false` for a timeout; an older adapter that returns `void` remains +fixed cadence. A custom wake-up adapter must return `true` for a notification +and `false` for a timeout. An older adapter that returns `void` remains compatible and keeps the fast cadence. Wake-ups reduce latency, while database polling remains the correctness path. diff --git a/docs/correctness.md b/docs/correctness.md index 1a67d1b..9f09bf2 100644 --- a/docs/correctness.md +++ b/docs/correctness.md @@ -12,12 +12,12 @@ recreated actor. A caller authorized before destruction receives `ActorDestroyed`; an unknown or forged reference remains unauthorized. - Graceful process shutdown and stale-process cleanup use the same atomic - ownership release: claimed messages return to ready membership, activations - are unfenced, and processing effect, reminder, and broadcast claims become - available again. A stale draining process is recoverable like a stale running - process. -- Permanent operation failure raises `MessageFailed` with the durable message - ID and persisted error details instead of treating actor code text as the + ownership release. Claimed messages return to ready membership. The runtime + unfences the activations. Effect, reminder, and broadcast claims become + available again. A stale process that drains is recoverable like a stale + process that runs. +- Permanent operation failure raises `MessageFailed`. The error carries the + durable message ID and the persisted error details. Actor code text is not the public exception contract. - Operation, lifecycle, observable, and payload callbacks retain their owning runtime through async context. Isolated runtimes therefore never fall back to @@ -31,14 +31,14 @@ - Activation passes are bounded. Yielding changes ready-membership polling order only; it neither changes durable message sequence nor makes future work due early. -- Idle hydrated actors remain fenced by the same renewable lease. Cache reuse - never bypasses claim membership or the commit fence, failed turns restore - their public fields before reuse, and conditional release cannot clear a - newer owner or generation. +- Idle hydrated actors keep the same renewable lease as their fence. Cache reuse + never bypasses claim membership or the commit fence. A failed turn restores + its public fields before reuse. Conditional release cannot clear a newer owner + or generation. - Actor setup completes before an attempt begins. A hydration, migration, or - activation failure atomically restores ready membership, releases its - activation fence, and restores the attempt count; an awaiting caller receives - the setup error. + activation failure restores ready membership, releases its activation fence, + and restores the attempt count. All three happen in one atomic step. A caller + that waits receives the setup error. - `guardApplicationDatabase()` rejects direct application writes during actor operations, observable and payload projections, and state migrations. It permits only `SELECT` through row-returning methods. Commit actions remain in @@ -49,10 +49,10 @@ successful snapshots and their nested JSON values are frozen copies. - Personalized payloads hydrate committed state separately for every payload name and subscriber. Each projection is read-only, size bounded, and fenced - independently by actor incarnation and revision. One denied, mutating, or - failing projection cannot stop its siblings or observable delivery. A state - change on an actor declaring payloads creates a revision broadcast even when - the actor declares no scalar observables. + independently by actor incarnation and revision. One projection that is + denied, that mutates, or that fails cannot stop its siblings or observable + delivery. A state change on an actor with payloads creates a revision + broadcast, even when that actor declares no scalar observables. ## Limitations and non-goals @@ -71,14 +71,13 @@ placement, capacity, database backups, and database failover. - Redis and PostgreSQL notifications reduce wake-up latency but do not replace durable polling or become a source of truth. -- `snapshotWithIncarnation`'s `createdAtMs` orders actor incarnations at - millisecond granularity, the same precision every adapter stores - `created_at_ms` at. Destroying and recreating the same actor identity - within the same database-clock millisecond produces two incarnations with - an equal `createdAtMs`; a caller fencing a derived write on it cannot - distinguish which of the two is current in that narrow case. `instanceId` - still changes and detects that a recreation happened; it is a random UUID - and carries no order of its own. +- `snapshotWithIncarnation`'s `createdAtMs` orders actor incarnations to the + millisecond. Every adapter stores `created_at_ms` at that same precision. If + you destroy and recreate the same actor identity inside one database-clock + millisecond, the two incarnations get an equal `createdAtMs`. In that narrow + case, a caller cannot tell which of the two is current, so it cannot fence a + derived write on that value. `instanceId` still changes and shows that a + recreation occurred. It is a random UUID and carries no order of its own. - Large documents, bulk pipelines, globally placed edge state, and global counters are outside the intended workload. Prefer an ordinary row transaction when it completely enforces the invariant. diff --git a/docs/dashboard.md b/docs/dashboard.md index 297b84c..547a525 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -115,9 +115,9 @@ with fresh random bytes, so tokens differ between requests while every form already open in the same session remains valid. POST requests without a valid token receive 403 and do not perform the action. -Every stored or request-derived string is escaped before entering HTML, -including JSON placed in chart attributes. HTML and statistics responses are -private and not cached. The dashboard sends a nonce-backed content security +The dashboard escapes every stored or request-derived string before that string +enters the HTML. This includes the JSON in chart attributes. HTML and statistics +responses are private, and no cache holds them. The dashboard sends a nonce-backed content security policy, denies framing, disables MIME sniffing, and limits referrers to the same origin. @@ -137,9 +137,9 @@ effects, broadcasts, and dead letters. Pause prevents workers from claiming new turns for that identity; a turn already executing may still commit. Resume clears the brake and normal polling resumes delivery. -Dead-letter retry calls `runtime.deadLetters.retry()`, retaining its durable -idempotency and actor-operation validation. A retry refused by the runtime is -shown on the detail page with status 422. +Dead-letter retry calls `runtime.deadLetters.retry()`. It keeps the durable +idempotency and the actor-operation validation of that method. If the runtime +refuses a retry, the detail page shows it with status 422. `HEAD /` performs only a schema reachability query and creates no CSRF session state. Use it for liveness checks instead of polling the full dashboard. diff --git a/docs/errors-and-recovery.md b/docs/errors-and-recovery.md index 80b9d34..aafc82a 100644 --- a/docs/errors-and-recovery.md +++ b/docs/errors-and-recovery.md @@ -19,18 +19,19 @@ do not parse error messages. | `PayloadTooLarge` | Arguments, state, result, snapshot getter, effect result, or personalized payload exceeded its configured limit. | Reduce the JSON value or deliberately raise the corresponding limit. | | `SyncInsideTransaction` | A committed call or message wait would self-deadlock inside this adapter's transaction. | Finish the transaction first or stage actor-owned work through a commit action. | -`this.reject()` accepts codes matching `[A-Za-z_][A-Za-z0-9_]*`, including -camelCase. An invalid code throws the non-retryable `InvalidRejectionCode`; the -operation fails on its first attempt and a synchronous caller receives -`MessageFailed` instead of waiting through retry backoff. +`this.reject()` accepts any code that matches `[A-Za-z_][A-Za-z0-9_]*`. +camelCase is valid. An invalid code throws the non-retryable +`InvalidRejectionCode`. The operation then fails on its first attempt, and a +synchronous caller receives `MessageFailed`. It does not wait through retry +backoff. `MessageReference.status()`, `result()`, and `wait()` reauthorize the stored operation. `result()` returns `undefined` while work is nonterminal, returns the committed result when complete, and raises `Rejected` or `MessageFailed` for a terminal refusal or failure. `wait()` blocks until the same terminal outcomes -or its deadline. A reference does not retain the authorization context used to -send it; supply the context to each of these methods so the stored operation is -reauthorized. +or its deadline. A reference does not keep the authorization context of the +original send. Supply the context to each of these methods, so that the runtime +can reauthorize the stored operation. ## Definition and programming errors @@ -97,8 +98,7 @@ linked replacement message; repeating the call returns the same replacement. Effects are different: they execute outside the actor transaction and are at least once. Deduplicate external work with the stable `EffectContext.id`. Success and failure callback operations receive the originally staged -`arguments` for actor-state correlation. A retryable failure scheduled into the -future is correctly considered idle for the present pass, so -`runtime.testing.drain()` does not advance retry backoff; use a -`NonRetryableError` when a test needs to exercise the exhausted failure callback -without waiting. +`arguments` for actor-state correlation. A retryable failure with a future +schedule is correctly idle for the present pass, so `runtime.testing.drain()` +does not advance retry backoff. Use a `NonRetryableError` when a test must reach +the exhausted failure callback immediately. diff --git a/docs/operations.md b/docs/operations.md index 0423ddf..89b4199 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -13,9 +13,9 @@ not cross a process boundary. When live processes share the database without a configured adapter, the runtime logs `solid_objects.polling_only_cross_process_wake_up` once. Use PostgreSQL notifications or optional Redis Pub/Sub when separate processes need prompt -delivery; without one, newly committed work can wait up to the current idle -polling interval. Notification errors are isolated and logged by role and error -class without failing the committed work. +delivery. Without one of them, newly committed work can wait for the current +idle polling interval. The runtime isolates notification errors and logs them by +role and error class. The committed work does not fail. The warning excludes process rows with the current hostname and host process ID. It can therefore appear during a rolling deployment or restart overlap when an @@ -53,8 +53,8 @@ isolated backlogs. `solid_objects.activation.yielded` reports the actor identity, turns processed, and remaining due membership count. `claimScanLimit` defaults to 100. Global claims inspect a bounded ordered set of -actor identities and continue after a lost lease race, preserving worker -parallelism without an unbounded scan. +actor identities. They continue after a lost lease race. Worker parallelism +stays, and the scan stays bounded. Workers retain a hydrated actor and its fenced lease for `idleDeactivationTimeoutMilliseconds`, which defaults to 30 seconds. Idle @@ -66,15 +66,17 @@ longer polling. Actors can override protected `onActivate()` and `onDeactivate()` methods for nondurable, process-local resources. Either hook may be asynchronous. Hook code -runs under the application-write guard, and `onDeactivate()` is best effort: -it may not run after a crash, cannot establish a correctness guarantee, and a -failure is logged without preventing lease release. +runs under the application-write guard. `onDeactivate()` is best effort: + +- it may not run after a crash; +- it cannot establish a correctness guarantee; +- the runtime logs a failure and still releases the lease. `runtime.administration.processes()` returns the same administration-authorized immutable process metadata as `runtime.processes.all()`, with hostname, host -process ID, Node and Solid Objects versions, and a current `stale` flag. It is -safe to call through the runtime's database adapter while workers are running; -the query is serialized with other database access and does not require a +process ID, Node and Solid Objects versions, and a current `stale` flag. You can +safely call it through the runtime's database adapter while the workers run. The +runtime serializes the query with the other database access, and it needs no second SQLite connection. Graceful shutdown first persists `draining` with a `shutdownRequestedAt` timestamp, then deactivates owned actors and atomically releases every role claim before persisting `stopped`. `cleanup()` reauthorizes @@ -82,7 +84,8 @@ separately and performs the same release for stale running or draining processes. The cleanup count is instrumented; application payloads are not. Committed calls and `message.wait()` apply `timeoutMilliseconds` to the entire -durable wait, beginning before enqueue or message lookup. Adapter deadlines +durable wait. The clock starts before enqueue or message lookup. Adapter +deadlines bound serialized SQLite access and lock waits, PostgreSQL pool acquisition, statements, and locks, and MySQL pool acquisition, queries, and transaction lock waits. A `SyncEnqueueTimeout` means the enqueue transaction did not commit @@ -112,10 +115,14 @@ ID and later calls return a reference to that same message. Self-scheduling actors need a low-frequency reconciler because application alarms can still be lost. `runtime.reconciliation` provides administration- -authorized, read-only views for active instances, quiet instances without -ready work, claimed work, or scheduled reminders, migrated state batches, and -orphaned actor IDs. Collection reads use a maximum page size of 1,000 and a -stable cursor. +authorized, read-only views for: + +- active instances; +- quiet instances with no ready work, claimed work, or scheduled reminder; +- migrated state batches; +- orphaned actor IDs. + +Collection reads use a maximum page size of 1,000 and a stable cursor. The host application supplies its current owner IDs to `orphaned()` because Node applications do not share an Active Record relation abstraction. Send @@ -140,9 +147,16 @@ actor type appears in `instanceRetentionByActorType`, and remains an explicit operator action because it deletes the entire actor incarnation. Pruning selects and rechecks at most `pruneBatchSize` rows per transaction. It -preserves ready and claimed messages, dead-letter originals and replacements, -unfinished effects and broadcasts, scheduled reminders, leased or paused -instances, and processes that still own a claim or activation. Instance +keeps: + +- ready and claimed messages; +- dead-letter originals and replacements; +- unfinished effects and broadcasts; +- scheduled reminders; +- leased or paused instances; +- processes that still own a claim or an activation. + +Instance expiration removes the entire actor incarnation and all of its retained history, so use it only for actor types whose state is safely disposable. diff --git a/docs/parity.md b/docs/parity.md index 3a42f54..20fbeee 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -11,20 +11,21 @@ earlier JavaScript releases. The Node `0.14.0` implementation has capability parity with that reference. Its relational runtime, correctness boundaries, administration, diagnostics, operator dashboard, realtime projections, browser behavior, and supported -adapters have native equivalents. Rails-specific rendering surfaces are -replaced by transport- and framework-neutral JavaScript APIs. The partial -guard and backpressure rows and the shared planned result-lookup row below -are explicit scope boundaries shared with the Ruby reference, not missing +adapters have native equivalents. Transport- and framework-neutral JavaScript +APIs replace the Rails-specific render surfaces. Three rows below are explicit +scope boundaries that the Ruby reference shares: the partial guard row, the +backpressure row, and the shared planned result-lookup row. They are not missing Ruby capabilities. `0.14.0` also adds `runtime.enqueueInternalMessage()`, `runtime.enqueueInternalMessageInTransaction()`, and `runtime.snapshotWithIncarnation()`. These are Node-only integration points for a host package (such as a future commercial scaling layer), not ported -Ruby capabilities: Ruby's equivalent primitives (`SolidObjects::Mailbox#enqueue`, -`ActorSnapshot`) are already reachable in-process without a dedicated public -API, since Ruby has no package-privacy boundary between a gem and its own -dependents the way Node's `exports` map enforces one. +Ruby capabilities. Ruby's equivalent primitives +(`SolidObjects::Mailbox#enqueue`, `ActorSnapshot`) are already reachable +in-process, and they need no dedicated public API. Node's `exports` map enforces +a package-privacy boundary between a package and its dependents. Ruby has no +such boundary between a gem and its dependents. ## Status vocabulary @@ -89,10 +90,10 @@ dependents the way Node's `exports` map enforces one. | Redis wake-up | Native | An optional `redis` peer provides role-specific Pub/Sub over separate lazy publisher/subscriber connections, with bounded failures and durable polling fallback. | Every wake-up adapter above is opt-in. Neither runtime selects one -automatically: an application that configures nothing keeps polling, and -each runtime warns once when live processes share a database without a -configured cross-process adapter. This is a shared, intentional limitation -of both runtimes, not a gap between them. +automatically. An application that configures nothing keeps polling. Each +runtime warns once when live processes share a database without a configured +cross-process adapter. This limit is intentional in both runtimes. It is not a +gap between them. ## Realtime and browser behavior diff --git a/docs/releasing.md b/docs/releasing.md index d551d66..4cba3ad 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -26,8 +26,8 @@ npm trust github solid-objects \ ## Release procedure -1. Update the version in `package.json` and `src/version.ts`, refresh the - lockfile when needed, and move the release notes out of the Unreleased +1. Update the version in `package.json` and `src/version.ts`. Refresh the + lockfile when necessary. Move the release notes out of the Unreleased section in `CHANGELOG.md` into a dated section for the new version. The publish job reads that section, so a version without one fails the release. 2. Run `pnpm run format:check`, `pnpm run check`, `pnpm run test:coverage`, @@ -48,7 +48,7 @@ quality, database, Redis, and browser job succeeds. It rejects tags that do not match `package.json`, safely skips versions already present in npm, and publishes new versions with npm provenance. -The job then builds the release notes with `scripts/release-notes.mjs`, which -prints the `CHANGELOG.md` section for the tagged version, and creates the GitHub -release for the tag. Re-running the job on a tag that npm already holds still -creates a missing release. +The job then builds the release notes with `scripts/release-notes.mjs`. That +script prints the `CHANGELOG.md` section for the tagged version. The job then +creates the GitHub release for the tag. If you run the job again on a tag that +npm already holds, it still creates a missing release. diff --git a/docs/state-and-lifecycle.md b/docs/state-and-lifecycle.md index f55ed1c..4d89fca 100644 --- a/docs/state-and-lifecycle.md +++ b/docs/state-and-lifecycle.md @@ -17,8 +17,8 @@ fields, then walks its prototype chain to discover methods and getters. - Operation, field, and getter names must not collide with the reference API. The constructor must establish every persisted field and must not depend on -external state. Solid Objects invokes it while validating the class, creating -defaults, hydrating state, and projecting a snapshot. +external state. Solid Objects invokes it at four points: class validation, +default creation, state hydration, and snapshot projection. ## Observable broadcast modes @@ -79,12 +79,16 @@ asynchronous work or write through a database wrapped by a failing migration, or state newer than the running code raises `StateMigrationError`. -For a destructive shape change, use expand/contract deployment: first deploy -readers that understand both shapes, then deploy the migration, wait for -operational evidence that actors have advanced, and only then remove old-shape -support. If old code cannot understand the new shape, drain it before new code -can persist the migration. Keep old migration steps so actors idle for several -releases can still advance one version at a time. +For a destructive shape change, use an expand/contract deployment: + +1. deploy readers that understand both shapes; +2. deploy the migration; +3. wait for operational evidence that the actors advanced; +4. remove the old-shape support. + +If old code cannot understand the new shape, drain it before new code persists +the migration. Keep the old migration steps. An actor that stays idle for +several releases can then still advance one version at a time. Actor state migration is separate from `runtime.install()`. The latter applies the package's relational schema migrations; it does not eagerly rewrite actor diff --git a/docs/support.md b/docs/support.md index a849cef..c9c87e4 100644 --- a/docs/support.md +++ b/docs/support.md @@ -28,19 +28,26 @@ can change. Prefer 24.15.0 or newer where the choice is free. ## What the matrix covers -The default suite exercises actor definitions, mailbox ordering, state -migrations, leases, fencing, retries, dead letters, effects, reminders, -realtime outboxes, administration, authorization, retention, lifecycle, -timeouts, and SQLite behavior. +The default suite exercises: + +- actor definitions, mailbox order, and state migrations; +- leases, fencing, retries, and dead letters; +- effects, reminders, and realtime outboxes; +- administration, authorization, and retention; +- lifecycle, timeouts, and SQLite behavior. Database jobs run the real adapter suites against PostgreSQL and MySQL servers. The Redis job runs wake-up behavior against a real Redis server. The browser job uses native WebSocket connections and Chromium for replay, payload, component, dashboard, and revision-fence behavior. -The quality job also builds the ESM package, inspects `npm pack`, installs the -generated tarball in a clean temporary project, runs its packaged SQLite -quickstart, and executes the multi-process recovery demonstration. +The quality job also: + +1. builds the ESM package; +2. inspects `npm pack`; +3. installs the tarball in a clean temporary project; +4. runs the packaged SQLite quickstart; +5. executes the multi-process recovery demonstration. ## Boundaries diff --git a/examples/quickstart-report.ts b/examples/quickstart-report.ts new file mode 100644 index 0000000..ad1766c --- /dev/null +++ b/examples/quickstart-report.ts @@ -0,0 +1,151 @@ +export interface QuickstartSummary { + sameIdentityCalls: number + sameIdentityFinalState: number + independentIdentitiesOverlapped: boolean + temporaryStateRemoved: boolean +} + +interface QuickstartCheck { + passed: boolean + title: string + detail: readonly string[] +} + +const INSTALL_COMMAND = "npm install solid-objects" +const AGENT_PROMPT = "where would the solid-objects library be best used in this app?" +const DOCUMENTATION_URL = "https://solidobjects.dev/node" + +export function formatQuickstartPlan(plan: { + sameIdentityCalls: number + actorSource: string +}): string { + return [ + "Solid Objects quickstart", + "", + "This command will:", + "", + " 1. create a temporary SQLite database;", + ` 2. send ${plan.sameIdentityCalls} concurrent calls to one identity;`, + " 3. run two other identities at the same time;", + " 4. close the runtime and delete the temporary database.", + "", + "No server, container, or configuration is needed.", + "", + "The actor it runs:", + "", + ...plan.actorSource.split("\n").map((line) => ` ${line}`.trimEnd()), + "", + "", + ].join("\n") +} + +export function formatQuickstartPrompt(): string { + return "Run it now? [Y/n] " +} + +export function answerAllowsRun(answer: string): boolean { + const value = answer.trim().toLowerCase() + if (value === "") return true + return value === "y" || value === "yes" +} + +export function formatQuickstartStop(): string { + return ["", "Nothing ran. No database and no files were created.", ""].join("\n") +} + +export function formatQuickstartReport(summary: QuickstartSummary): string { + const results = checks(summary) + return [ + "Results", + "", + ...results.flatMap(checkLines), + "", + "Each line above is an assertion, not a print. The command exits with a", + "non-zero code when one of them fails.", + "", + ...meaning(results), + "Add it to your app", + "", + ` ${INSTALL_COMMAND}`, + "", + "Or tell your agent", + "", + ` ${AGENT_PROMPT}`, + "", + `Docs: ${DOCUMENTATION_URL}`, + "", + ].join("\n") +} + +function meaning(results: readonly QuickstartCheck[]): string[] { + if (results.some((check) => !check.passed)) return [] + return [ + "What each PASS means", + "", + " Two requests on the same cart cannot overwrite each other.", + " That identity has one durable mailbox, so its calls commit one at a", + " time and no update is lost.", + "", + " Unrelated identities do not wait for that mailbox.", + " The order is per identity, not global.", + "", + " The state lives in an ordinary SQL database, so it survives a restart.", + "", + ] +} + +function checkLines(check: QuickstartCheck): string[] { + const result = check.passed ? "PASS" : "FAIL" + return [`${result} ${check.title}`, ...check.detail.map((line) => ` ${line}`)] +} + +function checks(summary: QuickstartSummary): QuickstartCheck[] { + return [ + { + passed: summary.sameIdentityFinalState === summary.sameIdentityCalls, + title: `${summary.sameIdentityCalls} concurrent calls to one identity`, + detail: sameIdentityDetail(summary), + }, + { + passed: summary.independentIdentitiesOverlapped, + title: "Two different identities ran at the same time", + detail: overlapDetail(summary), + }, + { + passed: summary.temporaryStateRemoved, + title: "Temporary state removed", + detail: cleanupDetail(summary), + }, + ] +} + +function sameIdentityDetail(summary: QuickstartSummary): string[] { + if (summary.sameIdentityFinalState !== summary.sameIdentityCalls) { + return [ + `The committed state is ${summary.sameIdentityFinalState} after ${summary.sameIdentityCalls} calls.`, + "The runtime lost an update.", + ] + } + return [ + "They ran in order on one mailbox.", + `The committed state is ${summary.sameIdentityFinalState}.`, + `The return values were the complete sequence 1 through ${summary.sameIdentityCalls}.`, + ] +} + +function overlapDetail(summary: QuickstartSummary): string[] { + if (!summary.independentIdentitiesOverlapped) { + return ["Their execution windows did not overlap."] + } + return [ + "Their execution windows overlapped, so an unrelated identity", + "never waits behind this one.", + ] +} + +function cleanupDetail(summary: QuickstartSummary): string[] { + if (!summary.temporaryStateRemoved) { + return ["The temporary SQLite database is still on disk."] + } + return ["The scoped temporary SQLite database was deleted at exit."] +} diff --git a/examples/sqlite-quickstart.ts b/examples/sqlite-quickstart.ts index 819880c..c908ea2 100644 --- a/examples/sqlite-quickstart.ts +++ b/examples/sqlite-quickstart.ts @@ -6,6 +6,17 @@ import { join, resolve } from "node:path" import { fileURLToPath } from "node:url" import { Actor, createRuntime } from "solid-objects" import { sqlite } from "solid-objects/database/sqlite" +import { createInterface } from "node:readline/promises" +import { + answerAllowsRun, + formatQuickstartPlan, + formatQuickstartPrompt, + formatQuickstartReport, + formatQuickstartStop, + type QuickstartSummary, +} from "./quickstart-report.js" + +const SAME_IDENTITY_CALLS = 25 class Counter extends Actor { static override readonly actorType = "QuickstartCounter" @@ -27,12 +38,61 @@ class Counter extends Actor { } } +const COUNTER_SOURCE = `class Counter extends Actor { + static override readonly actorType = "QuickstartCounter" + + count = 0 + + increment(): number { + this.count += 1 + return this.count + } + + async pause({ milliseconds }: { milliseconds: number }) { + const startedAt = performance.now() + await new Promise((done) => setTimeout(done, milliseconds)) + return { startedAt, finishedAt: performance.now() } + } +}` + +async function confirmRun(options: { + format?: "report" | "json" + confirm?: () => boolean | Promise +}): Promise { + if (options.format === "json") return true + if (options.confirm) return options.confirm() + if (!process.stdin.isTTY) return true + const questions = createInterface({ input: process.stdin, output: process.stdout }) + try { + return answerAllowsRun(await questions.question(formatQuickstartPrompt())) + } catch { + return false + } finally { + questions.close() + } +} + export async function runQuickstart( options: { signal?: AbortSignal write?: (value: string) => void + format?: "report" | "json" + confirm?: () => boolean | Promise } = {}, ): Promise { + const write = options.write ?? ((value: string) => process.stdout.write(value)) + if (options.format !== "json") { + write( + formatQuickstartPlan({ + sameIdentityCalls: SAME_IDENTITY_CALLS, + actorSource: COUNTER_SOURCE, + }), + ) + } + if (!(await confirmRun(options))) { + write(formatQuickstartStop()) + return + } const directory = await mkdtemp(join(tmpdir(), "solid-objects-quickstart-")) const databasePath = join(directory, "state.sqlite3") const runtime = createRuntime({ @@ -58,13 +118,15 @@ export async function runQuickstart( running = runtime.run(shutdown.signal) const counter = runtime.ref(Counter, "room-1") - const results = await Promise.all(Array.from({ length: 25 }, () => counter.increment())) + const results = await Promise.all( + Array.from({ length: SAME_IDENTITY_CALLS }, () => counter.increment()), + ) assert.deepEqual( [...results].sort((left, right) => left - right), - Array.from({ length: 25 }, (_value, index) => index + 1), + Array.from({ length: SAME_IDENTITY_CALLS }, (_value, index) => index + 1), ) sameIdentityFinalState = await counter.count - assert.equal(sameIdentityFinalState, 25) + assert.equal(sameIdentityFinalState, SAME_IDENTITY_CALLS) const pauses = await Promise.all([ runtime.ref(Counter, "room-2").send.pause({ milliseconds: 100 }), @@ -89,19 +151,17 @@ export async function runQuickstart( } assert.equal(existsSync(directory), false) - const write = options.write ?? ((value: string) => process.stdout.write(value)) - write( - `${JSON.stringify( - { - sameIdentityCalls: 25, - sameIdentityFinalState, - independentIdentitiesOverlapped, - temporaryStateRemoved: true, - }, - null, - 2, - )}\n`, - ) + const summary: QuickstartSummary = { + sameIdentityCalls: SAME_IDENTITY_CALLS, + sameIdentityFinalState, + independentIdentitiesOverlapped, + temporaryStateRemoved: true, + } + if (options.format === "json") { + write(`${JSON.stringify(summary, null, 2)}\n`) + return + } + write(formatQuickstartReport(summary)) } if (process.argv[1] && fileURLToPath(import.meta.url) === resolve(process.argv[1])) { diff --git a/scripts/release-artifact-smoke.mjs b/scripts/release-artifact-smoke.mjs index b171520..f17b72e 100644 --- a/scripts/release-artifact-smoke.mjs +++ b/scripts/release-artifact-smoke.mjs @@ -67,18 +67,59 @@ try { assert(resolvedModule.includes("/node_modules/solid-objects/dist/index.js")) assert.equal(resolvedModule.startsWith(`file://${repositoryRoot}`), false) - const quickstart = await run( + const quickstartJson = await run( join(projectDirectory, "node_modules/.bin/solid-objects"), - ["quickstart"], + ["quickstart", "--json"], { cwd: projectDirectory }, ) - const result = JSON.parse(quickstart) + const result = JSON.parse(quickstartJson) assert.deepEqual(result, { sameIdentityCalls: 25, sameIdentityFinalState: 25, independentIdentitiesOverlapped: true, temporaryStateRemoved: true, }) + + const quickstartReport = await run( + join(projectDirectory, "node_modules/.bin/solid-objects"), + ["quickstart"], + { cwd: projectDirectory }, + ) + for (const expectedText of [ + "This command will:", + "send 25 concurrent calls to one identity;", + "The actor it runs:", + "class Counter extends Actor {", + "PASS 25 concurrent calls to one identity", + "PASS Two different identities ran at the same time", + "PASS Temporary state removed", + "What each PASS means", + "npm install solid-objects", + "where would the solid-objects library be best used in this app?", + ]) { + assert(quickstartReport.includes(expectedText), `quickstart report is missing ${expectedText}`) + } + assert( + quickstartReport.indexOf("This command will:") < + quickstartReport.indexOf("PASS 25 concurrent calls to one identity"), + "quickstart must state its plan before it reports results", + ) + assert.equal( + quickstartReport.includes("Run it now?"), + false, + "quickstart must not wait for an answer when stdin is not a terminal", + ) + + const piped = await run( + "bash", + [ + "-c", + 'set -o pipefail; "$0" quickstart | head -3', + join(projectDirectory, "node_modules/.bin/solid-objects"), + ], + { cwd: projectDirectory }, + ) + assert(piped.includes("Solid Objects quickstart"), "a closed pipe must still print the heading") } finally { await rm(temporaryDirectory, { recursive: true }) } diff --git a/src/broken-pipe.ts b/src/broken-pipe.ts new file mode 100644 index 0000000..df229d2 --- /dev/null +++ b/src/broken-pipe.ts @@ -0,0 +1,11 @@ +import type { EventEmitter } from "node:events" + +export function ignoreBrokenPipe( + stream: EventEmitter, + options: { onBrokenPipe: () => void }, +): void { + stream.on("error", (error: NodeJS.ErrnoException) => { + if (error.code !== "EPIPE") throw error + options.onBrokenPipe() + }) +} diff --git a/src/cli.ts b/src/cli.ts index 877e78c..7069aa2 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -39,7 +39,7 @@ export async function runCli( const parsed = parseArguments(commandArguments) const write = options.write ?? ((value: string) => process.stdout.write(value)) if (command === "quickstart") { - assertOptions(parsed, { command }) + assertOptions(parsed, { command, flags: ["json", "yes"] }) assertNoPositionals(parsed, command) const module = (await import( new URL("./examples/sqlite-quickstart.js", import.meta.url).href @@ -47,11 +47,15 @@ export async function runCli( runQuickstart(options: { signal?: AbortSignal write: (value: string) => void + format: "report" | "json" + confirm?: () => boolean }): Promise } await module.runQuickstart({ ...(options.signal === undefined ? {} : { signal: options.signal }), + ...(parsed.flags.has("yes") ? { confirm: () => true } : {}), write, + format: parsed.flags.has("json") ? "json" : "report", }) return 0 } @@ -174,7 +178,7 @@ function parseArguments(argumentsValue: readonly string[]): ParsedArguments { continue } const name = argument === "-c" ? "config" : argument.slice(2) - if (name === "execute" || name === "skip-round-trip") { + if (name === "execute" || name === "skip-round-trip" || name === "json" || name === "yes") { flags.add(name) continue } @@ -250,7 +254,7 @@ function help(): string { return `Usage: solid-objects [options] Commands: - quickstart + quickstart [--json] [--yes] start doctor [--skip-round-trip] status diff --git a/src/executable.ts b/src/executable.ts index 935519f..ed4e479 100644 --- a/src/executable.ts +++ b/src/executable.ts @@ -1,7 +1,12 @@ #!/usr/bin/env node +import { ignoreBrokenPipe } from "./broken-pipe.js" import { runCli } from "./cli.js" +const endOnBrokenPipe = { onBrokenPipe: () => process.exit(0) } +ignoreBrokenPipe(process.stdout, endOnBrokenPipe) +ignoreBrokenPipe(process.stderr, endOnBrokenPipe) + const shutdown = new AbortController() process.once("SIGINT", () => shutdown.abort()) process.once("SIGTERM", () => shutdown.abort()) diff --git a/test/broken-pipe.test.ts b/test/broken-pipe.test.ts new file mode 100644 index 0000000..1789dc2 --- /dev/null +++ b/test/broken-pipe.test.ts @@ -0,0 +1,29 @@ +import { EventEmitter } from "node:events" +import { describe, expect, it } from "vitest" +import { ignoreBrokenPipe } from "../src/broken-pipe.js" + +function errorWithCode(code: string): NodeJS.ErrnoException { + const error: NodeJS.ErrnoException = new Error(code) + error.code = code + return error +} + +describe("ignoreBrokenPipe", () => { + it("reports a broken pipe instead of throwing", () => { + const stream = new EventEmitter() + let brokenPipes = 0 + ignoreBrokenPipe(stream, { onBrokenPipe: () => (brokenPipes += 1) }) + + expect(() => stream.emit("error", errorWithCode("EPIPE"))).not.toThrow() + expect(brokenPipes).toBe(1) + }) + + it("leaves every other stream error alone", () => { + const stream = new EventEmitter() + let brokenPipes = 0 + ignoreBrokenPipe(stream, { onBrokenPipe: () => (brokenPipes += 1) }) + + expect(() => stream.emit("error", errorWithCode("ENOSPC"))).toThrow("ENOSPC") + expect(brokenPipes).toBe(0) + }) +}) diff --git a/test/quickstart-report.test.ts b/test/quickstart-report.test.ts new file mode 100644 index 0000000..e0a42ea --- /dev/null +++ b/test/quickstart-report.test.ts @@ -0,0 +1,131 @@ +import { describe, expect, it } from "vitest" +import { + answerAllowsRun, + formatQuickstartPlan, + formatQuickstartPrompt, + formatQuickstartReport, + formatQuickstartStop, +} from "../examples/quickstart-report.js" + +const passing = { + sameIdentityCalls: 25, + sameIdentityFinalState: 25, + independentIdentitiesOverlapped: true, + temporaryStateRemoved: true, +} + +const actorSource = `class Counter extends Actor { + count = 0 + + increment(): number { + this.count += 1 + return this.count + } +}` + +describe("formatQuickstartPlan", () => { + it("shows the actor definition that the run will use", () => { + const plan = formatQuickstartPlan({ sameIdentityCalls: 25, actorSource }) + + expect(plan).toContain("The actor it runs") + expect(plan).toContain("class Counter extends Actor {") + expect(plan).toContain(" this.count += 1") + }) + + it("says what the command will do before it does it", () => { + const plan = formatQuickstartPlan({ sameIdentityCalls: 25, actorSource }) + + expect(plan).toContain("This command will") + expect(plan).toContain("create a temporary SQLite database") + expect(plan).toContain("send 25 concurrent calls to one identity") + expect(plan).toContain("run two other identities at the same time") + expect(plan).toContain("delete the temporary database") + }) + + it("states that the run needs no server and touches nothing else", () => { + const plan = formatQuickstartPlan({ sameIdentityCalls: 25, actorSource }) + + expect(plan).toContain("No server, container, or configuration is needed.") + }) +}) + +describe("the run confirmation", () => { + it("asks before it does the work", () => { + expect(formatQuickstartPrompt()).toContain("Run it now?") + }) + + it("accepts an empty answer and a yes", () => { + expect(answerAllowsRun("")).toBe(true) + expect(answerAllowsRun(" ")).toBe(true) + expect(answerAllowsRun("y")).toBe(true) + expect(answerAllowsRun("Y")).toBe(true) + expect(answerAllowsRun("yes")).toBe(true) + }) + + it("refuses any other answer", () => { + expect(answerAllowsRun("n")).toBe(false) + expect(answerAllowsRun("no")).toBe(false) + expect(answerAllowsRun("q")).toBe(false) + expect(answerAllowsRun("later")).toBe(false) + }) + + it("says that nothing ran when the answer refuses", () => { + expect(formatQuickstartStop()).toContain("Nothing ran.") + }) +}) + +describe("formatQuickstartReport", () => { + it("explains what the passing checks mean for an application", () => { + const report = formatQuickstartReport(passing) + + expect(report).toContain("What each PASS means") + expect(report).toContain("cannot overwrite each other") + expect(report).toContain("one durable mailbox") + expect(report).toContain("The order is per identity, not global.") + }) + + it("omits the meaning section when a check did not hold", () => { + const report = formatQuickstartReport({ ...passing, temporaryStateRemoved: false }) + + expect(report).not.toContain("What each PASS means") + }) + + it("explains what each assertion proved", () => { + const report = formatQuickstartReport(passing) + + expect(report).toContain("25 concurrent calls to one identity") + expect(report).toContain("committed state is 25") + expect(report).toContain("return values were the complete sequence 1 through 25") + expect(report).toContain("Two different identities ran at the same time") + expect(report).toContain("temporary SQLite database") + expect(report.match(/^PASS /gm)).toHaveLength(3) + }) + + it("marks a check that did not hold", () => { + const report = formatQuickstartReport({ ...passing, independentIdentitiesOverlapped: false }) + + expect(report).toContain("FAIL") + expect(report.match(/^PASS /gm)).toHaveLength(2) + }) + + it("reports the final state that the run committed", () => { + const report = formatQuickstartReport({ + ...passing, + sameIdentityCalls: 4, + sameIdentityFinalState: 4, + }) + + expect(report).toContain("4 concurrent calls to one identity") + expect(report).toContain("committed state is 4") + expect(report).toContain("sequence 1 through 4") + }) + + it("tells the reader how to add the package to an application", () => { + const report = formatQuickstartReport(passing) + + expect(report).toContain("Add it to your app") + expect(report).toContain("npm install solid-objects") + expect(report).toContain("Or tell your agent") + expect(report).toContain("where would the solid-objects library be best used in this app?") + }) +}) diff --git a/tsconfig.quickstart-build.json b/tsconfig.quickstart-build.json index 71b5001..a2660d8 100644 --- a/tsconfig.quickstart-build.json +++ b/tsconfig.quickstart-build.json @@ -8,5 +8,5 @@ "sourceMap": true, "noEmit": false }, - "include": ["examples/sqlite-quickstart.ts"] + "include": ["examples/sqlite-quickstart.ts", "examples/quickstart-report.ts"] }