From f5b4dfabd78eb3d0d04a5e7377ecc42f0c000b92 Mon Sep 17 00:00:00 2001 From: ElectronSz Date: Tue, 15 Sep 2026 11:26:20 +0200 Subject: [PATCH 1/5] Add SQL Server support, relationships, encryption and auto-migrate - Microsoft SQL Server as a first-class dialect (mssql v12): T-SQL query generation (OUTPUT INSERTED.*, OFFSET/FETCH pagination, MERGE upserts), schema generation via OBJECT_ID and sys.indexes probes in place of the CREATE IF NOT EXISTS forms SQL Server lacks, and automatic rewriting of the library's ? placeholders to @param0. - Model relationships: OneToOne, ManyToOne, OneToMany and ManyToMany, declared in the model config and eager-loaded in batched follow-up queries rather than per row. attach/detach/sync edit a join table without loading either side. - Column encryption: marking a column encrypted encrypts on write and decrypts on read with AES-256-GCM, including for rows served from cache. The key is read from ORM_ENCRYPTION_KEY with no fallback, so a missing key fails loudly instead of silently protecting nothing. - auto-migrate.ts: additive AutoMigrate that creates missing tables, adds missing columns and adds missing indexes. It never drops a column and never changes a column type. - validateAll, findOrFail/firstOrFail, raw clause builders, and connection retry with exponential backoff and jitter for read-only statements. Fixed: - after* hooks and both delete hooks never ran: getHooks() resolved the model through proto.constructor, but reads return plain rows, so the prototype was Object and the metadata lookup found nothing. - getRepository() opened a new, unclosed Redis connection on every call. - Cached find() results were written without their relations, so a cache hit returned a different shape than a miss. Adds test coverage for all of the above, plus integration suites for sqlite, mysql, mariadb, postgres, mssql, models, relations and the write paths, which skip themselves when no server answers. The docs move to their own repository (ElectronSz/stabilize-docs) and are now ignored here. --- .gitignore | 3 + CHANGELOG.md | 48 + README.md | 381 ++-- auto-migrate.ts | 698 ++++--- bun.lock | 217 +- client.ts | 373 +++- docker-compose.test.yml | 111 ++ hooks.ts | 19 +- index.ts | 34 +- migrations.ts | 211 +- model.ts | 81 +- mssql.d.ts | 52 + package.json | 6 +- query-builder.ts | 310 ++- repository.ts | 1974 +++++++++++++++---- stabilize-docs | 1 - tests/auto-migrate.primary-key.test.ts | 128 ++ tests/client.retry.test.ts | 89 + tests/encryption.test.ts | 106 + tests/integration.mariadb.test.ts | 529 +++++ tests/integration.models.test.ts | 196 ++ tests/integration.mssql.test.ts | 311 +++ tests/integration.mysql.test.ts | 391 ++++ tests/integration.postgres.test.ts | 292 +++ tests/integration.relations.test.ts | 508 +++++ tests/integration.sqlite.test.ts | 125 ++ tests/integration.write-paths.test.ts | 334 ++++ tests/metadata.bundle-boundary.test.ts | 172 ++ tests/mssql.dialect.test.ts | 431 ++++ tests/repository.cache-keys.test.ts | 164 ++ tests/repository.features.test.ts | 348 ++++ tests/repository.transaction-client.test.ts | 102 + tests/repository.validation.test.ts | 27 +- types.ts | 1 + utils/encryption.ts | 157 +- 35 files changed, 7958 insertions(+), 972 deletions(-) create mode 100644 docker-compose.test.yml create mode 100644 mssql.d.ts delete mode 160000 stabilize-docs create mode 100644 tests/auto-migrate.primary-key.test.ts create mode 100644 tests/client.retry.test.ts create mode 100644 tests/encryption.test.ts create mode 100644 tests/integration.mariadb.test.ts create mode 100644 tests/integration.models.test.ts create mode 100644 tests/integration.mssql.test.ts create mode 100644 tests/integration.mysql.test.ts create mode 100644 tests/integration.postgres.test.ts create mode 100644 tests/integration.relations.test.ts create mode 100644 tests/integration.sqlite.test.ts create mode 100644 tests/integration.write-paths.test.ts create mode 100644 tests/metadata.bundle-boundary.test.ts create mode 100644 tests/mssql.dialect.test.ts create mode 100644 tests/repository.cache-keys.test.ts create mode 100644 tests/repository.features.test.ts create mode 100644 tests/repository.transaction-client.test.ts diff --git a/.gitignore b/.gitignore index e9dc2fa..bdb14b5 100644 --- a/.gitignore +++ b/.gitignore @@ -32,3 +32,6 @@ report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json # Finder (MacOS) folder config .DS_Store + +# Documentation lives in its own repository (ElectronSz/stabilize-docs) +stabilize-docs/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 247e927..946fb5a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,54 @@ All notable changes to this project will be documented in this file. - Further features and improvements coming soon. +## [2.2.1] - 2026-09-15 + +Documentation-only release. No library code changed from 2.2.0. + +### Changed + +- **`README.md` now documents SQL Server.** 2.2.0 added Microsoft SQL Server as a first-class dialect but the README still described only PostgreSQL, MySQL and SQLite — there was not a single mention of SQL Server anywhere in it. Added a SQL Server badge, corrected the intro and the feature list, and documented that `DBType.MSSQL` uses the `mssql` v12 driver with T-SQL-specific query generation. +- Corrected the CLI badge (was pinned at 2.1.0) and pointed the license badge at `stabilize-orm`; it previously linked to the `stabilize-cli` repository. + +## [2.2.0] - 2026-09-14 + +### Added + +- **Microsoft SQL Server support** - SQL Server is now a first-class dialect alongside SQLite, MySQL and PostgreSQL. `DBType.MSSQL` selects the `mssql` driver (v12), and the placeholder rewriter maps the library's internal `?` placeholders to `@param0`-style named parameters automatically. + - T-SQL-specific query generation: `OUTPUT INSERTED.*` for reads-after-write, `OFFSET … FETCH` pagination with an `ORDER BY (SELECT NULL)` fallback when no ordering is supplied, and `MERGE INTO … USING … WHEN MATCHED / WHEN NOT MATCHED … OUTPUT INSERTED.*` for upserts. + - Schema generation uses `IF OBJECT_ID(…) IS NULL CREATE TABLE` in place of `CREATE TABLE IF NOT EXISTS`, and a `sys.indexes` probe in place of `CREATE INDEX IF NOT EXISTS`, neither of which SQL Server has. + - `poolStats()` reports borrowed / available / size for SQL Server connection pools. + - Connection strings use a comma between host and port: `Server=host,port;User Id=sa;Password=…;Database=…;TrustServerCertificate=true`. +- **Model Relationships** - `OneToOne`, `ManyToOne`, `OneToMany` and `ManyToMany` are declared in the model configuration and eager-loaded either by passing `relations` to a finder or by chaining `.withRelations()` onto the query builder. Relations are resolved in batched follow-up queries rather than per-row. +- **Many-to-many link management** - `attach()`, `detach()` and `sync()` edit a join table directly, without loading either side of the relationship into memory. +- **`validateAll()`** - runs every validator on an entity and returns all failures at once, instead of throwing on the first one. `validate()` keeps its fail-fast behaviour. +- **`findOrFail()` / `firstOrFail()`** - throw a `StabilizeError` with code `NOT_FOUND_ERROR` rather than returning `null`. +- **Raw clause builders** - `.orderByRaw()`, `.groupByRaw()` and `.havingRaw()` accept expressions that are not bare column names. +- **`auto-migrate.ts`** - a GORM-style `AutoMigrate` that creates missing tables, adds missing columns and adds missing indexes. It never drops columns and never alters column types - it only adds. +- **Column encryption** - `utils/encryption.ts` exposes `encrypt()` / `decrypt()`; marking a column as encrypted transparently encrypts on write and decrypts on read, including for rows served from cache. +- **Connection retry with exponential backoff and jitter** - `retryAttempts` (default 3) and `retryDelay` (default 1000ms) on the client config. Retries apply to read-only statements only; a failed write is never silently replayed. +- **`auto-migrate.primary-key` and `client.retry` test suites**, plus eight integration suites (`sqlite`, `mysql`, `mariadb`, `postgres`, `mssql`, `models`, `relations`, `write-paths`) that run against real database servers and skip themselves when no server answers. + +### Fixed + +- **`after*` hooks and both delete hooks never ran.** `getHooks()` looked the model up via `proto.constructor`, but reads return plain rows from the driver, so the prototype was `Object` and the metadata lookup found nothing. The lookup now resolves the caller's model when one is passed, and hooks take a `model` argument for exactly this case. Any code that relied on `afterCreate` / `afterUpdate` / `afterDelete` firing will now see it fire. +- **`getRepository()` opened a second, disconnected Redis connection on every call.** Repositories are now memoised per model, so repeated calls return the same instance instead of leaking a connection that nothing closed and `getCacheStats()` never counted. +- **Cached `find()` results were written without their relations.** Rows are now hydrated and have relations loaded before being written to the cache, so a cache hit returns the same shape as a cache miss. +- **`orderByRaw` / `groupByRaw` / `havingRaw` parameter handling**, and `OFFSET`-based pagination on dialects that require an `ORDER BY`. + +### Changed + +- `DBType` gains an `MSSQL` member. Since `DBType` is a string enum, existing persisted values are unaffected. +- `mssql` (v12) is now a runtime dependency. + +### Known Issues + +- **SQL Server `IDENTITY` columns reject an explicit primary key.** `create({ id: 99 })` fails with *"Cannot insert explicit value for identity column in table 'x' when IDENTITY_INSERT is set to OFF"*, and `upsert({ id: 98 }, ["title"])` fails with *"Cannot update identity column 'id'"*. The cause is that the generated `MERGE` statement places the identity column in both the `UPDATE SET` list and the `INSERT` column list, and SQL Server permits neither. Letting the database assign the key works correctly on every path. A fix requires the upsert builder to skip identity columns in the `UPDATE SET` list and rely on `OUTPUT INSERTED.*` for the generated value; it is documented rather than patched here so that the behaviour change is not shipped silently in a minor release. +- **`create()` with a JSON object throws on SQLite.** The SQLite write path binds values as given, while MySQL and SQL Server encode explicitly, so an object value reaches `bun:sqlite` unencoded and it rejects it with *"Binding expected string, TypedArray, boolean, number, bigint or null"*. Passing a pre-stringified value works on all three. +- **Declared column `length`, `precision` and `scale` are not enforced** at write time - they appear in the generated DDL only, and SQLite ignores them entirely. +- **`getValidators()` reports only `required` and `unique`**; other validators run during `validate()` but are not reflected in that summary. +- **A `STRING` primary key is generated as `UUID PRIMARY KEY` on PostgreSQL** but as `VARCHAR(255)` / `NVARCHAR(255)` / `TEXT` on the other dialects, so the column types differ across dialects for the same model. + ## [2.1.0] - 2026-04-02 ### Added diff --git a/README.md b/README.md index 6e8d139..d0ac12c 100644 --- a/README.md +++ b/README.md @@ -2,51 +2,54 @@ _A Modern, Type-Safe, and Expressive ORM for Bun_ -

- Stabilize ORM Logo -

-

NPM Version - License - Stabilize CLI + License + Stabilize CLI PostgreSQL MySQL SQLite + SQL Server Build Status MIT License

-**Stabilize** is a lightweight, feature-rich ORM designed for performance and developer experience. It provides a unified, database-agnostic API for **PostgreSQL**, **MySQL**, and **SQLite**. Powered by a robust query builder, programmatic model definitions, automatic versioning, and a full-featured command-line interface, Stabilize is built to scale with your app. +**Stabilize** is a lightweight, feature-rich ORM designed for performance and developer experience. It provides a unified, database-agnostic API for **PostgreSQL**, **MySQL/MariaDB**, **SQLite**, and **SQL Server**. Powered by a robust query builder, programmatic model definitions, automatic versioning, and a full-featured command-line interface, Stabilize is built to scale with your app. --- ## 🚀 Features -- **Unified API**: Write once, run on PostgreSQL, MySQL, or SQLite. +- **Unified API**: Write once, run on PostgreSQL, MySQL/MariaDB, SQLite, or SQL Server. - **Programmatic Model Definitions**: Define models and columns using the `defineModel` API with the `DataTypes` enum for database-agnostic schemas. - **Full-Featured CLI**: Generate models, manage migrations, seed data, and reset your database from the command line with [stabilize-cli](https://github.com/ElectronSz/stabilize-cli). - **Automatic Migrations**: Generate database-specific SQL schemas directly from your model definitions. +- **First-Class SQL Server Support**: `DBType.MSSQL` selects the `mssql` v12 driver, with T-SQL-specific query generation (`OUTPUT INSERTED.*`, `OFFSET … FETCH`, `MERGE` for upserts) and schema generation via `OBJECT_ID` / `sys.indexes` probes. Placeholders are rewritten automatically — you keep writing `?`. - **Versioned Models & Time-Travel**: Enable versioning in your model configuration for automatic history tables and snapshot queries. - **Retry Logic**: Automatic exponential backoff for database queries to handle transient connection issues. -- **Connection Pooling**: Efficient connection management for PostgreSQL and MySQL. +- **Connection Pooling**: Efficient connection management for PostgreSQL, MySQL, and SQL Server, with `poolStats()` reporting borrowed/available/size on SQL Server. - **Transactional Integrity**: Built-in support for atomic transactions with automatic rollback on failure. - **Advanced Query Builder**: Fluent, chainable API for building complex queries, including joins, filters, ordering, and pagination. - **Pagination Helper**: Easily paginate any query with `.paginate(page, pageSize)` and get `{ data, total, page, pageSize }`. - **Advanced Model Validation**: Enforce rules like `required`, `minLength`, `maxLength`, `pattern`, and custom validators—errors are thrown on invalid input. -- **Model Relationships**: Define `OneToOne`, `ManyToOne`, `OneToMany`, and `ManyToMany` relationships in the model configuration. +- **Model Relationships**: Define `OneToOne`, `ManyToOne`, `OneToMany`, and `ManyToMany` relationships in the model configuration, eager-loaded with `relations` or `.withRelations()`. +- **Many-to-Many Link Management**: `attach()`, `detach()` and `sync()` edit a join table directly, without loading either side. +- **validateAll**: Collect every validation failure at once, rather than only the first. +- **findOrFail / firstOrFail**: Throw a `NOT_FOUND_ERROR` instead of returning `null`. +- **Raw Clause Builders**: `.orderByRaw()`, `.groupByRaw()` and `.havingRaw()` for expressions that are not column names. - **Soft Deletes**: Enable soft deletes in the model configuration for transparent "deleted" flags and safe row removal. - **Lifecycle Hooks**: Define hooks in the model configuration or as class methods for lifecycle events like `beforeCreate`, `afterUpdate`, etc. - **Pluggable Logging**: Includes a robust `StabilizeLogger` with support for file-based, rotating logs. - **Custom Errors**: `StabilizeError` provides clear, consistent error handling. - **Caching Layer**: Optional Redis-backed caching with `cache-aside` and `write-through` strategies. +- **Column Encryption**: Mark a column with `encrypted: true` to transparently encrypt it on write and decrypt it on read with AES-256-GCM — including for rows served from cache. - **Custom Query Scopes**: Define reusable query conditions (scopes) in models for simplified, reusable filtering logic. - **Timestamps**: Automatically manage `createdAt` and `updatedAt` columns for tracking record creation and update times. - **SQL Default Expressions**: Support database-side default expressions (e.g., `gen_random_uuid()`, `NOW()`) for columns using the `sqlDefault()` helper. - **Nested Relations (Eager Loading)**: Load deeply nested relations using dot notation like `"roles.permissions"`. -- **AutoMigrate with Index Management**: Automatically create, detect, and remove indexes and unique constraints during migration. +- **AutoMigrate with Index Management**: Create missing tables, add missing columns, and add missing indexes and unique constraints in one pass. It is additive only — it never drops a column, never changes a column type, and never removes an index. - **Advanced Query Builder Filters**: Chainable `.orWhere()`, `.whereIn()`, `.whereNotIn()`, `.whereNull()`, `.whereNotNull()`, `.whereBetween()`, `.groupBy()`, `.having()`, `.lock()` methods. - **Optimistic Locking**: Add `optimisticLock: true` to a version column to automatically detect concurrent modification conflicts and throw `CONCURRENT_MODIFICATION` errors. - **findAndCount**: Get paginated results with a total count in one call. @@ -185,9 +188,9 @@ const User = defineModel({ tableName: "users", versioned: true, columns: { - id: { type: DataTypes.Integer, required: true }, + id: { type: DataTypes.INTEGER, required: true }, email: { - type: DataTypes.String, + type: DataTypes.STRING, length: 100, required: true, unique: true, @@ -198,7 +201,9 @@ const User = defineModel({ type: RelationType.OneToMany, target: () => UserRole, property: "roles", - foreignKey: "userId", + // The key lives on the target table, so the OneToMany side names it + // with inverseKey. `foreignKey` is accepted as a synonym here. + inverseKey: "userId", }, ], hooks: { @@ -247,9 +252,9 @@ Validation errors are thrown on create/update if data is invalid. const User = defineModel({ tableName: "users", columns: { - id: { type: DataTypes.Integer, required: true }, + id: { type: DataTypes.INTEGER, required: true }, email: { - type: DataTypes.String, + type: DataTypes.STRING, required: true, unique: true, minLength: 6, @@ -258,15 +263,71 @@ const User = defineModel({ val.endsWith("@offbytesecure.com") || "Must use an @offbytesecure.com email", }, - password: { type: DataTypes.String, minLength: 8 }, + password: { type: DataTypes.STRING, minLength: 8 }, + }, +}); +``` + +--- + +## 🔐 Column Encryption + +Mark a column with `encrypted: true` and the ORM encrypts it on the way in and +decrypts it on the way out. Your code keeps reading and writing ordinary +strings; the ciphertext is only ever visible in the database. + +```typescript +const User = defineModel({ + tableName: "users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, required: true, unique: true }, + nationalId: { type: DataTypes.STRING, encrypted: true }, }, }); + +await userRepository.create({ + email: "lwazicd@icloud.com", + nationalId: "9001015800085", // stored as v2::: +}); + +const user = await userRepository.findOne(1); +console.log(user.nationalId); // "9001015800085" — decrypted on read ``` +Encryption uses AES-256-GCM, so a value that has been tampered with or truncated +fails to decrypt rather than quietly returning corrupted plaintext. Values carry +a `v2:` prefix that identifies the format. Encrypted columns are also decrypted +when a row is served from cache, so a cache hit returns the same shape as a miss. + +### The encryption key + +The key is read from the `ORM_ENCRYPTION_KEY` environment variable on every call, +so it can be set after your modules are imported: + +```bash +# 32 bytes, or 64 hex characters +export ORM_ENCRYPTION_KEY="$(openssl rand -hex 32)" +``` + +There is deliberately no default. **If `ORM_ENCRYPTION_KEY` is unset, reading or +writing an encrypted column throws** rather than falling back to a built-in key. +Earlier versions did fall back to a constant compiled into the package, which +meant a deployment that never set the variable encrypted its columns with a value +anyone who read the published source could reproduce. Failing loudly is the +intended behaviour: it is a deployment mistake, not a runtime condition. + +If you have data written by one of those earlier versions, set the key to the +legacy value `f71a3c8e9b12d5a49c0a3f98b1f2e46d` to keep reading it, then re-save +those rows under a key of your own. Those rows use AES-CBC and are still read +correctly; everything newly written uses GCM. + --- ## ⏳ Versioning & Auditing +## ⏳ Versioning & Auditing + Enable automatic history tracking and time-travel queries by setting `versioned: true` in your model configuration. - Each change is recorded in a `_history` table with version, operation, and audit columns. @@ -281,8 +342,8 @@ const User = defineModel({ tableName: "users", versioned: true, columns: { - id: { type: DataTypes.Integer, required: true }, - name: { type: DataTypes.String, length: 100 }, + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, length: 100 }, }, }); @@ -316,10 +377,10 @@ import { defineModel, DataTypes } from "stabilize-orm"; const User = defineModel({ tableName: "users", columns: { - id: { type: DataTypes.Integer, required: true }, - name: { type: DataTypes.String, length: 100 }, - createdAt: { type: DataTypes.DateTime }, - updatedAt: { type: DataTypes.DateTime }, + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, length: 100 }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, }, hooks: { beforeCreate: (entity) => { @@ -348,101 +409,67 @@ Supported hooks: `beforeCreate`, `afterCreate`, `beforeUpdate`, `afterUpdate`, ` ## 💻 Command-Line Interface (CLI) -Stabilize includes a powerful CLI for managing your workflow. See: [stabilize-cli on GitHub](https://github.com/ElectronSz/stabilize-cli) - -### Generating Files - -- **Generate a model**: - - ```bash - stabilize-cli generate:model Product - ``` - -- **Generate a migration from a model**: - - ```bash - stabilize-cli generate:migration User - ``` - -- **Generate a seed file**: - - ```bash - stabilize-cli generate:seed InitialRoles - ``` - -- **Generate a REST API scaffold**: - ```bash - stabilize-cli generate:api User - ``` - -### Database & Migration Management - -- **Run all pending migrations**: - - ```bash - stabilize-cli migrate - ``` +Stabilize includes a powerful CLI with 31 commands. See: [stabilize-cli on GitHub](https://github.com/ElectronSz/stabilize-cli) -- **Roll back the last migration**: +### Generate - ```bash - stabilize-cli migrate:rollback - ``` - -- **Fresh migration (drop + re-migrate)**: - - ```bash - stabilize-cli migrate:fresh --force - ``` - -- **Run all pending seeds (in dependency order)**: - - ```bash - stabilize-cli seed - ``` +```bash +stabilize-cli generate:model User name:string email:string age:int --versioned # g:m +stabilize-cli generate:migration User # g:mg +stabilize-cli generate:seed User --count 10 # g:s +stabilize-cli generate:api Product --prefix /v1 # g:a +stabilize-cli generate:all Order userId:string total:decimal --count 20 # g:x +stabilize-cli generate:test User # g:t +``` -- **Check the status of migrations and seeds**: +### Migrate - ```bash - stabilize-cli status - ``` +```bash +stabilize-cli migrate +stabilize-cli migrate:rollback +stabilize-cli migrate:fresh --force +stabilize-cli migrate:status +stabilize-cli migrate:pending +stabilize-cli migrate:auto +``` -- **Reset the database (drop, migrate, seed)**: - ```bash - stabilize-cli db:reset - ``` +`migrate:auto` runs AutoMigrate against the live database — creating missing +tables, adding missing columns and adding missing indexes. Like the library API +it is additive only: it never drops a column and never changes a column type. -### Backup & Restore +### Database -- **Backup the database**: +```bash +stabilize-cli db:drop --force +stabilize-cli db:reset --force +stabilize-cli db:truncate users --force +stabilize-cli db:backup --output ./backups +stabilize-cli db:restore backups/backup.db --force +stabilize-cli db:tables +stabilize-cli db:size +stabilize-cli db:diff +stabilize-cli db:console +stabilize-cli db:table:info users +``` - ```bash - stabilize-cli db:backup - ``` +### Model & Config -- **Restore from a backup**: - ```bash - stabilize-cli db:restore backups/backup_20250101120000.db --force - ``` +```bash +stabilize-cli model:validate +stabilize-cli model:info User +stabilize-cli config:init --type postgres +``` ### Diagnostics -- **Database size statistics**: - - ```bash - stabilize-cli db:size - ``` - -- **Health check**: - - ```bash - stabilize-cli health - ``` - -- **CLI info**: - ```bash - stabilize-cli info - ``` +```bash +stabilize-cli seed +stabilize-cli status +stabilize-cli health +stabilize-cli health:json +stabilize-cli query 'SELECT * FROM users LIMIT 5' +stabilize-cli info +``` --- @@ -520,11 +547,11 @@ import { orm } from "./db"; const User = defineModel({ tableName: "users", columns: { - id: { type: DataTypes.Integer, required: true }, - email: { type: DataTypes.String, length: 100, required: true }, - isActive: { type: DataTypes.Boolean, required: true }, - createdAt: { type: DataTypes.DateTime }, - updatedAt: { type: DataTypes.DateTime }, + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, length: 100, required: true }, + isActive: { type: DataTypes.BOOLEAN, required: true }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, }, scopes: { active: (qb) => qb.where("isActive = ?", true), @@ -568,10 +595,10 @@ import { orm } from "./db"; const User = defineModel({ tableName: "users", columns: { - id: { type: DataTypes.Integer, required: true }, - email: { type: DataTypes.String, length: 100, required: true }, - createdAt: { type: DataTypes.DateTime }, - updatedAt: { type: DataTypes.DateTime }, + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, length: 100, required: true }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, }, timestamps: { createdAt: "createdAt", @@ -618,9 +645,9 @@ const User = defineModel({ tableName: "users", softDelete: true, columns: { - id: { type: DataTypes.Integer, required: true }, - email: { type: DataTypes.String, length: 100, required: true }, - deletedAt: { type: DataTypes.DateTime, softDelete: true }, + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, length: 100, required: true }, + deletedAt: { type: DataTypes.DATETIME, softDelete: true }, }, }); @@ -688,9 +715,9 @@ import { defineModel, DataTypes } from "stabilize-orm"; const User = defineModel({ tableName: "users", columns: { - id: { type: DataTypes.Integer, required: true }, - name: { type: DataTypes.String }, - version: { type: DataTypes.Integer, optimisticLock: true }, + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, }, }); @@ -730,9 +757,9 @@ const User = defineModel({ required: true, defaultExpression: sqlDefault("gen_random_uuid()"), }, - name: { type: DataTypes.String }, + name: { type: DataTypes.STRING }, createdAt: { - type: DataTypes.DateTime, + type: DataTypes.DATETIME, defaultExpression: sqlDefault("NOW()"), }, }, @@ -760,18 +787,118 @@ const results = await userRepository .execute(); ``` +### Raw Clauses + +`orderBy`, `groupBy` and `having` take a column name. When you need an +expression instead, use the raw variants: + +```typescript +const results = await orderRepository + .find() + .select("status", "COUNT(*) AS total") + .groupByRaw("strftime('%Y-%m', createdAt)") + .havingRaw("COUNT(*) > ?", 10) + .orderByRaw("CASE WHEN status = 'urgent' THEN 0 ELSE 1 END") + .execute(db.client); +``` + +`orderByRaw` takes an optional direction as its second argument +(`orderByRaw("LENGTH(title)", "DESC")`). Raw and plain clauses compose, and raw +parameters are bound in the order they appear. + --- -## 🔗 Nested Relations +## 🔗 Relations -Load deeply nested relations using dot notation: +Relations are eager-loaded, one batched query per relation. A to-many relation +comes back as an array (empty when there is nothing linked), a to-one relation +as the row or `null`. ```typescript const user = await userRepository.findOne(1, { relations: ["roles", "roles.permissions"], }); +// user.roles[0].permissions — nested paths use dot notation +``` + +Relations can also be requested on the query builder, alongside `where`, +`limit`, `orderBy` and `paginate`: + +```typescript +const users = await userRepository + .find() + .where("isActive = ?", true) + .withRelations("roles", "roles.permissions") + .limit(10) + .execute(db.client); ``` +`create`, `bulkCreate`, `findOne`, `findMany`, `findBy`, `findOneBy` and +`findAndCount` all accept a `relations` option: + +```typescript +const [user, post] = await userRepository.bulkCreate( + [{ email: "a@b.c" }, { email: "d@e.f" }], + { relations: ["roles"] }, +); +``` + +Related rows are read through the target model, so its soft-delete filter +applies — a deleted child is not returned as part of its parent. + +### Managing Many-to-Many Links + +`attach`, `detach` and `sync` write the join table directly, so a many-to-many +relation can be edited without loading and re-saving either side. + +```typescript +await postRepository.attach(postId, "tags", [1, 2]); // links 1 and 2 +await postRepository.detach(postId, "tags", [2]); // unlinks 2 +await postRepository.detach(postId, "tags"); // unlinks everything + +// Makes the link set exactly [3, 4]: adds what is missing, removes what is not +// in the list, leaves the rest alone. +const { attached, detached } = await postRepository.sync(postId, "tags", [3, 4]); +``` + +All three are idempotent, accept a single id or an array, and return how many +links they changed. `sync` runs in a transaction. + +--- + +## 🥇 findOrFail / firstOrFail + +`findOne` and `first` return `null` on a miss. The `*OrFail` variants throw a +`StabilizeError` with code `NOT_FOUND_ERROR` instead, so a miss cannot be +mistaken for an empty result. + +```typescript +const user = await userRepository.findOrFail(1); // never null +const admin = await userRepository.firstOrFail({ role: "admin" }); + +// Still loads relations +const post = await postRepository.findOrFail(7, { relations: ["author"] }); +``` + +--- + +## 🛡️ validateAll + +`create` and `update` validate and throw on the **first** failure. When +validating input from a form you usually want every failure at once: + +```typescript +const errors = userRepository.validateAll({ email: "nope", name: "ab" }); +// ["Field email does not match pattern", "Field name too short"] + +if (errors.length) { + return res.status(422).json({ errors }); +} +``` + +Pass `true` as the second argument to skip the `required` rules, which is how +an update validates a partial entity. + --- ## 📊 Aggregation Queries @@ -1084,7 +1211,7 @@ const id = generateUUID(); ## 📑 License -Licensed under the MIT License. See [LICENSE.md](./LICENSE.md) for details. +Licensed under the MIT License. See [LICENSE](./LICENSE) for details. --- @@ -1092,6 +1219,6 @@ Licensed under the MIT License. See [LICENSE.md](./LICENSE.md) for details. Created with ❤️ by **ElectronSz**
-File last updated: 2026-04-02 +File last updated: 2026-09-15 diff --git a/auto-migrate.ts b/auto-migrate.ts index af5ac64..f644666 100644 --- a/auto-migrate.ts +++ b/auto-migrate.ts @@ -1,86 +1,90 @@ /** * @file auto-migrate.ts - * @description Adds GORM-like AutoMigrate to Stabilize ORM with index and constraint management. + * @description GORM-like AutoMigrate: creates tables, adds missing columns, adds missing indexes. + * NEVER deletes columns or changes types - only adds. */ import { DBClient } from "./client"; -import { DBType, StabilizeError } from "./types"; - -type ColumnType = "string" | "number" | "boolean" | "date" | "json"; - -export interface ModelSchema { - tableName: string; - columns: Record< - string, - { - type: ColumnType; - primaryKey?: boolean; - autoIncrement?: boolean; - nullable?: boolean; - default?: any; - index?: string | boolean; - unique?: boolean; +import { DBType, DataTypes, StabilizeError } from "./types"; +import { MetadataStorage } from "./model"; +import { + createIndexIfNotExistsSQL, + createTableIfNotExistsSQL, + quoteIdentifier, +} from "./migrations"; + +async function tableExists(db: DBClient, table: string): Promise { + switch (db.config.type) { + case DBType.SQLite: { + const rows = await db.query( + `SELECT name FROM sqlite_master WHERE type='table' AND name = '${table}'`, + ); + return rows.length > 0; } - >; - indexes?: Array<{ - name: string; - columns: string[]; - unique?: boolean; - }>; -} - -function sqlType( - type: ColumnType, - dialect: "sqlite" | "mysql" | "postgres", -): string { - switch (dialect) { - case "sqlite": - return { - string: "TEXT", - number: "INTEGER", - boolean: "INTEGER", - date: "TEXT", - json: "TEXT", - }[type]!; - case "mysql": - return { - string: "VARCHAR(255)", - number: "INT", - boolean: "TINYINT(1)", - date: "DATETIME", - json: "JSON", - }[type]!; - case "postgres": - return { - string: "TEXT", - number: "INTEGER", - boolean: "BOOLEAN", - date: "TIMESTAMP", - json: "JSONB", - }[type]!; + case DBType.MySQL: { + const rows = await db.query( + `SELECT table_name FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name = ?`, + [table], + ); + return rows.length > 0; + } + case DBType.Postgres: { + const rows = await db.query( + `SELECT tablename FROM pg_tables WHERE schemaname = 'public' AND tablename = $1`, + [table], + ); + return rows.length > 0; + } + case DBType.MSSQL: { + const rows = await db.query( + `SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = ?`, + [table], + ); + return rows.length > 0; + } + default: + return false; } } async function getExistingColumns( db: DBClient, table: string, -): Promise> { +): Promise> { + const cols = new Map(); switch (db.config.type) { - case DBType.SQLite: - const rows = await db.query(`PRAGMA table_info(${table});`); - return new Set(rows.map((r: any) => r.name)); - case DBType.MySQL: - const cols = await db.query(`SHOW COLUMNS FROM \`${table}\`;`); - return new Set(cols.map((c: any) => c.Field)); - case DBType.Postgres: - const pgCols = await db.query( - `SELECT column_name FROM information_schema.columns WHERE table_name = $1`, + case DBType.SQLite: { + const rows = await db.query(`PRAGMA table_info(${table})`); + for (const r of rows) cols.set(r.name, r.type); + break; + } + case DBType.MySQL: { + const rows = await db.query(`SHOW COLUMNS FROM \`${table}\``); + for (const r of rows) cols.set(r.Field, r.Type); + break; + } + case DBType.Postgres: { + const rows = await db.query( + `SELECT column_name, data_type FROM information_schema.columns WHERE table_name = $1`, [table], ); - return new Set(pgCols.map((c: any) => c.column_name)); - default: - throw new StabilizeError("Unknown DB type", "MIGRATE_ERROR"); + for (const r of rows) cols.set(r.column_name, r.data_type); + break; + } + case DBType.MSSQL: { + // Aliased to the names the rest of this function reads, so the SQLite and + // MySQL shapes above stay untouched. + const rows = await db.query( + `SELECT COLUMN_NAME AS name, DATA_TYPE AS type, + IS_NULLABLE AS nullable, COLUMN_DEFAULT AS default_value + FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = ?`, + [table], + ); + for (const r of rows) cols.set(r.name, r.type); + break; + } } + return cols; } async function getExistingIndexes( @@ -90,10 +94,9 @@ async function getExistingIndexes( const indexes = new Map(); switch (db.config.type) { case DBType.SQLite: { - const rows = await db.query(`PRAGMA index_list(${table});`); + const rows = await db.query(`PRAGMA index_list(${table})`); for (const row of rows) { - if (row.origin === "u" || row.origin === "c") continue; - const info = await db.query(`PRAGMA index_info('${row.name}');`); + const info = await db.query(`PRAGMA index_info('${row.name}')`); indexes.set(row.name, { columns: info.map((i: any) => i.name), unique: !!row.unique, @@ -102,7 +105,7 @@ async function getExistingIndexes( break; } case DBType.MySQL: { - const rows = await db.query(`SHOW INDEX FROM \`${table}\`;`); + const rows = await db.query(`SHOW INDEX FROM \`${table}\``); for (const row of rows) { if (row.Key_name === "PRIMARY") continue; if (!indexes.has(row.Key_name)) { @@ -123,11 +126,32 @@ async function getExistingIndexes( ORDER BY i.relname, a.attnum`, [table], ); + for (const row of rows) { + if (!indexes.has(row.index_name)) { + indexes.set(row.index_name, { columns: [], unique: row.indisunique }); + } + indexes.get(row.index_name)!.columns.push(row.column_name); + } + break; + } + case DBType.MSSQL: { + // `sys.index_columns` carries one row per indexed column, so walking it + // in `index_column_id` order rebuilds each index's column list. The + // primary key is excluded, matching the PostgreSQL query above. + const rows = await db.query( + `SELECT i.name AS index_name, i.is_unique AS is_unique, c.name AS column_name + FROM sys.indexes i + JOIN sys.index_columns ic ON i.object_id = ic.object_id AND i.index_id = ic.index_id + JOIN sys.columns c ON ic.object_id = c.object_id AND ic.column_id = c.column_id + WHERE i.object_id = OBJECT_ID(?) AND i.is_primary_key = 0 AND i.name IS NOT NULL + ORDER BY i.name, ic.index_column_id`, + [table], + ); for (const row of rows) { if (!indexes.has(row.index_name)) { indexes.set(row.index_name, { columns: [], - unique: row.indisunique, + unique: !!row.is_unique, }); } indexes.get(row.index_name)!.columns.push(row.column_name); @@ -138,189 +162,414 @@ async function getExistingIndexes( return indexes; } -export function extractSchema(model: any): ModelSchema { - if (!model.schema) - throw new StabilizeError( - `Model "${model.name}" is missing static schema property.`, - "MIGRATE_ERROR", - ); - if (!model.schema.tableName) - throw new StabilizeError( - `Model "${model.name}" schema missing tableName.`, - "MIGRATE_ERROR", - ); - - return model.schema as ModelSchema; +function mapType( + dataType: string, + dialect: "sqlite" | "mysql" | "postgres" | "mssql", +): string { + const t = dataType.toUpperCase(); + const map: Record> = { + sqlite: { + STRING: "TEXT", + TEXT: "TEXT", + INTEGER: "INTEGER", + BIGINT: "INTEGER", + FLOAT: "REAL", + DOUBLE: "REAL", + DECIMAL: "NUMERIC", + BOOLEAN: "INTEGER", + DATE: "TEXT", + DATETIME: "TEXT", + JSON: "TEXT", + UUID: "TEXT", + BLOB: "BLOB", + }, + mysql: { + STRING: "VARCHAR(255)", + TEXT: "TEXT", + INTEGER: "INT", + BIGINT: "BIGINT", + FLOAT: "FLOAT", + DOUBLE: "DOUBLE", + DECIMAL: "DECIMAL(10,2)", + BOOLEAN: "TINYINT(1)", + DATE: "DATE", + DATETIME: "DATETIME", + JSON: "JSON", + UUID: "CHAR(36)", + BLOB: "BLOB", + }, + postgres: { + STRING: "TEXT", + TEXT: "TEXT", + INTEGER: "INTEGER", + BIGINT: "BIGINT", + FLOAT: "REAL", + DOUBLE: "DOUBLE PRECISION", + DECIMAL: "DECIMAL(10,2)", + BOOLEAN: "BOOLEAN", + DATE: "DATE", + DATETIME: "TIMESTAMP", + JSON: "JSONB", + UUID: "UUID", + BLOB: "BYTEA", + }, + mssql: { + STRING: "NVARCHAR(255)", + TEXT: "NVARCHAR(MAX)", + INTEGER: "INT", + BIGINT: "BIGINT", + FLOAT: "REAL", + DOUBLE: "FLOAT", + DECIMAL: "DECIMAL(10,2)", + BOOLEAN: "BIT", + DATE: "DATE", + DATETIME: "DATETIME2", + JSON: "NVARCHAR(MAX)", + UUID: "UNIQUEIDENTIFIER", + BLOB: "VARBINARY(MAX)", + }, + }; + // The fallback is a text column, and T-SQL's `TEXT` is both deprecated and + // unusable in most expressions, so SQL Server falls back to `NVARCHAR(MAX)`. + return map[dialect]?.[t] || (dialect === "mssql" ? "NVARCHAR(MAX)" : "TEXT"); } -function buildCreateTable( - schema: ModelSchema, - dialect: "sqlite" | "mysql" | "postgres", -) { - const parts: string[] = []; - - for (const [col, meta] of Object.entries(schema.columns)) { - let sql = `"${col}" ${sqlType(meta.type, dialect)}`; - - if (meta.primaryKey) sql += " PRIMARY KEY"; - - if (meta.autoIncrement) { - sql += - dialect === "mysql" - ? " AUTO_INCREMENT" - : dialect === "postgres" - ? " GENERATED ALWAYS AS IDENTITY" - : " AUTOINCREMENT"; - } - - if (!meta.nullable) sql += " NOT NULL"; - - if (meta.default !== undefined) - sql += ` DEFAULT ${JSON.stringify(meta.default)}`; - - parts.push(sql); - } - - return `CREATE TABLE IF NOT EXISTS "${schema.tableName}" (${parts.join(", ")})`; +/** + * The dialect's unbounded text type, used for the generated timestamp and + * history columns that `createTableFromMeta` and the history table add. + */ +function textType(dialect: "sqlite" | "mysql" | "postgres" | "mssql"): string { + return dialect === "mssql" ? "NVARCHAR(255)" : "TEXT"; } -function buildAddColumn( - table: string, - col: string, - meta: any, - dialect: "sqlite" | "mysql" | "postgres", -): string { - return `ALTER TABLE "${table}" ADD COLUMN "${col}" ${sqlType( - meta.type, - dialect, - )} ${meta.nullable ? "" : "NOT NULL"}`; +/** + * Resolves a column's declared type to its name. `type` is either a + * `DataTypes` enum member (the documented form) or a raw string. + */ +function resolveTypeName(type: any): string { + return typeof type === "string" ? type : (DataTypes[type] ?? "TEXT"); } -function buildCreateIndex( - table: string, - indexName: string, - columns: string[], - unique: boolean, - dialect: "sqlite" | "mysql" | "postgres", -): string { - const uniqueStr = unique ? "UNIQUE " : ""; - const colStr = columns.map((c) => `"${c}"`).join(", "); - return `CREATE ${uniqueStr}INDEX IF NOT EXISTS "${indexName}" ON "${table}" (${colStr})`; +/** Whether a resolved type name is an integer, and so can auto-increment. */ +function isIntegerType(typeName: string): boolean { + return typeName.toUpperCase() === "INTEGER" || typeName.toUpperCase() === "BIGINT"; } -function buildDropIndex( - indexName: string, - dialect: "sqlite" | "mysql" | "postgres", +function getAutoIncrementPK( + dialect: "sqlite" | "mysql" | "postgres" | "mssql", ): string { - if (dialect === "mysql") { - return `ALTER TABLE DROP INDEX \`${indexName}\``; + switch (dialect) { + case "mysql": + return "INT AUTO_INCREMENT PRIMARY KEY"; + case "postgres": + return "SERIAL PRIMARY KEY"; + case "mssql": + return "INT IDENTITY(1,1) PRIMARY KEY"; + default: + return "INTEGER PRIMARY KEY AUTOINCREMENT"; } - return `DROP INDEX IF EXISTS "${indexName}"`; -} - -function generateIndexName( - table: string, - columns: string[], - suffix: string = "idx", -): string { - return `${table}_${columns.join("_")}_${suffix}`; } +/** + * GORM-style AutoMigrate. + * + * - Creates table if it doesn't exist + * - Adds missing columns (never deletes or changes type) + * - Creates missing indexes + * - Creates history table if model is versioned + * + * Usage: + * ``` + * await orm.autoMigrate([User, Post, Comment]); + * ``` + */ export async function autoMigrate( db: DBClient, models: any | any[], ): Promise { const list = Array.isArray(models) ? models : [models]; + // Every identifier this file emits goes through `quoteIdentifier`, so the + // DDL parses on MySQL-family servers (backticks) as well as the rest (`"`). + // @see quoteIdentifier for why the two spellings are not interchangeable. + const q = (name: string) => quoteIdentifier(name, db.config.type); const dialect = db.config.type === DBType.SQLite ? "sqlite" : db.config.type === DBType.MySQL ? "mysql" - : "postgres"; + : db.config.type === DBType.MSSQL + ? "mssql" + : "postgres"; for (const model of list) { - const schema = extractSchema(model); + // Try MetadataStorage first (defineModel), fall back to model.schema + const meta = MetadataStorage.getModelMetadata(model); + const tableName = meta?.tableName || model.schema?.tableName; + + if (!tableName) { + throw new StabilizeError( + `Model is missing tableName. Use defineModel() or add static schema.`, + "MIGRATE_ERROR", + ); + } - const existing = await getExistingColumns(db, schema.tableName); - const existingIndexes = await getExistingIndexes(db, schema.tableName); + const exists = await tableExists(db, tableName); - if (existing.size === 0) { - const createSQL = buildCreateTable(schema, dialect); - await db.migrationQuery(createSQL); + if (!exists) { + // Create table from defineModel metadata + if (meta) { + await createTableFromMeta(db, meta, tableName, dialect); + } else { + // Legacy schema format + await createTableFromSchema(db, model.schema, tableName, dialect); + } } else { - for (const [col, meta] of Object.entries(schema.columns)) { - if (!existing.has(col)) { - const alter = buildAddColumn(schema.tableName, col, meta, dialect); - await db.migrationQuery(alter); + // Add missing columns + const existingCols = await getExistingColumns(db, tableName); + if (meta) { + for (const [key, col] of Object.entries(meta.columns)) { + const colName = (col as any).name || key; + if (!existingCols.has(colName)) { + const sqlType = mapType( + typeof (col as any).type === "string" + ? (col as any).type + : DataTypes[(col as any).type], + dialect, + ); + const notNull = (col as any).required ? " NOT NULL" : ""; + const defaultVal = + (col as any).defaultValue !== undefined + ? ` DEFAULT ${JSON.stringify((col as any).defaultValue)}` + : ""; + const unique = (col as any).unique ? " UNIQUE" : ""; + // `ADD COLUMN` is the spelling SQLite, MySQL and PostgreSQL share; + // T-SQL's grammar is `ADD `, with no `COLUMN` keyword. + const addColumn = + db.config.type === DBType.MSSQL ? "ADD" : "ADD COLUMN"; + await db.migrationQuery( + `ALTER TABLE ${q(tableName)} ${addColumn} ${q(colName)} ${sqlType}${notNull}${defaultVal}${unique}`, + ); + } } } } - const desiredIndexes = new Map< - string, - { columns: string[]; unique: boolean } - >(); - - for (const [col, meta] of Object.entries(schema.columns)) { - if (meta.index && typeof meta.index === "string") { - desiredIndexes.set(meta.index, { - columns: [col], - unique: !!meta.unique, - }); - } else if (meta.index === true) { - const idxName = generateIndexName(schema.tableName, [col]); - desiredIndexes.set(idxName, { columns: [col], unique: false }); - } - if (meta.unique && !meta.primaryKey) { - const idxName = generateIndexName(schema.tableName, [col], "uniq"); - desiredIndexes.set(idxName, { columns: [col], unique: true }); + // Create indexes. This read is what makes the statements below idempotent + // on MySQL and MariaDB, which have no `IF NOT EXISTS` clause to put on a + // `CREATE INDEX` and so cannot enforce it themselves — the name check here + // is the whole of the guarantee. @see createIndexIfNotExistsSQL. + const existingIndexes = await getExistingIndexes(db, tableName); + if (meta) { + for (const [key, col] of Object.entries(meta.columns)) { + const colName = (col as any).name || key; + if ((col as any).unique && key !== "id") { + const idxName = `${tableName}_${colName}_uniq`; + if (!existingIndexes.has(idxName)) { + await db.migrationQuery( + createIndexIfNotExistsSQL( + q(idxName), + q(tableName), + [q(colName)], + true, + db.config.type, + ), + ); + } + } + if ((col as any).index && typeof (col as any).index === "string") { + const idxName = (col as any).index; + if (!existingIndexes.has(idxName)) { + await db.migrationQuery( + createIndexIfNotExistsSQL( + q(idxName), + q(tableName), + [q(colName)], + false, + db.config.type, + ), + ); + } + } } } - if (schema.indexes) { - for (const idx of schema.indexes) { - desiredIndexes.set(idx.name, { - columns: idx.columns, - unique: !!idx.unique, - }); + // Create history table if versioned + if (meta?.versioned) { + const historyTable = `${tableName}_history`; + if (!(await tableExists(db, historyTable))) { + const columns = meta.columns; + const colDefs: string[] = []; + const historyColumnNames = new Set(); + for (const [key, col] of Object.entries(columns)) { + const colName = (col as any).name || key; + historyColumnNames.add(colName); + const sqlType = mapType( + typeof (col as any).type === "string" + ? (col as any).type + : DataTypes[(col as any).type], + dialect, + ); + colDefs.push(`${q(colName)} ${sqlType}`); + } + const historyText = textType(dialect); + colDefs.push(`${q("operation")} ${historyText}`); + // `writeHistory` always inserts a `version` column, so the table has + // to carry one even when the model declares no version column of its + // own — otherwise every versioned write fails with "no such column". + // The loop above already emitted it when the model declares one. + if (!historyColumnNames.has("version")) { + colDefs.push(`${q("version")} INTEGER`); + } + colDefs.push(`${q("valid_from")} ${historyText}`); + colDefs.push(`${q("valid_to")} ${historyText}`); + colDefs.push(`${q("modified_by")} ${historyText}`); + colDefs.push(`${q("modified_at")} ${historyText}`); + await db.migrationQuery( + createTableIfNotExistsSQL( + q(historyTable), + colDefs.join(", "), + db.config.type, + ), + ); } } + } +} - for (const [idxName, idxMeta] of desiredIndexes) { - if (!existingIndexes.has(idxName)) { - const createIdx = buildCreateIndex( - schema.tableName, - idxName, - idxMeta.columns, - idxMeta.unique, - dialect, - ); - await db.migrationQuery(createIdx); +async function createTableFromMeta( + db: DBClient, + meta: any, + tableName: string, + dialect: "sqlite" | "mysql" | "postgres" | "mssql", +) { + const colDefs: string[] = []; + const columns = meta.columns; + const q = (name: string) => quoteIdentifier(name, db.config.type); + + for (const [key, col] of Object.entries(columns)) { + const colName = (col as any).name || key; + const typeName = resolveTypeName((col as any).type); + + if (key === "id") { + // Only an integer `id` gets the database's auto-increment primary key. + // A declared STRING/UUID `id` — the pattern in the README, the docs site + // and what `generate:model` scaffolds — used to be overridden with + // `INTEGER PRIMARY KEY AUTOINCREMENT` regardless, so every + // `create({ id: generateUUID() })` failed with "datatype mismatch": + // SQLite will not store a UUID in a rowid column. Honour the declared + // type instead, and emit it as the primary key. + if (isIntegerType(typeName)) { + colDefs.push(`${q(colName)} ${getAutoIncrementPK(dialect)}`); + continue; } + const pkParts = [q(colName), mapType(typeName, dialect)]; + // SQLite's one historical quirk: a PRIMARY KEY that is not an INTEGER + // PRIMARY KEY may still hold NULL, so NOT NULL has to be explicit. + pkParts.push("NOT NULL", "PRIMARY KEY"); + colDefs.push(pkParts.join(" ")); + continue; } - for (const [existingIdxName, existingIdxMeta] of existingIndexes) { - let found = false; - for (const [desiredName, desiredMeta] of desiredIndexes) { - if (existingIdxName === desiredName) { - found = true; - break; - } - if ( - JSON.stringify(existingIdxMeta.columns.sort()) === - JSON.stringify(desiredMeta.columns.sort()) && - existingIdxMeta.unique === desiredMeta.unique - ) { - found = true; - break; - } - } - if (!found) { - const dropIdx = buildDropIndex(existingIdxName, dialect); - await db.migrationQuery(dropIdx); - } + const parts: string[] = [q(colName)]; + parts.push(mapType(typeName, dialect)); + if ((col as any).required) parts.push("NOT NULL"); + if ((col as any).unique) parts.push("UNIQUE"); + if ((col as any).defaultValue !== undefined) { + parts.push(`DEFAULT ${JSON.stringify((col as any).defaultValue)}`); } + colDefs.push(parts.join(" ")); } + + // Timestamps. Skipped when the model already declares the column in + // `columns` (the documented pattern), which would otherwise produce a + // duplicate column and make the CREATE TABLE fail. + if (meta.timestamps) { + const declared = new Set( + Object.entries(columns).map(([key, col]: [string, any]) => col.name || key), + ); + if (meta.timestamps.createdAt && !declared.has(meta.timestamps.createdAt)) { + colDefs.push(`${q(meta.timestamps.createdAt)} ${textType(dialect)}`); + } + if (meta.timestamps.updatedAt && !declared.has(meta.timestamps.updatedAt)) { + colDefs.push(`${q(meta.timestamps.updatedAt)} ${textType(dialect)}`); + } + } + + // Version column + if (meta.versioned) { + if (!columns.version) { + colDefs.push(`${q("version")} INTEGER DEFAULT 1`); + } + } + + await db.migrationQuery( + createTableIfNotExistsSQL( + q(tableName), + colDefs.join(", "), + db.config.type, + ), + ); +} + +async function createTableFromSchema( + db: DBClient, + schema: any, + tableName: string, + dialect: "sqlite" | "mysql" | "postgres" | "mssql", +) { + const parts: string[] = []; + const q = (name: string) => quoteIdentifier(name, db.config.type); + for (const [col, meta] of Object.entries(schema.columns)) { + const m = meta as any; + const dialectMap: Record> = { + sqlite: { + string: "TEXT", + number: "INTEGER", + boolean: "INTEGER", + date: "TEXT", + json: "TEXT", + }, + mysql: { + string: "VARCHAR(255)", + number: "INT", + boolean: "TINYINT(1)", + date: "DATETIME", + json: "JSON", + }, + postgres: { + string: "TEXT", + number: "INTEGER", + boolean: "BOOLEAN", + date: "TIMESTAMP", + json: "JSONB", + }, + mssql: { + string: "NVARCHAR(255)", + number: "INT", + boolean: "BIT", + date: "DATETIME2", + json: "NVARCHAR(MAX)", + }, + }; + let sql = `${q(col)} ${dialectMap[dialect]?.[m.type] || "TEXT"}`; + if (m.primaryKey) sql += " PRIMARY KEY"; + if (m.autoIncrement) { + sql += + dialect === "mysql" + ? " AUTO_INCREMENT" + : dialect === "postgres" + ? " GENERATED ALWAYS AS IDENTITY" + : dialect === "mssql" + ? " IDENTITY(1,1)" + : " AUTOINCREMENT"; + } + if (!m.nullable) sql += " NOT NULL"; + if (m.default !== undefined) sql += ` DEFAULT ${JSON.stringify(m.default)}`; + parts.push(sql); + } + await db.migrationQuery( + `CREATE TABLE IF NOT EXISTS ${q(tableName)} (${parts.join(", ")})`, + ); } (DBClient.prototype as any).autoMigrate = async function (models: any) { @@ -354,15 +603,20 @@ export async function resetDatabase( ): Promise { const list = Array.isArray(models) ? models : [models]; for (const model of list) { - const schema = extractSchema(model); - try { - await db.migrationQuery(`DROP TABLE IF EXISTS "${schema.tableName}"`); - } catch {} - try { - await db.migrationQuery( - `DROP TABLE IF EXISTS "${schema.tableName}_history"`, - ); - } catch {} + const meta = MetadataStorage.getModelMetadata(model); + const tableName = meta?.tableName || model.schema?.tableName; + if (tableName) { + try { + await db.migrationQuery( + `DROP TABLE IF EXISTS ${quoteIdentifier(tableName, db.config.type)}`, + ); + } catch {} + try { + await db.migrationQuery( + `DROP TABLE IF EXISTS ${quoteIdentifier(`${tableName}_history`, db.config.type)}`, + ); + } catch {} + } } await autoMigrate(db, list); } diff --git a/bun.lock b/bun.lock index 9a0b5a9..00d1e6d 100644 --- a/bun.lock +++ b/bun.lock @@ -4,34 +4,67 @@ "": { "name": "stabilize", "dependencies": { - "@types/pg": "^8.15.5", - "@types/uuid": "^11.0.0", - "commander": "^12.1.0", - "figlet": "^1.9.3", - "glob": "^11.0.0", - "ioredis": "^5.4.1", + "ioredis": "^5.8.1", + "mssql": "^12.7.2", "mysql2": "^3.15.2", "pg": "^8.16.3", - "reflect-metadata": "^0.2.2", - "uuid": "^13.0.0", }, "devDependencies": { - "@typescript-eslint/eslint-plugin": "^8.7.0", - "@typescript-eslint/parser": "^8.7.0", - "@vitest/coverage-v8": "^2.1.3", - "eslint": "^9.12.0", - "prettier": "^3.3.3", - "typescript": "^5.6.3", - "vitest": "^2.1.3", + "@types/pg": "^8.15.5", + "@types/uuid": "^11.0.0", + "@typescript-eslint/eslint-plugin": "^8.46.1", + "@typescript-eslint/parser": "^8.46.1", + "@vitest/coverage-v8": "^2.1.9", + "eslint": "^9.38.0", + "prettier": "^3.6.2", + "typescript": "^5.9.3", + "vitest": "^2.1.9", }, "peerDependencies": { "bun": ">=1.0.0", }, + "optionalPeers": [ + "bun", + ], }, }, "packages": { "@ampproject/remapping": ["@ampproject/remapping@2.3.0", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-30iZtAPgz+LTIYoeivqYo853f02jBYSd5uGnGpkFV0M3xOt9aN73erkgYAmZU43x4VfqcnLxW9Kpg3R5LC4YYw=="], + "@azure-rest/core-client": ["@azure-rest/core-client@2.9.0", "", { "dependencies": { "@azure/abort-controller": "^2.1.2", "@azure/core-auth": "^1.10.0", "@azure/core-rest-pipeline": "^1.24.0", "@azure/core-tracing": "^1.3.0", "@typespec/ts-http-runtime": "^0.3.8", "tslib": "^2.6.2" } }, "sha512-6933vNLqh06RR7rumnrq3UZIeBtRLeBHDPMrieDPIljiCaMMwt0nRw++nd3TOxiohtasNtm29rC+b/9ZVKCOqg=="], + + "@azure/abort-controller": ["@azure/abort-controller@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-fNAjWnA/nZ2jz31kxR/AqRaUT8ewHBw/WuBIosK0moMy1C9e5ValbDfFdIxJzVOOYaYkV/b2F1S4H/aHiqfVQg=="], + + "@azure/core-auth": ["@azure/core-auth@1.11.0", "", { "dependencies": { "@azure/abort-controller": "^2.1.2", "@azure/core-util": "^1.13.0", "tslib": "^2.6.2" } }, "sha512-IUZydyTUkDnYdstOW9pFOOUQlBjAepK5teihDE3x6yxsPJs/hsAaaYpeGxdxrgtOiJbBKSjKW7MDk7AEhb4LRg=="], + + "@azure/core-client": ["@azure/core-client@1.11.1", "", { "dependencies": { "@azure/abort-controller": "^2.1.2", "@azure/core-auth": "^1.10.0", "@azure/core-rest-pipeline": "^1.22.0", "@azure/core-tracing": "^1.3.0", "@azure/core-util": "^1.13.0", "@azure/logger": "^1.3.0", "tslib": "^2.6.2" } }, "sha512-2QygG2F76ZpMP2eMztiJvAiFMu71M9rDeU7vO/QKg5Css7MgM4frUOslFjhVjRhbGaCNPtz/S8M6y46/fFKVuQ=="], + + "@azure/core-lro": ["@azure/core-lro@2.7.2", "", { "dependencies": { "@azure/abort-controller": "^2.0.0", "@azure/core-util": "^1.2.0", "@azure/logger": "^1.0.0", "tslib": "^2.6.2" } }, "sha512-0YIpccoX8m/k00O7mDDMdJpbr6mf1yWo2dfmxt5A8XVZVVMz2SSKaEbMCeJRvgQ0IaSlqhjT47p4hVIRRy90xw=="], + + "@azure/core-paging": ["@azure/core-paging@1.7.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-7GEAoIsaoBr6KELNRb8nypowCqvk8dnCHFCYg4XD4lOQGY2GqjQg5IhkRjyBFRO18CGSMq05PaNqSOE9GQro3g=="], + + "@azure/core-process": ["@azure/core-process@1.0.0", "", {}, "sha512-/shnJ+ooO8WPxDhPEeI/2oRQuubn16gZ6CvlbpWbEswZfzwI9tI/sMAHmF3x1LuQ9yZYXfLW3TjzGMLEC5blKg=="], + + "@azure/core-rest-pipeline": ["@azure/core-rest-pipeline@1.25.0", "", { "dependencies": { "@azure/abort-controller": "^2.1.2", "@azure/core-auth": "^1.10.0", "@azure/core-tracing": "^1.3.0", "@azure/core-util": "^1.13.0", "@azure/logger": "^1.3.0", "@typespec/ts-http-runtime": "^0.3.4", "tslib": "^2.6.2" } }, "sha512-bMs8ekJLjX8wPV+9IPBges1SLPyuDtE9g5gLDWOpxzKcoOFQnpLGkbcT1tdw3FaAmDS1gnPmMmJ6y/T5B96kIA=="], + + "@azure/core-tracing": ["@azure/core-tracing@1.4.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-eGwxD0AtncrxeBM4tG8R55Pc3rdX1hNW2WibJAgYpCVA6E93mvvVH+LcssoVjOBrSKWS55yEIHsk0X8ctHmfOQ=="], + + "@azure/core-util": ["@azure/core-util@1.14.0", "", { "dependencies": { "@azure/abort-controller": "^2.1.2", "@typespec/ts-http-runtime": "^0.3.0", "tslib": "^2.6.2" } }, "sha512-9n2pWK61veAuN0V20t9lOuoV4CFMdyAZ1ygZzvBGk/pBBJRib/PjL9PLXa/aI2CcPpyHfqVsxxqLCYl6uZlfDw=="], + + "@azure/identity": ["@azure/identity@4.13.2", "", { "dependencies": { "@azure/abort-controller": "^2.0.0", "@azure/core-auth": "^1.9.0", "@azure/core-client": "^1.9.2", "@azure/core-process": "^1.0.0", "@azure/core-rest-pipeline": "^1.17.0", "@azure/core-tracing": "^1.0.0", "@azure/core-util": "^1.11.0", "@azure/logger": "^1.0.0", "@azure/msal-browser": "^5.5.0", "@azure/msal-node": "^5.1.5", "open": "^10.1.0", "tslib": "^2.2.0" } }, "sha512-NXL2/pCJctLxgw8bvrwwgge743kEq8LBT+O1pmV0vyUwetzFPH9auP6jhkU/cgZCPPtWoewAe3ncaGCgPo07fA=="], + + "@azure/keyvault-common": ["@azure/keyvault-common@2.1.0", "", { "dependencies": { "@azure-rest/core-client": "^2.3.3", "@azure/abort-controller": "^2.0.0", "@azure/core-auth": "^1.3.0", "@azure/core-rest-pipeline": "^1.8.0", "@azure/core-tracing": "^1.0.0", "@azure/core-util": "^1.10.0", "@azure/logger": "^1.1.4", "tslib": "^2.2.0" } }, "sha512-aCDidWuKY06LWQ4x7/8TIXK6iRqTaRWRL3t7T+LC+j1b07HtoIsOxP/tU90G4jCSBn5TAyUTCtA4MS/y5Hudaw=="], + + "@azure/keyvault-keys": ["@azure/keyvault-keys@4.10.2", "", { "dependencies": { "@azure-rest/core-client": "^2.3.3", "@azure/abort-controller": "^2.1.2", "@azure/core-auth": "^1.9.0", "@azure/core-lro": "^2.7.2", "@azure/core-paging": "^1.6.2", "@azure/core-rest-pipeline": "^1.19.0", "@azure/core-tracing": "^1.2.0", "@azure/core-util": "^1.11.0", "@azure/keyvault-common": "^2.1.0", "@azure/logger": "^1.1.4", "tslib": "^2.8.1" } }, "sha512-VmUSLbXRAbSzDD8grXHGPaknYs0SKr3yuf6U+d4XMpX4XuVYskNqbTTwXce0zR1LyxfTZm9rWEBcvs3vdYwCmQ=="], + + "@azure/logger": ["@azure/logger@1.4.0", "", { "dependencies": { "@typespec/ts-http-runtime": "^0.3.0", "tslib": "^2.6.2" } }, "sha512-rbAE25KUfjU/s3XHUdJgceoCP5dEOpMx85J04kF+QMdta73XkuG9JGHHinch+XIoKpBdqljin+KqURpJriSzLA=="], + + "@azure/msal-browser": ["@azure/msal-browser@5.21.0", "", { "dependencies": { "@azure/msal-common": "16.14.0" } }, "sha512-80OcuXDErmcEDAIH9pBtSqBsed2sPT/IWmbG3xHLoPMl5zc8TINd6SlJAbVSmN5huGa3xGAg5qR7VnpaIEK0Zw=="], + + "@azure/msal-common": ["@azure/msal-common@16.14.0", "", {}, "sha512-A4rb55hI86Q9tBl/+jBj7TMz7iX2RFgQs/nExFzcAtoI/BFRVdaH5SL/MivrYD7qvweMpN8AgVvVMHV8UBYxew=="], + + "@azure/msal-node": ["@azure/msal-node@5.6.0", "", { "dependencies": { "@azure/msal-common": "16.13.0", "jsonwebtoken": "^9.0.0" } }, "sha512-uFY9NxrWHw8PwZx7gAX6PDn+9vdfS05+levc/kwkx77IkjfaldnQbbcQzzDIZ5Hq5Zdr6/z92oAIoRWKp6MnOA=="], + "@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="], "@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.27.1", "", {}, "sha512-D2hP9eA+Sqx1kBZgzxZh0y1trbuU+JoDkiEwqhQ36nodYqJwyEIhPSdMNd7lOm/4io72luTPWH20Yda0xOuUow=="], @@ -116,10 +149,6 @@ "@ioredis/commands": ["@ioredis/commands@1.4.0", "", {}, "sha512-aFT2yemJJo+TZCmieA7qnYGQooOS7QfNmYrzGtsYd3g9j5iDP8AimYYAesf79ohjbLG12XxC4nG5DyEnC88AsQ=="], - "@isaacs/balanced-match": ["@isaacs/balanced-match@4.0.1", "", {}, "sha512-yzMTt9lEb8Gv7zRioUilSglI0c0smZ9k5D65677DLWLtWJaXIS3CqcGyUFByYKlnUj6TkjLVs54fBl6+TiGQDQ=="], - - "@isaacs/brace-expansion": ["@isaacs/brace-expansion@5.0.0", "", { "dependencies": { "@isaacs/balanced-match": "^4.0.1" } }, "sha512-ZT55BDLV0yv0RBm2czMiZ+SqCGO7AvmOM3G/w2xhVPH+te0aKgFjmBvGlL1dH+ql2tgGO3MVrbb3jCKyvpgnxA=="], - "@isaacs/cliui": ["@isaacs/cliui@8.0.2", "", { "dependencies": { "string-width": "^5.1.2", "string-width-cjs": "npm:string-width@^4.2.0", "strip-ansi": "^7.0.1", "strip-ansi-cjs": "npm:strip-ansi@^6.0.1", "wrap-ansi": "^8.1.0", "wrap-ansi-cjs": "npm:wrap-ansi@^7.0.0" } }, "sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA=="], "@istanbuljs/schema": ["@istanbuljs/schema@0.1.3", "", {}, "sha512-ZXRY4jNvVgSVQ8DL3LTcakaAtXwTVUxE81hslsyD2AtoXW/wVob10HkOJ1X/pAlcI7D+2YoZKg5do8G/w6RYgA=="], @@ -132,34 +161,14 @@ "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + "@js-joda/core": ["@js-joda/core@6.1.0", "", {}, "sha512-H8NTMRDJqad/leyv/D/A3kSOsf5/58Ydj4DJGDyaCWk9OU/zuZOLhndVffJgQjsgrn5GC0znHMHie7TfvPPG4w=="], + "@nodelib/fs.scandir": ["@nodelib/fs.scandir@2.1.5", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "^1.1.9" } }, "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g=="], "@nodelib/fs.stat": ["@nodelib/fs.stat@2.0.5", "", {}, "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A=="], "@nodelib/fs.walk": ["@nodelib/fs.walk@1.2.8", "", { "dependencies": { "@nodelib/fs.scandir": "2.1.5", "fastq": "^1.6.0" } }, "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg=="], - "@oven/bun-darwin-aarch64": ["@oven/bun-darwin-aarch64@1.3.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-WeXSaL29ylJEZMYHHW28QZ6rgAbxQ1KuNSZD9gvd3fPlo0s6s2PglvPArjjP07nmvIK9m4OffN0k4M98O7WmAg=="], - - "@oven/bun-darwin-x64": ["@oven/bun-darwin-x64@1.3.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-CFKjoUWQH0Oz3UHYfKbdKLq0wGryrFsTJEYq839qAwHQSECvVZYAnxVVDYUDa0yQFonhO2qSHY41f6HK+b7xtw=="], - - "@oven/bun-darwin-x64-baseline": ["@oven/bun-darwin-x64-baseline@1.3.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-+FSr/ub5vA/EkD3fMhHJUzYioSf/sXd50OGxNDAntVxcDu4tXL/81Ka3R/gkZmjznpLFIzovU/1Ts+b7dlkrfw=="], - - "@oven/bun-linux-aarch64": ["@oven/bun-linux-aarch64@1.3.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-WHthS/eLkCNcp9pk4W8aubRl9fIUgt2XhHyLrP0GClB1FVvmodu/zIOtG0NXNpzlzB8+gglOkGo4dPjfVf4Z+g=="], - - "@oven/bun-linux-aarch64-musl": ["@oven/bun-linux-aarch64-musl@1.3.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-HT5sr7N8NDYbQRjAnT7ISpx64y+ewZZRQozOJb0+KQObKvg4UUNXGm4Pn1xA4/WPMZDDazjO8E2vtOQw1nJlAQ=="], - - "@oven/bun-linux-x64": ["@oven/bun-linux-x64@1.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-sGEWoJQXO4GDr0x4t/yJQ/Bq1yNkOdX9tHbZZ+DBGJt3z3r7jeb4Digv8xQUk6gdTFC9vnGHuin+KW3/yD1Aww=="], - - "@oven/bun-linux-x64-baseline": ["@oven/bun-linux-x64-baseline@1.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-OmlEH3nlxQyv7HOvTH21vyNAZGv9DIPnrTznzvKiOQxkOphhCyKvPTlF13ydw4s/i18iwaUrhHy+YG9HSSxa4Q=="], - - "@oven/bun-linux-x64-musl": ["@oven/bun-linux-x64-musl@1.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-rtzUEzCynl3Rhgn/iR9DQezSFiZMcAXAbU+xfROqsweMGKwvwIA2ckyyckO08psEP8XcUZTs3LT9CH7PnaMiEA=="], - - "@oven/bun-linux-x64-musl-baseline": ["@oven/bun-linux-x64-musl-baseline@1.3.0", "", { "os": "linux", "cpu": "x64" }, "sha512-hrr7mDvUjMX1tuJaXz448tMsgKIqGJBY8+rJqztKOw1U5+a/v2w5HuIIW1ce7ut0ZwEn+KIDvAujlPvpH33vpQ=="], - - "@oven/bun-windows-x64": ["@oven/bun-windows-x64@1.3.0", "", { "os": "win32", "cpu": "x64" }, "sha512-xXwtpZVVP7T+vkxcF/TUVVOGRjEfkByO4mKveKYb4xnHWV4u4NnV0oNmzyMKkvmj10to5j2h0oZxA4ZVVv4gfA=="], - - "@oven/bun-windows-x64-baseline": ["@oven/bun-windows-x64-baseline@1.3.0", "", { "os": "win32", "cpu": "x64" }, "sha512-/jVZ8eYjpYHLDFNoT86cP+AjuWvpkzFY+0R0a1bdeu0sQ6ILuy1FV6hz1hUAP390E09VCo5oP76fnx29giHTtA=="], - "@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="], "@rollup/rollup-android-arm-eabi": ["@rollup/rollup-android-arm-eabi@4.52.4", "", { "os": "android", "cpu": "arm" }, "sha512-BTm2qKNnWIQ5auf4deoetINJm2JzvihvGb9R6K/ETwKLql/Bb3Eg2H1FBp1gUb4YGbydMA3jcmQTR73q7J+GAA=="], @@ -206,6 +215,8 @@ "@rollup/rollup-win32-x64-msvc": ["@rollup/rollup-win32-x64-msvc@4.52.4", "", { "os": "win32", "cpu": "x64" }, "sha512-bf9PtUa0u8IXDVxzRToFQKsNCRz9qLYfR/MpECxl4mRoWYjAeFjgxj1XdZr2M/GNVpT05p+LgQOHopYDlUu6/w=="], + "@tediousjs/connection-string": ["@tediousjs/connection-string@1.1.0", "", {}, "sha512-z9ZBWEG+8pIB5V1zYzlRPXx0oRJ5H7coPnMQK8EZOw03UTPI9Umn6viL36f5w+CuqkKsnCM50RVStpjZmR0Bng=="], + "@types/estree": ["@types/estree@1.0.8", "", {}, "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w=="], "@types/json-schema": ["@types/json-schema@7.0.15", "", {}, "sha512-5+fP8P8MFNC+AyZCDxrB2pkZFPGzqQWUzpSeuuVLvm8VMcorNYavBqoFcxK8bQz4Qsbn4oUEEem4wDLfcysGHA=="], @@ -214,6 +225,8 @@ "@types/pg": ["@types/pg@8.15.5", "", { "dependencies": { "@types/node": "*", "pg-protocol": "*", "pg-types": "^2.2.0" } }, "sha512-LF7lF6zWEKxuT3/OR8wAZGzkg4ENGXFNyiV/JeOt9z5B+0ZVwbql9McqX5c/WStFq1GaGso7H1AzP/qSzmlCKQ=="], + "@types/readable-stream": ["@types/readable-stream@4.0.24", "", { "dependencies": { "@types/node": "*" } }, "sha512-NRvUNC/JFGPJvqdAfEve8oginbM6V08u5NzLWpG8MwA2kTPOLnqk+wpwuPT+mp3aUsxyuT6m2gnrPuHYCruzEg=="], + "@types/uuid": ["@types/uuid@11.0.0", "", { "dependencies": { "uuid": "*" } }, "sha512-HVyk8nj2m+jcFRNazzqyVKiZezyhDKrGUA3jlEcg/nZ6Ms+qHwocba1Y/AaVaznJTAM9xpdFSh+ptbNrhOGvZA=="], "@typescript-eslint/eslint-plugin": ["@typescript-eslint/eslint-plugin@8.46.1", "", { "dependencies": { "@eslint-community/regexpp": "^4.10.0", "@typescript-eslint/scope-manager": "8.46.1", "@typescript-eslint/type-utils": "8.46.1", "@typescript-eslint/utils": "8.46.1", "@typescript-eslint/visitor-keys": "8.46.1", "graphemer": "^1.4.0", "ignore": "^7.0.0", "natural-compare": "^1.4.0", "ts-api-utils": "^2.1.0" }, "peerDependencies": { "@typescript-eslint/parser": "^8.46.1", "eslint": "^8.57.0 || ^9.0.0", "typescript": ">=4.8.4 <6.0.0" } }, "sha512-rUsLh8PXmBjdiPY+Emjz9NX2yHvhS11v0SR6xNJkm5GM1MO9ea/1GoDKlHHZGrOJclL/cZ2i/vRUYVtjRhrHVQ=="], @@ -236,6 +249,8 @@ "@typescript-eslint/visitor-keys": ["@typescript-eslint/visitor-keys@8.46.1", "", { "dependencies": { "@typescript-eslint/types": "8.46.1", "eslint-visitor-keys": "^4.2.1" } }, "sha512-ptkmIf2iDkNUjdeu2bQqhFPV1m6qTnFFjg7PPDjxKWaMaP0Z6I9l30Jr3g5QqbZGdw8YdYvLp+XnqnWWZOg/NA=="], + "@typespec/ts-http-runtime": ["@typespec/ts-http-runtime@0.3.9", "", { "dependencies": { "http-proxy-agent": "^7.0.0", "https-proxy-agent": "^7.0.0", "tslib": "^2.6.2" } }, "sha512-edSdeAqkdxBVzA1yL1LrLCml1YjyCVvPMtMqJpbF+6K609tHe8V6sQUzFQSGcYNhcuhOceZtjvN32+mpIth30A=="], + "@vitest/coverage-v8": ["@vitest/coverage-v8@2.1.9", "", { "dependencies": { "@ampproject/remapping": "^2.3.0", "@bcoe/v8-coverage": "^0.2.3", "debug": "^4.3.7", "istanbul-lib-coverage": "^3.2.2", "istanbul-lib-report": "^3.0.1", "istanbul-lib-source-maps": "^5.0.6", "istanbul-reports": "^3.1.7", "magic-string": "^0.30.12", "magicast": "^0.3.5", "std-env": "^3.8.0", "test-exclude": "^7.0.1", "tinyrainbow": "^1.2.0" }, "peerDependencies": { "@vitest/browser": "2.1.9", "vitest": "2.1.9" }, "optionalPeers": ["@vitest/browser"] }, "sha512-Z2cOr0ksM00MpEfyVE8KXIYPEcBFxdbLSs56L8PO0QQMxt/6bDj45uQfxoc96v05KW3clk7vvgP0qfDit9DmfQ=="], "@vitest/expect": ["@vitest/expect@2.1.9", "", { "dependencies": { "@vitest/spy": "2.1.9", "@vitest/utils": "2.1.9", "chai": "^5.1.2", "tinyrainbow": "^1.2.0" } }, "sha512-UJCIkTBenHeKT1TTlKMJWy1laZewsRIzYighyYiJKZreqtdxSos/S1t+ktRMQWu2CKqaarrkeszJx1cgC5tGZw=="], @@ -252,10 +267,14 @@ "@vitest/utils": ["@vitest/utils@2.1.9", "", { "dependencies": { "@vitest/pretty-format": "2.1.9", "loupe": "^3.1.2", "tinyrainbow": "^1.2.0" } }, "sha512-v0psaMSkNJ3A2NMrUEHFRzJtDPFn+/VWZ5WxImB21T9fjucJRmS7xCS3ppEnARb9y11OAzaD+P2Ps+b+BGX5iQ=="], + "abort-controller": ["abort-controller@3.0.0", "", { "dependencies": { "event-target-shim": "^5.0.0" } }, "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg=="], + "acorn": ["acorn@8.15.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg=="], "acorn-jsx": ["acorn-jsx@5.3.2", "", { "peerDependencies": { "acorn": "^6.0.0 || ^7.0.0 || ^8.0.0" } }, "sha512-rq9s+JNhf0IChjtDXxllJ7g41oZk5SlXtp0LHwyA5cejwn7vKmKp4pPri6YEePv2PU65sAsegbXtIinmDFDXgQ=="], + "agent-base": ["agent-base@7.1.4", "", {}, "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ=="], + "ajv": ["ajv@6.12.6", "", { "dependencies": { "fast-deep-equal": "^3.1.1", "fast-json-stable-stringify": "^2.0.0", "json-schema-traverse": "^0.4.1", "uri-js": "^4.2.2" } }, "sha512-j3fVLgvTo527anyYyJOGTYJbG+vnnQYvE0m5mmkc1TK+nxAppkCLMIL0aZ4dblVCNoGShhm+kzE4ZUykBoMg4g=="], "ansi-regex": ["ansi-regex@6.2.2", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="], @@ -270,11 +289,19 @@ "balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], + "base64-js": ["base64-js@1.5.1", "", {}, "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA=="], + + "bl": ["bl@6.1.6", "", { "dependencies": { "@types/readable-stream": "^4.0.0", "buffer": "^6.0.3", "inherits": "^2.0.4", "readable-stream": "^4.2.0" } }, "sha512-jLsPgN/YSvPUg9UX0Kd73CXpm2Psg9FxMeCSXnk3WBO3CMT10JMwijubhGfHCnFu6TPn1ei3b975dxv7K2pWVg=="], + "brace-expansion": ["brace-expansion@1.1.12", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg=="], "braces": ["braces@3.0.3", "", { "dependencies": { "fill-range": "^7.1.1" } }, "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA=="], - "bun": ["bun@1.3.0", "", { "optionalDependencies": { "@oven/bun-darwin-aarch64": "1.3.0", "@oven/bun-darwin-x64": "1.3.0", "@oven/bun-darwin-x64-baseline": "1.3.0", "@oven/bun-linux-aarch64": "1.3.0", "@oven/bun-linux-aarch64-musl": "1.3.0", "@oven/bun-linux-x64": "1.3.0", "@oven/bun-linux-x64-baseline": "1.3.0", "@oven/bun-linux-x64-musl": "1.3.0", "@oven/bun-linux-x64-musl-baseline": "1.3.0", "@oven/bun-windows-x64": "1.3.0", "@oven/bun-windows-x64-baseline": "1.3.0" }, "os": [ "linux", "win32", "darwin", ], "cpu": [ "x64", "arm64", ], "bin": { "bun": "bin/bun.exe", "bunx": "bin/bunx.exe" } }, "sha512-YI7mFs7iWc/VsGsh2aw6eAPD2cjzn1j+LKdYVk09x1CrdTWKYIHyd+dG5iQoN9//3hCDoZj8U6vKpZzEf5UARA=="], + "buffer": ["buffer@6.0.3", "", { "dependencies": { "base64-js": "^1.3.1", "ieee754": "^1.2.1" } }, "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA=="], + + "buffer-equal-constant-time": ["buffer-equal-constant-time@1.0.1", "", {}, "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA=="], + + "bundle-name": ["bundle-name@4.1.0", "", { "dependencies": { "run-applescript": "^7.0.0" } }, "sha512-tjwM5exMg6BGRI+kNmTntNsvdZS1X8BFYS6tnJ2hdH0kVxM6/eVZ2xy+FqStSWvYmtfFMDLIxurorHwDKfDz5Q=="], "cac": ["cac@6.7.14", "", {}, "sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ=="], @@ -292,7 +319,7 @@ "color-name": ["color-name@1.1.4", "", {}, "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="], - "commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="], + "commander": ["commander@11.1.0", "", {}, "sha512-yPVavfyCcRhmorC7rWlkHn15b4wDVgVmBA7kV4QVBsF7kv/9TKJAbAXVTxvTnwP8HHKjRCJDClKbciiYS7p0DQ=="], "concat-map": ["concat-map@0.0.1", "", {}, "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg=="], @@ -304,10 +331,18 @@ "deep-is": ["deep-is@0.1.4", "", {}, "sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ=="], + "default-browser": ["default-browser@5.5.1", "", { "dependencies": { "bundle-name": "^4.1.0", "default-browser-id": "^5.0.0" } }, "sha512-m1pAzaJgZ/gssEqlOhJkPJp8Xly7QyW6xcrkUa2KKcDeDSEMP7X8xipU3snUcfisTQx0w1AGae+9UtJSfVnXGw=="], + + "default-browser-id": ["default-browser-id@5.0.1", "", {}, "sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q=="], + + "define-lazy-prop": ["define-lazy-prop@3.0.0", "", {}, "sha512-N+MeXYoqr3pOgn8xfyRPREN7gHakLYjhsHhWGT3fWAiL4IkAt0iDw14QiiEm2bE30c5XX5q0FtAA3CK5f9/BUg=="], + "denque": ["denque@2.1.0", "", {}, "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw=="], "eastasianwidth": ["eastasianwidth@0.2.0", "", {}, "sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA=="], + "ecdsa-sig-formatter": ["ecdsa-sig-formatter@1.0.11", "", { "dependencies": { "safe-buffer": "^5.0.1" } }, "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ=="], + "emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="], "es-module-lexer": ["es-module-lexer@1.7.0", "", {}, "sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA=="], @@ -334,6 +369,10 @@ "esutils": ["esutils@2.0.3", "", {}, "sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g=="], + "event-target-shim": ["event-target-shim@5.0.1", "", {}, "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ=="], + + "events": ["events@3.3.0", "", {}, "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q=="], + "expect-type": ["expect-type@1.2.2", "", {}, "sha512-JhFGDVJ7tmDJItKhYgJCGLOWjuK9vPxiXoUFLwLDc99NlmklilbiQJwoctZtt13+xMw91MCk/REan6MWHqDjyA=="], "fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="], @@ -346,8 +385,6 @@ "fastq": ["fastq@1.19.1", "", { "dependencies": { "reusify": "^1.0.4" } }, "sha512-GwLTyxkCXjXbxqIhTsMI2Nui8huMPtnxg7krajPJAjnEG/iiOS7i+zCtWGZR9G0NBKbXKh6X9m9UIsYX/N6vvQ=="], - "figlet": ["figlet@1.9.3", "", { "dependencies": { "commander": "^14.0.0" }, "bin": { "figlet": "bin/index.js" } }, "sha512-majPgOpVtrZN1iyNGbsUP6bOtZ6eaJgg5HHh0vFvm5DJhh8dc+FJpOC4GABvMZ/A7XHAJUuJujhgUY/2jPWgMA=="], - "file-entry-cache": ["file-entry-cache@8.0.0", "", { "dependencies": { "flat-cache": "^4.0.0" } }, "sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ=="], "fill-range": ["fill-range@7.1.1", "", { "dependencies": { "to-regex-range": "^5.0.1" } }, "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg=="], @@ -364,7 +401,7 @@ "generate-function": ["generate-function@2.3.1", "", { "dependencies": { "is-property": "^1.0.2" } }, "sha512-eeB5GfMNeevm/GRYq20ShmsaGcmI81kIX2K9XQx5miC8KdHaC6Jm0qQ8ZNeGOi7wYB8OsdxKs+Y2oVuTFuVwKQ=="], - "glob": ["glob@11.0.3", "", { "dependencies": { "foreground-child": "^3.3.1", "jackspeak": "^4.1.1", "minimatch": "^10.0.3", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^2.0.0" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-2Nim7dha1KVkaiF4q6Dj+ngPPMdfvLJEOpZk/jKiUAkqKebpGAWQXAq9z1xu9HKu5lWfqw/FASuccEjyznjPaA=="], + "glob": ["glob@10.4.5", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-7Bv8RF0k6xjo7d4A/PxYLbUCfb6c+Vpd2/mB2yRDlew7Jb5hEXiCD9ibfO7wpk8i4sevK6DFny9h7EYbM3/sHg=="], "glob-parent": ["glob-parent@6.0.2", "", { "dependencies": { "is-glob": "^4.0.3" } }, "sha512-XxwI8EOhVQgWp6iDL+3b0r86f4d6AX6zSU55HfB4ydCEuXLXc5FcYeOu+nnGftS4TEju/11rt4KJPTMgbfmv4A=="], @@ -376,26 +413,40 @@ "html-escaper": ["html-escaper@2.0.2", "", {}, "sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg=="], + "http-proxy-agent": ["http-proxy-agent@7.0.2", "", { "dependencies": { "agent-base": "^7.1.0", "debug": "^4.3.4" } }, "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig=="], + + "https-proxy-agent": ["https-proxy-agent@7.0.6", "", { "dependencies": { "agent-base": "^7.1.2", "debug": "4" } }, "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw=="], + "iconv-lite": ["iconv-lite@0.7.0", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-cf6L2Ds3h57VVmkZe+Pn+5APsT7FpqJtEhhieDCvrE2MK5Qk9MyffgQyuxQTm6BChfeZNtcOLHp9IcWRVcIcBQ=="], + "ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="], + "ignore": ["ignore@7.0.5", "", {}, "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg=="], "import-fresh": ["import-fresh@3.3.1", "", { "dependencies": { "parent-module": "^1.0.0", "resolve-from": "^4.0.0" } }, "sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ=="], "imurmurhash": ["imurmurhash@0.1.4", "", {}, "sha512-JmXMZ6wuvDmLiHEml9ykzqO6lwFbof0GG4IkcGaENdCRDDmMVnny7s5HsIgHCbaq0w2MyPhDqkhTUgS2LU2PHA=="], + "inherits": ["inherits@2.0.4", "", {}, "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="], + "ioredis": ["ioredis@5.8.1", "", { "dependencies": { "@ioredis/commands": "1.4.0", "cluster-key-slot": "^1.1.0", "debug": "^4.3.4", "denque": "^2.1.0", "lodash.defaults": "^4.2.0", "lodash.isarguments": "^3.1.0", "redis-errors": "^1.2.0", "redis-parser": "^3.0.0", "standard-as-callback": "^2.1.0" } }, "sha512-Qho8TgIamqEPdgiMadJwzRMW3TudIg6vpg4YONokGDudy4eqRIJtDbVX72pfLBcWxvbn3qm/40TyGUObdW4tLQ=="], + "is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="], + "is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="], "is-fullwidth-code-point": ["is-fullwidth-code-point@3.0.0", "", {}, "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg=="], "is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="], + "is-inside-container": ["is-inside-container@1.0.0", "", { "dependencies": { "is-docker": "^3.0.0" }, "bin": { "is-inside-container": "cli.js" } }, "sha512-KIYLCCJghfHZxqjYBE7rEy0OBuTd5xCHS7tHVgvCLkx7StIoaxwNW3hCALgEUjFfeRk+MG/Qxmp/vtETEF3tRA=="], + "is-number": ["is-number@7.0.0", "", {}, "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng=="], "is-property": ["is-property@1.0.2", "", {}, "sha512-Ks/IoX00TtClbGQr4TWXemAnktAQvYB7HzcCxDGqEZU6oCmb2INHuOoKxbtR+HFkmYWBKv/dOZtGRiAjDhj92g=="], + "is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="], + "isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="], "istanbul-lib-coverage": ["istanbul-lib-coverage@3.2.2", "", {}, "sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg=="], @@ -406,7 +457,9 @@ "istanbul-reports": ["istanbul-reports@3.2.0", "", { "dependencies": { "html-escaper": "^2.0.0", "istanbul-lib-report": "^3.0.0" } }, "sha512-HGYWWS/ehqTV3xN10i23tkPkpH46MLCIMFNCaaKNavAXTF1RkqxawEPtnjnGZ6XKSInBKkiOA5BKS+aZiY3AvA=="], - "jackspeak": ["jackspeak@4.1.1", "", { "dependencies": { "@isaacs/cliui": "^8.0.2" } }, "sha512-zptv57P3GpL+O0I7VdMJNBZCu+BPHVQUk55Ft8/QCJjTVxrnJHuVuX/0Bl2A6/+2oyR/ZMEuFKwmzqqZ/U5nPQ=="], + "jackspeak": ["jackspeak@3.4.3", "", { "dependencies": { "@isaacs/cliui": "^8.0.2" }, "optionalDependencies": { "@pkgjs/parseargs": "^0.11.0" } }, "sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw=="], + + "js-md4": ["js-md4@0.3.2", "", {}, "sha512-/GDnfQYsltsjRswQhN9fhv3EMw2sCpUdrdxyWDOUK7eyD++r3gRhzgiQgc/x4MAv2i1iuQ4lxO5mvqM3vj4bwA=="], "js-yaml": ["js-yaml@4.1.0", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA=="], @@ -416,6 +469,12 @@ "json-stable-stringify-without-jsonify": ["json-stable-stringify-without-jsonify@1.0.1", "", {}, "sha512-Bdboy+l7tA3OGW6FjyFHWkP5LuByj1Tk33Ljyq0axyzdk9//JSi2u3fP1QSmd1KNwq6VOKYGlAu87CisVir6Pw=="], + "jsonwebtoken": ["jsonwebtoken@9.0.3", "", { "dependencies": { "jws": "^4.0.1", "lodash.includes": "^4.3.0", "lodash.isboolean": "^3.0.3", "lodash.isinteger": "^4.0.4", "lodash.isnumber": "^3.0.3", "lodash.isplainobject": "^4.0.6", "lodash.isstring": "^4.0.1", "lodash.once": "^4.0.0", "ms": "^2.1.1", "semver": "^7.5.4" } }, "sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g=="], + + "jwa": ["jwa@2.0.1", "", { "dependencies": { "buffer-equal-constant-time": "^1.0.1", "ecdsa-sig-formatter": "1.0.11", "safe-buffer": "^5.0.1" } }, "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg=="], + + "jws": ["jws@4.0.1", "", { "dependencies": { "jwa": "^2.0.1", "safe-buffer": "^5.0.1" } }, "sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA=="], + "keyv": ["keyv@4.5.4", "", { "dependencies": { "json-buffer": "3.0.1" } }, "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw=="], "levn": ["levn@0.4.1", "", { "dependencies": { "prelude-ls": "^1.2.1", "type-check": "~0.4.0" } }, "sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ=="], @@ -424,15 +483,29 @@ "lodash.defaults": ["lodash.defaults@4.2.0", "", {}, "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ=="], + "lodash.includes": ["lodash.includes@4.3.0", "", {}, "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w=="], + "lodash.isarguments": ["lodash.isarguments@3.1.0", "", {}, "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg=="], + "lodash.isboolean": ["lodash.isboolean@3.0.3", "", {}, "sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg=="], + + "lodash.isinteger": ["lodash.isinteger@4.0.4", "", {}, "sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA=="], + + "lodash.isnumber": ["lodash.isnumber@3.0.3", "", {}, "sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw=="], + + "lodash.isplainobject": ["lodash.isplainobject@4.0.6", "", {}, "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA=="], + + "lodash.isstring": ["lodash.isstring@4.0.1", "", {}, "sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw=="], + "lodash.merge": ["lodash.merge@4.6.2", "", {}, "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ=="], + "lodash.once": ["lodash.once@4.1.1", "", {}, "sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg=="], + "long": ["long@5.3.2", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="], "loupe": ["loupe@3.2.1", "", {}, "sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ=="], - "lru-cache": ["lru-cache@11.2.2", "", {}, "sha512-F9ODfyqML2coTIsQpSkRHnLSZMtkU8Q+mSfcaIyKwy58u+8k5nvAYeiNhsyMARvzNcXJ9QfWVrcPsC9e9rAxtg=="], + "lru-cache": ["lru-cache@7.18.3", "", {}, "sha512-jumlc0BIUrS3qJGgIkWZsyfAM7NCWiBcCDhnd+3NNM5KbBmLTgHVfWBcg6W+rLUsIpzpERPsvwUP7CckAQSOoA=="], "lru.min": ["lru.min@1.1.2", "", {}, "sha512-Nv9KddBcQSlQopmBHXSsZVY5xsdlZkdH/Iey0BlcBYggMd4two7cZnKOK9vmy3nY0O5RGH99z1PCeTpPqszUYg=="], @@ -452,14 +525,20 @@ "ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="], + "mssql": ["mssql@12.7.2", "", { "dependencies": { "@tediousjs/connection-string": "^1.0.0", "commander": "^11.0.0", "debug": "^4.3.3", "tarn": "^3.0.2", "tedious": "^19.2.2 || ^20.0.0" }, "bin": { "mssql": "bin/mssql" } }, "sha512-zhpPw+WXBWLw7d7J6Y0aWgQy0EV8HKDGNSvmkAjP8bSM4H41X0fweAeKWI+bg9Ri+JHCbsf/QScjmACBFcEo8A=="], + "mysql2": ["mysql2@3.15.2", "", { "dependencies": { "aws-ssl-profiles": "^1.1.1", "denque": "^2.1.0", "generate-function": "^2.3.1", "iconv-lite": "^0.7.0", "long": "^5.2.1", "lru.min": "^1.0.0", "named-placeholders": "^1.1.3", "seq-queue": "^0.0.5", "sqlstring": "^2.3.2" } }, "sha512-kFm5+jbwR5mC+lo+3Cy46eHiykWSpUtTLOH3GE+AR7GeLq8PgfJcvpMiyVWk9/O53DjQsqm6a3VOOfq7gYWFRg=="], "named-placeholders": ["named-placeholders@1.1.3", "", { "dependencies": { "lru-cache": "^7.14.1" } }, "sha512-eLoBxg6wE/rZkJPhU/xRX1WTpkFEwDJEN96oxFrTsqBdbT5ec295Q+CoHrL9IT0DipqKhmGcaZmwOt8OON5x1w=="], "nanoid": ["nanoid@3.3.11", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w=="], + "native-duplexpair": ["native-duplexpair@1.0.0", "", {}, "sha512-E7QQoM+3jvNtlmyfqRZ0/U75VFgCls+fSkbml2MpgWkWyz3ox8Y58gNhfuziuQYGNNQAbFZJQck55LHCnCK6CA=="], + "natural-compare": ["natural-compare@1.4.0", "", {}, "sha512-OWND8ei3VtNC9h7V60qff3SVobHr996CTwgxubgyQYEpg290h9J0buyECNNJexkFm5sOajh5G116RYA1c8ZMSw=="], + "open": ["open@10.2.0", "", { "dependencies": { "default-browser": "^5.2.1", "define-lazy-prop": "^3.0.0", "is-inside-container": "^1.0.0", "wsl-utils": "^0.1.0" } }, "sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA=="], + "optionator": ["optionator@0.9.4", "", { "dependencies": { "deep-is": "^0.1.3", "fast-levenshtein": "^2.0.6", "levn": "^0.4.1", "prelude-ls": "^1.2.1", "type-check": "^0.4.0", "word-wrap": "^1.2.5" } }, "sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g=="], "p-limit": ["p-limit@3.1.0", "", { "dependencies": { "yocto-queue": "^0.1.0" } }, "sha512-TYOanM3wGwNGsZN2cVTYPArw454xnXj5qmWF1bEoAc4+cU/ol7GVh7odevjp1FNHduHc3KZMcFduxU5Xc6uJRQ=="], @@ -474,7 +553,7 @@ "path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="], - "path-scurry": ["path-scurry@2.0.0", "", { "dependencies": { "lru-cache": "^11.0.0", "minipass": "^7.1.2" } }, "sha512-ypGJsmGtdXUOeM5u93TyeIEfEhM6s+ljAhrk5vAvSx8uyY/02OvrZnA0YNGUrPXfpJMgI1ODd3nwz8Npx4O4cg=="], + "path-scurry": ["path-scurry@1.11.1", "", { "dependencies": { "lru-cache": "^10.2.0", "minipass": "^5.0.0 || ^6.0.2 || ^7.0.0" } }, "sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA=="], "pathe": ["pathe@1.1.2", "", {}, "sha512-whLdWMYL2TwI08hn8/ZqAbrVemu0LNaNNJZX73O6qaIdCTfXutsLhMkjdENX0qhsQ9uIimo4/aQOmXkoon2nDQ=="], @@ -514,24 +593,30 @@ "prettier": ["prettier@3.6.2", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ=="], + "process": ["process@0.11.10", "", {}, "sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A=="], + "punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="], "queue-microtask": ["queue-microtask@1.2.3", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="], + "readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "redis-errors": ["redis-errors@1.2.0", "", {}, "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w=="], "redis-parser": ["redis-parser@3.0.0", "", { "dependencies": { "redis-errors": "^1.0.0" } }, "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A=="], - "reflect-metadata": ["reflect-metadata@0.2.2", "", {}, "sha512-urBwgfrvVP/eAyXx4hluJivBKzuEbSQs9rKWCrCkbSxNv8mxPcUZKeuoF3Uy4mJl3Lwprp6yy5/39VWigZ4K6Q=="], - "resolve-from": ["resolve-from@4.0.0", "", {}, "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g=="], "reusify": ["reusify@1.1.0", "", {}, "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw=="], "rollup": ["rollup@4.52.4", "", { "dependencies": { "@types/estree": "1.0.8" }, "optionalDependencies": { "@rollup/rollup-android-arm-eabi": "4.52.4", "@rollup/rollup-android-arm64": "4.52.4", "@rollup/rollup-darwin-arm64": "4.52.4", "@rollup/rollup-darwin-x64": "4.52.4", "@rollup/rollup-freebsd-arm64": "4.52.4", "@rollup/rollup-freebsd-x64": "4.52.4", "@rollup/rollup-linux-arm-gnueabihf": "4.52.4", "@rollup/rollup-linux-arm-musleabihf": "4.52.4", "@rollup/rollup-linux-arm64-gnu": "4.52.4", "@rollup/rollup-linux-arm64-musl": "4.52.4", "@rollup/rollup-linux-loong64-gnu": "4.52.4", "@rollup/rollup-linux-ppc64-gnu": "4.52.4", "@rollup/rollup-linux-riscv64-gnu": "4.52.4", "@rollup/rollup-linux-riscv64-musl": "4.52.4", "@rollup/rollup-linux-s390x-gnu": "4.52.4", "@rollup/rollup-linux-x64-gnu": "4.52.4", "@rollup/rollup-linux-x64-musl": "4.52.4", "@rollup/rollup-openharmony-arm64": "4.52.4", "@rollup/rollup-win32-arm64-msvc": "4.52.4", "@rollup/rollup-win32-ia32-msvc": "4.52.4", "@rollup/rollup-win32-x64-gnu": "4.52.4", "@rollup/rollup-win32-x64-msvc": "4.52.4", "fsevents": "~2.3.2" }, "bin": { "rollup": "dist/bin/rollup" } }, "sha512-CLEVl+MnPAiKh5pl4dEWSyMTpuflgNQiLGhMv8ezD5W/qP8AKvmYpCOKRRNOh7oRKnauBZ4SyeYkMS+1VSyKwQ=="], + "run-applescript": ["run-applescript@7.1.0", "", {}, "sha512-DPe5pVFaAsinSaV6QjQ6gdiedWDcRCbUuiQfQa2wmWV7+xC9bGulGI8+TdRmoFkAPaBXk8CrAbnlY2ISniJ47Q=="], + "run-parallel": ["run-parallel@1.2.0", "", { "dependencies": { "queue-microtask": "^1.2.2" } }, "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA=="], + "safe-buffer": ["safe-buffer@5.2.1", "", {}, "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ=="], + "safer-buffer": ["safer-buffer@2.1.2", "", {}, "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="], "semver": ["semver@7.7.3", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q=="], @@ -550,6 +635,8 @@ "split2": ["split2@4.2.0", "", {}, "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg=="], + "sprintf-js": ["sprintf-js@1.1.3", "", {}, "sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA=="], + "sqlstring": ["sqlstring@2.3.3", "", {}, "sha512-qC9iz2FlN7DQl3+wjwn3802RTyjCx7sDvfQEXchwa6CWOx07/WVfh91gBmQ9fahw8snwGEWU3xGzOt4tFyHLxg=="], "stackback": ["stackback@0.0.2", "", {}, "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw=="], @@ -562,6 +649,8 @@ "string-width-cjs": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="], + "string_decoder": ["string_decoder@1.3.0", "", { "dependencies": { "safe-buffer": "~5.2.0" } }, "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA=="], + "strip-ansi": ["strip-ansi@7.1.2", "", { "dependencies": { "ansi-regex": "^6.0.1" } }, "sha512-gmBGslpoQJtgnMAvOVqGZpEz9dyoKTCzy2nfz/n8aIFhN/jCE/rCmcxabB6jOOHV+0WNnylOxaxBQPSvcWklhA=="], "strip-ansi-cjs": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="], @@ -570,6 +659,10 @@ "supports-color": ["supports-color@7.2.0", "", { "dependencies": { "has-flag": "^4.0.0" } }, "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw=="], + "tarn": ["tarn@3.1.2", "", {}, "sha512-3RTvqKZcK/17jnJ8rMKFXbyNogywTs1z0gVPPwFsJGX46rkmUHOdIaSQ/aVO1rS7nH+soiXiWk7rvUXxndm8Dg=="], + + "tedious": ["tedious@20.0.0", "", { "dependencies": { "@azure/core-auth": "^1.10.1", "@azure/identity": "^4.13.1", "@azure/keyvault-keys": "^4.10.2", "@js-joda/core": "^6.0.1", "@types/node": ">=22", "bl": "^6.1.4", "iconv-lite": "^0.7.0", "js-md4": "^0.3.2", "native-duplexpair": "^1.0.0", "sprintf-js": "^1.1.3" } }, "sha512-bTR0aou0Ghucf0ytvZUJjnKHGKDV8tT57jPYtEkSpfTWFe++4uR1wxJLQ4mh5wlSvAYXzmgBxbI0vaE56qigXw=="], + "test-exclude": ["test-exclude@7.0.1", "", { "dependencies": { "@istanbuljs/schema": "^0.1.2", "glob": "^10.4.1", "minimatch": "^9.0.4" } }, "sha512-pFYqmTw68LXVjeWJMST4+borgQP2AyMNbg1BpZh9LbyhUeNkeaPF9gzfPGUAnSMV3qPYdWUwDIjjCLiSDOl7vg=="], "tinybench": ["tinybench@2.9.0", "", {}, "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg=="], @@ -586,6 +679,8 @@ "ts-api-utils": ["ts-api-utils@2.1.0", "", { "peerDependencies": { "typescript": ">=4.8.4" } }, "sha512-CUgTZL1irw8u29bzrOD/nH85jqyc74D6SshFgujOIA7osm2Rz7dYH77agkx7H4FBNxDq7Cjf+IjaX/8zwFW+ZQ=="], + "tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + "type-check": ["type-check@0.4.0", "", { "dependencies": { "prelude-ls": "^1.2.1" } }, "sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew=="], "typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], @@ -612,10 +707,14 @@ "wrap-ansi-cjs": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="], + "wsl-utils": ["wsl-utils@0.1.0", "", { "dependencies": { "is-wsl": "^3.1.0" } }, "sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw=="], + "xtend": ["xtend@4.0.2", "", {}, "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ=="], "yocto-queue": ["yocto-queue@0.1.0", "", {}, "sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q=="], + "@azure/msal-node/@azure/msal-common": ["@azure/msal-common@16.13.0", "", {}, "sha512-rOAy0KUcyBbdwVJ+f3uPpthXatFLLZN+/KWAsTLzk1aB23Xl9DRmmXYwSvBFOZyXj4jUQQ5FKxxRkhAFW1fOow=="], + "@eslint-community/eslint-utils/eslint-visitor-keys": ["eslint-visitor-keys@3.4.3", "", {}, "sha512-wpc+LXeiyiisxPlEkUzU6svyS1frIO3Mgxj1fdy7Pm8Ygzguax2N3Fa/D/ag1WqbOprdI+uY6wMUl8/a2G+iag=="], "@eslint/eslintrc/ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="], @@ -626,11 +725,9 @@ "fast-glob/glob-parent": ["glob-parent@5.1.2", "", { "dependencies": { "is-glob": "^4.0.1" } }, "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow=="], - "figlet/commander": ["commander@14.0.1", "", {}, "sha512-2JkV3gUZUVrbNA+1sjBOYLsMZ5cEEl8GTFP2a4AVz5hvasAMCQ1D2l2le/cX+pV4N6ZU17zjUahLpIXRrnWL8A=="], - - "glob/minimatch": ["minimatch@10.0.3", "", { "dependencies": { "@isaacs/brace-expansion": "^5.0.0" } }, "sha512-IPZ167aShDZZUMdRk66cyQAW3qr0WzbHkPdMYa8bzZhlHhO3jALbKdxcaak7W9FfT2rZNpQuUu4Od7ILEpXSaw=="], + "glob/minimatch": ["minimatch@9.0.5", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow=="], - "named-placeholders/lru-cache": ["lru-cache@7.18.3", "", {}, "sha512-jumlc0BIUrS3qJGgIkWZsyfAM7NCWiBcCDhnd+3NNM5KbBmLTgHVfWBcg6W+rLUsIpzpERPsvwUP7CckAQSOoA=="], + "path-scurry/lru-cache": ["lru-cache@10.4.3", "", {}, "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ=="], "string-width-cjs/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="], @@ -638,8 +735,6 @@ "strip-ansi-cjs/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="], - "test-exclude/glob": ["glob@10.4.5", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-7Bv8RF0k6xjo7d4A/PxYLbUCfb6c+Vpd2/mB2yRDlew7Jb5hEXiCD9ibfO7wpk8i4sevK6DFny9h7EYbM3/sHg=="], - "test-exclude/minimatch": ["minimatch@9.0.5", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow=="], "wrap-ansi/ansi-styles": ["ansi-styles@6.2.3", "", {}, "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg=="], @@ -650,18 +745,14 @@ "@typescript-eslint/typescript-estree/minimatch/brace-expansion": ["brace-expansion@2.0.2", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ=="], - "string-width-cjs/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="], - - "test-exclude/glob/jackspeak": ["jackspeak@3.4.3", "", { "dependencies": { "@isaacs/cliui": "^8.0.2" }, "optionalDependencies": { "@pkgjs/parseargs": "^0.11.0" } }, "sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw=="], + "glob/minimatch/brace-expansion": ["brace-expansion@2.0.2", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ=="], - "test-exclude/glob/path-scurry": ["path-scurry@1.11.1", "", { "dependencies": { "lru-cache": "^10.2.0", "minipass": "^5.0.0 || ^6.0.2 || ^7.0.0" } }, "sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA=="], + "string-width-cjs/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="], "test-exclude/minimatch/brace-expansion": ["brace-expansion@2.0.2", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ=="], "wrap-ansi-cjs/string-width/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="], "wrap-ansi-cjs/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="], - - "test-exclude/glob/path-scurry/lru-cache": ["lru-cache@10.4.3", "", {}, "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ=="], } } diff --git a/client.ts b/client.ts index 539179a..fe187da 100644 --- a/client.ts +++ b/client.ts @@ -7,6 +7,7 @@ import { Database, Statement } from "bun:sqlite"; import { Pool, type PoolClient } from "pg"; import mysql from "mysql2/promise"; +import sql from "mssql"; import { type DBConfig, StabilizeError, DBType } from "./types"; import { type Logger, StabilizeLogger } from "./logger"; @@ -28,6 +29,154 @@ function isMySQLConfig(config: DBConfig): boolean { return config.type === DBType.MySQL; } +/** + * Checks if the DB configuration is for SQL Server. + * @param config The database configuration object. + * @returns True if the configuration is for SQL Server, false otherwise. + */ +function isMSSQLConfig(config: DBConfig): boolean { + return config.type === DBType.MSSQL; +} + +/** + * Rewrites the library's `?` placeholders into a dialect's own parameter + * syntax. + * + * Every statement the ORM generates is written with `?`, and each driver + * numbers its parameters differently: PostgreSQL uses `$1`, SQL Server uses + * `@param0`. MySQL and SQLite take `?` as written, so their input is returned + * untouched. + * + * Exported, and kept free of any client state, so the rewrite can be asserted + * directly rather than through a live connection. + * + * @param query The SQL statement using `?` placeholders. + * @param dbType The target database dialect. + * @returns The statement with placeholders in the dialect's syntax. + */ +export function rewritePlaceholders(query: string, dbType: DBType): string { + if (dbType === DBType.Postgres) { + let paramIndex = 0; + return query.replace(/\?/g, () => `$${++paramIndex}`); + } + if (dbType === DBType.MSSQL) { + let paramIndex = 0; + return query.replace(/\?/g, () => `@param${paramIndex++}`); + } + return query; +} + +/** + * Binds one positional parameter to an mssql request. + * + * mssql infers a parameter's type from the value it is handed, and has nothing + * to infer from for `null` or `undefined` — the request would be sent with a + * type the server rejects. Those are therefore bound explicitly as a nullable + * `NVARCHAR`, which every column type accepts as a NULL. + * + * A plain object or array needs the same treatment for a different reason. + * Only `pg` serialises an object parameter to JSON on the way out; mssql has no + * such fallback and fails the whole statement with "Validation failed for + * parameter 'paramN'. Invalid string.", and `mysql2` does something worse still + * (@see bindMySQLParams). Encoding it here gives SQL Server the behaviour + * Postgres has. + * + * `Date` and `Buffer` are left to mssql, which infers a correct type for both. + * + * Exported so the binding rules can be asserted against a stub rather than a + * server. + * + * @param request The mssql request to bind onto. + * @param index The parameter's position, zero-based. + * @param value The value to bind. + */ +export function bindMSSQLParam( + request: { input: (name: string, typeOrValue: any, value?: any) => any }, + index: number, + value: any, +): void { + const name = `param${index}`; + if (value === null || value === undefined) { + request.input(name, sql.NVarChar, null); + } else if (isPlainJsonValue(value)) { + request.input(name, sql.NVarChar, JSON.stringify(value)); + } else { + request.input(name, value); + } +} + +/** + * Reports whether a value should be sent to a JSON column as JSON text. + * + * Only plain objects and arrays qualify. Anything with its own prototype — + * `Date`, `Buffer`, a class instance — carries meaning the drivers already know + * how to encode, and stringifying it would silently corrupt the column. + * + * @param value The parameter value. + * @returns True when the value should be JSON-encoded. + */ +export function isPlainJsonValue(value: any): boolean { + if (Array.isArray(value)) return true; + if (typeof value !== "object" || value === null) return false; + const proto = Object.getPrototypeOf(value); + return proto === Object.prototype || proto === null; +} + +/** + * Encodes the parameters of a MySQL-family statement for the driver. + * + * `mysql2` does not serialise an object to JSON the way `pg` does. It treats a + * plain object as a set of assignments — `{ nested: 1 }` binds as + * `` `nested` = 1 `` — which is meaningful only in an `UPDATE … SET` list and + * is a syntax error anywhere else. Bound inside a `VALUES` clause it rewrites + * the statement into one the server cannot parse: a single object parameter + * becomes several, and either the column count no longer matches ("Column + * count doesn't match value count at row 1", ER_WRONG_VALUE_COUNT_ON_ROW 1136) + * or the object's own keys are read as column names ("Unknown column 'nested' + * in 'field list'", ER_BAD_FIELD_ERROR 1054). A JSON column is therefore + * unwritable unless the value reaches the driver as text, which is what this + * does — the same answer `bindMSSQLParam` gives PostgreSQL's behaviour to SQL + * Server. + * + * The array is returned as a new list rather than mutated: callers reuse the + * parameter array they built, and on MySQL-less paths it must stay untouched. + * + * @param params The statement's positional parameters. + * @returns The parameters as the driver should receive them. + */ +export function bindMySQLParams(params: any[]): any[] { + return params.map((value) => + isPlainJsonValue(value) ? JSON.stringify(value) : value, + ); +} + +/** Statement prefixes that cannot change any data, and so can be replayed. */ +const READ_ONLY_PREFIXES = ["SELECT", "PRAGMA", "SHOW", "EXPLAIN", "VALUES"]; + +/** + * Checks whether a statement is safe to run more than once. + * + * Used to decide whether a failed statement may be retried. Anything not + * recognised as a read is treated as a write, so an unfamiliar statement is + * run once rather than risk being applied twice. + * + * @param query The SQL statement. + * @returns True when the statement only reads. + */ +function isReadOnlyStatement(query: string): boolean { + // Leading comments and whitespace are stripped so `/* hint */ SELECT …` + // is still recognised. + const stripped = query + .replace(/^\s*(?:\/\*[\s\S]*?\*\/|--[^\n]*\n|\s)+/, "") + .toUpperCase(); + return READ_ONLY_PREFIXES.some( + (prefix) => + stripped.startsWith(prefix) && + // Guard against a prefix matching a longer word, e.g. `SELECTED`. + !/^[A-Z_]/.test(stripped.slice(prefix.length)), + ); +} + /** * Checks if the given client is a MySQL pool. * @param client The database client. @@ -37,6 +186,25 @@ function isMySQLPool(client: any): client is mysql.Pool { return typeof client.getConnection === "function"; } +/** + * The shape of the mssql handles this client stores. + * + * Deliberately structural rather than `sql.ConnectionPool | sql.Transaction`. + * Naming those types would make the emitted `client.d.ts` import `mssql`, and + * every consumer of the published package — including one that only ever talks + * to SQLite — would then need declarations for a driver it does not use. A pool + * exposes `connect`, `close` and `request`; an open transaction exposes + * `begin`, `commit` and `rollback`, and is recognised by `begin`. + */ +interface MSSQLHandle { + connect?: () => Promise; + close?: () => Promise; + request?: () => unknown; + begin?: (...args: any[]) => any; + commit?: (...args: any[]) => any; + rollback?: (...args: any[]) => any; +} + /** * Provides a unified database client for interacting with PostgreSQL, MySQL, and SQLite. */ @@ -46,13 +214,21 @@ export class DBClient { | Pool | mysql.Pool | PoolClient - | mysql.PoolConnection; + | mysql.PoolConnection + | MSSQLHandle; private logger: Logger; public readonly config: DBConfig; private retryAttempts: number; private retryDelay: number; private maxJitter: number; + /** + * The in-flight `connect()` on the mssql pool, if one has been started. + * Held so that concurrent callers share a single connection attempt rather + * than each opening one. + */ + private mssqlConnectPromise: Promise | null = null; + private preparedStatements: Map = new Map(); public isTransactionClient: boolean = false; @@ -60,12 +236,17 @@ export class DBClient { * Constructs a new DBClient instance. * @param config The database configuration object. * @param logger Optional logger instance. Uses StabilizeLogger if not provided. - * @param existingClient Optional existing transaction client. + * @param existingClient Optional existing transaction client. For SQL Server + * this is the `sql.Transaction` the statements should run inside. */ constructor( config: DBConfig, logger: Logger = new StabilizeLogger(), - existingClient: PoolClient | mysql.PoolConnection | null = null, + existingClient: + | PoolClient + | mysql.PoolConnection + | MSSQLHandle + | null = null, ) { this.config = config; this.logger = logger; @@ -95,7 +276,69 @@ export class DBClient { } else if (config.type === DBType.Postgres) { this.client = new Pool({ connectionString: config.connectionString! }); this.logger.logDebug(`Initialized Postgres Pool client.`); + } else if (isMSSQLConfig(config)) { + // The pool object is built here but deliberately left unconnected: + // `ConnectionPool.connect()` is asynchronous and this method is called + // from the constructor, so awaiting it would make construction async for + // every driver. `ensureMSSQLConnected` opens it on first use instead. + this.client = new sql.ConnectionPool(config.connectionString); + this.logger.logDebug(`Initialized MSSQL Pool client.`); + } + } + + /** + * Opens the mssql pool, once, on first use. + * + * `initializeClient` cannot do this — see the note there — so every statement + * awaits it before building its request. The promise is memoised so that + * concurrent first queries share one connection attempt. + * + * A client holding an `sql.Transaction` has nothing to connect: the + * transaction already owns a pooled connection, and `connect()` does not + * exist on it. + */ + private async ensureMSSQLConnected(): Promise { + const pool = this.client as MSSQLHandle; + if (!pool || typeof pool.connect !== "function") return; + if (!this.mssqlConnectPromise) { + this.mssqlConnectPromise = pool.connect().then(() => { + this.logger.logDebug("MSSQL pool connected."); + }); + } + await this.mssqlConnectPromise; + } + + /** + * Builds the mssql request a statement should run through. + * + * A transaction-bound client holds an `sql.Transaction`, and `begin()` is + * what distinguishes it: a statement sent through a `Request` built from the + * transaction stays inside it, whereas one built from the pool would run on + * an unrelated connection and commit on its own. + */ + private async mssqlRequest(): Promise { + const handle = this.client as MSSQLHandle; + if (handle && typeof handle.begin === "function") { + return new sql.Request(handle as any); } + await this.ensureMSSQLConnected(); + return new sql.Request(handle as any); + } + + /** + * Sends one statement through a fresh mssql request and resolves its result. + * + * Parameter values are bound in order under the names the placeholder + * rewrite produced, so the statement the server sees carries neither more + * nor fewer parameters than were supplied. + */ + private async runMSSQL( + query: string, + params: any[], + ): Promise<{ recordset: any[]; rowsAffected: number[] }> { + const request = await this.mssqlRequest(); + params.forEach((value, index) => bindMSSQLParam(request, index, value)); + return request.query(rewritePlaceholders(query, DBType.MSSQL)); } /** @@ -114,7 +357,13 @@ export class DBClient { async query(query: string, params: any[] = []): Promise { const start = Date.now(); - for (let attempt = 1; attempt <= this.retryAttempts; attempt++) { + // Only reads are retried. Every write in the ORM — insert, update, delete, + // upsert — goes through this method, and a failure that happened *after* + // the database committed (a dropped connection on the way back, say) would + // be retried and applied a second time. A read is safe to repeat. + const attempts = isReadOnlyStatement(query) ? this.retryAttempts : 1; + + for (let attempt = 1; attempt <= attempts; attempt++) { try { let result: any; @@ -126,13 +375,22 @@ export class DBClient { } result = stmt.all(...params); } else if (this.config.type === DBType.MySQL) { - const [rows] = await (this.client as mysql.Pool).query(query, params); + const [rows] = await (this.client as mysql.Pool).query( + query, + bindMySQLParams(params), + ); result = rows; } else if (this.config.type === DBType.Postgres) { - let paramIndex = 0; - const pgQuery = query.replace(/\?/g, () => `$${++paramIndex}`); - const pgResult = await (this.client as Pool).query(pgQuery, params); + const pgResult = await (this.client as Pool).query( + rewritePlaceholders(query, DBType.Postgres), + params, + ); result = Array.isArray(pgResult.rows) ? pgResult.rows : []; + } else if (this.config.type === DBType.MSSQL) { + const mssqlResult = await this.runMSSQL(query, params); + result = Array.isArray(mssqlResult.recordset) + ? mssqlResult.recordset + : []; } else { throw new StabilizeError( "Unknown database client type", @@ -145,9 +403,9 @@ export class DBClient { return Array.isArray(result) ? (result as T[]) : []; } catch (error) { this.logger.logError(error as Error); - if (attempt === this.retryAttempts) { + if (attempt === attempts) { throw new StabilizeError( - `Query failed after ${this.retryAttempts} attempts: ${(error as Error).message}`, + `Query failed after ${attempts} attempt${attempts === 1 ? "" : "s"}: ${(error as Error).message}`, "QUERY_ERROR", ); } @@ -178,10 +436,63 @@ export class DBClient { if (this.isTransactionClient) return callback(this); if (this.client instanceof Database) { - const tx = this.client.transaction(async () => { - return await callback(this); - }); - return await tx(); + // `bun:sqlite`'s own `db.transaction()` is synchronous: it issues + // COMMIT as soon as the callback returns, and an async callback returns + // a pending promise at its first `await`. The COMMIT would therefore + // land before the work finished, so a later throw rolled nothing back + // and every write in the library ran non-atomically. Drive the + // transaction explicitly instead, which awaits properly. + this.logger.logDebug("Starting SQLite transaction."); + // SQLite runs on a single connection, so the callback receives this same + // client rather than a new one. Mark it as being inside a transaction for + // the duration: without that, a nested `transaction()` (a repository + // write inside the caller's transaction) would issue a second BEGIN and + // fail with "cannot start a transaction within a transaction". + const wasTransactionClient = this.isTransactionClient; + this.isTransactionClient = true; + await this.migrationQuery("BEGIN"); + try { + const result = await callback(this); + await this.migrationQuery("COMMIT"); + return result; + } catch (error) { + try { + await this.migrationQuery("ROLLBACK"); + } catch (rollbackError) { + this.logger.logError(rollbackError as Error); + } + throw error; + } finally { + this.isTransactionClient = wasTransactionClient; + } + } + + if (this.config.type === DBType.MSSQL) { + // SQL Server has no `BEGIN`/`COMMIT` text: the transaction is a + // server-side object opened on a borrowed pooled connection, and every + // statement inside it has to be sent through a `Request` built from that + // object. `Database` cannot appear here, so the pool handle is safe to + // treat as a pool — the constructor marks a transaction-bound client and + // the guard above returns early for it. + await this.ensureMSSQLConnected(); + const transaction = new sql.Transaction(this.client as any); + const txClient = new DBClient(this.config, this.logger, transaction); + this.logger.logDebug("Starting MSSQL transaction."); + await transaction.begin(); + try { + const result = await callback(txClient); + await transaction.commit(); + return result; + } catch (error) { + try { + await transaction.rollback(); + } catch (rollbackError) { + this.logger.logError(rollbackError as Error); + } + throw error; + } + // Nothing to release: unlike the pg and mysql pools, mssql returns the + // borrowed connection to the pool as part of commit/rollback. } if (isMySQLPool(this.client)) { @@ -234,6 +545,15 @@ export class DBClient { async close() { if (this.client instanceof Database) { this.client.close(); + } else if ( + this.config.type === DBType.MSSQL && + this.client && + "close" in this.client + ) { + // An mssql pool is torn down with `close()`, not the `end()` the pg and + // mysql pools expose — the generic branch below would silently skip it + // and leave the sockets open. + await (this.client as MSSQLHandle).close!(); } else if (this.client && "end" in this.client) { await (this.client as any).end(); } @@ -254,14 +574,20 @@ export class DBClient { } else if (this.config.type === DBType.MySQL) { const [mysqlResult] = await (this.client as mysql.Pool).query( query, - params, + bindMySQLParams(params), ); affectedRows = (mysqlResult as any).affectedRows ?? 0; } else if (this.config.type === DBType.Postgres) { - let paramIndex = 0; - const pgQuery = query.replace(/\?/g, () => `$${++paramIndex}`); - const pgResult = await (this.client as Pool).query(pgQuery, params); + const pgResult = await (this.client as Pool).query( + rewritePlaceholders(query, DBType.Postgres), + params, + ); affectedRows = pgResult.rowCount ?? 0; + } else if (this.config.type === DBType.MSSQL) { + const mssqlResult = await this.runMSSQL(query, params); + // mssql reports one entry per statement in the batch, so the first is the + // count for the statement that was sent. + affectedRows = mssqlResult.rowsAffected?.[0] ?? 0; } const executionTime = Date.now() - start; @@ -286,11 +612,14 @@ export class DBClient { } stmt.run(...params); } else if (this.config.type === DBType.MySQL) { - await (this.client as mysql.Pool).query(query, params); + await (this.client as mysql.Pool).query(query, bindMySQLParams(params)); } else if (this.config.type === DBType.Postgres) { - let paramIndex = 0; - const pgQuery = query.replace(/\?/g, () => `$${++paramIndex}`); - await (this.client as Pool).query(pgQuery, params); + await (this.client as Pool).query( + rewritePlaceholders(query, DBType.Postgres), + params, + ); + } else if (this.config.type === DBType.MSSQL) { + await this.runMSSQL(query, params); } const executionTime = Date.now() - start; diff --git a/docker-compose.test.yml b/docker-compose.test.yml new file mode 100644 index 0000000..15fa6ab --- /dev/null +++ b/docker-compose.test.yml @@ -0,0 +1,111 @@ +# Real-database test fleet for stabilize-orm. +# +# docker compose -f docker-compose.test.yml up -d --wait +# bun test +# docker compose -f docker-compose.test.yml down -v +# +# Host ports are deliberately non-standard so this never collides with a +# database you already run locally. Everything lives on its own network and +# every container is named `stabilize-test-*`, so `down` cannot touch anything +# else you have running. +# +# SQLite needs no container — `bun:sqlite` is built into the runtime. + +name: stabilize-test + +services: + postgres: + image: postgres:18-alpine + container_name: stabilize-test-postgres + environment: + POSTGRES_USER: stabilize + POSTGRES_PASSWORD: stabilize + POSTGRES_DB: stabilize_test + ports: + - "55432:5432" + healthcheck: + # -U matters: without it pg_isready checks the default user, not ours. + test: ["CMD-SHELL", "pg_isready -U stabilize -d stabilize_test"] + interval: 2s + timeout: 5s + retries: 30 + tmpfs: + # Keeps the test database in RAM. These are throwaway fixtures, and it + # makes the suite noticeably faster than a bind mount on Windows. + # + # The mount point is `/var/lib/postgresql`, not the older + # `/var/lib/postgresql/data`: from 18 onward the image keeps the cluster + # in a versioned subdirectory so `pg_upgrade --link` can cross the mount + # boundary. Mounting the old path makes 18 refuse to boot, since it finds + # data sitting in an "unused mount/volume". + - /var/lib/postgresql + networks: [stabilize-test] + + mysql: + image: mysql:8 + container_name: stabilize-test-mysql + environment: + MYSQL_ROOT_PASSWORD: stabilize + MYSQL_DATABASE: stabilize_test + MYSQL_USER: stabilize + MYSQL_PASSWORD: stabilize + ports: + - "53306:3306" + healthcheck: + # -p is attached to the flag on purpose; a space would warn about the + # password being visible in the process list. + test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-ustabilize", "-pstabilize"] + interval: 2s + timeout: 5s + retries: 40 + tmpfs: + - /var/lib/mysql + networks: [stabilize-test] + + mariadb: + image: mariadb:11 + container_name: stabilize-test-mariadb + environment: + MARIADB_ROOT_PASSWORD: stabilize + MARIADB_DATABASE: stabilize_test + MARIADB_USER: stabilize + MARIADB_PASSWORD: stabilize + ports: + - "53307:3306" + healthcheck: + test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"] + interval: 2s + timeout: 5s + retries: 40 + tmpfs: + - /var/lib/mysql + networks: [stabilize-test] + + mssql: + image: mcr.microsoft.com/mssql/server:2022-latest + container_name: stabilize-test-mssql + environment: + ACCEPT_EULA: "Y" + # SQL Server enforces a complexity policy: 8+ chars with upper, lower, + # digit and symbol. A simpler password makes the container exit on boot. + MSSQL_SA_PASSWORD: "Stabilize!Test123" + MSSQL_PID: Developer + ports: + - "51433:1433" + healthcheck: + # The image ships no sqlcmd on PATH by default in 2022+, so readiness is + # probed over TCP rather than by querying. /dev/tcp is a bash builtin. + test: + [ + "CMD-SHELL", + "bash -c 'exec 3<>/dev/tcp/127.0.0.1/1433' || exit 1", + ] + interval: 3s + timeout: 5s + retries: 40 + start_period: 20s + networks: [stabilize-test] + +networks: + stabilize-test: + driver: bridge diff --git a/hooks.ts b/hooks.ts index f1db9c7..31b52e9 100644 --- a/hooks.ts +++ b/hooks.ts @@ -52,17 +52,28 @@ export function registerHooks( * Combines hooks from MetadataStorage and class methods. * @param entity The entity instance. * @param type The hook type (e.g., 'beforeCreate'). + * @param model The model class the entity belongs to. Pass it whenever the + * entity may be a plain row rather than a class instance. * @returns An array of Hook objects to execute. */ -export function getHooks(entity: any, type: HookType): Hook[] { +export function getHooks( + entity: any, + type: HookType, + model?: Function, +): Hook[] { const hooks: Hook[] = []; if (!entity) return hooks; const proto = Object.getPrototypeOf(entity); - if (!proto) return hooks; - const model = proto.constructor; + if (!proto && !model) return hooks; + + // The caller's model wins over the entity's prototype. Reads return plain + // objects straight from the driver, so `proto.constructor` is `Object` and + // the metadata lookup below found nothing — which is why every `after*` + // hook, and both delete hooks, silently never ran. + const resolved = model ?? proto.constructor; // Get hooks from MetadataStorage - const config = MetadataStorage.getModelMetadata(model); + const config = MetadataStorage.getModelMetadata(resolved); if (config?.hooks?.[type]) { const callbacks = Array.isArray(config.hooks[type]) ? config.hooks[type] diff --git a/index.ts b/index.ts index 0def8e9..d8749e9 100644 --- a/index.ts +++ b/index.ts @@ -50,6 +50,8 @@ export class Stabilize { private cache: Cache | null; private logger: Logger; public events: StabilizeEmitter; + /** Repositories handed out so far, keyed by model. @see getRepository */ + private repositories = new Map>(); constructor( config: DBConfig, @@ -82,8 +84,22 @@ export class Stabilize { * ``` */ getRepository(model: new (...args: any[]) => T): Repository { - const cacheConfig = this.cache ? this.cache.config : undefined; - return new Repository(this.client, model, cacheConfig, this.logger); + // Memoised per model. Every Repository owns an optional cache handle, so + // building a new one on each call opened a second Redis connection that + // nothing disconnected and that `getCacheStats()` never saw — it reports on + // the ORM's own cache, which no repository was using. + const existing = this.repositories.get(model); + if (existing) return existing as Repository; + + const repository = new Repository( + this.client, + model, + this.cache?.config, + this.logger, + this.cache, + ); + this.repositories.set(model, repository); + return repository; } /** @@ -219,6 +235,20 @@ export class Stabilize { total: raw._allConnections.length, }; } + // `Stabilize.client` is the DBClient wrapper, so the driver's pool is one + // level down. Only the SQL Server branch below reads through it; the two + // checks above are left reading `raw` exactly as they always have. + const pool = raw.client ?? raw; + if ( + this.client.config.type === DBType.MSSQL && + typeof pool?.size === "number" + ) { + return { + active: pool.borrowed ?? 0, + idle: pool.available ?? 0, + total: pool.size ?? 0, + }; + } return { active: -1, idle: -1, total: -1 }; } } diff --git a/migrations.ts b/migrations.ts index 1b66a40..f6099f6 100644 --- a/migrations.ts +++ b/migrations.ts @@ -15,6 +15,41 @@ import { DataTypes, } from "./types"; +/** + * Quotes an identifier for the target dialect. + * + * `"x"` is an identifier only where the dialect's grammar says so. MySQL and + * MariaDB read it as a *string literal* unless the server runs with + * `ANSI_QUOTES` in `sql_mode` — off by default — so `CREATE TABLE "users" (…)` + * is a syntax error there and backticks are the spelling that always parses. + * Postgres and SQLite quote with `"`, and T-SQL accepts it too under + * `QUOTED_IDENTIFIER ON`, which is the default, so those three keep it. + * + * @param name The bare identifier. + * @param dbType The target database dialect. + * @returns The identifier wrapped for the dialect. + */ +export function quoteIdentifier(name: string, dbType: DBType): string { + return dbType === DBType.MySQL ? `\`${name}\`` : `"${name}"`; +} + +/** + * @internal + * Recovers the bare name from an identifier quoted for any dialect. + * + * SQL Server's catalogue functions take an unquoted name, so the quoting + * {@link quoteIdentifier} added has to come back off before one is built. + * Both spellings are stripped rather than just the one the current dialect + * uses, so a caller passing an already-quoted identifier gets the right answer + * whichever dialect produced it. + * + * @param name The quoted identifier. + * @returns The bare name. + */ +function unquoteIdentifier(name: string): string { + return name.replace(/^[`"]/, "").replace(/[`"]$/, ""); +} + /** * @internal * Formats a SQL query with placeholders for the target database dialect. @@ -30,6 +65,77 @@ function formatQuery(query: string, dbType: DBType): string { return query; } +/** + * Builds a `CREATE TABLE` that is a no-op when the table already exists. + * + * SQLite, MySQL and PostgreSQL spell this `CREATE TABLE IF NOT EXISTS`. T-SQL + * has no such clause and rejects the statement outright, so SQL Server gets the + * equivalent written as a leading existence check on the same batch instead. + * + * @param table The table identifier, already quoted for the dialect if needed. + * @param body The column definitions, without the surrounding parentheses. + * @param dbType The target database dialect. + * @returns The complete statement. + */ +export function createTableIfNotExistsSQL( + table: string, + body: string, + dbType: DBType, +): string { + if (dbType !== DBType.MSSQL) { + return `CREATE TABLE IF NOT EXISTS ${table} (${body})`; + } + // `OBJECT_ID` takes the bare name, not the quoted identifier, and the `N'…'` + // prefix keeps it Unicode so a non-ASCII table name still resolves. + const bare = unquoteIdentifier(table).replace(/'/g, "''"); + return `IF OBJECT_ID(N'${bare}', N'U') IS NULL CREATE TABLE ${table} (${body})`; +} + +/** + * Builds a `CREATE INDEX` that is a no-op when the index already exists. + * + * Only Postgres and SQLite support the clause itself. As with + * {@link createTableIfNotExistsSQL}, SQL Server has no `IF NOT EXISTS` clause + * to hang on the statement, so the check is a separate one against + * `sys.indexes` on the same batch. MySQL and MariaDB have neither the clause + * nor an inline substitute, so for them the caller does the checking — see the + * `DBType.MySQL` branch below. + * + * @param index The index identifier, already quoted for the dialect if needed. + * @param table The table identifier, already quoted for the dialect if needed. + * @param columns The indexed column identifiers, already quoted. + * @param unique Whether the index enforces uniqueness. + * @param dbType The target database dialect. + * @returns The complete statement. + */ +export function createIndexIfNotExistsSQL( + index: string, + table: string, + columns: string[], + unique: boolean, + dbType: DBType, +): string { + const kind = unique ? "UNIQUE INDEX" : "INDEX"; + const statement = `CREATE ${kind} ${index} ON ${table} (${columns.join(", ")})`; + if (dbType === DBType.MySQL) { + // MySQL and MariaDB have no `IF NOT EXISTS` clause on `CREATE INDEX` — the + // server rejects it as a syntax error however the identifiers are quoted — + // so the statement is issued plain, and the guarantee has to come from + // whoever calls this. `autoMigrate` is the only caller and satisfies it + // already: it reads the table's indexes from `information_schema` (via + // `SHOW INDEX`) and skips any name it finds, which is the pre-check, done + // once for every index rather than once per statement. Nothing else may + // call this for a MySQL target without doing the same. + return statement; + } + if (dbType !== DBType.MSSQL) { + return `CREATE ${kind} IF NOT EXISTS ${index} ON ${table} (${columns.join(", ")})`; + } + const bareIndex = unquoteIdentifier(index).replace(/'/g, "''"); + const bareTable = unquoteIdentifier(table).replace(/'/g, "''"); + return `IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = N'${bareIndex}' AND object_id = OBJECT_ID(N'${bareTable}')) ${statement}`; +} + /** * Maps an abstract data type to the correct SQL type string for the specified database dialect. * @param dt The data type to map. @@ -108,6 +214,38 @@ function mapDataTypeToSql(dt: DataTypes | string, dbType: DBType): string { return "TEXT"; } } + if (dbType === DBType.MSSQL) { + switch (type) { + case "string": + return "NVARCHAR(255)"; + case "text": + return "NVARCHAR(MAX)"; + case "integer": + return "INT"; + case "bigint": + return "BIGINT"; + case "float": + return "REAL"; + case "double": + return "FLOAT"; + case "decimal": + return "DECIMAL(10,2)"; + case "boolean": + return "BIT"; + case "date": + return "DATE"; + case "datetime": + return "DATETIME2"; + case "json": + return "NVARCHAR(MAX)"; + case "uuid": + return "UNIQUEIDENTIFIER"; + case "blob": + return "VARBINARY(MAX)"; + default: + return "NVARCHAR(MAX)"; + } + } if (dbType === DBType.SQLite) { switch (type) { case "string": @@ -155,6 +293,8 @@ function getAutoIncrementPK(dbType: DBType): string { return "SERIAL PRIMARY KEY"; case DBType.MySQL: return "INT AUTO_INCREMENT PRIMARY KEY"; + case DBType.MSSQL: + return "INT IDENTITY(1,1) PRIMARY KEY"; case DBType.SQLite: default: return "INTEGER PRIMARY KEY AUTOINCREMENT"; @@ -193,7 +333,28 @@ export async function generateMigration( if (key === "id") { defParts.push("id"); - defParts.push(getAutoIncrementPK(dbType)); + // `type` is typed as the DataTypes enum but may arrive as a literal + // string when the model came from another copy of the ORM. + const idTypeRaw: any = col.type; + const idTypeStr = + typeof idTypeRaw === "string" + ? idTypeRaw.toLowerCase() + : (DataTypes as any)[idTypeRaw]?.toLowerCase(); + if (idTypeStr === "string" || idTypeStr === "uuid") { + defParts.push( + dbType === DBType.Postgres + ? "UUID PRIMARY KEY" + : dbType === DBType.MySQL + ? "VARCHAR(255) PRIMARY KEY" + : dbType === DBType.MSSQL + ? idTypeStr === "uuid" + ? "UNIQUEIDENTIFIER PRIMARY KEY" + : "NVARCHAR(255) PRIMARY KEY" + : "TEXT PRIMARY KEY", + ); + } else { + defParts.push(getAutoIncrementPK(dbType)); + } } else { defParts.push(col.name || key); defParts.push(mapDataTypeToSql(col.type, dbType)); @@ -217,11 +378,23 @@ export async function generateMigration( columnDefs.push(defParts.join(" ")); } - // Add timestamp columns if enabled + // Add timestamp columns if enabled. A model may declare them in `columns` + // as well as in `timestamps` — the documented pattern — so skip any column + // that is already in the table definition rather than emitting it twice + // (which makes the CREATE TABLE fail with "duplicate column name"). if (timestamps) { + const declared = new Set( + Object.entries(columns).map(([key, col]) => col.name || key), + ); for (const [field, colName] of Object.entries(timestamps)) { + if (!colName || declared.has(colName)) continue; // Use the field name defined in the timestamps config - let sqlType = dbType === DBType.Postgres ? "TIMESTAMP" : "DATETIME"; + let sqlType = + dbType === DBType.Postgres + ? "TIMESTAMP" + : dbType === DBType.MSSQL + ? "DATETIME2" + : "DATETIME"; let def = `${colName} ${sqlType} NOT NULL`; // Set default value for createdAt, and optionally for updatedAt @@ -240,7 +413,7 @@ export async function generateMigration( } const up: string[] = [ - `CREATE TABLE IF NOT EXISTS ${tableName} (${columnDefs.join(", ")})`, + createTableIfNotExistsSQL(tableName, columnDefs.join(", "), dbType), ]; const down: string[] = [`DROP TABLE IF EXISTS ${tableName}`]; @@ -274,10 +447,17 @@ function generateHistoryMigration( let tsType = dbType === DBType.MySQL ? "DATETIME" - : dbType === DBType.SQLite - ? "TEXT" - : "TIMESTAMP"; - let modByType = dbType === DBType.MySQL ? "VARCHAR(255)" : "TEXT"; + : dbType === DBType.MSSQL + ? "DATETIME2" + : dbType === DBType.SQLite + ? "TEXT" + : "TIMESTAMP"; + let modByType = + dbType === DBType.MySQL + ? "VARCHAR(255)" + : dbType === DBType.MSSQL + ? "NVARCHAR(255)" + : "TEXT"; let modAtType = tsType + (dbType === DBType.Postgres ? " DEFAULT CURRENT_TIMESTAMP" : ""); @@ -296,7 +476,7 @@ function generateHistoryMigration( `modified_at ${modAtType}`, ]; return [ - `CREATE TABLE IF NOT EXISTS ${historyTable} (${historyColumns.join(", ")})`, + createTableIfNotExistsSQL(historyTable, historyColumns.join(", "), dbType), `DROP TABLE IF EXISTS ${historyTable}`, ]; } @@ -321,6 +501,14 @@ function getMigrationsTableSQL(dbType: DBType): string { name VARCHAR(255) UNIQUE NOT NULL, applied_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP )`; + case DBType.MSSQL: + return createTableIfNotExistsSQL( + "stabilize_migrations", + `id INT IDENTITY(1,1) PRIMARY KEY, + name NVARCHAR(255) UNIQUE NOT NULL, + applied_at DATETIME2 NOT NULL DEFAULT CURRENT_TIMESTAMP`, + DBType.MSSQL, + ); case DBType.SQLite: default: return `CREATE TABLE IF NOT EXISTS stabilize_migrations ( @@ -366,6 +554,11 @@ export async function runMigrations(config: DBConfig, migrations: Migration[]) { let appliedAt: string; if (dbType === DBType.MySQL) { appliedAt = new Date().toISOString().slice(0, 19).replace("T", " "); + } else if (dbType === DBType.MSSQL) { + // The trailing `Z` an ISO string carries is only meaningful for + // `datetimeoffset`; `datetime2` wants a space separator and no + // zone designator, which every server language parses the same way. + appliedAt = new Date().toISOString().slice(0, 23).replace("T", " "); } else { appliedAt = new Date().toISOString(); } diff --git a/model.ts b/model.ts index 5a36147..566b836 100644 --- a/model.ts +++ b/model.ts @@ -57,29 +57,94 @@ export interface ModelConfig { timestamps?: TimestampsConfig; // Auto-managed timestamp columns } +/** + * Key under which the model registry is stored on `globalThis`. + * + * The registry is process-wide rather than module-local so that every copy of + * this module shares it. If the ORM is bundled more than once (duplicate + * dependency, mixed CJS/ESM resolution, a CLI plus an app), each copy would + * otherwise get its own Map and metadata written by one copy would be + * invisible to the other — models would look as though they were never + * defined. + */ +const MODEL_REGISTRY_KEY = Symbol.for("stabilize-orm.model-registry"); + +function getModelRegistry(): Map { + const root = globalThis as any; + if (!root[MODEL_REGISTRY_KEY]) { + root[MODEL_REGISTRY_KEY] = new Map(); + } + return root[MODEL_REGISTRY_KEY]; +} + +/** + * Rebuilds a model configuration from the static properties mirrored onto the + * class by {@link MetadataStorage.setModelMetadata}. + * + * @param model - The class constructor for the model. + * @returns The reconstructed configuration, or undefined if the class carries + * no table name and therefore was never registered. + */ +function getStaticMetadata(model: Function): ModelConfig | undefined { + const candidate = model as any; + if (!candidate || !candidate.tableName) return undefined; + return { + tableName: candidate.tableName, + versioned: candidate.versioned || false, + softDelete: candidate.softDelete || false, + columns: candidate.columns || {}, + relations: Array.isArray(candidate.relations) ? candidate.relations : [], + scopes: candidate.scopes || {}, + timestamps: candidate.timestamps || {}, + hooks: candidate.hooks, + }; +} + /** * Metadata storage for models. * Stores and retrieves model configuration such as columns, relations, scopes, etc. */ export class MetadataStorage { - private static models: Map = new Map(); - /** * Associates model metadata with a class constructor. + * + * The configuration is stored both in the shared registry and as static + * properties on the class itself, so it survives being read from a different + * copy of the ORM. + * * @param model - The class constructor for the model. * @param config - The model configuration object. */ static setModelMetadata(model: Function, config: ModelConfig) { - this.models.set(model, config); + getModelRegistry().set(model, config); + + // The mirror is best effort: the registry above is authoritative. A frozen + // class, or one whose static `columns` is a getter without a setter, would + // otherwise make registration throw. + const target = model as any; + if (typeof target !== "function") return; + try { + target.tableName = config.tableName; + target.versioned = config.versioned || false; + target.softDelete = config.softDelete || false; + target.columns = config.columns; + target.relations = config.relations || []; + target.scopes = config.scopes || {}; + target.timestamps = config.timestamps || {}; + if (config.hooks) target.hooks = config.hooks; + } catch { + // Ignore: metadata is still available through the registry. + } } /** * Retrieves the model configuration for a given model class. * @param model - The class constructor for the model. - * @returns The model configuration or undefined if not found. + * @returns The model configuration, falling back to the class statics when + * the class was registered by a different copy of the ORM. */ static getModelMetadata(model: Function): ModelConfig | undefined { - return this.models.get(model); + return getModelRegistry().get(model) ?? getStaticMetadata(model); } /** @@ -173,7 +238,7 @@ export class MetadataStorage { * @returns The model constructor or undefined if not found. */ static getModelByTableName(tableName: string): Function | undefined { - for (const [model, config] of this.models) { + for (const [model, config] of getModelRegistry()) { if (config.tableName === tableName) { return model; } @@ -202,7 +267,8 @@ export function defineModel(config: ModelConfig) { } } - // Store metadata + // Stored in the shared registry and mirrored onto the class as statics so + // that metadata stays readable across bundle boundaries. MetadataStorage.setModelMetadata(Model, { tableName: config.tableName, versioned: config.versioned || false, @@ -211,6 +277,7 @@ export function defineModel(config: ModelConfig) { relations: config.relations || [], scopes: config.scopes || {}, timestamps: config.timestamps || {}, + hooks: config.hooks, }); return Model; diff --git a/mssql.d.ts b/mssql.d.ts new file mode 100644 index 0000000..c94ddb7 --- /dev/null +++ b/mssql.d.ts @@ -0,0 +1,52 @@ +/** + * @file mssql.d.ts + * @description Minimal ambient types for the `mssql` driver. + * + * `mssql` ships no declarations of its own, and taking `@types/mssql` as a + * dependency would leave every consumer of the published package needing it + * too. Only the surface the client actually touches is declared, in the same + * spirit as `bun-sqlite.d.ts`. + */ + +declare module "mssql" { + namespace sql { + /** A pool of connections to one SQL Server instance. */ + class ConnectionPool { + constructor(config: string | Record); + connect(): Promise; + close(): Promise; + request(): Request; + readonly connected: boolean; + readonly size: number; + readonly available: number; + readonly pending: number; + readonly borrowed: number; + } + + /** A server-side transaction, opened on a borrowed pooled connection. */ + class Transaction { + constructor(pool: ConnectionPool); + begin(isolationLevel?: number): Promise; + commit(): Promise; + rollback(): Promise; + } + + /** One batch of statements, bound to a pool or to an open transaction. */ + class Request { + constructor(parent?: ConnectionPool | Transaction); + input(name: string, value: any): Request; + input(name: string, type: any, value: any): Request; + query(command: string): Promise<{ + recordset: any[]; + recordsets: any[][]; + rowsAffected: number[]; + output: Record; + }>; + } + + /** Concrete data-type markers accepted by `Request.input`. */ + const NVarChar: any; + } + + export = sql; +} diff --git a/package.json b/package.json index bd9cfc5..20b4a4f 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "stabilize-orm", - "version": "2.1.0", + "version": "2.2.1", "description": "A modern, type-safe ORM for Bun, Node.js, and Deno with support for SQLite, MySQL, PostgreSQL, Redis caching, optimistic locking, cursor pagination, and a full-featured query builder.", "main": "dist/index.js", "types": "dist/index.d.ts", @@ -68,7 +68,8 @@ "test": "bun test", "format": "bunx prettier --write .", "clean": "rm -rf dist", - "release": "bun run build && npm publish" + "release": "bun run build && npm publish", + "release:d": "npm version patch && npm publish --otp=$OTP" }, "keywords": [ "orm", @@ -99,6 +100,7 @@ "license": "MIT", "dependencies": { "ioredis": "^5.8.1", + "mssql": "^12.7.2", "mysql2": "^3.15.2", "pg": "^8.16.3" }, diff --git a/query-builder.ts b/query-builder.ts index f52082c..e45c9d6 100644 --- a/query-builder.ts +++ b/query-builder.ts @@ -7,7 +7,7 @@ import { DBClient } from "./client"; import { Cache } from "./cache"; import { MetadataStorage } from "./model"; -import { StabilizeError } from "./types"; +import { DBType, StabilizeError } from "./types"; type JoinType = "INNER" | "LEFT" | "RIGHT" | "FULL" | "CROSS"; type LockMode = @@ -16,10 +16,75 @@ type LockMode = | "FOR NO KEY UPDATE" | "FOR KEY SHARE"; +/** + * Builds the clause that limits how many rows a statement returns. + * + * SQLite, MySQL and PostgreSQL all spell this `LIMIT … OFFSET …`, and the three + * are emitted here exactly as they always were. T-SQL has no `LIMIT`: it needs + * `OFFSET … ROWS FETCH NEXT … ROWS ONLY`, and both of those clauses are legal + * only on a statement that has an `ORDER BY`. A statement with no ordering is + * therefore given `ORDER BY (SELECT NULL)` — a constant ordering, which leaves + * the row order undefined exactly as an unordered `LIMIT` did — purely to + * satisfy that requirement. + * + * Pure, and exported, so every case can be asserted without a server. + * + * @param limit The `LIMIT` value, or null when none was set. + * @param offset The `OFFSET` value, or null when none was set. + * @param hasOrderBy Whether the statement already carries an `ORDER BY`. + * @param dbType The target database dialect. Defaults to the LIMIT dialects. + * @returns The clause to append, or an empty string when neither was set. + */ +export function buildLimitClause( + limit: number | null, + offset: number | null, + hasOrderBy: boolean, + dbType?: DBType, +): string { + if (limit === null && offset === null) return ""; + + if (dbType === DBType.MSSQL) { + const orderBy = hasOrderBy ? "" : "\nORDER BY (SELECT NULL)"; + if (offset === null) { + // `FETCH NEXT` cannot appear without an `OFFSET`, so a bare limit skips + // no rows rather than being translated into a different clause. + return `${orderBy}\nOFFSET 0 ROWS FETCH NEXT ${limit} ROWS ONLY`; + } + const offsetClause = `\nOFFSET ${offset} ROWS`; + return limit === null + ? `${orderBy}${offsetClause}` + : `${orderBy}${offsetClause} FETCH NEXT ${limit} ROWS ONLY`; + } + + let clause = ""; + if (limit !== null) { + clause += `\nLIMIT ${limit}`; + } + if (offset !== null) { + // SQLite and MySQL reject a bare OFFSET; every dialect accepts an + // explicitly large LIMIT, so supply one when no limit was set. + if (limit === null) { + clause += "\nLIMIT 9223372036854775807"; + } + clause += ` OFFSET ${offset}`; + } + return clause; +} + export class QueryBuilder { private table: string; + /** + * The dialect row limiting is spelled for. + * + * Left unset until a client is known — `execute` and its siblings set it from + * the client they are handed — so a builder that is only ever rendered + * without one keeps emitting the `LIMIT` form the three row-limiting dialects + * share. + */ + private dialect: DBType | null = null; private tableAlias: string | null = null; private selectFields: string[] = ["*"]; + private selectParams: any[] = []; private isDistinct = false; private joins: string[] = []; private whereConditions: string[] = []; @@ -32,6 +97,24 @@ export class QueryBuilder { private havingParams: any[] = []; private lockMode: LockMode | null = null; private eagerRelations: string[] = []; + /** + * Per-row post-processing applied to every result set this builder returns. + * The repository uses it to decrypt encrypted columns, so the transform has + * to run on all read paths rather than on a single hand-picked one. + */ + public rowTransform: ((rows: any[]) => any[]) | null = null; + /** + * Eager-loads {@link eagerRelations} onto a result set. + * + * Supplied by the repository, which owns the model metadata and the database + * access that loading a relation needs. Running it here — rather than in one + * repository method — means every builder a caller can reach honours + * `withRelations()`, and it runs before the result is cached so a cache hit + * carries the relations too. + */ + public relationLoader: + | ((rows: any[], relations: string[], client: DBClient) => Promise) + | null = null; private unions: { query: string; params: any[]; all: boolean }[] = []; private ctas: { name: string; @@ -53,7 +136,9 @@ export class QueryBuilder { selectRaw(expression: string, ...params: any[]): QueryBuilder { this.selectFields.push(expression); - this.whereParams.push(...params); + // Kept separate from `whereParams`: the SELECT list is emitted before the + // WHERE clause, so sharing one array would bind the values out of order. + this.selectParams.push(...params); return this; } @@ -62,6 +147,21 @@ export class QueryBuilder { return this; } + /** + * Declares the dialect this builder renders for, so dialect-specific clauses + * (today, only the row limit) are spelled correctly. + * + * `execute` sets this from the client it is given, so an ordinary + * `… .execute(db.client)` needs nothing. It is public for the cases where a + * builder is rendered without a client, and for a nested builder — one passed + * to `union` or `whereExists`, say — which is rendered before the outer + * builder ever sees a client. + */ + withDialect(dialect: DBType): QueryBuilder { + this.dialect = dialect; + return this; + } + as(alias: string): QueryBuilder { this.tableAlias = alias; return this; @@ -110,7 +210,14 @@ export class QueryBuilder { if (this.whereConditions.length === 0) { this.whereConditions.push(condition); } else { - this.whereConditions.push(`OR (${condition})`); + // Fold the conditions so far and this OR branch into a single group, so + // a later `where()` constrains the whole disjunction: + // (A OR B) AND C rather than A OR (B AND C) + // Without the group, SQL's AND precedence would drop the later filters + // from the first branch — including the soft-delete predicate. + this.whereConditions = [ + `(${this.whereConditions.join(" ")} OR (${condition}))`, + ]; } this.whereParams.push(...params); return this; @@ -259,7 +366,11 @@ export class QueryBuilder { /** Knex-style column-to-column comparison: .whereRef('orders.user_id', '=', 'users.id') */ whereRef(leftCol: string, op: string, rightCol: string): QueryBuilder { - this.whereConditions.push(`${leftCol} ${op} ${rightCol}`); + if (this.whereConditions.length > 0) { + this.whereConditions.push(`AND ${leftCol} ${op} ${rightCol}`); + } else { + this.whereConditions.push(`${leftCol} ${op} ${rightCol}`); + } return this; } @@ -302,7 +413,32 @@ export class QueryBuilder { // ─── ORDER BY ───────────────────────────────────────────────────── orderBy(column: string, direction: "ASC" | "DESC" = "ASC"): QueryBuilder { - this.orderByClauses.push(`${column} ${direction}`); + // Also accept the documented single-argument form, orderBy("createdAt DESC"), + // which would otherwise emit "ORDER BY createdAt DESC ASC". + const clause = column.trim(); + this.orderByClauses.push( + / (ASC|DESC)$/i.test(clause) ? clause : `${clause} ${direction}`, + ); + return this; + } + + /** + * Orders by an arbitrary SQL expression. + * + * `orderBy` only accepts a column name; an expression such as + * `CASE WHEN status = 'urgent' THEN 0 ELSE 1 END` has no column to name, and + * passing it to `orderBy` would have the direction appended to it. + * + * @param expression The SQL to order by, e.g. `"LENGTH(name)"`. + * @param direction Optional sort direction appended to the expression. + * @example + * ``` + * repo.find().orderByRaw("CASE WHEN status = 'urgent' THEN 0 ELSE 1 END") + * ``` + */ + orderByRaw(expression: string, direction?: "ASC" | "DESC"): QueryBuilder { + const clause = direction ? `${expression} ${direction}` : expression; + this.orderByClauses.push(clause); return this; } @@ -313,12 +449,44 @@ export class QueryBuilder { return this; } + /** + * Groups by an arbitrary SQL expression rather than a column name. + * @param expression The SQL to group by, e.g. `"strftime('%Y', createdAt)"`. + * @example + * ``` + * repo.find().groupByRaw("strftime('%Y-%m', createdAt)") + * ``` + */ + groupByRaw(expression: string): QueryBuilder { + this.groupByClauses.push(expression); + return this; + } + having(condition: string, ...params: any[]): QueryBuilder { - this.havingConditions.push(condition); + if (this.havingConditions.length > 0) { + this.havingConditions.push(`AND ${condition}`); + } else { + this.havingConditions.push(condition); + } this.havingParams.push(...params); return this; } + /** + * Adds a `HAVING` fragment that references an aggregate by its alias or + * position, which `having` cannot express without repeating the aggregate. + * + * Identical to `having` today; it exists so a caller reading `having("COUNT(*) + * > ?")` alongside `selectRaw` has the raw form spelled the same way as + * `whereRaw`, `orderByRaw` and `groupByRaw`. + * + * @param condition The raw SQL condition. + * @param params Values bound to its placeholders. + */ + havingRaw(condition: string, ...params: any[]): QueryBuilder { + return this.having(condition, ...params); + } + // ─── LIMIT / OFFSET (Prisma: take / skip) ───────────────────────── limit(limit: number): QueryBuilder { @@ -409,6 +577,44 @@ export class QueryBuilder { return this; } + // ─── EAGER RELATIONS ────────────────────────────────────────────── + + /** + * Eager-loads relations onto the result, as `findOne(id, { relations })` + * does. + * + * Nested paths use dot notation (`"roles.permissions"`). The builder records + * the paths and the repository loads them when `execute()` runs, so this + * composes with `where`, `limit` and `paginate`, and the loaded rows are + * cached with their relations. + * + * @param relations One or more relation paths. + * @example + * ``` + * const user = await userRepository + * .find() + * .where("isActive = ?", true) + * .withRelations("roles", "roles.permissions") + * .execute(db.client); + * ``` + */ + withRelations(...relations: (string | string[])[]): QueryBuilder { + for (const relation of relations.flat()) { + const path = relation?.trim(); + // A repeated path would load the same relation twice and overwrite the + // first result with an identical one. + if (path && !this.eagerRelations.includes(path)) { + this.eagerRelations.push(path); + } + } + return this; + } + + /** The relation paths requested via {@link withRelations}. */ + getRelations(): string[] { + return [...this.eagerRelations]; + } + // ─── SCOPE ──────────────────────────────────────────────────────── scope(name: string, ...args: any[]): QueryBuilder { @@ -433,16 +639,20 @@ export class QueryBuilder { q.selectFields = [...this.selectFields]; q.isDistinct = this.isDistinct; q.joins = [...this.joins]; + q.selectParams = [...this.selectParams]; q.whereConditions = [...this.whereConditions]; q.whereParams = [...this.whereParams]; q.orderByClauses = [...this.orderByClauses]; q.limitValue = this.limitValue; q.offsetValue = this.offsetValue; + q.dialect = this.dialect; q.groupByClauses = [...this.groupByClauses]; q.havingConditions = [...this.havingConditions]; q.havingParams = [...this.havingParams]; q.lockMode = this.lockMode; q.eagerRelations = [...this.eagerRelations]; + q.rowTransform = this.rowTransform; + q.relationLoader = this.relationLoader; q.unions = [...this.unions]; q.ctas = [...this.ctas]; return q; @@ -450,7 +660,14 @@ export class QueryBuilder { // ─── BUILD ──────────────────────────────────────────────────────── - build(): { query: string; params: any[] } { + /** + * Renders the statement and the values bound to its placeholders. + * + * @param dialect Overrides the dialect this builder renders for. Optional, so + * every existing call site renders exactly what it always did. + */ + build(dialect?: DBType): { query: string; params: any[] } { + const renderedFor = dialect ?? this.dialect ?? undefined; const params: any[] = []; let ctePrefix = ""; @@ -470,6 +687,10 @@ export class QueryBuilder { : this.table; const distinct = this.isDistinct ? " DISTINCT" : ""; + // The SELECT list is emitted before WHERE/GROUP BY/HAVING, so its params + // must be collected in that same order. + params.push(...this.selectParams); + let query = `${ctePrefix}SELECT${distinct} ${this.selectFields.join(", ")} FROM ${table}`; if (this.joins.length > 0) { @@ -494,12 +715,12 @@ export class QueryBuilder { query += "\nORDER BY " + this.orderByClauses.join(", "); } - if (this.limitValue !== null) { - query += `\nLIMIT ${this.limitValue}`; - } - if (this.offsetValue !== null) { - query += ` OFFSET ${this.offsetValue}`; - } + query += buildLimitClause( + this.limitValue, + this.offsetValue, + this.orderByClauses.length > 0, + renderedFor, + ); if (this.lockMode) { query += ` ${this.lockMode}`; @@ -513,8 +734,8 @@ export class QueryBuilder { return { query, params }; } - toSQL(): { query: string; params: any[] } { - return this.build(); + toSQL(dialect?: DBType): { query: string; params: any[] } { + return this.build(dialect); } // ─── EXECUTE ────────────────────────────────────────────────────── @@ -524,17 +745,36 @@ export class QueryBuilder { cache?: Cache, cacheKey?: string, ): Promise { + // Only the client knows which dialect this statement will be sent to, so + // the row-limiting clause is decided here rather than at build time. + this.dialect = client.config.type; const { query, params } = this.build(); - if (cache && cacheKey) { - const cached = await cache.get(cacheKey); + // Never cache a read taken inside an open transaction: the rows may be + // rolled back, and a cached copy would outlive the rollback. + const cacheable = + cache && cacheKey && !(client as any).isTransactionClient; + + if (cacheable) { + const cached = await cache.get(cacheKey!); if (cached) return cached; } - const results = await client.query(query, params); + let results = await client.query(query, params); + + // Transform before caching, so a cached row has the same shape a fresh + // read would produce. + if (this.rowTransform) results = this.rowTransform(results); - if (cache && cacheKey && results.length > 0) { - await cache.set(cacheKey, results, 60); + // Relations are loaded after the transform and before the cache write, so + // a cached result carries them — caching the pre-hydration rows made every + // later hit return a row with no relations on it. + if (this.relationLoader && this.eagerRelations.length > 0) { + results = await this.relationLoader(results, this.eagerRelations, client); + } + + if (cacheable && results.length > 0) { + await cache!.set(cacheKey!, results, cache!.config?.ttl ?? 60); } return results; @@ -542,19 +782,39 @@ export class QueryBuilder { async countExec(client: DBClient): Promise { const clone = this.clone(); - clone.selectFields = ["COUNT(*) AS __cnt"]; + clone.dialect = client.config.type; clone.orderByClauses = []; clone.limitValue = null; clone.offsetValue = null; - const results = await client.query( - clone.build().query, - clone.build().params, - ); + // Aggregate/limit clauses are meaningless in a COUNT and `FOR UPDATE` is + // rejected alongside aggregates by Postgres. + clone.lockMode = null; + + // GROUP BY/HAVING/UNION change what a row represents, so count the rows + // the query actually produces rather than the first group's count. + if (clone.groupByClauses.length > 0 || clone.unions.length > 0) { + const inner = clone.build(); + const results = await client.query( + `SELECT COUNT(*) AS __cnt FROM (${inner.query}) AS __cnt_sub`, + inner.params, + ); + return Number(results[0]?.__cnt ?? 0); + } + + // A join can multiply rows, so count distinct root rows. + clone.selectFields = [ + clone.joins.length > 0 + ? `COUNT(DISTINCT ${clone.tableAlias || clone.table}.id) AS __cnt` + : "COUNT(*) AS __cnt", + ]; + const built = clone.build(); + const results = await client.query(built.query, built.params); return Number(results[0]?.__cnt ?? 0); } async existsExec(client: DBClient): Promise { const clone = this.clone(); + clone.dialect = client.config.type; clone.selectFields = ["1"]; clone.orderByClauses = []; clone.limitValue = 1; diff --git a/repository.ts b/repository.ts index 227bf89..b0f8f90 100644 --- a/repository.ts +++ b/repository.ts @@ -21,6 +21,134 @@ import { decrypt, encrypt } from "./utils/encryption"; type VersionOperation = "insert" | "update" | "delete"; +/** + * How many key values go into one `IN (…)` when a relation is loaded. + * + * Relations are loaded with a single batched query per relation, so a + * `findMany` over ten thousand parents would otherwise build an `IN` list that + * long — past SQLite's default parameter limit, and past `max_allowed_packet` + * on MySQL. + */ +const RELATION_BATCH_SIZE = 500; + +/** + * Coerces a value into what the dialect's driver will accept as a parameter. + * + * A `Date` is the interesting case. SQLite rejects a raw `Date` at bind time, + * so leaving one in place fails outright; PostgreSQL's `TIMESTAMP` accepts ISO, + * and SQL Server's `DATETIME2` parses it. MySQL and MariaDB do not — their + * `DATETIME` under `STRICT_TRANS_TABLES` (the default on both) refuses the + * `T` and the `Z` of an ISO string with "Incorrect datetime value", so every + * write on those servers would fail. The space-separated, second-resolution + * spelling is what the whole MySQL family takes, and a `DATETIME` column is + * second-resolution by default anyway, so nothing is lost in the slice. + * + * Booleans become 1/0 because none of the dialects the library supports has a + * real boolean parameter type, and anything else that is not directly bindable + * (an object, a function) becomes NULL rather than reaching the driver and + * failing the statement. + * + * @param val The value to coerce. + * @param dbType The dialect the value is bound for. + * @returns A value the driver can bind. + */ +function sanitizeSqlValue( + val: any, + dbType: DBType, +): string | number | boolean | bigint | null { + if (val === undefined) return null; + if (val instanceof Date) { + if (dbType === DBType.MySQL) { + return val.toISOString().slice(0, 19).replace("T", " "); + } + return val.toISOString(); + } + if (typeof val === "boolean") return val ? 1 : 0; + if ( + typeof val === "string" || + typeof val === "number" || + typeof val === "bigint" + ) + return val; + return null; +} + +/** Splits values into `size`-long chunks. */ +function chunked(values: V[], size: number): V[][] { + if (values.length <= size) return [values]; + const chunks: V[][] = []; + for (let i = 0; i < values.length; i += size) { + chunks.push(values.slice(i, i + size)); + } + return chunks; +} + +/** + * The distinct, non-null values of `key` across `rows`, preserving order. + * + * Used to collect the keys a relation lookup should match on. Null keys are + * dropped: a parent with no foreign key has no related row, and `IN (NULL)` + * never matches anything anyway. + */ +function distinctKeys(rows: Record[], key: string): any[] { + const seen = new Set(); + for (const row of rows) { + const value = row?.[key]; + if (value === null || value === undefined) continue; + seen.add(value); + } + return [...seen]; +} + +/** + * Builds the T-SQL statement an upsert has to use. + * + * SQL Server supports neither `ON CONFLICT … DO UPDATE` nor + * `ON DUPLICATE KEY UPDATE`; the equivalent is `MERGE`. The payload is offered + * as a one-row `source` table whose columns are the bound parameters, and the + * `WHEN MATCHED` / `WHEN NOT MATCHED` branches then refer to those columns by + * name — so every value is bound exactly once, on the `USING` line, however + * many times the column is referenced afterwards. + * + * `OUTPUT INSERTED.*` is the T-SQL analogue of `RETURNING *`, and tells the + * caller which row the statement produced on either branch. + * + * Pure, and exported, so the rendered statement can be asserted without a + * server. + * + * @param table The table to merge into. + * @param columnNames The columns being written, in binding order. + * @param keyColumns The columns that decide whether a row already exists. + * @returns The `MERGE` statement, whose placeholders number one per column. + */ +export function buildMSSQLUpsertSQL( + table: string, + columnNames: string[], + keyColumns: string[], +): string { + if (keyColumns.length === 0) { + // With nothing to match on there is no row a `MERGE` could resolve + // against, so the statement can only ever insert. + return `INSERT INTO ${table} (${columnNames.join(", ")}) OUTPUT INSERTED.* VALUES (${columnNames.map(() => "?").join(", ")})`; + } + + const source = `SELECT ${columnNames.map((c) => `? AS ${c}`).join(", ")}`; + const on = keyColumns.map((c) => `target.${c} = source.${c}`).join(" AND "); + const updates = columnNames.filter((c) => !keyColumns.includes(c)); + const matched = + updates.length === 0 + ? "" + : `\nWHEN MATCHED THEN UPDATE SET ${updates + .map((c) => `target.${c} = source.${c}`) + .join(", ")}`; + const notMatched = `\nWHEN NOT MATCHED THEN INSERT (${columnNames.join( + ", ", + )}) VALUES (${columnNames.map((c) => `source.${c}`).join(", ")})`; + + // A `MERGE` statement has to be terminated by a semicolon. + return `MERGE INTO ${table} AS target\nUSING (${source}) AS source\nON (${on})${matched}${notMatched}\nOUTPUT INSERTED.*;`; +} + export class Repository { private client: DBClient; private cache: Cache | null; @@ -30,11 +158,15 @@ export class Repository { { name: string; type: string; + required?: boolean; + unique?: boolean; minLength?: number; maxLength?: number; pattern?: RegExp; customValidator?: (val: any) => boolean | string; encrypted?: boolean; + softDelete?: boolean; + optimisticLock?: boolean; } >; private validators: Record; @@ -54,6 +186,7 @@ export class Repository { private historyTable: string; private model: new (...args: any[]) => T; private optimisticLockField: string | null; + private autoIncrementField: string | null; private timestampsConfig: { createdAt?: string; updatedAt?: string } | null; constructor( @@ -61,9 +194,16 @@ export class Repository { model: new (...args: any[]) => T, cacheConfig: CacheConfig = { enabled: false, ttl: 60 }, logger: Logger = new StabilizeLogger(), + sharedCache?: Cache | null, ) { this.client = client; - this.cache = cacheConfig.enabled ? new Cache(cacheConfig, logger) : null; + // A caller that already owns a cache passes it in. Otherwise every + // repository built one of its own — a second Redis connection that nothing + // ever disconnected and that `getCacheStats()` never looked at, because it + // reports on the ORM's own cache instance. + this.cache = + sharedCache ?? + (cacheConfig.enabled ? new Cache(cacheConfig, logger) : null); this.table = MetadataStorage.getTableName(model); this.columns = Object.fromEntries( Object.entries(MetadataStorage.getColumns(model)).map(([key, col]) => [ @@ -71,6 +211,15 @@ export class Repository { { name: col.name ?? key, type: typeof col.type === "string" ? col.type : DataTypes[col.type], + required: col.required, + unique: col.unique, + minLength: col.minLength, + maxLength: col.maxLength, + pattern: col.pattern, + customValidator: col.customValidator, + encrypted: col.encrypted, + softDelete: col.softDelete, + optimisticLock: col.optimisticLock, }, ]), ); @@ -89,6 +238,7 @@ export class Repository { this.validators = MetadataStorage.getValidators(model); this.softDeleteField = MetadataStorage.getSoftDeleteField(model); this.optimisticLockField = this.getOptimisticLockField(model); + this.autoIncrementField = this.getAutoIncrementField(); this.timestampsConfig = MetadataStorage.getTimestamps(model); this.logger = logger; this.versioned = MetadataStorage.isVersioned(model); @@ -96,45 +246,167 @@ export class Repository { this.model = model; } + /** + * Builds the cache key under which a single row is stored. + * + * A `findOne` that eager-loads relations returns a different shape from one + * that does not, so the relation set is part of the key. The set is sorted so + * that the same relations requested in a different order share one entry, and + * the no-relations case gets the bare key rather than a `:undefined` suffix — + * the suffix that previously made every write-through `set` and every + * `invalidate` address a key that no read ever looked up. + */ + private rowCacheKey( + id: number | string, + relations?: string[], + ): string { + const base = `findOne:${this.table}:${id}`; + if (!relations || relations.length === 0) return base; + return `${base}:${[...relations].sort().join(",")}`; + } + + /** + * Invalidates every cached copy of a row: the bare key plus each + * relation-loaded variant, which differ only by a suffix. + */ + private async invalidateRowCache(id: number | string): Promise { + if (!this.cache) return; + const base = `findOne:${this.table}:${id}`; + await this.cache.invalidate([base, `find:${this.table}`]); + await this.cache.invalidatePattern(`${base}:*`); + // List/lookup queries for this table may embed the row too. + await this.cache.invalidatePattern(`find:${this.table}:*`); + } + + /** + * Clears every cached entry for this table. + * + * Multi-row writes (bulk update/delete, `deleteWhere`, `truncate`) cannot + * enumerate the affected ids, so the per-row entries have to go too — + * otherwise a bulk update leaves `findOne` serving the pre-update row. + */ + private async invalidateTableCache(): Promise { + if (!this.cache) return; + await this.cache.invalidate([`find:${this.table}`]); + await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.cache.invalidatePattern(`findOne:${this.table}:*`); + } + + /** Writes a row through to the cache (no-op unless strategy is write-through). */ + private async writeThroughRow(id: number | string, row: any): Promise { + if (!this.cache) return; + if (this.cache.getStrategy() !== "write-through") return; + // Cache the plain shape: a write-through row was not loaded with relations, + // so it must not occupy a relation variant's key. + await this.cache.set(this.rowCacheKey(id), [row], 60); + } + private getOptimisticLockField(model: Function): string | null { - const columns = MetadataStorage.getColumns(model); - for (const [key, col] of Object.entries(columns)) { - if ((col as any).optimisticLock) return key; + for (const [key, col] of Object.entries( + MetadataStorage.getColumns(model), + )) { + if (col.optimisticLock) return key; } return null; } + /** + * Finds the auto-incrementing primary key, if the model has one. + * + * The database generates this value, so `create()` must not require it from + * the caller. A string/UUID primary key is supplied by the caller and stays + * required. Mirrors the primary-key handling in `generateMigration`. + */ + private getAutoIncrementField(): string | null { + const idColumn = this.columns["id"]; + if (!idColumn) return null; + const type = String(idColumn.type).toUpperCase(); + if (type === "STRING" || type === "TEXT" || type === "UUID") return null; + return "id"; + } + + /** + * The SQL column backing the soft-delete property. + * + * A column may carry a `name` that differs from its property key, and rows + * read back from the database are keyed by SQL name. Interpolating the + * property key into a query therefore fails with "no such column" whenever + * the column is renamed. + */ + private get softDeleteColumn(): string | null { + return this.softDeleteField + ? (this.columns[this.softDeleteField]?.name ?? this.softDeleteField) + : null; + } + + /** True when any column of this model is encrypted. */ + private get hasEncryptedColumns(): boolean { + return Object.values(this.columns).some((col) => col.encrypted); + } + + /** The SQL column backing the optimistic-lock property. @see softDeleteColumn */ + private get optimisticLockColumn(): string | null { + return this.optimisticLockField + ? (this.columns[this.optimisticLockField]?.name ?? + this.optimisticLockField) + : null; + } + private getDBType(_client?: DBClient): DBType { const client = _client || this.client; return client.config.type; } - private validate(entity: Partial, skipRequired: boolean = false) { - for (const [key, rules] of Object.entries(this.validators)) { + /** + * The trailing clause that keeps a statement to a single row. + * + * T-SQL has no `LIMIT`. The history queries below always order their rows, so + * the `OFFSET … FETCH NEXT` pair — which requires an `ORDER BY` — is enough, + * with no need for the constant `ORDER BY (SELECT NULL)` a bare limit needs. + */ + private topOneClause(client: DBClient): string { + return this.getDBType(client) === DBType.MSSQL + ? " OFFSET 0 ROWS FETCH NEXT 1 ROWS ONLY" + : " LIMIT 1"; + } + + /** + * Collects every validation failure in `entity`, at most one per column. + * + * Shared by {@link validate}, which needs only the first failure, and + * {@link validateAll}, which needs all of them. + */ + private collectValidationErrors( + entity: Partial, + skipRequired: boolean = false, + ): string[] { + const errors: string[] = []; + // Iterate the column map rather than the validator rules: every column is + // guaranteed to be present here, so a column carrying only a length, + // pattern or custom validator is still checked. + for (const [key, column] of Object.entries(this.columns)) { const value = (entity as any)[key]; + const rules = this.validators[key] ?? []; if ( !skipRequired && - rules.includes("required") && + key !== this.autoIncrementField && + (column.required || rules.includes("required")) && (value === undefined || value === null) ) { - throw new StabilizeError( - `Field ${key} is required`, - "VALIDATION_ERROR", - ); + errors.push(`Field ${key} is required`); + continue; } if (value === undefined || value === null) continue; - const column = this.columns?.[key]; - if (!column) continue; - if ( column.minLength && typeof value === "string" && value.length < column.minLength ) { - throw new StabilizeError(`Field ${key} too short`, "VALIDATION_ERROR"); + errors.push(`Field ${key} too short`); + continue; } if ( @@ -142,7 +414,8 @@ export class Repository { typeof value === "string" && value.length > column.maxLength ) { - throw new StabilizeError(`Field ${key} too long`, "VALIDATION_ERROR"); + errors.push(`Field ${key} too long`); + continue; } if ( @@ -150,33 +423,138 @@ export class Repository { typeof value === "string" && !column.pattern.test(value) ) { - throw new StabilizeError( - `Field ${key} does not match pattern`, - "VALIDATION_ERROR", - ); + errors.push(`Field ${key} does not match pattern`); + continue; } if (typeof column.customValidator === "function") { const result = column.customValidator(value); if (result !== true) { - throw new StabilizeError(result as string, "VALIDATION_ERROR"); + errors.push(result as string); } } } + return errors; + } + + private validate(entity: Partial, skipRequired: boolean = false) { + const [first] = this.collectValidationErrors(entity, skipRequired); + if (first) { + // The first failure only: a write has nothing to do with the rest. + throw new StabilizeError(first, "VALIDATION_ERROR"); + } + } + + /** + * Validates an entity against the model's column rules and returns every + * failure, rather than throwing on the first one. + * + * `validate` stops at the first problem, which is what a write wants, but a + * caller validating a form wants the whole list instead of discovering one + * bad field per round trip. + * + * @param entity The values to check. + * @param skipRequired Skip the `required` rules, as an update does. + * @returns One message per invalid column; empty when the entity is valid. + * @example + * ``` + * const errors = repo.validateAll({ email: "nope" }); + * if (errors.length) res.status(422).json({ errors }); + * ``` + */ + validateAll(entity: Partial, skipRequired: boolean = false): string[] { + return this.collectValidationErrors(entity, skipRequired); } private async runHooks(entity: any, type: HookType): Promise { - for (const hook of getHooks(entity, type)) { + // The model is passed explicitly rather than inferred from the entity: + // `after*` hooks are handed a row read back from the database, whose + // prototype is `Object.prototype`, so the metadata lookup failed and the + // hooks never ran. + for (const hook of getHooks(entity, type, this.model)) { await hook.callback(entity); } } + /** + * Wraps a row read from the database in a model instance. + * + * Hooks receive entities, and class-method hooks (`async afterCreate() {}`) + * only resolve on a real instance, so a plain row is not enough. + */ + private hydrate(row: R | null): R { + if (row === null || row === undefined) return row as R; + return Object.assign(new this.model() as any, row) as R; + } + + /** + * Merges a hook-mutated instance back into the payload that will be written. + * + * A key is carried over when the caller supplied it, when a hook introduced + * it, or when a hook changed it. A key the hook left alone is skipped: it + * came from the existing row, which was decrypted on read, so writing it + * back would store plaintext in an encrypted column. + */ + private mergeHookOutput>( + payload: R, + instance: any, + previous: Record | null, + ): R { + const merged: Record = { ...payload }; + for (const key of Object.keys(instance)) { + if (!this.columns[key]) continue; + if (key in payload) { + merged[key] = instance[key]; + continue; + } + if (previous && key in previous && instance[key] === previous[key]) { + continue; + } + merged[key] = instance[key]; + } + return merged as R; + } + + /** The model's primary-key property. */ + private get primaryKeyField(): string { + return "id"; + } + + /** The SQL column backing the primary key. @see softDeleteColumn */ + private get primaryKeyColumn(): string { + return this.columns[this.primaryKeyField]?.name ?? this.primaryKeyField; + } + + /** + * Attaches the decrypting row transform to a builder owned by this + * repository. + * + * Applied wherever a builder is created rather than inside one hand-picked + * method, so `findBy`, `first`, `paginate`, `pluck`, `findDeleted` and the + * rest all return plaintext instead of ciphertext. + */ + private withRowTransform(qb: QueryBuilder): QueryBuilder { + if (this.hasEncryptedColumns) { + qb.rowTransform = (rows) => rows.map((row) => this.processForLoad(row)); + } + return qb; + } + find(): QueryBuilder { const qb = new QueryBuilder(this.table); + // `execute` stamps the dialect too, but a builder handed to `union` or + // `whereExists` is rendered while it is being attached — before any client + // is in sight — so it has to know the dialect from the outset. + qb.withDialect(this.client.config.type); if (this.softDeleteField) { - qb.where(`${this.table}.${this.softDeleteField} IS NULL`); + qb.where(`${this.table}.${this.softDeleteColumn} IS NULL`); } - return qb; + // Lets `find().withRelations(...)` work: the builder records the paths and + // calls back here to load them, since only the repository has the model + // metadata and the relation queries. + qb.relationLoader = (rows, relations, client) => + this.loadRelations(rows, relations, client); + return this.withRowTransform(qb); } scope(name: string, ...args: any[]): QueryBuilder { @@ -192,21 +570,16 @@ export class Repository { const client = _client || this.client; const start = performance.now(); this.logger.logDebug(`Finding one ${this.table} with ID ${id}`); - const queryBuilder = this.find().where(`${this.table}.id = ?`, id).limit(1); - if (options.relations) { - for (const rel of options.relations) { - if (rel.includes(".")) { - await this.loadNestedRelations(queryBuilder, rel); - } else { - await this.loadRelation(queryBuilder, rel); - } - } - // Use qualified SELECT to avoid ambiguous columns with joins - queryBuilder.select(`${this.table}.*`); - } - const cacheKey = `findOne:${this.table}:${id}:${options.relations?.join(",")}`; - const results = await queryBuilder.execute(client, this.cache!, cacheKey); - const result = this.processForLoad(results); + const cacheKey = this.rowCacheKey(id, options.relations); + // `execute` applies the row transform from `find()` (so the rows arrive + // decrypted) and calls back into `loadRelations` before writing the cache, + // so a cached entry holds the relations a later hit is asked for. + const result = await this.find() + .where(`${this.table}.id = ?`, id) + .limit(1) + .withRelations(options.relations ?? []) + .execute(client, this.cache!, cacheKey); + this.logger.logDebug( `Found ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, ); @@ -221,9 +594,13 @@ export class Repository { if (!this.versioned) throw new StabilizeError("Model is not versioned", "VERSIONING_ERROR"); const client = _client || this.client; + // Bind an ISO string rather than the `Date` itself: SQLite rejects a Date + // as a parameter, so every `asOf` call used to throw there, and the + // history timestamps are written as ISO strings anyway. + const at = asOfDate instanceof Date ? asOfDate.toISOString() : asOfDate; const rows = await client.query( - `SELECT * FROM ${this.historyTable} WHERE id = ? AND valid_from <= ? AND (valid_to IS NULL OR valid_to > ?) ORDER BY version DESC LIMIT 1`, - [id, asOfDate, asOfDate], + `SELECT * FROM ${this.historyTable} WHERE id = ? AND valid_from <= ? AND (valid_to IS NULL OR valid_to > ?) ORDER BY version DESC${this.topOneClause(client)}`, + [id, at, at], ); return rows[0] || null; } @@ -248,7 +625,7 @@ export class Repository { const client = _client || this.client; return client.transaction(async (txClient) => { const rows = await txClient.query( - `SELECT * FROM ${this.historyTable} WHERE id = ? AND version = ? LIMIT 1`, + `SELECT * FROM ${this.historyTable} WHERE id = ? AND version = ?${this.topOneClause(txClient)}`, [id, version], ); if (!rows.length) @@ -295,26 +672,6 @@ export class Repository { "modified_at", ]; - function sanitizeSqlValue( - val: any, - dbType: DBType, - ): string | number | boolean | bigint | null { - if (val === undefined) return null; - if (val instanceof Date) { - if (dbType === DBType.MySQL) { - return val.toISOString().slice(0, 19).replace("T", " "); - } - return val.toISOString(); - } - if (typeof val === "boolean") return val ? 1 : 0; - if ( - typeof val === "string" || - typeof val === "number" || - typeof val === "bigint" - ) - return val; - return null; - } const dbType = client.config.type; const values = propertyKeys.map((k) => sanitizeSqlValue(entity?.[k], dbType), @@ -345,21 +702,30 @@ export class Repository { async create( entity: Partial, options: { relations?: string[] } = {}, + _client?: DBClient, ): Promise { - return this.client.transaction(async (txClient) => { + return (_client || this.client).transaction(async (txClient) => { const instance = new this.model() as T; Object.assign(instance as object, entity); await this.runHooks(instance, "beforeCreate"); await this.runHooks(instance, "beforeSave"); - const result = await this._create(entity, options, txClient); + // Insert the hook-mutated instance rather than the argument. A + // `beforeCreate` that fills in a slug or normalises a field had its work + // discarded, because the original payload was what got written. + const result = await this._create(instance as Partial, options, txClient); + + const hydrated = this.hydrate(result); + await this.runHooks(hydrated, "afterCreate"); + await this.runHooks(hydrated, "afterSave"); - await this.runHooks(result, "afterCreate"); - await this.runHooks(result, "afterSave"); + await this.writeHistory(hydrated, "insert", txClient); - await this.writeHistory(result, "insert", txClient); - return result; + // The option was accepted and then ignored, so a caller asking for a + // relation got the bare row back with no error to explain it. + await this.loadRelations([hydrated], options.relations, txClient); + return hydrated; }); } @@ -373,16 +739,29 @@ export class Repository { `Creating ${this.table} with data: ${JSON.stringify(entity)}`, ); this.validate(entity); - const entityToSave = this.processForSave(entity); const timestamps = this.timestampsConfig; - const entityWithTimestamps = { ...entityToSave } as Record; + const entityWithTimestamps = { ...entity } as Record; if (timestamps?.createdAt && !entityWithTimestamps[timestamps.createdAt]) { - entityWithTimestamps[timestamps.createdAt] = new Date(); + entityWithTimestamps[timestamps.createdAt] = new Date().toISOString(); } if (timestamps?.updatedAt && !entityWithTimestamps[timestamps.updatedAt]) { - entityWithTimestamps[timestamps.updatedAt] = new Date(); + entityWithTimestamps[timestamps.updatedAt] = new Date().toISOString(); } + // Seed the optimistic lock so the first `update()` has a version to match + // on; without this the column stays NULL and `version = NULL` never + // matches, which makes every update look like a conflict. + if ( + this.optimisticLockField && + entityWithTimestamps[this.optimisticLockField] === undefined + ) { + entityWithTimestamps[this.optimisticLockField] = 1; + } + + // Encrypt after the timestamp and lock columns are in place, so everything + // bound below passes through the same coercion. + const entityToSave = this.processForSave(entityWithTimestamps); + Object.assign(entityWithTimestamps, entityToSave); const keys = Object.keys(entityWithTimestamps).filter( (k) => this.columns[k], @@ -390,15 +769,30 @@ export class Repository { const columnNames = keys.map((k) => this.columns[k]?.name).join(", "); const placeholders = keys.map(() => "?").join(", "); const params = keys.map((k) => (entityWithTimestamps as any)[k]); - let query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders})`; - - let insertedResult: T[] | undefined; - let id: number | string | undefined; const dbType = this.getDBType(client); + // `OUTPUT INSERTED.*` is the T-SQL spelling of `RETURNING *`, but the two + // do not sit in the same place: `RETURNING` trails the statement, whereas + // T-SQL wants `OUTPUT` between the column list and `VALUES`. Appending it + // is a syntax error ("Incorrect syntax near 'OUTPUT'"). + let query = `INSERT INTO ${this.table} (${columnNames})`; + if (dbType === DBType.MSSQL) { + query += " OUTPUT INSERTED.*"; + } + query += ` VALUES (${placeholders})`; if (dbType === DBType.Postgres) { query += " RETURNING *"; - insertedResult = await client.query(query, params); + } + + let insertedResult: T[] | undefined; + let id: number | string | undefined; + + if (dbType === DBType.Postgres || dbType === DBType.MSSQL) { + // Both clauses hand back raw column values, so encrypted columns need + // the same decoding a read would apply. + insertedResult = (await client.query(query, params)).map((row) => + this.processForLoad(row), + ); id = (insertedResult?.[0] as any)?.id; } else { await client.query(query, params); @@ -427,13 +821,8 @@ export class Repository { const result = insertedResult?.[0] ?? ((await this.findOne(id, options, client)) as T); - if (this.cache) { - const cacheKeys = [`find:${this.table}`, `findOne:${this.table}:${id}`]; - await this.cache.invalidate(cacheKeys); - if (this.cache.getStrategy() === "write-through") { - await this.cache.set(`findOne:${this.table}:${id}`, [result], 60); - } - } + await this.invalidateRowCache(id); + await this.writeThroughRow(id, result); this.logger.logDebug( `Created ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, @@ -444,8 +833,9 @@ export class Repository { async bulkCreate( entities: Partial[], options: { relations?: string[]; batchSize?: number } = {}, + _client?: DBClient, ): Promise { - return this.client.transaction(async (txClient) => { + return (_client || this.client).transaction(async (txClient) => { const preparedEntities = entities.map((data) => { const instance = new (this as any).model(); Object.assign(instance, data); @@ -457,15 +847,26 @@ export class Repository { await this.runHooks(entity, "beforeSave"); } - const results = await this._bulkCreate(entities, options, txClient); + const results = await this._bulkCreate( + preparedEntities as Partial[], + options, + txClient, + ); - for (const result of results) { - await this.runHooks(result, "afterCreate"); - await this.runHooks(result, "afterSave"); + for (let i = 0; i < results.length; i++) { + // Hydrated so `after*` hooks and class-method hooks resolve, and so a + // hook's mutation is visible on what `bulkCreate` returns. + const hydrated = this.hydrate(results[i]!); + await this.runHooks(hydrated, "afterCreate"); + await this.runHooks(hydrated, "afterSave"); if (this.versioned) { - await this.writeHistory(result, "insert", txClient); + await this.writeHistory(hydrated, "insert", txClient); } + results[i] = hydrated; } + + // Loaded once for the whole batch rather than per row. @see create + await this.loadRelations(results, options.relations, txClient); return results; }); } @@ -485,29 +886,45 @@ export class Repository { entities.forEach((entity) => this.validate(entity)); const timestamps = this.timestampsConfig; - const entitiesWithTimestamps = entities.map((entity) => ({ - ...entity, - ...(timestamps?.createdAt && - !(entity as Record)[timestamps.createdAt] - ? { [timestamps.createdAt]: new Date() } - : {}), - ...(timestamps?.updatedAt && - !(entity as Record)[timestamps.updatedAt] - ? { [timestamps.updatedAt]: new Date() } - : {}), - })) as Partial[]; + const prepared = entities.map((entity) => { + const row = { ...entity } as Record; + if (timestamps?.createdAt && !row[timestamps.createdAt]) { + row[timestamps.createdAt] = new Date().toISOString(); + } + if (timestamps?.updatedAt && !row[timestamps.updatedAt]) { + row[timestamps.updatedAt] = new Date().toISOString(); + } + if ( + this.optimisticLockField && + row[this.optimisticLockField] === undefined + ) { + row[this.optimisticLockField] = 1; + } + // The same coercion `create()` applies: encrypt encrypted columns and + // normalise Dates. Without it a bulk insert wrote plaintext into an + // encrypted column and bound a raw `Date`, which SQLite rejects. + return this.processForSave(row) as Partial; + }); const dbType = this.getDBType(client); + const pkField = this.primaryKeyField; + const pkColumn = this.primaryKeyColumn; const results: T[] = []; - for (let i = 0; i < entitiesWithTimestamps.length; i += batchSize) { - const batch = entitiesWithTimestamps.slice(i, i + batchSize); - const keys = Object.keys(batch[0]!).filter((k) => this.columns[k]); - const columnNames = keys.map((k) => this.columns[k]?.name).join(", "); - - let query: string; - let params: any[] = batch.flatMap((entity) => - keys.map((k) => (entity as any)[k]), + for (let i = 0; i < prepared.length; i += batchSize) { + const batch = prepared.slice(i, i + batchSize); + // Union the keys across the batch. Taking them from `batch[0]` alone + // silently dropped every column the first row happened not to carry, + // so a caller passing a mix of shapes lost data on the wider rows. + const keys: string[] = []; + for (const row of batch) { + for (const key of Object.keys(row)) { + if (this.columns[key] && !keys.includes(key)) keys.push(key); + } + } + const columnNames = keys.map((k) => this.columns[k]?.name ?? k).join(", "); + const params: any[] = batch.flatMap((row) => + keys.map((k) => (row as any)[k] ?? null), ); if (dbType === DBType.Postgres) { @@ -515,38 +932,55 @@ export class Repository { const valuePlaceholders = batch .map(() => `(${keys.map(() => `$${paramIdx++}`).join(", ")})`) .join(", "); - query = `INSERT INTO ${this.table} (${columnNames}) VALUES ${valuePlaceholders} RETURNING *`; + const query = `INSERT INTO ${this.table} (${columnNames}) VALUES ${valuePlaceholders} RETURNING *`; const batchResults = await client.query(query, params); - results.push(...batchResults); - } else { - const placeholders = `(${keys.map(() => "?").join(", ")})`; - query = `INSERT INTO ${this.table} (${columnNames}) VALUES ${batch.map(() => placeholders).join(", ")}`; - await client.query(query, params); - const ids = ( - await client.query<{ id: number }>( - `SELECT id FROM ${this.table} ORDER BY id DESC LIMIT ?`, - [batch.length], - ) - ).map((row) => row.id); - - let batchResults: T[] = []; - if (ids.length > 0) { - const queryBuilder = this.find().where( - `id IN (${ids.map(() => "?").join(", ")})`, - ...ids, - ); - if (options.relations) { - for (const rel of options.relations) { - await this.loadRelation(queryBuilder, rel); - } - } - batchResults = await queryBuilder.execute(client); - } - results.push(...batchResults); + // `RETURNING` hands back raw values, so encrypted columns need the + // same decoding a read applies. + results.push(...batchResults.map((row) => this.processForLoad(row))); + continue; + } + + const placeholders = `(${keys.map(() => "?").join(", ")})`; + const valuesClause = batch.map(() => placeholders).join(", "); + + if (dbType === DBType.MSSQL) { + // As in `_create`, `OUTPUT INSERTED.*` replaces the read-back the other + // dialects need, so the new keys are known without asking the server + // for an identity value afterwards. + const query = `INSERT INTO ${this.table} (${columnNames}) OUTPUT INSERTED.* VALUES ${valuesClause}`; + const batchResults = await client.query(query, params); + results.push(...batchResults.map((row) => this.processForLoad(row))); + continue; + } + + const query = `INSERT INTO ${this.table} (${columnNames}) VALUES ${valuesClause}`; + await client.query(query, params); + + const ids = await this.resolveInsertedIds( + batch, + client, + dbType, + pkField, + pkColumn, + ); + if (ids.length === 0) continue; + + const queryBuilder = this.find().where( + `${this.table}.${pkColumn} IN (${ids.map(() => "?").join(", ")})`, + ...ids, + ); + const fetched = await queryBuilder.execute(client); + await this.loadRelations(fetched, options.relations, client); + // The `IN` query returns rows in whatever order the planner picked, so + // re-order them to match the caller's input. + const byKey = new Map(fetched.map((row: any) => [row[pkColumn], row])); + for (const id of ids) { + const row = byKey.get(id); + if (row) results.push(row as T); } } - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); this.logger.logDebug( `Bulk created ${results.length} ${this.table} entities in ${(performance.now() - start).toFixed(2)}ms`, @@ -554,8 +988,61 @@ export class Repository { return results; } - async update(id: number | string, entity: Partial): Promise { - return this.client.transaction(async (txClient) => { + /** + * Works out the primary keys a single multi-row INSERT just created. + * + * These used to be guessed with `SELECT id FROM table ORDER BY id DESC LIMIT + * n`, which returns whatever rows happen to be newest — including rows a + * concurrent writer inserted in between — and cannot work at all when the + * primary key is a UUID rather than a counter. + */ + private async resolveInsertedIds( + batch: Partial[], + client: DBClient, + dbType: DBType, + pkField: string, + pkColumn: string, + ): Promise { + // A caller-supplied key (UUID, string id) is already known. + const explicit = batch.map( + (row) => (row as any)[pkField] ?? (row as any)[pkColumn], + ); + if (explicit.every((value) => value !== undefined && value !== null)) { + return explicit; + } + + if (!this.autoIncrementField) return []; + + // SQLite reports the rowid of the LAST row a multi-row INSERT assigned, + // MySQL the FIRST. Either way the batch occupies one contiguous run. + // SQL Server needs no probe of its own: every MSSQL insert path in this + // repository returns its rows through an `OUTPUT INSERTED.*` clause, so + // there is nothing left to look up here. + if (dbType === DBType.MSSQL) return []; + + const reported = + dbType === DBType.SQLite + ? ( + await client.query<{ id: number }>( + "SELECT last_insert_rowid() as id", + ) + )[0]?.id + : ( + await client.query("SELECT LAST_INSERT_ID() as id") + )[0]?.["LAST_INSERT_ID()"]; + + if (reported === undefined || reported === null) return []; + + const first = dbType === DBType.SQLite ? reported - batch.length + 1 : reported; + return Array.from({ length: batch.length }, (_, n) => first + n); + } + + async update( + id: number | string, + entity: Partial, + _client?: DBClient, + ): Promise { + return (_client || this.client).transaction(async (txClient) => { const before = await this.findOne(id, {}, txClient); if (!before) throw new StabilizeError("Not found", "UPDATE_ERROR"); const instance = new this.model() as T; @@ -564,21 +1051,29 @@ export class Repository { await this.runHooks(instance, "beforeUpdate"); await this.runHooks(instance, "beforeSave"); - const result = await this._update(id, entity, before, txClient); + // Write the hook-mutated values, not the raw argument, so a + // `beforeUpdate` that normalises a field is not thrown away. + const result = await this._update( + id, + this.mergeHookOutput({ ...entity }, instance, before as any), + before, + txClient, + ); - await this.runHooks(result, "afterUpdate"); - await this.runHooks(result, "afterSave"); + // Hydrated so `after*` hooks and class-method hooks resolve. + const hydrated = this.hydrate(result); + await this.runHooks(hydrated, "afterUpdate"); + await this.runHooks(hydrated, "afterSave"); await this.writeHistory( { - ...before, - ...entity, + ...hydrated, version: (before as any).version ? (before as any).version + 1 : 1, }, "update", txClient, ); - return result; + return hydrated; }); } @@ -595,7 +1090,44 @@ export class Repository { const timestamps = this.timestampsConfig; const entityWithTimestamps = { ...entity } as Record; if (timestamps?.updatedAt && !entityWithTimestamps[timestamps.updatedAt]) { - entityWithTimestamps[timestamps.updatedAt] = new Date(); + entityWithTimestamps[timestamps.updatedAt] = new Date().toISOString(); + } + + // Encrypt and coerce the payload. `processForSave` used to run only on + // create, so an update wrote plaintext into an encrypted column and bound + // raw `Date` objects that SQLite rejects. + Object.assign(entityWithTimestamps, this.processForSave(entityWithTimestamps)); + + // Advance the optimistic lock as part of the same UPDATE. This has to + // happen before the column list is derived, otherwise the version column + // is left out of the SET clause and never actually changes. + let lockValue: any; + let lockIsNull = false; + if (this.optimisticLockField) { + // Rows read back from the database are keyed by SQL column name, so a + // renamed column must be looked up under its `name` too — otherwise the + // value is undefined and the lock silently does nothing. + lockValue = + (before as any)[this.optimisticLockField] ?? + (this.optimisticLockColumn + ? (before as any)[this.optimisticLockColumn] + : undefined); + + // A caller that passes the version it read expects a conflict if someone + // else has written since. Guard on the caller's value, not the one just + // SELECTed inside this transaction, which would always match. + const callerVersion = (entity as any)[this.optimisticLockField]; + const expected = + callerVersion !== undefined && callerVersion !== null + ? callerVersion + : lockValue; + + if (expected !== undefined) { + lockValue = expected; + lockIsNull = expected === null; + entityWithTimestamps[this.optimisticLockField] = + typeof expected === "number" ? expected + 1 : 1; + } } const keys = Object.keys(entityWithTimestamps).filter( @@ -606,36 +1138,28 @@ export class Repository { .join(", "); const whereParts: string[] = ["id = ?"]; - const queryParams = [...keys.map((k) => (entity as any)[k]), id]; - - if (this.optimisticLockField) { - const lockValue = (before as any)[this.optimisticLockField]; - if (lockValue !== undefined) { - whereParts.push(`${this.optimisticLockField} = ?`); - queryParams.push(lockValue); - const newLockVal = - typeof lockValue === "number" ? lockValue + 1 : lockValue; - entityWithTimestamps[this.optimisticLockField] = newLockVal; - const lockColIdx = keys.findIndex( - (k) => k === this.optimisticLockField, - ); - if (lockColIdx !== -1) { - queryParams[lockColIdx] = newLockVal; - } - } + // Bind from `entityWithTimestamps`, not `entity`: timestamp and lock + // columns are injected above and would otherwise bind as undefined. + const queryParams = [...keys.map((k) => entityWithTimestamps[k]), id]; + + if (this.optimisticLockField && lockValue !== undefined) { + // `col = NULL` is never true in SQL, so a NULL version needs IS NULL. + whereParts.push( + lockIsNull + ? `${this.optimisticLockColumn} IS NULL` + : `${this.optimisticLockColumn} = ?`, + ); + if (!lockIsNull) queryParams.push(lockValue); } if (this.softDeleteField) { - whereParts.push(`${this.softDeleteField} IS NULL`); + whereParts.push(`${this.softDeleteColumn} IS NULL`); } const query = `UPDATE ${this.table} SET ${setClause} WHERE ${whereParts.join(" AND ")}`; const { affectedRows } = await client.queryExec(query, queryParams); - if ( - this.optimisticLockField && - (before as any)[this.optimisticLockField] !== undefined - ) { + if (this.optimisticLockField && lockValue !== undefined) { if (affectedRows === 0) { throw new StabilizeError( `Record was modified by another transaction (optimistic lock conflict on ${this.optimisticLockField})`, @@ -646,13 +1170,8 @@ export class Repository { const result = await this.findOne(id, {}, client); - if (this.cache) { - const cacheKeys = [`find:${this.table}`, `findOne:${this.table}:${id}`]; - await this.cache.invalidate(cacheKeys); - if (this.cache.getStrategy() === "write-through") { - await this.cache.set(`findOne:${this.table}:${id}`, [result], 60); - } - } + await this.invalidateRowCache(id); + await this.writeThroughRow(id, result); this.logger.logDebug( `Updated ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, @@ -663,8 +1182,9 @@ export class Repository { async bulkUpdate( updates: { where: { condition: string; params: any[] }; set: Partial }[], options: { batchSize?: number } = {}, + _client?: DBClient, ): Promise { - return this.client.transaction((txClient) => + return (_client || this.client).transaction((txClient) => this._bulkUpdate(updates, options, txClient), ); } @@ -681,7 +1201,9 @@ export class Repository { if (!updates.length) return; const batchSize = options.batchSize || 1000; - updates.forEach((update) => this.validate(update.set)); + // `set` is a partial patch, exactly like the payload of `update()`, so + // required columns that are not being changed must not be demanded here. + updates.forEach((update) => this.validate(update.set, true)); const timestamps = this.timestampsConfig; @@ -689,7 +1211,7 @@ export class Repository { const batch = updates.slice(i, i + batchSize); for (const update of batch) { const rows = await client.query<{ id: number | string }>( - `SELECT id FROM ${this.table} WHERE ${update.where.condition}${this.softDeleteField ? ` AND ${this.softDeleteField} IS NULL` : ""}`, + `SELECT id FROM ${this.table} WHERE ${update.where.condition}${this.softDeleteField ? ` AND ${this.softDeleteColumn} IS NULL` : ""}`, update.where.params, ); for (const { id } of rows) { @@ -710,7 +1232,7 @@ export class Repository { const setClause = keys .map((k) => `${this.columns[k]?.name} = ?`) .join(", "); - const query = `UPDATE ${this.table} SET ${setClause} WHERE id = ?${this.softDeleteField ? ` AND ${this.softDeleteField} IS NULL` : ""}`; + const query = `UPDATE ${this.table} SET ${setClause} WHERE id = ?${this.softDeleteField ? ` AND ${this.softDeleteColumn} IS NULL` : ""}`; const params = [ ...keys.map((k) => (updateWithTimestamps as any)[k]), id, @@ -738,15 +1260,19 @@ export class Repository { } } - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); this.logger.logDebug( `Bulk updated ${updates.length} ${this.table} entities in ${(performance.now() - start).toFixed(2)}ms`, ); } - async upsert(entity: Partial, keys: string[]): Promise { - return this.client.transaction((txClient) => + async upsert( + entity: Partial, + keys: string[], + _client?: DBClient, + ): Promise { + return (_client || this.client).transaction((txClient) => this._upsert(entity, keys, txClient), ); } @@ -763,40 +1289,23 @@ export class Repository { this.validate(entity); const dbType = this.getDBType(client); - const columns = Object.keys(entity).filter((k) => this.columns[k]); - const columnNames = columns.map((k) => this.columns[k]?.name).join(", "); - const placeholders = columns.map(() => "?").join(", "); - - const updateClause = columns - .filter((c) => !keys.includes(c)) - .map((c) => `${this.columns[c]?.name} = ?`) - .join(", "); - - let query: string; - const updateParams = columns - .filter((c) => !keys.includes(c)) - .map((k) => (entity as any)[k]); - const insertParams = columns.map((k) => (entity as any)[k]); - let params = [...insertParams, ...updateParams]; - - let before: T | null = null; - let isUpdate = false; - if (this.versioned && keys.length > 0) { - const whereClause = keys - .map((k) => `${this.columns[k]?.name} = ?`) - .join(" AND "); - const whereParams = keys.map((k) => (entity as any)[k]); - const found = await client.query( - `SELECT * FROM ${this.table} WHERE ${whereClause} LIMIT 1`, - whereParams, - ); - before = found[0] || null; - isUpdate = !!before; + const payload: Record = { ...entity }; + // Seed the optimistic lock before the hooks run, so a `beforeUpdate` sees + // the version the write will carry. + if ( + this.optimisticLockField && + payload[this.optimisticLockField] === undefined + ) { + payload[this.optimisticLockField] = 1; } - const instance = new this.model() as T; - Object.assign(instance as object, before || {}, entity); + // Which row this will land on is decided by the conflict keys, and that + // drives the hook pair and the history operation — so resolve it even when + // the model is not versioned, rather than calling every upsert an insert. + const before = await this.findRowByKeys(keys, payload, client); + const isUpdate = !!before; + const instance = this.hydrate({ ...(before || {}), ...payload } as T); if (isUpdate) { await this.runHooks(instance, "beforeUpdate"); await this.runHooks(instance, "beforeSave"); @@ -805,10 +1314,43 @@ export class Repository { await this.runHooks(instance, "beforeSave"); } + const writeValues = this.processForSave( + this.mergeHookOutput(payload, instance, before as any), + ); + const columns = Object.keys(writeValues).filter((k) => this.columns[k]); + const columnNames = columns.map((k) => this.columns[k]?.name).join(", "); + const placeholders = columns.map(() => "?").join(", "); + const conflictColumns = keys.map((k) => this.columns[k]!.name); + const insertParams = columns.map((k) => writeValues[k]); + const updateParams = columns + .filter((c) => !keys.includes(c)) + .map((k) => writeValues[k]); + + let query: string; + let params = [...insertParams, ...updateParams]; + if (dbType === DBType.SQLite) { - query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT(${keys.map((k) => this.columns[k]!.name).join(", ")}) DO UPDATE SET ${updateClause}`; + const updateClause = columns + .filter((c) => !keys.includes(c)) + .map((c) => `${this.columns[c]?.name} = ?`) + .join(", "); + query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT(${conflictColumns.join(", ")}) DO UPDATE SET ${updateClause}`; } else if (dbType === DBType.MySQL) { + const updateClause = columns + .filter((c) => !keys.includes(c)) + .map((c) => `${this.columns[c]?.name} = ?`) + .join(", "); query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON DUPLICATE KEY UPDATE ${updateClause}`; + } else if (dbType === DBType.MSSQL) { + // `MERGE` refers to each value by a source column name, so the parameters + // are the insert payload alone — the update payload is not bound a second + // time the way the `?`-based dialects above bind it. + query = buildMSSQLUpsertSQL( + this.table, + columns.map((c) => this.columns[c]!.name), + conflictColumns, + ); + params = insertParams; } else { const pgUpdateClause = columns .filter((c) => !keys.includes(c)) @@ -816,73 +1358,96 @@ export class Repository { (c) => `${this.columns[c]?.name} = EXCLUDED.${this.columns[c]?.name}`, ) .join(", "); - query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT (${keys.map((k) => this.columns[k]!.name).join(", ")}) DO UPDATE SET ${pgUpdateClause} RETURNING *`; + query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT (${conflictColumns.join(", ")}) DO UPDATE SET ${pgUpdateClause} RETURNING *`; params = insertParams; } - const results = await client.query(query, params); - let id: number | string | undefined = - (results[0] as any)?.id || (entity as any).id; + const results = (await client.query(query, params)).map((row) => + this.processForLoad(row), + ); - if (!id && dbType !== DBType.Postgres) { - if (dbType === DBType.SQLite) { - id = ( - await client.query<{ id: number }>("SELECT last_insert_rowid() as id") - )[0]?.id; - } else if (dbType === DBType.MySQL) { - const result = await client.query<{ "LAST_INSERT_ID()": number }>( - "SELECT LAST_INSERT_ID()", - ); - id = result[0]?.["LAST_INSERT_ID()"]; + // SQLite and MySQL report nothing for an `INSERT ... ON CONFLICT`, so the + // row has to be read back. It must be read back by the conflict keys: + // on the DO UPDATE path no insert happened, so `last_insert_rowid()` still + // holds an unrelated earlier statement's value and pointed at a row this + // upsert never touched. + let result: T | null = results[0] ?? null; + if (!result) result = await this.findRowByKeys(keys, writeValues, client); + if (!result && keys.length === 0) { + const id = await this.resolveInsertedIds( + [writeValues as Partial], + client, + dbType, + this.primaryKeyField, + this.primaryKeyColumn, + ); + if (id.length > 0) { + result = await this.findOne(id[0], {}, client); } } - if (!id) - throw new StabilizeError( - "Failed to retrieve upserted ID", - "UPSERT_ERROR", - ); + if (!result) + throw new StabilizeError("Failed to retrieve upserted row", "UPSERT_ERROR"); - const result = results[0] ?? ((await this.findOne(id, {}, client)) as T); + const id = + (result as any)[this.primaryKeyField] ?? + (result as any)[this.primaryKeyColumn]; + const hydrated = this.hydrate(result); if (isUpdate) { - await this.runHooks(result, "afterUpdate"); - await this.runHooks(result, "afterSave"); + await this.runHooks(hydrated, "afterUpdate"); + await this.runHooks(hydrated, "afterSave"); } else { - await this.runHooks(result, "afterCreate"); - await this.runHooks(result, "afterSave"); + await this.runHooks(hydrated, "afterCreate"); + await this.runHooks(hydrated, "afterSave"); } if (this.versioned) { await this.writeHistory( { - ...result, + ...hydrated, version: before ? (before as any).version ? (before as any).version + 1 : 1 : 1, }, - before ? "update" : "insert", + isUpdate ? "update" : "insert", client, ); } - if (this.cache) { - await this.cache.invalidatePattern(`find:${this.table}:*`); - if (this.cache.getStrategy() === "write-through") { - await this.cache.set(`findOne:${this.table}:${id}`, [result], 60); - } + if (id !== undefined && id !== null) { + await this.invalidateRowCache(id); + await this.writeThroughRow(id, hydrated); } this.logger.logDebug( `Upserted ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, ); - return result; + return hydrated; + } + + /** + * Reads the row a set of conflict keys identifies, or null when there is + * none. Returns null for an empty key list, which has nothing to match on. + */ + private async findRowByKeys( + keys: string[], + values: Record, + client: DBClient, + ): Promise { + if (keys.length === 0) return null; + const qb = this.find(); + for (const key of keys) { + qb.where(`${this.columns[key]?.name ?? key} = ?`, values[key]); + } + const found = await qb.limit(1).execute(client); + return (found[0] as T) ?? null; } - async delete(id: number | string): Promise { - return this.client.transaction(async (txClient) => { + async delete(id: number | string, _client?: DBClient): Promise { + return (_client || this.client).transaction(async (txClient) => { const before = await this.findOne(id, {}, txClient); if (!before) throw new StabilizeError("Not found", "DELETE_ERROR"); await this.runHooks(before, "beforeDelete"); @@ -899,18 +1464,15 @@ export class Repository { this.logger.logDebug(`Deleting ${this.table} with ID ${id}`); const query = this.softDeleteField - ? `UPDATE ${this.table} SET ${this.softDeleteField} = ? WHERE id = ?` + ? `UPDATE ${this.table} SET ${this.softDeleteColumn} = ? WHERE id = ?` : `DELETE FROM ${this.table} WHERE id = ?`; - const params = this.softDeleteField ? [new Date().toISOString(), id] : [id]; + const params = this.softDeleteField + ? [sanitizeSqlValue(new Date(), this.getDBType(client)), id] + : [id]; await client.query(query, params); - if (this.cache) { - await this.cache.invalidate([ - `find:${this.table}`, - `findOne:${this.table}:${id}`, - ]); - } + await this.invalidateRowCache(id); this.logger.logDebug( `Deleted ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, ); @@ -919,8 +1481,9 @@ export class Repository { async bulkDelete( ids: (number | string)[], options: { batchSize?: number } = {}, + _client?: DBClient, ): Promise { - return this.client.transaction((txClient) => + return (_client || this.client).transaction((txClient) => this._bulkDelete(ids, options, txClient), ); } @@ -944,10 +1507,10 @@ export class Repository { await this.runHooks(before, "beforeDelete"); const query = this.softDeleteField - ? `UPDATE ${this.table} SET ${this.softDeleteField} = ? WHERE id = ?` + ? `UPDATE ${this.table} SET ${this.softDeleteColumn} = ? WHERE id = ?` : `DELETE FROM ${this.table} WHERE id = ?`; const params = this.softDeleteField - ? [new Date().toISOString(), id] + ? [sanitizeSqlValue(new Date(), this.getDBType(client)), id] : [id]; await client.query(query, params); @@ -960,15 +1523,17 @@ export class Repository { } } - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); this.logger.logDebug( `Bulk deleted ${ids.length} ${this.table} entities in ${(performance.now() - start).toFixed(2)}ms`, ); } - async recover(id: number | string): Promise { - return this.client.transaction((txClient) => this._recover(id, txClient)); + async recover(id: number | string, _client?: DBClient): Promise { + return (_client || this.client).transaction((txClient) => + this._recover(id, txClient), + ); } private async _recover(id: number | string, client: DBClient): Promise { @@ -982,7 +1547,7 @@ export class Repository { } await client.query( - `UPDATE ${this.table} SET ${this.softDeleteField} = NULL WHERE id = ?`, + `UPDATE ${this.table} SET ${this.softDeleteColumn} = NULL WHERE id = ?`, [id], ); @@ -993,12 +1558,7 @@ export class Repository { "RECOVER_ERROR", ); - if (this.cache) { - await this.cache.invalidate([ - `find:${this.table}`, - `findOne:${this.table}:${id}`, - ]); - } + await this.invalidateRowCache(id); this.logger.logDebug( `Recovered ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, @@ -1016,92 +1576,268 @@ export class Repository { return result; } - private async loadRelation(queryBuilder: QueryBuilder, relation: string) { - this.logger.logDebug(`Loading relation ${relation} for ${this.table}`); - const rel = this.relations[relation]; - if (!rel) + /** The SQL column backing a property name. @see softDeleteColumn */ + private columnName(field: string): string { + return this.columns[field]?.name ?? field; + } + + /** + * Builds a repository for a relation's target model, bound to `client`. + * + * Loading a relation means reading the target table, so the target's own + * metadata is needed: its column names, its soft-delete filter, its encrypted + * columns. Building a repository supplies all three without restating them + * here. + * + * The client is the caller's, not `this.client`: a relation loaded inside a + * transaction has to be read on the transaction's own connection, or on + * MySQL and Postgres it would read from a different connection and miss (or + * deadlock on) the uncommitted rows it is meant to see. + */ + private relatedRepository( + rel: { targetModel: () => any }, + client: DBClient, + ): Repository { + const target = rel.targetModel(); + if (!MetadataStorage.getTableName(target)) { throw new StabilizeError( - `Relation ${relation} not found`, + `Relation target ${target?.name ?? "unknown"} is not a model defined with defineModel()`, "RELATION_ERROR", ); + } + return new Repository( + client, + target, + this.cache?.config, + this.logger, + this.cache, + ); + } - const relatedTable = MetadataStorage.getTableName(rel.targetModel()); - if ( - rel.type === RelationType.OneToOne || - rel.type === RelationType.ManyToOne - ) { - queryBuilder.join( - relatedTable, - `${this.table}.${rel.foreignKey} = ${relatedTable}.id`, - ); - } else if (rel.type === RelationType.OneToMany) { - queryBuilder.join( - relatedTable, - `${relatedTable}.${rel.inverseKey} = ${this.table}.id`, + /** + * Reads the target rows whose `column` holds one of `values`. + * + * Runs against the target model's own `find()`, so its soft-delete filter and + * column decryption apply to the related rows just as they do to a direct + * read. The key list is chunked, see {@link RELATION_BATCH_SIZE}. + */ + private async fetchRelatedWhereIn( + column: string, + values: any[], + client: DBClient, + ): Promise { + if (values.length === 0) return []; + const rows: any[] = []; + for (const chunk of chunked(values, RELATION_BATCH_SIZE)) { + const qb = this.find(); + qb.where( + `${this.table}.${column} IN (${chunk.map(() => "?").join(", ")})`, + ...chunk, ); - } else if (rel.type === RelationType.ManyToMany) { - queryBuilder - .join( - rel.joinTable!, - `${rel.joinTable}.${rel.foreignKey} = ${this.table}.id`, - ) - .join( - relatedTable, - `${relatedTable}.id = ${rel.joinTable}.${rel.inverseKey}`, + rows.push(...(await qb.execute(client))); + } + return rows; + } + + /** + * Eager-loads `relations` onto an already-fetched result set, attaching each + * result under the relation's property name. + * + * Relations used to be loaded by joining the target table onto the parent + * query. Nothing ever copied the joined columns onto the parent, so + * `findOne(1, { relations: ["posts"] })` resolved to a user with no `posts` + * key at all — the one thing the option was for. The join also multiplied + * each parent once per related row, so `LIMIT 1` truncated a to-many + * relation to a single row and `countExec` counted children instead of + * parents. One batched query per relation avoids both, and it is the only + * approach that composes with `LIMIT` at all. + * + * Paths are grouped by their first segment, so `findMany({ relations: + * ["posts.comments", "posts.tags"] })` reads `posts` once, and whatever + * follows the first segment is loaded recursively against the target model. + */ + private async loadRelations( + rows: R[], + relations: string[] | undefined, + client: DBClient, + ): Promise { + if (!relations?.length || rows.length === 0) return rows; + const parents = rows as unknown as Record[]; + + const grouped = new Map(); + for (const path of relations) { + const parts = path.split(".").map((p) => p.trim()).filter(Boolean); + const head = parts.shift(); + if (!head) continue; + const rest = grouped.get(head); + const tail = parts.join("."); + if (rest) rest.push(tail); + else grouped.set(head, [tail]); + } + + for (const [head, rest] of grouped) { + const rel = this.relations[head]; + if (!rel) { + throw new StabilizeError( + `Relation ${head} not found on ${this.table}`, + "RELATION_ERROR", ); + } + const related = this.relatedRepository(rel, client); + const children = await this.attachRelation( + parents, + head, + rel, + related, + client, + ); + + const deeper = rest.filter((path) => path.length > 0); + if (deeper.length > 0) { + await related.loadRelations(children, deeper, client); + } } + + return rows; } - private async loadNestedRelations( - queryBuilder: QueryBuilder, - relationPath: string, - ) { - const parts = relationPath.split("."); - let currentTable = this.table; - let currentRelations = this.relations; - - for (let i = 0; i < parts.length; i++) { - const relName = parts[i]!; - const rel = currentRelations[relName]; - if (!rel) + /** + * Loads one relation's rows and attaches them to each parent, returning the + * children so a nested path can descend into them. + */ + private async attachRelation( + rows: Record[], + name: string, + rel: { + type: RelationType; + foreignKey?: string; + inverseKey?: string; + joinTable?: string; + }, + related: Repository, + client: DBClient, + ): Promise { + this.logger.logDebug(`Loading relation ${name} for ${this.table}`); + + if (rel.type === RelationType.ManyToMany) { + return this.attachManyToMany(rows, name, rel, related, client); + } + + if (rel.type === RelationType.OneToMany) { + // The child holds the key, so match the children on the parent's own + // primary key. + // + // `inverseKey` is the documented name, but every example in the README + // and the docs site writes `foreignKey` for this side. Both mean "the + // column on the target table that points back here", so accept either + // rather than fail on a model copied from the documentation. + const inverseField = rel.inverseKey ?? rel.foreignKey; + if (!inverseField) { throw new StabilizeError( - `Relation ${relName} not found on ${currentTable}`, + `Relation ${name} on ${this.table} needs an inverseKey naming the column on the target table that points back at ${this.table}`, "RELATION_ERROR", ); - - const relatedTable = MetadataStorage.getTableName(rel.targetModel()); - if ( - rel.type === RelationType.OneToOne || - rel.type === RelationType.ManyToOne - ) { - queryBuilder.join( - relatedTable, - `${currentTable}.${rel.foreignKey} = ${relatedTable}.id`, - ); - } else if (rel.type === RelationType.OneToMany) { - queryBuilder.join( - relatedTable, - `${relatedTable}.${rel.inverseKey} = ${currentTable}.id`, - ); - } else if (rel.type === RelationType.ManyToMany) { - queryBuilder - .join( - rel.joinTable!, - `${rel.joinTable}.${rel.foreignKey} = ${currentTable}.id`, - ) - .join( - relatedTable, - `${relatedTable}.id = ${rel.joinTable}.${rel.inverseKey}`, - ); } + const inverseColumn = related.columnName(inverseField); + const parentKey = this.primaryKeyColumn; + const children = await related.fetchRelatedWhereIn( + inverseColumn, + distinctKeys(rows, parentKey), + client, + ); + const byParent = new Map(); + for (const child of children) { + const key = child[inverseColumn]; + const bucket = byParent.get(key); + if (bucket) bucket.push(child); + else byParent.set(key, [child]); + } + for (const row of rows) { + row[name] = byParent.get(row[parentKey]) ?? []; + } + return children; + } + + // OneToOne and ManyToOne both keep the key on this side. + if (!rel.foreignKey) { + throw new StabilizeError( + `Relation ${name} on ${this.table} is missing a foreignKey`, + "RELATION_ERROR", + ); + } + const foreignColumn = this.columnName(rel.foreignKey); + const relatedKey = related.primaryKeyColumn; + const children = await related.fetchRelatedWhereIn( + relatedKey, + distinctKeys(rows, foreignColumn), + client, + ); + const byId = new Map(children.map((child) => [child[relatedKey], child])); + for (const row of rows) { + row[name] = byId.get(row[foreignColumn]) ?? null; + } + return children; + } + + /** + * Loads a many-to-many relation through its join table and attaches the + * target rows to each parent, in join-table order. + */ + private async attachManyToMany( + rows: Record[], + name: string, + rel: { foreignKey?: string; inverseKey?: string; joinTable?: string }, + related: Repository, + client: DBClient, + ): Promise { + const { joinTable, foreignKey, inverseKey } = rel; + if (!joinTable || !foreignKey || !inverseKey) { + throw new StabilizeError( + `Relation ${name} on ${this.table} needs a joinTable, foreignKey and inverseKey`, + "RELATION_ERROR", + ); + } + + const parentKey = this.primaryKeyColumn; + const parentIds = distinctKeys(rows, parentKey); + if (parentIds.length === 0) { + for (const row of rows) row[name] = []; + return []; + } - if (i < parts.length - 1) { - const nestedModel = rel.targetModel(); - const nestedRels = MetadataStorage.getRelations(nestedModel); - currentRelations = nestedRels as any; - currentTable = relatedTable; + // The join table has no model of its own, so its key columns are named by + // the relation config rather than resolved through column metadata. + const links: { parent: any; child: any }[] = []; + for (const chunk of chunked(parentIds, RELATION_BATCH_SIZE)) { + const found = await client.query>( + `SELECT ${foreignKey} AS __parent, ${inverseKey} AS __child FROM ${joinTable} WHERE ${foreignKey} IN (${chunk.map(() => "?").join(", ")})`, + chunk, + ); + for (const link of found) { + links.push({ parent: link.__parent, child: link.__child }); } } + + const relatedKey = related.primaryKeyColumn; + const children = await related.fetchRelatedWhereIn( + relatedKey, + distinctKeys(links as any, "child"), + client, + ); + const byId = new Map(children.map((child) => [child[relatedKey], child])); + + const byParent = new Map(); + for (const link of links) { + const child = byId.get(link.child); + if (!child) continue; // Soft-deleted, or gone between the two reads. + const bucket = byParent.get(link.parent); + if (bucket) bucket.push(child); + else byParent.set(link.parent, [child]); + } + for (const row of rows) { + row[name] = byParent.get(row[parentKey]) ?? []; + } + return children; } async paginate( @@ -1114,7 +1850,7 @@ export class Repository { let countQuery = `SELECT COUNT(*) as count FROM ${this.table}`; let countParams: any[] = []; if (this.softDeleteField) { - countQuery += ` WHERE ${this.table}.${this.softDeleteField} IS NULL`; + countQuery += ` WHERE ${this.table}.${this.softDeleteColumn} IS NULL`; } const result = await this.client.query<{ count: number }>( countQuery, @@ -1131,6 +1867,15 @@ export class Repository { processed[key] = encrypt(processed[key]); } } + // A `Date` is normalised per dialect rather than to ISO for all of them, + // because MySQL and MariaDB reject the ISO form outright. @see + // sanitizeSqlValue for both reasons. + const dbType = this.getDBType(); + for (const key of Object.keys(processed)) { + if (processed[key] instanceof Date) { + processed[key] = sanitizeSqlValue(processed[key], dbType); + } + } return processed; } @@ -1138,10 +1883,17 @@ export class Repository { const processed = { ...row }; for (const [key, col] of Object.entries(this.columns)) { if ((col as any).encrypted && processed[key]) { + // A failure here means the key is wrong or the value was tampered + // with. Returning `null` instead made both look like an empty field, + // so the caller could neither notice nor react. try { processed[key] = decrypt(processed[key]); - } catch { - processed[key] = null; + } catch (error) { + throw new StabilizeError( + `Failed to decrypt column "${key}": ${(error as Error).message}`, + "DECRYPTION_ERROR", + error as Error, + ); } } } @@ -1153,14 +1905,11 @@ export class Repository { async findAndCount( options: { relations?: string[] } = {}, ): Promise<{ data: T[]; total: number }> { - const qb = this.find(); - if (options.relations) { - for (const rel of options.relations) { - await this.loadRelation(qb, rel); - } - } - const data = await qb.execute(this.client); - const total = await qb.clone().countExec(this.client); + const data = await this.find().execute(this.client); + await this.loadRelations(data, options.relations, this.client); + // Counted without loading relations: a join for a to-many relation would + // count the children rather than the parents. + const total = await this.find().countExec(this.client); return { data, total }; } @@ -1180,12 +1929,11 @@ export class Repository { qb.where(`${this.columns[key]?.name} = ?`, value); } } - if (options.relations) { - for (const rel of options.relations) { - await this.loadRelation(qb, rel); - } - } + // Safe to limit before loading: relations no longer multiply the parent + // rows, which previously made `LIMIT 1` truncate a to-many relation to + // whichever single child the planner happened to return first. const results = await qb.limit(1).execute(client); + await this.loadRelations(results, options.relations, client); return results[0] ?? null; } @@ -1194,7 +1942,9 @@ export class Repository { async findBy( conditions: Partial, options: { relations?: string[]; limit?: number; orderBy?: string } = {}, + _client?: DBClient, ): Promise { + const client = _client || this.client; const qb = this.find(); for (const [key, value] of Object.entries(conditions)) { if (value === null) { @@ -1203,14 +1953,10 @@ export class Repository { qb.where(`${this.columns[key]?.name} = ?`, value); } } - if (options.relations) { - for (const rel of options.relations) { - await this.loadRelation(qb, rel); - } - } if (options.limit) qb.limit(options.limit); if (options.orderBy) qb.orderBy(options.orderBy); - return qb.execute(this.client); + const results = await qb.execute(client); + return this.loadRelations(results, options.relations, client); } // ─── FEATURE 4: count (Prisma-style) ────────────────────────────── @@ -1218,11 +1964,15 @@ export class Repository { async count(conditions?: Partial): Promise { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { - if (value !== undefined && value !== null) { + // A `null` condition means IS NULL. Skipping it (as this used to) + // silently widened the count to every row. + if (value === null) { + qb.whereNull(this.columns[key]?.name ?? key); + } else if (value !== undefined) { qb.where(`${this.columns[key]?.name} = ?`, value); } } @@ -1243,7 +1993,7 @@ export class Repository { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { @@ -1327,7 +2077,7 @@ export class Repository { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } qb.select(...selectParts); const results = await qb.execute(this.client); @@ -1383,19 +2133,19 @@ export class Repository { if (options.take) qb.take(options.take); if (options.skip) qb.skip(options.skip); - if (options.relations) { - for (const rel of options.relations) { - await this.loadRelation(qb, rel); - } - } - return qb.execute(this.client); + const results = await qb.execute(this.client); + return this.loadRelations(results, options.relations, this.client); } // ─── FEATURE 7: bulk upsert (Prisma-style) ──────────────────────── - async bulkUpsert(entities: Partial[], keys: string[]): Promise { - return this.client.transaction(async (txClient) => { + async bulkUpsert( + entities: Partial[], + keys: string[], + _client?: DBClient, + ): Promise { + return (_client || this.client).transaction(async (txClient) => { const results: T[] = []; for (const entity of entities) { results.push(await this._upsert(entity, keys, txClient)); @@ -1409,11 +2159,15 @@ export class Repository { async exists(conditions?: Partial): Promise { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { - if (value !== undefined && value !== null) { + // As in `count`, a `null` condition is a real predicate (IS NULL), + // not one to drop. + if (value === null) { + qb.whereNull(this.columns[key]?.name ?? key); + } else if (value !== undefined) { qb.where(`${this.columns[key]?.name} = ?`, value); } } @@ -1423,26 +2177,24 @@ export class Repository { // ─── FEATURE 9: recoverAll (Stabilize-original) ─────────────────── - async recoverAll(): Promise { + async recoverAll(_client?: DBClient): Promise { if (!this.softDeleteField) { throw new StabilizeError( "Soft delete not enabled for this model", "RECOVER_ERROR", ); } - const result = await this.client.queryExec( - `UPDATE ${this.table} SET ${this.softDeleteField} = NULL WHERE ${this.softDeleteField} IS NOT NULL`, + const result = await (_client || this.client).queryExec( + `UPDATE ${this.table} SET ${this.softDeleteColumn} = NULL WHERE ${this.softDeleteColumn} IS NOT NULL`, ); return result.affectedRows; } // ─── FEATURE 10: truncate (Rails-style) ─────────────────────────── - async truncate(): Promise { - await this.client.queryExec(`DELETE FROM ${this.table}`); - if (this.cache) { - await this.cache.invalidatePattern(`find:${this.table}:*`); - } + async truncate(_client?: DBClient): Promise { + await (_client || this.client).queryExec(`DELETE FROM ${this.table}`); + await this.invalidateTableCache(); } // ─── FEATURE 11: seed framework (Laravel-style) ─────────────────── @@ -1450,13 +2202,15 @@ export class Repository { async seed( data: Partial[], options: { ignoreDuplicates?: boolean } = {}, + _client?: DBClient, ): Promise { if (data.length === 0) return []; - const existing = await this.find().execute(this.client); + const client = _client || this.client; + const existing = await this.find().execute(client); if (existing.length > 0 && options.ignoreDuplicates) return existing; - return this.bulkCreate(data); + return this.bulkCreate(data, {}, client); } // ─── FEATURE 12: healthCheck ────────────────────────────────────── @@ -1492,7 +2246,7 @@ export class Repository { const colName = this.columns[column]?.name || column; const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } qb.select(`COUNT(DISTINCT ${colName}) AS __cnt`); const results = await qb.execute(this.client); @@ -1505,24 +2259,34 @@ export class Repository { id: number | string, field: string, amount: number = 1, + _client?: DBClient, ): Promise { + const client = _client || this.client; const colName = this.columns[field]?.name || field; let query = `UPDATE ${this.table} SET ${colName} = ${colName} + ? WHERE id = ?`; - if (this.softDeleteField) query += ` AND ${this.softDeleteField} IS NULL`; - await this.client.queryExec(query, [amount, id]); - return (await this.findOne(id)) as T; + if (this.softDeleteField) query += ` AND ${this.softDeleteColumn} IS NULL`; + await client.queryExec(query, [amount, id]); + // These bypass the normal write path, so clear the row's cached copies + // before reading it back — otherwise the pre-increment value is returned. + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; } async decrement( id: number | string, field: string, amount: number = 1, + _client?: DBClient, ): Promise { + const client = _client || this.client; const colName = this.columns[field]?.name || field; let query = `UPDATE ${this.table} SET ${colName} = ${colName} - ? WHERE id = ?`; - if (this.softDeleteField) query += ` AND ${this.softDeleteField} IS NULL`; - await this.client.queryExec(query, [amount, id]); - return (await this.findOne(id)) as T; + if (this.softDeleteField) query += ` AND ${this.softDeleteColumn} IS NULL`; + await client.queryExec(query, [amount, id]); + // These bypass the normal write path, so clear the row's cached copies + // before reading it back — otherwise the pre-increment value is returned. + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; } // ─── FEATURE 15: pluck (Rails-style) ────────────────────────────── @@ -1532,7 +2296,7 @@ export class Repository { const qb = new QueryBuilder(this.table); qb.select(colName); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } const results = await qb.execute(this.client); return results.map((r: any) => r[colName]); @@ -1544,18 +2308,23 @@ export class Repository { const colNames = columns.map( (c) => this.columns[c as string]?.name || (c as string), ); - const qb = new QueryBuilder(this.table); + const qb = new QueryBuilder(this.table); qb.select(...colNames); if (this.softDeleteField) { - qb.where(`${this.softDeleteField} IS NULL`); + qb.where(`${this.softDeleteColumn} IS NULL`); } - const results = await qb.execute(this.client); + const results = await this.withRowTransform(qb).execute(this.client); return results as Partial[]; } // ─── FEATURE 17: toggle (Rails-style) ───────────────────────────── - async toggle(id: number | string, field: string): Promise { + async toggle( + id: number | string, + field: string, + _client?: DBClient, + ): Promise { + const client = _client || this.client; const colName = this.columns[field]?.name || field; const dbType = this.getDBType(); const expr = @@ -1565,26 +2334,72 @@ export class Repository { ? `NOT ${colName}` : `CASE WHEN ${colName} = 1 THEN 0 ELSE 1 END`; let query = `UPDATE ${this.table} SET ${colName} = ${expr} WHERE id = ?`; - if (this.softDeleteField) query += ` AND ${this.softDeleteField} IS NULL`; - await this.client.queryExec(query, [id]); - return (await this.findOne(id)) as T; + if (this.softDeleteField) query += ` AND ${this.softDeleteColumn} IS NULL`; + await client.queryExec(query, [id]); + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; } // ─── FEATURE 18: updateBy (TypeORM-style) ───────────────────────── - async updateBy(conditions: Partial, updates: Partial): Promise { - const setKeys = Object.keys(updates).filter((k) => this.columns[k]); - if (setKeys.length === 0) return 0; + /** + * Rejects a condition set that would match every row in the table. + * + * `updateBy({}, …)` and `deleteBy({})` compiled to a statement with no WHERE + * clause, so calling either with an empty filter silently rewrote or deleted + * the entire table. `truncate()` is the explicit way to do that. + */ + private assertConditions( + conditions: Record, + method: string, + ): void { + if (Object.keys(conditions).length === 0) { + throw new StabilizeError( + `${method}() requires at least one condition; refusing to affect every row of ${this.table}. Use truncate() for that.`, + "UNSAFE_QUERY", + ); + } + } + async updateBy( + conditions: Partial, + updates: Partial, + _client?: DBClient, + ): Promise { + const client = _client || this.client; + this.assertConditions(conditions, "updateBy"); + + const payload: Record = { ...updates }; const timestamps = this.timestampsConfig; if (timestamps?.updatedAt) { - (updates as any)[timestamps.updatedAt] = new Date(); + // Set before the key list is built. It used to be assigned afterwards, + // so it never made it into the SET clause and `updatedAt` never moved. + // An ISO string, not a `Date`: SQLite refuses to bind a Date object. + payload[timestamps.updatedAt] = new Date().toISOString(); + } + // Encrypt and coerce, exactly as a single-row `update()` does — otherwise a + // bulk update wrote plaintext straight into an encrypted column. + const writeValues = this.processForSave(payload); + + const setParts: string[] = []; + const setParams: any[] = []; + for (const key of Object.keys(writeValues)) { + if (!this.columns[key]) continue; + setParts.push(`${this.columns[key]!.name} = ?`); + setParams.push(writeValues[key]); + } + // A bulk update advances the version too, so the rows it touched are not + // left holding a version that no longer matches and rejecting the next + // ordinary write as a conflict. + if ( + this.optimisticLockField && + !(this.optimisticLockField in writeValues) + ) { + const lockColumn = this.columns[this.optimisticLockField]!.name; + setParts.push(`${lockColumn} = ${lockColumn} + 1`); } - - const setClause = setKeys - .map((k) => `${this.columns[k]?.name} = ?`) - .join(", "); - const setParams = setKeys.map((k) => (updates as any)[k]); + if (setParts.length === 0) return 0; + const setClause = setParts.join(", "); const whereParts: string[] = []; const whereParams: any[] = []; @@ -1597,24 +2412,26 @@ export class Repository { } } if (this.softDeleteField) { - whereParts.push(`${this.softDeleteField} IS NULL`); + whereParts.push(`${this.softDeleteColumn} IS NULL`); } const whereClause = whereParts.length > 0 ? ` WHERE ${whereParts.join(" AND ")}` : ""; const query = `UPDATE ${this.table} SET ${setClause}${whereClause}`; - const result = await this.client.queryExec(query, [ + const result = await client.queryExec(query, [ ...setParams, ...whereParams, ]); - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); return result.affectedRows; } // ─── FEATURE 19: deleteBy (TypeORM-style) ───────────────────────── - async deleteBy(conditions: Partial): Promise { + async deleteBy(conditions: Partial, _client?: DBClient): Promise { + const client = _client || this.client; + this.assertConditions(conditions, "deleteBy"); const whereParts: string[] = []; const whereParams: any[] = []; for (const [key, value] of Object.entries(conditions)) { @@ -1627,36 +2444,36 @@ export class Repository { } if (this.softDeleteField) { - whereParts.push(`${this.softDeleteField} IS NULL`); + whereParts.push(`${this.softDeleteColumn} IS NULL`); const whereClause = whereParts.length > 0 ? ` WHERE ${whereParts.join(" AND ")}` : ""; - const query = `UPDATE ${this.table} SET ${this.softDeleteField} = ?${whereClause}`; - const result = await this.client.queryExec(query, [ - new Date().toISOString(), + const query = `UPDATE ${this.table} SET ${this.softDeleteColumn} = ?${whereClause}`; + const result = await client.queryExec(query, [ + sanitizeSqlValue(new Date(), this.getDBType(client)), ...whereParams, ]); - if (this.cache) - await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); return result.affectedRows; } const whereClause = whereParts.length > 0 ? ` WHERE ${whereParts.join(" AND ")}` : ""; - const result = await this.client.queryExec( + const result = await client.queryExec( `DELETE FROM ${this.table}${whereClause}`, whereParams, ); - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + await this.invalidateTableCache(); return result.affectedRows; } // ─── FEATURE 20: restoreBy (soft-delete recovery by condition) ──── - async restoreBy(conditions: Partial): Promise { + async restoreBy(conditions: Partial, _client?: DBClient): Promise { if (!this.softDeleteField) { throw new StabilizeError("Soft delete not enabled", "RECOVER_ERROR"); } - const whereParts: string[] = [`${this.softDeleteField} IS NOT NULL`]; + const client = _client || this.client; + const whereParts: string[] = [`${this.softDeleteColumn} IS NOT NULL`]; const whereParams: any[] = []; for (const [key, value] of Object.entries(conditions)) { if (value !== undefined && value !== null) { @@ -1664,9 +2481,9 @@ export class Repository { whereParams.push(value); } } - const query = `UPDATE ${this.table} SET ${this.softDeleteField} = NULL WHERE ${whereParts.join(" AND ")}`; - const result = await this.client.queryExec(query, whereParams); - if (this.cache) await this.cache.invalidatePattern(`find:${this.table}:*`); + const query = `UPDATE ${this.table} SET ${this.softDeleteColumn} = NULL WHERE ${whereParts.join(" AND ")}`; + const result = await client.queryExec(query, whereParams); + await this.invalidateTableCache(); return result.affectedRows; } @@ -1677,14 +2494,14 @@ export class Repository { throw new StabilizeError("Soft delete not enabled", "QUERY_ERROR"); } const qb = new QueryBuilder(this.table); - qb.whereNotNull(this.softDeleteField); - return qb; + qb.whereNotNull(this.softDeleteColumn!); + return this.withRowTransform(qb); } // ─── FEATURE 22: withTrashed (include soft-deleted in query) ────── withTrashed(): QueryBuilder { - return new QueryBuilder(this.table); + return this.withRowTransform(new QueryBuilder(this.table)); } // ─── FEATURE 23: upsertMany (batch upsert) ──────────────────────── @@ -1693,11 +2510,13 @@ export class Repository { entities: Partial[], keys: string[], batchSize: number = 100, + _client?: DBClient, ): Promise { + const client = _client || this.client; const results: T[] = []; for (let i = 0; i < entities.length; i += batchSize) { const batch = entities.slice(i, i + batchSize); - const batchResults = await this.bulkUpsert(batch, keys); + const batchResults = await this.bulkUpsert(batch, keys, client); results.push(...batchResults); } return results; @@ -1766,7 +2585,11 @@ export class Repository { const client = _client || this.client; const dbType = this.getDBType(client); const qb = this.find().where("id = ?", id).limit(1); - if (dbType !== DBType.SQLite) { + if (dbType !== DBType.SQLite && dbType !== DBType.MSSQL) { + // T-SQL has no `FOR UPDATE`; its equivalent is a table hint + // (`WITH (UPDLOCK)`), which this builder cannot express. Skipping the + // clause keeps the statement valid on SQL Server, at the cost of the + // row lock the other dialects take. qb.lock("FOR UPDATE"); } const results = await qb.execute(client); @@ -1778,10 +2601,12 @@ export class Repository { async firstOrCreate( conditions: Partial, defaults: Partial = {}, + _client?: DBClient, ): Promise { - const existing = await this.findOneBy(conditions); + const client = _client || this.client; + const existing = await this.findOneBy(conditions, {}, client); if (existing) return existing; - return this.create({ ...conditions, ...defaults }); + return this.create({ ...conditions, ...defaults }, {}, client); } // ─── FEATURE 29: createOrGet (Laravel updateOrCreate) ───────────── @@ -1789,12 +2614,14 @@ export class Repository { async updateOrCreate( conditions: Partial, updates: Partial, + _client?: DBClient, ): Promise { - const existing = await this.findOneBy(conditions); + const client = _client || this.client; + const existing = await this.findOneBy(conditions, {}, client); if (existing) { - return this.update((existing as any).id, updates); + return this.update((existing as any).id, updates, client); } - return this.create({ ...conditions, ...updates }); + return this.create({ ...conditions, ...updates }, {}, client); } // ─── FEATURE 30: first (get first matching row) ─────────────────── @@ -1831,6 +2658,10 @@ export class Repository { let orderByExpr: string; if (dbType === DBType.MySQL) { orderByExpr = "RAND()"; + } else if (dbType === DBType.MSSQL) { + // T-SQL has no `RANDOM()`; ordering by a fresh `uniqueidentifier` is the + // usual way to shuffle rows. + orderByExpr = "NEWID()"; } else if (dbType === DBType.SQLite) { orderByExpr = "RANDOM()"; } else { @@ -1841,6 +2672,277 @@ export class Repository { return results[0] ?? null; } + // ─── findOrFail / firstOrFail ───────────────────────────────────── + + /** + * Like {@link findOne}, but throws when nothing matches. + * + * Every caller of `findOne` otherwise repeats the same null check, and the + * one that forgets it fails later, somewhere else, on a missing field. + * + * @throws StabilizeError with code `NOT_FOUND_ERROR`. + * @example + * ``` + * const user = await repo.findOrFail(id); // never null + * ``` + */ + async findOrFail( + id: number | string, + options: { relations?: string[] } = {}, + _client?: DBClient, + ): Promise { + const found = await this.findOne(id, options, _client); + if (!found) { + throw new StabilizeError( + `${this.table} with id ${id} not found`, + "NOT_FOUND_ERROR", + ); + } + return found; + } + + /** + * Like {@link first}, but throws when nothing matches. @see findOrFail + * + * @throws StabilizeError with code `NOT_FOUND_ERROR`. + */ + async firstOrFail( + conditions: Partial = {}, + options: { relations?: string[] } = {}, + _client?: DBClient, + ): Promise { + const found = await this.findOneBy(conditions, options, _client); + if (!found) { + throw new StabilizeError( + `No ${this.table} matched ${JSON.stringify(conditions)}`, + "NOT_FOUND_ERROR", + ); + } + return found; + } + + // ─── Many-to-many link management ───────────────────────────────── + + /** + * Resolves a relation that must be many-to-many, with its join table and + * both key columns present. + */ + private requireJoinRelation( + relation: string, + method: string, + ): { joinTable: string; foreignKey: string; inverseKey: string } { + const rel = this.relations[relation]; + if (!rel) { + throw new StabilizeError( + `Relation ${relation} not found on ${this.table}`, + "RELATION_ERROR", + ); + } + if (rel.type !== RelationType.ManyToMany) { + throw new StabilizeError( + `${method}() needs a ManyToMany relation, but ${relation} on ${this.table} is ${RelationType[rel.type]}`, + "RELATION_ERROR", + ); + } + const { joinTable, foreignKey, inverseKey } = rel; + if (!joinTable || !foreignKey || !inverseKey) { + throw new StabilizeError( + `Relation ${relation} on ${this.table} needs a joinTable, foreignKey and inverseKey`, + "RELATION_ERROR", + ); + } + return { joinTable, foreignKey, inverseKey }; + } + + /** + * Normalises the target ids of a link operation into a deduplicated list. + * + * Accepts a single id or a list, drops nullish entries, and collapses + * duplicates — a list naming the same id twice must link it once, and would + * otherwise insert two identical rows, since the join table carries no unique + * constraint to reject the second. + */ + private normalizeLinkTargets( + targetIds: number | string | (number | string)[], + ): (number | string)[] { + const targets: (number | string)[] = []; + const seen = new Set(); + for (const value of Array.isArray(targetIds) ? targetIds : [targetIds]) { + if (value === null || value === undefined) continue; + const key = String(value); + if (seen.has(key)) continue; + seen.add(key); + targets.push(value); + } + return targets; + } + + /** + * The raw values linked to `id` through a many-to-many relation's join table. + */ + private async fetchLinkedIds( + id: number | string, + relation: string, + client: DBClient, + ): Promise { + const { joinTable, foreignKey, inverseKey } = this.requireJoinRelation( + relation, + "fetchLinkedIds", + ); + const rows = await client.query>( + `SELECT ${inverseKey} FROM ${joinTable} WHERE ${foreignKey} = ?`, + [id], + ); + return rows.map((row) => row[inverseKey]); + } + + /** + * Links each of `targetIds` to `id` through a many-to-many relation's join + * table. + * + * Idempotent: a pair that is already linked is left alone, so calling this + * twice does not create the second link twice. + * + * @returns how many links were created. + * @example + * ``` + * await postRepo.attach(postId, "tags", [1, 2, 3]); + * ``` + */ + async attach( + id: number | string, + relation: string, + targetIds: number | string | (number | string)[], + _client?: DBClient, + ): Promise { + const client = _client || this.client; + const { joinTable, foreignKey, inverseKey } = this.requireJoinRelation( + relation, + "attach", + ); + const wanted = this.normalizeLinkTargets(targetIds); + if (wanted.length === 0) return 0; + + // Compared as strings: an id read back from the driver is a number while a + // caller may pass one from a URL as a string, and treating those as + // different would insert the same link twice. + const linked = new Set( + (await this.fetchLinkedIds(id, relation, client)).map((value) => + String(value), + ), + ); + const missing = wanted.filter((value) => !linked.has(String(value))); + if (missing.length === 0) return 0; + + await client.queryExec( + `INSERT INTO ${joinTable} (${foreignKey}, ${inverseKey}) VALUES ${missing + .map(() => "(?, ?)") + .join(", ")}`, + missing.flatMap((targetId) => [id, targetId]), + ); + await this.invalidateRowCache(id); + return missing.length; + } + + /** + * Removes links between `id` and `targetIds` from a many-to-many relation's + * join table. + * + * Omitting `targetIds` unlinks everything, which is the usual way to clear a + * relation. + * + * @returns how many links were removed. + * @example + * ``` + * await postRepo.detach(postId, "tags", [3]); // unlink one tag + * await postRepo.detach(postId, "tags"); // unlink them all + * ``` + */ + async detach( + id: number | string, + relation: string, + targetIds?: number | string | (number | string)[], + _client?: DBClient, + ): Promise { + const client = _client || this.client; + const { joinTable, foreignKey, inverseKey } = this.requireJoinRelation( + relation, + "detach", + ); + + let query = `DELETE FROM ${joinTable} WHERE ${foreignKey} = ?`; + const params: any[] = [id]; + if (targetIds !== undefined) { + const wanted = this.normalizeLinkTargets(targetIds); + if (wanted.length === 0) return 0; + query += ` AND ${inverseKey} IN (${wanted.map(() => "?").join(", ")})`; + params.push(...wanted); + } + + const { affectedRows } = await client.queryExec(query, params); + await this.invalidateRowCache(id); + return affectedRows; + } + + /** + * Makes the set of rows linked to `id` exactly `targetIds`: missing links are + * created, links absent from the list are removed, and links that are already + * correct are left untouched. + * + * Runs in one transaction, so a failure part way through cannot leave the + * relation half-updated. + * + * @returns how many links were added and how many removed. + * @example + * ``` + * await postRepo.sync(postId, "tags", [1, 2]); // ends up linked to 1 and 2 + * ``` + */ + async sync( + id: number | string, + relation: string, + targetIds: (number | string)[], + _client?: DBClient, + ): Promise<{ attached: number; detached: number }> { + const client = _client || this.client; + const { joinTable, foreignKey, inverseKey } = this.requireJoinRelation( + relation, + "sync", + ); + + return client.transaction(async (txClient) => { + const wanted = this.normalizeLinkTargets(targetIds ?? []); + const wantedKeys = new Set(wanted.map((value) => String(value))); + const existing = await this.fetchLinkedIds(id, relation, txClient); + const existingKeys = new Set(existing.map((value) => String(value))); + + const toAttach = wanted.filter( + (value) => !existingKeys.has(String(value)), + ); + const toDetach = existing.filter( + (value) => !wantedKeys.has(String(value)), + ); + + if (toAttach.length > 0) { + await txClient.queryExec( + `INSERT INTO ${joinTable} (${foreignKey}, ${inverseKey}) VALUES ${toAttach + .map(() => "(?, ?)") + .join(", ")}`, + toAttach.flatMap((targetId) => [id, targetId]), + ); + } + if (toDetach.length > 0) { + await txClient.queryExec( + `DELETE FROM ${joinTable} WHERE ${foreignKey} = ? AND ${inverseKey} IN (${toDetach.map(() => "?").join(", ")})`, + [id, ...toDetach], + ); + } + + await this.invalidateRowCache(id); + return { attached: toAttach.length, detached: toDetach.length }; + }); + } + // ─── FEATURE 33: columnExists helper ────────────────────────────── hasColumn(field: string): boolean { diff --git a/stabilize-docs b/stabilize-docs deleted file mode 160000 index bf3db79..0000000 --- a/stabilize-docs +++ /dev/null @@ -1 +0,0 @@ -Subproject commit bf3db79235980423ec2a1c001c3fa71e577ac324 diff --git a/tests/auto-migrate.primary-key.test.ts b/tests/auto-migrate.primary-key.test.ts new file mode 100644 index 0000000..cc1f069 --- /dev/null +++ b/tests/auto-migrate.primary-key.test.ts @@ -0,0 +1,128 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * `autoMigrate` used to give *every* model an + * `INTEGER PRIMARY KEY AUTOINCREMENT` for its `id`, whatever type the model + * declared. A model with a UUID primary key — the pattern in the README, on + * the docs site, and what `stabilize-cli generate:model` scaffolds — therefore + * got an integer column, and every `create({ id: generateUUID() })` failed + * with "datatype mismatch": SQLite will not store a UUID in a rowid column. + * + * The declared type also had to survive: `required` and `unique` were dropped + * on the `id` branch, leaving the key nullable. + */ + +const UuidUser = defineModel({ + tableName: "pk_uuid_users", + columns: { + id: { type: DataTypes.STRING, required: true, unique: true }, + email: { type: DataTypes.STRING, required: true }, + }, +}); + +const UuidModel = defineModel({ + tableName: "pk_uuid_widgets", + columns: { + // The dedicated UUID type maps to a different column per dialect, so it + // must not take the auto-increment path either. + id: { type: DataTypes.UUID, required: true }, + label: { type: DataTypes.STRING }, + }, +}); + +const IntModel = defineModel({ + tableName: "pk_int_widgets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING }, + }, +}); + +describe("autoMigrate primary keys", () => { + let db: any; + + beforeEach(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + }); + + afterEach(async () => { + await db?.close(); + }); + + const columnsOf = async (table: string) => + Object.fromEntries( + (await db.rawQuery(`PRAGMA table_info(${table})`)).map((c: any) => [ + c.name, + c, + ]), + ); + + it("keeps a declared STRING id a string, and makes it the primary key", async () => { + await db.autoMigrate([UuidUser]); + const id = (await columnsOf("pk_uuid_users")).id; + + expect(id.type).toBe("TEXT"); + expect(id.pk).toBe(1); + // SQLite allows NULL in a non-integer PRIMARY KEY, so NOT NULL is the only + // thing that actually enforces `required: true`. + expect(id.notnull).toBe(1); + }); + + it("creates a row with a UUID id", async () => { + await db.autoMigrate([UuidUser]); + const repo = db.getRepository(UuidUser); + const uuid = "550e8400-e29b-41d4-a716-446655440000"; + + // This is the assertion that failed with "datatype mismatch". + const created: any = await repo.create({ id: uuid, email: "a@b.c" }); + expect(created.id).toBe(uuid); + expect((await repo.findOne(uuid)).email).toBe("a@b.c"); + }); + + it("keeps a declared UUID id a string", async () => { + await db.autoMigrate([UuidModel]); + const id = (await columnsOf("pk_uuid_widgets")).id; + + expect(id.type).not.toBe("INTEGER"); + expect(id.pk).toBe(1); + expect(id.notnull).toBe(1); + + const repo = db.getRepository(UuidModel); + const uuid = "6ba7b810-9dad-11d1-80b4-00c04fd430c8"; + expect((await repo.create({ id: uuid, label: "w" })).id).toBe(uuid); + }); + + it("still auto-increments an integer id", async () => { + await db.autoMigrate([IntModel]); + const tableSql: string = ( + await db.rawQuery( + "SELECT sql FROM sqlite_master WHERE name = 'pk_int_widgets'", + ) + )[0].sql; + + // The original behaviour for an integer key has to be preserved: existing + // models rely on the database assigning the id. + expect(tableSql).toMatch(/AUTOINCREMENT/); + expect((await columnsOf("pk_int_widgets")).id.notnull).toBe(0); + + const repo = db.getRepository(IntModel); + const created: any = await repo.create({ label: "auto" }); + expect(created.id).toBe(1); + }); + + it("does not alter an existing table's primary key", async () => { + // AutoMigrate's contract is that it only ever adds. A table created by the + // old code keeps its integer key, and re-running must not fail on it. + await db.autoMigrate([UuidUser]); + await db.rawExec("DROP TABLE pk_uuid_users"); + await db.rawExec( + "CREATE TABLE pk_uuid_users (id INTEGER PRIMARY KEY AUTOINCREMENT, email TEXT NOT NULL)", + ); + + await db.autoMigrate([UuidUser]); + expect((await columnsOf("pk_uuid_users")).id.type).toBe("INTEGER"); + }); +}); diff --git a/tests/client.retry.test.ts b/tests/client.retry.test.ts new file mode 100644 index 0000000..ef8ab69 --- /dev/null +++ b/tests/client.retry.test.ts @@ -0,0 +1,89 @@ +import { describe, it, expect } from "vitest"; +import { DBClient } from "../client"; +import { DBType } from "../types"; +import type { Logger } from "../logger"; +import type { PoolMetrics } from "../types"; + +/** + * `DBClient.query()` retried every failed statement. Every write in the ORM + * goes through it, so a statement that failed *after* the database committed — + * a dropped connection on the way back, say — would be retried and applied a + * second time. Retries are now limited to statements that cannot write. + */ + +/** Counts the errors a client logs, one per failed attempt. */ +class RecordingLogger implements Logger { + public errors: string[] = []; + + logQuery(): void {} + logError(error: Error): void { + this.errors.push(error.message); + } + logMetrics(_metrics: PoolMetrics): void {} + logInfo(): void {} + logWarn(): void {} + logDebug(): void {} +} + +function makeClient(): { client: DBClient; logger: RecordingLogger } { + const logger = new RecordingLogger(); + const client = new DBClient( + { + type: DBType.SQLite, + connectionString: ":memory:", + // The real backoff is 1s, which would make this file take ten seconds. + // Only the attempt count is under test. + retryDelay: 1, + maxJitter: 0, + }, + logger, + ); + return { client, logger }; +} + +describe("query retry policy", () => { + it("retries a failed read", async () => { + const { client, logger } = makeClient(); + + await expect(client.query("SELECT * FROM missing_table")).rejects.toThrow(); + // One log per attempt proves the statement was replayed. + expect(logger.errors).toHaveLength(3); + + await client.close(); + }); + + it("does not retry a failed write", async () => { + const { client, logger } = makeClient(); + await client.query("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)"); + + // A write that fails must be attempted exactly once: replaying it could + // apply the change twice. + await expect( + client.query("INSERT INTO items (nope) VALUES (?)", ["x"]), + ).rejects.toThrow(); + expect(logger.errors).toHaveLength(1); + + await client.close(); + }); + + it("still retries a read behind a leading comment", async () => { + const { client, logger } = makeClient(); + + await expect( + client.query("/* pool hint */ SELECT * FROM missing_table"), + ).rejects.toThrow(); + expect(logger.errors).toHaveLength(3); + + await client.close(); + }); + + it("does not mistake a longer keyword for a read", async () => { + const { client, logger } = makeClient(); + + // `SELECTED` is not `SELECT`; it must not be treated as a replayable read. + await expect(client.query("SELECTED nonsense")).rejects.toThrow(); + expect(logger.errors).toHaveLength(1); + + await client.close(); + }); +}); diff --git a/tests/encryption.test.ts b/tests/encryption.test.ts new file mode 100644 index 0000000..2b5a9e5 --- /dev/null +++ b/tests/encryption.test.ts @@ -0,0 +1,106 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import crypto from "crypto"; +import { encrypt, decrypt } from "../utils/encryption"; + +/** + * Coverage for the column encryption helpers. + * + * Two defects lived here: the key fell back to a constant published in the + * source, and the cipher was unauthenticated CBC, so a tampered value decrypted + * to garbage rather than failing. + */ + +const KEY = "test-key-32-bytes-long-padding!!"; +/** The constant earlier versions used when no key was configured. */ +const LEGACY_KEY = "f71a3c8e9b12d5a49c0a3f98b1f2e46d"; + +/** Produces the unauthenticated `iv:ciphertext` format older versions wrote. */ +function encryptLegacyCbc(plaintext: string, key: string): string { + const iv = crypto.randomBytes(16); + const cipher = crypto.createCipheriv( + "aes-256-cbc", + Buffer.from(key, "utf8"), + iv, + ); + const encrypted = Buffer.concat([ + cipher.update(plaintext, "utf8"), + cipher.final(), + ]); + return `${iv.toString("base64")}:${encrypted.toString("base64")}`; +} + +describe("column encryption", () => { + const original = process.env.ORM_ENCRYPTION_KEY; + + beforeEach(() => { + process.env.ORM_ENCRYPTION_KEY = KEY; + }); + + afterEach(() => { + if (original === undefined) delete process.env.ORM_ENCRYPTION_KEY; + else process.env.ORM_ENCRYPTION_KEY = original; + }); + + it("round-trips a value", () => { + const encrypted = encrypt("111-11-1111"); + expect(encrypted).not.toBe("111-11-1111"); + expect(decrypt(encrypted)).toBe("111-11-1111"); + }); + + it("produces a different ciphertext each time", () => { + // A fixed IV would leak that two rows hold the same value. + expect(encrypt("same")).not.toBe(encrypt("same")); + }); + + it("writes the authenticated format", () => { + expect(encrypt("value").startsWith("v2:")).toBe(true); + }); + + it("rejects a tampered ciphertext", () => { + const encrypted = encrypt("111-11-1111"); + const parts = encrypted.split(":"); + const data = Buffer.from(parts[3]!, "base64"); + data[0] = data[0]! ^ 0xff; + parts[3] = data.toString("base64"); + + expect(() => decrypt(parts.join(":"))).toThrow(); + }); + + it("rejects a value encrypted under a different key", () => { + const encrypted = encrypt("secret"); + process.env.ORM_ENCRYPTION_KEY = "another-key-32-bytes-long-pad!!!"; + expect(() => decrypt(encrypted)).toThrow(); + }); + + it("still reads the legacy unauthenticated format", () => { + const legacy = encryptLegacyCbc("111-11-1111", LEGACY_KEY); + process.env.ORM_ENCRYPTION_KEY = LEGACY_KEY; + + // Existing rows must stay readable after the format change. + expect(decrypt(legacy)).toBe("111-11-1111"); + }); + + it("rejects malformed input", () => { + expect(() => decrypt("not-encrypted-at-all")).toThrow(/format/i); + }); + + it("refuses to operate without a configured key", () => { + delete process.env.ORM_ENCRYPTION_KEY; + + // Previously this silently used a key published in the package. + expect(() => encrypt("value")).toThrow(/ORM_ENCRYPTION_KEY/); + expect(() => decrypt(encryptLegacyCbc("value", LEGACY_KEY))).toThrow( + /ORM_ENCRYPTION_KEY/, + ); + }); + + it("rejects a key of the wrong length", () => { + process.env.ORM_ENCRYPTION_KEY = "too-short"; + expect(() => encrypt("value")).toThrow(/32 bytes/); + }); + + it("accepts a 64-character hex key", () => { + process.env.ORM_ENCRYPTION_KEY = crypto.randomBytes(32).toString("hex"); + expect(decrypt(encrypt("value"))).toBe("value"); + }); +}); diff --git a/tests/integration.mariadb.test.ts b/tests/integration.mariadb.test.ts new file mode 100644 index 0000000..8e100c1 --- /dev/null +++ b/tests/integration.mariadb.test.ts @@ -0,0 +1,529 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import mysql from "mysql2/promise"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * End-to-end coverage against a real MariaDB server. + * + * The MySQL dialect's SQL is unit-tested by asserting the string it produces, + * but a string assertion can only prove we emitted what we intended. Whether + * `?` parameters bind in order, whether `LAST_INSERT_ID()` really hands back + * the generated key, whether `ON DUPLICATE KEY UPDATE` updates in place — + * only a server can settle those. That is what this file is for, and it is + * the MySQL-dialect file plus a set of cases that exist *because MariaDB is + * not MySQL*; those are marked `MariaDB:` below. + * + * Where MariaDB differs from MySQL 8, the difference is usually one that a + * naive dialect check gets wrong: the version string, a `JSON` column that is + * really `LONGTEXT`, a server that accepts `INSERT … RETURNING` the dialect + * never emits. The suite pins those so a change made for one MySQL-family + * server is not silently made against the other's behaviour. + * + * The suite skips itself when no server is reachable, so `bun test` stays + * green on a machine without one. Start the fleet with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const CONNECTION = + process.env.MARIADB_URL || + "mysql://stabilize:stabilize@127.0.0.1:53307/stabilize_test"; + +/** Every table this suite creates, so a re-run starts from a clean schema. */ +const TABLES = [ + "maria_users_history", + "maria_posts_history", + "maria_types_history", + "maria_users", + "maria_posts", + "maria_types", +]; + +/** True when a MariaDB answers; decides whether the suite runs or skips. */ +async function probe(): Promise { + let connection: mysql.Connection | null = null; + try { + connection = await mysql.createConnection({ + uri: CONNECTION, + connectTimeout: 3000, + }); + await connection.query("SELECT 1 AS ok"); + return true; + } catch { + return false; + } finally { + // `end()` on a connection that never opened rejects; the probe's answer + // is already known by then, so the teardown failure is not worth surfacing. + await connection?.end().catch(() => {}); + } +} + +const available = await probe(); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No MariaDB at ${CONNECTION}. Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +const User = defineModel({ + tableName: "maria_users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true, minLength: 2 }, + email: { type: DataTypes.STRING, unique: true }, + age: { type: DataTypes.INTEGER }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +const Post = defineModel({ + tableName: "maria_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + // Unique, which the Postgres twin needs for a different reason: Postgres + // names its conflict target (`ON CONFLICT (title)`) and fails loudly when + // no unique constraint backs it; MySQL's `ON DUPLICATE KEY UPDATE` names + // no target and fires on *any* unique key, so without this index the + // "upsert" would silently append a second row instead of updating the + // first. + title: { type: DataTypes.STRING, required: true, unique: true }, + body: { type: DataTypes.TEXT }, + published: { type: DataTypes.BOOLEAN }, + meta: { type: DataTypes.JSON }, + score: { type: DataTypes.DECIMAL }, + createdAt: { type: DataTypes.DATETIME }, + }, +}); + +/** One column per mapped `DataTypes` member, to prove each survives a trip. */ +const Typed = defineModel({ + tableName: "maria_types", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + s: { type: DataTypes.STRING }, + t: { type: DataTypes.TEXT }, + i: { type: DataTypes.INTEGER }, + bi: { type: DataTypes.BIGINT }, + f: { type: DataTypes.FLOAT }, + d: { type: DataTypes.DOUBLE }, + // Not `dec`: `DEC` is a reserved word in MariaDB (a synonym for DECIMAL), + // so an unquoted `dec` in a column list is a syntax error. `autoMigrate` + // backticks the identifiers in its DDL, but the runtime query paths + // (`repository.ts`, `query-builder.ts`) build column lists bare, which + // makes the reserved-word set part of the dialect's real contract. + num: { type: DataTypes.DECIMAL }, + b: { type: DataTypes.BOOLEAN }, + dt: { type: DataTypes.DATE }, + ts: { type: DataTypes.DATETIME }, + j: { type: DataTypes.JSON }, + u: { type: DataTypes.UUID }, + payload: { type: DataTypes.BLOB }, + }, +}); + +/** + * Reads the local-time components of a value the driver handed back. + * + * `mysql2` parses `DATE` and `DATETIME` columns into a `Date` built from the + * literal digits in the column, using the *local* zone — not UTC. Comparing + * through `toISOString()` would therefore shift by the machine's offset and + * produce a test that passes in one timezone and fails in another. + */ +function localParts(value: any): number[] { + const d = value instanceof Date ? value : new Date(value); + return [ + d.getFullYear(), + d.getMonth() + 1, + d.getDate(), + d.getHours(), + d.getMinutes(), + d.getSeconds(), + ]; +} + +/** + * Runs `work` and returns the error it rejected with, or `undefined` if it + * resolved. Awaits, so it always settles. + * + * Deliberately not `await expect(promise).rejects.toThrow(...)`. Under + * `bun test` that form can hang until the runner's timeout when the rejection + * has travelled through a database driver's own async machinery — mysql2 hops + * through `setImmediate` internally — leaving the statement in flight and the + * connection checked out, and the run then tears down with the pool still + * open. The try/catch asserts exactly the same thing and always settles. + * Every negative case in this file goes through here. + */ +async function caught(work: () => Promise): Promise { + try { + await work(); + return undefined; + } catch (error) { + return error; + } +} + +suite("MariaDB integration", () => { + let db: any; + let repo: any; + let postRepo: any; + let typedRepo: any; + + beforeAll(async () => { + // Start from a clean schema: a previous run's rows would collide with the + // unique-constraint and count assertions below. + const reset = await mysql.createConnection({ uri: CONNECTION }); + for (const table of TABLES) { + await reset.query(`DROP TABLE IF EXISTS \`${table}\``); + } + + db = new Stabilize({ type: DBType.MySQL, connectionString: CONNECTION }); + // Not wrapped in a try/catch: the migration is what the schema case below + // is about, and letting it escape fails the suite loudly rather than + // leaving the rest of the file to run against a schema that is not there. + await db.autoMigrate([User, Post, Typed]); + + repo = db.getRepository(User); + postRepo = db.getRepository(Post); + typedRepo = db.getRepository(Typed); + + await reset.end(); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("connects and reports a health check", async () => { + const health = await db.healthCheck(); + expect(health.status).toBe("healthy"); + // The library has no MariaDB entry in `DBType` — it reaches this server + // through the MySQL driver, so the dialect (and therefore every branch + // keyed off `config.type`) reports "mysql". + expect(health.database).toBe("mysql"); + }); + + it("MariaDB: reports a MariaDB version string, not a MySQL one", async () => { + // The reason a separate suite exists at all. Any dialect check that keys + // off the server version — "MySQL 8 supports X" — sees a different string + // here, and a naive `startsWith("8.")` or `includes("MySQL")` would take + // the wrong branch. Asserting the shape documents what such a check faces. + const rows = await db.client.query("SELECT VERSION() AS version"); + const version: string = rows[0].version; + expect(version).toContain("MariaDB"); + }); + + it("autoMigrate creates the schema", async () => { + // Running it a second time must not throw. Here that is a real claim + // rather than a formality: MySQL and MariaDB have no `IF NOT EXISTS` + // clause to put on a `CREATE INDEX`, so the second run is only harmless + // because the dialect checks `SHOW INDEX` first and emits nothing. + await db.autoMigrate([User, Post, Typed]); + + const tables = await db.client.query( + "SELECT table_name AS name FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name LIKE 'maria\\_%'", + ); + expect(tables.map((t: any) => t.name).sort()).toEqual([ + "maria_posts", + "maria_types", + "maria_users", + ]); + }); + + it("inserts a row and returns it with a generated key", async () => { + // MySQL has no `RETURNING`, so the generated `id` arrives by a second + // round trip through `SELECT LAST_INSERT_ID()`. That statement is the + // only reason this works here, and MariaDB implements it. + const created: any = await repo.create({ name: "Ada", email: "ada@x.com" }); + + expect(created).toBeTruthy(); + expect(created.id).toBeGreaterThan(0); + expect(created.name).toBe("Ada"); + }); + + it("MariaDB: the server does support INSERT ... RETURNING, but the dialect does not use it", async () => { + // MariaDB accepts `INSERT ... RETURNING` (MySQL does not), so the dialect + // *could* take the Postgres path here. It does not — the repository keeps + // the MySQL branch, uses `ON DUPLICATE KEY UPDATE` and reads the key back + // with `LAST_INSERT_ID()`. Proving both halves matters: the server feature + // is real, and the fallback still works, so a future change that branches + // on `SELECT VERSION()` has a working target either way. + const rows = await db.client.query( + "INSERT INTO maria_posts (title, body) VALUES (?, ?) RETURNING id, title", + ["returning-probe", "x"], + ); + expect(rows[0].title).toBe("returning-probe"); + expect(Number(rows[0].id)).toBeGreaterThan(0); + + await db.client.query("DELETE FROM maria_posts WHERE title = ?", [ + "returning-probe", + ]); + }); + + it("reads the row back by id", async () => { + const created: any = await repo.create({ name: "Grace", email: "grace@x.com" }); + const found: any = await repo.findOne(created.id); + expect(found?.name).toBe("Grace"); + }); + + it("updates a row", async () => { + const created: any = await repo.create({ name: "Alan", email: "alan@x.com" }); + await repo.update(created.id, { name: "Alan Turing" }); + expect((await repo.findOne(created.id))?.name).toBe("Alan Turing"); + }); + + it("deletes a row", async () => { + // `maria_users` declares `deleted_at`, so this is the soft-delete write: + // an `UPDATE` whose bound timestamp has to be in the one spelling the + // MySQL family's `DATETIME` accepts. The ISO-8601 form the other dialects + // take is rejected here with ER_TRUNCATED_WRONG_VALUE. + const created: any = await repo.create({ name: "temp", email: "temp@x.com" }); + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + }); + + it("deletes a row outright when the model declares no soft-delete column", async () => { + // The other branch of `_delete`: `maria_posts` has no `deleted_at`, so the + // row is really removed rather than flagged. + const created: any = await postRepo.create({ title: "hard-delete", body: "x" }); + await postRepo.delete(created.id); + + expect(await postRepo.findOne(created.id)).toBeNull(); + const rows = await db.client.query( + "SELECT id FROM maria_posts WHERE id = ?", + [created.id], + ); + expect(rows.length).toBe(0); + }); + + it("stores and reads back every mapped column type", async () => { + const created: any = await typedRepo.create({ + s: "varchar", + t: "long text", + i: 42, + bi: 9007199254, + f: 1.5, + d: 2.25, + num: 12.5, + b: true, + dt: "2024-01-15", + ts: "2024-01-15 10:30:00", + j: { nested: [1, 2] }, + u: "3f2504e0-4f89-11d3-9a0c-0305e82c3301", + payload: Buffer.from("bytes"), + }); + + const found: any = await typedRepo.findOne(created.id); + + expect(found.s).toBe("varchar"); + expect(found.t).toBe("long text"); + expect(found.i).toBe(42); + // MySQL hands `BIGINT` back as a JS number, not a string. The value fits + // in a double, so an exact compare is safe at this magnitude. + expect(Number(found.bi)).toBe(9007199254); + expect(Number(found.f)).toBeCloseTo(1.5); + expect(Number(found.d)).toBeCloseTo(2.25); + // `DECIMAL` comes back as a string — mysql2 will not turn a fixed-point + // value into a lossy float on its own — so this compares numerically. + expect(Number(found.num)).toBeCloseTo(12.5); + // There is no real `BOOLEAN` in either MySQL or MariaDB: the dialect maps + // it to `TINYINT(1)`, which the driver hands back as the *number* 1 or 0. + // Comparing to `true` would fail; the column is only boolean by convention. + expect(Boolean(found.b)).toBe(true); + expect(localParts(found.dt).slice(0, 3)).toEqual([2024, 1, 15]); + expect(localParts(found.ts).slice(0, 5)).toEqual([2024, 1, 15, 10, 30]); + // `UUID` maps to `CHAR(36)` — there is no native type — so it round-trips + // as the plain string it was given. + expect(found.u).toBe("3f2504e0-4f89-11d3-9a0c-0305e82c3301"); + expect(Buffer.isBuffer(found.payload)).toBe(true); + expect(found.payload.toString()).toBe("bytes"); + }); + + it("round-trips a JSON column written as an object", async () => { + // MariaDB stores a `JSON` column as `LONGTEXT` with a `CHECK(json_valid())` + // — an alias, not the distinct type MySQL 8 has. The driver therefore sees + // a text column and hands back the raw string, where on MySQL it would + // have parsed the value into an object. The value survives either way; + // only its JS type differs, so this asserts the MariaDB half explicitly + // and parses before comparing. @see the information_schema case below. + const created: any = await postRepo.create({ + title: "json", + meta: { nested: { deep: [1, 2, 3] }, flag: false }, + }); + + const found: any = await postRepo.findOne(created.id); + expect(typeof found.meta).toBe("string"); + + const meta = + typeof found.meta === "string" ? JSON.parse(found.meta) : found.meta; + expect(meta).toEqual({ nested: { deep: [1, 2, 3] }, flag: false }); + }); + + it("MariaDB: a JSON column is reported as longtext, not as a JSON type", async () => { + // The alias is observable in `information_schema`, which is exactly where + // a MySQL-flavoured schema inspection goes wrong: code looking for + // `data_type = 'json'` finds nothing here, and code that branches on + // "is this column JSON" silently takes the text path. + const rows = await db.client.query( + "SELECT data_type AS type, column_type AS columnType FROM information_schema.columns WHERE table_schema = DATABASE() AND table_name = 'maria_posts' AND column_name = 'meta'", + ); + expect(rows[0].type).toBe("longtext"); + expect(rows[0].columnType).toBe("longtext"); + }); + + it("MariaDB: a JSON column still rejects invalid JSON", async () => { + // MariaDB is widely described as *not* enforcing JSON validity the way + // MySQL's native type does, but 11.x attaches `CHECK (json_valid(...))` to + // the alias. This asserts the server's actual behaviour rather than the + // folklore, so a future MariaDB that drops the check is noticed here. + const error = await caught(() => + db.client.query("INSERT INTO maria_posts (title, meta) VALUES (?, ?)", [ + "bad-json", + "not json at all", + ]), + ); + expect(error).toBeDefined(); + }); + + it("counts and aggregates", async () => { + const total = await repo.count(); + expect(total).toBeGreaterThan(0); + }); + + it("paginates with LIMIT/OFFSET", async () => { + for (let i = 0; i < 5; i++) { + await postRepo.create({ title: `paged-${i}`, body: "x" }); + } + + const page: any = await postRepo.paginate(1, 2); + expect(page.data.length).toBeLessThanOrEqual(2); + expect(page.total).toBeGreaterThanOrEqual(5); + }); + + it("filters by a where clause rather than returning every row", async () => { + // Deliberately not the first-inserted row: if the filter were dropped, a + // `limit(1)` would still return one row and the test would pass by luck. + const found = await postRepo.findBy({ title: "paged-3" }, { limit: 1 }); + expect(found.length).toBe(1); + expect(found[0].title).toBe("paged-3"); + + expect(await postRepo.findBy({ title: "no-such-title" })).toEqual([]); + }); + + it("rolls back every write when the transaction throws", async () => { + const before = await repo.count(); + + const thrown = await caught(() => + db.transaction(async (tx: any) => { + await repo.create({ name: "rolled-back", email: "rb@x.com" }, {}, tx); + throw new Error("boom"); + }), + ); + // @see `caught` for why this is not `.rejects.toThrow`. + expect(thrown?.message).toBe("boom"); + + expect(await repo.count()).toBe(before); + expect((await repo.findBy({ name: "rolled-back" })).length).toBe(0); + }); + + it("commits writes when the transaction succeeds", async () => { + await db.transaction(async (tx: any) => { + await repo.create({ name: "committed", email: "c@x.com" }, {}, tx); + }); + + expect((await repo.findBy({ name: "committed" })).length).toBe(1); + }); + + it("seeds the optimistic lock on create and increments on update", async () => { + const created: any = await repo.create({ name: "versioned", email: "v@x.com" }); + expect(created.version).toBe(1); + + const updated: any = await repo.update(created.id, { name: "versioned-2" }); + expect(updated?.version).toBe(2); + }); + + it("enforces a unique constraint", async () => { + await repo.create({ name: "unique-one", email: "dup@x.com" }); + + const error = await caught(() => + repo.create({ name: "unique-two", email: "dup@x.com" }), + ); + expect(error?.message).toMatch(/Duplicate entry/i); + }); + + it("soft deletes, hides from queries and recovers", async () => { + const created: any = await repo.create({ name: "soft", email: "soft@x.com" }); + + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.name).toBe("soft"); + }); + + it("upserts through ON DUPLICATE KEY UPDATE", async () => { + await postRepo.create({ title: "upsert-me", body: "first" }); + + const result: any = await postRepo.upsert( + { title: "upsert-me", body: "second" }, + ["title"], + ); + + expect(result).toBeDefined(); + + // The duplicate key must have matched and updated in place. Unlike + // Postgres's named conflict target, `ON DUPLICATE KEY UPDATE` fires on + // *any* unique key, so the row count is the only proof that `title`'s + // index — and not some other one — is what the clause collided against. + const found = await postRepo.findBy({ title: "upsert-me" }); + expect(found.length).toBe(1); + expect((found[0] as any).body).toBe("second"); + }); + + it("bulk deletes and soft deletes by condition", async () => { + // The other two soft-delete writes. Both bind the same timestamp as + // `delete()` does; `bulkDelete` does it one id at a time, `deleteBy` in a + // single UPDATE across every matching row. + const first: any = await repo.create({ name: "bulk-a", email: "bulka@x.com" }); + const second: any = await repo.create({ name: "bulk-b", email: "bulkb@x.com" }); + await repo.create({ name: "by-cond", email: "bycond@x.com" }); + + await repo.bulkDelete([first.id, second.id]); + const affected = await repo.deleteBy({ name: "by-cond" }); + + expect(affected).toBe(1); + expect(await repo.findOne(first.id)).toBeNull(); + expect(await repo.findOne(second.id)).toBeNull(); + expect((await repo.findBy({ name: "by-cond" })).length).toBe(0); + }); + + it("MariaDB: DATETIME takes a CURRENT_TIMESTAMP default, and ON UPDATE works", async () => { + // MySQL's older rule — one `TIMESTAMP` column per table may default to + // `CURRENT_TIMESTAMP` — is why the generator puts `DEFAULT + // CURRENT_TIMESTAMP` on `DATETIME` rather than `TIMESTAMP`. MariaDB + // relaxes that rule, but the shipped spelling has to parse on both, so + // assert the DDL works and that the server actually fills the column in. + await db.client.query("DROP TABLE IF EXISTS maria_datetime_probe"); + await db.client.query( + "CREATE TABLE maria_datetime_probe (" + + "id INT AUTO_INCREMENT PRIMARY KEY, " + + "createdAt DATETIME DEFAULT CURRENT_TIMESTAMP, " + + "updatedAt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP)", + ); + + await db.client.query("INSERT INTO maria_datetime_probe () VALUES ()"); + const rows = await db.client.query( + "SELECT createdAt, updatedAt FROM maria_datetime_probe", + ); + // The driver hands both back as `Date`s; a null means the default never + // applied, which is the failure this case is watching for. + expect(rows[0].createdAt).toBeInstanceOf(Date); + expect(rows[0].updatedAt).toBeInstanceOf(Date); + + await db.client.query("DROP TABLE maria_datetime_probe"); + }); +}); diff --git a/tests/integration.models.test.ts b/tests/integration.models.test.ts new file mode 100644 index 0000000..1f3dd5c --- /dev/null +++ b/tests/integration.models.test.ts @@ -0,0 +1,196 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; +import { decrypt } from "../utils/encryption"; + +/** + * Integration coverage for models that use the ORM's optional column + * features: auto-managed timestamps, renamed soft-delete and lock columns, + * encryption and version history. + * + * Each of these was broken in a way that only a real database exposes. + */ + +/** + * The README's timestamps pattern: declared in `columns` *and* `timestamps`. + * + * The table name is unique across the suite on purpose. `defineModel` writes + * into a process-global registry that `getModelByTableName` reads by table + * name, so two models sharing one table name collide and the last definition + * silently wins for whichever test runs second. + */ +const Article = defineModel({ + tableName: "article_models", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, + }, + timestamps: { createdAt: "createdAt", updatedAt: "updatedAt" }, +}); + +/** Soft-delete and lock columns whose SQL names differ from their keys. */ +const Doc = defineModel({ + tableName: "docs", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + parentId: { type: DataTypes.INTEGER }, + deletedAt: { type: DataTypes.DATETIME, softDelete: true, name: "deleted_at" }, + rev: { type: DataTypes.INTEGER, optimisticLock: true, name: "rev_no" }, + }, +}); + +const Secret = defineModel({ + tableName: "secrets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + ssn: { type: DataTypes.STRING, encrypted: true }, + }, +}); + +const Account = defineModel({ + tableName: "accounts", + versioned: true, + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + }, +}); + +describe("model feature integration", () => { + let db: any; + + /** `findDeleted()` and friends return a builder; run one against the client. */ + const rows = (qb: any) => qb.execute(db.client); + + beforeAll(async () => { + // Encrypted columns need a key. The ORM no longer falls back to a + // hard-coded one, so the suite supplies its own. + process.env.ORM_ENCRYPTION_KEY = "test-key-32-bytes-long-padding!!"; + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Article, Doc, Secret, Account]); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("migrates a model whose timestamp columns are also declared in `columns`", async () => { + // The DDL must not emit `createdAt` twice. + const rows = await db.client.query("PRAGMA table_info(article_models)"); + const names = rows.map((r: any) => r.name); + expect(names.filter((n: string) => n === "createdAt")).toHaveLength(1); + expect(names.filter((n: string) => n === "updatedAt")).toHaveLength(1); + }); + + it("fills timestamps on create and refreshes updatedAt on update", async () => { + const repo = db.getRepository(Article); + const created: any = await repo.create({ title: "first" }); + + expect(created.createdAt).toBeTruthy(); + expect(created.updatedAt).toBeTruthy(); + + const updated: any = await repo.update(created.id, { title: "second" }); + expect(updated?.updatedAt).toBeTruthy(); + expect(updated?.createdAt).toBe(created.createdAt); + }); + + it("uses the mapped column name for soft delete everywhere", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "doc" }); + + expect(await repo.count()).toBe(1); + await repo.delete(created.id); + + expect(await repo.count()).toBe(0); + expect(await repo.findOne(created.id)).toBeNull(); + expect(await rows(repo.findDeleted())).toHaveLength(1); + + await repo.recover(created.id); + expect(await repo.count()).toBe(1); + }); + + it("uses the mapped column name for the optimistic lock", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "locked" }); + + // The lock column is `rev_no`; the value must round-trip and advance. + expect(created.rev_no ?? created.rev).toBe(1); + + const updated: any = await repo.update(created.id, { title: "locked-2" }); + expect(updated.rev_no ?? updated.rev).toBe(2); + }); + + it("throws CONCURRENT_MODIFICATION when the caller's version is stale", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "conflict" }); + + // Someone else writes first. + await repo.update(created.id, { title: "other-writer" }); + + // Our caller still holds version 1. + await expect( + repo.update(created.id, { title: "mine", rev: 1 }), + ).rejects.toThrow(/concurrent|modified by another/i); + }); + + it("treats a null condition as IS NULL in count and exists", async () => { + const repo = db.getRepository(Doc); + // This table is shared with the tests above, so compare against a delta + // rather than assuming an empty table. + const before = await repo.count({ parentId: null }); + + await repo.create({ title: "with-parent", parentId: 7 }); + const orphan: any = await repo.create({ title: "orphan" }); + + expect(await repo.count({ parentId: null })).toBe(before + 1); + expect(await repo.exists({ parentId: null })).toBe(true); + expect(await repo.exists({ parentId: 7 })).toBe(true); + + await repo.delete(orphan.id); + expect(await repo.count({ parentId: null })).toBe(before); + }); + + it("encrypts on both create and update, and decrypts on read", async () => { + const repo = db.getRepository(Secret); + const created: any = await repo.create({ name: "a", ssn: "111-11-1111" }); + + const rawAfterCreate = await db.client.query( + "SELECT ssn FROM secrets WHERE id = ?", + [created.id], + ); + expect(rawAfterCreate[0].ssn).not.toBe("111-11-1111"); + expect(decrypt(rawAfterCreate[0].ssn)).toBe("111-11-1111"); + expect(created.ssn).toBe("111-11-1111"); + + await repo.update(created.id, { ssn: "222-22-2222" }); + + const rawAfterUpdate = await db.client.query( + "SELECT ssn FROM secrets WHERE id = ?", + [created.id], + ); + // An update must not write plaintext into an encrypted column. + expect(rawAfterUpdate[0].ssn).not.toBe("222-22-2222"); + expect(decrypt(rawAfterUpdate[0].ssn)).toBe("222-22-2222"); + + expect((await repo.findOne(created.id))?.ssn).toBe("222-22-2222"); + }); + + it("creates the version history table with a version column", async () => { + const cols = await db.client.query("PRAGMA table_info(accounts_history)"); + expect(cols.map((c: any) => c.name)).toContain("version"); + }); + + it("writes history rows for a versioned model", async () => { + const repo = db.getRepository(Account); + const created: any = await repo.create({ name: "acct" }); + + const history = await repo.history(created.id); + expect(history.length).toBeGreaterThanOrEqual(1); + expect(history[0].operation).toBe("insert"); + }); +}); diff --git a/tests/integration.mssql.test.ts b/tests/integration.mssql.test.ts new file mode 100644 index 0000000..571b01d --- /dev/null +++ b/tests/integration.mssql.test.ts @@ -0,0 +1,311 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import sql from "mssql"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * End-to-end coverage against a real SQL Server. + * + * The dialect's SQL generation is unit-tested in `mssql.dialect.test.ts`, but + * that suite can only prove the *strings* are what we intended. Everything + * that depends on the server agreeing — that `IF OBJECT_ID(…) IS NULL CREATE + * TABLE` parses, that `OUTPUT INSERTED.*` comes back as a `recordset`, that + * `MERGE` is accepted, that `OFFSET … FETCH` is legal — is only real if a + * server says so. That is what this file is for. + * + * The suite skips itself when no server is reachable, so `bun test` stays + * green on a machine without one. Start the fleet with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const HOST = process.env.MSSQL_HOST || "127.0.0.1,51433"; +const PASSWORD = process.env.MSSQL_PASSWORD || "Stabilize!Test123"; +const DB_NAME = "stabilize_test"; + +const masterConfig = `Server=${HOST};User Id=sa;Password=${PASSWORD};Database=master;TrustServerCertificate=true`; +const testConfig = `Server=${HOST};User Id=sa;Password=${PASSWORD};Database=${DB_NAME};TrustServerCertificate=true`; + +/** True when a SQL Server answers; decides whether the suite runs or skips. */ +async function probe(): Promise { + try { + const pool = new sql.ConnectionPool(masterConfig); + await pool.connect(); + await pool.request().query("SELECT 1 AS ok"); + await pool.close(); + return true; + } catch { + return false; + } +} + +const available = await probe(); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No SQL Server at ${HOST}. Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +const User = defineModel({ + tableName: "mssql_users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true, minLength: 2 }, + email: { type: DataTypes.STRING, unique: true }, + age: { type: DataTypes.INTEGER }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +const Post = defineModel({ + tableName: "mssql_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + body: { type: DataTypes.TEXT }, + published: { type: DataTypes.BOOLEAN }, + meta: { type: DataTypes.JSON }, + score: { type: DataTypes.DECIMAL }, + createdAt: { type: DataTypes.DATETIME }, + }, +}); + +suite("SQL Server integration", () => { + let db: any; + let repo: any; + let postRepo: any; + + /** `find()` and friends return a builder; run one against the live client. */ + const rows = (qb: any) => qb.execute(db.client); + + beforeAll(async () => { + // The container only ships `master`, so the test schema is created here. + // This is also the first thing that proves the driver can actually talk to + // the server, before any ORM code is involved. + const master = await sql.connect(masterConfig); + await master + .request() + .query( + `IF DB_ID('${DB_NAME}') IS NULL CREATE DATABASE [${DB_NAME}]`, + ); + await master.close(); + + // Start from a clean schema. The databases are throwaway, but a previous + // run's rows would still collide with the unique-constraint and count + // assertions below. + const reset = new sql.ConnectionPool(testConfig); + await reset.connect(); + for (const table of [ + "mssql_users_history", + "mssql_posts_history", + "mssql_users", + "mssql_posts", + ]) { + await reset + .request() + .query(`IF OBJECT_ID('${table}', 'U') IS NOT NULL DROP TABLE [${table}]`); + } + await reset.close(); + + db = new Stabilize({ type: DBType.MSSQL, connectionString: testConfig }); + await db.autoMigrate([User, Post]); + repo = db.getRepository(User); + postRepo = db.getRepository(Post); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("connects and reports a health check", async () => { + const health = await db.healthCheck(); + expect(health.status).toBe("healthy"); + expect(health.database).toBe("mssql"); + }); + + it("autoMigrate is idempotent", async () => { + // Running it a second time must not throw: the guards the dialect emits + // (`IF OBJECT_ID(…) IS NULL`, a `sys.indexes` check) are what make that + // true, and a server is the only thing that can confirm they parse. + // A second run must be a no-op rather than an error: that is the whole + // point of the `IF OBJECT_ID(…) IS NULL` guards this dialect emits. + await db.autoMigrate([User, Post]); + + const tables = await db.client.query( + "SELECT TABLE_NAME AS name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'mssql_%'", + ); + expect(tables.map((t: any) => t.name).sort()).toEqual([ + "mssql_posts", + "mssql_users", + ]); + }); + + it("inserts a row and returns it with a generated identity", async () => { + const created: any = await repo.create({ name: "Ada", email: "ada@x.com" }); + + // `OUTPUT INSERTED.*` is the only way this can be populated on SQL Server. + expect(created).toBeTruthy(); + expect(created.id).toBeGreaterThan(0); + expect(created.name).toBe("Ada"); + }); + + it("reads the row back by id", async () => { + const created: any = await repo.create({ name: "Grace", email: "grace@x.com" }); + const found: any = await repo.findOne(created.id); + expect(found?.name).toBe("Grace"); + }); + + it("updates a row", async () => { + const created: any = await repo.create({ name: "Alan", email: "alan@x.com" }); + await repo.update(created.id, { name: "Alan Turing" }); + expect((await repo.findOne(created.id))?.name).toBe("Alan Turing"); + }); + + it("deletes a row", async () => { + const created: any = await repo.create({ name: "Temp", email: "temp@x.com" }); + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + }); + + it("stores and reads back every mapped column type", async () => { + const created: any = await postRepo.create({ + title: "Typed", + body: "long text", + published: true, + meta: { tags: ["a", "b"] }, + score: 12.5, + }); + + const found: any = await postRepo.findOne(created.id); + expect(found.title).toBe("Typed"); + expect(found.body).toBe("long text"); + // TINYINT(1)-style booleans do not exist here — this must round-trip as a + // real BIT, which the mssql driver hands back as a JS boolean. + expect(Boolean(found.published)).toBe(true); + expect(Number(found.score)).toBeCloseTo(12.5); + }); + + it("counts and aggregates", async () => { + const total = await repo.count(); + expect(total).toBeGreaterThan(0); + }); + + it("paginates with OFFSET/FETCH", async () => { + for (let i = 0; i < 5; i++) { + await postRepo.create({ title: `paged-${i}`, body: "x" }); + } + + // A bare OFFSET is illegal in T-SQL without an ORDER BY, and `OFFSET … + // FETCH` requires one too — this is where the dialect's fallback ordering + // either works or does not. + const page: any = await postRepo.paginate(1, 2); + expect(page.data.length).toBeLessThanOrEqual(2); + expect(page.total).toBeGreaterThanOrEqual(5); + }); + + it("filters by a where clause rather than returning every row", async () => { + // Deliberately not the first-inserted row: if the filter were dropped, a + // `limit(1)` would still return one row and the test would pass by luck. + const found = await postRepo.findBy({ title: "paged-3" }, { limit: 1 }); + expect(found.length).toBe(1); + expect(found[0].title).toBe("paged-3"); + + expect(await postRepo.findBy({ title: "no-such-title" })).toEqual([]); + }); + + // Asserted with an explicit try/catch rather than `expect(...).rejects`, + // which is the idiom used everywhere else in this repo. + // + // Under `bun test`, an mssql rejection that arrives after the driver's + // internal `setImmediate` hop leaves `bun test` waiting on a promise that + // never settles: the test hangs until the runner's timeout and then Bun + // segfaults tearing the run down. It is not our code and not the library — + // the same file passes in full under Node, and a standalone `bun run` script + // performing this exact transaction rejects with "boom" and rolls back + // correctly. It is the *assertion form* that trips it: this test and the + // unique-constraint test below are the only two that wait on a rejection, + // and swapping in a try/catch fixes both while checking exactly the same + // thing. A plain `setImmediate` rejection does not reproduce it in isolation, + // so the trigger is somewhere in how the driver settles a failed request. + it("rolls back every write when the transaction throws", async () => { + const before = await repo.count(); + + let thrown: any; + try { + await db.transaction(async (tx: any) => { + await repo.create({ name: "rolled-back", email: "rb@x.com" }, {}, tx); + throw new Error("boom"); + }); + } catch (error) { + thrown = error; + } + expect(thrown?.message).toBe("boom"); + + // The row must not have survived the rollback. + expect(await repo.count()).toBe(before); + expect((await repo.findBy({ name: "rolled-back" })).length).toBe(0); + }); + + it("commits writes when the transaction succeeds", async () => { + await db.transaction(async (tx: any) => { + await repo.create({ name: "committed", email: "c@x.com" }, {}, tx); + }); + + expect((await repo.findBy({ name: "committed" })).length).toBe(1); + }); + + it("seeds the optimistic lock on create and increments on update", async () => { + const created: any = await repo.create({ name: "versioned", email: "v@x.com" }); + expect(created.version).toBe(1); + + const updated: any = await repo.update(created.id, { name: "versioned-2" }); + expect(updated?.version).toBe(2); + }); + + // The other rejection assertion in this file; see the note on the rollback + // test above for why it is written as a try/catch. + it("enforces a unique constraint", async () => { + await repo.create({ name: "unique-one", email: "dup@x.com" }); + + let thrown: any; + try { + await repo.create({ name: "unique-two", email: "dup@x.com" }); + } catch (error) { + thrown = error; + } + expect(thrown).toBeDefined(); + }); + + it("soft deletes, hides from queries and recovers", async () => { + const created: any = await repo.create({ name: "soft", email: "soft@x.com" }); + + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.name).toBe("soft"); + }); + + it("upserts through MERGE", async () => { + await postRepo.create({ title: "upsert-me", body: "first" }); + + // The MERGE statement has no `ON CONFLICT`/`ON DUPLICATE KEY` equivalent, + // so this is the one write path with entirely bespoke SQL per dialect. + const result: any = await postRepo.upsert( + { title: "upsert-me", body: "second" }, + ["title"], + ); + + expect(result).toBeDefined(); + + // The MERGE must have matched on `title` and updated in place. If the + // `ON` clause failed to match, this is where a duplicate row appears. + const found = await postRepo.findBy({ title: "upsert-me" }); + expect(found.length).toBe(1); + expect((found[0] as any).body).toBe("second"); + }); +}); diff --git a/tests/integration.mysql.test.ts b/tests/integration.mysql.test.ts new file mode 100644 index 0000000..c7a4e5d --- /dev/null +++ b/tests/integration.mysql.test.ts @@ -0,0 +1,391 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import mysql from "mysql2/promise"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * End-to-end coverage against a real MySQL 8 server. + * + * The SQL this dialect emits is unit-tested elsewhere, but a string assertion + * can only prove we produced what we intended. Whether `?` parameters bind in + * order, whether `LAST_INSERT_ID()` really hands back a generated key, whether + * `ON DUPLICATE KEY UPDATE` upserts instead of duplicating — only a server can + * settle those. That is what this file is for. + * + * MySQL is also the dialect where identifiers have to be backticked rather + * than double-quoted, and where a `DATETIME` column refuses the ISO-8601 form + * the other three take. Those are properties of the DDL builder and the + * parameter binding rather than of a query, so the schema below is created by + * `autoMigrate` itself — nothing is created by hand, and a regression in the + * generated DDL fails the suite at `beforeAll`. + * + * The suite skips itself when no server is reachable, so `bun test` stays + * green on a machine without one. Start the fleet with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const CONNECTION = + process.env.MYSQL_URL || + "mysql://stabilize:stabilize@127.0.0.1:53306/stabilize_test"; + +/** True when a MySQL answers; decides whether the suite runs or skips. */ +async function probe(): Promise { + const pool = mysql.createPool(CONNECTION); + try { + await pool.query("SELECT 1 AS ok"); + return true; + } catch { + return false; + } finally { + // `end()` on a pool that never connected can still reject; the probe's + // answer is already known by then, so the teardown failure is not worth + // surfacing. + await pool.end().catch(() => {}); + } +} + +const available = await probe(); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No MySQL at ${CONNECTION}. Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +const User = defineModel({ + tableName: "my_users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true, minLength: 2 }, + email: { type: DataTypes.STRING, unique: true }, + age: { type: DataTypes.INTEGER }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +const Post = defineModel({ + tableName: "my_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + // `unique` is load-bearing, and the reason differs from PostgreSQL's. A + // `UNIQUE` column is also what makes `autoMigrate` emit the second DDL + // statement a MySQL target needs — an index it creates itself rather than + // a constraint the server derives — which is the path the schema case + // below covers. At runtime, Postgres's `ON CONFLICT (title)` names its + // conflict target and the server looks up a unique index to match it + // against; MySQL's `ON DUPLICATE KEY UPDATE` has no target clause at all + // and fires on a violation of *any* unique or primary key. With no unique + // index on `title` there is nothing for a second insert to violate, so the + // upsert would silently degrade into a plain INSERT and duplicate the row. + title: { type: DataTypes.STRING, required: true, unique: true }, + body: { type: DataTypes.TEXT }, + published: { type: DataTypes.BOOLEAN }, + meta: { type: DataTypes.JSON }, + score: { type: DataTypes.DECIMAL }, + createdAt: { type: DataTypes.DATETIME }, + }, +}); + +/** Every table this suite creates, so a re-run starts from a clean schema. */ +const TABLES = ["my_users_history", "my_posts_history", "my_users", "my_posts"]; + +/** + * Runs `work` and returns the error it rejected with, or `undefined` if it + * resolved. Awaits, so it always settles. + * + * Deliberately not `await expect(promise).rejects.toThrow(...)`. Under + * `bun test` that form can hang until the runner's timeout when the rejection + * has travelled through a database driver's own async machinery — mysql2 hops + * through `setImmediate` internally — leaving the statement in flight and the + * connection checked out, and the run then tears down with the pool still + * open. The try/catch asserts exactly the same thing and always settles. + * Every negative case in this file goes through here. + */ +async function caught(work: () => Promise): Promise { + try { + await work(); + return undefined; + } catch (error) { + return error; + } +} + +suite("MySQL integration", () => { + let db: any; + let repo: any; + let postRepo: any; + + beforeAll(async () => { + // Start from a clean schema: a previous run's rows would collide with the + // unique-constraint and count assertions below. + const reset = mysql.createPool(CONNECTION); + for (const table of TABLES) { + await reset.query(`DROP TABLE IF EXISTS \`${table}\``); + } + await reset.end(); + + db = new Stabilize({ type: DBType.MySQL, connectionString: CONNECTION }); + // Not wrapped in a try/catch: the migration *is* what several of the cases + // below are about, and letting it escape fails the suite loudly instead of + // leaving the rest of the file to run against a schema that is not there. + await db.autoMigrate([User, Post]); + + repo = db.getRepository(User); + postRepo = db.getRepository(Post); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("connects and reports a health check", async () => { + const health = await db.healthCheck(); + expect(health.status).toBe("healthy"); + expect(health.database).toBe("mysql"); + }); + + it("autoMigrate builds the schema on MySQL", async () => { + // The whole `CREATE TABLE` path, which is only reachable on this server if + // every identifier in it was quoted with a backtick: MySQL reads `"x"` as + // a string literal unless `ANSI_QUOTES` is in `sql_mode`, and the stock + // server does not set it. + // + // `DATABASE()` rather than a hard-coded schema name: the pool connects + // straight to the database named in the connection string, so this follows + // a `MYSQL_URL` override instead of silently matching nothing. + const tables = await db.client.query( + "SELECT table_name AS name FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name LIKE 'my_%'", + ); + expect(tables.map((t: any) => t.name).sort()).toEqual([ + "my_posts", + "my_users", + ]); + + // The index half of the DDL, which takes a different statement from the + // other dialects: MySQL has no `IF NOT EXISTS` clause on `CREATE INDEX`, + // so this index exists only if the untargeted form was emitted. Asserting + // the name is what separates "it was created" from "the CREATE TABLE + // happened to carry a UNIQUE constraint with the server's own name". + const indexes = await db.client.query( + "SELECT DISTINCT index_name AS name FROM information_schema.statistics WHERE table_schema = DATABASE() AND table_name = 'my_posts'", + ); + expect(indexes.map((i: any) => i.name)).toContain("my_posts_title_uniq"); + }); + + it("autoMigrate is idempotent against information_schema", async () => { + // The tables exist by now, so this takes the "add missing columns" branch + // rather than the CREATE TABLE one, and — because the index created above + // is already listed — it must find every index present and emit no + // `CREATE INDEX` at all. On MySQL that check is the *only* thing standing + // between a second run and a duplicate-key error, since the statement + // itself carries no guard. + await db.autoMigrate([User, Post]); + + const tables = await db.client.query( + "SELECT table_name AS name FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name LIKE 'my_%'", + ); + expect(tables.map((t: any) => t.name).sort()).toEqual([ + "my_posts", + "my_users", + ]); + }); + + it("inserts a row and returns it with a generated key", async () => { + // MySQL has no `RETURNING *`; the driver reads `LAST_INSERT_ID()` back off + // the connection that ran the INSERT and issues a follow-up SELECT. If + // either half were missing the `id` would come back undefined. + const created: any = await repo.create({ name: "Ada", email: "ada@x.com" }); + + expect(created).toBeTruthy(); + expect(created.id).toBeGreaterThan(0); + expect(created.name).toBe("Ada"); + }); + + it("reads the row back by id", async () => { + const created: any = await repo.create({ name: "Grace", email: "grace@x.com" }); + const found: any = await repo.findOne(created.id); + expect(found?.name).toBe("Grace"); + }); + + it("updates a row", async () => { + const created: any = await repo.create({ name: "Alan", email: "alan@x.com" }); + await repo.update(created.id, { name: "Alan Turing" }); + expect((await repo.findOne(created.id))?.name).toBe("Alan Turing"); + }); + + it("deletes a row", async () => { + // `my_users` declares `deleted_at`, so this is the soft-delete write: an + // `UPDATE` whose bound timestamp has to be in the one spelling MySQL's + // `DATETIME` accepts. The ISO-8601 form the other dialects take is + // rejected here with ER_TRUNCATED_WRONG_VALUE. + const created: any = await repo.create({ name: "Temp", email: "temp@x.com" }); + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + }); + + it("stores and reads back every mapped column type", async () => { + const created: any = await postRepo.create({ + title: "Typed", + body: "long text", + published: true, + meta: { tags: ["a", "b"] }, + score: 12.5, + }); + + const found: any = await postRepo.findOne(created.id); + expect(found.title).toBe("Typed"); + expect(found.body).toBe("long text"); + // MySQL has no `BOOLEAN`; the dialect maps it to `TINYINT(1)`, which + // `mysql2` returns as a number, not a boolean. Comparing to `true` here + // would fail on `1`, so the value is coerced first. + expect(Boolean(found.published)).toBe(true); + // `DECIMAL` comes back as a string from `mysql2` to avoid a lossy float + // conversion, so this compares numerically rather than by type. The same + // trap exists on PostgreSQL. + expect(Number(found.score)).toBeCloseTo(12.5); + // MySQL's `JSON` is a real type, so the driver parses the column back into + // a value on read — no `JSON.parse` needed here, unlike MariaDB's alias. + expect(found.meta).toEqual({ tags: ["a", "b"] }); + }); + + it("round-trips a JSON column as a real object", async () => { + // Both halves have to work for this. On the write side the object is bound + // as JSON text: `mysql2` rewrites a plain object parameter into an + // assignment list (`` `nested` = 1 ``), which is meaningless inside a + // `VALUES` clause and fails the statement outright. On the read side the + // driver parses a `JSON` column back into an object. If either half were + // missing the column would be unwritable, or come back as a string. + const created: any = await postRepo.create({ + title: "json", + meta: { nested: { deep: [1, 2, 3] }, flag: false }, + }); + + const found: any = await postRepo.findOne(created.id); + expect(found.meta).toEqual({ nested: { deep: [1, 2, 3] }, flag: false }); + }); + + it("counts and aggregates", async () => { + const total = await repo.count(); + expect(total).toBeGreaterThan(0); + }); + + it("paginates with LIMIT/OFFSET", async () => { + for (let i = 0; i < 5; i++) { + await postRepo.create({ title: `paged-${i}`, body: "x" }); + } + + const page: any = await postRepo.paginate(1, 2); + expect(page.data.length).toBeLessThanOrEqual(2); + expect(page.total).toBeGreaterThanOrEqual(5); + }); + + it("filters by a where clause rather than returning every row", async () => { + // Deliberately not the first-inserted row: if the filter were dropped, a + // `limit(1)` would still return one row and the test would pass by luck. + const found = await postRepo.findBy({ title: "paged-3" }, { limit: 1 }); + expect(found.length).toBe(1); + expect(found[0].title).toBe("paged-3"); + + expect(await postRepo.findBy({ title: "no-such-title" })).toEqual([]); + }); + + it("rolls back every write when the transaction throws", async () => { + const before = await repo.count(); + + const thrown = await caught(() => + db.transaction(async (tx: any) => { + await repo.create({ name: "rolled-back", email: "rb@x.com" }, {}, tx); + throw new Error("boom"); + }), + ); + // @see `caught` for why this is not `.rejects.toThrow`. + expect(thrown?.message).toBe("boom"); + + expect(await repo.count()).toBe(before); + expect((await repo.findBy({ name: "rolled-back" })).length).toBe(0); + }); + + it("commits writes when the transaction succeeds", async () => { + await db.transaction(async (tx: any) => { + await repo.create({ name: "committed", email: "c@x.com" }, {}, tx); + }); + + expect((await repo.findBy({ name: "committed" })).length).toBe(1); + }); + + it("seeds the optimistic lock on create and increments on update", async () => { + const created: any = await repo.create({ name: "versioned", email: "v@x.com" }); + expect(created.version).toBe(1); + + const updated: any = await repo.update(created.id, { name: "versioned-2" }); + expect(updated?.version).toBe(2); + }); + + it("enforces a unique constraint", async () => { + await repo.create({ name: "unique-one", email: "dup@x.com" }); + + const error = await caught(() => + repo.create({ name: "unique-two", email: "dup@x.com" }), + ); + expect(error?.message).toMatch(/Duplicate entry/i); + }); + + it("soft deletes, hides from queries and recovers", async () => { + const created: any = await repo.create({ name: "soft", email: "soft@x.com" }); + + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.name).toBe("soft"); + }); + + it("upserts through ON DUPLICATE KEY", async () => { + await postRepo.create({ title: "upsert-me", body: "first" }); + + const result: any = await postRepo.upsert( + { title: "upsert-me", body: "second" }, + ["title"], + ); + + expect(result).toBeDefined(); + + // The unique key must have been violated and updated in place. If the + // `ON DUPLICATE KEY` clause were wrong, or the conflict key not actually + // unique, this is where a duplicate appears. + const found = await postRepo.findBy({ title: "upsert-me" }); + expect(found.length).toBe(1); + expect((found[0] as any).body).toBe("second"); + }); + + it("deletes rows in bulk and hides them from queries", async () => { + // `bulkDelete` reaches the same soft-delete UPDATE as `delete`, one row at + // a time, so it needs the same timestamp spelling. + const first: any = await repo.create({ name: "bulk-a", email: "bulka@x.com" }); + const second: any = await repo.create({ name: "bulk-b", email: "bulkb@x.com" }); + + await repo.bulkDelete([first.id, second.id]); + + expect(await repo.findOne(first.id)).toBeNull(); + expect(await repo.findOne(second.id)).toBeNull(); + expect((await repo.findBy({ name: "bulk-a" })).length).toBe(0); + }); + + it("soft deletes by condition in one statement", async () => { + // The third soft-delete write, and the one that reaches widest: a single + // UPDATE across every matching row rather than one id at a time, so it + // needs the same timestamp spelling as the two above. + await repo.create({ name: "by-cond-a", email: "bya@x.com" }); + await repo.create({ name: "by-cond-b", email: "byb@x.com" }); + + const affected = await repo.deleteBy({ name: "by-cond-a" }); + + expect(affected).toBe(1); + expect((await repo.findBy({ name: "by-cond-a" })).length).toBe(0); + expect((await repo.findBy({ name: "by-cond-b" })).length).toBe(1); + }); +}); diff --git a/tests/integration.postgres.test.ts b/tests/integration.postgres.test.ts new file mode 100644 index 0000000..033f1ad --- /dev/null +++ b/tests/integration.postgres.test.ts @@ -0,0 +1,292 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Pool } from "pg"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * End-to-end coverage against a real PostgreSQL server. + * + * The SQL this dialect emits is unit-tested elsewhere, but a string assertion + * can only prove we produced what we intended. Whether `$1`-style parameters + * bind in order, whether `RETURNING *` really hands back a generated key, + * whether `ON CONFLICT` upserts instead of duplicating — only a server can + * settle those. That is what this file is for. + * + * The suite skips itself when no server is reachable, so `bun test` stays + * green on a machine without one. Start the fleet with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const CONNECTION = + process.env.POSTGRES_URL || + "postgres://stabilize:stabilize@127.0.0.1:55432/stabilize_test"; + +/** True when a PostgreSQL answers; decides whether the suite runs or skips. */ +async function probe(): Promise { + const pool = new Pool({ connectionString: CONNECTION, connectionTimeoutMillis: 3000 }); + try { + await pool.query("SELECT 1 AS ok"); + return true; + } catch { + return false; + } finally { + // `end()` on a pool that never connected rejects; the probe's answer is + // already known by then, so the teardown failure is not worth surfacing. + await pool.end().catch(() => {}); + } +} + +const available = await probe(); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No PostgreSQL at ${CONNECTION}. Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +const User = defineModel({ + tableName: "pg_users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true, minLength: 2 }, + email: { type: DataTypes.STRING, unique: true }, + age: { type: DataTypes.INTEGER }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +const Post = defineModel({ + tableName: "pg_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + // `unique` is load-bearing: the upsert below names `title` as its conflict + // target, and PostgreSQL rejects `ON CONFLICT (title)` outright unless a + // unique index covers it. Without the constraint `autoMigrate` creates a + // plain column and the upsert fails with "there is no unique or exclusion + // constraint matching the ON CONFLICT specification". + title: { type: DataTypes.STRING, required: true, unique: true }, + body: { type: DataTypes.TEXT }, + published: { type: DataTypes.BOOLEAN }, + meta: { type: DataTypes.JSON }, + score: { type: DataTypes.DECIMAL }, + createdAt: { type: DataTypes.DATETIME }, + }, +}); + +suite("PostgreSQL integration", () => { + let db: any; + let repo: any; + let postRepo: any; + + beforeAll(async () => { + // Start from a clean schema: a previous run's rows would collide with the + // unique-constraint and count assertions below. + const reset = new Pool({ connectionString: CONNECTION }); + for (const table of [ + "pg_users_history", + "pg_posts_history", + "pg_users", + "pg_posts", + ]) { + await reset.query(`DROP TABLE IF EXISTS ${table} CASCADE`); + } + await reset.end(); + + db = new Stabilize({ type: DBType.Postgres, connectionString: CONNECTION }); + await db.autoMigrate([User, Post]); + repo = db.getRepository(User); + postRepo = db.getRepository(Post); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("connects and reports a health check", async () => { + const health = await db.healthCheck(); + expect(health.status).toBe("healthy"); + expect(health.database).toBe("postgres"); + }); + + it("autoMigrate is idempotent", async () => { + // Running it a second time must not throw. `CREATE TABLE IF NOT EXISTS` + // parses anywhere; that the server accepts it twice is the real claim. + await db.autoMigrate([User, Post]); + + const tables = await db.client.query( + "SELECT table_name AS name FROM information_schema.tables WHERE table_schema = 'public' AND table_name LIKE 'pg_%'", + ); + expect(tables.map((t: any) => t.name).sort()).toEqual([ + "pg_posts", + "pg_users", + ]); + }); + + it("inserts a row and returns it with a generated key", async () => { + // `RETURNING *` is what makes this possible; without it the generated + // `id` would have to be fetched by a second round trip, or guessed. + const created: any = await repo.create({ name: "Ada", email: "ada@x.com" }); + + expect(created).toBeTruthy(); + expect(created.id).toBeGreaterThan(0); + expect(created.name).toBe("Ada"); + }); + + it("reads the row back by id", async () => { + const created: any = await repo.create({ name: "Grace", email: "grace@x.com" }); + const found: any = await repo.findOne(created.id); + expect(found?.name).toBe("Grace"); + }); + + it("updates a row", async () => { + const created: any = await repo.create({ name: "Alan", email: "alan@x.com" }); + await repo.update(created.id, { name: "Alan Turing" }); + expect((await repo.findOne(created.id))?.name).toBe("Alan Turing"); + }); + + it("deletes a row", async () => { + const created: any = await repo.create({ name: "Temp", email: "temp@x.com" }); + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + }); + + it("stores and reads back every mapped column type", async () => { + const created: any = await postRepo.create({ + title: "Typed", + body: "long text", + published: true, + meta: { tags: ["a", "b"] }, + score: 12.5, + }); + + const found: any = await postRepo.findOne(created.id); + expect(found.title).toBe("Typed"); + expect(found.body).toBe("long text"); + expect(found.published).toBe(true); + // `NUMERIC` comes back as a string from `pg` to avoid a lossy float + // conversion, so this compares numerically rather than by type. + expect(Number(found.score)).toBeCloseTo(12.5); + }); + + it("round-trips a JSON column as a real object", async () => { + // `pg` parses `json`/`jsonb` back into a value on its own, and serialises + // an object parameter on the way out. If either half were missing the + // column would come back as a string. + const created: any = await postRepo.create({ + title: "json", + meta: { nested: { deep: [1, 2, 3] }, flag: false }, + }); + + const found: any = await postRepo.findOne(created.id); + expect(found.meta).toEqual({ nested: { deep: [1, 2, 3] }, flag: false }); + }); + + it("counts and aggregates", async () => { + const total = await repo.count(); + expect(total).toBeGreaterThan(0); + }); + + it("paginates with LIMIT/OFFSET", async () => { + for (let i = 0; i < 5; i++) { + await postRepo.create({ title: `paged-${i}`, body: "x" }); + } + + const page: any = await postRepo.paginate(1, 2); + expect(page.data.length).toBeLessThanOrEqual(2); + expect(page.total).toBeGreaterThanOrEqual(5); + }); + + it("filters by a where clause rather than returning every row", async () => { + // Deliberately not the first-inserted row: if the filter were dropped, a + // `limit(1)` would still return one row and the test would pass by luck. + const found = await postRepo.findBy({ title: "paged-3" }, { limit: 1 }); + expect(found.length).toBe(1); + expect(found[0].title).toBe("paged-3"); + + expect(await postRepo.findBy({ title: "no-such-title" })).toEqual([]); + }); + + it("rolls back every write when the transaction throws", async () => { + const before = await repo.count(); + + // Deliberately a `try`/`catch` rather than + // `expect(promise).rejects.toThrow("boom")`: under `bun test` the matcher + // hangs forever when the rejection comes back through `pg`'s async + // machinery, and the test then dies on the runner's timeout. The catch + // below asserts exactly the same thing. + let thrown: any; + try { + await db.transaction(async (tx: any) => { + await repo.create({ name: "rolled-back", email: "rb@x.com" }, {}, tx); + throw new Error("boom"); + }); + } catch (error) { + thrown = error; + } + + expect(thrown?.message).toBe("boom"); + expect(await repo.count()).toBe(before); + expect((await repo.findBy({ name: "rolled-back" })).length).toBe(0); + }); + + it("commits writes when the transaction succeeds", async () => { + await db.transaction(async (tx: any) => { + await repo.create({ name: "committed", email: "c@x.com" }, {}, tx); + }); + + expect((await repo.findBy({ name: "committed" })).length).toBe(1); + }); + + it("seeds the optimistic lock on create and increments on update", async () => { + const created: any = await repo.create({ name: "versioned", email: "v@x.com" }); + expect(created.version).toBe(1); + + const updated: any = await repo.update(created.id, { name: "versioned-2" }); + expect(updated?.version).toBe(2); + }); + + it("enforces a unique constraint", async () => { + await repo.create({ name: "unique-one", email: "dup@x.com" }); + + // Same reason as the rollback test: `.rejects.toThrow()` hangs under + // `bun test` when the failure surfaces through `pg`. + let thrown: any; + try { + await repo.create({ name: "unique-two", email: "dup@x.com" }); + } catch (error) { + thrown = error; + } + expect(thrown).toBeDefined(); + }); + + it("soft deletes, hides from queries and recovers", async () => { + const created: any = await repo.create({ name: "soft", email: "soft@x.com" }); + + await repo.delete(created.id); + expect(await repo.findOne(created.id)).toBeNull(); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.name).toBe("soft"); + }); + + it("upserts through ON CONFLICT", async () => { + await postRepo.create({ title: "upsert-me", body: "first" }); + + const result: any = await postRepo.upsert( + { title: "upsert-me", body: "second" }, + ["title"], + ); + + expect(result).toBeDefined(); + + // The conflict target must have matched and updated in place. If the + // `ON CONFLICT` clause were wrong this is where a duplicate appears. + const found = await postRepo.findBy({ title: "upsert-me" }); + expect(found.length).toBe(1); + expect((found[0] as any).body).toBe("second"); + }); +}); diff --git a/tests/integration.relations.test.ts b/tests/integration.relations.test.ts new file mode 100644 index 0000000..09e8bb9 --- /dev/null +++ b/tests/integration.relations.test.ts @@ -0,0 +1,508 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Cache, Repository, Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; + +/** + * End-to-end coverage for eager-loaded relations. + * + * Relations used to be "loaded" by joining the target table onto the parent + * query, and nothing ever copied the joined columns onto the parent. So + * `findOne(id, { relations: ["chapters"] })` resolved to a row with no + * `chapters` key at all. The join also multiplied each parent once per related + * row, which meant `LIMIT 1` truncated a to-many relation to a single row and + * `COUNT(*)` counted children instead of parents. + * + * These tests are written against real SQLite because every one of those + * symptoms is a property of the SQL that was generated. + */ + +const Author = defineModel({ + tableName: "rel_authors", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true }, + }, +}); + +const Tag = defineModel({ + tableName: "rel_tags", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING, required: true }, + }, + relations: [ + { + type: RelationType.ManyToMany, + target: () => Book, + property: "books", + joinTable: "rel_book_tags", + foreignKey: "tag_id", + inverseKey: "book_id", + }, + ], +}); + +const Book = defineModel({ + tableName: "rel_books", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // The property name deliberately differs from the column name: a relation + // names the *property*, and the old join put that name straight into the + // SQL, so a mapped key produced "no such column". + authorId: { type: DataTypes.INTEGER, name: "author_id" }, + }, + relations: [ + { + type: RelationType.ManyToOne, + target: () => Author, + property: "author", + foreignKey: "authorId", + }, + { + type: RelationType.OneToMany, + target: () => Chapter, + property: "chapters", + inverseKey: "bookId", + }, + { + type: RelationType.ManyToMany, + target: () => Tag, + property: "tags", + joinTable: "rel_book_tags", + foreignKey: "book_id", + inverseKey: "tag_id", + }, + ], +}); + +const Chapter = defineModel({ + tableName: "rel_chapters", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING }, + bookId: { type: DataTypes.INTEGER, name: "book_id" }, + removedAt: { type: DataTypes.DATETIME, softDelete: true }, + }, + relations: [ + { + type: RelationType.ManyToOne, + target: () => Book, + property: "book", + foreignKey: "bookId", + }, + ], +}); + +describe("relation loading", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Author, Book, Chapter, Tag]); + // Join tables have no model of their own, so `autoMigrate` does not create + // them; a real application declares one in a migration. + await db.rawExec( + "CREATE TABLE rel_book_tags (book_id INTEGER, tag_id INTEGER)", + ); + + await db.rawExec("INSERT INTO rel_authors (id, name) VALUES (1, 'Ada')"); + // Book ids deliberately differ from their author's, so a row whose columns + // were overwritten by the joined row is detectable. + await db.rawExec( + "INSERT INTO rel_books (id, title, author_id) VALUES (10, 'First', 1), (11, 'Second', 1), (12, 'Orphan', NULL)", + ); + await db.rawExec( + "INSERT INTO rel_chapters (id, title, book_id) VALUES (100, 'One', 10), (101, 'Two', 10), (102, 'Three', 10), (103, 'Other', 11)", + ); + await db.rawExec( + "INSERT INTO rel_tags (id, label) VALUES (1, 'alpha'), (2, 'beta')", + ); + // Tag 2 is attached to book 10 first; the relation must preserve that order. + await db.rawExec( + "INSERT INTO rel_book_tags (book_id, tag_id) VALUES (10, 2), (10, 1), (11, 1)", + ); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("attaches a to-many relation instead of dropping it", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters"] }); + + expect(book.chapters).toHaveLength(3); + expect(book.chapters.map((c: any) => c.title)).toEqual([ + "One", + "Two", + "Three", + ]); + }); + + it("keeps the parent row intact when a relation is loaded", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author"] }); + + // The join used to leave the parent's own columns unprojected, so the + // joined row's id could land in `book.id`. + expect(book.id).toBe(10); + expect(book.title).toBe("First"); + expect(book.author).toEqual({ id: 1, name: "Ada" }); + }); + + it("resolves a foreign key declared under a mapped column name", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author"] }); + + // `authorId` is stored as `author_id`, so this only works at all if the + // relation translates property names to column names — the old join put + // `rel_books.authorId` into the SQL and failed with "no such column". + expect(book.author.name).toBe("Ada"); + + // Note: the base row's own key is the *column* name. Reads everywhere in + // this library hand back driver rows, which are keyed by SQL column, so a + // mapped column is not reachable under its property name. That is a + // pre-existing wart across all read paths, not specific to relations. + expect(book.author_id).toBe(1); + }); + + it("gives a to-one relation null when the key is unset", async () => { + const orphan: any = await db + .getRepository(Book) + .findOne(12, { relations: ["author"] }); + + // `null`, not `undefined`: the relation was loaded and is genuinely empty. + expect(orphan.author).toBeNull(); + }); + + it("counts parents, not joined children", async () => { + const { data, total } = await db + .getRepository(Book) + .findAndCount({ relations: ["chapters"] }); + + // Three chapters belong to one book; the join made COUNT(*) report them. + expect(total).toBe(3); + expect(data).toHaveLength(3); + expect(data[0].chapters).toHaveLength(3); + }); + + it("returns the requested number of parents, not of children", async () => { + const books = await db + .getRepository(Book) + .findMany({ relations: ["chapters"], take: 2 }); + + // The join produced one row per chapter, so `take: 2` used to return the + // same book twice rather than two books. + expect(books).toHaveLength(2); + expect(new Set(books.map((b: any) => b.id)).size).toBe(2); + }); + + it("loads a many-to-many relation through its join table", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["tags"] }); + + // Order follows the join table, not the target table's own ordering. + expect(book.tags.map((t: any) => t.label)).toEqual(["beta", "alpha"]); + }); + + it("returns an empty array for a relation with no links", async () => { + const book: any = await db + .getRepository(Book) + .findOne(12, { relations: ["tags"] }); + + expect(book.tags).toEqual([]); + }); + + it("supports the inverse side of a many-to-many relation", async () => { + const tag: any = await db + .getRepository(Tag) + .findOne(1, { relations: ["books"] }); + + expect(tag.books.map((b: any) => b.title).sort()).toEqual([ + "First", + "Second", + ]); + }); + + it("loads a nested path against the target model", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters.book"] }); + + expect(book.chapters).toHaveLength(3); + expect(book.chapters[0].book.title).toBe("First"); + }); + + it("loads several relations in one call", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author", "chapters", "tags"] }); + + expect(book.author.id).toBe(1); + expect(book.chapters).toHaveLength(3); + expect(book.tags).toHaveLength(2); + }); + + it("gives every parent its own children in a batch read", async () => { + const books = await db + .getRepository(Book) + .findMany({ relations: ["chapters"] }); + + const byTitle = Object.fromEntries( + books.map((b: any) => [b.title, b.chapters.map((c: any) => c.title)]), + ); + + // One batched query serves all parents, so the grouping has to be right. + expect(byTitle["First"]).toEqual(["One", "Two", "Three"]); + expect(byTitle["Second"]).toEqual(["Other"]); + expect(byTitle["Orphan"]).toEqual([]); + }); + + it("omits related rows the target model soft-deleted", async () => { + await db.rawExec( + "UPDATE rel_chapters SET removedAt = '2020-01-01T00:00:00.000Z' WHERE id = 101", + ); + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters"] }); + + expect(book.chapters.map((c: any) => c.id)).toEqual([100, 102]); + + await db.rawExec("UPDATE rel_chapters SET removedAt = NULL WHERE id = 101"); + }); + + it("loads relations for a findBy condition", async () => { + const books = await db + .getRepository(Book) + .findBy({ authorId: 1 }, { relations: ["author"] }); + + expect(books).toHaveLength(2); + expect(books[0].author.name).toBe("Ada"); + }); + + it("rejects an unknown relation name", async () => { + // Silently returning the bare row is what made the bug so hard to notice. + await expect( + db.getRepository(Book).findOne(10, { relations: ["nope"] }), + ).rejects.toThrow(/Relation nope not found/); + }); + + it("accepts foreignKey as the name of a OneToMany's inverse column", async () => { + // Every example in the README and the docs site writes `foreignKey` for + // the OneToMany side, so a model copied from the documentation has to work. + const Shelf = defineModel({ + tableName: "rel_shelves", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + }, + relations: [ + { + type: RelationType.OneToMany, + target: () => Book, + property: "books", + foreignKey: "authorId", + }, + ], + }); + await db.autoMigrate([Shelf]); + await db.rawExec("INSERT INTO rel_shelves (id, name) VALUES (1, 'Top')"); + + const shelf: any = await db + .getRepository(Shelf) + .findOne(1, { relations: ["books"] }); + + // Matched on `author_id`, i.e. the named property was translated to its + // column. `Orphan` has a null key, so it belongs to no shelf. + expect(shelf.books.map((b: any) => b.title).sort()).toEqual([ + "First", + "Second", + ]); + }); + + it("caches a relation-loaded read together with its relations", async () => { + // A `Cache` with no `redisUrl` is a no-op, so this uses an in-memory one: + // the point is that whatever `findOne` stores is what a later read gets. + const cache = new FakeCache(); + const repo = new Repository( + db.client, + Book, + cache.config, + undefined, + cache, + ); + + const first: any = await repo.findOne(10, { relations: ["chapters"] }); + expect(first.chapters).toHaveLength(3); + + // Remove a child behind the cache's back. The next read has to be served + // from the cache — which proves both that the cache was consulted and that + // the entry holds the relation. The un-hydrated rows used to be cached. + await db.rawExec("DELETE FROM rel_chapters WHERE id = 102"); + const cached: any = await repo.findOne(10, { relations: ["chapters"] }); + expect(cache.reads).toBeGreaterThan(0); + expect(cached.chapters).toHaveLength(3); + + await cache.invalidatePattern("findOne:*"); + const reread: any = await repo.findOne(10, { relations: ["chapters"] }); + expect(reread.chapters).toHaveLength(2); + + await db.rawExec( + "INSERT INTO rel_chapters (id, title, book_id) VALUES (102, 'Three', 10)", + ); + }); + + it("keeps a relation-loaded read out of the transaction cache", async () => { + // Rows read inside a transaction may be rolled back, so they must never be + // cached — the cache key is per row and would outlive the rollback. + const cache = new FakeCache(); + const repo = new Repository( + db.client, + Book, + cache.config, + undefined, + cache, + ); + + await db.transaction(async (txClient: any) => { + await repo.findOne(11, { relations: ["chapters"] }, txClient); + }); + + expect(cache.writes).toBe(0); + }); +}); + +describe("withRelations on the query builder", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Author, Book, Chapter, Tag]); + await db.rawExec( + "CREATE TABLE rel_book_tags (book_id INTEGER, tag_id INTEGER)", + ); + await db.rawExec("INSERT INTO rel_authors (id, name) VALUES (1, 'Ada')"); + await db.rawExec( + "INSERT INTO rel_books (id, title, author_id) VALUES (10, 'First', 1), (11, 'Second', 1)", + ); + await db.rawExec( + "INSERT INTO rel_chapters (id, title, book_id) VALUES (100, 'One', 10), (101, 'Two', 10), (102, 'Other', 11)", + ); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("loads relations requested on the builder", async () => { + // Documented in the README but never implemented: `withRelations` did not + // exist on QueryBuilder at all. + const books = await db + .getRepository(Book) + .find() + .withRelations("chapters") + .execute(db.client); + + expect(books).toHaveLength(2); + expect(books[0].chapters).toHaveLength(2); + expect(books[1].chapters).toHaveLength(1); + }); + + it("composes with where, limit and orderBy", async () => { + const books = await db + .getRepository(Book) + .find() + .where("rel_books.title = ?", "First") + .withRelations("author", "chapters") + .limit(1) + .execute(db.client); + + expect(books).toHaveLength(1); + expect(books[0].author.name).toBe("Ada"); + expect(books[0].chapters).toHaveLength(2); + }); + + it("loads a nested path", async () => { + const books = await db + .getRepository(Book) + .find() + .withRelations("chapters.book") + .execute(db.client); + + expect(books[0].chapters[0].book.title).toBe("First"); + }); + + it("accepts a list as well as separate arguments, and de-duplicates", async () => { + const qb = db + .getRepository(Book) + .find() + .withRelations(["chapters", "author"], "chapters"); + + expect(qb.getRelations()).toEqual(["chapters", "author"]); + }); + + it("survives a clone", async () => { + const qb = db.getRepository(Book).find().withRelations("chapters"); + expect(qb.clone().getRelations()).toEqual(["chapters"]); + + // countExec clones, and a count must not be affected by the relations. + expect(await qb.clone().countExec(db.client)).toBe(2); + }); + + it("is a no-op on a builder with no repository behind it", async () => { + // A standalone builder has no model metadata, so it simply cannot load + // relations — but asking must not throw or corrupt the query. + const { QueryBuilder } = await import("../query-builder"); + const rows = await new QueryBuilder("rel_books") + .withRelations("chapters") + .execute(db.client); + + expect(rows).toHaveLength(2); + expect(rows[0].chapters).toBeUndefined(); + }); +}); + +/** + * An in-memory stand-in for Redis. The real `Cache` no-ops without a + * `redisUrl`, which would make the caching assertions vacuous. + */ +class FakeCache extends Cache { + private store = new Map(); + public reads = 0; + public writes = 0; + + constructor() { + super({ enabled: false, ttl: 60 }); + } + + async get(key: string): Promise { + this.reads++; + const value = this.store.get(key); + return value === undefined ? null : (value as T); + } + + async set(key: string, value: T): Promise { + this.writes++; + // Through JSON, as Redis would: the cached shape is what a caller gets. + this.store.set(key, JSON.parse(JSON.stringify(value))); + } + + async invalidate(keys: string[]): Promise { + for (const key of keys) this.store.delete(key); + } + + async invalidatePattern(pattern: string): Promise { + const re = new RegExp( + `^${pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*")}$`, + ); + for (const key of [...this.store.keys()]) { + if (re.test(key)) this.store.delete(key); + } + } +} diff --git a/tests/integration.sqlite.test.ts b/tests/integration.sqlite.test.ts new file mode 100644 index 0000000..0681aa4 --- /dev/null +++ b/tests/integration.sqlite.test.ts @@ -0,0 +1,125 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * End-to-end coverage against a real in-memory SQLite database. + * + * These exercise the paths that unit tests with fake clients cannot reach: + * actual transaction control, constraint enforcement and the DDL emitted by + * `autoMigrate`. + */ + +const User = defineModel({ + tableName: "users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true, minLength: 2 }, + email: { type: DataTypes.STRING, unique: true }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +describe("in-memory SQLite integration", () => { + let db: any; + let repo: any; + + /** `find()` and friends return a builder; run one against the live client. */ + const rows = (qb: any) => qb.execute(db.client); + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([User]); + repo = db.getRepository(User); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("rolls back every write when the transaction throws", async () => { + const before = await repo.count(); + + await expect( + db.transaction(async (tx: any) => { + await repo.create({ name: "rolled-back" }, {}, tx); + throw new Error("boom"); + }), + ).rejects.toThrow("boom"); + + // The row must not have survived the rollback. + expect(await repo.count()).toBe(before); + expect( + (await repo.findBy({ name: "rolled-back" })).length, + ).toBe(0); + }); + + it("commits writes when the transaction succeeds", async () => { + await db.transaction(async (tx: any) => { + await repo.create({ name: "committed" }, {}, tx); + }); + + expect((await repo.findBy({ name: "committed" })).length).toBe(1); + }); + + it("rolls back a mix of create, update and delete together", async () => { + const created: any = await repo.create({ name: "victim" }); + const idsBefore = (await rows(repo.find())).map((r: any) => r.id); + + await expect( + db.transaction(async (tx: any) => { + await repo.update(created.id, { name: "renamed" }, tx); + await repo.create({ name: "bystander" }, {}, tx); + throw new Error("abort"); + }), + ).rejects.toThrow("abort"); + + const after = await rows(repo.find()); + expect(after.map((r: any) => r.id)).toEqual(idsBefore); + expect((await repo.findOne(created.id))?.name).toBe("victim"); + }); + + it("seeds the optimistic lock on create and increments on update", async () => { + const created: any = await repo.create({ name: "versioned" }); + + expect(created.version).toBe(1); + + // The caller does not pass `version`; the lock must still advance. + const updated: any = await repo.update(created.id, { name: "versioned-2" }); + expect(updated?.version).toBe(2); + + const again: any = await repo.update(created.id, { name: "versioned-3" }); + expect(again?.version).toBe(3); + }); + + it("rejects a create that fails validation", async () => { + await expect(repo.create({ name: "A" })).rejects.toThrow("too short"); + }); + + it("enforces a unique constraint", async () => { + await repo.create({ name: "unique-one", email: "dup@example.com" }); + + await expect( + repo.create({ name: "unique-two", email: "dup@example.com" }), + ).rejects.toThrow(); + }); + + it("soft deletes, hides from queries and recovers", async () => { + const created: any = await repo.create({ name: "soft" }); + + await repo.delete(created.id); + + expect(await repo.findOne(created.id)).toBeNull(); + expect( + (await rows(repo.findDeleted())).some((r: any) => r.id === created.id), + ).toBe(true); + expect( + (await rows(repo.withTrashed())).some((r: any) => r.id === created.id), + ).toBe(true); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.name).toBe("soft"); + }); +}); diff --git a/tests/integration.write-paths.test.ts b/tests/integration.write-paths.test.ts new file mode 100644 index 0000000..ba7b66c --- /dev/null +++ b/tests/integration.write-paths.test.ts @@ -0,0 +1,334 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; +import { decrypt } from "../utils/encryption"; + +/** + * End-to-end coverage for the write paths that are only observable against a + * real database: lifecycle hooks, `bulkCreate` and `upsert`. + * + * Each case here is a bug that a fake client could not expose — the hooks ran + * against objects with no model prototype, `bulkCreate` guessed which rows it + * had inserted, and `upsert` resolved its id from a value that describes an + * unrelated statement. + */ + +/** Records every hook invocation so the tests can assert on ordering. */ +const calls: string[] = []; + +const Doc = defineModel({ + tableName: "hooked_docs", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // Deliberately required and deliberately never supplied by the caller: the + // `beforeCreate` hook is what fills it in. + slug: { type: DataTypes.STRING, required: true }, + tag: { type: DataTypes.STRING }, + }, + hooks: { + beforeCreate: (entity: any) => { + entity.slug = String(entity.title).toLowerCase().replace(/\s+/g, "-"); + }, + afterCreate: (entity: any) => { + calls.push(`afterCreate:${entity.id}`); + }, + beforeUpdate: (entity: any) => { + calls.push(`beforeUpdate:${entity.id}`); + entity.tag = `${entity.tag}-touched`; + }, + afterSave: (entity: any) => { + calls.push(`afterSave:${entity.id}`); + }, + beforeDelete: (entity: any) => { + calls.push(`beforeDelete:${entity.id}`); + }, + afterDelete: (entity: any) => { + calls.push(`afterDelete:${entity.id}`); + }, + }, +}); + +// A hook declared as a class method rather than in the config. It only +// resolves on a real model instance, which is what `hydrate` provides. +(Doc.prototype as any).afterUpdate = function () { + calls.push(`methodAfterUpdate:${this.id}`); +}; + +const Account = defineModel({ + tableName: "upserted_accounts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, unique: true }, + name: { type: DataTypes.STRING }, + nickname: { type: DataTypes.STRING }, + }, +}); + +const Secret = defineModel({ + tableName: "bulk_secrets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING }, + ssn: { type: DataTypes.STRING, encrypted: true }, + }, +}); + +/** Timestamps, an optimistic lock and both bulk operations. */ +const Note = defineModel({ + tableName: "notes", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + body: { type: DataTypes.STRING }, + rev: { type: DataTypes.INTEGER, optimisticLock: true }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, + }, + timestamps: { createdAt: "createdAt", updatedAt: "updatedAt" }, +}); + +describe("write path integration", () => { + let db: any; + + const rows = (qb: any) => qb.execute(db.client); + + beforeAll(async () => { + // Encrypted columns need a key; there is no hard-coded fallback. + process.env.ORM_ENCRYPTION_KEY = "test-key-32-bytes-long-padding!!"; + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Doc, Account, Secret]); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("persists what a beforeCreate hook wrote", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Hello World" }); + + // The hook supplied `slug`, which is required — validation runs after the + // hooks, so the insert succeeds and the value is actually written. + expect(created.slug).toBe("hello-world"); + expect((await repo.findOne(created.id))?.slug).toBe("hello-world"); + + const raw = await db.client.query( + "SELECT slug FROM hooked_docs WHERE id = ?", + [created.id], + ); + expect(raw[0].slug).toBe("hello-world"); + }); + + it("runs afterCreate and afterSave with the created entity", async () => { + const repo = db.getRepository(Doc); + calls.length = 0; + + const created: any = await repo.create({ title: "Second" }); + + expect(calls).toContain(`afterCreate:${created.id}`); + expect(calls).toContain(`afterSave:${created.id}`); + }); + + it("runs beforeUpdate, the class-method afterUpdate and afterSave on update", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Third", tag: "x" }); + calls.length = 0; + + const updated: any = await repo.update(created.id, { title: "Third!" }); + + expect(calls).toContain(`beforeUpdate:${created.id}`); + expect(calls).toContain(`afterSave:${created.id}`); + expect(calls).toContain(`methodAfterUpdate:${created.id}`); + // The beforeUpdate mutation has to reach the database. + expect(updated.tag).toBe("x-touched"); + }); + + it("runs the delete hooks", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Fourth" }); + calls.length = 0; + + await repo.delete(created.id); + + expect(calls).toContain(`beforeDelete:${created.id}`); + expect(calls).toContain(`afterDelete:${created.id}`); + }); + + it("writes every column of a bulk batch, not just the first row's", async () => { + const repo = db.getRepository(Doc); + calls.length = 0; + + const created = await repo.bulkCreate([ + { title: "narrow", slug: "narrow" }, + { title: "wide", slug: "wide", tag: "kept" }, + ]); + + // The second row carries a column the first does not. Deriving the column + // list from `batch[0]` used to drop it silently. + const wide = created.find((doc: any) => doc.title === "wide"); + expect(wide?.tag).toBe("kept"); + + const raw = await db.client.query( + "SELECT tag FROM hooked_docs WHERE title = ?", + ["wide"], + ); + expect(raw[0].tag).toBe("kept"); + }); + + it("returns the rows a bulk insert actually created", async () => { + const repo = db.getRepository(Doc); + const created = await repo.bulkCreate([ + { title: "a1", slug: "a1" }, + { title: "a2", slug: "a2" }, + { title: "a3", slug: "a3" }, + ]); + + expect(created).toHaveLength(3); + expect(created.map((doc: any) => doc.title)).toEqual(["a1", "a2", "a3"]); + + // Every returned id must be a row that holds the matching value, which is + // what `ORDER BY id DESC LIMIT n` failed to guarantee. + for (const doc of created) { + const raw = await db.client.query( + "SELECT title FROM hooked_docs WHERE id = ?", + [doc.id], + ); + expect(raw[0].title).toBe(doc.title); + } + }); + + it("returns the row an upsert actually wrote", async () => { + const repo = db.getRepository(Account); + const first: any = await repo.upsert( + { email: "a@example.com", name: "first" }, + ["email"], + ); + + // An unrelated insert moves the connection's last-inserted rowid, which is + // what the old id resolution read back. + await repo.create({ email: "b@example.com", name: "other" }); + + const upserted: any = await repo.upsert( + { email: "a@example.com", name: "second" }, + ["email"], + ); + + expect(upserted.id).toBe(first.id); + expect(upserted.name).toBe("second"); + expect(upserted.email).toBe("a@example.com"); + + const other = await repo.findOneBy({ email: "b@example.com" }); + expect(other?.name).toBe("other"); + }); + + it("treats an upsert onto an existing key as an update", async () => { + const repo = db.getRepository(Account); + await repo.upsert({ email: "c@example.com", name: "one" }, ["email"]); + const before = await repo.count(); + + const second: any = await repo.upsert( + { email: "c@example.com", name: "two" }, + ["email"], + ); + + expect(await repo.count()).toBe(before); + expect(second.name).toBe("two"); + expect(await rows(repo.find())).toHaveLength(before); + }); + + it("encrypts a column written through bulkCreate", async () => { + const repo = db.getRepository(Secret); + await repo.bulkCreate([ + { label: "one", ssn: "111-11-1111" }, + { label: "two", ssn: "222-22-2222" }, + ]); + + const raw = await db.client.query( + "SELECT label, ssn FROM bulk_secrets ORDER BY id ASC", + ); + expect(raw[0].ssn).not.toBe("111-11-1111"); + expect(decrypt(raw[0].ssn)).toBe("111-11-1111"); + expect(decrypt(raw[1].ssn)).toBe("222-22-2222"); + }); +}); + +describe("bulk operation guards", () => { + let db: any; + + const rows = (qb: any) => qb.execute(db.client); + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Note]); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("advances updatedAt on updateBy", async () => { + const repo = db.getRepository(Note); + const created: any = await repo.create({ title: "one", body: "x" }); + + // The clock is coarse enough that a same-millisecond write is possible. + await new Promise((resolve) => setTimeout(resolve, 5)); + const affected = await repo.updateBy({ title: "one" }, { body: "y" }); + + expect(affected).toBe(1); + const after: any = await repo.findOne(created.id); + expect(after.body).toBe("y"); + expect(after.updatedAt).toBeTruthy(); + expect(after.updatedAt).not.toBe(created.updatedAt); + }); + + it("advances the optimistic lock on updateBy", async () => { + const repo = db.getRepository(Note); + const created: any = await repo.create({ title: "locked-here" }); + + await repo.updateBy({ title: "locked-here" }, { body: "z" }); + + const after: any = await repo.findOne(created.id); + expect(after.rev).toBe(2); + + // The row must still be writable: a stale version would reject this. + const updated: any = await repo.update(created.id, { body: "z2" }); + expect(updated.rev).toBe(3); + }); + + it("refuses updateBy with no conditions", async () => { + const repo = db.getRepository(Note); + await repo.create({ title: "survivor" }); + const before = await repo.count(); + + await expect(repo.updateBy({}, { body: "wiped" })).rejects.toThrow( + /at least one condition/i, + ); + expect(await repo.count()).toBe(before); + expect(await rows(repo.find())).toHaveLength(before); + }); + + it("refuses deleteBy with no conditions", async () => { + const repo = db.getRepository(Note); + const before = await repo.count(); + + await expect(repo.deleteBy({})).rejects.toThrow( + /at least one condition/i, + ); + expect(await repo.count()).toBe(before); + }); + + it("deletes only the rows deleteBy matches", async () => { + const repo = db.getRepository(Note); + await repo.create({ title: "delete-me" }); + await repo.create({ title: "keep-me" }); + const before = await repo.count(); + + const affected = await repo.deleteBy({ title: "delete-me" }); + + expect(affected).toBe(1); + expect(await repo.count()).toBe(before - 1); + expect(await repo.exists({ title: "keep-me" })).toBe(true); + }); +}); diff --git a/tests/metadata.bundle-boundary.test.ts b/tests/metadata.bundle-boundary.test.ts new file mode 100644 index 0000000..f09b1cf --- /dev/null +++ b/tests/metadata.bundle-boundary.test.ts @@ -0,0 +1,172 @@ +import { describe, it, expect } from "vitest"; +import { Repository } from "../repository"; +import { MetadataStorage, defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; + +/** + * A model class as it appears when it was created by a *different copy* of the + * ORM: the metadata lives on the class statics, while the MetadataStorage + * registry of this module knows nothing about it. + * + * This is what a duplicate dependency or a mixed CJS/ESM build produces, and it + * used to silently disable validation, optimistic locking and relation loading. + */ +class Post { + static tableName = "posts"; + constructor(public data?: any) {} +} + +class User { + static tableName = "users"; + static versioned = true; + static softDelete = false; + static columns: any = { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + username: { type: DataTypes.STRING, minLength: 3, maxLength: 10 }, + email: { + type: DataTypes.STRING, + pattern: /@example\.com$/, + }, + handle: { + type: DataTypes.STRING, + customValidator: (val: string) => + val.startsWith("@") || "Handle must start with @", + }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + version: { type: DataTypes.INTEGER, optimisticLock: true }, + }; + static relations: any = [ + { + type: RelationType.ManyToOne, + target: () => Post, + property: "posts", + foreignKey: "user_id", + }, + ]; + static timestamps = { createdAt: "created_at", updatedAt: "updated_at" }; + constructor(public data?: any) {} +} + +const fakeClient: any = { config: { type: DBType.SQLite } }; + +describe("metadata resolution across bundle boundaries", () => { + it("should read table name, versioning and timestamps from class statics", () => { + const repo: any = new Repository(fakeClient, User); + + expect(repo.table).toBe("users"); + expect(repo.versioned).toBe(true); + expect(repo.timestampsConfig).toEqual({ + createdAt: "created_at", + updatedAt: "updated_at", + }); + }); + + it("should detect the soft delete and optimistic lock columns", () => { + const repo: any = new Repository(fakeClient, User); + + expect(repo.softDeleteField).toBe("deleted_at"); + expect(repo.optimisticLockField).toBe("version"); + }); + + it("should key relations by property name rather than array index", () => { + const repo: any = new Repository(fakeClient, User); + + expect(Object.keys(repo.relations)).toEqual(["posts"]); + expect(repo.relations.posts.targetModel()).toBe(Post); + expect(repo.relations.posts.foreignKey).toBe("user_id"); + }); + + it("should enforce length, pattern and custom validators", () => { + const repo: any = new Repository(fakeClient, User); + + const base = { title: "Hello" }; + + expect(() => repo.validate({ ...base, username: "ab" })).toThrow("too short"); + expect(() => repo.validate({ ...base, username: "abcdefghijk" })).toThrow( + "too long", + ); + expect(() => repo.validate({ ...base, email: "x@nope.com" })).toThrow( + "does not match pattern", + ); + expect(() => repo.validate({ ...base, handle: "someone" })).toThrow( + "Handle must start with @", + ); + expect(() => + repo.validate({ + ...base, + email: "someone@example.com", + handle: "@someone", + }), + ).not.toThrow(); + }); + + it("should enforce required columns", () => { + const repo: any = new Repository(fakeClient, User); + + expect(() => repo.validate({})).toThrow("required"); + expect(() => repo.validate({ title: "Hello" })).not.toThrow(); + }); + + it("should exempt an auto-increment primary key from `required`", () => { + const repo: any = new Repository(fakeClient, User); + + // `id: { type: DataTypes.INTEGER, required: true }` is the documented + // spelling of an auto-increment PK, so the database supplies the value. + expect(repo.autoIncrementField).toBe("id"); + expect(() => repo.validate({ title: "Hello" })).not.toThrow(); + + // A string/UUID primary key is generated by the caller, so it stays required. + const Uuid = defineModel({ + tableName: "uuid_models", + columns: { + id: { type: DataTypes.UUID, required: true }, + title: { type: DataTypes.STRING, required: true }, + }, + }); + const uuidRepo: any = new Repository(fakeClient, Uuid); + expect(uuidRepo.autoIncrementField).toBeNull(); + expect(() => uuidRepo.validate({ title: "Hello" })).toThrow("required"); + }); +}); + +describe("MetadataStorage cross-copy sharing", () => { + it("should anchor the registry on globalThis so copies share one map", () => { + const registry = (globalThis as any)[ + Symbol.for("stabilize-orm.model-registry") + ]; + + expect(registry).toBeInstanceOf(Map); + // A model defined here must be visible to every copy reading that map. + const Article = defineModel({ + tableName: "articles", + columns: { id: { type: DataTypes.INTEGER, required: true } }, + }); + expect(registry.get(Article)).toBeDefined(); + expect(MetadataStorage.getModelByTableName("articles")).toBe(Article); + }); + + it("should resolve validators from class statics", () => { + expect(MetadataStorage.getValidators(User)).toEqual({ + id: ["required"], + title: ["required"], + username: [], + email: [], + handle: [], + deleted_at: [], + version: [], + }); + }); + + it("should prefer the registry entry over class statics", () => { + const Local = defineModel({ + tableName: "local_only", + columns: { id: { type: DataTypes.INTEGER, required: true } }, + }); + + expect(MetadataStorage.getTableName(Local)).toBe("local_only"); + expect(MetadataStorage.getModelMetadata(Local)?.tableName).toBe( + "local_only", + ); + }); +}); diff --git a/tests/mssql.dialect.test.ts b/tests/mssql.dialect.test.ts new file mode 100644 index 0000000..44082b8 --- /dev/null +++ b/tests/mssql.dialect.test.ts @@ -0,0 +1,431 @@ +import { describe, it, expect } from "vitest"; +import sql from "mssql"; +import { + generateMigration, + mapDataTypeToSql, + createTableIfNotExistsSQL, + createIndexIfNotExistsSQL, +} from "../migrations"; +import { rewritePlaceholders, bindMSSQLParam } from "../client"; +import { buildLimitClause, QueryBuilder } from "../query-builder"; +import { buildMSSQLUpsertSQL } from "../repository"; +import { DBType, DataTypes } from "../types"; +import { defineModel } from "../model"; + +/** + * These cover the parts of SQL Server support that are pure text: type + * mappings, placeholder rewriting, row-limit clauses, DDL and the `MERGE` + * upsert. Nothing here needs a server, and nothing here pretends to have run + * one. + */ + +const mssqlModel = defineModel({ + tableName: "mssql_widgets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING, required: true, unique: true }, + }, +}); + +const mssqlUuidModel = defineModel({ + tableName: "mssql_gadgets", + columns: { + id: { type: DataTypes.UUID, required: true }, + label: { type: DataTypes.STRING, required: true }, + }, +}); + +describe("mapDataTypeToSql for SQL Server", () => { + const mssqlMappings: [DataTypes, string][] = [ + [DataTypes.STRING, "NVARCHAR(255)"], + [DataTypes.TEXT, "NVARCHAR(MAX)"], + [DataTypes.INTEGER, "INT"], + [DataTypes.BIGINT, "BIGINT"], + [DataTypes.FLOAT, "REAL"], + [DataTypes.DOUBLE, "FLOAT"], + [DataTypes.DECIMAL, "DECIMAL(10,2)"], + [DataTypes.BOOLEAN, "BIT"], + [DataTypes.DATE, "DATE"], + [DataTypes.DATETIME, "DATETIME2"], + [DataTypes.JSON, "NVARCHAR(MAX)"], + [DataTypes.UUID, "UNIQUEIDENTIFIER"], + [DataTypes.BLOB, "VARBINARY(MAX)"], + ]; + + it.each(mssqlMappings)("maps %s", (dataType, expected) => { + expect(mapDataTypeToSql(dataType, DBType.MSSQL)).toBe(expected); + }); + + it("falls back to NVARCHAR(MAX) for an unknown type", () => { + expect(mapDataTypeToSql("somethingelse", DBType.MSSQL)).toBe( + "NVARCHAR(MAX)", + ); + }); + + it("accepts a raw string type name", () => { + expect(mapDataTypeToSql("uuid", DBType.MSSQL)).toBe("UNIQUEIDENTIFIER"); + }); + + it("leaves PostgreSQL mappings unchanged", () => { + expect(mapDataTypeToSql(DataTypes.STRING, DBType.Postgres)).toBe("TEXT"); + expect(mapDataTypeToSql(DataTypes.DOUBLE, DBType.Postgres)).toBe( + "DOUBLE PRECISION", + ); + expect(mapDataTypeToSql(DataTypes.JSON, DBType.Postgres)).toBe("JSONB"); + expect(mapDataTypeToSql(DataTypes.BLOB, DBType.Postgres)).toBe("BYTEA"); + expect(mapDataTypeToSql("unknown", DBType.Postgres)).toBe("TEXT"); + }); + + it("leaves MySQL mappings unchanged", () => { + expect(mapDataTypeToSql(DataTypes.STRING, DBType.MySQL)).toBe( + "VARCHAR(255)", + ); + expect(mapDataTypeToSql(DataTypes.BOOLEAN, DBType.MySQL)).toBe("TINYINT(1)"); + expect(mapDataTypeToSql("unknown", DBType.MySQL)).toBe("TEXT"); + }); + + it("leaves SQLite mappings unchanged", () => { + expect(mapDataTypeToSql(DataTypes.STRING, DBType.SQLite)).toBe("TEXT"); + expect(mapDataTypeToSql(DataTypes.DECIMAL, DBType.SQLite)).toBe("NUMERIC"); + expect(mapDataTypeToSql("unknown", DBType.SQLite)).toBe("TEXT"); + }); +}); + +describe("rewritePlaceholders", () => { + it("numbers SQL Server parameters from zero, in order", () => { + expect( + rewritePlaceholders( + "SELECT * FROM t WHERE a = ? AND b = ?", + DBType.MSSQL, + ), + ).toBe("SELECT * FROM t WHERE a = @param0 AND b = @param1"); + }); + + it("numbers a longer statement consecutively", () => { + const rewritten = rewritePlaceholders( + "INSERT INTO t (a, b, c, d, e) VALUES (?, ?, ?, ?, ?)", + DBType.MSSQL, + ); + expect(rewritten).toBe( + "INSERT INTO t (a, b, c, d, e) VALUES (@param0, @param1, @param2, @param3, @param4)", + ); + expect(rewritten.match(/@param\d+/g)).toEqual([ + "@param0", + "@param1", + "@param2", + "@param3", + "@param4", + ]); + }); + + it("leaves a statement with no placeholders alone", () => { + expect(rewritePlaceholders("SELECT 1 AS ok", DBType.MSSQL)).toBe( + "SELECT 1 AS ok", + ); + }); + + it("leaves PostgreSQL numbering unchanged", () => { + expect( + rewritePlaceholders("SELECT * FROM t WHERE a = ? AND b = ?", DBType.Postgres), + ).toBe("SELECT * FROM t WHERE a = $1 AND b = $2"); + }); + + it("leaves MySQL and SQLite statements unchanged", () => { + const query = "SELECT * FROM t WHERE a = ?"; + expect(rewritePlaceholders(query, DBType.MySQL)).toBe(query); + expect(rewritePlaceholders(query, DBType.SQLite)).toBe(query); + }); +}); + +describe("bindMSSQLParam", () => { + /** Stands in for an `sql.Request`, recording how each input was bound. */ + function recorder() { + const calls: { name: string; args: any[] }[] = []; + return { + calls, + input(name: string, ...args: any[]) { + calls.push({ name, args }); + return this; + }, + }; + } + + it("binds a value positionally", () => { + const request = recorder(); + bindMSSQLParam(request, 0, 42); + bindMSSQLParam(request, 1, "hello"); + + expect(request.calls).toEqual([ + { name: "param0", args: [42] }, + { name: "param1", args: ["hello"] }, + ]); + }); + + it("binds null and undefined as an explicitly typed NULL", () => { + const request = recorder(); + bindMSSQLParam(request, 0, null); + bindMSSQLParam(request, 1, undefined); + + expect(request.calls).toEqual([ + { name: "param0", args: [sql.NVarChar, null] }, + { name: "param1", args: [sql.NVarChar, null] }, + ]); + }); + + it("does not add a type for falsy-but-present values", () => { + const request = recorder(); + bindMSSQLParam(request, 0, 0); + bindMSSQLParam(request, 1, false); + bindMSSQLParam(request, 2, ""); + + expect(request.calls).toEqual([ + { name: "param0", args: [0] }, + { name: "param1", args: [false] }, + { name: "param2", args: [""] }, + ]); + }); +}); + +describe("buildLimitClause for SQL Server", () => { + it("emits nothing when neither limit nor offset was set", () => { + expect(buildLimitClause(null, null, false, DBType.MSSQL)).toBe(""); + expect(buildLimitClause(null, null, true, DBType.MSSQL)).toBe(""); + }); + + it("adds a constant ORDER BY when the statement has none", () => { + expect(buildLimitClause(5, null, false, DBType.MSSQL)).toBe( + "\nORDER BY (SELECT NULL)\nOFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY", + ); + }); + + it("keeps an existing ORDER BY and skips no rows for a bare limit", () => { + expect(buildLimitClause(5, null, true, DBType.MSSQL)).toBe( + "\nOFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY", + ); + }); + + it("emits OFFSET alone when only an offset was set", () => { + expect(buildLimitClause(null, 10, false, DBType.MSSQL)).toBe( + "\nORDER BY (SELECT NULL)\nOFFSET 10 ROWS", + ); + expect(buildLimitClause(null, 10, true, DBType.MSSQL)).toBe( + "\nOFFSET 10 ROWS", + ); + }); + + it("emits OFFSET and FETCH when both were set", () => { + expect(buildLimitClause(5, 10, false, DBType.MSSQL)).toBe( + "\nORDER BY (SELECT NULL)\nOFFSET 10 ROWS FETCH NEXT 5 ROWS ONLY", + ); + expect(buildLimitClause(5, 10, true, DBType.MSSQL)).toBe( + "\nOFFSET 10 ROWS FETCH NEXT 5 ROWS ONLY", + ); + }); + + it("never emits an ORDER BY fallback for the LIMIT dialects", () => { + for (const dialect of [DBType.SQLite, DBType.MySQL, DBType.Postgres]) { + expect(buildLimitClause(5, null, false, dialect)).toBe("\nLIMIT 5"); + expect(buildLimitClause(null, 10, false, dialect)).toBe( + "\nLIMIT 9223372036854775807 OFFSET 10", + ); + expect(buildLimitClause(5, 10, false, dialect)).toBe( + "\nLIMIT 5 OFFSET 10", + ); + expect(buildLimitClause(null, null, false, dialect)).toBe(""); + } + }); + + it("defaults to the LIMIT form when no dialect is given", () => { + expect(buildLimitClause(5, 10, false)).toBe("\nLIMIT 5 OFFSET 10"); + }); +}); + +describe("QueryBuilder row limiting", () => { + it("renders OFFSET/FETCH when built for SQL Server", () => { + const qb = new QueryBuilder("users").limit(5); + expect(qb.build(DBType.MSSQL).query).toBe( + "SELECT * FROM users\nORDER BY (SELECT NULL)\nOFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY", + ); + }); + + it("renders the LIMIT form when no dialect is given", () => { + const qb = new QueryBuilder("users").limit(5); + expect(qb.build().query).toBe("SELECT * FROM users\nLIMIT 5"); + expect(qb.toSQL().query).toBe("SELECT * FROM users\nLIMIT 5"); + }); + + it("keeps an explicit ORDER BY out of the fallback", () => { + const qb = new QueryBuilder("users").orderBy("id").limit(5).offset(2); + expect(qb.build(DBType.MSSQL).query).toBe( + "SELECT * FROM users\nORDER BY id ASC\nOFFSET 2 ROWS FETCH NEXT 5 ROWS ONLY", + ); + }); + + it("holds the dialect set by withDialect", () => { + const qb = new QueryBuilder("users").limit(1).withDialect(DBType.MSSQL); + expect(qb.build().query).toContain("OFFSET 0 ROWS FETCH NEXT 1 ROWS ONLY"); + }); + + it("takes the dialect from the client it executes against", async () => { + const sent: string[] = []; + const stubClient = { + config: { type: DBType.MSSQL }, + query: async (query: string) => { + sent.push(query); + return []; + }, + }; + + await new QueryBuilder("users").limit(1).execute(stubClient as any); + + expect(sent).toHaveLength(1); + expect(sent[0]).toBe( + "SELECT * FROM users\nORDER BY (SELECT NULL)\nOFFSET 0 ROWS FETCH NEXT 1 ROWS ONLY", + ); + }); +}); + +describe("buildMSSQLUpsertSQL", () => { + const columns = ["id", "email", "name"]; + + it("binds each value exactly once, in the USING clause", () => { + const statement = buildMSSQLUpsertSQL("users", columns, ["email"]); + expect(statement.match(/\?/g) ?? []).toHaveLength(columns.length); + }); + + it("matches on the key columns and updates the rest", () => { + const statement = buildMSSQLUpsertSQL("users", columns, ["email"]); + expect(statement).toContain("MERGE INTO users AS target"); + expect(statement).toContain( + "USING (SELECT ? AS id, ? AS email, ? AS name) AS source", + ); + expect(statement).toContain("ON (target.email = source.email)"); + expect(statement).toContain( + "WHEN MATCHED THEN UPDATE SET target.id = source.id, target.name = source.name", + ); + expect(statement).toContain( + "WHEN NOT MATCHED THEN INSERT (id, email, name) VALUES (source.id, source.email, source.name)", + ); + expect(statement).toContain("OUTPUT INSERTED.*;"); + }); + + it("matches on several keys", () => { + const statement = buildMSSQLUpsertSQL("users", columns, ["id", "email"]); + expect(statement).toContain( + "ON (target.id = source.id AND target.email = source.email)", + ); + expect(statement).toContain( + "WHEN MATCHED THEN UPDATE SET target.name = source.name", + ); + }); + + it("terminates the statement, as T-SQL requires", () => { + expect(buildMSSQLUpsertSQL("users", columns, ["email"]).endsWith(";")).toBe( + true, + ); + }); + + it("inserts alone when there is no key to match on", () => { + const statement = buildMSSQLUpsertSQL("users", ["id", "name"], []); + expect(statement).toBe( + "INSERT INTO users (id, name) OUTPUT INSERTED.* VALUES (?, ?)", + ); + expect(statement.match(/\?/g) ?? []).toHaveLength(2); + }); +}); + +describe("createTableIfNotExistsSQL", () => { + it("uses IF NOT EXISTS for the three original dialects", () => { + for (const dialect of [DBType.SQLite, DBType.MySQL, DBType.Postgres]) { + expect(createTableIfNotExistsSQL("users", "id INT", dialect)).toBe( + "CREATE TABLE IF NOT EXISTS users (id INT)", + ); + } + }); + + it("uses an existence check for SQL Server", () => { + expect(createTableIfNotExistsSQL("users", "id INT", DBType.MSSQL)).toBe( + "IF OBJECT_ID(N'users', N'U') IS NULL CREATE TABLE users (id INT)", + ); + }); + + it("unquotes the identifier for the existence check only", () => { + expect( + createTableIfNotExistsSQL('"users"', "id INT", DBType.MSSQL), + ).toBe( + "IF OBJECT_ID(N'users', N'U') IS NULL CREATE TABLE \"users\" (id INT)", + ); + }); +}); + +describe("createIndexIfNotExistsSQL", () => { + it("uses IF NOT EXISTS for the three original dialects", () => { + expect( + createIndexIfNotExistsSQL('"i"', '"t"', ['"c"'], true, DBType.SQLite), + ).toBe('CREATE UNIQUE INDEX IF NOT EXISTS "i" ON "t" ("c")'); + expect( + createIndexIfNotExistsSQL('"i"', '"t"', ['"c"'], false, DBType.Postgres), + ).toBe('CREATE INDEX IF NOT EXISTS "i" ON "t" ("c")'); + }); + + it("checks sys.indexes for SQL Server", () => { + expect( + createIndexIfNotExistsSQL('"t_c_uniq"', '"t"', ['"c"'], true, DBType.MSSQL), + ).toBe( + "IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = N't_c_uniq' AND object_id = OBJECT_ID(N't')) CREATE UNIQUE INDEX \"t_c_uniq\" ON \"t\" (\"c\")", + ); + }); +}); + +describe("generateMigration for SQL Server", () => { + it("generates an IDENTITY primary key", async () => { + const migration = await generateMigration( + mssqlModel, + "create_mssql_widgets", + DBType.MSSQL, + ); + + expect(migration.up[0]).toBe( + "IF OBJECT_ID(N'mssql_widgets', N'U') IS NULL CREATE TABLE mssql_widgets (id INT IDENTITY(1,1) PRIMARY KEY, label NVARCHAR(255) NOT NULL UNIQUE)", + ); + }); + + it("maps a UUID primary key to UNIQUEIDENTIFIER", async () => { + const migration = await generateMigration( + mssqlUuidModel, + "create_mssql_gadgets", + DBType.MSSQL, + ); + + expect(migration.up[0]).toContain( + "id UNIQUEIDENTIFIER PRIMARY KEY", + ); + expect(migration.up[0]).toContain("label NVARCHAR(255) NOT NULL"); + }); + + it("keeps the three original dialects byte-identical", async () => { + const sqlite = await generateMigration( + mssqlModel, + "create", + DBType.SQLite, + ); + expect(sqlite.up[0]).toBe( + "CREATE TABLE IF NOT EXISTS mssql_widgets (id INTEGER PRIMARY KEY AUTOINCREMENT, label TEXT NOT NULL UNIQUE)", + ); + + const postgres = await generateMigration( + mssqlModel, + "create", + DBType.Postgres, + ); + expect(postgres.up[0]).toBe( + "CREATE TABLE IF NOT EXISTS mssql_widgets (id SERIAL PRIMARY KEY, label TEXT NOT NULL UNIQUE)", + ); + + const mysql = await generateMigration(mssqlModel, "create", DBType.MySQL); + expect(mysql.up[0]).toBe( + "CREATE TABLE IF NOT EXISTS mssql_widgets (id INT AUTO_INCREMENT PRIMARY KEY, label VARCHAR(255) NOT NULL UNIQUE)", + ); + expect(mysql.down[0]).toBe("DROP TABLE IF EXISTS mssql_widgets"); + }); +}); diff --git a/tests/repository.cache-keys.test.ts b/tests/repository.cache-keys.test.ts new file mode 100644 index 0000000..4f66819 --- /dev/null +++ b/tests/repository.cache-keys.test.ts @@ -0,0 +1,164 @@ +import { describe, it, expect, beforeEach } from "vitest"; +import { Repository } from "../repository"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * The cache is write-through for single rows, so the key a read looks up must + * be the key a write stores and the key a write invalidates. + * + * These tests pin that agreement. The read key used to carry a + * `:${relations.join(",")}` suffix, which rendered as the literal string + * `undefined` for the common no-relations call — so every write-through `set` + * and every `invalidate` addressed a key that no read ever consulted, and a + * stale row was served until its TTL expired. + */ + +const User = defineModel({ + tableName: "users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +/** Records every cache interaction and serves back whatever was stored. */ +class RecordingCache { + public gets: string[] = []; + public sets: { key: string; value: any }[] = []; + public invalidated: string[] = []; + public patterns: string[] = []; + private store = new Map(); + public config = { enabled: true, ttl: 60, strategy: "write-through" as const }; + + getStrategy() { + return this.config.strategy; + } + + async get(key: string): Promise { + this.gets.push(key); + return (this.store.get(key) as T) ?? null; + } + + async set(key: string, value: T): Promise { + this.sets.push({ key, value }); + this.store.set(key, value); + } + + async invalidate(keys: string[]): Promise { + this.invalidated.push(...keys); + for (const key of keys) this.store.delete(key); + } + + async invalidatePattern(pattern: string): Promise { + this.patterns.push(pattern); + const prefix = pattern.replace(/\*$/, ""); + for (const key of [...this.store.keys()]) { + if (key.startsWith(prefix)) this.store.delete(key); + } + } +} + +/** Minimal client: `findOne` only needs `query` to answer the SELECT. */ +function makeClient(rows: any[] = []) { + return { + config: { type: DBType.SQLite }, + queries: [] as string[], + async query(sql: string) { + this.queries.push(sql); + return rows; + }, + async queryExec() { + return { affectedRows: 1 }; + }, + async transaction(fn: (c: any) => Promise): Promise { + return fn(this); + }, + } as any; +} + +describe("repository cache keys", () => { + let cache: RecordingCache; + let repo: any; + + beforeEach(() => { + cache = new RecordingCache(); + repo = new Repository(makeClient([{ id: 5, name: "Ada" }]), User); + repo.cache = cache; + }); + + it("reads and write-through-stores the same key when no relations are loaded", async () => { + await repo.findOne(5); + await repo.writeThroughRow(5, { id: 5, name: "Ada" }); + + const readKey = cache.gets[0]; + const writtenKey = cache.sets[0].key; + + expect(readKey).toBe("findOne:users:5"); + expect(writtenKey).toBe(readKey); + // The bug: a literal "undefined" segment made the two disagree. + expect(readKey).not.toContain("undefined"); + }); + + it("serves a write-through row back to the next read", async () => { + await repo.writeThroughRow(5, { id: 5, name: "Ada" }); + + const found = await repo.findOne(5); + + expect(found).toEqual({ id: 5, name: "Ada" }); + }); + + it("invalidates the key that reads use", async () => { + await repo.writeThroughRow(5, { id: 5, name: "Ada" }); + await repo.invalidateRowCache(5); + + expect(cache.invalidated).toContain("findOne:users:5"); + expect(await repo.findOne(5)).toEqual({ id: 5, name: "Ada" }); + // Re-read after invalidation must be a miss, then fall through to the DB. + expect(cache.gets).toContain("findOne:users:5"); + }); + + it("keys relation-loaded reads apart, but deterministically", async () => { + const plain = repo.rowCacheKey(5); + const withPosts = repo.rowCacheKey(5, ["posts"]); + const withTags = repo.rowCacheKey(5, ["tags"]); + + expect(plain).toBe("findOne:users:5"); + expect(withPosts).toBe("findOne:users:5:posts"); + expect(withTags).toBe("findOne:users:5:tags"); + + // Relation order must not produce two entries for the same result shape. + expect(repo.rowCacheKey(5, ["tags", "posts"])).toBe( + repo.rowCacheKey(5, ["posts", "tags"]), + ); + expect(repo.rowCacheKey(5, [])).toBe(plain); + }); + + it("clears relation-loaded variants when a row is invalidated", async () => { + await cache.set(repo.rowCacheKey(5, ["posts"]), [{ id: 5 }]); + await repo.invalidateRowCache(5); + + expect( + cache.patterns.some((p: string) => p.startsWith("findOne:users:5")), + ).toBe(true); + expect(await cache.get(repo.rowCacheKey(5, ["posts"]))).toBeNull(); + }); + + it("clears per-row entries on a multi-row write", async () => { + await cache.set(repo.rowCacheKey(5), [{ id: 5 }]); + await repo.invalidateTableCache(); + + expect(await cache.get(repo.rowCacheKey(5))).toBeNull(); + }); + + it("does not cache a read taken inside a transaction", async () => { + const txClient = makeClient([{ id: 5, name: "Ada" }]); + txClient.isTransactionClient = true; + + await repo.findOne(5, {}, txClient); + + expect(cache.gets).toEqual([]); + expect(cache.sets).toEqual([]); + }); +}); diff --git a/tests/repository.features.test.ts b/tests/repository.features.test.ts new file mode 100644 index 0000000..d136af5 --- /dev/null +++ b/tests/repository.features.test.ts @@ -0,0 +1,348 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { QueryBuilder, Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; + +/** + * Coverage for the features added on top of the audit fixes: aggregate + * validation, the `*OrFail` lookups, many-to-many link management, and the raw + * clause builders. + */ + +const Signup = defineModel({ + tableName: "feat_signups", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + email: { + type: DataTypes.STRING, + required: true, + pattern: /^[^@\s]+@[^@\s]+$/, + }, + name: { type: DataTypes.STRING, required: true, minLength: 3 }, + age: { + type: DataTypes.INTEGER, + customValidator: (value: any) => + value >= 18 ? true : "Signups must be 18 or over", + }, + }, +}); + +const Author = defineModel({ + tableName: "feat_authors", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true }, + }, +}); + +const Tag = defineModel({ + tableName: "feat_tags", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING, required: true }, + }, +}); + +const Post = defineModel({ + tableName: "feat_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + authorId: { type: DataTypes.INTEGER, name: "author_id" }, + }, + relations: [ + { + type: RelationType.ManyToMany, + target: () => Tag, + property: "tags", + joinTable: "feat_post_tags", + foreignKey: "post_id", + inverseKey: "tag_id", + }, + { + type: RelationType.ManyToOne, + target: () => Author, + property: "author", + foreignKey: "authorId", + }, + ], +}); + +describe("validation and lookup features", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Signup, Author, Tag, Post]); + await db.rawExec( + "CREATE TABLE feat_post_tags (post_id INTEGER, tag_id INTEGER)", + ); + await db.rawExec("INSERT INTO feat_authors (id, name) VALUES (1, 'Ada')"); + await db.rawExec( + "INSERT INTO feat_posts (id, title, author_id) VALUES (10, 'First', 1), (11, 'Second', NULL)", + ); + await db.rawExec( + "INSERT INTO feat_tags (id, label) VALUES (1, 'alpha'), (2, 'beta'), (3, 'gamma')", + ); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("reports every validation failure, not just the first", async () => { + const repo = db.getRepository(Signup); + + // Three separate problems, each of which the throwing form would hide + // behind whichever happened to be checked first. + expect(repo.validateAll({ email: "nope", name: "ab", age: 12 })).toEqual([ + "Field email does not match pattern", + "Field name too short", + "Signups must be 18 or over", + ]); + }); + + it("reports missing required fields", async () => { + expect(db.getRepository(Signup).validateAll({})).toEqual([ + "Field email is required", + "Field name is required", + ]); + }); + + it("returns no errors for a valid entity", async () => { + expect( + db + .getRepository(Signup) + .validateAll({ email: "a@b.c", name: "Ada", age: 30 }), + ).toEqual([]); + }); + + it("can skip the required rules, as an update does", async () => { + // A partial update supplies only the fields it changes. + expect(db.getRepository(Signup).validateAll({ age: 30 }, true)).toEqual([]); + }); + + it("still stops at the first failure when writing", async () => { + // The write path must not change behaviour: it throws, as before. + await expect( + db.getRepository(Signup).create({ email: "nope", name: "ab" }), + ).rejects.toThrow("Field email does not match pattern"); + }); + + it("findOrFail returns the row when it exists", async () => { + const post = await db.getRepository(Post).findOrFail(10); + expect(post.title).toBe("First"); + }); + + it("findOrFail throws a named error when it does not", async () => { + await expect(db.getRepository(Post).findOrFail(999)).rejects.toThrow( + /feat_posts with id 999 not found/, + ); + + // The code is what a caller branches on to turn a miss into a 404. + await expect(db.getRepository(Post).findOrFail(999)).rejects.toMatchObject( + { code: "NOT_FOUND_ERROR" }, + ); + }); + + it("findOrFail still loads relations on the way", async () => { + const post: any = await db + .getRepository(Post) + .findOrFail(10, { relations: ["author"] }); + expect(post.author.name).toBe("Ada"); + }); + + it("firstOrFail finds a match or throws", async () => { + const found = await db + .getRepository(Post) + .firstOrFail({ title: "Second" }); + expect(found.id).toBe(11); + + await expect( + db.getRepository(Post).firstOrFail({ title: "Nothing" }), + ).rejects.toMatchObject({ code: "NOT_FOUND_ERROR" }); + }); +}); + +describe("many-to-many link management", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Post, Tag, Author]); + await db.rawExec( + "CREATE TABLE feat_post_tags (post_id INTEGER, tag_id INTEGER)", + ); + await db.rawExec( + "INSERT INTO feat_posts (id, title) VALUES (20, 'Linked'), (21, 'Other')", + ); + await db.rawExec( + "INSERT INTO feat_tags (id, label) VALUES (1, 'a'), (2, 'b'), (3, 'c')", + ); + }); + + afterAll(async () => { + await db?.close(); + }); + + const labels = async (id: number) => + (await db.getRepository(Post).findOne(id, { relations: ["tags"] })).tags + .map((t: any) => t.label) + .sort(); + + it("attaches links", async () => { + const repo = db.getRepository(Post); + expect(await repo.attach(20, "tags", [1, 2])).toBe(2); + expect(await labels(20)).toEqual(["a", "b"]); + }); + + it("attaching the same target twice is a no-op", async () => { + const repo = db.getRepository(Post); + expect(await repo.attach(20, "tags", [2, 3])).toBe(1); + expect(await labels(20)).toEqual(["a", "b", "c"]); + }); + + it("matches a string id against a numeric one", async () => { + // Ids arrive as strings from a URL and as numbers from a read; treating + // those as different would insert the same link a second time. + const repo = db.getRepository(Post); + expect(await repo.attach(20, "tags", "1")).toBe(0); + expect(await labels(20)).toEqual(["a", "b", "c"]); + }); + + it("accepts a single id as well as a list", async () => { + const repo = db.getRepository(Post); + expect(await repo.attach(21, "tags", 1)).toBe(1); + expect(await labels(21)).toEqual(["a"]); + }); + + it("detaches specific links", async () => { + const repo = db.getRepository(Post); + expect(await repo.detach(20, "tags", [2])).toBe(1); + expect(await labels(20)).toEqual(["a", "c"]); + }); + + it("detaches every link when no targets are given", async () => { + const repo = db.getRepository(Post); + expect(await repo.detach(20, "tags")).toBe(2); + expect(await labels(20)).toEqual([]); + + // Only the named owner is affected. + expect(await labels(21)).toEqual(["a"]); + }); + + it("detaching from an owner with no links removes nothing", async () => { + expect(await db.getRepository(Post).detach(20, "tags")).toBe(0); + }); + + it("syncs to exactly the given set", async () => { + const repo = db.getRepository(Post); + + // From {1} to {2, 3}: one link removed, two added. + expect(await repo.sync(21, "tags", [2, 3])).toEqual({ + attached: 2, + detached: 1, + }); + expect(await labels(21)).toEqual(["b", "c"]); + }); + + it("sync leaves an already-correct set untouched", async () => { + const repo = db.getRepository(Post); + + // Nothing to do, so nothing is written. + expect(await repo.sync(21, "tags", [3, 2])).toEqual({ + attached: 0, + detached: 0, + }); + expect(await labels(21)).toEqual(["b", "c"]); + }); + + it("sync can clear the relation entirely", async () => { + const repo = db.getRepository(Post); + expect(await repo.sync(21, "tags", [])).toEqual({ + attached: 0, + detached: 2, + }); + expect(await labels(21)).toEqual([]); + }); + + it("sync ignores a repeated id in the list", async () => { + const repo = db.getRepository(Post); + expect(await repo.sync(21, "tags", [1, 1])).toEqual({ + attached: 1, + detached: 0, + }); + expect(await labels(21)).toEqual(["a"]); + }); + + it("refuses link management on a relation that is not many-to-many", async () => { + // Silently doing nothing would leave the caller believing it worked. + await expect( + db.getRepository(Post).attach(20, "author", [1]), + ).rejects.toThrow(/needs a ManyToMany relation/); + }); + + it("rejects an unknown relation name", async () => { + await expect( + db.getRepository(Post).attach(20, "nope", [1]), + ).rejects.toThrow(/Relation nope not found/); + }); +}); + +describe("raw clause builders", () => { + it("orders by an expression rather than a column name", () => { + const { query } = new QueryBuilder("posts") + .orderByRaw("CASE WHEN status = 'urgent' THEN 0 ELSE 1 END") + .toSQL(); + + expect(query).toContain( + "ORDER BY CASE WHEN status = 'urgent' THEN 0 ELSE 1 END", + ); + }); + + it("appends a direction to a raw order", () => { + const { query } = new QueryBuilder("posts") + .orderByRaw("LENGTH(title)", "DESC") + .toSQL(); + + expect(query).toContain("ORDER BY LENGTH(title) DESC"); + }); + + it("groups by an expression", () => { + const { query } = new QueryBuilder("posts") + .select("strftime('%Y-%m', createdAt) AS month", "COUNT(*) AS n") + .groupByRaw("strftime('%Y-%m', createdAt)") + .toSQL(); + + expect(query).toContain("GROUP BY strftime('%Y-%m', createdAt)"); + }); + + it("emits HAVING after GROUP BY, with its parameters in order", () => { + const { query, params } = new QueryBuilder("posts") + .select("status") + .where("author_id = ?", 7) + .groupBy("status") + .havingRaw("COUNT(*) > ?", 2) + .toSQL(); + + // Clause order has to be GROUP BY then HAVING, and the bound values must + // follow the same order as their placeholders. + expect(query.indexOf("GROUP BY")).toBeLessThan(query.indexOf("HAVING")); + expect(query).toContain("HAVING COUNT(*) > ?"); + expect(params).toEqual([7, 2]); + }); + + it("combines raw and plain modifiers", () => { + const { query } = new QueryBuilder("posts") + .groupBy("status") + .groupByRaw("strftime('%Y', createdAt)") + .orderByRaw("COUNT(*) DESC") + .orderBy("title") + .toSQL(); + + expect(query).toContain( + "GROUP BY status, strftime('%Y', createdAt)", + ); + expect(query).toContain("ORDER BY COUNT(*) DESC, title ASC"); + }); +}); diff --git a/tests/repository.transaction-client.test.ts b/tests/repository.transaction-client.test.ts new file mode 100644 index 0000000..b902286 --- /dev/null +++ b/tests/repository.transaction-client.test.ts @@ -0,0 +1,102 @@ +import { describe, it, expect, beforeEach } from "vitest"; +import { Repository } from "../repository"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * Writes issued through a repository must run on the client they were given. + * + * `stabilize.transaction(cb)` hands the callback a *separate* client bound to + * one pooled connection. On PostgreSQL and MySQL the repository used to reach + * for its own root client instead, so a write inside a transaction ran on a + * different connection and survived the rollback. + */ + +const User = defineModel({ + tableName: "users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + logins: { type: DataTypes.INTEGER }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +/** A client that records every statement it is asked to run. */ +function makeClient(rows: any[] = []) { + return { + config: { type: DBType.Postgres }, + isTransactionClient: false, + statements: [] as string[], + execs: [] as string[], + async query(sql: string) { + this.statements.push(sql); + return rows; + }, + async queryExec(sql: string) { + this.execs.push(sql); + return { affectedRows: 1 }; + }, + async transaction(fn: (c: any) => Promise): Promise { + return fn(this); + }, + } as any; +} + +describe("repository write routing", () => { + let root: any; + let tx: any; + let repo: any; + + beforeEach(() => { + root = makeClient([{ id: 1, name: "Ada" }]); + tx = makeClient([{ id: 1, name: "Ada" }]); + tx.isTransactionClient = true; + repo = new Repository(root, User); + repo.cache = null; + }); + + it("creates on the supplied transaction client, not the root client", async () => { + await repo.create({ id: 1, name: "Ada" }, {}, tx); + + expect(tx.statements.length).toBeGreaterThan(0); + expect(root.statements).toEqual([]); + expect(root.execs).toEqual([]); + }); + + it("updates on the supplied transaction client", async () => { + await repo.update(1, { name: "Ada L" }, tx); + + expect(tx.statements.length).toBeGreaterThan(0); + expect(root.statements).toEqual([]); + }); + + it("deletes on the supplied transaction client", async () => { + await repo.delete(1, tx); + + expect(tx.execs.length + tx.statements.length).toBeGreaterThan(0); + expect(root.execs).toEqual([]); + expect(root.statements).toEqual([]); + }); + + it("runs bulk and condition-based writes on the supplied client", async () => { + await repo.updateBy({ name: "Ada" }, { name: "Ada L" }, tx); + await repo.deleteBy({ name: "Ada" }, tx); + await repo.restoreBy({ name: "Ada" }, tx); + await repo.increment(1, "logins", 1, tx); + await repo.decrement(1, "logins", 1, tx); + await repo.toggle(1, "logins", tx); + await repo.truncate(tx); + + expect(root.execs).toEqual([]); + expect(root.statements).toEqual([]); + expect(tx.execs.length).toBeGreaterThanOrEqual(7); + }); + + it("still uses its own client when none is supplied", async () => { + await repo.updateBy({ name: "Ada" }, { name: "Ada L" }); + + expect(root.execs.length).toBe(1); + expect(tx.execs).toEqual([]); + }); +}); diff --git a/tests/repository.validation.test.ts b/tests/repository.validation.test.ts index c647866..309fc36 100644 --- a/tests/repository.validation.test.ts +++ b/tests/repository.validation.test.ts @@ -1,17 +1,18 @@ -import { describe, it, expect } from 'vitest'; -import { Repository } from '../repository'; -import { defineModel } from '../model'; -import { DataTypes, DBType } from '../types'; +import { describe, it, expect } from "vitest"; +import { Repository } from "../repository"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; -describe('Repository validation metadata mapping', () => { +describe("Repository validation metadata mapping", () => { const User = defineModel({ - tableName: 'users', + tableName: "users", columns: { id: { type: DataTypes.INTEGER, required: true }, username: { type: DataTypes.STRING, minLength: 3 }, email: { type: DataTypes.STRING, - customValidator: (val: string) => val.endsWith('@example.com') || 'Invalid email domain', + customValidator: (val: string) => + val.endsWith("@example.com") || "Invalid email domain", }, }, }); @@ -20,15 +21,19 @@ describe('Repository validation metadata mapping', () => { config: { type: DBType.SQLite }, }; - it('should enforce minLength from model column metadata', () => { + it("should enforce minLength from model column metadata", () => { const repo = new Repository(fakeClient, User); - expect(() => (repo as any).validate({ id: 1, username: 'ab' })).toThrow('too short'); + expect(() => (repo as any).validate({ id: 1, username: "ab" })).toThrow( + "too short", + ); }); - it('should enforce customValidator from model column metadata', () => { + it("should enforce customValidator from model column metadata", () => { const repo = new Repository(fakeClient, User); - expect(() => (repo as any).validate({ id: 1, email: 'test@not-example.com' })).toThrow('Invalid email domain'); + expect(() => + (repo as any).validate({ id: 1, email: "test@not-example.com" }), + ).toThrow("Invalid email domain"); }); }); diff --git a/types.ts b/types.ts index 8fb29f4..170b5b2 100644 --- a/types.ts +++ b/types.ts @@ -8,6 +8,7 @@ export enum DBType { Postgres = "postgres", MySQL = "mysql", SQLite = "sqlite", + MSSQL = "mssql", } export enum LogLevel { diff --git a/utils/encryption.ts b/utils/encryption.ts index 7cdd21a..8cd258a 100644 --- a/utils/encryption.ts +++ b/utils/encryption.ts @@ -1,56 +1,131 @@ import crypto from "crypto"; -const ENCRYPTION_KEY = - process.env.ORM_ENCRYPTION_KEY || "f71a3c8e9b12d5a49c0a3f98b1f2e46d"; // 32 bytes for AES-256 -const IV_LENGTH = 16; // AES block size in bytes +/** + * The environment variable that holds the encryption key. + * + * There is deliberately no default. The key used to fall back to a constant + * written into this file, so every deployment that never set the variable + * encrypted its columns with a value anyone who read the published package + * could reproduce — data that looked confidential and offered none of the + * protection. A missing key is a deployment mistake, and it now fails loudly + * instead of quietly protecting nothing. + */ +const KEY_ENV_VAR = "ORM_ENCRYPTION_KEY"; + +/** The key previously hard-coded here, kept only to explain the migration. */ +const LEGACY_KEY = "f71a3c8e9b12d5a49c0a3f98b1f2e46d"; + +/** IV length for AES-GCM, which is what `encrypt` writes today. */ +const GCM_IV_LENGTH = 12; +/** AES block size — the IV length of the legacy CBC format. */ +const CBC_IV_LENGTH = 16; +/** Prefix identifying the authenticated format. */ +const GCM_PREFIX = "v2"; + +/** + * Resolves the configured key. + * + * Read on every call rather than captured at module load, so an application + * that sets the variable after importing still gets it. + */ +function getKey(): Buffer { + const configured = process.env[KEY_ENV_VAR]; + if (!configured) { + throw new Error( + `${KEY_ENV_VAR} is not set, so encrypted columns cannot be read or written. ` + + `Set it to 32 bytes (or 64 hex characters). Data written before this check ` + + `existed used a hard-coded key: set ${KEY_ENV_VAR}="${LEGACY_KEY}" to keep ` + + `reading it, then re-save those rows under a key of your own.`, + ); + } + + const key = /^[0-9a-fA-F]{64}$/.test(configured) + ? Buffer.from(configured, "hex") + : Buffer.from(configured, "utf8"); + + if (key.length !== 32) { + throw new Error( + `${KEY_ENV_VAR} must be 32 bytes (256 bits) or 64 hex characters; got ${key.length} bytes.`, + ); + } + return key; +} /** - * Encrypts a UTF-8 string using AES-256-CBC. + * Encrypts a UTF-8 string with AES-256-GCM. + * + * GCM rather than the CBC this used to use: GCM authenticates the ciphertext, + * so a value that was tampered with or truncated fails to decrypt instead of + * silently yielding corrupted plaintext. + * * @param text - The plain text to encrypt. - * @returns {string} Base64-encoded string in the format "iv:encrypted". + * @returns `v2:::`, each part Base64. */ export function encrypt(text: string): string { - // Convert the encryption key properly (handle hex vs utf8) - const keyBuffer = Buffer.from(ENCRYPTION_KEY, "utf8"); - if (keyBuffer.length !== 32) { - throw new Error("ENCRYPTION_KEY must be 32 bytes (256 bits) long for AES-256"); - } - - const iv = crypto.randomBytes(IV_LENGTH); - const cipher = crypto.createCipheriv("aes-256-cbc", keyBuffer, iv); + const key = getKey(); + const iv = crypto.randomBytes(GCM_IV_LENGTH); + const cipher = crypto.createCipheriv("aes-256-gcm", key, iv); - const encrypted = Buffer.concat([ - cipher.update(text, "utf8"), - cipher.final(), - ]); + const encrypted = Buffer.concat([ + cipher.update(text, "utf8"), + cipher.final(), + ]); - return `${iv.toString("base64")}:${encrypted.toString("base64")}`; + return [ + GCM_PREFIX, + iv.toString("base64"), + cipher.getAuthTag().toString("base64"), + encrypted.toString("base64"), + ].join(":"); } /** - * Decrypts a string encrypted with `encrypt()`. - * @param text - The Base64-encoded "iv:encrypted" string. - * @returns {string} The decrypted plain text. + * Decrypts a string produced by {@link encrypt}. + * + * Also reads the unauthenticated `iv:ciphertext` CBC format written by earlier + * versions, so existing rows stay readable. Nothing writes that format any + * more. + * + * @param text - The encrypted value. + * @returns The decrypted plain text. */ export function decrypt(text: string): string { - const [ivPart, encryptedPart] = text.split(":"); - if (!ivPart || !encryptedPart) { - throw new Error("Invalid encrypted text format. Expected 'iv:encrypted'."); - } - - const iv = Buffer.from(ivPart, "base64"); - const encrypted = Buffer.from(encryptedPart, "base64"); - - const keyBuffer = Buffer.from(ENCRYPTION_KEY, "utf8"); - if (keyBuffer.length !== 32) { - throw new Error("ENCRYPTION_KEY must be 32 bytes (256 bits) long for AES-256"); - } - - const decipher = crypto.createDecipheriv("aes-256-cbc", keyBuffer, iv); - const decrypted = Buffer.concat([ - decipher.update(encrypted), - decipher.final(), - ]); - - return decrypted.toString("utf8"); + const key = getKey(); + const parts = text.split(":"); + + if (parts[0] === GCM_PREFIX && parts.length === 4) { + const [, ivPart, tagPart, dataPart] = parts; + const decipher = crypto.createDecipheriv( + "aes-256-gcm", + key, + Buffer.from(ivPart!, "base64"), + ); + decipher.setAuthTag(Buffer.from(tagPart!, "base64")); + return Buffer.concat([ + decipher.update(Buffer.from(dataPart!, "base64")), + decipher.final(), + ]).toString("utf8"); + } + + const [ivPart, dataPart] = parts; + if ( + parts.length !== 2 || + !ivPart || + !dataPart || + Buffer.from(ivPart, "base64").length !== CBC_IV_LENGTH + ) { + throw new Error( + `Invalid encrypted text format. Expected "${GCM_PREFIX}:iv:tag:ciphertext" or "iv:ciphertext".`, + ); + } + + const decipher = crypto.createDecipheriv( + "aes-256-cbc", + key, + Buffer.from(ivPart, "base64"), + ); + return Buffer.concat([ + decipher.update(Buffer.from(dataPart, "base64")), + decipher.final(), + ]).toString("utf8"); } From 681e341a8723f36123f16704db00856f2c9b62ca Mon Sep 17 00:00:00 2001 From: ElectronSz Date: Wed, 16 Sep 2026 10:54:12 +0200 Subject: [PATCH 2/5] Add a MongoDB backend: connection, translation layer, read path and schema MongoDB joins the four SQL dialects as a full backend rather than a side-car. The work is staged M0-M4 of the approved plan, each milestone green before the next began. M0 - connection lifecycle. The driver is an optional dependency and is imported lazily, so a project that only uses SQLite never needs it. A connect-time probe warns when the server is a standalone, because the ORM wraps every write in a transaction and a standalone rejects startTransaction - so the failure would otherwise appear only on the first create, with a driver message that names the rule but not the remedy. The Mongo handles are structural interfaces, so no driver type reaches an emitted .d.ts. M1 - mongo-query.ts, the pure translation layer. Structured predicates are recorded alongside the SQL fragments the existing methods already render, so the four SQL backends are untouched. Clauses that have no MongoDB equivalent (joins, CTEs, raw SQL) record a blocker and throw at execute() naming every offending method. M2 - the repository's ~24 raw where() calls became structured ones, so no SQL parsing is needed on the Mongo path. This surfaced four bugs where a lookup rendered the property key into SQL instead of the column name; only a renamed column could expose them, which is why tests/renamed-columns.test.ts builds models that have one and asserts each fix, with a companion assertion that the property-named column is genuinely absent so the test cannot pass for the wrong reason. M3 - the read path and integer auto-increment keys. Ids come from a stabilize_counters collection; one $inc reserves a whole bulkCreate batch, and a caller-supplied id advances the counter with $max so a later generated id cannot collide. Documents are keyed by _id. M4 - schema and migrations. Collections are created with a $jsonSchema validator and indexes. Two rules are load-bearing and are asserted against a real server: validationLevel "moderate", without which an update to a document lacking a newly declared required field is rejected, and sparse: true on unique indexes, without which the second document that omits a unique column collides where SQL would allow it. Migrations are data rather than closures, so a generated one is assertable without a server. Also fixes a bug the M4 tests found in shared code: the transaction error classifier rewrote every error as TX_ERROR, including errors the ORM raised itself, so a validation failure pointed the reader at the server's replica-set configuration. The four SQL branches all let their original error through; MongoDB now does too, while still recognising a standalone's rejection when the executor has already wrapped it. 380 pass / 0 fail, typecheck clean, build clean, no mongodb reference in dist/*.d.ts. --- auto-migrate.ts | 10 + bun.lock | 25 ++ client.ts | 719 ++++++++++++++++++++++++++++++- docker-compose.test.yml | 53 ++- index.ts | 15 +- migrations.ts | 8 + mongo-migrate.ts | 213 ++++++++++ mongo-query.ts | 720 ++++++++++++++++++++++++++++++++ mongo-repository.ts | 384 +++++++++++++++++ mongo-schema.ts | 469 +++++++++++++++++++++ package.json | 5 +- query-builder.ts | 426 ++++++++++++++++++- repository.ts | 165 +++++--- tests/integration.mongo.test.ts | 314 ++++++++++++++ tests/mongo.client.test.ts | 375 +++++++++++++++++ tests/mongo.dialect.test.ts | 609 +++++++++++++++++++++++++++ tests/mongo.migrations.test.ts | 449 ++++++++++++++++++++ tests/renamed-columns.test.ts | 235 +++++++++++ types.ts | 53 +++ 19 files changed, 5188 insertions(+), 59 deletions(-) create mode 100644 mongo-migrate.ts create mode 100644 mongo-query.ts create mode 100644 mongo-repository.ts create mode 100644 mongo-schema.ts create mode 100644 tests/integration.mongo.test.ts create mode 100644 tests/mongo.client.test.ts create mode 100644 tests/mongo.dialect.test.ts create mode 100644 tests/mongo.migrations.test.ts create mode 100644 tests/renamed-columns.test.ts diff --git a/auto-migrate.ts b/auto-migrate.ts index f644666..81a28c1 100644 --- a/auto-migrate.ts +++ b/auto-migrate.ts @@ -12,6 +12,7 @@ import { createTableIfNotExistsSQL, quoteIdentifier, } from "./migrations"; +import { mongoAutoMigrate } from "./mongo-schema"; async function tableExists(db: DBClient, table: string): Promise { switch (db.config.type) { @@ -288,6 +289,15 @@ export async function autoMigrate( models: any | any[], ): Promise { const list = Array.isArray(models) ? models : [models]; + + // Returned before the `dialect` union below is computed, so no SQL path is + // reachable for a MongoDB client. MongoDB has no `ADD COLUMN` and no + // `CREATE TABLE`; its equivalent is a collection plus a validator, and it + // lives in `mongo-schema` rather than in a branch of this function. + if (db.config.type === DBType.MongoDB) { + return mongoAutoMigrate(db, list); + } + // Every identifier this file emits goes through `quoteIdentifier`, so the // DDL parses on MySQL-family servers (backticks) as well as the rest (`"`). // @see quoteIdentifier for why the two spellings are not interchangeable. diff --git a/bun.lock b/bun.lock index 00d1e6d..656c999 100644 --- a/bun.lock +++ b/bun.lock @@ -20,6 +20,9 @@ "typescript": "^5.9.3", "vitest": "^2.1.9", }, + "optionalDependencies": { + "mongodb": "^6.20.0", + }, "peerDependencies": { "bun": ">=1.0.0", }, @@ -163,6 +166,8 @@ "@js-joda/core": ["@js-joda/core@6.1.0", "", {}, "sha512-H8NTMRDJqad/leyv/D/A3kSOsf5/58Ydj4DJGDyaCWk9OU/zuZOLhndVffJgQjsgrn5GC0znHMHie7TfvPPG4w=="], + "@mongodb-js/saslprep": ["@mongodb-js/saslprep@1.5.4", "", { "dependencies": { "sparse-bitfield": "^3.0.3" } }, "sha512-05UC0jQsjKAOuXQ0H9Ud9vUTJpZIg+n/FinpR30tI5I8pY2inTfPOZ5OF/cg3Ce/N9MoD1xhRCeOsJtuTbFYlw=="], + "@nodelib/fs.scandir": ["@nodelib/fs.scandir@2.1.5", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "^1.1.9" } }, "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g=="], "@nodelib/fs.stat": ["@nodelib/fs.stat@2.0.5", "", {}, "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A=="], @@ -229,6 +234,10 @@ "@types/uuid": ["@types/uuid@11.0.0", "", { "dependencies": { "uuid": "*" } }, "sha512-HVyk8nj2m+jcFRNazzqyVKiZezyhDKrGUA3jlEcg/nZ6Ms+qHwocba1Y/AaVaznJTAM9xpdFSh+ptbNrhOGvZA=="], + "@types/webidl-conversions": ["@types/webidl-conversions@7.0.3", "", {}, "sha512-CiJJvcRtIgzadHCYXw7dqEnMNRjhGZlYK05Mj9OyktqV8uVT8fD2BFOB7S1uwBE3Kj2Z+4UyPmFw/Ixgw/LAlA=="], + + "@types/whatwg-url": ["@types/whatwg-url@11.0.5", "", { "dependencies": { "@types/webidl-conversions": "*" } }, "sha512-coYR071JRaHa+xoEvvYqvnIHaVqaYrLPbsufM9BF63HkwI5Lgmy2QR8Q5K/lYDYo5AK82wOvSOS0UsLTpTG7uQ=="], + "@typescript-eslint/eslint-plugin": ["@typescript-eslint/eslint-plugin@8.46.1", "", { "dependencies": { "@eslint-community/regexpp": "^4.10.0", "@typescript-eslint/scope-manager": "8.46.1", "@typescript-eslint/type-utils": "8.46.1", "@typescript-eslint/utils": "8.46.1", "@typescript-eslint/visitor-keys": "8.46.1", "graphemer": "^1.4.0", "ignore": "^7.0.0", "natural-compare": "^1.4.0", "ts-api-utils": "^2.1.0" }, "peerDependencies": { "@typescript-eslint/parser": "^8.46.1", "eslint": "^8.57.0 || ^9.0.0", "typescript": ">=4.8.4 <6.0.0" } }, "sha512-rUsLh8PXmBjdiPY+Emjz9NX2yHvhS11v0SR6xNJkm5GM1MO9ea/1GoDKlHHZGrOJclL/cZ2i/vRUYVtjRhrHVQ=="], "@typescript-eslint/parser": ["@typescript-eslint/parser@8.46.1", "", { "dependencies": { "@typescript-eslint/scope-manager": "8.46.1", "@typescript-eslint/types": "8.46.1", "@typescript-eslint/typescript-estree": "8.46.1", "@typescript-eslint/visitor-keys": "8.46.1", "debug": "^4.3.4" }, "peerDependencies": { "eslint": "^8.57.0 || ^9.0.0", "typescript": ">=4.8.4 <6.0.0" } }, "sha512-6JSSaBZmsKvEkbRUkf7Zj7dru/8ZCrJxAqArcLaVMee5907JdtEbKGsZ7zNiIm/UAkpGUkaSMZEXShnN2D1HZA=="], @@ -297,6 +306,8 @@ "braces": ["braces@3.0.3", "", { "dependencies": { "fill-range": "^7.1.1" } }, "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA=="], + "bson": ["bson@6.10.4", "", {}, "sha512-WIsKqkSC0ABoBJuT1LEX+2HEvNmNKKgnTAyd0fL8qzK4SH2i9NXg+t08YtdZp/V9IZ33cxe3iV4yM0qg8lMQng=="], + "buffer": ["buffer@6.0.3", "", { "dependencies": { "base64-js": "^1.3.1", "ieee754": "^1.2.1" } }, "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA=="], "buffer-equal-constant-time": ["buffer-equal-constant-time@1.0.1", "", {}, "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA=="], @@ -515,6 +526,8 @@ "make-dir": ["make-dir@4.0.0", "", { "dependencies": { "semver": "^7.5.3" } }, "sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw=="], + "memory-pager": ["memory-pager@1.5.0", "", {}, "sha512-ZS4Bp4r/Zoeq6+NLJpP+0Zzm0pR8whtGPf1XExKLJBAczGMnSi3It14OiNCStjQjM6NU1okjQGSxgEZN8eBYKg=="], + "merge2": ["merge2@1.4.1", "", {}, "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg=="], "micromatch": ["micromatch@4.0.8", "", { "dependencies": { "braces": "^3.0.3", "picomatch": "^2.3.1" } }, "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA=="], @@ -523,6 +536,10 @@ "minipass": ["minipass@7.1.2", "", {}, "sha512-qOOzS1cBTWYF4BH8fVePDBOO9iptMnGUEZwNc/cMWnTV2nVLZ7VoNWEPHkYczZA0pdoA7dl6e7FL659nX9S2aw=="], + "mongodb": ["mongodb@6.21.0", "", { "dependencies": { "@mongodb-js/saslprep": "^1.3.0", "bson": "^6.10.4", "mongodb-connection-string-url": "^3.0.2" }, "peerDependencies": { "@aws-sdk/credential-providers": "^3.188.0", "@mongodb-js/zstd": "^1.1.0 || ^2.0.0", "gcp-metadata": "^5.2.0", "kerberos": "^2.0.1", "mongodb-client-encryption": ">=6.0.0 <7", "snappy": "^7.3.2", "socks": "^2.7.1" }, "optionalPeers": ["@aws-sdk/credential-providers", "@mongodb-js/zstd", "gcp-metadata", "kerberos", "mongodb-client-encryption", "snappy", "socks"] }, "sha512-URyb/VXMjJ4da46OeSXg+puO39XH9DeQpWCslifrRn9JWugy0D+DvvBvkm2WxmHe61O/H19JM66p1z7RHVkZ6A=="], + + "mongodb-connection-string-url": ["mongodb-connection-string-url@3.0.2", "", { "dependencies": { "@types/whatwg-url": "^11.0.2", "whatwg-url": "^14.1.0 || ^13.0.0" } }, "sha512-rMO7CGo/9BFwyZABcKAWL8UJwH/Kc2x0g72uhDWzG48URRax5TCIcJ7Rc3RZqffZzO/Gwff/jyKwCU9TN8gehA=="], + "ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="], "mssql": ["mssql@12.7.2", "", { "dependencies": { "@tediousjs/connection-string": "^1.0.0", "commander": "^11.0.0", "debug": "^4.3.3", "tarn": "^3.0.2", "tedious": "^19.2.2 || ^20.0.0" }, "bin": { "mssql": "bin/mssql" } }, "sha512-zhpPw+WXBWLw7d7J6Y0aWgQy0EV8HKDGNSvmkAjP8bSM4H41X0fweAeKWI+bg9Ri+JHCbsf/QScjmACBFcEo8A=="], @@ -633,6 +650,8 @@ "source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="], + "sparse-bitfield": ["sparse-bitfield@3.0.3", "", { "dependencies": { "memory-pager": "^1.0.2" } }, "sha512-kvzhi7vqKTfkh0PZU+2D2PIllw2ymqJKujUcyPMd9Y75Nv4nPbGJZXNhxsgdQab2BmlDct1YnfQCguEvHr7VsQ=="], + "split2": ["split2@4.2.0", "", {}, "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg=="], "sprintf-js": ["sprintf-js@1.1.3", "", {}, "sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA=="], @@ -677,6 +696,8 @@ "to-regex-range": ["to-regex-range@5.0.1", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="], + "tr46": ["tr46@5.1.1", "", { "dependencies": { "punycode": "^2.3.1" } }, "sha512-hdF5ZgjTqgAntKkklYw0R03MG2x/bSzTtkxmIRw/sTNV8YXsCJ1tfLAX23lhxhHJlEf3CRCOCGGWw3vI3GaSPw=="], + "ts-api-utils": ["ts-api-utils@2.1.0", "", { "peerDependencies": { "typescript": ">=4.8.4" } }, "sha512-CUgTZL1irw8u29bzrOD/nH85jqyc74D6SshFgujOIA7osm2Rz7dYH77agkx7H4FBNxDq7Cjf+IjaX/8zwFW+ZQ=="], "tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], @@ -697,6 +718,10 @@ "vitest": ["vitest@2.1.9", "", { "dependencies": { "@vitest/expect": "2.1.9", "@vitest/mocker": "2.1.9", "@vitest/pretty-format": "^2.1.9", "@vitest/runner": "2.1.9", "@vitest/snapshot": "2.1.9", "@vitest/spy": "2.1.9", "@vitest/utils": "2.1.9", "chai": "^5.1.2", "debug": "^4.3.7", "expect-type": "^1.1.0", "magic-string": "^0.30.12", "pathe": "^1.1.2", "std-env": "^3.8.0", "tinybench": "^2.9.0", "tinyexec": "^0.3.1", "tinypool": "^1.0.1", "tinyrainbow": "^1.2.0", "vite": "^5.0.0", "vite-node": "2.1.9", "why-is-node-running": "^2.3.0" }, "peerDependencies": { "@edge-runtime/vm": "*", "@types/node": "^18.0.0 || >=20.0.0", "@vitest/browser": "2.1.9", "@vitest/ui": "2.1.9", "happy-dom": "*", "jsdom": "*" }, "optionalPeers": ["@edge-runtime/vm", "@types/node", "@vitest/browser", "@vitest/ui", "happy-dom", "jsdom"], "bin": { "vitest": "vitest.mjs" } }, "sha512-MSmPM9REYqDGBI8439mA4mWhV5sKmDlBKWIYbA3lRb2PTHACE0mgKwA8yQ2xq9vxDTuk4iPrECBAEW2aoFXY0Q=="], + "webidl-conversions": ["webidl-conversions@7.0.0", "", {}, "sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g=="], + + "whatwg-url": ["whatwg-url@14.2.0", "", { "dependencies": { "tr46": "^5.1.0", "webidl-conversions": "^7.0.0" } }, "sha512-De72GdQZzNTUBBChsXueQUnPKDkg/5A5zp7pFDuQAj5UFoENpiACU0wlCvzpAGnTkj++ihpKwKyYewn/XNUbKw=="], + "which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], "why-is-node-running": ["why-is-node-running@2.3.0", "", { "dependencies": { "siginfo": "^2.0.0", "stackback": "0.0.2" }, "bin": { "why-is-node-running": "cli.js" } }, "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w=="], diff --git a/client.ts b/client.ts index fe187da..59da0da 100644 --- a/client.ts +++ b/client.ts @@ -205,6 +205,122 @@ interface MSSQLHandle { rollback?: (...args: any[]) => any; } +/** + * The shape of the MongoDB handles this client stores. + * + * Structural for the same reason `MSSQLHandle` is — naming the driver's own + * types would emit an import of the `mongodb` package into `client.d.ts`, and + * every consumer of the published package would then need declarations for a + * driver most of them never install. + * + * Unlike mssql, though, the driver *does* ship its own declarations, so there + * is deliberately no ambient shim module here: one would shadow the real types + * for the ORM build and for any consumer that does use the driver. + * + * A `MongoClient` is recognised by `db`, a `ClientSession` by `withTransaction`. + */ +interface MongoHandle { + connect?: () => Promise; + close?: () => Promise; + db?: (name?: string) => MongoDbHandle; + startSession?: () => MongoSessionHandle; +} + +/** A `ClientSession`, which is what an open transaction actually is. */ +interface MongoSessionHandle { + withTransaction?: (...args: any[]) => any; + endSession?: () => Promise; +} + +/** A `Db` — the handle collections are read from. */ +interface MongoDbHandle { + collection?: (name: string) => MongoCollectionHandle; + command?: ( + command: Record, + options?: Record, + ) => Promise; + listCollections?: (...args: any[]) => MongoCursorHandle; + createCollection?: (...args: any[]) => Promise; + admin?: () => { command: (command: Record) => Promise }; +} + +/** A `Collection`. Only the members the ORM actually reaches for. */ +interface MongoCollectionHandle { + find: (...args: any[]) => MongoCursorHandle; + findOne: (...args: any[]) => Promise; + insertOne: (...args: any[]) => Promise; + insertMany: (...args: any[]) => Promise; + updateOne: (...args: any[]) => Promise; + updateMany: (...args: any[]) => Promise; + deleteOne: (...args: any[]) => Promise; + deleteMany: (...args: any[]) => Promise; + countDocuments: (...args: any[]) => Promise; + distinct: (...args: any[]) => Promise; + aggregate: (...args: any[]) => MongoCursorHandle; + findOneAndUpdate: (...args: any[]) => Promise; + bulkWrite: (...args: any[]) => Promise; + createIndex: (...args: any[]) => Promise; + listIndexes: (...args: any[]) => MongoCursorHandle; + drop?: (...args: any[]) => Promise; + indexes?: (...args: any[]) => Promise; +} + +/** A `FindCursor` or `AggregationCursor`. */ +interface MongoCursorHandle { + toArray: () => Promise; + sort?: (...args: any[]) => MongoCursorHandle; + skip?: (...args: any[]) => MongoCursorHandle; + limit?: (...args: any[]) => MongoCursorHandle; + project?: (...args: any[]) => MongoCursorHandle; + hasNext?: () => Promise; + next?: () => Promise; + close?: () => Promise; +} + +/** What a mongo write reports back. Field-for-field the driver's own result. */ +interface MongoUpdateResult { + acknowledged?: boolean; + matchedCount?: number; + modifiedCount?: number; + upsertedCount?: number; + upsertedId?: any; + insertedCount?: number; + deletedCount?: number; +} + +/** + * Handles whose replica-set support has already been probed. + * + * A transaction-bound client shares its parent's `MongoClient` object, so + * without this the probe would run again on every transaction — and the probe + * is a round trip on the hottest path in the library. + */ +const replicaSetProbed = new WeakSet(); + +/** + * Loads the MongoDB driver, which is an optional dependency. + * + * The specifier is held in a variable on purpose. A literal dynamic import is + * resolved statically by the bundler and by `tsc`, neither of which should + * require the driver to be present for a build or typecheck of the SQL + * backends. + * + * @returns The driver module. + * @throws StabilizeError when the driver is not installed. + */ +async function loadMongoDriver(): Promise { + const specifier = "mongodb"; + try { + return await import(specifier); + } catch { + throw new StabilizeError( + "The MongoDB driver is not installed. DBType.MongoDB requires it as an " + + "optional peer: install it with `bun add mongodb` or `npm install mongodb`.", + "MONGO_DRIVER_MISSING", + ); + } +} + /** * Provides a unified database client for interacting with PostgreSQL, MySQL, and SQLite. */ @@ -215,7 +331,8 @@ export class DBClient { | mysql.Pool | PoolClient | mysql.PoolConnection - | MSSQLHandle; + | MSSQLHandle + | MongoHandle; private logger: Logger; public readonly config: DBConfig; private retryAttempts: number; @@ -229,6 +346,24 @@ export class DBClient { */ private mssqlConnectPromise: Promise | null = null; + /** + * The in-flight connect on the mongo client, held for the same reason as + * `mssqlConnectPromise`. It resolves to the connected handle so that callers + * that need to reach a collection do not have to re-derive it. + */ + private mongoConnectPromise: Promise | null = null; + + /** + * The mongo session every statement on this client should run inside. + * + * Kept beside `client` rather than *as* `client`, unlike mssql. A mongo + * transaction is not a different connection the way `sql.Transaction` is: the + * commands still go to the same `MongoClient`, and the session is passed + * alongside them as an option. Storing it separately is what lets a + * transaction-bound client still resolve its parent's collections. + */ + private mongoSession: MongoSessionHandle | null = null; + private preparedStatements: Map = new Map(); public isTransactionClient: boolean = false; @@ -237,7 +372,10 @@ export class DBClient { * @param config The database configuration object. * @param logger Optional logger instance. Uses StabilizeLogger if not provided. * @param existingClient Optional existing transaction client. For SQL Server - * this is the `sql.Transaction` the statements should run inside. + * this is the `sql.Transaction` the statements should run inside; for + * MongoDB it is the parent `MongoClient`, shared with the session below. + * @param mongoSession Optional session that scopes statements to a + * transaction. Only MongoDB uses it. */ constructor( config: DBConfig, @@ -246,7 +384,9 @@ export class DBClient { | PoolClient | mysql.PoolConnection | MSSQLHandle + | MongoHandle | null = null, + mongoSession: MongoSessionHandle | null = null, ) { this.config = config; this.logger = logger; @@ -257,6 +397,7 @@ export class DBClient { if (existingClient) { this.client = existingClient; this.isTransactionClient = true; + this.mongoSession = mongoSession; } else { this.initializeClient(config); } @@ -283,9 +424,200 @@ export class DBClient { // every driver. `ensureMSSQLConnected` opens it on first use instead. this.client = new sql.ConnectionPool(config.connectionString); this.logger.logDebug(`Initialized MSSQL Pool client.`); + } else if (config.type === DBType.MongoDB) { + // Same constraint as mssql, one step worse: the driver is an optional + // dependency, so reaching it needs `await import()` — which cannot happen + // from a constructor either. `ensureMongoConnected` does both the import + // and the connect on first use and fills `client` in then. Nothing may + // touch `this.client` for a mongo config before awaiting it. + this.client = null as unknown as MongoHandle; + this.logger.logDebug(`Deferred MongoDB client initialization.`); } } + /** + * Opens the mongo client, once, on first use. + * + * Performs the lazy `import()` of the optional driver and then `connect()`, + * mirroring `ensureMSSQLConnected`. The promise is memoised so concurrent + * first queries share one connection attempt. + * + * @returns The connected mongo handle. + * @throws StabilizeError when the driver is absent or a handle is malformed. + */ + private async ensureMongoConnected(): Promise { + if (!this.mongoConnectPromise) { + this.mongoConnectPromise = this.openMongoClient(); + } + return this.mongoConnectPromise; + } + + /** Builds and connects the mongo client. See `ensureMongoConnected`. */ + private async openMongoClient(): Promise { + let handle = this.client as MongoHandle | null; + + if (!handle || typeof handle.db !== "function") { + const driver = await loadMongoDriver(); + const options: Record = { + ...(this.config.mongoOptions ?? {}), + }; + // The URI's own database wins when it has one; the driver only consults + // `dbName` when the path is empty. + if (this.config.database && !options.dbName) { + options.dbName = this.config.database; + } + handle = new driver.MongoClient( + this.config.connectionString, + options, + ) as MongoHandle; + this.client = handle; + } + + if (typeof handle.connect === "function") { + await handle.connect(); + } + this.logger.logDebug("MongoDB client connected."); + await this.assertReplicaSetOrExplained(handle); + return handle; + } + + /** + * Warns, once per client, when the server cannot serve transactions. + * + * Every write in the ORM is wrapped in a transaction, and MongoDB only + * supports those on a replica set or sharded cluster. A standalone `mongod` + * accepts the connection, answers every read, and then rejects the first + * `startTransaction` with a bare `IllegalOperation` — so without this the + * failure surfaces as "create() does not work" with nothing pointing at the + * cause. A warning at connect time names it. + * + * Deliberately not fatal: reads work fine standalone, and refusing to connect + * would break the read-only use someone may legitimately have. + */ + private async assertReplicaSetOrExplained(handle: MongoHandle): Promise { + if (replicaSetProbed.has(handle)) return; + replicaSetProbed.add(handle); + + const db = handle.db?.(this.config.database); + const admin = db?.admin?.(); + if (!admin || typeof admin.command !== "function") return; + + try { + const hello = await admin.command({ hello: 1 }); + if (hello && !hello.setName && !hello.msg) { + this.logger.logWarn( + "MongoDB is running as a standalone server. Transactions require a " + + "replica set, so every write — including create(), which the ORM " + + "always wraps in one — will fail with an IllegalOperation error. " + + "Start the server with --replSet and run rs.initiate(), or connect " + + "to an existing replica set.", + ); + } + } catch (error) { + // A server that will not answer `hello` is not one this check can say + // anything useful about; the real error will surface on first use. + this.logger.logDebug( + `Could not probe MongoDB replica-set support: ${(error as Error).message}`, + ); + } + } + + /** + * Turns the driver's bare "Transaction numbers are only allowed on a replica + * set member or mongos" into an error that says what to do about it. + * + * Code 20 (`IllegalOperation`) is the one a standalone server returns from + * `startTransaction`. It is worth naming precisely because the symptom is so + * far from the cause: reads work, the connection is healthy, and only writes + * fail — because the ORM wraps every write in a transaction. + * + * @param error Whatever the driver threw. + * @returns A StabilizeError preserving the original as `cause`. + */ + private explainMongoTransactionFailure(error: unknown): StabilizeError { + const code = (error as { code?: number })?.code; + const message = (error as Error)?.message ?? String(error); + + // The infrastructure signature is looked for *first*, and deliberately not + // after the pass-through below. The executor family wraps every driver + // failure in a `MONGO_ERROR` whose own `code` is a string, so by the time a + // code-20 rejection gets here the number is gone and the driver's wording + // survives only inside the wrapper's message. Checking `instanceof` first + // would hand that wrapper straight back and report a standalone server as a + // generic mongo error. + if (code === 20 || /replica set|mongos/i.test(message)) { + return new StabilizeError( + "MongoDB transactions require a replica set or sharded cluster, and " + + "this server is a standalone. Every write goes through a transaction, " + + "so start the server with --replSet and run rs.initiate() (or point " + + `the connection at an existing replica set). Driver said: ${message}`, + "TX_ERROR", + error as Error, + ); + } + + // Anything the ORM has already classified — a validation failure, an + // optimistic-lock conflict, a row that is not there, or a write the driver + // refused — is the answer, and the four SQL branches all let theirs through + // untouched. Rewriting it as a transaction failure gave the caller the + // wrong code to branch on and the wrong thing to go and look at: a payload + // that failed validation sent the reader to the server's replica-set + // config. + if (error instanceof StabilizeError) return error; + + return new StabilizeError(message, "TX_ERROR", error as Error); + } + + /** + * Resolves the database handle statements should be issued against. + * @returns The connected `Db`. + * @throws StabilizeError when the handle exposes no `db()`. + */ + private async mongoDb(): Promise { + const handle = await this.ensureMongoConnected(); + const db = handle.db?.(this.config.database); + if (!db) { + throw new StabilizeError( + "MongoDB client did not provide a database handle.", + "MONGO_ERROR", + ); + } + return db; + } + + /** + * The options every mongo command must carry. + * + * A transaction-bound client contributes its session here; a plain one + * contributes nothing. Threading it through every executor is what makes a + * repository write performed inside `transaction()` actually participate in + * it, rather than silently committing on its own. + */ + private mongoOptions(): Record { + return this.mongoSession ? { session: this.mongoSession } : {}; + } + + /** + * Rejects a SQL statement sent to a MongoDB client. + * + * There is no fifth branch to add to `query`: a mongo command is a document, + * not a string, so there is nothing for the SQL path to dispatch on. Failing + * loudly here means a caller who reached for `rawQuery` against mongo gets a + * sentence explaining why, rather than a driver-level parse error. + * + * @throws StabilizeError always, when this client is a mongo client. + */ + private rejectSQLForMongo(): void { + throw new StabilizeError( + "Raw SQL is not available on MongoDB. The Mongo backend speaks commands " + + "and documents rather than statements, so rawQuery/rawExec and the " + + "query-builder's SQL-only clauses (join, union, whereRaw, orderByRaw, " + + "selectRaw, groupByRaw, having, whereExists) have no equivalent. Use " + + "the repository API or the query builder's structured methods instead.", + "MONGO_UNSUPPORTED", + ); + } + /** * Opens the mssql pool, once, on first use. * @@ -355,6 +687,7 @@ export class DBClient { * @throws StabilizeError if all retry attempts fail. */ async query(query: string, params: any[] = []): Promise { + if (this.config.type === DBType.MongoDB) this.rejectSQLForMongo(); const start = Date.now(); // Only reads are retried. Every write in the ORM — insert, update, delete, @@ -467,6 +800,45 @@ export class DBClient { } } + if (this.config.type === DBType.MongoDB) { + const handle = await this.ensureMongoConnected(); + if (typeof handle.startSession !== "function") { + throw new StabilizeError( + "MongoDB client cannot start a session, so transactions are unavailable.", + "TX_ERROR", + ); + } + + const session = handle.startSession(); + // The transaction-bound client keeps the parent's `MongoClient` and adds + // the session, because mongo commands carry a session rather than being + // sent through a different connection. + const txClient = new DBClient(this.config, this.logger, handle, session); + this.logger.logDebug("Starting MongoDB transaction."); + + try { + // `withTransaction` rather than an explicit start/commit pair: it + // replays the callback when the server reports a transient error, which + // is exactly what a write conflict on a per-table counter document + // produces when two creates allocate ids at once. Reproducing that by + // hand would mean re-running caller code from inside this method. + return await session.withTransaction!(() => callback(txClient), { + readConcern: { level: "snapshot" }, + writeConcern: { w: "majority" }, + }); + } catch (error) { + throw this.explainMongoTransactionFailure(error); + } finally { + // The session is a server-side resource and leaks if it is not ended, + // whether the transaction committed or not. + try { + await session.endSession?.(); + } catch (endError) { + this.logger.logError(endError as Error); + } + } + } + if (this.config.type === DBType.MSSQL) { // SQL Server has no `BEGIN`/`COMMIT` text: the transaction is a // server-side object opened on a borrowed pooled connection, and every @@ -554,10 +926,20 @@ export class DBClient { // mysql pools expose — the generic branch below would silently skip it // and leave the sockets open. await (this.client as MSSQLHandle).close!(); + } else if ( + this.config.type === DBType.MongoDB && + this.client && + typeof (this.client as MongoHandle).close === "function" + ) { + // Same trap as mssql: a `MongoClient` is closed with `close()`. A client + // that was never used holds no handle at all, so the guard is on the + // function rather than the config. + await (this.client as MongoHandle).close!(); } else if (this.client && "end" in this.client) { await (this.client as any).end(); } this.client = null!; + this.mongoConnectPromise = null; this.logger.logInfo("Database connection closed"); } @@ -565,6 +947,7 @@ export class DBClient { query: string, params: any[] = [], ): Promise<{ affectedRows: number }> { + if (this.config.type === DBType.MongoDB) this.rejectSQLForMongo(); const start = Date.now(); let affectedRows = 0; @@ -603,6 +986,7 @@ export class DBClient { * @returns Promise that resolves once the query is complete. */ async migrationQuery(query: string, params: any[] = []): Promise { + if (this.config.type === DBType.MongoDB) this.rejectSQLForMongo(); const start = Date.now(); if (this.client instanceof Database) { let stmt = this.preparedStatements.get(query); @@ -625,4 +1009,335 @@ export class DBClient { const executionTime = Date.now() - start; this.logger.logQuery(query, params, executionTime); } + + // --------------------------------------------------------------------------- + // MongoDB executors + // + // A parallel family to query/queryExec/migrationQuery rather than a fifth + // branch inside them: those take a SQL string to dispatch on, and a mongo + // command is a document. Everything below funnels through `mongoRun` so that + // logging, session threading and error wrapping are written once. + // + // Reads are not retried the way `query` retries them. A transaction is + // already replayed wholesale by `withTransaction`, and outside one a mongo + // read failure is a topology problem that retrying three times will not fix. + // --------------------------------------------------------------------------- + + /** + * Runs one mongo operation against a collection. + * + * @param label Short description used in logs and error messages. + * @param detail The filter, document or pipeline, for the log line. + * @param operation Receives the connected database handle. + * @returns Whatever the operation resolved to. + * @throws StabilizeError wrapping any driver failure. + */ + private async mongoRun( + label: string, + detail: unknown, + operation: (db: MongoDbHandle) => Promise, + ): Promise { + const start = Date.now(); + try { + const db = await this.mongoDb(); + const result = await operation(db); + this.logger.logQuery(label, [detail], Date.now() - start); + return result; + } catch (error) { + if (error instanceof StabilizeError) throw error; + this.logger.logError(error as Error); + throw new StabilizeError( + `MongoDB ${label} failed: ${(error as Error).message}`, + "MONGO_ERROR", + error as Error, + ); + } + } + + /** + * Resolves a collection, or throws if the handle has none. + * @param name The collection name. + * @param db The database handle. + */ + private mongoCollection( + name: string, + db: MongoDbHandle, + ): MongoCollectionHandle { + const collection = db.collection?.(name); + if (!collection) { + throw new StabilizeError( + `MongoDB database handle did not provide collection '${name}'.`, + "MONGO_ERROR", + ); + } + return collection; + } + + /** Merges caller options with this client's session, when it has one. */ + private withSession( + options: Record = {}, + ): Record { + return { ...options, ...this.mongoOptions() }; + } + + /** Reads every matching document. */ + async mongoFind( + collection: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("find", { collection, filter }, async (db) => { + const cursor = this.mongoCollection(collection, db).find( + filter, + this.withSession(options), + ); + return (await cursor.toArray()) ?? []; + }); + } + + /** Reads the first matching document, or null. */ + async mongoFindOne( + collection: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("findOne", { collection, filter }, async (db) => + this.mongoCollection(collection, db).findOne( + filter, + this.withSession(options), + ), + ); + } + + /** Inserts one document. */ + async mongoInsertOne( + collection: string, + document: Record, + options: Record = {}, + ): Promise { + return this.mongoRun("insertOne", { collection }, async (db) => + this.mongoCollection(collection, db).insertOne( + document, + this.withSession(options), + ), + ); + } + + /** + * Inserts many documents. + * + * `ordered: false` is deliberately *not* the default: a batch that fails + * halfway should leave the caller able to tell which half landed, and an + * unordered insert reports that only in aggregate. + */ + async mongoInsertMany( + collection: string, + documents: Record[], + options: Record = {}, + ): Promise { + return this.mongoRun("insertMany", { collection, count: documents.length }, async (db) => + this.mongoCollection(collection, db).insertMany( + documents, + this.withSession(options), + ), + ); + } + + /** Updates the first matching document. */ + async mongoUpdateOne( + collection: string, + filter: Record, + update: Record, + options: Record = {}, + ): Promise { + return this.mongoRun("updateOne", { collection, filter }, async (db) => + this.mongoCollection(collection, db).updateOne( + filter, + update, + this.withSession(options), + ), + ); + } + + /** Updates every matching document. */ + async mongoUpdateMany( + collection: string, + filter: Record, + update: Record, + options: Record = {}, + ): Promise { + return this.mongoRun("updateMany", { collection, filter }, async (db) => + this.mongoCollection(collection, db).updateMany( + filter, + update, + this.withSession(options), + ), + ); + } + + /** Deletes the first matching document. */ + async mongoDeleteOne( + collection: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("deleteOne", { collection, filter }, async (db) => + this.mongoCollection(collection, db).deleteOne( + filter, + this.withSession(options), + ), + ); + } + + /** Deletes every matching document. */ + async mongoDeleteMany( + collection: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("deleteMany", { collection, filter }, async (db) => + this.mongoCollection(collection, db).deleteMany( + filter, + this.withSession(options), + ), + ); + } + + /** Counts matching documents without materialising them. */ + async mongoCount( + collection: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("countDocuments", { collection, filter }, async (db) => + this.mongoCollection(collection, db).countDocuments( + filter, + this.withSession(options), + ), + ); + } + + /** Lists the distinct values of a field. */ + async mongoDistinct( + collection: string, + field: string, + filter: Record = {}, + options: Record = {}, + ): Promise { + return this.mongoRun("distinct", { collection, field }, async (db) => + this.mongoCollection(collection, db).distinct( + field, + filter, + this.withSession(options), + ), + ); + } + + /** Runs an aggregation pipeline. */ + async mongoAggregate( + collection: string, + pipeline: Record[], + options: Record = {}, + ): Promise { + return this.mongoRun("aggregate", { collection, stages: pipeline.length }, async (db) => { + const cursor = this.mongoCollection(collection, db).aggregate( + pipeline, + this.withSession(options), + ); + return (await cursor.toArray()) ?? []; + }); + } + + /** + * Applies an update and returns a document. + * + * The driver returns the document itself, not a `ModifyResult`, because + * `includeResultMetadata` has defaulted to false since driver 6 (NODE-3568). + * The return shape is whatever `returnDocument` asks for, so this deliberately + * does not normalise it — the caller that needs `$inc`'s new value and the one + * that needs the pre-image want different answers. + */ + async mongoFindOneAndUpdate( + collection: string, + filter: Record, + update: Record, + options: Record = {}, + ): Promise { + return this.mongoRun("findOneAndUpdate", { collection, filter }, async (db) => + this.mongoCollection(collection, db).findOneAndUpdate( + filter, + update, + this.withSession(options), + ), + ); + } + + /** Runs a bulk write, for counter bumps and M2M syncs that need one trip. */ + async mongoBulkWrite( + collection: string, + operations: Record[], + options: Record = {}, + ): Promise { + return this.mongoRun("bulkWrite", { collection, count: operations.length }, async (db) => + this.mongoCollection(collection, db).bulkWrite( + operations, + this.withSession(options), + ), + ); + } + + /** Creates an index. */ + async mongoCreateIndex( + collection: string, + spec: Record, + options: Record = {}, + ): Promise { + return this.mongoRun("createIndex", { collection, spec }, async (db) => + this.mongoCollection(collection, db).createIndex( + spec, + this.withSession(options), + ), + ); + } + + /** Lists a collection's indexes. */ + async mongoListIndexes(collection: string): Promise { + return this.mongoRun("listIndexes", { collection }, async (db) => { + const cursor = this.mongoCollection(collection, db).listIndexes(); + return (await cursor.toArray()) ?? []; + }); + } + + /** Lists a database's collections. */ + async mongoListCollections(): Promise { + return this.mongoRun("listCollections", {}, async (db) => { + const cursor = db.listCollections?.(); + if (!cursor) return []; + return (await cursor.toArray()) ?? []; + }); + } + + /** + * Runs a database command. + * + * The catch-all for operations with no collection to hang off — `collMod` to + * change a validator, `ping` for the health check, `hello` for topology. + * + * The command and the options are separate arguments, and the session belongs + * in the second: merged into the command document it becomes a field the + * server tries to serialise, and a `ClientSession` is not BSON. + */ + async mongoCommand( + command: Record, + ): Promise { + return this.mongoRun("command", command, async (db) => { + if (!db.command) { + throw new StabilizeError( + "MongoDB database handle did not provide command().", + "MONGO_ERROR", + ); + } + return db.command(command, this.mongoOptions()); + }); + } } diff --git a/docker-compose.test.yml b/docker-compose.test.yml index 15fa6ab..ab8fddb 100644 --- a/docker-compose.test.yml +++ b/docker-compose.test.yml @@ -106,6 +106,57 @@ services: start_period: 20s networks: [stabilize-test] + mongo: + image: mongo:7 + container_name: stabilize-test-mongo + # A single-node *replica set* rather than the standalone default. MongoDB + # only supports transactions on a replica set or mongos, and the ORM wraps + # every write — including create() — in one. A standalone would accept the + # connection, answer every read, and then fail every write with a bare + # IllegalOperation. + command: ["--replSet", "rs0", "--bind_ip_all"] + ports: + - "57017:27017" + healthcheck: + # The set must be initiated before it will serve transactions and there is + # no init container here to do it, so the first probe does it and then + # rethrows — `rs.status()` throws while no config exists, which leaves the + # container unhealthy for exactly one interval. The next probe sees a + # healthy set. Self-contained, and no dependency cycle. + # + # The member registers itself as 127.0.0.1:27017 — the *container's* port. + # A driver doing topology discovery is handed that address and tries to + # reach it from the host, where nothing is listening. Connect with + # `directConnection=true` to skip discovery; see tests/integration.mongo.test.ts. + test: + [ + "CMD-SHELL", + "mongosh --quiet --eval 'try { rs.status().ok } catch (e) { rs.initiate({_id:\"rs0\",members:[{_id:0,host:\"127.0.0.1:27017\"}]}); throw e }'", + ] + interval: 2s + timeout: 10s + retries: 60 + start_period: 5s + mongo-standalone: + # Deliberately *not* a replica set, and here for one reason: to pin the + # failure mode a standalone produces. It accepts the connection, answers + # every read, and then rejects every write — because the ORM wraps each one + # in a transaction. Without a server like this the behaviour could only be + # asserted from documentation rather than observed. + image: mongo:7 + container_name: stabilize-test-mongo-standalone + ports: + - "57018:27017" + healthcheck: + test: ["CMD-SHELL", "mongosh --quiet --eval 'db.adminCommand({ping:1}).ok'"] + interval: 2s + timeout: 10s + retries: 30 + start_period: 3s + tmpfs: + - /data/db + networks: [stabilize-test] + networks: stabilize-test: - driver: bridge + driver: bridge \ No newline at end of file diff --git a/index.ts b/index.ts index d8749e9..164087f 100644 --- a/index.ts +++ b/index.ts @@ -171,14 +171,21 @@ export class Stabilize { }> { const start = performance.now(); try { - const results = await this.client.query("SELECT 1 AS ok"); + // MongoDB has no `SELECT 1`; `ping` is its equivalent liveness command. + // Both are wrapped the same way so a slow or unreachable server lands in + // the same catch rather than escaping as a different error shape. + const isMongo = this.client.config.type === DBType.MongoDB; + const healthy = isMongo + ? (await this.client.mongoCommand({ ping: 1 })).ok === 1 + : (await this.client.query("SELECT 1 AS ok")).length > 0; + const cacheStatus = this.cache ? (await this.cache.get("healthcheck")) ? "connected" : "connected (miss)" : "disabled"; return { - status: results.length > 0 ? "healthy" : "unhealthy", + status: healthy ? "healthy" : "unhealthy", database: this.client.config.type, latencyMs: Number((performance.now() - start).toFixed(2)), cacheStatus, @@ -249,6 +256,10 @@ export class Stabilize { total: pool.size ?? 0, }; } + // MongoDB and SQLite land here deliberately. Neither exposes a pool whose + // occupancy can be read synchronously — the mongo driver's pool is internal + // and per-server, and SQLite has no pool at all — so the sentinel is the + // honest answer rather than a number invented to fill the shape. return { active: -1, idle: -1, total: -1 }; } } diff --git a/migrations.ts b/migrations.ts index f6099f6..dbcf921 100644 --- a/migrations.ts +++ b/migrations.ts @@ -14,6 +14,7 @@ import { DBType, DataTypes, } from "./types"; +import { runMongoMigrations } from "./mongo-migrate"; /** * Quotes an identifier for the target dialect. @@ -525,6 +526,13 @@ function getMigrationsTableSQL(dbType: DBType): string { * @param migrations An array of `Migration` objects to be executed. */ export async function runMigrations(config: DBConfig, migrations: Migration[]) { + // Branched before `getMigrationsTableSQL` can be asked about a dialect it has + // no answer for: there is no `CREATE TABLE` here, and a migration's Mongo + // half rides in `mongoUp`/`mongoDown` rather than in `up`/`down`. + if (config.type === DBType.MongoDB) { + return runMongoMigrations(config, migrations); + } + const client = new DBClient(config); try { const dbType = config.type; diff --git a/mongo-migrate.ts b/mongo-migrate.ts new file mode 100644 index 0000000..a48d504 --- /dev/null +++ b/mongo-migrate.ts @@ -0,0 +1,213 @@ +/** + * @file mongo-migrate.ts + * @description Runs and generates MongoDB migrations. + * @author ElectronSz + * + * Mirrors `runMigrations` step for step, with one divergence that has a reason: + * **the steps of a migration are not wrapped in a transaction.** `createIndex` + * is not permitted inside one, and DDL is not transactional in MongoDB at all, + * so a transaction here would either fail on the first index or give a false + * impression of atomicity. + * + * The consequence is stated rather than hidden: a migration that fails halfway + * leaves a partially migrated collection and **no ledger entry**, so re-running + * applies it again from the start. That is the same failure mode MySQL's + * auto-committing DDL already has on the SQL side, and the alternative — a + * rollback that cannot exist — would be a lie. + */ + +import { MetadataStorage } from "./model"; +import { DBClient } from "./client"; +import { StabilizeError, type DBConfig, type Migration } from "./types"; +import { + generateMongoSteps, + modelUsesGeneratedIds, + mongoCollectionName, + type MongoStep, +} from "./mongo-schema"; + +/** + * The collection the migration ledger lives in. + * + * No counter: a migration is identified by name, and the name is the `_id`, so + * uniqueness is enforced by the storage layer rather than by a read first. + */ +export const MONGO_MIGRATIONS_COLLECTION = "stabilize_migrations"; + +/** + * Runs the steps of one migration. + * + * Sequential, and deliberately not transactional. @see the note at the top of + * this file. + * + * @param db The client to run through. + * @param steps The steps to apply, in order. + */ +async function runSteps(db: DBClient, steps: MongoStep[]): Promise { + for (const step of steps) { + switch (step.kind) { + case "createCollection": { + // `create` on a collection that exists is a `NamespaceExists` error, and + // a migration that was interrupted just after creating a collection is + // the ordinary case. Asked for first rather than caught, so that a + // genuinely unexpected failure is not swallowed along with it. + const existing = await db.mongoListCollections(); + if (existing.some((entry: any) => entry?.name === step.collection)) { + break; + } + await db.mongoCommand({ + create: step.collection, + ...(step.validator ? { validator: step.validator } : {}), + validationLevel: "moderate", + validationAction: "error", + }); + break; + } + + case "createIndex": + await db.mongoCreateIndex(step.collection, step.spec, step.options ?? {}); + break; + + case "dropIndex": + await db.mongoCommand({ + dropIndexes: step.collection, + index: step.name, + }); + break; + + case "collMod": + await db.mongoCommand({ + collMod: step.collection, + validator: step.validator, + validationLevel: "moderate", + }); + break; + + case "dropCollection": + // `drop` on a missing collection is a `NamespaceNotFound` error. The + // collection not being there is the state a drop is asking for, so it + // is not a failure. + await db.mongoCommand({ drop: step.collection }).catch(() => {}); + break; + + case "createCounter": + await db.mongoUpdateOne( + "stabilize_counters", + { _id: step.collection }, + { $setOnInsert: { seq: 0 } }, + { upsert: true }, + ); + break; + + default: { + const unknown = step as { kind: string }; + throw new StabilizeError( + `Unknown MongoDB migration step '${unknown.kind}'.`, + "MIGRATE_ERROR", + ); + } + } + } +} + +/** + * Builds a migration for a model, in the shape the SQL migrations already use. + * + * `up` and `down` are empty on purpose: they are SQL, and there is no SQL here. + * The Mongo half rides in `mongoUp`/`mongoDown`, which `tests/migrations.test.ts` + * never sees and `runMigrations` never reads. + * + * @param model The model to generate for. + * @param name The migration's name, and its identity in the ledger. + */ +export function generateMongoMigration( + model: any, + name: string, +): Migration { + const meta = MetadataStorage.getModelMetadata(model); + if (!meta?.tableName) { + throw new StabilizeError( + `Model is missing tableName. Use defineModel() or add static schema.`, + "MIGRATE_ERROR", + ); + } + + return { + name, + up: [], + down: [], + mongoUp: buildUpSteps(model), + mongoDown: [{ kind: "dropCollection", collection: meta.tableName }], + }; +} + +/** + * The steps that bring a model's collection into existence. + * + * Adds the counter step on top of what `generateMongoSteps` derives: the + * collection plan describes the collection, and the counter lives in a different + * one, so it is not something a per-collection plan can express. + * + * @param model The model to build for. + */ +function buildUpSteps(model: any): MongoStep[] { + const steps = generateMongoSteps(model, "up"); + if (steps.length === 0) return steps; + + if (modelUsesGeneratedIds(model)) { + const collection = mongoCollectionName(model); + if (collection) steps.push({ kind: "createCounter", collection }); + } + return steps; +} + +/** + * Applies every pending migration, recording each in the ledger. + * + * @param config The database configuration. + * @param migrations The migrations to apply, in order. + */ +export async function runMongoMigrations( + config: DBConfig, + migrations: Migration[], +): Promise { + const client = new DBClient(config); + try { + for (const [index, migration] of migrations.entries()) { + const name = + migration.name || `migration_${index}_${new Date().getTime()}`; + + const applied = await client.mongoFindOne(MONGO_MIGRATIONS_COLLECTION, { + _id: name, + }); + if (applied) continue; + + console.log(`Applying migration: ${name}...`); + const steps: MongoStep[] = migration.mongoUp ?? []; + + // A migration that carries only SQL has nothing to do here. Saying so is + // better than reporting success for work that never happened. + if (steps.length === 0 && migration.up.length > 0) { + throw new StabilizeError( + `Migration '${name}' contains SQL but no MongoDB steps. ` + + `Generate it with generateMongoMigration() for a MongoDB target.`, + "MIGRATE_ERROR", + ); + } + + // Sequentially, and not in a transaction. @see the note at the top. + await runSteps(client, steps); + + // Recorded only after every step succeeded, so an interrupted migration + // is retried rather than skipped. + await client.mongoInsertOne(MONGO_MIGRATIONS_COLLECTION, { + _id: name, + applied_at: new Date(), + }); + + console.log(`Migration ${name} applied successfully.`); + } + } finally { + await client.close(); + } +} diff --git a/mongo-query.ts b/mongo-query.ts new file mode 100644 index 0000000..21ae7d6 --- /dev/null +++ b/mongo-query.ts @@ -0,0 +1,720 @@ +/** + * @file mongo-query.ts + * @description Translates the query builder's structured predicates into MongoDB + * filters, sort specs, projections and aggregation pipelines. + * @author ElectronSz + * + * Everything in this file is **pure** — no server, no driver, no client — which + * is the point. A filter that is silently wrong returns the wrong rows with no + * error at all, so the translation is the one part of the MongoDB backend that + * has to be verifiable without a database. `tests/mongo.dialect.test.ts` asserts + * it directly. + * + * There is deliberately no `mongodb` import here. Nothing in this module needs + * a driver type: the output is plain documents that the driver serialises. + */ + +import { StabilizeError } from "./types"; + +// ─── PREDICATE MODEL ────────────────────────────────────────────────── +// +// A predicate is what a `QueryBuilder.where*` call records *in addition to* the +// SQL fragment it renders. The SQL arrays are untouched, so the four working +// backends keep emitting byte-identical statements; these sit alongside them. + +/** The comparison operators `whereCompare` accepts. */ +export type CompareOp = "=" | "!=" | ">" | ">=" | "<" | "<="; + +/** + * The shapes a predicate can take. + * + * Each maps to a Mongo operator that has *exactly* SQL's three-valued-logic + * behaviour, including the NULL handling — that last part is where a naive + * translation goes wrong, so it is spelled out per operator in + * {@link renderPredicate}. + */ +export type PredicateOp = + | "cmp" + | "in" + | "nin" + | "null" + | "notNull" + | "between" + | "notBetween" + | "like" + | "notLike" + | "ilike" + | "regex"; + +/** + * One recorded condition. + * + * `column` is a column name as written at the call site — possibly qualified + * (`users.email`) and possibly a *property* key rather than a column name. It is + * not resolved until {@link translateField} runs, because resolution needs the + * model metadata the builder does not hold. + */ +export interface Predicate { + op: PredicateOp; + column: string; + /** The single operand for `cmp` and `like`/`ilike`/`regex`. */ + value?: any; + /** The operand list for `in`/`nin`. */ + values?: any[]; + /** The bounds for `between`/`notBetween`. */ + start?: any; + end?: any; + /** Which comparison `cmp` means. Defaults to `=`. */ + compare?: CompareOp; +} + +/** + * The recorded conditions, as a tree rather than a list. + * + * This shape is not a stylistic choice — it is what makes the translation + * faithful. SQL's `AND` binds tighter than `OR`, and `orWhere` in the builder + * does **not** append a disjunct: it folds everything accumulated so far into a + * single group, so + * + * where(A).where(B).orWhere(C).where(D) + * + * renders as `((A AND B) OR C) AND D`. A flat `[A, B, C(OR), D]` list cannot + * express that — read left-to-right with `AND` precedence it yields + * `(A AND B) OR (C AND D)`, which is a different set of rows. The `or` node + * therefore holds the *whole* left-hand side, exactly as the fold does. + */ +export type MongoFilterNode = + | { kind: "pred"; predicate: Predicate } + | { kind: "and"; items: MongoFilterNode[] } + | { kind: "or"; left: MongoFilterNode; right: MongoFilterNode }; + +/** + * Where a document field name differs from the SQL column name. + * + * The ORM stores documents keyed by **column name**, with the primary key mapped + * to `_id` (see `mongo-repository.ts`). Callers legitimately hand the builder + * either spelling — `repository.ts` passes column names, model-facing code often + * passes property keys — so both are resolved here rather than at ~30 call sites. + */ +export interface MongoFieldContext { + /** The collection name, so a `table.column` qualifier can be stripped. */ + table?: string; + /** The builder's alias, likewise stripped. */ + alias?: string | null; + /** The primary-key *column* name. Defaults to `"id"`. */ + primaryKey?: string; + /** The primary-key *property* key, when it differs from the column name. */ + idProperty?: string; + /** Property key → column name, for callers that hold property keys. */ + columns?: Record; +} + +/** A `count`/`sum`/`avg`/`min`/`max` shortcut recorded by the builder. */ +export interface MongoAggregate { + fn: "count" | "sum" | "avg" | "min" | "max"; + column: string; + alias: string; +} + +/** + * The SQL-only clauses a builder was asked for. + * + * Collected rather than thrown at once, so the error a caller sees can name + * *every* offending method instead of only the first — a query with a `join` and + * a `whereRaw` should report both. + */ +export interface MongoBlockers { + methods: string[]; + details: string[]; +} + +/** The filter, sort, projection and window a query resolved to. */ +export interface MongoQuerySpec { + filter: Record; + projection?: Record; + sort?: Record; + limit?: number; + skip?: number; +} + +/** Everything {@link buildMongoSpec} needs from a builder. */ +export interface MongoSpecInput { + filter?: MongoFilterNode | null; + orderBy?: string[] | null; + limit?: number | null; + offset?: number | null; + select?: string[] | null; + blockers?: MongoBlockers | null; + ctx?: MongoFieldContext; +} + +/** + * A filter that matches no documents. + * + * `$in` with an empty array is the one Mongo idiom for "nothing" that needs no + * field to exist and no server version to support. It stands in for the places + * where SQL's three-valued logic would also match nothing but Mongo has no + * direct spelling: `IN ()`, `NOT IN (1, NULL)`, and a comparison against NULL. + */ +export const MONGO_MATCHES_NOTHING: Record = Object.freeze({ + _id: { $in: [] as any[] }, +}); + +/** + * Returned by {@link sanitizeMongoValue} for values a write should **omit** + * rather than store as null. + * + * A symbol cannot collide with a real field value, so `if (v === OMIT)` is + * unambiguous where `null` would be. + */ +export const OMIT: unique symbol = Symbol("stabilize.mongo.omit"); + +// ─── BLOCKERS ───────────────────────────────────────────────────────── + +export function createMongoBlockers(): MongoBlockers { + return { methods: [], details: [] }; +} + +/** + * Records a method that has no MongoDB equivalent. + * + * @param blockers The list being accumulated on the builder. + * @param method The method name, as the caller would have written it. + * @param detail The offending fragment, quoted back in the error. + */ +export function recordMongoBlocker( + blockers: MongoBlockers, + method: string, + detail?: string, +): void { + if (!blockers.methods.includes(method)) blockers.methods.push(method); + if (detail !== undefined) { + blockers.details.push(`${method}: ${detail}`); + } +} + +/** + * Throws the error a blocked query produces. + * + * Raised when the query is *executed*, not when the clause is added — the + * builder is dialect-agnostic until a client appears, which is what lets one + * builder be rendered for SQL and Mongo. + */ +export function throwMongoUnsupported(blockers: MongoBlockers): never { + const methods = blockers.methods; + const noun = methods.length === 1 ? "has" : "have"; + const detail = blockers.details.length + ? `\n ${blockers.details.join("\n ")}` + : ""; + throw new StabilizeError( + `This query cannot be translated to MongoDB: ${methods.join(", ")} ${noun} ` + + `no MongoDB equivalent. MongoDB is a document store — it has no joins, ` + + `no set operations and no SQL text. Use withRelations() for related ` + + `documents, or run this query against a SQL backend.${detail}`, + "MONGO_UNSUPPORTED", + ); +} + +// ─── FIELD TRANSLATION ──────────────────────────────────────────────── + +/** + * Resolves a call-site field reference to a document field name. + * + * Three rewrites, in order, and the order matters: + * + * 1. A leading `table.` or `alias.` qualifier is stripped. It names the SQL + * table, which in Mongo is the collection — already implied by which + * collection is being queried. + * 2. The primary key, by *either* its property key or its column name, becomes + * `_id`. Checked before the column map so that a model which renames its id + * column (`id` property, `user_id` column) resolves whichever spelling the + * caller used. + * 3. A property key is resolved to its column name, since documents are stored + * under column names. + * + * A remaining dotted path is left alone: relations are separate collections and + * never appear as dotted paths, so a dot here means an embedded document field, + * which Mongo addresses by exactly that path. + */ +export function translateField( + field: string, + ctx: MongoFieldContext = {}, +): string { + let name = String(field).trim(); + + for (const qualifier of [ctx.alias, ctx.table]) { + if (qualifier && name.startsWith(`${qualifier}.`)) { + name = name.slice(qualifier.length + 1); + break; + } + } + + const primaryKey = ctx.primaryKey ?? "id"; + if (ctx.idProperty && name === ctx.idProperty) return "_id"; + if (name === primaryKey) return "_id"; + + if (ctx.columns && Object.prototype.hasOwnProperty.call(ctx.columns, name)) { + const column = ctx.columns[name] ?? name; + // A renamed id column still has to reach `_id`. + return column === primaryKey ? "_id" : column; + } + + return name; +} + +// ─── PREDICATE RENDERING ────────────────────────────────────────────── + +/** + * Translates a SQL `LIKE` pattern into a regular expression. + * + * SQL's wildcards are not regex metacharacters and vice versa, so a pattern has + * to be rebuilt rather than passed through: `%` becomes `.*`, `_` becomes `.`, + * and every regex metacharacter in the literal part is escaped. + * + * The `s` flag is not optional. In a regex `.` does not match a newline, but + * SQL's `%` does, so `LIKE '%foo%'` and `/foo/` disagree about any value + * containing a line break. + * + * @param pattern The SQL pattern, e.g. `"a_c%"`. + * @param caseInsensitive Adds the `i` flag, for `ILIKE`. + */ +export function translateLikePattern( + pattern: any, + caseInsensitive: boolean = false, +): { pattern: string; options: string } { + let body = ""; + for (const ch of String(pattern)) { + if (ch === "%") body += ".*"; + else if (ch === "_") body += "."; + else body += ch.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + } + return { pattern: `^${body}$`, options: caseInsensitive ? "is" : "s" }; +} + +/** + * Renders one predicate to a Mongo filter document. + * + * The recurring difficulty is that SQL and Mongo disagree about absent fields. + * SQL compares against `NULL` and yields `UNKNOWN`, which matches nothing; Mongo + * has a "field is missing" state that most operators happily match. Every + * negation below therefore carries an explicit `$ne: null` — without it, + * `NOT IN` and `NOT LIKE` would return documents SQL would exclude, which is the + * quiet kind of wrong that survives review. + */ +function renderPredicate( + p: Predicate, + ctx: MongoFieldContext, +): Record { + const field = translateField(p.column, ctx); + + switch (p.op) { + case "cmp": + return renderCompare(field, p.compare ?? "=", p.value); + + case "in": { + // SQL's `IN (1, NULL)` can never be satisfied by the NULL — `x = NULL` is + // UNKNOWN regardless of x — so the NULLs are dropped rather than matched. + // That makes `IN (1, NULL)` behave exactly like `IN (1)`, as SQL does. + const values = (p.values ?? []).filter((v) => v !== null && v !== undefined); + return values.length > 0 + ? { [field]: { $in: values } } + : { ...MONGO_MATCHES_NOTHING }; + } + + case "nin": { + // One NULL anywhere in the list makes `NOT IN` unsatisfiable in SQL, since + // every row's comparison against it is UNKNOWN. + const values = p.values ?? []; + if (values.length === 0) return {}; // SQL renders no clause at all here. + if (values.some((v) => v === null || v === undefined)) { + return { ...MONGO_MATCHES_NOTHING }; + } + // `$ne: null` is what excludes documents where the field is missing or + // null; `$nin` alone would match them. + return { [field]: { $nin: values, $ne: null } }; + } + + case "null": + // Matches missing *and* null, which is the intent: a document written + // before the field existed is not soft-deleted. + return { [field]: null }; + + case "notNull": + return { [field]: { $ne: null } }; + + case "between": + if (p.start === null || p.start === undefined) { + return { ...MONGO_MATCHES_NOTHING }; + } + if (p.end === null || p.end === undefined) { + return { ...MONGO_MATCHES_NOTHING }; + } + return { [field]: { $gte: p.start, $lte: p.end } }; + + case "notBetween": + if (p.start === null || p.start === undefined) { + return { ...MONGO_MATCHES_NOTHING }; + } + if (p.end === null || p.end === undefined) { + return { ...MONGO_MATCHES_NOTHING }; + } + // `$not` alone would match missing fields, which `NOT BETWEEN` does not. + return { + $and: [ + { [field]: { $not: { $gte: p.start, $lte: p.end } } }, + { [field]: { $ne: null } }, + ], + }; + + case "like": + case "ilike": { + const { pattern, options } = translateLikePattern( + p.value, + p.op === "ilike", + ); + return { [field]: { $regex: pattern, $options: options } }; + } + + case "notLike": { + // Expressed with `$nor` rather than `$not`. `$not` combined with `$regex` + // is documented inconsistently across server versions, whereas `$nor` over + // a regex is unambiguous — and `$ne: null` then supplies the NULL + // exclusion SQL's `NOT LIKE` performs. + const { pattern, options } = translateLikePattern(p.value, false); + return { + $and: [ + { $nor: [{ [field]: { $regex: pattern, $options: options } }] }, + { [field]: { $ne: null } }, + ], + }; + } + + case "regex": + return { [field]: { $regex: p.value, $options: "s" } }; + + default: { + // Exhaustiveness: a new PredicateOp without a branch is a type error here + // rather than a predicate that silently matches everything. + const never: never = p.op; + throw new StabilizeError( + `Unsupported predicate op: ${String(never)}`, + "MONGO_UNSUPPORTED", + ); + } + } +} + +/** + * Renders a `cmp` predicate for a single comparison operator. + * + * `!=` is the interesting one. `{field: {$ne: v}}` matches documents where the + * field is **missing**, but SQL's `<>` does not — its result there is UNKNOWN. + * The `$ne: null` alongside it is what closes that gap. + */ +function renderCompare( + field: string, + op: CompareOp, + value: any, +): Record { + switch (op) { + case "=": + // Already faithful: a missing field does not equal a value. + return { [field]: value }; + case "!=": + if (value === null || value === undefined) { + // `x <> NULL` is UNKNOWN for every x, so it matches nothing. + return { ...MONGO_MATCHES_NOTHING }; + } + // `$nin` with an explicit `null` rather than `$ne`, because `{f: {$ne: v}}` + // matches documents where `f` is *missing* and SQL's `<>` does not: `$in` + // treats a missing field as null, so excluding null excludes missing too. + return { [field]: { $nin: [value, null] } }; + case ">": + return comparison(field, "$gt", value); + case ">=": + return comparison(field, "$gte", value); + case "<": + return comparison(field, "$lt", value); + case "<=": + return comparison(field, "$lte", value); + default: { + const never: never = op; + throw new StabilizeError( + `Unsupported comparison operator: ${String(never)}`, + "MONGO_UNSUPPORTED", + ); + } + } +} + +/** A range comparison, which matches nothing when the bound is NULL. */ +function comparison( + field: string, + operator: string, + value: any, +): Record { + if (value === null || value === undefined) { + return { ...MONGO_MATCHES_NOTHING }; + } + return { [field]: { [operator]: value } }; +} + +/** + * Renders a predicate tree to a single filter document. + * + * `$and` and `$or` rather than merging into one object, and that is not + * cosmetic. Merging `{age: {$gt: 18}}` with `{age: {$lt: 65}}` produces + * `{age: {$lt: 65}}` — the first condition is silently dropped. `$and` keeps + * both, which is what makes repeated conditions on one field correct. + */ +export function buildMongoFilter( + node: MongoFilterNode | null | undefined, + ctx: MongoFieldContext = {}, +): Record { + if (!node) return {}; + + switch (node.kind) { + case "pred": + return renderPredicate(node.predicate, ctx); + + case "and": { + // `$and` must be a non-empty array, so the degenerate sizes are folded + // away rather than emitted. + const items = node.items.map((item) => buildMongoFilter(item, ctx)); + if (items.length === 0) return {}; + if (items.length === 1) return items[0]!; + return { $and: items }; + } + + case "or": { + // Both sides are always present: the builder only creates an `or` node by + // folding a non-empty left-hand side against the new condition. + return { + $or: [buildMongoFilter(node.left, ctx), buildMongoFilter(node.right, ctx)], + }; + } + + default: { + const never: never = node; + throw new StabilizeError( + `Unsupported filter node: ${JSON.stringify(never)}`, + "MONGO_UNSUPPORTED", + ); + } + } +} + +// ─── SORT / PROJECTION / SPEC ───────────────────────────────────────── + +/** A bare field path, optionally qualified: `users.created_at`. */ +const FIELD_PATH = /^[A-Za-z_$][A-Za-z0-9_$]*(\.[A-Za-z_$][A-Za-z0-9_$]*)*$/; + +/** + * Renders `ORDER BY` clauses to a Mongo sort document. + * + * A clause that is not ` ` came from `orderByRaw` and is an + * arbitrary SQL expression. It throws rather than being passed through: a sort + * key of `"LENGTH(name)"` would be accepted by the driver as a literal field + * name, sort every document as missing, and return rows in no particular order. + */ +export function buildMongoSort( + clauses: string[] | null | undefined, + ctx: MongoFieldContext = {}, +): Record | null { + if (!clauses || clauses.length === 0) return null; + + const sort: Record = {}; + for (const raw of clauses) { + const match = /^(.+?)\s+(ASC|DESC)$/i.exec(String(raw).trim()); + const field = match?.[1]?.trim(); + const direction = match?.[2]?.toUpperCase(); + if (!field || !direction || !FIELD_PATH.test(field)) { + throw new StabilizeError( + `ORDER BY "${raw}" has no MongoDB equivalent. MongoDB sorts by field ` + + `path only; an expression such as orderByRaw() cannot be translated.`, + "MONGO_UNSUPPORTED", + ); + } + sort[translateField(field, ctx)] = direction === "DESC" ? -1 : 1; + } + return sort; +} + +/** + * Renders a `SELECT` list to a Mongo projection. + * + * Returns null — "take the whole document" — when the list is `*` or contains an + * expression, since a projection document cannot hold one. `selectRaw` is + * recorded as a blocker by the builder, so an expression here is already + * reported; returning null keeps this function total. + * + * `_id` is always included explicitly. Mongo's inclusion projections carry it by + * default, and it is where the model's primary key lives, so a projection that + * silently dropped it would leave every row without an `id`. + */ +export function buildMongoProjection( + select: string[] | null | undefined, + ctx: MongoFieldContext = {}, +): Record | null { + if (!select || select.length === 0) return null; + // `String(...)` rather than `select[0].trim()`: the loop below coerces for the + // same reason, and a `select()` handed a non-string would otherwise throw a + // bare TypeError out of `buildMongo` instead of being reported as an + // unbuildable projection. + if (select.length === 1 && String(select[0]).trim() === "*") return null; + + const projection: Record = {}; + for (const raw of select) { + const field = String(raw).trim(); + if (!FIELD_PATH.test(field)) return null; + projection[translateField(field, ctx)] = 1; + } + projection._id = 1; + return projection; +} + +/** + * Assembles the full spec a query resolves to, or throws if it cannot. + * + * @throws StabilizeError `MONGO_UNSUPPORTED` when the builder recorded a clause + * Mongo cannot express, naming every such method at once. + */ +export function buildMongoSpec(input: MongoSpecInput): MongoQuerySpec { + if (input.blockers && input.blockers.methods.length > 0) { + throwMongoUnsupported(input.blockers); + } + + const ctx = input.ctx ?? {}; + const spec: MongoQuerySpec = { + filter: buildMongoFilter(input.filter, ctx), + }; + + const projection = buildMongoProjection(input.select, ctx); + if (projection) spec.projection = projection; + + const sort = buildMongoSort(input.orderBy, ctx); + if (sort) spec.sort = sort; + + if (typeof input.limit === "number") spec.limit = input.limit; + if (typeof input.offset === "number" && input.offset > 0) { + spec.skip = input.offset; + } + + // A skip with no ordering has no stable total order in Mongo — the server is + // free to return a different subset for the same query on consecutive runs, + // so a paging loop can revisit a document or never reach the end. `_id` is + // unique, so ordering by it is a total order and makes the page boundaries + // reproducible. The SQL path is left alone; SQL's unordered LIMIT is + // unspecified too, but changing it would alter existing behaviour. + if (spec.skip !== undefined && !spec.sort) { + spec.sort = { _id: 1 }; + } + + return spec; +} + +// ─── AGGREGATES ─────────────────────────────────────────────────────── + +/** + * Renders the aggregate shortcuts (`count`/`sum`/`avg`/`min`/`max`) to a + * `$group` stage. + * + * `COUNT(column)` counts non-NULL values only, so a bare `$sum: 1` would be + * wrong for anything but `COUNT(*)`. The `$cond` reproduces the SQL definition. + */ +export function buildMongoGroupStage( + aggregates: MongoAggregate[], + ctx: MongoFieldContext = {}, +): Record { + const group: Record = { _id: null }; + + for (const aggregate of aggregates) { + if (aggregate.fn === "count") { + group[aggregate.alias] = + aggregate.column === "*" + ? { $sum: 1 } + : { + $sum: { + $cond: [ + { $ne: [`$${translateField(aggregate.column, ctx)}`, null] }, + 1, + 0, + ], + }, + }; + continue; + } + group[aggregate.alias] = { + [`$${aggregate.fn}`]: `$${translateField(aggregate.column, ctx)}`, + }; + } + + return { $group: group }; +} + +/** + * Builds the aggregation pipeline for a query carrying aggregates. + * + * `limit`/`skip` are deliberately not applied. They belong to the row window, + * and every aggregate shortcut here groups to a single `_id: null` row, so the + * window has nothing to narrow — the same one row SQL would return. + */ +export function buildMongoAggregatePipeline( + filter: Record, + aggregates: MongoAggregate[], + ctx: MongoFieldContext = {}, +): Record[] { + const projection: Record = { _id: 0 }; + for (const aggregate of aggregates) projection[aggregate.alias] = 1; + + return [ + { $match: filter }, + buildMongoGroupStage(aggregates, ctx), + { $project: projection }, + ]; +} + +// ─── VALUE COERCION ─────────────────────────────────────────────────── + +/** + * Prepares a value for storage in a Mongo document. + * + * This is **not** `sanitizeSqlValue` and must not be folded into it. Every one + * of that function's coercions is actively wrong here: + * + * - A `Date` stringified for SQLite fails a `{bsonType: "date"}` validator and + * turns an indexed range query into a string comparison. + * - `boolean` → `1|0` fails `{bsonType: "bool"}`, and makes + * `whereEq("published", true)` match nothing. + * - `undefined` → explicit `NULL` collides in a sparse unique index, which is + * exactly the case a unique-but-optional column creates. + * + * So this passes `Date`, `boolean`, plain objects, arrays and `Buffer` through + * untouched, and reports omission with {@link OMIT} instead of writing a null. + * + * Not recursive: it governs whether a *field* is written, not what is inside a + * document-valued field, where the caller's structure is preserved verbatim. + */ +export function sanitizeMongoValue(value: any): any { + if (value === undefined || value === null) return OMIT; + if (typeof value === "function" || typeof value === "symbol") return OMIT; + return value; +} + +/** + * Rewrites a stored document into the shape the rest of the ORM expects. + * + * `_id` is the Mongo primary key; the model calls it `id`. The rename happens + * here, before `rowTransform` and the relation loaders run, so decryption and + * relation key collection see the id under the name they look for. + */ +export function normalizeMongoDoc( + doc: any, + idColumn: string = "id", +): T { + if (!doc || typeof doc !== "object") return doc; + if (!Object.prototype.hasOwnProperty.call(doc, "_id")) return doc; + + const { _id, ...rest } = doc; + return { ...rest, [idColumn]: _id } as T; +} diff --git a/mongo-repository.ts b/mongo-repository.ts new file mode 100644 index 0000000..74e17b7 --- /dev/null +++ b/mongo-repository.ts @@ -0,0 +1,384 @@ +/** + * @file mongo-repository.ts + * @description The MongoDB bodies behind `Repository`'s write paths. + * @author ElectronSz + * + * Kept beside `repository.ts` rather than inside it because the two storage + * models agree on almost nothing at the statement level: `INSERT … VALUES` and + * `insertOne(doc)` share no structure to factor out. What they *do* share — + * validation, hooks, timestamps, the optimistic-lock seed, encryption, relation + * loading and cache invalidation — stays on the repository and reaches these + * bodies through {@link MongoRepositoryHost}, so none of it is reimplemented. + * + * Documents are keyed by the model's **column name** (`col.name ?? key`), with + * the primary key stored as `_id`. That is the decision that keeps this file + * small: every relation loader, cache key, history writer and transform in the + * repository is already written against column names, so they work unchanged. + */ + +import { DBClient } from "./client"; +import { StabilizeError } from "./types"; +import { OMIT, normalizeMongoDoc, sanitizeMongoValue } from "./mongo-query"; + +/** + * The collection auto-increment ids are drawn from. + * + * One document per table, `_id` being the table name — so uniqueness is + * enforced by the storage layer instead of by a read-then-write the caller + * would have to get right. + */ +export const MONGO_COUNTERS_COLLECTION = "stabilize_counters"; + +/** A column as the write bodies need to see it. */ +export interface MongoColumn { + name: string; + encrypted?: boolean; +} + +/** + * The parts of a `Repository` the MongoDB write bodies depend on. + * + * Deliberately narrow and structural: `Repository` satisfies it through an + * adapter object built inside the class, which keeps these bodies testable + * without a repository and keeps every member it touches explicit. + */ +export interface MongoRepositoryHost { + /** The collection this repository writes to. */ + table: string; + /** Property key → column. Always normalised, see the `Repository` constructor. */ + columns: Record; + /** The primary-key property name. `"id"` by convention. */ + idProperty: string; + /** The primary-key column name, which `_id` stores. */ + idColumn: string; + /** + * The property auto-increment ids are generated for, or null when the key is + * caller-supplied (a UUID or string id). + */ + autoIncrementField: string | null; + logger: { logDebug(message: string): void }; + /** Throws when the entity fails the model's validators. */ + validate(entity: any): void; + /** Applies the timestamp and optimistic-lock defaults a create writes. */ + seedCreateDefaults(entity: Record): Record; + /** Encrypts encrypted columns and normalises values for storage. */ + processForSave(entity: Record): Record; + /** Decrypts encrypted columns. */ + processForLoad(row: any): any; + findOne( + id: number | string, + options: { relations?: string[] }, + client: DBClient, + ): Promise; + loadRelations( + rows: any[], + relations: string[] | undefined, + client: DBClient, + ): Promise; + invalidateRowCache(id: number | string): Promise; + invalidateTableCache(): Promise; + writeThroughRow(id: number | string, row: any): Promise; +} + +/** + * Reads the sequence number out of a `findOneAndUpdate` reply. + * + * Driver 6 returns the document itself — `includeResultMetadata` has defaulted + * to false since NODE-3568 — where driver 5 returned a `ModifyResult` wrapping + * it in `value`. Both are read rather than one being assumed, and neither being + * present is an error: a silent `undefined` here would allocate `NaN` ids and + * fail much later, somewhere else. + * + * @param reply Whatever the driver handed back. + * @param table The collection the counter belongs to, for the message. + * @throws StabilizeError `MONGO_COUNTER_ERROR` when no sequence number is present. + */ +function readCounterSeq(reply: any, table: string): number { + const seq = reply?.seq ?? reply?.value?.seq; + if (typeof seq !== "number" || !Number.isFinite(seq)) { + throw new StabilizeError( + `MongoDB counter for '${table}' did not return a sequence number. ` + + `Expected a numeric 'seq' field on the counters document.`, + "MONGO_COUNTER_ERROR", + ); + } + return seq; +} + +/** + * Reports whether an error is a duplicate-key violation. + * + * The driver's own error is usually wrapped by `mongoRun` in a `StabilizeError`, + * so the code is looked for on the error and on everything it was caused by. + * + * @param error The error to inspect. + */ +function isDuplicateKey(error: unknown): boolean { + let current: any = error; + for (let depth = 0; current && depth < 5; depth++) { + if (current.code === 11000) return true; + if (typeof current.message === "string" && current.message.includes("E11000")) { + return true; + } + current = current.cause; + } + return false; +} + +/** + * Reserves a contiguous block of ids for one collection. + * + * One `$inc` allocates the whole block, so a batch of twenty costs one round + * trip and yields exactly twenty consecutive ids — where the SQL path has to + * guess which rows a multi-row `INSERT` produced. + * + * `$inc` against a field that does not exist yet initialises it to the + * increment, so a brand-new collection yields ids starting at 1. + * + * @param client The client to allocate through. + * @param table The collection the ids belong to. + * @param count How many ids to reserve. Zero or fewer allocates nothing. + * @returns The reserved ids, ascending. + * @throws StabilizeError `MONGO_COUNTER_ERROR` if the counter reports no sequence. + */ +export async function allocateMongoIds( + client: DBClient, + table: string, + count: number, +): Promise { + if (count <= 0) return []; + + const bump = () => + client.mongoFindOneAndUpdate( + MONGO_COUNTERS_COLLECTION, + { _id: table }, + { $inc: { seq: count } }, + { upsert: true, returnDocument: "after" }, + ); + + let reply: any; + try { + reply = await bump(); + } catch (error) { + // Two upserts racing on the same counter `_id`: the loser gets a + // duplicate-key error instead of an id. One retry settles it, because by + // then the document exists and the same call is an ordinary `$inc`. + if (!isDuplicateKey(error)) throw error; + reply = await bump(); + } + + const last = readCounterSeq(reply, table); + const first = last - count + 1; + return Array.from({ length: count }, (_, offset) => first + offset); +} + +/** + * Raises a counter so it is at least `maxId`. + * + * A caller-supplied id has to move the counter, or the next generated id would + * collide with it. SQLite gets this for free from `sqlite_sequence`; MongoDB + * needs it said out loud. `$max` rather than a write, so an id below the current + * sequence — a caller re-inserting a row it read — leaves the counter alone. + * + * @param client The client to write through. + * @param table The collection the counter belongs to. + * @param maxId The highest caller-supplied id in this write. + */ +export async function advanceMongoCounter( + client: DBClient, + table: string, + maxId: number, +): Promise { + await client.mongoUpdateOne( + MONGO_COUNTERS_COLLECTION, + { _id: table }, + { $max: { seq: maxId } }, + { upsert: true }, + ); +} + +/** + * Turns a prepared entity into the document that gets stored. + * + * The primary key is left out: it is written as `_id` by the caller, which is + * what gives the collection a unique index on it for free. + * + * `sanitizeMongoValue` returns {@link OMIT} for `null` and `undefined`, and the + * key is dropped rather than stored as an explicit null. Storing it would put an + * explicit null in every column the caller did not mention, and a sparse unique + * index treats two explicit nulls as a collision — so a second document omitting + * a unique column would be rejected. + * + * @param host The repository the entity belongs to. + * @param entity An entity already through `processForSave`. + */ +function buildMongoDocument( + host: MongoRepositoryHost, + entity: Record, +): Record { + const document: Record = {}; + for (const [key, value] of Object.entries(entity)) { + const column = host.columns[key]; + if (!column) continue; + if (key === host.idProperty) continue; + const sanitized = sanitizeMongoValue(value); + if (sanitized === OMIT) continue; + document[column.name] = sanitized; + } + return document; +} + +/** + * Splits an array into chunks of at most `size`. + * + * @param items The array to split. + * @param size The largest chunk to produce. + */ +function chunked(items: T[], size: number): T[][] { + const chunks: T[][] = []; + for (let i = 0; i < items.length; i += size) { + chunks.push(items.slice(i, i + size)); + } + return chunks; +} + +/** + * Inserts one entity. + * + * The read-back goes through `findOne` rather than reusing the document that + * was just inserted, so the row a caller receives has been through exactly the + * same decryption and relation loading a later read would apply. + * + * @param host The repository being written through. + * @param entity The entity, after the create hooks have run. + * @param options Relation paths to eager-load onto the result. + * @param client The client, which supplies the session and the collection. + * @returns The stored entity. + */ +export async function mongoCreate( + host: MongoRepositoryHost, + entity: Record, + options: { relations?: string[] }, + client: DBClient, +): Promise { + const start = Date.now(); + host.logger.logDebug( + `Creating ${host.table} with data: ${JSON.stringify(entity)}`, + ); + host.validate(entity); + + const prepared = host.processForSave(host.seedCreateDefaults(entity)); + const document = buildMongoDocument(host, prepared); + + const explicit = prepared[host.idProperty]; + if (explicit !== undefined && explicit !== null) { + document._id = explicit; + // A numeric key has to move the counter past it, or the next generated id + // is handed out again and collides with this row. + if (typeof explicit === "number" && Number.isFinite(explicit)) { + await advanceMongoCounter(client, host.table, explicit); + } + } else if (host.autoIncrementField) { + document._id = (await allocateMongoIds(client, host.table, 1))[0]; + } + + await client.mongoInsertOne(host.table, document); + const id = document._id; + + const result = await host.findOne(id, options, client); + + await host.invalidateRowCache(id); + await host.writeThroughRow(id, result); + + host.logger.logDebug( + `Created ${host.table} with ID ${id} in ${Date.now() - start}ms`, + ); + return result; +} + +/** + * Inserts many entities. + * + * Two divergences from the SQL path, both because documents are independent: + * the key sets are not unioned across the batch (that exists to build a single + * multi-row `VALUES` list, which has no analogue here), and the results are not + * re-read from the server. The SQL path re-reads because a multi-row `INSERT` + * does not say which keys it generated; here they are allocated up front and the + * documents that were sent *are* the rows, so a round trip would only be able to + * agree with what is already in hand. + * + * @param host The repository being written through. + * @param entities The entities, after the create hooks have run. + * @param options Batch size and relation paths to eager-load. + * @param client The client, which supplies the session and the collection. + * @returns The stored entities, in the order they were given. + */ +export async function mongoBulkCreate( + host: MongoRepositoryHost, + entities: Record[], + options: { relations?: string[]; batchSize?: number }, + client: DBClient, +): Promise { + const start = Date.now(); + host.logger.logDebug( + `Bulk creating ${entities.length} ${host.table} entities`, + ); + if (!entities.length) return []; + + const batchSize = options.batchSize || 1000; + entities.forEach((entity) => host.validate(entity)); + + const prepared = entities.map((entity) => + host.processForSave(host.seedCreateDefaults(entity)), + ); + + const results: any[] = []; + + for (const batch of chunked(prepared, batchSize)) { + const explicitIds = batch + .map((row) => row[host.idProperty]) + .filter( + (value): value is number => + typeof value === "number" && Number.isFinite(value), + ); + if (explicitIds.length > 0) { + await advanceMongoCounter( + client, + host.table, + Math.max(...explicitIds), + ); + } + + const generatedCount = batch.filter( + (row) => row[host.idProperty] === undefined || row[host.idProperty] === null, + ).length; + const generated = await allocateMongoIds(client, host.table, generatedCount); + + let next = 0; + const documents = batch.map((row) => { + const document = buildMongoDocument(host, row); + const explicit = row[host.idProperty]; + if (explicit !== undefined && explicit !== null) { + document._id = explicit; + } else { + document._id = generated[next++]; + } + return document; + }); + + await client.mongoInsertMany(host.table, documents); + results.push( + ...documents.map((document) => + host.processForLoad(normalizeMongoDoc(document, host.idColumn)), + ), + ); + } + + await host.invalidateTableCache(); + await host.loadRelations(results, options.relations, client); + + host.logger.logDebug( + `Bulk created ${results.length} ${host.table} entities in ${Date.now() - start}ms`, + ); + return results; +} diff --git a/mongo-schema.ts b/mongo-schema.ts new file mode 100644 index 0000000..6d0a479 --- /dev/null +++ b/mongo-schema.ts @@ -0,0 +1,469 @@ +/** + * @file mongo-schema.ts + * @description Derives MongoDB collections, indexes and validators from a model. + * @author ElectronSz + * + * A fourth type mapper rather than a branch inside the three the SQL side + * already has. The three agree on a shape — a type name goes in, a SQL type name + * comes out — and a BSON schema is not that: it is a document, it carries the + * validators the model already declares, and two of its rules (`INTEGER` spans + * several BSON types, an encrypted column is always a string) are properties no + * SQL dialect has. Folding it in would have made the SQL mappers harder to read + * for no SQL benefit. + * + * Two rules here are load-bearing, and both exist so that `autoMigrate` cannot + * break data it has already written: + * + * 1. `validationLevel: "moderate"` on every validator. Under the default + * `"strict"`, an update to a document that lacks a newly declared required + * field is *rejected* — so `repo.update()` would fail on exactly the rows + * the migration just declared a field for. + * 2. `sparse: true` on every unique index. MongoDB treats two documents that + * both lack the field as both null, so they collide; SQL treats two NULLs + * as distinct. Without `sparse`, `email: { unique: true }` would reject the + * second document that simply omits `email`. + */ + +import { DBClient } from "./client"; +import { MetadataStorage, type ColumnConfig } from "./model"; +import { + DataTypes, + RelationType, + StabilizeError, + type MongoStep, +} from "./types"; +import { MONGO_COUNTERS_COLLECTION } from "./mongo-repository"; + +/** Re-exported so schema callers do not have to reach into `types` for it. */ +export type { MongoStep }; + +/** + * Maps a declared column type onto a BSON schema. + * + * `INTEGER` accepts three BSON types, not one: a JavaScript integer becomes an + * `int` inside 32-bit range and a `double` beyond it, so accepting only `"int"` + * would reject every id past two billion. `BIGINT` is deliberately absent for + * the same reason in reverse — a driver cannot tell the caller's `1` was meant + * to be a 64-bit value. + * + * An encrypted column is a string whatever it was declared as, because the + * ciphertext is what is stored. Validating it against its declared type would + * reject every write. + * + * @param column The column to map. + * @returns The `bsonType` (or `bsonType` list) for the column's schema. + */ +export function mapColumnToBsonSchema( + column: ColumnConfig, +): Record { + const declared = + typeof column.type === "string" ? column.type : DataTypes[column.type]; + const type = String(declared).toUpperCase(); + + const schema: Record = {}; + + if (column.encrypted) { + // Ciphertext, always. See above. + schema.bsonType = "string"; + } else { + switch (type) { + case "INTEGER": + schema.bsonType = ["int", "long", "double"]; + break; + case "BIGINT": + schema.bsonType = ["int", "long", "double"]; + break; + case "FLOAT": + case "DOUBLE": + case "DECIMAL": + // MongoDB has no exact decimal unless the caller passes a `Decimal128`. + // A `DECIMAL` column is therefore a double, and a validator that + // demanded exactness would reject everything the ORM writes. + schema.bsonType = ["double", "int", "long", "decimal"]; + break; + case "BOOLEAN": + schema.bsonType = "bool"; + break; + case "DATE": + case "DATETIME": + // A native date, never a string. A string fails this and also defeats + // every range query that could have used an index. + schema.bsonType = ["date", "string"]; + break; + case "JSON": + schema.bsonType = ["object", "array", "string"]; + break; + case "BLOB": + schema.bsonType = ["binData", "string"]; + break; + case "UUID": + case "STRING": + case "TEXT": + default: + schema.bsonType = "string"; + break; + } + } + + // The model's own validators, promoted to server-enforced ones. This is a + // free win: the same rules that `collectValidationErrors` checks in process + // are then checked by the server as well, so a writer that bypasses the ORM + // cannot store a value the model says is invalid. + if (typeof column.minLength === "number") schema.minLength = column.minLength; + if (typeof column.maxLength === "number") schema.maxLength = column.maxLength; + if (typeof column.length === "number" && typeof column.minLength !== "number") { + schema.maxLength = column.length; + } + if (column.pattern) { + // A `RegExp` does not survive a round trip through an aggregation; its + // source does. + schema.pattern = column.pattern.source; + } + + return schema; +} + +/** + * Builds the `$jsonSchema` validator for a model. + * + * `additionalProperties` is deliberately **not** `false`. A collection that + * rejects undeclared fields stops being schemaless, which is the one property + * that makes a document store worth using alongside the four SQL backends — and + * it would make `autoMigrate` a one-way door, because a field written before it + * was declared could never be written again. + * + * `required` includes the primary key: `_id` is the key the collection is + * indexed on, and a document without one has no identity to address. + * + * @param meta The model's metadata. + * @param idColumn The column name the primary key is stored under. + */ +export function buildValidatorFromColumns( + columns: Record, + idColumn: string, +): Record { + const properties: Record = {}; + const required: string[] = ["_id"]; + + for (const [key, column] of Object.entries(columns)) { + const name = column.name ?? key; + // The primary key lives in `_id`, so the column name is not a field. + if (key === "id" || name === idColumn) continue; + properties[name] = mapColumnToBsonSchema(column); + if (column.required && !column.softDelete) required.push(name); + } + + return { + $jsonSchema: { + bsonType: "object", + required, + properties, + }, + }; +} + +/** A collection's derived shape: its validator and the indexes it needs. */ +export interface MongoCollectionPlan { + collection: string; + validator: Record; + indexes: { + spec: Record; + options: Record; + }[]; +} + +/** + * Derives everything a model needs in the database. + * + * Pure: no client, no server. That is what makes `generateMongoMigration` able + * to produce a migration without connecting, and what lets the plan be asserted + * in a unit test. + * + * @param model The model to derive from. + * @returns The collection plan, or null when the model has no table name. + */ +export function planMongoCollection(model: any): MongoCollectionPlan | null { + const meta = MetadataStorage.getModelMetadata(model); + if (!meta?.tableName) return null; + + const columns = meta.columns as Record; + const idColumn = columns["id"]?.name ?? "id"; + const indexes: MongoCollectionPlan["indexes"] = []; + + for (const [key, column] of Object.entries(columns)) { + const name = column.name ?? key; + if (key === "id" || name === idColumn) continue; + if (column.unique) { + // Sparse, always. See the note at the top of this file. + indexes.push({ spec: { [name]: 1 }, options: { unique: true, sparse: true } }); + } else if (column.index) { + indexes.push({ spec: { [name]: 1 }, options: { name: column.index } }); + } + } + + return { + collection: meta.tableName, + validator: buildValidatorFromColumns(columns, idColumn), + indexes, + }; +} + +/** + * Derives the link collection a many-to-many relation needs. + * + * The join table has no model of its own, so `autoMigrate` is the only thing + * that can create it. + * + * The link is keyed by a **compound `_id`** rather than a pair of fields with a + * unique index over them. MongoDB enforces `_id` uniqueness exactly, so + * `attach` becomes idempotent at the storage layer instead of by a read the + * caller has to perform first — and `sync` can report `{attached: 0}` on a + * second call without having checked anything. + * + * @param relation The relation metadata. + */ +export function planMongoLinkCollection(relation: any): MongoCollectionPlan | null { + if (relation.type !== RelationType.ManyToMany) return null; + if (!relation.joinTable || !relation.foreignKey || !relation.inverseKey) { + return null; + } + return { + collection: relation.joinTable, + validator: { + $jsonSchema: { + bsonType: "object", + required: ["_id"], + properties: { + _id: { bsonType: "object" }, + [relation.foreignKey]: {}, + [relation.inverseKey]: {}, + }, + }, + }, + indexes: [ + { spec: { [relation.inverseKey]: 1 }, options: {} }, + { spec: { [relation.foreignKey]: 1 }, options: {} }, + ], + }; +} + +/** + * Lists the link collections every many-to-many relation in a model set needs. + * + * @param models The models being migrated, which may reference each other. + */ +export function planMongoLinkCollections(models: any[]): MongoCollectionPlan[] { + const plans: MongoCollectionPlan[] = []; + const seen = new Set(); + + for (const model of models) { + const relations = MetadataStorage.getModelMetadata(model)?.relations ?? []; + for (const relation of relations) { + const plan = planMongoLinkCollection(relation); + if (plan && !seen.has(plan.collection)) { + seen.add(plan.collection); + plans.push(plan); + } + } + } + + return plans; +} + +/** + * Reports whether a collection already exists. + * + * @param db The client to ask. + * @param collection The collection name. + */ +async function collectionExists( + db: DBClient, + collection: string, +): Promise { + const collections = await db.mongoListCollections(); + return collections.some((entry: any) => entry?.name === collection); +} + +/** + * Creates a collection if it is not there yet, and installs its validator. + * + * An existing collection is not re-validated here. `collMod` is what changes a + * validator, and running it on every `autoMigrate` would replace a validator a + * DBA had tightened by hand. + * + * @param db The client to write through. + * @param plan The collection to ensure. + */ +async function ensureCollection( + db: DBClient, + plan: MongoCollectionPlan, +): Promise { + if (await collectionExists(db, plan.collection)) return; + await db.mongoCommand({ + create: plan.collection, + validator: plan.validator, + // Moderate, always. See the note at the top of this file: "strict" would + // reject updates to documents that predate a newly required field. + validationLevel: "moderate", + validationAction: "error", + }); +} + +/** + * Installs the indexes a collection plan asks for. + * + * `createIndex` is idempotent in MongoDB — asking for an index that already + * exists with the same spec and options is a no-op — so a second `autoMigrate` + * is safe. Asking for the *same name* with different options is an error + * (`IndexOptionsConflict`), which is the honest outcome: the change would need a + * drop first, and guessing that here would drop an index in production. + * + * @param db The client to write through. + * @param plan The collection whose indexes to install. + */ +async function ensureIndexes( + db: DBClient, + plan: MongoCollectionPlan, +): Promise { + for (const index of plan.indexes) { + await db.mongoCreateIndex(plan.collection, index.spec, index.options); + } +} + +/** + * Pre-creates a table's counter document. + * + * The steady-state allocation path upserts, so this is not required for + * correctness — but without it the *first* write on a table has two concurrent + * requests both upserting the same `_id`, and one of them gets a duplicate-key + * error. Creating it up front (`$setOnInsert`, so an existing counter is left + * alone) means the concurrent path is always a plain `$inc`. + * + * @param db The client to write through. + * @param collection The table whose counter to create. + */ +async function ensureCounter(db: DBClient, collection: string): Promise { + await db.mongoUpdateOne( + MONGO_COUNTERS_COLLECTION, + { _id: collection }, + { $setOnInsert: { seq: 0 } }, + { upsert: true }, + ); +} + +/** + * Whether a model's key is generated rather than supplied by the caller. + * + * Mirrors `Repository#getAutoIncrementField`, and must keep mirroring it: if + * this created a counter for a table whose keys are strings, the counter would + * simply never be read. + * + * @param columns The model's columns. + */ +function usesGeneratedIds(columns: Record): boolean { + const idColumn = columns["id"]; + if (!idColumn) return false; + const type = ( + typeof idColumn.type === "string" ? idColumn.type : DataTypes[idColumn.type] + ).toUpperCase(); + return type !== "STRING" && type !== "TEXT" && type !== "UUID"; +} + +/** + * Brings a set of models' collections in line with their declarations. + * + * The add-only contract the SQL side has is expressed here against the + * validator rather than against sampled documents, because a document either + * carries a field or does not — there is nothing to backfill. A document without + * the field reads back `undefined`, which is exactly what `ADD COLUMN` with no + * default produces anyway. + * + * @param db The client to migrate through. + * @param models The models to bring up to date. + */ +export async function mongoAutoMigrate( + db: DBClient, + models: any[], +): Promise { + const plans: MongoCollectionPlan[] = []; + for (const model of models) { + const plan = planMongoCollection(model); + if (!plan) { + throw new StabilizeError( + `Model is missing tableName. Use defineModel() or add static schema.`, + "MIGRATE_ERROR", + ); + } + plans.push(plan); + } + plans.push(...planMongoLinkCollections(models)); + + for (const plan of plans) { + await ensureCollection(db, plan); + await ensureIndexes(db, plan); + } + + // The counters collection itself, so a fresh database has somewhere for the + // first allocation to land. + await ensureCollection(db, { + collection: MONGO_COUNTERS_COLLECTION, + validator: {}, + indexes: [], + }); + + for (const model of models) { + const meta = MetadataStorage.getModelMetadata(model); + if (!meta) continue; + if (usesGeneratedIds(meta.columns as Record)) { + await ensureCounter(db, meta.tableName); + } + } +} + +/** + * Builds the migration steps that would bring a model's collection up to date. + * + * Pure, so a migration can be generated — and asserted — without a server. + * + * @param model The model to generate for. + * @param direction Whether this is the `up` or the `down` direction. + */ +export function generateMongoSteps( + model: any, + direction: "up" | "down", +): MongoStep[] { + const plan = planMongoCollection(model); + if (!plan) return []; + + if (direction === "down") { + // Dropping the collection is the only inverse that is actually total: the + // indexes and the validator go with it, so a re-`up` rebuilds from the + // declaration rather than from whatever survived. + return [{ kind: "dropCollection", collection: plan.collection }]; + } + + const steps: MongoStep[] = [ + { kind: "createCollection", collection: plan.collection, validator: plan.validator }, + ]; + for (const index of plan.indexes) { + steps.push({ + kind: "createIndex", + collection: plan.collection, + spec: index.spec, + options: index.options, + }); + } + return steps; +} + +/** Reports whether a model's key is generated. @see usesGeneratedIds */ +export function modelUsesGeneratedIds(model: any): boolean { + const meta = MetadataStorage.getModelMetadata(model); + if (!meta) return false; + return usesGeneratedIds(meta.columns as Record); +} + +/** The collection a model is stored in. */ +export function mongoCollectionName(model: any): string | null { + return MetadataStorage.getModelMetadata(model)?.tableName ?? null; +} diff --git a/package.json b/package.json index 20b4a4f..b99e8e4 100644 --- a/package.json +++ b/package.json @@ -61,7 +61,7 @@ ], "scripts": { "build": "bun run build:js && bun run build:types", - "build:js": "bun build index.ts repository.ts query-builder.ts client.ts cache.ts model.ts types.ts hooks.ts logger.ts migrations.ts auto-migrate.ts utils/encryption.ts --outdir dist --target bun --sourcemap=external --minify", + "build:js": "bun build index.ts repository.ts query-builder.ts client.ts cache.ts model.ts types.ts hooks.ts logger.ts migrations.ts auto-migrate.ts mongo-query.ts mongo-repository.ts mongo-schema.ts mongo-migrate.ts utils/encryption.ts --outdir dist --target bun --sourcemap=external --minify --external mongodb", "build:types": "bun tsc -p tsconfig.build.json", "prepublishOnly": "bun run build", "typecheck": "bun tsc --noEmit", @@ -104,6 +104,9 @@ "mysql2": "^3.15.2", "pg": "^8.16.3" }, + "optionalDependencies": { + "mongodb": "^6.20.0" + }, "peerDependencies": { "bun": ">=1.0.0" }, diff --git a/query-builder.ts b/query-builder.ts index e45c9d6..96f7783 100644 --- a/query-builder.ts +++ b/query-builder.ts @@ -8,6 +8,21 @@ import { DBClient } from "./client"; import { Cache } from "./cache"; import { MetadataStorage } from "./model"; import { DBType, StabilizeError } from "./types"; +import { + buildMongoAggregatePipeline, + buildMongoProjection, + buildMongoSpec, + createMongoBlockers, + normalizeMongoDoc, + recordMongoBlocker, + type CompareOp, + type MongoAggregate, + type MongoBlockers, + type MongoFieldContext, + type MongoFilterNode, + type MongoQuerySpec, + type Predicate, +} from "./mongo-query"; type JoinType = "INNER" | "LEFT" | "RIGHT" | "FULL" | "CROSS"; type LockMode = @@ -122,6 +137,15 @@ export class QueryBuilder { params: any[]; recursive: boolean; }[] = []; + /** + * The conditions this builder recorded for MongoDB, as a tree. + * @see MongoFilterNode for why this is not a flat list. + */ + private mongoAndList: MongoFilterNode[] = []; + /** Clauses that were asked for and have no MongoDB equivalent. */ + private mongoBlockers: MongoBlockers = createMongoBlockers(); + /** The aggregate this builder selects, when it used a shortcut for one. */ + private mongoAggregates: MongoAggregate[] | null = null; constructor(table: string) { this.table = table; @@ -135,6 +159,7 @@ export class QueryBuilder { } selectRaw(expression: string, ...params: any[]): QueryBuilder { + this.markMongoUnsupported("selectRaw", expression); this.selectFields.push(expression); // Kept separate from `whereParams`: the SELECT list is emitted before the // WHERE clause, so sharing one array would bind the values out of order. @@ -143,6 +168,10 @@ export class QueryBuilder { } distinct(): QueryBuilder { + // `SELECT DISTINCT` is a statement-wide modifier, whereas Mongo distincts a + // single field and returns a value list rather than documents. Reported + // rather than dropped, which would return duplicates SQL would not. + this.markMongoUnsupported("distinct"); this.isDistinct = true; return this; } @@ -171,32 +200,46 @@ export class QueryBuilder { count(column: string = "*", alias: string = "count"): QueryBuilder { this.selectFields = [`COUNT(${column}) AS ${alias}`]; + // Recorded alongside, so `buildMongo` can emit a `$group` stage instead of + // reporting the SQL aggregate expression as untranslatable. Set rather than + // pushed: these shortcuts replace `selectFields`, so a second call replaces + // the first aggregate rather than accumulating one. + this.mongoAggregates = [{ fn: "count", column, alias }]; return this; } sum(column: string, alias: string = "sum"): QueryBuilder { this.selectFields = [`SUM(${column}) AS ${alias}`]; + this.mongoAggregates = [{ fn: "sum", column, alias }]; return this; } avg(column: string, alias: string = "avg"): QueryBuilder { this.selectFields = [`AVG(${column}) AS ${alias}`]; + this.mongoAggregates = [{ fn: "avg", column, alias }]; return this; } min(column: string, alias: string = "min"): QueryBuilder { this.selectFields = [`MIN(${column}) AS ${alias}`]; + this.mongoAggregates = [{ fn: "min", column, alias }]; return this; } max(column: string, alias: string = "max"): QueryBuilder { this.selectFields = [`MAX(${column}) AS ${alias}`]; + this.mongoAggregates = [{ fn: "max", column, alias }]; return this; } // ─── WHERE ──────────────────────────────────────────────────────── where(condition: string, ...params: any[]): QueryBuilder { + // There is no SQL text to translate, and no parser here that could be + // trusted with one — a `where("a = 1 OR b = 2")` misread would return the + // wrong rows silently. Reported at execute time instead, alongside any + // other blocked clause. + this.markMongoUnsupported("where", condition); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${condition}`); } else { @@ -207,6 +250,7 @@ export class QueryBuilder { } orWhere(condition: string, ...params: any[]): QueryBuilder { + this.markMongoUnsupported("orWhere", condition); if (this.whereConditions.length === 0) { this.whereConditions.push(condition); } else { @@ -224,6 +268,7 @@ export class QueryBuilder { } whereNot(condition: string, ...params: any[]): QueryBuilder { + this.markMongoUnsupported("whereNot", condition); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND NOT (${condition})`); } else { @@ -234,6 +279,7 @@ export class QueryBuilder { } whereIn(column: string, values: any[]): QueryBuilder { + this.addMongoPredicate({ op: "in", column, values }); if (values.length === 0) { if (this.whereConditions.length > 0) { this.whereConditions.push("AND 1 = 0"); @@ -253,6 +299,7 @@ export class QueryBuilder { } whereNotIn(column: string, values: any[]): QueryBuilder { + this.addMongoPredicate({ op: "nin", column, values }); if (values.length === 0) return this; const placeholders = values.map(() => "?").join(", "); if (this.whereConditions.length > 0) { @@ -265,6 +312,7 @@ export class QueryBuilder { } whereNull(column: string): QueryBuilder { + this.addMongoPredicate({ op: "null", column }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} IS NULL`); } else { @@ -274,6 +322,7 @@ export class QueryBuilder { } whereNotNull(column: string): QueryBuilder { + this.addMongoPredicate({ op: "notNull", column }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} IS NOT NULL`); } else { @@ -283,6 +332,7 @@ export class QueryBuilder { } whereBetween(column: string, start: any, end: any): QueryBuilder { + this.addMongoPredicate({ op: "between", column, start, end }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} BETWEEN ? AND ?`); } else { @@ -293,6 +343,7 @@ export class QueryBuilder { } whereNotBetween(column: string, start: any, end: any): QueryBuilder { + this.addMongoPredicate({ op: "notBetween", column, start, end }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} NOT BETWEEN ? AND ?`); } else { @@ -303,6 +354,7 @@ export class QueryBuilder { } whereLike(column: string, pattern: string): QueryBuilder { + this.addMongoPredicate({ op: "like", column, value: pattern }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} LIKE ?`); } else { @@ -313,6 +365,7 @@ export class QueryBuilder { } whereILike(column: string, pattern: string): QueryBuilder { + this.addMongoPredicate({ op: "ilike", column, value: pattern }); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${column} ILIKE ?`); } else { @@ -322,7 +375,63 @@ export class QueryBuilder { return this; } + // ─── STRUCTURED COMPARISONS ─────────────────────────────────────── + // + // The methods above take a column and operands; `where` takes SQL text. These + // cover the one shape that has no structured form yet — a bare comparison — + // so that a caller never has to reach for `where("a > ?")` and lose the + // MongoDB translation. They render to SQL exactly as the equivalent `where` + // call would. + + whereEq(column: string, value: any): QueryBuilder { + this.addMongoPredicate({ op: "cmp", column, compare: "=", value }); + return this.pushComparison(column, "=", value); + } + + whereNotEq(column: string, value: any): QueryBuilder { + this.addMongoPredicate({ op: "cmp", column, compare: "!=", value }); + return this.pushComparison(column, "!=", value); + } + + whereCompare(column: string, op: CompareOp, value: any): QueryBuilder { + this.addMongoPredicate({ op: "cmp", column, compare: op, value }); + return this.pushComparison(column, op, value); + } + + orWhereEq(column: string, value: any): QueryBuilder { + this.foldMongoOr({ kind: "pred", predicate: { op: "cmp", column, compare: "=", value } }); + return this.pushComparison(column, "=", value, true); + } + + orWhereCompare(column: string, op: CompareOp, value: any): QueryBuilder { + this.foldMongoOr({ kind: "pred", predicate: { op: "cmp", column, compare: op, value } }); + return this.pushComparison(column, op, value, true); + } + + orWhereNull(column: string): QueryBuilder { + this.foldMongoOr({ kind: "pred", predicate: { op: "null", column } }); + return this.pushCondition(`${column} IS NULL`, [], true); + } + + orWhereNotNull(column: string): QueryBuilder { + this.foldMongoOr({ kind: "pred", predicate: { op: "notNull", column } }); + return this.pushCondition(`${column} IS NOT NULL`, [], true); + } + + orWhereIn(column: string, values: any[]): QueryBuilder { + this.foldMongoOr({ kind: "pred", predicate: { op: "in", column, values } }); + if (values.length === 0) { + return this.pushCondition("1 = 0", [], true); + } + const placeholders = values.map(() => "?").join(", "); + return this.pushCondition(`${column} IN (${placeholders})`, values, true); + } + whereExists(builderOrSql: string | QueryBuilder): QueryBuilder { + this.markMongoUnsupported( + "whereExists", + typeof builderOrSql === "string" ? builderOrSql : "subquery", + ); const sql = typeof builderOrSql === "string" ? builderOrSql @@ -339,6 +448,10 @@ export class QueryBuilder { } whereNotExists(builderOrSql: string | QueryBuilder): QueryBuilder { + this.markMongoUnsupported( + "whereNotExists", + typeof builderOrSql === "string" ? builderOrSql : "subquery", + ); const sql = typeof builderOrSql === "string" ? builderOrSql @@ -355,6 +468,7 @@ export class QueryBuilder { } whereRaw(rawSql: string, ...params: any[]): QueryBuilder { + this.markMongoUnsupported("whereRaw", rawSql); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${rawSql}`); } else { @@ -366,6 +480,10 @@ export class QueryBuilder { /** Knex-style column-to-column comparison: .whereRef('orders.user_id', '=', 'users.id') */ whereRef(leftCol: string, op: string, rightCol: string): QueryBuilder { + // Column-to-column comparison is what `$expr` does, but only for the four + // arithmetic operators, and the caller may pass any SQL operator. Reported + // rather than half-translated. + this.markMongoUnsupported("whereRef", `${leftCol} ${op} ${rightCol}`); if (this.whereConditions.length > 0) { this.whereConditions.push(`AND ${leftCol} ${op} ${rightCol}`); } else { @@ -380,32 +498,35 @@ export class QueryBuilder { type: JoinType, table: string, condition: string, + method: string, ): QueryBuilder { + this.markMongoUnsupported(method, `${type} JOIN ${table} ON ${condition}`); this.joins.push(`${type} JOIN ${table} ON ${condition}`); return this; } join(table: string, condition: string): QueryBuilder { - return this.addJoin("LEFT", table, condition); + return this.addJoin("LEFT", table, condition, "join"); } innerJoin(table: string, condition: string): QueryBuilder { - return this.addJoin("INNER", table, condition); + return this.addJoin("INNER", table, condition, "innerJoin"); } leftJoin(table: string, condition: string): QueryBuilder { - return this.addJoin("LEFT", table, condition); + return this.addJoin("LEFT", table, condition, "leftJoin"); } rightJoin(table: string, condition: string): QueryBuilder { - return this.addJoin("RIGHT", table, condition); + return this.addJoin("RIGHT", table, condition, "rightJoin"); } fullJoin(table: string, condition: string): QueryBuilder { - return this.addJoin("FULL", table, condition); + return this.addJoin("FULL", table, condition, "fullJoin"); } crossJoin(table: string): QueryBuilder { + this.markMongoUnsupported("crossJoin", table); this.joins.push(`CROSS JOIN ${table}`); return this; } @@ -437,6 +558,7 @@ export class QueryBuilder { * ``` */ orderByRaw(expression: string, direction?: "ASC" | "DESC"): QueryBuilder { + this.markMongoUnsupported("orderByRaw", expression); const clause = direction ? `${expression} ${direction}` : expression; this.orderByClauses.push(clause); return this; @@ -458,11 +580,17 @@ export class QueryBuilder { * ``` */ groupByRaw(expression: string): QueryBuilder { + this.markMongoUnsupported("groupByRaw", expression); this.groupByClauses.push(expression); return this; } having(condition: string, ...params: any[]): QueryBuilder { + // HAVING filters groups; Mongo filters documents with `$match` and groups + // with `$group`, and a post-group filter needs `$match` placed *after* the + // `$group` stage. Translating the condition itself is the blocker, and the + // condition is SQL text — so this is reported rather than guessed at. + this.markMongoUnsupported("having", condition); if (this.havingConditions.length > 0) { this.havingConditions.push(`AND ${condition}`); } else { @@ -538,6 +666,7 @@ export class QueryBuilder { // ─── SET OPERATIONS ─────────────────────────────────────────────── union(builder: QueryBuilder): QueryBuilder { + this.markMongoUnsupported("union"); this.unions.push({ query: builder.build().query, params: builder.build().params, @@ -547,6 +676,7 @@ export class QueryBuilder { } unionAll(builder: QueryBuilder): QueryBuilder { + this.markMongoUnsupported("unionAll"); this.unions.push({ query: builder.build().query, params: builder.build().params, @@ -558,6 +688,7 @@ export class QueryBuilder { // ─── COMMON TABLE EXPRESSIONS ───────────────────────────────────── with(name: string, builder: QueryBuilder): QueryBuilder { + this.markMongoUnsupported("with", name); this.ctas.push({ name, query: builder.build().query, @@ -568,6 +699,7 @@ export class QueryBuilder { } withRecursive(name: string, builder: QueryBuilder): QueryBuilder { + this.markMongoUnsupported("withRecursive", name); this.ctas.push({ name, query: builder.build().query, @@ -655,6 +787,16 @@ export class QueryBuilder { q.relationLoader = this.relationLoader; q.unions = [...this.unions]; q.ctas = [...this.ctas]; + // The nodes are treated as immutable once recorded — the recording methods + // replace the tree rather than mutate a node in place — so sharing them with + // the clone is safe. The blocker *arrays* are not shared: a clone that + // records a new blocker must not make the original report it. + q.mongoAndList = [...this.mongoAndList]; + q.mongoBlockers = { + methods: [...this.mongoBlockers.methods], + details: [...this.mongoBlockers.details], + }; + q.mongoAggregates = this.mongoAggregates ? [...this.mongoAggregates] : null; return q; } @@ -738,8 +880,273 @@ export class QueryBuilder { return this.build(dialect); } + // ─── MONGODB ────────────────────────────────────────────────────── + // + // The builder stays dialect-agnostic: it renders SQL fragments and records + // structured predicates side by side, and only decides which to use once a + // client appears. Nothing here changes what `build()` emits. + + /** Appends a comparison fragment, ANDed or ORed into the WHERE clause. */ + private pushComparison( + column: string, + op: CompareOp, + value: any, + isOr: boolean = false, + ): QueryBuilder { + return this.pushCondition(`${column} ${op} ?`, [value], isOr); + } + + /** + * Appends a condition, mirroring the fold `orWhere` performs. + * + * Shared by the structured comparison methods so their SQL output is + * character-for-character what the equivalent `where`/`orWhere` call + * produced, which is what keeps the four SQL backends unaffected. + */ + private pushCondition( + condition: string, + params: any[], + isOr: boolean = false, + ): QueryBuilder { + if (isOr) { + if (this.whereConditions.length === 0) { + this.whereConditions.push(condition); + } else { + this.whereConditions = [ + `(${this.whereConditions.join(" ")} OR (${condition}))`, + ]; + } + } else if (this.whereConditions.length > 0) { + this.whereConditions.push(`AND ${condition}`); + } else { + this.whereConditions.push(condition); + } + this.whereParams.push(...params); + return this; + } + + /** Records a structured condition for MongoDB, ANDed with the rest. */ + private addMongoPredicate(predicate: Predicate): void { + this.mongoAndList.push({ kind: "pred", predicate }); + } + + /** + * Folds a condition into the MongoDB tree as a disjunct. + * + * Deliberately identical to what `orWhere` does to the SQL fragment array: + * an empty list takes the condition bare, and otherwise the *whole* list so + * far becomes the left-hand side of the `or`. A flat list with an `OR` flag + * would not survive the round trip — see {@link MongoFilterNode}. + */ + private foldMongoOr(node: MongoFilterNode): void { + if (this.mongoAndList.length === 0) { + this.mongoAndList.push(node); + return; + } + this.mongoAndList = [ + { + kind: "or", + left: { kind: "and", items: this.mongoAndList }, + right: node, + }, + ]; + } + + /** Records a clause that cannot be expressed against MongoDB. */ + private markMongoUnsupported(method: string, detail?: string): void { + recordMongoBlocker(this.mongoBlockers, method, detail); + } + + /** + * The field-translation context, resolved from the model metadata. + * + * The primary key needs no flag on the column config: it is `id` by the same + * convention the repository already relies on when it writes `id = ?` and + * asks `getAutoIncrementField()` for the auto-increment column. + */ + private mongoContext(): MongoFieldContext { + const ctx: MongoFieldContext = { table: this.table, alias: this.tableAlias }; + const model = MetadataStorage.getModelByTableName(this.table); + if (!model) return ctx; + + const columns = MetadataStorage.getColumns(model); + const columnNames: Record = {}; + for (const [key, config] of Object.entries(columns)) { + columnNames[key] = config.name ?? key; + } + ctx.columns = columnNames; + ctx.idProperty = "id"; + ctx.primaryKey = columnNames["id"] ?? "id"; + return ctx; + } + + /** The aggregate this builder selects, if it used one of the shortcuts. */ + getMongoAggregates(): MongoAggregate[] | null { + return this.mongoAggregates ? [...this.mongoAggregates] : null; + } + + /** + * Renders this query as a MongoDB spec. + * + * Collected blockers are raised here rather than when the offending clause was + * added, because a builder is dialect-agnostic right up until a client is + * known — that is what lets one builder render for either backend. + * + * A `lock()`/`forUpdate()` is **not** reported. MongoDB has no row locking to + * map it onto and the call is a no-op, matching what the SQL Server path + * already does with it; the repository logs that where it has a logger. + * + * @throws StabilizeError `MONGO_UNSUPPORTED` naming every clause that has no + * MongoDB equivalent. + */ + buildMongo(): MongoQuerySpec { + // `selectRaw` records its own blocker, but `select()` accepts an expression + // too. A projection that cannot be built has to be reported rather than + // quietly widened to the whole document. + const selectsEverything = + this.selectFields.length === 1 && this.selectFields[0] === "*"; + if (!selectsEverything && !this.mongoAggregates) { + const ctx = this.mongoContext(); + if (!buildMongoProjection(this.selectFields, ctx)) { + recordMongoBlocker( + this.mongoBlockers, + "select", + this.selectFields.join(", "), + ); + } + } + + return buildMongoSpec({ + filter: { kind: "and", items: this.mongoAndList }, + orderBy: this.orderByClauses, + limit: this.limitValue, + offset: this.offsetValue, + select: this.selectFields, + blockers: this.mongoBlockers, + ctx: this.mongoContext(), + }); + } + // ─── EXECUTE ────────────────────────────────────────────────────── + /** + * Reads through MongoDB. + * + * Shares its shape with the SQL path on purpose: transform, then relations, + * then cache. Caching before the transform would cache raw column values, and + * caching before the relation load would cache a row with no relations on it, + * for the same reasons the SQL path orders it this way. + * + * @param client The client, which supplies the collection and any session. + * @param cache Optional cache to read through. + * @param cacheKey Key to read and write the cache under. + */ + private async executeMongo( + client: DBClient, + cache?: Cache, + cacheKey?: string, + ): Promise { + const cacheable = + cache && cacheKey && !(client as any).isTransactionClient; + + if (cacheable) { + const cached = await cache.get(cacheKey!); + if (cached) return cached; + } + + const spec = this.buildMongo(); + const idColumn = this.mongoContext().primaryKey ?? "id"; + + let results: T[]; + if (this.mongoAggregates) { + // An aggregate replaces the projection: the caller asked for one number + // per group, not for the documents behind it. + const rows = await client.mongoAggregate( + this.table, + buildMongoAggregatePipeline( + spec.filter, + this.mongoAggregates, + this.mongoContext(), + ), + ); + results = rows as T[]; + } else { + const options: Record = {}; + if (spec.projection) options.projection = spec.projection; + if (spec.sort) options.sort = spec.sort; + if (spec.limit !== undefined) options.limit = spec.limit; + if (spec.skip !== undefined) options.skip = spec.skip; + + const rows = await client.mongoFind(this.table, spec.filter, options); + results = rows.map((row) => normalizeMongoDoc(row, idColumn)); + } + + if (this.rowTransform) results = this.rowTransform(results); + + if (this.relationLoader && this.eagerRelations.length > 0) { + results = await this.relationLoader(results, this.eagerRelations, client); + } + + if (cacheable && results.length > 0) { + await cache!.set(cacheKey!, results, cache!.config?.ttl ?? 60); + } + + return results; + } + + /** + * Counts matching documents. + * + * The filter is rebuilt without the sort, limit and skip: a count answers + * "how many match", and carrying the paging clauses over would either count a + * page or make the `skip`-implies-`_id`-sort rule add an order the caller + * never asked for. A grouped query is counted from the group stage instead, + * because there the rows are the groups. + * + * @param client The client to count through. + */ + private async countMongo(client: DBClient): Promise { + const filter = buildMongoSpec({ + filter: { kind: "and", items: this.mongoAndList }, + select: ["*"], + blockers: this.mongoBlockers, + ctx: this.mongoContext(), + }).filter; + + if (this.mongoAggregates) { + const rows: any[] = await client.mongoAggregate( + this.table, + buildMongoAggregatePipeline(filter, this.mongoAggregates, this.mongoContext()), + ); + return Number(rows[0]?.[this.mongoAggregates[0]!.alias] ?? 0); + } + + return client.mongoCount(this.table, filter); + } + + /** + * Reports whether any document matches. + * + * Reads one `_id` rather than counting: the answer is the same and the server + * can stop at the first match. + * + * @param client The client to probe through. + */ + private async existsMongo(client: DBClient): Promise { + const filter = buildMongoSpec({ + filter: { kind: "and", items: this.mongoAndList }, + select: ["*"], + blockers: this.mongoBlockers, + ctx: this.mongoContext(), + }).filter; + + const rows = await client.mongoFind(this.table, filter, { + projection: { _id: 1 }, + limit: 1, + }); + return rows.length > 0; + } + async execute( client: DBClient, cache?: Cache, @@ -748,6 +1155,9 @@ export class QueryBuilder { // Only the client knows which dialect this statement will be sent to, so // the row-limiting clause is decided here rather than at build time. this.dialect = client.config.type; + if (this.dialect === DBType.MongoDB) { + return this.executeMongo(client, cache, cacheKey); + } const { query, params } = this.build(); // Never cache a read taken inside an open transaction: the rows may be @@ -783,6 +1193,9 @@ export class QueryBuilder { async countExec(client: DBClient): Promise { const clone = this.clone(); clone.dialect = client.config.type; + if (clone.dialect === DBType.MongoDB) { + return clone.countMongo(client); + } clone.orderByClauses = []; clone.limitValue = null; clone.offsetValue = null; @@ -815,6 +1228,9 @@ export class QueryBuilder { async existsExec(client: DBClient): Promise { const clone = this.clone(); clone.dialect = client.config.type; + if (clone.dialect === DBType.MongoDB) { + return clone.existsMongo(client); + } clone.selectFields = ["1"]; clone.orderByClauses = []; clone.limitValue = 1; diff --git a/repository.ts b/repository.ts index 24679d8..050a60e 100644 --- a/repository.ts +++ b/repository.ts @@ -18,9 +18,22 @@ import { import { MetadataStorage } from "./model"; import { getHooks, type HookType } from "./hooks"; import { decrypt, encrypt } from "./utils/encryption"; +import { + mongoBulkCreate, + mongoCreate, + type MongoRepositoryHost, +} from "./mongo-repository"; type VersionOperation = "insert" | "update" | "delete"; +/** + * The property name a model's primary key is declared under. + * + * A convention rather than metadata: `ColumnConfig` carries no primary-key flag, + * so `id` is the key by definition across every backend. + */ +const ID_PROPERTY = "id"; + /** * How many key values go into one `IN (…)` when a relation is loaded. * @@ -547,7 +560,7 @@ export class Repository { // is in sight — so it has to know the dialect from the outset. qb.withDialect(this.client.config.type); if (this.softDeleteField) { - qb.where(`${this.table}.${this.softDeleteColumn} IS NULL`); + qb.whereNull(`${this.table}.${this.softDeleteColumn}`); } // Lets `find().withRelations(...)` work: the builder records the paths and // calls back here to load them, since only the repository has the model @@ -575,7 +588,7 @@ export class Repository { // decrypted) and calls back into `loadRelations` before writing the cache, // so a cached entry holds the relations a later hit is asked for. const result = await this.find() - .where(`${this.table}.id = ?`, id) + .whereEq(`${this.table}.id`, id) .limit(1) .withRelations(options.relations ?? []) .execute(client, this.cache!, cacheKey); @@ -729,34 +742,79 @@ export class Repository { }); } + /** + * Fills in the columns a create writes that the caller did not supply. + * + * Shared by the SQL and MongoDB insert paths, so the two cannot drift on + * which defaults a new row carries. + * + * @param entity The entity as the create hooks left it. + * @returns A copy carrying the seeded timestamps and optimistic lock. + */ + private seedCreateDefaults(entity: Partial): Record { + const timestamps = this.timestampsConfig; + const row = { ...entity } as Record; + if (timestamps?.createdAt && !row[timestamps.createdAt]) { + row[timestamps.createdAt] = new Date().toISOString(); + } + if (timestamps?.updatedAt && !row[timestamps.updatedAt]) { + row[timestamps.updatedAt] = new Date().toISOString(); + } + // Seed the optimistic lock so the first `update()` has a version to match + // on; without this the column stays NULL and `version = NULL` never + // matches, which makes every update look like a conflict. + if ( + this.optimisticLockField && + row[this.optimisticLockField] === undefined + ) { + row[this.optimisticLockField] = 1; + } + return row; + } + + /** + * The repository's own members, narrowed to what the MongoDB write bodies + * need. Built here rather than handed over as `this` because most of what + * they reach for is private, and because the object literal is a single + * readable list of exactly what the Mongo path depends on. + */ + private get mongoCtx(): MongoRepositoryHost { + return { + table: this.table, + columns: this.columns, + idProperty: ID_PROPERTY, + idColumn: this.columns[ID_PROPERTY]?.name ?? ID_PROPERTY, + autoIncrementField: this.autoIncrementField, + logger: this.logger, + validate: (entity) => this.validate(entity), + seedCreateDefaults: (entity) => this.seedCreateDefaults(entity as Partial), + processForSave: (entity) => this.processForSave(entity), + processForLoad: (row) => this.processForLoad(row), + findOne: (id, options, client) => this.findOne(id, options, client), + loadRelations: (rows, relations, client) => + this.loadRelations(rows, relations, client), + invalidateRowCache: (id) => this.invalidateRowCache(id), + invalidateTableCache: () => this.invalidateTableCache(), + writeThroughRow: (id, row) => this.writeThroughRow(id, row), + }; + } + private async _create( entity: Partial, options: { relations?: string[] }, client: DBClient, ): Promise { + if (this.getDBType(client) === DBType.MongoDB) { + return mongoCreate(this.mongoCtx, entity as Record, options, client) as Promise; + } + const start = performance.now(); this.logger.logDebug( `Creating ${this.table} with data: ${JSON.stringify(entity)}`, ); this.validate(entity); - const timestamps = this.timestampsConfig; - const entityWithTimestamps = { ...entity } as Record; - if (timestamps?.createdAt && !entityWithTimestamps[timestamps.createdAt]) { - entityWithTimestamps[timestamps.createdAt] = new Date().toISOString(); - } - if (timestamps?.updatedAt && !entityWithTimestamps[timestamps.updatedAt]) { - entityWithTimestamps[timestamps.updatedAt] = new Date().toISOString(); - } - // Seed the optimistic lock so the first `update()` has a version to match - // on; without this the column stays NULL and `version = NULL` never - // matches, which makes every update look like a conflict. - if ( - this.optimisticLockField && - entityWithTimestamps[this.optimisticLockField] === undefined - ) { - entityWithTimestamps[this.optimisticLockField] = 1; - } + const entityWithTimestamps = this.seedCreateDefaults(entity); // Encrypt after the timestamp and lock columns are in place, so everything // bound below passes through the same coercion. @@ -876,6 +934,15 @@ export class Repository { options: { relations?: string[]; batchSize?: number }, client: DBClient, ): Promise { + if (this.getDBType(client) === DBType.MongoDB) { + return mongoBulkCreate( + this.mongoCtx, + entities as Record[], + options, + client, + ) as Promise; + } + const start = performance.now(); this.logger.logDebug( `Bulk creating ${entities.length} ${this.table} entities`, @@ -965,9 +1032,9 @@ export class Repository { ); if (ids.length === 0) continue; - const queryBuilder = this.find().where( - `${this.table}.${pkColumn} IN (${ids.map(() => "?").join(", ")})`, - ...ids, + const queryBuilder = this.find().whereIn( + `${this.table}.${pkColumn}`, + ids, ); const fetched = await queryBuilder.execute(client); await this.loadRelations(fetched, options.relations, client); @@ -1440,7 +1507,7 @@ export class Repository { if (keys.length === 0) return null; const qb = this.find(); for (const key of keys) { - qb.where(`${this.columns[key]?.name ?? key} = ?`, values[key]); + qb.whereEq(this.columns[key]?.name ?? key, values[key]); } const found = await qb.limit(1).execute(client); return (found[0] as T) ?? null; @@ -1630,10 +1697,7 @@ export class Repository { const rows: any[] = []; for (const chunk of chunked(values, RELATION_BATCH_SIZE)) { const qb = this.find(); - qb.where( - `${this.table}.${column} IN (${chunk.map(() => "?").join(", ")})`, - ...chunk, - ); + qb.whereIn(`${this.table}.${column}`, chunk); rows.push(...(await qb.execute(client))); } return rows; @@ -1870,7 +1934,12 @@ export class Repository { // A `Date` is normalised per dialect rather than to ISO for all of them, // because MySQL and MariaDB reject the ISO form outright. @see // sanitizeSqlValue for both reasons. + // + // MongoDB is the exception in the other direction: it has a native date + // type, and an ISO string would both fail a `{bsonType: "date"}` validator + // and defeat every range query that could have used an index. const dbType = this.getDBType(); + if (dbType === DBType.MongoDB) return processed; for (const key of Object.keys(processed)) { if (processed[key] instanceof Date) { processed[key] = sanitizeSqlValue(processed[key], dbType); @@ -1924,9 +1993,9 @@ export class Repository { const qb = this.find(); for (const [key, value] of Object.entries(conditions)) { if (value === null) { - qb.whereNull(key); + qb.whereNull(this.columns[key]?.name ?? key); } else { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } // Safe to limit before loading: relations no longer multiply the parent @@ -1948,9 +2017,9 @@ export class Repository { const qb = this.find(); for (const [key, value] of Object.entries(conditions)) { if (value === null) { - qb.whereNull(key); + qb.whereNull(this.columns[key]?.name ?? key); } else { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } if (options.limit) qb.limit(options.limit); @@ -1964,7 +2033,7 @@ export class Repository { async count(conditions?: Partial): Promise { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { @@ -1973,7 +2042,7 @@ export class Repository { if (value === null) { qb.whereNull(this.columns[key]?.name ?? key); } else if (value !== undefined) { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } } @@ -1993,12 +2062,12 @@ export class Repository { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { if (value !== undefined && value !== null) { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } } @@ -2077,7 +2146,7 @@ export class Repository { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } qb.select(...selectParts); const results = await qb.execute(this.client); @@ -2105,9 +2174,9 @@ export class Repository { if (options.where) { for (const [key, value] of Object.entries(options.where)) { if (value === null) { - qb.whereNull(key); + qb.whereNull(this.columns[key]?.name ?? key); } else { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } } @@ -2117,9 +2186,9 @@ export class Repository { const colName = this.columns[field]?.name || field; const dir = options.orderBy?.direction || "ASC"; if (direction === "forward") { - qb.where(`${colName} ${dir === "ASC" ? ">" : "<"} ?`, value); + qb.whereCompare(colName, dir === "ASC" ? ">" : "<", value); } else { - qb.where(`${colName} ${dir === "ASC" ? "<" : ">"} ?`, value); + qb.whereCompare(colName, dir === "ASC" ? "<" : ">", value); } if (options.orderBy) { qb.orderBy(colName, options.orderBy.direction); @@ -2159,7 +2228,7 @@ export class Repository { async exists(conditions?: Partial): Promise { const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } if (conditions) { for (const [key, value] of Object.entries(conditions)) { @@ -2168,7 +2237,7 @@ export class Repository { if (value === null) { qb.whereNull(this.columns[key]?.name ?? key); } else if (value !== undefined) { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } } @@ -2246,7 +2315,7 @@ export class Repository { const colName = this.columns[column]?.name || column; const qb = new QueryBuilder(this.table); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } qb.select(`COUNT(DISTINCT ${colName}) AS __cnt`); const results = await qb.execute(this.client); @@ -2296,7 +2365,7 @@ export class Repository { const qb = new QueryBuilder(this.table); qb.select(colName); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } const results = await qb.execute(this.client); return results.map((r: any) => r[colName]); @@ -2311,7 +2380,7 @@ export class Repository { const qb = new QueryBuilder(this.table); qb.select(...colNames); if (this.softDeleteField) { - qb.where(`${this.softDeleteColumn} IS NULL`); + qb.whereNull(this.softDeleteColumn!); } const results = await this.withRowTransform(qb).execute(this.client); return results as Partial[]; @@ -2584,7 +2653,7 @@ export class Repository { ): Promise { const client = _client || this.client; const dbType = this.getDBType(client); - const qb = this.find().where("id = ?", id).limit(1); + const qb = this.find().whereEq("id", id).limit(1); if (dbType !== DBType.SQLite && dbType !== DBType.MSSQL) { // T-SQL has no `FOR UPDATE`; its equivalent is a table hint // (`WITH (UPDLOCK)`), which this builder cannot express. Skipping the @@ -2631,9 +2700,9 @@ export class Repository { if (conditions) { for (const [key, value] of Object.entries(conditions)) { if (value === null) { - qb.whereNull(key); + qb.whereNull(this.columns[key]?.name ?? key); } else { - qb.where(`${this.columns[key]?.name} = ?`, value); + qb.whereEq(this.columns[key]?.name ?? key, value); } } } diff --git a/tests/integration.mongo.test.ts b/tests/integration.mongo.test.ts new file mode 100644 index 0000000..a428e1c --- /dev/null +++ b/tests/integration.mongo.test.ts @@ -0,0 +1,314 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * End-to-end coverage for the MongoDB backend, against a real server. + * + * The suite skips itself when the replica set is not running, so `bun test` + * stays green on a machine without the fleet: + * + * docker compose -f docker-compose.test.yml up -d --wait + * + * Two properties of the probe are deliberate. The driver import is *inside* the + * `try`, because `mongodb` is an optional dependency and a top-level import + * would break `bun test` for anyone who has not installed it. And the probe + * asks `hello` for `setName` rather than pinging: a standalone answers every + * read and then fails every write, because the ORM wraps each one in a + * transaction — so a ping would call a server "available" that cannot store + * anything. + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. See the note above. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** + * A model whose key is generated, so the counter is what produces the id. + * + * `fullName` is renamed to prove the column mapping holds on the way in and on + * the way out, which is the part a document store makes easy to get subtly + * wrong. + */ +const Widget = defineModel({ + tableName: "m3_widgets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true }, + fullName: { type: DataTypes.STRING, name: "full_name" }, + active: { type: DataTypes.BOOLEAN }, + }, +}); + +/** A model with a caller-supplied string key, so nothing is allocated. */ +const Coupon = defineModel({ + tableName: "m3_coupons", + columns: { + id: { type: DataTypes.STRING, required: true }, + label: { type: DataTypes.STRING }, + }, +}); + +const COLLECTIONS = [ + "m3_widgets", + "m3_coupons", + MONGO_COUNTERS_COLLECTION, +]; + +suite("MongoDB integration", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + // Dropped rather than assumed absent: the counters collection is what makes + // `id === 1` a meaningful assertion, and a leftover counter from an earlier + // run would hand out a higher id and turn this into a false failure. + for (const name of COLLECTIONS) { + await db.client.mongoDeleteMany(name, {}); + } + }); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoDeleteMany(name, {}).catch(() => {}); + } + await db.close(); + }); + + it("is serving transactions, which every write depends on", async () => { + const hello = await db.client.mongoCommand({ hello: 1 }); + expect(hello.setName).toBe("rs0"); + }); + + // ─── the counter ─────────────────────────────────────────────────── + + it("generates id 1 for the first document in a fresh collection", async () => { + const created: any = await db + .getRepository(Widget) + .create({ name: "first", fullName: "First Widget" }); + + expect(created.id).toBe(1); + expect(Number.isInteger(created.id)).toBe(true); + expect(created.full_name).toBe("First Widget"); + }); + + it("stores the generated id as _id", async () => { + const raw = await db.client.mongoFindOne("m3_widgets", { + _id: 1, + }); + expect(raw).not.toBeNull(); + expect(raw.name).toBe("first"); + // The id is the storage key, not a field beside it. + expect(raw.id).toBeUndefined(); + }); + + it("reads the row back by id", async () => { + const found: any = await db.getRepository(Widget).findOne(1); + expect(found).not.toBeNull(); + expect(found.id).toBe(1); + expect(found.name).toBe("first"); + }); + + it("keeps counting up from the counter document", async () => { + const repo = db.getRepository(Widget); + const second: any = await repo.create({ name: "second" }); + const third: any = await repo.create({ name: "third" }); + expect(second.id).toBe(2); + expect(third.id).toBe(3); + + const counter = await db.client.mongoFindOne(MONGO_COUNTERS_COLLECTION, { + _id: "m3_widgets", + }); + expect(counter.seq).toBe(3); + }); + + it("allocates a contiguous block for one bulkCreate", async () => { + await db.client.mongoDeleteMany("m3_bulk", {}); + const Bulk = defineModel({ + tableName: "m3_bulk", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + }, + }); + const repo = db.getRepository(Bulk); + + const created: any[] = await repo.bulkCreate( + Array.from({ length: 20 }, (_, i) => ({ name: `row-${i}` })), + ); + + // Contiguous and in input order. The SQL path has to work out which keys a + // multi-row INSERT produced; here one `$inc` reserved the whole block. + expect(created.map((row) => row.id)).toEqual( + Array.from({ length: 20 }, (_, i) => i + 1), + ); + + const counter = await db.client.mongoFindOne(MONGO_COUNTERS_COLLECTION, { + _id: "m3_bulk", + }); + expect(counter.seq).toBe(20); + await db.client.mongoDeleteMany("m3_bulk", {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "m3_bulk", + }); + }); + + it("advances the counter past a caller-supplied id", async () => { + const repo = db.getRepository(Widget); + const explicit: any = await repo.create({ id: 500, name: "explicit" }); + expect(explicit.id).toBe(500); + + // Without the `$max` bump this would be handed 4 and collide with a + // document that already exists. + const next: any = await repo.create({ name: "after-explicit" }); + expect(next.id).toBe(501); + }); + + it("honours a caller-supplied key without touching the counter", async () => { + const repo = db.getRepository(Coupon); + const created: any = await repo.create({ id: "SAVE10", label: "Ten off" }); + expect(created.id).toBe("SAVE10"); + expect((await repo.findOne("SAVE10")).label).toBe("Ten off"); + + const counter = await db.client.mongoFindOne(MONGO_COUNTERS_COLLECTION, { + _id: "m3_coupons", + }); + expect(counter).toBeNull(); + }); + + it("surfaces a write failure with its own error code", async () => { + // Every write goes through a transaction, and the transaction used to + // rewrite whatever came back out of it as TX_ERROR. So a payload that + // failed validation was reported as a transaction problem, with a message + // about replica sets — the wrong code to branch on and the wrong thing to + // go and look at. The four SQL backends all let the original through. + let code = ""; + let message = ""; + try { + await db.getRepository(Widget).create({}); + } catch (error) { + code = (error as any).code; + message = (error as Error).message; + } + + expect(code).toBe("VALIDATION_ERROR"); + expect(message).toContain("name"); + expect(message).not.toContain("replica set"); + }); + + // ─── the read path ───────────────────────────────────────────────── + + it("counts and reports existence through the query builder", async () => { + const repo = db.getRepository(Widget); + expect(await repo.count()).toBe(5); + expect(await repo.exists({ name: "first" })).toBe(true); + expect(await repo.exists({ name: "nobody" })).toBe(false); + }); + + it("filters, sorts and limits", async () => { + const repo = db.getRepository(Widget); + const rows: any[] = await repo.find().whereEq("name", "second").execute(db.client); + expect(rows).toHaveLength(1); + expect(rows[0].id).toBe(2); + + const ordered: any[] = await repo.find().orderBy("id", "DESC").limit(2).execute(db.client); + expect(ordered.map((row) => row.id)).toEqual([501, 500]); + }); + + it("projects only the columns asked for", async () => { + const repo = db.getRepository(Widget); + const rows: any[] = await repo + .find() + .select("name") + .whereEq("id", 1) + .execute(db.client); + expect(rows[0].name).toBe("first"); + // The id is always projected: a row with no key is not addressable. + expect(rows[0].id).toBe(1); + expect(rows[0].full_name).toBeUndefined(); + }); + + it("reads a null-valued column as absent rather than as a stored null", async () => { + const repo = db.getRepository(Widget); + const created: any = await repo.create({ name: "no-full-name" }); + const raw = await db.client.mongoFindOne("m3_widgets", { + _id: created.id, + }); + // Omitted, not `full_name: null` — an explicit null would collide in a + // sparse unique index, which is what the schema relies on. + expect("full_name" in raw).toBe(false); + expect(created.full_name ?? null).toBeNull(); + }); + + it("keeps booleans as booleans", async () => { + const repo = db.getRepository(Widget); + const created: any = await repo.create({ name: "flagged", active: true }); + const raw = await db.client.mongoFindOne("m3_widgets", { + _id: created.id, + }); + // The SQL path coerces a boolean to 1/0, which on Mongo would fail a + // `{bsonType: "bool"}` validator and make `whereEq("active", true)` match + // nothing. + expect(raw.active).toBe(true); + }); + + it("applies the beforeCreate hook before the document is built", async () => { + const Slugged = defineModel({ + tableName: "m3_slugged", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + slug: { type: DataTypes.STRING }, + }, + hooks: { + beforeCreate: (entity: any) => { + entity.slug = String(entity.title).toLowerCase().replace(/\s+/g, "-"); + }, + }, + }); + const created: any = await db + .getRepository(Slugged) + .create({ title: "Hello World" }); + expect(created.slug).toBe("hello-world"); + expect((await db.getRepository(Slugged).findOne(created.id)).slug).toBe( + "hello-world", + ); + + await db.client.mongoDeleteMany("m3_slugged", {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "m3_slugged", + }); + }); +}); diff --git a/tests/mongo.client.test.ts b/tests/mongo.client.test.ts new file mode 100644 index 0000000..a50da6b --- /dev/null +++ b/tests/mongo.client.test.ts @@ -0,0 +1,375 @@ +import { describe, it, expect, afterAll } from "vitest"; +import { DBClient } from "../client"; +import { DBType, StabilizeError, type Logger, type PoolMetrics } from "../types"; + +/** + * Client-level coverage for the MongoDB backend. + * + * Repository coverage lives in `integration.mongo.test.ts`; this file is about + * the layer beneath it — connecting, the raw-SQL guard, and above all + * *transactions*, which for MongoDB are the whole ballgame. Every write in the + * ORM is wrapped in one, so a server that cannot serve them cannot store + * anything, and the way that fails is quiet: the connection is fine, reads + * work, and only writes break. + * + * Two servers are needed to pin that down, both from the compose fleet: + * + * - a single-node replica set on 57017, where transactions work; + * - a standalone on 57018, where they cannot, so the warning and the error + * message that explain the situation can be asserted rather than assumed. + * + * Each suite skips itself when its server is absent, so `bun test` stays green + * on a machine without the fleet. Start it with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +const STANDALONE_URL = + process.env.MONGO_STANDALONE_URL || + "mongodb://127.0.0.1:57018/stabilize_test?directConnection=true"; + +/** + * A logger that keeps what it was told, so a warning can be asserted. + * + * `StabilizeLogger` writes to the console, which would make the assertion a + * matter of reading test output by eye. + */ +class RecordingLogger implements Logger { + public warnings: string[] = []; + public debugMessages: string[] = []; + + logQuery(): void {} + logError(): void {} + logMetrics(_metrics: PoolMetrics): void {} + logInfo(): void {} + logWarn(message: string): void { + this.warnings.push(message); + } + logDebug(message: string): void { + this.debugMessages.push(message); + } +} + +/** + * Reports whether a server answers *and* has the replica-set property asked for. + * + * Both halves matter, and the second is the one that would be missed. A + * standalone pings perfectly well — so a probe that only pinged would call the + * standalone "available" and hand the replica-set suite a server on which every + * write fails. Asking `hello` for `setName` is what tells the two apart. + * + * The driver import is inside the `try` on purpose: `mongodb` is an optional + * dependency, and a top-level import would break `bun test` for anyone who has + * not installed it. + */ +async function probe(url: string, wantReplicaSet: boolean): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return wantReplicaSet ? Boolean(hello.setName) : !hello.setName; + } catch { + return false; + } finally { + // `close()` on a client that never connected rejects; the answer is already + // known by then, so that failure is not worth surfacing. + await client?.close().catch(() => {}); + } +} + +const hasReplicaSet = await probe(REPLICA_SET_URL, true); +const hasStandalone = await probe(STANDALONE_URL, false); + +const suite = hasReplicaSet ? describe : describe.skip; +const standaloneSuite = hasStandalone ? describe : describe.skip; + +if (!hasReplicaSet) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} +if (!hasStandalone) { + console.warn( + `[skip] No standalone MongoDB at ${STANDALONE_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** Collection names this file owns; dropped on the way out. */ +const COLLECTIONS = ["m0_commit", "m0_rollback", "m0_duplicate"]; + +afterAll(async () => { + if (!hasReplicaSet) return; + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + for (const name of COLLECTIONS) { + await client.mongoDeleteMany(name, {}); + } + } finally { + await client.close(); + } +}); + +suite("DBClient against MongoDB", () => { + it("connects and answers a command", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + const pong = await client.mongoCommand({ ping: 1 }); + expect(pong.ok).toBe(1); + } finally { + await client.close(); + } + }); + + it("reads back a document it wrote outside a transaction", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + await client.mongoDeleteMany("m0_commit", {}); + await client.mongoInsertOne("m0_commit", { _id: 1, value: "plain" }); + + const docs = await client.mongoFind("m0_commit", {}); + expect(docs).toHaveLength(1); + expect(docs[0].value).toBe("plain"); + // `_id` is what a mongo document's primary key is called; the ORM maps it + // to the model's `id` further up. Pinning the raw shape here keeps that + // mapping honest. + expect(docs[0]._id).toBe(1); + } finally { + await client.close(); + } + }); + + it("commits a transaction", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + await client.mongoDeleteMany("m0_commit", {}); + await client.transaction(async (tx) => { + await tx.mongoInsertOne("m0_commit", { _id: 10, value: "committed" }); + }); + + const docs = await client.mongoFind("m0_commit", {}); + expect(docs.map((d) => d.value)).toEqual(["committed"]); + } finally { + await client.close(); + } + }); + + it("rolls back a transaction", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + await client.mongoDeleteMany("m0_rollback", {}); + + // try/catch rather than `.rejects`: a rejection assertion that the driver + // never settles leaves Bun's runner hanging. + let thrown: unknown = null; + try { + await client.transaction(async (tx) => { + await tx.mongoInsertOne("m0_rollback", { _id: 20, value: "nope" }); + throw new Error("deliberate"); + }); + } catch (error) { + thrown = error; + } + + expect(thrown).toBeInstanceOf(Error); + expect((thrown as Error).message).toBe("deliberate"); + + const docs = await client.mongoFind("m0_rollback", {}); + expect(docs).toHaveLength(0); + } finally { + await client.close(); + } + }); + + it("runs a write inside a transaction through the session", async () => { + // The distinction being pinned: without the session option the insert would + // still land, but *outside* the transaction, so a later rollback would + // leave it behind. This asserts the session actually reaches the command. + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + await client.mongoDeleteMany("m0_rollback", {}); + + let readInside: unknown[] | null = null; + try { + await client.transaction(async (tx) => { + await tx.mongoInsertOne("m0_rollback", { _id: 21, value: "in-flight" }); + // Visible to the transaction's own session. + readInside = await tx.mongoFind("m0_rollback", { _id: 21 }); + throw new Error("deliberate"); + }); + } catch { + // Expected; the assertion is on what the rollback left behind. + } + + expect(readInside).toHaveLength(1); + expect(await client.mongoFind("m0_rollback", { _id: 21 })).toHaveLength(0); + } finally { + await client.close(); + } + }); + + it("refuses raw SQL with MONGO_UNSUPPORTED", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + const caught: StabilizeError[] = []; + for (const attempt of [ + () => client.query("SELECT 1"), + () => client.queryExec("DELETE FROM nothing"), + () => client.migrationQuery("CREATE TABLE nothing (id INT)"), + ]) { + try { + await attempt(); + } catch (error) { + caught.push(error as StabilizeError); + } + } + + expect(caught).toHaveLength(3); + for (const error of caught) { + expect(error).toBeInstanceOf(StabilizeError); + expect(error.code).toBe("MONGO_UNSUPPORTED"); + } + } finally { + await client.close(); + } + }); + + it("lets a write the driver refused keep its own error code", async () => { + // The other half of the rule the standalone suite pins from the far side. A + // duplicate key is not a transaction problem, and reporting it as one would + // send the reader to the server's replica-set configuration when the answer + // is "that id is taken". The distinction that makes this reachable: the + // executor wraps the driver's rejection in a `MONGO_ERROR` whose `code` is a + // string, so its own number is gone by the time the classifier runs. + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + try { + await client.mongoDeleteMany("m0_duplicate", {}); + await client.mongoInsertOne("m0_duplicate", { _id: 1 }); + + let caught: StabilizeError | null = null; + try { + await client.transaction(async (tx) => { + await tx.mongoInsertOne("m0_duplicate", { _id: 1 }); + }); + } catch (error) { + caught = error as StabilizeError; + } + + expect(caught).toBeInstanceOf(StabilizeError); + expect(caught!.code).toBe("MONGO_ERROR"); + // The driver's own reason survives, so the failure stays diagnosable. + expect(caught!.message).toMatch(/duplicate key|E11000/i); + expect(caught!.message).not.toMatch(/replica set/i); + } finally { + await client.mongoDeleteMany("m0_duplicate", {}); + await client.close(); + } + }); + + it("closes a client that was never connected", async () => { + // The handle is filled in lazily, so `close()` runs against `null` when + // nothing ever used the client. It has to be a no-op rather than a throw. + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await client.close(); + }); +}); + +standaloneSuite("DBClient against a standalone MongoDB", () => { + it("warns at connect time that transactions are unavailable", async () => { + const logger = new RecordingLogger(); + const client = new DBClient( + { type: DBType.MongoDB, connectionString: STANDALONE_URL }, + logger, + ); + try { + // The warning is emitted during the lazy connect, which the first command + // triggers. + await client.mongoCommand({ ping: 1 }); + + const warnings = logger.warnings.join("\n"); + expect(warnings).toContain("standalone"); + expect(warnings).toContain("replica set"); + } finally { + await client.close(); + } + }); + + it("reports a transaction failure as a TX_ERROR naming the fix", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: STANDALONE_URL, + }); + try { + let caught: StabilizeError | null = null; + try { + await client.transaction(async (tx) => { + // A real write, not a `ping`: mongo rejects some administrative + // commands inside a transaction for reasons unrelated to the replica + // set ("This command is not supported in transactions"), which would + // mask the failure actually under test. + await tx.mongoInsertOne("m0_standalone_tx", { _id: 1 }); + }); + } catch (error) { + caught = error as StabilizeError; + } + + expect(caught).toBeInstanceOf(StabilizeError); + expect(caught!.code).toBe("TX_ERROR"); + // The point of the custom message: the driver's own wording ("Transaction + // numbers are only allowed on a replica set member or mongos") names the + // rule but not the remedy. + expect(caught!.message).toMatch(/replica set/i); + expect(caught!.message).toMatch(/rs\.initiate/); + } finally { + await client.close(); + } + }); + + it("still serves reads, which is why the warning is not fatal", async () => { + const client = new DBClient({ + type: DBType.MongoDB, + connectionString: STANDALONE_URL, + }); + try { + await client.mongoInsertOne("m0_standalone", { _id: 1 }); + expect(await client.mongoCount("m0_standalone", {})).toBe(1); + await client.mongoDeleteMany("m0_standalone", {}); + } finally { + await client.close(); + } + }); +}); diff --git a/tests/mongo.dialect.test.ts b/tests/mongo.dialect.test.ts new file mode 100644 index 0000000..2ab83e2 --- /dev/null +++ b/tests/mongo.dialect.test.ts @@ -0,0 +1,609 @@ +import { describe, it, expect } from "vitest"; +import { + buildMongoAggregatePipeline, + buildMongoFilter, + buildMongoGroupStage, + buildMongoProjection, + buildMongoSort, + buildMongoSpec, + createMongoBlockers, + MONGO_MATCHES_NOTHING, + normalizeMongoDoc, + OMIT, + recordMongoBlocker, + sanitizeMongoValue, + throwMongoUnsupported, + translateField, + translateLikePattern, + type MongoAggregate, + type MongoFieldContext, + type MongoFilterNode, + type Predicate, + type PredicateOp, +} from "../mongo-query"; +import { StabilizeError } from "../types"; + +/** + * The MongoDB translation layer, with no server involved. + * + * This file exists because a mis-translated filter does not fail — it returns + * the wrong rows. Every assertion here is one of two kinds: a SQL operator whose + * NULL handling differs from its Mongo counterpart, or a place where a naive + * implementation silently drops a condition. + * + * `NOTHING` is the filter that matches no documents, spelled once. + */ +const NOTHING = { _id: { $in: [] } }; + +/** A predicate node, for assembling filter trees by hand. */ +function pred(predicate: Predicate): MongoFilterNode { + return { kind: "pred", predicate }; +} + +/** An `AND` group. */ +function and(...items: MongoFilterNode[]): MongoFilterNode { + return { kind: "and", items }; +} + +/** The error a call throws, or null if it returned. */ +function catchError(fn: () => any): StabilizeError | null { + try { + fn(); + return null; + } catch (error) { + return error as StabilizeError; + } +} + +describe("translateField", () => { + const ctx: MongoFieldContext = { + table: "users", + alias: "u", + primaryKey: "id", + columns: { firstName: "first_name", id: "id", userId: "user_id" }, + }; + + it("maps the primary key to _id, however it was spelled", () => { + expect(translateField("id", ctx)).toBe("_id"); + expect(translateField("id", { columns: { id: "user_id" }, primaryKey: "user_id" })).toBe("_id"); + expect( + translateField("id", { idProperty: "id", primaryKey: "user_id" }), + ).toBe("_id"); + }); + + it("strips a table or alias qualifier", () => { + expect(translateField("users.email", ctx)).toBe("email"); + expect(translateField("u.email", ctx)).toBe("email"); + expect(translateField("users.id", ctx)).toBe("_id"); + }); + + it("resolves a property key to its column name", () => { + expect(translateField("firstName", ctx)).toBe("first_name"); + expect(translateField("users.firstName", ctx)).toBe("first_name"); + }); + + it("leaves an undeclared field and an embedded path alone", () => { + expect(translateField("email", ctx)).toBe("email"); + // A dot here means an embedded document, not a table qualifier: relations + // are separate collections and never appear as dotted paths. + expect(translateField("profile.bio", ctx)).toBe("profile.bio"); + expect(translateField(" email ", ctx)).toBe("email"); + }); +}); + +describe("buildMongoFilter", () => { + it("matches everything when there is nothing to match", () => { + expect(buildMongoFilter(null)).toEqual({}); + expect(buildMongoFilter(and())).toEqual({}); + }); + + it("inlines a single condition rather than wrapping it", () => { + expect( + buildMongoFilter(pred({ op: "cmp", column: "age", value: 30 })), + ).toEqual({ age: 30 }); + expect( + buildMongoFilter(and(pred({ op: "cmp", column: "age", value: 30 }))), + ).toEqual({ age: 30 }); + }); + + it("combines repeated conditions on one field with $and", () => { + // The defect this guards: merging into a single object would keep only the + // last assignment, so `age > 18 AND age < 65` would silently become + // `age < 65` and return children. + const filter = buildMongoFilter( + and( + pred({ op: "cmp", column: "age", compare: ">", value: 18 }), + pred({ op: "cmp", column: "age", compare: "<", value: 65 }), + ), + ); + expect(filter).toEqual({ + $and: [{ age: { $gt: 18 } }, { age: { $lt: 65 } }], + }); + expect(Object.keys(filter)).toEqual(["$and"]); + }); + + it("renders an OR node as $or", () => { + const filter = buildMongoFilter({ + kind: "or", + left: and(pred({ op: "cmp", column: "a", value: 1 })), + right: and(pred({ op: "cmp", column: "b", value: 2 })), + }); + expect(filter).toEqual({ $or: [{ a: 1 }, { b: 2 }] }); + }); + + it("reproduces the builder's OR fold exactly", () => { + // where(A).where(B).orWhere(C).where(D) renders in SQL as + // ((A AND B) OR (C)) AND D + // A flat predicate list read with AND-precedence would give + // (A AND B) OR (C AND D) + // which is a different set of rows. The tree has to keep the fold. + const A = pred({ op: "cmp", column: "a", value: 1 }); + const B = pred({ op: "cmp", column: "b", value: 2 }); + const C = pred({ op: "cmp", column: "c", value: 3 }); + const D = pred({ op: "cmp", column: "d", value: 4 }); + + const folded: MongoFilterNode = and({ + kind: "or", + left: and(A, B), + right: and(C), + }, D); + + expect(buildMongoFilter(folded)).toEqual({ + $and: [ + { $or: [{ $and: [{ a: 1 }, { b: 2 }] }, { c: 3 }] }, + { d: 4 }, + ], + }); + }); +}); + +describe("comparison predicates", () => { + const ctx: MongoFieldContext = { primaryKey: "id" }; + + it("translates = to equality", () => { + expect(buildMongoFilter(pred({ op: "cmp", column: "email", value: "a" }), ctx)).toEqual( + { email: "a" }, + ); + }); + + it("translates <> so that missing fields are excluded, as SQL does", () => { + // `{email: {$ne: "a"}}` matches documents with no `email` field at all; + // SQL's `email <> 'a'` does not. The null in the $nin list is the fix. + expect( + buildMongoFilter(pred({ op: "cmp", column: "email", compare: "!=", value: "a" }), ctx), + ).toEqual({ email: { $nin: ["a", null] } }); + }); + + it("translates the range operators", () => { + const cases: [any, any][] = [ + [">", { $gt: 5 }], + [">=", { $gte: 5 }], + ["<", { $lt: 5 }], + ["<=", { $lte: 5 }], + ]; + for (const [op, expected] of cases) { + expect( + buildMongoFilter(pred({ op: "cmp", column: "n", compare: op, value: 5 }), ctx), + ).toEqual({ n: expected }); + } + }); + + it("matches nothing for any comparison against NULL", () => { + // SQL evaluates every one of these to UNKNOWN, which filters the row out. + const nullish = [null, undefined]; + for (const value of nullish) { + for (const op of ["!=", ">", ">=", "<", "<="] as const) { + expect( + buildMongoFilter(pred({ op: "cmp", column: "n", compare: op, value }), ctx), + ).toEqual(NOTHING); + } + } + }); +}); + +describe("IN and NOT IN", () => { + it("translates IN to $in", () => { + expect( + buildMongoFilter(pred({ op: "in", column: "id", values: [1, 2, 3] })), + ).toEqual({ _id: { $in: [1, 2, 3] } }); + }); + + it("drops NULLs from an IN list, because SQL's cannot be satisfied by one", () => { + // `x IN (1, NULL)` is true only for x = 1: `x = NULL` is UNKNOWN either way, + // so the NULL contributes nothing and behaves as if it were not written. + expect( + buildMongoFilter(pred({ op: "in", column: "id", values: [1, null, 2] })), + ).toEqual({ _id: { $in: [1, 2] } }); + }); + + it("matches nothing for an empty IN, as SQL's `1 = 0` does", () => { + expect(buildMongoFilter(pred({ op: "in", column: "id", values: [] }))).toEqual( + NOTHING, + ); + expect( + buildMongoFilter(pred({ op: "in", column: "id", values: [null] })), + ).toEqual(NOTHING); + }); + + it("translates NOT IN to $nin plus a null exclusion", () => { + expect( + buildMongoFilter(pred({ op: "nin", column: "id", values: [1, 2] })), + ).toEqual({ _id: { $nin: [1, 2], $ne: null } }); + }); + + it("is a no-op for an empty NOT IN, matching the SQL builder", () => { + // `whereNotIn(col, [])` returns early in the SQL builder and adds no clause. + expect(buildMongoFilter(pred({ op: "nin", column: "id", values: [] }))).toEqual( + {}, + ); + }); + + it("matches nothing when a NOT IN list contains NULL", () => { + // One NULL makes every comparison UNKNOWN, so `NOT IN (1, NULL)` is never + // true — a different result from the empty case above, deliberately. + expect( + buildMongoFilter(pred({ op: "nin", column: "id", values: [1, null] })), + ).toEqual(NOTHING); + }); +}); + +describe("null predicates", () => { + it("matches missing and null for IS NULL", () => { + // Intentional: a document written before the soft-delete field existed is + // not soft-deleted, and Mongo reports both states as null. + expect(buildMongoFilter(pred({ op: "null", column: "deleted_at" }))).toEqual( + { deleted_at: null }, + ); + }); + + it("excludes missing and null for IS NOT NULL", () => { + expect( + buildMongoFilter(pred({ op: "notNull", column: "deleted_at" })), + ).toEqual({ deleted_at: { $ne: null } }); + }); +}); + +describe("BETWEEN predicates", () => { + it("translates BETWEEN inclusively", () => { + expect( + buildMongoFilter(pred({ op: "between", column: "n", start: 1, end: 9 })), + ).toEqual({ n: { $gte: 1, $lte: 9 } }); + }); + + it("translates NOT BETWEEN so that missing fields are excluded", () => { + // `$not` on its own matches a document with no `n` field; SQL's + // `n NOT BETWEEN 1 AND 9` evaluates to UNKNOWN there and excludes it. + expect( + buildMongoFilter(pred({ op: "notBetween", column: "n", start: 1, end: 9 })), + ).toEqual({ + $and: [{ n: { $not: { $gte: 1, $lte: 9 } } }, { n: { $ne: null } }], + }); + }); + + it("matches nothing when a bound is NULL", () => { + for (const op of ["between", "notBetween"] as PredicateOp[]) { + expect( + buildMongoFilter(pred({ op, column: "n", start: null, end: 9 })), + ).toEqual(NOTHING); + expect( + buildMongoFilter(pred({ op, column: "n", start: 1, end: undefined })), + ).toEqual(NOTHING); + } + }); +}); + +describe("translateLikePattern", () => { + it("turns SQL wildcards into regex and anchors the result", () => { + expect(translateLikePattern("a%")).toEqual({ pattern: "^a.*$", options: "s" }); + expect(translateLikePattern("%a%")).toEqual({ pattern: "^.*a.*$", options: "s" }); + expect(translateLikePattern("a_c")).toEqual({ pattern: "^a.c$", options: "s" }); + }); + + it("escapes regex metacharacters in the literal part", () => { + expect(translateLikePattern("a.b").pattern).toBe("^a\\.b$"); + expect(translateLikePattern("50%+").pattern).toBe("^50.*\\+$"); + expect(translateLikePattern("(x)").pattern).toBe("^\\(x\\)$"); + }); + + it("enables dotAll so that % matches a newline, as SQL's does", () => { + // Without the `s` flag `.` stops at a line break and `LIKE '%a%'` would miss + // a multi-line value SQL would have matched. + const { options } = translateLikePattern("%a%"); + expect(options).toContain("s"); + expect(translateLikePattern("%a%", true).options).toBe("is"); + }); + + it("coerces a non-string pattern", () => { + expect(translateLikePattern(42).pattern).toBe("^42$"); + }); +}); + +describe("LIKE predicates", () => { + it("renders LIKE and ILIKE as regex, differing only in case sensitivity", () => { + expect(buildMongoFilter(pred({ op: "like", column: "name", value: "A%" }))).toEqual( + { name: { $regex: "^A.*$", $options: "s" } }, + ); + expect(buildMongoFilter(pred({ op: "ilike", column: "name", value: "A%" }))).toEqual( + { name: { $regex: "^A.*$", $options: "is" } }, + ); + }); + + it("renders NOT LIKE with $nor, excluding missing fields", () => { + // `$nor` rather than `$not`, because `$not` over `$regex` is inconsistently + // supported; and the `$ne: null` supplies the NULL exclusion `NOT LIKE` + // performs, which `$nor` alone would not. + expect( + buildMongoFilter(pred({ op: "notLike", column: "name", value: "A%" })), + ).toEqual({ + $and: [ + { $nor: [{ name: { $regex: "^A.*$", $options: "s" } }] }, + { name: { $ne: null } }, + ], + }); + }); + + it("passes a raw regex through with dotAll", () => { + expect(buildMongoFilter(pred({ op: "regex", column: "name", value: "^a" }))).toEqual( + { name: { $regex: "^a", $options: "s" } }, + ); + }); +}); + +describe("buildMongoSort", () => { + const ctx: MongoFieldContext = { table: "users", primaryKey: "id" }; + + it("translates ordered clauses, defaulting the direction to ASC", () => { + expect(buildMongoSort(["name ASC"], ctx)).toEqual({ name: 1 }); + expect(buildMongoSort(["name DESC"], ctx)).toEqual({ name: -1 }); + expect(buildMongoSort(["name ASC", "id DESC"], ctx)).toEqual({ + name: 1, + _id: -1, + }); + }); + + it("strips a qualifier and maps the primary key", () => { + expect(buildMongoSort(["users.created_at DESC"], ctx)).toEqual({ + created_at: -1, + }); + expect(buildMongoSort(["users.id ASC"], ctx)).toEqual({ _id: 1 }); + }); + + it("returns null when there is no ordering", () => { + expect(buildMongoSort(null)).toBeNull(); + expect(buildMongoSort([])).toBeNull(); + }); + + it("throws on a raw expression instead of sorting by a nonexistent field", () => { + // Passing "LENGTH(name) DESC" through would have the driver look for a field + // literally named `LENGTH(name)`, find nothing in every document, and return + // the rows in arbitrary order with no error at all. + const error = catchError(() => buildMongoSort(["LENGTH(name) DESC"], ctx)); + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("LENGTH(name) DESC"); + }); +}); + +describe("buildMongoProjection", () => { + const ctx: MongoFieldContext = { table: "users", primaryKey: "id" }; + + it("returns null for SELECT *", () => { + expect(buildMongoProjection(["*"], ctx)).toBeNull(); + expect(buildMongoProjection([], ctx)).toBeNull(); + expect(buildMongoProjection(null, ctx)).toBeNull(); + }); + + it("projects listed fields and always keeps _id", () => { + // Mongo includes _id in an inclusion projection by default, and _id is where + // the model's primary key lives — dropping it would strip every row's id. + expect(buildMongoProjection(["name", "email"], ctx)).toEqual({ + name: 1, + email: 1, + _id: 1, + }); + expect(buildMongoProjection(["name", "id"], ctx)).toEqual({ + name: 1, + _id: 1, + }); + }); + + it("returns null for an expression, which a projection cannot hold", () => { + expect(buildMongoProjection(["COUNT(*)"], ctx)).toBeNull(); + }); +}); + +describe("buildMongoSpec", () => { + const ctx: MongoFieldContext = { primaryKey: "id" }; + + it("assembles filter, projection, sort and window", () => { + const spec = buildMongoSpec({ + filter: and(pred({ op: "cmp", column: "active", value: true })), + select: ["name"], + orderBy: ["name ASC"], + limit: 10, + offset: 20, + ctx, + }); + expect(spec).toEqual({ + filter: { active: true }, + projection: { name: 1, _id: 1 }, + sort: { name: 1 }, + limit: 10, + skip: 20, + }); + }); + + it("adds an _id tiebreaker when paging without an ordering", () => { + // Unordered skip/limit has no stable total order in Mongo, so `eachBatch`'s + // paging loop could revisit a document or never terminate. + const spec = buildMongoSpec({ filter: and(), offset: 100, limit: 50, ctx }); + expect(spec.sort).toEqual({ _id: 1 }); + expect(spec.skip).toBe(100); + }); + + it("leaves an explicit ordering alone and ignores a zero offset", () => { + expect( + buildMongoSpec({ filter: and(), orderBy: ["name DESC"], offset: 100, ctx }).sort, + ).toEqual({ name: -1 }); + expect(buildMongoSpec({ filter: and(), offset: 0, ctx }).skip).toBeUndefined(); + expect(buildMongoSpec({ filter: and(), ctx }).sort).toBeUndefined(); + }); + + it("throws, naming every offending method at once", () => { + const blockers = createMongoBlockers(); + recordMongoBlocker(blockers, "join", "LEFT JOIN posts ON posts.user_id = users.id"); + recordMongoBlocker(blockers, "whereRaw", "LOWER(name) = 'a'"); + + const error = catchError(() => buildMongoSpec({ filter: and(), blockers, ctx })); + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("join"); + expect(error!.message).toContain("whereRaw"); + // The quoted fragment, so the caller can see which call is at fault. + expect(error!.message).toContain("LEFT JOIN posts"); + }); +}); + +describe("blockers", () => { + it("records a method once but every fragment", () => { + const blockers = createMongoBlockers(); + recordMongoBlocker(blockers, "join", "A"); + recordMongoBlocker(blockers, "join", "B"); + recordMongoBlocker(blockers, "union", "C"); + expect(blockers.methods).toEqual(["join", "union"]); + expect(blockers.details).toEqual(["join: A", "join: B", "union: C"]); + }); + + it("reads sensibly for one method and for several", () => { + const one = createMongoBlockers(); + recordMongoBlocker(one, "join"); + expect(catchError(() => throwMongoUnsupported(one))!.message).toContain( + "join has no MongoDB equivalent", + ); + + const two = createMongoBlockers(); + recordMongoBlocker(two, "join"); + recordMongoBlocker(two, "union"); + expect(catchError(() => throwMongoUnsupported(two))!.message).toContain( + "join, union have no MongoDB equivalent", + ); + }); +}); + +describe("aggregates", () => { + const ctx: MongoFieldContext = { primaryKey: "id" }; + + it("counts rows for COUNT(*)", () => { + expect(buildMongoGroupStage([{ fn: "count", column: "*", alias: "count" }], ctx)).toEqual({ + $group: { _id: null, count: { $sum: 1 } }, + }); + }); + + it("counts non-null values for COUNT(column), as SQL does", () => { + const stage = buildMongoGroupStage( + [{ fn: "count", column: "email", alias: "count" }], + ctx, + ); + expect(stage).toEqual({ + $group: { + _id: null, + count: { $sum: { $cond: [{ $ne: ["$email", null] }, 1, 0] } }, + }, + }); + }); + + it("translates the remaining aggregates and the id field", () => { + const aggregates: MongoAggregate[] = [ + { fn: "sum", column: "total", alias: "sum" }, + { fn: "avg", column: "total", alias: "avg" }, + { fn: "min", column: "total", alias: "min" }, + { fn: "max", column: "total", alias: "max" }, + { fn: "count", column: "id", alias: "ids" }, + ]; + expect(buildMongoGroupStage(aggregates, ctx)).toEqual({ + $group: { + _id: null, + sum: { $sum: "$total" }, + avg: { $avg: "$total" }, + min: { $min: "$total" }, + max: { $max: "$total" }, + ids: { $sum: { $cond: [{ $ne: ["$_id", null] }, 1, 0] } }, + }, + }); + }); + + it("orders the pipeline match → group → project and drops the group key", () => { + const pipeline = buildMongoAggregatePipeline({ active: true }, [ + { fn: "sum", column: "total", alias: "total" }, + ]); + expect(pipeline).toEqual([ + { $match: { active: true } }, + { $group: { _id: null, total: { $sum: "$total" } } }, + { $project: { _id: 0, total: 1 } }, + ]); + expect(pipeline).toHaveLength(3); + }); +}); + +describe("sanitizeMongoValue", () => { + it("passes through the types SQL coercion would destroy", () => { + const date = new Date("2024-01-01T00:00:00Z"); + // A stringified Date fails {bsonType: "date"} and defeats an indexed range + // query; a 1/0 boolean makes whereEq("published", true) match nothing. + expect(sanitizeMongoValue(date)).toBe(date); + expect(sanitizeMongoValue(true)).toBe(true); + expect(sanitizeMongoValue(false)).toBe(false); + expect(sanitizeMongoValue({ a: 1 })).toEqual({ a: 1 }); + expect(sanitizeMongoValue([1, 2])).toEqual([1, 2]); + }); + + it("keeps falsy values that are real data", () => { + // The reason this is not `if (!value) return OMIT`. + expect(sanitizeMongoValue(0)).toBe(0); + expect(sanitizeMongoValue("")).toBe(""); + expect(sanitizeMongoValue(NaN)).toBeNaN(); + }); + + it("reports null, undefined and un-storable values as omitted", () => { + expect(sanitizeMongoValue(null)).toBe(OMIT); + expect(sanitizeMongoValue(undefined)).toBe(OMIT); + expect(sanitizeMongoValue(() => {})).toBe(OMIT); + expect(sanitizeMongoValue(Symbol("x"))).toBe(OMIT); + }); +}); + +describe("normalizeMongoDoc", () => { + it("renames _id to the id column and keeps the rest", () => { + expect(normalizeMongoDoc({ _id: 7, name: "a" })).toEqual({ id: 7, name: "a" }); + }); + + it("lets _id win over a stale id field", () => { + expect(normalizeMongoDoc({ _id: 7, id: 9, name: "a" })).toEqual({ + id: 7, + name: "a", + }); + }); + + it("accepts a renamed id column", () => { + expect(normalizeMongoDoc({ _id: 7, name: "a" }, "user_id")).toEqual({ + user_id: 7, + name: "a", + }); + }); + + it("leaves a document without _id, and a non-document, untouched", () => { + expect(normalizeMongoDoc({ name: "a" })).toEqual({ name: "a" }); + expect(normalizeMongoDoc(null)).toBeNull(); + expect(normalizeMongoDoc(5)).toBe(5); + }); +}); + +describe("MONGO_MATCHES_NOTHING", () => { + it("is frozen and matches nothing by construction", () => { + // Frozen so that one caller spreading it cannot poison the constant for + // every later query. + expect(Object.isFrozen(MONGO_MATCHES_NOTHING)).toBe(true); + expect({ ...MONGO_MATCHES_NOTHING }).toEqual(NOTHING); + expect({ ...MONGO_MATCHES_NOTHING }).not.toBe(MONGO_MATCHES_NOTHING); + }); +}); diff --git a/tests/mongo.migrations.test.ts b/tests/mongo.migrations.test.ts new file mode 100644 index 0000000..9d67f4d --- /dev/null +++ b/tests/mongo.migrations.test.ts @@ -0,0 +1,449 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; +import { + buildValidatorFromColumns, + generateMongoSteps, + mapColumnToBsonSchema, + planMongoCollection, + planMongoLinkCollections, +} from "../mongo-schema"; +import { + MONGO_MIGRATIONS_COLLECTION, + generateMongoMigration, + runMongoMigrations, +} from "../mongo-migrate"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * Schema derivation and migrations for the MongoDB backend. + * + * The first half is pure — no server, no client — because the whole point of + * expressing a MongoDB migration as data rather than as closures is that it can + * be asserted by reading it. The second half needs a real server, because the + * two rules that matter most (`validationLevel: "moderate"` and `sparse: true`) + * are only observable in what the server then *lets through*: a validator that + * was requested but never installed is indistinguishable from one that was, and + * a non-sparse unique index only misbehaves on the second document that omits + * the field. + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +// ─── pure: the derived schema ──────────────────────────────────────── + +const Account = defineModel({ + tableName: "m4_accounts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + // Unique but deliberately *not* required: a sparse unique index only + // matters for a column a document is allowed to omit. + email: { type: DataTypes.STRING, unique: true }, + handle: { type: DataTypes.STRING, required: true, index: "idx_handle" }, + age: { type: DataTypes.INTEGER }, + nickname: { type: DataTypes.STRING }, + bio: { type: DataTypes.TEXT, maxLength: 280 }, + secret: { type: DataTypes.STRING, encrypted: true }, + }, +}); + +const Post = defineModel({ + tableName: "m4_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING }, + }, + relations: [ + { + type: RelationType.ManyToMany, + target: () => Account, + property: "accounts", + joinTable: "m4_post_accounts", + foreignKey: "post_id", + inverseKey: "account_id", + }, + ], +}); + +describe("mongo schema derivation", () => { + it("accepts the several BSON types a JavaScript integer becomes", () => { + // A driver turns a JS integer into `int` inside 32-bit range and `double` + // beyond it, so a schema demanding only `int` would reject every id past + // two billion. + expect(mapColumnToBsonSchema({ type: DataTypes.INTEGER }).bsonType).toEqual([ + "int", + "long", + "double", + ]); + }); + + it("validates an encrypted column as a string whatever it was declared", () => { + // The ciphertext is what is stored; validating it against the declared type + // would reject every write. + expect( + mapColumnToBsonSchema({ type: DataTypes.INTEGER, encrypted: true }) + .bsonType, + ).toBe("string"); + }); + + it("folds the model's own validators into the BSON schema", () => { + const bio = mapColumnToBsonSchema({ type: DataTypes.TEXT, maxLength: 280 }); + expect(bio.maxLength).toBe(280); + + const code = mapColumnToBsonSchema({ + type: DataTypes.STRING, + minLength: 2, + pattern: /^[A-Z]+$/, + }); + expect(code.minLength).toBe(2); + // A `RegExp` does not survive a round trip through an aggregation; its + // source does. + expect(code.pattern).toBe("^[A-Z]+$"); + }); + + it("keeps a DATETIME a date rather than a string", () => { + expect(mapColumnToBsonSchema({ type: DataTypes.DATETIME }).bsonType).toEqual( + ["date", "string"], + ); + }); + + it("requires _id and the required columns, and does not seal the object", () => { + const plan = planMongoCollection(Account)!; + const schema = plan.validator.$jsonSchema; + + expect(schema.required).toContain("_id"); + expect(schema.required).toContain("handle"); + expect(schema.required).not.toContain("email"); + expect(schema.required).not.toContain("nickname"); + + // Deliberately not false: rejecting undeclared fields would stop the + // collection being schemaless, and would make autoMigrate a one-way door. + expect(schema.additionalProperties).toBeUndefined(); + // The primary key is `_id`; there is no separate `id` field to declare. + expect(schema.properties.id).toBeUndefined(); + }); + + it("marks every unique index sparse", () => { + const plan = planMongoCollection(Account)!; + const unique = plan.indexes.filter((index) => index.options.unique); + + expect(unique).toHaveLength(1); + expect(unique[0]!.spec).toEqual({ email: 1 }); + // Without `sparse`, two documents that both omit `email` are both null to + // MongoDB and collide — where SQL treats two NULLs as distinct. + expect(unique[0]!.options.sparse).toBe(true); + }); + + it("honours a declared index name", () => { + const plan = planMongoCollection(Account)!; + const named = plan.indexes.find((index) => index.options.name === "idx_handle"); + expect(named!.spec).toEqual({ handle: 1 }); + }); + + it("keys a many-to-many link by a compound _id", () => { + const plans = planMongoLinkCollections([Post]); + expect(plans).toHaveLength(1); + + const link = plans[0]!; + expect(link.collection).toBe("m4_post_accounts"); + // Compound `_id` so `attach` is idempotent at the storage layer rather than + // by a read the caller has to perform first. + expect(link.validator.$jsonSchema.properties._id).toEqual({ + bsonType: "object", + }); + expect(link.indexes.map((index) => Object.keys(index.spec)[0])).toEqual([ + "account_id", + "post_id", + ]); + }); + + it("generates a migration whose steps are data", () => { + const migration = generateMongoMigration(Account, "create_m4_accounts"); + + // The SQL halves stay empty: they are SQL, and there is none here. + expect(migration.up).toEqual([]); + expect(migration.down).toEqual([]); + expect(migration.mongoDown).toEqual([ + { kind: "dropCollection", collection: "m4_accounts" }, + ]); + + const kinds = migration.mongoUp!.map((step) => step.kind); + expect(kinds[0]).toBe("createCollection"); + expect(kinds).toContain("createIndex"); + // The counter step comes last, and only because the key is generated. + expect(kinds[kinds.length - 1]).toBe("createCounter"); + }); + + it("generates no counter for a caller-supplied key", () => { + const Token = defineModel({ + tableName: "m4_tokens", + columns: { + id: { type: DataTypes.STRING, required: true }, + value: { type: DataTypes.STRING }, + }, + }); + const migration = generateMongoMigration(Token, "create_m4_tokens"); + expect(migration.mongoUp!.map((step) => step.kind)).not.toContain( + "createCounter", + ); + }); + + it("drops the collection as the only inverse", () => { + // Dropping is total in a way that dropping indexes one by one is not: the + // validator goes with it, so a re-`up` rebuilds from the declaration. + expect(generateMongoSteps(Account, "down")).toEqual([ + { kind: "dropCollection", collection: "m4_accounts" }, + ]); + }); + + it("builds a validator with no id column to declare", () => { + const validator = buildValidatorFromColumns( + { id: { type: DataTypes.INTEGER }, name: { type: DataTypes.STRING } }, + "id", + ); + expect(validator.$jsonSchema.required).toEqual(["_id"]); + expect(Object.keys(validator.$jsonSchema.properties)).toEqual(["name"]); + }); +}); + +// ─── against a server ──────────────────────────────────────────────── + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +const COLLECTIONS = [ + "m4_accounts", + "m4_posts", + "m4_post_accounts", + MONGO_COUNTERS_COLLECTION, + MONGO_MIGRATIONS_COLLECTION, +]; + +suite("mongo autoMigrate", () => { + let db: any; + + const dropAll = async () => { + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + }; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + await dropAll(); + }); + + afterAll(async () => { + if (!db) return; + await dropAll().catch(() => {}); + await db.close(); + }); + + it("creates the collection, its indexes and its validator", async () => { + await db.autoMigrate([Account, Post]); + + const names = (await db.client.mongoListCollections()).map( + (entry: any) => entry.name, + ); + expect(names).toContain("m4_accounts"); + // The many-to-many join table has no model of its own, so autoMigrate is + // the only thing that can create it. + expect(names).toContain("m4_post_accounts"); + + const indexes = await db.client.mongoListIndexes("m4_accounts"); + const email = indexes.find((index: any) => index.key?.email === 1); + expect(email).toBeDefined(); + expect(email.unique).toBe(true); + // The assertion that matters: a non-sparse unique index only misbehaves on + // the second document that omits the field. + expect(email.sparse).toBe(true); + }); + + it("installs the validator, not just asks for one", async () => { + const info = await db.client.mongoCommand({ listCollections: 1 }); + const account = info.cursor.firstBatch.find( + (entry: any) => entry.name === "m4_accounts", + ); + expect(account.options.validationLevel).toBe("moderate"); + expect(account.options.validator.$jsonSchema.bsonType).toBe("object"); + }); + + it("rejects a document the schema says is the wrong type", async () => { + // Proves the validator runs, rather than merely being present in options. + let rejected = false; + try { + await db.client.mongoInsertOne("m4_accounts", { + _id: 9901, + // `handle` is supplied so the document is rejected for the wrong type + // rather than for the missing required field. + handle: "typed", + email: "typed@example.com", + age: "not a number", + }); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + }); + + it("lets a document omit a unique column twice", async () => { + const repo = db.getRepository(Account); + await repo.create({ email: "first@example.com", handle: "first" }); + // Two documents that both *omit* `email` are both null to MongoDB. Without + // `sparse` the second of these is rejected, where SQL would allow it. + const second: any = await repo.create({ handle: "second" }); + expect(second.id).toBeGreaterThan(0); + + const raw = await db.client.mongoFindOne("m4_accounts", { + _id: second.id, + }); + expect("email" in raw).toBe(false); + }); + + it("updates a document that predates a newly declared required field", async () => { + // The `moderate` assertion. Under the default `"strict"`, this update is + // rejected — so `repo.update()` would fail on exactly the rows autoMigrate + // just declared a field for. + const Extended = defineModel({ + tableName: "m4_accounts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, unique: true }, + handle: { type: DataTypes.STRING, required: true, index: "idx_handle" }, + age: { type: DataTypes.INTEGER }, + nickname: { type: DataTypes.STRING }, + bio: { type: DataTypes.TEXT, maxLength: 280 }, + secret: { type: DataTypes.STRING, encrypted: true }, + // Newly declared and required. The documents written above do not have + // it. + region: { type: DataTypes.STRING, required: true }, + }, + }); + + await db.autoMigrate([Extended]); + + const raw = await db.client.mongoFindOne("m4_accounts", { + handle: "second", + }); + expect(raw.region).toBeUndefined(); + + // The whole point: this must not throw. + await db.client.mongoUpdateOne( + "m4_accounts", + { _id: raw._id }, + { $set: { nickname: "still no region" } }, + ); + const after = await db.client.mongoFindOne("m4_accounts", { _id: raw._id }); + expect(after.nickname).toBe("still no region"); + }); + + it("is a no-op the second time", async () => { + // No `NamespaceExists` and no `IndexOptionsConflict`, which is what a + // naive re-run would produce. + await db.autoMigrate([Account, Post]); + await db.autoMigrate([Account, Post]); + const names = (await db.client.mongoListCollections()).map( + (entry: any) => entry.name, + ); + expect(names.filter((name: string) => name === "m4_accounts")).toHaveLength(1); + }); + + it("pre-creates a counter so the first concurrent write cannot race", async () => { + const counter = await db.client.mongoFindOne(MONGO_COUNTERS_COLLECTION, { + _id: "m4_accounts", + }); + expect(counter.seq).toBeGreaterThanOrEqual(0); + }); + + it("runs a generated migration and records it in the ledger", async () => { + await db.client.mongoCommand({ drop: "m4_tokens" }).catch(() => {}); + await db.client.mongoDeleteMany(MONGO_MIGRATIONS_COLLECTION, { + _id: "create_m4_tokens", + }); + + const Token = defineModel({ + tableName: "m4_tokens", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + value: { type: DataTypes.STRING, unique: true }, + }, + }); + + const migration = generateMongoMigration(Token, "create_m4_tokens"); + await runMongoMigrations( + { type: DBType.MongoDB, connectionString: REPLICA_SET_URL }, + [migration], + ); + + const ledger = await db.client.mongoFindOne(MONGO_MIGRATIONS_COLLECTION, { + _id: "create_m4_tokens", + }); + expect(ledger).not.toBeNull(); + expect(ledger.applied_at).toBeInstanceOf(Date); + + const indexes = await db.client.mongoListIndexes("m4_tokens"); + expect(indexes.some((index: any) => index.key?.value === 1)).toBe(true); + + // Idempotent: the ledger entry is what stops a second application. + await runMongoMigrations( + { type: DBType.MongoDB, connectionString: REPLICA_SET_URL }, + [migration], + ); + + await db.client.mongoCommand({ drop: "m4_tokens" }).catch(() => {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "m4_tokens", + }); + }); + + it("refuses a migration that carries SQL for a MongoDB target", async () => { + // Reporting success for steps that were never run would be worse than + // failing. + let message = ""; + try { + await runMongoMigrations( + { type: DBType.MongoDB, connectionString: REPLICA_SET_URL }, + [ + { + name: "sql_only_m4", + up: ["CREATE TABLE nope (id INT)"], + down: [], + }, + ], + ); + } catch (error) { + message = (error as Error).message; + } + expect(message).toContain("no MongoDB steps"); + }); +}); diff --git a/tests/renamed-columns.test.ts b/tests/renamed-columns.test.ts new file mode 100644 index 0000000..6b4da1d --- /dev/null +++ b/tests/renamed-columns.test.ts @@ -0,0 +1,235 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; + +/** + * A column declared as `firstName: { name: "first_name" }` is stored as + * `first_name`, so every predicate the repository builds has to name the + * *column*. Four lookup paths named the *property* instead — `findOneBy`, + * `findBy`, `findMany({ where })` and `first` each pushed the raw key into + * `whereNull`, so `findOneBy({ firstName: null })` rendered + * `WHERE firstName IS NULL` and the database rejected the statement with + * "no such column: firstName". A null lookup by property name was simply + * unusable on a mapped column; the equality path next to it was not, which is + * what made the failure look arbitrary. + * + * The models below rename every column they filter on. That is the only shape + * that can expose the leak: when the property and the column happen to share a + * name, the wrong one is still valid SQL and the bug is invisible. + */ + +const Person = defineModel({ + tableName: "renamed_people", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + firstName: { type: DataTypes.STRING, name: "first_name" }, + lastName: { type: DataTypes.STRING, name: "last_name" }, + email: { type: DataTypes.STRING, name: "email_address", unique: true }, + }, +}); + +/** A renamed soft-delete column, so the implicit `IS NULL` filter is renamed too. */ +const Ticket = defineModel({ + tableName: "renamed_tickets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING }, + removedOn: { type: DataTypes.DATETIME, name: "removed_on", softDelete: true }, + }, +}); + +/** A renamed sort key, exercised through the cursor branch of `findMany`. */ +const Metric = defineModel({ + tableName: "renamed_metrics", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + seqNo: { type: DataTypes.INTEGER, name: "seq_no" }, + score: { type: DataTypes.INTEGER }, + }, +}); + +describe("renamed columns", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ type: DBType.SQLite, connectionString: ":memory:" }); + await db.autoMigrate([Person, Ticket, Metric]); + + const people = db.getRepository(Person); + // `firstName` is omitted, so the column is NULL — the row every + // `{ firstName: null }` lookup below has to find. + await people.create({ lastName: "Lovelace", email: "ada@example.com" }); + await people.create({ + firstName: "Grace", + lastName: "Hopper", + email: "grace@example.com", + }); + + const tickets = db.getRepository(Ticket); + await tickets.create({ title: "open" }); + await tickets.create({ title: "closed" }); + + const metrics = db.getRepository(Metric); + await metrics.create({ seqNo: 1, score: 10 }); + await metrics.create({ seqNo: 2, score: 20 }); + await metrics.create({ seqNo: 3, score: 30 }); + }); + + afterAll(async () => { + await db?.close(); + }); + + it("stores the fixtures under the column name, not the property name", async () => { + // Without this the rest of the file could pass vacuously: if `name` were + // ignored, `first_name` would not exist and neither would the bug. + const rows = await db.client.query( + "SELECT first_name, email_address FROM renamed_people ORDER BY id", + ); + expect(rows).toEqual([ + { first_name: null, email_address: "ada@example.com" }, + { first_name: "Grace", email_address: "grace@example.com" }, + ]); + }); + + it("has no column named after the property", async () => { + // The direct proof that a pass below came from translating the property + // rather than from the two names coinciding. + let leaked = false; + try { + await db.client.query("SELECT firstName FROM renamed_people"); + } catch { + leaked = true; + } + expect(leaked).toBe(true); + }); + + // ─── the four sites that named the property ──────────────────────── + // + // Rows come back keyed by column name, which is the existing convention — + // see `author_id` in integration.relations.test.ts. + + it("findOneBy resolves a null property to its column", async () => { + const found = await db.getRepository(Person).findOneBy({ firstName: null }); + expect(found).not.toBeNull(); + expect(found.last_name).toBe("Lovelace"); + expect(found.first_name).toBeNull(); + }); + + it("findBy resolves a null property to its column", async () => { + const found = await db.getRepository(Person).findBy({ firstName: null }); + expect(found).toHaveLength(1); + expect(found[0].last_name).toBe("Lovelace"); + }); + + it("findMany resolves a null property to its column", async () => { + const found = await db + .getRepository(Person) + .findMany({ where: { firstName: null } }); + expect(found).toHaveLength(1); + expect(found[0].last_name).toBe("Lovelace"); + }); + + it("first resolves a null property to its column", async () => { + const found = await db.getRepository(Person).first({ firstName: null }); + expect(found).not.toBeNull(); + expect(found.last_name).toBe("Lovelace"); + }); + + it("count resolves a null property to its column", async () => { + // This one was already correct; it is here so the fix cannot quietly + // regress into the same shape it just left. + expect(await db.getRepository(Person).count({ firstName: null })).toBe(1); + }); + + // ─── the equality half, which has to keep working ────────────────── + + it("resolves a non-null property to its column on every lookup", async () => { + const people = db.getRepository(Person); + expect((await people.findOneBy({ firstName: "Grace" })).last_name).toBe( + "Hopper", + ); + expect(await people.findBy({ firstName: "Grace" })).toHaveLength(1); + expect( + await people.findMany({ where: { firstName: "Grace" } }), + ).toHaveLength(1); + expect((await people.first({ firstName: "Grace" })).last_name).toBe( + "Hopper", + ); + expect(await people.count({ firstName: "Grace" })).toBe(1); + }); + + it("finds nothing for a value no row holds", async () => { + const people = db.getRepository(Person); + expect(await people.findOneBy({ firstName: "Nobody" })).toBeNull(); + expect(await people.findBy({ firstName: "Nobody" })).toEqual([]); + }); + + it("plucks a renamed column by property name", async () => { + const names = await db.getRepository(Person).pluck("firstName"); + expect(names).toEqual([null, "Grace"]); + }); + + // ─── renamed columns through the soft-delete filter ──────────────── + + // Each of these seeds its own rows under a title of its own, so none of + // them depends on what a neighbour left behind — the table is shared. + + it("filters and recovers on a renamed soft-delete column", async () => { + const tickets = db.getRepository(Ticket); + const open = await tickets.create({ title: "sd-open" }); + const closed = await tickets.create({ title: "sd-closed" }); + + await tickets.delete(closed.id); + expect(await tickets.findBy({ title: "sd-closed" })).toEqual([]); + + const deleted = await tickets.findDeleted().execute(db.client); + expect(deleted.map((t: any) => t.title)).toContain("sd-closed"); + + await tickets.recover(closed.id); + expect(await tickets.findBy({ title: "sd-closed" })).toHaveLength(1); + expect((await tickets.findOne(open.id)).title).toBe("sd-open"); + }); + + it("counts only the rows the renamed soft-delete filter admits", async () => { + const tickets = db.getRepository(Ticket); + await tickets.create({ title: "cnt-keep" }); + const doomed = await tickets.create({ title: "cnt-doomed" }); + + const before = await tickets.count(); + await tickets.delete(doomed.id); + expect(await tickets.count()).toBe(before - 1); + }); + + it("deleteBy and restoreBy resolve a renamed condition", async () => { + const tickets = db.getRepository(Ticket); + await tickets.create({ title: "bulk-same" }); + await tickets.create({ title: "bulk-same" }); + + expect(await tickets.deleteBy({ title: "bulk-same" })).toBe(2); + expect(await tickets.findBy({ title: "bulk-same" })).toEqual([]); + + expect(await tickets.restoreBy({ title: "bulk-same" })).toBe(2); + expect(await tickets.findBy({ title: "bulk-same" })).toHaveLength(2); + }); + + // ─── renamed columns through the cursor branch ───────────────────── + + it("pages by a renamed sort column", async () => { + const metrics = db.getRepository(Metric); + const page = await metrics.findMany({ + cursor: { field: "seqNo", value: 1 }, + orderBy: { field: "seqNo", direction: "ASC" }, + take: 10, + }); + expect(page.map((m: any) => m.seq_no)).toEqual([2, 3]); + }); + + it("orders by a renamed column", async () => { + const metrics = db.getRepository(Metric); + const desc = await metrics.findMany({ + orderBy: { field: "seqNo", direction: "DESC" }, + }); + expect(desc.map((m: any) => m.seq_no)).toEqual([3, 2, 1]); + }); +}); diff --git a/types.ts b/types.ts index 170b5b2..cfe0947 100644 --- a/types.ts +++ b/types.ts @@ -9,6 +9,7 @@ export enum DBType { MySQL = "mysql", SQLite = "sqlite", MSSQL = "mssql", + MongoDB = "mongodb", } export enum LogLevel { @@ -51,6 +52,24 @@ export interface DBConfig { retryAttempts?: number; retryDelay?: number; maxJitter?: number; + /** + * The MongoDB database to operate on. + * + * Only meaningful for `DBType.MongoDB`. Every other backend takes its + * database from the connection string, but a mongo URI may legitimately omit + * one (and often does in development), so it is accepted separately as well. + * When both are present the URI's own path wins and this is ignored. + */ + database?: string; + /** + * Extra options handed verbatim to the MongoDB driver's `MongoClient`. + * + * For `DBType.MongoDB` only. This is the escape hatch for driver settings the + * ORM has no opinion about — `tls`, `authSource`, `maxPoolSize`, `retryWrites` + * and the rest — without the ORM having to model, and stay current with, the + * driver's full option surface. + */ + mongoOptions?: Record; } export interface CacheConfig { @@ -88,10 +107,44 @@ export interface CacheStats { keys: number; } +/** + * One schema change against MongoDB, as data rather than as a closure. + * + * A discriminated union so a generated migration is serializable and assertable + * without a server — the same reason `buildLimitClause` and + * `buildMSSQLUpsertSQL` are pure. It lives here rather than beside the schema + * derivation because `Migration` needs it, and `mongo-schema` already imports + * from this module; declaring it there would make the two files import each + * other. + */ +export type MongoStep = + | { kind: "createCollection"; collection: string; validator?: any } + | { + kind: "createIndex"; + collection: string; + spec: Record; + options?: Record; + } + | { kind: "dropIndex"; collection: string; name: string } + | { kind: "collMod"; collection: string; validator: any } + | { kind: "dropCollection"; collection: string } + | { kind: "createCounter"; collection: string }; + export interface Migration { name: string; up: string[]; down: string[]; + /** + * The MongoDB steps this migration applies. + * + * A sidecar rather than a replacement for `up`/`down`, because those are SQL + * and `tests/migrations.test.ts` asserts their exact strings. A migration + * generated for a SQL target leaves both of these undefined, and + * `runMigrations` never reads them. + */ + mongoUp?: MongoStep[]; + /** The MongoDB inverse of {@link mongoUp}. */ + mongoDown?: MongoStep[]; } export class StabilizeError extends Error { From 57eed517a64f236ca1d72c7176a59c94791ef056 Mon Sep 17 00:00:00 2001 From: ElectronSz Date: Wed, 16 Sep 2026 12:43:30 +0200 Subject: [PATCH 3/5] Add a MongoDB backend: write path, relations, versioning and escape hatches MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the backend begun in 681e341. The four SQL backends are untouched: every MongoDB body sits behind an early return in repository.ts, and the shared code above it — validation, hooks, timestamps, optimistic locking, cache invalidation, relation algorithms, encryption — is reused, not duplicated. Write path: create/bulkCreate, update/bulkUpdate, upsert, delete/bulkDelete, recover, plus updateBy/deleteBy/restoreBy, increment/decrement/toggle, aggregate, countDistinct, paginate, random and the findMany cursor. Relations: attach/fetch/sync for many-to-many, with the link collection keyed by a compound {parent, child} _id so attach is idempotent at the storage layer rather than by pre-read. Link reads are explicitly sorted, because MongoDB's to-many order is unspecified and not stable between two reads of unchanged data. Versioning: asOf/history/rollback/writeHistory against a history collection with native Date validity windows, so the range comparisons stay index-backed. A delete now records max(sent, newestRecorded + 1): on MongoDB the version is half the compound _id, so recording the row's current version collided with the row it was journalling and failed the delete it was meant to record. Escape hatches: the 22 clauses with no MongoDB equivalent throw MONGO_UNSUPPORTED naming every offending method, and an inverse table proves an allowed clause is not refused. lock()/forUpdate() is a no-op that Repository.lockForUpdate now warns about, since the read it returns is indistinguishable from a locked one. Also fixes aggregate() on an empty collection, where $group over no input produces no document at all and the count came back undefined rather than 0. Flagged, not fixed: processForLoad reads processed[key] while rows arrive keyed by column name, so an encrypted column that declares name: is not decrypted on read. This affects all five backends and is left for a separate change. --- .gitignore | 3 + README.md | 135 +++- client.ts | 15 +- index.ts | 13 + mongo-migrate.ts | 4 +- mongo-repository.ts | 1197 +++++++++++++++++++++++++++++- mongo-schema.ts | 138 +++- query-builder.ts | 16 +- repository.ts | 374 ++++++++-- tests/integration.mongo.test.ts | 55 ++ tests/mongo.blockers.test.ts | 679 +++++++++++++++++ tests/mongo.relations.test.ts | 618 +++++++++++++++ tests/mongo.transactions.test.ts | 484 ++++++++++++ tests/mongo.versioning.test.ts | 641 ++++++++++++++++ tests/mongo.write-paths.test.ts | 786 ++++++++++++++++++++ types.ts | 14 + 16 files changed, 5077 insertions(+), 95 deletions(-) create mode 100644 tests/mongo.blockers.test.ts create mode 100644 tests/mongo.relations.test.ts create mode 100644 tests/mongo.transactions.test.ts create mode 100644 tests/mongo.versioning.test.ts create mode 100644 tests/mongo.write-paths.test.ts diff --git a/.gitignore b/.gitignore index bdb14b5..eabe4f2 100644 --- a/.gitignore +++ b/.gitignore @@ -5,10 +5,13 @@ node_modules out dist *.tgz +*.sql + # code coverage coverage *.lcov +*table.json # logs logs diff --git a/README.md b/README.md index a2e5957..a90d8a1 100644 --- a/README.md +++ b/README.md @@ -16,13 +16,14 @@ _A Modern, Type-Safe, and Expressive ORM for Bun_ MIT License

-**Stabilize** is a lightweight, feature-rich ORM designed for performance and developer experience. It provides a unified, database-agnostic API for **PostgreSQL**, **MySQL/MariaDB**, **SQLite**, and **SQL Server**. Powered by a robust query builder, programmatic model definitions, automatic versioning, and a full-featured command-line interface, Stabilize is built to scale with your app. +**Stabilize** is a lightweight, feature-rich ORM designed for performance and developer experience. It provides a unified, database-agnostic API for **PostgreSQL**, **MySQL/MariaDB**, **SQLite**, and **SQL Server**, plus a **MongoDB** document backend with the boundaries spelled out [below](#-mongodb). Powered by a robust query builder, programmatic model definitions, automatic versioning, and a full-featured command-line interface, Stabilize is built to scale with your app. --- ## 🚀 Features - **Unified API**: Write once, run on PostgreSQL, MySQL/MariaDB, SQLite, or SQL Server. +- **MongoDB Backend**: `DBType.MongoDB` runs the same repositories, relations, hooks and structured query builder against a document store, through the optional `mongodb` driver. Raw SQL, joins, unions, CTEs and raw clauses are refused with a `MONGO_UNSUPPORTED` error rather than mistranslated; the [boundaries](#-mongodb) are documented in full. - **Programmatic Model Definitions**: Define models and columns using the `defineModel` API with the `DataTypes` enum for database-agnostic schemas. - **Full-Featured CLI**: Generate models, manage migrations, seed data, and reset your database from the command line with [stabilize-cli](https://github.com/ElectronSz/stabilize-cli). - **Automatic Migrations**: Generate database-specific SQL schemas directly from your model definitions. @@ -173,6 +174,138 @@ export const orm = new Stabilize(dbConfig, cacheConfig, loggerConfig); --- +## 🍃 MongoDB + +`DBType.MongoDB` selects a document backend rather than a fifth SQL dialect. Its +driver is the only optional dependency in the package, so install it alongside: + +```bash +bun add mongodb +``` + +```typescript +// config/database.ts +import { DBType, type DBConfig } from "stabilize-orm"; + +const dbConfig: DBConfig = { + type: DBType.MongoDB, + connectionString: process.env.MONGO_URL || "mongodb://localhost:27017/mydb", + // Optional. The fallback database for a URI that omits one from its path, + // which is how a mongo URI is usually written in development. + database: "mydb", + // Optional. Passed verbatim to the driver's `MongoClient` — `tls`, + // `authSource`, `maxPoolSize`, `retryWrites` and anything else the ORM has no + // opinion about. + mongoOptions: { maxPoolSize: 20 }, +}; + +export default dbConfig; +``` + +Models, repositories, relations, hooks, versioning, soft deletes, validation, +encryption, aggregates and transactions work as they do on SQL. A query is +written with the query builder's *structured* methods — the ones that record a +condition rather than SQL text: + +```typescript +const users = await orm + .getRepository(User) + .find() + .whereEq("isActive", true) + .whereIn("role", ["admin", "editor"]) + .orderBy("createdAt", "DESC") + .withRelations("roles") + .execute(orm.client); +``` + +`healthCheck()` pings the server. `poolStats()` returns +`{ active: -1, idle: -1, total: -1 }`: the driver's pool is internal and +per-server, so there is no honest number to report and the sentinel says so +rather than inventing one. + +### What MongoDB cannot do + +A document store is not a SQL engine, and Stabilize refuses to guess where the +two disagree. Every one of these is deliberate, and each is reported rather than +silently mistranslated — a dropped `join()` would return the wrong rows with no +error to notice. + +- **Raw SQL is refused.** `rawQuery()`, `rawExec()`, `query()` and `queryExec()` + throw a `StabilizeError` with code `MONGO_UNSUPPORTED`. Use the repository API + or the query builder's structured methods instead. + + ```typescript + await orm.rawQuery("SELECT * FROM users WHERE age > ?", [25]); + // StabilizeError: Raw SQL is not available on MongoDB. ... + ``` + +- **Joins, unions, CTEs and raw clauses throw.** `join()`, `innerJoin()`, + `leftJoin()`, `rightJoin()`, `fullJoin()`, `crossJoin()`, `union()`, + `unionAll()`, `with()`, `withRecursive()`, `whereRaw()`, `whereRef()`, + `whereExists()`, `whereNotExists()`, `selectRaw()`, `orderByRaw()`, + `groupByRaw()`, `having()`, `distinct()` and the SQL-text forms of + `where()`/`orWhere()`/`whereNot()` have no MongoDB equivalent. The throw + happens when the query is **executed**, not when the clause is added, and it + names every offending method at once: + + ```typescript + await repo + .find() + .innerJoin("posts", "posts.user_id = users.id") + .whereRaw("LOWER(name) = 'ada'") + .execute(orm.client); + // StabilizeError: This query cannot be translated to MongoDB: innerJoin, + // whereRaw have no MongoDB equivalent. ... Use withRelations() for related + // documents, or run this query against a SQL backend. + ``` + + For a join, the replacement is `withRelations()`, which loads related + documents with batched reads rather than one statement: + + ```typescript + await repo.find().withRelations("posts", "posts.comments").execute(orm.client); + ``` + +- **`lock()` / `forUpdate()` is a no-op.** MongoDB has no row lock to map it + onto, so the clause is not rendered and the query runs unlocked rather than + failing. `lockForUpdate()` therefore reads the row without protecting it — + use `updateBy()` with a condition, or an optimistic lock column, for a + read-modify-write that has to be safe. + +- **`DECIMAL` is stored as a `double`.** MongoDB has no exact decimal unless the + caller supplies a `Decimal128`, so a `DECIMAL` column loses precision the way + a binary float does. For money, store the smallest unit as an `INTEGER`/`BIGINT` + or the value as a `STRING`. + +- **Auto-increment ids come from a counters collection — and they roll back.** + Ids are reserved by a `$inc` against `stabilize_counters`, keyed by collection + name, rather than by the server. Because that reservation runs inside the same + transaction as the write, an aborted transaction gives its ids back: the next + insert re-uses them. MySQL behaves the opposite way — InnoDB's auto-increment + counter is not transactional, so an aborted insert leaks the gap. + +- **Transactions need a replica set or a sharded cluster.** A standalone + `mongod` serves reads but rejects every transaction — and every repository + write (`create()`, `update()`, `delete()`, `bulkCreate()`, `upsert()` …) runs + inside one, so a standalone makes writes fail generally, not only + explicitly-transactional code. The client warns at connect time and reports + the failure as `TX_ERROR`: + + ```typescript + // Standalone mongod, no replica set: + await repo.create({ name: "Ada" }); + // StabilizeError (TX_ERROR): MongoDB transactions require a replica set or + // sharded cluster, and this server is a standalone. Every write goes through + // a transaction, so start the server with --replSet and run rs.initiate() + // (or point the connection at an existing replica set). + ``` + + Run a single-node replica set in development + (`rs.initiate()` on a `mongod --replSet rs0`) and every write path works + unchanged. + +--- + ## 🏗️ Models & Relationships Define your tables as classes using the `defineModel` function. The `DataTypes` enum ensures database-agnostic schemas. diff --git a/client.ts b/client.ts index 59da0da..3df2743 100644 --- a/client.ts +++ b/client.ts @@ -1143,11 +1143,18 @@ export class DBClient { ); } - /** Updates the first matching document. */ + /** + * Updates the first matching document. + * + * `update` also accepts a pipeline array. A pipeline is the only way to + * express a value computed from the document's own current contents — which + * is what `toggle()` needs, and there is no update operator that flips a + * field in place. + */ async mongoUpdateOne( collection: string, filter: Record, - update: Record, + update: Record | Record[], options: Record = {}, ): Promise { return this.mongoRun("updateOne", { collection, filter }, async (db) => @@ -1159,11 +1166,11 @@ export class DBClient { ); } - /** Updates every matching document. */ + /** Updates every matching document. @see mongoUpdateOne for the pipeline form. */ async mongoUpdateMany( collection: string, filter: Record, - update: Record, + update: Record | Record[], options: Record = {}, ): Promise { return this.mongoRun("updateMany", { collection, filter }, async (db) => diff --git a/index.ts b/index.ts index 164087f..41069c5 100644 --- a/index.ts +++ b/index.ts @@ -41,6 +41,12 @@ import { type StabilizeEventHandler, StabilizeEmitter, generateUUID, + // The two MongoDB types this package re-exports. Both are reached through + // `types.ts`, the shared type surface: `mongo-query` and `mongo-schema` are + // not published entry points, so neither is somewhere a consumer could name + // them from. + type Predicate, + type MongoStep, } from "./types"; import { defineModel, MetadataStorage } from "./model"; import type { Hook } from "./hooks"; @@ -305,4 +311,11 @@ export type { QueryLogEntry, StabilizeEvent, StabilizeEventHandler, + // The two MongoDB types a consumer can legitimately need to name: the + // predicate the query builder records, and the serializable migration step. + // Both are reached through `types.ts`, the shared type surface, rather than + // through `mongo-query`/`mongo-schema` — neither of which is a published + // entry point. + Predicate, + MongoStep, }; diff --git a/mongo-migrate.ts b/mongo-migrate.ts index a48d504..d8c7075 100644 --- a/mongo-migrate.ts +++ b/mongo-migrate.ts @@ -137,7 +137,9 @@ export function generateMongoMigration( up: [], down: [], mongoUp: buildUpSteps(model), - mongoDown: [{ kind: "dropCollection", collection: meta.tableName }], + // Derived, not hand-written, so the inverse of a migration is the inverse of + // exactly what it created — a versioned model's history collection included. + mongoDown: generateMongoSteps(model, "down"), }; } diff --git a/mongo-repository.ts b/mongo-repository.ts index 74e17b7..b964412 100644 --- a/mongo-repository.ts +++ b/mongo-repository.ts @@ -18,7 +18,13 @@ import { DBClient } from "./client"; import { StabilizeError } from "./types"; -import { OMIT, normalizeMongoDoc, sanitizeMongoValue } from "./mongo-query"; +import { + type MongoAggregate, + OMIT, + buildMongoAggregatePipeline, + normalizeMongoDoc, + sanitizeMongoValue, +} from "./mongo-query"; /** * The collection auto-increment ids are drawn from. @@ -35,6 +41,34 @@ export interface MongoColumn { encrypted?: boolean; } +/** + * The three names a many-to-many link is addressed by. + * + * The join collection has no model of its own, so nothing here resolves + * through column metadata — these names come from the relation config and are + * used verbatim, exactly as the SQL path uses them as table and column names. + */ +export interface MongoLinkRelation { + joinTable: string; + foreignKey: string; + inverseKey: string; +} + +/** + * The composite `_id` a link document is keyed by. + * + * `_id` uniqueness is the one uniqueness constraint MongoDB enforces on every + * collection without an index being declared, so keying the link by the pair + * makes `attach` idempotent at the storage layer: the second insert of the same + * pair cannot land, whether or not the caller checked first. The SQL join table + * has no such constraint and needs the pre-read to avoid a duplicate row. + * + * One function, so the shape the writer builds is the shape the reader matches. + */ +function linkId(parent: any, child: any): { p: any; c: any } { + return { p: parent, c: child }; +} + /** * The parts of a `Repository` the MongoDB write bodies depend on. * @@ -56,9 +90,24 @@ export interface MongoRepositoryHost { * caller-supplied (a UUID or string id). */ autoIncrementField: string | null; + /** The soft-delete property key, or null when the model does not soft-delete. */ + softDeleteField: string | null; + /** The soft-delete *column* name. @see softDeleteField */ + softDeleteColumn: string | null; + /** The optimistic-lock property key, or null when the model has no lock. */ + optimisticLockField: string | null; + /** The optimistic-lock *column* name. @see optimisticLockField */ + optimisticLockColumn: string | null; + /** The timestamp property keys, or null when timestamps are off. */ + timestamps: { createdAt?: string; updatedAt?: string } | null; + /** + * The collection a versioned model's history documents are written to. Always + * set, because only a versioned repository reaches the bodies that use it. + */ + historyTable: string; logger: { logDebug(message: string): void }; /** Throws when the entity fails the model's validators. */ - validate(entity: any): void; + validate(entity: any, skipRequired?: boolean): void; /** Applies the timestamp and optimistic-lock defaults a create writes. */ seedCreateDefaults(entity: Record): Record; /** Encrypts encrypted columns and normalises values for storage. */ @@ -242,6 +291,121 @@ function chunked(items: T[], size: number): T[][] { return chunks; } +/** + * Turns a prepared patch into `$set`/`$unset` halves. + * + * The difference from {@link buildMongoDocument} is what a null means. On an + * insert it means "this column was not supplied", so the key is dropped. On an + * update it means "clear this column", which on a document store is `$unset` — + * dropping the key would leave the old value in place and silently ignore the + * caller. SQL spells both the same way (`SET col = NULL`), which is why the two + * builders cannot be one. + * + * @param host The repository the patch belongs to. + * @param prepared A patch already through `processForSave`. + */ +function buildMongoPatch( + host: MongoRepositoryHost, + prepared: Record, +): { $set: Record; $unset: Record } { + const $set: Record = {}; + const $unset: Record = {}; + for (const [key, value] of Object.entries(prepared)) { + const column = host.columns[key]; + // The primary key is `_id`, which is never patched: a change of identity is + // a delete and an insert, not an update. + if (!column || key === host.idProperty) continue; + const sanitized = sanitizeMongoValue(value); + if (sanitized === OMIT) $unset[column.name] = ""; + else $set[column.name] = sanitized; + } + return { $set, $unset }; +} + +/** + * Assembles the update document, leaving out the halves that are empty. + * + * MongoDB rejects `$set: {}` as an empty operator, so an absent half has to be + * an absent key rather than an empty object. + */ +function updateOperation( + $set: Record, + $unset: Record, +): Record { + const operation: Record = {}; + if (Object.keys($set).length > 0) operation.$set = $set; + if (Object.keys($unset).length > 0) operation.$unset = $unset; + return operation; +} + +/** + * Turns a set of equality conditions into a filter. + * + * The soft-delete clause is deliberately *not* added here: the operations that + * want it differ in the direction they want it, so each caller adds its own. + * + * @param host The repository the conditions belong to. + * @param conditions Property key → value, as the public methods take them. + * @param options `skipNull` matches `restoreBy`'s behaviour of ignoring a null + * condition rather than treating it as `IS NULL`. + * @throws StabilizeError `UNSAFE_QUERY` when the conditions name no known + * column: the repository has already refused a *caller* that passed nothing, + * but a condition set of unknown keys would otherwise become an empty filter, + * which is the same statement with no WHERE clause this is meant to prevent. + */ +function buildMongoConditions( + host: MongoRepositoryHost, + conditions: Record, + options: { skipNull?: boolean } = {}, +): Record { + const filter: Record = {}; + for (const [key, value] of Object.entries(conditions)) { + const column = host.columns[key]; + if (!column) continue; + if (value === null || value === undefined) { + if (options.skipNull) continue; + // `{field: null}` matches a document where the field is null *or absent*, + // which is what SQL's `IS NULL` means for a column that always exists. + filter[column.name] = null; + continue; + } + filter[column.name] = sanitizeMongoValue(value); + } + + if (Object.keys(conditions).length > 0 && Object.keys(filter).length === 0) { + throw new StabilizeError( + `None of the conditions ${JSON.stringify(Object.keys(conditions))} name a column of ${host.table}; refusing to affect every document.`, + "UNSAFE_QUERY", + ); + } + return filter; +} + +/** + * Reads an affected-row count out of a driver write result. + * + * Driver 6 reports `matchedCount` for an update and `deletedCount` for a + * delete. Driver 5 nested both under `result.n`, so that spelling is read too — + * a silent zero here would make `updateBy` report having changed nothing while + * the write had actually landed. + */ +function affectedRows( + result: any, + field: "matchedCount" | "deletedCount", +): number { + const direct = result?.[field]; + if (typeof direct === "number") return direct; + const legacy = result?.result?.n; + return typeof legacy === "number" ? legacy : 0; +} + +/** The soft-delete filter that keeps a write to rows that are not deleted. */ +function notDeletedFilter(host: MongoRepositoryHost): Record { + const column = host.softDeleteColumn; + if (!host.softDeleteField || !column) return {}; + return { [column]: null }; +} + /** * Inserts one entity. * @@ -382,3 +546,1032 @@ export async function mongoBulkCreate( ); return results; } + +/** + * Applies a patch to one row. + * + * The optimistic lock is advanced inside the same update and matched on in the + * filter, which is the whole mechanism: a filter that no longer matches is a + * conflict, and `matchedCount === 0` is how it is noticed. Doing the read and + * the compare as separate round trips would let two writers pass the check + * together and lose one of the updates. + * + * @param host The repository being written through. + * @param id The row to patch. + * @param entity The caller's patch. + * @param before The row as it was read, for the lock value the caller did not + * supply. + * @param client The client, which supplies the session and the collection. + * @returns The stored entity. + * @throws StabilizeError `CONCURRENT_MODIFICATION` when the lock filter misses. + */ +export async function mongoUpdate( + host: MongoRepositoryHost, + id: number | string, + entity: Record, + before: any, + client: DBClient, +): Promise { + const start = Date.now(); + host.logger.logDebug(`Updating ${host.table} with ID ${id}`); + // The payload is a partial patch, exactly like the SQL path's: required + // columns that are not being changed must not be demanded here. + host.validate(entity, true); + + const prepared: Record = { ...entity }; + const timestamps = host.timestamps; + if (timestamps?.updatedAt && !prepared[timestamps.updatedAt]) { + prepared[timestamps.updatedAt] = new Date().toISOString(); + } + // Encrypt and coerce after the timestamp is in place, so it passes through + // the same treatment everything else does. + Object.assign(prepared, host.processForSave(prepared)); + + let lockValue: any; + if (host.optimisticLockField) { + // Rows can arrive keyed by property or by column, so both are read — a + // renamed lock column would otherwise be undefined and the lock would + // silently do nothing. + lockValue = + before?.[host.optimisticLockField] ?? + (host.optimisticLockColumn + ? before?.[host.optimisticLockColumn] + : undefined); + + // A caller that passes the version it read expects a conflict if someone + // else has written since, so the caller's value wins over the one just + // read inside this transaction, which would always match. + const callerVersion = entity[host.optimisticLockField]; + const expected = + callerVersion !== undefined && callerVersion !== null + ? callerVersion + : lockValue; + + if (expected !== undefined) { + lockValue = expected; + prepared[host.optimisticLockField] = + typeof expected === "number" ? expected + 1 : 1; + } + } + + const { $set, $unset } = buildMongoPatch(host, prepared); + const operation = updateOperation($set, $unset); + + const filter: Record = { _id: id }; + if (host.optimisticLockField && lockValue !== undefined) { + // A null version is matched with `{col: null}`, which covers both a stored + // null and an absent field. SQL needs `IS NULL` there because `= NULL` is + // never true; here the one filter already means both. + filter[host.optimisticLockColumn!] = lockValue ?? null; + } + if (host.softDeleteField) { + filter[host.softDeleteColumn!] = null; + } + + // An update with no operators is rejected outright by the driver, so a patch + // that changes nothing writes nothing. The SQL path produces `SET` with an + // empty list there, which is a syntax error — but only for a payload that + // carries just the key, and doing nothing is the honest reading of that. + if (Object.keys(operation).length > 0) { + const result = await client.mongoUpdateOne(host.table, filter, operation); + if ( + host.optimisticLockField && + lockValue !== undefined && + affectedRows(result, "matchedCount") === 0 + ) { + throw new StabilizeError( + `Record was modified by another transaction (optimistic lock conflict on ${host.optimisticLockField})`, + "CONCURRENT_MODIFICATION", + ); + } + } + + const result = await host.findOne(id, {}, client); + + await host.invalidateRowCache(id); + await host.writeThroughRow(id, result); + + host.logger.logDebug( + `Updated ${host.table} with ID ${id} in ${Date.now() - start}ms`, + ); + return result; +} + +/** + * Removes one row, softly when the model soft-deletes. + * + * The soft-delete stamp is a native `Date`, not an ISO string: the "is this row + * deleted" filter is a comparison, and a string would compare lexically against + * whatever else is in the column. + * + * @param host The repository being written through. + * @param id The row to remove. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoDeleteRow( + host: MongoRepositoryHost, + id: number | string, + client: DBClient, +): Promise { + const start = Date.now(); + host.logger.logDebug(`Deleting ${host.table} with ID ${id}`); + + if (host.softDeleteField) { + await client.mongoUpdateOne( + host.table, + { _id: id }, + { $set: { [host.softDeleteColumn!]: new Date() } }, + ); + } else { + await client.mongoDeleteOne(host.table, { _id: id }); + } + + await host.invalidateRowCache(id); + host.logger.logDebug( + `Deleted ${host.table} with ID ${id} in ${Date.now() - start}ms`, + ); +} + +/** + * Clears a row's soft-delete stamp. + * + * `$unset` rather than a stored null, for the same reason the insert path omits + * null keys: an absent field is what "not deleted" looks like everywhere else + * in this backend, and writing an explicit null would make the state + * unrepresentable in a sparse index later. + * + * @param host The repository being written through. + * @param id The row to recover. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoRecover( + host: MongoRepositoryHost, + id: number | string, + client: DBClient, +): Promise { + await client.mongoUpdateOne( + host.table, + { _id: id }, + { $unset: { [host.softDeleteColumn!]: "" } }, + ); +} + +/** + * Writes the row an upsert lands on, and reads it back. + * + * The conflict keys are the filter and the payload is the update, in one + * `findOneAndUpdate` with `upsert: true` — so two callers racing on a key that + * does not exist yet cannot both insert. `before` is passed in only because the + * repository has already resolved it to choose the hook pair; the write does + * not depend on it being right. + * + * The generated `_id` is allocated before the write, which means the loser of + * such a race burns an id. That gap is the same one MySQL and SQLite leave when + * a rolled-back insert consumes an auto-increment value, and the alternative — + * reading the counter back after inserting — needs an `_id` that does not exist + * yet. + * + * @param host The repository being written through. + * @param keys The conflict-key property names. Empty means an unconditional + * insert, since there is nothing to match on and an empty filter would match + * an arbitrary document. + * @param values The payload, already through `processForSave`. + * @param before The row the conflict keys resolved to, or null. + * @param client The client, which supplies the session and the collection. + * @returns The stored row, normalised. + */ +export async function mongoUpsertRow( + host: MongoRepositoryHost, + keys: string[], + values: Record, + before: any, + client: DBClient, +): Promise { + const existingId = + before?.[host.idProperty] ?? + (host.idColumn ? before?.[host.idColumn] : undefined); + + if (existingId !== undefined && existingId !== null) { + const { $set, $unset } = buildMongoPatch(host, values); + const operation = updateOperation($set, $unset); + if (Object.keys(operation).length > 0) { + await client.mongoUpdateOne(host.table, { _id: existingId }, operation); + } + return host.processForLoad( + normalizeMongoDoc( + await client.mongoFindOne(host.table, { _id: existingId }), + host.idColumn, + ), + ); + } + + const document = buildMongoDocument(host, values); + const explicit = values[host.idProperty]; + + if (explicit !== undefined && explicit !== null) { + document._id = explicit; + if (typeof explicit === "number" && Number.isFinite(explicit)) { + await advanceMongoCounter(client, host.table, explicit); + } + } else if (host.autoIncrementField) { + document._id = (await allocateMongoIds(client, host.table, 1))[0]; + } + + if (keys.length === 0) { + await client.mongoInsertOne(host.table, document); + return host.processForLoad(normalizeMongoDoc(document, host.idColumn)); + } + + const conflict = buildMongoConditions( + host, + Object.fromEntries(keys.map((key) => [key, values[key]])), + ); + + const { $set, $unset } = buildMongoPatch(host, values); + const operation = updateOperation($set, $unset); + Object.assign(operation, { $setOnInsert: { _id: document._id } }); + + try { + const reply = await client.mongoFindOneAndUpdate(host.table, conflict, operation, { + upsert: true, + returnDocument: "after", + }); + return host.processForLoad(normalizeMongoDoc(reply, host.idColumn)); + } catch (error) { + if (!isDuplicateKey(error)) throw error; + // The other upsert won the insert between the read and this write, so the + // key exists now and the same operation is an ordinary update. The id + // allocated above goes unused, which is the gap noted on this function. + const reply = await client.mongoFindOneAndUpdate( + host.table, + conflict, + updateOperation($set, $unset), + { returnDocument: "after" }, + ); + return host.processForLoad(normalizeMongoDoc(reply, host.idColumn)); + } +} + +/** + * Adds `amount` to one numeric column. + * + * `$inc` is server-side, so two concurrent callers both land — which the SQL + * `col = col + ?` also guarantees. One divergence is worth knowing: `$inc` + * against a field that is *absent* initialises it to the increment, where SQL + * would compute `NULL + n` and leave the column NULL. On SQL the caller gets a + * null back; here it gets the amount. Neither is wrong, but a caller relying on + * the SQL behaviour will see a different value. + * + * @param host The repository being written through. + * @param id The row to change. + * @param column The column property name. + * @param amount How much to add. Negative values subtract. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoIncrement( + host: MongoRepositoryHost, + id: number | string, + column: string, + amount: number, + client: DBClient, +): Promise { + const field = host.columns[column]?.name ?? column; + await client.mongoUpdateOne( + host.table, + { _id: id, ...notDeletedFilter(host) }, + { $inc: { [field]: amount } }, + ); +} + +/** + * Flips a boolean column. + * + * A pipeline update, because there is no update operator that reads a field in + * order to write it. The comparison is against `true`, which is the value the + * ORM stores for a boolean column on this backend — the SQL path's + * `CASE WHEN col = 1` is about the 1/0 that SQLite and MySQL are handed, and a + * document that has never been written reads as `false` and flips to `true`, + * which is what a missing column means here anyway. + * + * @param host The repository being written through. + * @param id The row to change. + * @param column The column property name. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoToggle( + host: MongoRepositoryHost, + id: number | string, + column: string, + client: DBClient, +): Promise { + const field = host.columns[column]?.name ?? column; + await client.mongoUpdateOne( + host.table, + { _id: id, ...notDeletedFilter(host) }, + [ + { + $set: { + [field]: { $cond: [{ $eq: [`$${field}`, true] }, false, true] }, + }, + }, + ], + ); +} + +/** + * Applies one patch to every row a condition set matches. + * + * @param host The repository being written through. + * @param conditions Property key → value. + * @param values The patch, already through `processForSave`. + * @param client The client, which supplies the session and the collection. + * @returns How many rows matched, which is zero when the patch is empty. + */ +export async function mongoUpdateMany( + host: MongoRepositoryHost, + conditions: Record, + values: Record, + client: DBClient, +): Promise { + const { $set, $unset } = buildMongoPatch(host, values); + const operation = updateOperation($set, $unset); + + // A bulk update advances the version too, so the rows it touched are not left + // holding a version that no longer matches and rejecting the next ordinary + // write as a conflict. `$inc` and `$set` cannot touch the same path, so this + // is only added when the patch did not set the lock itself. + if (host.optimisticLockField && !(host.optimisticLockField in values)) { + const lockField = host.columns[host.optimisticLockField]!.name; + operation.$inc = { [lockField]: 1 }; + } + + if (Object.keys(operation).length === 0) return 0; + + const filter = buildMongoConditions(host, conditions); + if (host.softDeleteField) filter[host.softDeleteColumn!] = null; + + const result = await client.mongoUpdateMany(host.table, filter, operation); + return affectedRows(result, "matchedCount"); +} + +/** + * Removes every row a condition set matches, softly when the model soft-deletes. + * + * @param host The repository being written through. + * @param conditions Property key → value. + * @param client The client, which supplies the session and the collection. + * @returns How many rows matched. + */ +export async function mongoDeleteManyBy( + host: MongoRepositoryHost, + conditions: Record, + client: DBClient, +): Promise { + const filter = buildMongoConditions(host, conditions); + + if (host.softDeleteField) { + filter[host.softDeleteColumn!] = null; + const result = await client.mongoUpdateMany(host.table, filter, { + $set: { [host.softDeleteColumn!]: new Date() }, + }); + return affectedRows(result, "matchedCount"); + } + + const result = await client.mongoDeleteMany(host.table, filter); + return affectedRows(result, "deletedCount"); +} + +/** + * Clears the soft-delete stamp on every row a condition set matches. + * + * A null or undefined condition is skipped rather than matched, which is what + * the SQL path does: `restoreBy({tag: null})` restores everything whose tag is + * null *or* everything at all, depending on the backend, so the condition is + * dropped instead of being given a meaning the callers never agreed on. + * + * @param host The repository being written through. + * @param conditions Property key → value. + * @param client The client, which supplies the session and the collection. + * @returns How many rows matched. + */ +export async function mongoRestoreManyBy( + host: MongoRepositoryHost, + conditions: Record, + client: DBClient, +): Promise { + const filter = buildMongoConditions(host, conditions, { skipNull: true }); + filter[host.softDeleteColumn!] = { $ne: null }; + + const result = await client.mongoUpdateMany(host.table, filter, { + $unset: { [host.softDeleteColumn!]: "" }, + }); + return affectedRows(result, "matchedCount"); +} + +/** + * Runs the aggregates `aggregate()` was asked for, as one `$group`. + * + * The alias names match the SQL path's, because they are what the caller reads + * the answer out of — `count_all` for `count: "*"` and `${fn}_${column}` + * otherwise. + * + * @param host The repository being written through. + * @param options The requested aggregates. + * @param client The client, which supplies the session and the collection. + * @returns One row of results, or `{}` when nothing was asked for. + */ +export async function mongoAggregateRows( + host: MongoRepositoryHost, + options: { + count?: string | string[]; + sum?: string[]; + avg?: string[]; + min?: string[]; + max?: string[]; + }, + client: DBClient, +): Promise> { + const aggregates: MongoAggregate[] = []; + + for (const column of options.count + ? Array.isArray(options.count) + ? options.count + : [options.count] + : []) { + aggregates.push({ + fn: "count", + column: column === "*" ? "*" : (host.columns[column]?.name ?? column), + alias: column === "*" ? "count_all" : `count_${column}`, + }); + } + for (const [fn, columns] of [ + ["sum", options.sum], + ["avg", options.avg], + ["min", options.min], + ["max", options.max], + ] as const) { + for (const column of columns ?? []) { + aggregates.push({ + fn, + column: host.columns[column]?.name ?? column, + alias: `${fn}_${column}`, + }); + } + } + + if (aggregates.length === 0) return {}; + + const rows = await client.mongoAggregate( + host.table, + buildMongoAggregatePipeline(notDeletedFilter(host), aggregates, { + primaryKey: host.idColumn, + idProperty: host.idProperty, + }), + ); + if (rows[0]) return rows[0]; + + // A `$group` over an empty input produces *no documents at all*, where SQL's + // aggregate query returns one row of zeroes and NULLs. Without this the count + // of nothing comes back `undefined` rather than 0, and a caller doing + // arithmetic on it gets `NaN` — a wrong answer with no error, on the one + // input (an empty table) that a fresh install always has. + const empty: Record = {}; + for (const aggregate of aggregates) { + empty[aggregate.alias] = aggregate.fn === "count" ? 0 : null; + } + return empty; +} + +/** + * Counts the distinct non-null values of one column. + * + * SQL's `COUNT(DISTINCT col)` skips NULLs; Mongo's `distinct` returns them as a + * value like any other, so an explicit `null` would make the count one too high + * for every column that has one. + * + * @param host The repository being written through. + * @param column The column property name. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoCountDistinct( + host: MongoRepositoryHost, + column: string, + client: DBClient, +): Promise { + const field = host.columns[column]?.name ?? column; + const values = await client.mongoDistinct( + host.table, + field, + notDeletedFilter(host), + ); + return values.filter((value) => value !== null && value !== undefined).length; +} + +/** + * Picks a row at random, by its id. + * + * `$sample` chooses the document server-side, and then the id goes back through + * the ordinary `findOne`, so the answer has been through decryption, the row + * transform and relation loading exactly as any other read would. Returning the + * sampled document directly would skip all three. + * + * @param host The repository being written through. + * @param client The client, which supplies the session and the collection. + * @returns The row, or null when the collection is empty. + */ +export async function mongoRandom( + host: MongoRepositoryHost, + client: DBClient, +): Promise { + const sampled = await client.mongoAggregate(host.table, [ + { $match: notDeletedFilter(host) }, + { $sample: { size: 1 } }, + { $project: { _id: 1 } }, + ]); + if (!sampled.length) return null; + return host.findOne(sampled[0]._id, {}, client); +} + +/** + * Empties a collection. + * + * `deleteMany` rather than `drop`: dropping takes the indexes and the validator + * with it, so a truncated collection would silently stop enforcing the schema + * and stop being unique where the model says it is. SQL's `DELETE FROM` leaves + * both in place, which is the behaviour being matched. + * + * @param host The repository being written through. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoTruncate( + host: MongoRepositoryHost, + client: DBClient, +): Promise { + await client.mongoDeleteMany(host.table, {}); + await host.invalidateTableCache(); +} + +// ─── VERSIONING & HISTORY ───────────────────────────────────────────── + +/** + * The operations a history row records. Mirrors the union on `Repository`, and + * is spelled out here rather than imported from it: `repository.ts` already + * imports this file, and the storage layer's vocabulary is allowed to be its + * own. + */ +export type MongoVersionOperation = "insert" | "update" | "delete"; + +/** + * The compound `_id` a history document is keyed by. + * + * A row's history is one document per version, so the row's own key is not + * unique within the collection — the same problem the link collection has, and + * the same answer. `_id` uniqueness is the one constraint MongoDB enforces on + * every collection without an index being declared, so keying a version by the + * pair makes the version number the thing that cannot be stored twice. + * + * One function, so the shape the writer builds is the shape a reader matches. + */ +function historyId( + host: MongoRepositoryHost, + idValue: any, + version: number, +): Record { + return { [host.idColumn]: idValue, version }; +} + +/** + * Drops the compound `_id` off a history document. + * + * The identity is already on the row under its column name, which is where every + * reader of a history row looks for it. Handing the `_id` back as well would put + * a `{, version}` object on an entity whose `id` is a scalar. + */ +function historyRow(row: any): any { + if (!row || typeof row !== "object") return row ?? null; + const { _id, ...rest } = row; + return rest; +} + +/** + * Turns an entity into the history document a version is recorded as. + * + * The same treatment {@link buildMongoDocument} gives the live row — each column + * under its column name, with {@link OMIT} dropping the keys an entity does not + * carry — plus the audit fields the version is identified by. So a history row + * is the row it recorded, with the window it was valid for attached, and the + * relation loaders and row transforms that read column names work on it + * unchanged. + * + * The primary key is written twice on purpose: into the compound `_id` that + * makes the version unique, and under its own column name, because every read + * path here — {@link mongoAsOf}, {@link mongoHistory}, {@link mongoRollback} — + * addresses a history row by column name, and a dotted path into `_id` would be + * the one place that did not. + * + * `valid_from` and `modified_at` are native `Date`s rather than the ISO strings + * the SQL path binds: the "as of" window is a range query, and a string fails a + * `{bsonType: "date"}` validator *and* compares lexically against a field the + * index was built for as a date. + * + * `valid_to` is written only when there is one, and nothing writes one yet: it + * is what an explicit close of a version would set, and until something does + * that the newest row for a key is the open one. Writing a null says the same + * thing at the cost of a field, and this backend's reads already treat an absent + * field and a null one alike (`{valid_to: null}` matches both). + * + * @param host The repository the version belongs to. + * @param entity The row as it now stands, or as it stood for a delete. + * @param operation What the write did. + * @param version The version number this row records. + * @param user Who made the change, when the caller named someone. + */ +function buildHistoryDocument( + host: MongoRepositoryHost, + entity: Record, + operation: MongoVersionOperation, + version: number, + user?: string, +): Record { + const at = new Date(); + const idValue = entity?.[host.idProperty] ?? entity?.[host.idColumn]; + const document: Record = { + _id: historyId(host, idValue, version), + }; + + for (const [key, column] of Object.entries(host.columns)) { + // Rows arrive from both directions: a hydrated read is keyed by column name, + // and an entity a caller built is keyed by property. Both are read, so a + // renamed column is recorded rather than stored as its absence. + const sanitized = sanitizeMongoValue( + entity?.[key] ?? entity?.[column.name], + ); + if (sanitized === OMIT) continue; + document[column.name] = sanitized; + } + + // After the columns, so the history's own version is the one that lands when a + // model declares a `version` column of its own. The two hold the same number + // on every path through `writeHistory`, and the audit field is the one that + // has to be right. + document.operation = operation; + document.version = version; + document.valid_from = at; + document.modified_by = user || "system"; + document.modified_at = at; + + return document; +} + +/** + * Appends one version of a row to its history collection. + * + * The one place that decides what a history row is, so the four paths that + * record one — create, update, delete and rollback — cannot disagree about it. + * + * The version the history row is recorded under is not simply the version on the + * entity it was handed. A delete carries the row's *current* version, which the + * version it was created or last updated at already occupies — on SQL that is + * two rows sharing a number, which nothing forbids and which leaves the audit + * trail readable only by its `operation`. Here the version is half the compound + * `_id`, so the same number twice is a duplicate-key error that fails the write + * it was recording. A delete is a write like any other, and the number it is + * recorded under is the next one. + * + * Counting past the newest *recorded* version rather than off the entity is what + * keeps the three paths that already passed a fresh number on it: create sends + * 1 against an empty history, update (and upsert) send the version the lock just + * advanced to, and rollback sends one past the newest. Only a write that reuses + * a number — the delete — moves. + * + * The read is taken inside whatever transaction the caller is in, so the number + * is decided against the same snapshot as the write it accompanies, and a + * retried transaction re-decides it rather than reusing a stale one. + * + * @param host The repository being written through. + * @param entity The row as it now stands, or as it stood for a delete. + * @param operation What the write did. + * @param client The client, which supplies the session and the collection. + * @param user Who made the change, when the caller named someone. + */ +export async function mongoWriteHistory( + host: MongoRepositoryHost, + entity: Record, + operation: MongoVersionOperation, + client: DBClient, + user?: string, +): Promise { + // `|| 1`, not `?? 1`: the SQL path counts the same way, and a version of zero + // or an empty string is a row nobody recorded a version for. + const sent = Number(entity?.version) || 1; + const idValue = entity?.[host.idProperty] ?? entity?.[host.idColumn]; + const newest = await client.mongoFindOne( + host.historyTable, + { [host.idColumn]: idValue }, + { sort: { version: -1 }, projection: { version: 1 } }, + ); + const version = Math.max(sent, (Number(newest?.version) || 0) + 1); + + await client.mongoInsertOne( + host.historyTable, + buildHistoryDocument(host, entity, operation, version, user), + ); +} + +/** + * Reads the version of a row that was current at an instant. + * + * `{valid_to: null}` is the open window — a version whose end was never written, + * or was written as an explicit null. Both are the same absence, and the pair + * covers what the SQL path's `valid_to IS NULL OR valid_to > ?` covers. + * + * The sort is what stops a version that was never closed from coming back as an + * arbitrary one: every version that started before the instant still matches the + * open-window half of the filter, so without it the answer would be whichever + * document the collection happened to hand over first. + * + * @param host The repository being read through. + * @param id The row whose version is wanted. + * @param asOfDate The instant to read the row as of. + * @param client The client, which supplies the session and the collection. + * @returns The version, or null when the row has no version covering the instant. + */ +export async function mongoAsOf( + host: MongoRepositoryHost, + id: number | string, + asOfDate: Date | string, + client: DBClient, +): Promise { + // Coerced rather than bound as given. A caller's ISO string compared against a + // stored date is a comparison between BSON *types*, and MongoDB orders those by + // kind before value — so every date would look older than every string and the + // window would swallow versions that started after the instant. + const at = asOfDate instanceof Date ? asOfDate : new Date(asOfDate); + + const row = await client.mongoFindOne( + host.historyTable, + { + [host.idColumn]: id, + valid_from: { $lte: at }, + $or: [{ valid_to: null }, { valid_to: { $gt: at } }], + }, + { sort: { version: -1 } }, + ); + + return historyRow(row); +} + +/** + * Reads every version of a row, oldest first. + * + * Sorted rather than left to the server, because this is a *history*: a caller + * reading it is reasoning about what happened in what order, and MongoDB gives + * no order to a read that does not ask for one. + * + * @param host The repository being read through. + * @param id The row whose history is wanted. + * @param client The client, which supplies the session and the collection. + */ +export async function mongoHistory( + host: MongoRepositoryHost, + id: number | string, + client: DBClient, +): Promise { + const rows = await client.mongoFind( + host.historyTable, + { [host.idColumn]: id }, + { sort: { version: 1 } }, + ); + return rows.map(historyRow); +} + +/** + * Restores a row to a version it used to hold, as a new version. + * + * The whole operation is one transaction, because it is three writes that only + * mean anything together: the live document, the history row that records the + * restore, and the read that returns what the caller just wrote. + * + * The new version is the one after the *newest* recorded version, not the one + * after the version being restored. `version + 1` — what the SQL path writes — + * names a version number that is usually already recorded when a caller rolls + * back to anything but the latest version, and here the compound `_id` makes + * that a duplicate-key error rather than a second row. Counting past the newest + * both advances the version and leaves the versions in between in the audit + * trail, which is the point of having one. + * + * The optimistic lock is the one column that is advanced rather than restored. + * Everywhere else in this design a live row's version and the newest history + * row's version are the same number — create, update and upsert all leave them + * equal — and restoring the old one would hand a caller that had already written + * it a version it could match on twice. + * + * @param host The repository being written through. + * @param id The row to restore. + * @param version The version to restore it to. + * @param client The client, which supplies the session and the collection. + * @returns The reloaded row, as the ordinary read path would return it. + * @throws StabilizeError `ROLLBACK_ERROR` when the version or the row is gone. + */ +export async function mongoRollback( + host: MongoRepositoryHost, + id: number | string, + version: number, + client: DBClient, +): Promise { + const start = Date.now(); + host.logger.logDebug(`Rolling ${host.table} with ID ${id} back to version ${version}`); + + return client.transaction(async (tx) => { + const target = await tx.mongoFindOne(host.historyTable, { + [host.idColumn]: id, + version, + }); + if (!target) { + throw new StabilizeError("Version not found", "ROLLBACK_ERROR"); + } + + // Read inside the same transaction as the write it decides, so two rollbacks + // racing cannot both pick the same new version number. + const newest = await tx.mongoFindOne( + host.historyTable, + { [host.idColumn]: id }, + { sort: { version: -1 }, projection: { version: 1 } }, + ); + const next = + Math.max(Number(target.version) || 1, Number(newest?.version) || 0) + 1; + + // Every column the version recorded goes back onto the live document, keyed + // by its column name. A column the version does not carry is cleared rather + // than left alone: the row is being restored to a state, and a column that + // state does not have is part of it. + const restored: Record = {}; + for (const [key, column] of Object.entries(host.columns)) { + // The identity is never restored: a change of key is a delete and an + // insert, which is why no other write path in this file touches `_id`. + if (key === host.idProperty) continue; + if (key === host.optimisticLockField) continue; + const sanitized = sanitizeMongoValue( + target[key] ?? target[column.name] ?? null, + ); + restored[key] = sanitized === OMIT ? null : sanitized; + } + if (host.optimisticLockField) { + restored[host.optimisticLockField] = next; + } + + const { $set, $unset } = buildMongoPatch(host, restored); + const operation = updateOperation($set, $unset); + if (Object.keys(operation).length > 0) { + const result = await tx.mongoUpdateOne(host.table, { _id: id }, operation); + if (affectedRows(result, "matchedCount") === 0) { + throw new StabilizeError("Not found", "ROLLBACK_ERROR"); + } + } + + // The version this restore *creates* is the row as it was restored, recorded + // like any other write. It is written after the live document, so a failure + // between the two rolls both back rather than leaving a history row for a + // state the row was never put into. + await mongoWriteHistory( + host, + { ...target, version: next }, + "update", + tx, + ); + + const reloaded = await host.findOne(id, {}, tx); + await host.invalidateRowCache(id); + await host.writeThroughRow(id, reloaded); + + host.logger.logDebug( + `Rolled ${host.table} with ID ${id} back to version ${version} in ${Date.now() - start}ms`, + ); + return reloaded; + }); +} + +// ─── MANY-TO-MANY LINKS ─────────────────────────────────────────────── + +/** + * Reads the links from `parentIds` outward, one chunk at a time. + * + * The projection is limited to the two key fields: the composite `_id` is a + * duplicate of them, and a link collection has nothing else on it, so reading + * whole documents would only move the same data twice. + * + * Sorted on `_id`, which is `{p, c}` — so a parent's children come back in + * ascending child order, and the same read twice gives the same answer. A SQL + * join table read with no `ORDER BY` has no order to match, and Mongo's is not + * merely unspecified but liable to differ between two reads of unchanged data. + * + * @param client The client, which also supplies the transaction's session. + * @param link The join collection and the two fields that name a link. + * @param parentIds One chunk of parent ids, already deduplicated. + */ +export async function mongoFindLinks( + client: DBClient, + link: MongoLinkRelation, + parentIds: (number | string)[], +): Promise<{ parent: any; child: any }[]> { + if (parentIds.length === 0) return []; + const { joinTable, foreignKey, inverseKey } = link; + const docs = await client.mongoFind( + joinTable, + { [foreignKey]: { $in: parentIds } }, + { + projection: { [foreignKey]: 1, [inverseKey]: 1 }, + sort: { _id: 1 }, + }, + ); + return docs.map((doc) => ({ + parent: doc[foreignKey], + child: doc[inverseKey], + })); +} + +/** + * The raw values linked to `id` through a many-to-many relation. + * + * Sorted for the same reason as {@link mongoFindLinks}: `attach` and `sync` + * diff this list against what the caller asked for, and a list whose order + * changes between calls would make those diffs look different when they are not. + * + * @param client The client, which also supplies the transaction's session. + * @param link The join collection and the two fields that name a link. + * @param id The parent whose links are wanted. + */ +export async function mongoFetchLinkedIds( + client: DBClient, + link: MongoLinkRelation, + id: number | string, +): Promise { + const { joinTable, foreignKey, inverseKey } = link; + const docs = await client.mongoFind( + joinTable, + { [foreignKey]: id }, + { projection: { [inverseKey]: 1 }, sort: { _id: 1 } }, + ); + return docs.map((doc) => doc[inverseKey]); +} + +/** + * Creates the links from `id` to each of `childIds`. + * + * Written as one upsert per pair rather than as a batch of inserts. The caller + * has already read the existing links and filtered them out, so an insert would + * normally succeed — but two callers can pass that check at the same time, and + * the losing insert would fail the whole statement on the duplicate `_id`. An + * upsert that matches an existing link simply changes nothing. + * + * @param client The client, which also supplies the transaction's session. + * @param link The join collection and the two fields that name a link. + * @param id The parent to link from. + * @param childIds The children to link to, already deduplicated. + */ +export async function mongoAttachLinks( + client: DBClient, + link: MongoLinkRelation, + id: number | string, + childIds: (number | string)[], +): Promise { + if (childIds.length === 0) return; + const { joinTable, foreignKey, inverseKey } = link; + await client.mongoBulkWrite( + joinTable, + childIds.map((childId) => ({ + updateOne: { + filter: { _id: linkId(id, childId) }, + update: { + $setOnInsert: { [foreignKey]: id, [inverseKey]: childId }, + }, + upsert: true, + }, + })), + ); +} + +/** + * Removes the links from `id`, or just those naming one of `childIds`. + * + * @param client The client, which also supplies the transaction's session. + * @param link The join collection and the two fields that name a link. + * @param id The parent to unlink from. + * @param childIds The children to unlink, or `undefined` for all of them. + * @returns how many links were removed. + */ +export async function mongoDetachLinks( + client: DBClient, + link: MongoLinkRelation, + id: number | string, + childIds?: (number | string)[], +): Promise { + const { joinTable, foreignKey, inverseKey } = link; + const filter: Record = { [foreignKey]: id }; + if (childIds !== undefined) { + if (childIds.length === 0) return 0; + filter[inverseKey] = { $in: childIds }; + } + const result = await client.mongoDeleteMany(joinTable, filter); + return affectedRows(result, "deletedCount"); +} diff --git a/mongo-schema.ts b/mongo-schema.ts index 6d0a479..95ed7f3 100644 --- a/mongo-schema.ts +++ b/mongo-schema.ts @@ -172,6 +172,110 @@ export interface MongoCollectionPlan { }[]; } +/** + * Builds the `$jsonSchema` validator for a versioned model's history collection. + * + * The model's columns are described exactly as they are in the main collection — + * a history row *is* the row it recorded — with the audit columns the history + * writer adds appended. + * + * `valid_from` is both declared as a date and required, which is what gives the + * native-`Date` rule teeth: an ISO string satisfies no `bsonType: "date"` and is + * rejected by the server on insert, rather than quietly turning every "as of" + * range query into a comparison between strings. + * + * The model's own `required` columns are deliberately **not** required here. + * `validationLevel: "moderate"` is what keeps a row that predates a newly + * declared column updatable, and demanding that column of the history row too + * would then reject the *recording* of exactly the update the level exists to + * allow. + * + * @param columns The model's columns. + */ +export function buildHistoryValidator( + columns: Record, +): Record { + const properties: Record = {}; + + for (const [key, column] of Object.entries(columns)) { + // The primary key is a field here, unlike in the main collection: the + // document's `_id` is the pair that makes one version unique, so the row's + // own key has nowhere else to live. + properties[column.name ?? key] = mapColumnToBsonSchema(column); + } + + properties.operation = { bsonType: "string" }; + properties.version = { bsonType: ["int", "long", "double"] }; + properties.valid_from = { bsonType: "date" }; + properties.valid_to = { bsonType: "date" }; + properties.modified_by = { bsonType: "string" }; + properties.modified_at = { bsonType: "date" }; + + return { + $jsonSchema: { + bsonType: "object", + required: ["_id", "version", "valid_from"], + properties, + }, + }; +} + +/** + * Derives the history collection a versioned model needs. + * + * The history lives beside the model rather than in it — one collection keyed by + * the row and the version — and has no model of its own, so `autoMigrate` is the + * only thing that can create it. A model that is not versioned derives nothing. + * + * Both indexes serve a read that exists: `{row, version}` is `history()` and the + * newest-version lookup `rollback` makes, and `{row, valid_from}` is the window + * `asOf` ranges over. Without the second one the window comparison still works + * and stops being a range scan of one row's versions. + * + * @param model The model to derive from. + * @returns The history collection plan, or null when the model is not versioned. + */ +export function planMongoHistoryCollection( + model: any, +): MongoCollectionPlan | null { + const meta = MetadataStorage.getModelMetadata(model); + if (!meta?.tableName || !meta.versioned) return null; + + const columns = meta.columns as Record; + const idColumn = columns["id"]?.name ?? "id"; + + return { + collection: `${meta.tableName}_history`, + validator: buildHistoryValidator(columns), + indexes: [ + { spec: { [idColumn]: 1, version: 1 }, options: {} }, + { spec: { [idColumn]: 1, valid_from: 1 }, options: {} }, + ], + }; +} + +/** + * Lists the history collections a model set needs. + * + * @param models The models being migrated. + */ +export function planMongoHistoryCollections( + models: any[], +): MongoCollectionPlan[] { + const plans: MongoCollectionPlan[] = []; + const seen = new Set(); + + for (const model of models) { + const plan = planMongoHistoryCollection(model); + if (plan && !seen.has(plan.collection)) { + seen.add(plan.collection); + plans.push(plan); + } + } + + return plans; +} + /** * Derives everything a model needs in the database. * @@ -397,6 +501,7 @@ export async function mongoAutoMigrate( plans.push(plan); } plans.push(...planMongoLinkCollections(models)); + plans.push(...planMongoHistoryCollections(models)); for (const plan of plans) { await ensureCollection(db, plan); @@ -435,23 +540,38 @@ export function generateMongoSteps( const plan = planMongoCollection(model); if (!plan) return []; + // A versioned model's history collection is part of what a migration brings + // into existence, so it travels with the collection it belongs to in both + // directions. Left out of the `down`, a dropped model would keep its audit + // trail. + const history = planMongoHistoryCollection(model); + const collections = history ? [plan, history] : [plan]; + if (direction === "down") { // Dropping the collection is the only inverse that is actually total: the // indexes and the validator go with it, so a re-`up` rebuilds from the // declaration rather than from whatever survived. - return [{ kind: "dropCollection", collection: plan.collection }]; + return collections.map((each) => ({ + kind: "dropCollection" as const, + collection: each.collection, + })); } - const steps: MongoStep[] = [ - { kind: "createCollection", collection: plan.collection, validator: plan.validator }, - ]; - for (const index of plan.indexes) { + const steps: MongoStep[] = []; + for (const each of collections) { steps.push({ - kind: "createIndex", - collection: plan.collection, - spec: index.spec, - options: index.options, + kind: "createCollection", + collection: each.collection, + validator: each.validator, }); + for (const index of each.indexes) { + steps.push({ + kind: "createIndex", + collection: each.collection, + spec: index.spec, + options: index.options, + }); + } } return steps; } diff --git a/query-builder.ts b/query-builder.ts index 96f7783..4a65cc3 100644 --- a/query-builder.ts +++ b/query-builder.ts @@ -993,8 +993,20 @@ export class QueryBuilder { * known — that is what lets one builder render for either backend. * * A `lock()`/`forUpdate()` is **not** reported. MongoDB has no row locking to - * map it onto and the call is a no-op, matching what the SQL Server path - * already does with it; the repository logs that where it has a logger. + * map it onto, so the clause is simply not rendered and the statement runs + * unlocked — the same walk-past the SQL Server path already takes, where a + * `FOR UPDATE` would be invalid T-SQL. Unlike a blocked clause this changes + * what the query guarantees without changing its result, so it is documented + * in the README rather than thrown: an error here would refuse queries that + * have a perfectly good answer. + * + * The builder cannot warn about the dropped lock itself — it holds no logger, + * and it is dialect-agnostic right up until `execute()`, by which point the + * caller has stopped listening. `Repository.lockForUpdate` warns on its own + * behalf instead, since that is the path that adds the clause for a caller + * who asked for a lock by name. A `qb.lock()` written by hand and handed + * straight to `execute()` is still walked past in silence; that residual is + * why the README says so in prose as well. * * @throws StabilizeError `MONGO_UNSUPPORTED` naming every clause that has no * MongoDB equivalent. diff --git a/repository.ts b/repository.ts index 050a60e..67a9d09 100644 --- a/repository.ts +++ b/repository.ts @@ -19,8 +19,29 @@ import { MetadataStorage } from "./model"; import { getHooks, type HookType } from "./hooks"; import { decrypt, encrypt } from "./utils/encryption"; import { + mongoAggregateRows, + mongoAsOf, + mongoAttachLinks, mongoBulkCreate, + mongoCountDistinct, mongoCreate, + mongoDeleteManyBy, + mongoDeleteRow, + mongoDetachLinks, + mongoFetchLinkedIds, + mongoFindLinks, + mongoHistory, + mongoIncrement, + mongoRandom, + mongoRecover, + mongoRestoreManyBy, + mongoRollback, + mongoToggle, + mongoTruncate, + mongoUpdate, + mongoUpdateMany, + mongoUpsertRow, + mongoWriteHistory, type MongoRepositoryHost, } from "./mongo-repository"; @@ -607,6 +628,9 @@ export class Repository { if (!this.versioned) throw new StabilizeError("Model is not versioned", "VERSIONING_ERROR"); const client = _client || this.client; + if (this.getDBType(client) === DBType.MongoDB) { + return mongoAsOf(this.mongoCtx, id, asOfDate, client) as Promise; + } // Bind an ISO string rather than the `Date` itself: SQLite rejects a Date // as a parameter, so every `asOf` call used to throw there, and the // history timestamps are written as ISO strings anyway. @@ -622,6 +646,9 @@ export class Repository { if (!this.versioned) throw new StabilizeError("Model is not versioned", "VERSIONING_ERROR"); const client = _client || this.client; + if (this.getDBType(client) === DBType.MongoDB) { + return mongoHistory(this.mongoCtx, id, client) as Promise; + } return client.query( `SELECT * FROM ${this.historyTable} WHERE id = ? ORDER BY version ASC`, [id], @@ -636,6 +663,9 @@ export class Repository { if (!this.versioned) throw new StabilizeError("Model is not versioned", "VERSIONING_ERROR"); const client = _client || this.client; + if (this.getDBType(client) === DBType.MongoDB) { + return mongoRollback(this.mongoCtx, id, version, client) as Promise; + } return client.transaction(async (txClient) => { const rows = await txClient.query( `SELECT * FROM ${this.historyTable} WHERE id = ? AND version = ?${this.topOneClause(txClient)}`, @@ -672,6 +702,16 @@ export class Repository { ) { if (!this.versioned) return; + if (this.getDBType(client) === DBType.MongoDB) { + return mongoWriteHistory( + this.mongoCtx, + entity as Record, + operation, + client, + user, + ); + } + const propertyKeys = Object.keys(this.columns); const sqlColumnNames = propertyKeys.map((k) => this.columns[k]!.name); @@ -785,8 +825,14 @@ export class Repository { idProperty: ID_PROPERTY, idColumn: this.columns[ID_PROPERTY]?.name ?? ID_PROPERTY, autoIncrementField: this.autoIncrementField, + softDeleteField: this.softDeleteField, + softDeleteColumn: this.softDeleteColumn, + optimisticLockField: this.optimisticLockField, + optimisticLockColumn: this.optimisticLockColumn, + timestamps: this.timestampsConfig, + historyTable: this.historyTable, logger: this.logger, - validate: (entity) => this.validate(entity), + validate: (entity, skipRequired) => this.validate(entity, skipRequired), seedCreateDefaults: (entity) => this.seedCreateDefaults(entity as Partial), processForSave: (entity) => this.processForSave(entity), processForLoad: (row) => this.processForLoad(row), @@ -1150,6 +1196,10 @@ export class Repository { before: T, client: DBClient, ): Promise { + if (this.getDBType(client) === DBType.MongoDB) { + return mongoUpdate(this.mongoCtx, id, entity as Record, before, client) as Promise; + } + const start = performance.now(); this.logger.logDebug(`Updating ${this.table} with ID ${id}`); this.validate(entity, true); @@ -1251,6 +1301,16 @@ export class Repository { options: { batchSize?: number } = {}, _client?: DBClient, ): Promise { + // Each entry carries a raw SQL `WHERE` fragment and its parameters, and + // there is no MongoDB statement to translate them into — a filter is a + // document, not text. Refused before the transaction opens, since a session + // that can only fail is not worth starting. + if (this.getDBType(_client) === DBType.MongoDB) { + throw new StabilizeError( + `bulkUpdate() takes a raw SQL WHERE condition, which has no MongoDB equivalent. Use updateBy() with a condition object instead.`, + "MONGO_UNSUPPORTED", + ); + } return (_client || this.client).transaction((txClient) => this._bulkUpdate(updates, options, txClient), ); @@ -1393,63 +1453,81 @@ export class Repository { .filter((c) => !keys.includes(c)) .map((k) => writeValues[k]); - let query: string; - let params = [...insertParams, ...updateParams]; + let result: T | null; - if (dbType === DBType.SQLite) { - const updateClause = columns - .filter((c) => !keys.includes(c)) - .map((c) => `${this.columns[c]?.name} = ?`) - .join(", "); - query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT(${conflictColumns.join(", ")}) DO UPDATE SET ${updateClause}`; - } else if (dbType === DBType.MySQL) { - const updateClause = columns - .filter((c) => !keys.includes(c)) - .map((c) => `${this.columns[c]?.name} = ?`) - .join(", "); - query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON DUPLICATE KEY UPDATE ${updateClause}`; - } else if (dbType === DBType.MSSQL) { - // `MERGE` refers to each value by a source column name, so the parameters - // are the insert payload alone — the update payload is not bound a second - // time the way the `?`-based dialects above bind it. - query = buildMSSQLUpsertSQL( - this.table, - columns.map((c) => this.columns[c]!.name), - conflictColumns, - ); - params = insertParams; + if (dbType === DBType.MongoDB) { + // One `findOneAndUpdate` with `upsert`, rather than the read-then-write + // the SQL branches compile to: two callers racing on a key that does not + // exist yet would otherwise both insert. The tail below — the hook pair, + // the history entry and the cache writes — is shared, so it still runs + // against whatever row the write landed on. + result = (await mongoUpsertRow( + this.mongoCtx, + keys, + writeValues, + before, + client, + )) as T | null; } else { - const pgUpdateClause = columns - .filter((c) => !keys.includes(c)) - .map( - (c) => `${this.columns[c]?.name} = EXCLUDED.${this.columns[c]?.name}`, - ) - .join(", "); - query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT (${conflictColumns.join(", ")}) DO UPDATE SET ${pgUpdateClause} RETURNING *`; - params = insertParams; - } + let query: string; + let params = [...insertParams, ...updateParams]; - const results = (await client.query(query, params)).map((row) => - this.processForLoad(row), - ); + if (dbType === DBType.SQLite) { + const updateClause = columns + .filter((c) => !keys.includes(c)) + .map((c) => `${this.columns[c]?.name} = ?`) + .join(", "); + query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT(${conflictColumns.join(", ")}) DO UPDATE SET ${updateClause}`; + } else if (dbType === DBType.MySQL) { + const updateClause = columns + .filter((c) => !keys.includes(c)) + .map((c) => `${this.columns[c]?.name} = ?`) + .join(", "); + query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON DUPLICATE KEY UPDATE ${updateClause}`; + } else if (dbType === DBType.MSSQL) { + // `MERGE` refers to each value by a source column name, so the + // parameters are the insert payload alone — the update payload is not + // bound a second time the way the `?`-based dialects above bind it. + query = buildMSSQLUpsertSQL( + this.table, + columns.map((c) => this.columns[c]!.name), + conflictColumns, + ); + params = insertParams; + } else { + const pgUpdateClause = columns + .filter((c) => !keys.includes(c)) + .map( + (c) => + `${this.columns[c]?.name} = EXCLUDED.${this.columns[c]?.name}`, + ) + .join(", "); + query = `INSERT INTO ${this.table} (${columnNames}) VALUES (${placeholders}) ON CONFLICT (${conflictColumns.join(", ")}) DO UPDATE SET ${pgUpdateClause} RETURNING *`; + params = insertParams; + } - // SQLite and MySQL report nothing for an `INSERT ... ON CONFLICT`, so the - // row has to be read back. It must be read back by the conflict keys: - // on the DO UPDATE path no insert happened, so `last_insert_rowid()` still - // holds an unrelated earlier statement's value and pointed at a row this - // upsert never touched. - let result: T | null = results[0] ?? null; - if (!result) result = await this.findRowByKeys(keys, writeValues, client); - if (!result && keys.length === 0) { - const id = await this.resolveInsertedIds( - [writeValues as Partial], - client, - dbType, - this.primaryKeyField, - this.primaryKeyColumn, + const results = (await client.query(query, params)).map((row) => + this.processForLoad(row), ); - if (id.length > 0) { - result = await this.findOne(id[0], {}, client); + + // SQLite and MySQL report nothing for an `INSERT ... ON CONFLICT`, so the + // row has to be read back. It must be read back by the conflict keys: + // on the DO UPDATE path no insert happened, so `last_insert_rowid()` + // still holds an unrelated earlier statement's value and pointed at a row + // this upsert never touched. + result = results[0] ?? null; + if (!result) result = await this.findRowByKeys(keys, writeValues, client); + if (!result && keys.length === 0) { + const id = await this.resolveInsertedIds( + [writeValues as Partial], + client, + dbType, + this.primaryKeyField, + this.primaryKeyColumn, + ); + if (id.length > 0) { + result = await this.findOne(id[0], {}, client); + } } } @@ -1526,10 +1604,25 @@ export class Repository { }); } + /** + * Removes one row, which is what both `delete()` and `bulkDelete()` need. + * + * Shared rather than repeated so the two cannot disagree on what a delete + * means for a soft-deleting model — the bulk path used to carry its own copy + * of this statement. + */ private async _delete(id: number | string, client: DBClient): Promise { const start = performance.now(); this.logger.logDebug(`Deleting ${this.table} with ID ${id}`); + if (this.getDBType(client) === DBType.MongoDB) { + await mongoDeleteRow(this.mongoCtx, id, client); + this.logger.logDebug( + `Deleted ${this.table} with ID ${id} in ${(performance.now() - start).toFixed(2)}ms`, + ); + return; + } + const query = this.softDeleteField ? `UPDATE ${this.table} SET ${this.softDeleteColumn} = ? WHERE id = ?` : `DELETE FROM ${this.table} WHERE id = ?`; @@ -1573,14 +1666,7 @@ export class Repository { await this.runHooks(before, "beforeDelete"); - const query = this.softDeleteField - ? `UPDATE ${this.table} SET ${this.softDeleteColumn} = ? WHERE id = ?` - : `DELETE FROM ${this.table} WHERE id = ?`; - const params = this.softDeleteField - ? [sanitizeSqlValue(new Date(), this.getDBType(client)), id] - : [id]; - - await client.query(query, params); + await this._delete(id, client); await this.runHooks(before, "afterDelete"); @@ -1613,10 +1699,14 @@ export class Repository { ); } - await client.query( - `UPDATE ${this.table} SET ${this.softDeleteColumn} = NULL WHERE id = ?`, - [id], - ); + if (this.getDBType(client) === DBType.MongoDB) { + await mongoRecover(this.mongoCtx, id, client); + } else { + await client.query( + `UPDATE ${this.table} SET ${this.softDeleteColumn} = NULL WHERE id = ?`, + [id], + ); + } const result = await this.findOne(id, {}, client); if (!result) @@ -1871,8 +1961,17 @@ export class Repository { // The join table has no model of its own, so its key columns are named by // the relation config rather than resolved through column metadata. + const mongo = this.getDBType(client) === DBType.MongoDB; + const linkRelation = { joinTable, foreignKey, inverseKey }; const links: { parent: any; child: any }[] = []; for (const chunk of chunked(parentIds, RELATION_BATCH_SIZE)) { + if (mongo) { + // One `$in` over the link collection per chunk, the same shape the SQL + // path sends — which is why relations need no rewrite for a document + // store: they were already batched reads rather than joins. + links.push(...(await mongoFindLinks(client, linkRelation, chunk))); + continue; + } const found = await client.query>( `SELECT ${foreignKey} AS __parent, ${inverseKey} AS __child FROM ${joinTable} WHERE ${foreignKey} IN (${chunk.map(() => "?").join(", ")})`, chunk, @@ -1911,16 +2010,24 @@ export class Repository { ): Promise<{ data: T[]; total: number; page: number; pageSize: number }> { const qb = this.find(); const data = await qb.paginate(page, pageSize).execute(this.client); - let countQuery = `SELECT COUNT(*) as count FROM ${this.table}`; - let countParams: any[] = []; - if (this.softDeleteField) { - countQuery += ` WHERE ${this.table}.${this.softDeleteColumn} IS NULL`; + let count: number; + if (this.getDBType() === DBType.MongoDB) { + // The count is the same predicate the page itself was drawn with, so it + // goes through `count()` rather than a second, hand-written filter — the + // two would otherwise be free to disagree about soft-deleted rows. + count = await this.count(); + } else { + let countQuery = `SELECT COUNT(*) as count FROM ${this.table}`; + const countParams: any[] = []; + if (this.softDeleteField) { + countQuery += ` WHERE ${this.table}.${this.softDeleteColumn} IS NULL`; + } + const result = await this.client.query<{ count: number }>( + countQuery, + countParams, + ); + count = result?.[0]?.count ?? 0; } - const result = await this.client.query<{ count: number }>( - countQuery, - countParams, - ); - const count = result?.[0]?.count ?? 0; return { data, total: Number(count), page, pageSize }; } @@ -2100,6 +2207,10 @@ export class Repository { min?: string[]; max?: string[]; }): Promise> { + if (this.getDBType() === DBType.MongoDB) { + return mongoAggregateRows(this.mongoCtx, options, this.client); + } + const selectParts: string[] = []; if (options.count) { @@ -2262,7 +2373,12 @@ export class Repository { // ─── FEATURE 10: truncate (Rails-style) ─────────────────────────── async truncate(_client?: DBClient): Promise { - await (_client || this.client).queryExec(`DELETE FROM ${this.table}`); + const client = _client || this.client; + if (this.getDBType(client) === DBType.MongoDB) { + await mongoTruncate(this.mongoCtx, client); + return; + } + await client.queryExec(`DELETE FROM ${this.table}`); await this.invalidateTableCache(); } @@ -2312,6 +2428,10 @@ export class Repository { // ─── FEATURE 13: distinct count ─────────────────────────────────── async countDistinct(column: string): Promise { + if (this.getDBType() === DBType.MongoDB) { + return mongoCountDistinct(this.mongoCtx, column, this.client); + } + const colName = this.columns[column]?.name || column; const qb = new QueryBuilder(this.table); if (this.softDeleteField) { @@ -2332,6 +2452,12 @@ export class Repository { ): Promise { const client = _client || this.client; const colName = this.columns[field]?.name || field; + if (this.getDBType(client) === DBType.MongoDB) { + await mongoIncrement(this.mongoCtx, id, field, amount, client); + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; + } + let query = `UPDATE ${this.table} SET ${colName} = ${colName} + ? WHERE id = ?`; if (this.softDeleteField) query += ` AND ${this.softDeleteColumn} IS NULL`; await client.queryExec(query, [amount, id]); @@ -2349,6 +2475,12 @@ export class Repository { ): Promise { const client = _client || this.client; const colName = this.columns[field]?.name || field; + if (this.getDBType(client) === DBType.MongoDB) { + await mongoIncrement(this.mongoCtx, id, field, -amount, client); + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; + } + let query = `UPDATE ${this.table} SET ${colName} = ${colName} - ? WHERE id = ?`; if (this.softDeleteField) query += ` AND ${this.softDeleteColumn} IS NULL`; await client.queryExec(query, [amount, id]); @@ -2396,6 +2528,11 @@ export class Repository { const client = _client || this.client; const colName = this.columns[field]?.name || field; const dbType = this.getDBType(); + if (dbType === DBType.MongoDB) { + await mongoToggle(this.mongoCtx, id, field, client); + await this.invalidateRowCache(id); + return (await this.findOne(id, {}, client)) as T; + } const expr = dbType === DBType.Postgres ? `NOT ${colName}` @@ -2450,6 +2587,17 @@ export class Repository { // bulk update wrote plaintext straight into an encrypted column. const writeValues = this.processForSave(payload); + if (this.getDBType(client) === DBType.MongoDB) { + const count = await mongoUpdateMany( + this.mongoCtx, + conditions as Record, + writeValues, + client, + ); + await this.invalidateTableCache(); + return count; + } + const setParts: string[] = []; const setParams: any[] = []; for (const key of Object.keys(writeValues)) { @@ -2501,6 +2649,15 @@ export class Repository { async deleteBy(conditions: Partial, _client?: DBClient): Promise { const client = _client || this.client; this.assertConditions(conditions, "deleteBy"); + if (this.getDBType(client) === DBType.MongoDB) { + const count = await mongoDeleteManyBy( + this.mongoCtx, + conditions as Record, + client, + ); + await this.invalidateTableCache(); + return count; + } const whereParts: string[] = []; const whereParams: any[] = []; for (const [key, value] of Object.entries(conditions)) { @@ -2542,6 +2699,15 @@ export class Repository { throw new StabilizeError("Soft delete not enabled", "RECOVER_ERROR"); } const client = _client || this.client; + if (this.getDBType(client) === DBType.MongoDB) { + const count = await mongoRestoreManyBy( + this.mongoCtx, + conditions as Record, + client, + ); + await this.invalidateTableCache(); + return count; + } const whereParts: string[] = [`${this.softDeleteColumn} IS NOT NULL`]; const whereParams: any[] = []; for (const [key, value] of Object.entries(conditions)) { @@ -2654,7 +2820,18 @@ export class Repository { const client = _client || this.client; const dbType = this.getDBType(client); const qb = this.find().whereEq("id", id).limit(1); - if (dbType !== DBType.SQLite && dbType !== DBType.MSSQL) { + if (dbType === DBType.MongoDB) { + // MongoDB has no `SELECT … FOR UPDATE`, and nothing stands in for it: a + // document read cannot be held against a concurrent writer. The + // atomicity a read-modify-write actually wants lives in an atomic update + // or in a transaction, so the caller has to be told the lock is absent — + // the method's name promises one, and its *result* cannot show the + // difference, which is a lost update with nothing to notice. + this.logger.logWarn( + `lockForUpdate on ${this.table} takes no lock on MongoDB: ` + + `use a transaction or an atomic update for read-modify-write`, + ); + } else if (dbType !== DBType.SQLite && dbType !== DBType.MSSQL) { // T-SQL has no `FOR UPDATE`; its equivalent is a table hint // (`WITH (UPDLOCK)`), which this builder cannot express. Skipping the // clause keeps the statement valid on SQL Server, at the cost of the @@ -2724,6 +2901,12 @@ export class Repository { async random(): Promise { const dbType = this.getDBType(); + if (dbType === DBType.MongoDB) { + // `ORDER BY RANDOM() LIMIT 1` has no MongoDB spelling; the server-side + // `$sample` stage is the equivalent, and it does not scale with the + // collection the way sorting every document would. + return mongoRandom(this.mongoCtx, this.client) as Promise; + } let orderByExpr: string; if (dbType === DBType.MySQL) { orderByExpr = "RAND()"; @@ -2858,6 +3041,13 @@ export class Repository { relation, "fetchLinkedIds", ); + if (this.getDBType(client) === DBType.MongoDB) { + return mongoFetchLinkedIds( + client, + { joinTable, foreignKey, inverseKey }, + id, + ); + } const rows = await client.query>( `SELECT ${inverseKey} FROM ${joinTable} WHERE ${foreignKey} = ?`, [id], @@ -2903,6 +3093,17 @@ export class Repository { const missing = wanted.filter((value) => !linked.has(String(value))); if (missing.length === 0) return 0; + if (this.getDBType(client) === DBType.MongoDB) { + await mongoAttachLinks( + client, + { joinTable, foreignKey, inverseKey }, + id, + missing, + ); + await this.invalidateRowCache(id); + return missing.length; + } + await client.queryExec( `INSERT INTO ${joinTable} (${foreignKey}, ${inverseKey}) VALUES ${missing .map(() => "(?, ?)") @@ -2939,6 +3140,19 @@ export class Repository { "detach", ); + if (this.getDBType(client) === DBType.MongoDB) { + const removed = await mongoDetachLinks( + client, + { joinTable, foreignKey, inverseKey }, + id, + targetIds === undefined + ? undefined + : this.normalizeLinkTargets(targetIds), + ); + await this.invalidateRowCache(id); + return removed; + } + let query = `DELETE FROM ${joinTable} WHERE ${foreignKey} = ?`; const params: any[] = [id]; if (targetIds !== undefined) { @@ -2992,6 +3206,14 @@ export class Repository { (value) => !wantedKeys.has(String(value)), ); + if (this.getDBType(txClient) === DBType.MongoDB) { + const linkRelation = { joinTable, foreignKey, inverseKey }; + await mongoAttachLinks(txClient, linkRelation, id, toAttach); + await mongoDetachLinks(txClient, linkRelation, id, toDetach); + await this.invalidateRowCache(id); + return { attached: toAttach.length, detached: toDetach.length }; + } + if (toAttach.length > 0) { await txClient.queryExec( `INSERT INTO ${joinTable} (${foreignKey}, ${inverseKey}) VALUES ${toAttach diff --git a/tests/integration.mongo.test.ts b/tests/integration.mongo.test.ts index a428e1c..ea170a1 100644 --- a/tests/integration.mongo.test.ts +++ b/tests/integration.mongo.test.ts @@ -311,4 +311,59 @@ suite("MongoDB integration", () => { _id: "m3_slugged", }); }); + + it("visits every row exactly once when paging an unordered query", async () => { + // `eachBatch` walks the result set with `skip`/`limit` in a loop and sets + // no sort, which is safe on SQLite only because a rowid scan is stable. + // Mongo's `skip` over an unordered query has no such guarantee: the server + // is free to hand back a different order per call, so the second page can + // repeat rows from the first and drop others entirely — with no error, and + // a callback that has already been applied to whatever it was given. + // + // The fix is the implicit `{_id: 1}` sort the builder adds whenever a skip + // is set, which makes the order total and therefore the paging exact. That + // the sort is emitted is pinned without a server in + // `mongo.dialect.test.ts`; what this case adds is that the loop built on it + // terminates and stops at the short page. + const Paged = defineModel({ + tableName: "m3_paged", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + }, + }); + await db.client.mongoDeleteMany("m3_paged", {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "m3_paged", + }); + + const repo = db.getRepository(Paged); + await repo.bulkCreate( + Array.from({ length: 250 }, (_, i) => ({ name: `row-${i}` })), + ); + + const seen: number[] = []; + let batches = 0; + await repo.eachBatch( + repo.find(), + (batch: any[]) => { + batches++; + for (const row of batch) seen.push(row.id); + }, + 100, + ); + + // 250 rows at 100 per page: 100, 100, 50 — the short final page is what + // stops the loop, so a page that came back full forever would hang here. + expect(batches).toBe(3); + expect(seen).toHaveLength(250); + expect(new Set(seen).size).toBe(250); + expect(Math.min(...seen)).toBe(1); + expect(Math.max(...seen)).toBe(250); + + await db.client.mongoDeleteMany("m3_paged", {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "m3_paged", + }); + }); }); diff --git a/tests/mongo.blockers.test.ts b/tests/mongo.blockers.test.ts new file mode 100644 index 0000000..e4a4b51 --- /dev/null +++ b/tests/mongo.blockers.test.ts @@ -0,0 +1,679 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { QueryBuilder } from "../query-builder"; +import { Repository } from "../repository"; +import { defineModel } from "../model"; +import { DataTypes, DBType, StabilizeError } from "../types"; +import type { Predicate, MongoStep } from "../index"; + +/** + * Escape hatches on MongoDB: the clauses that have no equivalent, and the error + * a caller gets for reaching for one. + * + * `tests/mongo.dialect.test.ts` already proves `buildMongoSpec` throws when it + * is *handed* a blocker. What that cannot prove is the thing this file exists + * for: that the builder actually wires the blockers it records into + * `buildMongoSpec`, and that `execute()` actually reaches `buildMongoSpec`. A + * method that records a blocker nothing consults is worse than one that does not + * — a `join()` that is silently dropped returns the wrong rows with no error at + * all, and a caller has no way to notice. + * + * So the assertions run over the real API twice: once against `buildMongo()` + * with no server (which isolates the builder → spec wiring), and once against + * `execute(client)` (which adds the execute → builder wiring). The first half + * runs everywhere; the second skips itself when the replica set is not up. + * + * The two names this backend adds to the published entry point are pinned here + * too, for want of a file that owns the public surface. + * + * Start the fleet with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** Collections this file owns. The shared counters collection is not one. */ +const COLLECTIONS = ["b9_docs", "b9_posts"]; + +const B9Doc = defineModel({ + tableName: "b9_docs", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + }, +}); + +const B9Post = defineModel({ + tableName: "b9_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + docId: { type: DataTypes.INTEGER, name: "doc_id" }, + }, +}); + +/** + * Every method that records a blocker, with a call that exercises it. + * + * Exhaustive rather than illustrative: the failure this guards against is a + * *single* method whose blocker never reaches the spec, and a table that named + * only the interesting ones would not catch it. + */ +const BLOCKING: { + method: string; + detail?: string; + call: (q: QueryBuilder) => QueryBuilder; +}[] = [ + { + method: "selectRaw", + detail: "UPPER(title) AS shout", + call: (q) => q.selectRaw("UPPER(title) AS shout"), + }, + { method: "distinct", call: (q) => q.distinct() }, + { + method: "where", + detail: "title = 'a'", + call: (q) => q.where("title = 'a'"), + }, + { + method: "orWhere", + detail: "title = 'b'", + call: (q) => q.orWhere("title = 'b'"), + }, + { + method: "whereNot", + detail: "title = 'c'", + call: (q) => q.whereNot("title = 'c'"), + }, + { + method: "whereExists", + detail: "SELECT 1", + call: (q) => q.whereExists("SELECT 1"), + }, + { + method: "whereNotExists", + detail: "SELECT 1", + call: (q) => q.whereNotExists("SELECT 1"), + }, + { + method: "whereRaw", + detail: "LOWER(title) = 'a'", + call: (q) => q.whereRaw("LOWER(title) = 'a'"), + }, + { + method: "whereRef", + detail: "b9_docs.title = b9_posts.title", + call: (q) => q.whereRef("b9_docs.title", "=", "b9_posts.title"), + }, + { + method: "join", + detail: "LEFT JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + call: (q) => q.join("b9_posts", "b9_posts.doc_id = b9_docs.id"), + }, + { + method: "innerJoin", + detail: "INNER JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + call: (q) => q.innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"), + }, + { + method: "leftJoin", + detail: "LEFT JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + call: (q) => q.leftJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"), + }, + { + method: "rightJoin", + detail: "RIGHT JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + call: (q) => q.rightJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"), + }, + { + method: "fullJoin", + detail: "FULL JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + call: (q) => q.fullJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"), + }, + { + method: "crossJoin", + detail: "b9_posts", + call: (q) => q.crossJoin("b9_posts"), + }, + { + method: "orderByRaw", + detail: "LENGTH(title)", + call: (q) => q.orderByRaw("LENGTH(title)"), + }, + { + method: "groupByRaw", + detail: "LOWER(title)", + call: (q) => q.groupByRaw("LOWER(title)"), + }, + { + method: "having", + detail: "COUNT(*) > 1", + call: (q) => q.having("COUNT(*) > 1"), + }, + { method: "union", call: (q) => q.union(new QueryBuilder("b9_posts")) }, + { method: "unionAll", call: (q) => q.unionAll(new QueryBuilder("b9_posts")) }, + { method: "with", call: (q) => q.with("c", new QueryBuilder("b9_posts")) }, + { + method: "withRecursive", + call: (q) => q.withRecursive("c", new QueryBuilder("b9_posts")), + }, +]; + +/** + * Clauses that *are* translatable. + * + * The other half of the rule: a blocker table that reported too much would make + * MongoDB unusable in a way no thrown-error test would notice, because every + * one of those tests passes when everything throws. + */ +const ALLOWED: { + name: string; + call: (q: QueryBuilder) => QueryBuilder; +}[] = [ + { name: "whereEq", call: (q) => q.whereEq("title", "a") }, + { name: "whereNotEq", call: (q) => q.whereNotEq("title", "a") }, + { name: "whereCompare", call: (q) => q.whereCompare("id", "<", 3) }, + { name: "orWhereEq", call: (q) => q.orWhereEq("title", "a") }, + { name: "orWhereCompare", call: (q) => q.orWhereCompare("id", ">", 1) }, + { name: "orWhereNull", call: (q) => q.orWhereNull("title") }, + { name: "orWhereNotNull", call: (q) => q.orWhereNotNull("title") }, + { name: "orWhereIn", call: (q) => q.orWhereIn("title", ["a", "b"]) }, + { name: "whereIn", call: (q) => q.whereIn("title", ["a", "b"]) }, + { name: "whereNotIn", call: (q) => q.whereNotIn("title", ["a"]) }, + { name: "whereNull", call: (q) => q.whereNull("title") }, + { name: "whereNotNull", call: (q) => q.whereNotNull("title") }, + { name: "whereBetween", call: (q) => q.whereBetween("id", 1, 2) }, + { name: "whereNotBetween", call: (q) => q.whereNotBetween("id", 1, 2) }, + { name: "whereLike", call: (q) => q.whereLike("title", "a%") }, + { name: "whereILike", call: (q) => q.whereILike("title", "a%") }, + { name: "orderBy", call: (q) => q.orderBy("title", "DESC") }, + { name: "groupBy", call: (q) => q.groupBy("title") }, + { name: "limit", call: (q) => q.limit(5) }, + { name: "offset", call: (q) => q.offset(5) }, + { name: "paginate", call: (q) => q.paginate(2, 10) }, + { name: "take", call: (q) => q.take(1) }, + { name: "skip", call: (q) => q.skip(1) }, + { name: "first", call: (q) => q.first() }, + { name: "select", call: (q) => q.select("id", "title") }, + { name: "as", call: (q) => q.as("d").select("d.id", "d.title") }, + { name: "withRelations", call: (q) => q.withRelations("posts") }, + { name: "count", call: (q) => q.count("id", "n") }, + { name: "sum", call: (q) => q.sum("id", "n") }, + { name: "avg", call: (q) => q.avg("id", "n") }, + { name: "min", call: (q) => q.min("id", "n") }, + { name: "max", call: (q) => q.max("id", "n") }, + // Row locking is the documented no-op: Mongo has no row lock to map it + // onto, and the SQL Server path already treats it the same way. + { name: "lock", call: (q) => q.lock("FOR UPDATE") }, + { name: "forUpdate", call: (q) => q.forUpdate() }, + { name: "forShare", call: (q) => q.forShare() }, +]; + +/** Runs `call` and returns whatever it threw, or null. */ +function caught(fn: () => unknown): StabilizeError | null { + try { + fn(); + return null; + } catch (error) { + return error as StabilizeError; + } +} + +// ─── PUBLIC SURFACE ─────────────────────────────────────────────────── + +describe("mongo public surface", () => { + it("re-exports the predicate and the migration step", () => { + // Both are types, so there is nothing here for the runner to assert: the + // check is that the two imports above resolve at all. `tests/` sits outside + // the repo's tsconfig, so `bun tsc --noEmit` never sees them; the gate that + // does is `bun run build`, which must emit both names into + // `dist/index.d.ts`. The lines below only pin their shapes. + const predicate: Predicate = { op: "cmp", column: "title", value: "first" }; + const step: MongoStep = { kind: "dropCollection", collection: "b9_docs" }; + + expect(predicate.op).toBe("cmp"); + expect(step.kind).toBe("dropCollection"); + }); +}); + +// ─── BUILDER → SPEC WIRING (no server) ──────────────────────────────── + +describe("mongo blockers: the builder wires what it records", () => { + for (const { method, detail, call } of BLOCKING) { + it(`refuses ${method}() in buildMongo()`, () => { + const error = caught(() => + call(new QueryBuilder("b9_docs")).buildMongo(), + ); + + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + // The method is named, not just "something". + expect(error!.message).toContain(method); + if (detail !== undefined) { + // And the offending fragment is quoted back, so the caller can find it. + expect(error!.message).toContain(`${method}: ${detail}`); + } + // The message has to say what to reach for instead, or a caller's only + // option is to guess. + expect(error!.message).toContain("withRelations()"); + }); + } + + it("names every blocked clause at once, not only the first", () => { + const error = caught(() => + new QueryBuilder("b9_docs") + .innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id") + .whereRaw("LOWER(title) = 'a'") + .orderByRaw("LENGTH(title)") + .buildMongo(), + ); + + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + for (const method of ["innerJoin", "whereRaw", "orderByRaw"]) { + expect(error!.message).toContain(method); + } + expect(error!.message).toContain("have no MongoDB equivalent"); + }); + + it("keeps rendering the SQL it always did", () => { + // Blockers are recorded *alongside* the SQL arrays, not instead of them, so + // the four working backends keep emitting byte-identical statements. + const qb = new QueryBuilder("b9_docs").innerJoin( + "b9_posts", + "b9_posts.doc_id = b9_docs.id", + ); + const { query, params } = qb.build(DBType.Postgres); + + expect(query).toContain( + "INNER JOIN b9_posts ON b9_posts.doc_id = b9_docs.id", + ); + expect(params).toEqual([]); + // …and the same builder still refuses to be one. + expect(caught(() => qb.buildMongo())!.code).toBe("MONGO_UNSUPPORTED"); + }); + + it("reports a projection that cannot be built rather than widening to *", () => { + // `select()` has no blocker of its own — the expression is only unbuildable + // once someone asks what it projects, which is what `buildMongo` checks. + const error = caught(() => + new QueryBuilder("b9_docs").select("UPPER(title)").buildMongo(), + ); + + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("select"); + expect(error!.message).toContain("UPPER(title)"); + }); + + it("accepts every clause that does translate", () => { + for (const { name, call } of ALLOWED) { + const error = caught(() => + call(new QueryBuilder("b9_docs")).buildMongo(), + ); + expect( + error, + `${name}() should not be reported as unsupported`, + ).toBeNull(); + } + }); + + it("carries blockers into a clone, without leaking them back", () => { + // `countExec` and `existsExec` both clone before they build, so a blocker + // that did not survive the clone would be dropped on those two paths only. + const original = new QueryBuilder("b9_docs"); + const clone = original + .clone() + .innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"); + + expect(caught(() => clone.buildMongo())!.code).toBe("MONGO_UNSUPPORTED"); + expect(caught(() => original.buildMongo())).toBeNull(); + }); + + it("turns a translatable query into the spec it should be", () => { + // The control for every assertion above: this builder is capable of + // producing a spec, so their failures are about the blockers. + const spec = new QueryBuilder("b9_docs") + .whereEq("title", "first") + .orderBy("id", "DESC") + .limit(2) + .offset(3) + .buildMongo(); + + expect(spec.filter).toEqual({ title: "first" }); + expect(spec.sort).toEqual({ _id: -1 }); + expect(spec.limit).toBe(2); + expect(spec.skip).toBe(3); + }); +}); + +// ─── EXECUTE → BUILDER → SPEC WIRING (needs a server) ───────────────── + +suite("mongo blockers: execute() consults them", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + // Two rows in each, so a *silenced* join would visibly return more (or + // fewer) rows than the caller asked for rather than failing loudly. + await db.client.mongoInsertMany("b9_docs", [ + { _id: 1, title: "first" }, + { _id: 2, title: "second" }, + ]); + await db.client.mongoInsertMany("b9_posts", [ + { _id: 10, title: "p1", doc_id: 1 }, + { _id: 11, title: "p2", doc_id: 1 }, + { _id: 12, title: "p3", doc_id: 2 }, + ]); + }); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await db.close(); + }); + + it("reads plainly when nothing is blocked", async () => { + // The control. Without this, every assertion below would also pass against + // an `execute()` that threw unconditionally. + const docs = await db + .getRepository(B9Doc) + .find() + .orderBy("id", "ASC") + .execute(db.client); + + expect(docs.map((d: any) => d.title)).toEqual(["first", "second"]); + }); + + it("calls buildMongo() on the way through execute()", async () => { + // Read as a wiring assertion rather than a behaviour one. The table below + // proves each blocker is *recorded* and that `buildMongo()` raises it; this + // proves `execute()` goes through `buildMongo()` at all, which is the edge + // that a builder recording blockers no one consults would break. + const qb = new QueryBuilder("b9_docs"); + const real = qb.buildMongo.bind(qb); + let calls = 0; + qb.buildMongo = () => { + calls += 1; + return real(); + }; + + const docs = await qb.whereEq("title", "first").execute(db.client); + + expect(calls).toBe(1); + expect(docs).toHaveLength(1); + }); + + it("throws for a projection only `buildMongo()` can judge", async () => { + // `select()` records nothing when it is called — an expression is only + // unbuildable once someone asks what it projects. So this case fails if + // `execute()` skips `buildMongo()` in a way the table above would not: it + // pins the execute → build edge itself, not just the blocker wiring. + const error = await (async () => { + try { + await db + .getRepository(B9Doc) + .find() + .select("UPPER(title)") + .execute(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("UPPER(title)"); + }); + + for (const { method, call } of BLOCKING) { + it(`throws MONGO_UNSUPPORTED out of execute() for ${method}()`, async () => { + const repo = db.getRepository(B9Doc); + const error = await (async () => { + try { + await call(repo.find()).execute(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect( + error, + `${method}() did not throw — a silenced clause returns the wrong rows`, + ).not.toBeNull(); + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain(method); + }); + } + + it("throws out of countExec() and existsExec(), which build a clone", async () => { + const join = (q: QueryBuilder) => + q.innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id"); + + const countError = await (async () => { + try { + await join(new QueryBuilder("b9_docs")).countExec(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + expect(countError!.code).toBe("MONGO_UNSUPPORTED"); + expect(countError!.message).toContain("innerJoin"); + + const existsError = await (async () => { + try { + await join(new QueryBuilder("b9_docs")).existsExec(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + expect(existsError!.code).toBe("MONGO_UNSUPPORTED"); + expect(existsError!.message).toContain("innerJoin"); + }); + + it("still throws when an aggregate and a blocker are combined", async () => { + // `count()` replaces the projection, and `executeMongo` then takes a + // different branch entirely — it sends a pipeline rather than a find. The + // blocker has to be consulted before either branch is chosen. + const error = await (async () => { + try { + await db + .getRepository(B9Doc) + .find() + .count("id", "n") + .innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id") + .execute(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("innerJoin"); + }); + + it("still throws when relations were requested too", async () => { + // `withRelations()` is the documented replacement for a join, but it must + // not be read as *permission*: a query that asked for both is still one the + // server cannot answer as written. + const error = await (async () => { + try { + await db + .getRepository(B9Doc) + .find() + .withRelations("posts") + .innerJoin("b9_posts", "b9_posts.doc_id = b9_docs.id") + .execute(db.client); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("innerJoin"); + }); + + it("treats lock()/forUpdate() as a no-op rather than a blocker", async () => { + const docs = await db + .getRepository(B9Doc) + .find() + .forUpdate() + .orderBy("id", "ASC") + .execute(db.client); + + expect(docs).toHaveLength(2); + }); + + it("accepts lockForUpdate() and returns the row unprotected", async () => { + // The repository's own spelling of the same no-op. It reads as a lock and + // takes none, which is the reason it is documented rather than thrown: + // refusing would break a read that has a correct answer. + const repo = db.getRepository(B9Doc); + const doc = await repo.lockForUpdate(1); + + expect(doc?.title).toBe("first"); + }); + + it("warns that lockForUpdate() takes no lock, rather than doing it silently", async () => { + // The read above is correct either way, which is exactly the hazard: a + // caller who asked for a lock by name gets a result indistinguishable from + // one that was locked, does read-modify-write on the strength of it, and + // loses the update against a concurrent writer with nothing to notice. + // Throwing would refuse a query that has a good answer, so the diagnostic + // is the whole remedy — and a diagnostic nothing asserts is a diagnostic + // that can be deleted by accident. + const warnings: string[] = []; + const stubLogger = { + logError: () => {}, + logInfo: () => {}, + logWarn: (message: string) => warnings.push(message), + logDebug: () => {}, + } as any; + const repo = new Repository( + db.client, + B9Doc as any, + { enabled: false, ttl: 60 }, + stubLogger, + ); + + const doc = await repo.lockForUpdate(1); + + expect(doc?.title).toBe("first"); + expect(warnings).toHaveLength(1); + expect(warnings[0]).toContain("lockForUpdate"); + expect(warnings[0]).toContain("b9_docs"); + expect(warnings[0]).toContain("no lock on MongoDB"); + }); + + it("answers count() and exists() without SQL", async () => { + const repo = db.getRepository(B9Doc); + expect(await repo.count({})).toBe(2); + expect(await repo.count({ title: "first" })).toBe(1); + expect(await repo.exists({ title: "first" })).toBe(true); + expect(await repo.exists({ title: "nope" })).toBe(false); + }); + + // ─── raw SQL escape hatches ──────────────────────────────────────── + + it("refuses rawQuery() on the repository with MONGO_UNSUPPORTED", async () => { + const error = await (async () => { + try { + await db.getRepository(B9Doc).rawQuery("SELECT * FROM b9_docs"); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + // The message has to point somewhere, or "no" is all the caller learns. + expect(error!.message).toMatch(/structured methods|repository API/); + }); + + it("refuses rawQuery()/rawExec() on the Stabilize instance", async () => { + for (const attempt of [ + () => db.rawQuery("SELECT * FROM b9_docs"), + () => db.rawExec("DELETE FROM b9_docs"), + ]) { + const error = await (async () => { + try { + await attempt(); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).not.toBe(""); + } + }); + + it("tells the caller to use updateBy() for a raw WHERE", async () => { + // The one place the message names the exact replacement, because there is + // exactly one. + const error = await (async () => { + try { + await db + .getRepository(B9Doc) + .bulkUpdate([ + { + where: { condition: "title = ?", params: ["first"] }, + set: { title: "x" }, + }, + ]); + return null; + } catch (thrown) { + return thrown as StabilizeError; + } + })(); + + expect(error).toBeInstanceOf(StabilizeError); + expect(error!.code).toBe("MONGO_UNSUPPORTED"); + expect(error!.message).toContain("updateBy()"); + }); +}); diff --git a/tests/mongo.relations.test.ts b/tests/mongo.relations.test.ts new file mode 100644 index 0000000..82442a3 --- /dev/null +++ b/tests/mongo.relations.test.ts @@ -0,0 +1,618 @@ +import { + describe, + it, + expect, + beforeAll, + beforeEach, + afterAll, +} from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * Eager-loaded relations, against a real MongoDB server. + * + * Relations were the one feature that needed no algorithm change for a document + * store, because they were already batched `IN` reads rather than joins: the + * same shape `$in` gives. What this file has to prove is therefore narrower than + * the SQL original — that the batched reads still land on the right rows, that + * the grouping still gives each parent its own children, and that the link + * collection the many-to-many side now reads and writes is the one + * `autoMigrate` created. + * + * The original is `integration.relations.test.ts`, and each case below carries + * over the failure it was written to catch. + * + * The suite skips itself when the replica set is not running. Start it with: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** Collections this file owns; dropped on the way out. */ +const COLLECTIONS = [ + "rel_authors", + "rel_books", + "rel_chapters", + "rel_tags", + "rel_shelves", + "rel_book_tags", + MONGO_COUNTERS_COLLECTION, +]; + +const Author = defineModel({ + tableName: "rel_authors", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true }, + }, +}); + +const Tag = defineModel({ + tableName: "rel_tags", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING, required: true }, + }, + relations: [ + { + type: RelationType.ManyToMany, + target: () => Book, + property: "books", + joinTable: "rel_book_tags", + foreignKey: "tag_id", + inverseKey: "book_id", + }, + ], +}); + +const Book = defineModel({ + tableName: "rel_books", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // The property name deliberately differs from the column name. Documents + // are keyed by column name, so a relation that looked the property up + // verbatim would filter on a field no document has and come back empty — + // a wrong answer with no error. + authorId: { type: DataTypes.INTEGER, name: "author_id" }, + }, + relations: [ + { + type: RelationType.ManyToOne, + target: () => Author, + property: "author", + foreignKey: "authorId", + }, + { + type: RelationType.OneToMany, + target: () => Chapter, + property: "chapters", + inverseKey: "bookId", + }, + { + type: RelationType.ManyToMany, + target: () => Tag, + property: "tags", + joinTable: "rel_book_tags", + foreignKey: "book_id", + inverseKey: "tag_id", + }, + ], +}); + +const Chapter = defineModel({ + tableName: "rel_chapters", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING }, + bookId: { type: DataTypes.INTEGER, name: "book_id" }, + removedAt: { type: DataTypes.DATETIME, softDelete: true }, + }, + relations: [ + { + type: RelationType.ManyToOne, + target: () => Book, + property: "book", + foreignKey: "bookId", + }, + ], +}); + +describe("mongo relation loading", () => { + let db: any; + + /** The exact call the SQL file makes with `rawExec`, against collections. */ + const insert = (collection: string, docs: Record[]) => + db.client.mongoInsertMany(collection, docs); + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + + // The link collection is created by `autoMigrate` here, not by hand: the + // join table has no model, so this is the only thing that can create one, + // and the many-to-many reads below depend on it existing. + await db.autoMigrate([Author, Book, Chapter, Tag]); + + await insert("rel_authors", [{ _id: 1, name: "Ada" }]); + // Book ids deliberately differ from their author's, so a child attached to + // the wrong side of the grouping is detectable. + await insert("rel_books", [ + { _id: 10, title: "First", author_id: 1 }, + { _id: 11, title: "Second", author_id: 1 }, + { _id: 12, title: "Orphan" }, + ]); + await insert("rel_chapters", [ + { _id: 100, title: "One", book_id: 10 }, + { _id: 101, title: "Two", book_id: 10 }, + { _id: 102, title: "Three", book_id: 10 }, + { _id: 103, title: "Other", book_id: 11 }, + ]); + await insert("rel_tags", [ + { _id: 1, label: "alpha" }, + { _id: 2, label: "beta" }, + ]); + // Written raw, in the same shape `attach` writes: the composite `_id` is + // what makes the pair unique, so a link document without one would not be + // the document this code reads. + await insert("rel_book_tags", [ + { _id: { p: 10, c: 2 }, book_id: 10, tag_id: 2 }, + { _id: { p: 10, c: 1 }, book_id: 10, tag_id: 1 }, + { _id: { p: 11, c: 1 }, book_id: 11, tag_id: 1 }, + ]); + }); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await db.close(); + }); + + it("attaches a to-many relation instead of dropping it", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters"] }); + + expect(book.chapters).toHaveLength(3); + // Sorted by the test rather than expected in insertion order: a to-many + // read is `find({book_id: {$in: [...]}})` with no sort, and unlike SQLite's + // rowid scan a document store makes no promise about the order it returns. + expect(book.chapters.map((c: any) => c.id).sort()).toEqual([ + 100, 101, 102, + ]); + }); + + it("keeps the parent row intact when a relation is loaded", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author"] }); + + expect(book.id).toBe(10); + expect(book.title).toBe("First"); + expect(book.author).toEqual({ id: 1, name: "Ada" }); + }); + + it("resolves a foreign key declared under a mapped column name", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author"] }); + + // `authorId` is stored as `author_id`, so this only works if the relation + // translates the property to its column before building the filter. + expect(book.author.name).toBe("Ada"); + expect(book.author_id).toBe(1); + }); + + it("gives a to-one relation null when the key is unset", async () => { + const orphan: any = await db + .getRepository(Book) + .findOne(12, { relations: ["author"] }); + + // `null`, not `undefined`: the relation was loaded and is genuinely empty. + expect(orphan.author).toBeNull(); + }); + + it("counts parents, not children", async () => { + const { data, total } = await db + .getRepository(Book) + .findAndCount({ relations: ["chapters"] }); + + expect(total).toBe(3); + expect(data).toHaveLength(3); + + const byTitle = Object.fromEntries( + data.map((b: any) => [b.title, b.chapters.length]), + ); + expect(byTitle).toEqual({ First: 3, Second: 1, Orphan: 0 }); + }); + + it("returns the requested number of parents, not of children", async () => { + const books = await db + .getRepository(Book) + .findMany({ relations: ["chapters"], take: 2 }); + + expect(books).toHaveLength(2); + expect(new Set(books.map((b: any) => b.id)).size).toBe(2); + }); + + it("loads a many-to-many relation through its join collection", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["tags"] }); + + // Sorted on the link `_id` rather than returned in insertion order, which + // is the difference from the SQL file's assertion: a join table read with + // no `ORDER BY` has no order at all, and Mongo's is not even stable between + // two reads of unchanged data. + expect(book.tags.map((t: any) => t.label)).toEqual(["alpha", "beta"]); + }); + + it("returns an empty array for a relation with no links", async () => { + const book: any = await db + .getRepository(Book) + .findOne(12, { relations: ["tags"] }); + + expect(book.tags).toEqual([]); + }); + + it("supports the inverse side of a many-to-many relation", async () => { + const tag: any = await db + .getRepository(Tag) + .findOne(1, { relations: ["books"] }); + + // The inverse orientation reads the same collection by the other field. + expect(tag.books.map((b: any) => b.title).sort()).toEqual([ + "First", + "Second", + ]); + }); + + it("loads a nested path against the target model", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters.book"] }); + + expect(book.chapters).toHaveLength(3); + // Every chapter of book 10 points back at book 10, so the nested read + // resolved the target model rather than reusing the parent. + expect(book.chapters.every((c: any) => c.book?.title === "First")).toBe( + true, + ); + }); + + it("loads several relations in one call", async () => { + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["author", "chapters", "tags"] }); + + expect(book.author.id).toBe(1); + expect(book.chapters).toHaveLength(3); + expect(book.tags).toHaveLength(2); + }); + + it("gives every parent its own children in a batch read", async () => { + const books = await db + .getRepository(Book) + .findMany({ relations: ["chapters"] }); + + const byTitle = Object.fromEntries( + books.map((b: any) => [b.title, b.chapters.map((c: any) => c.title)]), + ); + + // One batched read serves every parent, so the grouping is what this pins. + expect(byTitle["First"]).toEqual(["One", "Two", "Three"]); + expect(byTitle["Second"]).toEqual(["Other"]); + expect(byTitle["Orphan"]).toEqual([]); + }); + + it("omits related rows the target model soft-deleted", async () => { + const chapters = db.getRepository(Chapter); + await chapters.update(101, { removedAt: new Date() } as any); + + const book: any = await db + .getRepository(Book) + .findOne(10, { relations: ["chapters"] }); + + expect(book.chapters.map((c: any) => c.id)).toEqual([100, 102]); + + await chapters.recover(101); + }); + + it("loads relations for a findBy condition", async () => { + const books = await db + .getRepository(Book) + .findBy({ authorId: 1 }, { relations: ["author"] }); + + expect(books).toHaveLength(2); + expect(books[0].author.name).toBe("Ada"); + }); + + it("rejects an unknown relation name", async () => { + // Silently returning the bare row is what made the original bug hard to + // notice. try/catch rather than `.rejects`: a rejection assertion the + // driver never settles leaves Bun's runner hanging. + let caught: Error | null = null; + try { + await db.getRepository(Book).findOne(10, { relations: ["nope"] }); + } catch (error) { + caught = error as Error; + } + + expect(caught?.message).toMatch(/Relation nope not found/); + }); + + it("accepts foreignKey as the name of a OneToMany's inverse column", async () => { + const Shelf = defineModel({ + tableName: "rel_shelves", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING }, + }, + relations: [ + { + type: RelationType.OneToMany, + target: () => Book, + property: "books", + foreignKey: "authorId", + }, + ], + }); + await db.autoMigrate([Shelf]); + await insert("rel_shelves", [{ _id: 1, name: "Top" }]); + + const shelf: any = await db + .getRepository(Shelf) + .findOne(1, { relations: ["books"] }); + + // Matched on `author_id`: the named property was translated to its column. + // `Orphan` has no key, so it belongs to no shelf. + expect(shelf.books.map((b: any) => b.title).sort()).toEqual([ + "First", + "Second", + ]); + }); + + it("loads relations requested on the builder", async () => { + const books = await db + .getRepository(Book) + .find() + .withRelations("chapters") + .execute(db.client); + + expect(books).toHaveLength(3); + const byTitle = Object.fromEntries( + books.map((b: any) => [b.title, b.chapters.length]), + ); + expect(byTitle).toEqual({ First: 3, Second: 1, Orphan: 0 }); + }); + + it("composes with a condition, limit and relations", async () => { + // `whereEq` rather than the SQL file's `where("rel_books.title = ?")`: + // there is no SQL text to send on this backend, and the raw-clause methods + // refuse outright. + const books = await db + .getRepository(Book) + .find() + .whereEq("title", "First") + .withRelations("author", "chapters") + .limit(1) + .execute(db.client); + + expect(books).toHaveLength(1); + expect(books[0].author.name).toBe("Ada"); + expect(books[0].chapters).toHaveLength(3); + }); + + it("loads a nested path from the builder", async () => { + const books = await db + .getRepository(Book) + .find() + .withRelations("chapters.book") + .execute(db.client); + + const first = books.find((b: any) => b.title === "First"); + expect(first.chapters).toHaveLength(3); + expect(first.chapters.every((c: any) => c.book?.title === "First")).toBe( + true, + ); + }); +}); + +describe("mongo many-to-many link management", () => { + let db: any; + + const reset = async () => { + await db.client.mongoDeleteMany("rel_book_tags", {}); + }; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + // Dropped rather than reused: the block above seeded the same ids, and an + // insert of an existing `_id` fails outright. + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await db.autoMigrate([Author, Book, Chapter, Tag]); + await db.client.mongoInsertMany("rel_books", [ + { _id: 10, title: "First", author_id: 1 }, + { _id: 11, title: "Second", author_id: 1 }, + { _id: 12, title: "Orphan" }, + ]); + await db.client.mongoInsertMany("rel_tags", [ + { _id: 1, label: "alpha" }, + { _id: 2, label: "beta" }, + { _id: 3, label: "gamma" }, + ]); + }); + + beforeEach(reset); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await db.close(); + }); + + it("attaches, and reports nothing to do the second time", async () => { + const repo = db.getRepository(Book); + + expect(await repo.attach(10, "tags", [1, 2])).toBe(2); + // Idempotent: the pair is already linked, so nothing is created. + expect(await repo.attach(10, "tags", [1, 2])).toBe(0); + expect(await repo.attach(10, "tags", 3)).toBe(1); + + const book: any = await repo.findOne(10, { relations: ["tags"] }); + expect(book.tags.map((t: any) => t.label)).toEqual([ + "alpha", + "beta", + "gamma", + ]); + }); + + it("links a repeated target once", async () => { + const repo = db.getRepository(Book); + + // The join collection carries no unique index on the pair other than its + // `_id`, so a duplicate in the caller's list would otherwise insert twice. + expect(await repo.attach(10, "tags", [1, 1, 1])).toBe(1); + + const links = await db.client.mongoFind("rel_book_tags", { book_id: 10 }); + expect(links).toHaveLength(1); + }); + + it("keys a link by the pair, so the storage layer enforces uniqueness", async () => { + await db.getRepository(Book).attach(10, "tags", [1]); + + const link = await db.client.mongoFindOne("rel_book_tags", { + book_id: 10, + tag_id: 1, + }); + // `_id` is the pair, not a generated value: that is what makes the second + // insert of the same pair impossible rather than merely unlikely. + expect(link._id).toEqual({ p: 10, c: 1 }); + + // And a direct duplicate is refused by the server, which is the property + // the SQL join table does not have. + let caught: any = null; + try { + await db.client.mongoInsertOne("rel_book_tags", { + _id: { p: 10, c: 1 }, + book_id: 10, + tag_id: 1, + }); + } catch (error) { + caught = error; + } + expect(caught).not.toBeNull(); + }); + + it("detaches one target", async () => { + const repo = db.getRepository(Book); + await repo.attach(10, "tags", [1, 2, 3]); + + expect(await repo.detach(10, "tags", [2])).toBe(1); + + const book: any = await repo.findOne(10, { relations: ["tags"] }); + expect(book.tags.map((t: any) => t.label)).toEqual(["alpha", "gamma"]); + }); + + it("detaches everything when no targets are named", async () => { + const repo = db.getRepository(Book); + await repo.attach(10, "tags", [1, 2]); + await repo.attach(11, "tags", [1]); + + expect(await repo.detach(10, "tags")).toBe(2); + + // Only the named parent was touched, so the filter is not "every link". + expect( + await db.client.mongoCount("rel_book_tags", { book_id: 11 }), + ).toBe(1); + }); + + it("syncs to exactly the given set", async () => { + const repo = db.getRepository(Book); + await repo.attach(10, "tags", [1, 2]); + + expect(await repo.sync(10, "tags", [2, 3])).toEqual({ + attached: 1, + detached: 1, + }); + + const book: any = await repo.findOne(10, { relations: ["tags"] }); + expect(book.tags.map((t: any) => t.label)).toEqual(["beta", "gamma"]); + }); + + it("reports nothing to do when sync runs twice", async () => { + const repo = db.getRepository(Book); + await repo.sync(10, "tags", [1, 2]); + + // The second call is the idempotency assertion: if the diff were computed + // against anything but the stored links it would report work every time. + expect(await repo.sync(10, "tags", [1, 2])).toEqual({ + attached: 0, + detached: 0, + }); + expect( + await db.client.mongoCount("rel_book_tags", { book_id: 10 }), + ).toBe(2); + }); + + it("syncs to nothing", async () => { + const repo = db.getRepository(Book); + await repo.attach(10, "tags", [1, 2]); + + expect(await repo.sync(10, "tags", [])).toEqual({ + attached: 0, + detached: 2, + }); + expect(await repo.findOne(10, { relations: ["tags"] })).toMatchObject({ + tags: [], + }); + }); +}); diff --git a/tests/mongo.transactions.test.ts b/tests/mongo.transactions.test.ts new file mode 100644 index 0000000..d7a0b9c --- /dev/null +++ b/tests/mongo.transactions.test.ts @@ -0,0 +1,484 @@ +import { describe, it, expect, beforeAll, beforeEach, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType, RelationType } from "../types"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * Relations read *inside* a transaction, against a real MongoDB server. + * + * The client-level suite (`mongo.client.test.ts`) already proves a write inside + * a transaction goes through the session, by rolling the transaction back and + * watching the document disappear. What it cannot prove is the other half: that + * a *read the ORM performs on the caller's behalf* goes through the same + * session. `loadRelations`, `attachRelation`, `fetchRelatedWhereIn` and + * `attachManyToMany` all take a `client` argument, and if any of them reached + * for `this.client` instead, a relation loaded mid-transaction would read from + * outside it: the parent would come back with an empty relation while the rows + * it was asked for sat uncommitted a session away, and nothing would raise. + * + * The equivalent case exists for SQLite/Postgres in + * `integration.relations.test.ts` ("keeps a relation-loaded read out of the + * transaction cache"), where the failure was a cross-connection read that on + * MySQL/Postgres deadlocks rather than merely answering wrongly. On MongoDB the + * failure is quieter — the read simply does not see the transaction's rows — + * which is exactly why it needs a test rather than a reviewer. + * + * The suite skips itself when the replica set is not running, so `bun test` + * stays green on a machine without the fleet: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** + * Collections this file owns; dropped on the way out. + * + * `stabilize_counters` is deliberately *not* dropped here: it is shared with + * every other mongo test file running in the same process, and dropping it + * would reset counters another suite is mid-way through allocating from. The + * per-table documents this file owns are deleted individually instead, see + * `reset`. + */ +const COLLECTIONS = ["t8_users", "t8_posts", "t8_tags", "t8_post_tags"]; + +/** The counter documents this file is allowed to remove. */ +const COUNTER_KEYS = ["t8_users", "t8_posts", "t8_tags"]; + +const User = defineModel({ + tableName: "t8_users", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + name: { type: DataTypes.STRING, required: true }, + }, + relations: [ + { + type: RelationType.OneToMany, + target: () => Post, + property: "posts", + inverseKey: "authorId", + }, + ], +}); + +const Tag = defineModel({ + tableName: "t8_tags", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING, required: true }, + }, + relations: [ + { + type: RelationType.ManyToMany, + target: () => Post, + property: "posts", + joinTable: "t8_post_tags", + foreignKey: "tag_id", + inverseKey: "post_id", + }, + ], +}); + +const Post = defineModel({ + tableName: "t8_posts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // Named differently from its column on purpose, so a relation loader that + // skipped the name translation would filter a field that does not exist and + // return nothing — the same silent wrong answer this file is about. + authorId: { type: DataTypes.INTEGER, name: "author_id" }, + }, + relations: [ + { + type: RelationType.ManyToOne, + target: () => User, + property: "author", + foreignKey: "authorId", + }, + { + type: RelationType.ManyToMany, + target: () => Tag, + property: "tags", + joinTable: "t8_post_tags", + foreignKey: "post_id", + inverseKey: "tag_id", + }, + ], +}); + +suite("mongo relations inside a transaction", () => { + let db: any; + let users: any; + let posts: any; + let tags: any; + + /** + * An empty set of rows and a counter back at zero, so every case can assert + * on absolute ids rather than on what the previous case left behind. + * + * `deleteMany` rather than `drop`: the validator and the indexes `autoMigrate` + * installed are part of what these cases rely on, and a drop takes them with + * it. The counter is reset per table, never as a collection. + */ + const reset = async () => { + for (const name of COLLECTIONS) { + await db.client.mongoDeleteMany(name, {}); + } + for (const table of COUNTER_KEYS) { + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: table, + }); + } + // Reference data, rewritten on every case so a case that links tags cannot + // be affected by one that did. + await db.client.mongoInsertMany("t8_tags", [ + { _id: 1, label: "alpha" }, + { _id: 2, label: "beta" }, + { _id: 3, label: "gamma" }, + ]); + }; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + + // `autoMigrate` is what creates `t8_post_tags`: the join table has no model + // of its own, so nothing else can. The many-to-many cases below depend on it + // existing, and on the link documents `attach` writes having the shape the + // read expects. + await db.autoMigrate([User, Post, Tag]); + + users = db.getRepository(User); + posts = db.getRepository(Post); + tags = db.getRepository(Tag); + }); + + beforeEach(reset); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + // Only this file's counter documents, for the reason given on COUNTER_KEYS. + try { + for (const table of COUNTER_KEYS) { + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: table, + }); + } + } catch { + // Nothing to clean if the connection is already gone. + } + await db.close(); + }); + + // ─── the gate: relations read on the transaction's own client ────── + + it("loads a relation on the client it was given, not the repository's own", async () => { + // The sharpest form of the question, because the parent is committed and + // visible to both clients: only the *children* differ between the two reads, + // so nothing about the parent read can explain the answer either way. + // + // Reading it on the root client must show the child that is already + // committed. Reading it on the transaction's client must show that child + // *and* the one this transaction has not committed yet. `loadRelations` + // reaches the children through `fetchRelatedWhereIn`, which builds the + // target model's query and executes it against the client it was handed — + // through `relatedRepository`, which binds the target model to that same + // client. Had either reached for `this.client`, the second read would agree + // with the first, and this is the assertion that would catch it. + const user: any = await users.create({ name: "Ada" }); + await posts.create({ title: "committed", authorId: user.id }); + + await db.transaction(async (tx: any) => { + await posts.create({ title: "in-flight", authorId: user.id }, {}, tx); + + const outside: any = await users.findOne(user.id, { + relations: ["posts"], + }); + expect(outside.posts.map((post: any) => post.title)).toEqual([ + "committed", + ]); + + const inside: any = await users.findOne( + user.id, + { relations: ["posts"] }, + tx, + ); + expect(inside.posts.map((post: any) => post.title).sort()).toEqual([ + "committed", + "in-flight", + ]); + }); + }); + + it("sees a one-to-many relation built inside the transaction", async () => { + let sawOutside: boolean | null = null; + + await db.transaction(async (tx: any) => { + const user: any = await users.create({ name: "Ada" }, {}, tx); + await posts.create({ title: "One", authorId: user.id }, {}, tx); + await posts.create({ title: "Two", authorId: user.id }, {}, tx); + + // The assertion this file exists for. `loadRelations` reaches the children + // through `fetchRelatedWhereIn`, which runs the target model's `find()` + // against the client it was handed. Handed the root client instead, this + // read would run outside the transaction, match nothing, and attach an + // empty array — the parent would still be found, so only the relation + // count tells the two apart. + const read: any = await users.findOne( + user.id, + { relations: ["posts"] }, + tx, + ); + expect(read.posts.map((post: any) => post.title).sort()).toEqual([ + "One", + "Two", + ]); + + // Control. The same document read through the *root* client is not there, + // so the visibility above is the session's doing and not a dirty read that + // would have made the assertion pass for the wrong reason. + sawOutside = (await users.findOne(user.id)) !== null; + }); + + expect(sawOutside).toBe(false); + + // And the commit did make it all permanent, so the case is not passing by + // leaving everything uncommitted. + const committed: any = await users.findOne(1, { relations: ["posts"] }); + expect(committed.posts).toHaveLength(2); + }); + + it("sees a many-to-many link attached inside the transaction", async () => { + await db.transaction(async (tx: any) => { + const post: any = await posts.create({ title: "Linked" }, {}, tx); + + // `attach` reads the existing links through `mongoFetchLinkedIds` and + // writes the missing ones through `mongoAttachLinks`, both with the client + // it was given. On the root client the read would see no links, every + // pair would look missing, and the write would land outside the + // transaction — so this count is the session reaching the link collection. + const attached = await posts.attach(post.id, "tags", [1, 2], tx); + expect(attached).toBe(2); + + // Read back through `mongoFindLinks`, the other half of the same + // question: the link collection was written on the session, and it has to + // be read on the session too. + const read: any = await posts.findOne( + post.id, + { relations: ["tags"] }, + tx, + ); + expect(read.tags.map((tag: any) => tag.label)).toEqual([ + "alpha", + "beta", + ]); + + // The inverse orientation reads the same documents by the other field, so + // it exercises a differently-shaped link read rather than the same one + // twice. + const inverse: any = await tags.findOne(1, { relations: ["posts"] }, tx); + expect(inverse.posts.map((p: any) => p.title)).toEqual(["Linked"]); + }); + + // Committed, so the link outlives the transaction it was made in. + const after: any = await posts.findOne(1, { relations: ["tags"] }); + expect(after.tags.map((tag: any) => tag.label)).toEqual(["alpha", "beta"]); + expect(await db.client.mongoCount("t8_post_tags", {})).toBe(2); + }); + + // ─── rollback ────────────────────────────────────────────────────── + + it("rolls back the parent, the children and the link together", async () => { + let userId: any = null; + let postId: any = null; + + // try/catch rather than `.rejects`: a rejection assertion the driver never + // settles leaves Bun's runner hanging. + let thrown: unknown = null; + try { + await db.transaction(async (tx: any) => { + const user: any = await users.create({ name: "doomed" }, {}, tx); + userId = user.id; + const post: any = await posts.create( + { title: "doomed", authorId: user.id }, + {}, + tx, + ); + postId = post.id; + await posts.attach(post.id, "tags", [1, 2], tx); + + // Read inside, so the case also pins that the rows were genuinely there + // before the rollback rather than never written at all. + const read: any = await users.findOne( + user.id, + { relations: ["posts"] }, + tx, + ); + expect(read.posts).toHaveLength(1); + + throw new Error("deliberate"); + }); + } catch (error) { + thrown = error; + } + + expect((thrown as Error)?.message).toBe("deliberate"); + + // None of the three collections kept anything, which is the property the + // session is supposed to give: one rollback undoes the parent, the child + // that points at it, and the link between the child and a tag — the three + // writes being one unit of work rather than three unrelated statements. + expect(await db.client.mongoFindOne("t8_users", { _id: userId })).toBeNull(); + expect(await db.client.mongoFindOne("t8_posts", { _id: postId })).toBeNull(); + expect(await db.client.mongoCount("t8_post_tags", {})).toBe(0); + + // Nothing is visible through the repository either, which is the answer a + // caller would actually receive. + expect(await users.findOne(userId)).toBeNull(); + expect(await posts.findOne(postId, { relations: ["tags"] })).toBeNull(); + expect(await users.count()).toBe(0); + }); + + it("rolls back the id allocation with the transaction", async () => { + // A deliberate divergence from MySQL and SQLite, where auto-increment does + // not roll back: an id burned by an aborted transaction is gone for good + // there. Here the counter `$inc` runs through the transaction's session, so + // the reservation is undone with everything else and the next write reuses + // the id. Pinned as a decision, not left as an accident — a caller that + // assumed the SQL behaviour would see ids repeat across a rollback. + let allocatedId: any = null; + + try { + await db.transaction(async (tx: any) => { + const user: any = await users.create({ name: "never" }, {}, tx); + allocatedId = user.id; + expect(allocatedId).toBe(1); + throw new Error("deliberate"); + }); + } catch { + // Expected; the assertion is on what the counter holds afterwards. + } + + expect(await users.count()).toBe(0); + + // Had the `$inc` committed on its own, this would be 2. + const survivor: any = await users.create({ name: "survivor" }); + expect(survivor.id).toBe(1); + }); + + // ─── snapshot isolation ──────────────────────────────────────────── + + it("does not see a row written outside the transaction after its snapshot", async () => { + // `withTransaction` opens the transaction with `readConcern: {level: + // "snapshot"}`, which fixes what the transaction can see at its first + // operation. Deterministic because it is the read concern that decides it, + // not a race: the outside write is made *after* the snapshot and is read + // back from outside before the transaction asks again. + let sawLateRow: boolean | null = null; + let lateRowIsCommitted: boolean | null = null; + + await db.transaction(async (tx: any) => { + await posts.create({ title: "early" }, {}, tx); + // First read: the snapshot exists from here on. + expect(await posts.findOne(1, {}, tx)).not.toBeNull(); + + // Written with a raw insert rather than through the repository, so it + // does not touch the counter document the open transaction has just + // incremented — an `$inc` on that document from outside would be a write + // conflict, and `withTransaction` would replay this callback rather than + // let the case observe anything. + // The validator `autoMigrate` installed requires `title` and types every + // declared column, so this carries exactly what `t8_posts` allows. An + // explicit `author_id: null` would be rejected as a document-validation + // failure, not as the conflict being tested for. + await db.client.mongoInsertOne("t8_posts", { + _id: 9001, + title: "late", + }); + + // Committed and readable from outside the transaction, so its absence + // below cannot be explained by the write not having happened. + lateRowIsCommitted = + (await db.client.mongoFindOne("t8_posts", { _id: 9001 })) !== null; + + sawLateRow = (await posts.findOne(9001, {}, tx)) !== null; + }); + + expect(lateRowIsCommitted).toBe(true); + expect(sawLateRow).toBe(false); + + // The row is there for everyone else, which is the point: the transaction + // read its own snapshot rather than the current state. + expect(await db.client.mongoFindOne("t8_posts", { _id: 9001 })).not.toBeNull(); + }); + + it("holds its own uncommitted writes apart from a same-collection read outside", async () => { + // The complement of the case above, from the other side: a transaction's + // own writes are invisible to reads outside it, while its relation reads + // still see them. Both directions together are what makes the visibility + // asserted earlier attributable to the session rather than to the read + // concern. + let outsideSawParent: boolean | null = null; + let outsideSawChild: boolean | null = null; + + await db.transaction(async (tx: any) => { + const user: any = await users.create({ name: "in-flight" }, {}, tx); + await posts.create({ title: "in-flight", authorId: user.id }, {}, tx); + + outsideSawParent = (await users.findOne(user.id)) !== null; + // Raw, so the answer does not depend on the repository's soft-delete and + // relation handling: the transaction's child is not in the collection yet. + outsideSawChild = (await db.client.mongoFind("t8_posts", {})).length > 0; + + // Still visible inside, in the same breath. + const read: any = await users.findOne( + user.id, + { relations: ["posts"] }, + tx, + ); + expect(read.posts).toHaveLength(1); + }); + + expect(outsideSawParent).toBe(false); + expect(outsideSawChild).toBe(false); + }); +}); diff --git a/tests/mongo.versioning.test.ts b/tests/mongo.versioning.test.ts new file mode 100644 index 0000000..ea54898 --- /dev/null +++ b/tests/mongo.versioning.test.ts @@ -0,0 +1,641 @@ +import { describe, it, expect, beforeAll, beforeEach, afterAll } from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; +import { Repository } from "../repository"; +import { decrypt } from "../utils/encryption"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * Versioning, encryption and caching, against a real MongoDB server. + * + * Versioning is the one feature this backend could not carry across by leaving + * it alone: `asOf`, `history` and `rollback` are raw SQL statements on the + * repository, and `writeHistory` builds an `INSERT`, so any model declared + * `versioned: true` threw `MONGO_UNSUPPORTED` on its first `create()` — the + * dispatch never existed. The cases below pin the three reads down to *values* + * rather than to the absence of an error, because a versioning feature that + * silently returns the wrong version is worse than one that throws: the older + * title has to come back from `asOf`, `history` has to come back in order with + * the operation that made each version, and `rollback` has to leave the row + * holding the old value *and* a version that moved forward. + * + * Two things about this backend's history collection are load-bearing and are + * asserted directly rather than inferred: + * + * - a history document is keyed by **column name**, so a column declared with + * a `name:` mapping is recorded under that column name — every reader of a + * history row addresses it that way, and a rollback restores from it; + * - the validity window is a native `Date`, not the ISO string the SQL path + * binds. A string fails the collection's `{bsonType: "date"}` validator (one + * case below proves the server really rejects it) and would turn the window + * comparison into a comparison between BSON types. + * + * The suite skips itself when the replica set is not running, so `bun test` + * stays green on a machine without the fleet: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up --dry-run`, + ); +} + +/** Lets two writes land in different milliseconds, which the windows need. */ +const tick = (ms = 8) => new Promise((resolve) => setTimeout(resolve, ms)); + +const Post = defineModel({ + tableName: "v7_posts", + versioned: true, + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // Renamed, deliberately. The property key and the column name differ, so a + // history document written under either one can be told apart — and the + // rollback below restores this column, which only works if the history row + // carries it under the name the live document uses. + subtitle: { type: DataTypes.STRING, name: "subtitle_col" }, + // The lock the versioning and the lock share: `writeHistory` records the + // version number as the same field, and a rollback has to leave the live row + // and the newest history row agreeing on it. + version: { type: DataTypes.INTEGER, optimisticLock: true }, + }, +}); + +// The encrypted column is deliberately *not* renamed. +// +// `processForLoad` walks the property keys and reads `row[propertyKey]`, while a +// row arrives keyed by column name — so a column carrying both `encrypted: true` +// and a `name:` mapping is never decrypted on read, on any of the five backends. +// A model here with `ssn: { encrypted: true, name: "ssn_col" }` would fail the +// round-trip below and pin a bug this milestone is not the place to fix. +const Secret = defineModel({ + tableName: "v7_secrets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING }, + ssn: { type: DataTypes.STRING, encrypted: true }, + }, +}); + +const HISTORY = "v7_posts_history"; + +/** Everything this file creates. The counters collection is *not* in here. */ +const COLLECTIONS = ["v7_posts", HISTORY, "v7_secrets"]; + +/** + * A cache that records what it was asked and keeps what it was given. + * + * The ORM's own cache is Redis-backed and this file must not require a Redis to + * prove that a write reaches the cache and that a later read is served from it, + * so the repository is handed one directly — the same shape + * `tests/repository.cache-keys.test.ts` uses against a fake client, here over a + * real one. + */ +class RecordingCache { + public gets: string[] = []; + public sets: { key: string; value: any }[] = []; + public invalidated: string[] = []; + public patterns: string[] = []; + public store = new Map(); + public config = { enabled: true, ttl: 60, strategy: "write-through" as const }; + + getStrategy() { + return this.config.strategy; + } + + async get(key: string): Promise { + this.gets.push(key); + return (this.store.get(key) as T) ?? null; + } + + async set(key: string, value: T): Promise { + this.sets.push({ key, value }); + this.store.set(key, value); + } + + async invalidate(keys: string[]): Promise { + this.invalidated.push(...keys); + for (const key of keys) this.store.delete(key); + } + + async invalidatePattern(pattern: string): Promise { + this.patterns.push(pattern); + const prefix = pattern.replace(/\*$/, ""); + for (const key of [...this.store.keys()]) { + if (key.startsWith(prefix)) this.store.delete(key); + } + } +} + +/** One stored history row, oldest first. */ +const storedHistory = (db: any, id: number) => + db.client.mongoFind(HISTORY, { id }, { sort: { version: 1 } }); + +const storedRow = (db: any, id: number) => + db.client.mongoFindOne("v7_posts", { _id: id }); + +suite("mongo versioning", () => { + let db: any; + + const reset = async () => { + for (const name of COLLECTIONS) { + await db.client.mongoDeleteMany(name, {}); + } + // Only this file's own counter documents. `stabilize_counters` is shared + // with every other suite running beside this one, so dropping it would take + // their counters — and their next generated id — with it. + for (const table of ["v7_posts", "v7_secrets"]) { + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: table, + }); + } + }; + + beforeAll(async () => { + // Encrypted columns need a key; there is no hard-coded fallback. + process.env.ORM_ENCRYPTION_KEY = "test-key-32-bytes-long-padding!!"; + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await reset(); + // `autoMigrate` is what creates the history collection, so a versioned model + // that never migrates would fail on its first write rather than on its first + // read. That it also installs the window's validator is proved below. + await db.autoMigrate([Post, Secret]); + }); + + afterAll(async () => { + if (!db) return; + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } + await reset().catch(() => {}); + await db.close(); + }); + + beforeEach(reset); + + // ─── the history collection ──────────────────────────────────────── + + it("creates the history collection with its validator and indexes", async () => { + const names = (await db.client.mongoListCollections()).map( + (entry: any) => entry.name, + ); + expect(names).toContain(HISTORY); + + const indexes = await db.client.mongoListIndexes(HISTORY); + // The index the "as of" window ranges over. Without it the comparison still + // answers correctly and stops being a one-row lookup. + expect( + indexes.some( + (index: any) => index.key?.id === 1 && index.key?.valid_from === 1, + ), + ).toBe(true); + expect( + indexes.some( + (index: any) => index.key?.id === 1 && index.key?.version === 1, + ), + ).toBe(true); + }); + + it("refuses a history row whose validity window is a string", async () => { + // The SQL path binds `new Date().toISOString()`. Doing that here would fail + // this validator (which is why `writeHistory` does not), and would have been + // accepted silently by a collection with no validator — leaving an "as of" + // range query comparing strings. + let rejected = false; + try { + await db.client.mongoInsertOne(HISTORY, { + _id: { id: 7001, version: 1 }, + id: 7001, + title: "stringly typed", + version: 1, + operation: "insert", + valid_from: new Date().toISOString(), + modified_at: new Date().toISOString(), + }); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + expect(await db.client.mongoCount(HISTORY, { id: 7001 })).toBe(0); + }); + + // ─── writing history ─────────────────────────────────────────────── + + it("records a create as version 1, keyed by column name", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ + title: "first", + subtitle: "sub", + }); + expect(created.id).toBe(1); + + const rows = await storedHistory(db, created.id); + expect(rows).toHaveLength(1); + + const [row] = rows; + expect(row.operation).toBe("insert"); + expect(row.version).toBe(1); + expect(row.title).toBe("first"); + // The renamed column, under its *column* name. A history document written + // under the property key would be invisible to every reader of it. + expect(row.subtitle_col).toBe("sub"); + expect(row.subtitle).toBeUndefined(); + expect(row.modified_by).toBe("system"); + + // Native dates, both of them. @see the string case above. + expect(row.valid_from).toBeInstanceOf(Date); + expect(row.modified_at).toBeInstanceOf(Date); + // The open window is an absent field, not a stored null — the same "absence + // means null" shape every filter in this backend reads. + expect("valid_to" in row).toBe(false); + + // The compound `_id` that makes one *version* unique when the row's own key + // is not. + expect(row._id).toEqual({ id: 1, version: 1 }); + }); + + it("records every version a row goes through, in order", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "first", subtitle: "sub" }); + await repo.update(created.id, { title: "second", subtitle: "sub2" }); + await repo.update(created.id, { title: "third", subtitle: "sub3" }); + + const history: any[] = await repo.history(created.id); + + expect(history).toHaveLength(3); + expect(history.map((row) => row.version)).toEqual([1, 2, 3]); + expect(history.map((row) => row.operation)).toEqual([ + "insert", + "update", + "update", + ]); + expect(history.map((row) => row.title)).toEqual([ + "first", + "second", + "third", + ]); + expect(history.map((row) => row.subtitle_col)).toEqual([ + "sub", + "sub2", + "sub3", + ]); + + // Ascending, and not merely by accident: a history read that does not ask + // for an order has none to give. + expect(history[0].valid_from.getTime()).toBeLessThanOrEqual( + history[1].valid_from.getTime(), + ); + expect(history[1].valid_from.getTime()).toBeLessThanOrEqual( + history[2].valid_from.getTime(), + ); + // The identity is a field on the row, not the compound `_id` object. + expect(history.map((row) => row.id)).toEqual([1, 1, 1]); + expect(history[0]._id).toBeUndefined(); + }); + + it("returns an empty history for a row that has none", async () => { + const repo = db.getRepository(Post); + expect(await repo.history(4242)).toEqual([]); + }); + + it("records a delete as its own version", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "doomed" }); + await repo.delete(created.id); + + const history: any[] = await repo.history(created.id); + // Its own version, not a second row under the version the insert recorded. + // The compound key makes the reuse a duplicate-key error rather than a + // second row, so the delete has to be recorded as the next version. + expect(history.map((row) => row.version)).toEqual([1, 2]); + expect(history.map((row) => row.operation)).toEqual(["insert", "delete"]); + expect(history[1].title).toBe("doomed"); + expect(history[0].operation).toBe("insert"); + expect(await storedRow(db, created.id)).toBeNull(); + }); + + // ─── asOf ────────────────────────────────────────────────────────── + + it("returns the version that was current at the instant asked for", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "first", subtitle: "sub" }); + + // The instant between the two writes. The clock is coarse enough that a + // same-millisecond update would land inside the window and make this + // ambiguous, so the writes are separated rather than raced. + await tick(); + const between = new Date(); + await tick(); + + await repo.update(created.id, { title: "second", subtitle: "sub2" }); + + const then: any = await repo.asOf(created.id, between); + expect(then.title).toBe("first"); + expect(then.subtitle_col).toBe("sub"); + expect(then.version).toBe(1); + + const now: any = await repo.asOf(created.id, new Date()); + expect(now.title).toBe("second"); + expect(now.version).toBe(2); + + // Before the row existed there is no version to return — not the oldest one, + // which is what a filter that had lost its lower bound would hand back. + expect(await repo.asOf(created.id, new Date(0))).toBeNull(); + expect(await repo.asOf(999, new Date())).toBeNull(); + }); + + it("accepts an ISO string for the instant, rather than comparing types", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "first" }); + await tick(); + const between = new Date().toISOString(); + await tick(); + await repo.update(created.id, { title: "second" }); + + // A string compared against a stored date is a comparison between BSON + // *types*, which MongoDB orders by kind before value — every date would look + // older than every string, and the window would match the newest version for + // any instant. `asOf` is typed as taking a `Date`; the coercion is what the + // SQL path's ISO-string binding needs replacing, so it is asserted rather + // than assumed. + const then: any = await repo.asOf(created.id, between as any); + expect(then.title).toBe("first"); + }); + + // ─── rollback ────────────────────────────────────────────────────── + + it("restores a version's values and advances the version", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "first", subtitle: "sub" }); + await repo.update(created.id, { title: "second", subtitle: "sub2" }); + const third: any = await repo.update(created.id, { + title: "third", + subtitle: "sub3", + }); + expect(third.version).toBe(3); + + const restored: any = await repo.rollback(created.id, 1); + + // The old values, every column of them — the renamed one included, which is + // only reachable because the history row is keyed by column name. + expect(restored.title).toBe("first"); + expect(restored.subtitle_col).toBe("sub"); + // And the version moved *forward*: past the newest version recorded, which + // is what makes the restore a version of its own rather than a rewind. + expect(restored.version).toBe(4); + + // Asserted against the collection too, not just against what the call + // returned. + const raw = await storedRow(db, created.id); + expect(raw.title).toBe("first"); + expect(raw.subtitle_col).toBe("sub"); + expect(raw.version).toBe(4); + + const history: any[] = await repo.history(created.id); + expect(history.map((row) => row.version)).toEqual([1, 2, 3, 4]); + expect(new Set(history.map((row) => row.version)).size).toBe(4); + expect(history.map((row) => row.title)).toEqual([ + "first", + "second", + "third", + "first", + ]); + expect(history[3].operation).toBe("update"); + // The audit of what was rolled *over* survives: version 3 still records + // "third". + expect(history[2].title).toBe("third"); + + // The next ordinary write carries on from the advanced version rather than + // colliding with it. + const next: any = await repo.update(created.id, { title: "fourth" }); + expect(next.version).toBe(5); + expect((await repo.history(created.id)).at(-1).version).toBe(5); + }); + + it("clears a column the restored version never had", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "bare" }); + await repo.update(created.id, { subtitle: "added" }); + expect((await storedRow(db, created.id)).subtitle_col).toBe("added"); + + await repo.rollback(created.id, 1); + + // The row is restored to a *state*, and a column that state does not have is + // part of it — leaving the later value in place would restore the version's + // title and keep someone else's subtitle. + expect("subtitle_col" in (await storedRow(db, created.id))).toBe(false); + }); + + it("reports a version it has never recorded", async () => { + const repo = db.getRepository(Post); + const created: any = await repo.create({ title: "only" }); + + // try/catch rather than `.rejects`: a rejection assertion that never settles + // leaves Bun's runner hanging. + let code = ""; + let message = ""; + try { + await repo.rollback(created.id, 99); + } catch (error) { + code = (error as any).code; + message = (error as Error).message; + } + + expect(code).toBe("ROLLBACK_ERROR"); + expect(message).toBe("Version not found"); + // Nothing was written: the row still holds what it held. + const raw = await storedRow(db, created.id); + expect(raw.title).toBe("only"); + expect(raw.version).toBe(1); + expect(await repo.history(created.id)).toHaveLength(1); + }); + + it("refuses the three reads on a model that is not versioned", async () => { + // The guard sits above the dispatch, so a MongoDB model reaches the same + // error every other backend raises rather than a query with no history + // collection behind it. + const repo = db.getRepository(Secret); + const codes: string[] = []; + for (const call of [ + () => repo.asOf(1, new Date()), + () => repo.history(1), + () => repo.rollback(1, 1), + ]) { + try { + await call(); + } catch (error) { + codes.push((error as any).code); + } + } + expect(codes).toEqual([ + "VERSIONING_ERROR", + "VERSIONING_ERROR", + "VERSIONING_ERROR", + ]); + }); + + // ─── encryption ──────────────────────────────────────────────────── + + it("round-trips an encrypted column through create and read-back", async () => { + const repo = db.getRepository(Secret); + const created: any = await repo.create({ + label: "employee", + ssn: "123-45-6789", + }); + + // The stored bytes are ciphertext... + const raw = await db.client.mongoFindOne("v7_secrets", { _id: created.id }); + expect(raw.ssn).not.toBe("123-45-6789"); + expect(decrypt(raw.ssn)).toBe("123-45-6789"); + + // ...and every read of it is plaintext, through the read-back `create` + // returns, through `findOne`, and through the query builder. + expect(created.ssn).toBe("123-45-6789"); + expect((await repo.findOne(created.id))?.ssn).toBe("123-45-6789"); + expect( + (await repo.find().whereEq("v7_secrets.id", created.id).execute(db.client))[0] + .ssn, + ).toBe("123-45-6789"); + + // A failed decrypt is raised rather than turned into a null the caller + // cannot tell from an empty field. + await db.client.mongoUpdateOne( + "v7_secrets", + { _id: created.id }, + { $set: { ssn: "not-ciphertext" } }, + ); + let code = ""; + try { + await repo.findOne(created.id); + } catch (error) { + code = (error as any).code; + } + expect(code).toBe("DECRYPTION_ERROR"); + }); + + it("updates an encrypted column without losing the previous ciphertext", async () => { + const repo = db.getRepository(Secret); + const created: any = await repo.create({ label: "x", ssn: "111-11-1111" }); + + const updated: any = await repo.update(created.id, { ssn: "222-22-2222" }); + + expect(updated.ssn).toBe("222-22-2222"); + const raw = await db.client.mongoFindOne("v7_secrets", { _id: created.id }); + expect(decrypt(raw.ssn)).toBe("222-22-2222"); + }); + + // ─── cache ───────────────────────────────────────────────────────── + + it("writes a created row through to the cache and serves it back", async () => { + const cache = new RecordingCache(); + const repo: any = new Repository( + db.client, + Post, + cache.config as any, + undefined, + cache as any, + ); + + const created: any = await repo.create({ title: "cached", subtitle: "s" }); + const key = `findOne:v7_posts:${created.id}`; + + // Write-through: the row the caller was handed is the row the cache holds, + // so a read that hits the cache gets a decrypted row rather than ciphertext + // or a raw column set. + expect(cache.store.get(key)?.[0]?.title).toBe("cached"); + expect(cache.store.get(key)?.[0]?.subtitle_col).toBe("s"); + + // Proves the cache was *consulted*: the row behind it is changed, and the + // read still answers with what was cached. + await db.client.mongoUpdateOne( + "v7_posts", + { _id: created.id }, + { $set: { title: "behind-the-cache" } }, + ); + expect((await repo.findOne(created.id)).title).toBe("cached"); + expect(cache.gets).toContain(key); + }); + + it("does not leave a stale cached row behind an update or a delete", async () => { + const cache = new RecordingCache(); + const repo: any = new Repository( + db.client, + Post, + cache.config as any, + undefined, + cache as any, + ); + const created: any = await repo.create({ title: "first" }); + const key = `findOne:v7_posts:${created.id}`; + + await repo.update(created.id, { title: "second" }); + + // The update both invalidates the cached row and writes the new one through, + // so the next read is the new value by either route. + expect(cache.invalidated).toContain(key); + expect(cache.store.get(key)?.[0]?.title).toBe("second"); + expect((await repo.findOne(created.id)).title).toBe("second"); + + await repo.delete(created.id); + + expect(cache.store.get(key)).toBeUndefined(); + expect(await repo.findOne(created.id)).toBeNull(); + }); + + it("leaves the cache holding the row a rollback restored", async () => { + const cache = new RecordingCache(); + const repo: any = new Repository( + db.client, + Post, + cache.config as any, + undefined, + cache as any, + ); + const created: any = await repo.create({ title: "first", subtitle: "sub" }); + const key = `findOne:v7_posts:${created.id}`; + await repo.update(created.id, { title: "second" }); + expect(cache.store.get(key)?.[0]?.title).toBe("second"); + + await repo.rollback(created.id, 1); + + // A rollback writes the live document, so a cached row that survived it + // would serve the pre-rollback value to every later read. + expect(cache.store.get(key)?.[0]?.title).toBe("first"); + expect(cache.store.get(key)?.[0]?.version).toBe(3); + expect((await repo.findOne(created.id)).title).toBe("first"); + }); +}); diff --git a/tests/mongo.write-paths.test.ts b/tests/mongo.write-paths.test.ts new file mode 100644 index 0000000..ecbae38 --- /dev/null +++ b/tests/mongo.write-paths.test.ts @@ -0,0 +1,786 @@ +import { + describe, + it, + expect, + beforeAll, + beforeEach, + afterAll, +} from "vitest"; +import { Stabilize } from "../index"; +import { defineModel } from "../model"; +import { DataTypes, DBType } from "../types"; +import { decrypt } from "../utils/encryption"; +import { MONGO_COUNTERS_COLLECTION } from "../mongo-repository"; + +/** + * The write paths, against a real MongoDB server. + * + * The SQL cases this mirrors (`integration.write-paths.test.ts`) each pinned a + * bug that a fake client could not expose. Re-running them here is what proves + * the MongoDB bodies preserve those fixes rather than re-introducing them in a + * second dialect: hooks still run above the dispatch, `bulkCreate` still writes + * every column of a batch, and `upsert` still resolves the row it actually + * wrote instead of a value that describes an unrelated statement. + * + * The assertions on stored bytes go through `mongoFind`/`mongoFindOne` rather + * than `client.query`, because there is no SQL to send and `query()` refuses + * outright on this backend. + * + * The suite skips itself when the replica set is not running, so `bun test` + * stays green on a machine without the fleet: + * + * docker compose -f docker-compose.test.yml up -d --wait + */ + +const REPLICA_SET_URL = + process.env.MONGO_URL || + "mongodb://127.0.0.1:57017/stabilize_test?directConnection=true&replicaSet=rs0"; + +/** The driver import must stay inside the try. It is an optional dependency. */ +async function hasReplicaSet(url: string): Promise { + let client: any = null; + try { + const { MongoClient } = await import("mongodb"); + client = new MongoClient(url, { serverSelectionTimeoutMS: 3000 }); + await client.connect(); + const hello = await client.db().admin().command({ hello: 1 }); + return Boolean(hello.setName); + } catch { + return false; + } finally { + await client?.close().catch(() => {}); + } +} + +const available = await hasReplicaSet(REPLICA_SET_URL); +const suite = available ? describe : describe.skip; + +if (!available) { + console.warn( + `[skip] No replica-set MongoDB at ${REPLICA_SET_URL}. ` + + `Run: docker compose -f docker-compose.test.yml up -d --wait`, + ); +} + +/** Records every hook invocation so the tests can assert on ordering. */ +const calls: string[] = []; + +const Doc = defineModel({ + tableName: "w5_docs", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + // Deliberately required and deliberately never supplied by the caller: the + // `beforeCreate` hook is what fills it in. + slug: { type: DataTypes.STRING, required: true }, + tag: { type: DataTypes.STRING }, + }, + hooks: { + beforeCreate: (entity: any) => { + entity.slug = String(entity.title).toLowerCase().replace(/\s+/g, "-"); + }, + afterCreate: (entity: any) => { + calls.push(`afterCreate:${entity.id}`); + }, + beforeUpdate: (entity: any) => { + calls.push(`beforeUpdate:${entity.id}`); + entity.tag = `${entity.tag}-touched`; + }, + afterSave: (entity: any) => { + calls.push(`afterSave:${entity.id}`); + }, + beforeDelete: (entity: any) => { + calls.push(`beforeDelete:${entity.id}`); + }, + afterDelete: (entity: any) => { + calls.push(`afterDelete:${entity.id}`); + }, + }, +}); + +// A hook declared as a class method rather than in the config. It only resolves +// on a real model instance, which is what `hydrate` provides. +(Doc.prototype as any).afterUpdate = function () { + calls.push(`methodAfterUpdate:${this.id}`); +}; + +const Account = defineModel({ + tableName: "w5_accounts", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + email: { type: DataTypes.STRING, unique: true }, + name: { type: DataTypes.STRING }, + nickname: { type: DataTypes.STRING }, + }, +}); + +const Secret = defineModel({ + tableName: "w5_secrets", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + label: { type: DataTypes.STRING }, + ssn: { type: DataTypes.STRING, encrypted: true }, + }, +}); + +/** Timestamps, an optimistic lock and both bulk operations. */ +const Note = defineModel({ + tableName: "w5_notes", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true }, + body: { type: DataTypes.STRING }, + rev: { type: DataTypes.INTEGER, optimisticLock: true }, + createdAt: { type: DataTypes.DATETIME }, + updatedAt: { type: DataTypes.DATETIME }, + }, + timestamps: { createdAt: "createdAt", updatedAt: "updatedAt" }, +}); + +const COLLECTIONS = [ + "w5_docs", + "w5_accounts", + "w5_secrets", + "w5_notes", + "w5_tasks", + MONGO_COUNTERS_COLLECTION, +]; + +const dropAll = async (db: any) => { + // Dropped rather than emptied: the counters and the indexes are what make + // `id === 1` and the unique-email assertions mean anything, and both survive + // a `deleteMany`. + for (const name of COLLECTIONS) { + await db.client.mongoCommand({ drop: name }).catch(() => {}); + } +}; + +describe("mongo write paths", () => { + let db: any; + + beforeAll(async () => { + // Encrypted columns need a key; there is no hard-coded fallback. + process.env.ORM_ENCRYPTION_KEY = "test-key-32-bytes-long-padding!!"; + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + await dropAll(db); + await db.autoMigrate([Doc, Account, Secret, Note]); + }); + + afterAll(async () => { + if (!db) return; + await dropAll(db).catch(() => {}); + await db.close(); + }); + + // ─── hooks ───────────────────────────────────────────────────────── + + it("persists what a beforeCreate hook wrote", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Hello World" }); + + // The hook supplied `slug`, which is required — validation runs after the + // hooks, so the insert succeeds and the value is actually written. + expect(created.slug).toBe("hello-world"); + expect((await repo.findOne(created.id))?.slug).toBe("hello-world"); + + const raw = await db.client.mongoFindOne("w5_docs", { _id: created.id }); + expect(raw.slug).toBe("hello-world"); + }); + + it("runs afterCreate and afterSave with the created entity", async () => { + const repo = db.getRepository(Doc); + calls.length = 0; + + const created: any = await repo.create({ title: "Second" }); + + expect(calls).toContain(`afterCreate:${created.id}`); + expect(calls).toContain(`afterSave:${created.id}`); + }); + + it("runs beforeUpdate, the class-method afterUpdate and afterSave on update", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Third", tag: "x" }); + calls.length = 0; + + const updated: any = await repo.update(created.id, { title: "Third!" }); + + expect(calls).toContain(`beforeUpdate:${created.id}`); + expect(calls).toContain(`afterSave:${created.id}`); + expect(calls).toContain(`methodAfterUpdate:${created.id}`); + // The beforeUpdate mutation has to reach the database. + expect(updated.tag).toBe("x-touched"); + const raw = await db.client.mongoFindOne("w5_docs", { _id: created.id }); + expect(raw.tag).toBe("x-touched"); + }); + + it("runs the delete hooks", async () => { + const repo = db.getRepository(Doc); + const created: any = await repo.create({ title: "Fourth" }); + calls.length = 0; + + await repo.delete(created.id); + + expect(calls).toContain(`beforeDelete:${created.id}`); + expect(calls).toContain(`afterDelete:${created.id}`); + expect(await db.client.mongoFindOne("w5_docs", { _id: created.id })).toBeNull(); + }); + + // ─── bulk create ─────────────────────────────────────────────────── + + it("writes every column of a bulk batch, not just the first row's", async () => { + const repo = db.getRepository(Doc); + calls.length = 0; + + const created = await repo.bulkCreate([ + { title: "narrow", slug: "narrow" }, + { title: "wide", slug: "wide", tag: "kept" }, + ]); + + // The second row carries a column the first does not. Documents being + // independent makes this free here, but the SQL path had to union the key + // sets, and dropping that union is exactly how the bug would come back. + const wide = created.find((doc: any) => doc.title === "wide"); + expect(wide?.tag).toBe("kept"); + + const raw = await db.client.mongoFindOne("w5_docs", { title: "wide" }); + expect(raw.tag).toBe("kept"); + expect("tag" in (await db.client.mongoFindOne("w5_docs", { title: "narrow" }))).toBe( + false, + ); + }); + + it("returns the rows a bulk insert actually created", async () => { + const repo = db.getRepository(Doc); + const created = await repo.bulkCreate([ + { title: "a1", slug: "a1" }, + { title: "a2", slug: "a2" }, + { title: "a3", slug: "a3" }, + ]); + + expect(created).toHaveLength(3); + expect(created.map((doc: any) => doc.title)).toEqual(["a1", "a2", "a3"]); + + // Every returned id must be a row that holds the matching value. On SQL + // this needed `ORDER BY id DESC LIMIT n` to be replaced by something that + // actually named the rows; here the ids are allocated up front, so the + // assertion is that the allocation and the documents agree. + for (const doc of created) { + const raw = await db.client.mongoFindOne("w5_docs", { _id: doc.id }); + expect(raw.title).toBe(doc.title); + } + }); + + // ─── upsert ──────────────────────────────────────────────────────── + + it("returns the row an upsert actually wrote", async () => { + const repo = db.getRepository(Account); + const first: any = await repo.upsert( + { email: "a@example.com", name: "first" }, + ["email"], + ); + + // An unrelated insert, which is what moved the connection's + // last-inserted-rowid on the SQL path. + await repo.create({ email: "b@example.com", name: "other" }); + + const upserted: any = await repo.upsert( + { email: "a@example.com", name: "second" }, + ["email"], + ); + + expect(upserted.id).toBe(first.id); + expect(upserted.name).toBe("second"); + expect(upserted.email).toBe("a@example.com"); + + const other = await repo.findOneBy({ email: "b@example.com" }); + expect(other?.name).toBe("other"); + }); + + it("treats an upsert onto an existing key as an update", async () => { + const repo = db.getRepository(Account); + await repo.upsert({ email: "c@example.com", name: "one" }, ["email"]); + const before = await repo.count(); + + const second: any = await repo.upsert( + { email: "c@example.com", name: "two" }, + ["email"], + ); + + expect(await repo.count()).toBe(before); + expect(second.name).toBe("two"); + expect(await repo.find().execute(db.client)).toHaveLength(before); + }); + + it("encrypts a column written through bulkCreate", async () => { + const repo = db.getRepository(Secret); + await repo.bulkCreate([ + { label: "one", ssn: "111-11-1111" }, + { label: "two", ssn: "222-22-2222" }, + ]); + + const raw = await db.client.mongoFind("w5_secrets", {}, { sort: { _id: 1 } }); + expect(raw[0].ssn).not.toBe("111-11-1111"); + expect(decrypt(raw[0].ssn)).toBe("111-11-1111"); + expect(decrypt(raw[1].ssn)).toBe("222-22-2222"); + }); +}); + +describe("mongo bulk operation guards", () => { + let db: any; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + await dropAll(db); + await db.autoMigrate([Note]); + }); + + afterAll(async () => { + if (!db) return; + await dropAll(db).catch(() => {}); + await db.close(); + }); + + it("advances updatedAt on updateBy", async () => { + const repo = db.getRepository(Note); + const created: any = await repo.create({ title: "one", body: "x" }); + + // The clock is coarse enough that a same-millisecond write is possible. + await new Promise((resolve) => setTimeout(resolve, 5)); + const affected = await repo.updateBy({ title: "one" }, { body: "y" }); + + expect(affected).toBe(1); + const after: any = await repo.findOne(created.id); + expect(after.body).toBe("y"); + expect(after.updatedAt).toBeTruthy(); + expect(after.updatedAt).not.toBe(created.updatedAt); + }); + + it("advances the optimistic lock on updateBy", async () => { + const repo = db.getRepository(Note); + const created: any = await repo.create({ title: "locked-here" }); + + await repo.updateBy({ title: "locked-here" }, { body: "z" }); + + const after: any = await repo.findOne(created.id); + expect(after.rev).toBe(2); + + // The row must still be writable: a stale version would reject this. + const updated: any = await repo.update(created.id, { body: "z2" }); + expect(updated.rev).toBe(3); + }); + + it("reports a genuine optimistic-lock conflict", async () => { + // The other half of the lock: matching on the version is only meaningful if + // a stale version is actually refused. + const repo = db.getRepository(Note); + const created: any = await repo.create({ title: "contested" }); + expect(created.rev).toBe(1); + + let caught: any = null; + try { + // The version the caller read, after someone else has already moved it. + await repo.update(created.id, { body: "stale", rev: 1 } as any); + await repo.update(created.id, { body: "staler", rev: 1 } as any); + } catch (error) { + caught = error; + } + + expect(caught?.code).toBe("CONCURRENT_MODIFICATION"); + expect((await repo.findOne(created.id))?.body).toBe("stale"); + }); + + it("refuses updateBy with no conditions", async () => { + const repo = db.getRepository(Note); + await repo.create({ title: "survivor" }); + const before = await repo.count(); + + // try/catch rather than `.rejects`: a rejection assertion that never + // settles leaves Bun's runner hanging. + let message = ""; + try { + await repo.updateBy({}, { body: "wiped" }); + } catch (error) { + message = (error as Error).message; + } + + expect(message).toMatch(/at least one condition/i); + expect(await repo.count()).toBe(before); + expect(await repo.find().execute(db.client)).toHaveLength(before); + }); + + it("refuses deleteBy with no conditions", async () => { + const repo = db.getRepository(Note); + const before = await repo.count(); + + let message = ""; + try { + await repo.deleteBy({}); + } catch (error) { + message = (error as Error).message; + } + + expect(message).toMatch(/at least one condition/i); + expect(await repo.count()).toBe(before); + }); + + it("deletes only the rows deleteBy matches", async () => { + const repo = db.getRepository(Note); + await repo.create({ title: "delete-me" }); + await repo.create({ title: "keep-me" }); + const before = await repo.count(); + + const affected = await repo.deleteBy({ title: "delete-me" }); + + expect(affected).toBe(1); + expect(await repo.count()).toBe(before - 1); + expect(await repo.exists({ title: "keep-me" })).toBe(true); + }); +}); + +/** + * The soft-deleting model for the operations below. + * + * `deleted_at` carries the flag, and the two columns exist to give the + * column-level operations something with a shape of its own: `done` is a + * boolean, which this backend stores as a boolean rather than as 1/0, and + * `status` is the column the "clear it" case unsets. + */ +const Task = defineModel({ + tableName: "w5_tasks", + columns: { + id: { type: DataTypes.INTEGER, required: true }, + title: { type: DataTypes.STRING, required: true, index: "idx_w5_task_title" }, + status: { type: DataTypes.STRING }, + done: { type: DataTypes.BOOLEAN }, + views: { type: DataTypes.INTEGER }, + score: { type: DataTypes.INTEGER }, + deleted_at: { type: DataTypes.DATETIME, softDelete: true }, + }, +}); + +/** + * The column-level and set-level write operations, which the ported suite above + * does not reach. + * + * A silently-wrong filter is the failure mode this backend is most exposed to: + * it returns the wrong rows and reports no error, so nothing downstream is in a + * position to notice. These cases each pin a filter down to one row or to an + * exact count for that reason. + */ +describe("mongo write operations", () => { + let db: any; + + /** Each case starts from an empty collection and a counter back at zero. */ + const reset = async () => { + await db.client.mongoDeleteMany("w5_tasks", {}); + await db.client.mongoDeleteMany(MONGO_COUNTERS_COLLECTION, { + _id: "w5_tasks", + }); + }; + + beforeAll(async () => { + db = new Stabilize({ + type: DBType.MongoDB, + connectionString: REPLICA_SET_URL, + }); + await db.client.mongoCommand({ ping: 1 }); + await dropAll(db); + await db.autoMigrate([Task]); + }); + + beforeEach(reset); + + afterAll(async () => { + if (!db) return; + await dropAll(db).catch(() => {}); + await db.close(); + }); + + // ─── soft delete ─────────────────────────────────────────────────── + + it("soft deletes, hides the row, and recovers it", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "soft" }); + + await repo.delete(created.id); + + expect(await repo.findOne(created.id)).toBeNull(); + expect( + (await repo.findDeleted().execute(db.client)).some( + (row: any) => row.id === created.id, + ), + ).toBe(true); + + // Deleted, not removed: the document is still there, carrying the stamp. + const raw = await db.client.mongoFindOne("w5_tasks", { _id: created.id }); + expect(raw).not.toBeNull(); + expect(raw.deleted_at).toBeInstanceOf(Date); + + await repo.recover(created.id); + expect((await repo.findOne(created.id))?.title).toBe("soft"); + + // `$unset`, not a stored null — the same "absent means not deleted" shape + // every other filter in this backend reads. + const recovered = await db.client.mongoFindOne("w5_tasks", { + _id: created.id, + }); + expect("deleted_at" in recovered).toBe(false); + }); + + it("bulk deletes softly and leaves the rows in place", async () => { + const repo = db.getRepository(Task); + const first: any = await repo.create({ title: "b1" }); + const second: any = await repo.create({ title: "b2" }); + + await repo.bulkDelete([first.id, second.id]); + + expect(await repo.count()).toBe(0); + expect(await db.client.mongoCount("w5_tasks", {})).toBe(2); + }); + + it("restores by condition and refuses to touch anything else", async () => { + const repo = db.getRepository(Task); + const kept: any = await repo.create({ title: "kept" }); + const restored: any = await repo.create({ title: "restored" }); + await repo.delete(kept.id); + await repo.delete(restored.id); + + const affected = await repo.restoreBy({ title: "restored" }); + + expect(affected).toBe(1); + expect((await repo.findOne(restored.id))?.title).toBe("restored"); + expect(await repo.findOne(kept.id)).toBeNull(); + }); + + // ─── column-level operations ─────────────────────────────────────── + + it("clears a column when a patch passes null", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "cleared", status: "open" }); + expect( + (await db.client.mongoFindOne("w5_tasks", { _id: created.id })).status, + ).toBe("open"); + + await repo.update(created.id, { status: null } as any); + + // SQL spells this `SET status = NULL`. A document store spells it `$unset`, + // and dropping the key from the patch instead would have left "open" in + // place while reporting success. + const raw = await db.client.mongoFindOne("w5_tasks", { _id: created.id }); + expect("status" in raw).toBe(false); + expect((await repo.findOne(created.id))?.status ?? null).toBeNull(); + }); + + it("stores a boolean as a boolean, not as 1 and 0", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "typed", done: true }); + + await repo.update(created.id, { done: false } as any); + + // Every SQL backend coerces a boolean to 1|0, and reusing that coercion + // here would fail a `{bsonType: "bool"}` validator and make + // `whereEq("done", true)` match nothing — a read that answers "no rows" + // for rows that are plainly there. + const raw = await db.client.mongoFindOne("w5_tasks", { _id: created.id }); + expect(raw.done).toBe(false); + }); + + it("toggles a boolean in place", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "toggled", done: false }); + + const on: any = await repo.toggle(created.id, "done"); + expect(on.done).toBe(true); + const off: any = await repo.toggle(created.id, "done"); + expect(off.done).toBe(false); + + // The round trip through the driver, not just the value handed back. + expect( + (await db.client.mongoFindOne("w5_tasks", { _id: created.id })).done, + ).toBe(false); + }); + + it("increments and decrements a counter column", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "counted", views: 5 }); + + expect((await repo.increment(created.id, "views", 3)).views).toBe(8); + expect((await repo.decrement(created.id, "views", 2)).views).toBe(6); + }); + + it("will not increment a soft-deleted row", async () => { + // The filter the SQL path carries as `AND deleted_at IS NULL`. Dropping it + // would let a write reach a row every read says is gone. + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "gone", views: 1 }); + await repo.delete(created.id); + + await repo.increment(created.id, "views", 10); + + const raw = await db.client.mongoFindOne("w5_tasks", { _id: created.id }); + expect(raw.views).toBe(1); + }); + + // ─── set-level reads and writes ──────────────────────────────────── + + it("aggregates over the collection", async () => { + const repo = db.getRepository(Task); + await repo.create({ title: "a", score: 10 }); + await repo.create({ title: "b", score: 20 }); + + const aggregated = await repo.aggregate({ + count: "*", + sum: ["score"], + avg: ["score"], + min: ["score"], + max: ["score"], + }); + + // Same aliases the SQL path produces: the caller reads the answer out of + // these names, so they are part of the contract rather than a detail. + expect(aggregated.count_all).toBe(2); + expect(aggregated.sum_score).toBe(30); + expect(aggregated.avg_score).toBe(15); + expect(aggregated.min_score).toBe(10); + expect(aggregated.max_score).toBe(20); + }); + + it("aggregates an empty collection into zeroes rather than nothing", async () => { + const aggregated = await db.getRepository(Task).aggregate({ + count: "*", + sum: ["score"], + }); + + // `$group` over an empty input produces no documents at all, where SQL's + // aggregate query still returns one row. Reporting `undefined` for the + // count of nothing would make `count_all + 1` come out `NaN` — on the one + // input a fresh install always has. + expect(aggregated.count_all).toBe(0); + expect(aggregated.sum_score).toBeNull(); + }); + + it("aggregates nothing into an empty object", async () => { + expect(await db.getRepository(Task).aggregate({})).toEqual({}); + }); + + it("counts distinct values, skipping the ones that are null", async () => { + const repo = db.getRepository(Task); + await repo.create({ title: "d1", status: "open" }); + await repo.create({ title: "d2", status: "open" }); + await repo.create({ title: "d3", status: "closed" }); + await repo.create({ title: "d4" }); // no status at all + + expect(await repo.countDistinct("status")).toBe(2); + + // SQL's `COUNT(DISTINCT …)` counts values, and a stored null is not one. + // Mongo's `distinct` hands it back like any other value, so this count + // would be 2 — one too many — without the explicit filter. Written raw + // because the write path never stores a null: it unsets the key instead. + await db.client.mongoInsertOne("w5_tasks", { + _id: 900, + title: "raw", + loose: null, + }); + await db.client.mongoInsertOne("w5_tasks", { + _id: 901, + title: "raw", + loose: 7, + }); + + expect(await repo.countDistinct("loose")).toBe(1); + }); + + it("paginates with a total that excludes soft-deleted rows", async () => { + const repo = db.getRepository(Task); + await repo.create({ title: "p1" }); + await repo.create({ title: "p2" }); + await repo.create({ title: "p3" }); + const gone: any = await repo.create({ title: "p4" }); + await repo.delete(gone.id); + + const page = await repo.paginate(1, 2); + + // The count and the page have to agree on what a row is. A `COUNT(*)` + // that forgot the soft-delete clause would report 4 here. + expect(page.total).toBe(3); + expect(page.data).toHaveLength(2); + expect(page.page).toBe(1); + expect(page.pageSize).toBe(2); + }); + + it("returns a row at random, and null when there are none", async () => { + const repo = db.getRepository(Task); + expect(await repo.random()).toBeNull(); + + const only: any = await repo.create({ title: "only" }); + const picked: any = await repo.random(); + expect(picked?.id).toBe(only.id); + expect(picked?.title).toBe("only"); + }); + + it("pages through a cursor", async () => { + const repo = db.getRepository(Task); + await repo.create({ title: "c1" }); + await repo.create({ title: "c2" }); + await repo.create({ title: "c3" }); + + const first = await repo.findMany({ + orderBy: { field: "id", direction: "ASC" }, + take: 2, + }); + expect(first.map((row: any) => row.id)).toEqual([1, 2]); + + const next = await repo.findMany({ + cursor: { field: "id", value: first[1].id }, + orderBy: { field: "id", direction: "ASC" }, + take: 2, + }); + expect(next.map((row: any) => row.id)).toEqual([3]); + }); + + it("truncates without dropping the model's indexes or validator", async () => { + const repo = db.getRepository(Task); + await repo.create({ title: "doomed" }); + expect(await repo.count()).toBe(1); + + await repo.truncate(); + + expect(await repo.count()).toBe(0); + // `drop()` would be the obvious way to empty a collection and the wrong + // one: it takes the indexes and the validator with it, so the truncated + // collection would silently stop enforcing the model. + const indexes = await db.client.mongoListIndexes("w5_tasks"); + expect(indexes.some((index: any) => index.key?.title === 1)).toBe(true); + + const info = await db.client.mongoCommand({ listCollections: 1 }); + const entry = info.cursor.firstBatch.find( + (each: any) => each.name === "w5_tasks", + ); + expect(entry.options.validationLevel).toBe("moderate"); + }); + + it("refuses bulkUpdate, which needs a raw SQL condition", async () => { + const repo = db.getRepository(Task); + const created: any = await repo.create({ title: "untouched" }); + + let code = ""; + try { + await repo.bulkUpdate([ + { where: { condition: "title = ?", params: ["untouched"] }, set: { status: "x" } }, + ]); + } catch (error) { + code = (error as any).code; + } + + expect(code).toBe("MONGO_UNSUPPORTED"); + expect((await repo.findOne(created.id))?.status ?? null).toBeNull(); + }); +}); diff --git a/types.ts b/types.ts index cfe0947..74f6389 100644 --- a/types.ts +++ b/types.ts @@ -107,6 +107,20 @@ export interface CacheStats { keys: number; } +/** + * The predicate shape a `QueryBuilder` records for MongoDB. + * + * Declared in `mongo-query`, where it is translated into a filter, and + * re-exported here because this module is the package's public type surface: + * `./types` is a published entry point and `./mongo-query` is not, so a consumer + * that wants to name the shape — in a helper that builds conditions, or to + * annotate a custom scope — would otherwise have no way to reach it. + * + * The re-export is type-only and therefore erased, so it adds no runtime edge + * back into `mongo-query` (which imports `StabilizeError` from here). + */ +export type { Predicate } from "./mongo-query"; + /** * One schema change against MongoDB, as data rather than as a closure. * From 247a280b269ad366a576fdad836d392775b95f46 Mon Sep 17 00:00:00 2001 From: ElectronSz Date: Wed, 16 Sep 2026 12:43:38 +0200 Subject: [PATCH 4/5] Release 3.0.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Major version for the MongoDB backend, which is a new supported database rather than a change to the existing four. The version number is the only purely mechanical part: no SQL behaviour changed in this release, so a 2.x consumer that does not opt into MongoDB has no migration to perform. The bump signals that defineModel and the Repository API now carry a backend whose guarantees differ in documented ways — raw SQL is refused, transactions require a replica set, and DECIMAL is stored as a double. The description now names SQL Server and MongoDB, neither of which it mentioned despite both being supported. --- package.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/package.json b/package.json index b99e8e4..7b8e3f2 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "stabilize-orm", - "version": "2.2.1", - "description": "A modern, type-safe ORM for Bun, Node.js, and Deno with support for SQLite, MySQL, PostgreSQL, Redis caching, optimistic locking, cursor pagination, and a full-featured query builder.", + "version": "3.0.0", + "description": "A modern, type-safe ORM for Bun, Node.js, and Deno with support for SQLite, MySQL, PostgreSQL, SQL Server and MongoDB, Redis caching, optimistic locking, cursor pagination, and a full-featured query builder.", "main": "dist/index.js", "types": "dist/index.d.ts", "exports": { From 4cdc64b92f5c37071c018fdfc84b57b430c72d7b Mon Sep 17 00:00:00 2001 From: ElectronSz Date: Wed, 16 Sep 2026 14:04:34 +0200 Subject: [PATCH 5/5] Track stabilize-cli as a real submodule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It was recorded as a bare gitlink with no .gitmodules, so a fresh clone produced an empty stabilize-cli directory with nothing to tell git where the contents should come from. The recorded pointer was also stale: it named 6a92aa13 while the nested repo's HEAD was ac5e810. That commit is a descendant, so the pointer was simply behind rather than pointing at something that never existed — and ac5e810 is already on origin/main, so nothing needed pushing to make it reachable. Adds the missing .gitmodules and advances the pointer. The nested repo's working tree is left as it is: it has uncommitted changes of its own, which belong to its own history rather than to this pointer. --- .gitmodules | 3 +++ stabilize-cli | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) create mode 100644 .gitmodules diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..c73ea03 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "stabilize-cli"] + path = stabilize-cli + url = https://github.com/ElectronSz/stabilize-cli.git diff --git a/stabilize-cli b/stabilize-cli index 6a92aa1..ac5e810 160000 --- a/stabilize-cli +++ b/stabilize-cli @@ -1 +1 @@ -Subproject commit 6a92aa13b49fe874771419a47c7cdf254f85d4fe +Subproject commit ac5e8106f97bdb914af0d2a3108c56db219181ca