diff --git a/AGENTS.md b/AGENTS.md index aad82fd874..5f149be4de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (428 fixtures; 36 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (449 fixtures; 38 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived. diff --git a/CHANGELOG.md b/CHANGELOG.md index 5cfebfbee3..ea04c89a9b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ here.** ## [Unreleased] +**`metamodelVersion` moves to `1.3`.** This release adds vocabulary (`field.duration`, +`validator.exactlyOne`, `@constraintName`, `@condition`, `@maxLength` on `field.enum`, +`@sequenceStart`, `@sequenceName`, `@flattenedColumnPrefix`, the `upperFirst` column rule) and +nothing that loaded before stops +loading. A port that reports `1.2` does not know that vocabulary; upgrade every port together. + ### Added - **A model explorer site, built by `docsSiteFile()`.** The docs site now shows what the @@ -31,15 +37,17 @@ here.** `WARN_REQUIREMENT_OBJECT_UNCLAIMED` row, `""` on every other row, and each finding in a `requirementLedger` row carries its `message`. See `docs/features/explorer-site.md`. -- **Generated record models enforce the four cross-field validators in every port.** - `validator.comparison`, `validator.requiredWhen`, `validator.presentIff` and - `validator.atLeastOne` used to render only as DB `CHECK` constraints, so a store with no `CHECK` - support (a graph or document database) enforced none of them and the application had to repeat the - rules in its writer. The full create/insert model now enforces them with the language's native +- **Generated record models enforce the cross-field validators in every port.** + `validator.comparison`, `validator.requiredWhen`, `validator.presentIff`, + `validator.atLeastOne` and the new `validator.exactlyOne` used to render only as DB `CHECK` + constraints, so a store with no `CHECK` support (a graph or document database) enforced none of + them and the application had to repeat the rules in its writer. The full create/insert model now enforces them with the language's native mechanism: a Zod refinement on the insert schema (TypeScript), a class-level Bean Validation constraint on the create DTO (Java, Kotlin), `IValidatableObject` (C#) and a Pydantic `model_validator` (Python). Semantics mirror the `CHECK` derivation: null-safe, `comparison` passes - when either side is null, `@equals` is read per the gating field's subtype. PATCH and + when either side is null, `@equals` is read per the gating field's subtype, and a rule with a + `@condition` is checked only when the condition is true (false or null lets the record through, as + the `CHECK`'s `NOT (condition) OR (rule)` does). PATCH and partial-update models are unchanged (a partial body cannot see the merged row, so the database `CHECK` remains the enforcement there), and `CHECK` emission is unchanged. A record that violates a declared rule is now rejected at the model where it used to reach the database. Gated by @@ -68,84 +76,49 @@ here.** `attr-shadow-descendant`, `attr-shadow-extend`, `common-attr-conflict`, `common-attr-per-type-after`, `common-attr-per-type-before` and the `attr-ownership/` sub-corpus. Fixes #527. -### Fixed - -- **TypeScript: every `meta` command now loads the providers in `metaobjects.config.ts`.** - `meta export`, `meta deps`, `meta fmt`, `meta types` and `meta upgrade` loaded the core - vocabulary only, so a project's own subtype or attribute failed there while `meta gen` accepted - it. A config that exists but fails to load is now reported as its own error by `meta verify` - (exit 2, in every mode) and `meta prompt-snapshot`, instead of continuing without its providers - and reporting an unknown subtype. -- **Python: `metaobjects fmt` reads the configured providers, and `--provider` adds to them.** - `fmt` ignored `providers:` in `metaobjects.config.yaml` (it also gains `--config`), and - `metaobjects verify --provider` replaced the configured list instead of adding to it. -- **`ERR_COLUMN_NAME_COLLISION` no longer rejects a read-only view whose columns come from `@sql`.** - The 1.2.0 check treated every field of a sourced object as a column MetaObjects creates. For a - `source.rdb @kind: view` with an `@sql` body, the author's SQL defines the columns and the fields - only read them, so two fields on one column are legal and nothing can be written. That object is - now exempt in all four ports (TypeScript, Java, C#, Python), provided every source of the object - is read-only. A derived view (no `@sql`), which emits a select list from its fields, and an object - with a writable table source plus an `@sql` view replica are still checked. When both colliding fields already carry an - explicit `@column`, the message now says to rename one field or point one `@column` at a different - column, instead of suggesting a `@column` they already have. The error code is unchanged. New - conformance fixtures: `column-name-shared-in-sql-view-ok`, `error-column-name-collision-derived-view` and - `error-column-name-collision-table-with-sql-view`. -- **C# codegen: an entity that extends another no longer loses the parent's properties.** The member - loop skipped every inherited field but the class declaration never named the base, so the fields - appeared nowhere. A child of a **concrete** parent now declares the parent's properties itself and - names no base: each concrete entity owns a table with all its columns (what `meta migrate` creates - and the other ports' models carry), and `class Child : Parent` between two mapped entities is an EF - inheritance hierarchy (one shared table, or a child table holding only its own columns). A child of - an **abstract** parent with `EmitAbstractShapes` on now emits `class Child : Base`, as the - generator always meant to, and declares only its own members. Regenerating changes those two shapes - of C# output only. An adopter override of `EmitClassDeclarationLine` that names a concrete parent as - the base now gets that parent's properties twice; drop the base from the override. -- **Test flakes fixed at their cause.** `GenCommandTests.Gen_writes_the_hash_manifest_beside_the_metadata_not_in_cwd` - (C# Cli) failed about one run in five because `DependencyCliTests` runs `gen` in process, which - changes the process-wide working directory, without joining the serial collection every other - such class is in; the two classes ran in parallel and one read the other's directory as the cwd. - The codegen-ts tests that type-check generated output with the real TypeScript compiler (28 files, - the slowest case measured at 3.3s against bun's 5s default) now run under a 30s timeout whose value - lives in one place, `test/compile-timeout.ts`. Each of those files calls `setDefaultTimeout` with - it, because bun applies that call only to the file that makes it, so the timeout holds in a plain - `bun test` as well as in CI, and a loaded machine no longer reports a timeout for correct code. -- **A cross-field `validator.comparison` between fields of different kinds is skipped in every - port, never rejected and never a 500.** Comparing a `field.date` with a `field.timestamp` (or any - two strings or temporals of different subtype) has no ordering. Java and Kotlin cast both sides - to `Comparable` inside the class-level validity check, so the cast threw `ClassCastException` and - a create answered 500 instead of 400; TypeScript compared the two as strings and Python raised a - `TypeError` that surfaced as an invalid body. All five ports now follow one rule: numerics compare - with each other, strings and temporals only with their own subtype and `@localTime` flag (a - `field.timestamp` is an instant and a `@localTime` one a wall clock, which is a different native - type in every port), boolean and enum fields test - `eq` / `ne` only (also on two `uuid`, `uri` or `inet` fields), and any other pair emits no rule, so an - ordering test on two booleans or two enums is no longer enforced by the generated validators (C# never - did). The database `CHECK` still covers the pair where the database can compare it. New shared cases - `comparison-across-kinds-is-skipped`, `comparison-instant-vs-local-timestamp-is-skipped`, - `uuid-ne-rejects-equal-ids` and `uuid-ne-accepts-different-ids` - in `fixtures/validation-conformance/`, run by every port. -- **Test coverage for cross-field validators.** A subtype form on a TPH hierarchy that declares a - cross-field rule now has a test proving the generated form binds the refined body schema (the rule - itself was enforced by the cross-field build). The Java check that a Patch is not held to cross-field - validators now runs the generated classes (the violating body is rejected by the Dto and accepted - by the Patch) instead of comparing generated source text. - -- **`meta migrate` refuses, at plan time, to drop a column a view it does not own still reads.** - With `--from-db`, a declared-`@unmanaged` view or one outside the scoped schemas whose live body - names the dropped column now blocks the `drop-column`, and the message names the view and the - column. Before, the migration was written and failed at apply on both Postgres (`cannot drop - column … other objects depend on it`) and SQLite (`error in view … after drop column`). A view with - no recorded body is not matched, and an offline run cannot see live views. Pass - `--allow drop-column-read-by-view` (config token `drop-column-read-by-view`) to proceed. -- **The SQLite and D1 down migration for `drop-column` re-adds the column.** It wrote a TODO comment - and still recreated the view with the old SQL, so the view failed with `no such column` after a - rollback. It now runs `ADD COLUMN` from the recorded column, as Postgres does; the data is not - restored, and a `NOT NULL` column with no default comes back nullable, which is all SQLite allows. -- **Docs: edit an `@sql` body when you remove a column it reads.** The tool never parses the body; - Postgres fails at apply and SQLite applies and leaves a broken view. - -### Added - +- **`field.duration`: a fixed-length span of time, in every port.** `TimeSpan` in C#, + `java.time.Duration` in Java and Kotlin, `timedelta` in Python, and a string in TypeScript; + a Postgres `interval` column, and a 64-bit integer of microseconds on SQLite, D1 and MySQL, so + sorting and range filters follow the span on every database. On the wire each port writes its own + language's standard form by default; `"wire": { "durations": "iso" }` makes every port write one + canonical ISO-8601 form (`PT1H30M`, `PT26H`, `-PT1M30S`, `PT0S`) at its own precision: as many + fractional digits as its duration type holds (6 in TypeScript and Python, 7 in C#, 9 in Java and + Kotlin). The database keeps microseconds, so a C#, Java or Kotlin server can answer a POST with + more digits than it stored; read the row back for the stored value. A week is 7 days and a day 24 + hours; years and months are refused. Postgres does not accept a + signed ISO-8601 duration as input, so a value bound as text, and a declared `@default`, use + its own `HH:MM:SS` spelling, which it returns unchanged. Filters: every numeric operator. + Shared vectors in `fixtures/duration-wire/`; see `docs/features/field-types.md`. Each port is + limited only by its own duration type and each database by its own column. + - **`wire.durations`: `"native"` (default) or `"iso"`**, beside `decimals` and `nullMembers` in + `.metaobjects/config.json` (ADR-0019 Amendment 2). `native` is the language's own serializer, + both ways: System.Text.Json's `TimeSpan` form in C# (`"01:30:00"`), `Duration.toString()` in + Java and Kotlin, Pydantic's `timedelta` form in Python (which folds 24 hours into a day, + `"P1DT2H"`), and the canonical string in TypeScript. `iso` is the canonical form in every port. + A filter operand is ISO-8601 in both modes. The api-contract and persistence corpora pin `iso`, + and a new `api-contract-conformance/duration/` sub-corpus gates it through every port's + generated routes on Postgres and SQLite (and MySQL and D1 where a port has them). + - An owned (ejected) generator copy from before this release does not know `field.duration`: + re-eject it (`meta eject --force`, or merge) before a model uses one. Models without a + duration are unaffected. + - A live Postgres `interval` column modelled as a `field.string`, the only way to model one + until now, is still read as text by `meta migrate` and `meta verify`, so it does not become + drift. Change it to `field.duration` to get the native column and type. +- **CHECK names, conditions and "exactly one" from validators.** `@constraintName` on a validator + that generates a CHECK (`numeric`, `length`, `regex` and the cross-field validators), and on a + `field.enum` for its membership CHECK, names the CHECK `meta migrate` + generates, for a database whose CHECKs were named by hand. A cross-field validator + (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) takes `@condition`, an + expression over the entity's fields, so the rule holds only where it does. The new + `validator.exactlyOne` says exactly one of its `@fields` is present, and becomes a + `num_nonnulls(...) = 1` CHECK (a sum of presence tests on SQLite and D1). The generated create + model enforces both as well, in every port (see the record-model entry above): an `exactlyOne` + rule, and a conditional rule only where its condition is true. They are optional; a model without + them generates what it did. A `@condition` naming a field the entity does not have, one that is not + a boolean expression, one comparing a field with a literal of another kind (or with `null`), or an + `exactlyOne` naming fewer than two fields, is refused at load (`ERR_BAD_ATTR_VALUE`) in every port. A declared CHECK name another constraint on the table + already has, or one over 63 bytes on Postgres, is refused by `meta migrate` and `meta verify` + (`ERR_CHECK_NAME`). See `docs/features/migrations-and-drift.md`, "CHECKs from validators". - **Contract emitters: JSON Schema and OpenAPI generators (FR-022).** `jsonSchemaFile()` (`contract-schema`) writes JSON Schema 2020-12 for each object's read, create and update shapes, and a strict structured-output subset with `profile: "strict"`. `openApiFile()` (`openapi`) writes one @@ -172,6 +145,76 @@ here.** Gated by the new `fixtures/explorer-conformance/` corpus. See `docs/features/explorer-data.md`. No metamodel vocabulary changes, and `metamodelVersion` does not move. +- **`@flattenedColumnPrefix`: name a flattened value object's columns.** On a `field.object` with + `@storage: "flattened"`, it replaces the default prefix (the field's column and `_`) verbatim; + `""` keeps the members' own column names. It sits on the embedding field because one value + object embedded twice needs two prefixes. Flattened column names now come from core naming in + every port (`resolveFlattenedColumnName` in TypeScript, `PhysicalNaming.ResolveFlattenedColumnName` + in C#, `ColumnNaming.resolveFlattened` on the JVM, `resolve_flattened_column_name` in Python), and + `meta migrate`, the Prisma schema, the C# EF mapping and the Kotlin Exposed tables all use it. Set + on any other storage, it is refused at load (`ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE`). + See `docs/features/field-types.md`, "Embedded value objects". +- **Names artifacts carry a flattened value object's member columns.** Each member of a + `@storage: "flattened"` value object is a column of the owning table, and the owner's names + artifact now has a constant for each, named by core naming: `fields.shipTo.members.street.column` + in TypeScript, `ShipToStreetColumn` in C#, `SHIP_TO_STREET_COLUMN` in Java and Kotlin, + `SHIPMENT_SHIP_TO_STREET_COLUMN` in Python, keyed `shipTo.street` in the column-by-field maps. + The generated EF Core and Exposed bindings spell these columns through the same core naming + function. A member constant + that would share its name with another constant is left out with a warning, so every model + that generated before still generates. In C# and Java, a names class no longer extends its + base's when that would make a base constant that is not a column of its own table reachable + through it (a flattened field redeclared as jsonb, say); it is emitted complete instead. +- **`@sequenceStart` and `@sequenceName` on `identity.primary`.** An `@generation: increment` + key's sequence can be given its first value and its physical name, so a model can reproduce a + table that keeps a low id range for seeded rows, or adopt a sequence that kept its name across + a table rename. On Postgres `meta migrate` creates + `GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME "…" START WITH n)`, reads the live sequence + back from the catalog, and alters one whose declared start or name differs in place: a new + start is set and the counter moved up to it (never down), a new name is a rename. Only what + the model declares is compared, on the live, shadow and snapshot paths alike, so a model that + declares neither sees no new change and writes the same SQL and snapshots. Every port + registers both attributes and refuses either without `@generation: increment` + (`ERR_SEQUENCE_WITHOUT_INCREMENT`) and a start below 1 or past the key's range + (`ERR_SEQUENCE_START_OUT_OF_RANGE`); a name two tables declare is refused as a duplicate + database name. See `docs/features/migrations-and-drift.md`. +- **A string-stored enum's column length: `@maxLength` on `field.enum`, and `enums.maxLength`.** + `@maxLength` on a `field.enum` makes its column `VARCHAR(n)`, and `"enums": { "maxLength": n }` + in `.metaobjects/config.json` gives every enum without one a default; with neither, the column + stays unbounded text, as before. The ports that size a column resolve the length through one + core function (`resolveEnumMaxLength` in TypeScript, `EnumLength.ResolveMaxLength` in C#, + `EnumLength.resolveMaxLength` on the JVM), and `meta migrate`, the Drizzle, Prisma, EF Core and + Exposed bindings use it. Python emits no column types, so it accepts the `enums` block and + sizes nothing. A member longer than the length is refused + (`ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH`): at load for `@maxLength`, when the column is sized for + the project default, which `meta migrate` and `meta verify` report by name. `@maxLength` beside + `@intValueMap` is refused at load (`ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP`). A new shared + corpus, `fixtures/enum-length-conformance/`, pins the resolution in those ports. +- **An `upperFirst` column rule.** `"naming": { "columns": "upperFirst" }` in + `.metaobjects/config.json` upper-cases a field name's first letter and changes nothing else + (`createdAt` → `CreatedAt`, `requestURL` → `RequestURL`, `address_line1` → `Address_line1`), in + every port. A schema whose columns are the field names with a capital first letter (EF Core's + default, for one) then needs `@column` only where the database spells a name differently. A + column rule only; `literal` tables keep an object name verbatim. + See `docs/features/migrations/physical-naming.md`. +- **Schema conventions for the database objects no code addresses.** `migrate.conventions` in + `.metaobjects/config.json` picks how `meta migrate` names primary-key and foreign-key + constraints, whether each foreign key gets an index, and whether a unique key an FK references + is a constraint or an index. `default` is what migrate has always emitted, so nothing changes + unless you set it. `efcore` reproduces EF Core's conventions (`PK_`, `FK_`, `IX_`, its + 63-character `~` truncation, alternate keys as constraints), so a database built by EF Core + migrations and one built by `meta migrate` match. A project can supply its own conventions + object from `metaobjects.config.ts`; an unknown preset name, or an object missing a member, is + refused by name. The PK name is now introspected too, so a `baseline --from-db` snapshot of a + database whose primary keys are not named `_pkey` records their names (not compared; + nothing churns). See `docs/features/migrations-and-drift.md`, "Schema conventions". +- **`migrate.checks.skipRedundant`: leave out the CHECKs a column type already enforces.** Set in + `.metaobjects/config.json` (default `false`), `meta migrate` and `meta verify --db` leave out a + `validator.length` maximum the column's `VARCHAR(n)` enforces (not on SQLite or D1, which do not + enforce one) and a length minimum of 0 or less, for a database whose CHECKs were written by hand + only where the column type does not already enforce the rule. A validator with a declared + `@constraintName` keeps its CHECK. A malformed `checks` block is refused when the config loads. + See `docs/features/migrations-and-drift.md`, "CHECKs from validators". - **A recipe for checking an API change against the last release.** Emit the document at the last tag and at `HEAD`, then run oasdiff. See `docs/recipes/check-api-against-last-release.md`. - **A `pydantic` extra on the Python package.** `pip install "metaobjects[pydantic]"` installs @@ -183,7 +226,6 @@ here.** The proof is a test file that `tsc` or pytest runs, so nothing reads your source and there is no new command. `examples/brownfield/` holds a runnable project for each, and the recipe ships in the `metaobjects-authoring` skill. - - **A project's own provider class, wired in every port's config, loaded by every command.** A project that declares its own vocabulary (a new subtype, or attributes on a core type) writes a provider class in each language whose tools load the metadata, and names it once: @@ -216,6 +258,37 @@ here.** `ERR_UNKNOWN_SUBTYPE` and then each `ERR_UNKNOWN_ATTR`. Java reported the attribute errors before the subtype error; it now reports them after it, as TypeScript and C# do. A load that failed still fails; only the list of errors differs. +- **`meta migrate` refuses a foreign key that references columns the target does not hold unique.** + Behaviour change: an `identity.reference` whose dotted `@references` names target fields that are + not the target's primary key, a plain `identity.secondary` (no `@where`, no `@expr`) or a single + `@unique` field used to emit an FK the database refuses at apply time ("there is no unique + constraint matching given keys"), after earlier statements had already run. `meta migrate` and + `meta verify --db` now fail first with `FkTargetNotUniqueError`, which lists every offending + reference at once (entity, reference, target and fields); both report it as a metadata refusal + with a hint. The check is against the table the model actually builds: a key declared on a TPH + subtype is not emitted, so it does not count (the message says to declare it on the base), and + a bare reference into a composite primary key with fewer FK fields is caught too. An FK on a + table this run does not govern (imported, or outside `migrate.scope`) is warned about, not + refused, and an FK into a table the model does not build (`@unmanaged`) is not checked. Postgres + refuses the DDL and D1 always enforces foreign keys, so both refuse. SQLite only warns: it accepts + the `CREATE TABLE`, and checks foreign keys only with `PRAGMA foreign_keys` on, which is off by + default, so the FK fails (`foreign key mismatch`) on a write with enforcement on. `meta verify + --db` gives the same verdict. Fix: declare an `identity.secondary` on those fields of the target. + This is a `meta migrate` correction; it moves no metamodel version. +- **A constraint or index whose only change is its name is renamed in place on Postgres.** A + drop plus an add of the same definition under another name, on a table that is not itself + renamed (switching schema conventions renames every FK), used to be a gated `drop-fk` / + `drop-index` that dropped and rebuilt the object. It is now one `rename-constraints` change: + `ALTER TABLE … RENAME CONSTRAINT` / `ALTER INDEX … RENAME`, and the down renames it back. SQLite + cannot rename a constraint and keeps the rebuild. The primary key's name is now compared too, + so a convention switch, or a table rename (which keeps the old `
_pkey`), renames it, and + `meta verify --db` reports a mismatch it used to pass. A unique key that changes only between + plain index and UNIQUE constraint is promoted in place (`ADD CONSTRAINT … UNIQUE USING INDEX`); + the reverse, when the conventions ask for a plain index, drops the constraint, re-creates the + index, and drops and re-adds each FK onto it, behind `--allow drop-index` and `--allow drop-fk`. + The default conventions accept either kind, so a database built with Drizzle's `unique()` + constraints is left as it is. A key that changes both its name and its kind + is renamed, then promoted or demoted, in the same migration. - **A prompt or output template that names a variable not on its payload now raises at render time.** Behaviour change: a generated render helper used to render `Hi {{nmae}}` as `Hi ` with no error when the payload declares `name`; it now raises `ERR_VAR_NOT_ON_PAYLOAD (nmae)` (`RenderError` in @@ -233,6 +306,106 @@ here.** ### Fixed +- **TypeScript: every `meta` command now loads the providers in `metaobjects.config.ts`.** + `meta export`, `meta deps`, `meta fmt`, `meta types` and `meta upgrade` loaded the core + vocabulary only, so a project's own subtype or attribute failed there while `meta gen` accepted + it. A config that exists but fails to load is now reported as its own error by `meta verify` + (exit 2, in every mode) and `meta prompt-snapshot`, instead of continuing without its providers + and reporting an unknown subtype. +- **Python: `metaobjects fmt` reads the configured providers, and `--provider` adds to them.** + `fmt` ignored `providers:` in `metaobjects.config.yaml` (it also gains `--config`), and + `metaobjects verify --provider` replaced the configured list instead of adding to it. +- **`ERR_COLUMN_NAME_COLLISION` no longer rejects a read-only view whose columns come from `@sql`.** + The 1.2.0 check treated every field of a sourced object as a column MetaObjects creates. For a + `source.rdb @kind: view` with an `@sql` body, the author's SQL defines the columns and the fields + only read them, so two fields on one column are legal and nothing can be written. That object is + now exempt in all four ports (TypeScript, Java, C#, Python), provided every source of the object + is read-only. A derived view (no `@sql`), which emits a select list from its fields, and an object + with a writable table source plus an `@sql` view replica are still checked. When both colliding fields already carry an + explicit `@column`, the message now says to rename one field or point one `@column` at a different + column, instead of suggesting a `@column` they already have. The error code is unchanged. New + conformance fixtures: `column-name-shared-in-sql-view-ok`, `error-column-name-collision-derived-view` and + `error-column-name-collision-table-with-sql-view`. +- **C# codegen: an entity that extends another no longer loses the parent's properties.** The member + loop skipped every inherited field but the class declaration never named the base, so the fields + appeared nowhere. A child of a **concrete** parent now declares the parent's properties itself and + names no base: each concrete entity owns a table with all its columns (what `meta migrate` creates + and the other ports' models carry), and `class Child : Parent` between two mapped entities is an EF + inheritance hierarchy (one shared table, or a child table holding only its own columns). A child of + an **abstract** parent with `EmitAbstractShapes` on now emits `class Child : Base`, as the + generator always meant to, and declares only its own members. Regenerating changes those two shapes + of C# output only. An adopter override of `EmitClassDeclarationLine` that names a concrete parent as + the base now gets that parent's properties twice; drop the base from the override. +- **Test flakes fixed at their cause.** `GenCommandTests.Gen_writes_the_hash_manifest_beside_the_metadata_not_in_cwd` + (C# Cli) failed about one run in five because `DependencyCliTests` runs `gen` in process, which + changes the process-wide working directory, without joining the serial collection every other + such class is in; the two classes ran in parallel and one read the other's directory as the cwd. + The codegen-ts tests that type-check generated output with the real TypeScript compiler (28 files, + the slowest case measured at 3.3s against bun's 5s default) now run under a 30s timeout whose value + lives in one place, `test/compile-timeout.ts`. Each of those files calls `setDefaultTimeout` with + it, because bun applies that call only to the file that makes it, so the timeout holds in a plain + `bun test` as well as in CI, and a loaded machine no longer reports a timeout for correct code. +- **A cross-field `validator.comparison` between fields of different kinds is skipped in every + port, never rejected and never a 500.** Comparing a `field.date` with a `field.timestamp` (or any + two strings or temporals of different subtype) has no ordering. Java and Kotlin cast both sides + to `Comparable` inside the class-level validity check, so the cast threw `ClassCastException` and + a create answered 500 instead of 400; TypeScript compared the two as strings and Python raised a + `TypeError` that surfaced as an invalid body. All five ports now follow one rule: numerics compare + with each other, strings and temporals only with their own subtype and `@localTime` flag (a + `field.timestamp` is an instant and a `@localTime` one a wall clock, which is a different native + type in every port), boolean and enum fields test + `eq` / `ne` only (also on two `uuid`, `uri` or `inet` fields), and any other pair emits no rule, so an + ordering test on two booleans or two enums is no longer enforced by the generated validators (C# never + did). The database `CHECK` still covers the pair where the database can compare it. New shared cases + `comparison-across-kinds-is-skipped`, `comparison-instant-vs-local-timestamp-is-skipped`, + `uuid-ne-rejects-equal-ids` and `uuid-ne-accepts-different-ids` + in `fixtures/validation-conformance/`, run by every port. +- **Two `field.duration` fields order by length in every port, and a comparison inside `@condition` + follows the same kind rule.** The kind rule above listed no durations, so a `validator.comparison` + between two durations emitted no rule in any port. Durations now order with their own subtype, by + length rather than by their ISO-8601 text (`PT45M` is shorter than `PT1H`). A comparison inside an + `@condition` that reads two fields of different kinds (a date against a timestamp, or an enum + against a string) now leaves the whole validator to the database, as a field it cannot see already + did, where TypeScript, Java and Kotlin had compared the two as strings and C# had compared an enum + with a string. An `eq` / `ne` inside `@condition` on two uuid, uri or inet fields now tests + equality in every port: Python ordered the two values, so an IPv4 against an IPv6 address read as + unknown and the rule was skipped, and C# skipped those comparisons altogether. C# also gave + `field.uri` and `field.inet` no filter operators, so `@filterable` on either, or any `@condition` + that reads one, failed to load; they now take the same bands as in the other ports (uri the string + band, inet the uuid band), and the shared `filter-ops-matrix` fixture lists both. New shared cases + `duration-comparison-rejects-a-longer-span`, `duration-comparison-orders-by-length-not-text`, + `condition-comparing-across-kinds-leaves-rule-to-database`, + `condition-ne-on-ipv4-and-ipv6-addresses-enforces-the-rule` and + `condition-ne-on-equal-addresses-lets-the-record-through`. +- **Test coverage for cross-field validators.** A subtype form on a TPH hierarchy that declares a + cross-field rule now has a test proving the generated form binds the refined body schema (the rule + itself was enforced by the cross-field build). The Java check that a Patch is not held to cross-field + validators now runs the generated classes (the violating body is rejected by the Dto and accepted + by the Patch) instead of comparing generated source text. + +- **`meta migrate` refuses, at plan time, to drop a column a view it does not own still reads.** + With `--from-db`, a declared-`@unmanaged` view or one outside the scoped schemas whose live body + names the dropped column now blocks the `drop-column`, and the message names the view and the + column. Before, the migration was written and failed at apply on both Postgres (`cannot drop + column … other objects depend on it`) and SQLite (`error in view … after drop column`). A view with + no recorded body is not matched, and an offline run cannot see live views. Pass + `--allow drop-column-read-by-view` (config token `drop-column-read-by-view`) to proceed. +- **The SQLite and D1 down migration for `drop-column` re-adds the column.** It wrote a TODO comment + and still recreated the view with the old SQL, so the view failed with `no such column` after a + rollback. It now runs `ADD COLUMN` from the recorded column, as Postgres does; the data is not + restored, and a `NOT NULL` column with no default comes back nullable, which is all SQLite allows. +- **Docs: edit an `@sql` body when you remove a column it reads.** The tool never parses the body; + Postgres fails at apply and SQLite applies and leaves a broken view. + +- **Postgres introspection reads a composite foreign key's columns once each, in order.** It joined + `information_schema.key_column_usage` to `constraint_column_usage`, which has no column position, so + every FK column was paired with every referenced column: a two-column FK read back as four columns and + a three-column one as nine. `meta migrate` reads its expected side back from the shadow database, so + the doubled columns reached the emitted DDL, which Postgres refused ("foreign key referenced-columns + list must not contain duplicates"); `verify --db` misreported every composite FK. It now reads + `pg_constraint`, pairing `conkey` with `confkey` by position. As before, only same-schema foreign + keys are read (the FK descriptor carries no referenced schema). + - **Python: an array field's `default` generated a string.** A model with `field.string[]: { default: "{}" }` generated `tags: list[str] = '{}'`, which fails on first use. It now generates `tags: list[str] = []`, from the same four spellings TypeScript accepts, or diff --git a/agent-context/skills/metaobjects-audit/SKILL.md b/agent-context/skills/metaobjects-audit/SKILL.md index 21a2973750..525029b69c 100644 --- a/agent-context/skills/metaobjects-audit/SKILL.md +++ b/agent-context/skills/metaobjects-audit/SKILL.md @@ -186,8 +186,8 @@ When folding hand validators into metadata, apply human judgment per constraint. A constraint enters **shared metadata** only if it is a **true cross-language domain invariant**; a one-consumer preference stays in a thin local refinement layer. -Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `requiredWhen` / -`presentIff`); ratification decides *which* belong in shared metadata. Output a +Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `exactlyOne` / +`requiredWhen` / `presentIff`); ratification decides *which* belong in shared metadata. Output a **ratification table**: KEEP-IN-METADATA / LOCAL-REFINEMENT / DROP + rationale — human-approved, never applied silently. Distinguish `required` from has-a-safe-default (`@default` often fixes the over-requiring bug). A core attr ripples cross-port; for a diff --git a/agent-context/skills/metaobjects-audit/references/capability-checklist.md b/agent-context/skills/metaobjects-audit/references/capability-checklist.md index e145d1ccf6..71672d2a81 100644 --- a/agent-context/skills/metaobjects-audit/references/capability-checklist.md +++ b/agent-context/skills/metaobjects-audit/references/capability-checklist.md @@ -182,7 +182,8 @@ soft-delete / status / type view it models. `validator.regex`** — hunt hand field-level validation (presence, length, numeric range, array bounds, `@pattern` regex) the validator subtypes model. - **Cross-field validators** — `validator.comparison` (`@left`/`@op`/`@right`, e.g. - "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.requiredWhen` + "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.exactlyOne` (exactly one of N + present), `validator.requiredWhen` (conditional-required), `validator.presentIff` (mutual presence). Hunt these as hand-coded multi-field rules — they ARE modelable (see the Semantic-constraint ratification section in `SKILL.md` to decide what belongs in shared metadata vs port-local). diff --git a/agent-context/skills/metaobjects-audit/references/drift-signatures.md b/agent-context/skills/metaobjects-audit/references/drift-signatures.md index 351893deb6..5148b26db4 100644 --- a/agent-context/skills/metaobjects-audit/references/drift-signatures.md +++ b/agent-context/skills/metaobjects-audit/references/drift-signatures.md @@ -35,7 +35,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md b/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/agent-context/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/agent-context/skills/metaobjects-codegen/references/physical-names.md b/agent-context/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/agent-context/skills/metaobjects-codegen/references/physical-names.md +++ b/agent-context/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/agent-context/skills/metaobjects-codegen/references/wire-format.md b/agent-context/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/agent-context/skills/metaobjects-codegen/references/wire-format.md +++ b/agent-context/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/agent-context/skills/metaobjects-verify/references/migration.md b/agent-context/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/agent-context/skills/metaobjects-verify/references/migration.md +++ b/agent-context/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index cf6542d422..9ca93357ae 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -1,6 +1,6 @@ # Conformance coverage -The MetaObjects standard ships **36 shared conformance corpora** under +The MetaObjects standard ships **38 shared conformance corpora** under [`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to it* and asserts the same expected behaviour against the same fixtures. **This page is the inverse index**: fixture → feature doc + per-port pass status, and it is the @@ -26,7 +26,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 428 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 449 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 16 | 16 / 16 | 15 / 16 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 15 / 16 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 16 / 16 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 45 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -36,9 +36,10 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 119 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 121 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal + 2 duration) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | -| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 20 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | +| [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | +| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/object-model-conformance/`](../fixtures/object-model-conformance/) | 1 shared metadata fixture (per-port scenarios) | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/codegen-conformance/`](../fixtures/codegen-conformance/) | 6 shared inputs; `shared-provided-enum` carries the enum-type decision table and `cross-package-binding` the reference-binding decision table (ADR-0059) every port answers | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -58,6 +59,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/requirement-test-identity-conformance/`](../fixtures/requirement-test-identity-conformance/) (the identity, skip state and digest of each generated requirement test, and the filter seam, ADR-0057) | 26 cases | ✓ (reference) | ✓ | identity function inherits via Java; emitted names asserted by its own generator test | ✓ | ✓ | | [`fixtures/naming-conformance/`](../fixtures/naming-conformance/) | 8 cases | ✓ | ✓ | inherits via Java (`RouteNaming.pluralize`) | ✓ | ✓ | | [`fixtures/wire-conformance/`](../fixtures/wire-conformance/) (the `wire` setting, ADR-0019 Amendment 2) | 6 cases + 8 refusals | ✓ | ✓ | inherits via Java (one config reader) | ✓ | ✓ | +| [`fixtures/enum-length-conformance/`](../fixtures/enum-length-conformance/) (a string-stored enum's column length: `@maxLength`, else `enums.maxLength`) | 5 cases + 1 overflow + 5 refusals; run by a port that sizes the column | ✓ | ✓ | inherits via Java (one config reader, one resolver) | ✓ | — (sizes no column: emits no DDL or column type; the config reader accepts and shape-checks the `enums` block) | | [`fixtures/codegen-noop/`](../fixtures/codegen-noop/) (FR-044 — reporting vocabulary: what is lowered, what stays inert) | 1 model pair (`reporting/with` vs `reporting/without`) | ✓ (codegen + migrate) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/cube-model/`](../fixtures/cube-model/) (FR-044 Plan 4 — the `cube-model` generator's golden Cube files, plus the canonical model's golden) | 46 cases (36 trees + 10 errors) + 1 canonical golden | ✓ (the exporter is TypeScript-only, spec R6; its `cube` lane also loads the files into a real Cube) | — | — | — | — | | [`fixtures/contract-conformance/`](../fixtures/contract-conformance/) (FR-022 — the `contract-schema` and `openapi` generators' golden trees and refusals) | 19 cases (14 trees + 5 errors) | ✓ (one neutral engine, ADR-0034 Amendment 10; the generated API lanes also hold real traffic to the documents) | — | — | — | — | @@ -286,7 +288,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (428) +### `fixtures/conformance/` — metamodel loader + canonical serializer (449) | Fixture prefix | Feature doc | |---|---| @@ -382,7 +384,7 @@ The corpus also covers the 9 cross-port filter operators (`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `like`, `isNull`) plus the implicit-AND combinator and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the URL grammar `?filter[][]=` (FR-009). On top of the 31 core -scenarios the corpus carries eight sub-corpora — `tph/` (10, single-table +scenarios the corpus carries nine sub-corpora — `tph/` (10, single-table inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), `jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), @@ -390,13 +392,15 @@ each corpus alone could not reach, and 1 pinning the collection-URL spelling), read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named `id` is addressed by it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one -canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (37, +canonical string, read exactly, filtered exactly and refused past its declared digits), `duration/` (2, a +`field.duration` under `"wire": { "durations": "iso" }` is one canonical ISO-8601 string, refused +in a calendar or clock spelling, and filtered and sorted as a span on every engine) and `report/` (37, FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java, Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and the port's own GENERATED API artifact booted over HTTP. `write-through/`, -`secret-fields/`, `projection/`, `decimal/` and `report/` run the generated lane only, on all five ports: what +`secret-fields/`, `projection/`, `decimal/`, `duration/` and `report/` run the generated lane only, on all five ports: what they test is whether a port's generator emits the routes. The `report/` sub-corpus is the one that reads real views, so it runs full stack on Postgres, SQLite and MySQL (`report/schema.sqlite.sql`, `report/schema.mysql.sql`, both TypeScript-produced) in every @@ -629,8 +633,8 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) -The fixtures in the nine corpora mapped above (metamodel 428 + yaml 16 + verify 45 -+ render 18 + persistence 59 + api-contract 119 + source-resolution 25 + scope 10 + +The fixtures in the nine corpora mapped above (metamodel 449 + yaml 16 + verify 45 ++ render 18 + persistence 59 + api-contract 121 + source-resolution 25 + scope 10 + dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 4f3f91f9f3..a4d965895e 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -431,24 +431,24 @@ compares equal: ### The wire setting -What a generated API writes for a decimal and for a value object's null member is a project +What a generated API writes for a decimal, a duration and a value object's null member is a project setting, `wire` in `.metaobjects/config.json`, read by every port's codegen the way `naming` is ([ADR-0019 Amendment 2](../../spec/decisions/ADR-0019-runtime-return-type-contract.md)): ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` -Both keys default to `native`: **each port writes what its language's standard serializer +Every key defaults to `native`: **each port writes what its language's standard serializer writes**, so a C# API looks like a C# API and a Python API like a Python API. Identical output across ports is something you opt into: ```json -{ "schema_version": 1, "wire": { "decimals": "string", "nullMembers": "omit" } } +{ "schema_version": 1, "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" } } ``` -That is the form 1.1.0 and 1.1.1 wrote, and the form the cross-port conformance corpora pin in -their own config. Choose it when clients written in several languages share one contract, or when +For decimals and null members that is the form 1.1.0 and 1.1.1 wrote, and it is the form the +cross-port conformance corpora pin in their own config. Choose it when clients written in several languages share one contract, or when a browser client must read a decimal exactly: `JSON.parse` turns a JSON number into a double before your code sees it. @@ -466,6 +466,20 @@ before your code sees it. exactly. - `nullMembers` covers the members of an `object.value` only, and so also the jsonb that a typed serializer writes. A top-level entity field that is null is always written as `null`. +- `durations` covers `field.duration` in a request and response body, both ways: + +| Port | `durations: native` (default) | `durations: iso` | +|---|---|---| +| C# | System.Text.Json's `TimeSpan` form, both ways: `"01:30:00"`, `"1.02:00:00"`, `"-00:01:30"` | the canonical ISO-8601 form: `"PT1H30M"`, `"PT26H"`, `"-PT1M30S"` | +| Java | `Duration.toString()` and `Duration.parse`, as Spring Boot configures Jackson (`WRITE_DURATIONS_AS_TIMESTAMPS` off): `"PT1H30M"`, `"PT26H"`, `"PT-1M-30S"` | the canonical form | +| Kotlin | same as Java | the canonical form | +| Python | ISO-8601 as Pydantic v2 writes a `timedelta`, which folds 24 hours into a day: `"PT1H30M"`, `"P1DT2H"`, `"-PT1M30S"`; it also reads `"01:30:00"` | the canonical form | +| TypeScript | the canonical form: a duration is a `string` in TypeScript, so there is no other standard spelling | the canonical form | + + A filter operand (`?filter[timeout][gt]=PT1H`) is an ISO-8601 duration in every mode, because it + is the query grammar, not the body serializer. The canonical form and the accepted one are in + [`fixtures/duration-wire/`](../../fixtures/duration-wire/). Choose `iso` when clients written in + several languages share one contract. - There is no `"number"` value for `decimals`. In C#, Java and Kotlin, `native` already is a number; in Python and TypeScript, a number would be a lossy spelling those languages do not use. diff --git a/docs/features/entities.md b/docs/features/entities.md index d0f0d2f728..bde41e9586 100644 --- a/docs/features/entities.md +++ b/docs/features/entities.md @@ -83,6 +83,10 @@ YAML is desugared to canonical JSON at load time. See > an `ERR_TOO_MANY_OCCURRENCES` load error. `identity.secondary` / `identity.reference` > are not singletons and DO require an explicit `name`. +An `@generation: increment` key's sequence can be given a start value (`@sequenceStart`) and +a physical name (`@sequenceName`); see +[migrations-and-drift.md](migrations-and-drift.md#an-increment-keys-sequence-sequencestart-and-sequencename). + ## `extends:` for shared abstract bases Common base fields (`id`, `createdAt`, `updatedAt`) live on an abstract entity that @@ -321,9 +325,9 @@ found = om.find_by_id("Author", row["id"]) ## Cross-field validators -`validator.comparison`, `validator.requiredWhen`, `validator.presentIff` and `validator.atLeastOne` -are declared as children of the entity and reference sibling fields by name. They are enforced in -two places: +`validator.comparison`, `validator.requiredWhen`, `validator.presentIff`, `validator.atLeastOne` +and `validator.exactlyOne` are declared as children of the entity and reference sibling fields by +name. They are enforced in two places: - **The generated full record (create) model**, in every port, with the language's native mechanism: a Zod refinement on the insert schema (TypeScript), a class-level Bean Validation @@ -335,15 +339,35 @@ two places: The two share one set of semantics: absent or null is SQL `NULL`; `comparison` passes when either side is null; `requiredWhen` is violated only when the gate equals `@equals` and the target is null; `presentIff` requires the target to be present exactly when the gate equals `@equals`; `atLeastOne` -needs one non-null field. `@equals` is read per the gating field's subtype. The shared cases in -`fixtures/validation-conformance/cross-field-cases.json` gate all five ports. +needs one non-null field; `exactlyOne` needs exactly one. `@equals` is read per the gating field's +subtype. + +Any of the five can carry a `@condition`, an expression over the entity's fields that says when the +rule applies (the grammar and its load-time checks are in +[migrations-and-drift.md](migrations-and-drift.md#checks-from-validators-names-conditions-how-many)). The rule is checked only +when the condition is true. A condition that is false, or unknown because a field it reads is null, +lets the record through, exactly as the `CHECK`'s `NOT (condition) OR (rule)` does in SQL. + +```json +{ "object.entity": { "name": "Shift", "children": [ + { "field.int": { "name": "startDay" } }, + { "field.int": { "name": "endDay" } }, + { "field.int": { "name": "startMin" } }, + { "field.int": { "name": "endMin" } }, + { "validator.comparison": { "name": "sameDayTimes", "@left": "endMin", "@op": "gt", "@right": "startMin", + "@condition": { "op": "eq", "left": { "field": "startDay" }, "right": { "field": "endDay" } } } } +] } } +``` + +The shared cases in `fixtures/validation-conformance/cross-field-cases.json` gate all five ports. One difference: a `comparison` orders only fields of the same kind. Numerics compare with each other; -strings and temporals compare only with their own subtype and `@localTime` flag. `eq` and `ne` also +strings, temporals and durations compare only with their own subtype and `@localTime` flag. `eq` and `ne` also apply to two booleans, enums, `uuid`, `uri` or `inet` fields, and ordering on a boolean or enum is not enforced. Any other pair is skipped in the generated model, never rejected, but the DB `CHECK` still -covers it where the database can compare the two columns. The full rule is in -`fixtures/validation-conformance/README.md`. +covers it where the database can compare the two columns. A comparison inside an `@condition` follows +the same rule: when two fields it reads are of different kinds, the generated model leaves the whole +validator to the database. The full rule is in `fixtures/validation-conformance/README.md`. PATCH and partial-update models do **not** enforce these rules, because a partial body cannot see the merged row. The database `CHECK` is the enforcement there. diff --git a/docs/features/field-types.md b/docs/features/field-types.md index 6623784a89..8c847d42b1 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -24,14 +24,15 @@ across ports — a `field.currency` is integer minor units everywhere; a | `field.uuid` | `string` | `UUID` | `UUID` | `Guid` | `UUID` | `uuid` | | `field.uri` | `string` | `java.net.URI` | `java.net.URI` | `Uri` | `AnyUrl` | `text` | | `field.inet` | `string` | `InetAddress` | `InetAddress` | `IPAddress` | `IPvAnyAddress` | `inet` | +| `field.duration` | `string` (ISO-8601) | `java.time.Duration` | `java.time.Duration` | `TimeSpan` | `timedelta` | `interval` | | `field.enum` | union + `z.enum` | `Enum` | `enum class` | `enum` | `Literal` inline, `Enum` with `UPPER_CASE` members when shared by `extends` | `varchar` + `CHECK` | | `field.object` | nested type | nested class | nested data class | nested class | nested Pydantic model | per `@storage` | | `field.map` | `Record` | `Map` | `Map` | `Dictionary` | `dict[str, V]` | `jsonb` | -That is the whole registered vocabulary — 17 concrete subtypes. `field.base` is an +That is the whole registered vocabulary — 18 concrete subtypes. `field.base` is an abstract registry anchor, never authored (`ERR_ABSTRACT_SUBTYPE_AUTHORED`). -Four rows need a footnote: +Five rows need a footnote: - **`field.decimal`** is exact in every port's own type. TypeScript's type is a string. Postgres and MySQL keep the declared `numeric(@precision, @scale)` exactly; SQLite and D1 have @@ -48,6 +49,29 @@ Four rows need a footnote: which has none (the same reason `field.uuid` is a `string` there). `@lenient: true` degrades both to a plain string with no validator, and takes the `inet` column down to `text`. +- **`field.duration`** is a fixed-length span: days, hours, minutes and seconds. A calendar + span ("every 3 months") is not one, because a month has no fixed length: model it as an int + and a unit. On the wire each port writes its own language's standard form by default (a C# + `TimeSpan` is `"01:30:00"`), and `"wire": { "durations": "iso" }` in `.metaobjects/config.json` + makes every port write one ISO-8601 form ([the wire setting](api-contract.md#the-wire-setting)). + That form is `[-]PT[nH][nM][n[.f]S]`: hours are never folded into days (`PT26H`), and zero is `PT0S`. + It is one form at each port's own precision: a port writes as many fractional digits as its + own type holds (6 in TypeScript and Python, 7 in C#, 9 in Java and Kotlin) and truncates a + longer fraction toward zero. A Postgres `interval` and the integer column below both hold + microseconds, so a C#, Java or Kotlin server can answer a POST with more digits than the + database kept: `PT0.1234567S` is answered as sent and read back as `PT0.123456S`. Read the row + back when the stored value matters. The Java types above are generated code's: Java's + metadata-driven runtime (OMDB) holds a duration as its canonical ISO-8601 `String`, and + `java.time.Duration` appears only inside its JDBC codec. Every port reads that form and also leading + week and day counts (`P1W` is 7 days, `P1DT2H` is 26 hours). Years and months are refused, and + so are lowercase designators and fractional hours. The column is a Postgres `interval`; SQLite, D1 and MySQL store a 64-bit + integer count of microseconds, the way Django stores a `DurationField` outside Postgres. Both + order and compare as the span does, so every filter operator a number has (`eq`, `ne`, `gt`, + `gte`, `lt`, `lte`, `in`, `isNull`) and `sort` work on every database. There is no shared upper + bound: each port is limited only by its own type (a .NET `TimeSpan`, a Python `timedelta`) and + each database by its own column, and refuses what it cannot hold rather than wrapping it. The + rule and its cross-port vectors are in + [`fixtures/duration-wire/`](../../fixtures/duration-wire/). - **`field.map`** is the typed dict: string keys, and a value type set by exactly one of `@valueType` (a scalar subtype) or `@objectRef` (a value object) — `V` above. It is one jsonb column holding the JSON object; `isArray` does not apply, so no port wraps the map @@ -111,7 +135,8 @@ A field has **two names**: the one your code calls it (`createdAt`, the metadata database calls it. `@column` sets the second. It never changes the first. With no `@column`, the physical name comes from the project's **column rule**: -`snake_case` (the default), `literal` (the field name verbatim) or `kebab-case`. That +`snake_case` (the default), `literal` (the field name verbatim), `kebab-case` or +`upperFirst` (the first letter upper-cased, the rest verbatim). That is CONFIG, not metadata, on purpose: the same model has to be able to drive a snake_case Postgres schema and a literal-column one. It is ONE setting for the whole project, `naming` in `.metaobjects/config.json` (ADR-0058): @@ -189,6 +214,26 @@ must be a non-empty set of unique members matching `^[A-Za-z_][A-Za-z0-9_]*$`. The loader enforces members own-only and emits `ERR_BAD_ATTR_VALUE` on a bad member or `ERR_MISSING_REQUIRED_ATTR` on missing `@values`. +**Column length:** a string-stored enum is an unbounded text column unless it has a length. +`@maxLength` on the field gives it one, and `"enums": { "maxLength": n }` in +`.metaobjects/config.json` gives every enum without its own `@maxLength` a default: + +```json +{ "field.enum": { "name": "status", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], "@maxLength": 16 } } +``` + +The column is then `VARCHAR(n)`: `meta migrate` creates it, and the generated column bindings +that carry a length name the same one: TypeScript's Drizzle and Prisma, C#'s EF Core +(`HasMaxLength(n)`) and Kotlin's Exposed. Java and Python generate no such binding; their loaders +apply the same member-length refusals, and the database column holds the length. Every member must fit. A member +longer than `@maxLength` is refused at load, and one longer than the project default is refused +when the column is sized, both as `ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH`, with the member named. +`@maxLength` beside `@intValueMap` is refused (`ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP`): an +int-backed enum's column is an integer. An array of enums stays a `text[]` column. +Turning a length on for a column that already exists (text to `VARCHAR(n)`) fails on any longer +row, so `meta migrate` gates it behind `--allow type-change`; turning it off widens the column and +is allowed. SQLite does not enforce the length, so there is nothing to change there. + **Int-backed storage:** `@intValueMap` switches the DB column from string to integer while preserving the string wire format and generated enum type. Keys must match `@values` exactly; values must be unique integers. Display labels and native Postgres `ENUM` types remain @@ -258,9 +303,10 @@ declaration. The `@storage` attr controls how the embedded shape persists. | `@storage` | Persistence shape | Use case | |---|---|---| -| `flattened` | One DB column per sub-field: `_` | EF Core `OwnsOne` / DDD value-objects | +| `flattened` | One DB column per sub-field: ``, the prefix `_` by default | EF Core `OwnsOne` / DDD value-objects | | `jsonb` | One `jsonb` column | Structured-but-evolvable user-defined shapes | -| `subdocument` (default — back-compat) | Single jsonb column | Pre-`@storage` projects | +| `subdocument` | Single jsonb column | A hint for document-store targets | +| not set (the default) | One `jsonb` column, as `jsonb` | Pre-`@storage` projects | Example: an `Author` with an embedded `Address` flattened to per-sub-field columns: @@ -275,6 +321,35 @@ Example: an `Author` with an embedded `Address` flattened to per-sub-field colum For `flattened`, the generated table gets `address_street`, `address_city`, `address_postal_code` columns instead of one `address` jsonb. +Each member's column is a **prefix** followed by the member's own column (its `@column`, else +its name under the project's column rule). The prefix defaults to the embedding field's column +and `_` (`address_`; with `@column: "bill"`, `bill_`; under the `upperFirst` rule, `Address_`, +EF Core's own default). **`@flattenedColumnPrefix`** on the embedding field replaces it, +verbatim: + +```json +{ "field.object": { "name": "homeAddress", "@objectRef": "Address", "@storage": "flattened", + "@flattenedColumnPrefix": "" } } +``` + +gives `street`, `city`, `postal_code`; `"@flattenedColumnPrefix": "home_"` gives `home_street`, and +so on. The prefix belongs to the embedding field, not the value object, because it is a fact +about the owning table: one `Address` embedded twice needs two prefixes. Every port names these +columns through the same core function (`resolveFlattenedColumnName` and its per-port +equivalents), so `meta migrate`, the Prisma schema, the C# EF mapping and the Kotlin Exposed +tables agree, and the owning object's names artifact carries each one as a constant +(`ShipmentNames.ShipToStreetColumn` in C#; see the codegen skill's physical-names reference +for every port). The field itself has no column constant, because the table has no column of +that name. On a field whose `@storage` is not `flattened` the attribute is refused at load +(`ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE`): no other storage gives the members +columns of their own. + +Two member columns that come out the same are refused at load (`ERR_COLUMN_NAME_COLLISION`): +two embeddings that share a prefix (two `""` embeddings of one `Address` both give `street`), or +a `""` member beside a field of the same name. When two files both declare the same field (a +merge, not an `overlay`) with different prefixes, `""` included, the loader reports +`ERR_MERGE_CONFLICT`. + ## What each port generates ### TypeScript @@ -447,6 +522,8 @@ The following conformance fixtures gate this feature's behavior across ports: - [`fixtures/conformance/field-object-storage-jsonb-single/`](../../fixtures/conformance/field-object-storage-jsonb-single/) — `@storage=jsonb` single-record column - [`fixtures/conformance/field-object-storage-jsonb-array/`](../../fixtures/conformance/field-object-storage-jsonb-array/) — `@storage=jsonb` on array field - [`fixtures/conformance/error-field-object-storage-flattened-array/`](../../fixtures/conformance/error-field-object-storage-flattened-array/) — `@storage=flattened` on array field is illegal +- [`fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/`](../../fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/) and [`…-default-storage/`](../../fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/) — `@flattenedColumnPrefix` without `@storage=flattened` is illegal +- [`fixtures/naming-conformance/physical-names/`](../../fixtures/naming-conformance/physical-names/) — flattened member columns under every column rule, with and without `@flattenedColumnPrefix` - [`fixtures/conformance/error-field-object-storage-no-object-ref/`](../../fixtures/conformance/error-field-object-storage-no-object-ref/) — `field.object` without `@objectRef` rejected Cross-port runner coverage: TS / Java / Kotlin / C# / Python all execute these diff --git a/docs/features/metadata-sources.md b/docs/features/metadata-sources.md index 6dfda56930..690ce0155a 100644 --- a/docs/features/metadata-sources.md +++ b/docs/features/metadata-sources.md @@ -352,6 +352,20 @@ out-of-scope table that already exists would become a proposed `DROP TABLE`. declaring object's FQN, and an object whose FQN is unknown was never proven to be anyone else's. +### `migrate.conventions` — names for objects no code addresses + +Beside `scope`, the `migrate` block takes `conventions`: the name of a schema-conventions preset +that decides primary-key and foreign-key constraint names, whether a foreign key gets an index, +and whether a unique key is an index or a constraint. `default` (absent) is what migrate has +always emitted; `efcore` is a reference preset that reproduces EF Core's naming. A project can +supply its own conventions from `metaobjects.config.ts` instead. Table, view and column names are +not conventions: they come from `naming`. See +[Schema conventions](migrations-and-drift.md#schema-conventions-names-for-objects-no-code-addresses). + +```jsonc +"migrate": { "scope": ["acme::billing::**"], "conventions": "efcore" } +``` + ### Put the `migrate` block where the ledger lives **Whoever holds `.metaobjects/migrations/` and the schema snapshot owns the schema.** diff --git a/docs/features/migrations-and-drift.md b/docs/features/migrations-and-drift.md index 7e38e0aea7..652e909524 100644 --- a/docs/features/migrations-and-drift.md +++ b/docs/features/migrations-and-drift.md @@ -282,6 +282,47 @@ sequence, or pass `--allow drop-identity-default` if removing auto-increment is intentional. An identity that *does* declare `@generation: increment` never reaches this gate (its default diff is skipped), so only the undeclared case fires. +#### An increment key's sequence: `@sequenceStart` and `@sequenceName` + +An `@generation: increment` key draws from a sequence. Two attributes on the +`identity.primary` declare the sequence's first value and its physical name, so the model can +state both instead of taking the database's defaults: + +```json +{ "identity.primary": { "@fields": "id", "@generation": "increment", + "@sequenceStart": 1000000000, "@sequenceName": "Ticket_Id_seq" } } +``` + +- `@sequenceStart` is the first value handed out. Absent: 1. For example, a table that keeps a + low id range for rows it seeds itself. +- `@sequenceName` is the sequence's physical name. Absent: the database's own default + (`
__seq` on Postgres). For example, a sequence that kept its name when its + table was renamed. + +On Postgres `meta migrate` creates the column as +`GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME "…" START WITH n)` (and adds an identity to an +existing column with the same options), and `meta verify --db` reads the live sequence back +from the catalog. A live sequence whose start or name differs from the declaration is altered +in place, never dropped: + +- a different start sets it (`ALTER SEQUENCE … START WITH n`) and moves the counter up to it + when it has not got there (`setval(…, n, false)`), so the next insert takes the declared + start. It never moves the counter down, which could hand out an id already in use; the down + migration restores the old start value and leaves the counter where it is. +- a different name renames it (`ALTER SEQUENCE … RENAME TO …`); the down renames it back. + +Only what the model declares is compared, on every path (a live database, a shadow database, +a committed snapshot): a database whose sequence kept an old name or a non-default start sees +no change until the model names it, and a model that declares neither writes the same SQL and +snapshots it always did. SQLite and D1 have no sequences, so both attributes change nothing +there; `meta migrate` and `meta verify --db` print an advisory when they create such a column +there. At load, either attribute without `@generation: increment` is refused +(`ERR_SEQUENCE_WITHOUT_INCREMENT`), and so is a `@sequenceStart` below 1 or past what the key +can hold, 2147483647 for a `field.int` (`ERR_SEQUENCE_START_OUT_OF_RANGE`). A declared name +shares the namespace of tables, views and indexes, so two tables declaring one (an abstract +base's identity extended twice, say) is refused when the schema is built +(`ERR_DUPLICATE_SQL_NAME`). + ### Java Schema migrations for Java projects are owned by the **TypeScript toolchain** @@ -370,6 +411,135 @@ opened by an older SQLite (Python 3.10's stdlib, Ubuntu 22.04's `sqlite3`). The behind `--allow drop-check` and raises no data hazard, because the rule is the same — and the re-diff is empty afterwards. Postgres keeps `IS [NOT] DISTINCT FROM`. +### CHECKs from validators: names, conditions, how many + +A `validator.*` node and a `field.enum`'s member list each become a CHECK constraint. Three +attributes shape it, all database-only (no generated code reads them): + +- **`@constraintName`** names the CHECK, for a database whose CHECKs were named by hand. On the + validators that generate a CHECK (`numeric`, `length`, `regex`, `comparison`, `requiredWhen`, + `presentIff`, `atLeastOne`, `exactlyOne`; `required` is a NOT NULL, not a CHECK), and on a + `field.enum` for its + membership CHECK. Absent, the name is derived (`
___chk`, + `
__chk` for an enum), as before. A declared name is refused (`ERR_CHECK_NAME`) when + another constraint on the table already has it, or, on Postgres, when it is over 63 bytes + (Postgres would truncate it, and the CHECK would never match its declaration). A CHECK with a + declared name is always emitted, even where `migrate.checks.skipRedundant` (below) would leave + it out. +- **`@condition`** on a cross-field validator (`comparison`, `requiredWhen`, `presentIff`, + `atLeastOne`, `exactlyOne`) is an expression, in the grammar `origin.computed` uses, over the entity's fields. + The rule holds only where the condition does: the CHECK is `(NOT condition) OR (rule)`, with + the negation folded in (`kind = 'Owned'` becomes `kind <> 'Owned'`). It is checked at load + (`ERR_BAD_ATTR_VALUE`): every field it names is a field of the entity, it is a boolean + expression, a literal compared with a field is of that field's kind (and, for an enum, one of + its `@values`), and no comparison has a `null` literal (`x = NULL` is never true: use `isNull` + or `isNotNull`). An int-backed enum's member is compared as its stored integer. +- **`validator.exactlyOne`** says exactly one of its `@fields` is present: + `num_nonnulls(a, b) = 1` on Postgres, a sum of `IS NOT NULL` tests on SQLite and D1. Its CHECK + is `
__exactlyone_chk` after the first field. It must name two or more fields + (`ERR_BAD_ATTR_VALUE`): exactly one of one field is a required field. + +```json +{ "object.entity": { "name": "Note", "children": [ + { "source.rdb": { "@table": "Notes" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "kind", "@values": ["Owned", "Shared"], "@constraintName": "CK_Notes_Kind" } }, + { "field.long": { "name": "customerId" } }, + { "field.long": { "name": "accountId" } }, + { "identity.primary": { "name": "pk", "@fields": "id", "@generation": "increment" } }, + { "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"], + "@constraintName": "CK_Notes_Owner", + "@condition": { "op": "eq", "left": { "field": "kind" }, "right": { "value": "Owned" } } } } +] } } +``` + +On Postgres the last one is `CK_Notes_Owner CHECK (("kind" <> 'Owned') OR (num_nonnulls("customer_id", "account_id") = 1))`. + +Renaming a CHECK this way is a drop and an add of the same rule on Postgres, under +`--allow drop-check`. On SQLite and D1 a CHECK is matched by its expression when the name does not +match, so a rename alone proposes nothing and the database keeps the old name. + +**Leaving out the CHECKs a column type already enforces.** By default every validator that has +a SQL form becomes a CHECK, including one the column already guarantees. For a database whose +CHECKs were written by hand only where the column type does not already enforce the rule, set: + +```json +{ "schema_version": 1, "migrate": { "checks": { "skipRedundant": true } } } +``` + +`meta migrate` and `meta verify --db` then leave out a `validator.length` maximum the column's +`VARCHAR(n)` enforces (n at most the maximum; not on SQLite or D1, which store a longer value in a +`VARCHAR(n)` column anyway) and a length minimum of 0 or less; the rest of the validator's CHECK +stands. A validator with a declared `@constraintName` keeps its CHECK whole: an explicit +declaration beats a project default, as a declared name beats a convention. Default `false`. A +`checks` that is not an object, a key other than `skipRedundant`, or a non-boolean value is +refused when the config loads. Turning it on for a database migrate already built proposes a +`drop-check` for each CHECK it now leaves out, which is gated behind `--allow drop-check`. + +## Schema conventions: names for objects no code addresses + +Some database objects have names that no program ever reads: a primary-key constraint, most +foreign-key constraints, the index on a foreign key. Their names, and a few physical choices, +come from the project's **schema conventions**, selected in `.metaobjects/config.json`: + +```json +{ "schema_version": 1, "migrate": { "conventions": "efcore" } } +``` + +| Decides | `default` (absent) | `efcore` | +|---|---|---| +| Primary-key constraint name (Postgres; SQLite and D1 emit an unnamed primary key) | `
_pkey` | `PK_
` | +| Foreign-key constraint name | `
__fk` | `FK_
__` | +| Index on a foreign key | none | `IX_
_`, unless the primary key or another index (a partial one included) already starts with the FK's columns; an FK index another FK index covers is dropped, so FK order does not matter | +| Unique key (`identity.secondary`, a `@unique` field) | a unique index; a live UNIQUE constraint is also accepted | a UNIQUE constraint when a foreign key references it, else a unique index | + +`efcore` is a reference preset. It reproduces one ORM's documented naming to show what the +mechanism does; the mechanism is general, and a project whose database was built by another tool +supplies its own conventions (below). `efcore` follows EF Core's own conventions, including, on +PostgreSQL (Npgsql), its truncation of a +name longer than 63 characters to 62 characters and `~`. EF Core's SQLite provider truncates +nothing, so on SQLite and D1 neither does `efcore`. A database first built by EF Core +migrations and one built by `meta migrate` then agree on these objects. Where the EF Core model +carried something its conventions do not (a hand-named key, an index dropped by raw SQL), declare +it in metadata. A unique key keeps the name its metadata node declares (EF Core would call an +alternate key `AK_…`; name the node that if you want EF's). A key with a `DESC` column or a +non-btree `@using` stays an index, since a UNIQUE constraint carries neither. `default` is what +migrate has always emitted, and it has no preference between the two kinds of unique key: a +database whose keys are UNIQUE constraints (Drizzle's `unique()` makes them) is left as it is. +A conventions object says which kind it wants with `uniqueKeyKind`: `index`, `constraint`, or +`either`, which is `default`'s answer. + +Names code does address are not conventions. Tables, views and columns resolve in core +naming (`naming`, and `@table` / `@column`), the same in every port. A constraint or index a +program compares against (a unique-violation handler checking the constraint name) is +declared in metadata: `@constraintName` on an `identity.reference`, or the index node's own +name. A declared name always wins over a convention. + +A project can supply its own conventions from `metaobjects.config.ts`, which wins over the +preset name. Start from a preset so every member is present; an object missing one is refused +by name: + +```ts +import { defineConfig } from "@metaobjectsdev/cli"; +import { efcoreConventions } from "@metaobjectsdev/migrate-ts"; + +export default defineConfig({ + generators: [/* … */], + schemaConventions: { ...efcoreConventions, id: "acme", primaryKeyName: ({ table }) => `pk_${table}` }, +}); +``` + +**Changing the conventions of an existing database.** On Postgres, a primary key, FK, CHECK or +index whose definition is unchanged and whose name differs is renamed in place (`RENAME +CONSTRAINT` / `ALTER INDEX … RENAME`), never dropped and re-created; new FK indexes are created. +A unique key that is now wanted as a UNIQUE constraint instead of a plain index is promoted in +place (`ADD CONSTRAINT … UNIQUE USING INDEX`), so an FK that relies on it keeps working. A key the +conventions want as a plain index (`uniqueKeyKind` answers `index`, as `efcore` does for a key +no FK references) is converted the other way, which has no in-place form: the constraint is dropped and the index re-created, and each FK +onto that key is dropped first and added back after, so the plan is gated behind `--allow +drop-index` and `--allow drop-fk`. `meta verify --db` reports each of these as drift. SQLite +cannot rename a constraint and rebuilds the table instead. + ## How CHECK constraints and expression indexes are compared Postgres rewrites an expression before it stores it (`"k" IN ('x')` becomes `k = 'x'::text`, diff --git a/docs/features/migrations/physical-naming.md b/docs/features/migrations/physical-naming.md index dbfbcd34ac..e5576c84f0 100644 --- a/docs/features/migrations/physical-naming.md +++ b/docs/features/migrations/physical-naming.md @@ -25,11 +25,18 @@ every runtime `ObjectManager` and `verify` read it, so they always produce the s | Key | Values | Applies to | Default | |---|---|---|---| -| `columns` | `snake_case`, `literal`, `kebab-case` | a field with no `@column` | `snake_case` | +| `columns` | `snake_case`, `literal`, `kebab-case`, `upperFirst` | a field with no `@column` | `snake_case` | | `tables` | `snake_case`, `literal` | a source with no `@table`/`@view`/`@proc`/`@function`: its `name`, else the object's name made plural | `snake_case` | An unknown key or value fails when the file is read. Index and constraint names are unchanged. +`upperFirst` upper-cases the field name's first letter and changes nothing +else: `createdAt` becomes `CreatedAt`, `requestURL` becomes `RequestURL` and `address_line1` +becomes `Address_line1`. It is not a word split, so it is not PascalCase. A schema whose columns +are the field names with a capital first letter (EF Core's default, for one) then needs `@column` +only where the database spells a name differently. It is a column rule only; `literal` tables +already keep an object name verbatim. + **Upgrade every port of an estate together.** A 1.0.x TypeScript tool refuses the new `naming` key, but a 1.0.x C#, Java or Python tool ignores an unknown top-level key silently and keeps the old column names, so a mixed estate disagrees about the schema without any error. diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index 02f28069b3..0c1de34c9c 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -381,6 +381,25 @@ generator copies `DecimalWire.cs` and `DecimalRangeEndpointFilter.cs`. Do not st attributes or delete `DecimalWire.g.cs` to change the output: they also check input. Set `wire` and regenerate. +A `field.duration` is a `TimeSpan` carrying `[DurationWire.Iso]`, which reads and writes the +ISO-8601 form every port shares (`PT1H30M`, and on input `P1W`, `P2DT3H`) instead of +System.Text.Json's `01:30:00`. It writes up to seven fractional digits, a `TimeSpan` tick, and +truncates a longer input fraction toward zero. On Npgsql it is an `interval` column. EF reads an +interval through Npgsql's own `TimeSpan` conversion, which (as of Npgsql 8.0.5) silently drops a +negative month or year part (`'-1 mons'` reads as `PT0S`) and wraps a day count past `TimeSpan`'s +range, so the generated context maps each duration column through `NpgsqlInterval` instead +(`DurationIntervalMapping`, `DurationIntervalArrayMapping`, emitted beside `AppDbContext`) and +refuses either one on read. LINQ still translates as it does on Npgsql's own mapping: `TimeSpan` +members (`TotalSeconds`, `Hours`, ...), a timestamp plus a duration, comparison, ordering, `Min`, +`Max`, `EF.Functions.Sum`, and `Contains` / `Count` / `Any` on a duration array. A compiled model +(`dotnet ef dbcontext optimize`) keeps the mapping and the refusal; EF Core 8's compiled model +carries no element type, so there a duration array reads and writes but is not queryable as a +collection. On any other provider a value converter stores the 64-bit microsecond count, so sorting +and range filters follow the span. Both columns hold microseconds, so a tick below one is dropped +on write. `TimeSpan`'s own range is the only limit: a longer span is refused, never wrapped. +`DurationWire.g.cs` is emitted once per namespace that has a duration, and an owned `routes` +generator also copies `DurationWire.cs`. + ### `Names` — the physical names, as constants `names` is opt-in — list `new MetaObjects.Codegen.Generators.NamesGenerator()` in @@ -443,6 +462,11 @@ no port's codegen names a primary key (Postgres migrations hardcode `
_pke emits an unnamed PK), and carrying that would restate a migrate-only formula in the artifact built to stop exactly that. +A `@storage: flattened` value-object field gets one constant per member column +(`ShipToStreetColumn`, keyed `shipTo.street` in `ColumnsByField`). Its own `ShipToColumn` is still +emitted but `[Obsolete]`: the table has no column by that name. Declaring it raises no warning; +reading it does, so a project that never reads it builds clean under `TreatWarningsAsErrors`. + `const`, not `static readonly`: a `[Table("...")]`/`[Column("...")]` attribute argument must be a compile-time constant, and those two attributes are the whole reason this artifact can replace a literal there rather than sit beside @@ -495,7 +519,9 @@ complete on every class, inherited entries included. Column names come from the project's `naming` setting in `.metaobjects/config.json` (ADR-0058), the one every port reads, default `snake_case` (what `meta migrate` creates). A database whose columns are named exactly like the fields declares -`"naming": { "columns": "literal" }`; through 1.0 that was this port's default. `dotnet meta gen` +`"naming": { "columns": "literal" }`; through 1.0 that was this port's default. One whose columns +are the field names with the first letter upper-cased (`requestURL` → `RequestURL`) declares +`"upperFirst"`. `dotnet meta gen` and `dotnet meta verify --codegen` both read the setting, so a plain run and the drift gate resolve the identical column strings; `--column-naming` is refused ([physical-naming.md](../features/migrations/physical-naming.md)). `verify --codegen` says so diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-allowlist.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-allowlist.ts index 386b10e1e4..e349d9cf03 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-allowlist.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-allowlist.ts @@ -15,6 +15,7 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, type FilterOp, } from "@metaobjectsdev/metadata"; export { FILTER_OPS, OPS_BY_SUBTYPE, opsForSubType, type FilterOp }; @@ -33,6 +34,7 @@ export const FILTER_VALUE_FORMATS = [ FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, ] as const; export type FilterValueFormat = (typeof FILTER_VALUE_FORMATS)[number]; diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts index c4b3066f5d..644e9a9616 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts @@ -15,8 +15,11 @@ import { ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT, looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant, } from "./filter-value-format.js"; -import { FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_TIMESTAMP } from "@metaobjectsdev/metadata"; +import { FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_TIMESTAMP } from "@metaobjectsdev/metadata"; import { canonicalDecimal } from "../decimal-wire.js"; +import { + DURATION_MICROS_MAX, DURATION_MICROS_MIN, durationToStoredMicros, formatPostgresInterval, parseDurationLiteral, +} from "../duration-wire.js"; // biome-ignore lint/suspicious/noExplicitAny: dynamic dispatch over user's Drizzle table type AnyTable = any; @@ -344,6 +347,16 @@ function coerce(value: unknown, rule: FilterFieldRule, col: unknown, field: stri if (dialect === "mysql") return sql`CAST(${canonical} AS DECIMAL(65,30))`; return dialect === "sqlite" ? Number(canonical) : canonical; } + // A duration bound is spelled the way the column stores it: the microsecond integer on SQLite / + // D1 / MySQL, the interval spelling on Postgres (which refuses a signed ISO-8601 duration). Both + // compare and order as the span does. + if (rule.format === FIELD_SUBTYPE_DURATION) { + const micros = parseDurationLiteral(s) as bigint; + // A well-formed span past 64 bits of microseconds is no stored value on any database, so it is + // a bad operand (400), not an error out of the binder. + if (micros > DURATION_MICROS_MAX || micros < DURATION_MICROS_MIN) throw invalidValue(field, op, s, DURATION_IN_RANGE); + return dialect === "sqlite" || dialect === "mysql" ? durationToStoredMicros(micros) : formatPostgresInterval(micros); + } // A timestamp bound is compared in the same UTC spelling the generated schemas store: // on SQLite/D1 the column is TEXT and compares as text. An instant field (`instant`, i.e. // not @localTime) reads a zoneless or date-only bound as UTC, as its write schema does — @@ -364,6 +377,7 @@ function coerce(value: unknown, rule: FilterFieldRule, col: unknown, field: stri /** `expected` for a non-whole bound on an integer-valued field (`integer: true`). */ const INTEGER_EXPECTED = "integer (a whole number)"; +const DURATION_IN_RANGE = "duration within the 64-bit microsecond range"; function coerceAs(value: unknown, subType: string, field: string, op: string, dateValues?: boolean): unknown { if (value === null || value === undefined) return null; diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-value-format.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-value-format.ts index 5def0974f6..cf8a3f1097 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-value-format.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-value-format.ts @@ -17,9 +17,11 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, } from "@metaobjectsdev/metadata"; import type { FilterValueFormat } from "./filter-allowlist.js"; import { canonicalDecimal } from "../decimal-wire.js"; +import { parseDurationLiteral } from "../duration-wire.js"; const DATE_RE = /^(\d{4})-(\d{2})-(\d{2})$/; const TIME_RE = /^(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d{1,9})?)?$/; @@ -48,6 +50,7 @@ const CHECKS: Readonly boolean>> = { [FIELD_SUBTYPE_TIMESTAMP]: isTimestamp, [FIELD_SUBTYPE_UUID]: (s) => UUID_RE.test(s), [FIELD_SUBTYPE_DECIMAL]: (s) => canonicalDecimal(s) !== undefined, + [FIELD_SUBTYPE_DURATION]: (s) => parseDurationLiteral(s) !== undefined, }; /** The `expected` member of the `invalid_filter_value` envelope, per format. */ @@ -57,6 +60,7 @@ export const FORMAT_EXPECTED: Readonly> = { [FIELD_SUBTYPE_TIMESTAMP]: "timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", [FIELD_SUBTYPE_UUID]: "uuid (8-4-4-4-12 hex)", [FIELD_SUBTYPE_DECIMAL]: "decimal number (digits, optional fraction, optional exponent)", + [FIELD_SUBTYPE_DURATION]: "duration (ISO 8601, e.g. PT1H30M; no years or months)", }; /** `expected` for a temporal rule that does not say WHICH temporal format it is. */ diff --git a/examples/advanced-modeling/codegen/runtime/duration-wire.ts b/examples/advanced-modeling/codegen/runtime/duration-wire.ts new file mode 100644 index 0000000000..732659f9b0 --- /dev/null +++ b/examples/advanced-modeling/codegen/runtime/duration-wire.ts @@ -0,0 +1,7 @@ +// The field.duration wire rule lives in @metaobjectsdev/metadata (shared with migrate-ts); +// re-exported here so the runtime's callers import it beside the decimal wire helpers. +export { + DURATION_PATTERN, DURATION_RE, parseDurationLiteral, formatDuration, formatPostgresInterval, + canonicalDuration, parseDurationInput, parsePostgresInterval, durationToWire, durationToStoredMicros, + DURATION_MICROS_MIN, DURATION_MICROS_MAX, +} from "@metaobjectsdev/metadata"; diff --git a/examples/advanced-modeling/codegen/templates/drizzle-schema.ts b/examples/advanced-modeling/codegen/templates/drizzle-schema.ts index 93e85d56a0..74df06cb0d 100644 --- a/examples/advanced-modeling/codegen/templates/drizzle-schema.ts +++ b/examples/advanced-modeling/codegen/templates/drizzle-schema.ts @@ -18,7 +18,7 @@ import { import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { - mapColumnType, PG_IDENTITY_CAPABLE_FNS, + mapColumnType, PG_IDENTITY_CAPABLE_FNS, DURATION_COLUMN_FN, type ColumnSpec, type EnumIntCustomType, } from "@metaobjectsdev/codegen-ts/engine"; import { tableNameFromEntity } from "@metaobjectsdev/codegen-ts/engine"; @@ -28,6 +28,7 @@ import { } from "@metaobjectsdev/codegen-ts/engine"; import { resolveTableSchema } from "@metaobjectsdev/metadata"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; import { collectTphSubtypeFields, tphConcreteSubtypes } from "./tph-discriminator.js"; @@ -46,7 +47,8 @@ export function renderDrizzleSchema( obj: MetaObject, ctx: RenderContext, /** - * OUT param: the int-backed-enum codec const names this table DECLARES. A caller that + * OUT param: the local column-codec const names this table DECLARES (int-backed enums, and + * the duration codec). A caller that * emits another artifact referencing the same codecs into the SAME module — entity-file's * write-through replica view — passes a set here and hands it to that emitter, which then * references the consts instead of re-declaring them. A duplicate module-scope `const` is @@ -211,6 +213,8 @@ export function renderDrizzleSchema( // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by // const name so a shared enum used by two fields of the SAME entity emits once. const enumIntTypes = new Map(); + // The field.duration codec: one declaration serves every duration column of the table. + let usesDurationCodec = false; for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -225,6 +229,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, columnNameExpr(child, spec.dbName), child, ctx, isPk, pkGeneration, fkInfo, isComposite, pkg, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -244,6 +249,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, columnNameExpr(child, spec.dbName), child, ctx, false, undefined, fkMap.get(child.name), isComposite, pkg, obj.name, true, @@ -403,9 +409,11 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} .map((t) => renderEnumIntCustomType(t, importModule)); if (declaredEnumIntCodecs !== undefined) { for (const name of enumIntTypes.keys()) declaredEnumIntCodecs.add(name); + if (usesDurationCodec) declaredEnumIntCodecs.add(DURATION_COLUMN_FN); } - const blocks: Code[] = [...enumIntBlocks, tableBlock]; + const durationBlocks = usesDurationCodec ? [renderDurationCustomType(dialect, importModule)] : []; + const blocks: Code[] = [...durationBlocks, ...enumIntBlocks, tableBlock]; if (relationsBlock !== null) blocks.push(relationsBlock); return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); } @@ -518,7 +526,9 @@ function renderColumn( const fnSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); let baseCall: Code; if (spec.fnOptions !== undefined && Object.keys(spec.fnOptions).length > 0) { diff --git a/examples/advanced-modeling/codegen/templates/duration-wire-helper.ts b/examples/advanced-modeling/codegen/templates/duration-wire-helper.ts new file mode 100644 index 0000000000..81aeaac06e --- /dev/null +++ b/examples/advanced-modeling/codegen/templates/duration-wire-helper.ts @@ -0,0 +1,213 @@ +// The wire rule for a field.duration, as text a generated model module carries as local code. +// Generated model code imports nothing from MetaObjects, so the rule is emitted, not imported; +// metadata/src/core/field/duration-wire.ts holds the same rule and both are held to +// fixtures/duration-wire/vectors.json (test/duration-wire-helper.test.ts evaluates the text +// emitted here against them). +// +// Accepted: optional weeks (exactly 7 days) and days, then hours, minutes and seconds; the seconds +// fraction may have any number of digits. Canonical: `[-]PT[H][M][[.]S]` — hours never +// folded into days, zero components left out, zero is `PT0S`, one leading `-`. A span is counted in +// microseconds, so the fraction keeps six digits and the rest is truncated toward zero. +// +// The emitted code counts a span in microseconds as a `bigint`, so it has no limit of its own: a +// span is as long as the stack it is stored in allows. The only limit here is a column's — the +// 64-bit integer a SQLite, D1 or MySQL column holds (refused past it, never wrapped). Postgres's +// interval has its own range, which Postgres enforces. The emitted code calls `BigInt(...)` and +// writes no bigint literal, so it needs the ES2020 `BigInt` global and nothing newer. + +import { code, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_PATTERN } from "@metaobjectsdev/metadata"; +import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_COLUMN_FN } from "@metaobjectsdev/codegen-ts/engine"; + +/** The accepted ISO-8601 duration form as a JS regex literal — the metadata's own pattern. */ +export const DURATION_REGEX_LITERAL = `/${DURATION_PATTERN}/`; + +/** The Zod expression that accepts a field.duration as sent (and as read back): the accepted form. */ +export const ZOD_DURATION_EXPR = `z.string().regex(${DURATION_REGEX_LITERAL})`; + +export const DURATION_WRITE_HELPER = "durationWrite"; + +/** The Zod expression of a duration WRITE field: the accepted form, rewritten to the canonical one, + * so equal spans are equal strings on the wire and in a jsonb value object. */ +export const ZOD_DURATION_WRITE_EXPR = `${ZOD_DURATION_EXPR}.transform(${DURATION_WRITE_HELPER})`; + +/** The accepted form with its components captured: sign, weeks, days, hours, minutes, seconds, + * fraction. Equivalent to DURATION_PATTERN (the same lookaheads refuse `P`, `PT` and `P1DT`). */ +const ISO_PARTS_SOURCE = String.raw`/^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/`; + +/** The default (`IntervalStyle = postgres`) text of a Postgres interval: signed year / mon / day + * parts, then an optionally signed `H:MM:SS[.ffffff]`. */ +const PG_INTERVAL_SOURCE = String.raw`/^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/`; + +/** Indent every line of `block` by `n` spaces (blank lines stay blank). */ +function indent(block: string, n: number): string { + const pad = " ".repeat(n); + return block.split("\n").map((l) => (l === "" ? l : pad + l)).join("\n"); +} + +/** Microsecond unit constants, as `BigInt(...)` calls (no bigint literal). */ +const UNIT_LINES = [ + "const ZERO = BigInt(0);", + "const SECOND = BigInt(1000000);", + "const MINUTE = BigInt(60000000);", + "const HOUR = BigInt(3600000000);", + "const DAY = BigInt(86400000000);", + "const WEEK = BigInt(604800000000);", +].join("\n"); + +/** `parseIso(text)`: the microseconds of an accepted ISO-8601 duration, or undefined. */ +const PARSE_ISO_FN = [ + "const parseIso = (text: string): bigint | undefined => {", + ` const p = ${ISO_PARTS_SOURCE}.exec(text);`, + " if (p === null) return undefined;", + " const mag = BigInt(p[2] ?? 0) * WEEK + BigInt(p[3] ?? 0) * DAY + BigInt(p[4] ?? 0) * HOUR", + ' + BigInt(p[5] ?? 0) * MINUTE + BigInt(p[6] ?? 0) * SECOND + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6));', + ' return p[1] === "-" ? -mag : mag;', + "};", +].join("\n"); + +/** `isoText(micros)`: the canonical wire form. */ +const ISO_TEXT_FN = [ + "const isoText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + " const s = rest / SECOND;", + " const f = rest % SECOND;", + ' let out = "PT";', + ' if (h > ZERO) out += String(h) + "H";', + ' if (m > ZERO) out += String(m) + "M";', + ' if (s > ZERO || f > ZERO) out += String(s) + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + "S";', + ' if (out === "PT") return "PT0S";', + ' return neg ? "-" + out : out;', + "};", +].join("\n"); + +export const DURATION_WRITE_HELPER_DECL = [ + "/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten", + " * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one", + " * leading \"-\" for a negative span. Counted in bigint microseconds, so any length is exact and a", + " * fraction past six digits is truncated toward zero. Generated. */", + `function ${DURATION_WRITE_HELPER}(text: string): string {`, + indent(UNIT_LINES, 2), + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + " const micros = parseIso(text);", + " return micros === undefined ? text : isoText(micros);", + "}", +].join("\n"); + +/** The physical column type a duration column's `customType` declares, per dialect: the + * Postgres-native interval; a 64-bit integer count of microseconds on SQLite / D1 (INTEGER) and + * MySQL (BIGINT) — the persistence corpus's canonical schemas, and what `meta migrate` writes. */ +export function durationColumnDataType(dialect: Dialect): string { + return dialect === "postgres" ? "interval" : dialect === "mysql" ? "bigint" : "integer"; +} + +/** + * The body of the duration column's declaration, around the `customType` reference (which the + * caller supplies as an import). `[before, after]`: `before + customType + after` is the whole + * declaration. Exported so a test can evaluate the exact text a generated module carries. + * + * - `toDriver` binds what the column holds: on Postgres the interval spelling `[-]HH:MM:SS[.f]` + * (Postgres refuses a signed ISO-8601 duration, and returns a value written this way in exactly + * this spelling); on SQLite, D1 and MySQL the microsecond count — a number when it is a safe + * integer, else a bigint — refused past the 64-bit range, which is that column's limit. It takes + * the value as an ISO-8601 duration (what the generated write schema hands it), or as what + * runtime-ts already binds a filter operand in (the Postgres text; the microsecond integer), and + * throws for anything else. + * - `fromDriver` reads what the driver returns — the microsecond integer (a number, a bigint or a + * digit string), ISO-8601 text, Postgres interval text in the default `IntervalStyle`, or a + * node-postgres interval object — as the canonical ISO-8601 form. An interval with a year or + * month part is not a fixed-length span: reading one throws. + */ +export function durationColumnDeclParts(dialect: Dialect): [string, string] { + const postgres = dialect === "postgres"; + const before = [ + "/** A field.duration column. Binds " + (postgres + ? "the Postgres interval spelling [-]HH:MM:SS[.f] (Postgres refuses a signed ISO-8601 duration);" + : "the span as a 64-bit integer count of microseconds (refused past that range);"), + " * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month", + " * part. Counted in bigint microseconds, so the column is the only limit. Generated. */", + `const ${DURATION_COLUMN_FN} = (() => {`, + indent(UNIT_LINES, 2), + ` const INT64_MAX = BigInt("9223372036854775807");`, + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + indent([ + "const pgText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + ' const two = (n: bigint): string => String(n).padStart(2, "0");', + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + ' let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND);', + " const f = rest % SECOND;", + ' if (f > ZERO) out += "." + String(f).padStart(6, "0").replace(/0+$/, "");', + ' return neg ? "-" + out : out;', + "};", + "const stored = (micros: bigint): number | bigint => {", + " if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) {", + ' throw new Error("the span " + isoText(micros) + " does not fit the 64-bit microsecond column a field.duration is stored in");', + " }", + " const n = Number(micros);", + " return Number.isSafeInteger(n) ? n : micros;", + "};", + "const notFixed = (v: unknown): Error =>", + ' new Error("the interval " + JSON.stringify(v) + " has a year or month part, so it is not a fixed-length field.duration");', + "const read = (v: unknown): bigint => {", + ' if (typeof v === "bigint") return v;', + ' if (typeof v === "number" && Number.isInteger(v)) return BigInt(v);', + ' if (typeof v === "string") {', + " if (/^-?\\d+$/.test(v)) return BigInt(v);", + " const iso = parseIso(v);", + " if (iso !== undefined) return iso;", + ` const p = v.length === 0 ? null : ${PG_INTERVAL_SOURCE}.exec(v);`, + ' if (p === null) throw new Error("not a field.duration value: " + JSON.stringify(v));', + " if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) throw notFixed(v);", + " let time = ZERO;", + " if (p[5] !== undefined) {", + ' time = BigInt(p[5]) * HOUR + BigInt(p[6] ?? 0) * MINUTE + BigInt(p[7] ?? 0) * SECOND + BigInt((p[8] ?? "").padEnd(6, "0"));', + ' if (p[4] === "-") time = -time;', + " }", + " return BigInt(p[3] ?? 0) * DAY + time;", + " }", + ' if (v !== null && typeof v === "object") {', + " const o = v as Record;", + " if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) throw notFixed(v);", + " const whole = (n: number | undefined): bigint => BigInt(Math.trunc(n ?? 0));", + " // postgres-interval carries microseconds as a fractional millisecond count.", + " return whole(o.days) * DAY + whole(o.hours) * HOUR + whole(o.minutes) * MINUTE + whole(o.seconds) * SECOND", + " + BigInt(Math.round((o.milliseconds ?? 0) * 1000));", + " }", + ' throw new Error("not a field.duration value: " + JSON.stringify(v));', + "};", + ].join("\n"), 2), + " return ", + ].join("\n"); + const after = [ + `<{ data: string; driverData: ${postgres ? "string" : "number | bigint | string"} }>({`, + ` dataType: () => ${JSON.stringify(durationColumnDataType(dialect))},`, + ` toDriver: (value) => ${postgres ? "pgText" : "stored"}(read(value)),`, + " fromDriver: (value) => isoText(read(value)),", + " });", + "})();", + ].join("\n"); + return [before, after]; +} + +/** A duration column's Drizzle `customType` declaration, emitted ahead of the table or view that + * uses it (once per module — see enum-int-codec.ts for why the order and the once matter). */ +export function renderDurationCustomType(dialect: Dialect, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const [before, after] = durationColumnDeclParts(dialect); + return code` +${before}${customTypeSym}${after} +`; +} diff --git a/examples/advanced-modeling/codegen/templates/field-meta.ts b/examples/advanced-modeling/codegen/templates/field-meta.ts index 6182a31306..72549fd876 100644 --- a/examples/advanced-modeling/codegen/templates/field-meta.ts +++ b/examples/advanced-modeling/codegen/templates/field-meta.ts @@ -22,6 +22,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STRING_FORMAT, @@ -43,6 +44,7 @@ import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; import { zodEnumExpr } from "./zod-enum.js"; import { viewForContext } from "@metaobjectsdev/codegen-ts/engine"; import { ZOD_INET_EXPR } from "./net-regex.js"; +import { ZOD_DURATION_EXPR } from "./duration-wire-helper.js"; // --------------------------------------------------------------------------- // inferViewKind @@ -140,6 +142,9 @@ export function zodTypeFor(field: MetaField, timestampMode: "date" | "string" = // (v4 or v6 literal). Zod-version-agnostic regex union (Zod 4 removed // `z.string().ip()`); see net-regex.ts. @lenient → a plain string. return field.attr(FIELD_ATTR_LENIENT) === true ? "z.string()" : ZOD_INET_EXPR; + case FIELD_SUBTYPE_DURATION: + // An ISO-8601 duration string; the column codec reads every row in the canonical form. + return ZOD_DURATION_EXPR; case FIELD_SUBTYPE_BOOLEAN: return "z.boolean()"; case FIELD_SUBTYPE_DATE: diff --git a/examples/advanced-modeling/codegen/templates/filter-allowlist.ts b/examples/advanced-modeling/codegen/templates/filter-allowlist.ts index 4a0cd087fd..ec7c195b7c 100644 --- a/examples/advanced-modeling/codegen/templates/filter-allowlist.ts +++ b/examples/advanced-modeling/codegen/templates/filter-allowlist.ts @@ -13,6 +13,7 @@ import { FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_ATTR_LOCAL_TIME, opsForField, @@ -70,6 +71,9 @@ const VALUE_FORMAT_SUBTYPES = new Set([ FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_DECIMAL, + // An ISO-8601 duration: runtime-ts refuses a value that is not one, and the column codec binds + // the dialect's spelling of it. + FIELD_SUBTYPE_DURATION, ]); /** Subtypes whose values are whole numbers — `field.currency` is integer minor units. */ diff --git a/examples/advanced-modeling/codegen/templates/inferred-types.ts b/examples/advanced-modeling/codegen/templates/inferred-types.ts index 1eec56f318..4b87f2f9bc 100644 --- a/examples/advanced-modeling/codegen/templates/inferred-types.ts +++ b/examples/advanced-modeling/codegen/templates/inferred-types.ts @@ -27,6 +27,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_REQUIRED, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_VALUE_TYPE, @@ -215,6 +216,8 @@ const SCALAR_TS_BY_SUBTYPE: Record = { // type, same as uuid). Other ports bind to their native URI/IP type. [FIELD_SUBTYPE_URI]: "string", [FIELD_SUBTYPE_INET]: "string", + // field.duration is an ISO-8601 duration string: TS has no native duration type. + [FIELD_SUBTYPE_DURATION]: "string", [FIELD_SUBTYPE_INT]: "number", [FIELD_SUBTYPE_LONG]: "number", [FIELD_SUBTYPE_DOUBLE]: "number", diff --git a/examples/advanced-modeling/codegen/templates/projection-decl.ts b/examples/advanced-modeling/codegen/templates/projection-decl.ts index e9ab1e45d2..a7ccbf0f93 100644 --- a/examples/advanced-modeling/codegen/templates/projection-decl.ts +++ b/examples/advanced-modeling/codegen/templates/projection-decl.ts @@ -21,7 +21,7 @@ import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; -import { physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; +import { columnExpr, physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; import { renderFilterAllowlist, renderSortAllowlist } from "./filter-allowlist.js"; import { renderFilterType } from "./filter-type.js"; import { buildUiFieldDescriptor, resourcePath } from "@metaobjectsdev/codegen-ts/engine"; @@ -156,11 +156,8 @@ export function renderProjectionDecl( // A6 — reference the constant whenever the artifact is in the run AND carries this // field. A lookup MISS (a TPH subtype field the base's names artifact never saw) is // normal, not a divergence — fall back to the literal, same as every other §A6 site. - const namesEntry = names?.resolved.fields[f.name]; - const dbColExpr: Code = - names !== undefined && namesEntry !== undefined - ? code`${names.symbol}.fields.${f.name}.column` - : code`${JSON.stringify(dbCol)}`; + // A flattened field has no `column` of its own (its members do), so it takes the literal. + const dbColExpr: Code = columnExpr(names, f.name, dbCol); // #356 — a projection's descriptor is the same shape the entity descriptor emits, // and now literally the SAME derivation: `buildUiFieldDescriptor` is what // `renderEntityConstants` reads, so the view kind, the label and the currency meta diff --git a/examples/advanced-modeling/codegen/templates/view-decl.ts b/examples/advanced-modeling/codegen/templates/view-decl.ts index da4a523058..eec5e9ed44 100644 --- a/examples/advanced-modeling/codegen/templates/view-decl.ts +++ b/examples/advanced-modeling/codegen/templates/view-decl.ts @@ -14,8 +14,9 @@ import { type MetaField, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, } from "@metaobjectsdev/metadata"; import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; -import { mapColumnType, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; +import { mapColumnType, DURATION_COLUMN_FN, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { zodTypeFor } from "./field-meta.js"; import { columnExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; @@ -25,7 +26,8 @@ export interface ViewDeclOpts { /** Drives the timestamp column TS type (Date vs string) in the view declaration. */ readonly timestampMode: "date" | "string"; /** - * Int-backed-enum codec const names ALREADY declared in the module this view lands in. + * Local column-codec const names (int-backed enums, the duration codec) ALREADY declared in + * the module this view lands in. * A write-through entity's table and its replica view share one file, and both reference * the same codecs — so the view references these by name and declares only what the table * did not (a derived field's codec, which the write table omits). Absent = the view owns @@ -88,6 +90,8 @@ function viewColumnLine( opts: ViewDeclOpts, /** Codecs this view needs, collected for the caller to DECLARE — see below. */ enumIntTypes: Map, + /** The duration codec's name, when a column needs it — declared by the caller, as above. */ + usesDuration: Set, ): Code { const { dialect, columnNamingStrategy, timestampMode } = opts; const spec = mapColumnType(f, dialect, columnNamingStrategy, timestampMode); @@ -99,10 +103,14 @@ function viewColumnLine( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + // The field.duration codec is a local const too; collected under its name, declared once. + if (spec.durationCustomType !== undefined) usesDuration.add(spec.durationCustomType.fnConstName); const colSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const optsArg = spec.fnOptions && Object.keys(spec.fnOptions).length > 0 ? `, ${JSON.stringify(spec.fnOptions)}` @@ -164,7 +172,8 @@ export function renderExistingViewDecl( const { viewFn, core: viewModule } = dialectModule(opts.dialect); const viewSym = imp(`${viewFn}@${viewModule}`); const enumIntTypes = new Map(); - const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes)); + const usesDuration = new Set(); + const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes, usesDuration)); const viewNameExpr = typeof viewName === "string" ? code`${JSON.stringify(viewName)}` : viewName; // @schema — a view lands in a schema exactly as a table does, and migrate qualifies the @@ -188,6 +197,9 @@ export function renderExistingViewDecl( .filter((t) => alreadyDeclared === undefined || !alreadyDeclared.has(t.fnConstName)) .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) .map((t) => renderEnumIntCustomType(t, viewModule)); + if (usesDuration.has(DURATION_COLUMN_FN) && (alreadyDeclared === undefined || !alreadyDeclared.has(DURATION_COLUMN_FN))) { + enumIntBlocks.unshift(renderDurationCustomType(opts.dialect, viewModule)); + } const viewBlock = code` // View declaration — Drizzle uses this for typed SELECT queries. diff --git a/examples/advanced-modeling/codegen/templates/zod-validators.ts b/examples/advanced-modeling/codegen/templates/zod-validators.ts index 917a215da5..195fddfacb 100644 --- a/examples/advanced-modeling/codegen/templates/zod-validators.ts +++ b/examples/advanced-modeling/codegen/templates/zod-validators.ts @@ -17,6 +17,7 @@ import { FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DECIMAL, FIELD_ATTR_PRECISION, FIELD_ATTR_SCALE, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_ARRAY, IDENTITY_ATTR_FIELDS, IDENTITY_ATTR_GENERATION, @@ -28,7 +29,9 @@ import { AUTO_SET_ON_CREATE, AUTO_SET_ON_UPDATE, VALIDATOR_ATTR_MAX, VALIDATOR_ATTR_MIN, VALIDATOR_ATTR_PATTERN, VALIDATOR_SUBTYPE_COMPARISON, VALIDATOR_SUBTYPE_REQUIRED_WHEN, - VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_SUBTYPE_EXACTLY_ONE, + VALIDATOR_ATTR_CONDITION, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, EXPR_FN_COALESCE, + FILTER_OP_IS_NULL, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, parseDurationLiteral, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, GENERATION_INCREMENT, GENERATION_UUID, @@ -48,6 +51,7 @@ import { } from "./date-time-regex.js"; import { renderDocsFor } from "./jsdoc.js"; import { DECIMAL_WRITE_HELPER_DECL, zodDecimalWriteExpr } from "./decimal-wire-helper.js"; +import { DURATION_WRITE_HELPER_DECL, ZOD_DURATION_EXPR, ZOD_DURATION_WRITE_EXPR } from "./duration-wire-helper.js"; import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumZodConstName } from "./enums-file.js"; @@ -86,6 +90,13 @@ function decimalHelperSince(since: number): string { return decimalWriteUses > since ? `${DECIMAL_WRITE_HELPER_DECL}\n\n` : ""; } +// The same device for the duration write rule (duration-wire-helper.ts). +let durationWriteUses = 0; + +function durationHelperSince(since: number): string { + return durationWriteUses > since ? `${DURATION_WRITE_HELPER_DECL}\n\n` : ""; +} + /** True when this object declares at least one @autoSet timestamp field * (onCreate or onUpdate). Drives whether codegen emits the #203 @@ -260,13 +271,16 @@ function isInsertRequiredPk(field: MetaField, assignedPk: Set): boolean } // ── Entity-scoped cross-field validators on the full (insert) record ────────── -// validator.comparison / requiredWhen / presentIff / atLeastOne are enforced on the -// INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged +// validator.comparison / requiredWhen / presentIff / atLeastOne / exactlyOne are enforced on +// the INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged // row, so the Update schema is deliberately left alone (the DB CHECK stays the guard // there). Semantics mirror the migrate-ts CHECK derivation: absent / null / undefined is // SQL NULL, "" is NOT null; a comparison with a NULL side passes; requiredWhen is violated // only when the gate EQUALS the literal and the target is null; presentIff requires -// (target != null) === (gate equals literal), a null gate being "not equal". +// (target != null) === (gate equals literal), a null gate being "not equal"; exactlyOne +// requires exactly one non-null field. A rule with an @condition is checked only when the +// condition is TRUE: the CHECK is NOT (condition) OR (rule), so a condition that is false or +// NULL (three-valued, as SQL evaluates it) lets the row through. const CROSS_FIELD_OPS: Record = { gt: ">", gte: ">=", lt: "<", lte: "<=", ne: "!==", eq: "===", @@ -278,7 +292,7 @@ const NUMERIC_SUBTYPES: ReadonlySet = new Set([ ]); const ORDERED_SUBTYPES: ReadonlySet = new Set([ - FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, + FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_DURATION, ]); /** Subtypes with equality but no ordering: a pair of the same one may be tested with eq / ne. */ @@ -287,7 +301,7 @@ const EQUALITY_SUBTYPES: ReadonlySet = new Set([ ]); /** Whether a comparison between these two fields has a sound meaning, as in every port: - * numerics compare with each other, strings and temporals only with their own subtype and + * numerics compare with each other, strings, temporals and durations only with their own subtype and * @localTime flag (a date against a timestamp, or an instant against a wall clock, compares * two differently shaped strings, which is meaningless), * and boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, never rejected. */ @@ -329,10 +343,47 @@ function crossFieldLiteral(raw: unknown, when: MetaField): { lit: string; numeri * subtype also exports `InsertBaseSchema` (unrefined) and `InsertBodySchema` * (refined, pinned discriminator omitted). */ export function hasInsertRefinements(obj: MetaObject): boolean { - return crossFieldRefinementBody(obj).length > 0; + return crossFieldRefinement(obj).blocks.length > 0; } -function crossFieldRefinementBody(obj: MetaObject): string[] { +/** + * How a value is compared, from the field it is or is compared with: numerically, as a + * duration's microseconds (the canonical string does not order), or as written (strings, + * dates and times in their ISO form, booleans, enum members). + */ +type CompareKind = "number" | "duration" | "plain"; + +function compareKind(f: MetaField | undefined): CompareKind { + if (f === undefined) return "plain"; + if (NUMERIC_SUBTYPES.has(f.subType)) return "number"; + if (f.subType === FIELD_SUBTYPE_DURATION) return "duration"; + return "plain"; +} + +/** The local helpers a refinement body uses, declared once at its top when needed. */ +const HELPER_CMP = ` const cmp = (a: unknown, b: unknown): number | null => { + if (a == null || b == null) return null; + if (typeof a === "number" || typeof b === "number") return Number(a) - Number(b); + if (a instanceof Date || b instanceof Date) return new Date(a as string).getTime() - new Date(b as string).getTime(); + return String(a) < String(b) ? -1 : String(a) > String(b) ? 1 : 0; + };`; +const HELPER_TRI = ` const all = (...v: Array): boolean | null => v.includes(false) ? false : v.includes(null) ? null : true; + const any = (...v: Array): boolean | null => v.includes(true) ? true : v.includes(null) ? null : false; + const not = (v: boolean | null): boolean | null => v === null ? null : !v;`; +// An ISO-8601 duration (weeks, days, hours, minutes, seconds) as microseconds, or null when the +// text is not one (the accepted form: `P`, `PT` and `P1DT` are not durations). +const HELPER_MICROS = ` const micros = (d: unknown): number | null => { + if (d == null) return null; + const m = /^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$/.exec(String(d)); + if (m === null) return null; + const hours = (Number(m[2] ?? 0) * 7 + Number(m[3] ?? 0)) * 24 + Number(m[4] ?? 0); + const us = ((hours * 60 + Number(m[5] ?? 0)) * 60 + Number(m[6] ?? 0)) * 1e6 + Number((m[7] ?? "").padEnd(6, "0").slice(0, 6)); + return m[1] === "-" ? -us : us; + };`; + +interface RefinementBuild { blocks: string[]; helpers: Set } + +function crossFieldRefinement(obj: MetaObject): RefinementBuild { const inSchema = insertSchemaFieldNameSet(obj); const byName = new Map(obj.fields().map((f) => [f.name, f] as const)); // A reference must name a field that exists AND is in the insert schema; otherwise skip @@ -350,9 +401,116 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { : `row[${JSON.stringify(f.name)}]`; const issue = (path: string, msg: string) => `ctx.addIssue({ code: "custom", path: [${path}], message: ${JSON.stringify(msg)} });`; + // The helpers the rule being built uses; merged into `helpers` only when the rule is kept, + // so a rule left to the database declares nothing that nothing reads. + const helpers = new Set(); + let ruleHelpers = new Set(); + /** A value compared as `kind`. */ + const asKind = (expr: string, kind: CompareKind): string => { + if (kind === "number") return `(${expr} == null ? null : Number(${expr}))`; + if (kind === "duration") { ruleHelpers.add(HELPER_MICROS); return `micros(${expr})`; } + return expr; + }; + + /** The field a node compares as: itself, or a coalesce's first field. */ + const nodeField = (node: unknown): MetaField | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") return byName.get(n["field"]); + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + for (const a of n["args"] as unknown[]) { const f = nodeField(a); if (f !== undefined) return f; } + } + return undefined; + }; + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + const nodeFields = (node: unknown): MetaField[] => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { const f = byName.get(n["field"]); return f === undefined ? [] : [f]; } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) return (n["args"] as unknown[]).flatMap(nodeFields); + return []; + }; + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + const conditionComparable = (left: unknown, right: unknown, opName: string): boolean => { + const fields = [...nodeFields(left), ...nodeFields(right)]; + return fields.every((a, i) => fields.slice(i + 1).every((b) => crossComparable(a, b, opName))); + }; + /** A value node (field, literal, coalesce) as a TS expression compared as `kind`, or undefined. */ + const valueTs = (node: unknown, kind: CompareKind): string | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { + const f = ref(n["field"]); + return f === undefined ? undefined : asKind(acc(f), kind); + } + if ("value" in n) { + const value = n["value"]; + if (kind === "duration" && typeof value === "string") { + const us = parseDurationLiteral(value); + return us === undefined ? undefined : String(Number(us)); + } + return JSON.stringify(value); + } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map((a) => valueTs(a, kind)); + if (args.some((a) => a === undefined)) return undefined; + return `([${args.join(", ")}].find((x) => x != null) ?? null)`; + } + return undefined; + }; + /** A condition node as a TS expression of type `boolean | null`, or undefined when it names + * a field the body does not carry (the rule is then left to the database). */ + const condTs = (node: unknown): string | undefined => { + const n = (node ?? {}) as Record; + const op = n["op"]; + if (typeof op === "string" && CROSS_FIELD_OPS[op] !== undefined) { + if (!conditionComparable(n["left"], n["right"], op)) return undefined; + const kind = compareKind(nodeField(n["left"]) ?? nodeField(n["right"])); + const l = valueTs(n["left"], kind); + const r = valueTs(n["right"], kind); + if (l === undefined || r === undefined) return undefined; + ruleHelpers.add(HELPER_CMP); + return `((c) => c === null ? null : c ${CROSS_FIELD_OPS[op]} 0)(cmp(${l}, ${r}))`; + } + if (op === FILTER_OP_IS_NULL || op === EXPR_OP_IS_NOT_NULL) { + const a = valueTs(n["arg"], "plain"); + if (a === undefined) return undefined; + return op === FILTER_OP_IS_NULL ? `(${a} == null)` : `(${a} != null)`; + } + if (op === EXPR_OP_NOT) { + const a = condTs(n["arg"]); + if (a === undefined) return undefined; + ruleHelpers.add(HELPER_TRI); + return `not(${a})`; + } + if ((op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map(condTs); + if (args.some((a) => a === undefined)) return undefined; + ruleHelpers.add(HELPER_TRI); + return `${op === FILTER_COMPOSE_AND ? "all" : "any"}(${args.join(", ")})`; + } + return undefined; + }; + const blocks: string[] = []; + /** Add a rule's block, guarded by its @condition when it has one. */ + const push = (v: { attr(name: string): unknown }, block: string): void => { + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + const guard = condition === undefined ? undefined : condTs(condition); + if (condition !== undefined && guard === undefined) return; + for (const h of ruleHelpers) helpers.add(h); + if (guard === undefined) { blocks.push(block); return; } + const inner = block.split("\n").map((l) => (l === "" ? l : ` ${l}`)).join("\n"); + blocks.push(` if (${guard} === true) {\n${inner}\n }`); + }; + /** The @fields of an atLeastOne / exactlyOne, each in the body, or undefined. */ + const fieldList = (raw: unknown): MetaField[] | undefined => { + const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); + const refs = names.map(ref); + if (refs.length === 0 || refs.some((r) => r === undefined)) return undefined; + return refs as MetaField[]; + }; // ADR-0039: entity.validators() is the resolving accessor (never ownValidators). for (const v of obj.validators()) { + ruleHelpers = new Set(); switch (v.subType) { case VALIDATOR_SUBTYPE_COMPARISON: { const left = ref(v.attr(VALIDATOR_ATTR_LEFT)); @@ -361,12 +519,15 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const op = CROSS_FIELD_OPS[opName]; if (!left || !right || !op || left.resolvedIsArray() || right.resolvedIsArray()) break; if (!crossComparable(left, right, opName)) break; - const numeric = NUMERIC_SUBTYPES.has(left.subType); - const c = numeric + const kind = compareKind(left); + const c = kind === "number" || kind === "duration" ? "Number(l) - Number(r)" : "l instanceof Date && r instanceof Date ? l.getTime() - r.getTime() : String(l) < String(r) ? -1 : String(l) > String(r) ? 1 : 0"; - blocks.push( - ` {\n const l: unknown = ${acc(left)};\n const r: unknown = ${acc(right)};\n` + + // A duration side is read as its microseconds; text that is not a duration reads as + // null, so the rule is not checked (the field's own schema refuses that text). + const readKind: CompareKind = kind === "duration" ? "duration" : "plain"; + push(v, + ` {\n const l: unknown = ${asKind(acc(left), readKind)};\n const r: unknown = ${asKind(acc(right), readKind)};\n` + ` if (l != null && r != null) {\n const c = ${c};\n` + ` if (!(c ${op} 0)) ${issue(JSON.stringify(left.name), `${left.name} must be ${opName} ${right.name}`)}\n }\n }`, ); @@ -382,35 +543,41 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const path = JSON.stringify(target.name); const decl = ` const g: unknown = ${acc(when)};\n const t: unknown = ${acc(target)};\n const open = ${gate};\n`; if (v.subType === VALIDATOR_SUBTYPE_REQUIRED_WHEN) { - blocks.push(` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } else { - blocks.push(` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } break; } case VALIDATOR_SUBTYPE_AT_LEAST_ONE: { - const raw = v.attr(VALIDATOR_ATTR_FIELDS); - const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); - const refs = names.map(ref); - if (refs.length === 0 || refs.some((r) => r === undefined)) break; - const fields = refs as MetaField[]; + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; const anyPresent = fields.map((f) => `${acc(f)} != null`).join(" || "); - blocks.push(` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + push(v, ` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + break; + } + case VALIDATOR_SUBTYPE_EXACTLY_ONE: { + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; + const present = fields.map((f) => `${acc(f)}`).join(", "); + push(v, ` if ([${present}].filter((x) => x != null).length !== 1) ${issue(JSON.stringify(fields[0]!.name), `exactly one of ${fields.map((f) => f.name).join(", ")} is required`)}`); break; } default: break; } } - return blocks; + return { blocks, helpers }; } /** `.superRefine(...)` suffix for an insert-shaped schema, or "" when the object has no * enforceable cross-field validator (output is then byte-identical to before). */ function insertRefinementSuffix(obj: MetaObject): string { - const blocks = crossFieldRefinementBody(obj); + const { blocks, helpers } = crossFieldRefinement(obj); if (blocks.length === 0) return ""; - return `.superRefine((row, ctx) => {\n${blocks.join("\n")}\n})`; + // Declared in a fixed order, so the output does not depend on which rule needed one first. + const decls = [HELPER_CMP, HELPER_TRI, HELPER_MICROS].filter((h) => helpers.has(h)); + return `.superRefine((row, ctx) => {\n${[...decls, ...blocks].join("\n")}\n})`; } /** @@ -426,6 +593,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); @@ -477,7 +645,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const docsPrefix = docs ? `${docs}\n` : ""; return code` -${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ +${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${durationHelperSince(durationUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ ${joinCode(insertFieldLines, { on: ",\n" })} })${insertRefinementSuffix(obj)}; `; @@ -586,6 +754,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const pkFields = primaryKeyFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); @@ -728,7 +897,7 @@ export type ${objName}CreatePreserving = ${z}.input DURATION_MICROS_MAX || micros < DURATION_MICROS_MIN) throw invalidValue(field, op, s, DURATION_IN_RANGE); + return dialect === "sqlite" || dialect === "mysql" ? durationToStoredMicros(micros) : formatPostgresInterval(micros); + } // A timestamp bound is compared in the same UTC spelling the generated schemas store: // on SQLite/D1 the column is TEXT and compares as text. An instant field (`instant`, i.e. // not @localTime) reads a zoneless or date-only bound as UTC, as its write schema does — @@ -364,6 +377,7 @@ function coerce(value: unknown, rule: FilterFieldRule, col: unknown, field: stri /** `expected` for a non-whole bound on an integer-valued field (`integer: true`). */ const INTEGER_EXPECTED = "integer (a whole number)"; +const DURATION_IN_RANGE = "duration within the 64-bit microsecond range"; function coerceAs(value: unknown, subType: string, field: string, op: string, dateValues?: boolean): unknown { if (value === null || value === undefined) return null; diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/filter-value-format.ts b/examples/showcase/codegen/runtime/drizzle-fastify/filter-value-format.ts index 5def0974f6..cf8a3f1097 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/filter-value-format.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/filter-value-format.ts @@ -17,9 +17,11 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, } from "@metaobjectsdev/metadata"; import type { FilterValueFormat } from "./filter-allowlist.js"; import { canonicalDecimal } from "../decimal-wire.js"; +import { parseDurationLiteral } from "../duration-wire.js"; const DATE_RE = /^(\d{4})-(\d{2})-(\d{2})$/; const TIME_RE = /^(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d{1,9})?)?$/; @@ -48,6 +50,7 @@ const CHECKS: Readonly boolean>> = { [FIELD_SUBTYPE_TIMESTAMP]: isTimestamp, [FIELD_SUBTYPE_UUID]: (s) => UUID_RE.test(s), [FIELD_SUBTYPE_DECIMAL]: (s) => canonicalDecimal(s) !== undefined, + [FIELD_SUBTYPE_DURATION]: (s) => parseDurationLiteral(s) !== undefined, }; /** The `expected` member of the `invalid_filter_value` envelope, per format. */ @@ -57,6 +60,7 @@ export const FORMAT_EXPECTED: Readonly> = { [FIELD_SUBTYPE_TIMESTAMP]: "timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", [FIELD_SUBTYPE_UUID]: "uuid (8-4-4-4-12 hex)", [FIELD_SUBTYPE_DECIMAL]: "decimal number (digits, optional fraction, optional exponent)", + [FIELD_SUBTYPE_DURATION]: "duration (ISO 8601, e.g. PT1H30M; no years or months)", }; /** `expected` for a temporal rule that does not say WHICH temporal format it is. */ diff --git a/examples/showcase/codegen/runtime/duration-wire.ts b/examples/showcase/codegen/runtime/duration-wire.ts new file mode 100644 index 0000000000..732659f9b0 --- /dev/null +++ b/examples/showcase/codegen/runtime/duration-wire.ts @@ -0,0 +1,7 @@ +// The field.duration wire rule lives in @metaobjectsdev/metadata (shared with migrate-ts); +// re-exported here so the runtime's callers import it beside the decimal wire helpers. +export { + DURATION_PATTERN, DURATION_RE, parseDurationLiteral, formatDuration, formatPostgresInterval, + canonicalDuration, parseDurationInput, parsePostgresInterval, durationToWire, durationToStoredMicros, + DURATION_MICROS_MIN, DURATION_MICROS_MAX, +} from "@metaobjectsdev/metadata"; diff --git a/examples/showcase/codegen/templates/drizzle-schema.ts b/examples/showcase/codegen/templates/drizzle-schema.ts index 93e85d56a0..74df06cb0d 100644 --- a/examples/showcase/codegen/templates/drizzle-schema.ts +++ b/examples/showcase/codegen/templates/drizzle-schema.ts @@ -18,7 +18,7 @@ import { import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { - mapColumnType, PG_IDENTITY_CAPABLE_FNS, + mapColumnType, PG_IDENTITY_CAPABLE_FNS, DURATION_COLUMN_FN, type ColumnSpec, type EnumIntCustomType, } from "@metaobjectsdev/codegen-ts/engine"; import { tableNameFromEntity } from "@metaobjectsdev/codegen-ts/engine"; @@ -28,6 +28,7 @@ import { } from "@metaobjectsdev/codegen-ts/engine"; import { resolveTableSchema } from "@metaobjectsdev/metadata"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; import { collectTphSubtypeFields, tphConcreteSubtypes } from "./tph-discriminator.js"; @@ -46,7 +47,8 @@ export function renderDrizzleSchema( obj: MetaObject, ctx: RenderContext, /** - * OUT param: the int-backed-enum codec const names this table DECLARES. A caller that + * OUT param: the local column-codec const names this table DECLARES (int-backed enums, and + * the duration codec). A caller that * emits another artifact referencing the same codecs into the SAME module — entity-file's * write-through replica view — passes a set here and hands it to that emitter, which then * references the consts instead of re-declaring them. A duplicate module-scope `const` is @@ -211,6 +213,8 @@ export function renderDrizzleSchema( // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by // const name so a shared enum used by two fields of the SAME entity emits once. const enumIntTypes = new Map(); + // The field.duration codec: one declaration serves every duration column of the table. + let usesDurationCodec = false; for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -225,6 +229,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, columnNameExpr(child, spec.dbName), child, ctx, isPk, pkGeneration, fkInfo, isComposite, pkg, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -244,6 +249,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, columnNameExpr(child, spec.dbName), child, ctx, false, undefined, fkMap.get(child.name), isComposite, pkg, obj.name, true, @@ -403,9 +409,11 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} .map((t) => renderEnumIntCustomType(t, importModule)); if (declaredEnumIntCodecs !== undefined) { for (const name of enumIntTypes.keys()) declaredEnumIntCodecs.add(name); + if (usesDurationCodec) declaredEnumIntCodecs.add(DURATION_COLUMN_FN); } - const blocks: Code[] = [...enumIntBlocks, tableBlock]; + const durationBlocks = usesDurationCodec ? [renderDurationCustomType(dialect, importModule)] : []; + const blocks: Code[] = [...durationBlocks, ...enumIntBlocks, tableBlock]; if (relationsBlock !== null) blocks.push(relationsBlock); return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); } @@ -518,7 +526,9 @@ function renderColumn( const fnSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); let baseCall: Code; if (spec.fnOptions !== undefined && Object.keys(spec.fnOptions).length > 0) { diff --git a/examples/showcase/codegen/templates/duration-wire-helper.ts b/examples/showcase/codegen/templates/duration-wire-helper.ts new file mode 100644 index 0000000000..81aeaac06e --- /dev/null +++ b/examples/showcase/codegen/templates/duration-wire-helper.ts @@ -0,0 +1,213 @@ +// The wire rule for a field.duration, as text a generated model module carries as local code. +// Generated model code imports nothing from MetaObjects, so the rule is emitted, not imported; +// metadata/src/core/field/duration-wire.ts holds the same rule and both are held to +// fixtures/duration-wire/vectors.json (test/duration-wire-helper.test.ts evaluates the text +// emitted here against them). +// +// Accepted: optional weeks (exactly 7 days) and days, then hours, minutes and seconds; the seconds +// fraction may have any number of digits. Canonical: `[-]PT[H][M][[.]S]` — hours never +// folded into days, zero components left out, zero is `PT0S`, one leading `-`. A span is counted in +// microseconds, so the fraction keeps six digits and the rest is truncated toward zero. +// +// The emitted code counts a span in microseconds as a `bigint`, so it has no limit of its own: a +// span is as long as the stack it is stored in allows. The only limit here is a column's — the +// 64-bit integer a SQLite, D1 or MySQL column holds (refused past it, never wrapped). Postgres's +// interval has its own range, which Postgres enforces. The emitted code calls `BigInt(...)` and +// writes no bigint literal, so it needs the ES2020 `BigInt` global and nothing newer. + +import { code, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_PATTERN } from "@metaobjectsdev/metadata"; +import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_COLUMN_FN } from "@metaobjectsdev/codegen-ts/engine"; + +/** The accepted ISO-8601 duration form as a JS regex literal — the metadata's own pattern. */ +export const DURATION_REGEX_LITERAL = `/${DURATION_PATTERN}/`; + +/** The Zod expression that accepts a field.duration as sent (and as read back): the accepted form. */ +export const ZOD_DURATION_EXPR = `z.string().regex(${DURATION_REGEX_LITERAL})`; + +export const DURATION_WRITE_HELPER = "durationWrite"; + +/** The Zod expression of a duration WRITE field: the accepted form, rewritten to the canonical one, + * so equal spans are equal strings on the wire and in a jsonb value object. */ +export const ZOD_DURATION_WRITE_EXPR = `${ZOD_DURATION_EXPR}.transform(${DURATION_WRITE_HELPER})`; + +/** The accepted form with its components captured: sign, weeks, days, hours, minutes, seconds, + * fraction. Equivalent to DURATION_PATTERN (the same lookaheads refuse `P`, `PT` and `P1DT`). */ +const ISO_PARTS_SOURCE = String.raw`/^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/`; + +/** The default (`IntervalStyle = postgres`) text of a Postgres interval: signed year / mon / day + * parts, then an optionally signed `H:MM:SS[.ffffff]`. */ +const PG_INTERVAL_SOURCE = String.raw`/^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/`; + +/** Indent every line of `block` by `n` spaces (blank lines stay blank). */ +function indent(block: string, n: number): string { + const pad = " ".repeat(n); + return block.split("\n").map((l) => (l === "" ? l : pad + l)).join("\n"); +} + +/** Microsecond unit constants, as `BigInt(...)` calls (no bigint literal). */ +const UNIT_LINES = [ + "const ZERO = BigInt(0);", + "const SECOND = BigInt(1000000);", + "const MINUTE = BigInt(60000000);", + "const HOUR = BigInt(3600000000);", + "const DAY = BigInt(86400000000);", + "const WEEK = BigInt(604800000000);", +].join("\n"); + +/** `parseIso(text)`: the microseconds of an accepted ISO-8601 duration, or undefined. */ +const PARSE_ISO_FN = [ + "const parseIso = (text: string): bigint | undefined => {", + ` const p = ${ISO_PARTS_SOURCE}.exec(text);`, + " if (p === null) return undefined;", + " const mag = BigInt(p[2] ?? 0) * WEEK + BigInt(p[3] ?? 0) * DAY + BigInt(p[4] ?? 0) * HOUR", + ' + BigInt(p[5] ?? 0) * MINUTE + BigInt(p[6] ?? 0) * SECOND + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6));', + ' return p[1] === "-" ? -mag : mag;', + "};", +].join("\n"); + +/** `isoText(micros)`: the canonical wire form. */ +const ISO_TEXT_FN = [ + "const isoText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + " const s = rest / SECOND;", + " const f = rest % SECOND;", + ' let out = "PT";', + ' if (h > ZERO) out += String(h) + "H";', + ' if (m > ZERO) out += String(m) + "M";', + ' if (s > ZERO || f > ZERO) out += String(s) + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + "S";', + ' if (out === "PT") return "PT0S";', + ' return neg ? "-" + out : out;', + "};", +].join("\n"); + +export const DURATION_WRITE_HELPER_DECL = [ + "/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten", + " * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one", + " * leading \"-\" for a negative span. Counted in bigint microseconds, so any length is exact and a", + " * fraction past six digits is truncated toward zero. Generated. */", + `function ${DURATION_WRITE_HELPER}(text: string): string {`, + indent(UNIT_LINES, 2), + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + " const micros = parseIso(text);", + " return micros === undefined ? text : isoText(micros);", + "}", +].join("\n"); + +/** The physical column type a duration column's `customType` declares, per dialect: the + * Postgres-native interval; a 64-bit integer count of microseconds on SQLite / D1 (INTEGER) and + * MySQL (BIGINT) — the persistence corpus's canonical schemas, and what `meta migrate` writes. */ +export function durationColumnDataType(dialect: Dialect): string { + return dialect === "postgres" ? "interval" : dialect === "mysql" ? "bigint" : "integer"; +} + +/** + * The body of the duration column's declaration, around the `customType` reference (which the + * caller supplies as an import). `[before, after]`: `before + customType + after` is the whole + * declaration. Exported so a test can evaluate the exact text a generated module carries. + * + * - `toDriver` binds what the column holds: on Postgres the interval spelling `[-]HH:MM:SS[.f]` + * (Postgres refuses a signed ISO-8601 duration, and returns a value written this way in exactly + * this spelling); on SQLite, D1 and MySQL the microsecond count — a number when it is a safe + * integer, else a bigint — refused past the 64-bit range, which is that column's limit. It takes + * the value as an ISO-8601 duration (what the generated write schema hands it), or as what + * runtime-ts already binds a filter operand in (the Postgres text; the microsecond integer), and + * throws for anything else. + * - `fromDriver` reads what the driver returns — the microsecond integer (a number, a bigint or a + * digit string), ISO-8601 text, Postgres interval text in the default `IntervalStyle`, or a + * node-postgres interval object — as the canonical ISO-8601 form. An interval with a year or + * month part is not a fixed-length span: reading one throws. + */ +export function durationColumnDeclParts(dialect: Dialect): [string, string] { + const postgres = dialect === "postgres"; + const before = [ + "/** A field.duration column. Binds " + (postgres + ? "the Postgres interval spelling [-]HH:MM:SS[.f] (Postgres refuses a signed ISO-8601 duration);" + : "the span as a 64-bit integer count of microseconds (refused past that range);"), + " * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month", + " * part. Counted in bigint microseconds, so the column is the only limit. Generated. */", + `const ${DURATION_COLUMN_FN} = (() => {`, + indent(UNIT_LINES, 2), + ` const INT64_MAX = BigInt("9223372036854775807");`, + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + indent([ + "const pgText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + ' const two = (n: bigint): string => String(n).padStart(2, "0");', + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + ' let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND);', + " const f = rest % SECOND;", + ' if (f > ZERO) out += "." + String(f).padStart(6, "0").replace(/0+$/, "");', + ' return neg ? "-" + out : out;', + "};", + "const stored = (micros: bigint): number | bigint => {", + " if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) {", + ' throw new Error("the span " + isoText(micros) + " does not fit the 64-bit microsecond column a field.duration is stored in");', + " }", + " const n = Number(micros);", + " return Number.isSafeInteger(n) ? n : micros;", + "};", + "const notFixed = (v: unknown): Error =>", + ' new Error("the interval " + JSON.stringify(v) + " has a year or month part, so it is not a fixed-length field.duration");', + "const read = (v: unknown): bigint => {", + ' if (typeof v === "bigint") return v;', + ' if (typeof v === "number" && Number.isInteger(v)) return BigInt(v);', + ' if (typeof v === "string") {', + " if (/^-?\\d+$/.test(v)) return BigInt(v);", + " const iso = parseIso(v);", + " if (iso !== undefined) return iso;", + ` const p = v.length === 0 ? null : ${PG_INTERVAL_SOURCE}.exec(v);`, + ' if (p === null) throw new Error("not a field.duration value: " + JSON.stringify(v));', + " if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) throw notFixed(v);", + " let time = ZERO;", + " if (p[5] !== undefined) {", + ' time = BigInt(p[5]) * HOUR + BigInt(p[6] ?? 0) * MINUTE + BigInt(p[7] ?? 0) * SECOND + BigInt((p[8] ?? "").padEnd(6, "0"));', + ' if (p[4] === "-") time = -time;', + " }", + " return BigInt(p[3] ?? 0) * DAY + time;", + " }", + ' if (v !== null && typeof v === "object") {', + " const o = v as Record;", + " if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) throw notFixed(v);", + " const whole = (n: number | undefined): bigint => BigInt(Math.trunc(n ?? 0));", + " // postgres-interval carries microseconds as a fractional millisecond count.", + " return whole(o.days) * DAY + whole(o.hours) * HOUR + whole(o.minutes) * MINUTE + whole(o.seconds) * SECOND", + " + BigInt(Math.round((o.milliseconds ?? 0) * 1000));", + " }", + ' throw new Error("not a field.duration value: " + JSON.stringify(v));', + "};", + ].join("\n"), 2), + " return ", + ].join("\n"); + const after = [ + `<{ data: string; driverData: ${postgres ? "string" : "number | bigint | string"} }>({`, + ` dataType: () => ${JSON.stringify(durationColumnDataType(dialect))},`, + ` toDriver: (value) => ${postgres ? "pgText" : "stored"}(read(value)),`, + " fromDriver: (value) => isoText(read(value)),", + " });", + "})();", + ].join("\n"); + return [before, after]; +} + +/** A duration column's Drizzle `customType` declaration, emitted ahead of the table or view that + * uses it (once per module — see enum-int-codec.ts for why the order and the once matter). */ +export function renderDurationCustomType(dialect: Dialect, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const [before, after] = durationColumnDeclParts(dialect); + return code` +${before}${customTypeSym}${after} +`; +} diff --git a/examples/showcase/codegen/templates/field-meta.ts b/examples/showcase/codegen/templates/field-meta.ts index 6182a31306..72549fd876 100644 --- a/examples/showcase/codegen/templates/field-meta.ts +++ b/examples/showcase/codegen/templates/field-meta.ts @@ -22,6 +22,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STRING_FORMAT, @@ -43,6 +44,7 @@ import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; import { zodEnumExpr } from "./zod-enum.js"; import { viewForContext } from "@metaobjectsdev/codegen-ts/engine"; import { ZOD_INET_EXPR } from "./net-regex.js"; +import { ZOD_DURATION_EXPR } from "./duration-wire-helper.js"; // --------------------------------------------------------------------------- // inferViewKind @@ -140,6 +142,9 @@ export function zodTypeFor(field: MetaField, timestampMode: "date" | "string" = // (v4 or v6 literal). Zod-version-agnostic regex union (Zod 4 removed // `z.string().ip()`); see net-regex.ts. @lenient → a plain string. return field.attr(FIELD_ATTR_LENIENT) === true ? "z.string()" : ZOD_INET_EXPR; + case FIELD_SUBTYPE_DURATION: + // An ISO-8601 duration string; the column codec reads every row in the canonical form. + return ZOD_DURATION_EXPR; case FIELD_SUBTYPE_BOOLEAN: return "z.boolean()"; case FIELD_SUBTYPE_DATE: diff --git a/examples/showcase/codegen/templates/filter-allowlist.ts b/examples/showcase/codegen/templates/filter-allowlist.ts index 4a0cd087fd..ec7c195b7c 100644 --- a/examples/showcase/codegen/templates/filter-allowlist.ts +++ b/examples/showcase/codegen/templates/filter-allowlist.ts @@ -13,6 +13,7 @@ import { FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_ATTR_LOCAL_TIME, opsForField, @@ -70,6 +71,9 @@ const VALUE_FORMAT_SUBTYPES = new Set([ FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_DECIMAL, + // An ISO-8601 duration: runtime-ts refuses a value that is not one, and the column codec binds + // the dialect's spelling of it. + FIELD_SUBTYPE_DURATION, ]); /** Subtypes whose values are whole numbers — `field.currency` is integer minor units. */ diff --git a/examples/showcase/codegen/templates/inferred-types.ts b/examples/showcase/codegen/templates/inferred-types.ts index 1eec56f318..4b87f2f9bc 100644 --- a/examples/showcase/codegen/templates/inferred-types.ts +++ b/examples/showcase/codegen/templates/inferred-types.ts @@ -27,6 +27,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_REQUIRED, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_VALUE_TYPE, @@ -215,6 +216,8 @@ const SCALAR_TS_BY_SUBTYPE: Record = { // type, same as uuid). Other ports bind to their native URI/IP type. [FIELD_SUBTYPE_URI]: "string", [FIELD_SUBTYPE_INET]: "string", + // field.duration is an ISO-8601 duration string: TS has no native duration type. + [FIELD_SUBTYPE_DURATION]: "string", [FIELD_SUBTYPE_INT]: "number", [FIELD_SUBTYPE_LONG]: "number", [FIELD_SUBTYPE_DOUBLE]: "number", diff --git a/examples/showcase/codegen/templates/projection-decl.ts b/examples/showcase/codegen/templates/projection-decl.ts index e9ab1e45d2..a7ccbf0f93 100644 --- a/examples/showcase/codegen/templates/projection-decl.ts +++ b/examples/showcase/codegen/templates/projection-decl.ts @@ -21,7 +21,7 @@ import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; -import { physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; +import { columnExpr, physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; import { renderFilterAllowlist, renderSortAllowlist } from "./filter-allowlist.js"; import { renderFilterType } from "./filter-type.js"; import { buildUiFieldDescriptor, resourcePath } from "@metaobjectsdev/codegen-ts/engine"; @@ -156,11 +156,8 @@ export function renderProjectionDecl( // A6 — reference the constant whenever the artifact is in the run AND carries this // field. A lookup MISS (a TPH subtype field the base's names artifact never saw) is // normal, not a divergence — fall back to the literal, same as every other §A6 site. - const namesEntry = names?.resolved.fields[f.name]; - const dbColExpr: Code = - names !== undefined && namesEntry !== undefined - ? code`${names.symbol}.fields.${f.name}.column` - : code`${JSON.stringify(dbCol)}`; + // A flattened field has no `column` of its own (its members do), so it takes the literal. + const dbColExpr: Code = columnExpr(names, f.name, dbCol); // #356 — a projection's descriptor is the same shape the entity descriptor emits, // and now literally the SAME derivation: `buildUiFieldDescriptor` is what // `renderEntityConstants` reads, so the view kind, the label and the currency meta diff --git a/examples/showcase/codegen/templates/view-decl.ts b/examples/showcase/codegen/templates/view-decl.ts index da4a523058..eec5e9ed44 100644 --- a/examples/showcase/codegen/templates/view-decl.ts +++ b/examples/showcase/codegen/templates/view-decl.ts @@ -14,8 +14,9 @@ import { type MetaField, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, } from "@metaobjectsdev/metadata"; import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; -import { mapColumnType, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; +import { mapColumnType, DURATION_COLUMN_FN, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { zodTypeFor } from "./field-meta.js"; import { columnExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; @@ -25,7 +26,8 @@ export interface ViewDeclOpts { /** Drives the timestamp column TS type (Date vs string) in the view declaration. */ readonly timestampMode: "date" | "string"; /** - * Int-backed-enum codec const names ALREADY declared in the module this view lands in. + * Local column-codec const names (int-backed enums, the duration codec) ALREADY declared in + * the module this view lands in. * A write-through entity's table and its replica view share one file, and both reference * the same codecs — so the view references these by name and declares only what the table * did not (a derived field's codec, which the write table omits). Absent = the view owns @@ -88,6 +90,8 @@ function viewColumnLine( opts: ViewDeclOpts, /** Codecs this view needs, collected for the caller to DECLARE — see below. */ enumIntTypes: Map, + /** The duration codec's name, when a column needs it — declared by the caller, as above. */ + usesDuration: Set, ): Code { const { dialect, columnNamingStrategy, timestampMode } = opts; const spec = mapColumnType(f, dialect, columnNamingStrategy, timestampMode); @@ -99,10 +103,14 @@ function viewColumnLine( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + // The field.duration codec is a local const too; collected under its name, declared once. + if (spec.durationCustomType !== undefined) usesDuration.add(spec.durationCustomType.fnConstName); const colSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const optsArg = spec.fnOptions && Object.keys(spec.fnOptions).length > 0 ? `, ${JSON.stringify(spec.fnOptions)}` @@ -164,7 +172,8 @@ export function renderExistingViewDecl( const { viewFn, core: viewModule } = dialectModule(opts.dialect); const viewSym = imp(`${viewFn}@${viewModule}`); const enumIntTypes = new Map(); - const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes)); + const usesDuration = new Set(); + const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes, usesDuration)); const viewNameExpr = typeof viewName === "string" ? code`${JSON.stringify(viewName)}` : viewName; // @schema — a view lands in a schema exactly as a table does, and migrate qualifies the @@ -188,6 +197,9 @@ export function renderExistingViewDecl( .filter((t) => alreadyDeclared === undefined || !alreadyDeclared.has(t.fnConstName)) .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) .map((t) => renderEnumIntCustomType(t, viewModule)); + if (usesDuration.has(DURATION_COLUMN_FN) && (alreadyDeclared === undefined || !alreadyDeclared.has(DURATION_COLUMN_FN))) { + enumIntBlocks.unshift(renderDurationCustomType(opts.dialect, viewModule)); + } const viewBlock = code` // View declaration — Drizzle uses this for typed SELECT queries. diff --git a/examples/showcase/codegen/templates/zod-validators.ts b/examples/showcase/codegen/templates/zod-validators.ts index 917a215da5..195fddfacb 100644 --- a/examples/showcase/codegen/templates/zod-validators.ts +++ b/examples/showcase/codegen/templates/zod-validators.ts @@ -17,6 +17,7 @@ import { FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DECIMAL, FIELD_ATTR_PRECISION, FIELD_ATTR_SCALE, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_ARRAY, IDENTITY_ATTR_FIELDS, IDENTITY_ATTR_GENERATION, @@ -28,7 +29,9 @@ import { AUTO_SET_ON_CREATE, AUTO_SET_ON_UPDATE, VALIDATOR_ATTR_MAX, VALIDATOR_ATTR_MIN, VALIDATOR_ATTR_PATTERN, VALIDATOR_SUBTYPE_COMPARISON, VALIDATOR_SUBTYPE_REQUIRED_WHEN, - VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_SUBTYPE_EXACTLY_ONE, + VALIDATOR_ATTR_CONDITION, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, EXPR_FN_COALESCE, + FILTER_OP_IS_NULL, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, parseDurationLiteral, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, GENERATION_INCREMENT, GENERATION_UUID, @@ -48,6 +51,7 @@ import { } from "./date-time-regex.js"; import { renderDocsFor } from "./jsdoc.js"; import { DECIMAL_WRITE_HELPER_DECL, zodDecimalWriteExpr } from "./decimal-wire-helper.js"; +import { DURATION_WRITE_HELPER_DECL, ZOD_DURATION_EXPR, ZOD_DURATION_WRITE_EXPR } from "./duration-wire-helper.js"; import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumZodConstName } from "./enums-file.js"; @@ -86,6 +90,13 @@ function decimalHelperSince(since: number): string { return decimalWriteUses > since ? `${DECIMAL_WRITE_HELPER_DECL}\n\n` : ""; } +// The same device for the duration write rule (duration-wire-helper.ts). +let durationWriteUses = 0; + +function durationHelperSince(since: number): string { + return durationWriteUses > since ? `${DURATION_WRITE_HELPER_DECL}\n\n` : ""; +} + /** True when this object declares at least one @autoSet timestamp field * (onCreate or onUpdate). Drives whether codegen emits the #203 @@ -260,13 +271,16 @@ function isInsertRequiredPk(field: MetaField, assignedPk: Set): boolean } // ── Entity-scoped cross-field validators on the full (insert) record ────────── -// validator.comparison / requiredWhen / presentIff / atLeastOne are enforced on the -// INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged +// validator.comparison / requiredWhen / presentIff / atLeastOne / exactlyOne are enforced on +// the INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged // row, so the Update schema is deliberately left alone (the DB CHECK stays the guard // there). Semantics mirror the migrate-ts CHECK derivation: absent / null / undefined is // SQL NULL, "" is NOT null; a comparison with a NULL side passes; requiredWhen is violated // only when the gate EQUALS the literal and the target is null; presentIff requires -// (target != null) === (gate equals literal), a null gate being "not equal". +// (target != null) === (gate equals literal), a null gate being "not equal"; exactlyOne +// requires exactly one non-null field. A rule with an @condition is checked only when the +// condition is TRUE: the CHECK is NOT (condition) OR (rule), so a condition that is false or +// NULL (three-valued, as SQL evaluates it) lets the row through. const CROSS_FIELD_OPS: Record = { gt: ">", gte: ">=", lt: "<", lte: "<=", ne: "!==", eq: "===", @@ -278,7 +292,7 @@ const NUMERIC_SUBTYPES: ReadonlySet = new Set([ ]); const ORDERED_SUBTYPES: ReadonlySet = new Set([ - FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, + FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_DURATION, ]); /** Subtypes with equality but no ordering: a pair of the same one may be tested with eq / ne. */ @@ -287,7 +301,7 @@ const EQUALITY_SUBTYPES: ReadonlySet = new Set([ ]); /** Whether a comparison between these two fields has a sound meaning, as in every port: - * numerics compare with each other, strings and temporals only with their own subtype and + * numerics compare with each other, strings, temporals and durations only with their own subtype and * @localTime flag (a date against a timestamp, or an instant against a wall clock, compares * two differently shaped strings, which is meaningless), * and boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, never rejected. */ @@ -329,10 +343,47 @@ function crossFieldLiteral(raw: unknown, when: MetaField): { lit: string; numeri * subtype also exports `InsertBaseSchema` (unrefined) and `InsertBodySchema` * (refined, pinned discriminator omitted). */ export function hasInsertRefinements(obj: MetaObject): boolean { - return crossFieldRefinementBody(obj).length > 0; + return crossFieldRefinement(obj).blocks.length > 0; } -function crossFieldRefinementBody(obj: MetaObject): string[] { +/** + * How a value is compared, from the field it is or is compared with: numerically, as a + * duration's microseconds (the canonical string does not order), or as written (strings, + * dates and times in their ISO form, booleans, enum members). + */ +type CompareKind = "number" | "duration" | "plain"; + +function compareKind(f: MetaField | undefined): CompareKind { + if (f === undefined) return "plain"; + if (NUMERIC_SUBTYPES.has(f.subType)) return "number"; + if (f.subType === FIELD_SUBTYPE_DURATION) return "duration"; + return "plain"; +} + +/** The local helpers a refinement body uses, declared once at its top when needed. */ +const HELPER_CMP = ` const cmp = (a: unknown, b: unknown): number | null => { + if (a == null || b == null) return null; + if (typeof a === "number" || typeof b === "number") return Number(a) - Number(b); + if (a instanceof Date || b instanceof Date) return new Date(a as string).getTime() - new Date(b as string).getTime(); + return String(a) < String(b) ? -1 : String(a) > String(b) ? 1 : 0; + };`; +const HELPER_TRI = ` const all = (...v: Array): boolean | null => v.includes(false) ? false : v.includes(null) ? null : true; + const any = (...v: Array): boolean | null => v.includes(true) ? true : v.includes(null) ? null : false; + const not = (v: boolean | null): boolean | null => v === null ? null : !v;`; +// An ISO-8601 duration (weeks, days, hours, minutes, seconds) as microseconds, or null when the +// text is not one (the accepted form: `P`, `PT` and `P1DT` are not durations). +const HELPER_MICROS = ` const micros = (d: unknown): number | null => { + if (d == null) return null; + const m = /^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$/.exec(String(d)); + if (m === null) return null; + const hours = (Number(m[2] ?? 0) * 7 + Number(m[3] ?? 0)) * 24 + Number(m[4] ?? 0); + const us = ((hours * 60 + Number(m[5] ?? 0)) * 60 + Number(m[6] ?? 0)) * 1e6 + Number((m[7] ?? "").padEnd(6, "0").slice(0, 6)); + return m[1] === "-" ? -us : us; + };`; + +interface RefinementBuild { blocks: string[]; helpers: Set } + +function crossFieldRefinement(obj: MetaObject): RefinementBuild { const inSchema = insertSchemaFieldNameSet(obj); const byName = new Map(obj.fields().map((f) => [f.name, f] as const)); // A reference must name a field that exists AND is in the insert schema; otherwise skip @@ -350,9 +401,116 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { : `row[${JSON.stringify(f.name)}]`; const issue = (path: string, msg: string) => `ctx.addIssue({ code: "custom", path: [${path}], message: ${JSON.stringify(msg)} });`; + // The helpers the rule being built uses; merged into `helpers` only when the rule is kept, + // so a rule left to the database declares nothing that nothing reads. + const helpers = new Set(); + let ruleHelpers = new Set(); + /** A value compared as `kind`. */ + const asKind = (expr: string, kind: CompareKind): string => { + if (kind === "number") return `(${expr} == null ? null : Number(${expr}))`; + if (kind === "duration") { ruleHelpers.add(HELPER_MICROS); return `micros(${expr})`; } + return expr; + }; + + /** The field a node compares as: itself, or a coalesce's first field. */ + const nodeField = (node: unknown): MetaField | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") return byName.get(n["field"]); + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + for (const a of n["args"] as unknown[]) { const f = nodeField(a); if (f !== undefined) return f; } + } + return undefined; + }; + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + const nodeFields = (node: unknown): MetaField[] => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { const f = byName.get(n["field"]); return f === undefined ? [] : [f]; } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) return (n["args"] as unknown[]).flatMap(nodeFields); + return []; + }; + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + const conditionComparable = (left: unknown, right: unknown, opName: string): boolean => { + const fields = [...nodeFields(left), ...nodeFields(right)]; + return fields.every((a, i) => fields.slice(i + 1).every((b) => crossComparable(a, b, opName))); + }; + /** A value node (field, literal, coalesce) as a TS expression compared as `kind`, or undefined. */ + const valueTs = (node: unknown, kind: CompareKind): string | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { + const f = ref(n["field"]); + return f === undefined ? undefined : asKind(acc(f), kind); + } + if ("value" in n) { + const value = n["value"]; + if (kind === "duration" && typeof value === "string") { + const us = parseDurationLiteral(value); + return us === undefined ? undefined : String(Number(us)); + } + return JSON.stringify(value); + } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map((a) => valueTs(a, kind)); + if (args.some((a) => a === undefined)) return undefined; + return `([${args.join(", ")}].find((x) => x != null) ?? null)`; + } + return undefined; + }; + /** A condition node as a TS expression of type `boolean | null`, or undefined when it names + * a field the body does not carry (the rule is then left to the database). */ + const condTs = (node: unknown): string | undefined => { + const n = (node ?? {}) as Record; + const op = n["op"]; + if (typeof op === "string" && CROSS_FIELD_OPS[op] !== undefined) { + if (!conditionComparable(n["left"], n["right"], op)) return undefined; + const kind = compareKind(nodeField(n["left"]) ?? nodeField(n["right"])); + const l = valueTs(n["left"], kind); + const r = valueTs(n["right"], kind); + if (l === undefined || r === undefined) return undefined; + ruleHelpers.add(HELPER_CMP); + return `((c) => c === null ? null : c ${CROSS_FIELD_OPS[op]} 0)(cmp(${l}, ${r}))`; + } + if (op === FILTER_OP_IS_NULL || op === EXPR_OP_IS_NOT_NULL) { + const a = valueTs(n["arg"], "plain"); + if (a === undefined) return undefined; + return op === FILTER_OP_IS_NULL ? `(${a} == null)` : `(${a} != null)`; + } + if (op === EXPR_OP_NOT) { + const a = condTs(n["arg"]); + if (a === undefined) return undefined; + ruleHelpers.add(HELPER_TRI); + return `not(${a})`; + } + if ((op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map(condTs); + if (args.some((a) => a === undefined)) return undefined; + ruleHelpers.add(HELPER_TRI); + return `${op === FILTER_COMPOSE_AND ? "all" : "any"}(${args.join(", ")})`; + } + return undefined; + }; + const blocks: string[] = []; + /** Add a rule's block, guarded by its @condition when it has one. */ + const push = (v: { attr(name: string): unknown }, block: string): void => { + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + const guard = condition === undefined ? undefined : condTs(condition); + if (condition !== undefined && guard === undefined) return; + for (const h of ruleHelpers) helpers.add(h); + if (guard === undefined) { blocks.push(block); return; } + const inner = block.split("\n").map((l) => (l === "" ? l : ` ${l}`)).join("\n"); + blocks.push(` if (${guard} === true) {\n${inner}\n }`); + }; + /** The @fields of an atLeastOne / exactlyOne, each in the body, or undefined. */ + const fieldList = (raw: unknown): MetaField[] | undefined => { + const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); + const refs = names.map(ref); + if (refs.length === 0 || refs.some((r) => r === undefined)) return undefined; + return refs as MetaField[]; + }; // ADR-0039: entity.validators() is the resolving accessor (never ownValidators). for (const v of obj.validators()) { + ruleHelpers = new Set(); switch (v.subType) { case VALIDATOR_SUBTYPE_COMPARISON: { const left = ref(v.attr(VALIDATOR_ATTR_LEFT)); @@ -361,12 +519,15 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const op = CROSS_FIELD_OPS[opName]; if (!left || !right || !op || left.resolvedIsArray() || right.resolvedIsArray()) break; if (!crossComparable(left, right, opName)) break; - const numeric = NUMERIC_SUBTYPES.has(left.subType); - const c = numeric + const kind = compareKind(left); + const c = kind === "number" || kind === "duration" ? "Number(l) - Number(r)" : "l instanceof Date && r instanceof Date ? l.getTime() - r.getTime() : String(l) < String(r) ? -1 : String(l) > String(r) ? 1 : 0"; - blocks.push( - ` {\n const l: unknown = ${acc(left)};\n const r: unknown = ${acc(right)};\n` + + // A duration side is read as its microseconds; text that is not a duration reads as + // null, so the rule is not checked (the field's own schema refuses that text). + const readKind: CompareKind = kind === "duration" ? "duration" : "plain"; + push(v, + ` {\n const l: unknown = ${asKind(acc(left), readKind)};\n const r: unknown = ${asKind(acc(right), readKind)};\n` + ` if (l != null && r != null) {\n const c = ${c};\n` + ` if (!(c ${op} 0)) ${issue(JSON.stringify(left.name), `${left.name} must be ${opName} ${right.name}`)}\n }\n }`, ); @@ -382,35 +543,41 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const path = JSON.stringify(target.name); const decl = ` const g: unknown = ${acc(when)};\n const t: unknown = ${acc(target)};\n const open = ${gate};\n`; if (v.subType === VALIDATOR_SUBTYPE_REQUIRED_WHEN) { - blocks.push(` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } else { - blocks.push(` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } break; } case VALIDATOR_SUBTYPE_AT_LEAST_ONE: { - const raw = v.attr(VALIDATOR_ATTR_FIELDS); - const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); - const refs = names.map(ref); - if (refs.length === 0 || refs.some((r) => r === undefined)) break; - const fields = refs as MetaField[]; + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; const anyPresent = fields.map((f) => `${acc(f)} != null`).join(" || "); - blocks.push(` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + push(v, ` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + break; + } + case VALIDATOR_SUBTYPE_EXACTLY_ONE: { + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; + const present = fields.map((f) => `${acc(f)}`).join(", "); + push(v, ` if ([${present}].filter((x) => x != null).length !== 1) ${issue(JSON.stringify(fields[0]!.name), `exactly one of ${fields.map((f) => f.name).join(", ")} is required`)}`); break; } default: break; } } - return blocks; + return { blocks, helpers }; } /** `.superRefine(...)` suffix for an insert-shaped schema, or "" when the object has no * enforceable cross-field validator (output is then byte-identical to before). */ function insertRefinementSuffix(obj: MetaObject): string { - const blocks = crossFieldRefinementBody(obj); + const { blocks, helpers } = crossFieldRefinement(obj); if (blocks.length === 0) return ""; - return `.superRefine((row, ctx) => {\n${blocks.join("\n")}\n})`; + // Declared in a fixed order, so the output does not depend on which rule needed one first. + const decls = [HELPER_CMP, HELPER_TRI, HELPER_MICROS].filter((h) => helpers.has(h)); + return `.superRefine((row, ctx) => {\n${[...decls, ...blocks].join("\n")}\n})`; } /** @@ -426,6 +593,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); @@ -477,7 +645,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const docsPrefix = docs ? `${docs}\n` : ""; return code` -${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ +${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${durationHelperSince(durationUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ ${joinCode(insertFieldLines, { on: ",\n" })} })${insertRefinementSuffix(obj)}; `; @@ -586,6 +754,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const pkFields = primaryKeyFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); @@ -728,7 +897,7 @@ export type ${objName}CreatePreserving = ${z}.inputNames` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md index 21a2973750..525029b69c 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -186,8 +186,8 @@ When folding hand validators into metadata, apply human judgment per constraint. A constraint enters **shared metadata** only if it is a **true cross-language domain invariant**; a one-consumer preference stays in a thin local refinement layer. -Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `requiredWhen` / -`presentIff`); ratification decides *which* belong in shared metadata. Output a +Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `exactlyOne` / +`requiredWhen` / `presentIff`); ratification decides *which* belong in shared metadata. Output a **ratification table**: KEEP-IN-METADATA / LOCAL-REFINEMENT / DROP + rationale — human-approved, never applied silently. Distinguish `required` from has-a-safe-default (`@default` often fixes the over-requiring bug). A core attr ripples cross-port; for a diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index e145d1ccf6..71672d2a81 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -182,7 +182,8 @@ soft-delete / status / type view it models. `validator.regex`** — hunt hand field-level validation (presence, length, numeric range, array bounds, `@pattern` regex) the validator subtypes model. - **Cross-field validators** — `validator.comparison` (`@left`/`@op`/`@right`, e.g. - "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.requiredWhen` + "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.exactlyOne` (exactly one of N + present), `validator.requiredWhen` (conditional-required), `validator.presentIff` (mutual presence). Hunt these as hand-coded multi-field rules — they ARE modelable (see the Semantic-constraint ratification section in `SKILL.md` to decide what belongs in shared metadata vs port-local). diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md index 351893deb6..5148b26db4 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md @@ -35,7 +35,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/physical-names.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/physical-names.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/wire-format.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/wire-format.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-verify/references/migration.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-verify/references/migration.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md index 21a2973750..525029b69c 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -186,8 +186,8 @@ When folding hand validators into metadata, apply human judgment per constraint. A constraint enters **shared metadata** only if it is a **true cross-language domain invariant**; a one-consumer preference stays in a thin local refinement layer. -Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `requiredWhen` / -`presentIff`); ratification decides *which* belong in shared metadata. Output a +Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `exactlyOne` / +`requiredWhen` / `presentIff`); ratification decides *which* belong in shared metadata. Output a **ratification table**: KEEP-IN-METADATA / LOCAL-REFINEMENT / DROP + rationale — human-approved, never applied silently. Distinguish `required` from has-a-safe-default (`@default` often fixes the over-requiring bug). A core attr ripples cross-port; for a diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index e145d1ccf6..71672d2a81 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -182,7 +182,8 @@ soft-delete / status / type view it models. `validator.regex`** — hunt hand field-level validation (presence, length, numeric range, array bounds, `@pattern` regex) the validator subtypes model. - **Cross-field validators** — `validator.comparison` (`@left`/`@op`/`@right`, e.g. - "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.requiredWhen` + "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.exactlyOne` (exactly one of N + present), `validator.requiredWhen` (conditional-required), `validator.presentIff` (mutual presence). Hunt these as hand-coded multi-field rules — they ARE modelable (see the Semantic-constraint ratification section in `SKILL.md` to decide what belongs in shared metadata vs port-local). diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md index 351893deb6..5148b26db4 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md @@ -35,7 +35,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/physical-names.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/physical-names.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/wire-format.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/wire-format.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-verify/references/migration.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-verify/references/migration.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md index 21a2973750..525029b69c 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -186,8 +186,8 @@ When folding hand validators into metadata, apply human judgment per constraint. A constraint enters **shared metadata** only if it is a **true cross-language domain invariant**; a one-consumer preference stays in a thin local refinement layer. -Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `requiredWhen` / -`presentIff`); ratification decides *which* belong in shared metadata. Output a +Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `exactlyOne` / +`requiredWhen` / `presentIff`); ratification decides *which* belong in shared metadata. Output a **ratification table**: KEEP-IN-METADATA / LOCAL-REFINEMENT / DROP + rationale — human-approved, never applied silently. Distinguish `required` from has-a-safe-default (`@default` often fixes the over-requiring bug). A core attr ripples cross-port; for a diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index e145d1ccf6..71672d2a81 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -182,7 +182,8 @@ soft-delete / status / type view it models. `validator.regex`** — hunt hand field-level validation (presence, length, numeric range, array bounds, `@pattern` regex) the validator subtypes model. - **Cross-field validators** — `validator.comparison` (`@left`/`@op`/`@right`, e.g. - "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.requiredWhen` + "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.exactlyOne` (exactly one of N + present), `validator.requiredWhen` (conditional-required), `validator.presentIff` (mutual presence). Hunt these as hand-coded multi-field rules — they ARE modelable (see the Semantic-constraint ratification section in `SKILL.md` to decide what belongs in shared metadata vs port-local). diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md index 351893deb6..5148b26db4 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md @@ -35,7 +35,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md index 21a2973750..525029b69c 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -186,8 +186,8 @@ When folding hand validators into metadata, apply human judgment per constraint. A constraint enters **shared metadata** only if it is a **true cross-language domain invariant**; a one-consumer preference stays in a thin local refinement layer. -Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `requiredWhen` / -`presentIff`); ratification decides *which* belong in shared metadata. Output a +Cross-field rules **are** modelable (`comparison` / `atLeastOne` / `exactlyOne` / +`requiredWhen` / `presentIff`); ratification decides *which* belong in shared metadata. Output a **ratification table**: KEEP-IN-METADATA / LOCAL-REFINEMENT / DROP + rationale — human-approved, never applied silently. Distinguish `required` from has-a-safe-default (`@default` often fixes the over-requiring bug). A core attr ripples cross-port; for a diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index e145d1ccf6..71672d2a81 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -182,7 +182,8 @@ soft-delete / status / type view it models. `validator.regex`** — hunt hand field-level validation (presence, length, numeric range, array bounds, `@pattern` regex) the validator subtypes model. - **Cross-field validators** — `validator.comparison` (`@left`/`@op`/`@right`, e.g. - "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.requiredWhen` + "end ≥ start"), `validator.atLeastOne` (one-of-N present), `validator.exactlyOne` (exactly one of N + present), `validator.requiredWhen` (conditional-required), `validator.presentIff` (mutual presence). Hunt these as hand-coded multi-field rules — they ARE modelable (see the Semantic-constraint ratification section in `SKILL.md` to decide what belongs in shared metadata vs port-local). diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md index 351893deb6..5148b26db4 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/drift-signatures.md @@ -35,7 +35,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Hunt from the declaration outward, never from code shapes inward.** (1) Build the value list from the generated artifacts, because they already carry every derived value after every transform — naming strategy, pluralized path, humanized label: read `Names` and, on TypeScript, the `.meta.ts` twin. Where the port emits no names artifact yet, derive the list from the metadata directly (`@table` / `@schema` / `@column`, else the naming strategy applied to the field name; `@values`; `@title`) and record the un-wired generator as the first finding (below). (2) Grep the non-generated tree once — code AND checked-in config, fixtures, YAML, JSON, `.env`, OpenAPI fragments — with an alternation of those VALUES as literals; distinctive values (`"email_address"`, `"/subscribers"`, `"edited_approved"`) score on the hit, generic ones (`"id"`, `"name"`, `255`) only by reading the consumer, never by count. This step is what finds the topic name, the index field, the `COPY` header and the flag key without knowing they exist. (3) Trace each hit to its CONSUMER and ask what that consumer matches it against. Matched against the schema, the generated route table, a generated wire shape or a generated label → the metadata owns it, apply the discriminator. Matched against something the app chose — its own page-route table, its own copy deck — → not owned, not this finding. (4) Only then run the shape greps — raw SQL, `fetch(` / `app.get(` / a `queryKey`, a hand validator — as the second pass, for what a value grep cannot see: a literal that has ALREADY drifted from the declared value, a concatenated or interpolated name, a case-transformed one. - **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a flattened value-object composite column belongs to no single field of either object; a relationship-synthesized FK column is derived, never declared; a write-through entity's replica view name is not in the artifact, which holds the PRIMARY source's. A hand-written repository that imports `Names` is the gold standard — call it out as good. + **Do not flag — one rule, and its consequences.** A literal is exempt when (a) it is resolved, not compared: a typed ORM handle (a Drizzle column object, an Exposed `Column`, an EF property, a Pydantic attribute), a typed property access (`row.email`, `form.input.email`), a typed filter object, an enum member narrowed by the generated union — rewriting any of these to `Subscriber.email.name` or a string lookup swaps a checked reference for an unchecked one and is a regression (the emitted `options: [...] as const` populates a dropdown; it does not replace checked literals); (b) the metadata does not OWN it — the value coincides with a derived one but the app may change it without the metadata being wrong: a client-side page route or link (``, ``) is a navigation URL the app chose, measured on an estate where all 11 apparent `$path` hits were exactly this and the true count was 0, and converting one couples the site's URL structure to the API's; likewise the deployment prefix of a fetch URL, which the constants surface deliberately does not carry — `/api` is supplied at runtime through the client provider's `baseUrl`, never frozen at `meta gen` — so `fetch("/api/subscribers")` decomposes into a prefix with no constant by design and a `/subscribers` that is `$path`; (c) no generated constant exists for it — next paragraph; (d) it sits where a constant cannot be imported or names something no single declaration owns: a `source.rdb` `@sql` body is metadata, fingerprinted precisely because it can reference nothing; a relationship-synthesized FK column is derived, never declared. (A flattened value object's member column and a write-through entity's replica view name are NOT exempt: the artifact carries both, as the field's member column constant and the source in role `replica`.) A hand-written repository that imports `Names` is the gold standard — call it out as good. **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md index 0457d764eb..bad20b4e25 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/vocabulary-hygiene.md @@ -39,6 +39,10 @@ such, surfaced in the roadmap, but **non-failing** (the code works; the form is URL validation) — a distinct native type + behavior is a subtype, not a validated string. - A `field.string` that **holds an IP address** → recommend **`field.inet`** (native IP type; Postgres `inet` column). +- A `field.string` (or a CLR/JVM type override) that **holds a time span** (`TimeSpan`, an + `interval` column, "seconds until…") → recommend **`field.duration`** (native duration type; + Postgres `interval` column; ISO-8601 on the wire). Not a calendar span: months and years + are not durations. **Custom-provider vocabulary (adopters who register their own types/attrs):** check new/custom vocab against the ADR-0037 procedure (advisory) — e.g. *a custom subtype diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index 01fb198e0d..1979d161f2 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -245,6 +245,7 @@ Reach for these before inventing anything. When no row fits, run the decision pr | Open JSON, no reader narrows it | `field.string` + `@dbColumnType: jsonb` | the LAST rung — see "A JSON column" | | URL / URI | `field.uri` | native `URI`/`Uri`; `text` column; strict absolute-URI validation (add `@lenient: true` to store any string) — a real native type + behavior, so a subtype (not a validated string) | | IP address | `field.inet` | native IP type; Postgres `inet` column; strict IPv4/IPv6-literal validation (add `@lenient: true` for a plain-string `text` column) | +| Elapsed time / timeout / delay | `field.duration` | native `TimeSpan`/`Duration`/`timedelta`; Postgres `interval` column; wire form is the port's native one by default, canonical ISO-8601 (`PT1H30M`) under `wire.durations: iso`. Not for a calendar span ("every 3 months"): that is an int and a unit. Never a string or a count of seconds | | Validated plain string (email / hostname) | `field.string` + `@stringFormat` | `@stringFormat: email` or `@stringFormat: hostname` — idiomatic per-port validation; don't hand-write the `validator.regex` | Four rules behind those rows, in short (full text: `references/choosing-a-shape.md`): @@ -285,10 +286,10 @@ Reuse a constraint set across entities with an abstract `field.enum` + `extends` `field.object` embeds another `object` declaration. `@objectRef` names it; `@storage` controls persistence: -- `flattened` — one DB column per sub-field (`address_street`, `address_city`, …). - Illegal on array fields. -- `jsonb` — one `jsonb` column. -- `subdocument` (default, back-compat) — single jsonb column. +- `flattened` — one DB column per sub-field (`address_street`, …); illegal on arrays. `@flattenedColumnPrefix` + replaces the `_` prefix verbatim (`""` keeps the members' own columns). +- `jsonb` — one `jsonb` column; also the default when `@storage` is not set. +- `subdocument` — single jsonb column (a hint for document-store targets). ```json { "field.object": { "name": "address", "@objectRef": "Address", "@storage": "flattened" } } @@ -434,16 +435,21 @@ store with no `CHECK` support is still covered. PATCH and partial-update models | Subtype | Rule | Key attrs | |---|---|---| -| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings and temporals only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | +| `validator.comparison` | two fields stand in a relational order (`@left @op @right`). Numerics compare with each other; strings, temporals and durations only with the same subtype and `@localTime`; boolean, enum, uuid, uri and inet only `eq`/`ne`. Any other pair is skipped, never rejected. | `@left`, `@op` (`gt`/`gte`/`lt`/`lte`/`ne`/`eq`), `@right` | | `validator.requiredWhen` | `@field` is required when `@when` equals `@equals` | `@field`, `@when`, `@equals` | | `validator.presentIff` | `@field` is present **iff** `@when` equals `@equals` (biconditional) | `@field`, `@when`, `@equals` | | `validator.atLeastOne` | at least one of `@fields` (2+) is present | `@fields` | +| `validator.exactlyOne` | exactly one of `@fields` (2+) is present | `@fields` | + +All five take an optional `@condition` (the `origin.computed` grammar): the rule holds only where it does. +Every validator that becomes a CHECK takes an optional `@constraintName`. ```json { "validator.comparison": { "name": "hpInRange", "@left": "currentHp", "@op": "lte", "@right": "maxHp" } } { "validator.requiredWhen": { "name": "reasonIfRejected", "@field": "rejectReason", "@when": "status", "@equals": "rejected" } } { "validator.presentIff": { "name": "usedAtWhenUsed", "@field": "usedAt", "@when": "isUsed", "@equals": "true" } } { "validator.atLeastOne": { "name": "emailOrPhone", "@fields": ["email", "phone"] } } +{ "validator.exactlyOne": { "name": "oneOwner", "@fields": ["customerId", "accountId"] } } ``` These are children of `object.entity`, alongside its fields and identities. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/physical-names.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/physical-names.md index 83f8364241..27bf59a63b 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/physical-names.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/physical-names.md @@ -89,14 +89,26 @@ literally. **Java and Python generate no SQL at all**: their DTOs and models car names, and persistence is the repository interface/`Protocol` you implement. There the artifact exists *for your code*, which is the only place a physical name appears. -Two categories stay literal, deliberately, and the gate pins them as such rather than -exempting them: - -- a **flattened value-object column** (`@storage: flattened`) is a composite — - `_` — belonging to no single field of either object, - so there is no one constant to reference; -- a **write-through entity's replica view name**: the artifact holds the object's PRIMARY - source's name (its table), and a write-through entity has two physical names. +**A flattened value object's columns are constants of the OWNING object's artifact.** A +`@storage: flattened` field has no column of its own: each member of the value object is a +column of the owner's table, named by core naming (the field's `@flattenedColumnPrefix`, +else `_`, then the member's column). The artifact carries one constant per +member, and the field itself has none, because the table has no column of that name: + +| Port | Reads as | +|---|---| +| TypeScript | `ShipmentNames.fields.shipTo.members.street.column` | +| C# | `ShipmentNames.ShipToStreetColumn` | +| Java / Kotlin | `ShipmentNames.SHIP_TO_STREET_COLUMN` | +| Python | `SHIPMENT_SHIP_TO_STREET_COLUMN` | + +The column-by-field maps (`ColumnsByField`, `COLUMNS_BY_FIELD`) key these `shipTo.street`. +A member constant that would share its name with another constant (a field `shipToStreet` +beside a flattened `shipTo.street` with a custom prefix) is left out with a warning; rename +one to get it. The generated EF Core and Exposed bindings still spell these columns as +literals, because an owned copy of an older names generator does not emit the member +constants. Without the artifact, compute the name with core naming's +`resolveFlattenedColumnName` (or its port equivalent), never by concatenating. **But prefer a typed handle where one exists — this rule has a real limit.** If the ORM gives you a type-checked object for the same thing, use that. Replacing a Drizzle column diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/wire-format.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/wire-format.md index a27288b3d1..b0b2f91c8a 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/wire-format.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/wire-format.md @@ -21,13 +21,16 @@ not "fix" a port to match another one, or to match the corpus. loaded root; there is no CLI flag and no generator argument. ```json -{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native" } } +{ "schema_version": 1, "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" } } ``` - `decimals`: `native` (default) or `string`, the canonical string (`"12.5"`, zero `"0"`, no exponent, no trailing zeros). - `nullMembers`: `native` (default), `emit` or `omit`. It covers the members of an `object.value` only. A top-level entity field that is null is always written as `null`. +- `durations`: `native` (default) or `iso`, for `field.duration`. `native` writes and reads each + port's standard duration form (C# `TimeSpan` is `"01:30:00"`). `iso` writes and reads the + canonical ISO-8601 form (`PT1H30M`) in every port. - An absent block or key means `native`. An unknown key or value is refused when the config is read. There is no `"number"` value: in C#, Java and Kotlin `native` already is a number. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-verify/references/migration.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-verify/references/migration.md index 6828d995d3..f589708490 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-verify/references/migration.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-verify/references/migration.md @@ -283,6 +283,6 @@ emitted, created, diffed and dropped** — derived from the model, never written SQL. `field.enum` membership (`IN (…)`, the mapped integers for an `@intValueMap` enum), `validator.numeric` min/max, `validator.length` min/max, `validator.regex` (Postgres only — SQLite has no regex operator), and the entity-level cross-field -validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`) each become a +validators (`comparison`, `requiredWhen`, `presentIff`, `atLeastOne`, `exactlyOne`) each become a named `CHECK`. Array columns get none (the element rules are enforced app-side). (Partial + descending **indexes** are supported too — see Index modeling above.) diff --git a/fixtures/api-contract-conformance/.metaobjects/config.json b/fixtures/api-contract-conformance/.metaobjects/config.json index d6d2b1359d..3a5a786b8c 100644 --- a/fixtures/api-contract-conformance/.metaobjects/config.json +++ b/fixtures/api-contract-conformance/.metaobjects/config.json @@ -1,5 +1,5 @@ { "schema_version": 1, "naming": { "columns": "literal" }, - "wire": { "decimals": "string", "nullMembers": "omit" } + "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" } } diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 7574beeab5..2e49a231e3 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -52,8 +52,8 @@ fixtures/api-contract-conformance/ └── filter-invalid-value.yaml # uncoercible value → 400; the fourth envelope, previously ungated ``` -Beside the 31 core scenarios above sit eight **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (88 scenarios in all, 119 with the +Beside the 31 core scenarios above sit nine **sub-corpora**, each a directory with its +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (90 scenarios in all, 121 with the core): | Directory | Scenarios | Gates | Lanes | @@ -66,6 +66,7 @@ core): | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | | `decimal/` | 11 | `field.decimal` is an exact canonical string on the wire (ADR-0019 Amendment 1): read, filter, sort, write and a report over decimals. A scenario's `engines:` key is exact; the `-15` twins run on SQLite and D1 as well. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | +| `duration/` | 2 | `field.duration` is a canonical ISO-8601 span on the wire under `"wire": { "durations": "iso" }`: write, read, refuse, and filter and sort as a span on every engine. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` package: diff --git a/fixtures/api-contract-conformance/duration/README.md b/fixtures/api-contract-conformance/duration/README.md new file mode 100644 index 0000000000..82026ccaa3 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/README.md @@ -0,0 +1,46 @@ +# `api-contract-conformance/duration/` — a duration is an ISO-8601 span on the wire + +Cross-port REST contract for `field.duration`, over one writable entity: `Timer` (table `timers`, +served at `/api/timers`) with a nullable, filterable, sortable `timeout`. The corpus config +(`../.metaobjects/config.json`) pins `"wire": { "durations": "iso" }`, so every port writes the +same canonical form. Under the default `native` setting each port writes its own serializer's form +and this corpus does not apply (`docs/features/api-contract.md`, "The wire setting"). + +## The rule (D1 to D4) + +- **D1, emit.** A duration is a JSON string in the canonical form `[-]PT[nH][nM][n[.f]S]`: hours + are never folded into days (`PT26H`), zero components are left out, zero is `PT0S`, and a + fraction has no trailing zeros. `null` stays `null`. +- **D2, accept.** A request body carries the accepted form: optional leading weeks (`P1W` is 7 days) + and days (`P1DT2H` is 26 hours), then hours, minutes and seconds. It is stored as the span and + answered canonical (`PT5400S` is answered `PT1H30M`). +- **D3, refuse.** Years and months (a calendar span has no fixed length), a clock spelling + (`01:30:00`), lowercase designators and a JSON number are `400 {"error": "validation"}` and + persist nothing. +- **D4, order.** A filter operand uses the D2 grammar and compares spans, never text: `gt`, `lte` + and a sort take in a negative span, `eq PT5400S` matches `PT1H30M`, and `in` matches across + spellings. A bad operand is `400 {"error": "invalid_filter_value", "field": "timeout"}`. Postgres + holds an `interval`; SQLite, D1 and MySQL a BIGINT count of microseconds, which orders the same way. + +The wire grammar and its vectors are in [`fixtures/duration-wire/`](../../duration-wire/). Neither +scenario carries an `engines:` key, so both run on every engine a port's lane has. + +## Scenarios + +| Scenario | What it pins | +|---|---| +| `duration-write-read` | POST and GET by id answer the canonical form for an hour-and-minutes span, days, seconds past an hour, a week, a negative span, zero and a fraction; an absent `timeout` reads `null`; PATCH replaces; `P1M`, `01:30:00`, `pt1h` and `5400` are `400 validation` and persist nothing | +| `duration-filter-sort` | five rows written by POST, then `gt` with a descending sort, `lte` with an ascending sort over a negative span, `eq` and `in` across spellings, `isNull` both ways with `limit`, and two bad operands | + +## Files + +``` +duration/ +├── README.md +├── meta.json # Timer (/api/timers) +├── seed.json # `timers` (empty, so the first POST is id 1) +├── schema.postgres.sql # TypeScript-produced: `timeout` is INTERVAL +├── schema.sqlite.sql # the same, for SQLite and D1: `timeout` is INTEGER (microseconds) +├── schema.mysql.sql # MySQL 8.4: the hand-written table, `timeout` BIGINT (microseconds) +└── scenarios/ +``` diff --git a/fixtures/api-contract-conformance/duration/meta.json b/fixtures/api-contract-conformance/duration/meta.json new file mode 100644 index 0000000000..a3d0a27584 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/meta.json @@ -0,0 +1,12 @@ +{ "metadata.root": { + "package": "acme::ops", + "children": [ + { "object.entity": { "name": "Timer", "children": [ + { "source.rdb": { "@table": "timers" } }, + { "field.long": { "name": "id", "@filterable": true, "@sortable": true } }, + { "field.string": { "name": "label", "@required": true, "@maxLength": 16, "@filterable": true } }, + { "field.duration": { "name": "timeout", "@filterable": true, "@sortable": true } }, + { "identity.primary": { "name": "pk", "@fields": "id", "@generation": "increment" } } + ] } } + ] +} } diff --git a/fixtures/api-contract-conformance/duration/scenarios/duration-filter-sort.yaml b/fixtures/api-contract-conformance/duration/scenarios/duration-filter-sort.yaml new file mode 100644 index 0000000000..67a419d8f7 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/scenarios/duration-filter-sort.yaml @@ -0,0 +1,80 @@ +name: duration-filter-sort +description: > + A duration filters and sorts as the span it is, never as text, on every engine (D4): Postgres keeps + an interval, SQLite, D1 and MySQL a 64-bit count of microseconds. A range filter takes in a + negative span, equality holds across spellings (PT5400S is PT1H30M), and a bad operand is + 400 invalid_filter_value naming the field. +requests: + - { id: s1, method: POST, path: /api/timers, body: { label: thirty, timeout: "PT30M" }, expect: { status: 201 } } + - { id: s2, method: POST, path: /api/timers, body: { label: ninety, timeout: "PT1H30M" }, expect: { status: 201 } } + - { id: s3, method: POST, path: /api/timers, body: { label: minus, timeout: "-PT1M30S" }, expect: { status: 201 } } + - { id: s4, method: POST, path: /api/timers, body: { label: long, timeout: "PT26H" }, expect: { status: 201 } } + - { id: s5, method: POST, path: /api/timers, body: { label: none }, expect: { status: 201 } } + - id: r1 + method: GET + path: /api/timers?filter[timeout][gt]=PT1H&sort=timeout:desc + expect: + status: 200 + body: + equals: + - { id: 4, label: long, timeout: "PT26H" } + - { id: 2, label: ninety, timeout: "PT1H30M" } + - id: r2 + method: GET + path: /api/timers?filter[timeout][lte]=PT30M&sort=timeout:asc + expect: + status: 200 + body: + equals: + - { id: 3, label: minus, timeout: "-PT1M30S" } + - { id: 1, label: thirty, timeout: "PT30M" } + - id: r3 + method: GET + path: /api/timers?filter[timeout][eq]=PT5400S + expect: + status: 200 + body: + equals: + - { id: 2, label: ninety, timeout: "PT1H30M" } + - id: r4 + method: GET + path: /api/timers?filter[timeout][in]=PT30M,P1DT2H&sort=timeout:asc + expect: + status: 200 + body: + equals: + - { id: 1, label: thirty, timeout: "PT30M" } + - { id: 4, label: long, timeout: "PT26H" } + - id: r5 + method: GET + path: /api/timers?filter[timeout][isNull]=true + expect: + status: 200 + body: + equals: + - { id: 5, label: none, timeout: null } + - id: r6 + method: GET + path: /api/timers?filter[timeout][isNull]=false&sort=timeout:asc&limit=2 + expect: + status: 200 + body: + equals: + - { id: 3, label: minus, timeout: "-PT1M30S" } + - { id: 1, label: thirty, timeout: "PT30M" } + - id: r7 + method: GET + path: /api/timers?filter[timeout][gt]=P1M + expect: + status: 400 + body: + error: invalid_filter_value + field: timeout + - id: r8 + method: GET + path: /api/timers?filter[timeout][gt]=soon + expect: + status: 400 + body: + error: invalid_filter_value + field: timeout diff --git a/fixtures/api-contract-conformance/duration/scenarios/duration-write-read.yaml b/fixtures/api-contract-conformance/duration/scenarios/duration-write-read.yaml new file mode 100644 index 0000000000..669f7e1087 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/scenarios/duration-write-read.yaml @@ -0,0 +1,132 @@ +name: duration-write-read +description: > + A field.duration is written and read as an ISO-8601 duration under "wire": { "durations": "iso" } + (D1, D2). Every port answers the one canonical form, [-]PT[nH][nM][n[.f]S]: hours are never folded + into days, a week is 7 days and a day 24 hours, and zero is PT0S. Years, months, a clock spelling, + lowercase designators and a JSON number are 400 validation (D3) and persist nothing. +requests: + - id: r1 + method: POST + path: /api/timers + body: { label: plain, timeout: "PT1H30M" } + expect: + status: 201 + body: + row: { id: 1, label: plain, timeout: "PT1H30M" } + - id: r2 + method: GET + path: /api/timers/1 + expect: + status: 200 + body: + row: { id: 1, label: plain, timeout: "PT1H30M" } + - id: r3 + method: POST + path: /api/timers + body: { label: days, timeout: "P1DT2H" } + expect: + status: 201 + body: + row: { label: days, timeout: "PT26H" } + - id: r4 + method: POST + path: /api/timers + body: { label: seconds, timeout: "PT5400S" } + expect: + status: 201 + body: + row: { label: seconds, timeout: "PT1H30M" } + - id: r5 + method: POST + path: /api/timers + body: { label: week, timeout: "P1W" } + expect: + status: 201 + body: + row: { label: week, timeout: "PT168H" } + - id: r6 + method: POST + path: /api/timers + body: { label: negative, timeout: "-PT1M30S" } + expect: + status: 201 + body: + row: { label: negative, timeout: "-PT1M30S" } + - id: r7 + method: POST + path: /api/timers + body: { label: zero, timeout: "PT0S" } + expect: + status: 201 + body: + row: { label: zero, timeout: "PT0S" } + - id: r8 + method: POST + path: /api/timers + body: { label: fraction, timeout: "PT1.5S" } + expect: + status: 201 + body: + row: { label: fraction, timeout: "PT1.5S" } + - id: r9 + method: POST + path: /api/timers + body: { label: unset } + expect: + status: 201 + body: + row: { label: unset, timeout: null } + - id: r10 + method: PATCH + path: /api/timers/1 + body: { timeout: "PT45M" } + expect: + status: 200 + body: + row: { id: 1, label: plain, timeout: "PT45M" } + - id: r11 + method: GET + path: /api/timers/1 + expect: + status: 200 + body: + row: { id: 1, label: plain, timeout: "PT45M" } + - id: r12 + method: POST + path: /api/timers + body: { label: bad, timeout: "P1M" } + expect: + status: 400 + body: + error: validation + - id: r13 + method: POST + path: /api/timers + body: { label: bad, timeout: "01:30:00" } + expect: + status: 400 + body: + error: validation + - id: r14 + method: POST + path: /api/timers + body: { label: bad, timeout: "pt1h" } + expect: + status: 400 + body: + error: validation + - id: r15 + method: POST + path: /api/timers + body: { label: bad, timeout: 5400 } + expect: + status: 400 + body: + error: validation + - id: r16 + method: GET + path: /api/timers?filter[label][eq]=bad + expect: + status: 200 + body: + length: 0 diff --git a/fixtures/api-contract-conformance/duration/schema.mysql.sql b/fixtures/api-contract-conformance/duration/schema.mysql.sql new file mode 100644 index 0000000000..fec083bf12 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/schema.mysql.sql @@ -0,0 +1,9 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- api-contract duration sub-corpus schema (MySQL 8.4, literal column naming). +-- Regenerate: `bun run gen:duration-api-schema` in server/typescript/packages/integration-tests. + +CREATE TABLE `timers` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `label` VARCHAR(16) NOT NULL, + `timeout` BIGINT +); diff --git a/fixtures/api-contract-conformance/duration/schema.postgres.sql b/fixtures/api-contract-conformance/duration/schema.postgres.sql new file mode 100644 index 0000000000..887a56ab0e --- /dev/null +++ b/fixtures/api-contract-conformance/duration/schema.postgres.sql @@ -0,0 +1,10 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- api-contract duration sub-corpus schema (Postgres, literal column naming). +-- Regenerate: `bun run gen:duration-api-schema` in server/typescript/packages/integration-tests. + +CREATE TABLE "timers" ( + "id" BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL, + "label" VARCHAR(16) NOT NULL, + "timeout" INTERVAL, + CONSTRAINT "timers_pkey" PRIMARY KEY ("id") +); diff --git a/fixtures/api-contract-conformance/duration/schema.sqlite.sql b/fixtures/api-contract-conformance/duration/schema.sqlite.sql new file mode 100644 index 0000000000..26a265173a --- /dev/null +++ b/fixtures/api-contract-conformance/duration/schema.sqlite.sql @@ -0,0 +1,9 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- api-contract duration sub-corpus schema (SQLite, literal column naming). +-- Regenerate: `bun run gen:duration-api-schema` in server/typescript/packages/integration-tests. + +CREATE TABLE "timers" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "label" VARCHAR(16) NOT NULL, + "timeout" INTEGER +); diff --git a/fixtures/api-contract-conformance/duration/seed.json b/fixtures/api-contract-conformance/duration/seed.json new file mode 100644 index 0000000000..a60fbf99e8 --- /dev/null +++ b/fixtures/api-contract-conformance/duration/seed.json @@ -0,0 +1 @@ +{ "timers": [] } diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 25ae559688..e9ad9cce50 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -54,6 +54,11 @@ "ERR_TEMPLATE_ENGINE_UNSUPPORTED": "Build-time verify and render (ADR-0061): a template body is written for an engine this port does not ship — a Jinja body (.jinja / .j2 / .jinja2) in the Java, Kotlin or C# port, which render Mustache only. path = the template ref.", "ERR_BAD_ATTR_FILTER": "An attr.filter references a field that is not addressable in that context or uses a disallowed op: a dataGrid @filter over a non-filterable field or an op disallowed for the field's subtype, or a projection row-scope @filter over a non-declared (dangling) or aggregate-derived field (#207). FR-044: a segment / measure.aggregate / object.report @filter names a field its entity (or @from) lacks or an op illegal for the field (S1); a relative-date value { \"now\": \"\" } appears outside the @filter of a segment, measure.aggregate or object.report (F1); or a relative-date value sits on a field that is not field.date/field.timestamp, under an op other than gt/gte/lt/lte, carries a value that is not an ISO-8601 duration, or has a startOf other than day, week or month, or a key besides now and startOf (F2).", "ERR_STORAGE_FLATTENED_ARRAY": "@storage \"flattened\" cannot be combined with isArray=true.", + "ERR_SEQUENCE_START_OUT_OF_RANGE": "An identity.primary @sequenceStart is below 1, or above the largest value its key field can hold (2147483647 for a field.int). Postgres would refuse the sequence when the migration is applied, so the model is refused at load.", + "ERR_SEQUENCE_WITHOUT_INCREMENT": "An identity.primary sets @sequenceStart or @sequenceName, but its @generation is not \"increment\". Only an increment key draws from a sequence, so the attribute would change nothing: the identity is refused rather than silently ignored.", + "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH": "A string-stored field.enum declares @maxLength (its column length) shorter than one of its @values members, which could not be stored in the column. The longest offending member is named.", + "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP": "A field.enum sets both @maxLength and @intValueMap. An @intValueMap enum is stored as an integer, so a column length would change nothing: the field is refused rather than silently ignored.", + "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE": "@flattenedColumnPrefix is set on a field.object whose @storage is not \"flattened\" (jsonb, subdocument, or absent, which is jsonb). Only a flattened value object puts its members' columns in the owning table, so the prefix would change nothing: the field is refused rather than silently ignored.", "ERR_STORAGE_WITHOUT_OBJECT_REF": "@storage was set on a field that has no @objectRef.", "ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF": "ADR-0013: a field.object declares no @objectRef. A field.object models a typed nested value and REQUIRES @objectRef. The message states the whole ladder rather than only the escape hatch: a known key set is an object.value the field @objectRefs; dynamic keys over a known value type are a field.map (@objectRef for a value object, @valueType for a scalar); only a bag no reader pins a key in is @dbColumnType: jsonb on a field.string. Naming the escape hatch alone is what taught adopters to reach for the bag by default.", "ERR_UNRESOLVED_OBJECT_REF": "ADR-0042: a field.object / field.map @objectRef does not resolve to any object in the loaded tree (a dangling target). The ref resolves package-locally when bare (referrer's package, else root-level) and exactly when FQN — a bare cross-package ref no longer binds elsewhere. The error names same-short-name objects in other packages so the author can qualify it.", diff --git a/fixtures/conformance/column-name-flattened-prefixes-ok/expected.json b/fixtures/conformance/column-name-flattened-prefixes-ok/expected.json new file mode 100644 index 0000000000..11a209760f --- /dev/null +++ b/fixtures/conformance/column-name-flattened-prefixes-ok/expected.json @@ -0,0 +1,65 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.object": { + "name": "billTo", + "@flattenedColumnPrefix": "", + "@objectRef": "test::Address", + "@storage": "flattened" + } + }, + { + "field.object": { + "name": "shipTo", + "@flattenedColumnPrefix": "ship_", + "@objectRef": "test::Address", + "@storage": "flattened" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/column-name-flattened-prefixes-ok/input/meta.orders.json b/fixtures/conformance/column-name-flattened-prefixes-ok/input/meta.orders.json new file mode 100644 index 0000000000..9ba446ee6d --- /dev/null +++ b/fixtures/conformance/column-name-flattened-prefixes-ok/input/meta.orders.json @@ -0,0 +1,63 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.object": { + "name": "billTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "" + } + }, + { + "field.object": { + "name": "shipTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "ship_" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/column-name-flattened-prefixes-ok/providers.json b/fixtures/conformance/column-name-flattened-prefixes-ok/providers.json new file mode 100644 index 0000000000..350e327ab5 --- /dev/null +++ b/fixtures/conformance/column-name-flattened-prefixes-ok/providers.json @@ -0,0 +1,4 @@ +[ + "metaobjects-core-types", + "metaobjects-db" +] diff --git a/fixtures/conformance/enum-max-length/expected.json b/fixtures/conformance/enum-max-length/expected.json new file mode 100644 index 0000000000..f48725ddb3 --- /dev/null +++ b/fixtures/conformance/enum-max-length/expected.json @@ -0,0 +1,38 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "status", + "@maxLength": 9, + "@values": [ + "DRAFT", + "PUBLISHED", + "ARCHIVED" + ] + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-max-length/input/meta.enums.json b/fixtures/conformance/enum-max-length/input/meta.enums.json new file mode 100644 index 0000000000..fad79bf83b --- /dev/null +++ b/fixtures/conformance/enum-max-length/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@maxLength": 9 + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/expected-errors.json b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/expected-errors.json new file mode 100644 index 0000000000..e3930c61b4 --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/expected-errors.json @@ -0,0 +1,25 @@ +{ + "errors": [ + { + "code": "ERR_COLUMN_NAME_COLLISION", + "source": { + "format": "json", + "files": [ + "meta.orders.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[3]['field.object']" + } + }, + { + "code": "ERR_COLUMN_NAME_COLLISION", + "source": { + "format": "json", + "files": [ + "meta.orders.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[3]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/input/meta.orders.json b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/input/meta.orders.json new file mode 100644 index 0000000000..fa9c4c31ac --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/input/meta.orders.json @@ -0,0 +1,63 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.object": { + "name": "billTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "" + } + }, + { + "field.object": { + "name": "shipTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/providers.json b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/providers.json new file mode 100644 index 0000000000..350e327ab5 --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-empty-prefix/providers.json @@ -0,0 +1,4 @@ +[ + "metaobjects-core-types", + "metaobjects-db" +] diff --git a/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/expected-errors.json b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/expected-errors.json new file mode 100644 index 0000000000..ff210c30a7 --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLUMN_NAME_COLLISION", + "source": { + "format": "json", + "files": [ + "meta.orders.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[3]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/input/meta.orders.json b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/input/meta.orders.json new file mode 100644 index 0000000000..07cdff7559 --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/input/meta.orders.json @@ -0,0 +1,60 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "street" + } + }, + { + "field.object": { + "name": "home", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/providers.json b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/providers.json new file mode 100644 index 0000000000..350e327ab5 --- /dev/null +++ b/fixtures/conformance/error-column-name-collision-flattened-member-vs-field/providers.json @@ -0,0 +1,4 @@ +[ + "metaobjects-core-types", + "metaobjects-db" +] diff --git a/fixtures/conformance/error-enum-max-length-int-value-map/expected-errors.json b/fixtures/conformance/error-enum-max-length-int-value-map/expected-errors.json new file mode 100644 index 0000000000..412a138aba --- /dev/null +++ b/fixtures/conformance/error-enum-max-length-int-value-map/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-max-length-int-value-map/input/meta.enums.json b/fixtures/conformance/error-enum-max-length-int-value-map/input/meta.enums.json new file mode 100644 index 0000000000..b455875959 --- /dev/null +++ b/fixtures/conformance/error-enum-max-length-int-value-map/input/meta.enums.json @@ -0,0 +1,24 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 1 }, + "@maxLength": 9 + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/expected-errors.json b/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/expected-errors.json new file mode 100644 index 0000000000..166e3a509e --- /dev/null +++ b/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/expected-errors.json @@ -0,0 +1,25 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + }, + { + "code": "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[2]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/input/meta.enums.json b/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/input/meta.enums.json new file mode 100644 index 0000000000..6069a85876 --- /dev/null +++ b/fixtures/conformance/error-enum-value-exceeds-max-length-every-field/input/meta.enums.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT", "AWAITING_REVIEW"], "@maxLength": 9 } }, + { "field.enum": { "name": "channel", "@values": ["EMAIL", "TELEPHONE"], "@maxLength": 5 } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-value-exceeds-max-length/expected-errors.json b/fixtures/conformance/error-enum-value-exceeds-max-length/expected-errors.json new file mode 100644 index 0000000000..caf054c0c3 --- /dev/null +++ b/fixtures/conformance/error-enum-value-exceeds-max-length/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-value-exceeds-max-length/input/meta.enums.json b/fixtures/conformance/error-enum-value-exceeds-max-length/input/meta.enums.json new file mode 100644 index 0000000000..d6f75adf1f --- /dev/null +++ b/fixtures/conformance/error-enum-value-exceeds-max-length/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "AWAITING_REVIEW"], + "@maxLength": 9 + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/expected-errors.json b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/expected-errors.json new file mode 100644 index 0000000000..9c0ece68e4 --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE", + "source": { + "format": "json", + "files": [ + "meta.storage.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/input/meta.storage.json b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/input/meta.storage.json new file mode 100644 index 0000000000..098810c27b --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/input/meta.storage.json @@ -0,0 +1,57 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.object": { + "name": "billTo", + "@objectRef": "test::Address", + "@flattenedColumnPrefix": "" + } + }, + { + "field.object": { + "name": "returnTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "ret_" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/providers.json b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/providers.json new file mode 100644 index 0000000000..f9bb435a37 --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-default-storage/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/expected-errors.json b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/expected-errors.json new file mode 100644 index 0000000000..9c0ece68e4 --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE", + "source": { + "format": "json", + "files": [ + "meta.storage.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/input/meta.storage.json b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/input/meta.storage.json new file mode 100644 index 0000000000..720dae57a1 --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/input/meta.storage.json @@ -0,0 +1,58 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.object": { + "name": "shipTo", + "@objectRef": "test::Address", + "@storage": "jsonb", + "@flattenedColumnPrefix": "ship_" + } + }, + { + "field.object": { + "name": "returnTo", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "ret_" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/providers.json b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/providers.json new file mode 100644 index 0000000000..f9bb435a37 --- /dev/null +++ b/fixtures/conformance/error-field-object-flattened-column-prefix-without-flattened-storage/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-identity-sequence-start-out-of-range/expected-errors.json b/fixtures/conformance/error-identity-sequence-start-out-of-range/expected-errors.json new file mode 100644 index 0000000000..6479c9d69c --- /dev/null +++ b/fixtures/conformance/error-identity-sequence-start-out-of-range/expected-errors.json @@ -0,0 +1,25 @@ +{ + "errors": [ + { + "code": "ERR_SEQUENCE_START_OUT_OF_RANGE", + "source": { + "format": "json", + "files": [ + "meta.seq.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[2]['identity.primary']" + } + }, + { + "code": "ERR_SEQUENCE_START_OUT_OF_RANGE", + "source": { + "format": "json", + "files": [ + "meta.seq.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[2]['identity.primary']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-identity-sequence-start-out-of-range/input/meta.seq.json b/fixtures/conformance/error-identity-sequence-start-out-of-range/input/meta.seq.json new file mode 100644 index 0000000000..276ebac20b --- /dev/null +++ b/fixtures/conformance/error-identity-sequence-start-out-of-range/input/meta.seq.json @@ -0,0 +1,53 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Small", + "children": [ + { + "source.rdb": {} + }, + { + "field.int": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id", + "@generation": "increment", + "@sequenceStart": 3000000000 + } + } + ] + } + }, + { + "object.entity": { + "name": "Zero", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id", + "@generation": "increment", + "@sequenceStart": 0 + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-identity-sequence-without-increment/expected-errors.json b/fixtures/conformance/error-identity-sequence-without-increment/expected-errors.json new file mode 100644 index 0000000000..57289aed65 --- /dev/null +++ b/fixtures/conformance/error-identity-sequence-without-increment/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_SEQUENCE_WITHOUT_INCREMENT", + "source": { + "format": "json", + "files": [ + "meta.seq.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[2]['identity.primary']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-identity-sequence-without-increment/input/meta.seq.json b/fixtures/conformance/error-identity-sequence-without-increment/input/meta.seq.json new file mode 100644 index 0000000000..53ce09e514 --- /dev/null +++ b/fixtures/conformance/error-identity-sequence-without-increment/input/meta.seq.json @@ -0,0 +1,32 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { + "source.rdb": { + "@table": "tickets" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id", + "@generation": "assigned", + "@sequenceStart": 1000000000 + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/expected-errors.json b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/expected-errors.json new file mode 100644 index 0000000000..5de76c39a7 --- /dev/null +++ b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_MERGE_CONFLICT", + "source": { + "format": "merged", + "files": [ + "meta.a.json", + "meta.b.json" + ] + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.a.json b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.a.json new file mode 100644 index 0000000000..3b999009c4 --- /dev/null +++ b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.a.json @@ -0,0 +1,55 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "source.rdb": { + "@table": "orders" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.object": { + "name": "home", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.b.json b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.b.json new file mode 100644 index 0000000000..3c69b752da --- /dev/null +++ b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/input/meta.b.json @@ -0,0 +1,22 @@ +{ + "metadata.root": { + "package": "test", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.object": { + "name": "home", + "@objectRef": "test::Address", + "@storage": "flattened", + "@flattenedColumnPrefix": "home_" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/providers.json b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/providers.json new file mode 100644 index 0000000000..350e327ab5 --- /dev/null +++ b/fixtures/conformance/error-merge-conflict-flattened-column-prefix-empty/providers.json @@ -0,0 +1,4 @@ +[ + "metaobjects-core-types", + "metaobjects-db" +] diff --git a/fixtures/conformance/error-validator-at-least-one-max-unknown/expected-errors.json b/fixtures/conformance/error-validator-at-least-one-max-unknown/expected-errors.json new file mode 100644 index 0000000000..4bb7627e76 --- /dev/null +++ b/fixtures/conformance/error-validator-at-least-one-max-unknown/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_UNKNOWN_ATTR", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[7]['validator.atLeastOne']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-at-least-one-max-unknown/input/meta.checks.json b/fixtures/conformance/error-validator-at-least-one-max-unknown/input/meta.checks.json new file mode 100644 index 0000000000..04158bd523 --- /dev/null +++ b/fixtures/conformance/error-validator-at-least-one-max-unknown/input/meta.checks.json @@ -0,0 +1,62 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Visit", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "field.boolean": { + "name": "active" + } + }, + { + "field.enum": { + "name": "kind", + "@values": [ + "Owned", + "Shared" + ] + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id" + } + }, + { + "validator.atLeastOne": { + "name": "none", + "@fields": [ + "storeId", + "kioskId" + ], + "@max": 0 + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-validator-condition-duration-literal/expected-errors.json b/fixtures/conformance/error-validator-condition-duration-literal/expected-errors.json new file mode 100644 index 0000000000..33942fa7d5 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-duration-literal/expected-errors.json @@ -0,0 +1,55 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.timers.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[6]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.timers.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[8]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.timers.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[9]['validator.requiredWhen']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.timers.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[10]['validator.presentIff']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.timers.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[11]['validator.requiredWhen']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-condition-duration-literal/input/meta.timers.json b/fixtures/conformance/error-validator-condition-duration-literal/input/meta.timers.json new file mode 100644 index 0000000000..e257fc7548 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-duration-literal/input/meta.timers.json @@ -0,0 +1,73 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Timer", + "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "storeId" } }, + { "field.long": { "name": "kioskId" } }, + { "field.duration": { "name": "timeout" } }, + { "identity.primary": { "name": "pk", "@fields": "id" } }, + { + "validator.atLeastOne": { + "name": "conditionNotIso", + "@fields": ["storeId", "kioskId"], + "@condition": { "op": "gt", "left": { "field": "timeout" }, "right": { "value": "1 hour" } } + } + }, + { + "validator.atLeastOne": { + "name": "conditionIso", + "@fields": ["storeId", "kioskId"], + "@condition": { "op": "gte", "left": { "field": "timeout" }, "right": { "value": "P1DT2H" } } + } + }, + { + "validator.atLeastOne": { + "name": "conditionLiteralLeft", + "@fields": ["storeId", "kioskId"], + "@condition": { "op": "lt", "left": { "value": "PT" }, "right": { "field": "timeout" } } + } + }, + { + "validator.requiredWhen": { + "name": "requiredWhenNotIso", + "@field": "storeId", + "@when": "timeout", + "@equals": "90 minutes" + } + }, + { + "validator.presentIff": { + "name": "presentIffMonths", + "@field": "kioskId", + "@when": "timeout", + "@equals": "P1M" + } + }, + { + "validator.requiredWhen": { + "name": "requiredWhenTrailingNewline", + "@field": "storeId", + "@when": "timeout", + "@equals": "PT1H\n" + } + }, + { + "validator.requiredWhen": { + "name": "requiredWhenIso", + "@field": "storeId", + "@when": "timeout", + "@equals": "-PT1M30S" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-validator-condition-type/expected-errors.json b/fixtures/conformance/error-validator-condition-type/expected-errors.json new file mode 100644 index 0000000000..1d444732f3 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-type/expected-errors.json @@ -0,0 +1,55 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[7]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[8]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[9]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[10]['validator.atLeastOne']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[11]['validator.atLeastOne']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-condition-type/input/meta.checks.json b/fixtures/conformance/error-validator-condition-type/input/meta.checks.json new file mode 100644 index 0000000000..d900247e36 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-type/input/meta.checks.json @@ -0,0 +1,136 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Visit", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "field.boolean": { + "name": "active" + } + }, + { + "field.enum": { + "name": "kind", + "@values": [ + "Owned", + "Shared" + ] + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id" + } + }, + { + "validator.atLeastOne": { + "name": "notBoolean", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "field": "storeId" + } + } + }, + { + "validator.atLeastOne": { + "name": "booleanAsString", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "op": "eq", + "left": { + "field": "active" + }, + "right": { + "value": "true" + } + } + } + }, + { + "validator.atLeastOne": { + "name": "nullLiteral", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "op": "ne", + "left": { + "field": "storeId" + }, + "right": { + "value": null + } + } + } + }, + { + "validator.atLeastOne": { + "name": "notAMember", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "op": "eq", + "left": { + "field": "kind" + }, + "right": { + "value": "Nope" + } + } + } + }, + { + "validator.atLeastOne": { + "name": "numberAsString", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "op": "eq", + "left": { + "value": "5" + }, + "right": { + "field": "storeId" + } + } + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-validator-condition-unknown-field/expected-errors.json b/fixtures/conformance/error-validator-condition-unknown-field/expected-errors.json new file mode 100644 index 0000000000..a9b446bf90 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-unknown-field/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[5]['validator.atLeastOne']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-condition-unknown-field/input/meta.checks.json b/fixtures/conformance/error-validator-condition-unknown-field/input/meta.checks.json new file mode 100644 index 0000000000..d5c768ea92 --- /dev/null +++ b/fixtures/conformance/error-validator-condition-unknown-field/input/meta.checks.json @@ -0,0 +1,53 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Visit", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id" + } + }, + { + "validator.atLeastOne": { + "name": "oneSource", + "@fields": [ + "storeId", + "kioskId" + ], + "@condition": { + "op": "isNotNull", + "arg": { + "field": "nosuch" + } + } + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-validator-condition-unknown-fields/expected-errors.json b/fixtures/conformance/error-validator-condition-unknown-fields/expected-errors.json new file mode 100644 index 0000000000..d0e328844e --- /dev/null +++ b/fixtures/conformance/error-validator-condition-unknown-fields/expected-errors.json @@ -0,0 +1,35 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[7]['validator.requiredWhen']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[8]['validator.presentIff']" + } + }, + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[9]['validator.comparison']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-condition-unknown-fields/input/meta.checks.json b/fixtures/conformance/error-validator-condition-unknown-fields/input/meta.checks.json new file mode 100644 index 0000000000..b6e333729a --- /dev/null +++ b/fixtures/conformance/error-validator-condition-unknown-fields/input/meta.checks.json @@ -0,0 +1,105 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Visit", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "field.boolean": { + "name": "active" + } + }, + { + "field.enum": { + "name": "kind", + "@values": [ + "Owned", + "Shared" + ] + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id" + } + }, + { + "validator.requiredWhen": { + "name": "rw", + "@field": "storeId", + "@when": "kind", + "@equals": "Owned", + "@condition": { + "op": "and", + "args": [ + { + "op": "isNull", + "arg": { + "field": "nosuch" + } + }, + { + "op": "isNull", + "arg": { + "field": "other" + } + } + ] + } + } + }, + { + "validator.presentIff": { + "name": "pi", + "@field": "kioskId", + "@when": "kind", + "@equals": "Shared", + "@condition": { + "op": "isNotNull", + "arg": { + "field": "gone" + } + } + } + }, + { + "validator.comparison": { + "name": "cmp", + "@left": "storeId", + "@op": "lt", + "@right": "kioskId", + "@condition": { + "op": "isNotNull", + "arg": { + "field": "missing" + } + } + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-validator-exactly-one-one-field/expected-errors.json b/fixtures/conformance/error-validator-exactly-one-one-field/expected-errors.json new file mode 100644 index 0000000000..4fbd98187a --- /dev/null +++ b/fixtures/conformance/error-validator-exactly-one-one-field/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.checks.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[5]['validator.exactlyOne']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-validator-exactly-one-one-field/input/meta.checks.json b/fixtures/conformance/error-validator-exactly-one-one-field/input/meta.checks.json new file mode 100644 index 0000000000..eabd77def0 --- /dev/null +++ b/fixtures/conformance/error-validator-exactly-one-one-field/input/meta.checks.json @@ -0,0 +1,46 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Visit", + "children": [ + { + "source.rdb": {} + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id" + } + }, + { + "validator.exactlyOne": { + "name": "oneSource", + "@fields": [ + "storeId" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/field-duration-basic/expected.json b/fixtures/conformance/field-duration-basic/expected.json new file mode 100644 index 0000000000..f2873e60d0 --- /dev/null +++ b/fixtures/conformance/field-duration-basic/expected.json @@ -0,0 +1,50 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Timer", + "children": [ + { + "source.rdb": { + "@table": "timers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.duration": { + "name": "timeout", + "@required": true + } + }, + { + "field.duration": { + "name": "grace" + } + }, + { + "field.duration": { + "name": "backoff", + "isArray": true + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ], + "@generation": "increment" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/field-duration-basic/input/meta.timers.json b/fixtures/conformance/field-duration-basic/input/meta.timers.json new file mode 100644 index 0000000000..f2270b4812 --- /dev/null +++ b/fixtures/conformance/field-duration-basic/input/meta.timers.json @@ -0,0 +1,20 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Timer", + "children": [ + { "source.rdb": { "@table": "timers" } }, + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "timeout", "@required": true } }, + { "field.duration": { "name": "grace" } }, + { "field.duration": { "name": "backoff", "isArray": true } }, + { "identity.primary": { "name": "pk", "@fields": "id", "@generation": "increment" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json index e305a32cfd..79eeccf8be 100644 --- a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json +++ b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json @@ -17,10 +17,13 @@ { "field.boolean": { "name": "fBoolean", "@filterable": true } }, { "field.date": { "name": "fDate", "@filterable": true } }, { "field.time": { "name": "fTime", "@filterable": true } }, + { "field.duration": { "name": "fDuration", "@filterable": true } }, { "field.timestamp": { "name": "fTimestamp", "@filterable": true } }, { "field.enum": { "name": "fEnum", "@values": ["a", "b", "c"], "@filterable": true } }, { "field.enum": { "name": "fEnumInt", "@values": ["a", "b", "c"], "@intValueMap": { "a": 0, "b": 1, "c": 2 }, "@filterable": true } }, { "field.uuid": { "name": "fUuid", "@filterable": true } }, + { "field.uri": { "name": "fUri", "@filterable": true } }, + { "field.inet": { "name": "fInet", "@filterable": true } }, { "source.rdb": { "@table": "matrix" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } }, { "identity.secondary": { "name": "byString", "@fields": ["fString"] } }, diff --git a/fixtures/conformance/filter-ops-matrix/script.json b/fixtures/conformance/filter-ops-matrix/script.json index acd7fe8414..e3aa56bf4c 100644 --- a/fixtures/conformance/filter-ops-matrix/script.json +++ b/fixtures/conformance/filter-ops-matrix/script.json @@ -12,6 +12,12 @@ { "navigate": ["object:Matrix", "field:fUuid"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "in", "isNull"] } }, + { "navigate": ["object:Matrix", "field:fUri"], + "invoke": "field.filter-ops", + "expect": { "names": ["eq", "ne", "in", "like", "isNull"] } }, + { "navigate": ["object:Matrix", "field:fInet"], + "invoke": "field.filter-ops", + "expect": { "names": ["eq", "ne", "in", "isNull"] } }, { "navigate": ["object:Matrix", "field:fInt"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"] } }, @@ -36,6 +42,9 @@ { "navigate": ["object:Matrix", "field:fTime"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"] } }, + { "navigate": ["object:Matrix", "field:fDuration"], + "invoke": "field.filter-ops", + "expect": { "names": ["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"] } }, { "navigate": ["object:Matrix", "field:fTimestamp"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"] } }, diff --git a/fixtures/conformance/identity-primary-sequence/expected.json b/fixtures/conformance/identity-primary-sequence/expected.json new file mode 100644 index 0000000000..2819942a8a --- /dev/null +++ b/fixtures/conformance/identity-primary-sequence/expected.json @@ -0,0 +1,35 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { + "source.rdb": { + "@table": "tickets" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ], + "@generation": "increment", + "@sequenceName": "Ticket_Id_seq", + "@sequenceStart": 1000000000 + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/identity-primary-sequence/input/meta.seq.json b/fixtures/conformance/identity-primary-sequence/input/meta.seq.json new file mode 100644 index 0000000000..86a6973911 --- /dev/null +++ b/fixtures/conformance/identity-primary-sequence/input/meta.seq.json @@ -0,0 +1,25 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { "name": "id" } }, + { + "identity.primary": { + "name": "pk", + "@fields": "id", + "@generation": "increment", + "@sequenceStart": 1000000000, + "@sequenceName": "Ticket_Id_seq" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/validator-constraint-name-condition-exactly-one/expected.json b/fixtures/conformance/validator-constraint-name-condition-exactly-one/expected.json new file mode 100644 index 0000000000..11eb6910f8 --- /dev/null +++ b/fixtures/conformance/validator-constraint-name-condition-exactly-one/expected.json @@ -0,0 +1,151 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "StoreHours", + "children": [ + { + "source.rdb": { + "@table": "StoreHours" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "kind", + "@constraintName": "CK_StoreHours_Kind", + "@values": [ + "InStore", + "Phone" + ] + } + }, + { + "field.int": { + "name": "startDay", + "children": [ + { + "validator.numeric": { + "name": "weekday", + "@constraintName": "CK_StoreHours_StartDay", + "@max": 6, + "@min": 0 + } + } + ] + } + }, + { + "field.int": { + "name": "endDay" + } + }, + { + "field.time": { + "name": "startTime" + } + }, + { + "field.time": { + "name": "endTime" + } + }, + { + "field.string": { + "name": "code", + "children": [ + { + "validator.length": { + "name": "len", + "@constraintName": "CK_StoreHours_Code", + "@max": 8 + } + }, + { + "validator.regex": { + "name": "shape", + "@constraintName": "CK_StoreHours_CodeShape", + "@pattern": "^[A-Z0-9]+$" + } + } + ] + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ], + "@generation": "increment" + } + }, + { + "validator.comparison": { + "name": "sameDayTimes", + "@condition": { + "op": "eq", + "left": { + "field": "startDay" + }, + "right": { + "field": "endDay" + } + }, + "@constraintName": "CK_StoreHours_SameDayTimes", + "@left": "endTime", + "@op": "gt", + "@right": "startTime" + } + }, + { + "validator.exactlyOne": { + "name": "oneSource", + "@condition": { + "op": "eq", + "left": { + "field": "kind" + }, + "right": { + "value": "InStore" + } + }, + "@constraintName": "CK_StoreHours_ExactlyOneLocation", + "@fields": [ + "storeId", + "kioskId" + ] + } + }, + { + "validator.atLeastOne": { + "name": "anyTime", + "@constraintName": "CK_StoreHours_AnyTime", + "@fields": [ + "startTime", + "endTime" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/validator-constraint-name-condition-exactly-one/input/meta.checks.json b/fixtures/conformance/validator-constraint-name-condition-exactly-one/input/meta.checks.json new file mode 100644 index 0000000000..54722be071 --- /dev/null +++ b/fixtures/conformance/validator-constraint-name-condition-exactly-one/input/meta.checks.json @@ -0,0 +1,149 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "StoreHours", + "children": [ + { + "source.rdb": { + "@table": "StoreHours" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "kind", + "@values": [ + "InStore", + "Phone" + ], + "@constraintName": "CK_StoreHours_Kind" + } + }, + { + "field.int": { + "name": "startDay", + "children": [ + { + "validator.numeric": { + "name": "weekday", + "@min": 0, + "@max": 6, + "@constraintName": "CK_StoreHours_StartDay" + } + } + ] + } + }, + { + "field.int": { + "name": "endDay" + } + }, + { + "field.time": { + "name": "startTime" + } + }, + { + "field.time": { + "name": "endTime" + } + }, + { + "field.string": { + "name": "code", + "children": [ + { + "validator.length": { + "name": "len", + "@max": 8, + "@constraintName": "CK_StoreHours_Code" + } + }, + { + "validator.regex": { + "name": "shape", + "@pattern": "^[A-Z0-9]+$", + "@constraintName": "CK_StoreHours_CodeShape" + } + } + ] + } + }, + { + "field.long": { + "name": "storeId" + } + }, + { + "field.long": { + "name": "kioskId" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": "id", + "@generation": "increment" + } + }, + { + "validator.comparison": { + "name": "sameDayTimes", + "@left": "endTime", + "@op": "gt", + "@right": "startTime", + "@constraintName": "CK_StoreHours_SameDayTimes", + "@condition": { + "op": "eq", + "left": { + "field": "startDay" + }, + "right": { + "field": "endDay" + } + } + } + }, + { + "validator.exactlyOne": { + "name": "oneSource", + "@fields": [ + "storeId", + "kioskId" + ], + "@constraintName": "CK_StoreHours_ExactlyOneLocation", + "@condition": { + "op": "eq", + "left": { + "field": "kind" + }, + "right": { + "value": "InStore" + } + } + } + }, + { + "validator.atLeastOne": { + "name": "anyTime", + "@fields": [ + "startTime", + "endTime" + ], + "@constraintName": "CK_StoreHours_AnyTime" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/case.json b/fixtures/contract-conformance/all-fields-wire-string-emit/case.json index d65a4fc249..a46445a727 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/case.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/case.json @@ -1 +1 @@ -{ "meta": "fixtures/contract-conformance/all-fields/meta.json", "wire": { "decimals": "string", "nullMembers": "emit" }, "strictExclude": ["Bag", "Tree"] } +{ "meta": "fixtures/contract-conformance/all-fields/meta.json", "wire": { "decimals": "string", "nullMembers": "emit", "durations": "iso" }, "strictExclude": ["Bag", "Tree"] } diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/openapi.json b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/openapi.json index ee79a8a7ec..abb6483b2d 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/openapi.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/openapi.json @@ -857,6 +857,85 @@ "type": "boolean" } }, + { + "name": "filter[holdFor]", + "in": "query", + "description": "Sugar for the eq operator.", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][eq]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][ne]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][in]", + "in": "query", + "schema": { + "type": "string", + "description": "Comma-separated values." + } + }, + { + "name": "filter[holdFor][isNull]", + "in": "query", + "schema": { + "type": "boolean" + } + }, { "name": "sort", "in": "query", @@ -904,7 +983,10 @@ "status:desc", "stage", "stage:asc", - "stage:desc" + "stage:desc", + "holdFor", + "holdFor:asc", + "holdFor:desc" ] } }, @@ -1716,6 +1798,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { @@ -1800,6 +1890,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", @@ -1913,6 +2004,11 @@ } ] }, + "holdFor": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "$ref": "#/components/schemas/AddressCreate" }, @@ -2116,6 +2212,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.create.schema.json b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.create.schema.json index 9617555a35..6b274c2876 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.create.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.create.schema.json @@ -103,6 +103,11 @@ } ] }, + "holdFor": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "$ref": "./Address.create.schema.json" }, diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.schema.json b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.schema.json index e9b3852a45..2336826976 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.schema.json @@ -151,6 +151,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { @@ -235,6 +243,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.strict.schema.json b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.strict.schema.json index 61e121e4bf..3f403bcb48 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.strict.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.strict.schema.json @@ -129,6 +129,13 @@ "null" ] }, + "holdFor": { + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months). Pattern: ^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$.", + "type": [ + "string", + "null" + ] + }, "shipTo": { "description": "A postal address.", "type": [ @@ -235,6 +242,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.update.schema.json b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.update.schema.json index 5e867a0dcf..9b3af60d63 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.update.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-emit/expected/schemas/acme/contract/AllFields.update.schema.json @@ -150,6 +150,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/case.json b/fixtures/contract-conformance/all-fields-wire-string-omit/case.json index d68f000263..af019c30f0 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/case.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/case.json @@ -1 +1 @@ -{ "meta": "fixtures/contract-conformance/all-fields/meta.json", "wire": { "decimals": "string", "nullMembers": "omit" }, "strictExclude": ["Bag", "Tree"] } +{ "meta": "fixtures/contract-conformance/all-fields/meta.json", "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" }, "strictExclude": ["Bag", "Tree"] } diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/openapi.json b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/openapi.json index 0f3e79d1c5..0df2287e3d 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/openapi.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/openapi.json @@ -857,6 +857,85 @@ "type": "boolean" } }, + { + "name": "filter[holdFor]", + "in": "query", + "description": "Sugar for the eq operator.", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][eq]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][ne]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][in]", + "in": "query", + "schema": { + "type": "string", + "description": "Comma-separated values." + } + }, + { + "name": "filter[holdFor][isNull]", + "in": "query", + "schema": { + "type": "boolean" + } + }, { "name": "sort", "in": "query", @@ -904,7 +983,10 @@ "status:desc", "stage", "stage:asc", - "stage:desc" + "stage:desc", + "holdFor", + "holdFor:asc", + "holdFor:desc" ] } }, @@ -1708,6 +1790,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { @@ -1792,6 +1882,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", @@ -1905,6 +1996,11 @@ } ] }, + "holdFor": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "$ref": "#/components/schemas/AddressCreate" }, @@ -2108,6 +2204,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.create.schema.json b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.create.schema.json index 9617555a35..6b274c2876 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.create.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.create.schema.json @@ -103,6 +103,11 @@ } ] }, + "holdFor": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "$ref": "./Address.create.schema.json" }, diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.schema.json b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.schema.json index e9b3852a45..2336826976 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.schema.json @@ -151,6 +151,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { @@ -235,6 +243,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.strict.schema.json b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.strict.schema.json index 61e121e4bf..3f403bcb48 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.strict.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.strict.schema.json @@ -129,6 +129,13 @@ "null" ] }, + "holdFor": { + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months). Pattern: ^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$.", + "type": [ + "string", + "null" + ] + }, "shipTo": { "description": "A postal address.", "type": [ @@ -235,6 +242,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.update.schema.json b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.update.schema.json index 5e867a0dcf..9b3af60d63 100644 --- a/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.update.schema.json +++ b/fixtures/contract-conformance/all-fields-wire-string-omit/expected/schemas/acme/contract/AllFields.update.schema.json @@ -150,6 +150,14 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields/expected/openapi.json b/fixtures/contract-conformance/all-fields/expected/openapi.json index 975be7c0ab..5a0735ad62 100644 --- a/fixtures/contract-conformance/all-fields/expected/openapi.json +++ b/fixtures/contract-conformance/all-fields/expected/openapi.json @@ -857,6 +857,85 @@ "type": "boolean" } }, + { + "name": "filter[holdFor]", + "in": "query", + "description": "Sugar for the eq operator.", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][eq]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][ne]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][gte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lt]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][lte]", + "in": "query", + "schema": { + "type": "string", + "pattern": "^[-+]?P(([0-9]+W([0-9]+D)?|[0-9]+D)(T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))?|T([0-9]+H([0-9]+M)?([0-9]+(\\.[0-9]+)?S)?|[0-9]+M([0-9]+(\\.[0-9]+)?S)?|[0-9]+(\\.[0-9]+)?S))$", + "description": "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months)." + } + }, + { + "name": "filter[holdFor][in]", + "in": "query", + "schema": { + "type": "string", + "description": "Comma-separated values." + } + }, + { + "name": "filter[holdFor][isNull]", + "in": "query", + "schema": { + "type": "boolean" + } + }, { "name": "sort", "in": "query", @@ -904,7 +983,10 @@ "status:desc", "stage", "stage:asc", - "stage:desc" + "stage:desc", + "holdFor", + "holdFor:asc", + "holdFor:desc" ] } }, @@ -1716,6 +1798,13 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "anyOf": [ { @@ -1800,6 +1889,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", @@ -1913,6 +2003,10 @@ } ] }, + "holdFor": { + "type": "string", + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "$ref": "#/components/schemas/AddressCreate" }, @@ -2116,6 +2210,13 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.create.schema.json b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.create.schema.json index 9617555a35..254d4b06ac 100644 --- a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.create.schema.json +++ b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.create.schema.json @@ -103,6 +103,10 @@ } ] }, + "holdFor": { + "type": "string", + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "$ref": "./Address.create.schema.json" }, diff --git a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.schema.json b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.schema.json index 3b7f7d4c4b..c15b124832 100644 --- a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.schema.json +++ b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.schema.json @@ -152,6 +152,13 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "anyOf": [ { @@ -236,6 +243,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.strict.schema.json b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.strict.schema.json index 61e121e4bf..b7d000c544 100644 --- a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.strict.schema.json +++ b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.strict.schema.json @@ -129,6 +129,13 @@ "null" ] }, + "holdFor": { + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages.", + "type": [ + "string", + "null" + ] + }, "shipTo": { "description": "A postal address.", "type": [ @@ -235,6 +242,7 @@ "homepage", "looseLink", "address", + "holdFor", "shipTo", "stops", "tags", diff --git a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.update.schema.json b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.update.schema.json index 5e867a0dcf..9ff9fd5721 100644 --- a/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.update.schema.json +++ b/fixtures/contract-conformance/all-fields/expected/schemas/acme/contract/AllFields.update.schema.json @@ -150,6 +150,13 @@ } ] }, + "holdFor": { + "type": [ + "string", + "null" + ], + "description": "A span of time, in the serving language's own form. Set wire.durations to \"iso\" to fix ISO-8601 across server languages." + }, "shipTo": { "anyOf": [ { diff --git a/fixtures/contract-conformance/all-fields/meta.json b/fixtures/contract-conformance/all-fields/meta.json index 19cb18136e..467b76126c 100644 --- a/fixtures/contract-conformance/all-fields/meta.json +++ b/fixtures/contract-conformance/all-fields/meta.json @@ -48,6 +48,7 @@ { "field.uri": { "name": "homepage" } }, { "field.uri": { "name": "looseLink", "@lenient": true } }, { "field.inet": { "name": "address" } }, + { "field.duration": { "name": "holdFor", "@filterable": true } }, { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, { "field.object": { "name": "stops", "@objectRef": "Address", "isArray": true, "@storage": "jsonb", "children": [ { "validator.array": { "name": "stopsCount", "@min": 1, "@max": 5 } } diff --git a/fixtures/contract-conformance/blog/case.json b/fixtures/contract-conformance/blog/case.json index ee85dd0638..34c8c69c5e 100644 --- a/fixtures/contract-conformance/blog/case.json +++ b/fixtures/contract-conformance/blog/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/decimal/case.json b/fixtures/contract-conformance/decimal/case.json index 099ba18278..36f114890d 100644 --- a/fixtures/contract-conformance/decimal/case.json +++ b/fixtures/contract-conformance/decimal/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/decimal/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/decimal/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/jsonb/case.json b/fixtures/contract-conformance/jsonb/case.json index 410b0c92a6..b5a484e78d 100644 --- a/fixtures/contract-conformance/jsonb/case.json +++ b/fixtures/contract-conformance/jsonb/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/jsonb/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"}, "strictExclude": ["Document", "Pricing"] } +{ "meta": "fixtures/api-contract-conformance/jsonb/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"}, "strictExclude": ["Document", "Pricing"] } diff --git a/fixtures/contract-conformance/m2m/case.json b/fixtures/contract-conformance/m2m/case.json index 9c456f6100..d8bf82969c 100644 --- a/fixtures/contract-conformance/m2m/case.json +++ b/fixtures/contract-conformance/m2m/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/m2m/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/m2m/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/projection/case.json b/fixtures/contract-conformance/projection/case.json index 7df31120b3..f80698c768 100644 --- a/fixtures/contract-conformance/projection/case.json +++ b/fixtures/contract-conformance/projection/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/projection/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/projection/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/report/case.json b/fixtures/contract-conformance/report/case.json index ebebb502a7..9f0e1526b1 100644 --- a/fixtures/contract-conformance/report/case.json +++ b/fixtures/contract-conformance/report/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/report/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"}, "strictExclude": ["Ticket", "TicketsByTags"], "reportExport": true } +{ "meta": "fixtures/api-contract-conformance/report/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"}, "strictExclude": ["Ticket", "TicketsByTags"], "reportExport": true } diff --git a/fixtures/contract-conformance/secret-fields/case.json b/fixtures/contract-conformance/secret-fields/case.json index 51ec47303f..c347962dda 100644 --- a/fixtures/contract-conformance/secret-fields/case.json +++ b/fixtures/contract-conformance/secret-fields/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/secret-fields/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/secret-fields/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/tph/case.json b/fixtures/contract-conformance/tph/case.json index 986490c531..9e6bcbac62 100644 --- a/fixtures/contract-conformance/tph/case.json +++ b/fixtures/contract-conformance/tph/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/tph/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/tph/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/contract-conformance/write-through/case.json b/fixtures/contract-conformance/write-through/case.json index 1b7e6bd8cb..b381bc8226 100644 --- a/fixtures/contract-conformance/write-through/case.json +++ b/fixtures/contract-conformance/write-through/case.json @@ -1 +1 @@ -{ "meta": "fixtures/api-contract-conformance/write-through/meta.json", "wire": {"decimals": "string", "nullMembers": "omit"} } +{ "meta": "fixtures/api-contract-conformance/write-through/meta.json", "wire": {"decimals": "string", "nullMembers": "omit", "durations": "iso"} } diff --git a/fixtures/duration-wire/README.md b/fixtures/duration-wire/README.md new file mode 100644 index 0000000000..ded2676da3 --- /dev/null +++ b/fixtures/duration-wire/README.md @@ -0,0 +1,44 @@ +# field.duration wire vectors + +The wire rule for `field.duration` (a fixed-length span of time), and the codecs some ports need +to store one. [`vectors.json`](./vectors.json) has two parts. + +## `wire`: every port passes every vector + +The ISO-8601 text a caller sends and a port answers under `"wire": { "durations": "iso" }`. Clients depend on it, so every port agrees in that mode. Under the default `native` mode each port writes its own standard form (see [the wire setting](../../docs/features/api-contract.md#the-wire-setting)). + +- **`parse`**: a duration a caller may send, and the canonical form a port answers for it. The + canonical form is `[-]PT[H][M][[.]S]`: hours are never folded into days, zero + components are left out, zero is `PT0S`, trailing zeros of the fraction are dropped, and a + negative duration has one leading `-`. A day (`D`) is exactly 24 hours and a week (`W`) exactly + 7 days. +- **`precision`**: the fraction may have any number of digits. A port keeps as many as its own + duration type holds and truncates the rest toward zero, never rounding. `out` is keyed by that + number of digits, and a port checks the entry for its own: 6 for TypeScript and Python + (microseconds), 7 for C# (a `TimeSpan` tick is 100 ns), 9 for Java and Kotlin (`java.time.Duration`). +- **`refused`**: input that is not a duration. Years and months are refused because a calendar + span has no fixed length. So are lowercase designators, fractional hours, minutes or weeks, a + comma decimal separator, per-component signs, out-of-order components and whitespace. + +There is no shared upper bound. Each port is limited only by its own type (a .NET `TimeSpan`, a +`timedelta`) and each database by its own column; a port refuses what it cannot hold rather than +wrapping it, and tests that limit in its own suite. + +## `codecs`: a port passes a group only when it has that codec + +| Group | What it pins | Passed by | +|---|---|---| +| `postgres` | the text a Postgres `interval` column returns under the default `IntervalStyle` (`postgres`), and its canonical form | a port that reads an interval as text, or checks a native value (Npgsql `TimeSpan`, a JDBC `PGInterval`) against the same rule | +| `postgresRefused` | an interval with a year or month part; reading one is an error, never a guess at the length of a month | the same ports | +| `postgresText` | a canonical duration and its Postgres spelling, `[-]HH:MM:SS[.f]` with unbounded hours and no day part | a port that binds a duration to Postgres as text, and `meta migrate` for a declared `@default`. Postgres refuses a signed ISO-8601 duration (`-PT1M30S` is "invalid input syntax for type interval"), and returns an interval written this way in the same spelling, so a default never drifts | +| `storedMicros` | a duration and the integer count of microseconds a SQLite, D1 or MySQL column stores for it (`BIGINT`), sub-microsecond digits truncated toward zero | a port that stores durations on one of those databases | + +An integer column orders and compares as the span does, so sorting, range filters and equality are +right on every database. This is how Django stores a `DurationField` outside Postgres. The 64-bit +column holds about 2.56 billion hours; a port refuses to bind a longer span there. Postgres keeps its +native `interval`, which holds microseconds. + +The Postgres text readers assume the server's default `IntervalStyle`, `postgres`. A session set to +`iso_8601`, `sql_standard` or `postgres_verbose` returns another spelling. A reader accepts it only +where it means the same span (an unsigned `iso_8601` value is plain ISO-8601) and refuses the rest, +such as `iso_8601`'s per-component signs (`PT-1M-30S`), rather than misreading it. diff --git a/fixtures/duration-wire/vectors.json b/fixtures/duration-wire/vectors.json new file mode 100644 index 0000000000..bd284d12e0 --- /dev/null +++ b/fixtures/duration-wire/vectors.json @@ -0,0 +1,105 @@ +{ + "$comment": "The field.duration wire rule and the codecs some ports need. Every port passes `wire`; a port passes a `codecs` group only when it has that codec. See README.md.", + "wire": { + "parse": [ + { "in": "PT1H30M", "out": "PT1H30M" }, + { "in": "PT90M", "out": "PT1H30M" }, + { "in": "PT5400S", "out": "PT1H30M" }, + { "in": "P1D", "out": "PT24H" }, + { "in": "P1DT2H", "out": "PT26H" }, + { "in": "P10DT0.5S", "out": "PT240H0.5S" }, + { "in": "PT1.5S", "out": "PT1.5S" }, + { "in": "PT0.000001S", "out": "PT0.000001S" }, + { "in": "PT1.100S", "out": "PT1.1S" }, + { "in": "PT1H5S", "out": "PT1H5S" }, + { "in": "PT0S", "out": "PT0S" }, + { "in": "P0D", "out": "PT0S" }, + { "in": "-PT0S", "out": "PT0S" }, + { "in": "+PT2M", "out": "PT2M" }, + { "in": "-PT90S", "out": "-PT1M30S" }, + { "in": "-P1DT1H", "out": "-PT25H" }, + { "in": "PT876000H", "out": "PT876000H" }, + { "in": "P1W", "out": "PT168H" }, + { "in": "P2W", "out": "PT336H" }, + { "in": "P1W1DT1H", "out": "PT193H" }, + { "in": "-P1W", "out": "-PT168H" } + ], + "precision": [ + { "in": "PT1.123456789S", "out": { "6": "PT1.123456S", "7": "PT1.1234567S", "9": "PT1.123456789S" } }, + { "in": "PT0.0000001S", "out": { "6": "PT0S", "7": "PT0.0000001S", "9": "PT0.0000001S" } }, + { "in": "PT0.000000001S", "out": { "6": "PT0S", "7": "PT0S", "9": "PT0.000000001S" } }, + { "in": "PT1.1234567890123S", "out": { "6": "PT1.123456S", "7": "PT1.1234567S", "9": "PT1.123456789S" } }, + { "in": "PT0.99999999999999S", "out": { "6": "PT0.999999S", "7": "PT0.9999999S", "9": "PT0.999999999S" } }, + { "in": "-PT1.9999999999S", "out": { "6": "-PT1.999999S", "7": "-PT1.9999999S", "9": "-PT1.999999999S" } }, + { "in": "PT1.10000000S", "out": { "6": "PT1.1S", "7": "PT1.1S", "9": "PT1.1S" } } + ], + "refused": [ + "", + "P", + "PT", + "P1DT", + "P1Y", + "P1M", + "P1Y2M3D", + "pt1h", + "PT1.5H", + "PT1,5S", + "PT1.S", + "PT.5S", + "1:30:00", + "PT1H 30M", + " PT1H", + "PT-1H", + "--PT1H", + "PT1M1H", + "P1D1W", + "P1.5W", + "PT1W" + ] + }, + "codecs": { + "postgres": [ + { "in": "01:30:00", "out": "PT1H30M" }, + { "in": "00:00:00", "out": "PT0S" }, + { "in": "00:00:01.5", "out": "PT1.5S" }, + { "in": "00:00:00.000001", "out": "PT0.000001S" }, + { "in": "-00:01:30", "out": "-PT1M30S" }, + { "in": "26:00:00", "out": "PT26H" }, + { "in": "1 day", "out": "PT24H" }, + { "in": "1 day 02:00:00", "out": "PT26H" }, + { "in": "3 days", "out": "PT72H" }, + { "in": "-1 days", "out": "-PT24H" }, + { "in": "-1 days +02:00:00", "out": "-PT22H" }, + { "in": "1 day -01:00:00", "out": "PT23H" }, + { "in": "876000:00:00", "out": "PT876000H" } + ], + "postgresRefused": [ + "1 year", + "2 mons", + "1 year 2 mons 3 days 04:05:06", + "1 mon 00:00:01", + "abc" + ], + "postgresText": [ + { "in": "PT1H30M", "out": "01:30:00" }, + { "in": "PT0S", "out": "00:00:00" }, + { "in": "PT1.5S", "out": "00:00:01.5" }, + { "in": "PT0.000001S", "out": "00:00:00.000001" }, + { "in": "-PT1M30S", "out": "-00:01:30" }, + { "in": "PT26H", "out": "26:00:00" }, + { "in": "-PT26H0.5S", "out": "-26:00:00.5" }, + { "in": "PT876000H", "out": "876000:00:00" } + ], + "storedMicros": [ + { "in": "PT0S", "out": "0" }, + { "in": "PT0.000001S", "out": "1" }, + { "in": "PT1.5S", "out": "1500000" }, + { "in": "PT1H30M", "out": "5400000000" }, + { "in": "-PT1M30S", "out": "-90000000" }, + { "in": "PT26H", "out": "93600000000" }, + { "in": "PT876000H", "out": "3153600000000000" }, + { "in": "PT1.123456789S", "out": "1123456" }, + { "in": "-PT1.0000015S", "out": "-1000001" } + ] + } +} diff --git a/fixtures/enum-length-conformance/README.md b/fixtures/enum-length-conformance/README.md new file mode 100644 index 0000000000..593a0b2862 --- /dev/null +++ b/fixtures/enum-length-conformance/README.md @@ -0,0 +1,30 @@ +# Enum-length conformance + +A string-stored `field.enum` is a `VARCHAR(n)` column when it has a length: its own `@maxLength`, +else the project's default, `"enums": { "maxLength": n }` in `.metaobjects/config.json`. With +neither it stays an unbounded text column, as before. An `@intValueMap` enum is an integer +column and takes no length. Every member must fit: the loader refuses an `@maxLength` a member +is longer than (`ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH`, pinned in `fixtures/conformance/`), and the +core resolver refuses a member longer than the project default, so every consumer that sizes +the column refuses the same model. The `@maxLength` refusals are metamodel rules every port's +loader applies; the corpus here is for a port that sizes the column. + +[`cases.json`](./cases.json) holds the `enums` blocks a port must accept, with the length +each enum in [`meta.enums.json`](./meta.enums.json) resolves to under them, the overflow a +port must refuse, and the blocks a port must refuse when it reads the file. A whole +number is accepted however JSON spells it (`32`, `32.0`, `1e2`), because a JSON reader that +parses numbers as doubles cannot tell those apart. Each port that sizes the column runs it +through its own config reader and core resolver: + +| Port | Test | +|---|---| +| TypeScript | `server/typescript/packages/sdk/test/enum-length-conformance.test.ts` | +| C# | `server/csharp/MetaObjects.Codegen.Tests/EnumLengthConformanceTests.cs` | +| Java / Kotlin | `server/java/metadata/src/test/java/com/metaobjects/database/EnumLengthConformanceTest.java` | + +Python does not run it: no Python generator or run-time path emits a column type or DDL, so +nothing there has a column to size. Its config reader still accepts an `enums` block (a project +shared with another port has one) and refuses a malformed one, and reads nothing from it. + +The column each port's bindings emit is described in +[`docs/features/field-types.md`](../../docs/features/field-types.md). diff --git a/fixtures/enum-length-conformance/cases.json b/fixtures/enum-length-conformance/cases.json new file mode 100644 index 0000000000..203a457ce4 --- /dev/null +++ b/fixtures/enum-length-conformance/cases.json @@ -0,0 +1,45 @@ +{ + "_comment": "The column length of a string-stored field.enum, per port. Each case is the `enums` block of a project's .metaobjects/config.json (null: no block). Each port reads it with its own config reader, puts it on the loaded meta.enums.json root, and resolves every Ticket enum's length through its core resolver (resolveEnumMaxLength and its per-port equivalents). null in `lengths` means no length: an unbounded text column, or for an array enum a text[] column, whose elements carry no length (so an array enum's members are never checked against the default). `overflow` cases must be refused when the named field's length is resolved, with ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH. Each `invalid` block must be refused when the config is read.", + "cases": [ + { + "name": "no enums block: only an explicit @maxLength sizes a column", + "enums": null, + "lengths": { "status": null, "channel": 20, "priority": 8, "rank": null, "tags": null } + }, + { + "name": "an empty block is the same as none", + "enums": {}, + "lengths": { "status": null, "channel": 20, "priority": 8, "rank": null, "tags": null } + }, + { + "name": "the project default sizes every enum without its own @maxLength", + "enums": { "maxLength": 32 }, + "lengths": { "status": 32, "channel": 20, "priority": 8, "rank": null, "tags": null } + }, + { + "name": "an integer written with a decimal point is that integer", + "enums": { "maxLength": 32.0 }, + "lengths": { "status": 32, "channel": 20, "priority": 8, "rank": null, "tags": null } + }, + { + "name": "an integer written with an exponent is that integer", + "enums": { "maxLength": 1e2 }, + "lengths": { "status": 100, "channel": 20, "priority": 8, "rank": null, "tags": null } + } + ], + "overflow": [ + { + "name": "a member longer than the project default is refused", + "enums": { "maxLength": 10 }, + "field": "status", + "member": "IN_PROGRESS" + } + ], + "invalid": [ + { "name": "maxLength is a positive integer, not zero", "enums": { "maxLength": 0 } }, + { "name": "maxLength is an integer", "enums": { "maxLength": 12.5 } }, + { "name": "maxLength is a number, not a string", "enums": { "maxLength": "32" } }, + { "name": "an unknown key is refused", "enums": { "length": 32 } }, + { "name": "enums is an object", "enums": 32 } + ] +} diff --git a/fixtures/enum-length-conformance/meta.enums.json b/fixtures/enum-length-conformance/meta.enums.json new file mode 100644 index 0000000000..6df2d37984 --- /dev/null +++ b/fixtures/enum-length-conformance/meta.enums.json @@ -0,0 +1,30 @@ +{ + "metadata.root": { + "package": "acme::enums", + "children": [ + { + "field.enum": { + "name": "Priority", + "abstract": true, + "@values": ["LOW", "HIGH"], + "@maxLength": 8 + } + }, + { + "object.entity": { + "name": "Ticket", + "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["OPEN", "IN_PROGRESS", "CLOSED"] } }, + { "field.enum": { "name": "channel", "@values": ["EMAIL", "PHONE"], "@maxLength": 20 } }, + { "field.enum": { "name": "priority", "extends": "acme::enums::Priority" } }, + { "field.enum": { "name": "rank", "@values": ["A", "B"], "@intValueMap": { "A": 1, "B": 2 } } }, + { "field.enum": { "name": "tags", "isArray": true, "@values": ["URGENT", "FOLLOW_UP_NEEDED"] } }, + { "identity.primary": { "name": "pk", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index 46ff01d311..1d5bb0f58f 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -34,6 +34,7 @@ children and the cardinality of that subtype. Documentation attributes | `field.date` | Calendar date (no time-of-day). Binds to the native date/temporal type; DB column is DATE. | [types/field.md#fielddate](types/field.md#fielddate) | | `field.decimal` | Precision-exact decimal (use @precision/@scale). Native TS binding is string (lossless); DB column is NUMERIC(p,s); the wire form is a string. Classified DATA_TYPE_STRING so an exact decimal is never silently rounded through a double. | [types/field.md#fielddecimal](types/field.md#fielddecimal) | | `field.double` | Double-precision (64-bit) IEEE-754 floating point. Binds to the native double/number type; DB column is DOUBLE PRECISION. Not for money — use field.currency or field.decimal. | [types/field.md#fielddouble](types/field.md#fielddouble) | +| `field.duration` | A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1. | [types/field.md#fieldduration](types/field.md#fieldduration) | | `field.enum` | String-backed enumeration constrained to a closed set of member symbols (@values). Each member is its own stored string with no name/value divergence. | [types/field.md#fieldenum](types/field.md#fieldenum) | | `field.float` | Single-precision floating point. Binds to the native double/number type (TS has no distinct float); DB column is REAL. Not for money. | [types/field.md#fieldfloat](types/field.md#fieldfloat) | | `field.inet` | An IP-address string (IPv4 or IPv6) (ADR-0036/0037 — a concept with a native type + behavior, so a subtype not a @stringFormat). Binds to TS string (TS has no native IP type, same as field.uuid); other ports bind to their native IP type (Java java.net.InetAddress, C# System.Net.IPAddress, Python ipaddress, Kotlin java.net.InetAddress). DB column is the Postgres-native inet type. Codegen emits an IPv4-or-IPv6 literal validator (a Zod regex union in TS — z.string().ip() was removed in Zod 4 — native IP-type parsing in the other ports); add @lenient to bind a plain string with no validator. v1 ships WITHOUT @kind — the native type subsumes ipv4/ipv6; @kind is reserved for a later ipv4\|ipv6 value-constraint. | [types/field.md#fieldinet](types/field.md#fieldinet) | @@ -82,6 +83,7 @@ children and the cardinality of that subtype. Documentation attributes | `validator.atLeastOne` | Cardinality of presence: at least one of the named fields (@fields) must be present (NOT NULL). Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). | [types/validator.md#validatoratleastone](types/validator.md#validatoratleastone) | | `validator.base` | Abstract base validator — the shared root subtype concrete validators specialize. Carries the @min/@max bounds attrs but enforces no rule of its own. Not authored directly: a `validator.base` node fails to load (ERR_ABSTRACT_SUBTYPE_AUTHORED) — this subtype is a registry anchor concrete subtypes inherit from, never a node in a document. | [types/validator.md#validatorbase](types/validator.md#validatorbase) | | `validator.comparison` | Cross-field ordering: requires two sibling fields of the owning entity stand in a relational order (@left @op @right), e.g. current_hp <= max_hp or expires_at > created_at. Entity-scoped; references fields by name. Backends derive the rule (CHECK constraint, cross-field assertion) — no raw expression is stored. | [types/validator.md#validatorcomparison](types/validator.md#validatorcomparison) | +| `validator.exactlyOne` | Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1. | [types/validator.md#validatorexactlyone](types/validator.md#validatorexactlyone) | | `validator.length` | Bounds string length / collection size via @min/@max. | [types/validator.md#validatorlength](types/validator.md#validatorlength) | | `validator.numeric` | Bounds a numeric value's magnitude via @min/@max. | [types/validator.md#validatornumeric](types/validator.md#validatornumeric) | | `validator.presentIff` | Biconditional presence: the target field (@field) is present (NOT NULL) if and only if the gating field (@when) equals @equals. Models paired flag/companion-column invariants, e.g. used_at present iff is_used=true. Entity-scoped; references fields by name. | [types/validator.md#validatorpresentiff](types/validator.md#validatorpresentiff) | diff --git a/fixtures/metamodel-docs/expected/providers.md b/fixtures/metamodel-docs/expected/providers.md index ee43c8bddc..3e7f4ec5a2 100644 --- a/fixtures/metamodel-docs/expected/providers.md +++ b/fixtures/metamodel-docs/expected/providers.md @@ -13,14 +13,14 @@ provider owns. This is the ownership lens over the same vocabulary Core metaobjects metamodel types and subtypes. -**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.intMap`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.report`, `object.value`, `origin.aggregate`, `origin.base`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` +**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.intMap`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.duration`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.report`, `object.value`, `origin.aggregate`, `origin.base`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.exactlyOne`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` **Contributes attributes:** - `field.base`: `@default`, `@mutability`, `@required`, `@unique` - `field.currency`: `@currency` - `field.decimal`: `@precision`, `@scale` -- `field.enum`: `@intValueMap`, `@provided`, `@values` +- `field.enum`: `@intValueMap`, `@maxLength`, `@provided`, `@values` - `field.inet`: `@lenient` - `field.map`: `@objectRef`, `@valueType` - `field.object`: `@objectRef` @@ -44,14 +44,15 @@ Core metaobjects metamodel types and subtypes. - `template.prompt`: `@format`, `@maxChars`, `@maxTokens`, `@model`, `@owner`, `@payloadRef`, `@promptStyle`, `@requiredSlots`, `@requiredTags`, `@responseFormat`, `@responseRef`, `@since`, `@textRef` - `template.toolcall`: `@maxTokens`, `@owner`, `@payloadRef`, `@since`, `@toolName` - `validator.array`: `@max`, `@min` -- `validator.atLeastOne`: `@fields` +- `validator.atLeastOne`: `@condition`, `@fields` - `validator.base`: `@max`, `@min` -- `validator.comparison`: `@left`, `@op`, `@right` +- `validator.comparison`: `@condition`, `@left`, `@op`, `@right` +- `validator.exactlyOne`: `@condition`, `@fields` - `validator.length`: `@max`, `@min` - `validator.numeric`: `@max`, `@min` -- `validator.presentIff`: `@equals`, `@field`, `@when` +- `validator.presentIff`: `@condition`, `@equals`, `@field`, `@when` - `validator.regex`: `@max`, `@min`, `@pattern` -- `validator.requiredWhen`: `@equals`, `@field`, `@when` +- `validator.requiredWhen`: `@condition`, `@equals`, `@field`, `@when` ## metaobjects-db @@ -65,22 +66,32 @@ DB-domain attributes — @column / @db.indexed / @dbColumnType on every field, @ - `field.date`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType` - `field.decimal`: `@column`, `@db.indexed`, `@dbColumnType` - `field.double`: `@column`, `@db.indexed`, `@dbColumnType` -- `field.enum`: `@column`, `@db.indexed`, `@dbColumnType` +- `field.duration`: `@column`, `@db.indexed`, `@dbColumnType` +- `field.enum`: `@column`, `@constraintName`, `@db.indexed`, `@dbColumnType` - `field.float`: `@column`, `@db.indexed`, `@dbColumnType` - `field.inet`: `@column`, `@db.indexed`, `@dbColumnType` - `field.int`: `@column`, `@db.indexed`, `@dbColumnType` - `field.long`: `@column`, `@db.indexed`, `@dbColumnType` - `field.map`: `@column`, `@db.indexed`, `@dbColumnType` -- `field.object`: `@column`, `@db.indexed`, `@dbColumnType`, `@storage` +- `field.object`: `@column`, `@db.indexed`, `@dbColumnType`, `@flattenedColumnPrefix`, `@storage` - `field.string`: `@column`, `@db.indexed`, `@dbColumnType` - `field.time`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType` - `field.timestamp`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType`, `@localTime` - `field.uri`: `@column`, `@db.indexed`, `@dbColumnType` - `field.uuid`: `@column`, `@db.indexed`, `@dbColumnType` +- `identity.primary`: `@sequenceName`, `@sequenceStart` - `identity.reference`: `@constraintName`, `@onDelete`, `@onUpdate` - `identity.secondary`: `@expr`, `@orders`, `@using`, `@where` - `index.lookup`: `@expr`, `@orders`, `@using`, `@where` - `source.rdb`: `@function`, `@kind`, `@materializedView`, `@parameterRef`, `@proc`, `@role`, `@schema`, `@sql`, `@table`, `@unmanaged`, `@view` +- `validator.atLeastOne`: `@constraintName` +- `validator.comparison`: `@constraintName` +- `validator.exactlyOne`: `@constraintName` +- `validator.length`: `@constraintName` +- `validator.numeric`: `@constraintName` +- `validator.presentIff`: `@constraintName` +- `validator.regex`: `@constraintName` +- `validator.requiredWhen`: `@constraintName` ## metaobjects-documentation @@ -100,6 +111,7 @@ Prompt / AI + serialization domain — @xmlText / @example / @instruction field - `field.date`: `@example`, `@instruction`, `@xmlText` - `field.decimal`: `@example`, `@instruction`, `@xmlText` - `field.double`: `@example`, `@instruction`, `@xmlText` +- `field.duration`: `@example`, `@instruction`, `@xmlText` - `field.enum`: `@coerceDefault`, `@enumAlias`, `@enumDoc`, `@example`, `@instruction`, `@normalize`, `@xmlText` - `field.float`: `@example`, `@instruction`, `@xmlText` - `field.inet`: `@example`, `@instruction`, `@xmlText` @@ -126,6 +138,7 @@ UI/query-surface domain — @filterable / @sortable / @sortableDefaultOrder fiel - `field.date`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` - `field.decimal`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` - `field.double`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` +- `field.duration`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` - `field.enum`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` - `field.float`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` - `field.inet`: `@filterable`, `@formExclude`, `@sortable`, `@sortableDefaultOrder` diff --git a/fixtures/metamodel-docs/expected/types/field.md b/fixtures/metamodel-docs/expected/types/field.md index f087e964f3..1d03426759 100644 --- a/fixtures/metamodel-docs/expected/types/field.md +++ b/fixtures/metamodel-docs/expected/types/field.md @@ -214,6 +214,39 @@ Double-precision (64-bit) IEEE-754 floating point. Binds to the native double/nu - `validator.*` — 0..* - `view.*` — 0..* +### field.duration + +A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1. + +**Owning provider:** metaobjects-core-types + +**When to use:** A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span ("every 3 months") is not a duration: model it as an int and a unit. + +**Attributes** + +| Attribute | Type | Required | Default | Allowed values | Provider | Description | +| --- | --- | --- | --- | --- | --- | --- | +| `@column` | string | no | | | metaobjects-db | Physical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the "naming" setting in .metaobjects/config.json). | +| `@db.indexed` | boolean | no | | | metaobjects-db | When true, suppress the @filterable-without-index Loader warning (the field is indexed by other means). | +| `@dbColumnType` | string | no | | `uuid`, `jsonb` | metaobjects-db | Physical DB column-type override (ADR-0013 escape hatch). Legal values are uuid \| jsonb, both on field.string (uuid = native Postgres uuid column over a string-typed field; jsonb = genuinely-open JSON column). The logical field type and its native binding are unchanged. Native SQL arrays (uuid[]/text[]) are NOT declared here — they are derived from a field subtype + isArray (ADR-0036 Wave 1). The retired timestamp_with_tz value is gone — timezone-awareness lives in field.timestamp (instant by default) + @localTime (the naive opt-out), per ADR-0036 Wave 2. | +| `@default` | any | no | | | metaobjects-core-types | Default value applied to the column when no value is supplied. Its type follows the field's own subtype (string / boolean / number / ...). Converted at consumption time via MetaField.defaultValue(). | +| `@example` | string | no | | | metaobjects-prompt | FR-010: an example value for this field, shown in the generated output-format prompt fragment. | +| `@filterable` | boolean | no | | | metaobjects-ui | When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer). | +| `@formExclude` | boolean | no | | | metaobjects-ui | When true, the field is omitted from generated forms, and the generated create, PATCH and PUT routes ignore a value sent for it (reads of it are unchanged); inert on fields for which no form is generated (e.g. projection/derived fields). | +| `@instruction` | string | no | | | metaobjects-prompt | FR-010: a short instruction for this field, shown in the generated output-format prompt fragment. | +| `@mutability` | string | no | | `readWrite`, `writeOnce`, `readOnly` | metaobjects-core-types | FR-037 R1: who may write this field, and when. 'readWrite' (the default when absent) — the caller may set it on create and change it on update. 'writeOnce' — the caller sets it on create; it is excluded from the update shape thereafter, so a value presented on PATCH is ignored rather than rejected. 'readOnly' — nobody writes it: codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas omit it from input variants; the value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. The three are mutually exclusive modes of ONE axis — who may write, and when — so the illegal pair is unrepresentable and inheritance has a total order: a subtype may TIGHTEN an inherited mode (readWrite < writeOnce < readOnly) and never loosen it (ERR_MUTABILITY_DOWNGRADE). Pairing a non-readWrite mode with @autoSet is ERR_MUTABILITY_AUTOSET_CONFLICT: @autoSet already says the SERVER supplies the value, which is a different axis from who may write it. | +| `@required` | boolean | no | | | metaobjects-core-types | When true, the field is NOT NULL. Equivalent to attaching a validator.required child. On a non-array string, generated wire-tier input validation (create/patch) additionally rejects the empty string by default — whitespace is accepted — unless an explicit validator.length @min: 0 opts back to presence-only. In-process read models never enforce this at construction. | +| `@sortable` | boolean | no | | | metaobjects-ui | When true, the field is exposed in generated CRUD sort allowlists. Inherits from @filterable by default; set false to opt out. | +| `@sortableDefaultOrder` | string | no | | `asc`, `desc` | metaobjects-ui | Direction a sort takes when it names this field but omits the order: a ?sort= with no :order, and a layout.dataGrid whose @defaultSortField names it with no @defaultSortOrder. Defaults to 'asc'. | +| `@unique` | boolean | no | | | metaobjects-core-types | When true, the field gets a column-level UNIQUE constraint. | +| `@xmlText` | boolean | no | | | metaobjects-prompt | When true, this field receives its element's XML TEXT CONTENT during tolerant extract (JAXB @XmlValue / Jackson @JacksonXmlText / .NET [XmlText]) instead of a same-named child. No effect when the eliciting prompt's @responseFormat is json. | + +**Allowed children** + +- `origin.*` — 0..* +- `validator.*` — 0..* +- `view.*` — 0..* + ### field.enum String-backed enumeration constrained to a closed set of member symbols (@values). Each member is its own stored string with no name/value divergence. @@ -230,6 +263,7 @@ String-backed enumeration constrained to a closed set of member symbols (@values | --- | --- | --- | --- | --- | --- | --- | | `@coerceDefault` | string | no | | | metaobjects-prompt | Fallback enum member used by tolerant extract when a present value cannot be coerced; must be one of the field's @values. | | `@column` | string | no | | | metaobjects-db | Physical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the "naming" setting in .metaobjects/config.json). | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@db.indexed` | boolean | no | | | metaobjects-db | When true, suppress the @filterable-without-index Loader warning (the field is indexed by other means). | | `@dbColumnType` | string | no | | `uuid`, `jsonb` | metaobjects-db | Physical DB column-type override (ADR-0013 escape hatch). Legal values are uuid \| jsonb, both on field.string (uuid = native Postgres uuid column over a string-typed field; jsonb = genuinely-open JSON column). The logical field type and its native binding are unchanged. Native SQL arrays (uuid[]/text[]) are NOT declared here — they are derived from a field subtype + isArray (ADR-0036 Wave 1). The retired timestamp_with_tz value is gone — timezone-awareness lives in field.timestamp (instant by default) + @localTime (the naive opt-out), per ADR-0036 Wave 2. | | `@default` | any | no | | | metaobjects-core-types | Default value applied to the column when no value is supplied. Its type follows the field's own subtype (string / boolean / number / ...). Converted at consumption time via MetaField.defaultValue(). | @@ -240,6 +274,7 @@ String-backed enumeration constrained to a closed set of member symbols (@values | `@formExclude` | boolean | no | | | metaobjects-ui | When true, the field is omitted from generated forms, and the generated create, PATCH and PUT routes ignore a value sent for it (reads of it are unchanged); inert on fields for which no form is generated (e.g. projection/derived fields). | | `@instruction` | string | no | | | metaobjects-prompt | FR-010: a short instruction for this field, shown in the generated output-format prompt fragment. | | `@intValueMap` | intMap | no | | | metaobjects-core-types | Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language. | +| `@maxLength` | int | no | | | metaobjects-core-types | Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1. | | `@mutability` | string | no | | `readWrite`, `writeOnce`, `readOnly` | metaobjects-core-types | FR-037 R1: who may write this field, and when. 'readWrite' (the default when absent) — the caller may set it on create and change it on update. 'writeOnce' — the caller sets it on create; it is excluded from the update shape thereafter, so a value presented on PATCH is ignored rather than rejected. 'readOnly' — nobody writes it: codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas omit it from input variants; the value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. The three are mutually exclusive modes of ONE axis — who may write, and when — so the illegal pair is unrepresentable and inheritance has a total order: a subtype may TIGHTEN an inherited mode (readWrite < writeOnce < readOnly) and never loosen it (ERR_MUTABILITY_DOWNGRADE). Pairing a non-readWrite mode with @autoSet is ERR_MUTABILITY_AUTOSET_CONFLICT: @autoSet already says the SERVER supplies the value, which is a different axis from who may write it. | | `@normalize` | string | no | `strip` | `none`, `collapse`, `strip` | metaobjects-prompt | ASCII normalization mode for tolerant enum extract (none\|collapse\|strip, default strip). On field.enum it is per-field; on object.value it is the default for the object's enum fields. | | `@provided` | boolean | no | | | metaobjects-core-types | FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration. | @@ -444,6 +479,7 @@ A nested structured value (set @objectRef to the target object). Storage is gove | `@default` | any | no | | | metaobjects-core-types | Default value applied to the column when no value is supplied. Its type follows the field's own subtype (string / boolean / number / ...). Converted at consumption time via MetaField.defaultValue(). | | `@example` | string | no | | | metaobjects-prompt | FR-010: an example value for this field, shown in the generated output-format prompt fragment. | | `@filterable` | boolean | no | | | metaobjects-ui | When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer). | +| `@flattenedColumnPrefix` | string | no | | | metaobjects-db | On a field.object with @storage "flattened": the text put before each member's column, verbatim ("" allowed, so the members keep their own column names). Absent: this field's resolved column name and "_" (with @column "bill", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1. | | `@formExclude` | boolean | no | | | metaobjects-ui | When true, the field is omitted from generated forms, and the generated create, PATCH and PUT routes ignore a value sent for it (reads of it are unchanged); inert on fields for which no form is generated (e.g. projection/derived fields). | | `@instruction` | string | no | | | metaobjects-prompt | FR-010: a short instruction for this field, shown in the generated output-format prompt fragment. | | `@mutability` | string | no | | `readWrite`, `writeOnce`, `readOnly` | metaobjects-core-types | FR-037 R1: who may write this field, and when. 'readWrite' (the default when absent) — the caller may set it on create and change it on update. 'writeOnce' — the caller sets it on create; it is excluded from the update shape thereafter, so a value presented on PATCH is ignored rather than rejected. 'readOnly' — nobody writes it: codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas omit it from input variants; the value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. The three are mutually exclusive modes of ONE axis — who may write, and when — so the illegal pair is unrepresentable and inheritance has a total order: a subtype may TIGHTEN an inherited mode (readWrite < writeOnce < readOnly) and never loosen it (ERR_MUTABILITY_DOWNGRADE). Pairing a non-readWrite mode with @autoSet is ERR_MUTABILITY_AUTOSET_CONFLICT: @autoSet already says the SERVER supplies the value, which is a different axis from who may write it. | diff --git a/fixtures/metamodel-docs/expected/types/identity.md b/fixtures/metamodel-docs/expected/types/identity.md index 74456e20fb..35dbe622dd 100644 --- a/fixtures/metamodel-docs/expected/types/identity.md +++ b/fixtures/metamodel-docs/expected/types/identity.md @@ -24,6 +24,8 @@ The primary key — one per entity; @fields names its column(s), @generation the | --- | --- | --- | --- | --- | --- | --- | | `@fields` | string[] | yes | | | metaobjects-core-types | The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite. | | `@generation` | string | no | | `increment`, `uuid`, `assigned` | metaobjects-core-types | Primary-key value generation strategy: 'increment' (auto-increment), 'uuid', or 'assigned' (caller-supplied). | +| `@sequenceName` | string | no | | | metaobjects-db | The physical name of an @generation "increment" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | +| `@sequenceStart` | long | no | | | metaobjects-db | The first value an @generation "increment" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | **Allowed children** diff --git a/fixtures/metamodel-docs/expected/types/validator.md b/fixtures/metamodel-docs/expected/types/validator.md index 22dfddab1c..5857bd44da 100644 --- a/fixtures/metamodel-docs/expected/types/validator.md +++ b/fixtures/metamodel-docs/expected/types/validator.md @@ -43,6 +43,8 @@ Cardinality of presence: at least one of the named fields (@fields) must be pres | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@condition` | expression | no | | | metaobjects-core-types | The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1. | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@fields` | string[] | yes | | | metaobjects-core-types | Names of the candidate fields; at least one must be present. | **Allowed children** @@ -80,6 +82,8 @@ Cross-field ordering: requires two sibling fields of the owning entity stand in | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@condition` | expression | no | | | metaobjects-core-types | The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1. | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@left` | string | yes | | | metaobjects-core-types | Name of the left-hand field of the owning entity. | | `@op` | string | yes | | `gt`, `gte`, `lt`, `lte`, `ne`, `eq` | metaobjects-core-types | Relational operator: gt (>), gte (>=), lt (<), lte (<=), ne (<>), eq (=). | | `@right` | string | yes | | | metaobjects-core-types | Name of the right-hand field of the owning entity. | @@ -88,6 +92,28 @@ Cross-field ordering: requires two sibling fields of the owning entity stand in _No structural children._ +### validator.exactlyOne + +Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1. + +**Owning provider:** metaobjects-core-types + +**Rules:** @fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null. + +**When to use:** A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks. + +**Attributes** + +| Attribute | Type | Required | Default | Allowed values | Provider | Description | +| --- | --- | --- | --- | --- | --- | --- | +| `@condition` | expression | no | | | metaobjects-core-types | The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1. | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | +| `@fields` | string[] | yes | | | metaobjects-core-types | Names of the alternative fields; exactly one must be present. | + +**Allowed children** + +_No structural children._ + ### validator.length Bounds string length / collection size via @min/@max. @@ -100,6 +126,7 @@ Bounds string length / collection size via @min/@max. | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@max` | int | no | | | metaobjects-core-types | Maximum allowed value (length, numeric value, or array element count depending on the validator subtype). | | `@min` | int | no | | | metaobjects-core-types | Minimum allowed value (length, numeric value, or array element count depending on the validator subtype). | @@ -119,6 +146,7 @@ Bounds a numeric value's magnitude via @min/@max. | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@max` | int | no | | | metaobjects-core-types | Maximum allowed value (length, numeric value, or array element count depending on the validator subtype). | | `@min` | int | no | | | metaobjects-core-types | Minimum allowed value (length, numeric value, or array element count depending on the validator subtype). | @@ -140,6 +168,8 @@ Biconditional presence: the target field (@field) is present (NOT NULL) if and o | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@condition` | expression | no | | | metaobjects-core-types | The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1. | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@equals` | string | yes | | | metaobjects-core-types | The gating value; @field is present exactly when @when equals this. | | `@field` | string | yes | | | metaobjects-core-types | Name of the field whose presence is governed by the condition. | | `@when` | string | yes | | | metaobjects-core-types | Name of the gating field. | @@ -160,6 +190,7 @@ Requires the value match a regular expression (@pattern). | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@max` | int | no | | | metaobjects-core-types | Maximum allowed value (length, numeric value, or array element count depending on the validator subtype). | | `@min` | int | no | | | metaobjects-core-types | Minimum allowed value (length, numeric value, or array element count depending on the validator subtype). | | `@pattern` | string | no | | | metaobjects-core-types | Regular expression the value must match. | @@ -198,6 +229,8 @@ One-directional conditional presence: when the gating field (@when) equals @equa | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@condition` | expression | no | | | metaobjects-core-types | The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1. | +| `@constraintName` | string | no | | | metaobjects-db | The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. | | `@equals` | string | yes | | | metaobjects-core-types | The gating value; when @when equals this, @field must be present. | | `@field` | string | yes | | | metaobjects-core-types | Name of the field that becomes required when the condition holds. | | `@when` | string | yes | | | metaobjects-core-types | Name of the gating field whose value triggers the requirement. | diff --git a/fixtures/naming-conformance/physical-names/cases.json b/fixtures/naming-conformance/physical-names/cases.json index fbc0c77920..4a8444d1d5 100644 --- a/fixtures/naming-conformance/physical-names/cases.json +++ b/fixtures/naming-conformance/physical-names/cases.json @@ -1,61 +1,90 @@ { - "_comment": "Cross-port physical naming (ADR-0058). Each case is the `naming` block of a project's .metaobjects/config.json. A port parses it with the reader it uses for that file, loads meta.naming.json, and resolves every table and column below through the resolver its codegen, migrate (TypeScript), runtime and verify use. `tables` keys are object names; `columns` keys are `.`. Overrides (@table, a source's structural name, @column) win under every rule; the entity-name fallback keeps the frozen legacy pluralizer under every rule. `invalid` blocks must be refused when the config is read.", + "_comment": "Cross-port physical naming (ADR-0058). Each case is the `naming` block of a project's .metaobjects/config.json. A port parses it with the reader it uses for that file, loads meta.naming.json, and resolves every table and column below through the resolver its codegen, migrate (TypeScript), runtime and verify use. `tables` keys are object names; `columns` keys are `.`. Overrides (@table, a source's structural name, @column) win under every rule; the entity-name fallback keeps the frozen legacy pluralizer under every rule. `invalid` blocks must be refused when the config is read. `upperFirst` upper-cases the field name's first letter and changes nothing else (`requestURL` → `RequestURL`). A key `..` names a member of a `@storage: flattened` value object: its column is the flattened prefix (`@flattenedColumnPrefix` on the embedding field, verbatim, `\"\"` allowed; else that field's resolved column and `_`) followed by the member's resolved column; a prefix inherited through `extends` counts.", "cases": [ { "name": "default", "naming": null, - "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests" }, + "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests", "Shipment": "shipments" }, "columns": { "OrderItem.id": "id", "OrderItem.createdAt": "created_at", "OrderItem.unitPriceCents": "unit_price_cents", "OrderItem.sku": "SKU_CODE", "Person.firstName": "first_name", "Person.lastName": "family_name", - "AuditEntry.eventType": "event_type", "Category.displayName": "display_name", "HTTPRequest.requestURL": "request_url" + "AuditEntry.eventType": "event_type", + "Category.displayName": "display_name", + "HTTPRequest.requestURL": "request_url", + "Shipment.id": "id", "Shipment.shipTo.street": "ship_to_street", "Shipment.shipTo.postalCode": "ship_to_postal_code", "Shipment.shipTo.country": "ship_to_CTRY", "Shipment.billTo.postalCode": "bill_postal_code", "Shipment.returnTo.postalCode": "postal_code", "Shipment.giftTo.postalCode": "g_postal_code", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_postal_code" } }, { "name": "empty block is the default", "naming": {}, - "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests" }, + "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests", "Shipment": "shipments" }, "columns": { "OrderItem.id": "id", "OrderItem.createdAt": "created_at", "OrderItem.unitPriceCents": "unit_price_cents", "OrderItem.sku": "SKU_CODE", "Person.firstName": "first_name", "Person.lastName": "family_name", - "AuditEntry.eventType": "event_type", "Category.displayName": "display_name", "HTTPRequest.requestURL": "request_url" + "AuditEntry.eventType": "event_type", + "Category.displayName": "display_name", + "HTTPRequest.requestURL": "request_url", + "Shipment.id": "id", "Shipment.shipTo.street": "ship_to_street", "Shipment.shipTo.postalCode": "ship_to_postal_code", "Shipment.shipTo.country": "ship_to_CTRY", "Shipment.billTo.postalCode": "bill_postal_code", "Shipment.returnTo.postalCode": "postal_code", "Shipment.giftTo.postalCode": "g_postal_code", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_postal_code" } }, { "name": "literal columns", "naming": { "columns": "literal" }, - "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests" }, + "tables": { "OrderItem": "order_items", "Person": "people", "AuditEntry": "audit_log", "Category": "categories", "HTTPRequest": "http_requests", "Shipment": "shipments" }, "columns": { "OrderItem.id": "id", "OrderItem.createdAt": "createdAt", "OrderItem.unitPriceCents": "unitPriceCents", "OrderItem.sku": "SKU_CODE", "Person.firstName": "firstName", "Person.lastName": "family_name", - "AuditEntry.eventType": "eventType", "Category.displayName": "displayName", "HTTPRequest.requestURL": "requestURL" + "AuditEntry.eventType": "eventType", + "Category.displayName": "displayName", + "HTTPRequest.requestURL": "requestURL", + "Shipment.id": "id", "Shipment.shipTo.street": "shipTo_street", "Shipment.shipTo.postalCode": "shipTo_postalCode", "Shipment.shipTo.country": "shipTo_CTRY", "Shipment.billTo.postalCode": "bill_postalCode", "Shipment.returnTo.postalCode": "postalCode", "Shipment.giftTo.postalCode": "g_postalCode", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_postalCode" } }, { "name": "literal tables", "naming": { "tables": "literal" }, - "tables": { "OrderItem": "OrderItems", "Person": "people", "AuditEntry": "AuditLog", "Category": "Categories", "HTTPRequest": "HTTPRequests" }, + "tables": { "OrderItem": "OrderItems", "Person": "people", "AuditEntry": "AuditLog", "Category": "Categories", "HTTPRequest": "HTTPRequests", "Shipment": "Shipments" }, "columns": { "OrderItem.id": "id", "OrderItem.createdAt": "created_at", "OrderItem.unitPriceCents": "unit_price_cents", "OrderItem.sku": "SKU_CODE", "Person.firstName": "first_name", "Person.lastName": "family_name", - "AuditEntry.eventType": "event_type", "Category.displayName": "display_name", "HTTPRequest.requestURL": "request_url" + "AuditEntry.eventType": "event_type", + "Category.displayName": "display_name", + "HTTPRequest.requestURL": "request_url", + "Shipment.id": "id", "Shipment.shipTo.street": "ship_to_street", "Shipment.shipTo.postalCode": "ship_to_postal_code", "Shipment.shipTo.country": "ship_to_CTRY", "Shipment.billTo.postalCode": "bill_postal_code", "Shipment.returnTo.postalCode": "postal_code", "Shipment.giftTo.postalCode": "g_postal_code", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_postal_code" } }, { "name": "kebab-case columns, literal tables", "naming": { "columns": "kebab-case", "tables": "literal" }, - "tables": { "OrderItem": "OrderItems", "Person": "people", "AuditEntry": "AuditLog", "Category": "Categories", "HTTPRequest": "HTTPRequests" }, + "tables": { "OrderItem": "OrderItems", "Person": "people", "AuditEntry": "AuditLog", "Category": "Categories", "HTTPRequest": "HTTPRequests", "Shipment": "Shipments" }, "columns": { "OrderItem.id": "id", "OrderItem.createdAt": "created-at", "OrderItem.unitPriceCents": "unit-price-cents", "OrderItem.sku": "SKU_CODE", "Person.firstName": "first-name", "Person.lastName": "family_name", - "AuditEntry.eventType": "event-type", "Category.displayName": "display-name", "HTTPRequest.requestURL": "request-url" + "AuditEntry.eventType": "event-type", + "Category.displayName": "display-name", + "HTTPRequest.requestURL": "request-url", + "Shipment.id": "id", "Shipment.shipTo.street": "ship-to_street", "Shipment.shipTo.postalCode": "ship-to_postal-code", "Shipment.shipTo.country": "ship-to_CTRY", "Shipment.billTo.postalCode": "bill_postal-code", "Shipment.returnTo.postalCode": "postal-code", "Shipment.giftTo.postalCode": "g_postal-code", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_postal-code" + } + }, + { + "name": "upperFirst columns, literal tables", + "naming": { "columns": "upperFirst", "tables": "literal" }, + "tables": { "OrderItem": "OrderItems", "Person": "people", "AuditEntry": "AuditLog", "Category": "Categories", "HTTPRequest": "HTTPRequests", "Shipment": "Shipments" }, + "columns": { + "OrderItem.id": "Id", "OrderItem.createdAt": "CreatedAt", "OrderItem.unitPriceCents": "UnitPriceCents", "OrderItem.sku": "SKU_CODE", + "Person.firstName": "FirstName", "Person.lastName": "family_name", + "AuditEntry.eventType": "EventType", + "Category.displayName": "DisplayName", + "HTTPRequest.requestURL": "RequestURL", + "Shipment.id": "Id", "Shipment.shipTo.street": "ShipTo_Street", "Shipment.shipTo.postalCode": "ShipTo_PostalCode", "Shipment.shipTo.country": "ShipTo_CTRY", "Shipment.billTo.postalCode": "bill_PostalCode", "Shipment.returnTo.postalCode": "PostalCode", "Shipment.giftTo.postalCode": "g_PostalCode", "Shipment.giftTo.country": "g_CTRY", "Shipment.homeTo.postalCode": "addr_PostalCode" } } ], "invalid": [ { "name": "unknown column rule", "naming": { "columns": "camelCase" } }, { "name": "kebab-case is not a table rule", "naming": { "tables": "kebab-case" } }, + { "name": "upperFirst is not a table rule", "naming": { "tables": "upperFirst" } }, { "name": "unknown key", "naming": { "column": "literal" } }, - { "name": "not an object", "naming": "snake_case" } + { "name": "not an object", "naming": "snake_case"} ] } diff --git a/fixtures/naming-conformance/physical-names/meta.naming.json b/fixtures/naming-conformance/physical-names/meta.naming.json index 45402809c7..f3e394ca94 100644 --- a/fixtures/naming-conformance/physical-names/meta.naming.json +++ b/fixtures/naming-conformance/physical-names/meta.naming.json @@ -59,6 +59,32 @@ { "identity.primary": { "name": "id", "@fields": "id" } } ] } + }, + { "field.object": { "name": "AddrCols", "abstract": true, "@objectRef": "acme::naming::Address", "@flattenedColumnPrefix": "addr_" } }, + { + "object.value": { + "name": "Address", + "children": [ + { "field.string": { "name": "street" } }, + { "field.string": { "name": "postalCode" } }, + { "field.string": { "name": "country", "@column": "CTRY" } } + ] + } + }, + { + "object.entity": { + "name": "Shipment", + "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "field.object": { "name": "billTo", "@objectRef": "Address", "@storage": "flattened", "@column": "bill" } }, + { "field.object": { "name": "returnTo", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "" } }, + { "field.object": { "name": "giftTo", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "g_" } }, + { "field.object": { "name": "homeTo", "extends": "acme::naming::AddrCols", "@storage": "flattened" } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } } ] } diff --git a/fixtures/persistence-conformance/.metaobjects/config.json b/fixtures/persistence-conformance/.metaobjects/config.json index d6d2b1359d..3a5a786b8c 100644 --- a/fixtures/persistence-conformance/.metaobjects/config.json +++ b/fixtures/persistence-conformance/.metaobjects/config.json @@ -1,5 +1,5 @@ { "schema_version": 1, "naming": { "columns": "literal" }, - "wire": { "decimals": "string", "nullMembers": "omit" } + "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" } } diff --git a/fixtures/persistence-conformance/canonical/meta.fitness.json b/fixtures/persistence-conformance/canonical/meta.fitness.json index 12dbeb55ab..cd2aa0a35c 100644 --- a/fixtures/persistence-conformance/canonical/meta.fitness.json +++ b/fixtures/persistence-conformance/canonical/meta.fitness.json @@ -347,6 +347,7 @@ { "field.uri": { "name": "uriVal", "@required": true } }, { "field.inet": { "name": "inetVal", "@required": true } }, { "field.inet": { "name": "inet6Val", "@required": true } }, + { "field.duration": { "name": "durationVal" } }, { "field.object": { "name": "settings", "@objectRef": "Settings", "@storage": "jsonb" } }, { "field.object": { "name": "labels", "@objectRef": "Label", "@storage": "jsonb", "isArray": true } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "uuid" } }, diff --git a/fixtures/persistence-conformance/canonical/schema.mysql.sql b/fixtures/persistence-conformance/canonical/schema.mysql.sql index 8d2a314303..3f2037f6fc 100644 --- a/fixtures/persistence-conformance/canonical/schema.mysql.sql +++ b/fixtures/persistence-conformance/canonical/schema.mysql.sql @@ -102,6 +102,7 @@ CREATE TABLE `all_types` ( `uriVal` VARCHAR(255) NOT NULL, `inetVal` VARCHAR(45) NOT NULL, `inet6Val` VARCHAR(45) NOT NULL, + `durationVal` BIGINT, `settings` JSON, `labels` JSON ); diff --git a/fixtures/persistence-conformance/canonical/schema.postgres.sql b/fixtures/persistence-conformance/canonical/schema.postgres.sql index 3631a660ad..a5a9bd9317 100644 --- a/fixtures/persistence-conformance/canonical/schema.postgres.sql +++ b/fixtures/persistence-conformance/canonical/schema.postgres.sql @@ -132,6 +132,7 @@ CREATE TABLE "all_types" ( "uriVal" TEXT NOT NULL, "inetVal" INET NOT NULL, "inet6Val" INET NOT NULL, + "durationVal" INTERVAL, "settings" JSONB, "labels" JSONB, CONSTRAINT "all_types_pkey" PRIMARY KEY ("id"), diff --git a/fixtures/persistence-conformance/canonical/schema.sqlite.sql b/fixtures/persistence-conformance/canonical/schema.sqlite.sql index c7cc934302..c61ed55174 100644 --- a/fixtures/persistence-conformance/canonical/schema.sqlite.sql +++ b/fixtures/persistence-conformance/canonical/schema.sqlite.sql @@ -133,6 +133,7 @@ CREATE TABLE "all_types" ( "uriVal" TEXT NOT NULL, "inetVal" TEXT NOT NULL, "inet6Val" TEXT NOT NULL, + "durationVal" INTEGER, "settings" TEXT, "labels" TEXT, CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')), diff --git a/fixtures/persistence-conformance/normalization.md b/fixtures/persistence-conformance/normalization.md index ee9fadf4ee..e467925fb3 100644 --- a/fixtures/persistence-conformance/normalization.md +++ b/fixtures/persistence-conformance/normalization.md @@ -38,6 +38,7 @@ byte equality check. | `TIMESTAMP` | string `"YYYY-MM-DDTHH:MM:SS[.fff]"` | `"2026-05-25T10:30:00"` (no Z) | | `TIMESTAMPTZ` | string `"YYYY-MM-DDTHH:MM:SS[.fff]Z"` | `"2026-05-25T14:30:00Z"` (UTC always) | | `UUID` | string (lowercase canonical) | `"550e8400-e29b-41d4-a716-446655440000"` | +| `INTERVAL` (`field.duration`) | string (canonical ISO-8601 duration, [`fixtures/duration-wire`](../duration-wire/)) | `"PT24H0.5S"` | | `JSON`, `JSONB` | re-serialized with **sorted keys** | `{"a": 1, "b": 2}` not `{"b": 2, "a": 1}` | | `BYTEA` | base64 string | `"aGVsbG8="` | | `NULL` | JSON `null` | | diff --git a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml index 93ff84047d..31ccf6024d 100644 --- a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml +++ b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml @@ -15,7 +15,7 @@ description: | string / int / long / double / float / decimal / boolean / date / time / timestamp / timestamp(tz) / currency / enum / uuid / - uri / inet (IPv4 + IPv6) / object(@objectRef, jsonb storage) + uri / inet (IPv4 + IPv6) / duration / object(@objectRef, jsonb storage) Tricky wire cases asserted here: - decimal: exact, canonical (no trailing zeros) → "123.456". @@ -26,6 +26,11 @@ description: | precision loss — a port routing these through a JS number / 32-bit int / double corrupts the low bits and fails. - uuid: written upper-case, reads back lowercase-canonical. + - duration: written in an accepted ISO-8601 form, read back canonical + (fixtures/duration-wire): a day folds into hours (`P1DT0.5S` -> `PT24H0.5S`) + and a negative span carries one leading sign (`-PT90S` -> `-PT1M30S`). + Postgres refuses a signed ISO duration as interval input, so this also + proves each port binds the value in a form Postgres reads. - id (uuid PK): NOT supplied on insert; gen_random_uuid() fills it, the runtime returns it, and the read-back is by that generated PK — so a server-generated PK is exercised on the write path too. @@ -77,6 +82,7 @@ queries: uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" inet6Val: "2001:db8::1" + durationVal: "P1DT0.5S" settings: { theme: "dark", notify: true, retries: 3 } # array-of-VO stored as JSONB (field.object @isArray @storage:jsonb). Element # order is preserved; each element's keys are sorted on read (like settings). @@ -102,6 +108,7 @@ queries: uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" inet6Val: "2001:db8::1" + durationVal: "PT24H0.5S" settings: { notify: true, retries: 3, theme: "dark" } labels: - { key: "alpha", weight: 10 } @@ -129,6 +136,7 @@ queries: uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" inet6Val: "fe80::1" + durationVal: "-PT90S" settings: { theme: "light", notify: false, retries: 0 } # empty array-of-VO: an empty JSON array must round-trip as [] (distinct from null). labels: [] @@ -151,6 +159,7 @@ queries: uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" inet6Val: "fe80::1" + durationVal: "-PT1M30S" settings: { notify: false, retries: 0, theme: "light" } labels: [] @@ -190,6 +199,7 @@ queries: # Authored in the long (uncompressed) form; Postgres returns the canonical # compressed form, so the expected read-back is `2001:db8::8a2e:370:7334`. inet6Val: "2001:0db8:0000:0000:0000:8a2e:0370:7334" + durationVal: "PT876000H" settings: { theme: "dark", notify: true, retries: 9 } labels: - { key: "solo", weight: 99 } @@ -212,6 +222,7 @@ queries: uriVal: "https://example.org/path#frag" inetVal: "255.255.255.255" inet6Val: "2001:db8::8a2e:370:7334" + durationVal: "PT876000H" settings: { notify: true, retries: 9, theme: "dark" } labels: - { key: "solo", weight: 99 } diff --git a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml index 35edbcfe3b..7eff2f9059 100644 --- a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml +++ b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml @@ -80,6 +80,7 @@ queries: uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" inet6Val: "2001:db8::dead:beef" + durationVal: "PT2700S" settings: { theme: "dark", notify: true, retries: 7 } expect: id: "22222222-2222-4222-8222-222222222222" @@ -101,6 +102,7 @@ queries: uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" inet6Val: "2001:db8::dead:beef" + durationVal: "PT45M" settings: { notify: true, retries: 7, theme: "dark" } # labels (array-of-VO jsonb) is neither seeded nor updated here → reads back null. labels: null @@ -130,6 +132,7 @@ queries: uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" inet6Val: "2001:db8::dead:beef" + durationVal: "PT45M" settings: { notify: true, retries: 7, theme: "dark" } # labels (array-of-VO jsonb) is neither seeded nor updated here → reads back null. labels: null diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 631dfcee54..898c776a48 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -1,6 +1,6 @@ { - "registeredSubTypeCount": 65, - "exercisedSubTypeCount": 53, + "registeredSubTypeCount": 66, + "exercisedSubTypeCount": 58, "untestedSubTypes": [ "attr.boolean", "attr.class", @@ -9,11 +9,7 @@ "attr.filter", "attr.int", "attr.intMap", - "attr.long", - "validator.atLeastOne", - "validator.comparison", - "validator.presentIff", - "validator.requiredWhen" + "attr.long" ], "untestedAttrsByExercisedSubType": [ { @@ -73,7 +69,6 @@ "formExclude", "instruction", "mutability", - "sortable", "sortableDefaultOrder", "unique", "xmlText" @@ -95,6 +90,23 @@ "xmlText" ] }, + { + "key": "field.duration", + "untestedAttrs": [ + "column", + "db.indexed", + "dbColumnType", + "default", + "example", + "formExclude", + "instruction", + "mutability", + "sortable", + "sortableDefaultOrder", + "unique", + "xmlText" + ] + }, { "key": "field.enum", "untestedAttrs": [ @@ -121,7 +133,6 @@ "formExclude", "instruction", "mutability", - "sortable", "sortableDefaultOrder", "unique", "xmlText" @@ -148,7 +159,6 @@ { "key": "field.int", "untestedAttrs": [ - "column", "dbColumnType", "example", "formExclude", @@ -188,7 +198,6 @@ "sortable", "sortableDefaultOrder", "unique", - "valueType", "xmlText" ] }, @@ -211,7 +220,6 @@ { "key": "field.string", "untestedAttrs": [ - "formExclude", "sortableDefaultOrder", "xmlText" ] @@ -324,12 +332,6 @@ "symmetric" ] }, - { - "key": "source.rdb", - "untestedAttrs": [ - "materializedView" - ] - }, { "key": "template.output", "untestedAttrs": [ @@ -355,12 +357,24 @@ "since" ] }, + { + "key": "validator.presentIff", + "untestedAttrs": [ + "constraintName" + ] + }, { "key": "validator.regex", "untestedAttrs": [ "max", "min" ] + }, + { + "key": "validator.requiredWhen", + "untestedAttrs": [ + "constraintName" + ] } ] } diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index e0fe66f7ef..da9d8569e9 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -1,5 +1,5 @@ { - "metamodelVersion": "1.2", + "metamodelVersion": "1.3", "types": [ { "type": "attr", @@ -1024,6 +1024,148 @@ } ] }, + { + "type": "field", + "subType": "duration", + "description": "A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.", + "whenToUse": "A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span (\"every 3 months\") is not a duration: model it as an int and a unit.", + "attrs": [ + { + "name": "column", + "valueType": "string", + "isArray": false, + "required": false, + "description": "Physical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the \"naming\" setting in .metaobjects/config.json)." + }, + { + "name": "db.indexed", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, suppress the @filterable-without-index Loader warning (the field is indexed by other means)." + }, + { + "name": "dbColumnType", + "valueType": "string", + "isArray": false, + "required": false, + "allowedValues": [ + "uuid", + "jsonb" + ], + "description": "Physical DB column-type override (ADR-0013 escape hatch). Legal values are uuid | jsonb, both on field.string (uuid = native Postgres uuid column over a string-typed field; jsonb = genuinely-open JSON column). The logical field type and its native binding are unchanged. Native SQL arrays (uuid[]/text[]) are NOT declared here — they are derived from a field subtype + isArray (ADR-0036 Wave 1). The retired timestamp_with_tz value is gone — timezone-awareness lives in field.timestamp (instant by default) + @localTime (the naive opt-out), per ADR-0036 Wave 2." + }, + { + "name": "default", + "valueType": null, + "isArray": false, + "required": false, + "description": "Default value applied to the column when no value is supplied. Its type follows the field's own subtype (string / boolean / number / ...). Converted at consumption time via MetaField.defaultValue()." + }, + { + "name": "example", + "valueType": "string", + "isArray": false, + "required": false, + "description": "FR-010: an example value for this field, shown in the generated output-format prompt fragment." + }, + { + "name": "filterable", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer)." + }, + { + "name": "formExclude", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, the field is omitted from generated forms, and the generated create, PATCH and PUT routes ignore a value sent for it (reads of it are unchanged); inert on fields for which no form is generated (e.g. projection/derived fields)." + }, + { + "name": "instruction", + "valueType": "string", + "isArray": false, + "required": false, + "description": "FR-010: a short instruction for this field, shown in the generated output-format prompt fragment." + }, + { + "name": "mutability", + "valueType": "string", + "isArray": false, + "required": false, + "allowedValues": [ + "readWrite", + "writeOnce", + "readOnly" + ], + "description": "FR-037 R1: who may write this field, and when. 'readWrite' (the default when absent) — the caller may set it on create and change it on update. 'writeOnce' — the caller sets it on create; it is excluded from the update shape thereafter, so a value presented on PATCH is ignored rather than rejected. 'readOnly' — nobody writes it: codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas omit it from input variants; the value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. The three are mutually exclusive modes of ONE axis — who may write, and when — so the illegal pair is unrepresentable and inheritance has a total order: a subtype may TIGHTEN an inherited mode (readWrite < writeOnce < readOnly) and never loosen it (ERR_MUTABILITY_DOWNGRADE). Pairing a non-readWrite mode with @autoSet is ERR_MUTABILITY_AUTOSET_CONFLICT: @autoSet already says the SERVER supplies the value, which is a different axis from who may write it." + }, + { + "name": "required", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, the field is NOT NULL. Equivalent to attaching a validator.required child. On a non-array string, generated wire-tier input validation (create/patch) additionally rejects the empty string by default — whitespace is accepted — unless an explicit validator.length @min: 0 opts back to presence-only. In-process read models never enforce this at construction." + }, + { + "name": "sortable", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, the field is exposed in generated CRUD sort allowlists. Inherits from @filterable by default; set false to opt out." + }, + { + "name": "sortableDefaultOrder", + "valueType": "string", + "isArray": false, + "required": false, + "allowedValues": [ + "asc", + "desc" + ], + "description": "Direction a sort takes when it names this field but omits the order: a ?sort= with no :order, and a layout.dataGrid whose @defaultSortField names it with no @defaultSortOrder. Defaults to 'asc'." + }, + { + "name": "unique", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, the field gets a column-level UNIQUE constraint." + }, + { + "name": "xmlText", + "valueType": "boolean", + "isArray": false, + "required": false, + "description": "When true, this field receives its element's XML TEXT CONTENT during tolerant extract (JAXB @XmlValue / Jackson @JacksonXmlText / .NET [XmlText]) instead of a same-named child. No effect when the eliciting prompt's @responseFormat is json." + } + ], + "children": [ + { + "childType": "origin", + "childSubType": "*", + "childName": "*", + "min": 0, + "max": null + }, + { + "childType": "validator", + "childSubType": "*", + "childName": "*", + "min": 0, + "max": null + }, + { + "childType": "view", + "childSubType": "*", + "childName": "*", + "min": 0, + "max": null + } + ] + }, { "type": "field", "subType": "enum", @@ -1045,6 +1187,13 @@ "required": false, "description": "Physical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the \"naming\" setting in .metaobjects/config.json)." }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "db.indexed", "valueType": "boolean", @@ -1119,6 +1268,13 @@ "required": false, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." }, + { + "name": "maxLength", + "valueType": "int", + "isArray": false, + "required": false, + "description": "Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1." + }, { "name": "mutability", "valueType": "string", @@ -2005,6 +2161,13 @@ "required": false, "description": "When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer)." }, + { + "name": "flattenedColumnPrefix", + "valueType": "string", + "isArray": false, + "required": false, + "description": "On a field.object with @storage \"flattened\": the text put before each member's column, verbatim (\"\" allowed, so the members keep their own column names). Absent: this field's resolved column name and \"_\" (with @column \"bill\", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1." + }, { "name": "formExclude", "valueType": "boolean", @@ -2901,6 +3064,20 @@ "assigned" ], "description": "Primary-key value generation strategy: 'increment' (auto-increment), 'uuid', or 'assigned' (caller-supplied)." + }, + { + "name": "sequenceName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, + { + "name": "sequenceStart", + "valueType": "long", + "isArray": false, + "required": false, + "description": "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } ], "children": [] @@ -4650,6 +4827,20 @@ "rules": "@fields names two or more fields of the owning entity. Satisfied when any one of them is non-null.", "whenToUse": "At least one of several fields must be present (email or phone). Declare it instead of hand-written OR-presence checks.", "attrs": [ + { + "name": "condition", + "valueType": "expression", + "isArray": false, + "required": false, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "fields", "valueType": "string", @@ -4689,6 +4880,20 @@ "rules": "@left and @right must name fields of the owning entity. @op is one of gt/gte/lt/lte/ne/eq. The comparison is null-tolerant where the backend's relational operator is (SQL: a NULL operand yields no violation).", "whenToUse": "Two sibling fields must stand in an order (start <= end, currentHp <= maxHp). Declare it instead of hand-writing the cross-field assertion.", "attrs": [ + { + "name": "condition", + "valueType": "expression", + "isArray": false, + "required": false, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "left", "valueType": "string", @@ -4721,12 +4926,50 @@ ], "children": [] }, + { + "type": "validator", + "subType": "exactlyOne", + "description": "Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.", + "rules": "@fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.", + "whenToUse": "A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.", + "attrs": [ + { + "name": "condition", + "valueType": "expression", + "isArray": false, + "required": false, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, + { + "name": "fields", + "valueType": "string", + "isArray": true, + "required": true, + "description": "Names of the alternative fields; exactly one must be present." + } + ], + "children": [] + }, { "type": "validator", "subType": "length", "description": "Bounds string length / collection size via @min/@max.", "whenToUse": "Bound a string's length or a collection's size. Use @min/@max instead of hand-written guards.", "attrs": [ + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "max", "valueType": "int", @@ -4750,6 +4993,13 @@ "description": "Bounds a numeric value's magnitude via @min/@max.", "whenToUse": "Bound a number's magnitude. Use @min/@max instead of hand-written range checks.", "attrs": [ + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "max", "valueType": "int", @@ -4774,6 +5024,20 @@ "rules": "@field and @when must name fields of the owning entity. @equals is rendered per @when's field subtype. Stricter than requiredWhen — also forbids @field when the condition is false.", "whenToUse": "A field must be present exactly when a condition holds, absent otherwise (biconditional). Declare it instead of hand-written presence logic.", "attrs": [ + { + "name": "condition", + "valueType": "expression", + "isArray": false, + "required": false, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "equals", "valueType": "string", @@ -4804,6 +5068,13 @@ "description": "Requires the value match a regular expression (@pattern).", "whenToUse": "A string must match a pattern (email, slug, code). Set @pattern instead of hand-writing the regex test.", "attrs": [ + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "max", "valueType": "int", @@ -4843,6 +5114,20 @@ "rules": "@field and @when must name fields of the owning entity. @equals is the gating value, compared against @when's value (rendered per @when's field subtype — boolean true/false, enum/string literal, numeric literal).", "whenToUse": "A field becomes required only when another field has a given value. Declare it instead of conditional validation code.", "attrs": [ + { + "name": "condition", + "valueType": "expression", + "isArray": false, + "required": false, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + }, + { + "name": "constraintName", + "valueType": "string", + "isArray": false, + "required": false, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, { "name": "equals", "valueType": "string", diff --git a/fixtures/registry-conformance/providers.json b/fixtures/registry-conformance/providers.json index 10930e07b4..94e5457142 100644 --- a/fixtures/registry-conformance/providers.json +++ b/fixtures/registry-conformance/providers.json @@ -23,13 +23,23 @@ "projectsOnto": [ "field.*", "field.date", + "field.enum", "field.object", "field.time", "field.timestamp", + "identity.primary", "identity.reference", "identity.secondary", "index.lookup", - "source.rdb" + "source.rdb", + "validator.atLeastOne", + "validator.comparison", + "validator.exactlyOne", + "validator.length", + "validator.numeric", + "validator.presentIff", + "validator.regex", + "validator.requiredWhen" ] }, "metaobjects-documentation": { diff --git a/fixtures/requirement-harness/csharp/RequirementHarnessTests.cs b/fixtures/requirement-harness/csharp/RequirementHarnessTests.cs index 56a08ae700..f152f72dea 100644 --- a/fixtures/requirement-harness/csharp/RequirementHarnessTests.cs +++ b/fixtures/requirement-harness/csharp/RequirementHarnessTests.cs @@ -1,6 +1,6 @@ // @generated by scripts/generate-requirement-harness.ts — DO NOT EDIT IN PLACE. // -// One slot per authored promise in metaobjects/meta.requirements.yaml (115 of them). +// One slot per authored promise in metaobjects/meta.requirements.yaml (117 of them). // Every slot is SKIPPED: this is a to-do list with a drift gate behind it, not a // claim that nothing is verified — the conformance corpora already check many of // these. To FILL one, move it into C#'s own suite and write the assertion the @@ -64,6 +64,10 @@ public void declare_constraints_validatorAtLeastOne() { /* metaobjects.declare.c [Fact(Skip = "unfilled slot — write an assertion that fails when: An end date before its start date, stored, and every report that spans them silently empty.")] public void declare_constraints_validatorComparison() { /* metaobjects.declare.constraints.validatorComparison */ } + // Exactly one — An exactly-one rule requires that one member of a declared group is given, and no other. + [Fact(Skip = "unfilled slot — write an assertion that fails when: A booking tied to both a store and a kiosk, so neither can tell whose it is.")] + public void declare_constraints_validatorExactlyOne() { /* metaobjects.declare.constraints.validatorExactlyOne */ } + // Length bounds — A length rule bounds how long a value may be, and an explicitly declared bound is authoritative over any implicit floor. [Fact(Skip = "unfilled slot — write an assertion that fails when: A declared minimum of zero raised to one behind the author's back, so provided-but-empty becomes inexpressible.")] public void declare_constraints_validatorLength() { /* metaobjects.declare.constraints.validatorLength */ } @@ -380,6 +384,10 @@ public void declare_typedFieldVocabulary_fieldDecimal() { /* metaobjects.declare [Fact(Skip = "unfilled slot — write an assertion that fails when: A running total accumulated in one, where the sum of the parts no longer equals the whole.")] public void declare_typedFieldVocabulary_fieldDouble() { /* metaobjects.declare.typedFieldVocabulary.fieldDouble */ } + // Span of time — A duration field holds a fixed length of time as a span, not as a count whose unit only the code knows. + [Fact(Skip = "unfilled slot — write an assertion that fails when: A timeout read as seconds by one service and as milliseconds by another.")] + public void declare_typedFieldVocabulary_fieldDuration() { /* metaobjects.declare.typedFieldVocabulary.fieldDuration */ } + // Enumeration — An enumeration field holds one member of a closed declared set, and the member symbol is what the wire and the runtime see whatever the column stores. [Fact(Skip = "unfilled slot — write an assertion that fails when: A stored value that is no declared member, handed back as though it were one.")] public void declare_typedFieldVocabulary_fieldEnum() { /* metaobjects.declare.typedFieldVocabulary.fieldEnum */ } diff --git a/fixtures/requirement-harness/java/RequirementHarnessTest.java b/fixtures/requirement-harness/java/RequirementHarnessTest.java index f02b52d9d3..90a7342772 100644 --- a/fixtures/requirement-harness/java/RequirementHarnessTest.java +++ b/fixtures/requirement-harness/java/RequirementHarnessTest.java @@ -1,6 +1,6 @@ // @generated by scripts/generate-requirement-harness.ts — DO NOT EDIT IN PLACE. // -// One slot per authored promise in metaobjects/meta.requirements.yaml (115 of them). +// One slot per authored promise in metaobjects/meta.requirements.yaml (117 of them). // Every slot is SKIPPED: this is a to-do list with a drift gate behind it, not a // claim that nothing is verified — the conformance corpora already check many of // these. To FILL one, move it into Java's own suite and write the assertion the @@ -77,6 +77,11 @@ void declare_constraints_validatorAtLeastOne() { /* metaobjects.declare.constrai @Disabled("unfilled slot — write an assertion that fails when: An end date before its start date, stored, and every report that spans them silently empty.") void declare_constraints_validatorComparison() { /* metaobjects.declare.constraints.validatorComparison */ } + // Exactly one — An exactly-one rule requires that one member of a declared group is given, and no other. + @Test + @Disabled("unfilled slot — write an assertion that fails when: A booking tied to both a store and a kiosk, so neither can tell whose it is.") + void declare_constraints_validatorExactlyOne() { /* metaobjects.declare.constraints.validatorExactlyOne */ } + // Length bounds — A length rule bounds how long a value may be, and an explicitly declared bound is authoritative over any implicit floor. @Test @Disabled("unfilled slot — write an assertion that fails when: A declared minimum of zero raised to one behind the author's back, so provided-but-empty becomes inexpressible.") @@ -472,6 +477,11 @@ void declare_typedFieldVocabulary_fieldDecimal() { /* metaobjects.declare.typedF @Disabled("unfilled slot — write an assertion that fails when: A running total accumulated in one, where the sum of the parts no longer equals the whole.") void declare_typedFieldVocabulary_fieldDouble() { /* metaobjects.declare.typedFieldVocabulary.fieldDouble */ } + // Span of time — A duration field holds a fixed length of time as a span, not as a count whose unit only the code knows. + @Test + @Disabled("unfilled slot — write an assertion that fails when: A timeout read as seconds by one service and as milliseconds by another.") + void declare_typedFieldVocabulary_fieldDuration() { /* metaobjects.declare.typedFieldVocabulary.fieldDuration */ } + // Enumeration — An enumeration field holds one member of a closed declared set, and the member symbol is what the wire and the runtime see whatever the column stores. @Test @Disabled("unfilled slot — write an assertion that fails when: A stored value that is no declared member, handed back as though it were one.") diff --git a/fixtures/requirement-harness/kotlin/RequirementHarnessTest.kt b/fixtures/requirement-harness/kotlin/RequirementHarnessTest.kt index 38830b4bfc..6ed84f006c 100644 --- a/fixtures/requirement-harness/kotlin/RequirementHarnessTest.kt +++ b/fixtures/requirement-harness/kotlin/RequirementHarnessTest.kt @@ -1,6 +1,6 @@ // @generated by scripts/generate-requirement-harness.ts — DO NOT EDIT IN PLACE. // -// One slot per authored promise in metaobjects/meta.requirements.yaml (115 of them). +// One slot per authored promise in metaobjects/meta.requirements.yaml (117 of them). // Every slot is SKIPPED: this is a to-do list with a drift gate behind it, not a // claim that nothing is verified — the conformance corpora already check many of // these. To FILL one, move it into Kotlin's own suite and write the assertion the @@ -77,6 +77,11 @@ class RequirementHarnessTest { @Disabled("unfilled slot — write an assertion that fails when: An end date before its start date, stored, and every report that spans them silently empty.") fun declare_constraints_validatorComparison() { /* metaobjects.declare.constraints.validatorComparison */ } + // Exactly one — An exactly-one rule requires that one member of a declared group is given, and no other. + @Test + @Disabled("unfilled slot — write an assertion that fails when: A booking tied to both a store and a kiosk, so neither can tell whose it is.") + fun declare_constraints_validatorExactlyOne() { /* metaobjects.declare.constraints.validatorExactlyOne */ } + // Length bounds — A length rule bounds how long a value may be, and an explicitly declared bound is authoritative over any implicit floor. @Test @Disabled("unfilled slot — write an assertion that fails when: A declared minimum of zero raised to one behind the author's back, so provided-but-empty becomes inexpressible.") @@ -472,6 +477,11 @@ class RequirementHarnessTest { @Disabled("unfilled slot — write an assertion that fails when: A running total accumulated in one, where the sum of the parts no longer equals the whole.") fun declare_typedFieldVocabulary_fieldDouble() { /* metaobjects.declare.typedFieldVocabulary.fieldDouble */ } + // Span of time — A duration field holds a fixed length of time as a span, not as a count whose unit only the code knows. + @Test + @Disabled("unfilled slot — write an assertion that fails when: A timeout read as seconds by one service and as milliseconds by another.") + fun declare_typedFieldVocabulary_fieldDuration() { /* metaobjects.declare.typedFieldVocabulary.fieldDuration */ } + // Enumeration — An enumeration field holds one member of a closed declared set, and the member symbol is what the wire and the runtime see whatever the column stores. @Test @Disabled("unfilled slot — write an assertion that fails when: A stored value that is no declared member, handed back as though it were one.") diff --git a/fixtures/requirement-harness/python/test_requirement_harness.py b/fixtures/requirement-harness/python/test_requirement_harness.py index 0e5fdfd90e..bc494c15ce 100644 --- a/fixtures/requirement-harness/python/test_requirement_harness.py +++ b/fixtures/requirement-harness/python/test_requirement_harness.py @@ -1,6 +1,6 @@ # @generated by scripts/generate-requirement-harness.ts — DO NOT EDIT IN PLACE. # -# One slot per authored promise in metaobjects/meta.requirements.yaml (115 of them). +# One slot per authored promise in metaobjects/meta.requirements.yaml (117 of them). # Every slot is SKIPPED: this is a to-do list with a drift gate behind it, not a # claim that nothing is verified — the conformance corpora already check many of # these. To FILL one, move it into Python's own suite and write the assertion the @@ -72,6 +72,11 @@ def test_declare_constraints_validatorAtLeastOne(): def test_declare_constraints_validatorComparison(): """metaobjects.declare.constraints.validatorComparison""" +# Exactly one — An exactly-one rule requires that one member of a declared group is given, and no other. +@pytest.mark.skip(reason="unfilled slot — write an assertion that fails when: A booking tied to both a store and a kiosk, so neither can tell whose it is.") +def test_declare_constraints_validatorExactlyOne(): + """metaobjects.declare.constraints.validatorExactlyOne""" + # Length bounds — A length rule bounds how long a value may be, and an explicitly declared bound is authoritative over any implicit floor. @pytest.mark.skip(reason="unfilled slot — write an assertion that fails when: A declared minimum of zero raised to one behind the author's back, so provided-but-empty becomes inexpressible.") def test_declare_constraints_validatorLength(): @@ -467,6 +472,11 @@ def test_declare_typedFieldVocabulary_fieldDecimal(): def test_declare_typedFieldVocabulary_fieldDouble(): """metaobjects.declare.typedFieldVocabulary.fieldDouble""" +# Span of time — A duration field holds a fixed length of time as a span, not as a count whose unit only the code knows. +@pytest.mark.skip(reason="unfilled slot — write an assertion that fails when: A timeout read as seconds by one service and as milliseconds by another.") +def test_declare_typedFieldVocabulary_fieldDuration(): + """metaobjects.declare.typedFieldVocabulary.fieldDuration""" + # Enumeration — An enumeration field holds one member of a closed declared set, and the member symbol is what the wire and the runtime see whatever the column stores. @pytest.mark.skip(reason="unfilled slot — write an assertion that fails when: A stored value that is no declared member, handed back as though it were one.") def test_declare_typedFieldVocabulary_fieldEnum(): diff --git a/fixtures/requirement-harness/typescript/requirement-harness.test.ts b/fixtures/requirement-harness/typescript/requirement-harness.test.ts index 487abe99db..1e7d21f463 100644 --- a/fixtures/requirement-harness/typescript/requirement-harness.test.ts +++ b/fixtures/requirement-harness/typescript/requirement-harness.test.ts @@ -1,6 +1,6 @@ // @generated by scripts/generate-requirement-harness.ts — DO NOT EDIT IN PLACE. // -// One slot per authored promise in metaobjects/meta.requirements.yaml (115 of them). +// One slot per authored promise in metaobjects/meta.requirements.yaml (117 of them). // Every slot is SKIPPED: this is a to-do list with a drift gate behind it, not a // claim that nothing is verified — the conformance corpora already check many of // these. To FILL one, move it into TypeScript's own suite and write the assertion the @@ -72,6 +72,11 @@ test.skip("metaobjects.declare.constraints.validatorComparison", () => { // unfilled slot — write an assertion that fails when: An end date before its start date, stored, and every report that spans them silently empty. }); +// Exactly one — An exactly-one rule requires that one member of a declared group is given, and no other. +test.skip("metaobjects.declare.constraints.validatorExactlyOne", () => { + // unfilled slot — write an assertion that fails when: A booking tied to both a store and a kiosk, so neither can tell whose it is. +}); + // Length bounds — A length rule bounds how long a value may be, and an explicitly declared bound is authoritative over any implicit floor. test.skip("metaobjects.declare.constraints.validatorLength", () => { // unfilled slot — write an assertion that fails when: A declared minimum of zero raised to one behind the author's back, so provided-but-empty becomes inexpressible. @@ -467,6 +472,11 @@ test.skip("metaobjects.declare.typedFieldVocabulary.fieldDouble", () => { // unfilled slot — write an assertion that fails when: A running total accumulated in one, where the sum of the parts no longer equals the whole. }); +// Span of time — A duration field holds a fixed length of time as a span, not as a count whose unit only the code knows. +test.skip("metaobjects.declare.typedFieldVocabulary.fieldDuration", () => { + // unfilled slot — write an assertion that fails when: A timeout read as seconds by one service and as milliseconds by another. +}); + // Enumeration — An enumeration field holds one member of a closed declared set, and the member symbol is what the wire and the runtime see whatever the column stores. test.skip("metaobjects.declare.typedFieldVocabulary.fieldEnum", () => { // unfilled slot — write an assertion that fails when: A stored value that is no declared member, handed back as though it were one. diff --git a/fixtures/validation-conformance/.metaobjects/config.json b/fixtures/validation-conformance/.metaobjects/config.json new file mode 100644 index 0000000000..5c95a8d2c8 --- /dev/null +++ b/fixtures/validation-conformance/.metaobjects/config.json @@ -0,0 +1,4 @@ +{ + "schema_version": 1, + "wire": { "durations": "iso" } +} diff --git a/fixtures/validation-conformance/README.md b/fixtures/validation-conformance/README.md index 0382d6982d..a13a58e731 100644 --- a/fixtures/validation-conformance/README.md +++ b/fixtures/validation-conformance/README.md @@ -113,11 +113,12 @@ FR-036 pin cases (the two semantic pins + the length-precedence pin): ## Cross-field validators — `meta.cross-field.json` + `cross-field-cases.json` -A second, separate pair of files pins the four entity-scoped cross-field validators +A second, separate pair of files pins the five entity-scoped cross-field validators (`validator.comparison`, `validator.requiredWhen`, `validator.presentIff`, -`validator.atLeastOne`) on the generated **full record (create) model**. They are separate files -so the run-time runners, which read `cases.json`, are unaffected. The entity is `Event`; every field -is optional, so absence is expressible in every port. +`validator.atLeastOne`, `validator.exactlyOne`) and their `@condition` on the generated **full +record (create) model**. They are separate files so the run-time runners, which read `cases.json`, +are unaffected. There are two entities, `Event` (the unconditional rules) and `Shift` (`exactlyOne` +and the conditional rules); every field is optional, so absence is expressible in every port. The semantics mirror the DB `CHECK` derivation in `migrate-ts/src/expected-schema.ts`, with an absent or null value standing for SQL `NULL` (and `""` NOT null). One deliberate difference: a comparison the @@ -129,13 +130,25 @@ can compare the two column types: date against a timestamp, is declared on `Event` and must accept a body where the date is later). Two temporals of the same subtype are still different kinds when only one carries `@localTime` (`loggedAt lte wallClock`, an instant against a wall clock, is skipped the same way). Ordering applies to numerics with each other and to - strings and temporals of the same subtype; `eq` / `ne` also apply to two fields of the same boolean, - enum, uuid, uri or inet subtype (`ownerId ne approverId` rejects two equal ids). + strings, temporals and durations of the same subtype; `eq` / `ne` also apply to two fields of the same boolean, + enum, uuid, uri or inet subtype (`ownerId ne approverId` rejects two equal ids). Two durations order by + length, not by their text (`Shift.actual lte planned` accepts `PT45M` against `PT1H`); the duration + payloads are ISO-8601, and the corpus pins `wire.durations: iso` in its own `.metaobjects/config.json`, + which a runner whose port defaults to another form reads. - `requiredWhen`: violated only when the gating field equals `@equals` and the target is null; a null gate is unconstrained. Both a string gate (`kind = BIRTH`) and a boolean gate (`flag = true`). - `presentIff`: the target is present exactly when the gate equals `@equals`; a null gate counts as not equal, so the target must be absent. - `atLeastOne` (`email`, `phone`): at least one is non-null. +- `exactlyOne` (`Shift`: `storeId`, `kioskId`): exactly one is non-null; none and both are invalid. +- `@condition` (`Shift`): the rule is checked only when the condition is TRUE under SQL three-valued + logic, since the CHECK is `NOT (condition) OR (rule)`. A false condition and a NULL one (a field it + compares is absent) both skip the rule. The cases cover a field-to-field `eq`, `not` over `isNull`, + and `and` / `or` / `ne` / `isNull` over a `coalesce` with a literal default. A comparison inside a + condition follows the `comparison` rule above: when two fields it reads are of different kinds, the + whole rule is left to the database (`lateCallNeedsPhone` compares a date with a timestamp, so a `CALL` + without a phone is valid). `eq` / `ne` on two equality-only fields test equality and never order them + (`offSiteCallNeedsPhone` compares an IPv4 with an IPv6 address, which have no order, and is enforced). - A rule naming a field the create body never carries (`reviewedBy`, `@mutability: readOnly`; likewise a derived field or a store-generated key) is skipped, so `review = DONE` without a reviewer is valid. @@ -184,6 +197,16 @@ So the rule is: Python fuses both (Pydantic construct-or-`ValidationError`); the Java/Kotlin/C# runners wrap the bind step so a native-parse failure maps to `valid=false`. +## `field.duration` accepted form + +`Account.holdFor` (`field.duration`, OPTIONAL) pins the duration accept/reject verdicts. ACCEPT an +ISO-8601 duration of days, hours, minutes and seconds (`PT1H30M`, `P1DT2H`, `-PT30S`, `PT0.5S`, +`PT0S`). REJECT a calendar span (`P1M`, `P1Y`, `P2W`), a clock form (`1:30:00`), a fractional hour +(`PT1.5H`), lowercase designators, and a `T` with no component (`PT`). The full rule, with the +canonical form every port writes, is in [`fixtures/duration-wire/`](../duration-wire/); a port +that binds a native duration type (`TimeSpan`, `java.time.Duration`, `timedelta`) still refuses +what that type's own parser would accept beyond this form. + ## Run-time runners (TypeScript and Python) Two ports also ship a metadata-driven **run-time** runner that needs no generated code: diff --git a/fixtures/validation-conformance/cases.json b/fixtures/validation-conformance/cases.json index 2f65ea2514..04b8b49be9 100644 --- a/fixtures/validation-conformance/cases.json +++ b/fixtures/validation-conformance/cases.json @@ -109,6 +109,18 @@ { "name": "lenient-inet-accepts-valid", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "reportedIp": "10.0.0.1" }, "expectValid": true }, { "name": "assigned-pk-present", "entity": "Ledger", "payload": { "code": "L-1", "label": "ok" }, "expectValid": true }, - { "name": "assigned-pk-missing", "entity": "Ledger", "payload": { "label": "ok" }, "expectValid": false } + { "name": "assigned-pk-missing", "entity": "Ledger", "payload": { "label": "ok" }, "expectValid": false }, + { "name": "duration-accept-hours-minutes", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "PT1H30M" }, "expectValid": true }, + { "name": "duration-accept-days", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "P1DT2H" }, "expectValid": true }, + { "name": "duration-accept-negative", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "-PT30S" }, "expectValid": true }, + { "name": "duration-accept-fraction", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "PT0.5S" }, "expectValid": true }, + { "name": "duration-accept-zero", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "PT0S" }, "expectValid": true }, + { "name": "duration-reject-months", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "P1M" }, "expectValid": false }, + { "name": "duration-reject-years", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "P1Y" }, "expectValid": false }, + { "name": "duration-accept-weeks", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "P2W" }, "expectValid": true }, + { "name": "duration-reject-clock", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "1:30:00" }, "expectValid": false }, + { "name": "duration-reject-fractional-hours", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "PT1.5H" }, "expectValid": false }, + { "name": "duration-reject-lowercase", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "pt1h" }, "expectValid": false }, + { "name": "duration-reject-empty-time", "payload": { "name": "Acme", "code": "ABC", "score": 50, "tags": ["x"], "note": "ok", "holdFor": "PT" }, "expectValid": false } ] } diff --git a/fixtures/validation-conformance/cross-field-cases.json b/fixtures/validation-conformance/cross-field-cases.json index dae54a0eef..d077a6763b 100644 --- a/fixtures/validation-conformance/cross-field-cases.json +++ b/fixtures/validation-conformance/cross-field-cases.json @@ -318,6 +318,244 @@ "approverId": "9b7d4e22-0c1a-4f3b-8d65-2a7e9c5f1b34" }, "expectValid": true + }, + { + "name": "shift-valid-baseline", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "DESK" + }, + "expectValid": true + }, + { + "name": "condition-true-rule-violated", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 120, + "endMin": 60, + "storeId": 7, + "kind": "DESK" + }, + "expectValid": false + }, + { + "name": "condition-false-rule-skipped", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 2, + "startMin": 120, + "endMin": 60, + "storeId": 7, + "kind": "DESK" + }, + "expectValid": true + }, + { + "name": "condition-null-rule-skipped", + "entity": "Shift", + "payload": { + "endDay": 1, + "startMin": 120, + "endMin": 60, + "storeId": 7, + "kind": "DESK" + }, + "expectValid": true + }, + { + "name": "exactly-one-none", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "kind": "DESK" + }, + "expectValid": false + }, + { + "name": "exactly-one-both", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "DESK", + "kioskId": 3 + }, + "expectValid": false + }, + { + "name": "exactly-one-other", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "kind": "DESK", + "kioskId": 3 + }, + "expectValid": true + }, + { + "name": "condition-not-is-null-true-violated", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "PHONE" + }, + "expectValid": false + }, + { + "name": "condition-not-is-null-false-skipped", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "kind": "PHONE", + "kioskId": 3 + }, + "expectValid": true + }, + { + "name": "condition-and-true-violated", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "DESK", + "priority": 3 + }, + "expectValid": false + }, + { + "name": "condition-and-satisfied", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "DESK", + "priority": 3, + "phone": "555" + }, + "expectValid": true + }, + { + "name": "condition-coalesce-default-skipped", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "DESK" + }, + "expectValid": true + }, + { + "name": "condition-and-false-by-or-skipped", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "kind": "BATCH", + "priority": 3 + }, + "expectValid": true + }, + { + "name": "condition-or-null-branch-violated", + "entity": "Shift", + "payload": { + "startDay": 1, + "endDay": 1, + "startMin": 60, + "endMin": 120, + "storeId": 7, + "priority": 3 + }, + "expectValid": false + }, + { + "name": "condition-comparing-across-kinds-leaves-rule-to-database", + "entity": "Shift", + "payload": { + "storeId": 7, + "kind": "CALL", + "shiftDate": "2030-01-02", + "loggedAt": "2020-01-01T00:00:00Z" + }, + "expectValid": true + }, + { + "name": "condition-ne-on-ipv4-and-ipv6-addresses-enforces-the-rule", + "entity": "Shift", + "payload": { + "storeId": 7, + "kind": "CALL", + "clientIp": "10.0.0.1", + "siteIp": "::1" + }, + "expectValid": false + }, + { + "name": "condition-ne-on-equal-addresses-lets-the-record-through", + "entity": "Shift", + "payload": { + "storeId": 7, + "kind": "CALL", + "clientIp": "10.0.0.1", + "siteIp": "10.0.0.1" + }, + "expectValid": true + }, + { + "name": "duration-comparison-rejects-a-longer-span", + "entity": "Shift", + "payload": { + "storeId": 7, + "planned": "PT1H", + "actual": "PT1H30M" + }, + "expectValid": false + }, + { + "name": "duration-comparison-orders-by-length-not-text", + "entity": "Shift", + "payload": { + "storeId": 7, + "planned": "PT1H", + "actual": "PT45M" + }, + "expectValid": true } ] } diff --git a/fixtures/validation-conformance/meta.cross-field.json b/fixtures/validation-conformance/meta.cross-field.json index a47faea675..093886570b 100644 --- a/fixtures/validation-conformance/meta.cross-field.json +++ b/fixtures/validation-conformance/meta.cross-field.json @@ -36,6 +36,48 @@ { "validator.atLeastOne": { "name": "reachable", "@fields": ["email", "phone"] } }, { "validator.requiredWhen": { "name": "reviewNeedsReviewer", "@field": "reviewedBy", "@when": "review", "@equals": "DONE" } } ] + }}, + { "object.entity": { + "name": "Shift", + "children": [ + { "source.rdb": { "@table": "shifts" } }, + { "field.long": { "name": "id" } }, + { "field.int": { "name": "startDay" } }, + { "field.int": { "name": "endDay" } }, + { "field.int": { "name": "startMin" } }, + { "field.int": { "name": "endMin" } }, + { "field.long": { "name": "storeId" } }, + { "field.long": { "name": "kioskId" } }, + { "field.string": { "name": "kind" } }, + { "field.string": { "name": "note" } }, + { "field.int": { "name": "priority" } }, + { "field.string": { "name": "email" } }, + { "field.string": { "name": "phone" } }, + { "field.date": { "name": "shiftDate" } }, + { "field.timestamp": { "name": "loggedAt" } }, + { "field.duration": { "name": "planned" } }, + { "field.duration": { "name": "actual" } }, + { "field.inet": { "name": "clientIp" } }, + { "field.inet": { "name": "siteIp" } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, + { "validator.comparison": { "name": "sameDayTimes", "@left": "endMin", "@op": "gt", "@right": "startMin", + "@condition": { "op": "eq", "left": { "field": "startDay" }, "right": { "field": "endDay" } } } }, + { "validator.comparison": { "name": "withinPlan", "@left": "actual", "@op": "lte", "@right": "planned" } }, + { "validator.exactlyOne": { "name": "oneLocation", "@fields": ["storeId", "kioskId"] } }, + { "validator.requiredWhen": { "name": "storePhoneNeedsNote", "@field": "note", "@when": "kind", "@equals": "PHONE", + "@condition": { "op": "not", "arg": { "op": "isNull", "arg": { "field": "storeId" } } } } }, + { "validator.requiredWhen": { "name": "offSiteCallNeedsPhone", "@field": "phone", "@when": "kind", "@equals": "CALL", "@condition": { "op": "ne", "left": { "field": "clientIp" }, "right": { "field": "siteIp" } } } }, + { "validator.requiredWhen": { "name": "lateCallNeedsPhone", "@field": "phone", "@when": "kind", "@equals": "CALL", + "@condition": { "op": "gt", "left": { "field": "shiftDate" }, "right": { "field": "loggedAt" } } } }, + { "validator.atLeastOne": { "name": "urgentReachable", "@fields": ["email", "phone"], + "@condition": { "op": "and", "args": [ + { "op": "gt", "left": { "fn": "coalesce", "args": [{ "field": "priority" }, { "value": 0 }] }, "right": { "value": 2 } }, + { "op": "or", "args": [ + { "op": "ne", "left": { "field": "kind" }, "right": { "value": "BATCH" } }, + { "op": "isNull", "arg": { "field": "kind" } } + ] } + ] } } } + ] }} ] } diff --git a/fixtures/validation-conformance/meta.json b/fixtures/validation-conformance/meta.json index a8a4daf79d..3f696a5c98 100644 --- a/fixtures/validation-conformance/meta.json +++ b/fixtures/validation-conformance/meta.json @@ -28,6 +28,7 @@ { "field.inet": { "name": "sourceIp" } }, { "field.uri": { "name": "citationUrl", "@lenient": true } }, { "field.inet": { "name": "reportedIp", "@lenient": true } }, + { "field.duration": { "name": "holdFor" } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } ] }}, diff --git a/fixtures/validation-conformance/runtime-errors.json b/fixtures/validation-conformance/runtime-errors.json index 1f70ea54ca..4ee0515d58 100644 --- a/fixtures/validation-conformance/runtime-errors.json +++ b/fixtures/validation-conformance/runtime-errors.json @@ -207,6 +207,60 @@ "rule": "required", "message": "'code' is required" } + ], + "duration-reject-months": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "P1M" + } + ], + "duration-reject-years": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "P1Y" + } + ], + "duration-reject-clock": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "1:30:00" + } + ], + "duration-reject-fractional-hours": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "PT1.5H" + } + ], + "duration-reject-lowercase": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "pt1h" + } + ], + "duration-reject-empty-time": [ + { + "field": "holdFor", + "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", + "received": "PT" + } ] } } diff --git a/fixtures/wire-conformance/README.md b/fixtures/wire-conformance/README.md index 17ad57a549..e39a32c1c3 100644 --- a/fixtures/wire-conformance/README.md +++ b/fixtures/wire-conformance/README.md @@ -1,11 +1,12 @@ # Wire conformance The `wire` block of `.metaobjects/config.json` (ADR-0019 Amendment 2) chooses the OUTPUT form of -a generated REST surface: how a `field.decimal` is written, and whether a value object's null -member is written. The default is `native` for both keys, which means each port writes what its -language's standard serializer writes. `"decimals": "string"` and `"nullMembers": "omit"` give -identical output across the five ports; the api-contract and persistence corpora pin exactly -that form in their own `.metaobjects/config.json`. +a generated REST surface: how a `field.decimal` is written, whether a value object's null +member is written, and how a `field.duration` is written and read. The default is `native` for +every key, which means each port writes what its language's standard serializer writes. +`"decimals": "string"`, `"nullMembers": "omit"` and `"durations": "iso"` give identical output +across the five ports; the api-contract and persistence corpora pin exactly that form in their +own `.metaobjects/config.json`. [`config-blocks.json`](./config-blocks.json) holds the blocks every port must accept, with the values they resolve to, and the blocks every port must refuse when it reads the file. Each port diff --git a/fixtures/wire-conformance/config-blocks.json b/fixtures/wire-conformance/config-blocks.json index eaa35234bc..385d05f251 100644 --- a/fixtures/wire-conformance/config-blocks.json +++ b/fixtures/wire-conformance/config-blocks.json @@ -1,12 +1,13 @@ { - "_comment": "Cross-port wire-format setting (ADR-0019 Amendment 2). Each case is the `wire` block of a project's .metaobjects/config.json (null = the key is absent). A port parses it with the reader it uses for that file and must resolve `decimals` and `nullMembers` to the values below; the default is `native` for both, which means each port's own language-standard output. `invalid` blocks must be refused when the config is read.", + "_comment": "Cross-port wire-format setting (ADR-0019 Amendment 2). Each case is the `wire` block of a project's .metaobjects/config.json (null = the key is absent). A port parses it with the reader it uses for that file and must resolve `decimals`, `nullMembers` and `durations` to the values below; the default is `native` for each, which means each port's own language-standard output. `invalid` blocks must be refused when the config is read.", "cases": [ - { "name": "default", "wire": null, "decimals": "native", "nullMembers": "native" }, - { "name": "empty block is the default", "wire": {}, "decimals": "native", "nullMembers": "native" }, - { "name": "explicit native", "wire": { "decimals": "native", "nullMembers": "native" }, "decimals": "native", "nullMembers": "native" }, - { "name": "portable form (what the corpora pin)", "wire": { "decimals": "string", "nullMembers": "omit" }, "decimals": "string", "nullMembers": "omit" }, - { "name": "decimals alone", "wire": { "decimals": "string" }, "decimals": "string", "nullMembers": "native" }, - { "name": "emit null members", "wire": { "nullMembers": "emit" }, "decimals": "native", "nullMembers": "emit" } + { "name": "default", "wire": null, "decimals": "native", "nullMembers": "native", "durations": "native" }, + { "name": "empty block is the default", "wire": {}, "decimals": "native", "nullMembers": "native", "durations": "native" }, + { "name": "explicit native", "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" }, "decimals": "native", "nullMembers": "native", "durations": "native" }, + { "name": "portable form (what the corpora pin)", "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" }, "decimals": "string", "nullMembers": "omit", "durations": "iso" }, + { "name": "decimals alone", "wire": { "decimals": "string" }, "decimals": "string", "nullMembers": "native", "durations": "native" }, + { "name": "emit null members", "wire": { "nullMembers": "emit" }, "decimals": "native", "nullMembers": "emit", "durations": "native" }, + { "name": "iso durations alone", "wire": { "durations": "iso" }, "decimals": "native", "nullMembers": "native", "durations": "iso" } ], "invalid": [ { "name": "wire is not an object", "wire": "string" }, @@ -16,6 +17,8 @@ { "name": "unknown decimals value", "wire": { "decimals": "canonical" } }, { "name": "decimals is not a string", "wire": { "decimals": true } }, { "name": "unknown nullMembers value", "wire": { "nullMembers": "skip" } }, - { "name": "nullMembers is not a string", "wire": { "nullMembers": null } } + { "name": "nullMembers is not a string", "wire": { "nullMembers": null } }, + { "name": "unknown durations value", "wire": { "durations": "iso8601" } }, + { "name": "durations is not a string", "wire": { "durations": false } } ] } diff --git a/metaobjects/meta.requirements.yaml b/metaobjects/meta.requirements.yaml index dcd229a300..fedf9a3b70 100644 --- a/metaobjects/meta.requirements.yaml +++ b/metaobjects/meta.requirements.yaml @@ -206,6 +206,18 @@ metadata.root: counterexample: >- An opening hour that cannot be declared without inventing a date to hang it on. + - requirement.functional: + name: fieldDuration + title: Span of time + level: 4 + status: live + statement: >- + A duration field holds a fixed length of time as a + span, not as a count whose unit only the code + knows. + counterexample: >- + A timeout read as seconds by one service and as + milliseconds by another. - requirement.functional: name: fieldTimestamp title: Instant @@ -1370,6 +1382,17 @@ metadata.root: counterexample: >- A contact saved with neither an email address nor a phone number, and no way to reach them. + - requirement.functional: + name: validatorExactlyOne + title: Exactly one + level: 4 + status: live + statement: >- + An exactly-one rule requires that one member of a + declared group is given, and no other. + counterexample: >- + A booking tied to both a store and a kiosk, so + neither can tell whose it is. - requirement.functional: name: presentation diff --git a/scripts/check-doc-examples.ts b/scripts/check-doc-examples.ts index 97ea0ee1a6..239c65f50d 100644 --- a/scripts/check-doc-examples.ts +++ b/scripts/check-doc-examples.ts @@ -154,6 +154,11 @@ const FAIL_CODES = new Set([ "ERR_MUTABILITY_DOWNGRADE", "ERR_READONLY_ASSIGNED_PRIMARY", "ERR_STORAGE_FLATTENED_ARRAY", + "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE", + "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", + "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP", + "ERR_SEQUENCE_WITHOUT_INCREMENT", + "ERR_SEQUENCE_START_OUT_OF_RANGE", "ERR_PHYSICAL_NAME_KIND_MISMATCH", "ERR_PHYSICAL_NAME_MULTIPLE", "ERR_SQL_BODY_ON_WRITABLE_KIND", diff --git a/scripts/check-requirements-ledger.ts b/scripts/check-requirements-ledger.ts index 7c0506bebd..838533a25e 100644 --- a/scripts/check-requirements-ledger.ts +++ b/scripts/check-requirements-ledger.ts @@ -48,7 +48,7 @@ const EXPECTED_WARNING = "WARN_REQUIREMENT_NOTHING_IMPLEMENTS"; * the neighbour — became `Number("metaobjects")`, i.e. NaN. Every comparison against * NaN is unequal, so the gate failed reporting "expected NaN" and a remedy paragraph * about updating the pin. */ -const DEFAULT_EXPECTED_WARNING_COUNT = 148; +const DEFAULT_EXPECTED_WARNING_COUNT = 150; function expectedWarnings(argv: readonly string[]): number { const flag = argv.find((a) => a.startsWith("--expect-warnings=")); diff --git a/scripts/prerelease-version.mjs b/scripts/prerelease-version.mjs index ac359d4f11..e54510abf0 100644 --- a/scripts/prerelease-version.mjs +++ b/scripts/prerelease-version.mjs @@ -46,7 +46,8 @@ export function releasedVersion({ npmLatest, tags, major }) { * The version this cut is FOR: `{ base, source }`, or throws an Error whose message says why not. * * - A CHANGELOG section newer than the release is unpublished work; the lowest one is the next - * release, and the cut is for it. + * release, and the cut is for it. Only sections on the released major line or the next one + * count: the historical Maven-only [7.x] sections are not this line's work. * - `--base` may name that version, or any version when every section is published. It may not * name an already released version, and it may not SKIP an unpublished section: that is the * mis-numbering this exists to refuse, reached by hand instead of by guess. @@ -61,7 +62,13 @@ export function decideBase({ released, changelog, explicit }) { "Pass --base (the CHANGELOG section being prepared).", ); } - const unpublished = changelog.filter((v) => cmpVer(v, released) > 0).sort(cmpVer); + // Only sections on the released major line or the next one (a major being prepared): the + // CHANGELOG also carries the old Maven-only [7.x] sections, which sort above every 1.x + // number and were proposed as the next base. + const major = Number(released.split(".")[0]); + const unpublished = changelog + .filter((v) => Number(v.split(".")[0]) <= major + 1 && cmpVer(v, released) > 0) + .sort(cmpVer); if (explicit != null) { if (cmpVer(explicit, released) <= 0) throw new Error(`--base ${explicit} is not newer than the released ${released}`); diff --git a/scripts/prerelease.mjs b/scripts/prerelease.mjs index e9a1b284da..db5c71a18f 100644 --- a/scripts/prerelease.mjs +++ b/scripts/prerelease.mjs @@ -9,7 +9,8 @@ // bun run prerelease:publish --iter 5 # pin the iteration number // bun run prerelease:publish --base 0.25.0 # override the in-development version // -// The base version is the lowest CHANGELOG.md `## [x.y.z]` section that is not released yet — +// The base version is the lowest CHANGELOG.md `## [x.y.z]` section on the released major line (or +// the next major) that is not released yet — // the repository's own declaration of what is being worked on — falling back to minor+1 when // every section is released. "Released" is read from public npm and the release tags, never // from package.json (release prep bumps that first). `--base` may override, but never to skip @@ -394,7 +395,11 @@ if (wants("java")) { "examples/showcase/jvm/pom.xml", ]; touch(...poms); - await run(`sed -i 's/${javaReleased.replace(/\./g, "\\.")}/${V.maven}/g' ${poms.map((f) => `'${f}'`).join(" ")}`); + // In JS rather than `sed -i`, whose in-place flag GNU and BSD (macOS) sed spell differently. + for (const f of poms) { + const path = join(ROOT, f); + writeFileSync(path, readFileSync(path, "utf8").replaceAll(javaReleased, V.maven)); + } await run("scripts/check-pom-versions.sh"); ok(`java poms → ${V.maven}`); const settings = join(scratch("m2"), "settings.xml"); diff --git a/scripts/test-prerelease-version.mjs b/scripts/test-prerelease-version.mjs index d417c7a0e5..0b07f458bd 100644 --- a/scripts/test-prerelease-version.mjs +++ b/scripts/test-prerelease-version.mjs @@ -9,6 +9,8 @@ // P2 the iteration counter reads every page of the registry's listing. // P3 a SIGTERM restores the version files, as exit and SIGINT already did, and stops a step that // is still running instead of letting the run publish the remaining registries first. +// P4 only CHANGELOG sections on the released major line or the next one count; the old Maven-only [7.x] sections +// made a cut after 1.2.0 propose 7.20.12. import assert from "node:assert/strict"; import { spawn } from "node:child_process"; import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; @@ -36,6 +38,22 @@ await test("the release tag is the highest plain tag on the npm major line", () assert.equal(latestReleaseTag(["v7.20.12"], 1), null); }); +await test("P4: a section from another major line (the old Maven-only [7.x]) is never the base", () => { + // 1.2.0 released; the CHANGELOG still carries historical Maven-only sections such as [7.20.12], + // which sort above every 1.x number. The cut must be 1.3.0 (nothing on 1.x is unpublished), not 7.20.12. + const md = "## [Unreleased]\n\n## [1.2.0] — 2026-10-30\n\n## [1.1.4]\n\n## [7.20.12] — 2026-08-02\n"; + assert.deepEqual(decideBase({ released: "1.2.0", changelog: changelogVersions(md), explicit: null }), + { base: "1.3.0", source: "guessed from released 1.2.0" }); + // and an explicit --base is not refused for "skipping" a section on another line + assert.equal(decideBase({ released: "1.2.0", changelog: changelogVersions(md), explicit: "1.2.1" }).base, "1.2.1"); +}); + +await test("P4: the NEXT major's section, being prepared, is still the base", () => { + const md = "## [Unreleased]\n\n## [2.0.0]\n\n## [1.2.0] — 2026-10-30\n\n## [7.20.12] — 2026-08-02\n"; + assert.deepEqual(decideBase({ released: "1.2.0", changelog: changelogVersions(md), explicit: null }), + { base: "2.0.0", source: "CHANGELOG.md" }); +}); + await test("P1: release prep's package.json bump does not make 1.1.0 'released'", () => { // package.json says 1.1.0 (the prep PR); npm and the tags say 1.0.13. const released = releasedVersion({ npmLatest: "1.0.13", tags: ["v1.0.13", "v1.1.0-rc.2"], major: 1 }); diff --git a/server/csharp/MetaObjects.Cli.Tests/WireSettingCliTests.cs b/server/csharp/MetaObjects.Cli.Tests/WireSettingCliTests.cs index d3cde2ecb9..c81a6212b0 100644 --- a/server/csharp/MetaObjects.Cli.Tests/WireSettingCliTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/WireSettingCliTests.cs @@ -108,12 +108,12 @@ public void Verify_with_the_wrong_wire_setting_says_the_setting_differs_instead_ var output = stdout + stderr; Assert.Equal(1, exit); Assert.Contains( - "The committed output was generated with \"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\" }; " + - "this run used \"wire\": { \"decimals\": \"native\", \"nullMembers\": \"native\" } " + + "The committed output was generated with \"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\", \"durations\": \"native\" }; " + + "this run used \"wire\": { \"decimals\": \"native\", \"nullMembers\": \"native\", \"durations\": \"native\" } " + "(.metaobjects/config.json; each key defaults to native). Do not regenerate", output); Assert.Contains( - "Declare \"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\" } in .metaobjects/config.json", + "Declare \"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\", \"durations\": \"native\" } in .metaobjects/config.json", output); Assert.DoesNotContain("Run 'dotnet meta gen' to regenerate", output); } diff --git a/server/csharp/MetaObjects.Codegen.Tests/CrossFieldValidationConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/CrossFieldValidationConformanceTests.cs index 8ec4fd1b50..45617482c5 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/CrossFieldValidationConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/CrossFieldValidationConformanceTests.cs @@ -6,6 +6,8 @@ using MetaObjects.Codegen; using MetaObjects.Codegen.Generators; using MetaObjects.Loader; +using MetaObjects.Config; +using MetaObjects.Meta; using Xunit; namespace MetaObjects.Codegen.Tests; @@ -45,7 +47,7 @@ public static IEnumerable Cases() => AllCases().Select(c => new object[] { c.GetProperty("name").GetString()! }); [Fact] - public void Discovers_all_cross_field_cases() => Assert.Equal(24, AllCases().Length); + public void Discovers_all_cross_field_cases() => Assert.Equal(43, AllCases().Length); [Theory] [MemberData(nameof(Cases))] @@ -100,6 +102,10 @@ private static Assembly Compile() var loadResult = new MetaDataLoader().Load( [new FileSource(Path.Combine(CorpusDir(), "meta.cross-field.json"))]); Assert.Empty(loadResult.Errors); + // The corpus pins its wire form in its own .metaobjects/config.json (`wire.durations: + // iso`, the form its duration payloads take); C#'s native default is System.Text.Json's + // own TimeSpan form. + loadResult.Root.WireFormat = NeutralConfig.Read(CorpusDir())?.Wire ?? WireFormat.Default; var ctx = new GenContext { Entities = loadResult.Root.Objects(), diff --git a/server/csharp/MetaObjects.Codegen.Tests/DurationWireTests.cs b/server/csharp/MetaObjects.Codegen.Tests/DurationWireTests.cs new file mode 100644 index 0000000000..f3dbb7d6c5 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/DurationWireTests.cs @@ -0,0 +1,383 @@ +// field.duration — the C# half of the duration wire rule, gated against the shared vectors in +// fixtures/duration-wire/vectors.json. No database: the production wire helper +// (Runtime/DurationWire.cs, the SAME text the entity generator emits into every generated project) +// runs every `wire` vector (the `precision` ones at seven digits: a TimeSpan tick is 100 ns), and +// the emitted text is proven identical to it. Of the `codecs` groups C# has two in production: +// `storedMicros` (the generated value converter on SQLite and MySQL, here) and the Postgres read +// (`postgres` and `postgresRefused`, through the generated NpgsqlInterval mapping, gated against a +// live database in MetaObjects.IntegrationTests/Api/DurationWireGeneratedApiTests). C# never spells +// a duration as Postgres text (Npgsql binds a TimeSpan natively), so `postgresText` is not its codec. + +using System.Reflection; +using System.Text.Json; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using MetaObjects.Codegen.Generators; +using MetaObjects.Codegen.Runtime; +using MetaObjects.Core.Query; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; +using static MetaObjects.Core.Field.FieldConstants; + +namespace MetaObjects.Codegen.Tests; + +public class DurationWireTests +{ + private static JsonElement Vectors() => + JsonDocument.Parse(File.ReadAllText(Path.Combine( + CorpusPaths.RepoRoot(), "fixtures", "duration-wire", "vectors.json"))).RootElement; + + private static JsonElement Wire(string group) => Vectors().GetProperty("wire").GetProperty(group); + + // The `precision` key for this port: the fractional digits a TimeSpan holds. + private const string PrecisionKey = "7"; + + public static TheoryData ParseVectors() + { + var data = new TheoryData(); + foreach (var v in Wire("parse").EnumerateArray()) + data.Add(v.GetProperty("in").GetString()!, v.GetProperty("out").GetString()!); + return data; + } + + public static TheoryData PrecisionVectors() + { + var data = new TheoryData(); + foreach (var v in Wire("precision").EnumerateArray()) + data.Add(v.GetProperty("in").GetString()!, v.GetProperty("out").GetProperty(PrecisionKey).GetString()!); + return data; + } + + public static TheoryData RefusedVectors() + { + var data = new TheoryData(); + foreach (var v in Wire("refused").EnumerateArray()) data.Add(v.GetString()!); + return data; + } + + public sealed class Holder + { + [DurationWire.Iso] public TimeSpan Span { get; set; } + [DurationWire.Iso] public TimeSpan? Maybe { get; set; } + [DurationWire.Iso] public IList Many { get; set; } = new List(); + } + + [Theory, MemberData(nameof(ParseVectors))] + public void An_accepted_duration_parses_to_its_canonical_form(string input, string canonical) + { + Assert.True(DurationWire.TryParse(input, out var span), $"'{input}' was refused"); + Assert.Equal(canonical, DurationWire.Format(span)); + Assert.Equal(canonical, DurationWire.Canonical(input)); + // The canonical form is a fixed point. + Assert.Equal(canonical, DurationWire.Format(DurationWire.Parse(canonical))); + } + + [Theory, MemberData(nameof(PrecisionVectors))] + public void A_fraction_keeps_whole_ticks_and_truncates_the_rest_toward_zero(string input, string canonical) + { + Assert.True(DurationWire.TryParse(input, out var span), $"'{input}' was refused"); + Assert.Equal(canonical, DurationWire.Format(span)); + Assert.Equal(canonical, DurationWire.Format(DurationWire.Parse(canonical))); + } + + [Theory, MemberData(nameof(ParseVectors)), MemberData(nameof(PrecisionVectors))] + public void A_property_reads_the_accepted_form_and_writes_the_canonical_one(string input, string canonical) + { + var h = JsonSerializer.Deserialize($"{{\"Span\":\"{input}\",\"Maybe\":\"{input}\",\"Many\":[\"{input}\"]}}")!; + Assert.Equal($"{{\"Span\":\"{canonical}\",\"Maybe\":\"{canonical}\",\"Many\":[\"{canonical}\"]}}", JsonSerializer.Serialize(h)); + } + + [Theory, MemberData(nameof(RefusedVectors))] + public void A_refused_value_is_not_a_duration(string input) + { + Assert.False(DurationWire.TryParse(input, out _)); + Assert.Null(DurationWire.Canonical(input)); + Assert.Throws(() => DurationWire.Parse(input)); + var json = JsonSerializer.Serialize(input); + Assert.ThrowsAny(() => JsonSerializer.Deserialize($"{{\"Span\":{json}}}")); + Assert.ThrowsAny(() => JsonSerializer.Deserialize($"{{\"Maybe\":{json}}}")); + Assert.ThrowsAny(() => JsonSerializer.Deserialize($"{{\"Many\":[{json}]}}")); + } + + [Theory] + [InlineData("1")] // a number is not a duration + [InlineData("true")] + [InlineData("{}")] + [InlineData("\"01:30:00\"")] // System.Text.Json's own TimeSpan form + [InlineData("\"PT99999999999999999999H\"")] // past what a TimeSpan holds: refused, not wrapped + public void A_non_duration_token_is_refused(string json) => + Assert.ThrowsAny(() => JsonSerializer.Deserialize($"{{\"Span\":{json}}}")); + + [Fact] + public void Null_crosses_as_null_and_every_tick_is_written() + { + Assert.Null(JsonSerializer.Deserialize("{\"Maybe\":null}")!.Maybe); + Assert.Contains("\"Maybe\":null", JsonSerializer.Serialize(new Holder())); + Assert.Equal("PT0.0000009S", DurationWire.Format(TimeSpan.FromTicks(9))); + Assert.Equal("-PT0.0000009S", DurationWire.Format(TimeSpan.FromTicks(-9))); + Assert.Equal("PT0.0000019S", DurationWire.Format(TimeSpan.FromTicks(19))); + Assert.Equal("-PT0.0000019S", DurationWire.Format(TimeSpan.FromTicks(-19))); + Assert.Equal("-PT1M30S", DurationWire.Format(TimeSpan.FromSeconds(-90))); + Assert.Equal(TimeSpan.FromTicks(-19), DurationWire.Parse("-PT0.00000199S")); // truncated toward zero + } + + public static TheoryData StoredMicrosVectors() + { + var data = new TheoryData(); + foreach (var v in Vectors().GetProperty("codecs").GetProperty("storedMicros").EnumerateArray()) + data.Add(v.GetProperty("in").GetString()!, v.GetProperty("out").GetString()!); + return data; + } + + [Theory, MemberData(nameof(StoredMicrosVectors))] + public void On_a_database_without_an_interval_a_duration_is_its_microsecond_count(string duration, string micros) + { + var count = long.Parse(micros, System.Globalization.CultureInfo.InvariantCulture); + Assert.Equal(count, DurationWire.ToMicros(DurationWire.Parse(duration))); + // The column's precision is a microsecond: what reads back is the span to the microsecond. + Assert.Equal(TimeSpan.FromTicks(count * 10), DurationWire.FromMicros(count)); + Assert.Equal(count, DurationWire.ToMicros(DurationWire.FromMicros(count))); + } + + // C#'s own limit — a TimeSpan's range, about 10.7 million days — not a cross-port one: a span + // past it is refused, never wrapped, on parse and on a stored microsecond count. The whole + // range is reachable, to the tick, MinValue (one tick longer than MaxValue) included. + [Fact] + public void A_span_past_a_timespan_is_refused_never_wrapped() + { + Assert.Equal("PT256204778H48M5.4775807S", DurationWire.Format(TimeSpan.MaxValue)); + Assert.Equal("-PT256204778H48M5.4775808S", DurationWire.Format(TimeSpan.MinValue)); + Assert.Equal(TimeSpan.MaxValue, DurationWire.Parse("PT256204778H48M5.4775807S")); + Assert.Equal(TimeSpan.MaxValue, DurationWire.Parse("PT256204778H48M5.47758079999S")); // truncated, not rounded past + Assert.Equal(TimeSpan.MinValue, DurationWire.Parse("-PT256204778H48M5.4775808S")); + Assert.Equal("PT256204778H", DurationWire.Format(DurationWire.Parse("PT256204778H"))); + Assert.Equal("-PT256204778H", DurationWire.Format(DurationWire.Parse("-P10675199DT2H"))); + Assert.Equal("PT256204704H", DurationWire.Format(DurationWire.Parse("P1525028W"))); + foreach (var past in new[] { + "PT256204778H48M5.4775808S", "-PT256204778H48M5.4775809S", "PT256204779H", "P10675200D", + "P1525029W", "P1525028W4D", "-PT256204779H", "PT99999999999999999999S", "P99999999999999999999W" }) + Assert.False(DurationWire.TryParse(past, out _), past); + Assert.Equal(TimeSpan.MaxValue.Ticks / 10, DurationWire.ToMicros(TimeSpan.MaxValue)); + Assert.Throws(() => DurationWire.FromMicros(long.MaxValue)); + Assert.Throws(() => DurationWire.FromMicros(long.MinValue)); + Assert.Equal(-1, DurationWire.ToMicros(TimeSpan.FromTicks(-19))); // truncated toward zero + } + + // ---- filter operands ---------------------------------------------------------------------- + + public sealed class Row { public TimeSpan Wait { get; set; } public TimeSpan? Grace { get; set; } } + + private static FilterParseResult Check(string field, string op, object value) => + EfCoreFilterDispatch.Validate(FilterParseResult.Ok([new FilterPredicate(field, op, value)])); + + [Theory] + [InlineData("wait", "eq", "P1M")] + [InlineData("wait", "eq", "01:30:00")] + [InlineData("grace", "ne", "pt1h")] + [InlineData("grace", "eq", "")] + public void A_malformed_duration_operand_is_invalid_filter_value_naming_the_field(string field, string op, string operand) + { + var r = Check(field, op, operand); + Assert.Equal("invalid_filter_value", r.ErrorEnvelope); + Assert.Equal(field, r.Field); + } + + [Fact] + public void A_good_duration_operand_passes_and_an_in_list_is_checked_element_wise() + { + Assert.Null(Check("wait", "eq", "PT90M").ErrorEnvelope); + Assert.Null(Check("grace", "ne", "-P1DT0.5S").ErrorEnvelope); + Assert.Null(Check("wait", "in", new List { "PT1H", "P2D", "P1W" }).ErrorEnvelope); + Assert.Equal("invalid_filter_value", Check("wait", "in", new List { "PT1H", "P1Y" }).ErrorEnvelope); + } + + [Fact] + public void A_duration_filters_on_the_ordered_band() + { + Assert.Equal(["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"], QueryConstants.OpsForSubType(FIELD_SUBTYPE_DURATION)); + } + + // ---- generated code ------------------------------------------------------------------------- + + private const string Model = """ + { "metadata.root": { "package": "acme::ops", "children": [ + { "object.value": { "name": "Window", "children": [ + { "field.duration": { "name": "length", "@required": true } }, + { "field.duration": { "name": "slack" } }, + { "field.duration": { "name": "steps", "isArray": true } } + ]}}, + { "object.value": { "name": "Limits", "children": [ + { "field.duration": { "name": "cap" } }, + { "field.duration": { "name": "marks", "isArray": true } } + ]}}, + { "object.entity": { "name": "Timer", "children": [ + { "source.rdb": { "@table": "timers" } }, + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "timeout", "@required": true, "@default": "PT90M" } }, + { "field.duration": { "name": "grace", "@filterable": true } }, + { "field.duration": { "name": "backoff", "isArray": true } }, + { "field.object": { "name": "window", "@objectRef": "Window", "@storage": "jsonb" } }, + { "field.object": { "name": "limits", "@objectRef": "Limits", "@storage": "flattened" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}} + ]}} + """; + + private static List Emit(DurationWireForm durations = DurationWireForm.Iso) + { + var r = new MetaDataLoader().Load([new InMemoryStringSource(Model, id: "duration-model.json")]); + Assert.Empty(r.Errors); + r.Root.WireFormat = WireFormat.Default with { Durations = durations }; + var ctx = new GenContext + { + Entities = r.Root.Objects(), Root = r.Root, + Config = new GenConfig { OutDir = Path.Combine(Path.GetTempPath(), "unused-duration"), Namespace = "Acme.Ops" }, + }; + return new IGenerator[] { new EntityGenerator(), new DbContextGenerator(), new FilterAllowlistGenerator(), new RoutesGenerator() } + .SelectMany(g => g.Generate(ctx)).ToList(); + } + + private static Assembly Compile(IEnumerable files) + { + var trees = files.Select(f => CSharpSyntaxTree.ParseText(f.Content, new CSharpParseOptions(LanguageVersion.CSharp12), path: f.Path)); + var comp = CSharpCompilation.Create("duration_" + Guid.NewGuid().ToString("N"), trees, + DbContextCompileTests.BuildReferences(), new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + using var ms = new MemoryStream(); + var emit = comp.Emit(ms); + Assert.True(emit.Success, string.Join("\n", emit.Diagnostics.Where(d => d.Severity == DiagnosticSeverity.Error))); + return Assembly.Load(ms.ToArray()); + } + + [Fact] + public void The_emitted_helper_is_the_production_helper_with_only_the_namespace_changed() + { + var emitted = Emit().Single(f => f.Path == "DurationWire.g.cs"); + var production = File.ReadAllText(Path.Combine( + CorpusPaths.RepoRoot(), "server", "csharp", "MetaObjects.Codegen", "Runtime", "DurationWire.cs")).Replace("\r\n", "\n"); + Assert.Equal(production.Replace("namespace MetaObjects.Codegen.Runtime;", "namespace Acme.Ops;"), emitted.Content.Replace("\r\n", "\n")); + } + + [Fact] + public void A_duration_is_a_TimeSpan_with_the_wire_converter_an_interval_on_Postgres_and_text_elsewhere() + { + var files = Emit(); + var timer = files.Single(f => f.Path == "Timer.g.cs").Content; + Assert.Contains(" [DurationWire.Iso]\n public TimeSpan Timeout { get; set; } = DurationWire.Parse(\"PT90M\");", timer.Replace("\r\n", "\n")); + Assert.Contains("public TimeSpan? Grace { get; set; }", timer); + Assert.Contains("public IList Backoff { get; set; } = new List();", timer); + Assert.DoesNotContain("using MetaObjects", timer); + + var window = files.Single(f => f.Path == "Window.g.cs").Content; + Assert.Contains("[DurationWire.Iso]", window); + + var ctx = files.Single(f => f.Path == "AppDbContext.g.cs").Content.Replace("\r\n", "\n"); + const string Npgsql = "if (Database.ProviderName == \"Npgsql.EntityFrameworkCore.PostgreSQL\")"; + const string Micros = "new Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter(" + + "v => global::Acme.Ops.DurationWire.ToMicros(v), v => global::Acme.Ops.DurationWire.FromMicros(v))"; + // Npgsql: the native interval, read through the generated NpgsqlInterval mapping; anything + // else: an integer count of microseconds. + Assert.Contains( + $" {Npgsql}\n" + + " modelBuilder.Entity().Property(x => x.Timeout).HasColumnType(\"interval\").Metadata.SetTypeMapping(DurationInterval.Mapping);\n" + + " else\n" + + $" modelBuilder.Entity().Property(x => x.Timeout).HasConversion({Micros});", + ctx); + Assert.Contains( + $" {Npgsql}\n" + + " modelBuilder.Entity().PrimitiveCollection(x => x.Backoff).Metadata.SetTypeMapping(DurationInterval.ArrayMapping);\n" + + " else\n" + + $" modelBuilder.Entity().PrimitiveCollection(x => x.Backoff).ElementType().HasConversion({Micros});", + ctx); + Assert.Contains("internal static class DurationInterval", ctx); + Assert.DoesNotContain("using Npgsql", ctx); + // A duration member of a jsonb value object is stored as its canonical string on every provider. + Assert.Contains("b.Property(p => p.Length).HasConversion(new Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter(", ctx); + Assert.Contains("b.PrimitiveCollection(p => p.Steps).ElementType().HasConversion(new Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter(", ctx); + // ...and a duration member of a FLATTENED value object takes the top-level column mapping. + Assert.Contains( + " b.Property(p => p.Cap).HasColumnName(\"limits_cap\");\n" + + $" {Npgsql}\n" + + " b.Property(p => p.Cap).HasColumnType(\"interval\").Metadata.SetTypeMapping(DurationInterval.Mapping);\n" + + " else\n" + + $" b.Property(p => p.Cap).HasConversion({Micros});", + ctx); + Assert.Contains($" b.PrimitiveCollection(p => p.Marks).ElementType().HasConversion({Micros});", ctx); + + var allowlist = files.Single(f => f.Path == "TimerFilterAllowlist.g.cs").Content; + Assert.Contains("[\"grace\"] = new(System.StringComparer.Ordinal) { \"eq\", \"ne\", \"gt\", \"gte\", \"lt\", \"lte\", \"in\", \"isNull\" },", allowlist); + + // The PATCH merge reads a duration through the wire helper, not System.Text.Json's own form. + var routes = files.Single(f => f.Path == "TimerRoutes.g.cs").Content; + Assert.Contains("if (clr == typeof(System.TimeSpan))", routes); + Assert.Contains("DurationWire.TryParseElement(prop.Value, out var __dur)", routes); + } + + // Stand-ins for NpgsqlTypes.NpgsqlInterval: the shape the generated Npgsql mapping reads, and two + // that have drifted from it. + public readonly struct IntervalLike { public int Months { get; } public int Days { get; } public long Time { get; } } + public readonly struct IntervalWithoutMonths { public int Days { get; } public long Time { get; } } + public readonly struct IntervalWithIntTime { public int Months { get; } public int Days { get; } public int Time { get; } } + + [Fact] + public void The_npgsql_mapping_names_what_is_missing_when_the_interval_shape_has_changed() + { + var asm = Compile(Emit()); + var shape = asm.GetType("Acme.Ops.DurationInterval+IntervalShape")!; + var problem = shape.GetMethod("Problem", BindingFlags.NonPublic | BindingFlags.Static)!; + string? Check(Type? t) => (string?)problem.Invoke(null, [t]); + + Assert.Null(Check(typeof(IntervalLike))); + Assert.Contains("no type NpgsqlTypes.NpgsqlInterval was found", Check(null)); + var noMonths = Check(typeof(IntervalWithoutMonths))!; + Assert.Contains("has no Int32 Months property", noMonths); + Assert.Contains("tested with Npgsql 8.0.5", noMonths); + Assert.Contains("has no Int64 Time property", Check(typeof(IntervalWithIntTime))); + Assert.Equal("NpgsqlTypes.NpgsqlInterval, Npgsql", shape.GetField("TypeName", BindingFlags.NonPublic | BindingFlags.Static)!.GetValue(null)); + } + + [Fact] + public void The_generated_entity_compiles_and_crosses_json_in_the_canonical_form() + { + var asm = Compile(Emit()); + var timer = asm.GetType("Acme.Ops.Timer")!; + + var fresh = Activator.CreateInstance(timer)!; + Assert.Equal(TimeSpan.FromMinutes(90), timer.GetProperty("Timeout")!.GetValue(fresh)); // @default + + var row = JsonSerializer.Deserialize( + "{\"Id\":1,\"Timeout\":\"P1DT2H\",\"Grace\":\"-PT90S\",\"Backoff\":[\"PT1S\",\"PT0.5S\"],\"Window\":{\"length\":\"PT5400S\",\"steps\":[\"P1D\"]}}", + timer)!; + var json = JsonSerializer.Serialize(row, timer); + Assert.Contains("\"Timeout\":\"PT26H\"", json); + Assert.Contains("\"Grace\":\"-PT1M30S\"", json); + Assert.Contains("\"Backoff\":[\"PT1S\",\"PT0.5S\"]", json); + // A value object's members cross under their metadata (camelCase) names. + Assert.Contains("\"length\":\"PT1H30M\"", json); + Assert.Contains("\"slack\":null", json); + Assert.Contains("\"steps\":[\"PT24H\"]", json); + + Assert.ThrowsAny(() => JsonSerializer.Deserialize("{\"Timeout\":\"01:30:00\"}", timer)); + Assert.ThrowsAny(() => JsonSerializer.Deserialize("{\"Grace\":\"P1M\"}", timer)); + Assert.ThrowsAny(() => JsonSerializer.Deserialize("{\"Backoff\":[\"P1Y\"]}", timer)); + } + + // wire.durations: native (the default) puts no converter on the property: System.Text.Json's + // own TimeSpan form is the wire form, both ways. + [Fact] + public void Under_native_durations_the_property_has_no_converter_and_crosses_in_the_STJ_form() + { + var files = Emit(DurationWireForm.Native); + Assert.DoesNotContain("[DurationWire.Iso]", files.Single(f => f.Path == "Timer.g.cs").Content); + Assert.DoesNotContain("[DurationWire.Iso]", files.Single(f => f.Path == "Window.g.cs").Content); + var asm = Compile(files); + var timer = asm.GetType("Acme.Ops.Timer")!; + var row = JsonSerializer.Deserialize( + "{\"Id\":1,\"Timeout\":\"1.02:00:00\",\"Grace\":\"-00:01:30\",\"Backoff\":[\"00:00:01\"]}", timer)!; + var json = JsonSerializer.Serialize(row, timer); + Assert.Contains("\"Timeout\":\"1.02:00:00\"", json); + Assert.Contains("\"Grace\":\"-00:01:30\"", json); + Assert.Contains("\"Backoff\":[\"00:00:01\"]", json); + Assert.ThrowsAny(() => JsonSerializer.Deserialize("{\"Timeout\":\"PT1H\"}", timer)); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCliTests.cs b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCliTests.cs new file mode 100644 index 0000000000..0c51bf6f7f --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCliTests.cs @@ -0,0 +1,60 @@ +// A member longer than the project's enums.maxLength is a metadata/config defect: `dotnet meta +// gen` and `verify --codegen` report it by name and fail, the way TypeScript's `meta gen` does, +// never as an unhandled exception and a stack trace. +using MetaObjects.Codegen.Generators; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +[Collection("Console")] +public sealed class EnumLengthCliTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "mo-enum-cli-" + Guid.NewGuid().ToString("N")); + private string Meta => Path.Combine(_root, "metaobjects"); + private string Out => Path.Combine(_root, "gen"); + + public EnumLengthCliTests() + { + Directory.CreateDirectory(Meta); + Directory.CreateDirectory(Path.Combine(_root, ".metaobjects")); + File.WriteAllText(Path.Combine(_root, ".metaobjects", "config.json"), """{ "schema_version": 1, "enums": { "maxLength": 4 } }"""); + File.WriteAllText(Path.Combine(Meta, "meta.acme.json"), """ + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["OPEN", "CLOSED"] } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """); + } + + public void Dispose() { try { Directory.Delete(_root, recursive: true); } catch { } } + + private (int Code, string Text) Run(params string[] args) + { + var (prevOut, prevErr) = (Console.Out, Console.Error); + var (o, e) = (new StringWriter(), new StringWriter()); + Console.SetOut(o); + Console.SetError(e); + try { return (CodegenCli.Run(args, [new DbContextGenerator()]), o + "\n" + e); } + finally { Console.SetOut(prevOut); Console.SetError(prevErr); } + } + + [Fact] + public void Gen_reports_the_member_by_name_and_fails() + { + var (code, text) = Run("gen", Meta, "--out", Out, "--namespace", "Acme.Generated"); + Assert.NotEqual(0, code); + Assert.Contains("field.enum \"Ticket.status\" has the member \"CLOSED\"", text); + } + + [Fact] + public void Verify_codegen_reports_the_member_by_name_and_fails() + { + var (code, text) = Run("verify", "--codegen", Meta, "--out", Out, "--namespace", "Acme.Generated"); + Assert.NotEqual(0, code); + Assert.Contains("field.enum \"Ticket.status\" has the member \"CLOSED\"", text); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCodegenTests.cs new file mode 100644 index 0000000000..d5213eb399 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthCodegenTests.cs @@ -0,0 +1,55 @@ +// A string-stored enum with a column length (its @maxLength, else the project's +// enums.maxLength — EnumLength.ResolveMaxLength) maps with HasMaxLength(n), the VARCHAR(n) +// `meta migrate` creates; with none, the mapping is unchanged. +using MetaObjects.Codegen.Generators; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class EnumLengthCodegenTests +{ + private static string DbContextFor(string statusAttrs, long? projectDefault = null) + { + var model = $$$""" + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["OPEN", "CLOSED"]{{{statusAttrs}}} } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "enum-length.json")]); + Assert.Empty(r.Errors); + if (projectDefault is { } d) r.Root.EnumDefaults = new EnumDefaults(d); + var ctx = new GenContext + { + Entities = r.Root.Objects(), Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated" }, + }; + return new DbContextGenerator().Generate(ctx).Single().Content; + } + + [Fact] + public void An_enum_with_maxLength_maps_with_that_length() + { + Assert.Contains("HasConversion().HasMaxLength(16);", DbContextFor(""", "@maxLength": 16""")); + } + + [Fact] + public void The_project_default_sizes_an_enum_with_no_maxLength() + { + Assert.Contains("HasConversion().HasMaxLength(24);", DbContextFor("", projectDefault: 24)); + } + + [Fact] + public void No_length_anywhere_leaves_the_mapping_unchanged() + { + var dbContext = DbContextFor(""); + Assert.Contains("HasConversion();", dbContext); + Assert.DoesNotContain("HasMaxLength", dbContext); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/EnumLengthConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthConformanceTests.cs new file mode 100644 index 0000000000..fe0a3c2557 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/EnumLengthConformanceTests.cs @@ -0,0 +1,101 @@ +// EnumLengthConformanceTests — runs the shared cross-port enum-length corpus +// (fixtures/enum-length-conformance/) against the C# port. Each case's `enums` block is +// written to a throwaway project's .metaobjects/config.json and read with NeutralConfig (the +// reader every project-aware load uses), put on the loaded meta.enums.json root, and every +// Ticket enum's column length resolved through EnumLength.ResolveMaxLength. Each `overflow` +// case must be refused with ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, and each `invalid` block +// refused when the config is read. +using System.Text.Json; +using MetaObjects; +using MetaObjects.Config; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class EnumLengthConformanceTests +{ + private static string FixtureDir => + Path.Combine(CorpusPaths.RepoRoot(), "fixtures", "enum-length-conformance"); + + private static JsonDocument Corpus() => JsonDocument.Parse(File.ReadAllText(Path.Combine(FixtureDir, "cases.json"))); + + private static JsonElement Entry(string section, string name) + { + using var doc = Corpus(); + return doc.RootElement.GetProperty(section).EnumerateArray() + .Single(c => c.GetProperty("name").GetString() == name).Clone(); + } + + private static TheoryData Names(string section) + { + var data = new TheoryData(); + using var doc = Corpus(); + foreach (var el in doc.RootElement.GetProperty(section).EnumerateArray()) data.Add(el.GetProperty("name").GetString()!); + return data; + } + + public static TheoryData CaseNames() => Names("cases"); + public static TheoryData OverflowNames() => Names("overflow"); + public static TheoryData InvalidNames() => Names("invalid"); + + /// Reads as a project's config.json `enums` block. + private static EnumDefaults ReadEnums(JsonElement enums) + { + var dir = Path.Combine(Path.GetTempPath(), "mo-enums-conf-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(Path.Combine(dir, ".metaobjects")); + var body = enums.ValueKind == JsonValueKind.Null + ? """{ "schema_version": 1 }""" + : $$"""{ "schema_version": 1, "enums": {{enums.GetRawText()}} }"""; + File.WriteAllText(Path.Combine(dir, ".metaobjects", "config.json"), body); + try + { + return NeutralConfig.Read(dir)?.Enums ?? EnumDefaults.Default; + } + finally + { + Directory.Delete(dir, recursive: true); + } + } + + private static MetaField TicketField(EnumDefaults defaults, string name) + { + var result = new MetaDataLoader().Load([new FileSource(Path.Combine(FixtureDir, "meta.enums.json"))]); + Assert.True(result.Errors.Count == 0, string.Join("\n", result.Errors.Select(e => $"{e.Code}: {e.Message}"))); + result.Root.EnumDefaults = defaults; + return result.Root.FindObject("Ticket")!.Fields().Single(f => f.Name == name); + } + + [Theory] + [MemberData(nameof(CaseNames))] + public void Resolves_every_enum_column_length(string name) + { + var c = Entry("cases", name); + var defaults = ReadEnums(c.GetProperty("enums")); + foreach (var p in c.GetProperty("lengths").EnumerateObject()) + { + long? expected = p.Value.ValueKind == JsonValueKind.Null ? null : p.Value.GetInt64(); + Assert.Equal(expected, EnumLength.ResolveMaxLength(TicketField(defaults, p.Name))); + } + } + + [Theory] + [MemberData(nameof(OverflowNames))] + public void Refuses_a_member_longer_than_the_project_default(string name) + { + var o = Entry("overflow", name); + var field = TicketField(ReadEnums(o.GetProperty("enums")), o.GetProperty("field").GetString()!); + var ex = Assert.Throws(() => EnumLength.ResolveMaxLength(field)); + Assert.Equal(ErrorCode.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, ex.Code); + Assert.Contains(o.GetProperty("member").GetString()!, ex.Message); + } + + [Theory] + [MemberData(nameof(InvalidNames))] + public void Refuses_an_invalid_enums_block(string name) + { + var bad = Entry("invalid", name); + Assert.Throws(() => ReadEnums(bad.GetProperty("enums"))); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs b/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs index 5ae9bb5e7c..3beae425ea 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs @@ -886,4 +886,97 @@ public void GenConfig_IncludeNames_defaults_to_false_so_a_bare_config_falls_back // artifact's symbol appears anywhere in the emitted entity. Assert.DoesNotContain("SubscriberNames", src); } + + private const string FlattenedModel = """ + { "metadata.root": { "package": "acme", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } }, + { "field.string": { "name": "postalCode", "@column": "ZIP" } } + ]}}, + { "object.entity": { "name": "Shipment", "children": [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "field.object": { "name": "payload", "@objectRef": "Address" } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + + [Fact] + public void A_flattened_member_gets_a_column_constant_beside_its_field_constants() + { + var content = new NamesGenerator().Generate(Ctx(Load(FlattenedModel))).Single(f => f.Path == "ShipmentNames.g.cs").Content; + + Assert.Contains("public const string ShipToStreetColumn = \"ship_to_street\";", content); + Assert.Contains("public const string ShipToPostalCodeColumn = \"ship_to_ZIP\";", content); + Assert.Contains("public const string ShipToField = \"shipTo\";", content); + Assert.Contains("public const string PayloadColumn = \"payload\";", content); + // A flattened field has no column of its own: no own constant, no own map key, no Obsolete. + Assert.DoesNotContain("ShipToColumn", content); + Assert.DoesNotContain("[\"shipTo\"]", content); + Assert.DoesNotContain("Obsolete", content); + Assert.Contains("[\"shipTo.street\"] = ShipToStreetColumn,", content); + Assert.Contains("[\"shipTo.postalCode\"] = ShipToPostalCodeColumn,", content); + } + + [Fact] + public void A_flattened_member_constant_that_collides_with_a_field_constant_is_left_out_with_a_warning() + { + var model = FlattenedModel.Replace( + "\"@storage\": \"flattened\" }", + "\"@storage\": \"flattened\", \"@flattenedColumnPrefix\": \"st_\" } },\n { \"field.string\": { \"name\": \"shipToStreet\" }"); + var warnings = new List(); + var root = Load(model); + var ctx = new GenContext + { + Entities = root.Objects(), Root = root, Warn = warnings.Add, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = ColumnNamingStrategy.SnakeCase, IncludeNames = true }, + }; + var content = new NamesGenerator().Generate(ctx).Single(f => f.Path == "ShipmentNames.g.cs").Content; + + // The field's constant wins; the model generates, as it did before member constants. + Assert.Contains("public const string ShipToStreetColumn = \"ship_to_street\";", content); + Assert.DoesNotContain("\"st_street\"", content); + Assert.Contains("public const string ShipToPostalCodeColumn = \"st_ZIP\";", content); + Assert.Contains(warnings, w => w.Contains("ShipToStreetColumn")); + } + + [Fact] + public void DbContext_keeps_the_literal_flattened_columns() + { + // An owned (ejected) names generator from before member constants does not emit them, + // so the EF mapping must not reference them. + var dbContext = new DbContextGenerator().Generate(Ctx(Load(FlattenedModel))).Single().Content; + + Assert.Contains("b.Property(p => p.Street).HasColumnName(\"ship_to_street\");", dbContext); + Assert.DoesNotContain("ShipToStreetColumn", dbContext); + } + + [Fact] + public void A_child_that_redeclares_a_flattened_field_as_jsonb_does_not_inherit_its_member_constants() + { + const string model = """ + { "metadata.root": { "package": "acme", "children": [ + { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "street" } } ]}}, + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } } + ]}}, + { "object.entity": { "name": "Order", "extends": "Base", "children": [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + var files = new NamesGenerator().Generate(Ctx(Load(model))).ToList(); + var order = files.Single(f => f.Path == "OrderNames.g.cs").Content; + + // Extending BaseNames would make BaseNames.ShipToStreetColumn a member of OrderNames, + // naming a column `orders` does not have; the artifact is emitted complete instead. + Assert.Contains("public abstract class OrderNames\n", order.Replace("\r\n", "\n")); + Assert.DoesNotContain("ShipToStreetColumn", order); + Assert.Contains("public const string ShipToColumn = \"ship_to\";", order); + } } diff --git a/server/csharp/MetaObjects.Codegen.Tests/NamingConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/NamingConformanceTests.cs index 304daf0902..3755e1179f 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/NamingConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/NamingConformanceTests.cs @@ -10,7 +10,9 @@ // and CSharpNaming.Column — never a reimplementation. Each `invalid` block must be refused // when the config is read. using System.Text.Json; +using MetaObjects; using MetaObjects.Config; +using MetaObjects.Core.Field; using MetaObjects.Loader; using MetaObjects.Meta; using Xunit; @@ -121,14 +123,54 @@ public void Resolves_every_table_and_column_name(string name) foreach (var (key, expectedColumn) in c.Columns) { - var parts = key.Split('.', 2); + var parts = key.Split('.', 3); var obj = root.FindObject(parts[0]); Assert.NotNull(obj); // ADR-0039: Fields() is the resolving accessor (an inherited field must resolve). var field = obj!.Fields().FirstOrDefault(f => f.Name == parts[1]); Assert.NotNull(field); - // CSharpNaming.Column is the exact door the entity/names generators call. - Assert.Equal(expectedColumn, CSharpNaming.Column(field!, naming.Columns)); + if (parts.Length == 2) + { + // CSharpNaming.Column is the exact door the entity/names generators call. + Assert.Equal(expectedColumn, CSharpNaming.Column(field!, naming.Columns)); + continue; + } + // `..`: a member of the flattened value object the field + // embeds, named by core naming, the door the DbContext generator calls. + var target = NamingRefs.RefTarget(field!, FieldConstants.FIELD_ATTR_OBJECT_REF) as MetaObject; + Assert.NotNull(target); + var member = target!.Fields().FirstOrDefault(f => f.Name == parts[2]); + Assert.NotNull(member); + Assert.Equal(expectedColumn, PhysicalNaming.ResolveFlattenedColumnName(field!, member!, naming.Columns)); + } + } + + /// + /// The names artifact names every column in the corpus the way the resolvers above do: a + /// scalar field's Column, and each member of a flattened value object under its + /// field's Members. + /// + [Theory] + [MemberData(nameof(ValidCaseNames))] + public void Names_artifact_carries_every_column(string name) + { + var c = LoadValidCase(name); + var naming = ReadNaming(c.Naming); + var root = LoadModel(naming); + + foreach (var (key, expectedColumn) in c.Columns) + { + var parts = key.Split('.', 3); + var names = CSharpNaming.ResolveObjectNames(root.FindObject(parts[0])!, naming.Columns); + Assert.NotNull(names); + var field = names!.Fields[parts[1]]; + if (parts.Length == 2) + { + Assert.Equal(expectedColumn, field.Column); + continue; + } + Assert.NotNull(field.Members); + Assert.Equal(expectedColumn, field.Members![parts[2]].Column); } } diff --git a/server/csharp/MetaObjects.Codegen.Tests/ObjectFieldCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ObjectFieldCodegenTests.cs index a704ee3c23..9a36fe8c73 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ObjectFieldCodegenTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ObjectFieldCodegenTests.cs @@ -96,6 +96,75 @@ public void Referenced_value_object_is_emitted_as_a_plain_poco() Assert.DoesNotContain("[Key]", address); } + // @flattenedColumnPrefix replaces the default `_` prefix, verbatim ("" keeps the members' + // own columns), and the default prefix follows the project's column rule: the names + // `meta migrate` creates for the same model (core naming's ResolveFlattenedColumnPrefix). + [Theory] + [InlineData(ColumnNamingStrategy.Literal, "street", "Officestreet", "homeAddress_street")] + [InlineData(ColumnNamingStrategy.UpperFirst, "Street", "OfficeStreet", "HomeAddress_Street")] + public void DbContext_flattened_columns_follow_column_prefix_and_the_column_rule( + ColumnNamingStrategy strategy, string plain, string prefixed, string byDefault) + { + const string model = """ + { "metadata.root": { "package": "acme", "children": [ + { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "street" } } ]}}, + { "object.entity": { "name": "Customer", "children": [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "mailing", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "" } }, + { "field.object": { "name": "work", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "Office" } }, + { "field.object": { "name": "homeAddress", "@objectRef": "Address", "@storage": "flattened" } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "prefix.json")]); + Assert.Empty(r.Errors); + var ctx = new GenContext + { + Entities = r.Root.Objects(), Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = strategy }, + }; + var dbContext = new DbContextGenerator().Generate(ctx).Single().Content; + + Assert.Contains($"b.Property(p => p.Street).HasColumnName(\"{plain}\");", dbContext); + Assert.Contains($"b.Property(p => p.Street).HasColumnName(\"{prefixed}\");", dbContext); + Assert.Contains($"b.Property(p => p.Street).HasColumnName(\"{byDefault}\");", dbContext); + } + + // Each flattened member's column is core naming's, whole: the generator does not build the + // member half itself. A member whose @column is "" is the case that told them apart: core + // treats an empty @column as absent, so the column is `bill_street`, not `bill_`. + [Fact] + public void DbContext_flattened_member_columns_are_core_namings() + { + const string model = """ + { "metadata.root": { "package": "acme", "children": [ + { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "street", "@column": "" } } ]}}, + { "object.entity": { "name": "Customer", "children": [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "billing", "@objectRef": "Address", "@storage": "flattened", "@column": "bill" } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "member.json")]); + Assert.Empty(r.Errors); + var ctx = new GenContext + { + Entities = r.Root.Objects(), Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = ColumnNamingStrategy.SnakeCase }, + }; + var dbContext = new DbContextGenerator().Generate(ctx).Single().Content; + + var billing = r.Root.FindObject("Customer")!.Fields().Single(f => f.Name == "billing"); + var street = r.Root.FindObject("Address")!.Fields().Single(f => f.Name == "street"); + var expected = PhysicalNaming.ResolveFlattenedColumnName(billing, street, ColumnNamingStrategy.SnakeCase); + Assert.Equal("bill_street", expected); + Assert.Contains($"b.Property(p => p.Street).HasColumnName(\"{expected}\");", dbContext); + } + [Fact] public void DbContext_configures_owned_types_flattened_and_json() { diff --git a/server/csharp/MetaObjects.Codegen.Tests/ReportExportGeneratorTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ReportExportGeneratorTests.cs index 2c3fe6fe11..66313734d7 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ReportExportGeneratorTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ReportExportGeneratorTests.cs @@ -35,9 +35,25 @@ public class ReportExportGeneratorTests ] } } """; - private static MetaRoot Load() + // The same shop with a field.duration dimension on the served report. + private const string DurationModel = """ + { "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Line", "children": [ + { "source.rdb": { "@table": "lines" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.duration": { "name": "wait" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "wait", "@of": "Line.wait" } }, + { "measure.aggregate": { "name": "lines", "@agg": "count", "@of": "Line.id" } } + ] } }, + { "object.report": { "name": "WaitTotals", "@from": "Line", "@dimensions": ["wait"], "@measures": ["lines"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_wait_totals" } } ] } } + ] } } + """; + + private static MetaRoot Load(string model = Model) { - var r = new MetaDataLoader().Load([new InMemoryStringSource(Model, id: "meta.shop.json")]); + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "meta.shop.json")]); Assert.True(r.Errors.Count == 0, string.Join("\n", r.Errors.Select(e => e.Message))); return r.Root; } @@ -130,9 +146,9 @@ public static string Xml(object?[][] rows, string[] cols) } """; - private static MethodInfo[] Compile(ReportExportGenerator gen, string? decimals = null) + private static MethodInfo[] Compile(ReportExportGenerator gen, string? decimals = null, string model = Model) { - var root = Load(); + var root = Load(model); if (decimals == "string") root.WireFormat = new WireFormat(DecimalWireForm.String, NullMemberForm.Native); var writer = gen.Generate(Ctx(root)).Single(f => f.Path == "ReportExportWriter.g.cs").Content; var tpa = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!).Split(Path.PathSeparator) @@ -150,6 +166,32 @@ private static MethodInfo[] Compile(ReportExportGenerator gen, string? decimals private static string Csv(MethodInfo[] h, object?[][] rows, params string[] cols) => (string)h[0].Invoke(null, [rows, cols, null])!; private static string Xml(MethodInfo[] h, object?[][] rows, params string[] cols) => (string)h[1].Invoke(null, [rows, cols])!; + private static string Writer(string model) => + new ReportExportGenerator().Generate(Ctx(Load(model))).Single(f => f.Path == "ReportExportWriter.g.cs").Content; + + // field.duration: the writer of a model with no duration column is the writer from before + // durations existed, byte for byte (the hash of that writer for this model, pinned). + [Fact] + public void The_writer_of_a_model_without_a_duration_column_is_unchanged() + { + var writer = Writer(Model); + Assert.DoesNotContain("TimeSpan ts", writer); + Assert.DoesNotContain("duration", writer, StringComparison.OrdinalIgnoreCase); + Assert.DoesNotContain("@DURATION", writer); + var hash = Convert.ToHexString(System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(writer.Replace("\r\n", "\n")))); + Assert.Equal("78913C645BB9D25C334F2285C210D0192510618A63C564AA7833C8AEA4A0B5AF", hash); + } + + [Fact] + public void A_duration_column_is_written_in_its_canonical_form_and_never_guarded() + { + Assert.Contains("case TimeSpan ts: return new(Duration(ts), false);", Writer(DurationModel)); + var h = Compile(new ReportExportGenerator(), model: DurationModel); + Assert.Equal("w\r\nPT26H\r\n-PT1M30S\r\nPT0.5S\r\nPT0.0000005S\r\nPT0S\r\n-PT256204778H48M5.4775808S\r\n\r\n", + Csv(h, [[TimeSpan.FromHours(26)], [TimeSpan.FromSeconds(-90)], [TimeSpan.FromMilliseconds(500)], [TimeSpan.FromTicks(5)], [TimeSpan.Zero], [TimeSpan.MinValue], [null]], "w")); + Assert.Contains("-PT1M30S", Xml(h, [[TimeSpan.FromSeconds(-90)]], "w")); + } + [Fact] public void Csv_quotes_ends_every_record_with_crlf_and_guards_text_cells_only() { diff --git a/server/csharp/MetaObjects.Codegen.Tests/ValidationConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ValidationConformanceTests.cs index b2724518e0..88343c77af 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ValidationConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ValidationConformanceTests.cs @@ -48,7 +48,7 @@ public static IEnumerable Cases() [Fact] public void Discovers_all_validation_conformance_cases() { - Assert.Equal(42, Cases().Count()); + Assert.Equal(54, Cases().Count()); } [Theory] @@ -117,6 +117,8 @@ private static Assembly Compile() new FileSource(Path.Combine(CorpusDir(), "meta.json")), ]); Assert.Empty(loadResult.Errors); + // The corpus cases state durations in ISO-8601, the cross-port wire form. + loadResult.Root.WireFormat = WireFormat.Default with { Durations = DurationWireForm.Iso }; var ctx = new GenContext { diff --git a/server/csharp/MetaObjects.Codegen.Tests/WireConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/WireConformanceTests.cs index a2f320e423..f06081b5b3 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/WireConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/WireConformanceTests.cs @@ -68,6 +68,7 @@ public void Resolves_each_wire_block(string name) var wire = ReadWire(c.GetProperty("wire")); Assert.Equal(c.GetProperty("decimals").GetString(), WireFormat.Spelling(wire.Decimals)); Assert.Equal(c.GetProperty("nullMembers").GetString(), WireFormat.Spelling(wire.NullMembers)); + Assert.Equal(c.GetProperty("durations").GetString(), WireFormat.Spelling(wire.Durations)); } [Theory] @@ -82,7 +83,7 @@ public void Refuses_invalid_wire_blocks(string name) [Fact] public void A_project_without_the_setting_loads_with_the_native_default() { - Assert.Equal(new WireFormat(DecimalWireForm.Native, NullMemberForm.Native), WireFormat.Default); + Assert.Equal(new WireFormat(DecimalWireForm.Native, NullMemberForm.Native, DurationWireForm.Native), WireFormat.Default); var r = new MetaObjects.Loader.MetaDataLoader().Load([new MetaObjects.Loader.InMemoryStringSource( """{ "metadata.root": { "package": "acme", "children": [] } }""", id: "empty.json")]); Assert.Equal(WireFormat.Default, r.Root.WireFormat); diff --git a/server/csharp/MetaObjects.Codegen/CSharpNaming.cs b/server/csharp/MetaObjects.Codegen/CSharpNaming.cs index 29e96e4ed4..d8e99fa656 100644 --- a/server/csharp/MetaObjects.Codegen/CSharpNaming.cs +++ b/server/csharp/MetaObjects.Codegen/CSharpNaming.cs @@ -18,8 +18,16 @@ namespace MetaObjects.Codegen; -/// Physical name + logical field name for one field (§A2/§A3 names artifact). -public sealed record FieldNames(string Name, string Column); +/// +/// Physical name + logical field name for one field (§A2/§A3 names artifact). +/// is set only on a @storage: "flattened" value object, +/// and there it is the real column information: such a field has no column of its own, and +/// each member of the value object is a column of the owning table, named by core naming +/// (, +/// the function meta migrate names them with), keyed by the member's field name. +/// is null on a flattened field: it names no column of the table. +/// +public sealed record FieldNames(string Name, string? Column, IReadOnlyDictionary? Members = null); /// /// One source.rdb child, under the ROLE it plays. @@ -203,11 +211,14 @@ public static class CSharpNaming // the Postgres-native inet type). Both are string-backed on the wire. [FIELD_SUBTYPE_URI] = "Uri", [FIELD_SUBTYPE_INET] = "IPAddress", + // field.duration binds to the native System.TimeSpan value type (DB column is the + // Postgres-native interval, a BIGINT count of microseconds elsewhere); string-backed (ISO-8601) on the wire. + [FIELD_SUBTYPE_DURATION] = "TimeSpan", }; /// Value types that take a ? suffix when nullable (vs. reference types). private static readonly HashSet ValueTypes = new(StringComparer.Ordinal) - { "int", "long", "double", "float", "decimal", "bool", "DateOnly", "TimeOnly", "DateTime", "DateTimeOffset", "Guid" }; + { "int", "long", "double", "float", "decimal", "bool", "DateOnly", "TimeOnly", "DateTime", "DateTimeOffset", "Guid", "TimeSpan" }; /// /// ADR-0036 Wave 3 — the CANONICAL hostname matcher for @stringFormat: hostname, @@ -442,6 +453,7 @@ public static string Column(MetaField field, ColumnNamingStrategy strategy = Col ColumnNamingStrategy.Literal => name, ColumnNamingStrategy.SnakeCase => ToSnakeCase(name), ColumnNamingStrategy.KebabCase => ToSnakeCase(name).Replace('_', '-'), + ColumnNamingStrategy.UpperFirst => PhysicalNaming.ToUpperFirstColumn(name), _ => name, }; @@ -513,11 +525,11 @@ public static bool HasPrimarySource(MetaObject obj) => // ADR-0039: Fields() is the RESOLVING accessor — an inherited @column must // resolve here, or the constant disagrees with the column EF actually binds. foreach (var f in obj.Fields()) - fields[f.Name] = new FieldNames(f.Name, Column(f, strategy)); + fields[f.Name] = FieldNamesOf(f, strategy); // ADR-0039's sanctioned own-accessor use: what this artifact DECLARES. var ownFields = new Dictionary(StringComparer.Ordinal); foreach (var f in obj.OwnFields()) - ownFields[f.Name] = new FieldNames(f.Name, Column(f, strategy)); + ownFields[f.Name] = FieldNamesOf(f, strategy); var superObj = NamesArtifactSuperOf(obj); // Identity of the resolved source NODE, not equality of the resolved strings: a @@ -719,10 +731,10 @@ private static bool DeclaresNamesContent(MetaObject obj) => var fields = new Dictionary(StringComparer.Ordinal); foreach (var f in obj.Fields()) - fields[f.Name] = new FieldNames(f.Name, Column(f, strategy)); + fields[f.Name] = FieldNamesOf(f, strategy); var ownFields = new Dictionary(StringComparer.Ordinal); foreach (var f in own) - ownFields[f.Name] = new FieldNames(f.Name, Column(f, strategy)); + ownFields[f.Name] = FieldNamesOf(f, strategy); return new ObjectNames { @@ -810,6 +822,28 @@ public static string SourceMemberName(string role, string member) => public static string KeyMemberName(string type, string keyName, string member) => Pascal(type) + PascalToken(keyName) + PascalToken(member); + /// The constant member naming a flattened member's column: ShipToStreetColumn. + public static string FlattenedColumnMemberName(string fieldName, string memberName) => + FieldMemberName(fieldName + Pascal(memberName), MEMBER_COLUMN); + + /// + /// One field's names. The flattened test is migrate's own: a field.object whose + /// resolved @storage is flattened. A flattened field whose @objectRef + /// does not resolve has no members; the loader refuses that model before codegen runs. + /// + public static FieldNames FieldNamesOf(MetaField f, ColumnNamingStrategy strategy) + { + if (f.SubType != FIELD_SUBTYPE_OBJECT || f.Storage != STORAGE_FLATTENED) + return new FieldNames(f.Name, Column(f, strategy)); + var members = new Dictionary(StringComparer.Ordinal); + if (NamingRefs.RefTarget(f, FIELD_ATTR_OBJECT_REF) is MetaObject vo) + { + foreach (var m in vo.Fields()) + members[m.Name] = new FieldNames(m.Name, PhysicalNaming.ResolveFlattenedColumnName(f, m, strategy)); + } + return new FieldNames(f.Name, null, members); + } + /// One field member: EmailField / EmailColumn. See /// for why this stays on . public static string FieldMemberName(string fieldName, string member) => diff --git a/server/csharp/MetaObjects.Codegen/CodegenCli.cs b/server/csharp/MetaObjects.Codegen/CodegenCli.cs index 8bb057b73a..a393994dcf 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenCli.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenCli.cs @@ -269,7 +269,7 @@ public static GenOutcome RunGen( // A bad template ref / wrong --template-root surfaces as a clean error, not a stack trace. return new GenOutcome([$"{RenderFailedPrefix}{ex.Message}"], null); } - catch (Exception ex) when (ex is ArgumentException or InvalidOperationException) + catch (Exception ex) when (ex is ArgumentException or InvalidOperationException or MetaModelException) { // An output-pattern error or a duplicate output-path collision surfaces lazily // during the walk — a clean error, not a stack trace. @@ -482,6 +482,7 @@ private static int Unknown(string cmd) { ColumnNamingStrategy.SnakeCase => "snake_case", ColumnNamingStrategy.KebabCase => "kebab-case", + ColumnNamingStrategy.UpperFirst => "upperFirst", _ => "literal", }; @@ -809,12 +810,12 @@ private static int RunGenArgs(string[] rest, IReadOnlyList declared) /// Every naming other than , in the order verify probes them (the /// Python port probes the same order): the used table rule first, so a different column /// rule alone is tried before a different table rule; within each, the used column rule - /// first, then snake_case, literal, kebab-case. + /// first, then snake_case, literal, kebab-case, upperFirst. /// private static IEnumerable NamingCandidates(PhysicalNaming used) { ColumnNamingStrategy[] columns = - [ColumnNamingStrategy.SnakeCase, ColumnNamingStrategy.Literal, ColumnNamingStrategy.KebabCase]; + [ColumnNamingStrategy.SnakeCase, ColumnNamingStrategy.Literal, ColumnNamingStrategy.KebabCase, ColumnNamingStrategy.UpperFirst]; TableNamingRule[] tables = [TableNamingRule.SnakeCase, TableNamingRule.Literal]; foreach (var t in tables.OrderBy(t => t == used.Tables ? 0 : 1)) foreach (var c in columns.OrderBy(c => c == used.Columns ? 0 : 1)) @@ -844,7 +845,7 @@ private static string NamingDiffers(PhysicalNaming committed, PhysicalNaming use /// /// The other wire setting (ADR-0019 Amendment 2) that reproduces the committed output - /// exactly, or null. Candidates are every combination of the two keys other than + /// exactly, or null. Candidates are every combination of the three keys other than /// , native before string and native before emit before omit; /// emit is never offered, because in C# it generates what native does. /// @@ -854,14 +855,15 @@ private static string NamingDiffers(PhysicalNaming committed, PhysicalNaming use var generated = used.NullMembers == NullMemberForm.Emit ? used with { NullMembers = NullMemberForm.Native } : used; foreach (var decimals in new[] { DecimalWireForm.Native, DecimalWireForm.String }) foreach (var nulls in new[] { NullMemberForm.Native, NullMemberForm.Omit }) - { - var candidate = new WireFormat(decimals, nulls); - if (candidate == generated) continue; - var root = load(); - root.WireFormat = candidate; - var probe = CodegenDrift.Compute(config, root, suite, narrowed); - if (probe.Error is null && probe.Clean) return candidate; - } + foreach (var durations in new[] { DurationWireForm.Native, DurationWireForm.Iso }) + { + var candidate = new WireFormat(decimals, nulls, durations); + if (candidate == generated) continue; + var root = load(); + root.WireFormat = candidate; + var probe = CodegenDrift.Compute(config, root, suite, narrowed); + if (probe.Error is null && probe.Clean) return candidate; + } return null; } @@ -871,7 +873,7 @@ private static string NamingDiffers(PhysicalNaming committed, PhysicalNaming use private static string WireDiffers(WireFormat committed, WireFormat used) => $"The committed output was generated with {committed.Block()}; this run used {used.Block()} " + "(.metaobjects/config.json; each key defaults to native). Do not regenerate: that changes the " + - "JSON your API writes for decimals or for a value object's null members. Declare " + + "JSON your API writes for decimals, durations or a value object's null members. Declare " + $"{committed.Block()} in .metaobjects/config.json, and gen and verify --codegen both read it."; private static int RunVerifyArgs(string[] rest, IReadOnlyList declared) diff --git a/server/csharp/MetaObjects.Codegen/CodegenDrift.cs b/server/csharp/MetaObjects.Codegen/CodegenDrift.cs index 29711710e4..98b698ab07 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenDrift.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenDrift.cs @@ -98,7 +98,7 @@ public static Result Compute(GenConfig config, MetaRoot root, IReadOnlyList temps[roots.FindIndex(r => r.Committed == Path.GetFullPath(realRootFor(g)))]); } - catch (Exception ex) when (ex is ArgumentException or InvalidOperationException) + catch (Exception ex) when (ex is ArgumentException or InvalidOperationException or MetaModelException) { return new Result { Clean = false, Error = $"verify --codegen: {ex.Message}" }; } diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index aa604b2b1f..06b8fbb41f 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -151,6 +151,8 @@ public virtual IEnumerable Generate(GenContext ctx) // creates (ADR-0015). foreach (var f in p.Fields().Where(f => f.SubType == FIELD_SUBTYPE_MAP)) modelLines.Add(MapJsonbConfig(name, p, f, ctx)); + // A duration column on a view reads through the same provider-dependent mapping. + AddDurationConfig(name, p, p.Fields(), ctx, modelLines); } foreach (var e in objects.Where(o => o.IsEntity() && !o.IsReadOnlyProjection())) { @@ -275,6 +277,7 @@ public virtual IEnumerable Generate(GenContext ctx) // did not model jsonbObjectsOnly suppressing the flattened arm on a read model. if (modelLines.Any(l => l.Contains(MapJsonbHelperName))) EmitMapJsonbHelper(sb); sb.AppendLine("}"); + if (modelLines.Any(l => l.Contains(DurationIntervalHelperName + "."))) EmitDurationIntervalHelper(sb); return [new EmittedFile("AppDbContext.g.cs", sb.ToString())]; } @@ -926,7 +929,13 @@ private static bool NeedsUnmappedEnumHelper(IEnumerable objects, Met private static string EnumConversionCall(string owner, MetaObject entity, MetaField f, GenConfig config) { var intMap = IntValueMapOf(f); - if (intMap is null) return "HasConversion()"; + // A string-backed enum's column length (EnumLength.ResolveMaxLength: its @maxLength, else + // the project's enums.maxLength) is the VARCHAR(n) `meta migrate` creates. An array enum + // is a text[] column, whose elements carry no length. + if (intMap is null) + return !f.ResolvedIsArray() && EnumLength.ResolveMaxLength(f) is { } length + ? $"HasConversion().HasMaxLength({length})" + : "HasConversion()"; var members = f.EffectiveEnumValues ?? new List(); if (members.Count == 0) return "HasConversion()"; @@ -1081,10 +1090,220 @@ private void EmitFieldTypeConfig( $" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasColumnType(\"inet\");"); } + // field.duration — a System.TimeSpan. Npgsql maps it to the native `interval` (and an array + // to `interval[]`), reading an interval with a month or year part as an error. Every other + // provider stores the canonical ISO-8601 string in a text column (the TS-owned SQLite / MySQL + // schemas), through the entity namespace's DurationWire; EF's own TimeSpan text ("01:30:00") + // would not be the cross-port form, and MySQL's `time` cannot hold more than 838 hours. + AddDurationConfig(className, entity, fieldList, ctx, modelLines); + foreach (var f in fieldList.Where(f => !f.ResolvedIsArray() && f.DbColumnType is not null)) if (DbColumnTypeConfig(className, f) is { } cfg) modelLines.Add(cfg); } + // field.duration — a System.TimeSpan. Npgsql maps it to the native `interval` (and an array + // to `interval[]`), reading an interval with a month or year part as an error. Every other + // provider stores a 64-bit count of microseconds in an integer column (the TS-owned SQLite / + // MySQL schemas), through the entity namespace's DurationWire; EF's own TimeSpan text + // ("01:30:00") does not order as the span, and MySQL's `time` cannot hold more than 838 hours. + private static void AddDurationConfig( + string className, MetaObject entity, IEnumerable fields, GenContext ctx, List modelLines) + { + foreach (var f in fields.Where(f => f.SubType == FIELD_SUBTYPE_DURATION && f.DbColumnType is null)) + modelLines.Add(string.Join("\n", + DurationColumnLines($"modelBuilder.Entity<{className}>()", "x", CSharpNaming.Pascal(f.Name), f.ResolvedIsArray(), entity, ctx, " "))); + } + + // The provider-dependent mapping of one duration column (a top-level property, or a member of a + // flattened value object through its owned builder `target`): + // - Npgsql: the native `interval` (`interval[]` for an array), read through the generated + // DurationInterval type mapping, so a month or year part, or a day count past what a TimeSpan + // holds, is refused instead of read wrong (Npgsql's own TimeSpan read drops a negative month + // part and wraps such a day count). + // - any other provider (SQLite, MySQL): a 64-bit count of microseconds in an integer column + // (sub-microsecond ticks truncated toward zero; a count past a TimeSpan refused, not + // wrapped), which orders and compares as the span does. An array holds the counts. + private static IEnumerable DurationColumnLines( + string target, string v, string prop, bool isArray, MetaObject owner, GenContext ctx, string indent) + { + var micros = DurationMicrosConverter(owner, ctx); + yield return $"{indent}if (Database.ProviderName == \"{NpgsqlProviderName}\")"; + yield return isArray + ? $"{indent} {target}.PrimitiveCollection({v} => {v}.{prop}).Metadata.SetTypeMapping({DurationIntervalHelperName}.ArrayMapping);" + : $"{indent} {target}.Property({v} => {v}.{prop}).HasColumnType(\"interval\").Metadata.SetTypeMapping({DurationIntervalHelperName}.Mapping);"; + yield return $"{indent}else"; + yield return isArray + ? $"{indent} {target}.PrimitiveCollection({v} => {v}.{prop}).ElementType().HasConversion({micros});" + : $"{indent} {target}.Property({v} => {v}.{prop}).HasConversion({micros});"; + } + + // The namespace OWNER's DurationWire is emitted into (owner has a duration field, so it has one). + private static string DurationWireRef(MetaObject owner, GenContext ctx) => + "global::" + PackageBindingResolver.Resolve(ctx.Config, PackageBindingResolver.EffectivePackage(owner), owner.Name, fallbackContext: owner.Name) + + ".DurationWire"; + + // A TimeSpan <-> canonical ISO-8601 string converter: a duration inside a JSON document. + private static string DurationConverter(MetaObject owner, GenContext ctx) + { + var dw = DurationWireRef(owner, ctx); + return "new Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter(" + + $"v => {dw}.Format(v), v => {dw}.Parse(v))"; + } + + // A TimeSpan <-> microsecond-count converter: a duration column on a database without an interval type. + private static string DurationMicrosConverter(MetaObject owner, GenContext ctx) + { + var dw = DurationWireRef(owner, ctx); + return "new Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter(" + + $"v => {dw}.ToMicros(v), v => {dw}.FromMicros(v))"; + } + + private const string DurationIntervalHelperName = "DurationInterval"; + + // field.duration on Npgsql — the interval type mapping the duration columns are read through. + // Npgsql's own TimeSpan read guards the month part with `months > 0` (so '-1 mons' reads as zero) + // and adds the day part unchecked (so a day count past a TimeSpan wraps); this mapping reads the + // value as NpgsqlInterval instead and refuses both. Npgsql is reached by reflection only, so the + // generated context compiles, and runs, in a project with no Npgsql reference; the class is first + // touched in the Npgsql branch of OnModelCreating. Emitted after the context class: the two + // mapping types must be public and top-level for a compiled model (`dotnet ef dbcontext + // optimize`), which re-creates a property's mapping as `.Default.Clone(...)` and + // refuses a nested or non-public one. DurationIntervalEfCoreTests (integration) gates LINQ, + // interval arithmetic and a compiled model against a live Postgres. + private static void EmitDurationIntervalHelper(StringBuilder sb) => sb.AppendLine(DurationIntervalHelperSource); + + private const string DurationIntervalHelperSource = """ + +/// +/// field.duration on Npgsql: an interval column read through NpgsqlInterval, so a month +/// or year part, or a day count past what a TimeSpan holds, is refused rather than read wrong. +/// Npgsql is reached by reflection: a project on another provider needs no Npgsql reference. +/// +internal static class DurationInterval +{ + // The Npgsql shape this mapping reads, resolved and checked when the model is built (the + // first touch is SetTypeMapping in OnModelCreating, or the first query on a compiled model), + // so a missing or changed type fails there, naming what is missing, rather than on a row. + internal static readonly System.Type IntervalType = Resolve(); + private static readonly System.Reflection.PropertyInfo Months = IntervalType.GetProperty("Months")!; + private static readonly System.Reflection.PropertyInfo Days = IntervalType.GetProperty("Days")!; + private static readonly System.Reflection.PropertyInfo Time = IntervalType.GetProperty("Time")!; + internal static readonly System.Reflection.MethodInfo GetFieldValue = + typeof(System.Data.Common.DbDataReader).GetMethod(nameof(System.Data.Common.DbDataReader.GetFieldValue))!; + + internal static readonly Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping Mapping = DurationIntervalMapping.Default; + internal static readonly Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping ArrayMapping = DurationIntervalArrayMapping.Default; + + private static System.Type Resolve() + { + var type = System.Type.GetType(IntervalShape.TypeName, throwOnError: false); + if (IntervalShape.Problem(type) is { } problem) throw new System.InvalidOperationException(problem); + return type!; + } + + /// The check of the NpgsqlInterval shape, kept apart from the state above so it can run against any type. + internal static class IntervalShape + { + internal const string TypeName = "NpgsqlTypes.NpgsqlInterval, Npgsql"; + + /// Null when has the shape the mapping reads, else a message naming what is missing. + internal static string? Problem(System.Type? type) + { + const string Lead = "field.duration on Npgsql: the generated DurationInterval mapping reads an interval column as " + + "NpgsqlTypes.NpgsqlInterval (an Int32 Months, an Int32 Days and an Int64 Time in microseconds; " + + "tested with Npgsql 8.0.5), but "; + if (type is null) return Lead + "no type NpgsqlTypes.NpgsqlInterval was found in the loaded Npgsql assembly."; + var missing = new System.Collections.Generic.List(); + foreach (var (name, want) in new[] { ("Months", typeof(int)), ("Days", typeof(int)), ("Time", typeof(long)) }) + { + var property = type.GetProperty(name); + if (property is null || property.PropertyType != want) + missing.Add(want.Name + " " + name); + } + return missing.Count == 0 + ? null + : Lead + type.FullName + " has no " + string.Join(", no ", missing) + " property. Check the Npgsql version the project references."; + } + } + + /// An NpgsqlInterval as a TimeSpan: refused when it has a month or year part, or a span a TimeSpan cannot hold. + public static System.TimeSpan FromInterval(object interval) + { + if ((int)Months.GetValue(interval)! != 0) + throw new System.InvalidCastException("field.duration: the interval has a month or year part, so it is not a fixed-length span"); + var days = (int)Days.GetValue(interval)!; + var micros = (long)Time.GetValue(interval)!; + return System.TimeSpan.FromTicks(checked(days * System.TimeSpan.TicksPerDay + micros * 10)); + } + + /// An interval[] (an NpgsqlInterval array) as the list a duration array property holds. + public static System.Collections.Generic.IList FromIntervals(object intervals) + { + var list = new System.Collections.Generic.List(); + foreach (var interval in (System.Array)intervals) list.Add(FromInterval(interval!)); + return list; + } + + // A span as Postgres interval input: [-]HH:MM:SS[.f], hours unbounded, no day part. The column + // holds microseconds, so sub-microsecond ticks are truncated toward zero. + internal static string Literal(System.TimeSpan value) + { + long micros = value.Ticks / 10; + bool negative = micros < 0; + ulong rest = negative ? (ulong)(-(micros + 1)) + 1UL : (ulong)micros; + ulong frac = rest % 1_000_000UL, s = rest / 1_000_000UL % 60, m = rest / 60_000_000UL % 60, h = rest / 3_600_000_000UL; + var text = h.ToString("00", System.Globalization.CultureInfo.InvariantCulture) + ":" + m.ToString("00", System.Globalization.CultureInfo.InvariantCulture) + + ":" + s.ToString("00", System.Globalization.CultureInfo.InvariantCulture); + if (frac > 0) text += "." + frac.ToString("D6", System.Globalization.CultureInfo.InvariantCulture).TrimEnd('0'); + return "INTERVAL '" + (negative ? "-" : "") + text + "'"; + } +} + +/// The type mapping of a field.duration column on Npgsql (an interval, read through ). +public sealed class DurationIntervalMapping : Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping +{ + public static DurationIntervalMapping Default { get; } = new(); + + public DurationIntervalMapping() : base("interval", typeof(System.TimeSpan)) { } + private DurationIntervalMapping(RelationalTypeMappingParameters parameters) : base(parameters) { } + protected override Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping Clone(RelationalTypeMappingParameters parameters) => new DurationIntervalMapping(parameters); + public override System.Reflection.MethodInfo GetDataReaderMethod() => DurationInterval.GetFieldValue.MakeGenericMethod(DurationInterval.IntervalType); + public override System.Linq.Expressions.Expression CustomizeDataReaderExpression(System.Linq.Expressions.Expression expression) => + System.Linq.Expressions.Expression.Call(typeof(DurationInterval).GetMethod(nameof(DurationInterval.FromInterval))!, + System.Linq.Expressions.Expression.Convert(expression, typeof(object))); + protected override string GenerateNonNullSqlLiteral(object value) => DurationInterval.Literal((System.TimeSpan)value); +} + +/// The type mapping of an @isArray field.duration column on Npgsql (an interval[]). +public sealed class DurationIntervalArrayMapping : Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping +{ + private static readonly Microsoft.EntityFrameworkCore.ChangeTracking.ValueComparer> ListComparer = new( + (a, b) => a == null ? b == null : b != null && System.Linq.Enumerable.SequenceEqual(a, b), + v => System.Linq.Enumerable.Aggregate(v, 0, (h, e) => System.HashCode.Combine(h, e)), + v => System.Linq.Enumerable.ToList(v)); + + // After ListComparer: static initializers run in declaration order. The element mapping is the + // scalar one, which is what lets EF query the column as a primitive collection (Contains, Count, + // Any): a compiled model carries only what the mapping itself names. + public static DurationIntervalArrayMapping Default { get; } = new(); + + public DurationIntervalArrayMapping() : base(new RelationalTypeMappingParameters( + new CoreTypeMappingParameters(typeof(System.Collections.Generic.IList), comparer: ListComparer, keyComparer: ListComparer, + elementMapping: DurationIntervalMapping.Default), "interval[]")) { } + private DurationIntervalArrayMapping(RelationalTypeMappingParameters parameters) : base(parameters) { } + protected override Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping Clone(RelationalTypeMappingParameters parameters) => new DurationIntervalArrayMapping(parameters); + public override System.Reflection.MethodInfo GetDataReaderMethod() => DurationInterval.GetFieldValue.MakeGenericMethod(DurationInterval.IntervalType.MakeArrayType()); + public override System.Linq.Expressions.Expression CustomizeDataReaderExpression(System.Linq.Expressions.Expression expression) => + System.Linq.Expressions.Expression.Call(typeof(DurationInterval).GetMethod(nameof(DurationInterval.FromIntervals))!, + System.Linq.Expressions.Expression.Convert(expression, typeof(object))); + protected override string GenerateNonNullSqlLiteral(object value) => + "ARRAY[" + string.Join(",", System.Linq.Enumerable.Select((System.Collections.Generic.IEnumerable)value, DurationInterval.Literal)) + "]::interval[]"; +} +"""; + + // The EF Core provider name Npgsql registers (DbContext.Database.ProviderName). + private const string NpgsqlProviderName = "Npgsql.EntityFrameworkCore.PostgreSQL"; + // True when a value object declares a field.map, or holds a value object that does. private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet seen) { @@ -1099,7 +1318,7 @@ private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet s } // Owned-type config for an object-typed entity field. @storage flattened maps each - // nested scalar to "{parentCol}_{nestedCol}"; every other storage collapses to one + // nested scalar to its core-naming flattened column (PhysicalNaming.ResolveFlattenedColumnName); every other storage collapses to one // json column (.ToJson) — matching the TS-owned schema DDL. null (with a warning) // when @objectRef can't be resolved. is the emitted CLR class // name (the write entity, or a write-through entity's View read model — #214). @@ -1118,7 +1337,6 @@ private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet s } var strategy = ctx.Config.ColumnNamingStrategy; var nav = CSharpNaming.Pascal(field.Name); - var parentCol = CSharpNaming.Column(field, strategy); // A6 -- the field itself belongs to `entity` (the JSON column's own physical // name, not the nested value object's), so it hits entity's own names artifact // unconditionally in practice — a lookup, not an assumption, per ColumnRef. @@ -1164,14 +1382,14 @@ private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet s ? $" modelBuilder.Entity<{owner}>().OwnsOne(x => x.{nav}, b => b.ToJson({parentColRef}));" : OwnedJsonBlock("OwnsOne", owner, nav, parentColRef, jsonEnums); - // Flattened-column prefix for each nested member: `_`. This is - // migrate-ts's rule (expected-schema.ts flattenObjectField), and TS owns the schema - // (ADR-0015) — so it is the convention, NOT EF's, whose own owned-type default is - // `` + // A NESTED value object: migrate gives it ONE json column at `` // (subtypeToSqlType returns { kind: "json" } for field.object — a nested VO does // not flatten recursively). EF models it as a nested owned type, so it needs its // own ToJson pinned to that column, plus the enum conversions its own members @@ -1241,7 +1459,7 @@ private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet s } // A `field.map` member: the migration flattens it to a jsonb column at - // `_`, so it takes the SAME converter/comparer pair a top-level map + // ``, so it takes the SAME converter/comparer pair a top-level map // takes, pinned to that name. Previously this fell to a warning whose stated // reason was that the port configured a top-level map nowhere either — that is // no longer true, and an unmapped dictionary here binds EF's `
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1."); + + /// @constraintName on a field.enum — the database name of its membership CHECK. + public static readonly AttrSchema EnumConstraintNameSchema = new AttrSchema( + Name: Core.Validator.ValidatorConstants.VALIDATOR_ATTR_CONSTRAINT_NAME, + ValueType: AttrConstants.ATTR_SUBTYPE_STRING, + Required: false, + Description: "The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1."); + + /// @sequenceStart — the first value of an increment key's sequence; on identity.primary. + public static readonly AttrSchema SequenceStartSchema = new AttrSchema( + Name: DbConstants.IDENTITY_ATTR_SEQUENCE_START, + ValueType: AttrConstants.ATTR_SUBTYPE_LONG, + Required: false, + Description: "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1."); + + /// @sequenceName — the physical name of an increment key's sequence; on identity.primary. + public static readonly AttrSchema SequenceNameSchema = new AttrSchema( + Name: DbConstants.IDENTITY_ATTR_SEQUENCE_NAME, + ValueType: AttrConstants.ATTR_SUBTYPE_STRING, + Required: false, + Description: "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1."); + /// @constraintName — physical FK constraint-name override; on identity.reference. public static readonly AttrSchema ConstraintNameSchema = new AttrSchema( Name: DbConstants.IDENTITY_ATTR_CONSTRAINT_NAME, diff --git a/server/csharp/MetaObjects/RegistryManifest.cs b/server/csharp/MetaObjects/RegistryManifest.cs index b59670f15d..8f0ee03576 100644 --- a/server/csharp/MetaObjects/RegistryManifest.cs +++ b/server/csharp/MetaObjects/RegistryManifest.cs @@ -375,7 +375,7 @@ private static ManifestType ToManifestType(TypeDefinition def) => /// unstable (semver major-0). The 1.0 cut flips this to "1.0". /// Emitted as the FIRST top-level key in the registry manifest (C4). /// - public const string MetamodelVersion = "1.2"; + public const string MetamodelVersion = "1.3"; /// /// Emit the canonical registry manifest as a byte-stable JSON string. diff --git a/server/csharp/MetaObjects/SpecMetamodel/db.json b/server/csharp/MetaObjects/SpecMetamodel/db.json index 44c1048d73..4e087e72f0 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/db.json +++ b/server/csharp/MetaObjects/SpecMetamodel/db.json @@ -21,6 +21,7 @@ "type": "field", "subType": "object", "children": [ + { "type": "attr", "subType": "string", "name": "flattenedColumnPrefix", "min": 0, "max": 1, "description": "On a field.object with @storage \"flattened\": the text put before each member's column, verbatim (\"\" allowed, so the members keep their own column names). Absent: this field's resolved column name and \"_\" (with @column \"bill\", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "string", "name": "storage", "min": 0, "max": 1, "allowedValues": ["flattened", "jsonb", "subdocument"], "description": "Storage strategy for an object-typed field (set with @objectRef). \"flattened\" expands the nested value into prefixed columns on the parent table. \"jsonb\" stores the structured value in a single jsonb column (supports isArray=true for arrays of values). \"subdocument\" is a hint for document-store codegen targets and emits no Postgres column." } ] }, @@ -58,6 +59,28 @@ { "type": "attr", "subType": "string", "name": "using", "min": 0, "max": 1, "description": "Index access method (e.g. \"gin\", \"gist\", \"hash\"); default \"btree\" (not rendered). Pair with @expr for e.g. a GIN index over an array/jsonb expression. RDB-physical — contributed by the db provider." } ] }, + { + "type": "field", + "subType": "enum", + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": ["numeric", "length", "regex", "comparison", "requiredWhen", "presentIff", "atLeastOne", "exactlyOne"], + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "identity", + "subType": "primary", + "children": [ + { "type": "attr", "subType": "long", "name": "sequenceStart", "min": 0, "max": 1, "description": "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." }, + { "type": "attr", "subType": "string", "name": "sequenceName", "min": 0, "max": 1, "description": "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, { "type": "identity", "subType": "reference", diff --git a/server/csharp/MetaObjects/SpecMetamodel/field.json b/server/csharp/MetaObjects/SpecMetamodel/field.json index b054bda765..3b4a4ce9fc 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/field.json +++ b/server/csharp/MetaObjects/SpecMetamodel/field.json @@ -151,6 +151,7 @@ "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "int", "name": "maxLength", "min": 0, "max": 1, "description": "Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, @@ -183,6 +184,14 @@ "children": [ { "type": "attr", "subType": "boolean", "name": "lenient", "min": 0, "max": 1, "description": "#234: opt this field out of strict well-formedness enforcement. When true, codegen binds a plain string (no URL/IP validator, no native URI/InetAddress type; field.inet uses a text column, not the native inet type) so a not-strictly-valid value round-trips unchanged. Default (absent/false) is strict: an absolute-scheme URI / an IPv4-or-IPv6 literal." } ] + }, + { + "type": "field", + "subType": "duration", + "dataType": "string", + "extendsBase": true, + "description": "A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.", + "whenToUse": "A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span (\"every 3 months\") is not a duration: model it as an int and a unit." } ] } diff --git a/server/csharp/MetaObjects/SpecMetamodel/validator.json b/server/csharp/MetaObjects/SpecMetamodel/validator.json index 9520605bd7..4aba88dbd3 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/validator.json +++ b/server/csharp/MetaObjects/SpecMetamodel/validator.json @@ -66,7 +66,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "left", "min": 1, "max": 1, "description": "Name of the left-hand field of the owning entity." }, { "type": "attr", "subType": "string", "name": "op", "min": 1, "max": 1, "allowedValues": ["gt", "gte", "lt", "lte", "ne", "eq"], "description": "Relational operator: gt (>), gte (>=), lt (<), lte (<=), ne (<>), eq (=)." }, - { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." } + { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -78,7 +79,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field that becomes required when the condition holds." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field whose value triggers the requirement." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -90,7 +92,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field whose presence is governed by the condition." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -100,7 +103,19 @@ "whenToUse": "At least one of several fields must be present (email or phone). Declare it instead of hand-written OR-presence checks.", "rules": "@fields names two or more fields of the owning entity. Satisfied when any one of them is non-null.", "children": [ - { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." } + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": "exactlyOne", + "description": "Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.", + "whenToUse": "A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.", + "rules": "@fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.", + "children": [ + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the alternative fields; exactly one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] } ] diff --git a/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xRoundTripTest.kt b/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xRoundTripTest.kt index 2917c2d509..4cc7d4a19a 100644 --- a/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xRoundTripTest.kt +++ b/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xRoundTripTest.kt @@ -59,6 +59,8 @@ class Exposed1xRoundTripTest { { "field.uri": { "name": "homepage", "@required": true } }, { "field.inet": { "name": "address", "@required": true } }, { "field.string": { "name": "externalRef", "@required": true, "@dbColumnType": "uuid" } }, + { "field.duration": { "name": "holdFor", "@required": true } }, + { "field.duration": { "name": "retries", "@required": true, "isArray": true } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "uuid" } } ] }} @@ -102,12 +104,19 @@ class Exposed1xRoundTripTest { // property is String, never java.util.UUID). Lowercase + canonical dashed // form, matching what java.util.UUID.toString() always produces. val wantExternalRef = "9d3b7f2a-5c4e-4a1b-8e6f-1a2b3c4d5e6f" + // field.duration: a negative span with microseconds (Postgres refuses a signed + // ISO-8601 duration as input, so this proves the interval-spelling bind) and an + // INTERVAL[] array including a span past a day. + val wantHoldFor = java.time.Duration.ofSeconds(-90, -500_000_000) + val wantRetries = listOf(java.time.Duration.ofHours(26), java.time.Duration.ofNanos(1_000)) val insertedId = RoundTripProbeTable.insert { it[RoundTripProbeTable.createdAt] = wantCreatedAt it[RoundTripProbeTable.homepage] = wantHomepage it[RoundTripProbeTable.address] = wantAddress it[RoundTripProbeTable.externalRef] = wantExternalRef + it[RoundTripProbeTable.holdFor] = wantHoldFor + it[RoundTripProbeTable.retries] = wantRetries }[RoundTripProbeTable.id] val row = RoundTripProbeTable.selectAll() @@ -119,6 +128,12 @@ class Exposed1xRoundTripTest { // Statically typed String (not java.util.UUID) — this line would not even // COMPILE against the pre-fix native uuid(...)/javaUUID(...) Column. val gotExternalRef: String = row[RoundTripProbeTable.externalRef] + val gotHoldFor = row[RoundTripProbeTable.holdFor] + val gotRetries = row[RoundTripProbeTable.retries] + // An equality filter on the interval column binds through the same CAST marker. + val byHoldFor = RoundTripProbeTable.selectAll() + .where { RoundTripProbeTable.holdFor eq wantHoldFor } + .count() val problems = mutableListOf() if (gotCreatedAt != wantCreatedAt) problems += "createdAt: want=${'$'}wantCreatedAt got=${'$'}gotCreatedAt" @@ -130,6 +145,10 @@ class Exposed1xRoundTripTest { problems += "externalRef: want=${'$'}wantExternalRef got=${'$'}gotExternalRef" } + if (gotHoldFor != wantHoldFor) problems += "holdFor: want=${'$'}wantHoldFor got=${'$'}gotHoldFor" + if (gotRetries != wantRetries) problems += "retries: want=${'$'}wantRetries got=${'$'}gotRetries" + if (byHoldFor != 1L) problems += "holdFor eq filter: want=1 got=${'$'}byHoldFor" + if (problems.isEmpty()) "OK" else "MISMATCH: " + problems.joinToString("; ") } } diff --git a/server/java/codegen-kotlin/pom.xml b/server/java/codegen-kotlin/pom.xml index 723a221397..da709813b4 100644 --- a/server/java/codegen-kotlin/pom.xml +++ b/server/java/codegen-kotlin/pom.xml @@ -94,6 +94,14 @@ 1.6.0 test + + + org.postgresql + postgresql + 42.7.4 + test + KotlinEnumEmitter.kt KotlinDecimalWireEmitter.kt + KotlinDurationWireEmitter.kt KotlinExtractSchemaEmitter.kt KotlinOutputFormatSpecEmitter.kt KotlinCrossFieldRules.kt diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt index 554ef431b3..3d81165da4 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt @@ -132,6 +132,9 @@ private val DECIMAL_WIRE_EMIT: List = listOf(KotlinDecimalWireEmitter::c * TPH per-subtype create. */ private val CROSS_FIELD_EMIT: List = listOf(KotlinCrossFieldRules::class.java.name) +/** The duration wire file `entity`, `routes` and `exposed-table` each write beside their own output. */ +private val DURATION_WIRE_EMIT: List = listOf(KotlinDurationWireEmitter::class.java.name) + /** The rest of `api-docs`: the page IR (one file, `KotlinApiModel.kt`, which also holds the unit, symbol, field * and kind types), the builder that fills it, the renderer and the page paths. */ private val API_DOCS_EMIT: List = listOf( @@ -162,7 +165,7 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( layer = GeneratorLayer.MODEL, factory = ::KotlinEntityGenerator, ejectResourcePath = ejectPath("KotlinEntityGenerator"), - ejectWith = ENUM_EMIT + DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT, + ejectWith = ENUM_EMIT + DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT + DURATION_WIRE_EMIT, ), "routes" to GeneratorInfo( name = "routes", @@ -171,7 +174,7 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( layer = GeneratorLayer.API, factory = ::KotlinSpringControllerGenerator, ejectResourcePath = ejectPath("KotlinSpringControllerGenerator"), - ejectWith = DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT, + ejectWith = DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT + DURATION_WIRE_EMIT, requires = listOf("entity", "exposed-table", "filter-allowlist", "relations"), ), "repository" to GeneratorInfo( @@ -248,7 +251,7 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinExposedTableGenerator, ejectResourcePath = ejectPath("KotlinExposedTableGenerator"), - ejectWith = DECIMAL_WIRE_EMIT, + ejectWith = DECIMAL_WIRE_EMIT + DURATION_WIRE_EMIT, requires = listOf("entity"), ), "relations" to GeneratorInfo( diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinCrossFieldRules.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinCrossFieldRules.kt index a3560c8a3e..030cb6a58a 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinCrossFieldRules.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinCrossFieldRules.kt @@ -7,13 +7,15 @@ import com.metaobjects.field.CurrencyField import com.metaobjects.field.DateField import com.metaobjects.field.DecimalField import com.metaobjects.field.DoubleField +import com.metaobjects.field.DurationField +import com.metaobjects.field.DurationWire +import com.metaobjects.field.EnumField import com.metaobjects.field.FloatField +import com.metaobjects.field.InetField import com.metaobjects.field.IntegerField import com.metaobjects.field.LongField -import com.metaobjects.field.EnumField import com.metaobjects.field.MetaField import com.metaobjects.field.StringField -import com.metaobjects.field.InetField import com.metaobjects.field.TimeField import com.metaobjects.field.TimestampField import com.metaobjects.field.UriField @@ -21,17 +23,133 @@ import com.metaobjects.field.UuidField import com.metaobjects.`object`.MetaObject import com.metaobjects.validator.AtLeastOneValidator import com.metaobjects.validator.ComparisonValidator +import com.metaobjects.validator.ExactlyOneValidator import com.metaobjects.validator.MetaValidator import com.metaobjects.validator.PresentIffValidator import com.metaobjects.validator.RequiredWhenValidator import com.squareup.kotlinpoet.CodeBlock /** - * Entity-scoped cross-field validators (comparison / requiredWhen / presentIff / atLeastOne) as - * Kotlin boolean expressions. Read by the entity data class's `isXxxValid` properties and by the + * Entity-scoped cross-field validators (comparison / requiredWhen / presentIff / atLeastOne / + * exactlyOne, each with an optional `@condition`) as Kotlin boolean expressions. Read by the entity data class's `isXxxValid` properties and by the * TPH per-subtype create handler, which validates the union body rather than the subtype class. */ object KotlinCrossFieldRules { + private enum class CompareKind { NUMBER, DURATION, PLAIN } + + private fun compareKind(f: MetaField<*>?): CompareKind = when { + f == null -> CompareKind.PLAIN + isNumericCross(f) -> CompareKind.NUMBER + f is DurationField -> CompareKind.DURATION + else -> CompareKind.PLAIN + } + + /** The field a value node compares as: itself, or a coalesce's first field. */ + private fun nodeField(node: Any?, fields: Map>): MetaField<*>? { + val n = node as? Map<*, *> ?: return null + (n["field"] as? String)?.let { return fields[it] } + if (n["fn"] == "coalesce") { + for (a in (n["args"] as? Collection<*>).orEmpty()) nodeField(a, fields)?.let { return it } + } + return null + } + + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + private fun nodeFields(node: Any?, fields: Map>): List> { + val n = node as? Map<*, *> ?: return emptyList() + (n["field"] as? String)?.let { return listOfNotNull(fields[it]) } + if (n["fn"] == "coalesce") return (n["args"] as? Collection<*>).orEmpty().flatMap { nodeFields(it, fields) } + return emptyList() + } + + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + private fun conditionComparable(left: Any?, right: Any?, op: String, fields: Map>): Boolean { + val all = nodeFields(left, fields) + nodeFields(right, fields) + return all.indices.all { i -> (i + 1 until all.size).all { j -> comparable(all[i], all[j], op) } } + } + + /** A value node (field, literal, coalesce) as a Kotlin expression compared as [kind], or null. */ + private fun conditionValue( + node: Any?, kind: CompareKind, fields: Map>, ref: (MetaField<*>) -> String, + ): CodeBlock? { + val n = node as? Map<*, *> ?: return null + (n["field"] as? String)?.let { name -> + val f = fields[name] ?: return null + if (f.isArrayType) return null + val r = ref(f) + return when (kind) { + CompareKind.NUMBER -> CodeBlock.of("(%L?.let·{·java.math.BigDecimal(it.toString())·})", r) + CompareKind.DURATION -> CodeBlock.of("%L", r) + CompareKind.PLAIN -> CodeBlock.of("(%L?.toString())", r) + } + } + if (n.containsKey("value")) { + val text = n["value"]?.toString() ?: return null + return when (kind) { + CompareKind.NUMBER -> CodeBlock.of("java.math.BigDecimal(%S)", text) + CompareKind.DURATION -> try { + val d = DurationWire.parse(text) + CodeBlock.of("java.time.Duration.ofSeconds(%LL,·%LL)", d.seconds, d.nano) + } catch (e: RuntimeException) { + null + } + CompareKind.PLAIN -> CodeBlock.of("%S", text) + } + } + if (n["fn"] == "coalesce") { + val args = (n["args"] as? Collection<*>).orEmpty().map { conditionValue(it, kind, fields, ref) ?: return null } + return CodeBlock.builder().apply { + add("listOf(") + args.forEachIndexed { i, a -> if (i > 0) add(",·"); add("%L", a) } + add(").firstOrNull·{·it·!=·null·}") + }.build() + } + return null + } + + /** A condition node as a Kotlin `Boolean?` expression (null = SQL unknown), or null when unresolvable. */ + private fun conditionExpr( + node: Any?, fields: Map>, ref: (MetaField<*>) -> String, + ): CodeBlock? { + val n = node as? Map<*, *> ?: return null + val op = n["op"] as? String ?: return null + val kop = when (op) { + "gt" -> ">"; "gte" -> ">="; "lt" -> "<"; "lte" -> "<="; "ne" -> "!="; "eq" -> "==" + else -> null + } + if (kop != null) { + if (!conditionComparable(n["left"], n["right"], op, fields)) return null + val kind = compareKind(nodeField(n["left"], fields) ?: nodeField(n["right"], fields)) + val l = conditionValue(n["left"], kind, fields, ref) ?: return null + val r = conditionValue(n["right"], kind, fields, ref) ?: return null + return CodeBlock.of( + "((%L)·as·Comparable?)?.let·{·l_·->·(%L)?.let·{·r_·->·l_.compareTo(r_)·$kop·0·}·}", l, r, + ) + } + when (op) { + "isNull", "isNotNull" -> { + val a = conditionValue(n["arg"], CompareKind.PLAIN, fields, ref) ?: return null + return CodeBlock.of("(%L·${if (op == "isNull") "==" else "!="}·null)", a) + } + "not" -> { + val a = conditionExpr(n["arg"], fields, ref) ?: return null + return CodeBlock.of("(%L)?.not()", a) + } + "and", "or" -> { + val args = (n["args"] as? Collection<*>).orEmpty().map { conditionExpr(it, fields, ref) ?: return null } + val decisive = if (op == "and") "false" else "true" + val other = if (op == "and") "true" else "false" + return CodeBlock.builder().apply { + add("listOf(") + args.forEachIndexed { i, a -> if (i > 0) add(",·"); add("%L", a) } + add(").let·{·v_·->·if·($decisive·in·v_)·$decisive·else·if·(null·in·v_)·null·else·$other·}") + }.build() + } + } + return null + } + private fun isNumericCross(f: MetaField<*>): Boolean = f is IntegerField || f is LongField || f is DoubleField || f is FloatField || f is DecimalField || f is CurrencyField @@ -53,24 +171,34 @@ object KotlinCrossFieldRules { /** * The comparison for two fields, or null when the pair has no sound one. Mirrors the C# - * generator: numerics compare with each other, strings and temporals only with their own + * generator: numerics compare with each other, strings, temporals and durations only with their own * subtype and Kotlin type (a date has no ordering against a timestamp, nor an Instant against a * `@localTime` LocalDateTime, and casting both to Comparable throws * ClassCastException inside the validity property, turning a create into a 500), and * boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, like every other port. */ - private fun comparison(l: MetaField<*>, r: MetaField<*>, op: String, kop: String, ref: (MetaField<*>) -> String): CodeBlock? { + private fun orderableCross(f: MetaField<*>): Boolean = + isNumericCross(f) || f is StringField || isTemporalCross(f) || f is DurationField + + private fun equatableCross(f: MetaField<*>): Boolean = + f is BooleanField || f is EnumField || f is UuidField || f is UriField || f is InetField + + /** Whether the pair has a sound comparison (see [comparison]); emits nothing. */ + private fun comparable(l: MetaField<*>, r: MetaField<*>, op: String): Boolean { val ordering = op == "gt" || op == "gte" || op == "lt" || op == "lte" val same = l.subType == r.subType - if (isNumericCross(l) && isNumericCross(r) && !same) { + if (isNumericCross(l) && isNumericCross(r) && !same) return true + if (!same) return false + if (!orderableCross(l) && !(!ordering && equatableCross(l))) return false + return nativeShape(l) == nativeShape(r) + } + + private fun comparison(l: MetaField<*>, r: MetaField<*>, op: String, kop: String, ref: (MetaField<*>) -> String): CodeBlock? { + if (!comparable(l, r, op)) return null + if (isNumericCross(l) && isNumericCross(r) && l.subType != r.subType) { return CodeBlock.of("java.math.BigDecimal(%L.toString()).compareTo(java.math.BigDecimal(%L.toString())) $kop 0", ref(l), ref(r)) } - if (!same) return null - val comparable = isNumericCross(l) || l is StringField || isTemporalCross(l) - val equatable = !ordering && (l is BooleanField || l is EnumField || l is UuidField || l is UriField || l is InetField) - if (!comparable && !equatable) return null - if (nativeShape(l) != nativeShape(r)) return null - if (comparable) { + if (orderableCross(l)) { return CodeBlock.of("(%L as Comparable).compareTo(%L) $kop 0", ref(l), ref(r)) } return CodeBlock.of("(%L %L %L)", ref(l), if (op == "eq") "==" else "!=", ref(r)) @@ -129,7 +257,8 @@ object KotlinCrossFieldRules { if (v is RequiredWhenValidator) CodeBlock.of("!%L || %L != null", gate, ref(t)) else CodeBlock.of("(%L != null) == %L", ref(t), gate) } - is AtLeastOneValidator -> { + is AtLeastOneValidator, is ExactlyOneValidator -> { + // atLeastOne and exactlyOne both name @fields; they differ only in the count they allow. val raw = if (v.hasMetaAttr(AtLeastOneValidator.ATTR_FIELDS)) v.getMetaAttr(AtLeastOneValidator.ATTR_FIELDS).value else null val names = when (raw) { @@ -139,12 +268,25 @@ object KotlinCrossFieldRules { } if (names.isEmpty() || names.any { it !in fields }) continue CodeBlock.builder().apply { - names.forEachIndexed { i, n -> if (i > 0) add(" || "); add("%L != null", ref(fields.getValue(n))) } + if (v is AtLeastOneValidator) { + names.forEachIndexed { i, n -> if (i > 0) add(" || "); add("%L != null", ref(fields.getValue(n))) } + } else { + add("listOf(") + names.forEachIndexed { i, n -> if (i > 0) add(", "); add("%L", ref(fields.getValue(n))) } + add(").count·{·it·!=·null·}·==·1") + } }.build() } else -> continue } - out.add(v to expr) + // The CHECK is NOT (condition) OR (rule): the rule binds only when the condition is TRUE under SQL + // three-valued logic; FALSE and NULL (unknown) let the record through. + if (v.hasMetaAttr(MetaValidator.ATTR_CONDITION)) { + val cond = conditionExpr(v.getMetaAttr(MetaValidator.ATTR_CONDITION).value, fields, ref) ?: continue + out.add(v to CodeBlock.of("(%L)·!=·true·||·(%L)", cond, expr)) + } else { + out.add(v to expr) + } } return out } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireEmitter.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireEmitter.kt new file mode 100644 index 0000000000..5ae52ab1aa --- /dev/null +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireEmitter.kt @@ -0,0 +1,203 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.generator.util.GeneratedFileWriter +import java.nio.file.Path + +/** + * Writes the per-package `MetaDurationWire.kt` support file: the `field.duration` wire rule + * (`fixtures/duration-wire/`) the generated entity, table and controller share, in generated + * source, so the generated code imports nothing from MetaObjects at runtime. Mirrors + * [KotlinDecimalWireEmitter]. + * + * A `java.time.Duration` is written in the canonical ISO-8601 form `[-]PT[nH][nM][n[.f]S]` and read + * from the accepted form only; `Duration.parse` takes more (fractional hours, per-component signs, + * lowercase) and `Duration.toString()` differs from the canonical form for a negative span, so + * neither is the wire form. A value keeps nanoseconds (a `java.time.Duration`'s precision); the + * stored forms truncate to microseconds, which is what a Postgres interval and the BIGINT column hold. + * + * On Postgres, pgjdbc hands an interval column (scalar or array element) to Exposed as a + * `PGInterval`, so that object's text is the only interval spelling the helper reads; one with a + * year or month part is refused, never converted. + * + * The content is fixed, so several writers of the same path agree byte for byte. + */ +object KotlinDurationWireEmitter { + + /** File name (sans `.kt`) and the name of the `internal object` it declares. */ + const val FILE_NAME = "MetaDurationWire" + + /** Write `MetaDurationWire.kt` into [pkg] under [outRoot]. */ + fun write(pkg: String, outRoot: Path) { + val header = if (pkg.isEmpty()) "" else "package $pkg\n\n" + val outFile = outRoot.resolve(pkg.replace('.', '/')).resolve("$FILE_NAME.kt") + GeneratedFileWriter.write(outFile, header + BODY + "\n") + } + + /** + * The fixed body. `+`-concatenated messages and no Kotlin string templates (every `$` below is + * a regex anchor followed by `)` or `"`), so this raw string needs no `$` escaping. Depends on + * Jackson only, which a generated entity already imports. + */ + val BODY: String = """ + |import com.fasterxml.jackson.core.JsonGenerator + |import com.fasterxml.jackson.core.JsonParser + |import com.fasterxml.jackson.core.JsonToken + |import com.fasterxml.jackson.databind.DeserializationContext + |import com.fasterxml.jackson.databind.JsonNode + |import com.fasterxml.jackson.databind.SerializerProvider + |import com.fasterxml.jackson.databind.deser.std.StdDeserializer + |import com.fasterxml.jackson.databind.ser.std.StdSerializer + |import java.time.Duration + | + |/** + | * GENERATED — do not hand-edit. The field.duration wire rule: a duration is written as the + | * canonical ISO-8601 string [-]PT[nH][nM][n[.f]S] (hours never folded into days, up to nine + | * fractional digits, zero as PT0S) and read from the accepted form only: an optional sign, P, an + | * optional week count (W = exactly 7 days), an optional day count (D = exactly 24 hours), T and + | * hours/minutes/seconds (a fraction of any length, truncated toward zero to the nanosecond). + | * Years and months are refused: a calendar span has no fixed length. Dependency-free (Jackson only). + | */ + |internal object MetaDurationWire { + | private val ACCEPTED = Regex("^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$") + | /** The text of pgjdbc's PGInterval: what a JDBC read of an interval column, or of an interval[] element, hands back. */ + | private val PG_INTERVAL = Regex("^(-?\\d+) years (-?\\d+) mons (-?\\d+) days (-?\\d+) hours (-?\\d+) mins (-?)(\\d+)(?:\\.(\\d+))? secs$") + | + | /** What a refused value should have been. */ + | const val EXPECTED = "an ISO-8601 duration such as PT1H30M (no years or months)" + | + | /** True when [text] is in the accepted ISO-8601 form and fits a Duration (refused past it, never wrapped). */ + | fun isDuration(text: String?): Boolean = + | text != null && ACCEPTED.matches(text) && runCatching { parse(text) }.isSuccess + | + | /** Strict read of an accepted ISO-8601 duration (digits past a nanosecond truncated toward zero). */ + | fun parse(text: String?): Duration { + | val m = (if (text == null) null else ACCEPTED.matchEntire(text)) + | ?: throw IllegalArgumentException("field.duration: not " + EXPECTED + ": " + text) + | try { + | val g = m.groupValues + | val seconds = Math.addExact( + | Math.addExact(Math.multiplyExact(num(g[2]), 604_800L), Math.multiplyExact(num(g[3]), 86_400L)), + | Math.addExact(Math.multiplyExact(num(g[4]), 3_600L), Math.addExact(Math.multiplyExact(num(g[5]), 60L), num(g[6])))) + | val d = Duration.ofSeconds(seconds, nanos(g[7])) + | return if (g[1] == "-") d.negated() else d + | } catch (e: ArithmeticException) { + | throw IllegalArgumentException("field.duration: out of range: " + text) + | } catch (e: NumberFormatException) { + | throw IllegalArgumentException("field.duration: out of range: " + text) + | } + | } + | + | /** The canonical wire form of [value], to the nanosecond (trailing zeros dropped). */ + | fun format(value: Duration): String { + | val negative = value.isNegative + | val mag = if (negative) value.negated() else value + | val total = mag.seconds + | val nanos = mag.nano.toLong() + | if (total == 0L && nanos == 0L) return "PT0S" + | val b = StringBuilder(if (negative) "-PT" else "PT") + | if (total / 3_600L > 0) b.append(total / 3_600L).append('H') + | if (total % 3_600L / 60L > 0) b.append(total % 3_600L / 60L).append('M') + | if (total % 60L > 0 || nanos > 0) { + | b.append(total % 60L) + | fraction(b, nanos, 9) + | b.append('S') + | } + | return b.toString() + | } + | + | /** + | * The Postgres interval input spelling `[-]HH:MM:SS[.f]`: Postgres refuses a signed ISO-8601 + | * duration. An interval holds microseconds, so a sub-microsecond part is truncated toward zero. + | */ + | fun formatPostgres(value: Duration): String { + | val mag = if (value.isNegative) value.negated() else value + | val total = mag.seconds + | val micros = mag.nano / 1_000L + | val b = StringBuilder(if (value.isNegative && (total != 0L || micros != 0L)) "-" else "") + | val h = total / 3_600L + | if (h < 10) b.append('0') + | b.append(h).append(':') + | if (total % 3_600L / 60L < 10) b.append('0') + | b.append(total % 3_600L / 60L).append(':') + | if (total % 60L < 10) b.append('0') + | b.append(total % 60L) + | fraction(b, micros, 6) + | return b.toString() + | } + | + | /** + | * A stored value: the ISO-8601 string a text column holds, or the text of the PGInterval pgjdbc + | * reads a Postgres interval as. An interval with a year or month part is refused, never + | * guessed at: a calendar span is not a duration. + | */ + | fun fromStored(text: String): Duration { + | if (ACCEPTED.matches(text)) return parse(text) + | val pg = PG_INTERVAL.matchEntire(text) ?: throw IllegalArgumentException("field.duration: not a stored duration: " + text) + | val g = pg.groupValues + | require(g[1].toLong() == 0L && g[2].toLong() == 0L) { + | "field.duration: the interval " + text + " has a year or month part; a calendar span is not a duration" + | } + | val secs = Duration.ofSeconds(g[7].toLong(), nanos(g[8])) + | return Duration.ofDays(g[3].toLong()).plusHours(g[4].toLong()).plusMinutes(g[5].toLong()) + | .plus(if (g[6] == "-") secs.negated() else secs) + | } + | + | /** + | * The 64-bit count of microseconds a SQLite / MySQL column stores (sub-microsecond precision + | * truncated toward zero); past the 64-bit range (that column's limit) it is refused. + | */ + | fun toMicros(value: Duration): Long { + | try { + | val mag = if (value.isNegative) value.negated() else value + | val micros = Math.addExact(Math.multiplyExact(mag.seconds, 1_000_000L), mag.nano / 1_000L) + | return if (value.isNegative) -micros else micros + | } catch (e: ArithmeticException) { + | throw IllegalArgumentException("field.duration: " + value + " is past a 64-bit count of microseconds") + | } + | } + | + | /** A stored count of microseconds as a duration. */ + | fun fromMicros(micros: Long): Duration = + | Duration.ofSeconds(Math.floorDiv(micros, 1_000_000L), Math.floorMod(micros, 1_000_000L) * 1_000L) + | + | /** A duration from a JSON node: an accepted string, anything else is rejected. */ + | fun fromNode(node: JsonNode): Duration { + | if (node.isTextual) return parse(node.asText()) + | throw IllegalArgumentException("field.duration: not a string: " + node) + | } + | + | private fun num(digits: String): Long = if (digits.isEmpty()) 0L else digits.toLong() + | + | /** A fraction of any length as whole nanoseconds, the digits past the ninth truncated toward zero. */ + | private fun nanos(frac: String): Long = + | if (frac.isEmpty()) 0L else (frac + "000000000").substring(0, 9).toLong() + | + | /** [value] as [width] zero-padded ASCII digits after a `.`, trailing zeros dropped. */ + | private fun fraction(b: StringBuilder, value: Long, width: Int) { + | if (value == 0L) return + | b.append('.').append(value.toString().padStart(width, '0').trimEnd('0')) + | } + | + | /** Writes a duration as its canonical ISO-8601 string, whatever the host ObjectMapper would do. */ + | class Serializer : StdSerializer(Duration::class.java) { + | override fun serialize(value: Duration, gen: JsonGenerator, provider: SerializerProvider) { + | gen.writeString(format(value)) + | } + | } + | + | /** Reads the accepted form only; a number of seconds, a calendar span or a clock string is a 400. */ + | class Deserializer : StdDeserializer(Duration::class.java) { + | override fun deserialize(p: JsonParser, ctxt: DeserializationContext): Duration { + | if (p.currentToken() != JsonToken.VALUE_STRING) { + | return ctxt.handleUnexpectedToken(Duration::class.java, p) as Duration + | } + | return try { + | parse(p.text) + | } catch (e: IllegalArgumentException) { + | ctxt.handleWeirdStringValue(Duration::class.java, p.text, "%s", e.message) as Duration + | } + | } + | } + |} + """.trimMargin() +} diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt index 434f0df5b6..d384c4dd99 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt @@ -20,6 +20,7 @@ import com.metaobjects.MetaData import com.metaobjects.field.DecimalField import com.metaobjects.field.EnumField import com.metaobjects.field.InetField +import com.metaobjects.field.DurationField import com.metaobjects.field.MetaField import com.metaobjects.field.MapField import com.metaobjects.field.ObjectField @@ -94,6 +95,7 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { parseArgs() val outRoot = Paths.get(outDir.absolutePath) decimalWirePackages.clear() + durationWirePackages.clear() // ADR-0019 Amendment 2: the project's wire setting, read from the loaded root. wire = WireFormat.of(loader.root) fr019Config = Fr019SharedEnum.ProvidedEnumConfig.fromArgs(args) @@ -134,8 +136,13 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { emitNetJsonSupport(loader, outRoot) // ADR-0019 Amendment 1: the decimal wire rule, once per package that wrote a decimal property. for (pkg in decimalWirePackages) KotlinDecimalWireEmitter.write(pkg, outRoot, wire) + // field.duration: the canonical ISO-8601 wire rule, once per package that wrote a duration property. + for (pkg in durationWirePackages) KotlinDurationWireEmitter.write(pkg, outRoot) } + /** Packages whose emitted data classes carry a duration property: each gets one `MetaDurationWire.kt`. */ + private val durationWirePackages = linkedSetOf() + /** Packages whose emitted data classes carry a decimal property: each gets one `MetaDecimalWire.kt`. */ private val decimalWirePackages = linkedSetOf() @@ -313,6 +320,7 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { } decimalWireAnnotations(field, pkg, declaredRange = obj.subType == MetaObject.SUBTYPE_VALUE) .forEach { propBuilder.addAnnotation(it) } + durationWireAnnotations(field, pkg).forEach { propBuilder.addAnnotation(it) } typeBuilder.addProperty(propBuilder.build()) } @@ -326,6 +334,7 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { typeBuilder.addProperty( PropertySpec.builder(propName, propType).initializer(propName) .apply { decimalWireAnnotations(field, pkg).forEach { addAnnotation(it) } } + .apply { durationWireAnnotations(field, pkg).forEach { addAnnotation(it) } } .build(), ) } @@ -498,7 +507,7 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { /** * One `@get:AssertTrue @get:JsonIgnore val isXxxValid: Boolean` per entity-scoped cross-field - * validator (comparison / requiredWhen / presentIff / atLeastOne). Null-safe in the DB CHECK + * validator (comparison / requiredWhen / presentIff / atLeastOne / exactlyOne, with @condition). Null-safe in the DB CHECK * sense (SQL NULL = Kotlin null; "" is not null). A validator naming a missing field is * skipped. Body-declared, so it stays out of the data class's equals / copy / constructor. */ @@ -717,6 +726,28 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { return annotations } + /** + * `field.duration`: a `java.time.Duration` property is written as the canonical ISO-8601 string + * and read from the accepted form only, through the generated same-package `MetaDurationWire` + * (no MetaObjects import). Only under `wire.durations: iso`; under native the host + * ObjectMapper's own java.time handling applies. Both halves in iso mode: a host ObjectMapper's JavaTimeModule + * writes a Duration as decimal seconds (or `Duration.toString()`, which differs for a negative + * span), and `Duration.parse` accepts more than the accepted form. The serializer sits on the + * getter and the deserializer on the constructor parameter; `contentUsing` for an `isArray`. + */ + private fun durationWireAnnotations(field: MetaField<*>, pkg: String): List { + // wire.durations: native (the default) leaves the host ObjectMapper's own java.time handling. + if (field !is DurationField || !wire.durationsIso()) return emptyList() + durationWirePackages.add(pkg) + val member = if (field.isArrayType) "contentUsing" else "using" + return listOf( + AnnotationSpec.builder(JSON_SERIALIZE).useSiteTarget(AnnotationSpec.UseSiteTarget.GET) + .addMember("$member = %T::class", ClassName(pkg, KotlinDurationWireEmitter.FILE_NAME, "Serializer")).build(), + AnnotationSpec.builder(JSON_DESERIALIZE).useSiteTarget(AnnotationSpec.UseSiteTarget.PARAM) + .addMember("$member = %T::class", ClassName(pkg, KotlinDurationWireEmitter.FILE_NAME, "Deserializer")).build(), + ) + } + private fun isStrictNetField(field: MetaField<*>): Boolean = (field is UriField || field is InetField) && !KotlinTypeMapper.lenientNetField(field) diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt index 93da980132..821e09a443 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt @@ -170,6 +170,11 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase` builders), emitted in pass 3e. val packagesNeedingPortableHelper = sortedSetOf() + // Packages that emit at least one table carrying a `field.duration` column. Each gets ONE + // shared `MetaDurationColumnType.kt` (the engine-aware `Column` builders) plus the + // package's `MetaDurationWire.kt`, emitted in pass 3f. + val packagesNeedingDurationHelper = sortedSetOf() + // Pass 2: emit one Table per entity using its own metadata + the // inbound FKs accumulated in Pass 1. // FR-024 (ADR-0028): object.projection is emitted too — a projection with @@ -204,6 +209,9 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase` column types ([durationSupportBlock]) and, beside it, the + * package's `MetaDurationWire.kt` they format and parse through. + */ + private fun emitDurationSupportFile(pkg: String, outRoot: Path) { + KotlinDurationWireEmitter.write(pkg, outRoot) + val api = exposedApi() + emitSupportFile(pkg, outRoot, DURATION_SUPPORT_FILE_NAME, durationSupportFileImports(api), durationSupportBlock(api)) + } + private fun entityNeedsUuidStringHelper(entity: MetaObject, loader: MetaDataLoader): Boolean = entity.metaFields.any { it !is ObjectField && KotlinTypeMapper.usesUuidStringHelper(it) } @@ -1331,13 +1376,12 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBaseNames has no // constant for it. Always literal; useNames() does not reach here. @@ -1688,6 +1732,117 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase` over the Postgres-native + * `INTERVAL`, and elsewhere a BIGINT count of microseconds (so filters and sorts order it). Postgres refuses a signed + * ISO-8601 duration as interval input, so the value binds in the interval spelling + * `[-]HH:MM:SS[.f]`, as text under a `CAST(? AS INTERVAL)` parameter marker (a bare + * varchar parameter is refused where an interval goes). A read takes the + * driver's value — pgjdbc's `PGInterval` on Postgres, the microsecond count elsewhere — + * through `MetaDurationWire.fromStored`, which refuses an interval with a year or month part. + * - `MetaDurationArrayColumnType` — an `isArray` duration: `INTERVAL[]` on Postgres, a JSON + * text array of microsecond counts on MySQL and SQLite. + * + * The text differs by [ExposedApi] only in the array bind (and the imports). + */ + fun durationSupportBlock(api: ExposedApi): String { + // 0.x binds a typed array by element-type name; 1.x wants an ArrayColumnType there, so it + // binds through the raw JDBC statement, as MetaDecimalArrayColumnType does. + val arrayBind = if (api == ExposedApi.V1) { + " @Suppress(\"UNCHECKED_CAST\")\n" + + " val raw = (stmt as JdbcPreparedStatementImpl).statement\n" + + " raw.setArray(index, raw.connection.createArrayOf(\"interval\", value as Array))" + } else { + " @Suppress(\"UNCHECKED_CAST\")\n" + + " stmt.setArray(index, \"interval\", value as Array)" + } + return """ + |private fun isPostgres(): Boolean = currentDialect.name.equals("postgresql", ignoreCase = true) + | + |private fun isMySql(): Boolean = currentDialect.name.let { it.equals("mysql", ignoreCase = true) || it.equals("mariadb", ignoreCase = true) } + | + |/** The stored form of a duration: the interval spelling on Postgres, a 64-bit microsecond count elsewhere. */ + |private fun stored(value: Duration): Any = + | if (isPostgres()) MetaDurationWire.formatPostgres(value) else MetaDurationWire.toMicros(value) + | + |/** A stored value: a microsecond count (SQLite, MySQL), or interval / ISO text. */ + |private fun durationOf(value: Any): Duration = when (value) { + | is Duration -> value + | is Number -> MetaDurationWire.fromMicros(value.toLong()) + | else -> MetaDurationWire.fromStored(value.toString()) + |} + | + |/** + | * GENERATED — do not hand-edit. A `java.time.Duration` column: native `INTERVAL` on Postgres, and + | * elsewhere a BIGINT count of microseconds, so a range filter and a sort order it as a number. + | */ + |internal class MetaDurationColumnType : ColumnType() { + | override fun sqlType(): String = if (isPostgres()) "INTERVAL" else "BIGINT" + | override fun valueFromDB(value: Any): Duration = durationOf(value) + | override fun notNullValueToDB(value: Duration): Any = stored(value) + | override fun nonNullValueToString(value: Duration): String = + | if (isPostgres()) "'" + stored(value) + "'" else stored(value).toString() + | // The value binds as text; Postgres will not take a varchar where an interval goes, so the + | // marker casts it (an explicit cast from text to interval is always allowed). + | override fun parameterMarker(value: Duration?): String = if (isPostgres()) "CAST(? AS INTERVAL)" else "?" + |} + | + |/** Column builder for `field.duration`: a `Column`. */ + |internal fun Table.durationColumn(name: String): Column = + | registerColumn(name, MetaDurationColumnType()) + | + |/** + | * GENERATED — do not hand-edit. An `isArray` duration column: `INTERVAL[]` on Postgres, and elsewhere + | * a JSON text array of microsecond counts. + | */ + |internal class MetaDurationArrayColumnType : ColumnType>() { + | private fun element(value: Any): Duration = durationOf(value) + | private fun json(value: List): String = value.joinToString(",", "[", "]") { MetaDurationWire.toMicros(it).toString() } + | override fun sqlType(): String = if (isPostgres()) "INTERVAL[]" else if (isMySql()) "JSON" else "TEXT" + | override fun valueFromDB(value: Any): List = when (value) { + | is java.sql.Array -> (value.array as Array<*>).map { element(it!!) } + | is Array<*> -> value.map { element(it!!) } + | is List<*> -> value.map { element(it!!) } + | else -> com.fasterxml.jackson.databind.ObjectMapper().readTree(value.toString()).map { + | if (it.isIntegralNumber) MetaDurationWire.fromMicros(it.longValue()) else MetaDurationWire.fromNode(it) + | } + | } + | override fun notNullValueToDB(value: List): Any = + | if (isPostgres()) value.map { MetaDurationWire.formatPostgres(it) }.toTypedArray() else json(value) + | override fun nonNullValueToString(value: List): String = + | if (isPostgres()) value.joinToString(",", "ARRAY[", "]::interval[]") { "'" + MetaDurationWire.formatPostgres(it) + "'" } + | else "'" + json(value) + "'" + | override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { + | if (value is Array<*>) { + |$arrayBind + | } else { + | super.setParameter(stmt, index, value) + | } + | } + |} + | + |/** Column builder for an `isArray` `field.duration`: a `Column>`. */ + |internal fun Table.durationArrayColumn(name: String): Column> = + | registerColumn(name, MetaDurationArrayColumnType()) + |""".trimMargin() + } + /** File name (sans `.kt`) of the per-package support file for `field.uuid` and `field.decimal` columns. */ const val PORTABLE_SUPPORT_FILE_NAME = "MetaPortableColumnTypes" @@ -2438,7 +2593,7 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase $simple.valueOf(v as String) }, { e -> e.name })" // `; ` separates the branches: this is a one-line `when`, and space-separated // branches do not parse (`Foo.DRAFT 5 -> ...`). diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt index 62b74b74fc..1d7965d1e5 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt @@ -214,8 +214,20 @@ public object KotlinGenUtil { .firstOrNull { MetaSource.ROLE_PRIMARY == it.role } } - /** §A2/§A3 — physical name + logical field name for one field. */ - data class KotlinFieldNames(val name: String, val column: String) + /** + * §A2/§A3 — physical name + logical field name for one field. + * + * [members] is set only on a `@storage: "flattened"` value object, and there it is the + * only column information: such a field has no column of its own ([column] is null), and + * each member of the value object is a column of the owning table, named by core naming + * ([ColumnNaming.resolveFlattened], the rule `meta migrate` names them by), keyed by the + * member's field name. + */ + data class KotlinFieldNames( + val name: String, + val column: String?, + val members: Map? = null, + ) /** * One `source.rdb` child, under the ROLE it plays. @@ -448,7 +460,21 @@ public object KotlinGenUtil { } private fun fieldMap(fields: Collection>, strategy: String): Map = - fields.associate { f -> f.name to KotlinFieldNames(f.name, resolveColumnName(f, strategy)) } + fields.associate { f -> f.name to fieldNamesOf(f, strategy) } + + /** + * The flattened test is migrate's own: a `field.object` whose RESOLVED `@storage` is + * `flattened`. Members are the value object's resolving fields. + */ + private fun fieldNamesOf(f: MetaField<*>, strategy: String): KotlinFieldNames { + val flattened = f is ObjectField && f.hasMetaAttr(ObjectField.ATTR_STORAGE, true) && + ObjectField.STORAGE_FLATTENED == f.getMetaAttr(ObjectField.ATTR_STORAGE, true).valueAsString + if (!flattened) return KotlinFieldNames(f.name, resolveColumnName(f, strategy)) + val target = (f as ObjectField).objectRef + val members = target?.metaFields.orEmpty() + .associate { m -> m.name to KotlinFieldNames(m.name, ColumnNaming.resolveFlattened(f, m, strategy)) } + return KotlinFieldNames(f.name, null, members) + } /** * The names FRAGMENT for an object that a sourced object extends but which declares no @@ -896,6 +922,7 @@ public object KotlinGenUtil { const val COLUMN_NAMING_SNAKE_CASE: String = ColumnNaming.SNAKE_CASE const val COLUMN_NAMING_LITERAL: String = ColumnNaming.LITERAL const val COLUMN_NAMING_KEBAB_CASE: String = ColumnNaming.KEBAB_CASE + const val COLUMN_NAMING_UPPER_FIRST: String = ColumnNaming.UPPER_FIRST const val DEFAULT_COLUMN_NAMING: String = COLUMN_NAMING_SNAKE_CASE /** diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGenerator.kt index 4e7a84d258..492ed72cae 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGenerator.kt @@ -4,6 +4,7 @@ package com.metaobjects.generator.kotlin // com.metaobjects.generator.kotlin when this generator is copied out via // `mvn metaobjects:eject` and its own package is renamed — an explicit import, not // same-package bare-name resolution, is what keeps the ejected copy compiling. +import org.slf4j.LoggerFactory import com.metaobjects.generator.kotlin.KotlinGenUtil import com.metaobjects.generator.kotlin.KotlinNaming import com.metaobjects.generator.kotlin.PackageMapping @@ -148,8 +149,24 @@ open class KotlinNamesGenerator : val value: String, val path: String, val section: String, + /** A flattened value object's member column: left out, not refused, when it collides. */ + val flattened: Boolean = false, ) + + /** + * The flattened-member constants of [names] that would share a name with another + * constant of the same object — a field `shipToStreet` beside the member `street` of a + * flattened `shipTo`. They are left out, with a warning, rather than refused: such a model + * generated before member constants existed, and every other constant it had is still + * emitted. Computed over the WHOLE resolved set, so an object and its base agree. + */ + protected fun collidingFlattenedMembers(names: KotlinGenUtil.KotlinObjectNames): Set { + val raw = rawMembersOf(names, all = true) + val taken = raw.filter { !it.flattened }.mapTo(mutableSetOf()) { it.member } + return raw.filter { it.flattened && !taken.add(it.member) }.mapTo(sortedSetOf()) { it.member } + } + /** @return true when a file was written. */ protected open fun emit( entity: MetaObject, @@ -181,6 +198,11 @@ open class KotlinNamesGenerator : val resolved = membersOf(names, all = true) val own = membersOf(names, all = false) + for (skipped in collidingFlattenedMembers(names)) { + LOG.warn("names: {}: the flattened column constant {} would share its name with another " + + "constant of the object, so it is not emitted. Rename the field or the value-object " + + "member to get it.", entity.name, skipped) + } // Two nodes whose SCREAMING_SNAKE member forms collide would emit duplicate const // members. Kotlin would refuse to compile it, but the error would name a generated @@ -234,9 +256,19 @@ open class KotlinNamesGenerator : // It stays COMPLETE — every field, inherited included — because it is the // lookup surface, and a miss on an inherited field is exactly the // fallback-to-literal this artifact removes. - append("\n val COLUMNS_BY_FIELD: Map = mapOf(\n") - for (name in names.fields.keys.sorted()) { - append(" \"$name\" to ${KotlinNaming.namesMember(name)}_COLUMN,\n") + append("\n") + append(" val COLUMNS_BY_FIELD: Map = mapOf(\n") + val skippedMembers = collidingFlattenedMembers(names) + val rows = names.fields.values.flatMap { f -> + // A flattened field has no column of its own: only its members are keyed. + (if (f.members == null) listOf(f.name to "${KotlinNaming.namesMember(f.name)}_COLUMN") else emptyList()) + + // A flattened member is keyed `.`, the path the naming corpus uses. + f.members.orEmpty().values + .map { m -> "${f.name}.${m.name}" to KotlinNaming.flattenedColumnMember(f.name, m.name) } + .filter { (_, constant) -> constant !in skippedMembers } + }.sortedBy { it.first } + for ((key, constant) in rows) { + append(" \"$key\" to $constant,\n") } append(" )\n") append("}\n") @@ -260,6 +292,11 @@ open class KotlinNamesGenerator : * are re-exported, with no `inheritsSource` flag needed to say so. */ protected fun membersOf(names: KotlinGenUtil.KotlinObjectNames, all: Boolean): List { + val skipped = collidingFlattenedMembers(names) + return rawMembersOf(names, all).filterNot { it.flattened && it.member in skipped } + } + + private fun rawMembersOf(names: KotlinGenUtil.KotlinObjectNames, all: Boolean): List { val out = mutableListOf() val self = names.name out += NameConst("TYPE", names.type, self, "self") @@ -295,7 +332,14 @@ open class KotlinNamesGenerator : val member = KotlinNaming.namesMember(f.name) val path = "$self.fields.${f.name}" out += NameConst("${member}_FIELD", f.name, path, "fields") - out += NameConst("${member}_COLUMN", f.column, path, "fields") + val column = f.column + if (column != null) { + out += NameConst("${member}_COLUMN", column, path, "fields") + } + // A flattened field has no column of its own: one constant per member's column. + for (m in f.members.orEmpty().values.sortedBy { it.name }) { + out += NameConst(KotlinNaming.flattenedColumnMember(f.name, m.name), checkNotNull(m.column), "$path.${m.name}", "fields", flattened = true) + } } addKeys(out, self, "identities", if (all) names.identities else names.ownIdentities) @@ -348,4 +392,8 @@ open class KotlinNamesGenerator : PackageMapping.splitFqn(md.name).first.replace('.', '/') override fun getSingleOutputFilename(md: MetaObject): String = KotlinNaming.namesObjectName(PackageMapping.splitFqn(md.name).second) + ".kt" + + private companion object { + val LOG = LoggerFactory.getLogger(KotlinNamesGenerator::class.java) + } } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt index ba4ab410f1..3990c3b37f 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt @@ -127,6 +127,10 @@ object KotlinNaming { */ fun namesMember(fieldName: String): String = KotlinGenUtil.camelToSnake(fieldName).uppercase() + /** The constant naming a flattened member's column: `SHIP_TO_STREET_COLUMN`. */ + fun flattenedColumnMember(fieldName: String, memberName: String): String = + "${namesMember(fieldName)}_${namesMember(memberName)}_COLUMN" + /** * [KotlinSpringControllerGenerator] / [KotlinM2mSupport]: the REST collection segment — * the ENTITY NAME snake_cased and then pluralized (`Author` -> `authors`, diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGenerator.kt index 88361a7b38..3bc458ca92 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGenerator.kt @@ -11,6 +11,7 @@ import com.metaobjects.generator.kotlin.PackageMapping import com.metaobjects.field.DecimalField import com.metaobjects.field.EnumField import com.metaobjects.field.InetField +import com.metaobjects.field.DurationField import com.metaobjects.field.MapField import com.metaobjects.field.MetaField import com.metaobjects.field.StringField @@ -95,7 +96,13 @@ open class KotlinReportExportGenerator : MultiFileDirectGeneratorBase + val credentials = KotlinGenUtil.reportCredentialNames(entity).toSet() + entity.metaFields.any { it.name !in credentials && cellFactory(it) == "duration" } + } + write(outRoot, writerPackage, WRITER_NAME, writerSource(writerPackage, xmlNulls, csvBom, decimalsAsString, duration)) for (entity in served) { val (pkg, shortName) = PackageMapping.splitFqn(entity.name) write(outRoot, pkg, "${shortName}Export", exportSource(entity, pkg, shortName, writerPackage, maxRows)) @@ -180,6 +187,7 @@ open class KotlinReportExportGenerator : MultiFileDirectGeneratorBase "text" f is DecimalField -> "decimal" + f is DurationField -> "duration" else -> "raw" } } @@ -188,8 +196,13 @@ open class KotlinReportExportGenerator : MultiFileDirectGeneratorBase 0) b.append(total / 3600).append('H') + if (total % 3600 / 60 > 0) b.append(total % 3600 / 60).append('M') + if (total % 60 > 0 || nanos > 0) { + b.append(total % 60) + if (nanos > 0) b.append('.').append(nanos.toString().padStart(9, '0').trimEnd('0')) + b.append('S') + } + return Cell(b.toString(), false) + } + +""" + /** The simple name of the shared writer. */ const val WRITER_NAME = "ReportExportWriter" diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index edb545bc44..cc014650d1 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -20,6 +20,7 @@ import com.metaobjects.field.BooleanField import com.metaobjects.field.CurrencyField import com.metaobjects.field.DateField import com.metaobjects.field.DecimalField +import com.metaobjects.field.DurationField import com.metaobjects.field.DoubleField import com.metaobjects.field.EnumField import com.metaobjects.field.FloatField @@ -154,6 +155,8 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase() + val durationWirePackages = linkedSetOf() + durationsIso = WireFormat.of(loader.root).durationsIso() for (declared in loader.metaObjects) { // FR-044: a served report is emitted from its read model, the same object the @@ -204,6 +207,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase(TextColumnType())`; import both only when such a column exists // so entities without a filterable enum stay byte-identical. @@ -908,6 +919,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase(TextColumnType())`; same // conditional imports as the vanilla controller. @@ -1364,8 +1378,20 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase, fn: String, ktType: String): String = - if (field is DecimalField) { + if (field is DurationField && durationsIso) { + // field.duration: the accepted ISO-8601 form only, never the ObjectMapper's Duration.parse. + val read = if (field.isArrayType) { + "body.get(\"$fn\").also { require(it.isArray) }.map { MetaDurationWire.fromNode(it) }" + } else { + "MetaDurationWire.fromNode(body.get(\"$fn\"))" + } + "runCatching { $read }" + + ".getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\", \"field\" to \"$fn\") as Any) }" + } else if (field is DecimalField) { val read = if (field.isArrayType) { "body.get(\"$fn\").also { require(it.isArray) }.map { MetaDecimalWire.fromNode(it) }" } else { @@ -1605,6 +1631,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase out.append("coerce${shortName}Uuid(op, raw)\n") + // field.duration columns are Column: the operand is read by the accepted + // ISO-8601 rule (MetaDurationWire), a malformed one is `invalid_filter_value`. + DurationField.SUBTYPE_DURATION -> + out.append("coerce${shortName}Duration(op, raw)\n") else -> // Unrecognized subtype — fall through to a string pass-through. This branch // is unreachable for fields that are actually @filterable (those subtypes @@ -1640,6 +1670,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase( is DateField -> "${ExposedImports.javatime(exposedApi, "JavaLocalDateColumnType")}()" is TimestampField -> "${ExposedImports.javatime(exposedApi, "JavaInstantColumnType")}()" is CurrencyField -> "${ExposedImports.core(exposedApi, "LongColumnType")}()" - is EnumField -> "${ExposedImports.core(exposedApi, "VarCharColumnType")}(${KotlinTypeMapper.ENUM_VARCHAR_LEN})" + is EnumField -> KotlinTypeMapper.enumVarcharLength(field) + ?.let { "${ExposedImports.core(exposedApi, "VarCharColumnType")}($it)" } + ?: "${ExposedImports.core(exposedApi, "TextColumnType")}()" // issue #390: java.util.UUID columns MUST use the core.java UUIDColumnType under 1.x — // bare core.UUIDColumnType (if it even existed there) would be the kotlin.uuid.Uuid // binding the 0.x name no longer provides; see KotlinTypeMapper's [UuidField] branches. diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt index 21ccdeb380..05d62eb586 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt @@ -18,6 +18,7 @@ import com.metaobjects.field.TimestampField import com.metaobjects.field.UuidField import com.metaobjects.field.UriField import com.metaobjects.field.InetField +import com.metaobjects.field.DurationField import com.metaobjects.`object`.MetaObject import com.squareup.kotlinpoet.ANY import com.squareup.kotlinpoet.BOOLEAN @@ -48,9 +49,22 @@ import com.squareup.kotlinpoet.TypeName */ object KotlinTypeMapper { - /** Default VARCHAR width for string-backed `field.enum` storage (v1). */ + /** + * The width a string-backed `field.enum` column had when no length was configured. The + * packaged generators no longer use it (that case is `text` now, as `meta migrate` creates + * it); it stays so a generator ejected earlier, which still names it, keeps compiling. + */ const val ENUM_VARCHAR_LEN = 64 + /** + * The VARCHAR width of a string-backed `field.enum`'s column: core's + * [com.metaobjects.database.EnumLength.resolveMaxLength] (its `@maxLength`, else the + * project's `enums.maxLength`, refusing a member that does not fit). Null when no length + * is configured, and then the column is `text`, as `meta migrate` creates it. + */ + fun enumVarcharLength(field: EnumField): Long? = + com.metaobjects.database.EnumLength.resolveMaxLength(field) + /** * Threshold beyond which a [StringField]'s `@maxLength` is treated as unbounded text and * emitted as Exposed `text(name)` rather than `varchar(name, N)`. Chosen at 4000 — the @@ -154,6 +168,19 @@ object KotlinTypeMapper { */ const val EXPOSED_INET_FN = "inetColumn" + /** + * Name of the generated, file-local Exposed extension function emitted for a + * [DurationField]. Returns a `Column` whose `sqlType()` is the + * Postgres-native `INTERVAL` (elsewhere a BIGINT count of microseconds), so the + * column type MATCHES the `java.time.Duration` data-class property. + * [KotlinExposedTableGenerator] emits the supporting `ColumnType` + this extension + * into the package-shared `MetaDurationColumnType` support file. + */ + const val EXPOSED_DURATION_FN = "durationColumn" + + /** [EXPOSED_DURATION_FN] for an `isArray` duration: `INTERVAL[]` on Postgres, a JSON text array of microsecond counts elsewhere. */ + const val EXPOSED_DURATION_ARRAY_FN = "durationArrayColumn" + /** * Name of the generated, file-local Exposed extension function emitted for a * `field.string @dbColumnType=uuid` column (R6 Plan 2b physical escape hatch; ADR-0037: @@ -315,6 +342,10 @@ object KotlinTypeMapper { // column is the Postgres-native inet type; the file-local `inetColumn(...)` extension // gives a `Column`. #234: a @lenient field.inet degrades to a plain String. is InetField -> if (lenientNetField(field)) STRING else ClassName("java.net", "InetAddress") + // field.duration → native java.time.Duration property. The DB column is the Postgres-native + // interval (a BIGINT count of microseconds elsewhere); the file-local `durationColumn(...)` extension gives a + // `Column`, and the wire is the canonical ISO-8601 string (MetaDurationWire). + is DurationField -> ClassName("java.time", "Duration") // field.map → an open-keyed Map stored in a single jsonb column. // Keys are always String. The value type is the scalar named by @valueType, // or (for an @objectRef map) the referenced VO — which needs the loader to @@ -548,7 +579,7 @@ object KotlinTypeMapper { // Enum stored as VARCHAR for v1. Proper enum-column handling needs the generated // enum class (Exposed's `customEnumeration` / `enumerationByName` takes a KClass) // and is intentionally deferred. - is EnumField -> "varchar($colExpr, $ENUM_VARCHAR_LEN)" + is EnumField -> enumVarcharLength(field)?.let { "varchar($colExpr, $it)" } ?: "text($colExpr)" // field.uuid → Exposed's first-class `uuid(name)` under 0.x (native Postgres uuid // column, java.util.UUID) or `javaUUID(name)` under 1.x (issue #390 — 1.x's `uuid()` // binds `kotlin.uuid.Uuid` instead; `javaUUID` is the replacement that keeps @@ -575,6 +606,12 @@ object KotlinTypeMapper { // field.inet is a plain `text` column (the native inet column would reject a // not-strictly-valid value at INSERT). is InetField -> if (lenientNetField(field)) "text($colExpr)" else "$EXPOSED_INET_FN($colExpr)" + // field.duration → file-local `durationColumn(...)` (a `Column` over the + // Postgres-native INTERVAL, a BIGINT count of microseconds elsewhere) or, with `isArray`, + // `durationArrayColumn(...)` (INTERVAL[] / a JSON text array of microsecond counts). The helpers live in the package-shared + // MetaDurationColumnType.kt support file. + is DurationField -> + if (isArrayResolved(field)) "$EXPOSED_DURATION_ARRAY_FN($colExpr)" else "$EXPOSED_DURATION_FN($colExpr)" // field.map → a single Postgres `JSONB` column holding the JSON object. Same emission // as a `field.object` jsonb column (the typed-object JSONB path) — the Exposed // `jsonb(name, encoder, decoder)` extension encoded/decoded through the shared Jackson @@ -667,6 +704,12 @@ object KotlinTypeMapper { fun usesInetUriHelper(field: MetaField<*>): Boolean = (field is UriField || field is InetField) && !lenientNetField(field) + /** + * True iff [field] needs the package-shared `MetaDurationColumnType.kt` support file — a + * [DurationField] (custom `Column` over the Postgres-native interval, a BIGINT count of microseconds elsewhere). + */ + fun usesDurationHelper(field: MetaField<*>): Boolean = field is DurationField + /** * True iff [field] needs the package-shared `MetaUuidStringColumnType.kt` support file — * i.e. it is a non-array [StringField] carrying `@dbColumnType=uuid` (R6 Plan 2b). Mirrors diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinCodegenSnapshotTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinCodegenSnapshotTest.kt index 04d49efa58..61ebc5a64b 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinCodegenSnapshotTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinCodegenSnapshotTest.kt @@ -147,6 +147,6 @@ class KotlinCodegenSnapshotTest { val useNames = Regex("\"useNames\"\\s*:\\s*true").containsMatchIn(json) val decimals = if (Regex("\"wireDecimals\"\\s*:\\s*\"string\"").containsMatchIn(json)) WireFormat.Decimals.STRING else WireFormat.Decimals.NATIVE val nulls = if (Regex("\"wireNullMembers\"\\s*:\\s*\"omit\"").containsMatchIn(json)) WireFormat.NullMembers.OMIT else WireFormat.NullMembers.NATIVE - return FixtureConfig(gens, pkg, useNames, WireFormat(decimals, nulls)) + return FixtureConfig(gens, pkg, useNames, WireFormat(decimals, nulls, if (Regex("\"wireDurations\"\\s*:\\s*\"iso\"").containsMatchIn(json)) WireFormat.Durations.ISO else WireFormat.Durations.NATIVE)) } } diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt index 813db20c90..d32e964ab0 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt @@ -51,7 +51,7 @@ class KotlinDecimalWireTest { }""".trimIndent() /** The cross-port form the conformance corpora pin: canonical decimal strings, null members omitted. */ - private val pinned = WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT) + private val pinned = WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO) private fun emit(json: String, generators: List, wire: WireFormat = pinned): Map { val outDir = Files.createTempDirectory("kt-decimal-wire-") @@ -522,9 +522,9 @@ class KotlinDecimalWireTest { @Test fun `emit writes a null member as null like native, and omit leaves it out`() { - val emit = writeItem(WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.EMIT)) + val emit = writeItem(WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.EMIT, WireFormat.Durations.ISO)) assertTrue("\"note\":null" in emit, emit) - val omit = writeItem(WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.OMIT)) + val omit = writeItem(WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO)) assertFalse("note" in omit, omit) assertTrue("\"price\":12.50" in omit, omit) } diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireTest.kt new file mode 100644 index 0000000000..9911e4d5cd --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDurationWireTest.kt @@ -0,0 +1,242 @@ +package com.metaobjects.generator.kotlin + +import com.fasterxml.jackson.databind.JsonNode +import com.fasterxml.jackson.databind.ObjectMapper +import com.metaobjects.generator.Generator +import com.metaobjects.metadata.ktx.loadString +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import java.nio.file.Files +import java.nio.file.Path +import java.time.Duration +import java.util.TreeMap +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * `field.duration` as the Kotlin generators write it. The rule lives in a generated, per-package + * `MetaDurationWire.kt` (the PRODUCTION helper of this port, which imports nothing from + * MetaObjects), so the shared `fixtures/duration-wire/vectors.json` vectors run against the + * compiled generated object; the entity, table and controller that call it compile alongside. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class KotlinDurationWireTest { + + private val model = """{ + "metadata.root": { "package": "acme::ops", "children": [ + { "object.value": { "name": "Window", "children": [ + { "field.duration": { "name": "span", "@required": true } } + ] } }, + { "object.entity": { "name": "Job", "children": [ + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "holdFor", "@required": true, "@filterable": true } }, + { "field.duration": { "name": "grace", "@filterable": true } }, + { "field.duration": { "name": "retries", "isArray": true } }, + { "field.object": { "name": "window", "@objectRef": "Window", "@storage": "jsonb" } }, + { "source.rdb": { "@table": "jobs" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + + private fun emit( + generators: List, + durations: com.metaobjects.config.WireFormat.Durations = com.metaobjects.config.WireFormat.Durations.ISO, + ): Map { + val outDir = Files.createTempDirectory("kt-duration-wire-") + try { + val loader = loadString("duration-wire", model) + loader.root.wireFormat = com.metaobjects.config.WireFormat( + com.metaobjects.config.WireFormat.Decimals.NATIVE, + com.metaobjects.config.WireFormat.NullMembers.NATIVE, + durations, + ) + for (gen in generators) { + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loader) + } + val files = TreeMap() + Files.walk(outDir).use { s -> + s.filter { it.isRegularFile() }.forEach { files[outDir.relativize(it).toString()] = it.readText() } + } + return files + } finally { + outDir.toFile().deleteRecursively() + } + } + + private fun all() = listOf( + KotlinEntityGenerator(), KotlinExposedTableGenerator(), + KotlinFilterAllowlistGenerator(), KotlinSpringControllerGenerator(), + ) + + private fun compile(files: Map): ClassLoader { + val sources = files.filterKeys { it.endsWith(".kt") }.map { (path, src) -> SourceFile.kotlin(path.substringAfterLast('/'), src) } + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + return result.classLoader + } + + private fun vectors(): JsonNode { + var p: Path? = Path.of(System.getProperty("user.dir")).toAbsolutePath() + while (p != null && !Files.isRegularFile(p.resolve("fixtures/duration-wire/vectors.json"))) p = p.parent + requireNotNull(p) { "fixtures/duration-wire/vectors.json not found above the working directory" } + return ObjectMapper().readTree(p.resolve("fixtures/duration-wire/vectors.json").toFile()) + } + + @Test + fun `the generated entity, table and controller wire a duration through MetaDurationWire`() { + val files = emit(all()) + val entity = files.getValue("acme/ops/Job.kt") + assertTrue("@get:JsonSerialize(using = MetaDurationWire.Serializer::class)" in entity, entity) + assertTrue("@param:JsonDeserialize(using = MetaDurationWire.Deserializer::class)" in entity, entity) + assertTrue("@get:JsonSerialize(contentUsing = MetaDurationWire.Serializer::class)" in entity, entity) + val controller = files.getValue("acme/ops/JobController.kt") + assertTrue("MetaDurationWire.parse" in controller, controller) + assertTrue("MetaDurationWire.fromNode(body.get(\"holdFor\"))" in controller, controller) + val allowlist = files.getValue("acme/ops/JobFilterAllowlist.kt") + // A duration orders on every DB (interval on Postgres, microsecond count elsewhere). + val holdForOps = allowlist.substringAfter("\"holdFor\" to setOf(").substringBefore(")") + for (op in listOf("eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull")) assertTrue("\"$op\"" in holdForOps, allowlist) + assertTrue("JobTable.holdFor greater (p.value as Duration)" in controller, controller) + // Generated code names no MetaObjects package. + for ((path, src) in files) { + val code = src.lineSequence().filterNot { it.trimStart().let { l -> l.startsWith("*") || l.startsWith("/*") || l.startsWith("//") } } + .joinToString("\n") + assertFalse("com.metaobjects." in code, "$path references a MetaObjects package:\n$src") + } + // Spring is not on this module's test classpath: the controller compiles in + // integration-tests-kotlin (FitnessCorpusControllerCompileTest); the rest compiles here. + compile(files.filterKeys { !it.endsWith("Controller.kt") }) + } + + @Test + fun `the generated MetaDurationWire passes every shared wire vector`() { + val cl = compile(emit(all()).filterKeys { !it.endsWith("Controller.kt") }) + val wire = cl.loadClass("acme.ops.MetaDurationWire") + val instance = wire.getField("INSTANCE").get(null) + fun call(name: String, vararg args: Any?): Any? { + val m = wire.methods.first { it.name == name && it.parameterCount == args.size } + return try { m.invoke(instance, *args) } catch (e: java.lang.reflect.InvocationTargetException) { throw e.targetException } + } + // The generated Exposed column type: on Postgres, pgjdbc hands it a PGInterval. + val columnType = cl.loadClass("acme.ops.MetaDurationColumnType").getDeclaredConstructor().newInstance() + val valueFromDB = columnType.javaClass.methods.first { it.name == "valueFromDB" && it.parameterCount == 1 } + fun read(value: Any): Any? = + try { valueFromDB.invoke(columnType, value) } catch (e: java.lang.reflect.InvocationTargetException) { throw e.targetException } + + val v = vectors() + val wireVectors = v.get("wire") + for (c in wireVectors.get("parse")) { + val out = call("format", call("parse", c.get("in").asText())) + assertEquals(c.get("out").asText(), out, "parse ${c.get("in")}") + } + // A java.time.Duration holds nanoseconds: this port's precision key is "9". + for (c in wireVectors.get("precision")) { + val out = call("format", call("parse", c.get("in").asText())) + assertEquals(c.get("out").get("9").asText(), out, "precision ${c.get("in")}") + } + for (c in wireVectors.get("refused")) { + assertEquals(false, call("isDuration", c.asText()), "refused '${c.asText()}'") + assertFailsWith("refused '${c.asText()}'") { call("parse", c.asText()) } + } + + val codecs = v.get("codecs") + // Postgres: the interval text the server returns, as the PGInterval pgjdbc builds from it. + for (c in codecs.get("postgres")) { + val pg = org.postgresql.util.PGInterval(c.get("in").asText()) + assertEquals(c.get("out").asText(), call("format", read(pg)), "postgres ${c.get("in")} as $pg") + } + // A year or month part is refused on that native path, never converted to days. (pgjdbc + // keeps the months it read; text that is not an interval never comes back from a server.) + val calendar = codecs.get("postgresRefused").map { org.postgresql.util.PGInterval(it.asText()) } + .filter { it.years != 0 || it.months != 0 } + assertTrue(calendar.size >= 4, "postgresRefused calendar intervals: $calendar") + for (pg in calendar) { + assertFailsWith("postgresRefused '$pg'") { read(pg) } + } + // The interval spelling the column type binds (Postgres refuses a signed ISO-8601 duration). + for (c in codecs.get("postgresText")) { + assertEquals(c.get("out").asText(), call("formatPostgres", call("parse", c.get("in").asText())), "postgresText ${c.get("in")}") + } + // The stored microsecond count (SQLite / MySQL), sub-microsecond digits truncated toward zero. + for (c in codecs.get("storedMicros")) { + val d = call("parse", c.get("in").asText()) as Duration + val micros = c.get("out").asText().toLong() + assertEquals(micros, call("toMicros", d), "storedMicros ${c.get("in")}") + assertEquals(call("format", d.truncatedTo(java.time.temporal.ChronoUnit.MICROS)), call("format", read(micros))) + } + assertEquals(-1_000_000L, call("toMicros", Duration.ofSeconds(-1, -500))) + assertFailsWith { call("toMicros", Duration.ofSeconds(Long.MAX_VALUE / 1_000_000L + 1)) } + assertEquals("-PT1.123456789S", call("format", Duration.ofSeconds(-1, -123_456_789))) + assertEquals("-00:00:01.123456", call("formatPostgres", Duration.ofSeconds(-1, -123_456_789))) + // Java's Duration range is the only limit, refused past it, never wrapped. + assertEquals("PT2562047788015215H", call("format", call("parse", "PT2562047788015215H"))) + assertFailsWith { call("parse", "PT9223372036854775808S") } + assertEquals(false, call("isDuration", "PT99999999999999999999H")) + // No default locale changes the digits. + val saved = java.util.Locale.getDefault() + try { + java.util.Locale.setDefault(java.util.Locale.forLanguageTag("ar-EG-u-nu-arab")) + assertEquals("-PT1M30.000005S", call("format", Duration.ofSeconds(-90, -5_000))) + assertEquals("-00:01:30.000005", call("formatPostgres", Duration.ofSeconds(-90, -5_000))) + } finally { + java.util.Locale.setDefault(saved) + } + } + + @Test + fun `native durations carry no MetaObjects annotation and round-trip through the host mapper`() { + val files = emit(listOf(KotlinEntityGenerator()), com.metaobjects.config.WireFormat.Durations.NATIVE) + val entity = files.getValue("acme/ops/Job.kt") + assertFalse("MetaDurationWire" in entity, entity) + val cl = compile(files) + val job = cl.loadClass("acme.ops.Job") + // The host mapper as Spring Boot configures it: java.time Duration as Duration.toString() / Duration.parse. + val module = com.fasterxml.jackson.databind.module.SimpleModule() + .addSerializer(Duration::class.java, com.fasterxml.jackson.databind.ser.std.ToStringSerializer.instance) + .addDeserializer(Duration::class.java, object : com.fasterxml.jackson.databind.deser.std.StdDeserializer(Duration::class.java) { + override fun deserialize(p: com.fasterxml.jackson.core.JsonParser, ctxt: com.fasterxml.jackson.databind.DeserializationContext): Duration = + Duration.parse(p.text) + }) + val mapper = ObjectMapper().registerModule(com.fasterxml.jackson.module.kotlin.KotlinModule.Builder().build()).registerModule(module) + val read = mapper.readValue( + """{"id":1,"holdFor":"PT1H30M","retries":["PT1S"],"window":{"span":"PT5400S"}}""", job) + val out = mapper.valueToTree(read) + assertEquals("PT1H30M", out.get("holdFor").asText()) + // Duration.parse takes the looser form the iso rule refuses. + mapper.readValue("""{"id":1,"holdFor":"pt1h","window":{"span":"PT1S"}}""", job) + } + + @Test + fun `a duration property writes the canonical string and reads the accepted form only`() { + val cl = compile(emit(listOf(KotlinEntityGenerator()))) + val job = cl.loadClass("acme.ops.Job") + for (mapper in listOf( + ObjectMapper().registerModule(com.fasterxml.jackson.module.kotlin.KotlinModule.Builder().build()), + ObjectMapper().findAndRegisterModules(), + )) { + val read = mapper.readValue( + """{"id":1,"holdFor":"-PT90S","retries":["P1D","PT0.5S"],"window":{"span":"PT5400S"}}""", job) + val out = mapper.valueToTree(read) + assertEquals("-PT1M30S", out.get("holdFor").asText()) + assertEquals("PT24H", out.get("retries").get(0).asText()) + assertEquals("PT0.5S", out.get("retries").get(1).asText()) + assertEquals("PT1H30M", out.get("window").get("span").asText()) + for (bad in listOf("\"P1M\"", "\"1:30:00\"", "5400", "\"pt1h\"", "\"pt1h\"", "\"PT-1H\"", "\"P1.5W\"")) { + assertFailsWith("must refuse $bad") { + mapper.readValue("""{"id":1,"holdFor":$bad}""", job) + } + } + } + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableCrossPackageEnumTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableCrossPackageEnumTest.kt index 99dbee9a25..1a00f383fa 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableCrossPackageEnumTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableCrossPackageEnumTest.kt @@ -78,14 +78,14 @@ class KotlinExposedTableCrossPackageEnumTest { val orderTable = readGenerated(outDir, "acme/orders/OrderTable.kt") assertTrue(orderTable.contains("import acme.common.RecordStatus"), "OrderTable must import the cross-package shared enum; saw:\n$orderTable") - assertTrue(orderTable.contains("RecordStatus::class"), - "OrderTable still references the enum in enumerationByName; saw:\n$orderTable") + assertTrue(orderTable.contains("RecordStatus.valueOf"), + "OrderTable still references the enum in its text enum column; saw:\n$orderTable") val invoiceTable = readGenerated(outDir, "acme/billing/InvoiceTable.kt") assertTrue(invoiceTable.contains("import acme.common.RecordStatus"), "InvoiceTable must import the cross-package shared enum; saw:\n$invoiceTable") - assertTrue(invoiceTable.contains("RecordStatus::class"), - "InvoiceTable still references the enum in enumerationByName; saw:\n$invoiceTable") + assertTrue(invoiceTable.contains("RecordStatus.valueOf"), + "InvoiceTable still references the enum in its text enum column; saw:\n$invoiceTable") // TPH-fold path: the base ShipmentTable.kt (NOT AirShipmentTable — TPH subtypes emit // no table of their own) must carry the import for the SUBTYPE-only cross-package @@ -93,7 +93,7 @@ class KotlinExposedTableCrossPackageEnumTest { val shipmentTable = readGenerated(outDir, "acme/shipping/ShipmentTable.kt") assertTrue(shipmentTable.contains("import acme.common.RecordStatus"), "TPH-folded ShipmentTable must import the cross-package shared enum; saw:\n$shipmentTable") - assertTrue(shipmentTable.contains("val recordStatus = enumerationByName(\"record_status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, RecordStatus::class).nullable()"), + assertTrue(shipmentTable.contains("val recordStatus = customEnumeration(\"record_status\", \"TEXT\", { v -> RecordStatus.valueOf(v as String) }, { e -> e.name }).nullable()"), "TPH-folded status column must reference the shared enum; saw:\n$shipmentTable") assertTrue(!Files.exists(outDir.resolve("acme/shipping/AirShipmentTable.kt")), "a TPH subtype must NOT emit its own table") diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGeneratorTest.kt index da516f79e6..a656fad07c 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGeneratorTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGeneratorTest.kt @@ -3,6 +3,7 @@ package com.metaobjects.generator.kotlin import com.metaobjects.metadata.ktx.loadString import java.nio.file.Files import kotlin.test.Test +import kotlin.test.assertFalse import kotlin.test.assertTrue class KotlinExposedTableGeneratorTest { @@ -83,6 +84,45 @@ class KotlinExposedTableGeneratorTest { } } + @Test fun `field duration emits the duration columns and the shared support files`() { + // field.duration → `durationColumn(...)` (Column: native INTERVAL on Postgres, + // the canonical ISO-8601 string elsewhere) and `durationArrayColumn(...)` for isArray, both + // backed by the package-shared MetaDurationColumnType.kt + MetaDurationWire.kt. + val fixture = """{ + "metadata.root": { "package": "acme::demo", "children": [ + { "object.entity": { "name": "Timer", "children": [ + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "timeout", "@required": true } }, + { "field.duration": { "name": "grace" } }, + { "field.duration": { "name": "backoff", "isArray": true } }, + { "source.rdb": { "@table": "timers" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + val outDir = Files.createTempDirectory("ktbl-duration-") + try { + val gen = KotlinExposedTableGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loadString("test", fixture)) + + val src = Files.readString(outDir.resolve("acme/demo/TimerTable.kt")) + assertTrue("val timeout = durationColumn(\"timeout\")" in src, src) + assertTrue("val grace = durationColumn(\"grace\").nullable()" in src, src) + assertTrue("val backoff = durationArrayColumn(\"backoff\")" in src, src) + + val sup = Files.readString(outDir.resolve("acme/demo/MetaDurationColumnType.kt")) + assertTrue("internal class MetaDurationColumnType : ColumnType()" in sup, sup) + assertTrue("if (isPostgres()) \"INTERVAL\"" in sup, sup) + assertTrue("override fun parameterMarker(value: Duration?): String = if (isPostgres()) \"CAST(? AS INTERVAL)\" else \"?\"" in sup, sup) + assertTrue("internal fun Table.durationArrayColumn(name: String): Column>" in sup, sup) + val wire = Files.readString(outDir.resolve("acme/demo/MetaDurationWire.kt")) + assertTrue("internal object MetaDurationWire" in wire, wire) + } finally { + outDir.toFile().deleteRecursively() + } + } + @Test fun `skips entities without source rdb child`() { val noSource = """{ "metadata.root": { "package": "x", "children": [ @@ -256,6 +296,107 @@ class KotlinExposedTableGeneratorTest { } } + // @flattenedColumnPrefix replaces the default `_` prefix, verbatim — "" keeps the + // members' own columns — through core naming (ColumnNaming.resolveFlattened), the names + // `meta migrate` creates for the same model. + @Test fun flattenedColumnPrefixNamesTheFlattenedColumns() { + val outDir = Files.createTempDirectory("ktbl-flat-prefix-") + try { + val model = """{ + "metadata.root": { "package": "acme::demo", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street", "@required": true } } + ] } }, + { "object.entity": { "name": "User", "children": [ + { "field.long": { "name": "id" } }, + { "field.object": { "name": "home", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "" } }, + { "field.object": { "name": "work", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "office_" } }, + { "source.rdb": { "@table": "users" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + val gen = KotlinExposedTableGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loadString("flat-prefix", model)) + + val src = Files.readString(outDir.resolve("acme/demo/UserTable.kt")) + assertTrue("val homeStreet = text(\"street\")" in src, + "expected the \"\" prefix to keep the member's own column; saw:\n$src") + assertTrue("val workStreet = text(\"office_street\")" in src, + "expected the office_ prefix; saw:\n$src") + } finally { + outDir.toFile().deleteRecursively() + } + } + + @Test fun flattenedColumnsKeepTheirLiteralWithTheNamesArtifactInTheRun() { + val outDir = Files.createTempDirectory("ktbl-flat-names-") + try { + val model = """{ + "metadata.root": { "package": "acme::demo", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street", "@required": true } } + ] } }, + { "object.entity": { "name": "User", "children": [ + { "field.long": { "name": "id" } }, + { "field.object": { "name": "work", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "office_" } }, + { "source.rdb": { "@table": "users" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + val gen = KotlinExposedTableGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString(), KotlinGenUtil.ARG_USE_NAMES to "true")) + gen.execute(loadString("flat-names", model)) + + val src = Files.readString(outDir.resolve("acme/demo/UserTable.kt")) + // An owned (ejected) names generator from before member constants does not emit + // UserNames.WORK_STREET_COLUMN, so the table must not reference it. + assertTrue("val workStreet = text(\"office_street\")" in src, + "expected the flattened column's literal; saw:\n$src") + assertFalse("WORK_STREET_COLUMN" in src, "saw:\n$src") + } finally { + outDir.toFile().deleteRecursively() + } + } + + private fun enumTable(statusDecl: String, maxLength: Long? = null): String { + val outDir = Files.createTempDirectory("ktbl-enum-len-") + try { + val model = """{ + "metadata.root": { "package": "acme::demo", "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", $statusDecl } }, + { "source.rdb": { "@table": "tickets" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + val loader = loadString("enum-len", model) + if (maxLength != null) loader.root.enumDefaults = com.metaobjects.config.EnumDefaults(maxLength) + val gen = KotlinExposedTableGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loader) + return Files.readString(outDir.resolve("acme/demo/TicketTable.kt")) + } finally { + outDir.toFile().deleteRecursively() + } + } + + @Test fun anEnumColumnTakesItsLengthFromMaxLengthThenTheProjectDefault() { + val values = "\"@values\": [\"OPEN\", \"CLOSED\"]" + val own = enumTable("$values, \"@maxLength\": 16") + assertTrue("enumerationByName(\"status\", 16, " in own, own) + val byDefault = enumTable(values, maxLength = 24) + assertTrue("enumerationByName(\"status\", 24, " in byDefault, byDefault) + // No length anywhere: `text`, as `meta migrate` creates it. + val none = enumTable(values) + assertTrue("customEnumeration(\"status\", \"TEXT\", " in none, none) + assertFalse("enumerationByName(\"status\"" in none, none) + } + @Test fun storageJsonbEmitsJsonbColumn() { val outDir = Files.createTempDirectory("ktbl-jsonb-") try { @@ -339,8 +480,8 @@ class KotlinExposedTableGeneratorTest { val src = Files.readString(playerTable) // Typed enum column, not a varchar. assertTrue( - "val status = enumerationByName(\"status\", 64, PlayerStatus::class)" in src, - "expected enumerationByName column; saw:\n$src", + "val status = customEnumeration(\"status\", \"TEXT\", { v -> PlayerStatus.valueOf(v as String) }, { e -> e.name })" in src, + "expected text enum column; saw:\n$src", ) assertTrue("varchar(\"status\"" !in src, "expected NO varchar fallback for the enum column; saw:\n$src") diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt index 6746d9f5ca..23132044ca 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt @@ -81,10 +81,10 @@ class KotlinExposedTableIntBackedEnumTest { "priority must not emit the string-backed column; saw:\n$src") // The string-backed sibling is unchanged — this feature is additive. - assertTrue("enumerationByName(\"status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, OrderStatus::class)" in src, - "string-backed status must keep enumerationByName; saw:\n$src") - assertTrue("customEnumeration(\"status\"" !in src, - "string-backed status must not become customEnumeration; saw:\n$src") + assertTrue("customEnumeration(\"status\", \"TEXT\"" in src, + "string-backed status (no length configured) is a text column; saw:\n$src") + assertTrue("customEnumeration(\"status\", \"INTEGER\"" !in src, + "string-backed status must not become the int-backed form; saw:\n$src") // Compile gate: proves both lambdas type-check against the real Exposed API. val result = compile(outDir) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableNamesTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableNamesTest.kt index 796f129554..070780573f 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableNamesTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableNamesTest.kt @@ -122,16 +122,16 @@ class KotlinExposedTableNamesTest { // above) -- asserted here so a fix that only patched the enum branch, and broke // the plain-column branch it sits beside, can't pass this test either. assertTrue("varchar(TicketNames.REFERENCE_COLUMN, 40)" in src, src) - assertTrue("enumerationByName(TicketNames.STATUS_COLUMN, 64, TicketStatus::class)" in src, src) + assertTrue("customEnumeration(TicketNames.STATUS_COLUMN, \"TEXT\", { v -> TicketStatus.valueOf(v as String) }, { e -> e.name })" in src, src) // The literal the strategy WOULD have produced must be GONE -- a positive // assertion alone would still pass a generator emitting BOTH the constant // reference and the old literal side by side. - assertFalse("enumerationByName(\"status\"" in src, src) + assertFalse("customEnumeration(\"status\"" in src, src) } @Test fun `an own field-enum column keeps its literal by default`() { val src = ticketTableSrc() - assertTrue("enumerationByName(\"status\", 64, TicketStatus::class)" in src, src) + assertTrue("customEnumeration(\"status\", \"TEXT\", { v -> TicketStatus.valueOf(v as String) }, { e -> e.name })" in src, src) assertTrue("varchar(\"reference\", 40)" in src, src) assertFalse("TicketNames" in src, src) } @@ -279,11 +279,11 @@ class KotlinExposedTableNamesTest { // The base's own columns: this entity's artifact (a hit — never redirected). assertTrue("object VehicleTable : Table(VehicleNames.SOURCE_PRIMARY_TABLE)" in src, src) assertTrue("long(VehicleNames.ID_COLUMN)" in src, src) - assertTrue("enumerationByName(VehicleNames.KIND_COLUMN, 64, VehicleKind::class)" in src, src) + assertTrue("customEnumeration(VehicleNames.KIND_COLUMN, \"TEXT\", { v -> VehicleKind.valueOf(v as String) }, { e -> e.name })" in src, src) // A same-package concrete subtype's column, through the enum-free fold arm... assertTrue("integer(CarNames.DOORS_COLUMN).nullable()" in src, src) // ...and through the enum arm. - assertTrue("enumerationByName(CarNames.GRADE_COLUMN, 64, VehicleGrade::class).nullable()" in src, src) + assertTrue("customEnumeration(CarNames.GRADE_COLUMN, \"TEXT\", { v -> VehicleGrade.valueOf(v as String) }, { e -> e.name }).nullable()" in src, src) // A cross-package subtype's column: the reference is package-qualified. assertTrue("integer(acme.haulage.TruckNames.PAYLOAD_COLUMN).nullable()" in src, src) // A column declared on an ABSTRACT intermediate resolves to that object's fragment. @@ -301,7 +301,7 @@ class KotlinExposedTableNamesTest { val src = vehicleTableSrc() assertTrue("object VehicleTable : Table(\"phys_vehicles\")" in src, src) assertTrue("integer(\"phys_doors\").nullable()" in src, src) - assertTrue("enumerationByName(\"phys_grade\", 64, VehicleGrade::class).nullable()" in src, src) + assertTrue("customEnumeration(\"phys_grade\", \"TEXT\", { v -> VehicleGrade.valueOf(v as String) }, { e -> e.name }).nullable()" in src, src) assertTrue("integer(\"phys_payload\").nullable()" in src, src) assertTrue("integer(\"phys_wheels\").nullable()" in src, src) assertFalse("Names" in src, src) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinM2mTraversalCompileRunTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinM2mTraversalCompileRunTest.kt index 8a74397516..b291a4867d 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinM2mTraversalCompileRunTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinM2mTraversalCompileRunTest.kt @@ -243,7 +243,12 @@ class KotlinM2mTraversalCompileRunTest { // constructor whose first two params are (Long, String) and pass defaults for the rest. val ctor = clazz.declaredConstructors .firstOrNull { c -> - c.parameterTypes.size >= 2 && + // Never the synthetic defaults constructor (`..., int mask, DefaultConstructorMarker`): + // declaredConstructors has no defined order, and a null for its int mask is the + // NullPointerException this test used to hit at random. + !c.isSynthetic && + c.parameterTypes.none { it.name == "kotlin.jvm.internal.DefaultConstructorMarker" } && + c.parameterTypes.size >= 2 && c.parameterTypes[0] == java.lang.Long::class.java && c.parameterTypes[1] == String::class.java } ?: error("no (Long, String, ...) constructor on ${clazz.name}") diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGeneratorTest.kt index ebc1a0dc83..7f856d2fe8 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGeneratorTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinNamesGeneratorTest.kt @@ -525,4 +525,116 @@ class KotlinNamesGeneratorTest { assertTrue("CREATED_AT" in e.message!!, e.message!!) } + + // ------------------------------------------------------------------------- + // Flattened value objects: one constant per member's column, none for the field + // ------------------------------------------------------------------------- + + private val flattenedModel = """{ + "metadata.root": { "package": "acme::ship", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } }, + { "field.string": { "name": "postalCode", "@column": "ZIP" } } + ] } }, + { "object.entity": { "name": "Shipment", "children": [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "field.object": { "name": "payload", "@objectRef": "Address" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + + @Test fun `a flattened member gets a column constant beside its field constants`() { + val src = emit(flattenedModel).getValue("acme/ship/ShipmentNames.kt") + + assertTrue("const val SHIP_TO_STREET_COLUMN: String = \"ship_to_street\"" in src, src) + assertTrue("const val SHIP_TO_POSTAL_CODE_COLUMN: String = \"ship_to_ZIP\"" in src, src) + assertTrue("const val SHIP_TO_FIELD: String = \"shipTo\"" in src, src) + assertTrue("const val PAYLOAD_COLUMN: String = \"payload\"" in src, src) + // A flattened field has no column of its own, so no _COLUMN constant and no map key. + assertFalse("SHIP_TO_COLUMN" in src, src) + assertFalse("Deprecated" in src, src) + assertFalse("DEPRECATION" in src, src) + assertFalse("\"shipTo\" to" in src, src) + assertTrue("\"shipTo.street\" to SHIP_TO_STREET_COLUMN" in src, src) + assertTrue("\"shipTo.postalCode\" to SHIP_TO_POSTAL_CODE_COLUMN" in src, src) + } + + /** + * A flattened field has no column constant: the generated names objects, a base and its + * child, compile with every warning an error, and reading the field's own column constant + * is a compile error. + */ + @Test fun `a flattened field has no column constant to read`() { + val model = """{ + "metadata.root": { "package": "acme::ship", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } } + ] } }, + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } } + ] } }, + { "object.entity": { "name": "Order", "extends": "acme::ship::Base", "children": [ + { "source.rdb": { "@table": "orders" } }, + { "field.string": { "name": "note" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + val files = emit(model).filterKeys { it.endsWith(".kt") } + val order = files.getValue("acme/ship/OrderNames.kt") + assertFalse("SHIP_TO_COLUMN" in order, order) + + fun compile(extra: List) = KotlinCompilation().apply { + this.sources = files.map { (path, text) -> SourceFile.kotlin(path.substringAfterLast('/'), text) } + extra + inheritClassPath = true + allWarningsAsErrors = true + messageOutputStream = System.out + }.compile() + + val clean = compile(emptyList()) + assertEquals(KotlinCompilation.ExitCode.OK, clean.exitCode, "generated names objects warn:\n${clean.messages}") + + val use = compile(listOf(SourceFile.kotlin("UsesIt.kt", + "package acme.ship\nobject UsesIt { val c: String = OrderNames.SHIP_TO_COLUMN }\n"))) + assertEquals(KotlinCompilation.ExitCode.COMPILATION_ERROR, use.exitCode, use.messages) + assertTrue("SHIP_TO_COLUMN" in use.messages, use.messages) + } + + @Test fun `a flattened member constant colliding with a field constant is left out`() { + val model = flattenedModel.replace( + "\"@storage\": \"flattened\" }", + "\"@storage\": \"flattened\", \"@flattenedColumnPrefix\": \"st_\" } },\n" + + " { \"field.string\": { \"name\": \"shipToStreet\" }") + val src = emit(model).getValue("acme/ship/ShipmentNames.kt") + + // The field's constant wins; the model generates, as it did before member constants. + assertTrue("const val SHIP_TO_STREET_COLUMN: String = \"ship_to_street\"" in src, src) + assertFalse("\"st_street\"" in src, src) + assertTrue("const val SHIP_TO_POSTAL_CODE_COLUMN: String = \"st_ZIP\"" in src, src) + } + + /** The cross-port naming corpus: every `..` key is a constant, under every rule. */ + @Test fun `every flattened column in the naming corpus is a constant`() { + var dir: Path? = Path.of("").toAbsolutePath() + while (dir != null && !Files.isDirectory(dir.resolve("fixtures"))) dir = dir.parent + val corpus = requireNotNull(dir) { "could not locate fixtures/" }.resolve("fixtures/naming-conformance/physical-names") + val model = corpus.resolve("meta.naming.json").readText() + val cases = com.fasterxml.jackson.databind.ObjectMapper().readTree(corpus.resolve("cases.json").toFile()).get("cases") + for (c in cases) { + val naming = c.get("naming") + val rule = if (naming != null && naming.has("columns")) naming.get("columns").asText() else "snake_case" + val src = emit(model, columnsRule = rule).getValue("acme/naming/ShipmentNames.kt") + for ((key, value) in c.get("columns").fields()) { + val parts = key.split(".") + if (parts.size != 3 || parts[0] != "Shipment") continue + val member = "${KotlinNaming.namesMember(parts[1])}_${KotlinNaming.namesMember(parts[2])}_COLUMN" + val decl = "const val $member: String = \"${value.asText()}\"" + assertTrue(decl in src, "${c.get("name").asText()}: $decl\n$src") + } + } + } } diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionExtendsInheritanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionExtendsInheritanceTest.kt index 62b46f0ad1..02558d0c0c 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionExtendsInheritanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionExtendsInheritanceTest.kt @@ -94,9 +94,9 @@ class KotlinProjectionExtendsInheritanceTest { "publishedAt must inherit the instant/TZ-aware timestamp (instantWithTimeZone); saw:\n$table") // --- enum gets the projection's OWN self-contained type, not root Status --- - assertTrue("enumerationByName(\"status\", " in table && "ProgramViewStatus::class" in table, + assertTrue("customEnumeration(\"status\", " in table && "ProgramViewStatus.valueOf" in table, "status must reference the projection's own ProgramViewStatus enum; saw:\n$table") - assertFalse(", Status::class)" in table, + assertFalse("{ v -> Status.valueOf" in table, "status must NOT collapse to a root-package `Status` enum; saw:\n$table") // The projection's own enum class is materialized in the projection's package, diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionTwoHopEnumTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionTwoHopEnumTest.kt index be1411d841..90c023e9e1 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionTwoHopEnumTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinProjectionTwoHopEnumTest.kt @@ -98,14 +98,14 @@ class KotlinProjectionTwoHopEnumTest { // The projection table references the SHARED enum. val viewTable = outDir.resolve("acme/commerce/ProgramViewTable.kt").readText() - assertTrue("SharedStatus::class" in viewTable, + assertTrue("SharedStatus.valueOf" in viewTable, "projection table must reference the shared SharedStatus enum; saw:\n$viewTable") - assertFalse("ProgramViewStatus::class" in viewTable, + assertFalse("ProgramViewStatus.valueOf" in viewTable, "projection table must NOT reference a per-projection enum; saw:\n$viewTable") // The ENTITY table still collapses onto the shared enum (byte-identity guardrail). val programTable = outDir.resolve("acme/commerce/ProgramTable.kt").readText() - assertTrue("SharedStatus::class" in programTable, + assertTrue("SharedStatus.valueOf" in programTable, "entity table must still reference the shared enum; saw:\n$programTable") // Compile-gate: the whole generated tree must resolve. @@ -131,7 +131,7 @@ class KotlinProjectionTwoHopEnumTest { "consumer; files=${Files.walk(outDir).toList()}") val viewTable = outDir.resolve("acme/commerce/ProgramViewTable.kt").readText() - assertTrue("SharedStatus::class" in viewTable, + assertTrue("SharedStatus.valueOf" in viewTable, "projection table must reference the shared SharedStatus enum; saw:\n$viewTable") val result = compile(outDir) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGeneratorTest.kt index 01c5bfdb8b..79bb654709 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGeneratorTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportExportGeneratorTest.kt @@ -16,7 +16,9 @@ import kotlin.io.path.readText import kotlin.test.Test import kotlin.test.assertEquals import kotlin.test.assertFailsWith +import kotlin.test.assertFalse import kotlin.test.assertNotEquals +import kotlin.test.assertTrue /** * The `report-export` generator (the "Report export" section of `docs/features/api-contract.md`): @@ -102,6 +104,65 @@ class KotlinReportExportGeneratorTest { } /** The shared writer, generated and compiled, with the entity row classes it never touches. */ + /** A served report over an entity with a `field.duration` dimension. */ + private val durationModel = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Call", "children": [ + { "source.rdb": { "@table": "calls" } }, + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "holdFor" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "holdFor", "@of": "Call.holdFor" } }, + { "measure.aggregate": { "name": "calls", "@agg": "count", "@of": "Call.id" } } + ] } }, + { "object.report": { "name": "CallsByHold", "@from": "Call", "@dimensions": ["holdFor"], "@measures": ["calls"], + "children": [ { "source.rdb": { "@kind": "view", "@table": "v_calls_by_hold" } } ] } } + ] } + }""".trimIndent() + + private fun emitDuration(): Map { + val outDir = Files.createTempDirectory("report-export-duration-") + try { + val gen = KotlinReportExportGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(com.metaobjects.metadata.ktx.loadString("report-export-duration", durationModel)) + val files = TreeMap() + Files.walk(outDir).use { st -> + st.filter { it.isRegularFile() }.forEach { files[outDir.relativize(it).toString()] = it.readText() } + } + return files + } finally { + outDir.toFile().deleteRecursively() + } + } + + @Test + fun `the duration cell is in the writer only when a served report has a duration column`() { + // No duration in the model: the writer is the one it always was, with no duration text at all. + val plain = emit(listOf(KotlinReportExportGenerator())).getValue("acme/shop/ReportExportWriter.kt") + assertFalse("uration" in plain, plain) + // A duration column: the export calls the duration cell, and the writer differs from the + // plain one by exactly that factory. + val files = emitDuration() + val export = files.getValue("acme/shop/CallsByHoldExport.kt") + assertTrue("ReportExportWriter.duration(row.holdFor)" in export, export) + val withDuration = files.getValue("acme/shop/ReportExportWriter.kt") + assertTrue("fun duration(value: java.time.Duration?): Cell" in withDuration, withDuration) + val start = withDuration.indexOf(" /** A field.duration in its canonical") + val end = withDuration.indexOf(" /** A decimal as the wire setting writes it") + assertEquals(plain, withDuration.substring(0, start) + withDuration.substring(end)) + // And it compiles and writes the canonical form. + val result = KotlinCompilation().apply { + sources = listOf(SourceFile.kotlin("ReportExportWriter.kt", withDuration)) + inheritClassPath = true + messageOutputStream = java.io.OutputStream.nullOutputStream() + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + val w = W(result.classLoader.loadClass("acme.shop.ReportExportWriter")) + val cell = w.call("duration", java.time.Duration.ofSeconds(-90, -500_000_000))!! + assertEquals("-PT1M30.5S", cell.javaClass.getMethod("getText").invoke(cell)) + } + private fun writer(args: Map, wire: WireFormat = WireFormat.DEFAULT): Class<*> { val files = emit(listOf(KotlinReportExportGenerator()), args, wire) val result = KotlinCompilation().apply { @@ -155,7 +216,7 @@ class KotlinReportExportGeneratorTest { val native = W(writer(emptyMap())) assertEquals("a\r\n$jackson\r\n", native.csv(listOf("a"), listOf(listOf(native.decimal(value))))) // string: the canonical form (ADR-0019 Amendment 1) - val pinned = W(writer(emptyMap(), WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT))) + val pinned = W(writer(emptyMap(), WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO))) assertEquals( "a,b,c\r\n12.5,0,1000\r\n", pinned.csv(listOf("a", "b", "c"), listOf(listOf( diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt index cf830479b8..0d3e4f920c 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt @@ -237,7 +237,7 @@ class KotlinReportRestSurfaceTest { val row = files.getValue("fitness/ProgramsByMonth.kt") val table = files.getValue("fitness/ProgramsByMonthTable.kt") assertTrue(" public val status: ProgramStatus" in row, row) - assertTrue("ProgramStatus::class" in table, table) + assertTrue("ProgramStatus.valueOf(" in table, table) // No generator materializes an enum for the report. assertFalse(files.keys.any { "ProgramsByMonthStatus" in it }, files.keys.toString()) assertFalse("ProgramsByMonthStatus" in row, row) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt index 12234e2c2e..97c3df3aa7 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt @@ -155,7 +155,7 @@ class KotlinReportTableGeneratorTest { assertTrue(" val createdAtMonth = date(\"createdAtMonth\")\n" in src, src) // Program.status's own generated class — no generator emits a ProgramsByMonthStatus. assertTrue( - " val status = enumerationByName(\"status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, ProgramStatus::class)\n" in src, + " val status = customEnumeration(\"status\", \"TEXT\", { v -> ProgramStatus.valueOf(v as String) }, { e -> e.name })\n" in src, src, ) assertFalse("ProgramsByMonthStatus" in src, src) @@ -557,7 +557,7 @@ class KotlinReportTableGeneratorTest { // Store's class, and not nullable: Store.tier is required and the only hop is a required // reference, an INNER join (D5). assertTrue( - " val storeTier = enumerationByName(\"store_tier\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, StoreTier::class)\n" in src, + " val storeTier = customEnumeration(\"store_tier\", \"TEXT\", { v -> StoreTier.valueOf(v as String) }, { e -> e.name })\n" in src, src, ) // An int-backed enum keeps its mapping, typed by Sale's class. @@ -630,7 +630,7 @@ class KotlinReportTableGeneratorTest { val src = files.getValue("b/RTable.kt") assertTrue(" val kind = varchar(\"kind\", 12).nullable()\n" in src, src) // The enum class of the @from entity, which is the class the entity generator emits. - assertTrue("EvTier::class).nullable()\n" in src, src) + assertTrue("EvTier.valueOf(v as String) }, { e -> e.name }).nullable()\n" in src, src) assertTrue(" val events = long(\"events\")\n" in src, src) assertCompiles(files.filterKeys { !it.startsWith("b/Base") }) } diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapperTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapperTest.kt index 994f357d04..236b75f7c3 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapperTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapperTest.kt @@ -20,6 +20,7 @@ import com.metaobjects.field.TimestampField import com.metaobjects.field.UuidField import com.metaobjects.field.UriField import com.metaobjects.field.InetField +import com.metaobjects.field.DurationField import com.metaobjects.metadata.ktx.loadString import com.squareup.kotlinpoet.BOOLEAN import com.squareup.kotlinpoet.ClassName @@ -157,6 +158,21 @@ class KotlinTypeMapperTest { assertTrue(KotlinTypeMapper.usesInetUriHelper(f)) } + @Test fun `duration field maps to java time Duration and a file-local durationColumn`() { + // field.duration: native java.time.Duration property; the Exposed column is the file-local + // `durationColumn(...)` (Column over native INTERVAL, a BIGINT count of microseconds elsewhere) and, with + // isArray, `durationArrayColumn(...)`. + val f = DurationField("holdFor") + val tn = KotlinTypeMapper.kotlinTypeName(f) as ClassName + assertEquals("java.time", tn.packageName) + assertEquals("Duration", tn.simpleName) + assertEquals("durationColumn(\"hold_for\")", KotlinTypeMapper.exposedColumnSpec(f)) + assertTrue(KotlinTypeMapper.usesDurationHelper(f)) + val arr = DurationField("retries") + arr.setArray(true) + assertEquals("durationArrayColumn(\"retries\")", KotlinTypeMapper.exposedColumnSpec(arr)) + } + @Test fun `string field with isArray derives native text array column`() { // Phase 1 (dbColumnType slim-and-derive): `field.string` + `isArray:true` derives a // native Postgres `text[]` (TextColumnType element) — no `@dbColumnType=text_array`. @@ -354,11 +370,14 @@ class KotlinTypeMapperTest { assertEquals("long(\"price_cents\")", KotlinTypeMapper.exposedColumnSpec(f)) } - @Test fun `enum field maps to String and varchar(64) exposed column`() { + @Test fun `enum field maps to String and a text exposed column when no length is configured`() { val f = EnumField("status") - // v1 enum representation: String + VARCHAR; full enum-class emission deferred. + // v1 enum representation: String; full enum-class emission deferred. No @maxLength and + // no project enums.maxLength: `text`, as `meta migrate` creates it. assertEquals(STRING, KotlinTypeMapper.kotlinTypeName(f)) - assertEquals("varchar(\"status\", 64)", KotlinTypeMapper.exposedColumnSpec(f)) + assertEquals("text(\"status\")", KotlinTypeMapper.exposedColumnSpec(f)) + f.addMetaAttr(IntAttribute.create("maxLength", 32)) + assertEquals("varchar(\"status\", 32)", KotlinTypeMapper.exposedColumnSpec(f)) } @Test fun enumTypeNameComputesClassName() { diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ValidationConformanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ValidationConformanceTest.kt index 23f56987e1..92c30ff758 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ValidationConformanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ValidationConformanceTest.kt @@ -51,7 +51,7 @@ class ValidationConformanceTest { runCorpus("meta.json", "cases.json", "Account", "acme.auth") } - /** Entity-scoped cross-field validators enforced by class-level Bean Validation on the data class. */ + /** Entity-scoped cross-field validators (incl. exactlyOne and @condition) enforced by class-level Bean Validation on the data class. */ @Test fun `generated Event data class enforces the cross-field verdicts`() { runCorpus("meta.cross-field.json", "cross-field-cases.json", "Event", "acme.events") } @@ -64,6 +64,12 @@ class ValidationConformanceTest { try { // --- generate the Account data class with jakarta.validation annotations --- val loader = loadString("validation-conformance", metaJson) + // The corpus pins the cross-port duration rule (strict ISO-8601), which is wire.durations: iso. + loader.root.wireFormat = com.metaobjects.config.WireFormat( + com.metaobjects.config.WireFormat.Decimals.NATIVE, + com.metaobjects.config.WireFormat.NullMembers.NATIVE, + com.metaobjects.config.WireFormat.Durations.ISO, + ) val gen = KotlinEntityGenerator() gen.setArgs(mapOf("outputDir" to outDir.toString())) gen.execute(loader) diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-enum/acme/game/PlayerTable.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-enum/acme/game/PlayerTable.kt index 4e91a3997d..324a73d7b1 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-enum/acme/game/PlayerTable.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-enum/acme/game/PlayerTable.kt @@ -6,7 +6,7 @@ import org.jetbrains.exposed.sql.Table object PlayerTable : Table("players") { val id = long("id").autoIncrement() val username = varchar("username", 50).nullable() - val status = enumerationByName("status", 64, PlayerStatus::class).nullable() + val status = customEnumeration("status", "TEXT", { v -> PlayerStatus.valueOf(v as String) }, { e -> e.name }).nullable() override val primaryKey = PrimaryKey(id) } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthTable.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthTable.kt index fef003a3be..3c9fa910fb 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthTable.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthTable.kt @@ -5,7 +5,7 @@ import org.jetbrains.exposed.sql.Table /** GENERATED — do not hand-edit. Regenerated from metadata. */ object AuthTable : Table("auths") { val id = long("id").autoIncrement() - val type = enumerationByName("type", 64, AuthType::class).nullable() + val type = customEnumeration("type", "TEXT", { v -> AuthType.valueOf(v as String) }, { e -> e.name }).nullable() val reference = varchar("reference", 80) val quantity = integer("quantity").nullable() val copayAmount = portableDecimal("copay_amount", 10, 2).nullable() diff --git a/server/java/codegen-mustache/src/test/java/com/metaobjects/generator/mustache/HelperRegistryPhysicalNamingTest.java b/server/java/codegen-mustache/src/test/java/com/metaobjects/generator/mustache/HelperRegistryPhysicalNamingTest.java index 4cb8b7b7b3..7a2a90e6f2 100644 --- a/server/java/codegen-mustache/src/test/java/com/metaobjects/generator/mustache/HelperRegistryPhysicalNamingTest.java +++ b/server/java/codegen-mustache/src/test/java/com/metaobjects/generator/mustache/HelperRegistryPhysicalNamingTest.java @@ -76,6 +76,11 @@ public void tableAndColumnHelpersFollowEveryCorpusNamingRule() throws IOExceptio } for (Iterator> it = c.get("columns").fields(); it.hasNext(); ) { Map.Entry e = it.next(); + // `..` is a flattened value object's member column. It + // belongs to two fields, and dbColumnName takes one: no helper addresses it, so + // the corpus's flattened keys are core naming's (ColumnNaming.resolveFlattened) + // to pin, in PhysicalNamingConformanceTest. + if (e.getKey().chars().filter(ch -> ch == '.').count() > 1) continue; String[] parts = e.getKey().split("\\.", 2); MetaField field = object(loader, parts[0]).getMetaField(parts[1]); assertEquals(name + ": dbColumnName of " + e.getKey(), e.getValue().asText(), diff --git a/server/java/codegen-spring/pom.xml b/server/java/codegen-spring/pom.xml index 4b4949f0ee..b3e9ebd6cf 100644 --- a/server/java/codegen-spring/pom.xml +++ b/server/java/codegen-spring/pom.xml @@ -286,6 +286,9 @@ CanonicalDecimal.java CanonicalDecimalDeserializer.java CanonicalDecimalSerializer.java + CanonicalDuration.java + CanonicalDurationDeserializer.java + CanonicalDurationSerializer.java DeclaredDecimal.java ConstraintErrors.java FilterParseResult.java diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java index 817825ecb3..e733380680 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java @@ -269,11 +269,11 @@ public String toString() { } } - /** What the generated controller imports — the whole filter pipeline, the patch and + /** What the generated controller imports — the whole filter pipeline (with the decimal and duration operand parsers), the patch and * constraint error mapping, and the record-component escape. Declared before * {@code REGISTRY}: static initializers run in order. */ private static final List ROUTES_RUNTIME = List.of( - "CanonicalDecimal", "ConstraintErrors", "FilterParseResult", "FilterParser", "FilterPredicate", + "CanonicalDecimal", "CanonicalDuration", "ConstraintErrors", "FilterParseResult", "FilterParser", "FilterPredicate", "PatchValidationException", "RecordComponentNames", "ReportExport", "UnreadableBody"); /** The writer a generated type's package and class name are read from, and its base. */ @@ -441,13 +441,15 @@ private static Map buildRegistry() { register(m, "dto", SpringDtoGenerator.class.getName(), "Per-entity Spring DTO record.", Tier.NATIVE, Layer.MODEL, ejectPath(SpringDtoGenerator.class), List.of("CanonicalDecimal", "CanonicalDecimalDeserializer", - "CanonicalDecimalSerializer", "DeclaredDecimal", "PatchValidationException"), null, RECORD_EMIT); + "CanonicalDecimalSerializer", "CanonicalDuration", "CanonicalDurationDeserializer", + "CanonicalDurationSerializer", "DeclaredDecimal", "PatchValidationException"), null, RECORD_EMIT); register(m, "value-object", SpringValueObjectGenerator.class.getName(), "Per-value-object Spring record with jakarta constraints — the typed " + "component the DTO/Patch bind for a field.object @storage:jsonb column, " + "and a template's payload/response type.", Tier.NATIVE, Layer.MODEL, ejectPath(SpringValueObjectGenerator.class), List.of("CanonicalDecimal", - "CanonicalDecimalDeserializer", "CanonicalDecimalSerializer", "DeclaredDecimal"), null, RECORD_EMIT); + "CanonicalDecimalDeserializer", "CanonicalDecimalSerializer", "CanonicalDuration", + "CanonicalDurationDeserializer", "CanonicalDurationSerializer", "DeclaredDecimal"), null, RECORD_EMIT); register(m, "trace-helper", LlmTraceHelperGenerator.class.getName(), "Per-entity typed record LLM-trace helper (extract + buildLlmCallRow + persist; " + "LlmCallBase-derived entities only).", Tier.NATIVE, Layer.CAPABILITY, diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java index 8fd89f6a62..c57ad29df0 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java @@ -838,7 +838,7 @@ private static void appendListHandler(StringBuilder src, String shortName, Strin src.append(" }\n"); src.append(" FilterParseResult filter = FilterParser.parse(\n"); src.append(" request.getQueryString(), ").append(allowlistName).append(".FIELDS, ") - .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS);\n"); + .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS, ").append(allowlistName).append(".DURATION_FIELDS);\n"); src.append(" if (filter.error() != null) {\n"); src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", filter.error(), \"field\", filter.field()));\n"); src.append(" }\n"); @@ -940,7 +940,7 @@ private static void appendReportListHandlerBody(StringBuilder src, String shortN src.append(" }\n"); src.append(" FilterParseResult filter = FilterParser.parse(\n"); src.append(" request.getQueryString(), ").append(allowlistName).append(".FIELDS, ") - .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS);\n"); + .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS, ").append(allowlistName).append(".DURATION_FIELDS);\n"); src.append(" if (filter.error() != null) {\n"); src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", filter.error(), \"field\", filter.field()));\n"); src.append(" }\n"); @@ -1027,7 +1027,7 @@ private static void appendReportExportHandler(StringBuilder src, String shortNam src.append(" }\n"); src.append(" FilterParseResult filter = FilterParser.parse(\n"); src.append(" request.getQueryString(), ").append(allowlistName).append(".FIELDS, ") - .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS);\n"); + .append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS, ").append(allowlistName).append(".DURATION_FIELDS);\n"); src.append(" if (filter.error() != null) {\n"); src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", filter.error(), \"field\", filter.field()));\n"); src.append(" }\n"); @@ -1233,7 +1233,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append(" if (sortClause == null) return ResponseEntity.badRequest().body(Map.of(\"error\", \"invalid_sort\", \"field\", sort.split(\":\", 2)[0]));\n"); src.append(" }\n"); src.append(" FilterParseResult filter = FilterParser.parse(request.getQueryString(), ") - .append(allowlistName).append(".FIELDS, ").append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS);\n"); + .append(allowlistName).append(".FIELDS, ").append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS, ").append(allowlistName).append(".DURATION_FIELDS);\n"); src.append(" if (filter.error() != null) return ResponseEntity.badRequest().body(Map.of(\"error\", filter.error(), \"field\", filter.field()));\n"); src.append(" List filters = filter.predicates();\n"); src.append(" List<").append(dtoName).append("> rows = repository.list(actualLimit, actualOffset, sortClause, filters);\n"); @@ -1306,7 +1306,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append(" if (sortClause == null) return ResponseEntity.badRequest().body(Map.of(\"error\", \"invalid_sort\", \"field\", sort.split(\":\", 2)[0]));\n"); src.append(" }\n"); src.append(" FilterParseResult filter = FilterParser.parse(request.getQueryString(), ") - .append(allowlistName).append(".FIELDS, ").append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS);\n"); + .append(allowlistName).append(".FIELDS, ").append(allowlistName).append(".OPS_BY_FIELD, ").append(allowlistName).append(".DECIMAL_FIELDS, ").append(allowlistName).append(".DURATION_FIELDS);\n"); src.append(" if (filter.error() != null) return ResponseEntity.badRequest().body(Map.of(\"error\", filter.error(), \"field\", filter.field()));\n"); src.append(" return ResponseEntity.ok(").append(redactAll( "repository.listByType(\"" + disc + "\", actualLimit, actualOffset, sortClause, filter.predicates())", diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java index 01f96b6bad..f0d5e80bde 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java @@ -378,6 +378,8 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" Map assigned = new LinkedHashMap<>();\n"); boolean usesDecimalPatch = false; boolean usesDecimalListPatch = false; + boolean usesDurationPatch = false; + boolean usesDurationListPatch = false; for (MetaField field : settable) { String name = field.getName(); String type = patchComponentType(field, entity, dtoName); @@ -404,6 +406,16 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin usesDecimalListPatch = true; continue; } + if (wire.durationsIso() + && ("java.time.Duration".equals(type) || "java.util.List".equals(type))) { + // wire.durations: iso. field.duration binds through the strict accepted-form rule, not the mapper's + // Duration.parse (which takes fractional hours, lowercase and numbers of seconds). + boolean list = !"java.time.Duration".equals(type); + src.append(" bindDuration").append(list ? "List" : "").append("(body, assigned, \"") + .append(name).append("\", ").append(required).append(");\n"); + if (list) usesDurationListPatch = true; else usesDurationPatch = true; + continue; + } src.append(" bind(body, mapper, assigned, \"").append(name) .append("\", new TypeReference<").append(type).append(">() {}, ") .append(required).append(");\n"); @@ -475,6 +487,51 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" }\n"); } } + if (usesDurationPatch || usesDurationListPatch) { + String pve = "import " + runtimePackage + ".PatchValidationException;\n"; + int at = src.indexOf(pve); + src.insert(at, "import " + runtimePackage + ".CanonicalDurationDeserializer;\n"); + } + if (usesDurationPatch) { + src.append("\n private static void bindDuration(JsonNode body, Map assigned,\n"); + src.append(" String field, boolean required) {\n"); + src.append(" if (!body.has(field)) return; // absent → untouched\n"); + src.append(" JsonNode node = body.get(field);\n"); + src.append(" if (node.isNull()) {\n"); + src.append(" if (required) throw new PatchValidationException(field);\n"); + src.append(" assigned.put(field, null); // explicit null → clear\n"); + src.append(" return;\n"); + src.append(" }\n"); + src.append(" try {\n"); + src.append(" assigned.put(field, CanonicalDurationDeserializer.fromNode(node));\n"); + src.append(" } catch (IllegalArgumentException e) {\n"); + src.append(" throw new PatchValidationException(field);\n"); + src.append(" }\n"); + src.append(" }\n"); + } + if (usesDurationListPatch) { + src.append("\n private static void bindDurationList(JsonNode body, Map assigned,\n"); + src.append(" String field, boolean required) {\n"); + src.append(" if (!body.has(field)) return; // absent → untouched\n"); + src.append(" JsonNode node = body.get(field);\n"); + src.append(" if (node.isNull()) {\n"); + src.append(" if (required) throw new PatchValidationException(field);\n"); + src.append(" assigned.put(field, null); // explicit null → clear\n"); + src.append(" return;\n"); + src.append(" }\n"); + src.append(" if (!node.isArray()) throw new PatchValidationException(field);\n"); + src.append(" java.util.List items = new java.util.ArrayList<>();\n"); + src.append(" for (JsonNode item : node) {\n"); + src.append(" if (item.isNull()) throw new PatchValidationException(field);\n"); + src.append(" try {\n"); + src.append(" items.add(CanonicalDurationDeserializer.fromNode(item));\n"); + src.append(" } catch (IllegalArgumentException e) {\n"); + src.append(" throw new PatchValidationException(field);\n"); + src.append(" }\n"); + src.append(" }\n"); + src.append(" assigned.put(field, items);\n"); + src.append(" }\n"); + } src.append("}\n"); writeJavaFile(entity, outRoot, pkg, patchName, src.toString()); @@ -564,11 +621,18 @@ private void emitRecord(MetaObject entity, Path outRoot, List fields, // deserializer (JsonDeserialize is imported once, for either reason). boolean usesDecimal = fields.stream() .anyMatch(f -> SpringGeneratorSupport.isDecimalComponent(componentTypeFr019(f, entity))); + // field.duration: a Duration component binds the canonical serializer / strict deserializer. + boolean usesDuration = wire.durationsIso() && fields.stream() + .anyMatch(f -> SpringGeneratorSupport.isDurationComponent(componentTypeFr019(f, entity))); if (usesDecimal) { src.append(SpringGeneratorSupport.decimalAnnotationImports(runtimePackage, wire)); - } else if (usesNetDeser) { + } else if (usesNetDeser || usesDuration) { src.append("import com.fasterxml.jackson.databind.annotation.JsonDeserialize;\n"); } + if (usesDuration) { + src.append(SpringGeneratorSupport.durationAnnotationImports( + runtimePackage, usesDecimal && wire.decimalsAsString())); + } // Wildcard import keeps the emit simple — record components carry a small, // closed set of jakarta.validation.constraints annotations. if (usesValid) { @@ -577,7 +641,7 @@ private void emitRecord(MetaObject entity, Path outRoot, List fields, if (usesValidation || crossField) { src.append("import jakarta.validation.constraints.*;\n"); } - if (usesDecimal || usesNetDeser || usesValid || usesValidation || crossField) { + if (usesDecimal || usesNetDeser || usesDuration || usesValid || usesValidation || crossField) { src.append('\n'); } src.append("/** GENERATED — wire DTO for ").append(shortName) @@ -599,6 +663,8 @@ private void emitRecord(MetaObject entity, Path outRoot, List fields, if (!annotations.isEmpty()) src.append(annotations).append(' '); String decimalJson = SpringGeneratorSupport.decimalJsonAnnotations(type, runtimePackage, field, wire); if (!decimalJson.isEmpty()) src.append(decimalJson).append(' '); + String durationJson = SpringGeneratorSupport.durationJsonAnnotations(type, wire); + if (!durationJson.isEmpty()) src.append(durationJson).append(' '); if (!jsonName.isEmpty()) src.append(jsonName).append(' '); src.append(type).append(' ').append(component); if (i < fields.size() - 1) src.append(','); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java index 03af95a822..b89019041a 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java @@ -160,6 +160,11 @@ private static boolean isDecimalTyped(MetaField field) { return !field.isArrayType() && field instanceof com.metaobjects.field.DecimalField; } + /** True for a scalar (non-array) {@code field.duration}: the DTO declares it {@code java.time.Duration}. */ + private static boolean isDurationTyped(MetaField field) { + return !field.isArrayType() && field instanceof com.metaobjects.field.DurationField; + } + private static Set opsForField(MetaField field) { // Single source of truth — com.metaobjects.query.FilterOps (the same // band the loader's validation path reads). Returns a canonical-ordered @@ -211,6 +216,19 @@ protected void emit(MetaObject entity, Path outRoot, MetaDataLoader loader) { } } + // The duration-typed filterable fields. Under wire.durations: iso an operand is a canonical + // ISO-8601 span (the accepted form, compared as a span); under native it stays the raw string. + java.util.List durationFields = new java.util.ArrayList<>(); + if (loader.getRoot().getWireFormat().durationsIso()) { + java.util.List durationCandidates = new java.util.ArrayList<>(); + entity.getMetaFields().forEach(durationCandidates::add); + if (TphPlan.isTphBase(entity, loader)) durationCandidates.addAll(TphPlan.collectSubtypeFields(entity, loader)); + for (MetaField f : durationCandidates) { + if (opsByField.containsKey(f.getName()) && isDurationTyped(f) + && !durationFields.contains(f.getName())) durationFields.add(f.getName()); + } + } + StringBuilder src = new StringBuilder(); if (!pkg.isEmpty()) { src.append("package ").append(pkg).append(";\n\n"); @@ -276,6 +294,16 @@ protected void emit(MetaObject entity, Path outRoot, MetaDataLoader loader) { } src.append(");\n"); + src.append("\n /** Filterable fields typed duration under wire.durations: iso: their operands parse as a span. */\n"); + src.append(" public static final Set DURATION_FIELDS = Set.of("); + boolean firstDuration = true; + for (String f : durationFields) { + if (!firstDuration) src.append(", "); + firstDuration = false; + src.append('"').append(f).append('"'); + } + src.append(");\n"); + // No instances — pure constants holder. src.append("\n private ").append(className).append("() {}\n"); src.append("}\n"); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringGeneratorSupport.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringGeneratorSupport.java index ab308ab864..536aaf5a73 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringGeneratorSupport.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringGeneratorSupport.java @@ -14,9 +14,10 @@ import com.metaobjects.config.WireFormat; import com.metaobjects.field.BooleanField; import com.metaobjects.field.CurrencyField; -import com.metaobjects.field.DecimalField; import com.metaobjects.field.DateField; +import com.metaobjects.field.DecimalField; import com.metaobjects.field.DoubleField; +import com.metaobjects.field.DurationField; import com.metaobjects.field.EnumField; import com.metaobjects.field.FloatField; import com.metaobjects.field.IntegerField; @@ -38,6 +39,7 @@ import com.metaobjects.validator.ArrayValidator; import com.metaobjects.validator.AtLeastOneValidator; import com.metaobjects.validator.ComparisonValidator; +import com.metaobjects.validator.ExactlyOneValidator; import com.metaobjects.validator.PresentIffValidator; import com.metaobjects.validator.RequiredWhenValidator; import com.metaobjects.validator.LengthValidator; @@ -664,25 +666,37 @@ private static boolean isTemporalCrossField(MetaField f) { /** * The comparison expression for two fields, or null when the pair has no sound one. Mirrors the - * C# generator: numerics compare with each other, strings and temporals only with their own + * C# generator: numerics compare with each other, strings, temporals and durations only with their own * subtype and Java type (a date has no ordering against a timestamp, nor an Instant against a * {@code @localTime} LocalDateTime, and casting both to Comparable would * throw ClassCastException inside the constraint method, turning a create into a 500), and * boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, like every other port. */ - private static String crossComparison(MetaField l, MetaField r, String lc, String rc, String op, String jop) { + private static boolean orderableCrossField(MetaField f) { + return isNumericCrossField(f) || f instanceof StringField || isTemporalCrossField(f) || f instanceof DurationField; + } + + private static boolean equatableCrossField(MetaField f) { + return f instanceof BooleanField || f instanceof EnumField || f instanceof UuidField + || f instanceof UriField || f instanceof InetField; + } + + /** Whether the pair has a sound comparison (see {@link #crossComparison}); emits nothing. */ + private static boolean crossComparable(MetaField l, MetaField r, String op) { boolean ordering = op.equals("gt") || op.equals("gte") || op.equals("lt") || op.equals("lte"); boolean sameSubType = l.getSubType().equals(r.getSubType()); - if (isNumericCrossField(l) && isNumericCrossField(r) && !sameSubType) { + if (isNumericCrossField(l) && isNumericCrossField(r) && !sameSubType) return true; + if (!sameSubType) return false; + if (!orderableCrossField(l) && !(!ordering && equatableCrossField(l))) return false; + return SpringTypeMapper.javaTypeName(l).equals(SpringTypeMapper.javaTypeName(r)); + } + + private static String crossComparison(MetaField l, MetaField r, String lc, String rc, String op, String jop) { + if (!crossComparable(l, r, op)) return null; + if (isNumericCrossField(l) && isNumericCrossField(r) && !l.getSubType().equals(r.getSubType())) { return "new java.math.BigDecimal(String.valueOf(" + lc + ")).compareTo(new java.math.BigDecimal(String.valueOf(" + rc + "))) " + jop + " 0"; } - if (!sameSubType) return null; - boolean comparable = isNumericCrossField(l) || l instanceof StringField || isTemporalCrossField(l); - boolean equatable = !ordering && (l instanceof BooleanField || l instanceof EnumField - || l instanceof UuidField || l instanceof UriField || l instanceof InetField); - if (!comparable && !equatable) return null; - if (!SpringTypeMapper.javaTypeName(l).equals(SpringTypeMapper.javaTypeName(r))) return null; - if (comparable) { + if (orderableCrossField(l)) { return "((Comparable) (Object) " + lc + ").compareTo(" + rc + ") " + jop + " 0"; } return (op.equals("eq") ? "" : "!") + "java.util.Objects.equals(" + lc + ", " + rc + ")"; @@ -726,7 +740,8 @@ private static String crossEqualsExpr(String ref, MetaField when, String raw) /** * Record-body members enforcing the entity-scoped cross-field validators - * ({@code validator.comparison / requiredWhen / presentIff / atLeastOne}) on the FULL create + * ({@code validator.comparison / requiredWhen / presentIff / atLeastOne / exactlyOne}, each with an optional + * {@code @condition}) on the FULL create * DTO: one {@code @AssertTrue} {@code isXxxValid()} method per validator (Bean Validation * reads it as a class-level rule; {@code @JsonIgnore} keeps it off the wire). Null-safe in * the DB CHECK sense (SQL NULL = Java null; "" is not null). A validator whose referenced @@ -790,7 +805,8 @@ public static List> crossFieldRules( expr = v instanceof RequiredWhenValidator ? "!" + gate + " || " + tc + " != null" : "(" + tc + " != null) == " + gate; - } else if (v instanceof AtLeastOneValidator) { + } else if (v instanceof AtLeastOneValidator || v instanceof ExactlyOneValidator) { + // atLeastOne and exactlyOne both name @fields; they differ only in the count they allow. Object raw = v.hasMetaAttr(AtLeastOneValidator.ATTR_FIELDS) ? v.getMetaAttr(AtLeastOneValidator.ATTR_FIELDS).getValue() : null; List names = new ArrayList<>(); @@ -801,10 +817,19 @@ public static List> crossFieldRules( for (String n : names) { MetaField f = crossField(entity, n, components); if (f == null) { missing = true; break; } - parts.add(ref.apply(f) + " != null"); + parts.add(ref.apply(f) + (v instanceof AtLeastOneValidator ? " != null" : " != null ? 1 : 0")); } if (missing) continue; - expr = String.join(" || ", parts); + expr = v instanceof AtLeastOneValidator + ? String.join(" || ", parts) + : "((" + String.join(") + (", parts) + ")) == 1"; + } + if (expr != null && v.hasMetaAttr(MetaValidator.ATTR_CONDITION)) { + // The CHECK is NOT (condition) OR (rule): the rule binds only when the condition is TRUE + // under SQL three-valued logic; FALSE and NULL (unknown) let the record through. + String cond = conditionExpr(v.getMetaAttr(MetaValidator.ATTR_CONDITION).getValue(), entity, components, ref); + if (cond == null) continue; // names a field the create model lacks: left to the database + expr = "!Boolean.TRUE.equals(" + cond + ") || (" + expr + ")"; } if (expr == null) continue; out.add(Map.entry(v, expr)); @@ -812,6 +837,151 @@ public static List> crossFieldRules( return out; } + // --- @condition: a three-valued (Boolean, null = unknown) Java expression ---------------------- + // Generated code stays self-contained (JDK types only); there is no runtime expression interpreter. + + private enum CompareKind { NUMBER, DURATION, PLAIN } + + private static CompareKind compareKind(MetaField f) { + if (f == null) return CompareKind.PLAIN; + if (isNumericCrossField(f)) return CompareKind.NUMBER; + if (f instanceof com.metaobjects.field.DurationField) return CompareKind.DURATION; + return CompareKind.PLAIN; + } + + /** The field a value node compares as: itself, or a coalesce's first field. */ + private static MetaField nodeField(Object node, MetaObject entity, List components) { + if (!(node instanceof Map n)) return null; + if (n.get("field") instanceof String name) return crossField(entity, name, components); + if ("coalesce".equals(n.get("fn")) && n.get("args") instanceof java.util.Collection args) { + for (Object a : args) { + MetaField f = nodeField(a, entity, components); + if (f != null) return f; + } + } + return null; + } + + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + private static List> nodeFields(Object node, MetaObject entity, List components) { + List> out = new ArrayList<>(); + if (!(node instanceof Map n)) return out; + if (n.get("field") instanceof String name) { + MetaField f = crossField(entity, name, components); + if (f != null) out.add(f); + } else if ("coalesce".equals(n.get("fn")) && n.get("args") instanceof java.util.Collection args) { + for (Object a : args) out.addAll(nodeFields(a, entity, components)); + } + return out; + } + + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + private static boolean conditionComparable(Object left, Object right, String op, + MetaObject entity, List components) { + List> fields = nodeFields(left, entity, components); + fields.addAll(nodeFields(right, entity, components)); + for (int i = 0; i < fields.size(); i++) { + for (int j = i + 1; j < fields.size(); j++) { + if (!crossComparable(fields.get(i), fields.get(j), op)) return false; + } + } + return true; + } + + /** A value node (field, literal, coalesce) as a Java expression compared as {@code kind}, or null. */ + private static String conditionValue(Object node, CompareKind kind, MetaObject entity, + List components, java.util.function.Function, String> ref) { + if (!(node instanceof Map n)) return null; + if (n.get("field") instanceof String name) { + MetaField f = crossField(entity, name, components); + if (f == null || f.isArrayType()) return null; + String r = ref.apply(f); + return switch (kind) { + case NUMBER -> "(" + r + " == null ? null : new java.math.BigDecimal(String.valueOf(" + r + ")))"; + case DURATION -> r; + case PLAIN -> "(" + r + " == null ? null : String.valueOf(" + r + "))"; + }; + } + if (n.containsKey("value")) { + Object value = n.get("value"); + if (value == null) return null; + String text = String.valueOf(value); + switch (kind) { + case NUMBER: + return "new java.math.BigDecimal(\"" + escapeJavaString(text) + "\")"; + case DURATION: + try { + java.time.Duration d = com.metaobjects.field.DurationWire.parse(text); + return "java.time.Duration.ofSeconds(" + d.getSeconds() + "L, " + d.getNano() + "L)"; + } catch (RuntimeException e) { + return null; + } + default: + return "\"" + escapeJavaString(text) + "\""; + } + } + if ("coalesce".equals(n.get("fn")) && n.get("args") instanceof java.util.Collection args) { + List parts = new ArrayList<>(); + for (Object a : args) { + String part = conditionValue(a, kind, entity, components, ref); + if (part == null) return null; + parts.add(part); + } + return "java.util.Arrays.asList(" + String.join(", ", parts) + + ").stream().filter(java.util.Objects::nonNull).findFirst().orElse(null)"; + } + return null; + } + + /** A condition node as a Java {@code Boolean} expression (null = SQL unknown), or null when unresolvable. */ + private static String conditionExpr(Object node, MetaObject entity, List components, + java.util.function.Function, String> ref) { + if (!(node instanceof Map n)) return null; + Object op = n.get("op"); + if (!(op instanceof String opName)) return null; + String jop = switch (opName) { + case "gt" -> ">"; case "gte" -> ">="; case "lt" -> "<"; case "lte" -> "<="; + case "ne" -> "!="; case "eq" -> "=="; default -> null; }; + if (jop != null) { + if (!conditionComparable(n.get("left"), n.get("right"), opName, entity, components)) return null; + MetaField typed = nodeField(n.get("left"), entity, components); + if (typed == null) typed = nodeField(n.get("right"), entity, components); + CompareKind kind = compareKind(typed); + String l = conditionValue(n.get("left"), kind, entity, components, ref); + String r = conditionValue(n.get("right"), kind, entity, components, ref); + if (l == null || r == null) return null; + return "((java.util.function.BiFunction, Object, Boolean>) (l_, r_) -> " + + "l_ == null || r_ == null ? null : Boolean.valueOf(l_.compareTo(r_) " + jop + " 0))" + + ".apply((Comparable) (Object) " + l + ", " + r + ")"; + } + if (opName.equals("isNull") || opName.equals("isNotNull")) { + String a = conditionValue(n.get("arg"), CompareKind.PLAIN, entity, components, ref); + if (a == null) return null; + return "Boolean.valueOf(" + a + (opName.equals("isNull") ? " == null)" : " != null)"); + } + if (opName.equals("not")) { + String a = conditionExpr(n.get("arg"), entity, components, ref); + if (a == null) return null; + return "((java.util.function.Function) v_ -> v_ == null ? null : Boolean.valueOf(!v_)).apply(" + a + ")"; + } + if ((opName.equals("and") || opName.equals("or")) && n.get("args") instanceof java.util.Collection args) { + List parts = new ArrayList<>(); + for (Object a : args) { + String part = conditionExpr(a, entity, components, ref); + if (part == null) return null; + parts.add(part); + } + boolean and = opName.equals("and"); + String decisive = and ? "Boolean.FALSE" : "Boolean.TRUE"; + String other = and ? "Boolean.TRUE" : "Boolean.FALSE"; + return "((java.util.function.Function, Boolean>) v_ -> v_.contains(" + decisive + + ") ? " + decisive + " : v_.contains(null) ? null : " + other + + ").apply(java.util.Arrays.asList(" + String.join(", ", parts) + "))"; + } + return null; + } + /** Escape a regex/string literal for safe embedding in generated Java source. */ public static String escapeJavaString(String s) { return com.metaobjects.generator.util.JavaIdentifiers.escapeStringLiteral(s); @@ -944,4 +1114,41 @@ public static String decimalAnnotationImports(String runtimePackage, WireFormat + "import " + runtimePackage + ".CanonicalDecimalDeserializer;\n" + "import " + runtimePackage + ".DeclaredDecimal;\n"; } + + /** The element type string of a duration component (see {@link #durationJsonAnnotations}). */ + private static final String DURATION_TYPE = "java.time.Duration"; + private static final String DURATION_LIST_TYPE = "java.util.List"; + + /** True iff a record component of Java type {@code type} carries the duration wire binding. */ + public static boolean isDurationComponent(String type) { + return DURATION_TYPE.equals(type) || DURATION_LIST_TYPE.equals(type); + } + + /** + * {@code field.duration}: the Jackson annotations binding a {@code java.time.Duration} record + * component to the canonical ISO-8601 serializer and the strict deserializer shipped in the + * runtime package. Both halves ride in every wire mode: a host ObjectMapper's JavaTimeModule + * writes a Duration as decimal seconds (or {@code Duration.toString()}, which differs from the + * canonical form for negatives), and {@code Duration.parse} accepts more than the accepted form. + * A {@code List} binds its elements ({@code contentUsing}). Empty for any other type, + * and under {@code wire.durations: native} (the default), where the host ObjectMapper's own + * {@code java.time} handling writes and reads the duration with no override. + */ + public static String durationJsonAnnotations(String type, WireFormat wire) { + if (!wire.durationsIso() || !isDurationComponent(type)) return ""; + String key = DURATION_LIST_TYPE.equals(type) ? "contentUsing" : "using"; + return "@JsonSerialize(" + key + " = CanonicalDurationSerializer.class) " + + "@JsonDeserialize(" + key + " = CanonicalDurationDeserializer.class)"; + } + + /** + * The imports {@link #durationJsonAnnotations} needs beyond {@code JsonDeserialize} (which the + * caller imports once for every reason it is used): {@code JsonSerialize} unless + * {@code serializeImported}, and the two runtime classes. + */ + public static String durationAnnotationImports(String runtimePackage, boolean serializeImported) { + return (serializeImported ? "" : "import com.fasterxml.jackson.databind.annotation.JsonSerialize;\n") + + "import " + runtimePackage + ".CanonicalDurationDeserializer;\n" + + "import " + runtimePackage + ".CanonicalDurationSerializer;\n"; + } } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringNamesGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringNamesGenerator.java index fc13918616..0173de7d6b 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringNamesGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringNamesGenerator.java @@ -10,6 +10,7 @@ import com.metaobjects.database.ColumnNaming; import com.metaobjects.database.IndexNaming; import com.metaobjects.field.MetaField; +import com.metaobjects.field.ObjectField; import com.metaobjects.generator.EmitsPhysicalNameConstants; import com.metaobjects.generator.GeneratorException; import com.metaobjects.generator.GeneratorIOWriter; @@ -112,6 +113,8 @@ public class SpringNamesGenerator extends MultiFileDirectGeneratorBase implements EmitsPhysicalNameConstants { + private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(SpringNamesGenerator.class); + @Override protected Class getFilterClass() { return MetaObject.class; @@ -164,7 +167,17 @@ public void execute(MetaDataLoader loader) { // ------------------------------------------------------------------------- /** Physical name + logical field name for one field. */ - private record FieldNames(String name, String column) {} + /** + * One field's names. {@code members} is set only on a {@code @storage: "flattened"} value + * object, and there it is the only column information: such a field has no column of its + * own ({@code column} is null, and no {@code _COLUMN} constant is emitted for it), and each + * member of the value object is a column of the owning table, named by core naming + * ({@link ColumnNaming#resolveFlattened(MetaField, MetaField, String)}, the rule + * {@code meta migrate} names them by), keyed by the member's field name. + */ + private record FieldNames(String name, String column, Map members) { + FieldNames(String name, String column) { this(name, column, null); } + } /** * One {@code source.rdb} child, under the ROLE it plays. @@ -344,10 +357,37 @@ private static Map keysOf(Collection nodes return out; } + /** + * The flattened test is migrate's own: a {@code field.object} whose RESOLVED + * {@code @storage} is {@code flattened}. Members are the value object's resolving fields. + */ + private static FieldNames fieldNamesOf(MetaField field, String strategy) { + if (!(field instanceof ObjectField) + || !field.hasMetaAttr(ObjectField.ATTR_STORAGE, true) + || !ObjectField.STORAGE_FLATTENED.equals( + field.getMetaAttr(ObjectField.ATTR_STORAGE, true).getValueAsString())) { + return new FieldNames(field.getName(), ColumnNaming.resolve(field, strategy)); + } + Map members = new LinkedHashMap<>(); + MetaObject target = ((ObjectField) field).getObjectRef(); + if (target != null) { + for (MetaField member : target.getMetaFields()) { + members.put(member.getName(), new FieldNames(member.getName(), + ColumnNaming.resolveFlattened(field, member, strategy))); + } + } + return new FieldNames(field.getName(), null, members); + } + + /** The constant naming a flattened member's column: {@code SHIP_TO_STREET_COLUMN}. */ + static String flattenedColumnMember(String fieldName, String memberName) { + return namesMember(fieldName) + "_" + namesMember(memberName) + "_COLUMN"; + } + private static Map fieldMap(Collection src, String strategy) { Map out = new LinkedHashMap<>(); for (MetaField field : src) { - out.put(field.getName(), new FieldNames(field.getName(), ColumnNaming.resolve(field, strategy))); + out.put(field.getName(), fieldNamesOf(field, strategy)); } return out; } @@ -442,10 +482,32 @@ static String namesMember(String name) { * one blank line between the object's own identity, its sources, its fields, its * identities and its indexes. It is never read as data.

*/ - private record Member(String member, String literal, String path, String section) {} + private record Member(String member, String literal, String path, String section, boolean flattened) {} private static Member str(String member, String value, String path, String section) { - return new Member(member, "\"" + value + "\"", path, section); + return new Member(member, "\"" + value + "\"", path, section, false); + } + + /** A flattened value object's member column: left out, not refused, when it collides. */ + private static Member flattenedStr(String member, String value, String path) { + return new Member(member, "\"" + value + "\"", path, "fields", true); + } + + /** + * The flattened-member constants of {@code names} that would share a name with another + * constant of the same class — a field {@code shipToStreet} beside the member + * {@code street} of a flattened {@code shipTo}. They are left out, with a warning, rather + * than refused: such a model generated before member constants existed, and every other + * constant it had is still emitted. Computed over the WHOLE resolved set, so a class and + * its base agree about what is skipped. + */ + private static Set collidingFlattenedMembers(ObjectNames names) { + Set taken = new HashSet<>(); + Set skipped = new java.util.TreeSet<>(); + List all = rawMembersOf(names, true); + for (Member m : all) if (!m.flattened()) taken.add(m.member()); + for (Member m : all) if (m.flattened() && !taken.add(m.member())) skipped.add(m.member()); + return skipped; } /** @return true when a file was written. */ @@ -464,6 +526,29 @@ protected boolean emit(MetaObject entity, Path outRoot, String strategy, boolean ? null : SpringNaming.namesName(SpringNaming.splitFqn(names.superObject().getName())[1]); + for (String skipped : collidingFlattenedMembers(names)) { + LOG.warn("names: {}: the flattened column constant {} would share its name with another " + + "constant of the class, so it is not emitted. Rename the field or the value-object " + + "member to get it.", entity.getName(), skipped); + } + + // Java static inheritance makes every base constant reachable through this class too. + // When this object resolves a field differently from its base (a flattened field + // redeclared as jsonb, say), a base constant can name a column this table does not + // have; then the class is emitted complete, with no base, rather than inheriting it. + if (superClass != null) { + MetaObject sup = names.superObject(); + ObjectNames supNames = resolveObjectNames(sup, strategy); + if (supNames == null) supNames = resolveSuperFragmentNames(sup, strategy); + if (supNames != null) { + Set own = new HashSet<>(); + for (Member m : membersOf(names, true)) own.add(m.member()); + for (Member m : membersOf(supNames, true)) { + if (!own.contains(m.member())) { superClass = null; break; } + } + } + } + // Two nodes whose SCREAMING_SNAKE member forms collide would emit duplicate // constants. javac would refuse to compile the file, but the error would name a // generated .java and read as a codegen bug rather than a model one. Fail here, @@ -531,15 +616,28 @@ protected boolean emit(MetaObject entity, Path outRoot, String strategy, boolean // artifact removes. It repeats no LITERAL: an inherited entry's value is the base's // own constant, reached through Java's static-member inheritance. Hiding the base's // field of the same name is what makes Names.COLUMNS_BY_FIELD the complete one. + Set skippedMembers = collidingFlattenedMembers(names); List allRows = new ArrayList<>(); for (FieldNames f : names.fields().values()) { - allRows.add(new String[] { namesMember(f.name()), f.name() }); + // A flattened field has no column of its own: only its members are keyed. + if (f.members() == null) { + allRows.add(new String[] { namesMember(f.name()) + "_COLUMN", f.name() }); + } else { + // A flattened member is keyed `.`, the path the naming corpus uses. + for (FieldNames m : f.members().values()) { + String constant = flattenedColumnMember(f.name(), m.name()); + if (!skippedMembers.contains(constant)) { + allRows.add(new String[] { constant, f.name() + "." + m.name() }); + } + } + } } allRows.sort((a, b) -> a[1].compareTo(b[1])); - src.append("\n public static final Map COLUMNS_BY_FIELD = Map.ofEntries(\n"); + src.append("\n"); + src.append(" public static final Map COLUMNS_BY_FIELD = Map.ofEntries(\n"); for (int i = 0; i < allRows.size(); i++) { String[] row = allRows.get(i); - src.append(" Map.entry(\"").append(row[1]).append("\", ").append(row[0]).append("_COLUMN)"); + src.append(" Map.entry(\"").append(row[1]).append("\", ").append(row[0]).append(")"); src.append(i < allRows.size() - 1 ? ",\n" : "\n"); } src.append(" );\n"); @@ -576,6 +674,15 @@ protected boolean emit(MetaObject entity, Path outRoot, String strategy, boolean * are what a consumer reaches.

*/ private static List membersOf(ObjectNames names, boolean all) { + Set skipped = collidingFlattenedMembers(names); + List out = new ArrayList<>(); + for (Member m : rawMembersOf(names, all)) { + if (!(m.flattened() && skipped.contains(m.member()))) out.add(m); + } + return out; + } + + private static List rawMembersOf(ObjectNames names, boolean all) { List out = new ArrayList<>(); String self = names.name(); out.add(str("TYPE", names.type(), self, "self")); @@ -606,7 +713,16 @@ private static List membersOf(ObjectNames names, boolean all) { String member = namesMember(f.name()); String path = self + ".fields." + f.name(); out.add(str(member + "_FIELD", f.name(), path, "fields")); - out.add(str(member + "_COLUMN", f.column(), path, "fields")); + // A flattened field has no column of its own: one constant per member's column. + if (f.members() == null) { + out.add(str(member + "_COLUMN", f.column(), path, "fields")); + } else { + for (String m : sorted(f.members())) { + FieldNames mn = f.members().get(m); + out.add(flattenedStr(flattenedColumnMember(f.name(), mn.name()), mn.column(), + path + "." + mn.name())); + } + } } addKeys(out, self, "identities", all ? names.identities() : names.ownIdentities()); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringOutputParserGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringOutputParserGenerator.java index aaef1eeb25..ad398cfe60 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringOutputParserGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringOutputParserGenerator.java @@ -511,6 +511,7 @@ private static String coercerFor(MetaField field) { if (field instanceof com.metaobjects.field.InetField) return SpringTypeMapper.javaTypeName(field).endsWith("InetAddress") ? "coerceInet" : COERCE_STRING; + if (field instanceof com.metaobjects.field.DurationField) return "coerceDuration"; return COERCE_STRING; } @@ -672,6 +673,16 @@ private static void appendCoercionHelpers(StringBuilder src, java.util.Set null. */\n"); + src.append(" private static java.time.Duration coerceDuration(Object v) {\n"); + src.append(" if (v instanceof java.time.Duration t) return t;\n"); + src.append(" if (v == null) return null;\n"); + src.append(" try { return com.metaobjects.field.DurationWire.parse(v.toString().trim()); }\n"); + src.append(" catch (IllegalArgumentException e) { return null; }\n"); + src.append(" }\n"); + } if (used.contains("coerceInet")) { src.append("\n"); src.append(" /** IP LITERALS only — see the guard below. */\n"); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringReportExportGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringReportExportGenerator.java index 561f915a4a..0130cb7b90 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringReportExportGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringReportExportGenerator.java @@ -10,6 +10,7 @@ import com.metaobjects.field.DecimalField; import com.metaobjects.field.EnumField; import com.metaobjects.field.InetField; +import com.metaobjects.field.DurationField; import com.metaobjects.field.MapField; import com.metaobjects.field.MetaField; import com.metaobjects.field.StringField; @@ -162,7 +163,8 @@ public void execute(MetaDataLoader loader) { String writerPackage = getArg(ARG_WRITER_PACKAGE, sharedPackage(served)); boolean decimalsAsString = loader.getRoot().getWireFormat().decimalsAsString(); - write(outRoot, writerPackage, WRITER_NAME, writerSource(writerPackage, xmlNulls, csvBom, decimalsAsString)); + write(outRoot, writerPackage, WRITER_NAME, + writerSource(writerPackage, xmlNulls, csvBom, decimalsAsString, usesDurationCell(served))); for (MetaObject entity : served) { String[] split = SpringNaming.splitFqn(entity.getName()); write(outRoot, split[0], split[1] + "Export", @@ -269,6 +271,7 @@ private static String cellFactory(MetaField f) { return "text"; } if (DecimalField.SUBTYPE_DECIMAL.equals(sub)) return "decimal"; + if (DurationField.SUBTYPE_DURATION.equals(sub)) return "duration"; return "raw"; } @@ -277,7 +280,19 @@ private static String cellFactory(MetaField f) { // ----------------------------------------------------------------------- private static String writerSource(String pkg, String xmlNulls, boolean csvBom, boolean decimalsAsString) { - String text = WRITER_TEMPLATE + return writerSource(pkg, xmlNulls, csvBom, decimalsAsString, false); + } + + /** + * The shared writer; {@code duration} adds the {@code duration(Duration)} cell factory, and is + * set only when a served report has a scalar {@code field.duration} column, so a model without + * one gets the writer it always got, byte for byte. + */ + static String writerSource(String pkg, String xmlNulls, boolean csvBom, boolean decimalsAsString, + boolean duration) { + String template = duration ? WRITER_TEMPLATE.replace(DURATION_CELL_ANCHOR, DURATION_CELL + DURATION_CELL_ANCHOR) + : WRITER_TEMPLATE; + String text = template .replace("@PACKAGE@", pkg.isEmpty() ? "" : "package " + pkg + ";\n\n") .replace("@BOM@", String.valueOf(csvBom)) .replace("@NIL@", String.valueOf(XML_NULLS_NIL.equals(xmlNulls))) @@ -285,6 +300,48 @@ private static String writerSource(String pkg, String xmlNulls, boolean csvBom, return text; } + /** True when a served report has a scalar {@code field.duration} column (an array is a JSON cell). */ + static boolean usesDurationCell(List served) { + for (MetaObject entity : served) { + List credentials = SpringGeneratorSupport.credentialNames(entity); + for (MetaField f : SpringGeneratorSupport.dtoComponentFields(entity)) { + if (!credentials.contains(f.getName()) && "duration".equals(cellFactory(f))) return true; + } + } + return false; + } + + /** Where {@link #DURATION_CELL} goes in the writer: just before the decimal cell factory. */ + private static final String DURATION_CELL_ANCHOR = + " /** A decimal as the wire setting writes it: under \"string\" canonical plain notation (no\n"; + + /** The {@code field.duration} cell factory, emitted only for a model that has one. */ + private static final String DURATION_CELL = """ + /** A field.duration in its canonical ISO-8601 wire form ([-]PT[nH][nM][n[.f]S], to the + * nanosecond), as the list route writes it; Duration.toString() differs for a negative span. */ + public static Cell duration(java.time.Duration value) { + if (value == null) return NULL; + boolean negative = value.isNegative(); + java.time.Duration mag = negative ? value.negated() : value; + long total = mag.getSeconds(); + long nanos = mag.getNano(); + if (total == 0 && nanos == 0) return new Cell("PT0S", false); + StringBuilder b = new StringBuilder(negative ? "-PT" : "PT"); + if (total / 3600 > 0) b.append(total / 3600).append('H'); + if (total % 3600 / 60 > 0) b.append(total % 3600 / 60).append('M'); + if (total % 60 > 0 || nanos > 0) { + b.append(total % 60); + if (nanos > 0) { + String digits = Long.toString(nanos); + b.append('.').append(("000000000".substring(digits.length()) + digits).replaceAll("0+$", "")); + } + b.append('S'); + } + return new Cell(b.toString(), false); + } + +"""; + /** The shared writer, verbatim but for the four substitutions in {@link #writerSource}. */ private static final String WRITER_TEMPLATE = """ @PACKAGE@import com.fasterxml.jackson.databind.ObjectMapper; diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringTypeMapper.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringTypeMapper.java index 30732f0017..e9a2efb61a 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringTypeMapper.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringTypeMapper.java @@ -18,6 +18,7 @@ import com.metaobjects.field.UuidField; import com.metaobjects.field.UriField; import com.metaobjects.field.InetField; +import com.metaobjects.field.DurationField; import com.metaobjects.database.CoreDBMetaDataProvider; import com.metaobjects.identity.MetaIdentity; import com.metaobjects.object.MetaObject; @@ -127,6 +128,10 @@ public static String javaTypeName(MetaField field) { // stays a String; the DB column is the Postgres-native inet type. #234: a @lenient // field.inet degrades to a plain String (no native InetAddress type, no validator). if (field instanceof InetField) return isLenientNetField(field) ? "String" : "java.net.InetAddress"; + // duration — native java.time.Duration binding. The wire stays the canonical ISO-8601 + // string (bound per component by the CanonicalDuration serializer/deserializer, since a + // host ObjectMapper writes a Duration as seconds); the DB column is the Postgres interval. + if (field instanceof DurationField) return "java.time.Duration"; // field.object @objectRef → the referenced value object's generated record type // (Program D). A jsonb value-object column surfaces as the record (single) or // — via componentType wrapping — List<> (isArray). Fully-qualified so the diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java index f88164d0bb..982b294177 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java @@ -160,6 +160,14 @@ private void emit(MetaObject vo, Path outRoot) { if (usesDecimal) { src.append(SpringGeneratorSupport.decimalAnnotationImports(runtimePackage, wire)); } + // field.duration: a Duration component binds the canonical serializer / strict deserializer. + boolean usesDuration = wire.durationsIso() && fields.stream() + .anyMatch(f -> SpringGeneratorSupport.isDurationComponent(componentTypeFr019(f, vo))); + if (usesDuration) { + if (!usesDecimal) src.append("import com.fasterxml.jackson.databind.annotation.JsonDeserialize;\n"); + src.append(SpringGeneratorSupport.durationAnnotationImports( + runtimePackage, usesDecimal && wire.decimalsAsString())); + } if (usesValid) { src.append("import jakarta.validation.Valid;\n"); } @@ -188,6 +196,8 @@ private void emit(MetaObject vo, Path outRoot) { if (!a.isEmpty()) src.append(a).append(' '); String decimalJson = SpringGeneratorSupport.decimalJsonAnnotations(type, runtimePackage, field, wire); if (!decimalJson.isEmpty()) src.append(decimalJson).append(' '); + String durationJson = SpringGeneratorSupport.durationJsonAnnotations(type, wire); + if (!durationJson.isEmpty()) src.append(durationJson).append(' '); if (!jsonName.isEmpty()) src.append(jsonName).append(' '); src.append(type).append(' ').append(component); if (i < fields.size() - 1) src.append(','); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDuration.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDuration.java new file mode 100644 index 0000000000..55151b6bc1 --- /dev/null +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDuration.java @@ -0,0 +1,108 @@ +package com.metaobjects.generator.spring.runtime; + +import java.time.Duration; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * The {@code field.duration} wire rule for a generated record component + * ({@code fixtures/duration-wire/}): a {@link Duration} is written in the canonical ISO-8601 form + * {@code [-]PT[H][M][[.]S]} and read from the accepted form only. + * + *

{@link Duration#parse} accepts more than the accepted form (fractional hours, per-component + * signs, lowercase designators) and refuses weeks and more than nine fractional digits, and + * {@link Duration#toString()} differs from the canonical form for negative spans, so neither is + * used on the wire. A value keeps nanoseconds (a {@link Duration}'s precision); fractional digits + * past the ninth are truncated toward zero, never rounded.

+ * + *

Self-contained (no MetaObjects imports) so an ejected runtime copy compiles on its own; the + * same rule is {@code com.metaobjects.field.DurationWire} in the metadata module.

+ */ +public final class CanonicalDuration { + + private CanonicalDuration() {} + + /** + * The accepted ISO-8601 duration form: weeks (exactly 7 days), days (exactly 24 hours), hours, + * minutes, seconds with a fraction of any length; no years or months. + */ + public static final Pattern ACCEPTED = Pattern.compile( + "^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$"); + + /** What a refused value should have been. */ + public static final String EXPECTED = "an ISO-8601 duration such as PT1H30M (no years or months)"; + + /** A value that is not a duration. */ + public static final class InvalidDurationException extends IllegalArgumentException { + private static final long serialVersionUID = 1L; + public InvalidDurationException(String message) { super(message); } + } + + /** True iff {@code s} is in the accepted form and fits a {@link Duration} (refused past it, never wrapped). */ + public static boolean isDuration(String s) { + if (s == null || !ACCEPTED.matcher(s).matches()) return false; + try { + parse(s); + return true; + } catch (InvalidDurationException outOfRange) { + return false; + } + } + + /** + * Parse an accepted ISO-8601 duration (digits past a nanosecond truncated toward zero). A + * {@link Duration}'s own range is the only limit. + * + * @throws InvalidDurationException when {@code s} is not in the accepted form or overflows + */ + public static Duration parse(String s) { + Matcher m = s == null ? null : ACCEPTED.matcher(s); + if (m == null || !m.matches()) { + throw new InvalidDurationException("field.duration: not " + EXPECTED + ": \"" + s + "\""); + } + try { + long seconds = Math.addExact( + Math.addExact(Math.multiplyExact(num(m.group(2)), 604_800L), Math.multiplyExact(num(m.group(3)), 86_400L)), + Math.addExact(Math.multiplyExact(num(m.group(4)), 3_600L), + Math.addExact(Math.multiplyExact(num(m.group(5)), 60L), num(m.group(6))))); + String frac = m.group(7); + long nanos = frac == null ? 0L + : Long.parseLong(frac.length() >= 9 ? frac.substring(0, 9) : (frac + "000000000").substring(0, 9)); + Duration d = Duration.ofSeconds(seconds, nanos); + return "-".equals(m.group(1)) ? d.negated() : d; + } catch (ArithmeticException | NumberFormatException e) { + throw new InvalidDurationException("field.duration: out of range: \"" + s + "\""); + } + } + + /** The canonical wire form of {@code d}, to the nanosecond (trailing zeros dropped). */ + public static String format(Duration d) { + boolean negative = d.isNegative(); + Duration mag = negative ? d.negated() : d; + long total = mag.getSeconds(); + long nanos = mag.getNano(); + if (total == 0 && nanos == 0) return "PT0S"; + StringBuilder b = new StringBuilder(negative ? "-PT" : "PT"); + long h = total / 3_600L; + long mi = (total % 3_600L) / 60L; + long s = total % 60L; + if (h > 0) b.append(h).append('H'); + if (mi > 0) b.append(mi).append('M'); + if (s > 0 || nanos > 0) { + b.append(s); + if (nanos > 0) { + String digits = Long.toString(nanos); + String f = "000000000".substring(digits.length()) + digits; + int end = f.length(); + while (f.charAt(end - 1) == '0') end--; + b.append('.').append(f, 0, end); + } + b.append('S'); + } + return b.toString(); + } + + private static long num(String digits) { + return digits == null ? 0L : Long.parseLong(digits); + } +} diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationDeserializer.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationDeserializer.java new file mode 100644 index 0000000000..7549260d15 --- /dev/null +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationDeserializer.java @@ -0,0 +1,50 @@ +package com.metaobjects.generator.spring.runtime; + +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonToken; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; + +import java.io.IOException; +import java.time.Duration; + +/** + * Strict Jackson deserializer for a {@code field.duration} record component: a JSON string in + * the accepted ISO-8601 form ({@link CanonicalDuration#ACCEPTED}). Everything else — a number of + * seconds, a calendar span ({@code P1M}), a clock string ({@code 1:30:00}), a lowercase or + * fractional-hour form {@link Duration#parse} would take — is an error Spring maps to {@code 400}. + * JSON {@code null} never reaches this class: Jackson binds it to {@code null}. + */ +public final class CanonicalDurationDeserializer extends StdDeserializer { + + private static final long serialVersionUID = 1L; + + public CanonicalDurationDeserializer() { + super(Duration.class); + } + + @Override + public Duration deserialize(JsonParser p, DeserializationContext ctxt) throws IOException { + if (p.currentToken() != JsonToken.VALUE_STRING) { + return (Duration) ctxt.handleUnexpectedToken(Duration.class, p); + } + try { + return CanonicalDuration.parse(p.getText()); + } catch (CanonicalDuration.InvalidDurationException e) { + return (Duration) ctxt.handleWeirdStringValue(Duration.class, p.getText(), "%s", e.getMessage()); + } + } + + /** + * The same rule over an already-parsed tree node, for the generated {@code Patch} + * (which reads a body as a {@code JsonNode}). + * + * @throws CanonicalDuration.InvalidDurationException for a node that is not an accepted + * duration string + */ + public static Duration fromNode(JsonNode node) { + if (node.isTextual()) return CanonicalDuration.parse(node.asText()); + throw new CanonicalDuration.InvalidDurationException("field.duration: not a string: " + node); + } +} diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationSerializer.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationSerializer.java new file mode 100644 index 0000000000..520502341d --- /dev/null +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDurationSerializer.java @@ -0,0 +1,29 @@ +package com.metaobjects.generator.spring.runtime; + +import com.fasterxml.jackson.core.JsonGenerator; +import com.fasterxml.jackson.databind.SerializerProvider; +import com.fasterxml.jackson.databind.ser.std.StdSerializer; + +import java.io.IOException; +import java.time.Duration; + +/** + * Jackson serializer for a {@code field.duration} record component: writes the canonical + * ISO-8601 string ({@link CanonicalDuration#format}). A generated record binds it per component + * with {@code @JsonSerialize(using = CanonicalDurationSerializer.class)}, so a host + * {@code ObjectMapper} (whose JavaTimeModule writes a Duration as decimal seconds, or as + * {@link Duration#toString()}) cannot change the bytes. Null is written by Jackson before this runs. + */ +public final class CanonicalDurationSerializer extends StdSerializer { + + private static final long serialVersionUID = 1L; + + public CanonicalDurationSerializer() { + super(Duration.class); + } + + @Override + public void serialize(Duration value, JsonGenerator gen, SerializerProvider provider) throws IOException { + gen.writeString(CanonicalDuration.format(value)); + } +} diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/FilterParser.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/FilterParser.java index dd875d9053..a62f70c827 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/FilterParser.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/FilterParser.java @@ -81,6 +81,24 @@ public static FilterParseResult parse( Set allowedFields, Map> opsByField, Set decimalFields) { + return parse(rawQuery, allowedFields, opsByField, decimalFields, Set.of()); + } + + /** + * {@link #parse(String, Set, Map, Set)} with the allowlist's duration-typed fields (those served + * under {@code wire.durations: iso}). An operand on one of them uses the accepted ISO-8601 grammar + * ({@link CanonicalDuration}) and becomes a {@link java.time.Duration} the repository compares as + * a span; a malformed one (years, months, a clock spelling, lowercase) is {@code invalid_filter_value}. + * For {@code in} the predicate value is a {@code List}. + * + * @param durationFields typically {@code FilterAllowlist.DURATION_FIELDS} + */ + public static FilterParseResult parse( + String rawQuery, + Set allowedFields, + Map> opsByField, + Set decimalFields, + Set durationFields) { if (rawQuery == null || rawQuery.isEmpty()) return FilterParseResult.ok(List.of()); @@ -112,7 +130,8 @@ public static FilterParseResult parse( if (!allowedFields.contains(field)) return FilterParseResult.err("invalid_filter_field", field); Set ops = opsByField.get(field); if (ops == null || !ops.contains(op)) return FilterParseResult.err("invalid_filter_op", field); - Object coerced = decimalFields.contains(field) ? coerceDecimal(value, op) : coerceValue(value, op); + Object coerced = decimalFields.contains(field) ? coerceDecimal(value, op) + : durationFields.contains(field) ? coerceDuration(value, op) : coerceValue(value, op); if (coerced == INVALID_VALUE) return FilterParseResult.err("invalid_filter_value", field); if (op.equals("in") && coerced instanceof List list && list.size() > MAX_IN_LIST) { return FilterParseResult.err("filter.in_too_large", field); @@ -180,6 +199,21 @@ private static Object coerceDecimal(String raw, String op) { } } + private static Object coerceDuration(String raw, String op) { + if (op.equals("isNull")) return coerceValue(raw, op); + try { + if (op.equals("in")) { + String[] parts = raw.split(","); + List list = new ArrayList<>(parts.length); + for (String p : parts) list.add(CanonicalDuration.parse(p.trim())); + return list; + } + return CanonicalDuration.parse(raw); + } catch (CanonicalDuration.InvalidDurationException e) { + return INVALID_VALUE; + } + } + private static Object coerceValue(String raw, String op) { if (op.equals("isNull")) { if (raw.equals("true")) return Boolean.TRUE; diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedReportExportCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedReportExportCompileRunTest.java index c7bec8fc91..f01ed19a29 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedReportExportCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedReportExportCompileRunTest.java @@ -163,7 +163,7 @@ public void csvQuotesGuardsAndEndsEveryRecordWithCrLf() throws Exception { @Test public void numbersAreNeverGuardedAndStringWireDecimalsAreCanonical() throws Exception { - Loaded l = generate(Map.of(), new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT)); + Loaded l = generate(Map.of(), new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO)); String csv = l.csv(List.of("a", "b", "c", "d"), List.of(List.of(l.cell("raw", -7), l.cell("decimal", new BigDecimal("0.0")), l.cell("decimal", new BigDecimal("1E+3")), l.cell("decimal", new BigDecimal("12.50"))))); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedScalarExtractLockStepTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedScalarExtractLockStepTest.java index 3d40983493..5b5026ef81 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedScalarExtractLockStepTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedScalarExtractLockStepTest.java @@ -55,7 +55,7 @@ public class GeneratedScalarExtractLockStepTest extends SharedRegistryTestBase { */ private static final List SCALAR_SUBTYPES = List.of( "string", "int", "long", "double", "float", "decimal", "boolean", - "date", "time", "timestamp", "currency", "enum", "uuid", "uri", "inet"); + "date", "time", "timestamp", "currency", "enum", "uuid", "uri", "inet", "duration"); private static String fixture(String subType, boolean isArray) { String arr = isArray ? ", \"isArray\": true" : ""; diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java index cfdc53deb8..70dd8db76b 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java @@ -77,7 +77,7 @@ public void generatedDtoPatchAndAllowlistFollowTheDecimalWireRule() throws Excep Path classesDir = tmp.newFolder("classes").toPath(); MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder("fx").toPath(), "ledger", META); // The canonical output form is opt-in (ADR-0019 Amendment 2): pin the form the corpora pin. - loader.getRoot().setWireFormat(new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT)); + loader.getRoot().setWireFormat(new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO)); runGenerator(new SpringDtoGenerator(), loader, srcDir); runGenerator(new SpringValueObjectGenerator(), loader, srcDir); runGenerator(new SpringFilterAllowlistGenerator(), loader, srcDir); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDurationWireCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDurationWireCompileRunTest.java new file mode 100644 index 0000000000..ca556e66b6 --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDurationWireCompileRunTest.java @@ -0,0 +1,221 @@ +package com.metaobjects.generator.spring; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.metaobjects.config.WireFormat; +import com.metaobjects.loader.MetaDataLoader; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import javax.tools.DiagnosticCollector; +import javax.tools.JavaCompiler; +import javax.tools.JavaFileObject; +import javax.tools.ToolProvider; + +import java.io.File; +import java.lang.reflect.InvocationTargetException; +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * {@code field.duration}, end to end over GENERATED code: the DTO, the value-object record and the + * Patch type a duration as {@code java.time.Duration}, write the canonical ISO-8601 string and read + * the accepted form only — whatever the host ObjectMapper's JavaTimeModule would do — and the + * allowlist gives a duration the ordering band (eq/ne/gt/gte/lt/lte/in/isNull). + */ +public class SpringDurationWireCompileRunTest { + + private static final String PKG = "acme.ops"; + + private static final String META = """ + { "metadata.root": { "package": "acme::ops", "children": [ + { "object.value": { "name": "Window", "children": [ + { "field.duration": { "name": "span", "@required": true } } + ]}}, + { "object.entity": { "name": "Job", "children": [ + { "source.rdb": { "@table": "jobs" } }, + { "field.long": { "name": "id" } }, + { "field.duration": { "name": "holdFor", "@required": true, "@filterable": true } }, + { "field.duration": { "name": "retries", "isArray": true } }, + { "field.decimal": { "name": "cost", "@precision": 12, "@scale": 2 } }, + { "field.object": { "name": "window", "@objectRef": "Window", "@storage": "jsonb" } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ]}} + ]}} + """; + + @Rule + public TemporaryFolder tmp = new TemporaryFolder(); + + @Test + public void canonicalStringWireUnderBothDecimalModes() throws Exception { + for (WireFormat.Decimals decimals : WireFormat.Decimals.values()) { + run(decimals, new ObjectMapper()); + run(decimals, new ObjectMapper().findAndRegisterModules()); // a host mapper (JavaTimeModule when present) + } + } + + /** + * {@code wire.durations: native} (the default): no MetaObjects annotation is emitted and the host + * mapper's own java.time handling reads and writes the Duration ({@code Duration.toString()}). + */ + @Test + public void nativeDurationsUseTheHostMapper() throws Exception { + Path srcDir = tmp.newFolder().toPath(); + Path classesDir = tmp.newFolder().toPath(); + MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder().toPath(), "ops", META); + loader.getRoot().setWireFormat(new WireFormat( + WireFormat.Decimals.NATIVE, WireFormat.NullMembers.OMIT, WireFormat.Durations.NATIVE)); + runGenerator(new SpringDtoGenerator(), loader, srcDir); + runGenerator(new SpringValueObjectGenerator(), loader, srcDir); + compile(srcDir, classesDir); + String dtoSrc = Files.readString(srcDir.resolve("acme/ops/JobDto.java")); + assertFalse(dtoSrc, dtoSrc.contains("CanonicalDuration")); + + // A host mapper's java.time handling of a Duration: Duration.toString() / Duration.parse. + com.fasterxml.jackson.databind.module.SimpleModule hostDurations = + new com.fasterxml.jackson.databind.module.SimpleModule(); + hostDurations.addSerializer(Duration.class, com.fasterxml.jackson.databind.ser.std.ToStringSerializer.instance); + hostDurations.addDeserializer(Duration.class, + new com.fasterxml.jackson.databind.deser.std.StdDeserializer(Duration.class) { + @Override public Duration deserialize(com.fasterxml.jackson.core.JsonParser p, + com.fasterxml.jackson.databind.DeserializationContext c) throws java.io.IOException { + return Duration.parse(p.getText()); + } + }); + ObjectMapper mapper = new ObjectMapper().registerModule(hostDurations); + try (URLClassLoader cl = new URLClassLoader( + new URL[]{ classesDir.toUri().toURL() }, getClass().getClassLoader())) { + Class dto = cl.loadClass(PKG + ".JobDto"); + Object job = mapper.readValue("{\"holdFor\":\"PT1H30M\",\"window\":{\"span\":\"PT5400S\"}}", dto); + assertEquals(Duration.ofMinutes(90), dto.getMethod("holdFor").invoke(job)); + JsonNode out = mapper.valueToTree(job); + assertEquals(Duration.ofMinutes(90).toString(), out.get("holdFor").asText()); + // Jackson's own Duration.parse takes the looser forms the iso rule refuses. + Object loose = mapper.readValue("{\"holdFor\":\"pt1h\"}", dto); + assertEquals(Duration.ofHours(1), dto.getMethod("holdFor").invoke(loose)); + } + } + + private void run(WireFormat.Decimals decimals, ObjectMapper mapper) throws Exception { + Path srcDir = tmp.newFolder().toPath(); + Path classesDir = tmp.newFolder().toPath(); + MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder().toPath(), "ops", META); + loader.getRoot().setWireFormat(new WireFormat(decimals, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO)); + runGenerator(new SpringDtoGenerator(), loader, srcDir); + runGenerator(new SpringValueObjectGenerator(), loader, srcDir); + runGenerator(new SpringFilterAllowlistGenerator(), loader, srcDir); + compile(srcDir, classesDir); + + String dtoSrc = Files.readString(srcDir.resolve("acme/ops/JobDto.java")); + assertTrue(dtoSrc, dtoSrc.contains("@JsonSerialize(using = CanonicalDurationSerializer.class) " + + "@JsonDeserialize(using = CanonicalDurationDeserializer.class) java.time.Duration holdFor")); + assertTrue(dtoSrc, dtoSrc.contains("@JsonSerialize(contentUsing = CanonicalDurationSerializer.class)")); + + try (URLClassLoader cl = new URLClassLoader( + new URL[]{ classesDir.toUri().toURL() }, getClass().getClassLoader())) { + Class dto = cl.loadClass(PKG + ".JobDto"); + + // Read the accepted form, write the canonical one (a negative span keeps ONE sign; + // Duration.toString() would write PT-1M-30S). + Object job = mapper.readValue( + "{\"holdFor\":\"-PT90S\",\"retries\":[\"P1D\",\"PT0.5S\"],\"window\":{\"span\":\"PT5400S\"}}", dto); + assertEquals(Duration.ofSeconds(-90), dto.getMethod("holdFor").invoke(job)); + JsonNode out = mapper.valueToTree(job); + assertEquals("-PT1M30S", out.get("holdFor").asText()); + assertEquals("PT24H", out.get("retries").get(0).asText()); + assertEquals("PT0.5S", out.get("retries").get(1).asText()); + assertEquals("PT1H30M", out.get("window").get("span").asText()); + + // Refused at bind (Spring maps it to 400): calendar spans, a clock, a number of seconds, + // and the looser forms Duration.parse itself takes. + for (String bad : new String[]{"\"P1M\"", "\"1:30:00\"", "5400", "\"pt1h\"", "\"PT1.5H\"", "\"PT-1H\""}) { + try { + mapper.readValue("{\"holdFor\":" + bad + "}", dto); + fail("must refuse " + bad); + } catch (com.fasterxml.jackson.core.JsonProcessingException expected) { + // refused + } + try { + mapper.readValue("{\"window\":{\"span\":" + bad + "}}", dto); + fail("a value object must refuse " + bad); + } catch (com.fasterxml.jackson.core.JsonProcessingException expected) { + // refused + } + } + + // The Patch binds the same strict rule, scalar and per array element. + Class patch = cl.loadClass(PKG + ".JobPatch"); + java.lang.reflect.Method fromJson = patch.getMethod("fromJson", JsonNode.class, ObjectMapper.class); + Object p = fromJson.invoke(null, mapper.readTree("{\"holdFor\":\"P1DT2H\",\"retries\":[\"PT1S\"]}"), mapper); + assertEquals(Duration.ofHours(26), patch.getMethod("holdFor").invoke(p)); + assertEquals(List.of(Duration.ofSeconds(1)), patch.getMethod("retries").invoke(p)); + // A week is exactly 7 days; nanoseconds are kept. + Object w = fromJson.invoke(null, mapper.readTree("{\"holdFor\":\"P1WT0.123456789S\"}"), mapper); + assertEquals(Duration.ofDays(7).plusNanos(123_456_789), patch.getMethod("holdFor").invoke(w)); + for (String bad : new String[]{"{\"holdFor\":\"P1M\"}", "{\"holdFor\":\"P1.5W\"}", "{\"holdFor\":90}", "{\"holdFor\":null}", + "{\"retries\":[\"P1Y\"]}", "{\"retries\":[null]}", "{\"retries\":\"PT1S\"}"}) { + try { + fromJson.invoke(null, mapper.readTree(bad), mapper); + fail("a Patch must refuse " + bad); + } catch (InvocationTargetException e) { + assertEquals(bad, "PatchValidationException", e.getCause().getClass().getSimpleName()); + } + } + + // The filter band orders: a native interval on Postgres, a microsecond count elsewhere. + Class allow = cl.loadClass(PKG + ".JobFilterAllowlist"); + @SuppressWarnings("unchecked") + Map> ops = (Map>) allow.getField("OPS_BY_FIELD").get(null); + assertEquals(Set.of("eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"), ops.get("holdFor")); + } + } + + private static void runGenerator(Object generator, MetaDataLoader loader, Path outDir) { + Map args = new HashMap<>(); + args.put("outputDir", outDir.toString()); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).setArgs(args); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).execute(loader); + } + + private static void compile(Path srcDir, Path classesDir) throws Exception { + List sources; + try (Stream s = Files.walk(srcDir)) { + sources = s.filter(p -> p.toString().endsWith(".java")).map(Path::toFile) + .collect(Collectors.toList()); + } + assertTrue("expected generated .java sources", !sources.isEmpty()); + JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); + assertNotNull("JDK required — getSystemJavaCompiler() returned null", javac); + String cp = System.getProperty("java.class.path"); + DiagnosticCollector diags = new DiagnosticCollector<>(); + var fm = javac.getStandardFileManager(diags, null, StandardCharsets.UTF_8); + List opts = List.of("-classpath", cp, "-parameters", "-d", classesDir.toString()); + boolean ok = javac.getTask(null, fm, diags, opts, null, + fm.getJavaFileObjectsFromFiles(sources)).call(); + if (!ok) { + StringBuilder sb = new StringBuilder("generated sources failed to compile:\n"); + for (var d : diags.getDiagnostics()) { + sb.append(" ").append(d.getKind()).append(": ").append(d.getMessage(null)).append('\n'); + } + fail(sb.toString()); + } + } +} diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringNamesGeneratorTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringNamesGeneratorTest.java index efde171a77..ef2d00a40e 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringNamesGeneratorTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringNamesGeneratorTest.java @@ -683,6 +683,12 @@ public void anObjectWithMoreThanTenFieldsCompiles() throws Exception { * not succeed. Mirrors {@code SpringProjectionDtoCompileRunTest#compile}. */ private void compileAll(Path root) throws Exception { + String failure = compile(root, List.of()); + if (failure != null) fail(failure); + } + + /** Compile every {@code .java} under {@code root}; the diagnostics when it fails, else null. */ + private String compile(Path root, List extraOpts) throws Exception { List sources; try (Stream s = Files.walk(root)) { sources = s.filter(p -> p.toString().endsWith(".java")) @@ -694,11 +700,12 @@ private void compileAll(Path root) throws Exception { JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); assertNotNull("JDK (not JRE) required -- getSystemJavaCompiler() returned null", javac); - Path classes = tempFolder.newFolder("classes-" + root.getFileName()).toPath(); + Path classes = tempFolder.newFolder().toPath(); String cp = System.getProperty("java.class.path"); DiagnosticCollector diags = new DiagnosticCollector<>(); var fm = javac.getStandardFileManager(diags, null, null); - List opts = List.of("-classpath", cp, "-d", classes.toString()); + List opts = new java.util.ArrayList<>(List.of("-classpath", cp, "-d", classes.toString())); + opts.addAll(extraOpts); boolean ok = javac.getTask(null, fm, diags, opts, null, fm.getJavaFileObjectsFromFiles(sources)).call(); @@ -711,7 +718,162 @@ private void compileAll(Path root) throws Exception { .append(':').append(d.getLineNumber()).append('\n'); } } - fail(sb.toString()); + return sb.toString(); + } + return null; + } + + // ------------------------------------------------------------------------- + // Flattened value objects: one constant per member's column, none for the field + // ------------------------------------------------------------------------- + + private static final String FLATTENED_FIXTURE = """ + { + "metadata.root": { "package": "acme::ship", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } }, + { "field.string": { "name": "postalCode", "@column": "ZIP" } } + ] } }, + { "object.entity": { "name": "Shipment", "children": [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "field.object": { "name": "payload", "@objectRef": "Address" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }"""; + + @Test + public void aFlattenedMemberGetsAColumnConstantBesideItsFieldConstants() throws IOException { + String src = emit(FLATTENED_FIXTURE, "acme/ship/ShipmentNames.java", Map.of()); + + assertTrue(src, src.contains("public static final String SHIP_TO_STREET_COLUMN = \"ship_to_street\";")); + assertTrue(src, src.contains("public static final String SHIP_TO_POSTAL_CODE_COLUMN = \"ship_to_ZIP\";")); + assertTrue(src, src.contains("public static final String SHIP_TO_FIELD = \"shipTo\";")); + assertTrue(src, src.contains("public static final String PAYLOAD_COLUMN = \"payload\";")); + // A flattened field has no column of its own, so no _COLUMN constant and no map key. + assertFalse(src, src.contains("SHIP_TO_COLUMN")); + assertFalse(src, src.contains("Deprecated")); + assertFalse(src, src.contains("Map.entry(\"shipTo\",")); + assertTrue(src, src.contains("Map.entry(\"shipTo.street\", SHIP_TO_STREET_COLUMN)")); + assertTrue(src, src.contains("Map.entry(\"shipTo.postalCode\", SHIP_TO_POSTAL_CODE_COLUMN)")); + } + + /** + * A flattened field has no column constant: the generated names classes, a base and its + * subclass, compile under {@code -Xlint:all -Werror}, and reading the field's own column + * constant is a compile error. + */ + @Test + public void aFlattenedFieldHasNoColumnConstantToRead() throws Exception { + String model = """ + { + "metadata.root": { "package": "acme::ship", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } } + ] } }, + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } } + ] } }, + { "object.entity": { "name": "Order", "extends": "acme::ship::Base", "children": [ + { "source.rdb": { "@table": "orders" } }, + { "field.string": { "name": "note" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }"""; + Path outDir = tempFolder.newFolder().toPath(); + Path workspace = tempFolder.newFolder().toPath(); + SpringNamesGenerator gen = new SpringNamesGenerator(); + gen.setArgs(Map.of("outputDir", outDir.toString())); + gen.execute(SpringTestFixtures.loadFixture(workspace, "names-deprecation", model)); + String order = Files.readString(outDir.resolve("acme/ship/OrderNames.java")); + assertTrue(order, order.contains("public abstract class OrderNames extends BaseNames {")); + assertFalse(order, order.contains("SHIP_TO_COLUMN")); + + List strict = List.of("-Xlint:all", "-Werror"); + String failure = compile(outDir, strict); + assertTrue("the generated names compile with -Xlint:all -Werror:\n" + failure, failure == null); + + Files.writeString(outDir.resolve("acme/ship/UsesIt.java"), + "package acme.ship;\nclass UsesIt { String c = OrderNames.SHIP_TO_COLUMN; }\n"); + failure = compile(outDir, strict); + assertTrue("a use of the absent constant does not compile:\n" + failure, + failure != null && failure.contains("SHIP_TO_COLUMN")); + } + + @Test + public void aFlattenedMemberConstantCollidingWithAFieldConstantIsLeftOut() throws IOException { + String model = FLATTENED_FIXTURE.replace( + "\"@storage\": \"flattened\" }", + "\"@storage\": \"flattened\", \"@flattenedColumnPrefix\": \"st_\" } },\n" + + " { \"field.string\": { \"name\": \"shipToStreet\" }"); + String src = emit(model, "acme/ship/ShipmentNames.java", Map.of()); + + // The field's constant wins; the model generates, as it did before member constants. + assertTrue(src, src.contains("public static final String SHIP_TO_STREET_COLUMN = \"ship_to_street\";")); + assertFalse(src, src.contains("\"st_street\"")); + assertTrue(src, src.contains("public static final String SHIP_TO_POSTAL_CODE_COLUMN = \"st_ZIP\";")); + } + + @Test + public void aChildThatRedeclaresAFlattenedFieldAsJsonbDoesNotInheritItsMemberConstants() throws IOException { + String model = """ + { + "metadata.root": { "package": "acme::ship", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "street" } } + ] } }, + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "flattened" } } + ] } }, + { "object.entity": { "name": "Order", "extends": "acme::ship::Base", "children": [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }"""; + String src = emit(model, "acme/ship/OrderNames.java", Map.of()); + + // Extending BaseNames would make BaseNames.SHIP_TO_STREET_COLUMN reachable as + // OrderNames.SHIP_TO_STREET_COLUMN, naming a column `orders` does not have. + assertTrue(src, src.contains("public abstract class OrderNames {")); + assertFalse(src, src.contains("SHIP_TO_STREET_COLUMN")); + assertTrue(src, src.contains("public static final String SHIP_TO_COLUMN = \"ship_to\";")); + } + + /** + * The cross-port naming corpus: every {@code ..} key is a flattened + * column, and the names artifact must carry each one as {@code __COLUMN} + * holding exactly the column {@code meta migrate} creates, under every naming rule. + */ + @Test + public void everyFlattenedColumnInTheNamingCorpusIsAConstant() throws IOException { + Path dir = Path.of("").toAbsolutePath(); + while (dir != null && !Files.isDirectory(dir.resolve("fixtures"))) dir = dir.getParent(); + assertNotNull("could not locate the repository fixtures/ directory", dir); + Path corpus = dir.resolve("fixtures/naming-conformance/physical-names"); + String model = Files.readString(corpus.resolve("meta.naming.json")); + com.fasterxml.jackson.databind.JsonNode cases = new com.fasterxml.jackson.databind.ObjectMapper() + .readTree(corpus.resolve("cases.json").toFile()).get("cases"); + for (com.fasterxml.jackson.databind.JsonNode c : cases) { + com.fasterxml.jackson.databind.JsonNode naming = c.get("naming"); + String columnsRule = naming != null && naming.has("columns") ? naming.get("columns").asText() : "snake_case"; + String src = emit(model, "acme/naming/ShipmentNames.java", Map.of(), columnsRule); + var it = c.get("columns").fields(); + while (it.hasNext()) { + var e = it.next(); + String[] parts = e.getKey().split("\\."); + if (parts.length != 3 || !parts[0].equals("Shipment")) continue; + String member = SpringNamesGenerator.namesMember(parts[1]) + "_" + + SpringNamesGenerator.namesMember(parts[2]) + "_COLUMN"; + String decl = "public static final String " + member + " = \"" + e.getValue().asText() + "\";"; + assertTrue(c.get("name").asText() + ": " + decl + "\n" + src, src.contains(decl)); + } } } } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportExportDurationCellTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportExportDurationCellTest.java new file mode 100644 index 0000000000..ba86a823bd --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportExportDurationCellTest.java @@ -0,0 +1,26 @@ +package com.metaobjects.generator.spring; + +import org.junit.Test; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * The shared report writer carries the {@code field.duration} cell factory only when a served + * report has a duration column: without one the writer is exactly what it was, and with one it + * differs by that factory alone. + */ +public class SpringReportExportDurationCellTest { + + @Test + public void theDurationCellIsAddedOnlyWhenAReportHasADurationColumn() { + String plain = SpringReportExportGenerator.writerSource("acme.shop", "empty", false, true, false); + assertFalse(plain, plain.contains("uration")); + String withDuration = SpringReportExportGenerator.writerSource("acme.shop", "empty", false, true, true); + assertTrue(withDuration, withDuration.contains("public static Cell duration(java.time.Duration value)")); + int start = withDuration.indexOf(" /** A field.duration in its canonical"); + int end = withDuration.indexOf(" /** A decimal as the wire setting writes it"); + assertEquals(plain, withDuration.substring(0, start) + withDuration.substring(end)); + } +} diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringTypeMapperTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringTypeMapperTest.java index d0baa4209c..89455c1df5 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringTypeMapperTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringTypeMapperTest.java @@ -1,5 +1,6 @@ package com.metaobjects.generator.spring; +import com.metaobjects.config.WireFormat; import com.metaobjects.field.BooleanField; import com.metaobjects.field.CurrencyField; import com.metaobjects.field.DateField; @@ -155,6 +156,21 @@ public void inetFieldMapsToJavaNetInetAddress() { SpringTypeMapper.javaTypeName(new com.metaobjects.field.InetField("sourceIp"))); } + @Test + public void durationFieldMapsToJavaTimeDuration() { + // field.duration binds to java.time.Duration, with the canonical ISO-8601 wire binding. + assertEquals("java.time.Duration", + SpringTypeMapper.javaTypeName(new com.metaobjects.field.DurationField("holdFor"))); + assertEquals("@JsonSerialize(using = CanonicalDurationSerializer.class) " + + "@JsonDeserialize(using = CanonicalDurationDeserializer.class)", + SpringGeneratorSupport.durationJsonAnnotations("java.time.Duration", new WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.NATIVE, WireFormat.Durations.ISO))); + assertEquals("@JsonSerialize(contentUsing = CanonicalDurationSerializer.class) " + + "@JsonDeserialize(contentUsing = CanonicalDurationDeserializer.class)", + SpringGeneratorSupport.durationJsonAnnotations("java.util.List", new WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.NATIVE, WireFormat.Durations.ISO))); + assertEquals("", SpringGeneratorSupport.durationJsonAnnotations("String", new WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.NATIVE, WireFormat.Durations.ISO))); + assertEquals("", SpringGeneratorSupport.durationJsonAnnotations("java.time.Duration", WireFormat.DEFAULT)); + } + @Test public void stringFormatEmailEmitsJakartaEmail() { // ADR-0036/0037 Wave 3: @stringFormat:email → Jakarta @Email (codegen-owned matcher). diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringWireFormatCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringWireFormatCompileRunTest.java index 19dbfbe38b..f233c49a07 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringWireFormatCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringWireFormatCompileRunTest.java @@ -129,7 +129,7 @@ public void theDefaultIsJacksonsOwnShape() throws Exception { @Test public void stringAndOmitGiveTheCanonicalCrossPortForm() throws Exception { - generate(new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT), (src, cl) -> { + generate(new WireFormat(WireFormat.Decimals.STRING, WireFormat.NullMembers.OMIT, WireFormat.Durations.ISO), (src, cl) -> { String raw = writtenBytes(cl); JsonNode out = MAPPER.readTree(raw); assertEquals("12.5", out.get("amount").asText()); @@ -142,7 +142,7 @@ public void stringAndOmitGiveTheCanonicalCrossPortForm() throws Exception { @Test public void emitWritesTheNullMemberLikeNative() throws Exception { - generate(new WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.EMIT), (src, cl) -> { + generate(new WireFormat(WireFormat.Decimals.NATIVE, WireFormat.NullMembers.EMIT, WireFormat.Durations.ISO), (src, cl) -> { String raw = writtenBytes(cl); JsonNode out = MAPPER.readTree(raw); assertTrue(out.get("price").has("note")); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ValidationConformanceTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ValidationConformanceTest.java index 1b8dfc48cb..f1fa7451ae 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ValidationConformanceTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ValidationConformanceTest.java @@ -70,7 +70,7 @@ public void generatedDtoEnforcesCorpusVerdicts() throws Exception { runCorpus("meta.json", "cases.json", "Account", "acme.auth"); } - /** Entity-scoped cross-field validators (comparison / requiredWhen / presentIff / atLeastOne) + /** Entity-scoped cross-field validators (comparison / requiredWhen / presentIff / atLeastOne / exactlyOne, with @condition) * enforced by the class-level Bean Validation on the generated full DTO. */ @Test public void generatedDtoEnforcesCrossFieldVerdicts() throws Exception { @@ -90,6 +90,11 @@ private void runCorpus(String metaName, String casesName, String defaultEntity, MetaDataLoader loader = SpringTestFixtures.loadFixture( tmp.newFolder().toPath(), metaName.replace('.', '_'), Files.readString(metaFile)); + // The corpus pins the cross-port duration rule (strict ISO-8601): wire.durations: iso. + loader.getRoot().setWireFormat(new com.metaobjects.config.WireFormat( + com.metaobjects.config.WireFormat.Decimals.NATIVE, com.metaobjects.config.WireFormat.NullMembers.NATIVE, + com.metaobjects.config.WireFormat.Durations.ISO)); + // 2. Generate the Account DTO. Path gen = tmp.newFolder("gen-dto").toPath(); SpringDtoGenerator generator = new SpringDtoGenerator(); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/DurationWireTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/DurationWireTest.java new file mode 100644 index 0000000000..47219fbd8e --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/DurationWireTest.java @@ -0,0 +1,123 @@ +package com.metaobjects.generator.spring.runtime; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.annotation.JsonDeserialize; +import com.fasterxml.jackson.databind.annotation.JsonSerialize; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.time.Duration; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * The shared {@code fixtures/duration-wire/vectors.json} {@code wire} vectors ({@code parse}, + * {@code precision} at this port's key {@code "9"}, {@code refused}) against the PRODUCTION helpers + * a generated record binds ({@link CanonicalDuration} through {@link CanonicalDurationSerializer} / + * {@link CanonicalDurationDeserializer}). The {@code codecs} groups belong to the persistence side + * ({@code com.metaobjects.field.DurationWire}); this runtime has no storage codec. + */ +public class DurationWireTest { + + public record Holder( + @JsonSerialize(using = CanonicalDurationSerializer.class) + @JsonDeserialize(using = CanonicalDurationDeserializer.class) + Duration v) {} + + /** A plain mapper — no JavaTimeModule: the per-component binding carries the whole rule. */ + private static final ObjectMapper MAPPER = new ObjectMapper(); + + private static JsonNode vectors() throws IOException { + Path p = Paths.get(System.getProperty("user.dir")).toAbsolutePath(); + while (p != null && !Files.exists(p.resolve("fixtures/duration-wire/vectors.json"))) p = p.getParent(); + assertNotNull("fixtures/duration-wire/vectors.json not found above the working directory", p); + return MAPPER.readTree(p.resolve("fixtures/duration-wire/vectors.json").toFile()); + } + + @Test + public void parseVectorsRoundTripThroughTheRecordBinding() throws Exception { + for (JsonNode v : vectors().get("wire").get("parse")) { + String in = v.get("in").asText(); + String out = v.get("out").asText(); + assertEquals("format(parse(" + in + "))", out, CanonicalDuration.format(CanonicalDuration.parse(in))); + Holder h = MAPPER.readValue("{\"v\":" + MAPPER.writeValueAsString(in) + "}", Holder.class); + assertEquals("wire " + in, "{\"v\":\"" + out + "\"}", MAPPER.writeValueAsString(h)); + } + } + + @Test + public void precisionVectorsKeepNineDigits() throws Exception { + for (JsonNode v : vectors().get("wire").get("precision")) { + String in = v.get("in").asText(); + String out = v.get("out").get("9").asText(); + assertEquals("format(parse(" + in + "))", out, CanonicalDuration.format(CanonicalDuration.parse(in))); + Holder h = MAPPER.readValue("{\"v\":" + MAPPER.writeValueAsString(in) + "}", Holder.class); + assertEquals("wire " + in, "{\"v\":\"" + out + "\"}", MAPPER.writeValueAsString(h)); + } + } + + @Test + public void refusedVectorsAreA400AtTheBindTier() throws Exception { + for (JsonNode v : vectors().get("wire").get("refused")) { + String in = v.asText(); + assertFalse("refused: '" + in + "'", CanonicalDuration.isDuration(in)); + try { + MAPPER.readValue("{\"v\":" + MAPPER.writeValueAsString(in) + "}", Holder.class); + fail("must refuse '" + in + "'"); + } catch (com.fasterxml.jackson.databind.JsonMappingException expected) { + // refused at bind + } + } + } + + @Test + public void aNumberOfSecondsIsNotADuration() throws Exception { + try { + MAPPER.readValue("{\"v\":5400}", Holder.class); + fail("a JSON number must be refused"); + } catch (com.fasterxml.jackson.databind.JsonMappingException expected) { + // refused + } + assertTrue(MAPPER.readValue("{\"v\":null}", Holder.class).v() == null); + } + + @Test + public void formatKeepsNanos() { + assertEquals("-PT1.123456789S", CanonicalDuration.format(Duration.ofSeconds(-1, -123_456_789))); + assertEquals("PT0.000000999S", CanonicalDuration.format(Duration.ofNanos(999))); + } + + @Test + public void formattingIgnoresTheDefaultLocalesDigits() throws Exception { + java.util.Locale saved = java.util.Locale.getDefault(); + try { + java.util.Locale.setDefault(java.util.Locale.forLanguageTag("ar-EG-u-nu-arab")); + assertEquals("{\"v\":\"-PT1M30.000005S\"}", + MAPPER.writeValueAsString(new Holder(Duration.ofSeconds(-90, -5_000)))); + } finally { + java.util.Locale.setDefault(saved); + } + } + + @Test + public void javaDurationsOwnRangeIsTheOnlyLimit() { + assertEquals("PT2562047788015215H", CanonicalDuration.format(CanonicalDuration.parse("PT2562047788015215H"))); + for (String tooLong : new String[]{"PT9223372036854775808S", "PT99999999999999999999H"}) { + assertFalse(tooLong, CanonicalDuration.isDuration(tooLong)); + try { + CanonicalDuration.parse(tooLong); + fail("must refuse, never wrap: " + tooLong); + } catch (CanonicalDuration.InvalidDurationException expected) { + // refused + } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt index 1cbc4943be..3057078a35 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt @@ -33,7 +33,7 @@ internal class KotlinCodegenMatchesReferenceTest { ExpectedColumn("id", families = setOf("long")), ExpectedColumn("title", families = setOf("varchar")), ExpectedColumn("priceCents", families = setOf("long")), - ExpectedColumn("status", families = setOf("varchar", "enumerationByName")), + ExpectedColumn("status", families = setOf("varchar", "enumerationByName", "customEnumeration")), // Plain `field.timestamp` (no @dbColumnType) → Exposed `datetime(...)` (Postgres // `timestamp without time zone`, LocalDateTime). The TZ-aware opt-in is covered by // Asset.recordedAt below. @@ -74,7 +74,7 @@ internal class KotlinCodegenMatchesReferenceTest { // string default — not the old `varchar(255)`. The view's DDL is TS-canonical-owned; // this Exposed mapping is read-only. ExpectedColumn("title", families = setOf("text")), - ExpectedColumn("status", families = setOf("varchar", "enumerationByName")), + ExpectedColumn("status", families = setOf("varchar", "enumerationByName", "customEnumeration")), ), ), "ProgramStat" to EntityExpectation( @@ -137,7 +137,7 @@ internal class KotlinCodegenMatchesReferenceTest { "ProgramsByMonth" to EntityExpectation( columns = listOf( ExpectedColumn("createdAtMonth", families = setOf("date"), nullable = false), - ExpectedColumn("status", families = setOf("varchar", "enumerationByName"), nullable = false), + ExpectedColumn("status", families = setOf("varchar", "enumerationByName", "customEnumeration"), nullable = false), ExpectedColumn("programs", families = setOf("long"), nullable = false), ExpectedColumn("listValue", families = setOf("long"), nullable = true), ), @@ -253,7 +253,7 @@ internal class KotlinCodegenMatchesReferenceTest { "AllTypesBreakdown" to EntityExpectation( columns = listOf( ExpectedColumn("flag", families = setOf("bool"), nullable = false), - ExpectedColumn("level", families = setOf("varchar", "enumerationByName"), nullable = false), + ExpectedColumn("level", families = setOf("varchar", "enumerationByName", "customEnumeration"), nullable = false), ExpectedColumn("legacyStatus", families = setOf("customEnumeration"), nullable = true), ExpectedColumn("dateKey", families = setOf("date"), nullable = false), ExpectedColumn("uuidKey", families = setOf("uuid", "portableUuid"), nullable = false), @@ -280,7 +280,7 @@ internal class KotlinCodegenMatchesReferenceTest { ), "LiveIntroPrograms" to EntityExpectation( columns = listOf( - ExpectedColumn("status", families = setOf("varchar", "enumerationByName"), nullable = false), + ExpectedColumn("status", families = setOf("varchar", "enumerationByName", "customEnumeration"), nullable = false), ExpectedColumn("programs", families = setOf("long"), nullable = false), ExpectedColumn("notDrafts", families = setOf("long"), nullable = false), ExpectedColumn("listValue", families = setOf("long"), nullable = true), @@ -334,7 +334,7 @@ internal class KotlinCodegenMatchesReferenceTest { ), "AuthsByKind" to EntityExpectation( columns = listOf( - ExpectedColumn("kind", families = setOf("varchar", "enumerationByName"), nullable = true), + ExpectedColumn("kind", families = setOf("varchar", "enumerationByName", "customEnumeration"), nullable = true), ExpectedColumn("auths", families = setOf("long"), nullable = false), ExpectedColumn("distinctRefs", families = setOf("long"), nullable = false), ), diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Normalization.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Normalization.kt index ab967aaab9..58b93e66c5 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Normalization.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Normalization.kt @@ -51,6 +51,8 @@ object Normalization { is Double -> canonicalFloat(v) is BigDecimal -> canonicalDecimal(v) is UUID -> v.toString().lowercase() + // field.duration → the canonical ISO-8601 wire form (Duration.toString() differs for a negative span). + is java.time.Duration -> com.metaobjects.field.DurationWire.format(v) is ByteArray -> Base64.getEncoder().encodeToString(v) // DATE → "YYYY-MM-DD"; LocalDate.toString() is already this exact form. is LocalDate -> v.toString() diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioConformanceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioConformanceTest.kt index 15bf409491..33d64d8711 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioConformanceTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioConformanceTest.kt @@ -48,6 +48,16 @@ internal class QueryScenarioConformanceTest { */ private val DEFERRED_SCENARIOS = emptyMap() + /** + * `-Dmetaobjects.scenarios=a,b` narrows a run to the named scenarios (one Postgres + * container each, so a targeted run is far cheaper than the corpus). Unset = all. + */ + private fun selected(name: String): Boolean { + val only = System.getProperty("metaobjects.scenarios") + if (only.isNullOrBlank()) return true + return only.split(",").map { it.trim() }.contains(name) + } + @JvmStatic fun scenarios(): Stream { val corpus = ScenarioLoader.findCorpusRoot() @@ -60,6 +70,7 @@ internal class QueryScenarioConformanceTest { ) return all .filter { it.name !in DEFERRED_SCENARIOS.keys && it.runsOn(ScenarioEngine.POSTGRES) } + .filter { selected(it.name) } .stream() .map { Arguments.of(it.name, it) } } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index 13c4a09832..66283863d4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -10,6 +10,7 @@ import com.metaobjects.integration.kotlin.Scenarios.QueryScenario import com.metaobjects.integration.kotlin.Scenarios.QuerySpec import com.metaobjects.integration.kotlin.tables.AssetTable import com.metaobjects.integration.kotlin.tables.EntryTable +import com.metaobjects.integration.kotlin.tables.MetaDurationColumnType import com.metaobjects.integration.kotlin.tables.ReferenceDecimal import com.metaobjects.integration.kotlin.tables.AuthTable import com.metaobjects.integration.kotlin.tables.MeasurementTable @@ -484,6 +485,9 @@ object QueryScenarioRunner { // Int-backed field.enum: bind the member SYMBOL and let the column's codec encode it to // the stored int (see the same guard in `coerce`). if (col.columnType is IntBackedEnumColumnType) return raw.toString() + // field.duration: parse the ISO-8601 authoring literal (checked BEFORE the sqlType + // dispatch, whose `int` substring branch would match "interval"). + if (col.columnType is MetaDurationColumnType) return com.metaobjects.field.DurationWire.parse(raw.toString()) val type = col.columnType.sqlType().lowercase() return when { type == "uuid" -> if (raw is UUID) raw else UUID.fromString(raw.toString().lowercase()) @@ -684,6 +688,8 @@ object QueryScenarioRunner { // member SYMBOL and the column's own codec maps it to the stored int. Checked BEFORE the // sqlType dispatch, whose `int` branch would try "PUBLISHED".toInt(). if (col.columnType is IntBackedEnumColumnType) return raw.toString() + // A duration operand is the ISO-8601 literal (before the sqlType dispatch: "interval" contains "int"). + if (col.columnType is MetaDurationColumnType) return com.metaobjects.field.DurationWire.parse(raw.toString()) val type = col.columnType.sqlType().lowercase() return when { // uuid columns compare against java.util.UUID — the YAML supplies a string diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/duration/DurationGeneratedApiContractConformanceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/duration/DurationGeneratedApiContractConformanceTest.kt new file mode 100644 index 0000000000..dc8c7a59a6 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/duration/DurationGeneratedApiContractConformanceTest.kt @@ -0,0 +1,114 @@ +package com.metaobjects.integration.kotlin.api.duration + +import com.metaobjects.integration.kotlin.MySqlContainer +import com.metaobjects.integration.kotlin.PostgresContainer +import com.metaobjects.integration.kotlin.ScenarioEngine +import com.metaobjects.integration.kotlin.api.ApiContractAssertions +import com.metaobjects.integration.kotlin.api.ApiContractScenarioLoader +import com.metaobjects.integration.kotlin.api.ApiContractScenarios.ApiScenario +import com.metaobjects.integration.kotlin.api.report.generated.GeneratedReportControllerHarness +import com.metaobjects.integration.kotlin.api.report.generated.GeneratedReportControllerHarness.ReportDb +import org.junit.jupiter.api.AfterAll +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assumptions +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.BeforeAll +import org.junit.jupiter.api.DisplayName +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.assertDoesNotThrow +import org.junit.jupiter.api.io.TempDir +import org.junit.jupiter.params.ParameterizedTest +import org.junit.jupiter.params.provider.Arguments +import org.junit.jupiter.params.provider.MethodSource +import java.nio.file.Files +import java.nio.file.Path +import java.util.stream.Stream + +/** + * The Kotlin GENERATED-controller lane for the `duration/` api-contract sub-corpus, on real Postgres, + * SQLite and MySQL. A duration is a canonical ISO-8601 span on the wire (D1), a request takes the accepted + * form (D2) and refuses years, months, a clock spelling and a JSON number (D3), and a filter operand and a + * sort compare spans (D4). Reuses the report lane's harness (same mechanics: the committed schema executed + * verbatim, the emitted controllers hosted unmodified). Generated lane only, as every port runs it. + */ +@DisplayName("API contract duration - GENERATED Kotlin Spring controllers over embedded Tomcat, real databases") +internal class DurationGeneratedApiContractConformanceTest { + + @ParameterizedTest(name = "postgres {0}") + @MethodSource("scenarios") + fun scenarioOnPostgres(name: String, scenario: ApiScenario) = + run(scenario, ReportDb(ScenarioEngine.POSTGRES, PG.jdbcUrl, PG.username, PG.password)) + + @ParameterizedTest(name = "sqlite {0}") + @MethodSource("scenarios") + fun scenarioOnSqlite(name: String, scenario: ApiScenario, @TempDir dir: Path) = + run(scenario, ReportDb(ScenarioEngine.SQLITE, "jdbc:sqlite:${dir.resolve("duration.db")}", "", "")) + + @ParameterizedTest(name = "mysql {0}") + @MethodSource("scenarios") + fun scenarioOnMySql(name: String, scenario: ApiScenario) = + run(scenario, ReportDb(ScenarioEngine.MYSQL, MYSQL.jdbcUrl, MYSQL.username, MYSQL.password)) + + /** A fresh schema (real tables and views) and base-table seed per scenario, then the requests over HTTP. */ + private fun run(scenario: ApiScenario, db: ReportDb) { + // `engines:` is exact: a scenario that names other engines is not this engine's. + Assumptions.assumeTrue(scenario.runsOn(db.engine.name), "${scenario.name} does not run on ${db.engine}") + assertDoesNotThrow { + HARNESS.reset(db) + for (req in scenario.requests) { + val res = HARNESS.exchange(req.method, req.path, req.body) + val parsed = HARNESS.parseBody(res.body) + ApiContractAssertions.assertResponse(scenario.name, req, res.status, parsed) + } + } + } + + @Test + fun `the corpus has its two scenarios`() { + // A lane that silently ran fewer would still be green. + assertEquals(2, scenarios().count()) + } + + @Test + fun `the entity gets a controller`() { + for (name in SERVED) { + assertTrue("acme/ops/${name}Controller.kt" in HARNESS.emittedFiles, "$name was not generated") + } + } + + companion object { + // One server per engine for the class; each scenario rebuilds its schema on it. Lazy, so a + // run that selects one engine's tests does not boot the others' containers. + private val PG by lazy { PostgresContainer() } + private val MYSQL by lazy { MySqlContainer() } + + private val SERVED = listOf("Timer") + + private lateinit var HARNESS: GeneratedReportControllerHarness + + @BeforeAll + @JvmStatic + fun setUp() { + val corpus = ApiContractScenarioLoader.findCorpusRoot() + val genDir = Files.createTempDirectory("duration-generated-kotlin-controller") + HARNESS = GeneratedReportControllerHarness( + corpus, genDir, lane = "duration", entityPkg = "acme.ops", served = SERVED, + ) + } + + @AfterAll + @JvmStatic + fun tearDown() { + if (::HARNESS.isInitialized) HARNESS.close() + runCatching { PG.close() } + runCatching { MYSQL.close() } + } + + @JvmStatic + fun scenarios(): Stream { + val corpus = ApiContractScenarioLoader.findCorpusRoot() + val scenarios = ApiContractScenarioLoader.loadScenarios(corpus.resolve("duration/scenarios")) + return scenarios.stream().map { Arguments.of(it.name, it) } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/TphFullSuiteCompilesTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/TphFullSuiteCompilesTest.kt index 12c45efb04..c90f06be67 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/TphFullSuiteCompilesTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/TphFullSuiteCompilesTest.kt @@ -142,7 +142,8 @@ class TphFullSuiteCompilesTest { assertTrue("integer(BridgeAuthNames.QUANTITY_COLUMN).nullable()" in authTable, authTable) assertTrue("portableDecimal(CopayAuthNames.COPAY_AMOUNT_COLUMN, 10, 2).nullable()" in authTable, authTable) assertTrue("varchar(PriorAuthAuthNames.PRIOR_AUTH_NUMBER_COLUMN, 80).nullable()" in authTable, authTable) - assertTrue("enumerationByName(CopayAuthNames.TIER_COLUMN, 64, AuthTier::class).nullable()" in authTable, authTable) + // An enum with no configured length is a text column, as `meta migrate` creates it. + assertTrue("customEnumeration(CopayAuthNames.TIER_COLUMN, \"TEXT\", { v -> AuthTier.valueOf(v as String) }, { e -> e.name }).nullable()" in authTable, authTable) val sources = emitted.map { path -> SourceFile.kotlin(outDir.relativize(path).toString().replace('/', '_'), path.readText()) diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt index 6533b79c46..56ed69d11d 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt @@ -31,6 +31,7 @@ import org.jetbrains.exposed.sql.json.jsonb * - `field.enum` (@values LOW/MEDIUM/HIGH) → `varchar("enumVal", 64)` (text + CHECK in DDL) * - `field.enum` + `@intValueMap` (int-backed) → `intBackedEnum("intEnumVal", …)` (INTEGER + int CHECK) * - `field.uuid` (non-key, @required) → `uuid("uuidVal")` (Postgres native uuid) + * - `field.duration` → `durationColumn("durationVal")` (Postgres INTERVAL) * - `field.object` (@objectRef Settings, @storage jsonb) → `jsonb("settings", …)` (real Postgres JSONB) * - `field.object` (@objectRef Label, @storage jsonb, isArray) → `jsonb("labels", …)` (JSONB array) * @@ -80,6 +81,10 @@ object AllTypesTable : Table("all_types") { // canonical COMPRESSED IPv6) and binds the bare address on write. val inetVal = inetColumn("inetVal") val inet6Val = inetColumn("inet6Val") + // `field.duration` → the Postgres-native `INTERVAL` ([MetaDurationColumnType]): written in the + // interval spelling, read back as the canonical ISO-8601 duration. Nullable to match the + // canonical DDL (`"durationVal" INTERVAL`, no NOT NULL). + val durationVal = durationColumn("durationVal").nullable() // `@storage:jsonb` typed owned-object column. Raw-JSON-String identity encode/decode keeps // the JSON text; the runner serializes the authoring map on write and parses it back to a // Map on read so it re-serializes with sorted keys per the normalization contract. Nullable diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/MetaDurationColumnType.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/MetaDurationColumnType.kt new file mode 100644 index 0000000000..02c0155ca2 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/MetaDurationColumnType.kt @@ -0,0 +1,42 @@ +package com.metaobjects.integration.kotlin.tables + +import com.metaobjects.field.DurationWire +import org.jetbrains.exposed.sql.Column +import org.jetbrains.exposed.sql.ColumnType +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.vendors.currentDialect +import java.sql.ResultSet +import java.time.Duration + +/** + * Hand-written reference Exposed column type for `field.duration` — the test-harness analogue of + * the per-package `MetaDurationColumnType.kt` the + * [com.metaobjects.generator.kotlin.KotlinExposedTableGenerator] emits, over the metadata + * module's [DurationWire] (the generated copy carries its own `MetaDurationWire`). + * + * A `Column` over the Postgres-native `INTERVAL`, and elsewhere (SQLite, MySQL) + * a BIGINT count of microseconds. Postgres refuses a signed ISO-8601 duration as interval + * input, so the value binds in the interval spelling `[-]HH:MM:SS[.f]` under a + * `CAST(? AS INTERVAL)` marker; the read takes the interval's text (`ResultSet.getString`, the + * `postgres` IntervalStyle) and refuses one with a year or month part. + */ +internal class MetaDurationColumnType : ColumnType() { + private fun isPostgres(): Boolean = currentDialect.name.equals("postgresql", ignoreCase = true) + override fun sqlType(): String = if (isPostgres()) "INTERVAL" else "BIGINT" + override fun valueFromDB(value: Any): Duration = when (value) { + is Duration -> value + is Number -> DurationWire.fromMicros(value.toLong()) + else -> DurationWire.fromStored(value.toString()) + } + override fun notNullValueToDB(value: Duration): Any = + if (isPostgres()) DurationWire.formatPostgres(value) else DurationWire.toMicros(value) + override fun nonNullValueToString(value: Duration): String = + if (isPostgres()) "'" + notNullValueToDB(value) + "'" else notNullValueToDB(value).toString() + override fun parameterMarker(value: Duration?): String = if (isPostgres()) "CAST(? AS INTERVAL)" else "?" + /** Read the interval's text on Postgres (never pgjdbc's PGInterval object), the integer elsewhere. */ + override fun readObject(rs: ResultSet, index: Int): Any? = if (isPostgres()) rs.getString(index) else rs.getObject(index) +} + +/** Column builder for `field.duration`: a `Column`. */ +internal fun Table.durationColumn(name: String): Column = + registerColumn(name, MetaDurationColumnType()) diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/duration/GeneratedDurationRoundTripTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/duration/GeneratedDurationRoundTripTest.kt new file mode 100644 index 0000000000..903c39232f --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/duration/GeneratedDurationRoundTripTest.kt @@ -0,0 +1,135 @@ +package com.metaobjects.integration.kotlin.tables.duration + +import com.metaobjects.generator.kotlin.KotlinEntityGenerator +import com.metaobjects.generator.kotlin.KotlinExposedTableGenerator +import com.metaobjects.integration.kotlin.PostgresContainer +import com.metaobjects.metadata.ktx.loadString +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.io.TempDir +import java.nio.file.Files +import java.nio.file.Path +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.assertEquals + +/** + * The GENERATED (exposedApi=0) `field.duration` columns, compiled and run: a scalar and an + * `isArray` duration round-trip, a range filter and a sort order spans (never text), on SQLite (a + * BIGINT microsecond count, a JSON array of counts) and Postgres (INTERVAL, INTERVAL[]). + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class GeneratedDurationRoundTripTest { + + private val fixture = """{ + "metadata.root": { "package": "durdemo", "children": [ + { "object.entity": { "name": "Timer", "children": [ + { "source.rdb": { "@table": "timers" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.duration": { "name": "timeout", "@required": true } }, + { "field.duration": { "name": "backoff", "isArray": true } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] } } + ] } + }""".trimIndent() + + private val driverSource = """ + package durdemo + + import org.jetbrains.exposed.sql.Database + import org.jetbrains.exposed.sql.SchemaUtils + import org.jetbrains.exposed.sql.SortOrder + import org.jetbrains.exposed.sql.insert + import org.jetbrains.exposed.sql.selectAll + import org.jetbrains.exposed.sql.transactions.TransactionManager + import org.jetbrains.exposed.sql.transactions.transaction + import java.time.Duration + + object DurationRoundTripDriver { + fun run(jdbcUrl: String, driver: String, user: String, pass: String): String { + val db = Database.connect(jdbcUrl, driver = driver, user = user, password = pass) + return transaction(db) { + SchemaUtils.create(TimerTable) + val spans = mapOf( + 1L to Duration.ofMinutes(30), + 2L to Duration.ofMinutes(90), + 3L to Duration.ofSeconds(-90, -500_000_000), + 4L to Duration.ofHours(26), + ) + for ((id, d) in spans) { + TimerTable.insert { + it[TimerTable.id] = id + it[TimerTable.timeout] = d + it[TimerTable.backoff] = listOf(d, Duration.ofNanos(1_000)) + } + } + val longer = TimerTable.selectAll() + .where { TimerTable.timeout greater Duration.ofHours(1) } + .orderBy(TimerTable.timeout, SortOrder.DESC) + .map { it[TimerTable.id] } + val all = TimerTable.selectAll().orderBy(TimerTable.timeout, SortOrder.ASC).map { it[TimerTable.id] } + val r3 = TimerTable.selectAll().where { TimerTable.id eq 3L }.single() + val problems = mutableListOf() + if (longer != listOf(4L, 2L)) problems += "gt + desc: " + longer + if (all != listOf(3L, 1L, 2L, 4L)) problems += "asc: " + all + if (r3[TimerTable.timeout] != spans.getValue(3L)) problems += "timeout: " + r3[TimerTable.timeout] + if (r3[TimerTable.backoff] != listOf(spans.getValue(3L), Duration.ofNanos(1_000))) problems += "backoff: " + r3[TimerTable.backoff] + if (!jdbcUrl.startsWith("jdbc:postgresql")) { + var kind: String? = null + TransactionManager.current().exec("SELECT typeof(timeout), timeout FROM timers WHERE id = 3") { rs -> + rs.next(); kind = rs.getString(1) + " " + rs.getString(2) + } + if (kind != "integer -90500000") problems += "stored: " + kind + } + if (problems.isEmpty()) "OK" else problems.joinToString("; ") + } + } + } + """.trimIndent() + + private fun compileDriver(): ClassLoader { + val outDir = Files.createTempDirectory("duration-rt-") + try { + val loader = loadString("duration-rt", fixture) + for (g in listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator())) { + g.setArgs(mapOf("outputDir" to outDir.toString())) + g.execute(loader) + } + val sources = Files.walk(outDir).use { s -> + s.filter { it.isRegularFile() && it.toString().endsWith(".kt") } + .map { SourceFile.kotlin(outDir.relativize(it).toString().replace('/', '_'), it.readText()) } + .toList() + }.toMutableList() + sources.add(SourceFile.kotlin("DurationRoundTripDriver.kt", driverSource)) + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + return result.classLoader + } finally { + outDir.toFile().deleteRecursively() + } + } + + private fun drive(cl: ClassLoader, url: String, driver: String, user: String, pass: String): String { + val cls = cl.loadClass("durdemo.DurationRoundTripDriver") + return cls.getMethod("run", String::class.java, String::class.java, String::class.java, String::class.java) + .invoke(cls.getField("INSTANCE").get(null), url, driver, user, pass) as String + } + + @Test + fun `generated duration columns order and round-trip on SQLite`(@TempDir dir: Path) { + assertEquals("OK", drive(compileDriver(), "jdbc:sqlite:${dir.resolve("d.db")}", "org.sqlite.JDBC", "", "")) + } + + @Test + fun `generated duration columns order and round-trip on Postgres`() { + val cl = compileDriver() + PostgresContainer().use { pg -> + assertEquals("OK", drive(cl, pg.jdbcUrl, "org.postgresql.Driver", pg.username, pg.password)) + } + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioTests.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioTests.java index dd4c589699..2e9d171800 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioTests.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioTests.java @@ -40,7 +40,18 @@ final class QueryScenarioTests { private static final Path CORPUS = ScenarioLoader.findCorpusRoot(); private static final List SCENARIOS = ScenarioLoader.loadQueries(CORPUS.resolve("queries")).stream() - .filter(ScenarioEngine.POSTGRES::runs).toList(); + .filter(ScenarioEngine.POSTGRES::runs) + .filter(QueryScenarioTests::selected).toList(); + + /** + * {@code -Dmetaobjects.scenarios=a,b} narrows a run to the named scenarios (one Postgres + * container each, so a targeted run is far cheaper than the corpus). Unset = all. + */ + private static boolean selected(QueryScenario s) { + String only = System.getProperty("metaobjects.scenarios"); + if (only == null || only.isBlank()) return true; + return java.util.Arrays.stream(only.split(",")).map(String::trim).anyMatch(s.name()::equals); + } /** * Scenarios deferred until the underlying Java port gap is closed. diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/RoundtripWriter.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/RoundtripWriter.java index 550843b58a..684bd22871 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/RoundtripWriter.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/RoundtripWriter.java @@ -16,6 +16,7 @@ import com.metaobjects.field.TimeField; import com.metaobjects.field.TimestampField; import com.metaobjects.field.UriField; +import com.metaobjects.field.DurationField; import com.metaobjects.field.UuidField; import com.metaobjects.integration.Scenarios.QuerySpec; import com.metaobjects.manager.ObjectConnection; @@ -186,6 +187,11 @@ private static void setNative(ValueObject vo, MetaField mf, Object raw) { // inet column's NATIVE wire form on read-back (bare address, no ::text cast → no /32|/128 // mask, and Postgres returns the compressed canonical IPv6). vo.setString(name, raw.toString()); + } else if (mf instanceof DurationField) { + // field.duration is String-backed: author the ISO-8601 string verbatim. DurationCodec + // binds it canonically (text) / in the interval spelling (Postgres) and reads it back + // as the canonical wire string. + vo.setString(name, raw.toString()); } else if (mf instanceof UuidField) { // Author the upper-case UUID string; the write codec binds a java.util.UUID and the // read codec returns it lowercase-canonical (the cross-port contract). diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationCorpus.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationCorpus.java new file mode 100644 index 0000000000..8be4d476cc --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationCorpus.java @@ -0,0 +1,29 @@ +package com.metaobjects.integration.api; + +import java.io.IOException; +import java.io.UncheckedIOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; + +/** Locates the duration api-contract sub-corpus ({@code fixtures/api-contract-conformance/duration/}) and its committed per-engine schema. */ +final class DurationCorpus { + private DurationCorpus() {} + + static Path root() { + return ApiContractScenarioLoader.findCorpusRoot().resolve("duration"); + } + + static Path scenariosDir() { + return root().resolve("scenarios"); + } + + /** The committed TypeScript-produced schema of {@code engineId}, read verbatim. */ + static String schema(String engineId) { + try { + return Files.readString(root().resolve("schema." + engineId + ".sql"), StandardCharsets.UTF_8); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbApiContractLane.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbApiContractLane.java new file mode 100644 index 0000000000..d2453122a8 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbApiContractLane.java @@ -0,0 +1,165 @@ +package com.metaobjects.integration.api; + +import com.metaobjects.integration.JdbcTarget; +import com.metaobjects.integration.QueryScenarioRunner; +import com.metaobjects.integration.ScenarioEngine; +import com.metaobjects.integration.api.ApiContractScenarios.ApiRequest; +import com.metaobjects.integration.api.ApiContractScenarios.ApiScenario; +import com.metaobjects.integration.api.generated.GeneratedDurationControllerHarness; +import com.metaobjects.manager.db.ObjectManagerDB; +import com.metaobjects.object.value.ValueObject; +import com.metaobjects.registry.ObjectClassRegistry; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import javax.sql.DataSource; +import java.io.PrintWriter; +import java.nio.file.Files; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.sql.Statement; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.logging.Logger; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; +import static org.junit.jupiter.api.Assertions.assertFalse; + +/** + * The Java full-stack lane of the {@code duration/} api-contract sub-corpus on a real database; one + * subclass per engine. The artifacts under test are the GENERATED {@code TimerController}, DTO, + * allowlist and repository interface, mounted unmodified on embedded Tomcat over an OMDB-backed + * repository. The database executes the committed {@code duration/schema..sql} verbatim; + * {@code timers} is emptied and its counter reset before EVERY scenario (the scenarios write). + */ +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +abstract class DurationOmdbApiContractLane { + + private final List scenarios = ApiContractScenarioLoader.loadScenarios(DurationCorpus.scenariosDir()); + + private GeneratedDurationControllerHarness harness; + private JdbcTarget target; + + abstract ScenarioEngine engine(); + + abstract JdbcTarget openDatabase(); + + abstract void closeDatabase(); + + @BeforeAll + void setUp() throws Exception { + target = openDatabase(); + provisionSchema(); + + harness = new GeneratedDurationControllerHarness(DurationCorpus.root(), Files.createTempDirectory("duration-omdb-controllers")); + + ObjectClassRegistry reg = new ObjectClassRegistry(); + Map> bindings = new HashMap<>(); + bindings.put("acme::ops::Timer", ValueObject.class); + reg.register(() -> bindings); + ObjectClassRegistry.setGlobal(reg); + + ObjectManagerDB omdb = new ObjectManagerDB(); + omdb.setDatabaseDriver(engine().newDriver()); + omdb.setDataSource(dataSource()); + omdb.init(); + harness.start(omdb); + } + + @AfterAll + void tearDown() throws Exception { + try { + if (harness != null) harness.close(); + } finally { + closeDatabase(); + } + } + + Stream scenarios() { + return scenarios.stream() + .filter(s -> ApiContractScenarioLoader.runsOn(s, engine().id())) + .map(s -> Arguments.of(s.name(), s)); + } + + @Test + void theEngineRunsAtLeastOneScenario() { + // A scenarios directory that resolved to nothing, or an engines: filter that dropped + // everything, would leave this lane green and empty. + assertFalse(scenarios().findAny().isEmpty()); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + void scenario(String name, ApiScenario scenario) { + assertDoesNotThrow(() -> { + reseed(); + for (ApiRequest req : scenario.requests()) { + GeneratedDurationControllerHarness.Response res = + harness.exchange(req.method(), req.path(), req.body()); + Object parsed = harness.parseBody(res.body()); + ApiContractAssertions.assertResponse(scenario.name(), req, res.status(), parsed); + } + }); + } + + // ----------------------------------------------------------------------- + // database provisioning + // ----------------------------------------------------------------------- + + private void provisionSchema() throws SQLException { + ScenarioEngine engine = engine(); + String schema = DurationCorpus.schema(engine.id()); + try (Connection c = connection()) { + if (engine == ScenarioEngine.POSTGRES) { + try (Statement s = c.createStatement()) { s.execute(schema); } + } else { + List statements = ScenarioEngine.splitStatements(schema); + if (engine == ScenarioEngine.MYSQL) QueryScenarioRunner.dropSchema(c, statements); + try (Statement s = c.createStatement()) { + for (String stmt : statements) s.execute(stmt); + } + } + } + } + + /** Empty {@code timers} and restart its id counter, so the first POST of a scenario is id 1. */ + private void reseed() throws SQLException { + ScenarioEngine engine = engine(); + String q = engine.quote(); + try (Connection c = connection(); Statement s = c.createStatement()) { + s.execute((engine == ScenarioEngine.SQLITE ? "DELETE FROM " : "TRUNCATE TABLE ") + q + "timers" + q); + switch (engine) { + case POSTGRES -> s.execute("ALTER TABLE \"timers\" ALTER COLUMN \"id\" RESTART WITH 1"); + case MYSQL -> s.execute("ALTER TABLE `timers` AUTO_INCREMENT = 1"); + case SQLITE -> s.execute("DELETE FROM sqlite_sequence WHERE name = 'timers'"); + } + } + } + + private Connection connection() throws SQLException { + return DriverManager.getConnection(target.jdbcUrl(), target.username(), target.password()); + } + + private DataSource dataSource() { + return new DataSource() { + @Override public Connection getConnection() throws SQLException { return connection(); } + @Override public Connection getConnection(String u, String p) throws SQLException { return connection(); } + @Override public PrintWriter getLogWriter() { return null; } + @Override public void setLogWriter(PrintWriter out) {} + @Override public void setLoginTimeout(int seconds) {} + @Override public int getLoginTimeout() { return 0; } + @Override public Logger getParentLogger() { return Logger.getLogger("integration-tests"); } + @Override public T unwrap(Class iface) { return null; } + @Override public boolean isWrapperFor(Class iface) { return false; } + }; + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbMySqlApiContractConformanceTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbMySqlApiContractConformanceTest.java new file mode 100644 index 0000000000..6ff1710808 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbMySqlApiContractConformanceTest.java @@ -0,0 +1,24 @@ +package com.metaobjects.integration.api; + +import com.metaobjects.integration.MySqlContainer; +import com.metaobjects.integration.JdbcTarget; +import com.metaobjects.integration.ScenarioEngine; +import org.junit.jupiter.api.DisplayName; + +/** The duration api-contract corpus, GENERATED controllers over an OMDB repository on MySql (throwaway docker container; needs docker). */ +@DisplayName("API contract duration - GENERATED Spring controllers over OMDB on MySql") +final class DurationOmdbMySqlApiContractConformanceTest extends DurationOmdbApiContractLane { + + private MySqlContainer database; + + @Override ScenarioEngine engine() { return ScenarioEngine.MYSQL; } + + @Override JdbcTarget openDatabase() { + database = new MySqlContainer(); + return database; + } + + @Override void closeDatabase() { + if (database != null) database.close(); + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbPostgresApiContractConformanceTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbPostgresApiContractConformanceTest.java new file mode 100644 index 0000000000..87823695e0 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbPostgresApiContractConformanceTest.java @@ -0,0 +1,24 @@ +package com.metaobjects.integration.api; + +import com.metaobjects.integration.PostgresContainer; +import com.metaobjects.integration.JdbcTarget; +import com.metaobjects.integration.ScenarioEngine; +import org.junit.jupiter.api.DisplayName; + +/** The duration api-contract corpus, GENERATED controllers over an OMDB repository on Postgres (throwaway docker container; needs docker). */ +@DisplayName("API contract duration - GENERATED Spring controllers over OMDB on Postgres") +final class DurationOmdbPostgresApiContractConformanceTest extends DurationOmdbApiContractLane { + + private PostgresContainer database; + + @Override ScenarioEngine engine() { return ScenarioEngine.POSTGRES; } + + @Override JdbcTarget openDatabase() { + database = new PostgresContainer(); + return database; + } + + @Override void closeDatabase() { + if (database != null) database.close(); + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbSqliteApiContractConformanceTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbSqliteApiContractConformanceTest.java new file mode 100644 index 0000000000..d7d475e329 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/DurationOmdbSqliteApiContractConformanceTest.java @@ -0,0 +1,24 @@ +package com.metaobjects.integration.api; + +import com.metaobjects.integration.SqliteDatabase; +import com.metaobjects.integration.JdbcTarget; +import com.metaobjects.integration.ScenarioEngine; +import org.junit.jupiter.api.DisplayName; + +/** The duration api-contract corpus, GENERATED controllers over an OMDB repository on SQLite (embedded, no docker). */ +@DisplayName("API contract duration - GENERATED Spring controllers over OMDB on Sqlite") +final class DurationOmdbSqliteApiContractConformanceTest extends DurationOmdbApiContractLane { + + private SqliteDatabase database; + + @Override ScenarioEngine engine() { return ScenarioEngine.SQLITE; } + + @Override JdbcTarget openDatabase() { + database = new SqliteDatabase(); + return database; + } + + @Override void closeDatabase() { + if (database != null) database.close(); + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedDurationControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedDurationControllerHarness.java new file mode 100644 index 0000000000..c104887094 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedDurationControllerHarness.java @@ -0,0 +1,184 @@ +package com.metaobjects.integration.api.generated; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.SerializationFeature; +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule; +import com.metaobjects.generator.spring.SpringControllerGenerator; +import com.metaobjects.generator.spring.SpringDtoGenerator; +import com.metaobjects.generator.spring.SpringFilterAllowlistGenerator; +import com.metaobjects.generator.spring.SpringRepositoryGenerator; +import com.metaobjects.generator.spring.SpringValueObjectGenerator; +import com.metaobjects.integration.api.TomcatHost; +import com.metaobjects.loader.LoaderOptions; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.uri.URIHelper; +import com.metaobjects.manager.db.ObjectManagerDB; +import com.metaobjects.object.MetaObject; +import jakarta.validation.Validation; +import jakarta.validation.Validator; +import java.io.File; +import java.lang.reflect.Constructor; +import java.net.URI; +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import javax.tools.DiagnosticCollector; +import javax.tools.JavaCompiler; +import javax.tools.JavaFileObject; +import javax.tools.ToolProvider; + +/** + * Hosts the GENERATED Spring controller of the duration api-contract sub-corpus + * ({@code fixtures/api-contract-conformance/duration/}) over real HTTP (embedded Tomcat) and a real + * database through OMDB: the writable {@code Timer} entity ({@code /api/timers}). The controller, DTO, + * repository interface and filter allowlist are generated UNMODIFIED; the adopter-owned seam is + * {@link OmdbTimerRepositorySource}. Same shape as {@link GeneratedDecimalControllerHarness}. + */ +public final class GeneratedDurationControllerHarness implements AutoCloseable { + + private static final String PKG = "acme.ops"; + + private final ObjectMapper mapper = new ObjectMapper() + .registerModule(new JavaTimeModule()) + .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS); + private final Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); + private final MetaDataLoader loader; + private final URLClassLoader classLoader; + private final Constructor timerRepoCtor; + private final Constructor timerControllerCtor; + + private TomcatHost host; + + public GeneratedDurationControllerHarness(Path corpusRoot, Path genDir) throws Exception { + Path srcDir = genDir.resolve("src"); + Path classesDir = genDir.resolve("classes"); + Files.createDirectories(srcDir); + Files.createDirectories(classesDir); + + this.loader = loadCorpus(corpusRoot.resolve("meta.json")); + // ADR-0058: the committed config at the api-contract corpus root declares naming.columns: literal. + loader.getRoot().setPhysicalNaming( + com.metaobjects.config.SourceResolver.resolvePhysicalNaming(corpusRoot.getParent())); + // ADR-0019 Amendment 2 - the corpus's config.json (found by walking up from the corpus dir) pins the wire form (string / omit). + loader.getRoot().setWireFormat( + com.metaobjects.config.SourceResolver.discoverWireFormat(java.util.List.of((corpusRoot.getParent()).toUri())) + .orElse(com.metaobjects.config.WireFormat.DEFAULT)); + + runGenerator(new SpringControllerGenerator(), loader, srcDir); + runGenerator(new SpringDtoGenerator(), loader, srcDir); + runGenerator(new SpringRepositoryGenerator(), loader, srcDir); + runGenerator(new SpringFilterAllowlistGenerator(), loader, srcDir); + + Path pkgDir = srcDir.resolve(PKG.replace('.', '/')); + if (!Files.exists(pkgDir.resolve("TimerController.java"))) { + throw new IllegalStateException("codegen did not emit TimerController under " + pkgDir); + } + Files.writeString(pkgDir.resolve("OmdbTimerRepository.java"), OmdbTimerRepositorySource.SOURCE); + + compile(srcDir, classesDir); + + this.classLoader = new URLClassLoader(new URL[]{ classesDir.toUri().toURL() }, getClass().getClassLoader()); + Class timerRepo = classLoader.loadClass(PKG + ".TimerRepository"); + this.timerControllerCtor = classLoader.loadClass(PKG + ".TimerController") + .getDeclaredConstructor(timerRepo, ObjectMapper.class, Validator.class); + this.timerRepoCtor = classLoader.loadClass(OmdbTimerRepositorySource.FQCN) + .getDeclaredConstructor(ObjectManagerDB.class, MetaObject.class); + } + + /** Mount the generated controller over an OMDB-backed repository on {@code omdb}. */ + public void start(ObjectManagerDB omdb) throws Exception { + Object controller = timerControllerCtor.newInstance( + timerRepoCtor.newInstance(omdb, find("Timer")), mapper, validator); + if (host != null) host.close(); + this.host = TomcatHost.start(mapper, controller); + } + + private MetaObject find(String name) { + return loader.getMetaObjects().stream().filter(o -> o.getShortName().equals(name)).findFirst() + .orElseThrow(() -> new IllegalStateException("no " + name + " in the model")); + } + + public MetaDataLoader loader() { + return loader; + } + + public Response exchange(String method, String path, Object jsonBody) throws Exception { + TomcatHost.Response res = host.exchange(method, path, jsonBody == null ? null : mapper.writeValueAsString(jsonBody)); + return new Response(res.status(), res.body()); + } + + public Object parseBody(String body) { + return TomcatHost.parseBody(mapper, body); + } + + @Override + public void close() throws Exception { + if (host != null) host.close(); + classLoader.close(); + } + + public record Response(int status, String body) {} + + private static MetaDataLoader loadCorpus(Path metaJson) { + URI uri = URIHelper.toURI( + "model:file:" + metaJson.toAbsolutePath().toString().replace('\\', '/')); + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), + MetaDataLoader.SUBTYPE_MANUAL, + "api-contract-duration-generated"); + loader.setSourceURIs(List.of(uri)); + loader.init(); + return loader; + } + + private static void runGenerator(Object generator, MetaDataLoader loader, Path outDir) { + Map args = new HashMap<>(); + args.put("outputDir", outDir.toString()); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).setArgs(args); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).execute(loader); + } + + private static void compile(Path srcDir, Path classesDir) throws Exception { + List sources; + try (Stream s = Files.walk(srcDir)) { + sources = s.filter(p -> p.toString().endsWith(".java")) + .map(Path::toFile) + .collect(Collectors.toList()); + } + if (sources.isEmpty()) { + throw new IllegalStateException("no generated .java sources under " + srcDir); + } + + JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); + if (javac == null) { + throw new IllegalStateException( + "JDK (not JRE) required — ToolProvider.getSystemJavaCompiler() returned null"); + } + String cp = System.getProperty("java.class.path"); + DiagnosticCollector diags = new DiagnosticCollector<>(); + var fm = javac.getStandardFileManager(diags, null, StandardCharsets.UTF_8); + List opts = List.of("-classpath", cp, "-d", classesDir.toString(), "-parameters"); + + boolean ok = javac.getTask(null, fm, diags, opts, null, + fm.getJavaFileObjectsFromFiles(sources)).call(); + if (!ok) { + StringBuilder sb = new StringBuilder("generated sources failed to compile:\n"); + for (var d : diags.getDiagnostics()) { + sb.append(" ").append(d.getKind()).append(": ").append(d.getMessage(null)).append('\n'); + if (d.getSource() != null) { + sb.append(" at ").append(d.getSource().getName()) + .append(':').append(d.getLineNumber()).append('\n'); + } + } + throw new IllegalStateException(sb.toString()); + } + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/OmdbTimerRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/OmdbTimerRepositorySource.java new file mode 100644 index 0000000000..c12ff49faa --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/OmdbTimerRepositorySource.java @@ -0,0 +1,41 @@ +package com.metaobjects.integration.api.generated; + +/** + * The Java SOURCE for an OMDB-backed {@code acme.ops.TimerRepository}: the adopter-owned seam of the + * duration api-contract sub-corpus' writable entity. It is {@link OmdbEntryRepositorySource}'s seam + * re-aimed at {@code Timer} (a {@code label} and a {@code java.time.Duration timeout}); a duration is + * a {@link java.time.Duration} on the DTO, written to the model as its canonical text; a filter operand arrives already parsed to one. + */ +final class OmdbTimerRepositorySource { + + private OmdbTimerRepositorySource() {} + + static final String PKG = "acme.ops"; + static final String FQCN = PKG + ".OmdbTimerRepository"; + + static final String SOURCE = OmdbEntryRepositorySource.SOURCE + .replace("package acme.ledger;", "package " + PKG + ";") + .replace("if (dto.bucket() != null) vo.setString(\"bucket\", dto.bucket());", + "if (dto.label() != null) vo.setString(\"label\", dto.label());") + .replace("vo.setString(\"bucket\", dto.bucket());", "vo.setString(\"label\", dto.label());") + .replace("vo.setObject(\"amount\", dto.amount());", "vo.setObject(\"timeout\", span(dto.timeout()));") + .replace("if (dto.flagged() != null) vo.setBoolean(\"flagged\", dto.flagged());", "") + .replace("vo.setBoolean(\"flagged\", Boolean.TRUE.equals(dto.flagged()));", "") + .replace("if (patch.hasBucket()) vo.setString(\"bucket\", patch.bucket());", + "if (patch.hasLabel()) vo.setString(\"label\", patch.label());") + .replace("if (patch.hasAmount()) vo.setObject(\"amount\", patch.amount());", + "if (patch.hasTimeout()) vo.setObject(\"timeout\", span(patch.timeout()));") + .replace("if (patch.hasFlagged()) vo.setBoolean(\"flagged\", Boolean.TRUE.equals(patch.flagged()));", "") + .replace("if (value instanceof BigDecimal exact) return exact;", + "if (value instanceof java.time.Duration span) return span(span);") + .replace(" private Optional load(", + " /** A duration is stored from its canonical text, never Duration.toString (PT-1M-30S is not the accepted form). */\n" + + " private static String span(java.time.Duration d) {\n" + + " return d == null ? null : com.metaobjects.field.DurationWire.format(d);\n" + + " }\n\n" + + " private Optional load(") + .replace("OmdbEntryRepository", "OmdbTimerRepository") + .replace("EntryRepository", "TimerRepository") + .replace("EntryDto", "TimerDto") + .replace("EntryPatch", "TimerPatch"); +} diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java index f1df2868a6..a1c8a511ba 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java @@ -775,6 +775,10 @@ protected MetaDataLoader createLoader(ClassLoader projectClassLoader) { loader.getRoot().setWireFormat(resolveProjectConfig(srcDir) .map(com.metaobjects.config.NeutralConfig::getWireFormat) .orElse(com.metaobjects.config.WireFormat.DEFAULT)); + // The enum storage defaults (`enums`), from the same file, read by the table generators. + loader.getRoot().setEnumDefaults(resolveProjectConfig(srcDir) + .map(com.metaobjects.config.NeutralConfig::getEnumDefaults) + .orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); getLog().info("MetaData Mojo > Create Loader: " + loader.toString()); diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java index 964a29a5cb..2b85c1dacf 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java @@ -522,7 +522,7 @@ private void verifyCodegen() throws MojoExecutionException, MojoFailureException /** * ADR-0019 Amendment 2: the other {@code wire} setting under which the generators reproduce the * committed output exactly, or {@code null} when none does. Probes decimals {native, string} x - * nullMembers {native, omit} ({@code emit} writes what {@code native} writes on the JVM), skipping + * nullMembers {native, omit} x durations {native, iso} ({@code emit} writes what {@code native} writes on the JVM), skipping * the setting this run used, then puts that setting back on the root. Each probe runs fresh * generator instances, since a generator need not support a second {@code execute}. A probe that * fails answers "no match", so the ordinary drift report still prints. @@ -536,8 +536,9 @@ private WireFormat matchingWire(MetaDataLoader loader, ClassLoader projectClassL try { for (WireFormat.Decimals d : WireFormat.Decimals.values()) { for (WireFormat.NullMembers n : List.of(WireFormat.NullMembers.NATIVE, WireFormat.NullMembers.OMIT)) { - if (d == used.decimals() && n == usedNulls) continue; - loader.getRoot().setWireFormat(new WireFormat(d, n)); + for (WireFormat.Durations u : WireFormat.Durations.values()) { + if (d == used.decimals() && n == usedNulls && u == used.durations()) continue; + loader.getRoot().setWireFormat(new WireFormat(d, n, u)); for (GenTarget t : targets) deleteRecursively(t.tempDir); try { for (Generator gen : buildGenerators(projectClassLoader, overrides)) gen.execute(loader); @@ -550,7 +551,8 @@ private WireFormat matchingWire(MetaDataLoader loader, ClassLoader projectClassL if (ownHashes != null) { drift.addAll(strandedOutsideTargets(ownHashes, targets, projectRoot, new ArrayList<>())); } - if (drift.isEmpty()) return new WireFormat(d, n); + if (drift.isEmpty()) return new WireFormat(d, n, u); + } } } return null; @@ -562,14 +564,15 @@ private WireFormat matchingWire(MetaDataLoader loader, ClassLoader projectClassL /** The {@code wire} block that declares {@code wire}. */ static String wireBlock(WireFormat wire) { return "\"wire\": { \"decimals\": \"" + wire.decimals().wireName() - + "\", \"nullMembers\": \"" + wire.nullMembers().wireName() + "\" }"; + + "\", \"nullMembers\": \"" + wire.nullMembers().wireName() + + "\", \"durations\": \"" + wire.durations().wireName() + "\" }"; } /** The verify failure for committed output that another {@code wire} setting reproduces. */ static String wireDiffers(WireFormat committed, WireFormat used) { return "The committed output was generated with " + wireBlock(committed) + "; this run used " + wireBlock(used) + " (.metaobjects/config.json; each key defaults to native). Do not " - + "regenerate: that changes the JSON your API writes for decimals or for a value object's " + + "regenerate: that changes the JSON your API writes for decimals, durations or for a value object's " + "null members. Declare " + wireBlock(committed) + " in .metaobjects/config.json, and " + "generate and verify both read it."; } diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java index 41ac41b1ce..e7b9412f03 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java @@ -155,9 +155,9 @@ public void eachJavaGeneratorsOutputImportsExactlyTheRuntimeItsEntryDeclares() t if (!(gen instanceof com.metaobjects.generator.direct.MultiFileDirectGeneratorBase)) { a.put("outputFilename", e.stableName + ".out"); } - // The shared fitness corpus declares no decimal value-object member, and a value object - // names the decimal wire classes only for one: that generator runs over a small - // dedicated model that has one, so its declared set can be held exact too. + // The shared fitness corpus declares no decimal or duration value-object member, and a + // value object names the decimal / duration wire classes only for one: that generator runs + // over a small dedicated model that has both, so its declared set can be held exact too. gen.setArgs(a).execute("value-object".equals(e.stableName) ? decimalValueObjectLoader() : loader); Set all = imports(out); @@ -189,7 +189,8 @@ public void eachJavaGeneratorsOutputImportsExactlyTheRuntimeItsEntryDeclares() t private static final String DECIMAL_VALUE_OBJECT_MODEL = """ {"metadata.root":{"package":"acme","children":[ {"object.value":{"name":"Price","children":[ - {"field.decimal":{"name":"net","@precision":12,"@scale":2}} + {"field.decimal":{"name":"net","@precision":12,"@scale":2}}, + {"field.duration":{"name":"validFor"}} ]}} ]}}"""; diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectSupportTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectSupportTest.java index c3738617a5..d1a4530251 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectSupportTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectSupportTest.java @@ -328,7 +328,8 @@ public void everyKotlinEntryResolvesItsCopiedClassesToKotlinSources() { assertEquals(List.of("com.metaobjects.generator.kotlin.KotlinEntityGenerator", "com.metaobjects.generator.kotlin.KotlinEnumEmitter", "com.metaobjects.generator.kotlin.KotlinDecimalWireEmitter", - "com.metaobjects.generator.kotlin.KotlinCrossFieldRules"), entity.sources()); + "com.metaobjects.generator.kotlin.KotlinCrossFieldRules", + "com.metaobjects.generator.kotlin.KotlinDurationWireEmitter"), entity.sources()); assertTrue(entity.resourceOf("com.metaobjects.generator.kotlin.KotlinDecimalWireEmitter") .endsWith("/kotlin/KotlinDecimalWireEmitter.kt")); assertTrue(entity.resourceOf("com.metaobjects.generator.kotlin.KotlinEnumEmitter"), diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyWireTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyWireTest.java index 2433530c75..75941efdde 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyWireTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyWireTest.java @@ -87,8 +87,8 @@ public void verifyNamesTheWireSettingTheCommittedOutputWasGeneratedWith() throws } catch (MojoFailureException e) { String msg = e.getMessage(); assertTrue(msg, msg.contains("The committed output was generated with " - + "\"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\" }; this run used " - + "\"wire\": { \"decimals\": \"native\", \"nullMembers\": \"native\" }")); + + "\"wire\": { \"decimals\": \"string\", \"nullMembers\": \"omit\", \"durations\": \"native\" }; this run used " + + "\"wire\": { \"decimals\": \"native\", \"nullMembers\": \"native\", \"durations\": \"native\" }")); assertTrue(msg, msg.contains("Do not regenerate")); assertFalse(msg, msg.contains("generated code is stale")); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 3f2433e6c5..f3f7830ae2 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -299,6 +299,13 @@ public enum ErrorCode { /** @storage "flattened" cannot be combined with isArray=true. */ ERR_STORAGE_FLATTENED_ARRAY, + /** @flattenedColumnPrefix on a field.object whose @storage is not "flattened" (absent is jsonb). */ + ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE, + ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, + ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP, + ERR_SEQUENCE_WITHOUT_INCREMENT, + ERR_SEQUENCE_START_OUT_OF_RANGE, + /** @storage was set on a field that has no @objectRef. */ ERR_STORAGE_WITHOUT_OBJECT_REF, diff --git a/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java b/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java index ec99e0c8c5..de98496cd6 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java +++ b/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java @@ -213,6 +213,22 @@ public void setPhysicalNaming(com.metaobjects.database.PhysicalNaming physicalNa private transient com.metaobjects.config.WireFormat wireFormat = com.metaobjects.config.WireFormat.DEFAULT; + /** + * The project's enum storage defaults ({@code enums} in {@code .metaobjects/config.json}), + * set by the same project-aware load as the naming rules and read through + * {@link com.metaobjects.config.EnumDefaults#of(MetaData)}. A tree loaded with no project has none. + */ + private transient com.metaobjects.config.EnumDefaults enumDefaults = + com.metaobjects.config.EnumDefaults.DEFAULT; + + public com.metaobjects.config.EnumDefaults getEnumDefaults() { + return enumDefaults; + } + + public void setEnumDefaults(com.metaobjects.config.EnumDefaults enumDefaults) { + this.enumDefaults = (enumDefaults != null) ? enumDefaults : com.metaobjects.config.EnumDefaults.DEFAULT; + } + /** The project's wire-format setting for this tree. Never {@code null}. */ public com.metaobjects.config.WireFormat getWireFormat() { return wireFormat; diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/EnumDefaults.java b/server/java/metadata/src/main/java/com/metaobjects/config/EnumDefaults.java new file mode 100644 index 0000000000..2a0c19006c --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/config/EnumDefaults.java @@ -0,0 +1,35 @@ +package com.metaobjects.config; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaRoot; + +/** + * The project's enum storage defaults: {@code enums} in {@code .metaobjects/config.json}. + * {@code maxLength} is the column length (VARCHAR(n)) of a string-stored {@code field.enum} + * that declares no {@code @maxLength}; absent means no default (an unbounded text column). + * Carried on the loaded root beside the naming rules. Mirrors TS {@code enum-length.ts}. + */ +public final class EnumDefaults { + + /** No project default. */ + public static final EnumDefaults DEFAULT = new EnumDefaults(null); + + private final Long maxLength; + + public EnumDefaults(Long maxLength) { + this.maxLength = maxLength; + } + + /** The default column length, or {@code null} when the project declares none. */ + public Long getMaxLength() { + return maxLength; + } + + /** The enum defaults for the tree {@code node} belongs to: what its root carries, else none. */ + public static EnumDefaults of(MetaData node) { + for (MetaData n = node; n != null; n = n.getParent()) { + if (n instanceof MetaRoot root) return root.getEnumDefaults(); + } + return DEFAULT; + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java b/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java index 7c30658a46..f6d0282e6a 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java +++ b/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java @@ -69,6 +69,7 @@ public final class NeutralConfig { private final List migrateScope; private final PhysicalNaming physicalNaming; private final WireFormat wireFormat; + private EnumDefaults enumDefaults = EnumDefaults.DEFAULT; private NeutralConfig(List> sources, List> dependencies, List scopeInclude, List migrateScope, @@ -134,6 +135,11 @@ public WireFormat getWireFormat() { return wireFormat; } + /** The project's enum storage defaults ({@code enums} in this file). Never {@code null}. */ + public EnumDefaults getEnumDefaults() { + return enumDefaults; + } + /** * Returns empty when {@code /.metaobjects/config.json} does not exist. * A file that EXISTS but is malformed throws — swallowing it would make a typo'd @@ -328,8 +334,41 @@ public static NeutralConfig parse(String content, String origin) { // Unknown top-level keys, and every key of `scope`/`migrate` but the two read above, // are IGNORED by design — see the class javadoc. - return new NeutralConfig(specs, dependencies, scopeInclude, migrateScope, physicalNaming, + NeutralConfig cfg = new NeutralConfig(specs, dependencies, scopeInclude, migrateScope, physicalNaming, wireFormat); + cfg.enumDefaults = parseEnums(root.get("enums"), origin); + return cfg; + } + + /** + * The {@code enums} block: {@code maxLength}, a positive integer, is the column length of a + * string-stored enum with no {@code @maxLength}. An unknown key, a wrong-typed value, or a + * non-object block is refused, the same rule as {@code naming}. + */ + private static EnumDefaults parseEnums(JsonElement enums, String origin) { + if (enums == null) return EnumDefaults.DEFAULT; + if (!enums.isJsonObject()) { + throw new MetaDataException(origin + ": \"enums\" must be an object", ErrorCode.ERR_BAD_ATTR_VALUE); + } + Long maxLength = null; + JsonObject obj = enums.getAsJsonObject(); + for (String key : obj.keySet()) { + if (!"maxLength".equals(key)) { + throw new MetaDataException( + origin + ": \"enums\" has unknown key \"" + key + "\"; expected \"maxLength\"", + ErrorCode.ERR_BAD_ATTR_VALUE); + } + JsonElement v = obj.get(key); + java.math.BigDecimal n = v.isJsonPrimitive() && v.getAsJsonPrimitive().isNumber() + ? v.getAsBigDecimal() : null; + if (n == null || n.signum() <= 0 || n.stripTrailingZeros().scale() > 0) { + throw new MetaDataException( + origin + ": \"enums.maxLength\" must be a positive integer", + ErrorCode.ERR_BAD_ATTR_VALUE); + } + maxLength = n.longValueExact(); + } + return maxLength == null ? EnumDefaults.DEFAULT : new EnumDefaults(maxLength); } /** Mirrors the TS {@code DependencyName} regex (DESIGN §3.1). */ @@ -438,6 +477,7 @@ private static WireFormat parseWire(JsonElement wire, String origin) { JsonObject wireObj = wire.getAsJsonObject(); WireFormat.Decimals decimals = WireFormat.Decimals.NATIVE; WireFormat.NullMembers nullMembers = WireFormat.NullMembers.NATIVE; + WireFormat.Durations durations = WireFormat.Durations.NATIVE; for (String key : wireObj.keySet()) { JsonElement el = wireObj.get(key); String raw = el.isJsonPrimitive() && el.getAsJsonPrimitive().isString() ? el.getAsString() : null; @@ -458,12 +498,20 @@ private static WireFormat parseWire(JsonElement wire, String origin) { ErrorCode.ERR_BAD_ATTR_VALUE); } } + case "durations" -> { + durations = raw == null ? null : WireFormat.durationsOf(raw); + if (durations == null) { + throw new MetaDataException( + origin + ": \"wire.durations\" must be one of \"native\", \"iso\"", + ErrorCode.ERR_BAD_ATTR_VALUE); + } + } default -> throw new MetaDataException( origin + ": \"wire\" has unknown key \"" + key - + "\"; expected \"decimals\" and/or \"nullMembers\"", + + "\"; expected \"decimals\", \"nullMembers\" and/or \"durations\"", ErrorCode.ERR_BAD_ATTR_VALUE); } } - return new WireFormat(decimals, nullMembers); + return new WireFormat(decimals, nullMembers, durations); } } diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java b/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java index b9985dbc8a..fac3312f07 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java +++ b/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java @@ -225,6 +225,11 @@ public static java.util.Optional discoverWireFormat(List discoverEnumDefaults(List uris) { + return discoverConfig(uris).map(NeutralConfig::getEnumDefaults); + } + /** * ADR-0058 — the physical-naming rules of the {@code .metaobjects/config.json} found by * walking up from the first {@code file:} URI (stopping after a directory holding diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/WireFormat.java b/server/java/metadata/src/main/java/com/metaobjects/config/WireFormat.java index 044dccfb4f..2cb6f790e9 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/config/WireFormat.java +++ b/server/java/metadata/src/main/java/com/metaobjects/config/WireFormat.java @@ -23,12 +23,15 @@ * {@code .metaobjects/config.json}): the OUTPUT form of a generated REST surface. Input rules * (W2, W3) never change with it. * - *

Both keys default to {@code native}: what the language's standard serializer writes (for + *

Every key defaults to {@code native}: what the language's standard serializer writes (for * Java and Kotlin, Jackson as the host configures it: a decimal is a JSON number, a value * object's null member is {@code null} with the key present). {@code "decimals": "string"} and - * {@code "nullMembers": "omit"} are the cross-port form the conformance corpora pin.

+ * {@code "nullMembers": "omit"} and {@code "durations": "iso"} are the cross-port form the + * conformance corpora pin. A {@code native} duration is Jackson's own {@code java.time} form + * ({@code Duration.toString()} / {@code Duration.parse} under Spring Boot's default of durations + * not written as timestamps).

*/ -public record WireFormat(Decimals decimals, NullMembers nullMembers) { +public record WireFormat(Decimals decimals, NullMembers nullMembers, Durations durations) { /** How a {@code field.decimal} is written. */ public enum Decimals { @@ -62,12 +65,33 @@ public enum NullMembers { public String wireName() { return wireName; } } - /** Both keys at {@code native}: what a tree loaded with no project carries. */ - public static final WireFormat DEFAULT = new WireFormat(Decimals.NATIVE, NullMembers.NATIVE); + /** How a {@code field.duration} is written and read. */ + public enum Durations { + /** The language's standard serializer: Jackson's {@code java.time} handling of a {@code Duration}. */ + NATIVE("native"), + /** Canonical ISO-8601 ({@code PT1H30M}), the rule in {@code fixtures/duration-wire/}. */ + ISO("iso"); + + private final String wireName; + + Durations(String wireName) { this.wireName = wireName; } + + /** The value as spelled in {@code .metaobjects/config.json}. */ + public String wireName() { return wireName; } + } + + /** Every key at {@code native}: what a tree loaded with no project carries. */ + public static final WireFormat DEFAULT = new WireFormat(Decimals.NATIVE, NullMembers.NATIVE, Durations.NATIVE); public WireFormat { if (decimals == null) decimals = Decimals.NATIVE; if (nullMembers == null) nullMembers = NullMembers.NATIVE; + if (durations == null) durations = Durations.NATIVE; + } + + /** True when a duration is written and read as canonical ISO-8601 (the duration wire annotations are emitted). */ + public boolean durationsIso() { + return durations == Durations.ISO; } /** True when a decimal is written as the canonical string (the serializer annotation is emitted). */ @@ -92,6 +116,12 @@ public static NullMembers nullMembersOf(String raw) { return null; } + /** The {@code durations} value spelled {@code raw}, or {@code null} when it names none. */ + public static Durations durationsOf(String raw) { + for (Durations d : Durations.values()) if (d.wireName.equals(raw)) return d; + return null; + } + /** The wire setting {@code node}'s root carries, or {@link #DEFAULT} for a detached node. */ public static WireFormat of(MetaData node) { for (MetaData n = node; n != null; n = n.getParent()) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/database/ColumnNaming.java b/server/java/metadata/src/main/java/com/metaobjects/database/ColumnNaming.java index 0726e5bc07..690a051764 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/database/ColumnNaming.java +++ b/server/java/metadata/src/main/java/com/metaobjects/database/ColumnNaming.java @@ -45,6 +45,13 @@ private ColumnNaming() {} public static final String SNAKE_CASE = "snake_case"; /** camelCase / PascalCase to kebab-case. */ public static final String KEBAB_CASE = "kebab-case"; + /** + * The first letter upper-cased, nothing else changed ({@code requestURL} to + * {@code RequestURL}, {@code id} to {@code Id}): for a schema whose columns are named like + * the fields with an upper-case first letter, so a camelCase model maps onto it with no + * {@code @column}. Not a word split: one UTF-16 unit in, one out. + */ + public static final String UPPER_FIRST = "upperFirst"; /** The project default (ADR-0058) — what {@code meta migrate} has always created. */ public static final String DEFAULT = SNAKE_CASE; @@ -83,9 +90,16 @@ public static String apply(String name, String strategy) { if (LITERAL.equals(strategy)) return name; if (SNAKE_CASE.equals(strategy)) return toSnakeCase(name); if (KEBAB_CASE.equals(strategy)) return toSnakeCase(name).replace('_', '-'); + if (UPPER_FIRST.equals(strategy)) return toUpperFirstColumn(name); throw new IllegalArgumentException( "unknown column-naming strategy '" + strategy + "'; expected one of: " - + LITERAL + ", " + SNAKE_CASE + ", " + KEBAB_CASE); + + LITERAL + ", " + SNAKE_CASE + ", " + KEBAB_CASE + ", " + UPPER_FIRST); + } + + /** The {@link #UPPER_FIRST} rule: the first letter upper-cased, the rest verbatim. */ + public static String toUpperFirstColumn(String name) { + if (name == null || name.isEmpty()) return name; + return Character.toUpperCase(name.charAt(0)) + name.substring(1); } /** @@ -119,6 +133,43 @@ public static String resolve(MetaField field, String strategy) { return apply(field.getName(), strategy); } + /** + * The text before each member's column when {@code field} (a {@code field.object + * @storage "flattened"}) expands its value object into the owning table: its + * {@code @flattenedColumnPrefix} verbatim ({@code ""} allowed), else its own resolved column + * and {@code _}. Core naming, not one generator's: the migration creates these columns and + * the generated ORM mappings (the Kotlin Exposed tables) address them, so every one of them + * spells them here. + * + *

ADR-0039: read RESOLVING — the prefix and {@code @column} may be inherited through + * {@code extends}.

+ */ + public static String resolveFlattenedPrefix(MetaField field, String strategy) { + if (field.hasMetaAttr(CoreDBMetaDataProvider.FLATTENED_COLUMN_PREFIX, true)) { + Object prefix = field.getMetaAttr(CoreDBMetaDataProvider.FLATTENED_COLUMN_PREFIX, true).getValue(); + if (prefix instanceof String) return (String) prefix; + } + return resolve(field, strategy) + "_"; + } + + /** {@link #resolveFlattenedPrefix(MetaField, String)} under the project's rule. */ + public static String resolveFlattenedPrefix(MetaField field) { + return resolveFlattenedPrefix(field, PhysicalNaming.of(field).columns()); + } + + /** + * The column a member of a flattened value object gets in the owning table: the embedding + * field's {@link #resolveFlattenedPrefix prefix} followed by the member's own resolved column. + */ + public static String resolveFlattened(MetaField field, MetaField member, String strategy) { + return resolveFlattenedPrefix(field, strategy) + resolve(member, strategy); + } + + /** {@link #resolveFlattened(MetaField, MetaField, String)} under the project's rule. */ + public static String resolveFlattened(MetaField field, MetaField member) { + return resolveFlattened(field, member, PhysicalNaming.of(field).columns()); + } + /** * {@code displayName} to {@code display_name}, {@code URLPath} to {@code url_path}. * diff --git a/server/java/metadata/src/main/java/com/metaobjects/database/CoreDBMetaDataProvider.java b/server/java/metadata/src/main/java/com/metaobjects/database/CoreDBMetaDataProvider.java index 8e59a9d217..5ed395ec73 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/database/CoreDBMetaDataProvider.java +++ b/server/java/metadata/src/main/java/com/metaobjects/database/CoreDBMetaDataProvider.java @@ -1,6 +1,7 @@ package com.metaobjects.database; import com.metaobjects.attr.StringAttribute; +import java.util.List; import com.metaobjects.index.Index; import com.metaobjects.identity.SecondaryIdentity; import com.metaobjects.registry.MetaDataRegistry; @@ -46,6 +47,13 @@ public class CoreDBMetaDataProvider implements MetaDataTypeProvider { */ public static final String COLUMN = "column"; + /** + * {@code @flattenedColumnPrefix} on a {@code field.object @storage "flattened"}: the text before each + * member's column, verbatim ({@code ""} allowed). Absent: the field's resolved column and + * {@code _} ({@link ColumnNaming#resolveFlattenedPrefix(com.metaobjects.field.MetaField)}). + */ + public static final String FLATTENED_COLUMN_PREFIX = "flattenedColumnPrefix"; + /** * Physical column-type attribute (R6 Plan 2b, ADR-0013). Selects the DB column type * while leaving the logical field type and its idiomatic native binding untouched — @@ -109,7 +117,7 @@ public String getProviderId() { @Override public String[] getDependencies() { // Depends on field types, object types, identity types, and index types so it loads after them. - return new String[]{"field-types", "object-types", "identity-types", "index-types"}; + return new String[]{"field-types", "object-types", "identity-types", "index-types", "validator-types"}; } @Override @@ -141,6 +149,24 @@ public void registerTypes(MetaDataRegistry registry) { .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); }); + // --- @constraintName: the CHECK name of a CHECK-generating validator, and of a --- + // field.enum's membership (db.json's validator and field.enum blocks; identity.reference + // declares its own). required is NOT NULL; array and base generate no CHECK. + for (Class c : List.of( + com.metaobjects.validator.LengthValidator.class, + com.metaobjects.validator.RegexValidator.class, + com.metaobjects.validator.NumericValidator.class, + com.metaobjects.validator.ComparisonValidator.class, + com.metaobjects.validator.RequiredWhenValidator.class, + com.metaobjects.validator.PresentIffValidator.class, + com.metaobjects.validator.AtLeastOneValidator.class, + com.metaobjects.validator.ExactlyOneValidator.class, + com.metaobjects.field.EnumField.class)) { + registry.extendType(c, def -> def + .optionalAttributeWithConstraints(com.metaobjects.validator.MetaValidator.ATTR_CONSTRAINT_NAME) + .ofType(StringAttribute.SUBTYPE_STRING).asSingle()); + } + // --- index.lookup RDB-physical attrs --- registry.extendType(com.metaobjects.index.LookupIndex.class, def -> { diff --git a/server/java/metadata/src/main/java/com/metaobjects/database/EnumLength.java b/server/java/metadata/src/main/java/com/metaobjects/database/EnumLength.java new file mode 100644 index 0000000000..8867237376 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/database/EnumLength.java @@ -0,0 +1,66 @@ +package com.metaobjects.database; + +import com.metaobjects.ErrorCode; +import com.metaobjects.MetaData; +import com.metaobjects.MetaDataException; +import com.metaobjects.config.EnumDefaults; +import com.metaobjects.field.EnumField; +import com.metaobjects.field.MetaField; +import com.metaobjects.object.MetaObject; +import com.metaobjects.util.ErrorMessageConstants; + +import java.util.List; + +/** + * The column length of a string-stored {@code field.enum}: its {@code @maxLength}, else the + * project's {@code enums.maxLength}, else none (an unbounded text column). Mirrors TS + * {@code resolveEnumMaxLength}. + */ +public final class EnumLength { + + private EnumLength() {} + + /** + * The column length of {@code field}, a string-stored {@code field.enum}: its + * {@code @maxLength} (ADR-0039 — resolving), else the project default, else {@code null}. + * {@code null} for an {@code @intValueMap} enum, whose column is an integer, and for any + * other field. + * + *

The loader refuses an {@code @maxLength} a member does not fit; it does not see the + * project config, so a member longer than the project DEFAULT is refused here + * ({@code ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH}), by every consumer that sizes the column.

+ */ + public static Long resolveMaxLength(MetaField field) { + if (!(field instanceof EnumField) || field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null; + // An array enum is a text[] column, whose elements carry no length. + if (field.isArrayType()) return null; + if (field.hasMetaAttr(EnumField.ATTR_MAX_LENGTH)) { + Object own = field.getMetaAttr(EnumField.ATTR_MAX_LENGTH).getValue(); + if (own instanceof Number n) return n.longValue(); + } + Long fallback = EnumDefaults.of(field).getMaxLength(); + if (fallback == null) return null; + String longest = null; + Object raw = field.hasMetaAttr(EnumField.ATTR_VALUES) + ? field.getMetaAttr(EnumField.ATTR_VALUES).getValue() : null; + if (raw instanceof List values) { + for (Object o : values) { + String v = String.valueOf(o); + if (longest == null || v.length() > longest.length()) longest = v; + } + } + if (longest != null && longest.length() > fallback) { + MetaData owner = field.getParent(); + String label = owner instanceof MetaObject o + ? o.getShortName() + "." + field.getShortName() : field.getShortName(); + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH + + ": field.enum \"" + label + "\" has the member \"" + longest + "\" (" + + longest.length() + " characters), longer than the project's enums.maxLength " + + fallback + ". Give the field its own @maxLength of at least " + longest.length() + + ", raise enums.maxLength, or shorten the member.", + ErrorCode.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH); + } + return fallback; + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/database/PhysicalNaming.java b/server/java/metadata/src/main/java/com/metaobjects/database/PhysicalNaming.java index 87971a3180..a25be31c57 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/database/PhysicalNaming.java +++ b/server/java/metadata/src/main/java/com/metaobjects/database/PhysicalNaming.java @@ -46,17 +46,18 @@ private PhysicalNaming(String columns, String tables) { * whole schema to the wrong names and report success. * * @throws MetaDataException on a {@code columns} value outside - * {@link ColumnNaming#SNAKE_CASE}/{@link ColumnNaming#LITERAL}/{@link ColumnNaming#KEBAB_CASE}, - * or a {@code tables} value outside {@link #TABLE_SNAKE_CASE}/{@link #TABLE_LITERAL} - * (kebab-case is a column rule only — not a table rule). + * {@link ColumnNaming#SNAKE_CASE}/{@link ColumnNaming#LITERAL}/{@link ColumnNaming#KEBAB_CASE}/ + * {@link ColumnNaming#UPPER_FIRST}, or a {@code tables} value outside + * {@link #TABLE_SNAKE_CASE}/{@link #TABLE_LITERAL} (kebab-case and upperFirst are column + * rules only — not table rules). */ public static PhysicalNaming of(String columns, String tables) { if (!ColumnNaming.SNAKE_CASE.equals(columns) && !ColumnNaming.LITERAL.equals(columns) - && !ColumnNaming.KEBAB_CASE.equals(columns)) { + && !ColumnNaming.KEBAB_CASE.equals(columns) && !ColumnNaming.UPPER_FIRST.equals(columns)) { throw new MetaDataException( "unknown \"naming.columns\" rule \"" + columns + "\" in .metaobjects/config.json; " + "expected one of: " + ColumnNaming.SNAKE_CASE + ", " + ColumnNaming.LITERAL - + ", " + ColumnNaming.KEBAB_CASE, + + ", " + ColumnNaming.KEBAB_CASE + ", " + ColumnNaming.UPPER_FIRST, ErrorCode.ERR_BAD_ATTR_VALUE); } if (!TABLE_SNAKE_CASE.equals(tables) && !TABLE_LITERAL.equals(tables)) { @@ -68,7 +69,7 @@ public static PhysicalNaming of(String columns, String tables) { return new PhysicalNaming(columns, tables); } - /** The {@code naming.columns} rule — snake_case / literal / kebab-case. */ + /** The {@code naming.columns} rule — snake_case / literal / kebab-case / upperFirst. */ public String columns() { return columns; } diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/DurationField.java b/server/java/metadata/src/main/java/com/metaobjects/field/DurationField.java new file mode 100644 index 0000000000..e33ccb7b92 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/field/DurationField.java @@ -0,0 +1,78 @@ +/* + * Copyright 2004 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.field; + +import com.metaobjects.DataTypes; +import com.metaobjects.registry.MetaDataRegistry; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * A duration field — a fixed-length span of time (weeks, days, hours, minutes, seconds). + * + *

{@code field.duration} is a logical subtype (ADR-0036/0037): it fixes the value's + * meaning and drives idiomatic native binding per language — {@code java.time.Duration} + * on the JVM — independent of how the value is physically stored. Like {@link InetField} it is + * a bare scalar with no own attributes.

+ * + *

Wire / storage contract. The value is carried as a {@code String} + * ({@link DataTypes#STRING}): the canonical ISO-8601 duration {@link DurationWire} writes + * ({@code [-]PT[nH][nM][n[.f]S]}). The DB column is the Postgres-native {@code interval}, and + * elsewhere a 64-bit count of microseconds. A week is exactly 7 days and a day exactly 24 hours; years and months are refused: a + * calendar span has no fixed length.

+ * + * @since 1.2.0 + */ +@SuppressWarnings("serial") +public class DurationField extends PrimitiveField { + + private static final Logger log = LoggerFactory.getLogger(DurationField.class); + + /** duration field subtype constant — cross-language vocabulary ({@code field.duration}). */ + public static final String SUBTYPE_DURATION = "duration"; + + public DurationField(String name) { + // String-backed value: the wire/storage representation is the canonical ISO-8601 + // duration. The native java.time.Duration binding is a codegen concern. + super(SUBTYPE_DURATION, name, DataTypes.STRING); + } + + /** + * Register the {@code field.duration} type with the MetaDataRegistry. A bare scalar + * inheriting the common field attributes, with no own attrs. The cross-port description is + * sourced from {@code spec/metamodel/field.json}. + */ + public static void registerTypes(MetaDataRegistry registry) { + try { + registry.registerType(DurationField.class, def -> def + .type(TYPE_FIELD).subType(SUBTYPE_DURATION) + .description("duration field — fixed-length span of time; native java.time.Duration binding") + .inheritsFrom(TYPE_FIELD, SUBTYPE_BASE)); + if (log != null) log.debug("Registered DurationField type with unified registry"); + } catch (Exception e) { + if (log != null) log.error("Failed to register DurationField type with unified registry", e); + } + } + + /** + * Manually create a DurationField. + * @param name Name of the field + * @return New DurationField + */ + public static DurationField create(String name) { + return new DurationField(name); + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/DurationWire.java b/server/java/metadata/src/main/java/com/metaobjects/field/DurationWire.java new file mode 100644 index 0000000000..05e4a84c23 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/field/DurationWire.java @@ -0,0 +1,256 @@ +/* + * Copyright 2004 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.field; + +import java.time.Duration; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * The {@code field.duration} wire rule (shared vectors: {@code fixtures/duration-wire/}). + * + *
    + *
  • Accepted on read: {@link #ACCEPTED} — an optional sign, {@code P}, an optional week + * count ({@code W} = exactly 7 days), an optional day count ({@code D} = exactly 24 hours), an + * optional {@code T} followed by hours, minutes and seconds (a fraction of any length). Years + * and months are refused: a calendar span has no fixed length. Validated BEFORE any parse, + * and parsed here: {@link Duration#parse} accepts more (fractional hours, per-component signs, + * lowercase) and refuses weeks and more than nine fractional digits.
    • + *
  • Canonical on write ({@link #format}): {@code [-]PT[H][M][[.]S]} — + * hours never folded into days, zero components omitted, zero is {@code PT0S}, up to nine + * fractional digits (a {@link Duration} holds nanoseconds; trailing zeros dropped), one + * leading {@code -} for a negative span. {@link Duration#toString()} differs for negatives, + * so it is never the wire form.
    • + *
  • Postgres: an {@code interval} reads back (IntervalStyle {@code postgres}) as + * {@code [-]N days [-+]HH:MM:SS[.f]} — {@link #parsePostgres}; one with a year or month part + * is refused. Postgres refuses a signed ISO-8601 duration as input, so a TEXT bind uses + * {@link #formatPostgres} ({@code [-]HH:MM:SS[.f]}).
    • + *
+ * + *

A parsed value keeps nanoseconds (more fractional digits are truncated toward zero, never + * rounded). Storage is the database's limit, not this one: a Postgres interval and the + * SQLite / MySQL microsecond column both hold microseconds, so {@link #formatPostgres} and + * {@link #toMicros} truncate below a microsecond toward zero.

+ */ +public final class DurationWire { + + private DurationWire() {} + + /** The accepted ISO-8601 duration form. */ + public static final String ACCEPTED_REGEX = + "^[-+]?P(?!$)(\\d+W)?(\\d+D)?(T(?=\\d)(\\d+H)?(\\d+M)?(\\d+(\\.\\d+)?S)?)?$"; + + /** {@link #ACCEPTED_REGEX}, compiled. */ + public static final Pattern ACCEPTED = Pattern.compile(ACCEPTED_REGEX); + + /** Capturing form of {@link #ACCEPTED}: sign, weeks, days, hours, minutes, seconds, fraction. */ + private static final Pattern PARTS = Pattern.compile( + "^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$"); + + /** A Postgres interval under IntervalStyle {@code postgres}, with no year or month part. */ + private static final Pattern POSTGRES = Pattern.compile( + "^(?:([-+]?\\d+) days?(?: |$))?(?:([-+])?(\\d+):(\\d{2}):(\\d{2})(?:\\.(\\d{1,6}))?)?$"); + + private static final Pattern POSTGRES_CALENDAR = Pattern.compile("\\b(years?|mons?)\\b"); + + /** What a refused value is told it should be. */ + public static final String EXPECTED = + "an ISO-8601 duration such as PT1H30M (no years or months)"; + + /** + * True iff {@code s} is in the accepted ISO-8601 form and fits a {@link Duration} (its own range + * is the only limit; a value past it is refused, never wrapped). + */ + public static boolean isDuration(String s) { + if (s == null || !ACCEPTED.matcher(s).matches()) return false; + try { + parse(s); + return true; + } catch (IllegalArgumentException outOfRange) { + return false; + } + } + + /** + * Parse an accepted ISO-8601 duration to the nanosecond (fractional digits past the ninth are + * truncated toward zero, never rounded). + * + * @throws IllegalArgumentException when {@code s} is not in the accepted form, or overflows + */ + public static Duration parse(String s) { + if (s == null) throw new IllegalArgumentException("field.duration: null is not a duration"); + Matcher m = PARTS.matcher(s); + if (!m.matches()) { + throw new IllegalArgumentException("field.duration: not " + EXPECTED + ": " + quote(s)); + } + try { + long seconds = Math.addExact( + Math.addExact(Math.multiplyExact(num(m.group(2)), 604_800L), + Math.multiplyExact(num(m.group(3)), 86_400L)), + Math.addExact(Math.multiplyExact(num(m.group(4)), 3_600L), + Math.addExact(Math.multiplyExact(num(m.group(5)), 60L), num(m.group(6))))); + Duration d = Duration.ofSeconds(seconds, nanos(m.group(7))); + return "-".equals(m.group(1)) ? d.negated() : d; + } catch (ArithmeticException | NumberFormatException e) { + throw new IllegalArgumentException("field.duration: out of range: " + quote(s), e); + } + } + + /** {@link #parse} then {@link #format}: the canonical spelling of an accepted duration. */ + public static String canonicalize(String s) { + return format(parse(s)); + } + + /** The canonical wire form of {@code d}, to the nanosecond. */ + public static String format(Duration d) { + if (d == null) throw new IllegalArgumentException("field.duration: null is not a duration"); + boolean negative = d.isNegative(); + Duration mag = negative ? d.negated() : d; + long total = mag.getSeconds(); + long nanos = mag.getNano(); + if (total == 0 && nanos == 0) return "PT0S"; + long h = total / 3_600L; + long mi = (total % 3_600L) / 60L; + long s = total % 60L; + StringBuilder b = new StringBuilder(negative ? "-PT" : "PT"); + if (h > 0) b.append(h).append('H'); + if (mi > 0) b.append(mi).append('M'); + if (s > 0 || nanos > 0) { + b.append(s); + appendFraction(b, nanos, 9); + b.append('S'); + } + return b.toString(); + } + + /** + * The Postgres interval input spelling of {@code d}: {@code [-]HH:MM:SS[.f]}, no day part. An + * interval holds microseconds, so a sub-microsecond part is truncated toward zero here. + */ + public static String formatPostgres(Duration d) { + if (d == null) throw new IllegalArgumentException("field.duration: null is not a duration"); + boolean negative = d.isNegative(); + Duration mag = negative ? d.negated() : d; + long total = mag.getSeconds(); + long micros = mag.getNano() / 1_000L; + if (total == 0 && micros == 0) negative = false; + StringBuilder b = new StringBuilder(negative ? "-" : ""); + long h = total / 3_600L; + if (h < 10) b.append('0'); + b.append(h).append(':'); + two(b, (total % 3_600L) / 60L); + b.append(':'); + two(b, total % 60L); + appendFraction(b, micros, 6); + return b.toString(); + } + + /** + * Read the text a Postgres {@code interval} column returns (IntervalStyle {@code postgres}). + * + * @throws IllegalArgumentException for an interval with a year or month part (a calendar span + * is not a duration — never a guess at the length of a month) or unreadable text + */ + public static Duration parsePostgres(String s) { + if (s == null) throw new IllegalArgumentException("field.duration: null is not an interval"); + if (POSTGRES_CALENDAR.matcher(s).find()) { + throw new IllegalArgumentException( + "field.duration: the interval " + quote(s) + " has a year or month part; a calendar span is not a duration"); + } + Matcher m = POSTGRES.matcher(s); + if (s.isEmpty() || !m.matches() || (m.group(1) == null && m.group(3) == null)) { + throw new IllegalArgumentException("field.duration: not a Postgres interval: " + quote(s)); + } + try { + Duration days = Duration.ofSeconds(Math.multiplyExact(m.group(1) == null ? 0L : Long.parseLong(m.group(1)), 86_400L)); + if (m.group(3) == null) return days; + long secs = Math.addExact(Math.multiplyExact(Long.parseLong(m.group(3)), 3_600L), + Long.parseLong(m.group(4)) * 60L + Long.parseLong(m.group(5))); + Duration time = Duration.ofSeconds(secs, nanos(m.group(6))); + return days.plus("-".equals(m.group(2)) ? time.negated() : time); + } catch (ArithmeticException | NumberFormatException e) { + throw new IllegalArgumentException("field.duration: interval out of range: " + quote(s), e); + } + } + + /** + * A stored duration: the canonical (or any accepted) ISO-8601 string a text column holds, or + * the text a Postgres interval column returns. + */ + public static Duration fromStored(String s) { + return s != null && ACCEPTED.matcher(s).matches() ? parse(s) : parsePostgres(s); + } + + /** {@link #fromStored} then {@link #format}: the canonical wire string of a stored value. */ + public static String canonicalizeStored(String s) { + return format(fromStored(s)); + } + + /** + * The 64-bit count of microseconds a SQLite / MySQL column stores (sub-microsecond precision + * truncated toward zero). + * + * @throws IllegalArgumentException past the 64-bit range, which is that column's limit + */ + public static long toMicros(Duration d) { + try { + boolean negative = d.isNegative(); + Duration mag = negative ? d.negated() : d; + long micros = Math.addExact(Math.multiplyExact(mag.getSeconds(), 1_000_000L), mag.getNano() / 1_000L); + return negative ? -micros : micros; + } catch (ArithmeticException tooLong) { + throw new IllegalArgumentException( + "field.duration: " + d + " is past a 64-bit count of microseconds", tooLong); + } + } + + /** A stored count of microseconds as a duration. */ + public static Duration fromMicros(long micros) { + return Duration.ofSeconds(Math.floorDiv(micros, 1_000_000L), Math.floorMod(micros, 1_000_000L) * 1_000L); + } + + // ── helpers ── + + private static long num(String digits) { + return digits == null ? 0L : Long.parseLong(digits); + } + + /** A fraction of any length → whole nanoseconds, the digits past the ninth truncated toward zero. */ + private static long nanos(String frac) { + if (frac == null || frac.isEmpty()) return 0L; + String nine = frac.length() >= 9 ? frac.substring(0, 9) : (frac + "000000000").substring(0, 9); + return Long.parseLong(nine); + } + + /** {@code value} as {@code width} zero-padded ASCII digits after a {@code .}, trailing zeros dropped. */ + private static void appendFraction(StringBuilder b, long value, int width) { + if (value == 0) return; + String digits = Long.toString(value); + String f = "000000000".substring(0, width - digits.length()) + digits; + int end = f.length(); + while (end > 0 && f.charAt(end - 1) == '0') end--; + b.append('.').append(f, 0, end); + } + + private static void two(StringBuilder b, long v) { + if (v < 10) b.append('0'); + b.append(v); + } + + private static String quote(String s) { + return "\"" + s + "\""; + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java index f8d86f6fe4..9bae5e39bb 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java +++ b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java @@ -63,6 +63,9 @@ public class EnumField extends PrimitiveField { */ public static final String ATTR_VALUES = "values"; + /** The column length of a string-stored enum. Same name as {@link StringField#ATTR_MAX_LENGTH}. */ + public static final String ATTR_MAX_LENGTH = StringField.ATTR_MAX_LENGTH; + /** * Name of the optional off-vocabulary → canonical-member alias map (properties). * Consumed by FR-010 extract: maps raw input tokens to their canonical enum member. @@ -221,6 +224,13 @@ public static void registerTypes(MetaDataRegistry registry) { .ofType(com.metaobjects.attr.IntMapAttribute.SUBTYPE_INT_MAP) .asSingle(); + // Optional @maxLength — the column length of a string-stored enum (VARCHAR(n)). + // Every member must fit, and it is refused beside @intValueMap; both checked + // post-load in ValidationPhase.validateEnumMaxLength. + def.optionalAttributeWithConstraints(ATTR_MAX_LENGTH) + .ofType(com.metaobjects.attr.IntAttribute.SUBTYPE_INT) + .asSingle(); + // FR-033: @enumDoc and @coerceDefault are re-homed to the // metaobjects-prompt concern provider (reads // spec/metamodel/prompt.json's field.enum extends). diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/FieldTypesMetaDataProvider.java b/server/java/metadata/src/main/java/com/metaobjects/field/FieldTypesMetaDataProvider.java index 97db5f4c5a..10a946d607 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/field/FieldTypesMetaDataProvider.java +++ b/server/java/metadata/src/main/java/com/metaobjects/field/FieldTypesMetaDataProvider.java @@ -68,6 +68,7 @@ public void registerTypes(MetaDataRegistry registry) { UuidField.registerTypes(registry); UriField.registerTypes(registry); InetField.registerTypes(registry); + DurationField.registerTypes(registry); log.debug("Field types registered via provider"); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/ObjectField.java b/server/java/metadata/src/main/java/com/metaobjects/field/ObjectField.java index 2622f9b0cb..fba5bc3a88 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/field/ObjectField.java +++ b/server/java/metadata/src/main/java/com/metaobjects/field/ObjectField.java @@ -39,6 +39,10 @@ public class ObjectField extends MetaField public final static String ATTR_OBJECTREF = MetaObject.ATTR_OBJECT_REF; /** Persistence-side storage shape for owned-object data. Cross-port values: flattened / jsonb / subdocument. */ public final static String ATTR_STORAGE = "storage"; + /** {@code @storage "flattened"}: the nested object's columns expand into the parent table. */ + public final static String STORAGE_FLATTENED = "flattened"; + /** {@code @flattenedColumnPrefix} — see {@link com.metaobjects.database.CoreDBMetaDataProvider#FLATTENED_COLUMN_PREFIX}. */ + public final static String ATTR_FLATTENED_COLUMN_PREFIX = com.metaobjects.database.CoreDBMetaDataProvider.FLATTENED_COLUMN_PREFIX; /** * Register ObjectField type using the standardized registerTypes() pattern. @@ -58,6 +62,11 @@ public static void registerTypes(MetaDataRegistry registry) { // them), so ObjectField inherits both via the snapshot rather // than re-declaring them here. .inheritsFrom(TYPE_FIELD, SUBTYPE_BASE) + // @flattenedColumnPrefix (string, "" allowed) — the flattened-member column + // prefix. Scoped to field.object; description enriched from spec db.json. + .optionalAttributeWithConstraints(ATTR_FLATTENED_COLUMN_PREFIX) + .ofType(com.metaobjects.attr.StringAttribute.SUBTYPE_STRING) + .asSingle() // ADR-0042 — @objectRef is a cross-reference to a real object (the // nested value's shape); the loader's registry-derived validation // resolves it package-locally (dangling target = load error). Fires diff --git a/server/java/metadata/src/main/java/com/metaobjects/identity/PrimaryIdentity.java b/server/java/metadata/src/main/java/com/metaobjects/identity/PrimaryIdentity.java index 9ed59d7cb4..f30c678a7c 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/identity/PrimaryIdentity.java +++ b/server/java/metadata/src/main/java/com/metaobjects/identity/PrimaryIdentity.java @@ -17,6 +17,11 @@ */ public class PrimaryIdentity extends MetaIdentity { + /** The first value of an increment key's sequence. Absent: 1. */ + public static final String ATTR_SEQUENCE_START = "sequenceStart"; + /** The physical name of an increment key's sequence. Absent: the database's default. */ + public static final String ATTR_SEQUENCE_NAME = "sequenceName"; + private static final Logger log = LoggerFactory.getLogger(PrimaryIdentity.class); /** @@ -48,6 +53,11 @@ public static void registerTypes(MetaDataRegistry registry) { // the dedicated validateIdentityFieldsAndGeneration pass already enforces it). def.requiredAttributeWithConstraints(ATTR_FIELDS).ofType(StringAttribute.SUBTYPE_STRING).asArray(); def.optionalAttributeWithConstraints(ATTR_GENERATION).ofType(StringAttribute.SUBTYPE_STRING).withEnum(GENERATION_INCREMENT, GENERATION_UUID, GENERATION_ASSIGNED); + // The sequence an @generation "increment" key draws from: its first value and its + // physical name (spec/metamodel/db.json). Refused on any other generation, post-load + // in ValidationPhase.validateIdentitySequence. + def.optionalAttributeWithConstraints(ATTR_SEQUENCE_START).ofType(com.metaobjects.attr.LongAttribute.SUBTYPE_LONG).asSingle(); + def.optionalAttributeWithConstraints(ATTR_SEQUENCE_NAME).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); // ACCEPTS ANY ATTRIBUTES (for extensibility from service providers) // ADR-0051: NO any-attr wildcard. An undeclared attribute is ERR_UNKNOWN_ATTR in diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java index f09473d809..b1e04a4cb7 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java @@ -747,6 +747,9 @@ public static MetaDataLoader fromDirectory(String name, Path directory, loader.getRoot().setWireFormat(com.metaobjects.config.SourceResolver .discoverWireFormat(List.of(directory.toAbsolutePath().toUri())) .orElse(com.metaobjects.config.WireFormat.DEFAULT)); + loader.getRoot().setEnumDefaults(com.metaobjects.config.SourceResolver + .discoverEnumDefaults(List.of(directory.toAbsolutePath().toUri())) + .orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); } catch (MetaDataLoadingException e) { throw e; } catch (Exception e) { @@ -834,6 +837,8 @@ public static MetaDataLoader fromUris(String name, List uris, LoaderOptions // ADR-0019 Amendment 2: the wire setting from the same file, else the native default. loader.getRoot().setWireFormat(com.metaobjects.config.SourceResolver .discoverWireFormat(uris).orElse(com.metaobjects.config.WireFormat.DEFAULT)); + loader.getRoot().setEnumDefaults(com.metaobjects.config.SourceResolver + .discoverEnumDefaults(uris).orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); } catch (MetaDataLoadingException e) { throw e; } catch (Exception e) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index f4c4b8c615..3b6d30c504 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -26,6 +26,8 @@ import com.metaobjects.field.DateField; import com.metaobjects.field.DecimalField; import com.metaobjects.field.DoubleField; +import com.metaobjects.field.DurationField; +import com.metaobjects.field.DurationWire; import com.metaobjects.field.EnumField; import com.metaobjects.field.FloatField; import com.metaobjects.field.IntegerField; @@ -262,6 +264,9 @@ public static void run(MetaRoot root, MetaDataLoader loader) { // FR-024 B6 — derived-field providability (entity origin fields need a read source). pass(collected, () -> validateDerivedFieldProvidability(root)); pass(collected, () -> validateObjectFieldStorage(root)); + validateEnumMaxLength(root, collected); + validateIdentitySequence(root, collected); + validateCrossFieldValidators(root, collected); pass(collected, () -> validateFieldMap(root)); pass(collected, () -> validateIdentityFieldsAndGeneration(root)); // FR-024 B3/B4a — subtype rules (identity-name, value purity, projection licensing). @@ -290,7 +295,13 @@ public static void run(MetaRoot root, MetaDataLoader loader) { pass(collected, () -> validateSortableHasSupportedSubtype(root)); pass(collected, () -> validateIndexLookupFields(root)); // Two fields of one stored object resolving to the same physical column. - pass(collected, () -> validateColumnNameCollisions(root)); + // Collects every collision (a flattened embedding can collide on several members at + // once); keyed by message like the reporting pass, since they share code + envelope. + pass(collected, () -> { + List collisions = validateColumnNameCollisions(root); + reportingFindings.addAll(collisions); + collected.addAll(collisions); + }); // FR-039 — a retired requirement carries no @implementedBy (refused, not exempted, // so the dangling-ref class is unreachable) and @supersededBy is legal only there. pass(collected, () -> validateRetiredRequirementLinks(root)); @@ -2361,11 +2372,322 @@ private static List junctionReferenceFkFields(MetaObject junction) { // 2. @storage="flattened" + isArray → ERR_STORAGE_FLATTENED_ARRAY // (flattened storage materialises one column-per-field; arrays would // require a side table, which is what @storage="jsonb" is for.) + // 3. @flattenedColumnPrefix without @storage="flattened" → + // ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE (no other storage gives + // the members columns of their own to prefix). // // Only field.object nodes are inspected; @storage on other field subtypes is // already rejected by the constraint phase. // ========================================================================= + // ========================================================================= + // field.enum @maxLength — the column length of a string-stored enum: + // 1. every member of @values fits in it (ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH), the + // longest offender named; + // 2. it is refused beside @intValueMap (ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP). + // Read RESOLVING (ADR-0039), reported on the node that DECLARES one of the three, so an + // enum that only inherits them is not reported twice. Every field.enum in the tree is + // checked, root-level abstract enums included. Mirrors TS validateEnumMaxLength. + // ========================================================================= + + // ========================================================================= + // Cross-field validators (comparison / requiredWhen / presentIff / atLeastOne / exactlyOne), + // which the database enforces as CHECKs: exactlyOne names two or more @fields; + // every field a @condition names is a field of the owning entity; a requiredWhen / + // presentIff @equals gating a field.duration is an ISO-8601 duration; and the @condition can + // become a CHECK guard (see conditionProblem). ERR_BAD_ATTR_VALUE, one per validator, + // every violation collected. Mirrors TS validateCrossFieldValidators. + // ========================================================================= + static void validateCrossFieldValidators(MetaRoot root, List out) { + java.util.Set crossField = java.util.Set.of( + com.metaobjects.validator.ComparisonValidator.SUBTYPE_COMPARISON, + com.metaobjects.validator.RequiredWhenValidator.SUBTYPE_REQUIRED_WHEN, + com.metaobjects.validator.PresentIffValidator.SUBTYPE_PRESENT_IFF, + com.metaobjects.validator.AtLeastOneValidator.SUBTYPE_AT_LEAST_ONE, + com.metaobjects.validator.ExactlyOneValidator.SUBTYPE_EXACTLY_ONE); + for (MetaObject obj : root.getChildren(MetaObject.class, false)) { + java.util.Map> fields = new java.util.LinkedHashMap<>(); + for (MetaField f : obj.getMetaFields()) fields.put(f.getShortName(), f); + for (com.metaobjects.validator.MetaValidator v + : obj.getChildren(com.metaobjects.validator.MetaValidator.class, false)) { + if (!crossField.contains(v.getSubType())) continue; + String label = v.getSubType() + " \"" + obj.getShortName() + "." + v.getShortName() + "\""; + if (com.metaobjects.validator.ExactlyOneValidator.SUBTYPE_EXACTLY_ONE.equals(v.getSubType())) { + Object raw = v.hasMetaAttr(com.metaobjects.validator.ExactlyOneValidator.ATTR_FIELDS) + ? v.getMetaAttr(com.metaobjects.validator.ExactlyOneValidator.ATTR_FIELDS).getValue() : null; + int count = raw instanceof java.util.Collection c ? c.size() : raw == null ? 0 : 1; + if (count < 2) { + out.add(new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + ": validator." + label + " @fields names " + + count + " field" + (count == 1 ? "" : "s") + + "; it needs two or more. For one field, make that field required.", + ErrorCode.ERR_BAD_ATTR_VALUE, v.getSource())); + continue; + } + } + if (com.metaobjects.validator.RequiredWhenValidator.SUBTYPE_REQUIRED_WHEN.equals(v.getSubType()) + || com.metaobjects.validator.PresentIffValidator.SUBTYPE_PRESENT_IFF.equals(v.getSubType())) { + Object whenName = v.hasMetaAttr(com.metaobjects.validator.RequiredWhenValidator.ATTR_WHEN) + ? v.getMetaAttr(com.metaobjects.validator.RequiredWhenValidator.ATTR_WHEN).getValue() : null; + MetaField when = whenName instanceof String w ? fields.get(w) : null; + Object equals = v.hasMetaAttr(com.metaobjects.validator.RequiredWhenValidator.ATTR_EQUALS) + ? v.getMetaAttr(com.metaobjects.validator.RequiredWhenValidator.ATTR_EQUALS).getValue() : null; + if (when != null && DurationField.SUBTYPE_DURATION.equals(when.getSubType()) + && !(equals instanceof String e && DurationWire.ACCEPTED.matcher(e).matches())) { + out.add(new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + ": validator." + label + " @equals " + + conditionJsonLiteral(equals) + " gates field.duration \"" + when.getShortName() + + "\"; it needs " + DURATION_LITERAL_FORM + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, v.getSource())); + continue; + } + } + if (!v.hasMetaAttr(com.metaobjects.validator.MetaValidator.ATTR_CONDITION)) continue; + Object condition = v.getMetaAttr(com.metaobjects.validator.MetaValidator.ATTR_CONDITION).getValue(); + java.util.TreeSet unknown = new java.util.TreeSet<>(); + collectExprFieldRefs(condition, unknown); + unknown.removeAll(fields.keySet()); + if (!unknown.isEmpty()) { + StringBuilder names = new StringBuilder(); + for (String u : unknown) names.append(names.length() == 0 ? "" : ", ").append('"').append(u).append('"'); + out.add(new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + ": validator." + label + " @condition names " + names + + ", which " + (unknown.size() == 1 ? "is not a field" : "are not fields") + + " of \"" + obj.getShortName() + "\".", + ErrorCode.ERR_BAD_ATTR_VALUE, v.getSource())); + continue; + } + String problem = conditionProblem(condition, fields); + if (problem != null) { + out.add(new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + ": validator." + label + " @condition " + problem, + ErrorCode.ERR_BAD_ATTR_VALUE, v.getSource())); + } + } + } + } + + /** What a literal compared with a field.duration must be. */ + private static final String DURATION_LITERAL_FORM = "an ISO-8601 duration (such as PT1H30M; no years or months)"; + + /** Field subtypes whose values a condition compares with a number literal. */ + private static final java.util.Set CONDITION_NUMBER_SUBTYPES = java.util.Set.of( + IntegerField.SUBTYPE_INT, LongField.SUBTYPE_LONG, DoubleField.SUBTYPE_DOUBLE, + FloatField.SUBTYPE_FLOAT, DecimalField.SUBTYPE_DECIMAL, CurrencyField.SUBTYPE_CURRENCY); + + /** + * Why a cross-field validator's {@code @condition} cannot become a CHECK guard, or + * {@code null} when it can. Its names are already known to be fields. The rule, the same + * in every port: + * - it types cleanly (the origin.computed inference) and its root is boolean, since a + * CHECK negates it; + * - a literal compared with a field is of that field's kind (a boolean for field.boolean, + * a number for a numeric field, a string otherwise) and, for field.enum, one of its + * @values, for field.duration, an ISO-8601 duration; + * - no comparison has a null literal: {@code x = NULL} is never true, so use isNull / + * isNotNull. + * Mirrors TS conditionProblem. + */ + private static String conditionProblem(Object condition, java.util.Map> fields) { + ExpressionAttribute.InferResult inferred = ExpressionAttribute.inferExprType( + condition, name -> fields.containsKey(name) ? fields.get(name).getSubType() : null); + if (!inferred.errors.isEmpty()) { + return "is not a valid expression: " + String.join("; ", inferred.errors) + "."; + } + if (!BooleanField.SUBTYPE_BOOLEAN.equals(inferred.type)) { + return "must be a boolean expression (a comparison, a null test, or and / or / not of them), not a field." + + inferred.type + "."; + } + return conditionLiteralProblem(condition, fields); + } + + /** The first comparison in {@code node} whose literal side cannot be compared with its + * other side, as conditionProblem's message tail, or {@code null}. */ + private static String conditionLiteralProblem(Object node, java.util.Map> fields) { + if (!(node instanceof java.util.Map n)) return null; + if (n.get("left") != null && n.get("right") != null) { + Object[][] sides = { { n.get("left"), n.get("right") }, { n.get("right"), n.get("left") } }; + for (Object[] side : sides) { + if (!(side[0] instanceof java.util.Map lit) || !lit.containsKey("value")) continue; + Object value = lit.get("value"); + if (value == null) { + return "compares with null, which is never true: use isNull or isNotNull."; + } + MetaField field = side[1] instanceof java.util.Map ref && ref.get("field") instanceof String f + ? fields.get(f) : null; + if (field == null) continue; + String subType = field.getSubType(); + String kind = BooleanField.SUBTYPE_BOOLEAN.equals(subType) ? "boolean" + : CONDITION_NUMBER_SUBTYPES.contains(subType) ? "number" : "string"; + boolean matches = switch (kind) { + case "boolean" -> value instanceof Boolean; + case "number" -> value instanceof Number; + default -> value instanceof String; + }; + if (!matches) { + return "compares \"" + field.getShortName() + "\" (field." + subType + ") with " + + conditionJsonLiteral(value) + "; it needs a " + kind + "."; + } + if (EnumField.SUBTYPE_ENUM.equals(subType) && field.hasMetaAttr(EnumField.ATTR_VALUES) + && field.getMetaAttr(EnumField.ATTR_VALUES).getValue() instanceof java.util.Collection members) { + List names = new java.util.ArrayList<>(); + for (Object m : members) names.add(String.valueOf(m)); + if (!names.contains(String.valueOf(value))) { + return "compares \"" + field.getShortName() + "\" with " + conditionJsonLiteral(value) + + ", which is not one of its @values (" + String.join(", ", names) + ")."; + } + } + if (DurationField.SUBTYPE_DURATION.equals(subType) && !DurationWire.ACCEPTED.matcher((String) value).matches()) { + return "compares \"" + field.getShortName() + "\" with " + conditionJsonLiteral(value) + + ", which is not " + DURATION_LITERAL_FORM + "."; + } + } + } + for (String k : List.of("left", "right", "arg")) { + String p = conditionLiteralProblem(n.get(k), fields); + if (p != null) return p; + } + if (n.get("args") instanceof java.util.Collection args) { + for (Object a : args) { + String p = conditionLiteralProblem(a, fields); + if (p != null) return p; + } + } + return null; + } + + /** A condition literal as JSON, the way TS JSON.stringify renders it (an integral + * double without a fraction). */ + private static String conditionJsonLiteral(Object value) { + if ((value instanceof Double || value instanceof Float) && !Double.isInfinite(((Number) value).doubleValue()) + && ((Number) value).doubleValue() == Math.rint(((Number) value).doubleValue())) { + return Long.toString(((Number) value).longValue()); + } + return new com.google.gson.GsonBuilder().disableHtmlEscaping().create().toJson(value); + } + + /** The field names an expression tree (ExpressionAttribute's Map form) references. */ + private static void collectExprFieldRefs(Object node, java.util.Set out) { + if (node instanceof java.util.Map m) { + if (m.get("field") instanceof String f) out.add(f); + for (String k : List.of("left", "right", "arg")) collectExprFieldRefs(m.get(k), out); + if (m.get("args") instanceof java.util.Collection args) for (Object a : args) collectExprFieldRefs(a, out); + } + } + + // ========================================================================= + // identity.primary @sequenceStart / @sequenceName describe the sequence an @generation + // "increment" key draws from; on any other generation there is none, so either one is + // refused (ERR_SEQUENCE_WITHOUT_INCREMENT). Read RESOLVING (ADR-0039), reported on the + // identity that declares one of them, every offender collected. Mirrors TS. + // ========================================================================= + + static void validateIdentitySequence(MetaRoot root, List out) { + for (MetaObject obj : root.getChildren(MetaObject.class, false)) { + for (com.metaobjects.identity.PrimaryIdentity id + : obj.getChildren(com.metaobjects.identity.PrimaryIdentity.class, false)) { + List declared = new java.util.ArrayList<>(); + for (String a : List.of(com.metaobjects.identity.PrimaryIdentity.ATTR_SEQUENCE_START, + com.metaobjects.identity.PrimaryIdentity.ATTR_SEQUENCE_NAME)) { + if (id.hasMetaAttr(a, false)) declared.add("@" + a); + } + String generation = id.hasMetaAttr(com.metaobjects.identity.MetaIdentity.ATTR_GENERATION) + ? id.getMetaAttr(com.metaobjects.identity.MetaIdentity.ATTR_GENERATION).getValueAsString() : null; + if (declared.isEmpty()) continue; + if (com.metaobjects.identity.MetaIdentity.GENERATION_INCREMENT.equals(generation)) { + // The key field's own range bounds the start (Postgres refuses the sequence + // otherwise, at apply time): 1 to the largest value of a field.int / field.long. + if (id.hasMetaAttr(com.metaobjects.identity.PrimaryIdentity.ATTR_SEQUENCE_START)) { + Object raw = id.getMetaAttr(com.metaobjects.identity.PrimaryIdentity.ATTR_SEQUENCE_START).getValue(); + if (raw instanceof Number n) { + long start = n.longValue(); + List keys = id.getFields(); + String keyName = keys.isEmpty() ? null : keys.get(0); + com.metaobjects.field.MetaField key = null; + for (com.metaobjects.field.MetaField f : obj.getMetaFields()) { + if (f.getShortName().equals(keyName)) { key = f; break; } + } + Long max = key instanceof com.metaobjects.field.IntegerField ? (long) Integer.MAX_VALUE : null; + if (start < 1 || (max != null && start > max)) { + out.add(new MetaDataException( + ErrorMessageConstants.ERR_SEQUENCE_START_OUT_OF_RANGE + + ": identity.primary \"" + obj.getShortName() + "." + id.getShortName() + + "\" @sequenceStart " + start + " is out of range for its key \"" + keyName + "\"" + + (key != null ? " (field." + key.getSubType() + ")" : "") + + ": it must be at least 1" + (max != null ? " and at most " + max : "") + ".", + ErrorCode.ERR_SEQUENCE_START_OUT_OF_RANGE, id.getSource())); + } + } + } + continue; + } + String shown = generation == null ? "not set" : "\"" + generation + "\""; + String names = String.join(" and ", declared); + out.add(new MetaDataException( + ErrorMessageConstants.ERR_SEQUENCE_WITHOUT_INCREMENT + + ": identity.primary \"" + obj.getShortName() + "." + id.getShortName() + "\" sets " + + names + ", but its @generation is " + shown + "; only an @generation \"increment\"" + + " key draws from a sequence. Set @generation \"increment\", or remove " + names + ".", + ErrorCode.ERR_SEQUENCE_WITHOUT_INCREMENT, id.getSource())); + } + } + } + + /** Every offending enum is reported, as the other ports do, not only the first. */ + static void validateEnumMaxLength(MetaData node, List out) { + for (MetaData child : node.getChildren(MetaData.class, false)) { + if (child instanceof EnumField) { + try { + validateEnumMaxLengthNode((EnumField) child); + } catch (MetaDataException e) { + out.add(e); + } + } else if (!(child instanceof com.metaobjects.field.MetaField)) { + validateEnumMaxLength(child, out); + } + } + } + + private static void validateEnumMaxLengthNode(EnumField field) { + boolean declares = field.hasMetaAttr(EnumField.ATTR_MAX_LENGTH, false) + || field.hasMetaAttr(EnumField.ATTR_VALUES, false) + || field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false); + if (!declares || !field.hasMetaAttr(EnumField.ATTR_MAX_LENGTH)) return; + Object rawLength = field.getMetaAttr(EnumField.ATTR_MAX_LENGTH).getValue(); + if (!(rawLength instanceof Number)) return; + long maxLength = ((Number) rawLength).longValue(); + MetaData owner = field.getParent(); + String label = owner instanceof MetaObject + ? ((MetaObject) owner).getShortName() + "." + field.getShortName() + : field.getShortName(); + if (field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) { + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP + + ": field.enum \"" + label + "\" sets @maxLength " + maxLength + + " and @intValueMap; an @intValueMap enum is stored as an integer, so a column" + + " length does not apply. Remove @maxLength.", + ErrorCode.ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP, field.getSource()); + } + String tooLong = null; + Object raw = field.hasMetaAttr(EnumField.ATTR_VALUES) + ? field.getMetaAttr(EnumField.ATTR_VALUES).getValue() : null; + if (raw instanceof List) { + for (Object o : (List) raw) { + String v = String.valueOf(o); + if (v.length() > maxLength && (tooLong == null || v.length() > tooLong.length())) tooLong = v; + } + } + if (tooLong != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH + + ": field.enum \"" + label + "\" @maxLength " + maxLength + + " is shorter than its member \"" + tooLong + "\" (" + tooLong.length() + + " characters). Raise @maxLength to at least " + tooLong.length() + + ", or shorten the member.", + ErrorCode.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, field.getSource()); + } + } + static void validateObjectFieldStorage(MetaRoot root) { walkObjectFieldStorage(root); } @@ -2400,10 +2722,32 @@ private static void validateObjectFieldStorageNode(MetaData node) { } // ADR-0039: @storage is also an effective property — resolve it. - if (!node.hasMetaAttr(ObjectField.ATTR_STORAGE)) return; + Object storageVal = node.hasMetaAttr(ObjectField.ATTR_STORAGE) + ? node.getMetaAttr(ObjectField.ATTR_STORAGE).getValue() + : null; - Object storageVal = node.getMetaAttr(ObjectField.ATTR_STORAGE).getValue(); - if ("flattened".equals(String.valueOf(storageVal)) && field.isArrayType()) { + // @flattenedColumnPrefix names the columns a flattened value object adds to the owning + // table; under any other storage it would change nothing, so it is refused rather + // than silently ignored. + // Only a field an object owns is checked (as TS and C# do): a root-level abstract + // field.object is a template, and the concrete field that extends it — resolving both + // the prefix and @storage — is the one checked. + MetaData owner = node.getParent(); + if (owner instanceof MetaObject + && node.hasMetaAttr(ObjectField.ATTR_FLATTENED_COLUMN_PREFIX) + && !ObjectField.STORAGE_FLATTENED.equals(storageVal == null ? null : String.valueOf(storageVal))) { + String storageText = storageVal == null ? "not set (jsonb)" : "\"" + storageVal + "\""; + throw new MetaDataException( + ErrorMessageConstants.ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE + + ": field.object \"" + ((MetaObject) owner).getShortName() + "." + field.getName() + + "\" sets @flattenedColumnPrefix, but its @storage is " + storageText + + "; only a @storage \"flattened\" value object puts its members' columns in" + + " the owning table. Set @storage \"flattened\", or remove @flattenedColumnPrefix.", + ErrorCode.ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE, field.getSource()); + } + + if (storageVal == null) return; + if (ObjectField.STORAGE_FLATTENED.equals(String.valueOf(storageVal)) && field.isArrayType()) { throw new MetaDataException( ErrorMessageConstants.ERR_STORAGE_FLATTENED_ARRAY + ": field.object '" + field.getName() @@ -5406,8 +5750,10 @@ private static void walkRetiredRequirementLinks(MetaData node) { *

Only the RESOLVED name is compared, through {@link com.metaobjects.database.ColumnNaming#resolve}, * so an explicit {@code @column}, an inherited field and the project's column strategy are * honoured. Scope: a concrete object declaring at least one source (a source-less object - * generates nothing); derived fields and {@code field.object} (may flatten to several - * columns) are skipped. ADR-0039: resolving accessors, so an inherited field collides like + * generates nothing); derived fields are skipped. A {@code field.object @storage "flattened"} + * has no column of its own: each member of its value object is a column, named by + * {@link com.metaobjects.database.ColumnNaming#resolveFlattened} exactly as migrate creates + * it. Any other {@code field.object} is one jsonb column under its own resolved name. ADR-0039: resolving accessors, so an inherited field collides like * an own one.

* *

Exempt: an object whose every source is read-only and at least one is a @@ -5432,28 +5778,76 @@ private static boolean hasExplicitColumn(MetaField field) { return v != null && !v.isEmpty(); } - static void validateColumnNameCollisions(MetaRoot root) { + /** One physical column a field contributes to its object's table. */ + private static final class PhysicalColumn { + final String column; + /** {@code street}, or {@code shipTo.street} for a member of a flattened value object. */ + final String label; + /** The field that names the column: the field itself, or the flattened value object's member. */ + final MetaField named; + final boolean flattenedMember; + + PhysicalColumn(String column, String label, MetaField named, boolean flattenedMember) { + this.column = column; + this.label = label; + this.named = named; + this.flattenedMember = flattenedMember; + } + } + + private static List physicalColumnsOf(MetaField field) { + // ADR-0039: resolving - @storage and @objectRef may be inherited via extends. + if (ObjectField.SUBTYPE_OBJECT.equals(field.getSubType()) + && field.hasMetaAttr(ObjectField.ATTR_STORAGE) + && ObjectField.STORAGE_FLATTENED.equals( + String.valueOf(field.getMetaAttr(ObjectField.ATTR_STORAGE).getValue()))) { + MetaObject target = field.hasMetaAttr(ObjectField.ATTR_OBJECTREF) + ? References.refTarget(field, ObjectField.ATTR_OBJECTREF) : null; + List out = new ArrayList<>(); + if (target == null) return out; + // Every member, as migrate expands it. + for (MetaField member : target.getChildren(MetaField.class, true)) { + out.add(new PhysicalColumn( + com.metaobjects.database.ColumnNaming.resolveFlattened(field, member), + field.getName() + "." + member.getName(), member, true)); + } + return out; + } + return List.of(new PhysicalColumn( + com.metaobjects.database.ColumnNaming.resolve(field), field.getName(), field, false)); + } + + private static String collisionHint(PhysicalColumn a, PhysicalColumn b) { + if (a.flattenedMember || b.flattenedMember) { + return "Give one flattened field a different @flattenedColumnPrefix, or rename one field."; + } + return hasExplicitColumn(a.named) && hasExplicitColumn(b.named) + ? "Rename one field, or point one @column at a different column." + : "Rename one field, or give one an explicit @column so the two columns differ."; + } + + static List validateColumnNameCollisions(MetaRoot root) { + List errors = new ArrayList<>(); for (MetaObject obj : root.getChildren(MetaObject.class, false)) { if (isAbstract(obj)) continue; if (obj.getChildren(MetaSource.class, true).isEmpty()) continue; if (readsThroughSqlView(obj)) continue; - java.util.Map> seen = new java.util.HashMap<>(); + java.util.Map seen = new java.util.HashMap<>(); for (MetaField field : obj.getChildren(MetaField.class, true)) { - if (ObjectField.SUBTYPE_OBJECT.equals(field.getSubType()) || field.isDerived()) continue; - String column = com.metaobjects.database.ColumnNaming.resolve(field); - MetaField first = seen.putIfAbsent(column, field); - if (first == null) continue; - throw new MetaDataException( - ErrorMessageConstants.ERR_COLUMN_NAME_COLLISION - + ": fields \"" + first.getName() + "\" and \"" + field.getName() - + "\" on \"" + obj.getName() + "\" both resolve to physical column \"" - + column + "\". " - + (hasExplicitColumn(first) && hasExplicitColumn(field) - ? "Rename one field, or point one @column at a different column." - : "Rename one field, or give one an explicit @column so the two columns differ."), - ErrorCode.ERR_COLUMN_NAME_COLLISION, field.getSource()); + if (field.isDerived()) continue; + for (PhysicalColumn col : physicalColumnsOf(field)) { + PhysicalColumn first = seen.putIfAbsent(col.column, col); + if (first == null) continue; + errors.add(new MetaDataException( + ErrorMessageConstants.ERR_COLUMN_NAME_COLLISION + + ": fields \"" + first.label + "\" and \"" + col.label + + "\" on \"" + obj.getName() + "\" both resolve to physical column \"" + + col.column + "\". " + collisionHint(first, col), + ErrorCode.ERR_COLUMN_NAME_COLLISION, field.getSource())); + } } } + return errors; } static void validateIndexLookupFields(MetaRoot root) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java b/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java index 93873239a9..54f9d54c66 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java @@ -5,6 +5,7 @@ import com.google.gson.JsonObject; import com.google.gson.JsonParser; import com.google.gson.JsonPrimitive; +import com.metaobjects.database.CoreDBMetaDataProvider; import com.metaobjects.ErrorCode; import com.metaobjects.MetaData; import com.metaobjects.MetaDataException; @@ -1047,8 +1048,9 @@ private void detectAttrMergeConflicts(MetaData target, JsonObject body, String existingVal = preAttrs.get(attrName); if (existingVal == null) continue; - if (isEmptyJsonValue(entry.getValue())) continue; - if (existingVal.isEmpty()) continue; + boolean emptyIsAValue = EMPTY_STRING_IS_A_VALUE.contains(attrName); + if (isEmptyJsonValue(entry.getValue(), emptyIsAValue)) continue; + if (existingVal.isEmpty() && !emptyIsAValue) continue; // Normalize the new JSON value to the same comma-delimited form // that {@link MetaAttribute#getValueAsString()} returns for arrays, @@ -1057,7 +1059,7 @@ private void detectAttrMergeConflicts(MetaData target, JsonObject body, // parser's {@code convertJsonArrayToCommaDelimited} treatment of // array-shaped inline attrs. String newVal = normalizeAttrValueForCompare(entry.getValue()); - if (newVal == null || newVal.isEmpty()) continue; + if (newVal == null || (newVal.isEmpty() && !emptyIsAValue)) continue; if (existingVal.equals(newVal)) continue; // Real conflict — emit ERR_MERGE_CONFLICT with merged envelope. @@ -1180,11 +1182,17 @@ private String normalizeAttrValueForCompare(JsonElement el) { return raw; } - /** FR5c — empty-value check matching the TS reference. */ - private static boolean isEmptyJsonValue(JsonElement el) { + /** Attrs whose empty string is a meaningful value rather than "not set" (FR5c). */ + private static final java.util.Set EMPTY_STRING_IS_A_VALUE = + java.util.Set.of(CoreDBMetaDataProvider.FLATTENED_COLUMN_PREFIX); + + /** FR5c — empty-value check matching the TS reference. An attr in + * {@link #EMPTY_STRING_IS_A_VALUE} treats {@code ""} as a value, so {@code ""} beside + * {@code "home_"} is a conflict rather than last-loaded-wins. */ + private static boolean isEmptyJsonValue(JsonElement el, boolean emptyStringIsAValue) { if (el == null || el.isJsonNull()) return true; if (el.isJsonArray() && el.getAsJsonArray().size() == 0) return true; - if (el.isJsonPrimitive() && el.getAsJsonPrimitive().isString() + if (!emptyStringIsAValue && el.isJsonPrimitive() && el.getAsJsonPrimitive().isString() && el.getAsString().isEmpty()) return true; return false; } diff --git a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java index e970857173..09f77297e8 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java +++ b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java @@ -30,6 +30,7 @@ import com.metaobjects.field.UuidField; import com.metaobjects.field.UriField; import com.metaobjects.field.InetField; +import com.metaobjects.field.DurationField; import java.util.Collections; import java.util.LinkedHashMap; @@ -126,6 +127,8 @@ private static Set ordered(String... ops) { m.put(CurrencyField.SUBTYPE_CURRENCY, OPS_NUMERIC); m.put(DateField.SUBTYPE_DATE, OPS_NUMERIC); m.put(TimeField.SUBTYPE_TIME, OPS_NUMERIC); + // field.duration orders on every DB: a native interval on Postgres, a microsecond count elsewhere. + m.put(DurationField.SUBTYPE_DURATION, OPS_NUMERIC); m.put(TimestampField.SUBTYPE_TIMESTAMP, OPS_NUMERIC); m.put(BooleanField.SUBTYPE_BOOLEAN, OPS_BOOLEAN); OPS_BY_SUBTYPE = Collections.unmodifiableMap(m); diff --git a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java index 89c236a088..167bbef143 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java +++ b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java @@ -297,7 +297,7 @@ public static List metamodelProviders() { * the 1.0 cut will flip it to {@code "1.0"}. Mirrors the TS reference's * {@code METAMODEL_VERSION} constant. */ - public static final String METAMODEL_VERSION = "1.2"; + public static final String METAMODEL_VERSION = "1.3"; // ------------------------------------------------------------------ // SP-G Phase1 Units2-3 — manifest emitter exclusions (documented, uniform diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 41402721ff..ca76773a5a 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -168,6 +168,42 @@ private ErrorMessageConstants() { */ public static final String ERR_STORAGE_FLATTENED_ARRAY = "ERR_STORAGE_FLATTENED_ARRAY"; + /** + * Error code emitted when {@code @flattenedColumnPrefix} is set on a {@code field.object} + * whose {@code @storage} is not {@code "flattened"} (absent is jsonb): no other storage + * gives the members columns of their own to prefix. + * Cross-language contract: {@code ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE}. + */ + public static final String ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE = + "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE"; + + /** + * A string-stored {@code field.enum}'s {@code @maxLength} (its column length) is shorter + * than one of its {@code @values} members, or the project's {@code enums.maxLength} is. + * Cross-language contract: {@code ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH}. + */ + public static final String ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH = "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH"; + + /** + * A {@code field.enum} sets both {@code @maxLength} and {@code @intValueMap}: an int-stored + * enum's column is an integer, so a length would change nothing. + * Cross-language contract: {@code ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP}. + */ + public static final String ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP = "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP"; + + /** + * An {@code identity.primary} sets {@code @sequenceStart} or {@code @sequenceName} without + * {@code @generation "increment"}: only an increment key draws from a sequence. + * Cross-language contract: {@code ERR_SEQUENCE_WITHOUT_INCREMENT}. + */ + public static final String ERR_SEQUENCE_WITHOUT_INCREMENT = "ERR_SEQUENCE_WITHOUT_INCREMENT"; + + /** + * An {@code identity.primary} {@code @sequenceStart} is below 1, or past the largest value its + * key field can hold. Cross-language contract: {@code ERR_SEQUENCE_START_OUT_OF_RANGE}. + */ + public static final String ERR_SEQUENCE_START_OUT_OF_RANGE = "ERR_SEQUENCE_START_OUT_OF_RANGE"; + /** * Error code emitted when {@code field.object @storage} is set on a field that * has no {@code @objectRef} — storage shape only applies to referenced objects. diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/AtLeastOneValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/AtLeastOneValidator.java index ea7ceea0c4..be5aaae941 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/AtLeastOneValidator.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/AtLeastOneValidator.java @@ -42,6 +42,8 @@ public static void registerTypes(MetaDataRegistry registry) { .inheritsFrom(TYPE_VALIDATOR, SUBTYPE_BASE); def.requiredAttributeWithConstraints(ATTR_FIELDS).ofType(StringAttribute.SUBTYPE_STRING).asArray(); + def.optionalAttributeWithConstraints(ATTR_CONDITION) + .ofType(com.metaobjects.attr.ExpressionAttribute.SUBTYPE_EXPRESSION).asSingle(); }); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/ComparisonValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/ComparisonValidator.java index 82ddc56898..fee1a184c4 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/ComparisonValidator.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/ComparisonValidator.java @@ -50,6 +50,8 @@ public static void registerTypes(MetaDataRegistry registry) { def.requiredAttributeWithConstraints(ATTR_OP).ofType(StringAttribute.SUBTYPE_STRING) .withEnum("gt", "gte", "lt", "lte", "ne", "eq"); def.requiredAttributeWithConstraints(ATTR_RIGHT).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); + def.optionalAttributeWithConstraints(ATTR_CONDITION) + .ofType(com.metaobjects.attr.ExpressionAttribute.SUBTYPE_EXPRESSION).asSingle(); }); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/ExactlyOneValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/ExactlyOneValidator.java new file mode 100644 index 0000000000..09243591de --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/ExactlyOneValidator.java @@ -0,0 +1,57 @@ +/* + * Copyright 2003 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.validator; + +import com.metaobjects.attr.StringAttribute; +import com.metaobjects.registry.MetaDataRegistry; + +import static com.metaobjects.validator.MetaValidator.TYPE_VALIDATOR; +import static com.metaobjects.validator.MetaValidator.SUBTYPE_BASE; + +/** + * Cardinality of presence — exactly one of the named fields ({@code @fields}) + * is present (NOT NULL) and the others are null; it needs two or more fields. Entity-scoped; references fields by name (the same + * {@code @fields}-by-name pattern as {@code identity.*}). DB-enforced, so + * {@link #validate} is a no-op in the JVM. + */ +@SuppressWarnings("serial") +public class ExactlyOneValidator extends MetaValidator { + + public final static String SUBTYPE_EXACTLY_ONE = "exactlyOne"; + + /** Alternative field names; exactly one must be present. */ + public final static String ATTR_FIELDS = "fields"; + + public static void registerTypes(MetaDataRegistry registry) { + registry.registerType(ExactlyOneValidator.class, def -> { + def.type(TYPE_VALIDATOR).subType(SUBTYPE_EXACTLY_ONE) + .description("Presence cardinality — exactly one of @fields must be present") + .inheritsFrom(TYPE_VALIDATOR, SUBTYPE_BASE); + + def.requiredAttributeWithConstraints(ATTR_FIELDS).ofType(StringAttribute.SUBTYPE_STRING).asArray(); + def.optionalAttributeWithConstraints(ATTR_CONDITION) + .ofType(com.metaobjects.attr.ExpressionAttribute.SUBTYPE_EXPRESSION).asSingle(); + }); + } + + public ExactlyOneValidator(String name) { + super(SUBTYPE_EXACTLY_ONE, name); + } + + /** DB-enforced (CONSTRAINT CHECK); no JVM-side runtime validation. */ + public void validate(Object object, Object value) { + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/MetaValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/MetaValidator.java index 656b463353..c0ba1630c3 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/MetaValidator.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/MetaValidator.java @@ -43,6 +43,12 @@ public abstract class MetaValidator extends MetaData { /** Cross-port maximum-bound attribute ({@code @max}). */ public final static String ATTR_MAX = "max"; + /** A cross-field validator's @condition: the rule applies only where this expression holds. */ + public final static String ATTR_CONDITION = "condition"; + + /** The database name of the CHECK a CHECK-generating validator (or a field.enum's membership) generates. */ + public final static String ATTR_CONSTRAINT_NAME = "constraintName"; + /** * Register this type with the MetaDataRegistry (called by provider) * @param registry the MetaDataRegistry to register with diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/PresentIffValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/PresentIffValidator.java index 198bf92574..49f4cd049f 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/PresentIffValidator.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/PresentIffValidator.java @@ -50,6 +50,8 @@ public static void registerTypes(MetaDataRegistry registry) { def.requiredAttributeWithConstraints(ATTR_FIELD).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.requiredAttributeWithConstraints(ATTR_WHEN).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.requiredAttributeWithConstraints(ATTR_EQUALS).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); + def.optionalAttributeWithConstraints(ATTR_CONDITION) + .ofType(com.metaobjects.attr.ExpressionAttribute.SUBTYPE_EXPRESSION).asSingle(); }); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/RequiredWhenValidator.java b/server/java/metadata/src/main/java/com/metaobjects/validator/RequiredWhenValidator.java index c4b718d2f9..689ed22cda 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/RequiredWhenValidator.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/RequiredWhenValidator.java @@ -49,6 +49,8 @@ public static void registerTypes(MetaDataRegistry registry) { def.requiredAttributeWithConstraints(ATTR_FIELD).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.requiredAttributeWithConstraints(ATTR_WHEN).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.requiredAttributeWithConstraints(ATTR_EQUALS).ofType(StringAttribute.SUBTYPE_STRING).asSingle(); + def.optionalAttributeWithConstraints(ATTR_CONDITION) + .ofType(com.metaobjects.attr.ExpressionAttribute.SUBTYPE_EXPRESSION).asSingle(); }); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/validator/ValidatorTypesMetaDataProvider.java b/server/java/metadata/src/main/java/com/metaobjects/validator/ValidatorTypesMetaDataProvider.java index a353fc6a7d..572c49b5d7 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/validator/ValidatorTypesMetaDataProvider.java +++ b/server/java/metadata/src/main/java/com/metaobjects/validator/ValidatorTypesMetaDataProvider.java @@ -46,6 +46,7 @@ public void registerTypes(MetaDataRegistry registry) { RequiredWhenValidator.registerTypes(registry); PresentIffValidator.registerTypes(registry); AtLeastOneValidator.registerTypes(registry); + ExactlyOneValidator.registerTypes(registry); log.debug("Validator types registered via provider"); } diff --git a/server/java/metadata/src/test/java/com/metaobjects/config/WireConformanceTest.java b/server/java/metadata/src/test/java/com/metaobjects/config/WireConformanceTest.java index 9333e30ae9..ec4649f3dd 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/config/WireConformanceTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/config/WireConformanceTest.java @@ -63,9 +63,10 @@ public void everyValidBlockResolvesToItsValues() throws IOException { WireFormat wire = parse(c.get("wire"), "case:" + name); assertEquals(name + ": decimals", c.get("decimals").getAsString(), wire.decimals().wireName()); assertEquals(name + ": nullMembers", c.get("nullMembers").getAsString(), wire.nullMembers().wireName()); + assertEquals(name + ": durations", c.get("durations").getAsString(), wire.durations().wireName()); n++; } - assertEquals(6, n); + assertEquals(7, n); } @Test @@ -77,6 +78,6 @@ public void everyInvalidBlockIsRefused() throws IOException { assertThrows(name, MetaDataException.class, () -> parse(c.get("wire"), "invalid-case:" + name)); n++; } - assertEquals(8, n); + assertEquals(10, n); } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/database/ColumnNamingTest.java b/server/java/metadata/src/test/java/com/metaobjects/database/ColumnNamingTest.java index 49807ae50d..395e1d9866 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/database/ColumnNamingTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/database/ColumnNamingTest.java @@ -68,7 +68,7 @@ public void snakeCaseMatchesTheCrossPortAlgorithm() { public void anUnknownStrategyIsRefusedNotSilentlyDefaulted() { // A typo would otherwise bind a whole schema to the wrong columns and succeed. IllegalArgumentException ex = assertThrows(IllegalArgumentException.class, - () -> ColumnNaming.apply("createdAt", "PascalCase")); + () -> ColumnNaming.apply("createdAt", "camelCase")); org.junit.Assert.assertTrue(ex.getMessage(), ex.getMessage().contains("snake_case")); } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/database/EnumLengthConformanceTest.java b/server/java/metadata/src/test/java/com/metaobjects/database/EnumLengthConformanceTest.java new file mode 100644 index 0000000000..ae4386fc07 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/database/EnumLengthConformanceTest.java @@ -0,0 +1,112 @@ +package com.metaobjects.database; + +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import com.metaobjects.ErrorCode; +import com.metaobjects.MetaDataException; +import com.metaobjects.config.EnumDefaults; +import com.metaobjects.config.NeutralConfig; +import com.metaobjects.field.MetaField; +import com.metaobjects.loader.FileSource; +import com.metaobjects.loader.LoaderOptions; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.object.MetaObject; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.Test; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.List; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * The shared cross-port enum-length corpus ({@code fixtures/enum-length-conformance/}) run + * against the Java port: each case's {@code enums} block parsed with {@link NeutralConfig} + * (the reader every project-aware load uses), put on the loaded {@code meta.enums.json} root, + * and every Ticket enum's column length resolved through {@link EnumLength#resolveMaxLength}. + * Each {@code overflow} case is refused with {@code ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH}, and + * each {@code invalid} block when the config is read. + */ +public class EnumLengthConformanceTest extends SharedRegistryTestBase { + + private static Path corpusDir() { + Path dir = Paths.get("").toAbsolutePath(); + while (dir != null && !Files.isDirectory(dir.resolve("fixtures"))) dir = dir.getParent(); + if (dir == null) throw new IllegalStateException("could not locate the repository fixtures/ directory"); + return dir.resolve("fixtures/enum-length-conformance"); + } + + private static JsonObject corpus() throws IOException { + return JsonParser.parseString(new String( + Files.readAllBytes(corpusDir().resolve("cases.json")), StandardCharsets.UTF_8)).getAsJsonObject(); + } + + private static EnumDefaults parseEnums(JsonElement enums) { + JsonObject doc = new JsonObject(); + doc.addProperty("schema_version", 1); + if (enums != null && !enums.isJsonNull()) doc.add("enums", enums); + return NeutralConfig.parse(doc.toString(), "enum-length-corpus").getEnumDefaults(); + } + + private MetaField ticketField(EnumDefaults defaults, String name) throws IOException { + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "enum_length_conformance"); + loader.init(); + loader.load(List.of(new FileSource(corpusDir().resolve("meta.enums.json")))); + if (!loader.getErrors().isEmpty()) fail("expected a clean load, got: " + loader.getErrors()); + loader.getRoot().setEnumDefaults(defaults); + for (MetaObject obj : loader.getMetaObjects()) { + if (obj.getShortName().equals("Ticket")) return obj.getMetaField(name); + } + throw new AssertionError("no Ticket in the corpus model"); + } + + @Test + public void resolvesEveryEnumColumnLength() throws IOException { + for (JsonElement el : corpus().getAsJsonArray("cases")) { + JsonObject c = el.getAsJsonObject(); + EnumDefaults defaults = parseEnums(c.get("enums")); + for (Map.Entry e : c.getAsJsonObject("lengths").entrySet()) { + Long expected = e.getValue().isJsonNull() ? null : e.getValue().getAsLong(); + assertEquals(c.get("name").getAsString() + ": " + e.getKey(), + expected, EnumLength.resolveMaxLength(ticketField(defaults, e.getKey()))); + } + } + } + + @Test + public void refusesAMemberLongerThanTheProjectDefault() throws IOException { + for (JsonElement el : corpus().getAsJsonArray("overflow")) { + JsonObject o = el.getAsJsonObject(); + MetaField field = ticketField(parseEnums(o.get("enums")), o.get("field").getAsString()); + try { + EnumLength.resolveMaxLength(field); + fail(o.get("name").getAsString() + ": expected a refusal"); + } catch (MetaDataException e) { + assertEquals(java.util.Optional.of(ErrorCode.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH), e.getCode()); + assertTrue(e.getMessage(), e.getMessage().contains(o.get("member").getAsString())); + } + } + } + + @Test + public void refusesAnInvalidEnumsBlock() throws IOException { + for (JsonElement el : corpus().getAsJsonArray("invalid")) { + JsonObject bad = el.getAsJsonObject(); + try { + parseEnums(bad.get("enums")); + fail(bad.get("name").getAsString() + ": expected the config to be refused"); + } catch (MetaDataException expected) { + // refused when read + } + } + } +} diff --git a/server/java/metadata/src/test/java/com/metaobjects/database/PhysicalNamingConformanceTest.java b/server/java/metadata/src/test/java/com/metaobjects/database/PhysicalNamingConformanceTest.java index e66269d11f..6087ea55ac 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/database/PhysicalNamingConformanceTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/database/PhysicalNamingConformanceTest.java @@ -22,6 +22,7 @@ import com.metaobjects.MetaDataException; import com.metaobjects.config.NeutralConfig; import com.metaobjects.field.MetaField; +import com.metaobjects.field.ObjectField; import com.metaobjects.loader.LoaderOptions; import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.object.MetaObject; @@ -164,10 +165,18 @@ public void resolvesEveryTableAndColumnUnderThisNamingRule() throws IOException } for (Map.Entry e : testCase.columns().entrySet()) { - String[] parts = e.getKey().split("\\.", 2); + String[] parts = e.getKey().split("\\.", 3); MetaObject obj = findObject(loader, parts[0]); MetaField field = obj.getMetaField(parts[1]); - String actual = ColumnNaming.resolve(field); + String actual; + if (parts.length == 2) { + actual = ColumnNaming.resolve(field); + } else { + // `..`: a member of the flattened value object the + // field embeds, named by core naming. + MetaField member = ((ObjectField) field).getObjectRef().getMetaField(parts[2]); + actual = ColumnNaming.resolveFlattened(field, member); + } assertEquals(caseName + ": column name for " + e.getKey(), e.getValue(), actual); } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/field/DurationWireVectorsTest.java b/server/java/metadata/src/test/java/com/metaobjects/field/DurationWireVectorsTest.java new file mode 100644 index 0000000000..d4c5236fee --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/field/DurationWireVectorsTest.java @@ -0,0 +1,179 @@ +package com.metaobjects.field; + +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import org.junit.Test; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.time.temporal.ChronoUnit; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * The shared {@code field.duration} vectors ({@code fixtures/duration-wire/vectors.json}) against + * the production helper, {@link DurationWire}: every {@code wire} group, at this port's precision + * ({@code "9"}: a {@link Duration} holds nanoseconds), and the {@code codecs} groups OMDB uses in + * production — {@code JdbcCodecs.DurationCodec} reads a Postgres interval as its text + * ({@code postgres}, {@code postgresRefused}), {@code PostgresDriver} binds the interval spelling + * ({@code postgresText}), and SQLite / MySQL store a microsecond count ({@code storedMicros}). + */ +public class DurationWireVectorsTest { + + /** This port's key in {@code wire.precision[].out}: java.time.Duration holds nine fractional digits. */ + private static final String PRECISION = "9"; + + private static JsonObject file() throws Exception { + Path dir = Path.of(System.getProperty("user.dir")).toAbsolutePath(); + while (dir != null && !Files.isDirectory(dir.resolve("fixtures/duration-wire"))) dir = dir.getParent(); + if (dir == null) throw new IllegalStateException("fixtures/duration-wire not found above " + System.getProperty("user.dir")); + return JsonParser.parseString(Files.readString(dir.resolve("fixtures/duration-wire/vectors.json"))).getAsJsonObject(); + } + + private static JsonObject wire() throws Exception { + return file().getAsJsonObject("wire"); + } + + private static JsonObject codecs() throws Exception { + return file().getAsJsonObject("codecs"); + } + + @Test + public void parseYieldsTheCanonicalForm() throws Exception { + for (JsonElement e : wire().getAsJsonArray("parse")) { + String in = e.getAsJsonObject().get("in").getAsString(); + String out = e.getAsJsonObject().get("out").getAsString(); + assertTrue("accepted: " + in, DurationWire.isDuration(in)); + assertEquals("parse " + in, out, DurationWire.canonicalize(in)); + // The canonical form is a fixed point. + assertEquals("re-parse " + out, out, DurationWire.canonicalize(out)); + } + } + + @Test + public void precisionKeepsNineDigitsAndTruncatesTheRestTowardZero() throws Exception { + for (JsonElement e : wire().getAsJsonArray("precision")) { + String in = e.getAsJsonObject().get("in").getAsString(); + String out = e.getAsJsonObject().getAsJsonObject("out").get(PRECISION).getAsString(); + assertTrue("accepted: " + in, DurationWire.isDuration(in)); + assertEquals("precision " + in, out, DurationWire.canonicalize(in)); + assertEquals("re-parse " + out, out, DurationWire.canonicalize(out)); + } + } + + @Test + public void refusedInputIsRejected() throws Exception { + for (JsonElement e : wire().getAsJsonArray("refused")) { + String in = e.getAsJsonPrimitive().getAsString(); + assertFalse("refused: '" + in + "'", DurationWire.isDuration(in)); + try { + DurationWire.parse(in); + fail("parse must refuse '" + in + "'"); + } catch (IllegalArgumentException expected) { + // refused + } + } + } + + @Test + public void postgresTextReadsAsTheCanonicalForm() throws Exception { + for (JsonElement e : codecs().getAsJsonArray("postgres")) { + String in = e.getAsJsonObject().get("in").getAsString(); + String out = e.getAsJsonObject().get("out").getAsString(); + assertEquals("postgres " + in, out, DurationWire.format(DurationWire.parsePostgres(in))); + assertEquals("stored " + in, out, DurationWire.canonicalizeStored(in)); + } + } + + @Test + public void postgresCalendarIntervalsAreRefused() throws Exception { + for (JsonElement e : codecs().getAsJsonArray("postgresRefused")) { + String in = e.getAsJsonPrimitive().getAsString(); + try { + DurationWire.fromStored(in); + fail("must refuse the interval '" + in + "'"); + } catch (IllegalArgumentException expected) { + // refused + } + } + } + + @Test + public void postgresTextSpelling() throws Exception { + for (JsonElement e : codecs().getAsJsonArray("postgresText")) { + String in = e.getAsJsonObject().get("in").getAsString(); + String out = e.getAsJsonObject().get("out").getAsString(); + assertEquals("postgresText " + in, out, DurationWire.formatPostgres(DurationWire.parse(in))); + // An interval written that way reads back as the canonical value. + assertEquals("read back " + out, in, DurationWire.format(DurationWire.parsePostgres(out))); + } + } + + @Test + public void formatKeepsNanosAndTheStoredFormsTruncateToMicros() { + assertEquals("PT1.123456789S", DurationWire.format(Duration.ofSeconds(1, 123_456_789))); + assertEquals("-PT1.123456789S", DurationWire.format(Duration.ofSeconds(1, 123_456_789).negated())); + assertEquals("-PT0.000000999S", DurationWire.format(Duration.ofNanos(-999))); + assertEquals("-PT1M30S", DurationWire.format(Duration.ofSeconds(-90))); + // A Postgres interval holds microseconds: the interval spelling truncates toward zero. + assertEquals("00:00:01.123456", DurationWire.formatPostgres(Duration.ofSeconds(1, 123_456_789))); + assertEquals("-00:00:01.123456", DurationWire.formatPostgres(Duration.ofSeconds(1, 123_456_789).negated())); + assertEquals("00:00:00", DurationWire.formatPostgres(Duration.ofNanos(-999))); + } + + @Test + public void formattingIgnoresTheDefaultLocalesDigits() { + java.util.Locale saved = java.util.Locale.getDefault(); + try { + java.util.Locale.setDefault(java.util.Locale.forLanguageTag("ar-EG-u-nu-arab")); + assertEquals("-PT1M30.000005S", DurationWire.format(Duration.ofSeconds(-90, -5_000))); + assertEquals("-00:01:30.000005", DurationWire.formatPostgres(Duration.ofSeconds(-90, -5_000))); + } finally { + java.util.Locale.setDefault(saved); + } + } + + @Test + public void storedMicrosVectors() throws Exception { + for (JsonElement e : codecs().getAsJsonArray("storedMicros")) { + String in = e.getAsJsonObject().get("in").getAsString(); + long out = Long.parseLong(e.getAsJsonObject().get("out").getAsString()); + Duration d = DurationWire.parse(in); + assertEquals("micros " + in, out, DurationWire.toMicros(d)); + // The stored count reads back as the value truncated to microseconds, toward zero. + assertEquals("from micros " + out, DurationWire.format(d.truncatedTo(ChronoUnit.MICROS)), + DurationWire.format(DurationWire.fromMicros(out))); + } + // Sub-microsecond precision truncates toward zero on both sides of zero. + assertEquals(-1_000_000L, DurationWire.toMicros(Duration.ofSeconds(-1, -500))); + assertEquals(1_000_000L, DurationWire.toMicros(Duration.ofSeconds(1, 500))); + } + + @Test + public void javaDurationsOwnRangeIsTheOnlyLimitAndIsRefusedNeverWrapped() { + // Far past any other port's range: Java takes it. + assertEquals("PT2562047788015215H", DurationWire.canonicalize("PT2562047788015215H")); + // Past a long count of seconds: refused, never wrapped. + for (String tooLong : new String[]{"PT9223372036854775808S", "P106751991167301D", "PT99999999999999999999H"}) { + assertFalse(tooLong, DurationWire.isDuration(tooLong)); + try { + DurationWire.parse(tooLong); + fail("must refuse " + tooLong); + } catch (IllegalArgumentException expected) { + // refused + } + } + // A BIGINT column holds about 292,000 years of microseconds: past it, the write is refused. + try { + DurationWire.toMicros(Duration.ofSeconds(Long.MAX_VALUE / 1_000_000L + 1)); + fail("past a 64-bit microsecond count must be refused"); + } catch (IllegalArgumentException expected) { + // refused + } + } +} diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java index cab32aa873..d42ee883a2 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java @@ -44,6 +44,8 @@ import com.metaobjects.field.UuidField; import com.metaobjects.field.UriField; import com.metaobjects.field.InetField; +import com.metaobjects.field.DurationField; +import com.metaobjects.field.DurationWire; import java.math.BigDecimal; import java.sql.PreparedStatement; @@ -104,6 +106,9 @@ public static JdbcFieldCodec forField(MetaField f) { // type (a plain setString fails "operator does not exist: inet = varchar"), mirroring // the UuidCodec native-column pattern. register(InetField.class, new InetCodec()); + // field.duration ⇄ a microsecond count on an integer column; the Postgres driver binds the + // interval spelling to its native interval column (see DurationCodec). + register(DurationField.class, new DurationCodec()); register(ObjectField.class, DEFAULT); // identity with the fallback codec } @@ -541,6 +546,59 @@ static final class InetCodec implements JdbcFieldCodec { } } + /** + * {@code field.duration} ⇄ a 64-bit integer column holding the count of microseconds (SQLite, + * MySQL and any other engine without a native interval — Django's DurationField precedent), or + * — on Postgres — the native {@code interval} column. DurationField is backed by + * {@code DataTypes.STRING} (the canonical wire string); a {@link java.time.Duration} value is + * accepted too. + * + *

Read: an integer is a microsecond count; text is what a Postgres interval returns + * ({@code 1 day 02:00:00}) or a stored ISO string; all become the canonical string, and an + * interval with a year or month part is refused ({@link DurationWire#fromStored}). Write: the + * microsecond count (sub-microsecond truncated toward zero; past a 64-bit count is refused), so + * the column orders and compares as a number. Postgres refuses a signed ISO-8601 duration, so + * {@code PostgresDriver} binds {@link #postgresText} instead.

+ */ + public static final class DurationCodec implements JdbcFieldCodec { + @Override public int sqlType() { return Types.BIGINT; } + @Override public int sqlLength() { return 0; } + + @Override public void readInto(Object o, MetaField f, ResultSet rs, int j) throws SQLException { + Object raw = rs.getObject(j); + if (raw == null || rs.wasNull()) { f.setString(o, null); return; } + try { + f.setString(o, raw instanceof Number n + ? DurationWire.format(DurationWire.fromMicros(n.longValue())) + : DurationWire.canonicalizeStored(rs.getString(j))); + } catch (IllegalArgumentException e) { + throw new SQLException("field.duration '" + f.getName() + "' cannot be read: " + e.getMessage(), e); + } + } + @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { + if (v == null) { s.setNull(j, Types.BIGINT); return; } + try { + s.setLong(j, DurationWire.toMicros(toDuration(f, v))); + } catch (IllegalArgumentException e) { + throw new SQLException("field.duration '" + f.getName() + "' cannot be written: " + e.getMessage(), e); + } + } + + /** The Postgres interval input spelling of a duration value ({@code [-]HH:MM:SS[.f]}). */ + public static String postgresText(MetaField f, Object v) throws SQLException { + return DurationWire.formatPostgres(toDuration(f, v)); + } + + static java.time.Duration toDuration(MetaField f, Object v) throws SQLException { + if (v instanceof java.time.Duration d) return d; + try { + return DurationWire.parse(v.toString()); + } catch (IllegalArgumentException e) { + throw new SQLException("field.duration '" + f.getName() + "' cannot be written: " + e.getMessage(), e); + } + } + } + /** * True for a {@code field.timestamp} carrying {@code @localTime: true} — the naive * wall-clock opt-out (ADR-0036 Wave 2). Absent/false (the default) = an absolute diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/PostgresDriver.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/PostgresDriver.java index d76427820d..fff406cf60 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/PostgresDriver.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/PostgresDriver.java @@ -197,6 +197,25 @@ protected int jsonbNullSqlType() { return java.sql.Types.OTHER; } + /** + * {@code field.duration} → the native {@code interval} column. Postgres refuses a signed + * ISO-8601 duration as interval input ({@code '-PT1M30S'::interval} is "invalid input + * syntax"), so the value binds in the interval spelling {@code [-]HH:MM:SS[.f]} through + * {@code setObject(.., Types.OTHER)} (an untyped parameter, like jsonb). Postgres returns an + * interval in that same spelling, which {@code DurationCodec} reads back canonically. + */ + @Override + protected void setStatementValue(PreparedStatement s, com.metaobjects.field.MetaField f, int index, Object value) + throws SQLException { + if (f instanceof com.metaobjects.field.DurationField && !f.isArrayType()) { + if (value == null) s.setNull(index, java.sql.Types.OTHER); + else s.setObject(index, + com.metaobjects.manager.db.codec.JdbcCodecs.DurationCodec.postgresText(f, value), java.sql.Types.OTHER); + return; + } + super.setStatementValue(s, f, index, value); + } + /** * dbColumnType slim-and-derive (Phase 1): Postgres has native array columns, so a * {@code field.string isArray} → {@code text[]} and a {@code field.uuid isArray} → diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java index d28371d041..1446b261a8 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java @@ -390,6 +390,70 @@ public void uuidCodecReadsBackLowercaseCanonical() throws Exception { } } + /** + * {@code DurationCodec} on an engine without a native interval: a duration is stored as a + * 64-bit count of microseconds (so a range filter and a sort order it as a number), and reads + * back as the canonical wire string. The text a Postgres interval returns reads canonically too. + */ + @Test + public void durationCodecStoresAMicrosecondCountAndReadsTheCanonicalForm() throws Exception { + final java.util.Map store = new java.util.HashMap<>(); + com.metaobjects.field.DurationField durationField = new com.metaobjects.field.DurationField("holdFor") { + @Override public void setString(Object obj, String value) { store.put("v", value); } + }; + JdbcCodecs.DurationCodec codec = new JdbcCodecs.DurationCodec(); + + try (Connection conn = getConnection()) { + try (Statement st = conn.createStatement()) { + st.execute("CREATE TABLE codec_duration_probe (d BIGINT, t VARCHAR(64))"); + } + try (PreparedStatement ps = conn.prepareStatement("INSERT INTO codec_duration_probe (d) VALUES (?)")) { + codec.write(ps, durationField, 1, "P1DT0.5S"); + ps.executeUpdate(); + codec.write(ps, durationField, 1, java.time.Duration.ofSeconds(-90)); + ps.executeUpdate(); + codec.write(ps, durationField, 1, "PT1H30M"); + ps.executeUpdate(); + } + // A range filter binds through the same codec and compares numbers: > PT1H. + java.util.List stored = new java.util.ArrayList<>(); + java.util.List read = new java.util.ArrayList<>(); + try (PreparedStatement ps = conn.prepareStatement( + "SELECT d FROM codec_duration_probe WHERE d > ? ORDER BY d")) { + codec.write(ps, durationField, 1, "PT1H"); + try (ResultSet rs = ps.executeQuery()) { + while (rs.next()) { + stored.add(rs.getString(1)); + codec.readInto(new Object(), durationField, rs, 1); + read.add((String) store.get("v")); + } + } + } + assertEquals(java.util.List.of("5400000000", "86400500000"), stored); + assertEquals(java.util.List.of("PT1H30M", "PT24H0.5S"), read); + // Postgres interval text in a text column reads canonically. + try (Statement st = conn.createStatement()) { + st.execute("INSERT INTO codec_duration_probe (t) VALUES ('-1 days +02:00:00')"); + } + try (Statement st = conn.createStatement(); + ResultSet rs = st.executeQuery("SELECT t FROM codec_duration_probe WHERE t IS NOT NULL")) { + assertTrue(rs.next()); + codec.readInto(new Object(), durationField, rs, 1); + assertEquals("-PT22H", store.get("v")); + } + try (PreparedStatement ps = conn.prepareStatement("INSERT INTO codec_duration_probe (d) VALUES (?)")) { + codec.write(ps, durationField, 1, "P1M"); + fail("a calendar span is not a duration"); + } catch (SQLException expected) { + assertTrue(expected.getMessage(), expected.getMessage().contains("holdFor")); + } + assertEquals("-00:01:30", JdbcCodecs.DurationCodec.postgresText(durationField, "-PT90S")); + try (Statement st = conn.createStatement()) { + st.execute("DROP TABLE codec_duration_probe"); + } + } + } + /** * W2b zone-codec fix — {@code TimestampCodec} (plain TIMESTAMP) is zone-INDEPENDENT. * diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index d04d23e195..7dd23b66c0 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -2292,7 +2292,7 @@ def naming_candidates(used: NamingRules) -> list[NamingRules]: """Every naming other than ``used``, in the order ``verify --codegen`` probes them (the C# port probes the same order): the used table rule first, so a different column rule alone is tried before a different table rule; within each, the used column rule first, then - snake_case, literal, kebab-case.""" + snake_case, literal, kebab-case, upperFirst.""" tables = sorted(TABLE_NAMING_RULES, key=lambda t: t != used.tables) columns = sorted(COLUMN_NAMING_STRATEGIES, key=lambda c: c != used.columns) return [ @@ -2337,7 +2337,7 @@ def wire_differs_message(committed: WireFormat, used: WireFormat) -> str: f"The committed output was generated with {wire_block(committed)}; this run used " f"{wire_block(used)} (the wire block of .metaobjects/config.json; each key defaults to " f"native). Do not regenerate: that changes the JSON the generated routes write " - f"(decimals, value-object null members) for clients already reading it. " + f"(decimals, durations, value-object null members) for clients already reading it. " f"Declare {wire_block(committed)} in .metaobjects/config.json, and gen and verify " f"--codegen both read it." ) diff --git a/server/python/src/metaobjects/codegen/generator_registry.py b/server/python/src/metaobjects/codegen/generator_registry.py index 3c31e48881..b1cc10dc62 100644 --- a/server/python/src/metaobjects/codegen/generator_registry.py +++ b/server/python/src/metaobjects/codegen/generator_registry.py @@ -209,6 +209,8 @@ def _render_helper_default(ctx: GeneratorBuildContext) -> Generator: _FR010_FIELD_MAPPING = "fr010_field_mapping" _M2M_CODEGEN = "m2m_codegen" _FR019_SHARED_ENUM = "fr019_shared_enum" +#: The field.duration codec the entity tier emits as the generated package's ``_duration_wire.py``. +_DURATION_WIRE = "duration_wire" _EXTRACT_DELEGATE = "extract_delegate_emitter" _EXTRACT_SCHEMA = "extract_schema_emitter" _OUTPUT_FORMAT_SPEC = "output_format_spec_emitter" @@ -236,7 +238,7 @@ def _render_helper_default(ctx: GeneratorBuildContext) -> Generator: layer="model", factory=lambda _ctx: entity_model(), source=entity_model, - runtime=_SHARED_PREDICATES + (_FR019_SHARED_ENUM, _M2M_CODEGEN), + runtime=_SHARED_PREDICATES + (_FR019_SHARED_ENUM, _M2M_CODEGEN, _DURATION_WIRE), ), "routes": GeneratorEntry( name="routes", diff --git a/server/python/src/metaobjects/codegen/generators/entity_model.py b/server/python/src/metaobjects/codegen/generators/entity_model.py index 92eb5bc484..06455b476a 100644 --- a/server/python/src/metaobjects/codegen/generators/entity_model.py +++ b/server/python/src/metaobjects/codegen/generators/entity_model.py @@ -9,6 +9,7 @@ from __future__ import annotations import re +from pathlib import Path from metaobjects.meta.core.field.meta_field import MetaField from metaobjects.meta.core.object.meta_object import MetaObject @@ -21,6 +22,7 @@ # FR-037 R1 — THE mutability accessors. Every consumer deciding "may this be # written, and when?" goes through these, so the absent-means-readWrite default # lives in exactly one place per port. +from metaobjects.loader.validation_passes import EXPR_FN_COALESCE, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT from metaobjects.loader.validate_field_mutability import ( is_read_only_mutability, is_write_once_mutability, @@ -33,6 +35,7 @@ ) from metaobjects.meta.core.validator import validator_constants as vc from metaobjects.meta.persistence.db import db_constants as dbc +from metaobjects.wire import WIRE_DURATIONS_ISO, wire_of from metaobjects.meta.persistence.origin.origin_constants import ( AGG_ALL, AGG_ANY, @@ -42,7 +45,7 @@ ) from metaobjects.shared.base_types import TYPE_ORIGIN, TYPE_VALIDATOR from metaobjects.codegen.config import GenConfig -from metaobjects.codegen.constants import generated_header +from metaobjects.codegen.constants import GENERATED_MARKER, generated_header from metaobjects.codegen.type_map import field_is_array, py_type_for from metaobjects.codegen.format import ruff_format from metaobjects.codegen.generator import EmittedFile, GenContext, Generator, per_entity @@ -55,7 +58,12 @@ from ..runtime import fr019_shared_enum as fr019 from ..runtime.m2m_codegen import build_object_index, resolve_m2m_descriptors -from ..runtime.shared_predicates import array_default, has_literal_default +from ..runtime.shared_predicates import ( + DURATION_WIRE_MODULE, + array_default, + has_duration_field, + has_literal_default, +) def _is_int(value: object) -> bool: @@ -75,6 +83,45 @@ def _is_int(value: object) -> bool: ) +#: field.duration under ``wire.durations: iso``: a ``datetime.timedelta`` held to the ISO-8601 +#: wire rule (fixtures/duration-wire). Under ``native`` the field is a plain ``timedelta`` and +#: Pydantic's own parser and serializer apply. On input only an accepted ISO-8601 duration string (or a +#: ``timedelta`` in process) is taken, so Pydantic's own timedelta parser, which also takes +#: numbers, clock text and calendar spans, never sees a value; in JSON the canonical form is +#: written (Pydantic writes ``PT5400S`` for 90 minutes). A Python-mode dump keeps the native +#: ``timedelta``. +DURATION_TYPE = ( + "Annotated[datetime.timedelta, BeforeValidator(parse_duration_input), " + 'PlainSerializer(format_duration, return_type=str, when_used="json")]' +) +#: The generated package's copy of the duration codec, emitted beside the models when any of +#: them has a ``field.duration`` (:func:`duration_wire_module`). A copy rather than an import of +#: the installed ``metaobjects``, so the generated package runs on its own, as an owned router's +#: ``_runtime/`` does; the router imports its formatter from the same module. +DURATION_IMPORTS: tuple[str, ...] = ( + "import datetime", + "from typing import Annotated", + "from pydantic import BeforeValidator, PlainSerializer", + f"from .{DURATION_WIRE_MODULE} import format_duration, parse_duration_input", +) + +#: Where the codec source is read from: ``codegen/runtime/`` beside this generator, which is the +#: packaged ``metaobjects/codegen/runtime`` or, for an ejected copy, the adopter's own +#: ``codegen/runtime`` (eject copies ``duration_wire.py`` there with this generator). +_RUNTIME_SOURCE_DIR = Path(__file__).resolve().parent.parent / "runtime" + + +def duration_wire_module() -> EmittedFile: + """The ``_duration_wire.py`` module a model with a ``field.duration`` imports its codec from: + ``codegen/runtime/duration_wire.py``, verbatim, under a generated header.""" + src = _RUNTIME_SOURCE_DIR / "duration_wire.py" + return EmittedFile( + path=f"{DURATION_WIRE_MODULE}.py", + content=f"# {GENERATED_MARKER} — DO NOT EDIT. Copied from codegen/runtime/duration_wire.py: " + "edit that file and regenerate.\n" + src.read_text(encoding="utf-8"), + ) + + def _validators(field: MetaField, sub_type: str) -> list[MetaField]: """The field's own ``validator.`` children (effective, supers included).""" return [ @@ -162,7 +209,9 @@ def _validator_constraints(field: MetaField, *, wire: bool) -> dict[str, object] min_len = _first_attr(field, vc.VALIDATOR_SUBTYPE_LENGTH, vc.VALIDATOR_ATTR_MIN) max_len = _first_attr(field, vc.VALIDATOR_SUBTYPE_LENGTH, vc.VALIDATOR_ATTR_MAX) field_max = field.attrs().get(fc.FIELD_ATTR_MAX_LENGTH) - if _is_int(field_max): + # On a field.enum @maxLength is the column length, not an input rule: the Literal of its + # members is the rule, and every member fits. + if _is_int(field_max) and field.sub_type != fc.FIELD_SUBTYPE_ENUM: # FR-036 A3 — strictest-wins: when BOTH a validator.length @max and a field # @maxLength cap a string, the smaller bound governs (never the field attr # unconditionally, which could loosen a tighter validator). @@ -298,6 +347,16 @@ def _type_expr_for_field( ): imports.add("from pydantic import EmailStr") type_expr = "list[EmailStr]" if field_is_array(field) else "EmailStr" + if field.sub_type == fc.FIELD_SUBTYPE_DURATION: + # `wire.durations: native` leaves Pydantic's own timedelta handling in place, both ways; + # `iso` holds the field to the ISO-8601 wire rule. + if wire_of(field).durations == WIRE_DURATIONS_ISO: + imports.update(DURATION_IMPORTS) + duration_type = DURATION_TYPE + else: + imports.add("import datetime") + duration_type = "datetime.timedelta" + type_expr = f"list[{duration_type}]" if field_is_array(field) else duration_type if field.sub_type in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): # A field.object (-> VO) and a field.map with @objectRef (-> dict[str, VO]) both # reference a value-object; import its model so the emitted annotation resolves. @@ -470,12 +529,19 @@ def _assigned_pk_field_names(entity: MetaObject) -> set[str]: _NUMERIC_CROSS = _INT_SUBTYPES + _FLOAT_SUBTYPES + _DECIMAL_SUBTYPES _ORDERED_CROSS = ( fc.FIELD_SUBTYPE_STRING, fc.FIELD_SUBTYPE_DATE, fc.FIELD_SUBTYPE_TIME, fc.FIELD_SUBTYPE_TIMESTAMP, + fc.FIELD_SUBTYPE_DURATION, +) +# Subtypes whose values test only equality: compared with ``==``, never ordered (a uri or an +# inet value need not order, and ordering one raises TypeError). +_EQUALITY_ONLY_CROSS = ( + fc.FIELD_SUBTYPE_BOOLEAN, fc.FIELD_SUBTYPE_ENUM, + fc.FIELD_SUBTYPE_UUID, fc.FIELD_SUBTYPE_URI, fc.FIELD_SUBTYPE_INET, ) def _comparable(left: MetaField, right: MetaField, op: str) -> bool: """Whether a comparison between these two fields has a sound meaning, as in every port: numerics - compare with each other, strings and temporals only with their own subtype and @localTime flag (a + compare with each other, strings, temporals and durations only with their own subtype and @localTime flag (a date has no ordering against a timestamp, nor an aware datetime against a naive one, and Python raises TypeError on both), and boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, never rejected.""" @@ -490,15 +556,13 @@ def _comparable(left: MetaField, right: MetaField, op: str) -> bool: return False if a in _ORDERED_CROSS: return True - return op in ("eq", "ne") and a in ( - fc.FIELD_SUBTYPE_BOOLEAN, fc.FIELD_SUBTYPE_ENUM, - fc.FIELD_SUBTYPE_UUID, fc.FIELD_SUBTYPE_URI, fc.FIELD_SUBTYPE_INET, - ) + return op in ("eq", "ne") and a in _EQUALITY_ONLY_CROSS _CROSS_FIELD_SUBTYPES = ( vc.VALIDATOR_SUBTYPE_COMPARISON, vc.VALIDATOR_SUBTYPE_REQUIRED_WHEN, vc.VALIDATOR_SUBTYPE_PRESENT_IFF, vc.VALIDATOR_SUBTYPE_AT_LEAST_ONE, + vc.VALIDATOR_SUBTYPE_EXACTLY_ONE, ) _NUMERIC_LITERAL = re.compile(r"^-?\d+(\.\d+)?$") @@ -521,6 +585,182 @@ def _equals_literal(raw: object, when_field: MetaField, imports: set[str]) -> st return repr(text) +# --- @condition: a rule is checked only where its condition is TRUE ----------- +# The CHECK is ``NOT (condition) OR (rule)``, so a condition that is FALSE or NULL (SQL +# three-valued logic) lets the record through. The condition is generated as a small Python +# expression of type ``bool | None`` (``None`` is SQL NULL); the helpers below are declared +# inside the generated method, so the generated model imports nothing from MetaObjects. +_HELPER_CMP = ( + " def _cmp(a: object, b: object) -> int | None:\n" + " if a is None or b is None:\n" + " return None\n" + " try:\n" + " return (a > b) - (a < b) # type: ignore[operator]\n" + " except TypeError:\n" + " return None\n" +) +_HELPER_TRI = ( + " def _all(*v: bool | None) -> bool | None:\n" + " return False if False in v else None if None in v else True\n" + "\n" + " def _any(*v: bool | None) -> bool | None:\n" + " return True if True in v else None if None in v else False\n" + "\n" + " def _not(v: bool | None) -> bool | None:\n" + " return None if v is None else not v\n" +) +_CONDITION_HELPERS = (_HELPER_CMP, _HELPER_TRI) +_NUMERIC_SUBTYPES = (*_INT_SUBTYPES, *_FLOAT_SUBTYPES, *_DECIMAL_SUBTYPES) +EXPR_OP_IS_NULL, EXPR_OP_AND, EXPR_OP_OR = "isNull", "and", "or" +_COMPARE_NUMBER, _COMPARE_DURATION, _COMPARE_PLAIN = "number", "duration", "plain" +_TEMPORAL_PARSERS = { + fc.FIELD_SUBTYPE_DATE: "datetime.date.fromisoformat", + fc.FIELD_SUBTYPE_TIME: "datetime.time.fromisoformat", + fc.FIELD_SUBTYPE_TIMESTAMP: "datetime.datetime.fromisoformat", +} + + +def _compare_kind(field: MetaField | None) -> str: + if field is None: + return _COMPARE_PLAIN + if field.sub_type in _NUMERIC_SUBTYPES: + return _COMPARE_NUMBER + if field.sub_type == fc.FIELD_SUBTYPE_DURATION: + return _COMPARE_DURATION + return _COMPARE_PLAIN + + +_DURATION_LITERAL = re.compile( + r"^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$", + re.ASCII, +) + + +def _duration_micros(text: str) -> int | None: + """An ISO-8601 duration literal (weeks, days, hours, minutes, seconds; the accepted wire + form) as microseconds, or ``None`` when it is not one.""" + m = _DURATION_LITERAL.match(text) + if m is None: + return None + sign, w, d, h, mi, sec, frac = m.groups() + hours = (int(w or 0) * 7 + int(d or 0)) * 24 + int(h or 0) + micros = ((hours * 60 + int(mi or 0)) * 60 + int(sec or 0)) * 1_000_000 + int((frac or "").ljust(6, "0")[:6] or 0) + return -micros if sign == "-" else micros + + +class _ConditionBuilder: + """Lowers an ``@condition`` expression to a ``bool | None`` Python expression over the + model's fields; ``None`` from a method means a field is not in the model (skip the rule).""" + + def __init__(self, by_name: dict[str, MetaField], imports: set[str]) -> None: + self.by_name = by_name + self.imports = imports + self.helpers: set[str] = set() + + def _node_field(self, node: object) -> MetaField | None: + if not isinstance(node, dict): + return None + name = node.get("field") + if isinstance(name, str): + return self.by_name.get(name) + args = node.get("args") + if node.get("fn") == EXPR_FN_COALESCE and isinstance(args, list): + for a in args: + f = self._node_field(a) + if f is not None: + return f + return None + + def _node_fields(self, node: object) -> list[MetaField]: + """Every field a value node reads: the field itself, or each field among a coalesce's arguments.""" + if not isinstance(node, dict): + return [] + name = node.get("field") + if isinstance(name, str): + f = self.by_name.get(name) + return [] if f is None else [f] + args = node.get("args") + if node.get("fn") == EXPR_FN_COALESCE and isinstance(args, list): + return [f for a in args for f in self._node_fields(a)] + return [] + + def _pairs_comparable(self, left: object, right: object, op: str) -> bool: + """A comparison inside a condition follows the validator rule: every pair of fields it + reads must be comparable, or the whole rule is left to the database.""" + fields = self._node_fields(left) + self._node_fields(right) + return all(_comparable(a, b, op) for i, a in enumerate(fields) for b in fields[i + 1:]) + + def _literal(self, value: object, kind: str, field: MetaField | None) -> str | None: + if kind == _COMPARE_DURATION and isinstance(value, str): + micros = _duration_micros(value) + if micros is None: + return None + self.imports.add("import datetime") + return f"datetime.timedelta(microseconds={micros})" + if kind == _COMPARE_PLAIN and isinstance(value, str) and field is not None: + parser = _TEMPORAL_PARSERS.get(field.sub_type) + if parser is not None: + self.imports.add("import datetime") + return f"{parser}({value!r})" + return repr(value) + + def value(self, node: object, kind: str, ref: MetaField | None) -> str | None: + if not isinstance(node, dict): + return None + name = node.get("field") + if isinstance(name, str): + return f"self.{name}" if name in self.by_name else None + if "value" in node: + return self._literal(node["value"], kind, ref) + args = node.get("args") + if node.get("fn") == EXPR_FN_COALESCE and isinstance(args, list): + parts = [self.value(a, kind, ref) for a in args] + if any(p is None for p in parts): + return None + return f"next((x for x in ({', '.join(parts)},) if x is not None), None)" # type: ignore[arg-type] + return None + + def condition(self, node: object) -> str | None: + if not isinstance(node, dict): + return None + op = node.get("op") + if isinstance(op, str) and op in _COMPARISON_PY_OP: + if not self._pairs_comparable(node.get("left"), node.get("right"), op): + return None + ref = self._node_field(node.get("left")) or self._node_field(node.get("right")) + kind = _compare_kind(ref) + left = self.value(node.get("left"), kind, ref) + right = self.value(node.get("right"), kind, ref) + if left is None or right is None: + return None + if ref is not None and ref.sub_type in _EQUALITY_ONLY_CROSS: + return ( + f"(lambda a, b: None if a is None or b is None else a {_COMPARISON_PY_OP[op]} b)" + f"({left}, {right})" + ) + self.helpers.add(_HELPER_CMP) + return f"(lambda c: None if c is None else c {_COMPARISON_PY_OP[op]} 0)(_cmp({left}, {right}))" + if op in (EXPR_OP_IS_NULL, EXPR_OP_IS_NOT_NULL): + arg = self.value(node.get("arg"), _COMPARE_PLAIN, None) + if arg is None: + return None + return f"({arg} is {'' if op == EXPR_OP_IS_NULL else 'not '}None)" + if op == EXPR_OP_NOT: + arg = self.condition(node.get("arg")) + if arg is None: + return None + self.helpers.add(_HELPER_TRI) + return f"_not({arg})" + args = node.get("args") + if op in (EXPR_OP_AND, EXPR_OP_OR) and isinstance(args, list): + parts = [self.condition(a) for a in args] + if any(p is None for p in parts): + return None + self.helpers.add(_HELPER_TRI) + return f"{'_all' if op == EXPR_OP_AND else '_any'}({', '.join(parts)})" # type: ignore[arg-type] + return None + + def _cross_field_rules_method( entity: MetaObject, model_name: str, model_fields: set[str], imports: set[str] ) -> list[str]: @@ -529,9 +769,29 @@ def _cross_field_rules_method( referenced field is missing from the model is skipped silently (as the DB CHECK is).""" by_name = {f.name: f for f in entity.fields() if f.name in model_fields} checks: list[str] = [] + builder = _ConditionBuilder(by_name, imports) + + def guarded(v: MetaData, block: str) -> None: + """Add a rule's block, checked only where its ``@condition`` is TRUE (three-valued).""" + condition = v.get_meta_attr(vc.VALIDATOR_ATTR_CONDITION) + if condition is None: + checks.append(block) + return + guard = builder.condition(condition) + if guard is None: + # Left to the database: undo the imports and helpers this rule asked for, so a + # dropped rule leaves nothing unused in the generated module. + imports.intersection_update(kept[0]) + builder.helpers.intersection_update(kept[1]) + return + inner = "\n".join((" " + ln) if ln else ln for ln in block.split("\n")) + checks.append(f" if {guard} is True:\n{inner}") + for v in entity.children(): if v.type != TYPE_VALIDATOR or v.sub_type not in _CROSS_FIELD_SUBTYPES: continue + kept = (set(imports), set(builder.helpers)) # what the rules kept so far need + # ADR-0039: resolving read — a cross-field validator's attrs may be inherited. def attr(name: str, node: MetaData = v) -> object: return node.get_meta_attr(name) @@ -546,14 +806,15 @@ def attr(name: str, node: MetaData = v) -> object: continue if not _comparable(by_name[str(left)], by_name[str(right)], str(attr(vc.VALIDATOR_ATTR_OP))): continue - checks.append( + guarded( + v, f" if self.{left} is not None and self.{right} is not None:\n" f" try:\n" f" _ordered = self.{left} {op} self.{right}\n" f" except TypeError:\n" f" _ordered = False\n" f" if not _ordered:\n" - f" raise ValueError({label})" + f" raise ValueError({label})", ) elif v.sub_type in (vc.VALIDATOR_SUBTYPE_REQUIRED_WHEN, vc.VALIDATOR_SUBTYPE_PRESENT_IFF): target, when = attr(vc.VALIDATOR_ATTR_FIELD), attr(vc.VALIDATOR_ATTR_WHEN) @@ -563,28 +824,42 @@ def attr(name: str, node: MetaData = v) -> object: cmp_op = "is" if lit in ("True", "False") else "==" gate = f"(self.{when} is not None and self.{when} {cmp_op} {lit})" if v.sub_type == vc.VALIDATOR_SUBTYPE_REQUIRED_WHEN: - checks.append( + guarded( + v, f" if {gate} and self.{target} is None:\n" - f" raise ValueError({label})" + f" raise ValueError({label})", ) else: - checks.append( + guarded( + v, f" if (self.{target} is not None) != {gate}:\n" - f" raise ValueError({label})" + f" raise ValueError({label})", ) else: raw = attr(vc.VALIDATOR_ATTR_FIELDS) names = list(raw) if isinstance(raw, (list, tuple)) else ([raw] if isinstance(raw, str) else []) if not names or any(n not in by_name for n in names): continue - all_none = " and ".join(f"self.{n} is None" for n in names) - checks.append(f" if {all_none}:\n raise ValueError({label})") + if v.sub_type == vc.VALIDATOR_SUBTYPE_EXACTLY_ONE: + present = ", ".join(f"self.{n}" for n in names) + guarded( + v, + f" if sum(x is not None for x in ({present},)) != 1:\n" + f" raise ValueError({label})", + ) + else: + all_none = " and ".join(f"self.{n} is None" for n in names) + guarded(v, f" if {all_none}:\n raise ValueError({label})") if not checks: return [] + # Declared in a fixed order, so the output does not depend on which rule needed one first. + helper_src = "\n".join(h for h in _CONDITION_HELPERS if h in builder.helpers).rstrip("\n").split("\n") return [ "", ' @model_validator(mode="after")', f" def _cross_field_rules(self) -> {model_name}:", + *(helper_src if builder.helpers else []), + *([""] if builder.helpers else []), *checks, " return self", ] @@ -801,7 +1076,8 @@ def _emit_cross_field_rules( self, entity: MetaObject, create_lines: list[str], imports: set[str] ) -> list[str]: """The ``Create`` cross-field ``model_validator`` (comparison / requiredWhen / - presentIff / atLeastOne), keyed by the field lines the create model carries. + presentIff / atLeastOne / exactlyOne, each optionally under an ``@condition``), keyed by + the field lines the create model carries. Create only: a Patch body cannot see the merged row. Override to customize.""" carried = {m.group(1) for line in create_lines if (m := re.match(r" (\w+):", line))} return _cross_field_rules_method(entity, f"{model_class_name(entity)}Create", carried, imports) @@ -1054,6 +1330,8 @@ def generate(self, ctx: GenContext) -> list[EmittedFile]: enums_module = self._render_shared_enums_module(matched) if enums_module is not None: files.append(enums_module) + if has_duration_field(matched) and any(wire_of(e).durations == WIRE_DURATIONS_ISO for e in matched): + files.append(duration_wire_module()) return files diff --git a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py index 000b1cf10b..8230ff0790 100644 --- a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py +++ b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py @@ -94,7 +94,10 @@ def ops_for_subtype_ordered(sub_type: str | None) -> tuple[str, ...]: return _OPS_STRING if sub_type in (fc.FIELD_SUBTYPE_UUID, fc.FIELD_SUBTYPE_INET): return _OPS_UUID + # field.duration is an orderable span (a Postgres interval; a microsecond integer on + # SQLite, D1 and MySQL): the numeric band. if sub_type in ( + fc.FIELD_SUBTYPE_DURATION, fc.FIELD_SUBTYPE_INT, fc.FIELD_SUBTYPE_LONG, fc.FIELD_SUBTYPE_FLOAT, diff --git a/server/python/src/metaobjects/codegen/generators/names_generator.py b/server/python/src/metaobjects/codegen/generators/names_generator.py index 4e7f9999d5..d1be676c56 100644 --- a/server/python/src/metaobjects/codegen/generators/names_generator.py +++ b/server/python/src/metaobjects/codegen/generators/names_generator.py @@ -75,6 +75,8 @@ """ from __future__ import annotations +from collections.abc import Callable + import re from dataclasses import dataclass @@ -90,13 +92,21 @@ from metaobjects.meta.persistence.source.source_constants import ( PHYSICAL_NAME_ATTR_BY_KIND, ) +from metaobjects.meta.core.field.field_constants import ( + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_STORAGE, + FIELD_SUBTYPE_OBJECT, + STORAGE_FLATTENED, +) from metaobjects.naming import ( resolve_column_name, + resolve_flattened_column_name, resolve_index_name, strip_package, to_snake_case, ) from metaobjects.shared.base_types import TYPE_FIELD, TYPE_IDENTITY, TYPE_INDEX +from metaobjects.naming_refs import ref_target from metaobjects.source_resolution import primary_rdb_source #: The nodes whose DATABASE index name this module carries. @@ -221,7 +231,7 @@ def _key_consts(node: MetaData) -> list[_Const]: return out -def _field_consts(field: MetaData, column: str) -> list[_Const]: +def _field_consts(field: MetaData, column: str | None) -> list[_Const]: """One field. The ONE node kind with no type prefix: a field's sub_type does not change what ``COLUMN`` denotes. @@ -229,12 +239,49 @@ def _field_consts(field: MetaData, column: str) -> list[_Const]: moment a case boundary or a ``@column`` appears, and a consumer holding one cannot recover the other — ``@column`` is free-form, so ``callPurpose`` may map to ``purpose_code``, which is neither the logical name nor any transformation of it. + + ``column`` is ``None`` for a flattened field, which has no column of its own: only + ``FIELD`` is emitted and its members carry the columns. """ member = _member(field.name) path = f"field {field.name!r}" + consts = [_Const(f"{member}_FIELD", _q(field.name), path)] + if column is not None: + consts.append(_Const(f"{member}_COLUMN", _q(column), path)) + return consts + + +def _flattened_members(field: MetaData, strategy: str) -> list[tuple[str, str]] | None: + """The ``(member name, column)`` pairs of a ``@storage: "flattened"`` value object, or + ``None`` for any other field. + + The flattened test is migrate's own: a ``field.object`` whose RESOLVED ``@storage`` is + ``flattened`` (ADR-0039 — it may be inherited). Each member's column is core naming's + :func:`metaobjects.naming.resolve_flattened_column_name`, the rule ``meta migrate`` names + it by. A flattened field whose ``@objectRef`` does not resolve has no members; the loader + refuses that model before codegen runs. + """ + if field.sub_type != FIELD_SUBTYPE_OBJECT or field.get_meta_attr(FIELD_ATTR_STORAGE) != STORAGE_FLATTENED: + return None + target = ref_target(field, FIELD_ATTR_OBJECT_REF) + if not isinstance(target, MetaObject): + return [] + return [(m.name, resolve_flattened_column_name(field, m, strategy)) for m in target.fields()] + + +def _flattened_member_consts( + field: MetaData, members: list[tuple[str, str]] +) -> list[tuple[str, _Const]]: + """A flattened value object's ``__COLUMN`` constants, one per member, + emitted beside the field's own ``_FIELD`` constant (a flattened field has no column + constant of its own).""" + member = _member(field.name) return [ - _Const(f"{member}_FIELD", _q(field.name), path), - _Const(f"{member}_COLUMN", _q(column), path), + ( + f"{field.name}.{name}", + _Const(f"{member}_{_member(name)}_COLUMN", _q(column), f"field {field.name + '.' + name!r}"), + ) + for name, column in sorted(members) ] @@ -340,7 +387,13 @@ def _sources_by_role(entity: MetaObject) -> dict[str, MetaSource]: return out -def render_names(entity: MetaObject, strategy: str, *, fragment: bool = False) -> str | None: +def render_names( + entity: MetaObject, + strategy: str, + *, + fragment: bool = False, + warn: Callable[[str], None] | None = None, +) -> str | None: """Render the names module for ``entity``, or ``None`` when it declares (and inherits) no primary ``source.rdb`` at all (#248). @@ -404,11 +457,44 @@ def carried_by_super(node: MetaData) -> bool: # ADR-0039: fields() RESOLVES — an inherited field and an inherited @column both # must appear here, or the constant disagrees with what the runtime actually binds. + # (COLUMNS_BY_FIELD key, constant member) — a flattened member is keyed + # `.`, the path the naming corpus uses. columns: list[tuple[str, str]] = [] + flattened: list[tuple[MetaData, list[tuple[str, _Const]]]] = [] for field in sorted(entity.fields(), key=lambda f: f.name): - column = resolve_column_name(field, strategy) - columns.append((field.name, column)) - groups.append(("fields", _field_consts(field, column), carried_by_super(field))) + members = _flattened_members(field, strategy) + if members is None: + column = resolve_column_name(field, strategy) + columns.append((field.name, f"{_member(field.name)}_COLUMN")) + groups.append(("fields", _field_consts(field, column), carried_by_super(field))) + else: + # A flattened field has no column of its own: it keeps its `_FIELD` constant and + # its members carry the columns. + groups.append(("fields", _field_consts(field, None), carried_by_super(field))) + if members: + flattened.append((field, _flattened_member_consts(field, members))) + + # A flattened member's constant that would share a name with another constant — a field + # `shipToStreet` beside the member `street` of a flattened `shipTo` — is left out, with a + # warning, rather than refused: such a model generated before member constants existed, + # and every other constant it had is still emitted. + taken = {c.member for _s, consts, _i in groups for c in consts} + for field, consts in flattened: + kept = [] + for key, const in consts: + if const.member in taken: + if warn is not None: + warn( + f"names: {entity.name}: the flattened column constant {upper}_{const.member} " + "would share its name with another constant of the module, so it is not " + "emitted. Rename the field or the value-object member to get it." + ) + continue + taken.add(const.member) + kept.append(const) + columns.append((key, const.member)) + if kept: + groups.append(("fields", kept, carried_by_super(field))) for type_name in (TYPE_IDENTITY, TYPE_INDEX): keys = sorted( @@ -488,11 +574,11 @@ def carried_by_super(node: MetaData) -> bool: # module's re-exported reference to the base's own constant. Always emitted, even # when empty, so a consumer can read it unconditionally. lines.append(f"{upper}_COLUMNS_BY_FIELD: Final[dict[str, str]] = {{") - for name, _column in columns: + for key, member in sorted(columns): # _q, not a bare f-string: the KEY is an author-supplied field name, and every # other literal in this module goes through the escaper. Spliced raw, a name # holding a quote emitted a module that does not parse. - lines.append(f" {_q(name)}: {upper}_{_member(name)}_COLUMN,") + lines.append(f" {_q(key)}: {upper}_{member},") lines.append("}") lines.append("") return "\n".join(lines) @@ -508,7 +594,7 @@ def generate(self, ctx: GenContext) -> list[EmittedFile]: emitted: set[str] = set() def one(entity: MetaObject, *, fragment: bool) -> list[EmittedFile]: - source = render_names(entity, ctx.config.column_naming, fragment=fragment) + source = render_names(entity, ctx.config.column_naming, fragment=fragment, warn=ctx.warn) if source is None: return [] emitted.add(entity.resolution_key()) diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 227346832c..175d3beff3 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -75,7 +75,7 @@ from metaobjects.codegen.config import DEFAULT_API_PREFIX from metaobjects.naming import DEFAULT_COLUMN_NAMING from metaobjects.naming_refs import ref_target -from metaobjects.wire import WIRE_NATIVE, WireFormat, wire_of +from metaobjects.wire import WIRE_DURATIONS_ISO, WIRE_NATIVE, WireFormat, wire_of from metaobjects.shared.base_types import TYPE_IDENTITY from ..runtime.fr010_field_mapping import is_required @@ -85,7 +85,12 @@ pk_field_name, resolve_m2m_descriptors, ) -from ..runtime.shared_predicates import emits_router, is_read_only_routed +from ..runtime.shared_predicates import ( + DURATION_WIRE_MODULE, + emits_router, + has_duration_field, + is_read_only_routed, +) #: The repository ``Protocol`` declares a method named ``list`` (the cross-port contract keeps #: that name), and inside the class body a later bare ``list[...]`` annotation resolves to @@ -217,6 +222,27 @@ def _decimal_field_names(entities: list[MetaObject]) -> list[str]: return list(seen) +def _duration_field_names(entities: list[MetaObject]) -> list[str]: + """Names of the ``field.duration`` fields across *entities* (ADR-0039 resolving), in + declaration order, de-duplicated.""" + seen: dict[str, None] = {} + for entity in entities: + for f in entity.fields(): + if f.sub_type == fc.FIELD_SUBTYPE_DURATION: + seen.setdefault(f.name, None) + return list(seen) + + +def _duration_fields_line(entities: list[MetaObject], wire: WireFormat) -> str: + """The ``_DURATION_FIELDS`` constant a list handler hands the filter coercion: each + duration field with the parser its filter operand goes through (D4: ISO-8601 only, a + malformed one is a 400). Under ``wire.durations: native`` there is no ISO rule to hold an + operand to, so the map is empty and the repository's own handling applies.""" + names = _duration_field_names(entities) if wire.durations == WIRE_DURATIONS_ISO else [] + body = ", ".join(f"{n!r}: parse_duration" for n in names) + return f"_DURATION_FIELDS: dict[str, object] = {{{body}}}" + + def _decimal_limits(entities: list[MetaObject]) -> dict[str, tuple[int, int]]: """``{name: (precision, scale)}`` for each decimal field declaring ``@precision`` (ADR-0039 resolving; the first declaration of a name wins). The column's range (W3): @@ -322,9 +348,35 @@ def _wire_lines(entities: list[MetaObject], wire: WireFormat, name: str = "_WIRE args += f", shapes={shapes!r}" if hidden := _hidden_names(entities): args += f", hidden={tuple(hidden)!r}" + if has_duration_field(entities): + args += f", {_DURATION_WIRE_ARG if wire.durations == WIRE_DURATIONS_ISO else _NATIVE_DURATION_ARG}" return ["", "", f"{name} = WireOutput({args})"] +#: The ``WireOutput`` argument a response holding a ``field.duration`` is written with: its +#: ``timedelta`` leaves as the canonical ISO-8601 duration, not FastAPI's float of seconds. +_DURATION_WIRE_ARG = "duration=format_duration" + + +#: Under ``wire.durations: native`` a ``timedelta`` leaves as Pydantic itself writes it, not as +#: FastAPI's ``jsonable_encoder`` would (a float of seconds). +_NATIVE_DURATION_ARG = "duration=to_jsonable_python" + + +def _with_duration_import(parts: list[str]) -> list[str]: + """*parts* plus the import of the duration formatter, when a wire constant uses it. The + formatter is the generated package's own ``_duration_wire`` module, which the entity tier + emits beside any model with a ``field.duration``.""" + if any(_NATIVE_DURATION_ARG in line for line in parts): + at = next(i for i, line in enumerate(parts) if line.startswith("from .")) + return [*parts[:at], "from pydantic_core import to_jsonable_python", *parts[at:]] + if not any(_DURATION_WIRE_ARG in line for line in parts): + return parts + at = next(i for i, line in enumerate(parts) if line.startswith("from .")) + names = "format_duration, parse_duration" if any("parse_duration}" in line for line in parts) else "format_duration" + return [*parts[:at], f"from .{DURATION_WIRE_MODULE} import {names}", *parts[at:]] + + def _m2m_wire_name(target: str) -> str: """The wire constant an M:N traversal onto *target* writes the target's rows through.""" return f"_WIRE_{_snake_case(target).upper()}" @@ -902,7 +954,7 @@ def _report_list_with_export( " if filter_result.error_envelope is not None:", " return JSONResponse(status_code=400, content=filter_result.error_envelope)", " predicates, decimal_error = coerce_decimal_predicates(", - " filter_result.predicates, _DECIMAL_FIELDS", + " filter_result.predicates, _DECIMAL_FIELDS, _DURATION_FIELDS", " )", " if decimal_error is not None:", " return JSONResponse(status_code=400, content=decimal_error)", @@ -1015,7 +1067,7 @@ def _emit_route_handler( " if filter_result.error_envelope is not None:", " return JSONResponse(status_code=400, content=filter_result.error_envelope)", " predicates, decimal_error = coerce_decimal_predicates(", - " filter_result.predicates, _DECIMAL_FIELDS", + " filter_result.predicates, _DECIMAL_FIELDS, _DURATION_FIELDS", " )", " if decimal_error is not None:", " return JSONResponse(status_code=400, content=decimal_error)", @@ -1052,7 +1104,7 @@ def _emit_route_handler( " if filter_result.error_envelope is not None:", " return JSONResponse(status_code=400, content=filter_result.error_envelope)", " predicates, decimal_error = coerce_decimal_predicates(", - " filter_result.predicates, _DECIMAL_FIELDS", + " filter_result.predicates, _DECIMAL_FIELDS, _DURATION_FIELDS", " )", " if decimal_error is not None:", " return JSONResponse(status_code=400, content=decimal_error)", @@ -1248,7 +1300,7 @@ def _emit_tph_list_body( " if filter_result.error_envelope is not None:", " return JSONResponse(status_code=400, content=filter_result.error_envelope)", " predicates, decimal_error = coerce_decimal_predicates(", - " filter_result.predicates, _DECIMAL_FIELDS", + " filter_result.predicates, _DECIMAL_FIELDS, _DURATION_FIELDS", " )", " if decimal_error is not None:", " return JSONResponse(status_code=400, content=decimal_error)", @@ -1456,6 +1508,7 @@ def _render_tph_router( parts.append("") parts.append("") parts.append(f"_DECIMAL_FIELDS: frozenset[str] = {decimal_set_body}") + parts.append(_duration_fields_line([entity, *(st.entity for st in plan.subtypes)], wire)) parts.extend(_decimal_limits_lines([entity, *(st.entity for st in plan.subtypes)])) parts.extend(_wire_lines([entity, *(st.entity for st in plan.subtypes)], wire)) parts.extend(_m2m_wire_lines( @@ -1670,7 +1723,7 @@ def list_sig(fn: str, route: str) -> list[str]: )) parts.append("") - return "\n".join(parts) + return "\n".join(_with_duration_import(parts)) def render_router( self, @@ -1849,6 +1902,7 @@ def render_router( parts.append("") parts.append("") parts.append(f"_DECIMAL_FIELDS: frozenset[str] = {decimal_set_body}") + parts.append(_duration_fields_line([entity], wire)) parts.extend(_decimal_limits_lines([entity])) parts.extend(_wire_lines([entity], wire)) parts.extend(_m2m_wire_lines(m2m, object_index or {}, wire)) @@ -1914,7 +1968,7 @@ def render_router( parts.extend(self._emit_m2m_route(d, snake, pk_param, pk_type, repo_class)) parts.append("") - return "\n".join(parts) + return "\n".join(_with_duration_import(parts)) def _emit_readonly_reject_handlers( self, @@ -2077,6 +2131,7 @@ def _render_readonly_router( "_DECIMAL_FIELDS: frozenset[str] = " + _py_set_literal(_decimal_field_names([entity]), frozen=True) ) + parts.append(_duration_fields_line([entity], wire)) parts.extend(_wire_lines([entity], wire)) parts.append("") parts.append("") @@ -2136,7 +2191,7 @@ def _render_readonly_router( ) ) parts.append("") - return "\n".join(parts) + return "\n".join(_with_duration_import(parts)) def generate(self, ctx: GenContext) -> list[EmittedFile]: index = build_object_index(ctx.entities) diff --git a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py index 3131576954..984977c0d7 100644 --- a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py +++ b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py @@ -21,6 +21,7 @@ import dataclasses import json import re +from datetime import timedelta from decimal import Context, Decimal, InvalidOperation from collections.abc import Mapping from typing import Any, Callable, Iterable @@ -225,15 +226,31 @@ def _coerce_decimal(value: Any, limit: tuple[int, int] | None) -> Decimal: def coerce_decimal_predicates( - predicates: Iterable[Any], decimal_fields: frozenset[str] + predicates: Iterable[Any], + decimal_fields: frozenset[str], + duration_fields: Mapping[str, Any] | None = None, ) -> tuple[list[Any], dict[str, str] | None]: """W2 for a filter: each predicate (a dataclass with ``field`` / ``op`` / ``value``, the shape ``filter_parser.parse_filter`` returns) on a decimal field has its operand parsed exactly, so a repository compares a ``Decimal``, never a float or text. Returns ``(predicates, None)``, or ``([], envelope)`` where *envelope* is the corpus's existing - ``invalid_filter_value`` body naming the first malformed field.""" + ``invalid_filter_value`` body naming the first malformed field. An operand on a duration + field (D4) is likewise parsed by the parser the router names for it (ISO-8601 only, to a ``timedelta``), so a repository + compares a span, never text.""" out: list[Any] = [] + durations = duration_fields or {} for p in predicates: + if p.field in durations and p.op != "isNull": + try: + value = ( + [durations[p.field](v) for v in p.value] + if isinstance(p.value, list) + else durations[p.field](p.value) + ) + except ValueError: + return [], {"error": "invalid_filter_value", "field": p.field} + out.append(dataclasses.replace(p, value=value)) + continue if p.field in decimal_fields and p.op != "isNull": try: value = ( @@ -248,33 +265,42 @@ def coerce_decimal_predicates( return out, None -def _render(value: Any, decimal: Callable[[Decimal], str]) -> Any: +#: A ``timedelta`` (a ``field.duration``) as its wire string: the generated package's +#: ``_duration_wire.format_duration``, handed in by a router whose entity has a duration. +DurationFormat = Callable[[timedelta], str] + + +def _render(value: Any, decimal: Callable[[Decimal], str], duration: DurationFormat | None = None) -> Any: """*value* with every ``Decimal`` (at any depth of a dict, list or Pydantic model) as the - string *decimal* makes of it; everything else is left for the framework's encoder.""" + string *decimal* makes of it, and, when *duration* is given, every ``timedelta`` as the + string it makes of it (FastAPI's own encoder would write a float count of seconds); + everything else is left for the framework's encoder.""" if isinstance(value, Decimal): return decimal(value) + if duration is not None and isinstance(value, timedelta): + return duration(value) if isinstance(value, dict): - return {k: _render(v, decimal) for k, v in value.items()} + return {k: _render(v, decimal, duration) for k, v in value.items()} if isinstance(value, (list, tuple)): - return [_render(v, decimal) for v in value] + return [_render(v, decimal, duration) for v in value] dump = getattr(value, "model_dump", None) if callable(dump): - return _render(dump(), decimal) + return _render(dump(), decimal, duration) return value -def to_wire(value: Any) -> Any: +def to_wire(value: Any, duration: DurationFormat | None = None) -> Any: """W1 over a whole response: every ``Decimal`` in *value* (a row, a list of rows, the ``{"rows", "total"}`` envelope, a Pydantic model) as its canonical string.""" - return _render(value, format_decimal) + return _render(value, format_decimal, duration) -def to_native(value: Any) -> Any: +def to_native(value: Any, duration: DurationFormat | None = None) -> Any: """The ``native`` decimal form over a whole response: every ``Decimal`` as ``str(value)``, which is what Pydantic v2 writes for one and keeps the value's scale (``Decimal("12.50")`` is ``"12.50"``). FastAPI's own encoder would turn a ``Decimal`` in a plain dict row into a ``float``, which is lossy; a string is the exact form.""" - return _render(value, str) + return _render(value, str, duration) @dataclasses.dataclass(frozen=True) @@ -291,6 +317,11 @@ class WireOutput: shapes: dict[str, Any] = dataclasses.field(default_factory=dict) #: #484: credential fields, which no response carries (the key is absent, not null). hidden: tuple[str, ...] = () + #: field.duration: how a ``timedelta`` leaves, whatever *decimals* says: the canonical + #: ISO-8601 duration under ``wire.durations: iso``, Pydantic's own form under ``native`` + #: (the router hands in ``pydantic_core.to_jsonable_python``). ``None`` for an entity with + #: no duration field. + duration: DurationFormat | None = None def render_wire(value: Any, wire: WireOutput) -> Any: @@ -300,7 +331,7 @@ def render_wire(value: Any, wire: WireOutput) -> Any: value = _drop_hidden(value, wire.hidden) if wire.null_members != "native" and wire.shapes: value = _shape_members(value, wire) - return to_wire(value) if wire.decimals == "string" else to_native(value) + return to_wire(value, wire.duration) if wire.decimals == "string" else to_native(value, wire.duration) def _drop_hidden(value: Any, hidden: tuple[str, ...]) -> Any: diff --git a/server/python/src/metaobjects/codegen/runtime/duration_wire.py b/server/python/src/metaobjects/codegen/runtime/duration_wire.py new file mode 100644 index 0000000000..6d6ba9fa9c --- /dev/null +++ b/server/python/src/metaobjects/codegen/runtime/duration_wire.py @@ -0,0 +1,243 @@ +"""The ``field.duration`` wire rule: a fixed-length span of time is an ISO-8601 duration on the +wire, and every port answers the same canonical form for it. + +One production helper, used by the run-time (the ObjectManager's read and write codec and the +validator runner), the generated models and routers, and tested against the shared vectors +(``fixtures/duration-wire/vectors.json``). Standard library only, so an owned router can emit it. + +* **Canonical** (what this port writes) -- :func:`format_duration`: + ``[-]PT[H][M][[.]S]``. Hours are never folded into days, zero components are left + out, zero is ``PT0S``, up to six fractional digits (a ``timedelta`` holds microseconds; + trailing zeros are dropped), and a negative span has one leading ``-``. +* **Accepted** (what a caller may send) -- :func:`parse_duration`: an optional sign, ``P``, + optional weeks (``W`` is exactly 7 days), optional days (``D`` is exactly 24 hours), then an + optional ``T`` with hours, minutes and seconds in that order; at least one component. The + seconds fraction may have any number of digits: the first six are kept and the rest truncated + toward zero, never rounded. Years and months are refused: a calendar span has no fixed length. +* **Range** -- this port's own limit only: a ``timedelta`` spans at most 999999999 days either + way. A duration past it is refused, never wrapped. +* **Stored micros** -- :func:`to_stored_micros` / :func:`from_stored_micros`: on SQLite, D1 and + MySQL a duration is a 64-bit integer count of microseconds (the column's limit: past it is + refused). +* **Postgres** -- pg8000 binds a ``timedelta`` natively as an ``interval`` and hands one back, + or a ``PGInterval`` when the stored interval has a year or month part (it never guesses a + month's length), which :func:`duration_from_driver` refuses. :func:`parse_postgres_interval` + reads the default-style (``IntervalStyle = postgres``) text of an ``interval``, the spelling an + interval has once Postgres has put it in a JSON value, under the same rule. +""" +from __future__ import annotations + +import datetime as _dt +import re +from typing import Any + +#: The accepted ISO-8601 duration form, byte-identical to the TS ``DURATION_PATTERN``. +DURATION_PATTERN = r"^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$" + +# ``re.ASCII`` keeps ``\d`` to the ASCII digits, as JavaScript's ``\d`` is; ``fullmatch`` is used +# throughout because a Python ``$`` also matches before a trailing newline. +DURATION_RE = re.compile(DURATION_PATTERN, re.ASCII) + +_PARTS_RE = re.compile( + r"([-+])?P(?:([0-9]+)W)?(?:([0-9]+)D)?(?:T(?:([0-9]+)H)?(?:([0-9]+)M)?(?:([0-9]+)(?:\.([0-9]+))?S)?)?", +) + +# The default-style text of an interval: signed year / mon / day parts, then an optionally +# signed ``H:MM:SS[.ffffff]``. Years and months are matched so they can be refused by name. +_PG_INTERVAL_RE = re.compile( + r"(?:([-+]?[0-9]+) years? ?)?(?:([-+]?[0-9]+) mons? ?)?(?:([-+]?[0-9]+) days? ?)?" + r"(?:([-+])?([0-9]+):([0-9]{2}):([0-9]{2})(?:\.([0-9]{1,6}))?)?", +) + +_MICROS_PER_SECOND = 1_000_000 +_MICROS_PER_MINUTE = 60 * _MICROS_PER_SECOND +_MICROS_PER_HOUR = 60 * _MICROS_PER_MINUTE +_MICROS_PER_DAY = 24 * _MICROS_PER_HOUR +_MICROS_PER_WEEK = 7 * _MICROS_PER_DAY + +#: The range of a 64-bit integer column (SQLite INTEGER, MySQL BIGINT) holding microseconds. +_INT64_MIN = -(2**63) +_INT64_MAX = 2**63 - 1 + +#: The message a value that is not a duration is refused with (the run-time validator's text). +NOT_A_DURATION = "must be an ISO-8601 duration such as PT1H30M (no years or months)" + + +def is_duration_literal(text: str) -> bool: + """True when *text* is an accepted ISO-8601 duration a ``timedelta`` can hold.""" + micros = _micros_of_literal(text) + return micros is not None and _in_range(micros) + + +def _fraction_micros(digits: str | None) -> int: + """Fractional-second digits, any number of them, as microseconds: the first six are kept + and the rest truncated toward zero (the magnitude is truncated, so a negative span is too).""" + if not digits: + return 0 + return int(digits.ljust(6, "0")[:6]) + + +def _micros_of_literal(text: str) -> int | None: + if not isinstance(text, str) or DURATION_RE.fullmatch(text) is None: + return None + m = _PARTS_RE.fullmatch(text) + if m is None: + return None + sign, w, d, h, mi, s, f = m.groups() + magnitude = ( + int(w or 0) * _MICROS_PER_WEEK + + int(d or 0) * _MICROS_PER_DAY + + int(h or 0) * _MICROS_PER_HOUR + + int(mi or 0) * _MICROS_PER_MINUTE + + int(s or 0) * _MICROS_PER_SECOND + + _fraction_micros(f) + ) + return -magnitude if sign == "-" else magnitude + + +def micros_of(value: _dt.timedelta) -> int: + """The exact microsecond count of *value*.""" + return (value.days * 86_400 + value.seconds) * _MICROS_PER_SECOND + value.microseconds + + +_MIN_MICROS = micros_of(_dt.timedelta.min) +_MAX_MICROS = micros_of(_dt.timedelta.max) + + +def _in_range(micros: int) -> bool: + return _MIN_MICROS <= micros <= _MAX_MICROS + + +def _from_micros(micros: int) -> _dt.timedelta: + """A ``timedelta`` of *micros* microseconds; past a ``timedelta``'s range is a ``ValueError`` + (refused, never wrapped).""" + if not _in_range(micros): + raise ValueError( + f"the duration {_format_micros(micros)} is out of range: a Python timedelta spans at most " + "999999999 days either way" + ) + return _dt.timedelta(microseconds=micros) + + +def to_stored_micros(value: _dt.timedelta) -> int: + """*value* as the 64-bit microsecond count a SQLite / D1 / MySQL column stores. Past the + 64-bit range (the column's limit) is a ``ValueError``.""" + micros = micros_of(value) + if not _INT64_MIN <= micros <= _INT64_MAX: + raise ValueError( + f"the duration {_format_micros(micros)} is out of range for a 64-bit microsecond column" + ) + return micros + + +def from_stored_micros(micros: int) -> _dt.timedelta: + """The ``timedelta`` a stored microsecond count is.""" + return _from_micros(micros) + + +def parse_duration(text: str) -> _dt.timedelta: + """An accepted ISO-8601 duration as a ``timedelta``. Anything else is a ``ValueError``.""" + micros = _micros_of_literal(text) + if micros is None: + raise ValueError(f"{text!r} is not an ISO-8601 duration (such as PT1H30M; no years or months)") + return _from_micros(micros) + + +def format_duration(value: _dt.timedelta) -> str: + """The canonical wire form of *value*.""" + return _format_micros(micros_of(value)) + + +def _split(micros: int) -> tuple[bool, int, int, int, int]: + negative = micros < 0 + rest = -micros if negative else micros + h, rest = divmod(rest, _MICROS_PER_HOUR) + m, rest = divmod(rest, _MICROS_PER_MINUTE) + s, frac = divmod(rest, _MICROS_PER_SECOND) + return negative, h, m, s, frac + + +def _fraction_text(frac: int) -> str: + return "." + f"{frac:06d}".rstrip("0") if frac else "" + + +def _format_micros(micros: int) -> str: + negative, h, m, s, frac = _split(micros) + out = "PT" + if h: + out += f"{h}H" + if m: + out += f"{m}M" + if s or frac: + out += f"{s}{_fraction_text(frac)}S" + if out == "PT": + return "PT0S" + return "-" + out if negative else out + + +def canonical_duration(text: str) -> str | None: + """The canonical form of an accepted ISO-8601 duration, or ``None`` when *text* is not one + (a duration past a ``timedelta``'s range included).""" + micros = _micros_of_literal(text) + return None if micros is None or not _in_range(micros) else _format_micros(micros) + + +def _not_fixed_length(shown: str) -> ValueError: + return ValueError( + f"the interval {shown} has a year or month part, so it is not a fixed-length field.duration" + ) + + +def parse_postgres_interval(text: str) -> _dt.timedelta: + """The default-style text of a Postgres interval as a ``timedelta``. An interval with a year + or month part, and text that is not an interval, is a ``ValueError``.""" + m = _PG_INTERVAL_RE.fullmatch(text) if text else None + if m is None: + raise ValueError(f"not a Postgres interval: {text!r}") + years, months, days, time_sign, h, mi, s, f = m.groups() + if int(years or 0) != 0 or int(months or 0) != 0: + raise _not_fixed_length(repr(text)) + time = 0 + if h is not None: + time = int(h) * _MICROS_PER_HOUR + int(mi) * _MICROS_PER_MINUTE + int(s) * _MICROS_PER_SECOND + time += _fraction_micros(f) + if time_sign == "-": + time = -time + return _from_micros(int(days or 0) * _MICROS_PER_DAY + time) + + +def duration_from_driver(value: Any) -> _dt.timedelta: + """What a driver hands back for a duration column, as a ``timedelta``: a ``timedelta`` + (pg8000 for an interval), the microsecond count a SQLite / MySQL integer column holds, a + canonical ISO string (a duration inside a JSON value object), the default-style text of an + interval, or a pg8000 ``PGInterval`` (an interval pg8000 could not make a ``timedelta`` of + because it has a month part, which is refused).""" + if isinstance(value, _dt.timedelta): + return value + if isinstance(value, int) and not isinstance(value, bool): + return from_stored_micros(value) + if isinstance(value, str): + micros = _micros_of_literal(value) + return _from_micros(micros) if micros is not None else parse_postgres_interval(value) + to_timedelta = getattr(value, "to_timedelta", None) + if callable(to_timedelta): # pg8000's PGInterval + if getattr(value, "millennia", None) or getattr(value, "centuries", None) or getattr( + value, "decades", None + ) or getattr(value, "years", None) or getattr(value, "months", None): + raise _not_fixed_length(str(value)) + result = to_timedelta() + if isinstance(result, _dt.timedelta): + return result + raise ValueError(f"not a field.duration value: {value!r}") + + +def parse_duration_input(value: Any) -> _dt.timedelta: + """A caller's value for a duration field: an accepted ISO-8601 string, or a ``timedelta`` + already in process. A number is refused (it is not a duration, whatever its unit), and so + is every string outside the accepted form, including the ones Pydantic's own ``timedelta`` + parser would take (``1:30:00``, ``P1Y``).""" + if isinstance(value, _dt.timedelta): + return value + if isinstance(value, str): + return parse_duration(value) + raise ValueError(f"{value!r} is not an ISO-8601 duration (such as PT1H30M; no years or months)") diff --git a/server/python/src/metaobjects/codegen/runtime/fr010_field_mapping.py b/server/python/src/metaobjects/codegen/runtime/fr010_field_mapping.py index ae2d7510b5..d5a6257bb9 100644 --- a/server/python/src/metaobjects/codegen/runtime/fr010_field_mapping.py +++ b/server/python/src/metaobjects/codegen/runtime/fr010_field_mapping.py @@ -109,6 +109,8 @@ def scalar_kind(sub_type: str) -> str | None: # ADR-0036/0037 Wave 3: uri/inet are string scalars on the wire. fc.FIELD_SUBTYPE_URI, fc.FIELD_SUBTYPE_INET, + # field.duration: an ISO-8601 duration string on the wire. + fc.FIELD_SUBTYPE_DURATION, ): return "STRING" if sub_type == fc.FIELD_SUBTYPE_INT: diff --git a/server/python/src/metaobjects/codegen/runtime/shared_predicates.py b/server/python/src/metaobjects/codegen/runtime/shared_predicates.py index 9318226f40..32e9d94f3b 100644 --- a/server/python/src/metaobjects/codegen/runtime/shared_predicates.py +++ b/server/python/src/metaobjects/codegen/runtime/shared_predicates.py @@ -207,3 +207,14 @@ def emits_writable_router(entity: MetaObject) -> bool: and not is_read_only_routed(entity) and not is_tph_subtype(entity) ) + + +#: The generated package's copy of the ``field.duration`` codec (``codegen/runtime/duration_wire.py``, +#: emitted by the entity tier beside any model with a duration). The models import their +#: validator and serializer from it, and a router its response formatter. +DURATION_WIRE_MODULE = "_duration_wire" + + +def has_duration_field(objects: list[MetaObject]) -> bool: + """True when any of *objects* has a ``field.duration`` (ADR-0039 resolving).""" + return any(f.sub_type == fc.FIELD_SUBTYPE_DURATION for o in objects for f in o.fields()) diff --git a/server/python/src/metaobjects/codegen/type_map.py b/server/python/src/metaobjects/codegen/type_map.py index 33f0f6fbe7..1b260c5682 100644 --- a/server/python/src/metaobjects/codegen/type_map.py +++ b/server/python/src/metaobjects/codegen/type_map.py @@ -39,6 +39,11 @@ class PyType: # IPv4 + IPv6 — the Python analogue of TS Zod z.string().ip()). Wire/storage # form stays a string; DB column is the Postgres-native inet type. fc.FIELD_SUBTYPE_INET: PyType("IPvAnyAddress", ("from pydantic import IPvAnyAddress",)), + # field.duration binds the native datetime.timedelta. Its wire form is an ISO-8601 + # duration (fixtures/duration-wire), which Pydantic's own timedelta handling does not + # hold to (it accepts numbers, clock text and calendar spans, and writes its own form): + # the generated model wraps this type in the duration codec (entity_model.py). + fc.FIELD_SUBTYPE_DURATION: PyType("datetime.timedelta", ("import datetime",)), } diff --git a/server/python/src/metaobjects/config/neutral_config.py b/server/python/src/metaobjects/config/neutral_config.py index 93bfd7495c..cb0e043240 100644 --- a/server/python/src/metaobjects/config/neutral_config.py +++ b/server/python/src/metaobjects/config/neutral_config.py @@ -7,7 +7,7 @@ from metaobjects.errors import ErrorCode, ParseError from metaobjects.naming import COLUMN_NAMING_STRATEGIES, TABLE_NAMING_RULES, NamingRules -from metaobjects.wire import WIRE_DECIMALS, WIRE_NULL_MEMBERS, WireFormat +from metaobjects.wire import WIRE_DECIMALS, WIRE_DURATIONS, WIRE_NULL_MEMBERS, WireFormat #: The DEFAULT value of `sources` when the key is absent or empty — never a #: requirement, and never assumed to exist by any other code path. @@ -175,6 +175,7 @@ def read_neutral_config(config_dir: Path) -> NeutralConfig | None: libraries = _read_libraries(raw, path) naming = _read_naming(raw, path) wire = _read_wire(raw, path) + _check_enums(raw, path) # Unknown top-level keys are IGNORED by design — see the module docstring. return NeutralConfig( @@ -305,12 +306,48 @@ def _read_naming(raw: dict[str, object], path: Path) -> NamingRules: return NamingRules(columns=columns, tables=tables) +#: Every key the `enums` block accepts, and the ONLY ones it accepts (refused, not ignored, like `naming`). +_ENUMS_KEYS = ("maxLength",) + + +def _check_enums(raw: dict[str, object], path: Path) -> None: + """The `enums` block: `maxLength`, a positive integer, is the column length a port that + sizes a string-stored enum's column gives one with no @maxLength. This port sizes no + column (it emits no DDL and no column type), so it reads nothing from the block, but a + project shared with a port that does has one: the block is accepted, and its shape is + checked so a typo is refused here as it is there (a non-object block, an unknown key, or + a value that is not a positive integer).""" + if "enums" not in raw: + return + enums = raw["enums"] + if not isinstance(enums, dict): + raise ParseError(f"{path}: 'enums' must be an object", code=ErrorCode.ERR_BAD_ATTR_VALUE) + unknown = sorted(set(enums) - set(_ENUMS_KEYS)) + if unknown: + raise ParseError( + f"{path}: 'enums' has unknown key(s) {unknown}; accepted: {list(_ENUMS_KEYS)}", + code=ErrorCode.ERR_BAD_ATTR_VALUE, + ) + if "maxLength" not in enums: + return + value = enums["maxLength"] + # A whole number however JSON spells it (32, 32.0, 1e2): a reader that parses numbers as + # doubles (TypeScript's) cannot tell those apart, so every port accepts all three. + if isinstance(value, float) and value.is_integer(): + value = int(value) + if isinstance(value, bool) or not isinstance(value, int) or value <= 0: + raise ParseError( + f"{path}: 'enums.maxLength' must be a positive integer, got {value!r}", + code=ErrorCode.ERR_BAD_ATTR_VALUE, + ) + + #: Every key the `wire` block accepts, and the ONLY ones it accepts (refused, not ignored, like `naming`). -_WIRE_KEYS = ("decimals", "nullMembers") +_WIRE_KEYS = ("decimals", "nullMembers", "durations") def _read_wire(raw: dict[str, object], path: Path) -> WireFormat: - """ADR-0019 Amendment 2 — the `wire` block. Absent or `{}` ⇒ `native` for both keys. + """ADR-0019 Amendment 2 — the `wire` block. Absent or `{}` ⇒ `native` for every key. Refuses a non-object `wire`, an unknown key, or a value outside the closed set a key accepts: a mistake must not silently fall back to the default and change the JSON an API writes.""" @@ -322,11 +359,11 @@ def _read_wire(raw: dict[str, object], path: Path) -> WireFormat: for key in wire: if key not in _WIRE_KEYS: raise ParseError( - f'{path}: "wire" has an unknown key "{key}" (expected "decimals" and/or "nullMembers")', + f'{path}: "wire" has an unknown key "{key}" (expected "decimals", "nullMembers" and/or "durations")', code=ErrorCode.ERR_COLLECTION_NOT_FOUND, ) values: dict[str, str] = {} - for key, allowed in (("decimals", WIRE_DECIMALS), ("nullMembers", WIRE_NULL_MEMBERS)): + for key, allowed in (("decimals", WIRE_DECIMALS), ("nullMembers", WIRE_NULL_MEMBERS), ("durations", WIRE_DURATIONS)): value = wire.get(key, allowed[0]) if not isinstance(value, str) or value not in allowed: choices = ", ".join(f'"{a}"' for a in allowed) @@ -335,7 +372,7 @@ def _read_wire(raw: dict[str, object], path: Path) -> WireFormat: code=ErrorCode.ERR_COLLECTION_NOT_FOUND, ) values[key] = value - return WireFormat(decimals=values["decimals"], null_members=values["nullMembers"]) + return WireFormat(decimals=values["decimals"], null_members=values["nullMembers"], durations=values["durations"]) def _validate_dependency_spec(dep: object, path: Path) -> dict[str, str]: diff --git a/server/python/src/metaobjects/core_types.py b/server/python/src/metaobjects/core_types.py index 5a00288a48..f08b8d23f4 100644 --- a/server/python/src/metaobjects/core_types.py +++ b/server/python/src/metaobjects/core_types.py @@ -1078,6 +1078,8 @@ def _register_subtypes( AttrSchema(name=vc.VALIDATOR_ATTR_MIN, value_type=ATTR_SUBTYPE_INT), AttrSchema(name=vc.VALIDATOR_ATTR_MAX, value_type=ATTR_SUBTYPE_INT), ] +# A cross-field validator's optional @condition (an expression over the entity's fields). +_VALIDATOR_CONDITION_ATTR = AttrSchema(name=vc.VALIDATOR_ATTR_CONDITION, value_type=ATTR_SUBTYPE_EXPRESSION) _VALIDATOR_ATTRS_BY_SUBTYPE: dict[str, list[AttrSchema]] = { SUBTYPE_BASE: list(_VALIDATOR_MIN_MAX_ATTRS), vc.VALIDATOR_SUBTYPE_REQUIRED: [], @@ -1094,19 +1096,27 @@ def _register_subtypes( AttrSchema(name=vc.VALIDATOR_ATTR_OP, value_type=ATTR_SUBTYPE_STRING, required=True, allowed_values=("gt", "gte", "lt", "lte", "ne", "eq")), AttrSchema(name=vc.VALIDATOR_ATTR_RIGHT, value_type=ATTR_SUBTYPE_STRING, required=True), + _VALIDATOR_CONDITION_ATTR, ], vc.VALIDATOR_SUBTYPE_REQUIRED_WHEN: [ AttrSchema(name=vc.VALIDATOR_ATTR_FIELD, value_type=ATTR_SUBTYPE_STRING, required=True), AttrSchema(name=vc.VALIDATOR_ATTR_WHEN, value_type=ATTR_SUBTYPE_STRING, required=True), AttrSchema(name=vc.VALIDATOR_ATTR_EQUALS, value_type=ATTR_SUBTYPE_STRING, required=True), + _VALIDATOR_CONDITION_ATTR, ], vc.VALIDATOR_SUBTYPE_PRESENT_IFF: [ AttrSchema(name=vc.VALIDATOR_ATTR_FIELD, value_type=ATTR_SUBTYPE_STRING, required=True), AttrSchema(name=vc.VALIDATOR_ATTR_WHEN, value_type=ATTR_SUBTYPE_STRING, required=True), AttrSchema(name=vc.VALIDATOR_ATTR_EQUALS, value_type=ATTR_SUBTYPE_STRING, required=True), + _VALIDATOR_CONDITION_ATTR, ], vc.VALIDATOR_SUBTYPE_AT_LEAST_ONE: [ AttrSchema(name=vc.VALIDATOR_ATTR_FIELDS, value_type=ATTR_SUBTYPE_STRING, required=True, is_array=True), + _VALIDATOR_CONDITION_ATTR, + ], + vc.VALIDATOR_SUBTYPE_EXACTLY_ONE: [ + AttrSchema(name=vc.VALIDATOR_ATTR_FIELDS, value_type=ATTR_SUBTYPE_STRING, required=True, is_array=True), + _VALIDATOR_CONDITION_ATTR, ], } for _sub, _attrs in _VALIDATOR_ATTRS_BY_SUBTYPE.items(): diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index eac4041d32..3fdcb7d610 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -208,6 +208,13 @@ class ErrorCode(str, Enum): # folds `userId` and `userID` together, or two fields share an @column). ERR_COLUMN_NAME_COLLISION = "ERR_COLUMN_NAME_COLLISION" ERR_STORAGE_FLATTENED_ARRAY = "ERR_STORAGE_FLATTENED_ARRAY" + ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH = "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH" + ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP = "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP" + ERR_SEQUENCE_WITHOUT_INCREMENT = "ERR_SEQUENCE_WITHOUT_INCREMENT" + ERR_SEQUENCE_START_OUT_OF_RANGE = "ERR_SEQUENCE_START_OUT_OF_RANGE" + ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE = ( + "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE" + ) ERR_STORAGE_WITHOUT_OBJECT_REF = "ERR_STORAGE_WITHOUT_OBJECT_REF" # ADR-0013: a field.object REQUIRES @objectRef (open/untyped JSON uses the # physical @dbColumnType: jsonb escape hatch on field.string). diff --git a/server/python/src/metaobjects/loader/merge.py b/server/python/src/metaobjects/loader/merge.py index 95a61dd231..bc761c60dc 100644 --- a/server/python/src/metaobjects/loader/merge.py +++ b/server/python/src/metaobjects/loader/merge.py @@ -32,6 +32,7 @@ from typing import Optional from ..errors import ErrorCode, MetaError +from ..meta.core.field.field_constants import FIELD_ATTR_FLATTENED_COLUMN_PREFIX from ..meta.meta_data import MetaData from ..serializer_json import canonical_serialize from ..shared.separators import PACKAGE_SEP @@ -273,14 +274,19 @@ def _build_contributors(files: tuple[str, ...]) -> tuple[Contributor, ...]: ) -def _is_empty_value(v: object) -> bool: +#: Attrs whose empty string is a meaningful value rather than "not set" (FR5c). +_EMPTY_STRING_IS_A_VALUE: frozenset[str] = frozenset({FIELD_ATTR_FLATTENED_COLUMN_PREFIX}) + + +def _is_empty_value(v: object, attr_name: str) -> bool: """Mirror TS ``isEmptyValue``: ``None``, ``""``, and ``[]`` are empty; - everything else is set. + everything else is set. ``@flattenedColumnPrefix: ""`` is a value (keep the + members' own columns), so ``""`` against ``"home_"`` is a conflict. """ if v is None: return True if isinstance(v, str) and v == "": - return True + return attr_name not in _EMPTY_STRING_IS_A_VALUE if isinstance(v, list) and not v: return True return False @@ -333,7 +339,7 @@ def _detect_attr_merge_conflicts( existing_val = pre_attrs[attr_name] new_val = getattr(src_attr, "value", None) - if _is_empty_value(new_val) or _is_empty_value(existing_val): + if _is_empty_value(new_val, attr_name) or _is_empty_value(existing_val, attr_name): continue if _attr_values_equal(existing_val, new_val): continue diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index bb2c548cee..754e6605c5 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -7,6 +7,7 @@ """ from __future__ import annotations +import json import math import re from typing import Callable, NamedTuple @@ -23,14 +24,18 @@ from .validate_source_parameter_ref import validate_source_parameter_ref from .validate_source_escapes import validate_source_escapes from .validate_reporting import validate_reporting +from ..codegen.runtime.duration_wire import DURATION_RE from ..meta.core.field.field_constants import ( ENUM_MEMBER_PATTERN, FIELD_ATTR_COERCE_DEFAULT, FIELD_ATTR_DEFAULT, FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_REQUIRED, FIELD_ATTR_SORTABLE, + FIELD_ATTR_FLATTENED_COLUMN_PREFIX, + STORAGE_FLATTENED, FIELD_ATTR_STORAGE, FIELD_ATTR_VALUE_TYPE, FIELD_ATTR_VALUES, @@ -44,6 +49,7 @@ FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_STRING, @@ -85,6 +91,7 @@ TYPE_REQUIREMENT, TYPE_SOURCE, TYPE_TEMPLATE, + TYPE_VALIDATOR, ) from ..meta.template import template_constants as tc from ..meta.presentation.layout.layout_constants import ( @@ -247,6 +254,12 @@ def run_validations( # FR-015 — source.rdb @parameterRef typed-input rules. validate_source_parameter_ref(root, errors) _validate_field_object_storage(root, errors) + # field.enum @maxLength — every member fits; refused with @intValueMap. + _validate_enum_max_length(root, errors) + # identity.primary @sequenceStart / @sequenceName need @generation "increment". + _validate_identity_sequence(root, errors) + # cross-field validators — exactlyOne names two or more fields; @condition names entity fields. + _validate_cross_field_validators(root, errors) # field.map — exactly one of @valueType (scalar) or @objectRef (VO), and # @valueType (when set) must name a known scalar subtype. _validate_field_map(root, errors) @@ -1097,6 +1110,10 @@ def ops_for_subtype(field_subtype: str) -> frozenset[str]: return _OPS_STRING if field_subtype in ("uuid", FIELD_SUBTYPE_INET): return _OPS_UUID + # field.duration is an orderable span: the numeric/temporal band. Every engine orders it + # (a Postgres interval; a microsecond integer on SQLite, D1 and MySQL). + if field_subtype == FIELD_SUBTYPE_DURATION: + return _OPS_NUMERIC_TEMPORAL if field_subtype == "boolean": return _OPS_BOOLEAN if field_subtype in _NUMERIC_TEMPORAL_SUBTYPES: @@ -3711,6 +3728,307 @@ def _validate_sortable_has_supported_subtype( # skipped when @objectRef is absent). # 2. @storage="flattened" + isArray → ERR_STORAGE_FLATTENED_ARRAY (flattened # materialises one-column-per-field; arrays require @storage="jsonb"). +# 3. @flattenedColumnPrefix without @storage="flattened" → +# ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE (no other storage gives the +# members columns of their own to prefix). + + +# identity.primary @sequenceStart / @sequenceName describe the sequence an @generation +# "increment" key draws from; on any other generation there is none, so either one is refused +# (ERR_SEQUENCE_WITHOUT_INCREMENT). Read RESOLVING (ADR-0039), reported on the identity that +# declares one of them. Mirrors TS validateIdentitySequence. +#: The largest value a field.int key (a 32-bit integer column) can hold. +_SEQUENCE_INT_MAX = 2147483647 + + +def _validate_identity_sequence(root: MetaData, errors: list[MetaError]) -> None: + from ..meta.core.identity.identity_constants import ( + GENERATION_INCREMENT, + IDENTITY_ATTR_GENERATION, + IDENTITY_SUBTYPE_PRIMARY, + ) + from ..meta.persistence.db.db_constants import ( + IDENTITY_PRIMARY_ATTR_SEQUENCE_NAME, + IDENTITY_PRIMARY_ATTR_SEQUENCE_START, + ) + + for obj in root.children(): + if obj.type != TYPE_OBJECT: + continue + for ident in obj.own_children(): + if ident.type != TYPE_IDENTITY or ident.sub_type != IDENTITY_SUBTYPE_PRIMARY: + continue + declared = [ + f"@{a}" + for a in (IDENTITY_PRIMARY_ATTR_SEQUENCE_START, IDENTITY_PRIMARY_ATTR_SEQUENCE_NAME) + if ident.attr(a) is not None + ] + generation = ident.get_meta_attr(IDENTITY_ATTR_GENERATION) + if not declared: + continue + if generation == GENERATION_INCREMENT: + # The key field's own range bounds the start (Postgres refuses the sequence + # otherwise, at apply time): 1 to the largest value of a field.int / field.long. + start = ident.get_meta_attr(IDENTITY_PRIMARY_ATTR_SEQUENCE_START) + if isinstance(start, int) and not isinstance(start, bool): + fields = ident.get_meta_attr(IDENTITY_ATTR_FIELDS) + key_name = fields[0] if isinstance(fields, list) and fields else fields + key = next((f for f in obj.fields() if f.name == key_name), None) + maximum = _SEQUENCE_INT_MAX if key is not None and key.sub_type == FIELD_SUBTYPE_INT else None + if start < 1 or (maximum is not None and start > maximum): + kind = f" (field.{key.sub_type})" if key is not None else "" + upper = f" and at most {maximum}" if maximum is not None else "" + errors.append(MetaError( + code=ErrorCode.ERR_SEQUENCE_START_OUT_OF_RANGE, + message=( + f'identity.primary "{obj.name}.{ident.name}" @sequenceStart {start} is out ' + f'of range for its key "{key_name}"{kind}: it must be at least 1{upper}.' + ), + envelope=ident.source, + )) + continue + shown = f'"{generation}"' if isinstance(generation, str) else "not set" + names = " and ".join(declared) + errors.append(MetaError( + code=ErrorCode.ERR_SEQUENCE_WITHOUT_INCREMENT, + message=( + f'identity.primary "{obj.name}.{ident.name}" sets {names}, but its @generation ' + f'is {shown}; only an @generation "increment" key draws from a sequence. Set ' + f'@generation "increment", or remove {names}.' + ), + envelope=ident.source, + )) + + +# Cross-field validators (comparison / requiredWhen / presentIff / atLeastOne / exactlyOne), which +# the database enforces as CHECKs: exactlyOne @fields names two or more fields, +# every field a @condition names a field of the owning entity, a requiredWhen / presentIff +# @equals gating a field.duration an ISO-8601 duration, and a @condition that can +# become a CHECK guard (see _condition_problem) — all ERR_BAD_ATTR_VALUE. Read RESOLVING +# (ADR-0039). Mirrors TS validateCrossFieldValidators. +def _expr_field_refs(node: object, out: set[str]) -> None: + if not isinstance(node, dict): + return + if isinstance(node.get("field"), str): + out.add(node["field"]) + for key in ("left", "right", "arg"): + _expr_field_refs(node.get(key), out) + args = node.get("args") + if isinstance(args, list): + for a in args: + _expr_field_refs(a, out) + + +# What a literal compared with a field.duration must be. +_DURATION_LITERAL_FORM = "an ISO-8601 duration (such as PT1H30M; no years or months)" + +# Field subtypes whose values a condition compares with a number literal. +_CONDITION_NUMBER_SUBTYPES = frozenset({ + FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, + FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_CURRENCY, +}) + + +def _json_literal(value: object) -> str: + """A literal as JSON.stringify writes it, so the message matches the TS port's.""" + if isinstance(value, float) and value.is_integer(): + value = int(value) + return json.dumps(value, ensure_ascii=False, separators=(",", ":")) + + +def _literal_is_kind(value: object, kind: str) -> bool: + # bool is a subclass of int in Python: a bool literal is not a number, nor a number a bool. + if kind == "boolean": + return isinstance(value, bool) + if kind == "number": + return isinstance(value, (int, float)) and not isinstance(value, bool) + return isinstance(value, str) + + +def _condition_problem(condition: object, obj: MetaData) -> str | None: + """Why a cross-field validator's ``@condition`` cannot become a CHECK guard, or None when + it can. Its names are already known to be fields. The rule, the same in every port: + - it types cleanly (the origin.computed inference) and its root is boolean, since a + CHECK negates it; + - a literal compared with a field is of that field's kind (a boolean for field.boolean, + a number for a numeric field, a string otherwise) and, for field.enum, one of its + @values, for field.duration, an ISO-8601 duration; + - no comparison has a null literal: ``x = NULL`` is never true, so use isNull / isNotNull. + Mirrors TS conditionProblem.""" + fields = {f.name: f for f in obj.fields()} + inferred = infer_expr_type( + condition, lambda name: fields[name].sub_type if name in fields else None + ) + if inferred.errors: + return f"is not a valid expression: {'; '.join(inferred.errors)}." + if inferred.type != FIELD_SUBTYPE_BOOLEAN: + return ( + "must be a boolean expression (a comparison, a null test, or and / or / not of " + f"them), not a field.{inferred.type}." + ) + + def visit(node: object) -> str | None: + if not isinstance(node, dict): + return None + if node.get("left") is not None and node.get("right") is not None: + for lit, ref in ((node["left"], node["right"]), (node["right"], node["left"])): + if not isinstance(lit, dict) or "value" not in lit: + continue + value = lit["value"] + if value is None: + return "compares with null, which is never true: use isNull or isNotNull." + name = ref.get("field") if isinstance(ref, dict) else None + field = fields.get(name) if isinstance(name, str) else None + if field is None: + continue + sub = field.sub_type + kind = ( + "boolean" if sub == FIELD_SUBTYPE_BOOLEAN + else "number" if sub in _CONDITION_NUMBER_SUBTYPES + else "string" + ) + if not _literal_is_kind(value, kind): + return ( + f'compares "{field.name}" (field.{sub}) with {_json_literal(value)}; ' + f"it needs a {kind}." + ) + members = field.get_meta_attr(FIELD_ATTR_VALUES) if sub == FIELD_SUBTYPE_ENUM else None + if isinstance(members, list) and str(value) not in [str(m) for m in members]: + return ( + f'compares "{field.name}" with {_json_literal(value)}, which is not one ' + f"of its @values ({', '.join(str(m) for m in members)})." + ) + if sub == FIELD_SUBTYPE_DURATION and DURATION_RE.fullmatch(value) is None: + return ( + f'compares "{field.name}" with {_json_literal(value)}, which is not ' + f"{_DURATION_LITERAL_FORM}." + ) + for key in ("left", "right", "arg"): + found = visit(node.get(key)) + if found is not None: + return found + args = node.get("args") + if isinstance(args, list): + for a in args: + found = visit(a) + if found is not None: + return found + return None + + return visit(condition) + + +def _validate_cross_field_validators(root: MetaData, errors: list[MetaError]) -> None: + from ..meta.core.validator import validator_constants as vc + + for obj in root.children(): + if obj.type != TYPE_OBJECT: + continue + field_names = {f.name for f in obj.fields()} + for v in obj.own_children(): + if v.type != TYPE_VALIDATOR or v.sub_type not in vc.CROSS_FIELD_VALIDATOR_SUBTYPES: + continue + if v.sub_type == vc.VALIDATOR_SUBTYPE_EXACTLY_ONE: + fields = v.get_meta_attr(vc.VALIDATOR_ATTR_FIELDS) + count = len(fields) if isinstance(fields, list) else (0 if fields is None else 1) + if count < 2: + plural = "" if count == 1 else "s" + errors.append(MetaError( + code=ErrorCode.ERR_BAD_ATTR_VALUE, + message=( + f'validator.exactlyOne "{obj.name}.{v.name}" @fields names {count} ' + f"field{plural}; it needs two or more. For one field, make that field required." + ), + envelope=v.source, + )) + continue + if v.sub_type in (vc.VALIDATOR_SUBTYPE_REQUIRED_WHEN, vc.VALIDATOR_SUBTYPE_PRESENT_IFF): + when_name = v.get_meta_attr(vc.VALIDATOR_ATTR_WHEN) + when = next((f for f in obj.fields() if f.name == when_name), None) + equals = v.get_meta_attr(vc.VALIDATOR_ATTR_EQUALS) + if when is not None and when.sub_type == FIELD_SUBTYPE_DURATION and not ( + isinstance(equals, str) and DURATION_RE.fullmatch(equals) + ): + errors.append(MetaError( + code=ErrorCode.ERR_BAD_ATTR_VALUE, + message=( + f'validator.{v.sub_type} "{obj.name}.{v.name}" @equals {_json_literal(equals)} ' + f'gates field.duration "{when.name}"; it needs {_DURATION_LITERAL_FORM}.' + ), + envelope=v.source, + )) + continue + condition = v.get_meta_attr(vc.VALIDATOR_ATTR_CONDITION) + if condition is None: + continue + label = f'validator.{v.sub_type} "{obj.name}.{v.name}" @condition' + refs: set[str] = set() + _expr_field_refs(condition, refs) + unknown = sorted(refs - field_names) + if unknown: + names = ", ".join(f'"{u}"' for u in unknown) + verb = "is not a field" if len(unknown) == 1 else "are not fields" + errors.append(MetaError( + code=ErrorCode.ERR_BAD_ATTR_VALUE, + message=f'{label} names {names}, which {verb} of "{obj.name}".', + envelope=v.source, + )) + continue + problem = _condition_problem(condition, obj) + if problem is not None: + errors.append(MetaError( + code=ErrorCode.ERR_BAD_ATTR_VALUE, + message=f"{label} {problem}", + envelope=v.source, + )) + + +# field.enum @maxLength — the column length of a string-stored enum: +# 1. every member of @values fits in it (ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH), the longest +# offender named; +# 2. it is refused beside @intValueMap (ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP). +# Read RESOLVING (ADR-0039), reported on the node that DECLARES one of the three, so an enum +# that only inherits them is not reported twice. Mirrors TS validateEnumMaxLength. +def _validate_enum_max_length(root: MetaData, errors: list[MetaError]) -> None: + for node in _walk(root): + if node.type != TYPE_FIELD or node.sub_type != FIELD_SUBTYPE_ENUM: + continue + declares = any( + node.attr(a) is not None + for a in (FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP) + ) + max_length = node.get_meta_attr(FIELD_ATTR_MAX_LENGTH) # ADR-0039 resolving + if not declares or not isinstance(max_length, int) or isinstance(max_length, bool): + continue + owner = node.parent + label = f"{owner.name}.{node.name}" if owner is not None and owner.type == TYPE_OBJECT else node.name + if node.get_meta_attr(FIELD_ATTR_INT_VALUE_MAP) is not None: + errors.append(MetaError( + code=ErrorCode.ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP, + message=( + f'field.enum "{label}" sets @maxLength {max_length} and @intValueMap; an ' + f"@intValueMap enum is stored as an integer, so a column length does not " + f"apply. Remove @maxLength." + ), + envelope=node.source, + )) + continue + values = node.get_meta_attr(FIELD_ATTR_VALUES) + too_long = sorted( + (v for v in (values if isinstance(values, list) else []) if isinstance(v, str) and len(v) > max_length), + key=len, + reverse=True, + ) + if too_long: + errors.append(MetaError( + code=ErrorCode.ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, + message=( + f'field.enum "{label}" @maxLength {max_length} is shorter than its member ' + f'"{too_long[0]}" ({len(too_long[0])} characters). Raise @maxLength to at ' + f"least {len(too_long[0])}, or shorten the member." + ), + envelope=node.source, + )) def _validate_field_object_storage(root: MetaData, errors: list[MetaError]) -> None: @@ -3737,9 +4055,33 @@ def _validate_field_object_storage(root: MetaData, errors: list[MetaError]) -> N )) continue storage = node.get_meta_attr(FIELD_ATTR_STORAGE) # ADR-0039 resolving + # @flattenedColumnPrefix names the columns a flattened value object adds to the + # owning table; under any other storage it would change nothing, so it is refused + # rather than silently ignored. + # Only a field an object owns is checked (as TS and C# do): a root-level abstract + # field.object is a template, and the concrete field that extends it — resolving + # both the prefix and @storage — is the one checked. + owner = node.parent + if ( + owner is not None + and owner.type == TYPE_OBJECT + and node.get_meta_attr(FIELD_ATTR_FLATTENED_COLUMN_PREFIX) is not None + and storage != STORAGE_FLATTENED + ): + storage_text = "not set (jsonb)" if storage is None else f'"{storage}"' + errors.append(MetaError( + code=ErrorCode.ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE, + message=( + f'field.object "{owner.name}.{node.name}" sets @flattenedColumnPrefix, but its @storage ' + f"is {storage_text}; only a @storage \"flattened\" value object puts its " + f"members' columns in the owning table. Set @storage \"flattened\", or " + f"remove @flattenedColumnPrefix." + ), + envelope=node.source, + )) if storage is None: continue - if storage == "flattened" and node.resolved_is_array(): # ADR-0039 resolving isArray + if storage == STORAGE_FLATTENED and node.resolved_is_array(): # ADR-0039 resolving isArray errors.append(MetaError( code=ErrorCode.ERR_STORAGE_FLATTENED_ARRAY, message=( @@ -4205,6 +4547,46 @@ def _has_explicit_column(field: MetaData) -> bool: return isinstance(col, str) and col != "" +class _PhysicalColumn(NamedTuple): + """One physical column a field contributes to its object's table.""" + + column: str + #: ``street``, or ``shipTo.street`` for a member of a flattened value object. + label: str + #: The field that names it: the field itself, or the flattened value object's member. + named: MetaData + #: Set for a flattened member: the embedding field.object. + embedding: MetaData | None = None + + +def _physical_columns_of(field: MetaData) -> list[_PhysicalColumn]: + from ..naming import resolve_column_name, resolve_flattened_column_name + # ADR-0039: resolving -- @storage and @objectRef may be inherited via extends. + if field.sub_type == FIELD_SUBTYPE_OBJECT and field.get_meta_attr(FIELD_ATTR_STORAGE) == STORAGE_FLATTENED: + target = bound_ref_target(field, FIELD_ATTR_OBJECT_REF) + if not isinstance(target, MetaObject): + return [] + # Every member, as migrate's flattenObjectField expands it. + return [ + _PhysicalColumn( + resolve_flattened_column_name(field, member), + f"{field.name}.{member.name}", + member, + field, + ) + for member in target.fields() + ] + return [_PhysicalColumn(resolve_column_name(field), field.name, field)] + + +def _collision_hint(a: _PhysicalColumn, b: _PhysicalColumn) -> str: + if a.embedding is not None or b.embedding is not None: + return "Give one flattened field a different @flattenedColumnPrefix, or rename one field." + if _has_explicit_column(a.named) and _has_explicit_column(b.named): + return "Rename one field, or point one @column at a different column." + return "Rename one field, or give one an explicit @column so the two columns differ." + + def _validate_column_name_collisions(root: MetaData, errors: list[MetaError]) -> None: """ERR_COLUMN_NAME_COLLISION: two fields of one stored object resolving to the SAME physical column (``userId`` + ``userID`` both snake_case to ``user_id``, or two @@ -4213,8 +4595,11 @@ def _validate_column_name_collisions(root: MetaData, errors: list[MetaError]) -> Only the RESOLVED name is compared, through :func:`resolve_column_name`, so an explicit ``@column``, an inherited field and the project's column strategy are honoured. Scope: a concrete object declaring at least one source (a source-less - object generates nothing); derived fields and ``field.object`` (may flatten to - several columns) are skipped. ADR-0039: effective children (inherited fields count). + object generates nothing); derived fields are skipped. A ``field.object @storage + "flattened"`` has no column of its own: each member of its value object is a column, + named by :func:`resolve_flattened_column_name` exactly as migrate creates it. Any + other ``field.object`` is one column under its own resolved name. + ADR-0039: effective children (inherited fields count). Exempt: an object whose every source is read-only and at least one is a ``source.rdb @kind: view`` with an ``@sql`` body. The author's SQL defines that view's @@ -4223,7 +4608,6 @@ def _validate_column_name_collisions(root: MetaData, errors: list[MetaError]) -> from its fields, and an object with a writable source has a real table whose columns MetaObjects creates; both stay checked. """ - from ..naming import resolve_column_name from ..meta.core.field.meta_field import MetaField for obj in (c for c in root.children() if c.type == TYPE_OBJECT): if obj.is_abstract: @@ -4236,29 +4620,23 @@ def _validate_column_name_collisions(root: MetaData, errors: list[MetaError]) -> for s in sources ): continue - seen: dict[str, MetaData] = {} + seen: dict[str, _PhysicalColumn] = {} for field in obj.children(): - if not isinstance(field, MetaField) or field.sub_type == FIELD_SUBTYPE_OBJECT \ - or field.is_derived(): - continue - column = resolve_column_name(field) - first = seen.get(column) - if first is None: - seen[column] = field + if not isinstance(field, MetaField) or field.is_derived(): continue - errors.append(MetaError( - code=ErrorCode.ERR_COLUMN_NAME_COLLISION, - message=( - f'fields "{first.name}" and "{field.name}" on "{obj.name}" both resolve ' - f'to physical column "{column}". ' - + ( - "Rename one field, or point one @column at a different column." - if _has_explicit_column(first) and _has_explicit_column(field) - else "Rename one field, or give one an explicit @column so the two columns differ." - ) - ), - envelope=field.source, - )) + for col in _physical_columns_of(field): + first = seen.get(col.column) + if first is None: + seen[col.column] = col + continue + errors.append(MetaError( + code=ErrorCode.ERR_COLUMN_NAME_COLLISION, + message=( + f'fields "{first.label}" and "{col.label}" on "{obj.name}" both resolve ' + f'to physical column "{col.column}". ' + _collision_hint(first, col) + ), + envelope=field.source, + )) def _validate_retired_requirement_links(root: MetaData, errors: list[MetaError]) -> None: diff --git a/server/python/src/metaobjects/meta/core/field/field_constants.py b/server/python/src/metaobjects/meta/core/field/field_constants.py index 98f1e6d337..6cf388650b 100644 --- a/server/python/src/metaobjects/meta/core/field/field_constants.py +++ b/server/python/src/metaobjects/meta/core/field/field_constants.py @@ -35,6 +35,14 @@ # build-time codegen concern. DB column is the Postgres-native inet type. Ships # WITHOUT @kind in v1 (the native type subsumes ipv4/ipv6). Mirrors TS FIELD_SUBTYPE_INET. FIELD_SUBTYPE_INET = "inet" +# field.duration: a fixed-length span of time (days, hours, minutes, seconds). A +# concept WITH a native type + behavior → a subtype (the field.inet precedent). +# String-backed on the wire: an ISO-8601 duration, written in the canonical form +# ``[-]PT[nH][nM][n[.f]S]`` (fixtures/duration-wire). The native binding is +# ``datetime.timedelta``; DB column is the Postgres-native interval; SQLite and MySQL +# store a 64-bit integer count of microseconds. +# Calendar spans (years, months) are refused; a week is exactly 7 days. Mirrors TS FIELD_SUBTYPE_DURATION. +FIELD_SUBTYPE_DURATION = "duration" FIELD_SUBTYPES = ( SUBTYPE_BASE, @@ -54,6 +62,7 @@ FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, # Note: FIELD_SUBTYPE_ENUM is intentionally excluded here; it is registered # separately in core_types.py with its dedicated @values AttrSchema. ) @@ -129,9 +138,15 @@ NORMALIZE_MODES = (NORMALIZE_NONE, NORMALIZE_COLLAPSE, NORMALIZE_STRIP) # Persistence-side storage shape for owned field.object data. Cross-port values. FIELD_ATTR_STORAGE = "storage" -STORAGE_VALUES = ("flattened", "jsonb", "subdocument") +#: @storage "flattened": the nested object's columns expand into the parent table. +STORAGE_FLATTENED = "flattened" +STORAGE_VALUES = (STORAGE_FLATTENED, "jsonb", "subdocument") # Physical column name override (cross-port; renamed from @dbColumn). FIELD_ATTR_COLUMN = "column" +# @flattenedColumnPrefix on a field.object @storage "flattened": the text before each member's column, +# verbatim ("" allowed). Absent: the field's resolved column and "_" +# (metaobjects.naming.resolve_flattened_column_prefix). +FIELD_ATTR_FLATTENED_COLUMN_PREFIX = "flattenedColumnPrefix" # ADR-0036/0037 Wave 3 — @stringFormat: a closed validation format for a plain # field.string that has NO native type or behavior of its own (email | hostname). diff --git a/server/python/src/metaobjects/meta/core/field/meta_field.py b/server/python/src/metaobjects/meta/core/field/meta_field.py index 8f210c23e7..dd8ab1ed37 100644 --- a/server/python/src/metaobjects/meta/core/field/meta_field.py +++ b/server/python/src/metaobjects/meta/core/field/meta_field.py @@ -31,6 +31,9 @@ # are surfaced at codegen time (see codegen/type_map.py). fc.FIELD_SUBTYPE_URI: DataType.STRING, fc.FIELD_SUBTYPE_INET: DataType.STRING, + # field.duration is string-backed on the wire (an ISO-8601 duration); the native + # datetime.timedelta binding is surfaced at codegen time and by the runtime codec. + fc.FIELD_SUBTYPE_DURATION: DataType.STRING, } diff --git a/server/python/src/metaobjects/meta/core/validator/validator_constants.py b/server/python/src/metaobjects/meta/core/validator/validator_constants.py index 5c16eae8b2..522f802ea3 100644 --- a/server/python/src/metaobjects/meta/core/validator/validator_constants.py +++ b/server/python/src/metaobjects/meta/core/validator/validator_constants.py @@ -16,6 +16,7 @@ VALIDATOR_SUBTYPE_REQUIRED_WHEN = "requiredWhen" VALIDATOR_SUBTYPE_PRESENT_IFF = "presentIff" VALIDATOR_SUBTYPE_AT_LEAST_ONE = "atLeastOne" +VALIDATOR_SUBTYPE_EXACTLY_ONE = "exactlyOne" # Validator attr keys (read by codegen when lowering validator children). VALIDATOR_ATTR_MIN = "min" @@ -29,3 +30,15 @@ VALIDATOR_ATTR_WHEN = "when" VALIDATOR_ATTR_EQUALS = "equals" VALIDATOR_ATTR_FIELDS = "fields" +# A cross-field validator's @condition: the rule applies only where this expression holds. +VALIDATOR_ATTR_CONDITION = "condition" +# The database name of the CHECK a CHECK-generating validator generates (db provider). +VALIDATOR_ATTR_CONSTRAINT_NAME = "constraintName" +# The cross-field validators (entity-scoped; the database enforces them as CHECKs). +CROSS_FIELD_VALIDATOR_SUBTYPES = ( + VALIDATOR_SUBTYPE_COMPARISON, + VALIDATOR_SUBTYPE_REQUIRED_WHEN, + VALIDATOR_SUBTYPE_PRESENT_IFF, + VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_EXACTLY_ONE, +) diff --git a/server/python/src/metaobjects/meta/persistence/db/db_constants.py b/server/python/src/metaobjects/meta/persistence/db/db_constants.py index 968978fb64..6fc5ee76f2 100644 --- a/server/python/src/metaobjects/meta/persistence/db/db_constants.py +++ b/server/python/src/metaobjects/meta/persistence/db/db_constants.py @@ -76,6 +76,10 @@ IDENTITY_SECONDARY_ATTR_USING = "using" # identity.reference: physical FK constraint-name override. IDENTITY_REFERENCE_ATTR_CONSTRAINT_NAME = "constraintName" +#: identity.primary (@generation "increment"): the sequence's first value. Absent → 1. +IDENTITY_PRIMARY_ATTR_SEQUENCE_START = "sequenceStart" +#: identity.primary (@generation "increment"): the sequence's physical name. Absent → the database's default. +IDENTITY_PRIMARY_ATTR_SEQUENCE_NAME = "sequenceName" # identity.reference: referential action on parent delete — the explicit per-FK override (ADR-0047). IDENTITY_REFERENCE_ATTR_ON_DELETE = "onDelete" # identity.reference: referential action on key update — the explicit per-FK override (ADR-0047). diff --git a/server/python/src/metaobjects/meta/persistence/db/db_provider.py b/server/python/src/metaobjects/meta/persistence/db/db_provider.py index a6b8aaad26..25a3a4615c 100644 --- a/server/python/src/metaobjects/meta/persistence/db/db_provider.py +++ b/server/python/src/metaobjects/meta/persistence/db/db_provider.py @@ -16,19 +16,23 @@ """ from __future__ import annotations -from ...core.attr.attr_constants import ATTR_SUBTYPE_BOOLEAN, ATTR_SUBTYPE_STRING +from ...core.attr.attr_constants import ATTR_SUBTYPE_BOOLEAN, ATTR_SUBTYPE_LONG, ATTR_SUBTYPE_STRING from ...core.field import field_constants as fc from ...core.identity.identity_constants import ( + IDENTITY_SUBTYPE_PRIMARY, IDENTITY_SUBTYPE_REFERENCE, IDENTITY_SUBTYPE_SECONDARY, ) from ...core.index.index_constants import INDEX_SUBTYPE_LOOKUP +from ...core.validator import validator_constants as vc from ....provider import Provider from ....registry import AttrSchema, TypeRegistry -from ....shared.base_types import TYPE_FIELD, TYPE_IDENTITY, TYPE_INDEX +from ....shared.base_types import TYPE_FIELD, TYPE_IDENTITY, TYPE_INDEX, TYPE_VALIDATOR from .db_constants import ( FIELD_ATTR_DB_COLUMN_TYPE, FIELD_ATTR_LOCAL_TIME, + IDENTITY_PRIMARY_ATTR_SEQUENCE_NAME, + IDENTITY_PRIMARY_ATTR_SEQUENCE_START, IDENTITY_REFERENCE_ATTR_CONSTRAINT_NAME, IDENTITY_REFERENCE_ATTR_ON_DELETE, IDENTITY_REFERENCE_ATTR_ON_UPDATE, @@ -74,6 +78,15 @@ required=False, ) +# @flattenedColumnPrefix — the flattened-member column prefix (string, "" allowed). +# Registered on field.object only; the description is sourced from the embedded +# spec/metamodel/db.json by apply_spec_descriptions (single-source). +_FLATTENED_COLUMN_PREFIX_SCHEMA = AttrSchema( + name=fc.FIELD_ATTR_FLATTENED_COLUMN_PREFIX, + value_type=ATTR_SUBTYPE_STRING, + required=False, +) + # DB-domain physical attrs EXTENDING identity subtypes — RDB index / FK-constraint # concerns, NOT core identity. Mirrors spec/metamodel/db.json's identity extends and # the TS/Java/C# db providers. Descriptions are byte-identical to the canonical. @@ -121,6 +134,18 @@ "expression. RDB-physical — contributed by the db provider." ), ) +_SEQUENCE_START_SCHEMA = AttrSchema( + name=IDENTITY_PRIMARY_ATTR_SEQUENCE_START, + value_type=ATTR_SUBTYPE_LONG, + required=False, + description='The first value an @generation "increment" key\'s sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1.', +) +_SEQUENCE_NAME_SCHEMA = AttrSchema( + name=IDENTITY_PRIMARY_ATTR_SEQUENCE_NAME, + value_type=ATTR_SUBTYPE_STRING, + required=False, + description='The physical name of an @generation "increment" key\'s sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database\'s own default,

__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1.', +) _CONSTRAINT_NAME_SCHEMA = AttrSchema( name=IDENTITY_REFERENCE_ATTR_CONSTRAINT_NAME, value_type=ATTR_SUBTYPE_STRING, @@ -162,6 +187,38 @@ ) +_ENUM_CONSTRAINT_NAME_SCHEMA = AttrSchema( + name=vc.VALIDATOR_ATTR_CONSTRAINT_NAME, + value_type=ATTR_SUBTYPE_STRING, + required=False, + description=( + "The database name of the CHECK constraint this enum's member list generates. " + "Absent:
__chk. Lets a model adopt a database whose CHECKs were " + "named by hand. RDB-physical — contributed by the db provider. Justified in " + "ADR-0023 Amendment 1." + ), +) +_VALIDATOR_CONSTRAINT_NAME_SCHEMA = AttrSchema( + name=vc.VALIDATOR_ATTR_CONSTRAINT_NAME, + value_type=ATTR_SUBTYPE_STRING, + required=False, + description=( + "The database name of the CHECK constraint this validator generates (numeric, length, " + "regex and the cross-field validators; the others generate no CHECK). Absent: the " + "derived
___chk. Lets a model adopt a database whose CHECKs " + "were named by hand. RDB-physical — contributed by the db provider. Justified in " + "ADR-0023 Amendment 1." + ), +) +# The validators meta migrate renders as a CHECK: required is NOT NULL; array and base have none. +_VALIDATOR_SUBTYPES_FOR_CONSTRAINT_NAME = ( + vc.VALIDATOR_SUBTYPE_NUMERIC, + vc.VALIDATOR_SUBTYPE_LENGTH, + vc.VALIDATOR_SUBTYPE_REGEX, + *vc.CROSS_FIELD_VALIDATOR_SUBTYPES, +) + + def _register(registry: TypeRegistry) -> None: for sub_type in _DB_FIELD_SUBTYPES: registry.extend( @@ -176,11 +233,22 @@ def _register(registry: TypeRegistry) -> None: fc.FIELD_SUBTYPE_TIMESTAMP, attributes=[_LOCAL_TIME_SCHEMA], ) + # @flattenedColumnPrefix is field.object-only (mirrors db.json's field.object extends). + registry.extend( + TYPE_FIELD, + fc.FIELD_SUBTYPE_OBJECT, + attributes=[_FLATTENED_COLUMN_PREFIX_SCHEMA], + ) registry.extend( TYPE_IDENTITY, IDENTITY_SUBTYPE_SECONDARY, attributes=[_ORDERS_SCHEMA, _WHERE_SCHEMA, _EXPR_SCHEMA, _USING_SCHEMA], ) + registry.extend( + TYPE_IDENTITY, + IDENTITY_SUBTYPE_PRIMARY, + attributes=[_SEQUENCE_START_SCHEMA, _SEQUENCE_NAME_SCHEMA], + ) registry.extend( TYPE_IDENTITY, IDENTITY_SUBTYPE_REFERENCE, @@ -190,6 +258,10 @@ def _register(registry: TypeRegistry) -> None: _REFERENCE_ON_UPDATE_SCHEMA, ], ) + # @constraintName names the CHECK a validator / a field.enum's membership generates. + registry.extend(TYPE_FIELD, fc.FIELD_SUBTYPE_ENUM, attributes=[_ENUM_CONSTRAINT_NAME_SCHEMA]) + for sub_type in _VALIDATOR_SUBTYPES_FOR_CONSTRAINT_NAME: + registry.extend(TYPE_VALIDATOR, sub_type, attributes=[_VALIDATOR_CONSTRAINT_NAME_SCHEMA]) # index.lookup: same RDB physical attrs as identity.secondary (@orders/@where/@expr/@using). registry.extend( TYPE_INDEX, diff --git a/server/python/src/metaobjects/naming.py b/server/python/src/metaobjects/naming.py index 3ff359909b..04a6aadecf 100644 --- a/server/python/src/metaobjects/naming.py +++ b/server/python/src/metaobjects/naming.py @@ -37,18 +37,23 @@ COLUMN_NAMING_SNAKE_CASE = "snake_case" COLUMN_NAMING_LITERAL = "literal" COLUMN_NAMING_KEBAB_CASE = "kebab-case" +#: The first letter upper-cased, nothing else changed (``requestURL`` → ``RequestURL``), so +#: a camelCase model maps with no ``@column`` onto a schema whose columns are spelled that +#: way. Not a word split. +COLUMN_NAMING_UPPER_FIRST = "upperFirst" COLUMN_NAMING_STRATEGIES = ( COLUMN_NAMING_SNAKE_CASE, COLUMN_NAMING_LITERAL, COLUMN_NAMING_KEBAB_CASE, + COLUMN_NAMING_UPPER_FIRST, ) #: ADR-0058 — the project default, in every port: what ``meta migrate`` has always #: created. FLIPPED in 1.1.0 (was ``literal`` in this port only). DEFAULT_COLUMN_NAMING = COLUMN_NAMING_SNAKE_CASE -#: ADR-0058 — ``tables`` accepts only these two: no ``kebab-case`` table rule. +#: ADR-0058 — ``tables`` accepts only these two: no ``kebab-case`` or ``upperFirst`` table rule. TABLE_NAMING_SNAKE_CASE = "snake_case" TABLE_NAMING_LITERAL = "literal" @@ -132,6 +137,21 @@ def to_snake_case(name: str) -> str: return "".join(out) +def to_upper_first_column(name: str) -> str: + """The ``upperFirst`` column rule: the first letter upper-cased, the rest verbatim. + + One UTF-16 unit in, one out, as C#'s ``char.ToUpperInvariant`` and Java's + ``Character.toUpperCase`` do: a first character whose upper case is not one character + (``ß`` → ``SS``), or that is outside the Basic Multilingual Plane, is kept as written, so + every port names it alike. + """ + first = name[:1] + upper = first.upper() + if len(upper) != 1 or ord(first) > 0xFFFF: + upper = first + return upper + name[1:] + + def apply_column_naming_strategy(name: str, strategy: str = DEFAULT_COLUMN_NAMING) -> str: """Apply a strategy to a bare name. @@ -144,6 +164,8 @@ def apply_column_naming_strategy(name: str, strategy: str = DEFAULT_COLUMN_NAMIN return to_snake_case(name) if strategy == COLUMN_NAMING_KEBAB_CASE: return to_snake_case(name).replace("_", "-") + if strategy == COLUMN_NAMING_UPPER_FIRST: + return to_upper_first_column(name) raise ValueError( f"unknown column-naming strategy {strategy!r}; " f"expected one of: {', '.join(COLUMN_NAMING_STRATEGIES)}" @@ -169,6 +191,33 @@ def resolve_column_name(field: MetaField, strategy: str | None = None) -> str: return apply_column_naming_strategy(field.name, effective) +def resolve_flattened_column_prefix(field: MetaField, strategy: str | None = None) -> str: + """The text before each member's column when *field* (a ``field.object @storage + "flattened"``) expands its value object into the owning table: its + ``@flattenedColumnPrefix`` verbatim (``""`` allowed), else its own resolved column and + ``_``. Core naming, not one generator's: the migration creates these columns and the + generated ORM mappings address them, so every port spells them the same way. + + ADR-0039: read RESOLVING — the prefix and ``@column`` may be inherited through + ``extends``. ``strategy`` as for :func:`resolve_column_name`. + """ + prefix = field.get_meta_attr(fc.FIELD_ATTR_FLATTENED_COLUMN_PREFIX) + if isinstance(prefix, str): + return prefix + return resolve_column_name(field, strategy) + "_" + + +def resolve_flattened_column_name( + field: MetaField, member: MetaField, strategy: str | None = None +) -> str: + """The column a member of a flattened value object gets in the owning table: the + embedding field's :func:`resolve_flattened_column_prefix` followed by the member's own + resolved column. The member's rule is the embedding field's, so one project setting + names both halves.""" + effective = strategy if strategy is not None else naming_rules_of(field).columns + return resolve_flattened_column_prefix(field, effective) + resolve_column_name(member, effective) + + def strip_package(name: str) -> str: """The bare, package-less segment of a metadata name — ``acme::demo::by_name`` → ``by_name``. diff --git a/server/python/src/metaobjects/registry_manifest.py b/server/python/src/metaobjects/registry_manifest.py index 0340fb466b..9e99593953 100644 --- a/server/python/src/metaobjects/registry_manifest.py +++ b/server/python/src/metaobjects/registry_manifest.py @@ -255,7 +255,7 @@ def _sorted_children(definition: TypeDefinition) -> list[dict[str, object]]: return children -METAMODEL_VERSION = "1.2" +METAMODEL_VERSION = "1.3" """Rolled-up spec-version for the cross-port registry manifest. ``"0"`` = pre-1.0 / unstable (semver major-0). Flips to ``"1.0"`` at the diff --git a/server/python/src/metaobjects/runtime/dialect.py b/server/python/src/metaobjects/runtime/dialect.py index afb19f9af0..a7ad385479 100644 --- a/server/python/src/metaobjects/runtime/dialect.py +++ b/server/python/src/metaobjects/runtime/dialect.py @@ -13,7 +13,7 @@ turned into the native Python type the metadata declares (``decode_value``). The Postgres dialect is the identity on both value hooks: pg8000 already speaks native types -(ADR-0019). SQLite and MySQL decode by the FIELD's declared subtype, because neither engine +(ADR-0019), a ``field.duration``'s ``timedelta`` included (bound and read as an interval). SQLite and MySQL decode by the FIELD's declared subtype, because neither engine has the column types Postgres has (a SQLite boolean is ``0``/``1``, its date and uuid are ``TEXT``; a MySQL ``TIME`` arrives as a ``timedelta``, its ``SUM`` of an integer as a ``Decimal``), so the declared type is the only authority on what a value means. @@ -28,6 +28,7 @@ from typing import Any from ..codegen.runtime.decimal_wire import decimal_from_double15 +from ..codegen.runtime.duration_wire import to_stored_micros from ..meta.core.field import field_constants as fc from ..meta.core.field.meta_field import MetaField from ..meta.persistence.db import db_constants as dbc @@ -39,6 +40,22 @@ PG_OID_INT4 = 23 +class DurationParam(_dt.timedelta): + """A ``field.duration``'s bound value: the ``timedelta`` the write codec makes of it. + + The SQLite and MySQL dialects bind it as its 64-bit microsecond count (an INTEGER / BIGINT + column, so it orders and compares as a number), and they key that on THIS type, not on + ``timedelta``: only the duration write codec makes + one, so a ``timedelta`` handed in for any other column binds exactly as it did before. + pg8000 binds it as the ``timedelta`` it is (an interval).""" + + __slots__ = () + + @classmethod + def of(cls, value: _dt.timedelta) -> "DurationParam": + return cls(days=value.days, seconds=value.seconds, microseconds=value.microseconds) + + def sqlite_supports_returning() -> bool: """``INSERT/UPDATE ... RETURNING`` arrived in SQLite 3.35.""" return _sqlite3.sqlite_version_info >= (3, 35, 0) @@ -261,6 +278,9 @@ def encode_param(self, value: Any) -> Any: return value.isoformat() if isinstance(value, _dt.time): return value.isoformat() + if isinstance(value, DurationParam): + # field.duration: an INTEGER column holding the microsecond count. + return to_stored_micros(value) return value def decode_value(self, field: MetaField, value: Any) -> Any: @@ -298,6 +318,10 @@ def encode_param(self, value: Any) -> Any: return value.astimezone(_dt.timezone.utc).replace(tzinfo=None) if isinstance(value, _dt.time): return value.isoformat() + if isinstance(value, DurationParam): + # field.duration: a BIGINT column holding the microsecond count. PyMySQL would + # otherwise bind a timedelta as a TIME literal. + return to_stored_micros(value) return value def decode_value(self, field: MetaField, value: Any) -> Any: diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index 612108f02d..582ebdf15d 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -34,8 +34,15 @@ from collections.abc import Iterable, Mapping from typing import Any, Protocol +from ..codegen.runtime.duration_wire import ( + duration_from_driver, + format_duration, + parse_duration_input, + parse_postgres_interval, +) from ..meta.meta_root import MetaRoot from ..meta.core.object.meta_object import MetaObject +from ..naming_refs import ref_target from ..meta.core.object.object_constants import OBJECT_SUBTYPE_REPORT from ..meta.core.reporting.report_read_model import report_read_model from ..meta.core.field.meta_field import MetaField @@ -56,7 +63,7 @@ collect_symmetric_target_ids, resolve_n2m_descriptor, ) -from .dialect import MYSQL, POSTGRES, Dialect +from .dialect import MYSQL, POSTGRES, Dialect, DurationParam from .tph import TphSubtype, tph_subtype_of from .validator_runner import ValidationResult, run_validators @@ -116,6 +123,10 @@ class DbApiDriver: def __init__(self, conn: Connection) -> None: self._conn = conn + def _cursor(self) -> Any: + """A cursor for one statement (a driver may set it up for its engine).""" + return self._conn.cursor() + def _bind(self, params: tuple[Any, ...]) -> tuple[Any, ...]: return tuple(self.dialect.encode_param(p) for p in params) @@ -131,7 +142,7 @@ def _result(self, description: Any, rows: list[Any]) -> SelectResult: return SelectResult([{c: row[i] for i, c in enumerate(cols)} for row in rows], column_oids) def select(self, sql: str, params: tuple[Any, ...] = ()) -> SelectResult: - cur = self._conn.cursor() + cur = self._cursor() try: cur.execute(sql, self._bind(params)) return self._result(cur.description, list(cur.fetchall())) @@ -155,7 +166,7 @@ def insert_returning(self, sql: str, params: tuple[Any, ...] = ()) -> SelectResu wire discriminator into the serialization boundary, exactly like the read path — so an op:create asserting the RETURNING row sees BIGINT as a string. """ - cur = self._conn.cursor() + cur = self._cursor() try: cur.execute(sql, self._bind(params)) description = cur.description @@ -174,7 +185,7 @@ def update_returning( ``insert_returning``). The OIDs carry the int4-vs-int8 (BIGINT→string) wire discriminator into the serialization boundary, exactly like the read path — so a BIGINT/currency column updates back as a numeric string.""" - cur = self._conn.cursor() + cur = self._cursor() try: cur.execute(sql, self._bind(params)) description = cur.description @@ -190,7 +201,7 @@ def insert_last_id(self, sql: str, params: tuple[Any, ...] = ()) -> Any: """Run an INSERT on an engine with no ``RETURNING`` and answer the key the engine generated (``lastrowid``; ``0`` / ``None`` when the caller supplied the key). Commits on success.""" - cur = self._conn.cursor() + cur = self._cursor() try: cur.execute(sql, self._bind(params)) last = getattr(cur, "lastrowid", None) @@ -202,7 +213,7 @@ def insert_last_id(self, sql: str, params: tuple[Any, ...] = ()) -> Any: def execute_rowcount(self, sql: str, params: tuple[Any, ...] = ()) -> int: """Run a DML statement (DELETE / UPDATE) and return the affected row count. Commits on success.""" - cur = self._conn.cursor() + cur = self._cursor() try: cur.execute(sql, self._bind(params)) count = cur.rowcount @@ -213,10 +224,92 @@ def execute_rowcount(self, sql: str, params: tuple[Any, ...] = ()) -> int: class PostgresDriver(DbApiDriver): - """Wrap a pg8000 / psycopg connection (the Postgres dialect).""" + """Wrap a pg8000 / psycopg connection (the Postgres dialect). + + An ``interval`` is read exactly, and one with a year or month part is never turned into a + ``timedelta`` (a month has no fixed length, so a ``field.duration`` refuses it): + + * **pg8000** gets an exact ``interval`` / ``interval[]`` reader registered on the connection + (pg8000 has no per-cursor adapters). pg8000 1.31 drops the sign of a negative time part's + minutes and seconds (``-00:01:30`` reads as ``+90 s``, ``-01:30:00`` as ``-30 min``). The + reader takes the default-``IntervalStyle`` text exactly (:func:`parse_postgres_interval`) + and hands anything else to pg8000's own reader, so an interval with a year or month part + is still the ``PGInterval`` pg8000 makes of it, and every other interval reads as before. + * **psycopg** (3, and psycopg2) turns a month into 30 days and a year into 365. The reader + is registered on each cursor this driver opens, never on the connection: it reads the + text exactly and hands an interval it cannot read as a fixed span back as its text, which + a ``field.duration`` refuses.""" dialect = POSTGRES + def __init__(self, conn: Connection) -> None: + super().__init__(conn) + _register_pg8000_interval_reader(conn) + + def _cursor(self) -> Any: + cur = self._conn.cursor() + _register_psycopg_interval_reader(cur) + return cur + + +def _interval_or_text(text: str) -> Any: + """The default-style text of an interval as a ``timedelta``, or the text itself when it is + not one this reads exactly (a year or month part, another ``IntervalStyle``): a + ``field.duration`` read refuses that text rather than guessing.""" + try: + return parse_postgres_interval(text) + except ValueError: + return text + + +def _register_pg8000_interval_reader(conn: Any) -> None: + """Register the exact ``interval`` reader on a pg8000 connection; any other is untouched.""" + register = getattr(conn, "register_in_adapter", None) + if not callable(register) or not type(conn).__module__.startswith("pg8000"): + return + from pg8000 import converters as _pgc # type: ignore[import-untyped] # only with a pg8000 connection + + pg8000_interval_in = _pgc.interval_in + + def interval_in(text: str) -> Any: + value = _interval_or_text(text) + # Not read here: pg8000's own reading (a PGInterval for a year or month part). + return pg8000_interval_in(text) if isinstance(value, str) else value + + register(_pgc.INTERVAL, interval_in) + register(_pgc.INTERVAL_ARRAY, _pgc._array_in(interval_in)) + + +#: The psycopg 3 text loader for ``interval``, made on first use (psycopg is optional). +_PSYCOPG_INTERVAL_LOADER: Any = None + + +def _register_psycopg_interval_reader(cur: Any) -> None: + """Register the exact ``interval`` reader on a psycopg 3 or psycopg2 cursor.""" + module = type(cur).__module__ + if module.startswith("psycopg2"): + import psycopg2.extensions as _ext # type: ignore[import-untyped] # only with a psycopg2 cursor + + interval = _ext.new_type( + (1186,), "METAOBJECTS_INTERVAL", lambda v, _c: None if v is None else _interval_or_text(v) + ) + _ext.register_type(interval, cur) + _ext.register_type(_ext.new_array_type((1187,), "METAOBJECTS_INTERVAL_ARRAY", interval), cur) + return + adapters = getattr(cur, "adapters", None) + if adapters is None or not module.startswith("psycopg"): + return + global _PSYCOPG_INTERVAL_LOADER + if _PSYCOPG_INTERVAL_LOADER is None: + from psycopg.adapt import Loader # type: ignore[import-not-found, unused-ignore] # only with psycopg 3 + + class _IntervalLoader(Loader): # type: ignore[misc, unused-ignore] + def load(self, data: Any) -> Any: + return _interval_or_text(bytes(data).decode("ascii")) + + _PSYCOPG_INTERVAL_LOADER = _IntervalLoader + adapters.register_loader("interval", _PSYCOPG_INTERVAL_LOADER) + class ObjectManager: """Method-based read API. Translates Filter dicts → parameterized SQL.""" @@ -1044,6 +1137,16 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: return [v if isinstance(v, _uuid.UUID) else _uuid.UUID(str(v)) for v in value] return value if isinstance(value, _uuid.UUID) else _uuid.UUID(str(value)) + # field.duration: an accepted ISO-8601 duration (or a timedelta already in process) → + # datetime.timedelta, which pg8000 binds as an interval; the SQLite and MySQL dialects + # bind its microsecond count (Dialect.encode_param), so a filter operand compares and + # orders as a number there. Anything else — a calendar span, a + # clock string, a number — is refused here, not by the database. + if sub == fc.FIELD_SUBTYPE_DURATION: + if _field_is_array(field) and isinstance(value, list): + return [None if v is None else _duration_write_value(field, v) for v in value] + return _duration_write_value(field, value) + # temporal: parse the authoring string to the native type, with tz-awareness # driven by the field so the driver binds the correct physical type. ADR-0036 # Wave 2: field.timestamp is instant / tz-aware BY DEFAULT (→ timestamptz); the @@ -1075,7 +1178,9 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: if sub in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): storage = field.get_meta_attr(fc.FIELD_ATTR_STORAGE) # ADR-0039 resolving if storage != "flattened": # None / "jsonb" / "subdocument" → single jsonb column - return _json.dumps(value) + # A field.duration inside the value object is stored in its canonical form, so the + # column holds what every port writes (and refuses what is not a duration). + return _json.dumps(_vo_durations(field, value, _write_duration_member)) # field.enum carrying @intValueMap: the column is an INTEGER while the native # and wire contract stays the member SYMBOL (int-backing is a persistence-layer # concern, invisible above this codec), so encode symbol -> declared int here. @@ -1092,6 +1197,80 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: return value +def _write_duration_member(field: MetaField, value: Any) -> str: + """A ``field.duration`` member of a value object, as the canonical string its JSON holds.""" + try: + return format_duration(parse_duration_input(value)) + except ValueError as e: + raise ValueError(f"field.duration '{field.name}' cannot be written: {e}") from None + + +def _read_duration_member(field: MetaField, value: Any) -> str: + """A ``field.duration`` member of a stored value object, in its canonical form (a value + another writer stored in an accepted but non-canonical spelling is normalised).""" + try: + return format_duration(duration_from_driver(value)) + except ValueError as e: + raise ValueError(f"field.duration '{field.name}' read a stored value that is not one: {e}") from None + + +def _vo_has_duration(vo: MetaObject, seen: frozenset[int] = frozenset()) -> bool: + """True when value object *vo*, or one it embeds, has a ``field.duration`` member.""" + if id(vo) in seen: + return False + seen = seen | {id(vo)} + for f in vo.fields(): + if f.sub_type == fc.FIELD_SUBTYPE_DURATION: + return True + if f.sub_type in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): + target = ref_target(f, fc.FIELD_ATTR_OBJECT_REF) + if isinstance(target, MetaObject) and _vo_has_duration(target, seen): + return True + return False + + +def _vo_durations(field: MetaField, value: Any, convert: Any, seen: frozenset[int] = frozenset()) -> Any: + """The value of a jsonb ``field.object`` / ``field.map`` with *convert* applied to every + ``field.duration`` member of its value object, at any depth: a single object, an array of + them, the values of a map, and the value objects those embed. Anything that is not the + shape the metadata declares is left as it is (validation reports it, not this).""" + if value is None: + return None + target = ref_target(field, fc.FIELD_ATTR_OBJECT_REF) # ADR-0039 resolving @objectRef + if not isinstance(target, MetaObject) or id(target) in seen or not _vo_has_duration(target): + return value + seen = seen | {id(target)} + + def one(member: Any) -> Any: + if not isinstance(member, Mapping): + return member + out = dict(member) + for f in target.fields(): + v = out.get(f.name) + if v is None: + continue + if f.sub_type == fc.FIELD_SUBTYPE_DURATION: + out[f.name] = [None if x is None else convert(f, x) for x in v] if isinstance(v, list) else convert(f, v) + elif f.sub_type in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): + out[f.name] = _vo_durations(f, v, convert, seen) + return out + + if field.sub_type == fc.FIELD_SUBTYPE_MAP: + return {k: one(v) for k, v in value.items()} if isinstance(value, Mapping) else value + if isinstance(value, list): + return [one(v) for v in value] + return one(value) + + +def _duration_write_value(field: MetaField, value: Any) -> DurationParam: + """The bound value of a ``field.duration``: a :class:`DurationParam`, so the dialect that + binds a duration as text does so by the field, never by a value's type.""" + try: + return DurationParam.of(parse_duration_input(value)) + except ValueError as e: + raise ValueError(f"field.duration '{field.name}' cannot be written: {e}") from None + + def _int_value_map(field: MetaField) -> dict[Any, Any] | None: """The declared ``@intValueMap`` (symbol → int), or ``None`` when the enum is string-backed. @@ -1111,7 +1290,9 @@ def _int_value_map(field: MetaField) -> dict[Any, Any] | None: def _decode_read_value(field: MetaField, value: Any) -> Any: """Decode a stored value back to its authoring form on read. - Two cases. The inverse of :func:`_coerce_write_value`'s int-backed-enum branch: + A ``field.duration`` reads back as a ``datetime.timedelta`` whatever the engine handed + back (see the branch below). Beyond that, two cases. The inverse of + :func:`_coerce_write_value`'s int-backed-enum branch: the column holds the member's integer, callers expect the member SYMBOL. And ``field.currency``, whose native type is ``int`` minor units: a table column is BIGINT and already reads back as an ``int``, but a VIEW column need not be — @@ -1131,6 +1312,18 @@ def _decode_read_value(field: MetaField, value: Any) -> Any: """ if value is None: return None + if field.sub_type == fc.FIELD_SUBTYPE_DURATION: + # A duration reads back as a datetime.timedelta on every engine: pg8000 hands one back + # for an interval, a SQLite / MySQL integer column holds the microsecond count. An interval with a month + # part (a PGInterval from pg8000, its text from psycopg: see PostgresDriver) is not a fixed length: refused. + if isinstance(value, list): + return [None if v is None else duration_from_driver(v) for v in value] + return duration_from_driver(value) + if field.sub_type in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): + # A jsonb value object's duration members read back in their canonical form. + if field.get_meta_attr(fc.FIELD_ATTR_STORAGE) != "flattened": # ADR-0039 resolving + return _vo_durations(field, value, _read_duration_member) + return value if field.sub_type == fc.FIELD_SUBTYPE_CURRENCY: if isinstance(value, _decimal.Decimal) and value == value.to_integral_value(): return int(value) diff --git a/server/python/src/metaobjects/runtime/validator_runner.py b/server/python/src/metaobjects/runtime/validator_runner.py index 1f0577243c..a9243f9168 100644 --- a/server/python/src/metaobjects/runtime/validator_runner.py +++ b/server/python/src/metaobjects/runtime/validator_runner.py @@ -23,6 +23,7 @@ from dataclasses import dataclass from decimal import Decimal +from ..codegen.runtime.duration_wire import NOT_A_DURATION, is_duration_literal from ..meta.core.field import field_constants as fc from ..meta.core.identity import identity_constants as ic from ..meta.core.object.meta_object import MetaObject @@ -72,6 +73,7 @@ class ValidationResult: _INT64_FIELD_SUBTYPES = frozenset({fc.FIELD_SUBTYPE_LONG, fc.FIELD_SUBTYPE_CURRENCY}) _STRING_FIELD_SUBTYPES = frozenset({ fc.FIELD_SUBTYPE_STRING, fc.FIELD_SUBTYPE_UUID, fc.FIELD_SUBTYPE_URI, fc.FIELD_SUBTYPE_INET, + fc.FIELD_SUBTYPE_DURATION, }) # ASCII digits only — ``\d`` would also admit other Unicode digits, which JS ``\d`` does not. @@ -382,6 +384,11 @@ def _scalar_errors(field: MetaData, value: object, required: bool, label: str) - errors.append(ValidationFailure( label, "format", f"'{label}' must be an IPv4 or IPv6 address", "inet", value, )) + + # field.duration — an ISO-8601 duration with no years or months (no @lenient: the + # column is an interval on Postgres, so a value that is not a duration cannot be stored). + if isinstance(value, str) and field.sub_type == fc.FIELD_SUBTYPE_DURATION and not is_duration_literal(value): + errors.append(ValidationFailure(label, "format", f"'{label}' {NOT_A_DURATION}", "duration", value)) return errors diff --git a/server/python/src/metaobjects/spec_metamodel/db.json b/server/python/src/metaobjects/spec_metamodel/db.json index 44c1048d73..4e087e72f0 100644 --- a/server/python/src/metaobjects/spec_metamodel/db.json +++ b/server/python/src/metaobjects/spec_metamodel/db.json @@ -21,6 +21,7 @@ "type": "field", "subType": "object", "children": [ + { "type": "attr", "subType": "string", "name": "flattenedColumnPrefix", "min": 0, "max": 1, "description": "On a field.object with @storage \"flattened\": the text put before each member's column, verbatim (\"\" allowed, so the members keep their own column names). Absent: this field's resolved column name and \"_\" (with @column \"bill\", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "string", "name": "storage", "min": 0, "max": 1, "allowedValues": ["flattened", "jsonb", "subdocument"], "description": "Storage strategy for an object-typed field (set with @objectRef). \"flattened\" expands the nested value into prefixed columns on the parent table. \"jsonb\" stores the structured value in a single jsonb column (supports isArray=true for arrays of values). \"subdocument\" is a hint for document-store codegen targets and emits no Postgres column." } ] }, @@ -58,6 +59,28 @@ { "type": "attr", "subType": "string", "name": "using", "min": 0, "max": 1, "description": "Index access method (e.g. \"gin\", \"gist\", \"hash\"); default \"btree\" (not rendered). Pair with @expr for e.g. a GIN index over an array/jsonb expression. RDB-physical — contributed by the db provider." } ] }, + { + "type": "field", + "subType": "enum", + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": ["numeric", "length", "regex", "comparison", "requiredWhen", "presentIff", "atLeastOne", "exactlyOne"], + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "identity", + "subType": "primary", + "children": [ + { "type": "attr", "subType": "long", "name": "sequenceStart", "min": 0, "max": 1, "description": "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." }, + { "type": "attr", "subType": "string", "name": "sequenceName", "min": 0, "max": 1, "description": "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, { "type": "identity", "subType": "reference", diff --git a/server/python/src/metaobjects/spec_metamodel/field.json b/server/python/src/metaobjects/spec_metamodel/field.json index b054bda765..3b4a4ce9fc 100644 --- a/server/python/src/metaobjects/spec_metamodel/field.json +++ b/server/python/src/metaobjects/spec_metamodel/field.json @@ -151,6 +151,7 @@ "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "int", "name": "maxLength", "min": 0, "max": 1, "description": "Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, @@ -183,6 +184,14 @@ "children": [ { "type": "attr", "subType": "boolean", "name": "lenient", "min": 0, "max": 1, "description": "#234: opt this field out of strict well-formedness enforcement. When true, codegen binds a plain string (no URL/IP validator, no native URI/InetAddress type; field.inet uses a text column, not the native inet type) so a not-strictly-valid value round-trips unchanged. Default (absent/false) is strict: an absolute-scheme URI / an IPv4-or-IPv6 literal." } ] + }, + { + "type": "field", + "subType": "duration", + "dataType": "string", + "extendsBase": true, + "description": "A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.", + "whenToUse": "A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span (\"every 3 months\") is not a duration: model it as an int and a unit." } ] } diff --git a/server/python/src/metaobjects/spec_metamodel/validator.json b/server/python/src/metaobjects/spec_metamodel/validator.json index 9520605bd7..4aba88dbd3 100644 --- a/server/python/src/metaobjects/spec_metamodel/validator.json +++ b/server/python/src/metaobjects/spec_metamodel/validator.json @@ -66,7 +66,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "left", "min": 1, "max": 1, "description": "Name of the left-hand field of the owning entity." }, { "type": "attr", "subType": "string", "name": "op", "min": 1, "max": 1, "allowedValues": ["gt", "gte", "lt", "lte", "ne", "eq"], "description": "Relational operator: gt (>), gte (>=), lt (<), lte (<=), ne (<>), eq (=)." }, - { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." } + { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -78,7 +79,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field that becomes required when the condition holds." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field whose value triggers the requirement." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -90,7 +92,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field whose presence is governed by the condition." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -100,7 +103,19 @@ "whenToUse": "At least one of several fields must be present (email or phone). Declare it instead of hand-written OR-presence checks.", "rules": "@fields names two or more fields of the owning entity. Satisfied when any one of them is non-null.", "children": [ - { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." } + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": "exactlyOne", + "description": "Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.", + "whenToUse": "A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.", + "rules": "@fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.", + "children": [ + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the alternative fields; exactly one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] } ] diff --git a/server/python/src/metaobjects/wire.py b/server/python/src/metaobjects/wire.py index 3da8986727..8b025d378e 100644 --- a/server/python/src/metaobjects/wire.py +++ b/server/python/src/metaobjects/wire.py @@ -1,11 +1,11 @@ """The project ``wire`` setting (ADR-0019 Amendment 2): the OUTPUT form of a generated REST surface. -``wire`` in ``.metaobjects/config.json`` (``{"decimals": ..., "nullMembers": ...}``) is read by +``wire`` in ``.metaobjects/config.json`` (``{"decimals": ..., "nullMembers": ..., "durations": ...}``) is read by the same port-neutral reader as ``naming`` and carried by the loaded root (``MetaRoot.wire``); -codegen reads it with :func:`wire_of`. The default is ``native`` for both keys: what this +codegen reads it with :func:`wire_of`. The default is ``native`` for every key: what this port's own serializer writes (Pydantic's ``str(Decimal)``, a value object as the runtime holds -it). ``"decimals": "string"`` and ``"nullMembers": "omit"`` give the portable form the -conformance corpora pin. INPUT rules (W2/W3) do not depend on this setting. +it, Pydantic's own ``timedelta`` form). ``"decimals": "string"``, ``"nullMembers": "omit"`` and +``"durations": "iso"`` give the portable form the conformance corpora pin. INPUT rules (W2/W3) do not depend on this setting. """ from __future__ import annotations @@ -20,10 +20,12 @@ WIRE_DECIMALS_STRING = "string" WIRE_NULLS_EMIT = "emit" WIRE_NULLS_OMIT = "omit" +WIRE_DURATIONS_ISO = "iso" #: The closed sets each key accepts, in the order the refusal message lists them. WIRE_DECIMALS = (WIRE_NATIVE, WIRE_DECIMALS_STRING) WIRE_NULL_MEMBERS = (WIRE_NATIVE, WIRE_NULLS_EMIT, WIRE_NULLS_OMIT) +WIRE_DURATIONS = (WIRE_NATIVE, WIRE_DURATIONS_ISO) @dataclass(frozen=True) @@ -33,6 +35,7 @@ class WireFormat: decimals: str = WIRE_NATIVE null_members: str = WIRE_NATIVE + durations: str = WIRE_NATIVE def wire_of(node: "MetaData") -> WireFormat: @@ -47,7 +50,7 @@ def wire_of(node: "MetaData") -> WireFormat: def wire_block(wire: WireFormat) -> str: """A wire format as the ``wire`` block that declares it.""" - return f'"wire": {{ "decimals": "{wire.decimals}", "nullMembers": "{wire.null_members}" }}' + return f'"wire": {{ "decimals": "{wire.decimals}", "nullMembers": "{wire.null_members}", "durations": "{wire.durations}" }}' def wire_candidates(used: WireFormat) -> list[WireFormat]: @@ -56,7 +59,8 @@ def wire_candidates(used: WireFormat) -> list[WireFormat]: candidate for d in WIRE_DECIMALS for n in WIRE_NULL_MEMBERS - if (candidate := WireFormat(decimals=d, null_members=n)) != used + for u in WIRE_DURATIONS + if (candidate := WireFormat(decimals=d, null_members=n, durations=u)) != used ] diff --git a/server/python/tests/codegen/test_config_run_settings.py b/server/python/tests/codegen/test_config_run_settings.py index 4e7377e3fb..1762909c3b 100644 --- a/server/python/tests/codegen/test_config_run_settings.py +++ b/server/python/tests/codegen/test_config_run_settings.py @@ -318,12 +318,12 @@ def test_verify_with_another_wire_says_the_wire_block_differs_not_regenerate( err = capsys.readouterr().err assert ( 'The committed output was generated with "wire": { "decimals": "string", ' - '"nullMembers": "omit" }; this run used "wire": { "decimals": "native", ' - '"nullMembers": "native" } (the wire block of .metaobjects/config.json; each key ' + '"nullMembers": "omit", "durations": "native" }; this run used "wire": { "decimals": "native", ' + '"nullMembers": "native", "durations": "native" } (the wire block of .metaobjects/config.json; each key ' "defaults to native). Do not regenerate" in err ) assert ( - 'Declare "wire": { "decimals": "string", "nullMembers": "omit" } in ' + 'Declare "wire": { "decimals": "string", "nullMembers": "omit", "durations": "native" } in ' ".metaobjects/config.json" in err ) assert "regenerate (metaobjects gen)" not in err diff --git a/server/python/tests/codegen/test_duration_codegen.py b/server/python/tests/codegen/test_duration_codegen.py new file mode 100644 index 0000000000..861cf8c42f --- /dev/null +++ b/server/python/tests/codegen/test_duration_codegen.py @@ -0,0 +1,222 @@ +"""field.duration codegen: the native ``datetime.timedelta`` binding; ``wire.durations: iso`` holds it +to the ISO-8601 wire rule, ``native`` (the default) leaves Pydantic's own handling. + +The generated model takes only an accepted ISO-8601 duration (Pydantic's own ``timedelta`` +parser would also take a number, clock text and a calendar span) and writes the canonical form +in JSON (Pydantic's own is ``PT5400S`` for 90 minutes). Its codec is the generated package's +``_duration_wire.py``, a copy of the production helper. A router writes a duration in a response +through the same formatter, filters it in the ordered band and sorts by it. +""" +from __future__ import annotations + +import datetime as dt +import importlib +import json +import sys +from collections.abc import Iterator +from pathlib import Path +from typing import Any + +import pytest +from pydantic import TypeAdapter, ValidationError + +import metaobjects.core_types # noqa: F401 — side effect: registers core types +from metaobjects import load_string +from metaobjects.codegen.config import GenConfig +from metaobjects.codegen.generators.entity_model import entity_model +from metaobjects.codegen.generators.filter_allowlist_generator import ( + filter_allowlist_generator, + ops_for_field_ordered, +) +from metaobjects.codegen.generators.router_generator import router_generator +from metaobjects.codegen.runner import run_gen +from metaobjects.codegen.runtime import duration_wire +from metaobjects.codegen.runtime.decimal_wire import WireOutput, render_wire +from metaobjects.codegen.type_map import py_type_for +from metaobjects.loader.validation_passes import ops_for_subtype +from metaobjects.wire import WireFormat + +_META = { + "metadata.root": { + "package": "acme::ops", + "children": [ + { + "object.entity": { + "name": "Hold", + "children": [ + {"source.rdb": {"@table": "holds"}}, + {"field.long": {"name": "id"}}, + {"field.duration": {"name": "holdFor", "@required": True, "@filterable": True}}, + {"field.duration": {"name": "grace"}}, + {"field.duration": {"name": "steps", "isArray": True}}, + {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, + ], + } + }, + { + "object.entity": { + "name": "Plain", + "children": [ + {"source.rdb": {"@table": "plains"}}, + {"field.long": {"name": "id"}}, + {"field.string": {"name": "label"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, + ], + } + }, + ], + } +} + + +def _root(durations: str = "iso") -> Any: + result = load_string(json.dumps(_META)) + assert not result.errors, [str(e) for e in result.errors] + result.root.wire = WireFormat(durations=durations) + return result.root + + +def _entity(name: str) -> Any: + return next(c for c in _root().children() if getattr(c, "name", None) == name) + + +def _field(name: str) -> Any: + return next(f for f in _entity("Hold").fields() if f.name == name) + + +@pytest.fixture(scope="module") +def generated(tmp_path_factory: pytest.TempPathFactory) -> Iterator[dict[str, Any]]: + """Generate entity + filter-allowlist + routes for the model and import the package.""" + tmp = tmp_path_factory.mktemp("duration") + out = tmp / "gen_duration" + run_gen( + GenConfig(out_dir=str(out), emit_package_init=True), + _root(), + generators=[entity_model(), filter_allowlist_generator(), router_generator()], + ) + files = {p.relative_to(out).as_posix(): p.read_text(encoding="utf-8") for p in out.rglob("*.py")} + sys.path.insert(0, str(tmp)) + try: + hold = importlib.import_module("gen_duration.Hold") + router = importlib.import_module("gen_duration.hold_router") + finally: + sys.path.remove(str(tmp)) + yield {"files": files, "hold": hold, "router": router} + for name in [m for m in sys.modules if m.split(".")[0] == "gen_duration"]: + del sys.modules[name] + + +def test_the_native_type_is_timedelta() -> None: + assert py_type_for(_field("holdFor")).expr == "datetime.timedelta" + assert py_type_for(_field("steps")).expr == "list[datetime.timedelta]" + + +def test_the_codec_module_is_emitted_only_beside_a_duration(generated: dict[str, Any]) -> None: + files = generated["files"] + assert files["_duration_wire.py"].endswith( + Path(duration_wire.__file__).read_text(encoding="utf-8") + ) + assert "from ._duration_wire import format_duration, parse_duration_input" in files["Hold.py"] + assert "_duration_wire" not in files["Plain.py"] + + +def test_a_project_with_no_duration_emits_no_codec_module(tmp_path: Path) -> None: + meta = json.loads(json.dumps(_META)) + meta["metadata.root"]["children"] = meta["metadata.root"]["children"][1:] + run_gen(GenConfig(out_dir=str(tmp_path)), load_string(json.dumps(meta)).root, generators=[entity_model()]) + assert not (tmp_path / "_duration_wire.py").exists() + + +@pytest.mark.parametrize("model", ["Hold", "HoldCreate", "HoldPatch"]) +def test_the_model_reads_an_iso_duration_and_writes_the_canonical_form( + generated: dict[str, Any], model: str +) -> None: + cls = getattr(generated["hold"], model) + m = cls(holdFor="P1DT2H", grace="-PT90S", steps=["PT5400S", "PT0.5S"]) + assert m.holdFor == dt.timedelta(hours=26) + assert json.loads(m.model_dump_json()) | {"id": None} == { + "id": None, "holdFor": "PT26H", "grace": "-PT1M30S", "steps": ["PT1H30M", "PT0.5S"], + } + # In process the value stays the native timedelta. + assert m.model_dump()["grace"] == dt.timedelta(seconds=-90) + assert cls(holdFor=dt.timedelta(minutes=5)).holdFor == dt.timedelta(minutes=5) + # A week is exactly 7 days, and a fraction past microseconds is truncated, never rounded. + assert json.loads(cls(holdFor="P1W1DT1H", grace="PT0.99999999999S").model_dump_json())["holdFor"] == "PT193H" + assert json.loads(cls(holdFor="PT1H", grace="PT0.99999999999S").model_dump_json())["grace"] == "PT0.999999S" + + +@pytest.mark.parametrize("value", ["1:30:00", "P1Y", "P1M", "P1.5W", "PT1W", "PT1.5H", "pt1h", "PT", 90, 1.5]) +def test_the_model_refuses_what_pydantic_timedelta_would_take(generated: dict[str, Any], value: object) -> None: + with pytest.raises(ValidationError): + generated["hold"].HoldCreate(holdFor=value) + with pytest.raises(ValidationError): + generated["hold"].HoldCreate(holdFor="PT1H", steps=[value]) + + +def test_duration_takes_the_ordered_filter_band() -> None: + band = ("eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull") + assert ops_for_subtype("duration") == frozenset(band) + assert ops_for_field_ordered(_field("holdFor")) == band + + +def test_a_duration_is_a_sort_key(generated: dict[str, Any]) -> None: + assert {"holdFor", "grace"} <= generated["router"]._SORT_ALLOWLIST + + +def test_a_router_writes_a_duration_through_the_codec(generated: dict[str, Any]) -> None: + src = generated["files"]["hold_router.py"] + assert "from ._duration_wire import format_duration" in src + assert "duration=format_duration" in src + assert "_duration_wire" not in generated["files"]["plain_router.py"] + wire = generated["router"]._WIRE + row = {"id": 1, "holdFor": dt.timedelta(hours=26), "steps": [dt.timedelta(seconds=-90)]} + assert render_wire(row, wire) == {"id": 1, "holdFor": "PT26H", "steps": ["-PT1M30S"]} + assert render_wire(row, WireOutput(decimals="string", duration=wire.duration))["holdFor"] == "PT26H" + + +@pytest.fixture(scope="module") +def generated_native(tmp_path_factory: pytest.TempPathFactory) -> Iterator[dict[str, Any]]: + """The same model generated under the default ``wire.durations: native``.""" + tmp = tmp_path_factory.mktemp("duration_native") + out = tmp / "gen_duration_native" + run_gen( + GenConfig(out_dir=str(out), emit_package_init=True), + _root("native"), + generators=[entity_model(), filter_allowlist_generator(), router_generator()], + ) + files = {p.relative_to(out).as_posix(): p.read_text(encoding="utf-8") for p in out.rglob("*.py")} + sys.path.insert(0, str(tmp)) + try: + hold = importlib.import_module("gen_duration_native.Hold") + router = importlib.import_module("gen_duration_native.hold_router") + finally: + sys.path.remove(str(tmp)) + yield {"files": files, "hold": hold, "router": router} + for name in [m for m in sys.modules if m.split(".")[0] == "gen_duration_native"]: + del sys.modules[name] + + +def test_native_emits_no_codec_and_no_override(generated_native: dict[str, Any]) -> None: + files = generated_native["files"] + assert "_duration_wire.py" not in files + assert "_duration_wire" not in files["Hold.py"] + assert "BeforeValidator" not in files["Hold.py"] + assert "format_duration" not in files["hold_router.py"] + assert "duration=to_jsonable_python" in files["hold_router.py"] + + +def test_native_round_trip_is_pydantics_own_timedelta(generated_native: dict[str, Any]) -> None: + cls = generated_native["hold"].Hold + # Pydantic's own parser takes clock text and numbers, which the ISO rule refuses. + m = cls(holdFor="1:30:00", grace=90, steps=["PT5400S"]) + assert m.holdFor == dt.timedelta(hours=1, minutes=30) + assert m.grace == dt.timedelta(seconds=90) + assert json.loads(m.model_dump_json())["holdFor"] == TypeAdapter(dt.timedelta).dump_python(m.holdFor, mode="json") + assert cls.model_validate_json(m.model_dump_json()).holdFor == m.holdFor + + +def test_native_response_is_pydantics_own_form_and_iso_is_canonical(generated_native: dict[str, Any]) -> None: + value = dt.timedelta(hours=1, minutes=30) + native = generated_native["router"]._WIRE + assert render_wire({"d": value}, native) == {"d": TypeAdapter(dt.timedelta).dump_python(value, mode="json")} + assert render_wire({"d": value}, WireOutput(duration=duration_wire.format_duration)) == {"d": "PT1H30M"} diff --git a/server/python/tests/codegen/test_enum_max_length_model.py b/server/python/tests/codegen/test_enum_max_length_model.py new file mode 100644 index 0000000000..420d16f72e --- /dev/null +++ b/server/python/tests/codegen/test_enum_max_length_model.py @@ -0,0 +1,31 @@ +"""An enum's @maxLength is its column length, not an input rule: the generated model puts no +``max_length`` on it (the ``Literal`` of its members is the rule, and every member fits), while +a string's @maxLength still becomes ``max_length``.""" +from __future__ import annotations + +import json + +import metaobjects.core_types # noqa: F401 — side-effect: registers attr classes + +from metaobjects import InMemoryStringSource, MetaDataFormat, MetaDataLoader +from metaobjects.codegen.generators.entity_model import _validator_constraints +from metaobjects.shared.base_types import TYPE_OBJECT + +_MODEL = {"metadata.root": {"package": "acme", "children": [ + {"object.entity": {"name": "Ticket", "children": [ + {"source.rdb": {}}, + {"field.long": {"name": "id"}}, + {"field.enum": {"name": "status", "@values": ["OPEN", "CLOSED"], "@maxLength": 16}}, + {"field.string": {"name": "title", "@maxLength": 40}}, + {"identity.primary": {"name": "pk", "@fields": "id"}}, + ]}}, +]}} + + +def test_an_enum_column_length_is_not_a_model_constraint() -> None: + result = MetaDataLoader().load([InMemoryStringSource(json.dumps(_MODEL), format=MetaDataFormat.JSON)]) + assert not result.errors + ticket = next(c for c in result.root.children() if c.type == TYPE_OBJECT and c.name == "Ticket") + fields = {f.name: f for f in ticket.fields()} + assert "max_length" not in _validator_constraints(fields["status"], wire=False) + assert _validator_constraints(fields["title"], wire=False).get("max_length") == 40 diff --git a/server/python/tests/codegen/test_m2m_codegen.py b/server/python/tests/codegen/test_m2m_codegen.py index 946b9a4637..7082cc4da5 100644 --- a/server/python/tests/codegen/test_m2m_codegen.py +++ b/server/python/tests/codegen/test_m2m_codegen.py @@ -266,7 +266,7 @@ def test_an_unknown_strategy_is_refused_at_the_descriptor_too() -> None: # Fail loudly rather than fall back: a typo would otherwise bind junction SQL to # columns that do not exist and report success. with pytest.raises(ValueError, match="snake_case"): - resolve_m2m_descriptors(_ENTITIES["Post"], _INDEX, column_naming="PascalCase") + resolve_m2m_descriptors(_ENTITIES["Post"], _INDEX, column_naming="camelCase") # --------------------------------------------------------------------------- diff --git a/server/python/tests/codegen/test_names_flattened.py b/server/python/tests/codegen/test_names_flattened.py new file mode 100644 index 0000000000..c4df205ed8 --- /dev/null +++ b/server/python/tests/codegen/test_names_flattened.py @@ -0,0 +1,100 @@ +"""A flattened value object has no column of its own: each member is a column of the +owning table, named by core naming (``resolve_flattened_column_name``). The names module +carries one ``___COLUMN`` constant per member, beside the field's own +``_FIELD`` constant. The field has no ``_COLUMN`` constant and no ``COLUMNS_BY_FIELD`` key of its own. + +Mirrors the C# ``NamesGenerator``, the Java/Kotlin names generators and the TypeScript +``names.ts``; the corpus test runs the shared cross-port naming fixture. +""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +import metaobjects.core_types # noqa: F401 — side-effect: registers attr classes + +from metaobjects import FileSource, InMemoryStringSource, MetaDataFormat, MetaDataLoader +from metaobjects.codegen.generators.names_generator import render_names +from metaobjects.meta.core.object.meta_object import MetaObject +from metaobjects.naming import to_snake_case +from metaobjects.shared.base_types import TYPE_OBJECT + +REPO = Path(__file__).parents[4] +CORPUS = REPO / "fixtures" / "naming-conformance" / "physical-names" +_CASES: list[dict[str, object]] = json.loads((CORPUS / "cases.json").read_text())["cases"] + +_MODEL = { + "metadata.root": {"package": "acme::ship", "children": [ + {"object.value": {"name": "Address", "children": [ + {"field.string": {"name": "street"}}, + {"field.string": {"name": "postalCode", "@column": "ZIP"}}, + ]}}, + {"object.entity": {"name": "Shipment", "children": [ + {"source.rdb": {"@table": "shipments"}}, + {"field.long": {"name": "id"}}, + {"field.object": {"name": "shipTo", "@objectRef": "Address", "@storage": "flattened"}}, + {"field.object": {"name": "payload", "@objectRef": "Address"}}, + {"identity.primary": {"@fields": ["id"], "@generation": "increment"}}, + ]}}, + ]}, +} + + +def _object(root: MetaObject, name: str) -> MetaObject: + return next(c for c in root.children() if c.type == TYPE_OBJECT and c.name == name) + + +def _load(model: dict[str, object]) -> MetaObject: + result = MetaDataLoader().load( + [InMemoryStringSource(json.dumps(model), format=MetaDataFormat.JSON)] + ) + assert not result.errors, [e.message for e in result.errors] + return result.root + + +def test_a_flattened_member_gets_a_column_constant_beside_its_field_constants() -> None: + src = render_names(_object(_load(_MODEL), "Shipment"), "snake_case") + + assert 'SHIPMENT_SHIP_TO_STREET_COLUMN: Final[str] = "ship_to_street"' in src + assert 'SHIPMENT_SHIP_TO_POSTAL_CODE_COLUMN: Final[str] = "ship_to_ZIP"' in src + assert 'SHIPMENT_SHIP_TO_FIELD: Final[str] = "shipTo"' in src + assert 'SHIPMENT_PAYLOAD_COLUMN: Final[str] = "payload"' in src + # It names no column of the table, so the field has no column constant of its own. + assert "SHIPMENT_SHIP_TO_COLUMN" not in src + assert "Deprecated" not in src + assert '"shipTo":' not in src + assert '"shipTo.street": SHIPMENT_SHIP_TO_STREET_COLUMN,' in src + assert '"shipTo.postalCode": SHIPMENT_SHIP_TO_POSTAL_CODE_COLUMN,' in src + + +def test_a_flattened_member_constant_colliding_with_a_field_constant_is_left_out_with_a_warning() -> None: + model = json.loads(json.dumps(_MODEL)) + shipment = model["metadata.root"]["children"][1]["object.entity"]["children"] + shipment[2]["field.object"]["@flattenedColumnPrefix"] = "st_" + shipment.insert(3, {"field.string": {"name": "shipToStreet"}}) + warnings: list[str] = [] + src = render_names(_object(_load(model), "Shipment"), "snake_case", warn=warnings.append) + + # The field's constant wins; the model generates, as it did before member constants. + assert 'SHIPMENT_SHIP_TO_STREET_COLUMN: Final[str] = "ship_to_street"' in src + assert '"st_street"' not in src + assert 'SHIPMENT_SHIP_TO_POSTAL_CODE_COLUMN: Final[str] = "st_ZIP"' in src + assert any("SHIPMENT_SHIP_TO_STREET_COLUMN" in w for w in warnings) + + +@pytest.mark.parametrize("case", _CASES, ids=lambda c: c["name"]) +def test_every_flattened_column_in_the_naming_corpus_is_a_constant(case: dict[str, object]) -> None: + result = MetaDataLoader().load([FileSource(CORPUS / "meta.naming.json")]) + assert not result.errors, [e.message for e in result.errors] + naming = case["naming"] or {} + rule = naming.get("columns", "snake_case") # type: ignore[union-attr] + src = render_names(_object(result.root, "Shipment"), rule) + for key, column in case["columns"].items(): # type: ignore[union-attr] + parts = key.split(".") + if len(parts) != 3 or parts[0] != "Shipment": + continue + field, member = (to_snake_case(p).upper() for p in parts[1:]) + decl = f'SHIPMENT_{field}_{member}_COLUMN: Final[str] = "{column}"' + assert decl in src, f"{case['name']}: {decl}\n{src}" diff --git a/server/python/tests/codegen/test_validation_conformance.py b/server/python/tests/codegen/test_validation_conformance.py index e713f29662..7d51178a09 100644 --- a/server/python/tests/codegen/test_validation_conformance.py +++ b/server/python/tests/codegen/test_validation_conformance.py @@ -22,11 +22,13 @@ from pathlib import Path import pytest + +from metaobjects.wire import WireFormat from pydantic import BaseModel, ValidationError import metaobjects.core_types # noqa: F401 — side effect: registers core types from metaobjects import MetaDataLoader -from metaobjects.codegen.generators.entity_model import render_entity_model +from metaobjects.codegen.generators.entity_model import duration_wire_module, render_entity_model from metaobjects.meta.core.object.meta_object import MetaObject from metaobjects.shared.base_types import TYPE_OBJECT @@ -45,6 +47,9 @@ def _load_entity(name: str) -> MetaObject: meta = (_corpus_dir() / "meta.json").read_text() result = MetaDataLoader.from_string(meta) assert not result.errors, [str(e) for e in result.errors] + # The corpus' duration cases (duration-reject-*) pin the ISO-8601 input rule, which is what + # `wire.durations: iso` generates; under the `native` default Pydantic's own parser applies. + result.root.wire = WireFormat(durations="iso") # type: ignore[attr-defined] for child in result.root.children(): if child.type == TYPE_OBJECT and getattr(child, "name", None) == name: assert isinstance(child, MetaObject) @@ -68,15 +73,23 @@ def _build_generated_model(entity_name: str = DEFAULT_ENTITY) -> type[BaseModel] # resolves when ``__name__`` and the module's imports are in scope. A bare # exec namespace has no ``__name__`` (forward refs fall back to builtins and # fail — #234); a proper module import mirrors the TS runner's temp-module. - tmp_dir = tempfile.mkdtemp() - mod_name = f"generated_{entity_name.lower()}" - mod_path = Path(tmp_dir) / f"{mod_name}.py" + # + # The module lives in a package beside the generated ``_duration_wire.py`` companion the + # entity tier emits for a model with a ``field.duration`` (``Account.holdFor``), which the + # model imports package-relatively. + pkg_dir = Path(tempfile.mkdtemp()) / f"generated_pkg_{entity_name.lower()}" + pkg_dir.mkdir() + (pkg_dir / "__init__.py").write_text("") + companion = duration_wire_module() + (pkg_dir / companion.path).write_text(companion.content) + mod_name = f"{pkg_dir.name}.generated_{entity_name.lower()}" + mod_path = pkg_dir / f"generated_{entity_name.lower()}.py" mod_path.write_text(source) - spec = importlib.util.spec_from_file_location(mod_name, mod_path) - assert spec and spec.loader - module = importlib.util.module_from_spec(spec) - sys.modules[mod_name] = module - spec.loader.exec_module(module) + sys.path.insert(0, str(pkg_dir.parent)) + try: + module = importlib.import_module(mod_name) + finally: + sys.path.remove(str(pkg_dir.parent)) model = getattr(module, f"{entity_name}Create") assert isinstance(model, type) and issubclass(model, BaseModel) return model @@ -209,3 +222,41 @@ def test_cross_field_comparison_of_mixed_aware_and_naive_timestamps_is_a_validat module.WindowCreate(opensAt="2026-01-01T11:00:00Z", closesAt="2026-01-01T10:00:00Z") with pytest.raises(ValidationError): module.WindowCreate(opensAt="2026-01-01T10:00:00Z", closesAt="2026-01-01T11:00:00") + + +_DROPPED_RULE_MODEL = """ +{ "metadata.root": { "package": "acme::drop", "children": [ + { "object.entity": { "name": "Drop", "children": [ + { "source.rdb": { "@table": "drops" } }, + { "field.long": { "name": "id" } }, + { "field.int": { "name": "a" } }, + { "field.int": { "name": "b" } }, + { "field.string": { "name": "kind" } }, + { "field.string": { "name": "note" } }, + { "field.date": { "name": "d" } }, + { "field.timestamp": { "name": "t" } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, + { "validator.atLeastOne": { "name": "dropped", "@fields": ["note"], "@condition": { "op": "and", "args": [ + { "op": "gt", "left": { "field": "a" }, "right": { "field": "b" } }, + { "op": "gt", "left": { "field": "d" }, "right": { "field": "t" } } + ] } } }, + { "validator.requiredWhen": { "name": "kept", "@field": "note", "@when": "kind", "@equals": "X" } } + ]}} +]}} +""" + + +def test_a_rule_left_to_the_database_declares_no_helper() -> None: + # The dropped rule's condition asks for `_cmp` and `_all`, then compares a date with a timestamp + # (no order), so the rule is left to the database; nothing it asked for may reach the module. + result = MetaDataLoader.from_string(_DROPPED_RULE_MODEL) + assert not result.errors, [str(e) for e in result.errors] + entity = next(c for c in result.root.children() if c.type == TYPE_OBJECT and c.name == "Drop") + source = render_entity_model(entity) + assert "_cross_field_rules" in source + assert "def _cmp" not in source + assert "def _all" not in source + module = _import_source(source, "generated_cross_drop") + with pytest.raises(ValidationError): + module.DropCreate(kind="X") + module.DropCreate(a=2, b=1, kind="Y") diff --git a/server/python/tests/config/test_neutral_config.py b/server/python/tests/config/test_neutral_config.py index 474b7cab82..a1aa8447bf 100644 --- a/server/python/tests/config/test_neutral_config.py +++ b/server/python/tests/config/test_neutral_config.py @@ -177,3 +177,23 @@ def test_unknown_library_token_raises_loudly(tmp_path: Path) -> None: read_neutral_config(tmp_path) assert e.value.code == ErrorCode.ERR_UNKNOWN_LIBRARY assert "ai-trace" in str(e.value) + + +@pytest.mark.parametrize("enums", [{}, {"maxLength": 32}, {"maxLength": 32.0}, {"maxLength": 1e2}]) +def test_an_enums_block_is_accepted_and_ignored(tmp_path: Path, enums: object) -> None: + """A project shared with a port that sizes a string-stored enum's column has an `enums` + block. This port sizes no column, so it reads nothing from the block, but accepts it.""" + _write_config(tmp_path, {"schema_version": 1, "sources": [], "enums": enums}) + assert read_neutral_config(tmp_path) is not None + + +@pytest.mark.parametrize( + "enums", + [[], "32", {"maxLen": 32}, {"maxLength": 0}, {"maxLength": -1}, {"maxLength": 1.5}, {"maxLength": True}], + ids=repr, +) +def test_a_malformed_enums_block_is_refused(tmp_path: Path, enums: object) -> None: + _write_config(tmp_path, {"schema_version": 1, "sources": [], "enums": enums}) + with pytest.raises(ParseError) as e: + read_neutral_config(tmp_path) + assert e.value.code == ErrorCode.ERR_BAD_ATTR_VALUE diff --git a/server/python/tests/conformance/test_naming_conformance.py b/server/python/tests/conformance/test_naming_conformance.py index 053bb9f038..19f59dd3b1 100644 --- a/server/python/tests/conformance/test_naming_conformance.py +++ b/server/python/tests/conformance/test_naming_conformance.py @@ -18,7 +18,9 @@ from metaobjects.config.neutral_config import read_neutral_config from metaobjects.errors import ParseError from metaobjects.meta.core.object.meta_object import MetaObject -from metaobjects.naming import resolve_column_name +from metaobjects.meta.core.field.field_constants import FIELD_ATTR_OBJECT_REF +from metaobjects.naming import resolve_column_name, resolve_flattened_column_name +from metaobjects.naming_refs import ref_target from metaobjects.shared.base_types import TYPE_OBJECT from metaobjects.source_resolution import primary_rdb_source @@ -79,12 +81,19 @@ def test_physical_names_case(case: dict[str, object], tmp_path: Path) -> None: ) for key, expected_column in case["columns"].items(): # type: ignore[union-attr] - obj_name, field_name = key.split(".", 1) + obj_name, field_name, *member_name = key.split(".", 2) obj = objects[obj_name] field = next(f for f in obj.fields() if f.name == field_name) - assert resolve_column_name(field) == expected_column, ( - f"case {case['name']!r}: {key} column" - ) + if not member_name: + actual = resolve_column_name(field) + else: + # `..`: a member of the flattened value object the field + # embeds, named by core naming. + target = ref_target(field, FIELD_ATTR_OBJECT_REF) + assert isinstance(target, MetaObject), f"{key}: @objectRef does not resolve" + member = next(f for f in target.fields() if f.name == member_name[0]) + actual = resolve_flattened_column_name(field, member) + assert actual == expected_column, f"case {case['name']!r}: {key} column" @pytest.mark.parametrize("invalid", _INVALID, ids=lambda c: c["name"]) diff --git a/server/python/tests/conformance/test_wire_conformance.py b/server/python/tests/conformance/test_wire_conformance.py index 6ef0def705..2d247cbd09 100644 --- a/server/python/tests/conformance/test_wire_conformance.py +++ b/server/python/tests/conformance/test_wire_conformance.py @@ -1,7 +1,7 @@ """ADR-0019 Amendment 2 cross-port conformance: ``fixtures/wire-conformance/config-blocks.json``. Every port parses each case's ``wire`` block with its OWN config reader -(``metaobjects.config.neutral_config``) and must resolve ``decimals`` and ``nullMembers`` to +(``metaobjects.config.neutral_config``) and must resolve ``decimals``, ``nullMembers`` and ``durations`` to the recorded values; every ``invalid`` block must be refused when the config is read. """ from __future__ import annotations @@ -37,6 +37,7 @@ def test_wire_block_resolves(case: dict[str, object], tmp_path: Path) -> None: assert cfg is not None assert cfg.wire.decimals == case["decimals"] assert cfg.wire.null_members == case["nullMembers"] + assert cfg.wire.durations == case["durations"] @pytest.mark.parametrize("case", _INVALID, ids=lambda c: c["name"]) @@ -51,4 +52,4 @@ def test_the_corpora_pin_the_portable_form() -> None: for corpus in ("api-contract-conformance", "persistence-conformance"): cfg = read_neutral_config(REPO / "fixtures" / corpus) assert cfg is not None - assert (cfg.wire.decimals, cfg.wire.null_members) == ("string", "omit"), corpus + assert (cfg.wire.decimals, cfg.wire.null_members, cfg.wire.durations) == ("string", "omit", "iso"), corpus diff --git a/server/python/tests/integration/normalization.py b/server/python/tests/integration/normalization.py index 067b0edc08..1e7e42499a 100644 --- a/server/python/tests/integration/normalization.py +++ b/server/python/tests/integration/normalization.py @@ -5,7 +5,7 @@ This is the **serialization boundary** for the Python port (ADR-0019): the runtime ``ObjectManager`` returns native pg8000 values (``int`` / ``Decimal`` / -``datetime`` / ``date`` / ``time`` / ``uuid.UUID`` / ``dict`` / ``list``); the +``datetime`` / ``date`` / ``time`` / ``timedelta`` / ``uuid.UUID`` / ``dict`` / ``list``); the canonicalization to the cross-port wire form lives HERE, keyed by Python native type — mirroring the other ports' by-native-type runners (Java keys off ``Long`` vs ``Integer`` / ``BigDecimal`` / ``OffsetDateTime``; TS keys off @@ -30,6 +30,8 @@ from collections.abc import Mapping from typing import Any +from metaobjects.codegen.runtime.duration_wire import format_duration + # Stable Postgres type OIDs (see pg_type.dat) for the int8 family — the BIGINT→ # string wire discriminator. INTEGER (23) / SMALLINT (21) stay JSON numbers. _PG_OID_INT8 = 20 # BIGINT / BIGSERIAL @@ -80,6 +82,11 @@ def normalize_value(v: Any, column_oid: int | None = None, *, actual: bool = Fal return str(v).lower() if isinstance(v, (bytes, bytearray, memoryview)): return base64.b64encode(bytes(v)).decode("ascii") + if isinstance(v, datetime.timedelta): + # field.duration: the canonical ISO-8601 duration (fixtures/duration-wire), through the + # production codec. pg8000 reads an interval as a timedelta; the runtime decodes a + # SQLite / MySQL text column to one too. + return format_duration(v) if isinstance(v, datetime.datetime): # The contract distinguishes TIMESTAMP from TIMESTAMPTZ by tzinfo # (ADR-0019): pg8000 surfaces a TIMESTAMPTZ as a tz-aware datetime and a diff --git a/server/python/tests/integration/report_lane.py b/server/python/tests/integration/report_lane.py index 3fc4f672f9..ebb394522c 100644 --- a/server/python/tests/integration/report_lane.py +++ b/server/python/tests/integration/report_lane.py @@ -76,6 +76,12 @@ def reseed(self, *, empty: bool) -> None: f"SELECT setval(pg_get_serial_sequence('{d.quote(table)}', 'id'), " f"COALESCE((SELECT MAX({d.quote('id')}) FROM {d.quote(table)}), 0) + 1, false)" ) + elif self.engine == "sqlite": + # DELETE leaves the AUTOINCREMENT counter where it was: restart an empty-seeded + # table at 1 (the table has no sequence row until its first insert). + for table, seed_rows in self.base_tables.items(): + if not seed_rows: + cur.execute("DELETE FROM sqlite_sequence WHERE name = ?", (table,)) elif self.engine == "mysql": # DELETE leaves AUTO_INCREMENT where it was: restart an empty-seeded table at 1. for table, seed_rows in self.base_tables.items(): @@ -92,9 +98,10 @@ def _has_id(rows: list[dict[str, Any]]) -> bool: def load_seed(corpus_root: Path) -> tuple[dict[str, list[dict[str, Any]]], dict[str, list[dict[str, Any]]]]: parsed = json.loads((corpus_root / "seed.json").read_text()) - reports = parsed.get("reports") + # A corpus with no view (the duration corpus) carries no expected-`reports` half. + reports = parsed.get("reports", {}) if not isinstance(reports, dict): - raise RuntimeError("report seed.json: missing 'reports' object") + raise RuntimeError("report seed.json: 'reports' must be an object") base = {name: rows for name, rows in parsed.items() if name != "reports"} return base, reports diff --git a/server/python/tests/integration/test_api_contract_duration.py b/server/python/tests/integration/test_api_contract_duration.py new file mode 100644 index 0000000000..d10d46964a --- /dev/null +++ b/server/python/tests/integration/test_api_contract_duration.py @@ -0,0 +1,93 @@ +"""Cross-port API contract for ``field.duration`` (D1 to D4). + +Drives ``fixtures/api-contract-conformance/duration/`` against the GENERATED ``Timer`` router +on a REAL database: the corpus' ``schema..sql`` runs verbatim on Postgres, SQLite and +MySQL 8.4, ``timers`` is seeded from ``seed.json`` (empty, so the first POST is id 1), and the +generated consumer seam is an ``ObjectManager`` repository. The corpus config pins +``wire.durations: iso``. A scenario's top-level ``engines:`` is EXACT; without it a scenario +runs on every engine the lane has. Same harness as ``test_api_contract_decimal.py``. +""" +from __future__ import annotations + +import json +from collections.abc import Iterator +from pathlib import Path +from typing import Any + +import pytest +import yaml +from fastapi.testclient import TestClient + +from . import api_contract_assertions +from .generated_report_app import LaneModel, build_generated_report_app +from .mysql_container import MySqlInfo +from .report_lane import ENGINES, ReportLane, open_lane + +_CORPUS = next( + p / "fixtures" / "api-contract-conformance" / "duration" + for p in [Path(__file__).resolve(), *Path(__file__).resolve().parents] + if (p / "fixtures" / "api-contract-conformance" / "duration").is_dir() +) +_MODEL = LaneModel(reports=(), entities=(("Timer", "timer"),)) +_KNOWN_ENGINES = {"postgres", "mysql", "sqlite", "d1"} + + +def _load() -> list[tuple[str, dict[str, Any]]]: + out = [] + for path in sorted((_CORPUS / "scenarios").glob("*.yaml")): + raw = yaml.safe_load(path.read_text()) + engines = raw.get("engines") + if engines is not None and (not isinstance(engines, list) or not engines or set(engines) - _KNOWN_ENGINES): + raise RuntimeError(f"{path.name}: unknown or empty engines: {engines!r}") + out.append((raw["name"], raw)) + return out + + +_SCENARIOS = _load() + + +class _Lane: + def __init__(self, lane: ReportLane) -> None: + self.lane = lane + app, _ = build_generated_report_app(_CORPUS, lane.om, _MODEL) + self.client = TestClient(app) + + +@pytest.fixture(scope="module", params=ENGINES) +def lane(request: pytest.FixtureRequest, tmp_path_factory: pytest.TempPathFactory) -> Iterator[_Lane]: + engine: str = request.param + mysql: MySqlInfo | None = None + if engine == "mysql": + from .mysql_container import reset_database + + mysql = request.getfixturevalue("mysql_server") + reset_database(mysql) + with open_lane( + engine, _CORPUS, sqlite_path=tmp_path_factory.mktemp("duration") / "duration.db", mysql=mysql + ) as opened: + yield _Lane(opened) + + +@pytest.mark.parametrize("name,scenario", _SCENARIOS, ids=[n for n, _ in _SCENARIOS]) +def test_duration_scenario(lane: _Lane, name: str, scenario: dict[str, Any]) -> None: + engines = scenario.get("engines") + if engines is not None and lane.lane.engine not in engines: + pytest.skip(f"{name} does not run on {lane.lane.engine}") + lane.lane.reseed(empty=bool((scenario.get("setup") or {}).get("truncate"))) + for req in scenario.get("requests", []): + kwargs: dict[str, Any] = {} + if req.get("body") is not None: + kwargs["content"] = json.dumps(req["body"]) + kwargs["headers"] = {"content-type": "application/json"} + response = lane.client.request(str(req["method"]).upper(), str(req["path"]), **kwargs) + text = response.text + parsed = api_contract_assertions.parse_body(text) if text else None + expect = req["expect"] + api_contract_assertions.assert_response( + scenario_name=name, + request_id=str(req.get("id", "?")), + expect_status=int(expect["status"]), + expect_body=expect.get("body"), + status=response.status_code, + body=parsed, + ) diff --git a/server/python/tests/integration/test_duration_postgres.py b/server/python/tests/integration/test_duration_postgres.py new file mode 100644 index 0000000000..e20e85789d --- /dev/null +++ b/server/python/tests/integration/test_duration_postgres.py @@ -0,0 +1,93 @@ +"""field.duration on a live Postgres through the ObjectManager: the shared interval codec +vectors against each driver's native read path. Ordering, range filters and the canonical wire +form are gated by ``test_api_contract_duration.py`` (the ``api-contract-conformance/duration`` +corpus), which replaced the bespoke ordering test that lived here.""" +from __future__ import annotations + +import json +from contextlib import closing +from pathlib import Path +from typing import Any + +import pg8000.dbapi as pg8000 +import pytest + +from metaobjects import load_string +from metaobjects.codegen.runtime.duration_wire import format_duration +from metaobjects.naming import NamingRules +from metaobjects.runtime import ObjectManager, PostgresDriver + +from .postgres_container import PostgresContainer + +_META = { + "metadata.root": { + "package": "acme::ops", + "children": [ + { + "object.entity": { + "name": "Hold", + "children": [ + {"source.rdb": {"@table": "holds"}}, + {"field.long": {"name": "id"}}, + {"field.duration": {"name": "holdFor"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, + ], + } + } + ], + } +} + + +_VECTORS = json.loads( + (Path(__file__).resolve().parents[4] / "fixtures" / "duration-wire" / "vectors.json").read_text() +)["codecs"] + + +def _pg8000(info: Any) -> Any: + return pg8000.connect(host=info.host, port=info.port, user=info.user, password=info.password, database=info.database) + + +def _psycopg(info: Any) -> Any: + psycopg = pytest.importorskip("psycopg") + return psycopg.connect( + host=info.host, port=info.port, user=info.user, password=info.password, dbname=info.database, + ) + + +def _psycopg2(info: Any) -> Any: + psycopg2 = pytest.importorskip("psycopg2") + return psycopg2.connect( + host=info.host, port=info.port, user=info.user, password=info.password, dbname=info.database, + ) + + +@pytest.mark.parametrize("connect", [_pg8000, _psycopg, _psycopg2], ids=["pg8000", "psycopg", "psycopg2"]) +def test_an_interval_reads_as_its_span_and_one_with_a_month_part_is_refused(connect: Any) -> None: + """The shared ``postgres`` / ``postgresRefused`` vectors against the native read path: each + interval is stored by Postgres itself and read back through the driver and the + ObjectManager. Left to themselves pg8000 drops the sign of a negative time part + (``-00:01:30`` is +90 s) and psycopg makes a month 30 days; neither reaches a duration.""" + root: Any = load_string(json.dumps(_META)).root + root.naming = NamingRules(columns="literal") + with PostgresContainer() as pg: + with closing(connect(pg.info())) as conn: + cur = conn.cursor() + cur.execute('CREATE TABLE "holds" ("id" BIGINT PRIMARY KEY, "holdFor" INTERVAL)') + conn.commit() + driver = PostgresDriver(conn) + om = ObjectManager(root, driver) + dcur = driver._cursor() + dcur.execute("SELECT ARRAY[CAST('-01:30:00' AS interval), CAST('-00:00:01.5' AS interval)]") + assert [format_duration(d) for d in dcur.fetchone()[0]] == ["-PT1H30M", "-PT1.5S"] + for i, v in enumerate(_VECTORS["postgres"]): + cur.execute('INSERT INTO "holds" VALUES (%s, CAST(%s AS interval))', (i, v["in"])) + conn.commit() + assert format_duration(om.find_by_id("Hold", i)["holdFor"]) == v["out"], v["in"] + refused = [t for t in _VECTORS["postgresRefused"] if any(u in t for u in ("year", "mon"))] + assert refused, "the shared vectors have intervals with a year or month part" + for i, text in enumerate(refused, start=1000): + cur.execute('INSERT INTO "holds" VALUES (%s, CAST(%s AS interval))', (i, text)) + conn.commit() + with pytest.raises(ValueError, match="year or month part"): + om.find_by_id("Hold", i) diff --git a/server/python/tests/runtime/test_duration_wire.py b/server/python/tests/runtime/test_duration_wire.py new file mode 100644 index 0000000000..461d861086 --- /dev/null +++ b/server/python/tests/runtime/test_duration_wire.py @@ -0,0 +1,144 @@ +"""The Python half of the field.duration wire rule against the shared vectors +(``fixtures/duration-wire/vectors.json``), through the production helper: every ``wire`` group, +and the ``codecs`` groups this port has in production code -- ``postgres`` and +``postgresRefused`` (the interval text a JSON value holds once Postgres has written it, and the +pg8000 ``PGInterval`` path) and ``storedMicros`` (SQLite and MySQL). ``postgresText`` is not run: +pg8000 binds a ``timedelta`` natively, so this port never spells an interval as text.""" +from __future__ import annotations + +import datetime as dt +import json +from pathlib import Path + +import pytest + +from metaobjects.codegen.runtime.duration_wire import ( + canonical_duration, + duration_from_driver, + format_duration, + from_stored_micros, + is_duration_literal, + micros_of, + parse_duration, + parse_duration_input, + parse_postgres_interval, + to_stored_micros, +) + +_VECTORS = json.loads( + (Path(__file__).resolve().parents[4] / "fixtures" / "duration-wire" / "vectors.json").read_text() +) +_WIRE = _VECTORS["wire"] +_CODECS = _VECTORS["codecs"] +#: The ``wire.precision`` entry this port checks: a ``timedelta`` holds microseconds. +_PRECISION_KEY = "6" + + +@pytest.mark.parametrize("v", _WIRE["parse"], ids=lambda v: v["in"]) +def test_parse_answers_the_canonical_form(v: dict[str, str]) -> None: + assert is_duration_literal(v["in"]) + assert canonical_duration(v["in"]) == v["out"] + assert format_duration(parse_duration(v["in"])) == v["out"] + # The canonical form is a fixed point, and what a text column holds reads back as written. + assert canonical_duration(v["out"]) == v["out"] + assert format_duration(duration_from_driver(v["out"])) == v["out"] + + +@pytest.mark.parametrize("text", _WIRE["refused"], ids=lambda t: repr(t)) +def test_refused_is_not_a_duration(text: str) -> None: + assert not is_duration_literal(text) + assert canonical_duration(text) is None + with pytest.raises(ValueError): + parse_duration(text) + with pytest.raises(ValueError): + parse_duration_input(text) + + +@pytest.mark.parametrize("v", _CODECS["postgres"], ids=lambda v: v["in"]) +def test_postgres_interval_text_reads_as_canonical(v: dict[str, str]) -> None: + assert format_duration(parse_postgres_interval(v["in"])) == v["out"] + assert format_duration(duration_from_driver(v["in"])) == v["out"] + + +@pytest.mark.parametrize("text", _CODECS["postgresRefused"], ids=lambda t: t) +def test_postgres_interval_with_a_month_part_is_refused(text: str) -> None: + with pytest.raises(ValueError): + parse_postgres_interval(text) + with pytest.raises(ValueError): + duration_from_driver(text) + + +@pytest.mark.parametrize("v", _WIRE["precision"], ids=lambda v: v["in"]) +def test_a_fraction_of_any_length_is_truncated_to_microseconds(v: dict[str, object]) -> None: + out = v["out"][_PRECISION_KEY] # type: ignore[index] + assert is_duration_literal(v["in"]) # type: ignore[arg-type] + assert canonical_duration(v["in"]) == out # type: ignore[arg-type] + assert format_duration(parse_duration(v["in"])) == out # type: ignore[arg-type] + + +def test_a_native_timedelta_is_formatted_canonically() -> None: + assert format_duration(dt.timedelta(minutes=90)) == "PT1H30M" + assert format_duration(dt.timedelta(seconds=-90)) == "-PT1M30S" + assert format_duration(dt.timedelta(days=1, seconds=-1)) == "PT23H59M59S" + assert format_duration(dt.timedelta(0)) == "PT0S" + assert format_duration(dt.timedelta(microseconds=1)) == "PT0.000001S" + assert duration_from_driver(dt.timedelta(hours=2)) == dt.timedelta(hours=2) + + +def test_a_pg8000_interval_with_a_month_part_is_refused() -> None: + pg8000_types = pytest.importorskip("pg8000.types") + with pytest.raises(ValueError, match="year or month part"): + duration_from_driver(pg8000_types.PGInterval(months=1, days=2)) + assert format_duration(duration_from_driver(pg8000_types.PGInterval(days=1, hours=2))) == "PT26H" + + +@pytest.mark.parametrize("value", [90, 1.5, True, None, ["PT1H"]], ids=repr) +def test_a_non_string_input_is_not_a_duration(value: object) -> None: + with pytest.raises(ValueError): + parse_duration_input(value) + + +def test_an_out_of_range_duration_is_a_value_error() -> None: + with pytest.raises(ValueError): + parse_duration("PT99999999999999999999H") + + +@pytest.mark.parametrize("v", _CODECS["storedMicros"], ids=lambda v: v["in"]) +def test_stored_micros(v: dict[str, str]) -> None: + td = parse_duration(v["in"]) + assert to_stored_micros(td) == int(v["out"]) + # What the column holds reads back as the input, less the digits a microsecond count drops. + stored = format_duration(from_stored_micros(int(v["out"]))) + assert stored == canonical_duration(v["in"]) + assert format_duration(duration_from_driver(int(v["out"]))) == stored + + +# ── this port's own limits (not shared vectors) ────────────────────────────── + + +def test_the_timedelta_range_is_the_only_limit_and_past_it_is_refused() -> None: + # Far past what another port's native type holds: Python takes it. + assert format_duration(parse_duration("PT256204779H")) == "PT256204779H" + edge = f"P{999_999_999}DT23H59M59.999999S" + assert parse_duration(edge) == dt.timedelta.max + assert parse_duration(f"-P{999_999_999}D") == dt.timedelta.min + for text in (f"P{1_000_000_000}D", f"P{999_999_999}DT24H", f"-P{999_999_999}DT0.000001S"): + assert not is_duration_literal(text) + assert canonical_duration(text) is None + with pytest.raises(ValueError, match="out of range"): + parse_duration(text) + with pytest.raises(ValueError, match="out of range"): + from_stored_micros(micros_of(dt.timedelta.max) + 1) + + +def test_a_stored_micro_count_past_64_bits_is_refused() -> None: + assert to_stored_micros(dt.timedelta(microseconds=2**63 - 1)) == 2**63 - 1 + assert to_stored_micros(dt.timedelta(microseconds=-(2**63))) == -(2**63) + for td in (dt.timedelta(microseconds=2**63), dt.timedelta(microseconds=-(2**63) - 1), dt.timedelta.max): + with pytest.raises(ValueError, match="64-bit"): + to_stored_micros(td) + + +def test_a_bool_is_not_a_stored_micro_count() -> None: + with pytest.raises(ValueError): + duration_from_driver(True) diff --git a/server/python/tests/runtime/test_object_manager_duration.py b/server/python/tests/runtime/test_object_manager_duration.py new file mode 100644 index 0000000000..6b08834ef9 --- /dev/null +++ b/server/python/tests/runtime/test_object_manager_duration.py @@ -0,0 +1,252 @@ +"""field.duration through the ObjectManager: the write codec (an accepted ISO-8601 string becomes a +``timedelta``; anything else is refused), the per-engine binding (pg8000 binds the ``timedelta`` +as an interval, SQLite and MySQL an integer column holding the microsecond count, so it orders) +and the read codec (every engine reads back a ``timedelta``; an interval with a month part is +refused). A duration inside a jsonb value object stays the canonical ISO string. + +The live Postgres lane is ``tests/integration/test_query_scenarios.py`` +(``roundtrip-all-types``, ``update-delete-all-types``). +""" +from __future__ import annotations + +import datetime as dt +import json +import sqlite3 +from typing import Any + +import pytest + +from metaobjects import load_string +from metaobjects.naming import NamingRules +from metaobjects.runtime import MYSQL, POSTGRES, SQLITE, ObjectManager, SqliteDriver +from metaobjects.runtime.dialect import DurationParam +from metaobjects.runtime.object_manager import _coerce_write_value, _decode_read_value + +_META = { + "metadata.root": { + "package": "acme::ops", + "children": [ + { + "object.entity": { + "name": "Hold", + "children": [ + {"source.rdb": {"@table": "holds"}}, + {"field.long": {"name": "id"}}, + {"field.duration": {"name": "holdFor"}}, + {"field.duration": {"name": "steps", "isArray": True}}, + {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, + ], + } + } + ], + } +} + + +def _root() -> Any: + root: Any = load_string(json.dumps(_META)).root + root.naming = NamingRules(columns="literal") + return root + + +def _field(name: str) -> Any: + return next(f for f in _root().children()[0].fields() if f.name == name) + + +def _sqlite_om() -> tuple[ObjectManager, sqlite3.Connection]: + conn = sqlite3.connect(":memory:") + conn.execute('CREATE TABLE "holds" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "holdFor" INTEGER, "steps" TEXT)') + return ObjectManager(_root(), SqliteDriver(conn)), conn + + +def test_an_accepted_iso_duration_is_written_as_a_timedelta() -> None: + f = _field("holdFor") + assert _coerce_write_value(f, "P1DT0.5S") == dt.timedelta(days=1, milliseconds=500) + assert _coerce_write_value(f, "-PT90S") == dt.timedelta(seconds=-90) + assert _coerce_write_value(f, dt.timedelta(minutes=5)) == dt.timedelta(minutes=5) + assert _coerce_write_value(_field("steps"), ["PT1M", None]) == [dt.timedelta(minutes=1), None] + + +@pytest.mark.parametrize("value", ["P1M", "1:30:00", "PT1.5H", "pt1h", 90, 1.5]) +def test_a_value_that_is_not_a_duration_is_refused_on_write(value: object) -> None: + with pytest.raises(ValueError, match="field.duration 'holdFor' cannot be written"): + _coerce_write_value(_field("holdFor"), value) + + +def test_postgres_binds_the_native_timedelta() -> None: + td = dt.timedelta(seconds=-90) + assert POSTGRES.encode_param(td) is td + + +@pytest.mark.parametrize("dialect", [SQLITE, MYSQL], ids=lambda d: d.name) +def test_an_integer_engine_binds_the_microsecond_count(dialect: Any) -> None: + f = _field("holdFor") + assert dialect.encode_param(_coerce_write_value(f, "P1DT0.5S")) == 86_400_500_000 + assert dialect.encode_param(_coerce_write_value(f, dt.timedelta(seconds=-90))) == -90_000_000 + # Past the 64-bit column is refused (the column's limit), never wrapped. + with pytest.raises(ValueError, match="64-bit"): + dialect.encode_param(_coerce_write_value(f, dt.timedelta.max)) + + +@pytest.mark.parametrize("dialect", [SQLITE, MYSQL], ids=lambda d: d.name) +def test_the_integer_binding_is_by_field_not_by_value(dialect: Any) -> None: + # A timedelta the duration write codec did not make (a caller's value for some other + # column, an operand on an undeclared one) binds exactly as it did before. + td = dt.timedelta(minutes=5) + assert dialect.encode_param(td) is td + assert isinstance(_coerce_write_value(_field("holdFor"), "PT5M"), DurationParam) + + +def test_only_the_timedelta_range_limits_a_duration() -> None: + f = _field("holdFor") + assert _coerce_write_value(f, "PT256204779H") == dt.timedelta(hours=256_204_779) + assert _decode_read_value(f, "256204779:00:00") == dt.timedelta(hours=256_204_779) + with pytest.raises(ValueError, match="cannot be written.*out of range"): + _coerce_write_value(f, "P1000000000D") + with pytest.raises(ValueError, match="out of range"): + _decode_read_value(f, 10**20) + + +def test_every_engine_reads_back_a_timedelta() -> None: + f = _field("holdFor") + assert _decode_read_value(f, dt.timedelta(hours=26)) == dt.timedelta(hours=26) # pg8000 + assert _decode_read_value(f, -90_000_000) == dt.timedelta(seconds=-90) # an integer column + assert _decode_read_value(f, "1 day 02:00:00") == dt.timedelta(hours=26) # interval text + assert _decode_read_value(_field("steps"), [dt.timedelta(minutes=1), None]) == [dt.timedelta(minutes=1), None] + + +def test_an_interval_with_a_month_part_is_refused_on_read() -> None: + pg8000_types = pytest.importorskip("pg8000.types") + with pytest.raises(ValueError, match="year or month part"): + _decode_read_value(_field("holdFor"), pg8000_types.PGInterval(months=1)) + with pytest.raises(ValueError, match="year or month part"): + _decode_read_value(_field("holdFor"), "1 mon 00:00:01") + + +def test_sqlite_round_trip_stores_the_microsecond_count() -> None: + om, conn = _sqlite_om() + created = om.create("Hold", {"holdFor": "P1DT0.5S"}) + assert created["holdFor"] == dt.timedelta(days=1, milliseconds=500) + assert conn.execute('SELECT "holdFor" FROM "holds"').fetchone() == (86_400_500_000,) + (row,) = om.find_many("Hold", {"holdFor": {"eq": "PT24H0.5S"}}) + assert row["holdFor"] == dt.timedelta(hours=24, milliseconds=500) + # Equal spans are equal integers, so a filter in another accepted spelling still matches. + assert len(om.find_many("Hold", {"holdFor": {"in": ["P1DT0.5S", "PT1H"]}})) == 1 + + +def test_sqlite_orders_and_range_filters_a_duration() -> None: + om, _ = _sqlite_om() + # Inserted out of order, and spelled so that text order would differ from span order. + for d in ["PT2H", "-PT1M30S", "PT90M", "PT45S", "P1D", "PT0S"]: + om.create("Hold", {"holdFor": d}) + om.create("Hold", {}) + + def spans(rows: list[dict[str, Any]]) -> list[str | None]: + from metaobjects.codegen.runtime.duration_wire import format_duration + + return [None if r["holdFor"] is None else format_duration(r["holdFor"]) for r in rows] + + asc = om.find_many("Hold", sort=[("holdFor", "asc")]) + assert spans(asc) == ["-PT1M30S", "PT0S", "PT45S", "PT1H30M", "PT2H", "PT24H", None] + desc = om.find_many("Hold", sort=[("holdFor", "desc")]) + assert spans(desc) == [None, "PT24H", "PT2H", "PT1H30M", "PT45S", "PT0S", "-PT1M30S"] + + ranged = om.find_many( + "Hold", {"holdFor": {"gt": "PT0S", "lte": "PT120M"}}, sort=[("holdFor", "asc")] + ) + assert spans(ranged) == ["PT45S", "PT1H30M", "PT2H"] + assert spans(om.find_many("Hold", {"holdFor": {"gte": "PT2H"}}, sort=[("holdFor", "asc")])) == ["PT2H", "PT24H"] + assert spans(om.find_many("Hold", {"holdFor": {"lt": "PT0S"}})) == ["-PT1M30S"] + assert om.count("Hold", {"holdFor": {"ne": "PT90M"}}) == 5 + + +# ── a duration inside a jsonb value object ─────────────────────────────────── + +_VO_META = { + "metadata.root": { + "package": "acme::ops", + "children": [ + {"object.value": {"name": "Inner", "children": [{"field.duration": {"name": "wait"}}]}}, + { + "object.value": { + "name": "Window", + "children": [ + {"field.string": {"name": "label"}}, + {"field.duration": {"name": "timeout"}}, + {"field.duration": {"name": "steps", "isArray": True}}, + {"field.object": {"name": "inner", "@objectRef": "Inner"}}, + ], + } + }, + { + "object.entity": { + "name": "Plan", + "children": [ + {"source.rdb": {"@table": "plans"}}, + {"field.long": {"name": "id"}}, + {"field.object": {"name": "settings", "@objectRef": "Window", "@storage": "jsonb"}}, + {"field.object": {"name": "windows", "@objectRef": "Window", "@storage": "jsonb", "isArray": True}}, + {"field.map": {"name": "byName", "@objectRef": "Window"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, + ], + } + }, + ], + } +} + + +def _plan_om() -> tuple[ObjectManager, sqlite3.Connection]: + result = load_string(json.dumps(_VO_META)) + assert not result.errors, [str(e) for e in result.errors] + root: Any = result.root + root.naming = NamingRules(columns="literal") + conn = sqlite3.connect(":memory:") + conn.execute( + 'CREATE TABLE "plans" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, ' + '"settings" TEXT, "windows" TEXT, "byName" TEXT)' + ) + return ObjectManager(root, SqliteDriver(conn)), conn + + +def _window(timeout: Any, steps: list[Any], wait: Any) -> dict[str, Any]: + return {"label": "w", "timeout": timeout, "steps": steps, "inner": {"wait": wait}} + + +def test_a_value_object_duration_is_stored_and_read_canonical() -> None: + om, conn = _plan_om() + created = om.create("Plan", { + "settings": _window("PT90M", ["PT5400S", None], "P1D"), + "windows": [_window(dt.timedelta(seconds=-90), [], "PT0.500S")], + "byName": {"a": _window("PT1H", ["P1DT2H"], None)}, + }) + stored = dict(zip(("settings", "windows", "byName"), conn.execute( + 'SELECT "settings", "windows", "byName" FROM "plans"' + ).fetchone())) + expected = { + "settings": _window("PT1H30M", ["PT1H30M", None], "PT24H"), + "windows": [_window("-PT1M30S", [], "PT0.5S")], + "byName": {"a": _window("PT1H", ["PT26H"], None)}, + } + assert {k: json.loads(v) for k, v in stored.items()} == expected + assert {k: created[k] for k in expected} == expected + (row,) = om.find_many("Plan") + assert {k: row[k] for k in expected} == expected + + +def test_a_value_object_duration_stored_non_canonical_reads_back_canonical() -> None: + om, conn = _plan_om() + conn.execute( + 'INSERT INTO "plans" ("settings") VALUES (?)', + (json.dumps({"timeout": "PT90M", "steps": ["P1D"], "inner": {"wait": "PT60S"}}),), + ) + (row,) = om.find_many("Plan") + assert row["settings"] == {"timeout": "PT1H30M", "steps": ["PT24H"], "inner": {"wait": "PT1M"}} + + +def test_a_value_object_member_that_is_not_a_duration_is_refused_on_write() -> None: + om, _ = _plan_om() + with pytest.raises(ValueError, match="field.duration 'timeout' cannot be written"): + om.create("Plan", {"settings": {"timeout": "P1M"}}) + with pytest.raises(ValueError, match="field.duration 'wait' cannot be written"): + om.create("Plan", {"windows": [{"inner": {"wait": "P1000000000D"}}]}) diff --git a/server/python/tests/runtime/test_validator_runner.py b/server/python/tests/runtime/test_validator_runner.py index 2e728fb366..de2560b20a 100644 --- a/server/python/tests/runtime/test_validator_runner.py +++ b/server/python/tests/runtime/test_validator_runner.py @@ -341,6 +341,33 @@ def test_non_string_uri_or_inet_is_a_type_failure_lenient_or_not() -> None: assert _rules(_net(), {"website": 5, "reportedIp": 5}) == ["website:type", "reportedIp:type"] +# ── field.duration ─────────────────────────────────────────────────────────── + + +def _hold() -> MetaObject: + return _load({"field.duration": {"name": "holdFor"}}, {"field.duration": {"name": "steps", "isArray": True}}) + + +def test_duration_accepts_an_iso_duration_of_weeks_days_hours_minutes_and_seconds() -> None: + for d in ["PT1H30M", "P1DT2H", "-PT30S", "+PT2M", "PT0.5S", "PT0S", "P0D", "PT1.123456789S", "P2W", + "P1W1DT1H", "PT1.1234567890123S"]: + assert _rules(_hold(), {"holdFor": d}) == [] + + +def test_duration_rejects_calendar_spans_clock_text_and_what_timedelta_would_take() -> None: + for d in ["P1M", "P1Y", "PT1W", "P1.5W", "P1D1W", "1:30:00", "PT1.5H", "pt1h", "PT", "P", "", "PT1H\n", "90"]: + assert _errors(_hold(), {"holdFor": d}) == [{ + "field": "holdFor", "rule": "format", + "message": "'holdFor' must be an ISO-8601 duration such as PT1H30M (no years or months)", + "expected": "duration", "received": d, + }] + + +def test_duration_is_a_string_on_the_wire_and_each_array_element_is_checked() -> None: + assert _rules(_hold(), {"holdFor": 90}) == ["holdFor:type"] + assert _rules(_hold(), {"steps": ["PT1M", "P1M"]}) == ["steps[1]:format"] + + # ── assigned primary key ───────────────────────────────────────────────────── diff --git a/server/python/tests/unit/test_column_naming_strategy.py b/server/python/tests/unit/test_column_naming_strategy.py index 63bb83dafd..94c69acb4b 100644 --- a/server/python/tests/unit/test_column_naming_strategy.py +++ b/server/python/tests/unit/test_column_naming_strategy.py @@ -78,7 +78,7 @@ def test_an_unknown_strategy_is_refused_not_silently_defaulted() -> None: # A typo'd strategy would otherwise bind an entire schema to the wrong columns and # report success. with pytest.raises(ValueError, match="snake_case"): - apply_column_naming_strategy("createdAt", "PascalCase") + apply_column_naming_strategy("createdAt", "camelCase") # WHERE THE STRATEGY ACTUALLY REACHES, in this port. @@ -122,3 +122,12 @@ def test_object_manager_column_naming_argument_is_refused() -> None: "ObjectManager(column_naming=...). See " "https://github.com/metaobjectsdev/metaobjects/blob/v1.1.0/docs/features/migrations/physical-naming.md" ) + + +def test_upper_first_keeps_a_first_character_whose_upper_case_is_not_one_character() -> None: + # C# and Java map one UTF-16 unit to one; Python's str.upper gives "SS" for "ß". + from metaobjects.naming import to_upper_first_column + + assert to_upper_first_column("requestURL") == "RequestURL" + assert to_upper_first_column("ßeta") == "ßeta" + assert to_upper_first_column("") == "" diff --git a/server/typescript/packages/cli/src/commands/gen.ts b/server/typescript/packages/cli/src/commands/gen.ts index 6f96d480fc..b5c398f41b 100644 --- a/server/typescript/packages/cli/src/commands/gen.ts +++ b/server/typescript/packages/cli/src/commands/gen.ts @@ -34,6 +34,8 @@ import { describeError } from "../lib/error-text.js"; import { findRuntimeBoundaryCrossings, runtimeBoundaryWarnings } from "../lib/runtime-boundary-advisory.js"; import { hasLegacyDocsBlock, legacyDocsConfigMessage, NO_CONFIG_MESSAGE } from "../lib/docs-removed.js"; import { buildAgentSchemaInput } from "../lib/agent-schema.js"; +import { schemaBuildOptions, schemaBuildOptionsOrWarn, type SchemaBuildOptions } from "../lib/schema-build-options.js"; +import { isUnknownSchemaConventionsError } from "@metaobjectsdev/migrate-ts"; import { requirementPayload } from "../lib/requirement-payload.js"; import { existsSync } from "node:fs"; @@ -215,6 +217,19 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat : []; for (const w of unknownAttrs) { log.warn(`warning: ${w}`); warnings.push(w); } + // The schema settings `meta migrate` builds with, for the Prisma schema, the schema page and + // the migrate advice. A conventions preset that does not resolve is a config error here as in + // `meta migrate` and `meta verify`: generating with the default names instead would write a + // Prisma schema whose keys the database does not have. + let schemaOptions: SchemaBuildOptions; + try { + schemaOptions = schemaBuildOptions(genCollection, forgeConfig); + } catch (err) { + if (!isUnknownSchemaConventionsError(err)) throw err; + log.error(`gen: ${err.message}`); + return 2; + } + // FR-023 §11.1 item 2 — refuse a positional that names ONLY objects imported // from a dependency and excluded from output by the default exclusion rule. // Distinct from the runner's existing "no entities matched" WARNING @@ -259,7 +274,10 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat metadata, forgeConfig.dialect, (m) => { const w = `[agent-docs] ${m}`; log.warn(w); warnings.push(w); }, + schemaOptions, ), + // The Prisma schema names its keys by the same conventions migrate uses. + schemaSettings: schemaOptions, // FR-043 §6 — the shipped-library selection. Always passed, including as `[]`: // that is what tells the post-selection audit "this project opted into none", // which is a different statement from a programmatic caller that never said. @@ -326,6 +344,7 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat const migrateDefaults = await resolveMigrateDefaults(projectRoot); const migrateAdvice = await genMigrateAdvice({ + schema: schemaOptions, metadata, dialect: forgeConfig.dialect, migrateDialect: migrateDefaults.dialect, @@ -504,6 +523,7 @@ async function listCatalogCommand( reportLoadError(log, "failed to load metadata", err); return 1; } + const schemaOptions = schemaBuildOptionsOrWarn(genCollection, forgeConfig, () => {}); opts = { project: { @@ -526,11 +546,14 @@ async function listCatalogCommand( scope: genCollection.inScope, sourceFiles: genCollection.ownFiles, sourceRoots: sourceRootsOf(genCollection), - agentSchema: () => buildAgentSchemaInput( + agentSchema: () => schemaOptions === undefined ? undefined : buildAgentSchemaInput( metadata, forgeConfig.dialect, - () => {}, + () => {}, + schemaOptions, ), + // The Prisma schema names its keys by the same conventions migrate uses. + ...(schemaOptions !== undefined && { schemaSettings: schemaOptions }), }, }; } else { diff --git a/server/typescript/packages/cli/src/commands/migrate.ts b/server/typescript/packages/cli/src/commands/migrate.ts index a5d8bc30da..9806304313 100644 --- a/server/typescript/packages/cli/src/commands/migrate.ts +++ b/server/typescript/packages/cli/src/commands/migrate.ts @@ -1,4 +1,5 @@ import { resolve as resolvePath } from "node:path"; +import { ENUM_LENGTH_HINT, isEnumLengthRefusal } from "../lib/enum-length-refusal.js"; import { mkdir } from "node:fs/promises"; import { existsSync } from "node:fs"; import { spawn } from "node:child_process"; @@ -12,15 +13,22 @@ import { emitStructured, narrate, toonEncode } from "../lib/format.js"; import { buildKyselyFromUrl, inferDialect, redactUrl } from "../lib/kysely.js"; import { describeExprCanonPath, shadowSession } from "../lib/shadow.js"; import { log } from "../lib/log.js"; +import { warnUnenforcedFkTargets } from "../lib/fk-target-warning.js"; import { loadMemory, resolveCollection, resolveConfigDir, type Collection } from "@metaobjectsdev/sdk"; -import { loadMemoryOptionsFrom, loadMetaobjectsConfig, RemovedNamingConfigError, resolveGenConfigDir } from "../lib/load-metaobjects-config.js"; +import { configNotLoadedWarning, loadMemoryOptionsFrom, loadMetaobjectsConfig, RemovedNamingConfigError, resolveGenConfigDir } from "../lib/load-metaobjects-config.js"; import { collectionLoadOptions } from "../lib/collection-load-options.js"; +import { schemaBuildOptions, type SchemaBuildOptions } from "../lib/schema-build-options.js"; import { exclusionNotes, importedOption, migrateScopeMismatch } from "../lib/migrate-scope.js"; import { isBlockedChangesError, isPrimaryKeyChangeError, isDeclaredRenameError, isDuplicateSqlNameError, + isInvalidCheckNameError, + isFkTargetNotUniqueError, + FkTargetNotUniqueError, + type FkTargetViolation, + isUnknownSchemaConventionsError, buildExpectedSchemaWithProvenance, scopeExpectedSchema, scopedDiffInputs, @@ -50,6 +58,7 @@ import { type AmbiguousResolution, type Change, type DataHazard, + type DiffResult, type Dialect, type D1Binding, type EmitResult, @@ -69,12 +78,13 @@ import { defaultWranglerRunner, type WranglerRunner, } from "../lib/wrangler.js"; -import { buildProjectionViews } from "@metaobjectsdev/codegen-ts"; +import { buildProjectionViews, type MetaobjectsGenConfig } from "@metaobjectsdev/codegen-ts"; import { tokensToAllowOptions, blockedEntriesFor, blockedHintLines, viewRecreateNote } from "../lib/allow.js"; import { reportLoadError } from "../lib/load-error.js"; import { decimalPrecisionWarnings } from "../lib/decimal-precision-advisory.js"; +import { schemaAdvisoryLines } from "../lib/schema-advisories.js"; import { scanForReferentialActionConflicts } from "../lib/referential-action-advisory.js"; -import type { MetaData } from "@metaobjectsdev/metadata"; +import type { MetaData, MetaRoot } from "@metaobjectsdev/metadata"; import { describeError } from "../lib/error-text.js"; /** @@ -288,10 +298,10 @@ function sqlPreview(sql: { up: string; down: string }): string { * child's. Advisory: it never changes the exit code, and a scan that throws is dropped * (the schema build itself reports anything wrong with the model). */ -function warnReferentialActionConflicts(metadata: MetaData, collection: Collection): void { +function warnReferentialActionConflicts(metadata: MetaData, collection: Collection, schema: SchemaBuildOptions): void { let findings; try { - findings = scanForReferentialActionConflicts(metadata, { skip: collection.imported }); + findings = scanForReferentialActionConflicts(metadata, { skip: collection.imported, schema }); } catch { return; } @@ -351,6 +361,16 @@ function mapOnAmbiguous(v: "abort" | "rename" | "drop-add"): AmbiguousResolution return v === "drop-add" ? "drop+add" : v; } +/** + * FKs whose target is not a unique key, on tables this run does not govern (imported, or + * outside migrate.scope). This project never emits them, so they are reported, not refused. + */ +function warnUngovernedFkTargets(violations: readonly FkTargetViolation[] | undefined): void { + if (violations === undefined || violations.length === 0) return; + log.warn(`migrate: ${new FkTargetNotUniqueError([...violations]).message}`); + log.warn(`migrate: the table(s) holding ${violations.length === 1 ? "that foreign key are" : "those foreign keys are"} not governed by this run (imported, or outside migrate.scope), so nothing is refused.`); +} + /** * The report-and-exit for an engine refusal whose message the engine already made * actionable — the sibling of `refuseScopeMismatch` above, which is the same decision @@ -363,13 +383,46 @@ function refuseEngineError(err: Error, hint: string, fmt: OutputFormat): number return 1; } +const SCHEMA_CONVENTIONS_HINT = + "set migrate.conventions in .metaobjects/config.json to one of the presets, or supply your own conventions from metaobjects.config.ts"; + +/** + * The project's schema settings (`migrate.conventions`, a conventions object from + * `metaobjects.config.ts`, `migrate.checks`), or exit 2 after naming the config error: a + * preset this version does not ship, or a conventions object missing a member. + */ +function resolveSchemaBuild( + collection: Collection, + genConfig: MetaobjectsGenConfig | undefined, + fmt: OutputFormat, +): SchemaBuildOptions | number { + try { + return schemaBuildOptions(collection, genConfig); + } catch (err) { + if (!isUnknownSchemaConventionsError(err)) throw err; + refuseEngineError(err, SCHEMA_CONVENTIONS_HINT, fmt); + return 2; + } +} + const PK_CHANGE_HINT = "align the primary key manually, or reconcile the metadata identity to match the live table"; const DUPLICATE_SQL_NAME_HINT = "rename one of the colliding declarations; an index's database name is its identity.secondary / index.lookup name, not prefixed with its table"; +const CHECK_NAME_HINT = + "give each constraint on a table its own @constraintName, at most 63 bytes"; +const FK_TARGET_NOT_UNIQUE_HINT = + "declare an identity.secondary on the target entity with exactly the referenced fields, or reference its primary key"; const DECLARED_RENAME_HINT = "fix or remove the --rename-table / --rename-column flag; a table's name is its name in the metadata (the new name when it is renamed too)"; +/** Print the diff's advice that is not drift (`schemaAdvisoryLines`), one warning each. */ +function warnAdvisories(advisories: DiffResult["advisories"] | undefined): string[] { + const lines = schemaAdvisoryLines(advisories); + for (const w of lines) log.warn(`migrate: ${w}`); + return lines; +} + /** * Warn about changes that apply to an empty table and fail on a populated one. Warn, not * refuse: the diff cannot see rows, and a new required column on an empty table is fine. @@ -736,13 +789,17 @@ export async function migrateCommand( // providers. Column naming is not read here: it is `naming` in // `.metaobjects/config.json`, set on the loaded root (ADR-0058). let postgresLoadOptions: ReturnType = {}; + let postgresGenConfig: MetaobjectsGenConfig | undefined; try { - postgresLoadOptions = loadMemoryOptionsFrom(await loadMetaobjectsConfig(genRoot)); + postgresGenConfig = await loadMetaobjectsConfig(genRoot); + postgresLoadOptions = loadMemoryOptionsFrom(postgresGenConfig); } catch (e) { if (e instanceof RemovedNamingConfigError) { log.error(e.message); return 2; } + const warning = configNotLoadedWarning(genRoot, e, "migrate"); + if (warning !== undefined) log.warn(warning); postgresLoadOptions = {}; } @@ -759,6 +816,9 @@ export async function migrateCommand( return 2; } + const schema = resolveSchemaBuild(collection, postgresGenConfig, fmt); + if (typeof schema === "number") return schema; + let metadata; try { metadata = await loadMemory(collection.configDir, { @@ -769,7 +829,7 @@ export async function migrateCommand( reportLoadError(log, "failed to load metadata", err); return 1; } - warnReferentialActionConflicts(metadata, collection); + warnReferentialActionConflicts(metadata, collection, schema); let kysely; try { @@ -806,6 +866,7 @@ export async function migrateCommand( // there is no separate view-migration emitter. const expectedViews = buildProjectionViews(metadata, { dialect: kysely.dialect }); const built = buildExpectedSchemaWithProvenance(metadata, { + ...schema, dialect: kysely.dialect, views: expectedViews, }); @@ -818,6 +879,8 @@ export async function migrateCommand( const scoped = scopeExpectedSchema(built, collection.inMigrateScope, importedOption(collection)); const expected = scoped.snapshot; logOutOfScope(scoped.outOfScope, scoped.importedOutOfScope ?? [], fmt); + warnUngovernedFkTargets(scoped.ungovernedFkTargetViolations); + warnUnenforcedFkTargets("migrate", scoped.unenforcedFkTargetViolations); let actual; try { actual = await introspect(kysely.db, kysely.dialect); @@ -916,7 +979,7 @@ export async function migrateCommand( } changeCounts = summarizeChanges(diffResult.changes); - hazardWarnings = warnDataHazards(diffResult.hazards); + hazardWarnings = [...warnDataHazards(diffResult.hazards), ...warnAdvisories(diffResult.advisories)]; // #313 — refuse to AUTHOR a drop for an object the committed snapshot never // contained. This path diffs metadata against introspection and never reads the @@ -1151,6 +1214,12 @@ export async function migrateCommand( if (err instanceof AlreadyEmittedError) return err.exitCode; // A metadata defect, not a tool failure: two declarations generate one database name. if (isDuplicateSqlNameError(err)) return refuseEngineError(err, DUPLICATE_SQL_NAME_HINT, fmt); + // A metadata defect too: a declared CHECK name that duplicates another constraint or is too long. + if (isInvalidCheckNameError(err)) return refuseEngineError(err, CHECK_NAME_HINT, fmt); + // A metadata defect too: a foreign key whose target columns are not a unique key there. + if (isFkTargetNotUniqueError(err)) return refuseEngineError(err, FK_TARGET_NOT_UNIQUE_HINT, fmt); + // A metadata defect too: an enum member longer than the project's enums.maxLength. + if (isEnumLengthRefusal(err)) return refuseEngineError(err, ENUM_LENGTH_HINT, fmt); // Unexpected error: emit structured error on stdout in the active format, then exit 1. const msg = describeError(err) ?? String(err); log.error(`migrate: unexpected error: ${msg}`); @@ -1167,20 +1236,28 @@ export async function migrateCommand( * * Throws what loading throws; a `RemovedNamingConfigError` from the config propagates unchanged. */ -async function loadBaselineMetadata(metaRoot: string, genRoot: string) { +async function loadBaselineMetadata( + metaRoot: string, + genRoot: string, +): Promise<{ metadata: MetaRoot; schema: SchemaBuildOptions }> { let baselineLoadOptions: ReturnType = {}; + let cfg: MetaobjectsGenConfig | undefined; try { - const cfg = await loadMetaobjectsConfig(genRoot); + cfg = await loadMetaobjectsConfig(genRoot); baselineLoadOptions = loadMemoryOptionsFrom(cfg); } catch (e) { if (e instanceof RemovedNamingConfigError) throw e; // config absent — no custom providers, no libraries, default snake_case + const warning = configNotLoadedWarning(genRoot, e, "migrate"); + if (warning !== undefined) log.warn(warning); } const collection = await resolveCollection(metaRoot); - return await loadMemory(collection.configDir, { + const schema = schemaBuildOptions(collection, cfg); + const metadata = await loadMemory(collection.configDir, { ...collectionLoadOptions(collection), ...baselineLoadOptions, }); + return { metadata, schema }; } /** @@ -1201,8 +1278,8 @@ async function withGeneratedSpelling( ): Promise<{ snapshot: SchemaSnapshot; path?: ExprCanonPath; shadowUnavailable?: string }> { let expected: SchemaSnapshot; try { - const metadata = await loadBaselineMetadata(metaRoot, genRoot); - expected = baselineFromMetadata(metadata, dialect, buildProjectionViews(metadata, { dialect })); + const { metadata, schema } = await loadBaselineMetadata(metaRoot, genRoot); + expected = baselineFromMetadata(metadata, dialect, buildProjectionViews(metadata, { dialect }), schema); } catch { return { snapshot }; } @@ -1313,18 +1390,23 @@ export async function runBaseline( // would delete the entry, and removing the scope later would then propose // CREATE TABLE for a table that exists. let metadata; + let schema; try { - metadata = await loadBaselineMetadata(metaRoot, genRoot); + ({ metadata, schema } = await loadBaselineMetadata(metaRoot, genRoot)); } catch (err) { if (err instanceof RemovedNamingConfigError) { log.error(err.message); return 2; } + if (isUnknownSchemaConventionsError(err)) { + refuseEngineError(err, SCHEMA_CONVENTIONS_HINT, fmt); + return 2; + } reportLoadError(log, "migrate baseline: failed to load metadata", err); return 1; } const baselineViews = buildProjectionViews(metadata, { dialect: config.dialect }); - snapshot = baselineFromMetadata(metadata, config.dialect, baselineViews); + snapshot = baselineFromMetadata(metadata, config.dialect, baselineViews, schema); } if (config.dryRun) { @@ -1463,15 +1545,18 @@ export async function runOfflineGenerate( // mirroring the DB path (and `meta gen`) so offline generate resolves // config-registered custom subtypes too (#157). let offlineLoadOptions: ReturnType = {}; + let offlineGenConfig: MetaobjectsGenConfig | undefined; try { - const cfg = await loadMetaobjectsConfig(genRoot); - offlineLoadOptions = loadMemoryOptionsFrom(cfg); + offlineGenConfig = await loadMetaobjectsConfig(genRoot); + offlineLoadOptions = loadMemoryOptionsFrom(offlineGenConfig); } catch (e) { if (e instanceof RemovedNamingConfigError) { log.error(e.message); return 2; } // config absent — no custom providers, no libraries, default snake_case + const warning = configNotLoadedWarning(genRoot, e, "migrate"); + if (warning !== undefined) log.warn(warning); } let metadata; @@ -1486,7 +1571,9 @@ export async function runOfflineGenerate( reportLoadError(log, "migrate: failed to load metadata", err); return 1; } - warnReferentialActionConflicts(metadata, collection); + const schema = resolveSchemaBuild(collection, offlineGenConfig, fmt); + if (typeof schema === "number") return schema; + warnReferentialActionConflicts(metadata, collection, schema); warnDecimalPrecision(metadata, collection, config.dialect); const offlineDialect = config.dialect; @@ -1494,6 +1581,7 @@ export async function runOfflineGenerate( const scopeRc = refuseScopeMismatch( collection, () => buildExpectedSchemaWithProvenance(metadata, { + ...schema, dialect: offlineDialect, views: offlineViews, }).provenance, @@ -1534,6 +1622,7 @@ export async function runOfflineGenerate( let plan; try { plan = await planOffline({ + ...schema, metadata, dialect: config.dialect, snapshot, @@ -1594,7 +1683,7 @@ export async function runOfflineGenerate( finishMigrate(fmt, [`migrate: no changes`, ...narratedNotes(offlineNotes)], migrateResultToData(offlineResult({}))); return 0; } - const offlineWarnings = warnDataHazards(diffResult.hazards); + const offlineWarnings = [...warnDataHazards(diffResult.hazards), ...warnAdvisories(diffResult.advisories)]; const emitResult = emit(diffResult.changes, { dialect: config.dialect, @@ -1761,13 +1850,17 @@ async function runD1Migrate( // 3. Load metadata. Best-effort config read for consumer providers; falls // back to default core+forge bundle if metaobjects.config.ts is absent. let d1LoadOptions: ReturnType = {}; + let d1GenConfig: MetaobjectsGenConfig | undefined; try { - d1LoadOptions = loadMemoryOptionsFrom(await loadMetaobjectsConfig(genRoot)); + d1GenConfig = await loadMetaobjectsConfig(genRoot); + d1LoadOptions = loadMemoryOptionsFrom(d1GenConfig); } catch (e) { if (e instanceof RemovedNamingConfigError) { log.error(e.message); return 2; } + const warning = configNotLoadedWarning(genRoot, e, "migrate"); + if (warning !== undefined) log.warn(warning); d1LoadOptions = {}; } @@ -1792,18 +1885,22 @@ async function runD1Migrate( reportLoadError(log, "migrate: failed to load metadata", err); return 1; } - warnReferentialActionConflicts(metadata, collection); + const schema = resolveSchemaBuild(collection, d1GenConfig, fmt); + if (typeof schema === "number") return schema; + warnReferentialActionConflicts(metadata, collection, schema); warnDecimalPrecision(metadata, collection, "d1"); // 4. Build expected schema + introspect actual. const expectedViews = buildProjectionViews(metadata, { dialect: "d1" }); - const built = buildExpectedSchemaWithProvenance(metadata, { dialect: "d1", views: expectedViews }); + const built = buildExpectedSchemaWithProvenance(metadata, { ...schema, dialect: "d1", views: expectedViews }); const scopeRc = refuseScopeMismatch(collection, () => built.provenance, fmt); if (scopeRc !== undefined) return scopeRc; // Per-command scope — both-sided, exactly as on the Kysely path above. const scoped = scopeExpectedSchema(built, collection.inMigrateScope, importedOption(collection)); const expected = scoped.snapshot; logOutOfScope(scoped.outOfScope, scoped.importedOutOfScope ?? [], fmt); + warnUngovernedFkTargets(scoped.ungovernedFkTargetViolations); + warnUnenforcedFkTargets("migrate", scoped.unenforcedFkTargetViolations); let actual; try { actual = await introspectD1({ diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index 7a0c181fdd..f06809878e 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -7,11 +7,13 @@ // silently break a prompt" guarantee, enforced at the last fixed point before // the text ships. Required-slot misses are warnings (don't fail the build). +import { ENUM_LENGTH_HINT, isEnumLengthRefusal } from "../lib/enum-length-refusal.js"; import { existsSync } from "node:fs"; import { join, relative, resolve as resolvePath } from "node:path"; import { parseVerifyArgs, type MigrateFlags } from "../lib/args.js"; import { log } from "../lib/log.js"; import { requirementLedgerRows, type RequirementLedgerRow } from "../lib/requirement-ledger.js"; +import { warnUnenforcedFkTargets } from "../lib/fk-target-warning.js"; import { requirementCountsOf, requirementDiagnosticRow, requirementPayload, type RequirementCounts, type RequirementDiagnosticRow } from "../lib/requirement-payload.js"; import { emitStructured, type OutputFormat } from "../lib/format.js"; import { @@ -35,10 +37,12 @@ import { FileProvider, templateReferenceLine } from "../lib/file-provider.js"; import { derivePayloadFieldTree } from "../lib/payload-field-tree.js"; import { loadMemoryOptionsFrom, loadMetaobjectsConfig, RemovedNamingConfigError, resolveGenCollection, resolveGenConfigDir } from "../lib/load-metaobjects-config.js"; import { collectionLoadOptions } from "../lib/collection-load-options.js"; +import { schemaBuildOptions, type SchemaBuildOptions } from "../lib/schema-build-options.js"; import { computeCodegenDrift } from "../lib/codegen-drift.js"; import { hasLegacyDocsBlock, legacyDocsConfigMessage, REMOVED_VERIFY_DOCS } from "../lib/docs-removed.js"; import { buildAgentSchemaInput } from "../lib/agent-schema.js"; import { sourceRootsOf } from "./gen.js"; +import { schemaAdvisoryLines } from "../lib/schema-advisories.js"; import { checkRequirements, summariseRequirements, scanRequirements, type Diagnostic, } from "../lib/requirement-check.js"; @@ -93,6 +97,9 @@ import { postgresExprCanonProbe, type ShadowSchemaSource, withoutFlywayHistory, + isFkTargetNotUniqueError, + isInvalidCheckNameError, + isUnknownSchemaConventionsError, } from "@metaobjectsdev/migrate-ts"; import { DEFAULT_METAOBJECTS_DIR, @@ -122,8 +129,10 @@ import { physicalNamingOf, WIRE_DECIMALS, WIRE_NULL_MEMBERS, + WIRE_DURATIONS, WIRE_KEY_DECIMALS, WIRE_KEY_NULL_MEMBERS, + WIRE_KEY_DURATIONS, type WireFormat, } from "@metaobjectsdev/metadata"; import { @@ -169,6 +178,13 @@ function attrAsStringArray(attr: unknown): string[] { return []; } +/** An enum member longer than the project's enums.maxLength: a metadata defect, reported by name. */ +function refuseEnumLength(err: Error): number { + log.error(`verify: ${err.message}`); + log.error(`verify: ${ENUM_LENGTH_HINT}`); + return 1; +} + export async function verifyCommand( args: string[], cwd: string, @@ -396,6 +412,19 @@ export async function verifyCommand( return 1; } + // The project's schema settings, resolved once for every expected-schema build and drift + // computation below. A `migrate.conventions` preset this version does not ship is a config + // error on every verify run, not only one that builds the expected schema (a typo'd preset + // otherwise passes `meta verify` and changes nothing until `meta migrate`). + let schema: SchemaBuildOptions; + try { + schema = schemaBuildOptions(collection, forgeConfig); + } catch (err) { + if (!isUnknownSchemaConventionsError(err)) throw err; + log.error(`verify: ${err.message}`); + return 2; + } + // The schema gate governs exactly the objects `meta migrate` governs — ONE // declaration (`migrate.scope`), not a second key: a drift gate that fails on // tables migrate deliberately does not own is incoherent. Undefined ⇒ everything @@ -709,11 +738,19 @@ export async function verifyCommand( // passes no `governed` and gets the comparison exactly as it was. let governed: GovernedScope | undefined; if (schemaScope !== undefined) { - const built = buildExpectedSchemaWithProvenance(root, { - dialect, - views: buildProjectionViews(root, { dialect }), - }); - governed = scopeExpectedSchema(built, schemaScope, importedOption(collection)); + try { + const built = buildExpectedSchemaWithProvenance(root, { + ...schema, + dialect, + views: buildProjectionViews(root, { dialect }), + }); + governed = scopeExpectedSchema(built, schemaScope, importedOption(collection)); + } catch (err) { + if (isEnumLengthRefusal(err)) return refuseEnumLength(err); + if (isInvalidCheckNameError(err)) { log.error(`verify: ${err.message}`); return 1; } + if (isFkTargetNotUniqueError(err)) return refuseFkTarget(err); + throw err; + } } // `verifyReplay` calls `applyPending` itself. That is NOT a second replay: the @@ -1058,6 +1095,7 @@ export async function verifyCommand( unindexedFks = scanForUnindexedForeignKeys(root, { dialect: scanDialect, skip: collection.imported, + schema, }); } } catch { @@ -1070,7 +1108,7 @@ export async function verifyCommand( // cascade) silently has no effect. Same discipline as its siblings. let actionConflicts: ReferentialActionConflictFinding[] = []; try { - actionConflicts = scanForReferentialActionConflicts(root, { skip: collection.imported }); + actionConflicts = scanForReferentialActionConflicts(root, { skip: collection.imported, schema }); } catch { // advisory only — a model the schema builder refuses is migrate's to report. } @@ -1294,13 +1332,23 @@ export async function verifyCommand( // work at the top of `verifyCommand`, because `migrate.scope` governs only the // schema gate: a stale pattern must not fail a `--templates` run that never // consults it. - const scopeMismatch = migrateScopeMismatch(collection, () => { - const dialect: Dialect = usingD1 ? "d1" : (flags.dialect ?? inferDialect(flags.db as string)); - return buildExpectedSchemaWithProvenance(root, { - dialect, - views: buildProjectionViews(root, { dialect }), - }).provenance; - }); + let scopeMismatch: string | undefined; + try { + scopeMismatch = migrateScopeMismatch(collection, () => { + const dialect: Dialect = usingD1 ? "d1" : (flags.dialect ?? inferDialect(flags.db as string)); + return buildExpectedSchemaWithProvenance(root, { + ...schema, + dialect, + views: buildProjectionViews(root, { dialect }), + }).provenance; + }); + } catch (err) { + // Building the expected schema to check the scope raises the same metadata refusals + // the comparison below would; report them by name, not as an uncaught error. + if (isEnumLengthRefusal(err)) return refuseEnumLength(err); + if (isInvalidCheckNameError(err)) { log.error(`verify: ${err.message}`); return 1; } + throw err; + } if (scopeMismatch !== undefined) { log.error(`verify: ${scopeMismatch}`); return 2; @@ -1375,6 +1423,7 @@ export async function verifyCommand( actual = withoutFlywayHistory(actual); } driftResult = await computeDriftFromActual(actual, kysely.dialect, root, { + ...schema, allow, views: expectedViews, // The same question `meta migrate` asks, so the report names the change it plans. @@ -1388,11 +1437,20 @@ export async function verifyCommand( ...importedOption(collection), }); } catch (err) { + // Metadata refusals raised while building the expected side, not introspection failures. + if (isFkTargetNotUniqueError(err)) return refuseFkTarget(err); + if (isInvalidCheckNameError(err)) { + log.error(`verify: ${err.message}`); + return 1; + } + if (isEnumLengthRefusal(err)) return refuseEnumLength(err); schemaRunError = `failed to introspect ${kysely.displayUrl}: ${describeError(err)}`; log.error(`verify: ${schemaRunError}`); return 1; } + warnUnenforcedFkTargets("verify", driftResult.unenforcedFkTargetViolations); + if (driftResult.exprCanonPath !== undefined) { say(describeExprCanonPath(driftResult.exprCanonPath, driftResult.shadowUnavailable)); } @@ -1480,6 +1538,7 @@ export async function verifyCommand( let driftResult; try { driftResult = await computeDriftFromActual(actual, "d1", root, { + ...schema, allow, views: expectedViews, ...(schemaScope !== undefined ? { inScope: schemaScope } : {}), @@ -1617,6 +1676,8 @@ export async function verifyCommand( say(note); } + for (const line of schemaAdvisoryLines(driftResult.advisories)) log.warn(`verify: ${line}`); + const changes = driftResult.changes; schemaDrift = { changes: changes.map(toSchemaDriftRow), findings: ledgerDrift }; if (changes.length === 0 && ledgerDrift.length === 0) { @@ -1690,6 +1751,16 @@ export async function verifyCommand( return 2; } const config = forgeConfig; + // The schema page describes the schema `meta gen` would, so it takes this package's settings. + // A conventions preset that does not resolve fails here as it does in `meta gen`. + let codegenSchema: SchemaBuildOptions; + try { + codegenSchema = schemaBuildOptions(genCollection, config); + } catch (err) { + if (!isUnknownSchemaConventionsError(err)) throw err; + log.error(`verify --codegen: ${err.message}`); + return 2; + } // The explorer's requirement ledger, built only if the docs site asks for it. const requirementLedger = () => requirementPayload(codegenRoot, genCollection); let result; @@ -1700,7 +1771,9 @@ export async function verifyCommand( codegenRoot, config.dialect, (m) => log.warn(`verify --codegen: [agent-docs] ${m}`), + codegenSchema, ), + schemaSettings: codegenSchema, requirementLedger, }); } catch (err) { @@ -1723,7 +1796,8 @@ export async function verifyCommand( const probe = await computeCodegenDrift(config, codegenRoot, genConfigDir, genCollection.inScope, genCollection.ownFiles, { sourceRoots: sourceRootsOf(genCollection), - agentSchema: () => buildAgentSchemaInput(codegenRoot, config.dialect, () => {}), + agentSchema: () => buildAgentSchemaInput(codegenRoot, config.dialect, () => {}, codegenSchema), + schemaSettings: codegenSchema, requirementLedger, }); return probe.error === undefined && probe.clean; @@ -1879,6 +1953,7 @@ const DRIFT_PRESENTATION: Record { for (const decimals of WIRE_DECIMALS) { for (const nullMembers of WIRE_NULL_MEMBERS) { - if (decimals === used.decimals && nullMembers === used.nullMembers) continue; - const wire: WireFormat = { decimals, nullMembers }; - if (await reproduces(wire)) return wire; + for (const durations of WIRE_DURATIONS) { + if (decimals === used.decimals && nullMembers === used.nullMembers && durations === used.durations) continue; + const wire: WireFormat = { decimals, nullMembers, durations }; + if (await reproduces(wire)) return wire; + } } } return undefined; @@ -1977,7 +2056,7 @@ export function wireDiffersMessage(committed: WireFormat, used: WireFormat): str return ( `The committed output was generated with ${wireBlock(committed)}; this run used ${wireBlock(used)} ` + "(.metaobjects/config.json; each key defaults to native). Do not regenerate: that changes the JSON " + - "your API writes for decimals or for a value object's null members. Declare " + + "your API writes for decimals, durations or a value object's null members. Declare " + `${wireBlock(committed)} in .metaobjects/config.json, and 'meta gen' and 'meta verify --codegen' both read it.` ); } @@ -2185,3 +2264,10 @@ function buildVerifyPayload(input: { ], }; } + +/** An FK whose target is not a unique key, refused by name on every `--db` path. */ +function refuseFkTarget(err: Error): number { + log.error(`verify: ${err.message}`); + log.error("verify: declare an identity.secondary on the target entity with exactly the referenced fields, or reference its primary key"); + return 1; +} diff --git a/server/typescript/packages/cli/src/lib/agent-schema.ts b/server/typescript/packages/cli/src/lib/agent-schema.ts index 0da0e3bff2..125ca53f57 100644 --- a/server/typescript/packages/cli/src/lib/agent-schema.ts +++ b/server/typescript/packages/cli/src/lib/agent-schema.ts @@ -14,6 +14,7 @@ import type { AgentSchemaInput, Dialect, SchemaColumnLike } from "@metaobjectsde import { buildExpectedSchemaWithProvenance, columnTypeSql, qualifiedDbName } from "@metaobjectsdev/migrate-ts"; import type { MetaRoot } from "@metaobjectsdev/metadata"; import { describeError } from "./error-text.js"; +import type { SchemaBuildOptions } from "./schema-build-options.js"; /** * The schema for `agent/schema.md`, or undefined, after saying why through `warn`, when the @@ -24,6 +25,8 @@ export function buildAgentSchemaInput( root: MetaRoot, configured: Dialect | undefined, warn: (message: string) => void, + /** The project's schema settings, so the page describes the schema `meta migrate` builds. */ + schema: SchemaBuildOptions, ): AgentSchemaInput | undefined { // THE RUNNER'S OWN GUARD DECIDES THIS, not a default applied here. `DEFAULT_DIALECT` is // inert: `meta gen` refuses a model that emits database code with no dialect declared, so a @@ -45,6 +48,7 @@ export function buildAgentSchemaInput( const dialect = configured ?? DEFAULT_DIALECT; try { const built = buildExpectedSchemaWithProvenance(root, { + ...schema, dialect, // Views come from codegen-ts (migrate-ts never generates view DDL), exactly as // `verify --db` threads them. diff --git a/server/typescript/packages/cli/src/lib/allow.ts b/server/typescript/packages/cli/src/lib/allow.ts index a6db944138..7726f26eab 100644 --- a/server/typescript/packages/cli/src/lib/allow.ts +++ b/server/typescript/packages/cli/src/lib/allow.ts @@ -67,6 +67,8 @@ export function describeChange(c: Change): string { case "create-table": return c.table.name; case "drop-table": return c.table; case "rename-table": return `${c.from} → ${c.to}`; + case "rename-constraints": + return `${c.table} ${[...(c.constraintRenames ?? []), ...(c.indexRenames ?? [])].map((r) => `${r.from} → ${r.to}`).join(", ")}`; case "add-column": return `${c.table}.${c.column.name}`; case "drop-column": return c.readBy === undefined || c.readBy.length === 0 @@ -84,6 +86,11 @@ export function describeChange(c: Change): string { case "change-column-default": return `${c.table}.${c.column}`; case "change-column-identity": return `${c.table}.${c.column} (identity ${c.from ?? "none"} → ${c.to ?? "none"})`; case "change-comment": return `${c.column !== undefined ? `${c.table}.${c.column}` : c.table} (comment)`; + case "alter-sequence": return `${c.table}.${c.column} (sequence ${c.sequence}: ` + + [ + ...(c.toStart !== undefined ? [`start ${c.fromStart} → ${c.toStart}`] : []), + ...(c.toName !== undefined ? [`rename → ${c.toName}`] : []), + ].join(", ") + ")"; case "add-index": return `${c.table} idx ${c.index.name}`; case "drop-index": return `${c.table} idx ${c.index}`; case "add-fk": return `${c.table} fk ${c.fk.name}`; diff --git a/server/typescript/packages/cli/src/lib/catalog-listing.ts b/server/typescript/packages/cli/src/lib/catalog-listing.ts index 24d1689680..705328bf51 100644 --- a/server/typescript/packages/cli/src/lib/catalog-listing.ts +++ b/server/typescript/packages/cli/src/lib/catalog-listing.ts @@ -210,6 +210,8 @@ export interface CatalogListingOpts { sourceRoots?: readonly string[]; /** The physical schema `agent/schema.md` describes, as `meta gen` supplies it. */ agentSchema?: RunGenOpts["agentSchema"]; + /** The project's schema conventions and CHECK settings, as `meta gen` supplies them (Prisma). */ + schemaSettings?: RunGenOpts["schemaSettings"]; }; } @@ -226,7 +228,7 @@ async function probeOne( project: CatalogProject, probe: NonNullable, ): Promise<{ count: number; needsConfig?: string[] } | { error: string }> { - const { metadata, scope, sourceFiles, sourceRoots, agentSchema } = probe; + const { metadata, scope, sourceFiles, sourceRoots, agentSchema, schemaSettings } = probe; const count = async (config: MetaobjectsGenConfig): Promise => { const result = await runGen({ config: { ...config, generators: [entry.factory()] }, @@ -237,6 +239,7 @@ async function probeOne( ...(sourceFiles !== undefined ? { sourceFiles } : {}), ...(sourceRoots !== undefined ? { sourceRoots } : {}), ...(agentSchema !== undefined ? { agentSchema } : {}), + ...(schemaSettings !== undefined ? { schemaSettings } : {}), }); return result.files.length; }; diff --git a/server/typescript/packages/cli/src/lib/codegen-drift.ts b/server/typescript/packages/cli/src/lib/codegen-drift.ts index 259d53825e..e6ea2faee7 100644 --- a/server/typescript/packages/cli/src/lib/codegen-drift.ts +++ b/server/typescript/packages/cli/src/lib/codegen-drift.ts @@ -187,7 +187,7 @@ export async function computeCodegenDrift( projectRoot: string, scope?: (fqn: string) => boolean, sourceFiles?: readonly string[], - run?: Pick, + run?: Pick, ): Promise { const root = isAbsolute(projectRoot) ? projectRoot : resolve(projectRoot); @@ -282,6 +282,7 @@ export async function computeCodegenDrift( ...(sourceFiles !== undefined ? { sourceFiles } : {}), ...(run?.sourceRoots !== undefined ? { sourceRoots: run.sourceRoots } : {}), ...(run?.agentSchema !== undefined ? { agentSchema: run.agentSchema } : {}), + ...(run?.schemaSettings !== undefined ? { schemaSettings: run.schemaSettings } : {}), ...(run?.requirementLedger !== undefined ? { requirementLedger: run.requirementLedger } : {}), }); diff --git a/server/typescript/packages/cli/src/lib/collection-load-options.ts b/server/typescript/packages/cli/src/lib/collection-load-options.ts index c86d74433e..69f6173ff3 100644 --- a/server/typescript/packages/cli/src/lib/collection-load-options.ts +++ b/server/typescript/packages/cli/src/lib/collection-load-options.ts @@ -12,6 +12,7 @@ // `ERR_DEPENDENCY_PACKAGE_NOT_OWNED` refusal, so a node declared into a // dependency's package is silently excluded from that command's output instead // of being refused by name. +import { DEFAULT_ENUM_DEFAULTS } from "@metaobjectsdev/metadata"; import type { Collection, LoadMemoryOptions } from "@metaobjectsdev/sdk"; /** @@ -26,7 +27,7 @@ import type { Collection, LoadMemoryOptions } from "@metaobjectsdev/sdk"; */ export function collectionLoadOptions( collection: Collection, -): Required> { +): Required> { return { files: collection.files, fileIds: collection.fileIds, @@ -40,5 +41,8 @@ export function collectionLoadOptions( libraries: collection.libraries, physicalNaming: collection.physicalNaming, wireFormat: collection.wireFormat, + enumDefaults: collection.enumDefaults ?? DEFAULT_ENUM_DEFAULTS, + // The collection's migrate settings (`migrate.conventions`, `migrate.checks`) are not load + // options: they reach the expected-schema builds through `schemaBuildOptions`. }; } diff --git a/server/typescript/packages/cli/src/lib/enum-length-refusal.ts b/server/typescript/packages/cli/src/lib/enum-length-refusal.ts new file mode 100644 index 0000000000..9f20d669ef --- /dev/null +++ b/server/typescript/packages/cli/src/lib/enum-length-refusal.ts @@ -0,0 +1,16 @@ +/** + * A string-stored enum member longer than the project's `enums.maxLength`: core's + * resolveEnumMaxLength refuses it while the expected schema is built. A metadata (or config) + * defect, so `meta migrate` and `meta verify` report it by name rather than as an + * unexpected error. Matched by name and code, not `instanceof`: two physical copies of + * @metaobjectsdev/metadata in one process give the class two identities. + */ +export const ENUM_LENGTH_CODE = "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH"; + +export const ENUM_LENGTH_HINT = + "give the enum field its own @maxLength that fits its longest member, raise enums.maxLength in .metaobjects/config.json, or shorten the member"; + +export function isEnumLengthRefusal(err: unknown): err is Error { + return err instanceof Error && err.name === "MetaModelError" + && (err as { code?: unknown }).code === ENUM_LENGTH_CODE; +} diff --git a/server/typescript/packages/cli/src/lib/fk-index-advisory.ts b/server/typescript/packages/cli/src/lib/fk-index-advisory.ts index 4a3d94ff86..4a375ca589 100644 --- a/server/typescript/packages/cli/src/lib/fk-index-advisory.ts +++ b/server/typescript/packages/cli/src/lib/fk-index-advisory.ts @@ -30,6 +30,7 @@ import { type Dialect, type TableDescriptor, } from "@metaobjectsdev/migrate-ts"; +import type { SchemaBuildOptions } from "./schema-build-options.js"; export interface UnindexedFkFinding { /** The metadata file declaring the entity ("" when the loader recorded none). */ @@ -43,6 +44,8 @@ export interface UnindexedFkOptions { dialect: Dialect; /** Skip an entity by FQN — imported (dependency) metadata is not the adopter's to index. */ skip?: (fqn: string) => boolean; + /** The project's schema settings: the conventions decide which FK indexes the schema already has. */ + schema: SchemaBuildOptions; } const INDEX_LOOKUP = `${TYPE_INDEX}${TYPE_SUBTYPE_SEPARATOR}${INDEX_SUBTYPE_LOOKUP}`; @@ -63,6 +66,7 @@ function pascal(s: string): string { export function scanForUnindexedForeignKeys(root: MetaData, opts: UnindexedFkOptions): UnindexedFkFinding[] { const { snapshot, provenance } = buildExpectedSchemaWithProvenance(root, { + ...opts.schema, dialect: opts.dialect, }); // ADR-0058: the same column rule the expected schema just used, from the loaded root. diff --git a/server/typescript/packages/cli/src/lib/fk-target-warning.ts b/server/typescript/packages/cli/src/lib/fk-target-warning.ts new file mode 100644 index 0000000000..058397b236 --- /dev/null +++ b/server/typescript/packages/cli/src/lib/fk-target-warning.ts @@ -0,0 +1,13 @@ +import { FkTargetNotUniqueError, type FkTargetViolation } from "@metaobjectsdev/migrate-ts"; +import { log } from "./log.js"; + +/** + * FKs whose target is not a unique key, on a SQLite database that does not enforce the target's + * uniqueness when the table is created: reported, not refused. `prefix` is the command name that + * leads each line (`migrate`, `verify`). + */ +export function warnUnenforcedFkTargets(prefix: string, violations: readonly FkTargetViolation[] | undefined): void { + if (violations === undefined || violations.length === 0) return; + log.warn(`${prefix}: ${new FkTargetNotUniqueError([...violations]).message}`); + log.warn(`${prefix}: SQLite creates the table anyway and checks foreign keys only with PRAGMA foreign_keys on, so nothing is refused; a write with it on fails with "foreign key mismatch".`); +} diff --git a/server/typescript/packages/cli/src/lib/gen-migrate-advice.ts b/server/typescript/packages/cli/src/lib/gen-migrate-advice.ts index 14827acd58..068e9cd1db 100644 --- a/server/typescript/packages/cli/src/lib/gen-migrate-advice.ts +++ b/server/typescript/packages/cli/src/lib/gen-migrate-advice.ts @@ -30,6 +30,7 @@ import { type Change, type D1Binding, } from "@metaobjectsdev/migrate-ts"; +import type { SchemaBuildOptions } from "./schema-build-options.js"; /** The FQN separator; an object's short name is the segment after the last one. */ const FQN_SEPARATOR = "::"; @@ -52,6 +53,8 @@ export interface GenMigrateAdviceInput { migrateFormat: "default" | "flyway"; /** Paths (any form) of the files this run wrote, merged or removed. */ changedFiles: readonly string[]; + /** The project's schema settings, so the schema compared is the one `meta migrate` builds. */ + schema: SchemaBuildOptions; } /** The next-step line, or undefined when this run cannot have changed the schema. */ @@ -65,6 +68,7 @@ export async function genMigrateAdvice(input: GenMigrateAdviceInput): Promise; try { provenance = buildExpectedSchemaWithProvenance(input.metadata, { + ...input.schema, dialect: "postgres", }).provenance; } catch { diff --git a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts index 606dc55fc7..1053a219d8 100644 --- a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts +++ b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts @@ -52,6 +52,13 @@ const CLI_PKG_PATHS: Record = { dist: "node_modules/@metaobjectsdev/metadata/dist/index.js", src: "node_modules/@metaobjectsdev/metadata/src/index.ts", }, + // A config that supplies its own schema conventions starts from a preset + // (`efcoreConventions`) — aliased like the other codegen specifiers so the project + // needn't declare migrate-ts directly, and a global CLI still resolves it. + "@metaobjectsdev/migrate-ts": { + dist: "node_modules/@metaobjectsdev/migrate-ts/dist/index.js", + src: "node_modules/@metaobjectsdev/migrate-ts/src/index.ts", + }, "@metaobjectsdev/codegen-ts/generators": { dist: "node_modules/@metaobjectsdev/codegen-ts/dist/generators/index.js", src: "node_modules/@metaobjectsdev/codegen-ts/src/generators/index.ts", @@ -203,6 +210,18 @@ function rewriteImportSpecifiers(source: string, aliasMap: Record | undefined, ): { providers?: readonly MetaDataTypeProvider[] } { + // `schemaConventions` is not a load option: it reaches the expected-schema builds through + // `schemaBuildOptions` (schema-build-options.ts), never through the loaded model. return { ...(cfg?.providers !== undefined ? { providers: cfg.providers } : {}), }; diff --git a/server/typescript/packages/cli/src/lib/referential-action-advisory.ts b/server/typescript/packages/cli/src/lib/referential-action-advisory.ts index b93c9245cf..71f54366ee 100644 --- a/server/typescript/packages/cli/src/lib/referential-action-advisory.ts +++ b/server/typescript/packages/cli/src/lib/referential-action-advisory.ts @@ -20,6 +20,7 @@ import { describeReferentialActionConflict, type ReferentialActionConflict, } from "@metaobjectsdev/migrate-ts"; +import type { SchemaBuildOptions } from "./schema-build-options.js"; export interface ReferentialActionConflictFinding { /** The metadata file declaring the FK-owning entity ("" when the loader recorded none). */ @@ -32,14 +33,16 @@ export interface ReferentialActionConflictFinding { export interface ReferentialActionConflictOptions { /** Skip an entity by FQN — imported (dependency) metadata is not the adopter's to fix. */ skip?: (fqn: string) => boolean; + /** The project's schema settings: the build that reports the conflicts is the one migrate runs. */ + schema: SchemaBuildOptions; } export function scanForReferentialActionConflicts( root: MetaData, - opts: ReferentialActionConflictOptions = {}, + opts: ReferentialActionConflictOptions, ): ReferentialActionConflictFinding[] { const conflicts: ReferentialActionConflict[] = []; - buildExpectedSchema(root, { onReferentialActionConflict: (c) => conflicts.push(c) }); + buildExpectedSchema(root, { ...opts.schema, onReferentialActionConflict: (c) => conflicts.push(c) }); const out: ReferentialActionConflictFinding[] = []; const seen = new Set(); diff --git a/server/typescript/packages/cli/src/lib/schema-advisories.ts b/server/typescript/packages/cli/src/lib/schema-advisories.ts new file mode 100644 index 0000000000..e30c7f985a --- /dev/null +++ b/server/typescript/packages/cli/src/lib/schema-advisories.ts @@ -0,0 +1,19 @@ +// Advice from the schema diff that is not drift (migrate-ts `SchemaAdvisory`): `meta migrate` +// and `meta verify` print the same sentence for each. +import type { SchemaAdvisory } from "@metaobjectsdev/migrate-ts"; + +/** One sentence per advisory, each naming the column and what to do about it. */ +export function schemaAdvisoryLines(advisories: readonly SchemaAdvisory[] | undefined): string[] { + return (advisories ?? []).map((a) => { + const t = a.schema !== undefined ? `${a.schema}.${a.table}` : a.table; + switch (a.kind) { + case "interval-as-string": + return `${t}.${a.column} is a Postgres interval the metadata does not declare as a field.duration; ` + + `it is compared as text, so it is not drift. Model it as field.duration for the native type, ` + + `span ordering and range filters.`; + case "sequence-not-applied": + return `${t}.${a.column}: SQLite and D1 have no sequence, so the primary key's @sequenceStart / ` + + `@sequenceName is not applied and the key counts from 1.`; + } + }); +} diff --git a/server/typescript/packages/cli/src/lib/schema-build-options.ts b/server/typescript/packages/cli/src/lib/schema-build-options.ts new file mode 100644 index 0000000000..8067c6e57b --- /dev/null +++ b/server/typescript/packages/cli/src/lib/schema-build-options.ts @@ -0,0 +1,56 @@ +// server/typescript/packages/cli/src/lib/schema-build-options.ts +// +// The project settings every expected-schema build takes, resolved in ONE place. +// +// The schema conventions (`migrate.conventions` in `.metaobjects/config.json`, or a +// conventions object from `metaobjects.config.ts`) and `migrate.checks.skipRedundant` are +// migrate settings, not facts about the model, so the loaded model does not carry them: +// each command resolves them here and passes the result to every build and drift +// computation. The helpers that build an expected schema take it as a REQUIRED +// parameter, so a call site that forgets it is a compile error rather than a silent +// default — the way #333 happened to `libraries` (see `loadMemoryOptionsFrom`). +import type { MetaobjectsGenConfig } from "@metaobjectsdev/codegen-ts"; +import { isUnknownSchemaConventionsError, resolveSchemaConventions, type SchemaConventions } from "@metaobjectsdev/migrate-ts"; +import type { Collection } from "@metaobjectsdev/sdk"; + +/** What an expected-schema build needs from the project, every member resolved. */ +export interface SchemaBuildOptions { + conventions: SchemaConventions; + skipRedundantChecks: boolean; +} + +/** + * The project's schema settings. A conventions object from `metaobjects.config.ts` wins over + * the `migrate.conventions` preset name. Throws `UnknownSchemaConventionsError` for a preset + * this version does not ship and `InvalidSchemaConventionsError` for an object missing a + * member (`isUnknownSchemaConventionsError` recognises both). + */ +export function schemaBuildOptions( + collection: Pick, + tsConfig: Pick | undefined, +): SchemaBuildOptions { + return { + conventions: resolveSchemaConventions(tsConfig?.schemaConventions ?? collection.schemaConventions), + skipRedundantChecks: collection.skipRedundantChecks, + }; +} + +/** + * {@link schemaBuildOptions} for a command that only ADVISES from the schema (`meta gen`'s + * schema page and next-step line): conventions that do not resolve are warned about and the + * result is undefined, so the caller skips what needed them. `meta migrate` and `meta verify` + * refuse the same config with exit 2. + */ +export function schemaBuildOptionsOrWarn( + collection: Pick, + tsConfig: Pick | undefined, + warn: (message: string) => void, +): SchemaBuildOptions | undefined { + try { + return schemaBuildOptions(collection, tsConfig); + } catch (err) { + if (!isUnknownSchemaConventionsError(err)) throw err; + warn(`${err.message}; the schema-derived output (agent/schema.md, the migrate advice) is skipped`); + return undefined; + } +} diff --git a/server/typescript/packages/cli/test/integration/identity-sequence-pg.test.ts b/server/typescript/packages/cli/test/integration/identity-sequence-pg.test.ts new file mode 100644 index 0000000000..d587ae7326 --- /dev/null +++ b/server/typescript/packages/cli/test/integration/identity-sequence-pg.test.ts @@ -0,0 +1,131 @@ +/** + * identity.primary @sequenceStart / @sequenceName through the CLI against a real Postgres, on + * the shadow path `meta verify --db` and `meta migrate --from-db` take by default (env-gated on + * MIGRATE_TS_PG_URL; a throwaway database per test). + * + * - A model that declares NO sequence sees no change from one: a live sequence left behind by + * a table rename, with a non-default start, verifies clean and plans nothing, and a fresh + * CREATE stays plain `GENERATED BY DEFAULT AS IDENTITY`. + * - A declared start and name that differ from the live sequence are altered in place, and the + * counter really moves: the next insert takes the declared start. + */ +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import pg from "pg"; +import { run } from "../../src/index.js"; + +const PG_URL = process.env.MIGRATE_TS_PG_URL; + +function model(pk: Record): string { + return JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "Ticket" } }, + { "field.int": { name: "id", "@column": "Id" } }, + { "field.string": { name: "label", "@column": "Label" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment", ...pk } }, + ] } }, + ] } }); +} + +async function withDatabase(fn: (url: string, client: pg.Client) => Promise): Promise { + const admin = new pg.Client({ connectionString: PG_URL }); + await admin.connect(); + const db = `mt_seq_${Date.now()}_${Math.floor(Math.random() * 1e6)}`; + await admin.query(`CREATE DATABASE ${db}`); + const url = new URL(PG_URL!); + url.pathname = `/${db}`; + const client = new pg.Client({ connectionString: url.toString() }); + await client.connect(); + try { + await fn(url.toString(), client); + } finally { + await client.end(); + await admin.query(`DROP DATABASE IF EXISTS ${db} WITH (FORCE)`); + await admin.end(); + } +} + +function project(pk: Record): string { + const root = mkdtempSync(join(tmpdir(), "identity-sequence-pg-")); + mkdirSync(join(root, "metaobjects"), { recursive: true }); + mkdirSync(join(root, ".metaobjects", "migrations"), { recursive: true }); + writeFileSync(join(root, "metaobjects", "meta.acme.json"), model(pk), "utf8"); + writeFileSync(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, migrate: { dialect: "postgres" } }), "utf8"); + return root; +} + +async function captured(argv: string[]): Promise<{ exit: number; out: string }> { + const lines: string[] = []; + const keep = (...a: unknown[]): void => { lines.push(a.map(String).join(" ")); }; + const orig = [console.log, console.error, console.warn] as const; + console.log = keep; console.error = keep; console.warn = keep; + try { + return { exit: await run(argv), out: lines.join("\n") }; + } finally { + [console.log, console.error, console.warn] = orig; + } +} + +describe("identity.primary sequence on a real Postgres (env-gated)", () => { + if (PG_URL === undefined || PG_URL.length === 0) { + test.skip("skipped — MIGRATE_TS_PG_URL not set", () => {}); + return; + } + + test("a model that declares no sequence: a renamed table's live sequence verifies clean and plans nothing", async () => { + await withDatabase(async (url, client) => { + await client.query(`CREATE TABLE "OldTicket" ("Id" integer GENERATED BY DEFAULT AS IDENTITY (START WITH 1000) PRIMARY KEY, "Label" text)`); + await client.query(`ALTER TABLE "OldTicket" RENAME TO "Ticket"`); + // The primary key's name is compared too, so it follows the table; the sequence keeps its old name. + await client.query(`ALTER TABLE "Ticket" RENAME CONSTRAINT "OldTicket_pkey" TO "Ticket_pkey"`); + const root = project({}); + try { + const verify = await captured(["verify", "--cwd", root, "--db", url]); + expect(verify.out).not.toContain("alter-sequence"); + expect(verify.exit).toBe(0); + const plan = await captured(["migrate", "--cwd", root, "--db", url, "--from-db", "--dry-run"]); + expect(plan.out).not.toContain("SEQUENCE"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + }); + + test("a model that declares no sequence: a fresh CREATE is plain IDENTITY", async () => { + await withDatabase(async (url) => { + const root = project({}); + try { + const plan = await captured(["migrate", "--cwd", root, "--db", url, "--from-db", "--dry-run"]); + expect(plan.out).toContain("GENERATED BY DEFAULT AS IDENTITY"); + expect(plan.out).not.toContain("SEQUENCE NAME"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + }); + + test("a declared start and name that differ are altered in place, and the next insert takes the start", async () => { + await withDatabase(async (url, client) => { + await client.query(`CREATE TABLE "Ticket" ("Id" integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, "Label" text)`); + await client.query(`INSERT INTO "Ticket" ("Label") VALUES ('seeded')`); + const root = project({ "@sequenceStart": 1000000000, "@sequenceName": "Legacy_Id_seq" }); + try { + const drift = await captured(["verify", "--cwd", root, "--db", url]); + expect(drift.exit).toBe(1); + expect(drift.out).toContain("alter-sequence"); + const applied = await captured(["migrate", "--cwd", root, "--db", url, "--from-db", "--slug", "seq", "--apply"]); + expect(applied.exit).toBe(0); + const { rows } = await client.query(`INSERT INTO "Ticket" ("Label") VALUES ('next') RETURNING "Id"`); + expect(Number(rows[0].Id)).toBe(1000000000); + const seq = await client.query(`SELECT pg_get_serial_sequence('"Ticket"', 'Id') AS s`); + expect(seq.rows[0].s).toBe('public."Legacy_Id_seq"'); + const clean = await captured(["verify", "--cwd", root, "--db", url]); + expect(clean.exit).toBe(0); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + }); +}); diff --git a/server/typescript/packages/cli/test/integration/verify-codegen-wire.test.ts b/server/typescript/packages/cli/test/integration/verify-codegen-wire.test.ts index 40ca34b00c..166e7f88b6 100644 --- a/server/typescript/packages/cli/test/integration/verify-codegen-wire.test.ts +++ b/server/typescript/packages/cli/test/integration/verify-codegen-wire.test.ts @@ -84,8 +84,8 @@ describe("meta verify --codegen — the wire setting", () => { out = []; err = []; expect(await run(["verify", "--cwd", root, "--codegen"])).toBe(1); expect(all()).toContain( - 'The committed output was generated with "wire": { "decimals": "string", "nullMembers": "omit" }; ' + - 'this run used "wire": { "decimals": "native", "nullMembers": "native" }', + 'The committed output was generated with "wire": { "decimals": "string", "nullMembers": "omit", "durations": "native" }; ' + + 'this run used "wire": { "decimals": "native", "nullMembers": "native", "durations": "native" }', ); expect(all()).toContain("Do not regenerate"); expect(all()).not.toContain("Run 'meta gen' to regenerate"); diff --git a/server/typescript/packages/cli/test/migrate-checks-skip-redundant.test.ts b/server/typescript/packages/cli/test/migrate-checks-skip-redundant.test.ts new file mode 100644 index 0000000000..12c0789ab3 --- /dev/null +++ b/server/typescript/packages/cli/test/migrate-checks-skip-redundant.test.ts @@ -0,0 +1,121 @@ +/** + * `migrate.checks.skipRedundant` in .metaobjects/config.json reaches `meta migrate` and + * `meta verify --db`: a `validator.length` minimum of 0 is left out of the CHECK (on SQLite + * the maximum stays, since SQLite does not enforce VARCHAR(n)). It is not a schema + * convention: `migrate.conventions: "efcore"` alone emits the CHECK whole. A malformed + * `checks` block is refused as a config error. + */ +import { describe, test, expect, afterAll, beforeEach, afterEach, spyOn } from "bun:test"; +import { mkdtemp, rm, mkdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { migrateCommand } from "../src/commands/migrate.js"; +import { verifyCommand } from "../src/commands/verify.js"; +import { log } from "../src/lib/log.js"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +const MODEL = JSON.stringify({ + "metadata.root": { + package: "acme::platform", + children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { name: "src", "@table": "tickets" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "code", "@maxLength": 3, children: [ + { "validator.length": { name: "len", "@min": 0, "@max": 3 } }, + ] } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + ], + }, +}); + +async function writeConfig(root: string, migrate: Record): Promise { + await writeFile(join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, migrate: { dialect: "sqlite", ...migrate } }), "utf8"); +} + +async function project(migrate: Record): Promise { + const root = await mkdtemp(join(tmpdir(), "checks-skip-redundant-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.platform.json"), MODEL, "utf8"); + await mkdir(join(root, ".metaobjects", "migrations"), { recursive: true }); + await writeConfig(root, migrate); + return root; +} + +let out: string[] = []; +let spies: ReturnType[] = []; +beforeEach(() => { + out = []; + const push = (m: unknown) => { out.push(String(m)); }; + spies = [ + spyOn(log, "info").mockImplementation(push), + spyOn(log, "error").mockImplementation(push), + spyOn(log, "warn").mockImplementation(push), + spyOn(console, "log").mockImplementation(push), + spyOn(console, "error").mockImplementation(push), + ]; +}); +afterEach(() => { for (const s of spies) s.mockRestore(); }); + +const db = (root: string) => `file:${join(root, "t.db")}`; +const dryRun = (root: string) => + migrateCommand(["--from-db", "--db", db(root), "--dialect", "sqlite", "--dry-run"], root); + +const WHOLE = 'CHECK (length("code") >= 0 AND length("code") <= 3)'; +const SKIPPED = 'CHECK (length("code") <= 3)'; + +describe("migrate.checks.skipRedundant", () => { + test("meta migrate leaves the redundant part of the CHECK out", async () => { + const root = await project({ checks: { skipRedundant: true } }); + + expect(await dryRun(root)).toBe(0); + const text = out.join("\n"); + expect(text).toContain(SKIPPED); + expect(text).not.toContain(WHOLE); + }); + + test("absent, and under the efcore conventions alone, the CHECK is emitted whole", async () => { + for (const migrate of [{}, { conventions: "efcore" }, { checks: { skipRedundant: false } }]) { + out = []; + const root = await project(migrate); + expect(await dryRun(root)).toBe(0); + expect(out.join("\n")).toContain(WHOLE); + } + }); + + test("meta verify --db compares against the same setting", async () => { + const root = await project({ checks: { skipRedundant: true } }); + expect(await migrateCommand(["--from-db", "--db", db(root), "--dialect", "sqlite", "--slug", "init", "--apply"], root)).toBe(0); + + out = []; + expect(await verifyCommand(["--db", db(root), "--dialect", "sqlite"], root)).toBe(0); + + // The database was built without the redundant part; a project that wants it reports drift. + await writeConfig(root, {}); + out = []; + expect(await verifyCommand(["--db", db(root), "--dialect", "sqlite"], root)).toBe(1); + expect(out.join("\n")).toContain("tickets_code_length_chk"); + }); + + test("a malformed checks block is refused, naming the key", async () => { + for (const [checks, says] of [ + [true, "migrate.checks"], + [{ skipRedundnat: true }, "skipRedundnat"], + [{ skipRedundant: "yes" }, "migrate.checks.skipRedundant"], + ] as const) { + out = []; + const root = await project({ checks }); + const migrateCode = await dryRun(root); + expect(migrateCode).not.toBe(0); + expect(out.join("\n")).toContain(says); + out = []; + expect(await verifyCommand([], root)).not.toBe(0); + expect(out.join("\n")).toContain(says); + } + }); +}); diff --git a/server/typescript/packages/cli/test/migrate-enum-max-length.test.ts b/server/typescript/packages/cli/test/migrate-enum-max-length.test.ts new file mode 100644 index 0000000000..590d7afea2 --- /dev/null +++ b/server/typescript/packages/cli/test/migrate-enum-max-length.test.ts @@ -0,0 +1,65 @@ +/** + * `enums.maxLength` in .metaobjects/config.json reaches `meta migrate` through the + * project-aware load: an enum with no @maxLength gets VARCHAR(n), and a member longer than + * the default is refused by name — a metadata defect, not an "unexpected error". + */ +import { describe, test, expect, afterAll, beforeEach, afterEach, spyOn } from "bun:test"; +import { mkdtemp, rm, mkdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { migrateCommand } from "../src/commands/migrate.js"; +import { log } from "../src/lib/log.js"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +function model(values: string[]): string { + return JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", "@values": values } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + ] } }); +} + +async function project(values: string[], maxLength: number): Promise { + const root = await mkdtemp(join(tmpdir(), "enum-max-length-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.acme.json"), model(values), "utf8"); + await mkdir(join(root, ".metaobjects", "migrations"), { recursive: true }); + await writeFile(join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, enums: { maxLength }, migrate: { dialect: "postgres" } }), "utf8"); + return root; +} + +let out: string[] = []; +let spies: ReturnType[] = []; +beforeEach(() => { + out = []; + spies = (["info", "error", "warn"] as const).map((k) => + spyOn(log, k).mockImplementation((m: string) => { out.push(String(m)); })); +}); +afterEach(() => { for (const s of spies) s.mockRestore(); }); + +const dryRun = (root: string) => + migrateCommand(["--from-db", "--db", `file:${join(root, "t.db")}`, "--dialect", "sqlite", "--dry-run"], root); + +describe("meta migrate with enums.maxLength", () => { + test("an enum with no @maxLength gets the project's length", async () => { + const root = await project(["OPEN", "CLOSED"], 24); + expect(await dryRun(root)).toBe(0); + expect(out.join("\n")).toMatch(/"status" (VARCHAR|varchar)\(24\)/); + }); + + test("a member longer than the default is refused by name, not as an unexpected error", async () => { + const root = await project(["OPEN", "IN_PROGRESS"], 10); + const code = await dryRun(root); + const text = out.join("\n"); + expect(code).toBe(1); + expect(text).toContain('field.enum "Ticket.status" has the member "IN_PROGRESS"'); + expect(text).not.toContain("unexpected error"); + }); +}); diff --git a/server/typescript/packages/cli/test/migrate-fk-target-not-unique.test.ts b/server/typescript/packages/cli/test/migrate-fk-target-not-unique.test.ts new file mode 100644 index 0000000000..15fe6b68ab --- /dev/null +++ b/server/typescript/packages/cli/test/migrate-fk-target-not-unique.test.ts @@ -0,0 +1,136 @@ +/** + * An FK whose target columns are not a unique key is a metadata defect, not a tool + * failure. On Postgres `meta migrate` refuses it with the engine's message and a hint, the + * way it refuses two declarations that generate one database name, never as an "unexpected + * error". On SQLite the DDL is accepted, so it is warned and the run goes on. + */ +import { describe, test, expect, afterAll, beforeEach, afterEach, spyOn } from "bun:test"; +import { mkdtemp, rm, mkdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import pg from "pg"; +import { migrateCommand } from "../src/commands/migrate.js"; +import { log } from "../src/lib/log.js"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +const MODEL = JSON.stringify({ + "metadata.root": { + package: "acme::platform", + children: [ + { + "object.entity": { + name: "Person", + children: [ + { "source.rdb": { name: "src", "@table": "people" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "code" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ], + }, + }, + { + "object.entity": { + name: "Visit", + children: [ + { "source.rdb": { name: "src", "@table": "visits" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "personCode" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + { "identity.reference": { name: "ref_person", "@fields": ["personCode"], "@references": "Person.code" } }, + ], + }, + }, + ], + }, +}); + +async function project(migrate: Record = { dialect: "sqlite" }): Promise { + const root = await mkdtemp(join(tmpdir(), "fk-target-unique-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.platform.json"), MODEL, "utf8"); + await mkdir(join(root, ".metaobjects", "migrations"), { recursive: true }); + await writeFile(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, migrate }), "utf8"); + return root; +} + +let out: string[] = []; +let infoSpy: ReturnType; +let errSpy: ReturnType; +let warnSpy: ReturnType; +beforeEach(() => { + out = []; + infoSpy = spyOn(log, "info").mockImplementation((m: string) => { out.push(String(m)); }); + errSpy = spyOn(log, "error").mockImplementation((m: string) => { out.push(String(m)); }); + warnSpy = spyOn(log, "warn").mockImplementation((m: string) => { out.push(String(m)); }); +}); +afterEach(() => { infoSpy.mockRestore(); errSpy.mockRestore(); warnSpy.mockRestore(); }); + +describe("meta migrate on SQLite: an FK to a non-unique target is warned, not refused", () => { + // SQLite accepts the CREATE TABLE and checks the FK only on a write with PRAGMA + // foreign_keys on, so a refusal here would stop an estate that works (CR-9). + test("the reference and the fix are named, and the migration is still planned", async () => { + const root = await project(); + const code = await migrateCommand(["--from-db", "--db", `file:${join(root, "t.db")}`, "--dialect", "sqlite", "--dry-run"], root); + + const text = out.join("\n"); + expect(code).toBe(0); + expect(text).toContain('reference "ref_person" targets [code] on "Person"'); + expect(text).toContain("identity.secondary"); + expect(text).toContain("PRAGMA foreign_keys"); + expect(text).not.toContain("unexpected error"); + expect(text).toContain('CREATE TABLE "visits"'); + }); +}); + +const PG_URL = process.env.MIGRATE_TS_PG_URL; + +async function withPgDatabase(fn: (url: string) => Promise): Promise { + const admin = new pg.Client({ connectionString: PG_URL }); + await admin.connect(); + const db = `mt_fkt_${Date.now()}_${Math.floor(Math.random() * 1e6)}`; + await admin.query(`CREATE DATABASE ${db}`); + const url = new URL(PG_URL!); + url.pathname = `/${db}`; + try { + await fn(url.toString()); + } finally { + await admin.query(`DROP DATABASE IF EXISTS ${db} WITH (FORCE)`); + await admin.end(); + } +} + +describe("meta migrate on Postgres refuses an FK to a non-unique target (env-gated)", () => { + if (PG_URL === undefined || PG_URL.length === 0) { + test.skip("skipped — MIGRATE_TS_PG_URL not set", () => {}); + return; + } + + test("exits 1 with the reference named and the fix, not as an unexpected error", async () => { + await withPgDatabase(async (url) => { + const root = await project({ dialect: "postgres" }); + const code = await migrateCommand(["--from-db", "--db", url, "--dialect", "postgres", "--dry-run"], root); + + const text = out.join("\n"); + expect(code).toBe(1); + expect(text).toContain('reference "ref_person" targets [code] on "Person"'); + expect(text).toContain("identity.secondary"); + expect(text).not.toContain("unexpected error"); + expect(text).not.toContain("CREATE TABLE"); + }); + }); + + test("a holder outside migrate.scope is not this run's to fix: warned, and the run goes on", async () => { + await withPgDatabase(async (url) => { + const root = await project({ dialect: "postgres", scope: ["acme::platform::Person"] }); + const code = await migrateCommand(["--from-db", "--db", url, "--dialect", "postgres", "--dry-run"], root); + + const text = out.join("\n"); + expect(code).toBe(0); + expect(text).toContain('reference "ref_person" targets [code] on "Person"'); + expect(text).toContain("not governed by this run"); + }); + }); +}); diff --git a/server/typescript/packages/cli/test/migrate-schema-conventions.test.ts b/server/typescript/packages/cli/test/migrate-schema-conventions.test.ts new file mode 100644 index 0000000000..fb36b12ddb --- /dev/null +++ b/server/typescript/packages/cli/test/migrate-schema-conventions.test.ts @@ -0,0 +1,152 @@ +/** + * `migrate.conventions` (or a conventions object from metaobjects.config.ts) reaches + * `meta migrate` and `meta verify --db`: an `efcore` project gets EF Core's database-only + * names (here, the index EF puts on every foreign key), and a preset name the tool does not + * know is refused by name, as a config error. + */ +import { describe, test, expect, afterAll, beforeEach, afterEach, spyOn } from "bun:test"; +import { mkdtemp, rm, mkdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { migrateCommand } from "../src/commands/migrate.js"; +import { verifyCommand } from "../src/commands/verify.js"; +import { genCommand } from "../src/commands/gen.js"; +import { log } from "../src/lib/log.js"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +const MODEL = JSON.stringify({ + "metadata.root": { + package: "acme::platform", + children: [ + { "object.entity": { name: "Order", children: [ + { "source.rdb": { name: "src", "@table": "Orders" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "OrderLine", children: [ + { "source.rdb": { name: "src", "@table": "OrderLines" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.long": { name: "orderId", "@column": "OrderId" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + { "identity.reference": { name: "order", "@fields": ["orderId"], "@references": "Order" } }, + ] } }, + ], + }, +}); + +async function project(conventions: string | undefined): Promise { + const root = await mkdtemp(join(tmpdir(), "schema-conventions-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.platform.json"), MODEL, "utf8"); + await mkdir(join(root, ".metaobjects", "migrations"), { recursive: true }); + const migrate = conventions === undefined ? { dialect: "sqlite" } : { dialect: "sqlite", conventions }; + await writeFile(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, migrate }), "utf8"); + return root; +} + +let out: string[] = []; +let infoSpy: ReturnType; +let errSpy: ReturnType; +let warnSpy: ReturnType; +beforeEach(() => { + out = []; + infoSpy = spyOn(log, "info").mockImplementation((m: string) => { out.push(String(m)); }); + errSpy = spyOn(log, "error").mockImplementation((m: string) => { out.push(String(m)); }); + warnSpy = spyOn(log, "warn").mockImplementation((m: string) => { out.push(String(m)); }); +}); +afterEach(() => { infoSpy.mockRestore(); errSpy.mockRestore(); warnSpy.mockRestore(); }); + +const dryRun = (root: string) => + migrateCommand(["--from-db", "--db", `file:${join(root, "t.db")}`, "--dialect", "sqlite", "--dry-run"], root); + +describe("meta migrate with migrate.conventions", () => { + test("efcore: the foreign key gets EF Core's IX_ index", async () => { + const root = await project("efcore"); + + expect(await dryRun(root)).toBe(0); + expect(out.join("\n")).toContain('CREATE INDEX "IX_OrderLines_OrderId" ON "OrderLines" ("OrderId");'); + }); + + test("no conventions set: no FK index, as before", async () => { + const root = await project(undefined); + + expect(await dryRun(root)).toBe(0); + expect(out.join("\n")).not.toContain("IX_OrderLines_OrderId"); + }); + + test("an unknown preset is refused by name, not as an unexpected error", async () => { + const root = await project("hibernate"); + + const code = await dryRun(root); + const text = out.join("\n"); + expect(code).toBe(2); + expect(text).toContain('migrate.conventions "hibernate" is not a schema-conventions preset: expected one of default, efcore'); + expect(text).not.toContain("unexpected error"); + }); + + test("meta gen refuses an unknown preset too, rather than generating with the default names", async () => { + const root = await project("hibernate"); + await writeFile(join(root, "metaobjects.config.ts"), "export default { generators: [] };\n", "utf8"); + const code = await genCommand([], root); + expect(code).toBe(2); + expect(out.join("\n")).toContain('migrate.conventions "hibernate" is not a schema-conventions preset'); + }); + + test("a conventions object from metaobjects.config.ts applies, importing the preset from migrate-ts", async () => { + const root = await project(undefined); + await writeFile(join(root, "metaobjects.config.ts"), [ + 'import { efcoreConventions } from "@metaobjectsdev/migrate-ts";', + "export default { generators: [], schemaConventions: { ...efcoreConventions, id: \"acme\" } };", + "", + ].join("\n"), "utf8"); + + expect(await dryRun(root)).toBe(0); + expect(out.join("\n")).toContain('CREATE INDEX "IX_OrderLines_OrderId"'); + }); + + test("a metaobjects.config.ts that exists but does not load is warned about, not silently ignored", async () => { + const root = await project(undefined); + await writeFile(join(root, "metaobjects.config.ts"), 'throw new Error("config exploded");\n', "utf8"); + + expect(await dryRun(root)).toBe(0); + const text = out.join("\n"); + expect(text).toContain("metaobjects.config.ts exists but did not load"); + expect(text).toContain("config exploded"); + expect(text).toContain("schemaConventions"); + }); + + test("meta verify --db builds with the same conventions, from config.json or metaobjects.config.ts", async () => { + const consoleSpies = [ + spyOn(console, "log").mockImplementation((m: unknown) => { out.push(String(m)); }), + spyOn(console, "error").mockImplementation((m: unknown) => { out.push(String(m)); }), + ]; + try { + const db = (root: string) => `file:${join(root, "t.db")}`; + const verify = (root: string) => verifyCommand(["--db", db(root), "--dialect", "sqlite"], root); + + const preset = await project("efcore"); + expect(await migrateCommand(["--from-db", "--db", db(preset), "--dialect", "sqlite", "--slug", "init", "--apply"], preset)).toBe(0); + expect(await verify(preset)).toBe(0); + + const own = await project(undefined); + await writeFile(join(own, "metaobjects.config.ts"), [ + 'import { efcoreConventions } from "@metaobjectsdev/migrate-ts";', + "export default { generators: [], schemaConventions: { ...efcoreConventions, id: \"acme\" } };", + "", + ].join("\n"), "utf8"); + expect(await migrateCommand(["--from-db", "--db", db(own), "--dialect", "sqlite", "--slug", "init", "--apply"], own)).toBe(0); + expect(await verify(own)).toBe(0); + + // Without the conventions, the EF Core FK index is an object the metadata does not describe. + await rm(join(own, "metaobjects.config.ts")); + out = []; + expect(await verify(own)).toBe(1); + expect(out.join("\n")).toContain("IX_OrderLines_OrderId"); + } finally { + for (const s of consoleSpies) s.mockRestore(); + } + }); +}); diff --git a/server/typescript/packages/cli/test/unit/gen-migrate-advice.test.ts b/server/typescript/packages/cli/test/unit/gen-migrate-advice.test.ts index 6bd127dee0..375b4be8af 100644 --- a/server/typescript/packages/cli/test/unit/gen-migrate-advice.test.ts +++ b/server/typescript/packages/cli/test/unit/gen-migrate-advice.test.ts @@ -5,7 +5,7 @@ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { MetaDataLoader, InMemoryStringSource, type MetaRoot } from "@metaobjectsdev/metadata"; -import { buildExpectedSchema, snapshotPath, writeSnapshot } from "@metaobjectsdev/migrate-ts"; +import { buildExpectedSchema, defaultConventions, snapshotPath, writeSnapshot } from "@metaobjectsdev/migrate-ts"; import { genMigrateAdvice, type GenMigrateAdviceInput } from "../../src/lib/gen-migrate-advice.js"; const dirs: string[] = []; @@ -49,6 +49,7 @@ function input(over: Partial & Pick { expect(await verifyCommand(["--replay-snapshot"], root)).toBe(1); // tier 2: must not pass vacuously }); }); + +describe("verify --replay-snapshot under migrate.scope", () => { + // The scoped branch rebuilds the expected schema to decide what it governs, and that + // build refuses an FK into a non-unique target on Postgres. The refusal must come out + // by name, exit 1, as on every other --db path, not as an unexpected error. + test("an FK to a non-unique target on Postgres is refused by name", async () => { + const root = await project(); + const pgCfg = () => + ({ dialect: "postgres", outDir: MIGRATIONS, onAmbiguous: "abort", + allow: [], slug: "init", dryRun: false } as never); + await writeFile(join(root, "metaobjects", "meta.platform.json"), NO_TABLES, "utf8"); + await writeFile( + join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, migrate: { dialect: "postgres", outDir: MIGRATIONS } }), + "utf8", + ); + expect(await runBaseline(pgCfg(), root)).toBe(0); + await writeFile(join(root, "metaobjects", "meta.platform.json"), JSON.stringify({ "metadata.root": { + package: "acme::platform", + children: [ + { "object.entity": { name: "Person", children: [ + { "source.rdb": { name: "src", "@table": "people" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "code" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Visit", children: [ + { "source.rdb": { name: "src", "@table": "visits" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "personCode" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + { "identity.reference": { name: "ref_person", "@fields": ["personCode"], "@references": "Person.code" } }, + ] } }, + ], + } }), "utf8"); + await writeFile( + join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, migrate: { dialect: "postgres", outDir: MIGRATIONS, scope: ["acme::platform::*"] } }), + "utf8", + ); + + const lines: string[] = []; + const orig = [console.log, console.error] as const; + console.log = (...a: unknown[]) => { lines.push(a.map(String).join(" ")); }; + console.error = (...a: unknown[]) => { lines.push(a.map(String).join(" ")); }; + let code: number; + try { + code = await verifyCommand(["--replay-snapshot", "--skip-schema"], root); + } finally { + [console.log, console.error] = orig; + } + expect(code).toBe(1); + expect(lines.join("\n")).toContain('reference "ref_person" targets [code] on "Person"'); + }, 60_000); +}); diff --git a/server/typescript/packages/cli/test/verify-schema-refusals.test.ts b/server/typescript/packages/cli/test/verify-schema-refusals.test.ts new file mode 100644 index 0000000000..aa75a48ea1 --- /dev/null +++ b/server/typescript/packages/cli/test/verify-schema-refusals.test.ts @@ -0,0 +1,157 @@ +/** + * `meta verify` reports the two metadata/config refusals the expected-schema build can raise + * as what they are, the way `meta migrate` does: an FK whose target is not a unique key + * (exit 1 on Postgres, the reference named; a warning on SQLite, whose DDL accepts it), and an unknown `migrate.conventions` preset (exit 2, the + * presets named). Neither is "failed to introspect", and a typo'd preset is refused even + * on a run with no --db. + */ +import { describe, test, expect, afterAll } from "bun:test"; +import { mkdtemp, mkdir, writeFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import pg from "pg"; +import { verifyCommand } from "../src/commands/verify.js"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +function model(references: string): string { + return JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Person", children: [ + { "source.rdb": { "@table": "people" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "code" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Visit", children: [ + { "source.rdb": { "@table": "visits" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "personCode" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + { "identity.reference": { name: "ref_person", "@fields": ["personCode"], "@references": references } }, + ] } }, + ] } }); +} + +async function project(references: string, migrate: Record = {}): Promise { + const root = await mkdtemp(join(tmpdir(), "verify-schema-refusals-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.acme.json"), model(references), "utf8"); + await mkdir(join(root, ".metaobjects"), { recursive: true }); + await writeFile(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, migrate }), "utf8"); + return root; +} + +async function capture(fn: () => Promise): Promise<{ code: number; out: string }> { + const lines: string[] = []; + const log = console.log; + const err = console.error; + console.log = (...a: unknown[]) => { lines.push(a.map(String).join(" ")); }; + console.error = (...a: unknown[]) => { lines.push(a.map(String).join(" ")); }; + try { + const code = await fn(); + return { code, out: lines.join("\n") }; + } finally { + console.log = log; + console.error = err; + } +} + +describe("meta verify — schema refusals", () => { + test("--db on SQLite: an FK to a non-unique target is warned by name, not refused and not an introspection failure", async () => { + const root = await project("Person.code"); + const { out } = await capture(() => verifyCommand(["--db", `file:${join(root, "t.db")}`], root)); + + expect(out).toContain('reference "ref_person" targets [code] on "Person"'); + expect(out).toContain("PRAGMA foreign_keys"); + expect(out).not.toContain("failed to introspect"); + }); + + test("--db on Postgres: an FK to a non-unique target is refused by name (env-gated)", async () => { + const pgUrl = process.env.MIGRATE_TS_PG_URL; + if (pgUrl === undefined || pgUrl.length === 0) return; + const admin = new pg.Client({ connectionString: pgUrl }); + await admin.connect(); + const db = `mt_vfk_${Date.now()}_${Math.floor(Math.random() * 1e6)}`; + await admin.query(`CREATE DATABASE ${db}`); + try { + const url = new URL(pgUrl); + url.pathname = `/${db}`; + const root = await project("Person.code", { dialect: "postgres" }); + const { code, out } = await capture(() => verifyCommand(["--db", url.toString()], root)); + + expect(code).toBe(1); + expect(out).toContain('reference "ref_person" targets [code] on "Person"'); + expect(out).not.toContain("failed to introspect"); + } finally { + await admin.query(`DROP DATABASE IF EXISTS ${db} WITH (FORCE)`); + await admin.end(); + } + }); + + test("an unknown migrate.conventions preset is refused with exit 2, with or without --db", async () => { + const root = await project("Person", { conventions: "hibernate" }); + + const plain = await capture(() => verifyCommand([], root)); + expect(plain.code).toBe(2); + expect(plain.out).toContain('migrate.conventions "hibernate" is not a schema-conventions preset'); + + const withDb = await capture(() => verifyCommand(["--db", `file:${join(root, "t.db")}`], root)); + expect(withDb.code).toBe(2); + expect(withDb.out).not.toContain("failed to introspect"); + }); + + test("--db: an enum member longer than enums.maxLength is refused by name, not as an introspection failure", async () => { + const root = await mkdtemp(join(tmpdir(), "verify-schema-refusals-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.acme.json"), JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", "@values": ["OPEN", "IN_PROGRESS"] } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + ] } }, + ] } }), "utf8"); + await mkdir(join(root, ".metaobjects"), { recursive: true }); + await writeFile(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, enums: { maxLength: 10 } }), "utf8"); + + const { code, out } = await capture(() => verifyCommand(["--db", `file:${join(root, "t.db")}`], root)); + expect(code).toBe(1); + expect(out).toContain('field.enum "Ticket.status" has the member "IN_PROGRESS"'); + expect(out).not.toContain("failed to introspect"); + + // The same refusal under migrate.scope, whose scope check builds the expected schema first. + await writeFile(join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, enums: { maxLength: 10 }, migrate: { scope: ["acme::*"] } }), "utf8"); + const scoped = await capture(() => verifyCommand(["--db", `file:${join(root, "t.db")}`], root)); + expect(scoped.code).toBe(1); + expect(scoped.out).toContain('field.enum "Ticket.status" has the member "IN_PROGRESS"'); + }); + + test("--db: a declared CHECK name two validators share is refused by name, not as an introspection failure", async () => { + const root = await mkdtemp(join(tmpdir(), "verify-schema-refusals-")); + dirs.push(root); + await mkdir(join(root, "metaobjects"), { recursive: true }); + await writeFile(join(root, "metaobjects", "meta.acme.json"), JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Slot", children: [ + { "source.rdb": { "@table": "slots" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "a" } }, + { "field.long": { name: "b" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@constraintName": "CK_Slot" } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", "@constraintName": "CK_Slot" } }, + ] } }, + ] } }), "utf8"); + await mkdir(join(root, ".metaobjects"), { recursive: true }); + await writeFile(join(root, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1 }), "utf8"); + + const { code, out } = await capture(() => verifyCommand(["--db", `file:${join(root, "t.db")}`], root)); + expect(code).toBe(1); + expect(out).toContain("ERR_CHECK_NAME"); + expect(out).toContain('CHECK "CK_Slot" on table "slots"'); + expect(out).not.toContain("failed to introspect"); + }); +}); diff --git a/server/typescript/packages/codegen-ts/src/column-mapper.ts b/server/typescript/packages/codegen-ts/src/column-mapper.ts index df69132fa4..f0a16e07f0 100644 --- a/server/typescript/packages/codegen-ts/src/column-mapper.ts +++ b/server/typescript/packages/codegen-ts/src/column-mapper.ts @@ -21,6 +21,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, FIELD_ATTR_MAX_LENGTH, @@ -45,6 +46,7 @@ import { AGG_ANY, AGG_ALL, AGG_COLLECT, + resolveEnumMaxLength, } from "@metaobjectsdev/metadata"; import { columnNameFromField } from "./naming.js"; import { enumValues, intValueMapOf, intValueForMember } from "./enum-meta.js"; @@ -189,6 +191,7 @@ function sqliteJsonArrayElementTsType(subType: string): string | undefined { case FIELD_SUBTYPE_TIME: case FIELD_SUBTYPE_TIMESTAMP: case FIELD_SUBTYPE_DECIMAL: + case FIELD_SUBTYPE_DURATION: return "string"; case FIELD_SUBTYPE_INT: case FIELD_SUBTYPE_LONG: @@ -242,6 +245,23 @@ export interface EnumIntCustomType { intByMember: Record; } +/** + * A `field.duration` column: a generated Drizzle `customType` (the module-local const + * {@link DURATION_COLUMN_FN}) whose `toDriver` binds what the column holds — the Postgres + * interval text `[-]HH:MM:SS[.f]` (Postgres refuses a signed ISO-8601 duration), a 64-bit integer + * count of microseconds on SQLite / D1 / MySQL (it orders and compares as the span) — and whose + * `fromDriver` reads either back as the canonical ISO-8601 form. Drizzle's own pg-core `interval()` reads Postgres's text spelling + * (`01:30:00`), which is not the wire form. One declaration per module serves every duration + * column in it (templates/duration-wire-helper.ts). + */ +export const DURATION_COLUMN_FN = "durationColumn"; + +export interface DurationCustomType { + /** The local const name, {@link DURATION_COLUMN_FN}. */ + fnConstName: string; + dialect: Dialect; +} + export interface ColumnSpec { /** Drizzle function name, e.g., "text", "integer", "varchar". */ fnName: string; @@ -250,6 +270,8 @@ export interface ColumnSpec { * helper) rather than a Drizzle export — the renderer must NOT `imp()` it. */ enumIntCustomType?: EnumIntCustomType; + /** Set for a scalar `field.duration`: `fnName` is the LOCAL duration codec const — never `imp()` it. */ + durationCustomType?: DurationCustomType; /** DB column name (snake_case from field name, or @column override). */ dbName: string; /** Positional args after dbName (currently always empty; reserved). */ @@ -458,6 +480,7 @@ export const PG_IDENTITY_CAPABLE_FNS: ReadonlySet = new Set([ * with a `@default` (a `TEXT` column cannot take a literal default). Those are * `VARCHAR(255)`. * - `field.uuid` is `VARCHAR(36)` and `field.inet` `VARCHAR(45)`: MySQL has neither type. + * `field.duration` is a `BIGINT` count of microseconds, through the local duration codec. * - A string enum is `VARCHAR(n)`, `n` the longest member, typed as the member union. * - `field.timestamp` is `DATETIME(3)` in Drizzle's "date" mode (see normalizeTimestampMode). * - Arrays, `field.object`, `field.map` and `@dbColumnType: jsonb` are `JSON`. @@ -505,6 +528,10 @@ function mapMysqlColumn( return varchar(36); case FIELD_SUBTYPE_INET: return varchar(45); + case FIELD_SUBTYPE_DURATION: + // A BIGINT count of microseconds (MySQL has no interval type; an integer orders as the span). + // The column function is the local duration codec. + return { fnName: DURATION_COLUMN_FN }; case FIELD_SUBTYPE_OBJECT: case FIELD_SUBTYPE_MAP: return { fnName: "json" }; @@ -514,6 +541,10 @@ function mapMysqlColumn( const t = buildEnumIntCustomType(field, im); return t !== undefined ? { fnName: t.fnConstName, enumIntCustomType: t } : { fnName: "int" }; } + // The column `meta migrate` creates: the enum's length (core's resolveEnumMaxLength — + // its @maxLength, else the project's enums.maxLength), else the longest member. + const sized = resolveEnumMaxLength(field); + if (sized !== undefined) return varchar(sized); const values = enumValues(field) ?? []; const longest = values.reduce((n, v) => Math.max(n, v.length), 1); return varchar(longest); @@ -647,6 +678,10 @@ export function mapColumnType( // SQLite has no native uuid/inet type; store as TEXT (string native binding). fnName = "text"; break; + case FIELD_SUBTYPE_DURATION: + // A 64-bit INTEGER count of microseconds, through the local duration codec. + fnName = DURATION_COLUMN_FN; + break; default: fnName = "text"; break; @@ -734,6 +769,12 @@ export function mapColumnType( // the opt-out. fnName = field.attr(FIELD_ATTR_LENIENT) === true ? "text" : "inet"; break; + case FIELD_SUBTYPE_DURATION: + // The Postgres-native interval, through the local duration codec: it binds the + // interval spelling and reads the canonical ISO-8601 form. A duration array is + // interval[] (`.array()` below); Drizzle maps each element through the codec. + fnName = DURATION_COLUMN_FN; + break; case FIELD_SUBTYPE_DECIMAL: { // Drizzle pg `numeric` infers as a TS `string` (precision-exact); the // native TS binding for field.decimal is `string` to match. Read the @@ -781,7 +822,16 @@ export function mapColumnType( enumIntCustomType = buildEnumIntCustomType(field, im); fnName = enumIntCustomType?.fnConstName ?? "integer"; } else { - fnName = "text"; + // A length (core's resolveEnumMaxLength) is the VARCHAR(n) `meta migrate` + // creates; with none the column is unbounded text, as before. An array enum is + // a text[] column, whose elements carry no length (migrate's arrayElementSqlType). + const sized = isArray ? undefined : resolveEnumMaxLength(field); + if (sized !== undefined) { + fnName = "varchar"; + fnOptions = { length: sized }; + } else { + fnName = "text"; + } } } break; @@ -798,7 +848,9 @@ export function mapColumnType( // arrays use { mode: "json" }, and the enum members go through Zod // validation at the Insert/Update layer instead. Mirrors the Zod // emission, which already uses z.enum([...]). - if (subType === FIELD_SUBTYPE_ENUM && !isArray && (fnName === "text" || (dialect === "mysql" && fnName === "varchar"))) { + // A sized enum is a varchar on Postgres too (resolveEnumMaxLength); pg-core's varchar + // takes the same `enum` option. + if (subType === FIELD_SUBTYPE_ENUM && !isArray && (fnName === "text" || fnName === "varchar")) { const values = enumValues(field); if (values !== undefined && values.length > 0) { fnOptions = { ...(fnOptions ?? {}), enum: values }; @@ -936,6 +988,9 @@ export function mapColumnType( if (fnOptions !== undefined) result.fnOptions = fnOptions; if (defaultExpr !== undefined) result.defaultExpr = defaultExpr; if (enumIntCustomType !== undefined) result.enumIntCustomType = enumIntCustomType; + if (fnName === DURATION_COLUMN_FN && subType === FIELD_SUBTYPE_DURATION) { + result.durationCustomType = { fnConstName: DURATION_COLUMN_FN, dialect }; + } if (dollarTypeRef !== undefined) result.dollarTypeRef = dollarTypeRef; if (leadingComment !== undefined) result.leadingComment = leadingComment; diff --git a/server/typescript/packages/codegen-ts/src/contract/field-schema.ts b/server/typescript/packages/codegen-ts/src/contract/field-schema.ts index 82cae81a9f..539cf5c62e 100644 --- a/server/typescript/packages/codegen-ts/src/contract/field-schema.ts +++ b/server/typescript/packages/codegen-ts/src/contract/field-schema.ts @@ -13,6 +13,7 @@ import { type MetaField, type MetaObject, WIRE_DECIMALS_STRING, + WIRE_DURATIONS_ISO, type WireFormat, } from "@metaobjectsdev/metadata"; import { @@ -33,6 +34,7 @@ import { FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DOUBLE, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_INET, @@ -71,6 +73,22 @@ export type ContractShape = "read" | "create" | "update"; /** How a schema points at another object's schema: a relative file, or a component. */ export type RefFor = (target: MetaObject, shape: ContractShape) => string; +/** + * The accepted field.duration form (DURATION_PATTERN) as a JSON Schema `pattern` with no lookaround: + * DURATION_PATTERN's `(?!$)` and `(?=\d)` are not I-Regexp (RFC 9485), and an RE2-based validator or + * code generator refuses them. Spelled out instead: weeks and/or days (weeks first) with an optional + * time part, or a time part alone, where `T` is followed by at least one of hours, minutes and + * seconds, in that order, and the seconds may have a fraction of any length. Digits are `[0-9]` and + * groups capture, which I-Regexp also requires; the anchors make JSON Schema's search a full match. + * test/contract/duration-json-schema-pattern.test.ts holds the two to the same verdicts. + */ +export const DURATION_JSON_SCHEMA_PATTERN = (() => { + const seconds = "[0-9]+(\\.[0-9]+)?S"; + const time = `T([0-9]+H([0-9]+M)?(${seconds})?|[0-9]+M(${seconds})?|${seconds})`; + const date = "([0-9]+W([0-9]+D)?|[0-9]+D)"; + return `^[-+]?P(${date}(${time})?|${time})$`; +})(); + /** The canonical decimal string (ADR-0019 Amendment 1): plain notation, no trailing fraction zeros. */ export const DECIMAL_CANONICAL_PATTERN = "^-?(0|[1-9][0-9]*)(\\.[0-9]*[1-9])?$"; /** A decimal as a request may send it: a string in JSON-number grammar. */ @@ -143,6 +161,17 @@ export function scalarSchema(subType: string): JsonSchema { case FIELD_SUBTYPE_URI: return { type: "string", format: "uri" }; case FIELD_SUBTYPE_INET: return { type: "string", anyOf: [{ format: "ipv4" }, { format: "ipv6" }] }; case FIELD_SUBTYPE_DECIMAL: return { type: "string", pattern: DECIMAL_INPUT_PATTERN }; + // An ISO-8601 duration: the accepted form (weeks, days, hours, minutes, seconds; no years or months). + // A read answers the canonical form, which the same pattern matches. No `format: "duration"`: + // that format is RFC 3339's, which has no sign and no fractional seconds, so a validator that + // asserts formats (Ajv with ajv-formats) would refuse `-PT1M30S` and `PT1.5S`, both of which + // every server writes. + case FIELD_SUBTYPE_DURATION: + return { + type: "string", + pattern: DURATION_JSON_SCHEMA_PATTERN, + description: "An ISO-8601 duration: weeks, days, hours, minutes and seconds (no years or months).", + }; default: return { type: "string" }; } } @@ -159,6 +188,12 @@ function plainValueSchema(field: MetaField, shape: ContractShape, wire: WireForm switch (field.subType) { case FIELD_SUBTYPE_DECIMAL: return decimalSchema(field, shape, wire); + // `wire.durations: native` leaves the form to the serving port's own serializer (a C# + // server writes `01:30:00`), so only `iso` can state one. + case FIELD_SUBTYPE_DURATION: + return wire.durations === WIRE_DURATIONS_ISO + ? scalarSchema(field.subType) + : { type: "string", description: `A span of time, in the serving language's own form. Set wire.durations to "iso" to fix ISO-8601 across server languages.` }; case FIELD_SUBTYPE_TIMESTAMP: if (shape !== "read") return timestampInputSchema(field); return field.attr(FIELD_ATTR_LOCAL_TIME) === true diff --git a/server/typescript/packages/codegen-ts/src/cube/cube-members.ts b/server/typescript/packages/codegen-ts/src/cube/cube-members.ts index 439c976848..5971c61a25 100644 --- a/server/typescript/packages/codegen-ts/src/cube/cube-members.ts +++ b/server/typescript/packages/codegen-ts/src/cube/cube-members.ts @@ -19,6 +19,7 @@ import { FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_MAP, @@ -72,7 +73,8 @@ const STRING_SUBTYPES: ReadonlySet = new Set([ // The subtypes `dimensionOfField` maps, in the order the refusal below lists them — derived from // the dispatch sets and checks, so the message cannot go stale when a mapping is added or removed. const MAPPED_DIMENSION_SUBTYPES: readonly string[] = [ - ...STRING_SUBTYPES, ...NUMBER_SUBTYPES, FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIMESTAMP, + ...STRING_SUBTYPES, ...NUMBER_SUBTYPES, FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DATE, + FIELD_SUBTYPE_TIMESTAMP, ]; /** `.`: a member's address, as `@of` and `@via` spell it. */ @@ -130,6 +132,14 @@ export function dimensionColumn( } if (STRING_SUBTYPES.has(field.subType)) return { sql: col, type: "string" }; if (NUMBER_SUBTYPES.has(field.subType)) return { sql: col, type: "number" }; + if (field.subType === FIELD_SUBTYPE_DURATION) { + // Cube has no interval dimension type; a number of SECONDS sorts and filters by the span. + // Postgres: EXTRACT(EPOCH FROM interval) is an exact numeric. MySQL stores a BIGINT count of + // microseconds; its `/` keeps only the dividend's scale plus div_precision_increment (4 by + // default), so the dividend is widened to six fractional digits first to keep every microsecond. + const sql = d === "postgres" ? `EXTRACT(EPOCH FROM ${col})` : `CAST(${col} AS DECIMAL(65, 6)) / 1000000`; + return { sql, type: "number" }; + } if (field.subType === FIELD_SUBTYPE_BOOLEAN) return { sql: col, type: "boolean" }; if (field.subType === FIELD_SUBTYPE_TIMESTAMP) return { sql: col, type: "time" }; if (field.subType === FIELD_SUBTYPE_DATE) { diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index 1cf0c7754d..a6059e0104 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -1,3 +1,4 @@ +import type { SchemaBuildSettings } from "@metaobjectsdev/migrate-ts"; import type { ExposeOption } from "./routes-expose.js"; import type { MetaDataTypeProvider, MetaObject, MetaRoot, TypeRegistry } from "@metaobjectsdev/metadata"; import type { LibraryManifest } from "@metaobjectsdev/metadata/library"; @@ -127,6 +128,13 @@ export interface GenContext { * call, where a docs generator's own options decide. See docs-run.ts. */ docs?: DocsRunFacts; + /** + * The project's schema settings for a generator that builds the expected physical schema + * (the Prisma schema): `migrate.conventions` and `migrate.checks` from .metaobjects/config.json, + * or a conventions object from metaobjects.config.ts, resolved by the caller. Absent: the + * default conventions, every CHECK. + */ + schemaSettings?: SchemaBuildSettings; /** * The files this run generated, for a generator marked `readsRunOutput` (the docs site's * model explorer, which lists what was generated from each node). The runner runs such a diff --git a/server/typescript/packages/codegen-ts/src/generators/prisma-schema-file.ts b/server/typescript/packages/codegen-ts/src/generators/prisma-schema-file.ts index 8ce80aca18..96a7184aa6 100644 --- a/server/typescript/packages/codegen-ts/src/generators/prisma-schema-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/prisma-schema-file.ts @@ -97,6 +97,7 @@ function generatePrismaFiles(ctx: GenContext, options: PrismaSchemaFileOptions): // The generator's own filter, never the run's selection: see the header. matches: ctx.matches, checks: options.checks ?? "file", + ...(ctx.schemaSettings !== undefined && { schemaSettings: ctx.schemaSettings }), }); const fileName = options.fileName ?? DEFAULT_PRISMA_SCHEMA_FILE; const checksFile = spec.checks.length > 0 ? prismaChecksFileName(fileName) : undefined; diff --git a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts index 6fdd32a6ba..2e9394404a 100644 --- a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts +++ b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts @@ -1,3 +1,4 @@ +import type { SchemaConventions } from "@metaobjectsdev/migrate-ts"; import { physicalNamingRemovedMessage, type MetaDataTypeProvider } from "@metaobjectsdev/metadata"; import { physicalNamingGuideUrl } from "@metaobjectsdev/metadata/release-docs"; import type { RoutesExpose } from "./routes-expose.js"; @@ -181,6 +182,13 @@ export interface MetaobjectsGenConfig extends Omit> }; + +/** A field entry that names one column: a plain field, or a flattened member. */ +export interface ColumnFieldNames { + readonly name: string; + readonly column: string; +} + +/** + * The flattened test is migrate's own (`expected-schema.ts`): a `field.object` whose + * RESOLVED `@storage` is `flattened` (ADR-0039 — it may be inherited). A flattened field + * whose `@objectRef` does not resolve has no members; the loader refuses that model before + * codegen runs. + */ +function fieldNamesOf(f: MetaField, strategy: ColumnNamingStrategy | undefined): FieldNames { + if (f.subType !== FIELD_SUBTYPE_OBJECT || f.attr(FIELD_ATTR_STORAGE) !== STORAGE_FLATTENED) { + return { name: f.name, column: resolveColumnName(f, strategy) }; + } + const target = refTarget(f, FIELD_ATTR_OBJECT_REF); + const members: Record = {}; + if (target !== undefined && isMetaObject(target)) { + for (const m of target.fields()) { + members[m.name] = { name: m.name, column: resolveFlattenedColumnName(f, m, strategy) }; + } + } + return { name: f.name, members }; +} /** * One `source.rdb` child, under the ROLE it plays. @@ -325,12 +369,12 @@ export function resolveObjectNames( // ADR-0039: fields() is the RESOLVING accessor — inherited fields must appear, and an // inherited @column must resolve, or the constant disagrees with the DDL. for (const f of obj.fields()) { - fields[f.name] = { name: f.name, column: resolveColumnName(f, strategy) }; + fields[f.name] = fieldNamesOf(f, strategy); } // ADR-0039's sanctioned own-accessor use: what this artifact DECLARES. See ObjectNames. const ownFields: Record = {}; for (const f of ownFieldList) { - ownFields[f.name] = { name: f.name, column: resolveColumnName(f, strategy) }; + ownFields[f.name] = fieldNamesOf(f, strategy); } const superObj = namesArtifactSuperOf(obj); @@ -415,11 +459,11 @@ export function resolveSuperFragmentNames( const fields: Record = {}; for (const f of obj.fields()) { - fields[f.name] = { name: f.name, column: resolveColumnName(f, strategy) }; + fields[f.name] = fieldNamesOf(f, strategy); } const ownFields: Record = {}; for (const f of ownFieldList) { - ownFields[f.name] = { name: f.name, column: resolveColumnName(f, strategy) }; + ownFields[f.name] = fieldNamesOf(f, strategy); } const superObj = namesArtifactSuperOf(obj); return { @@ -624,7 +668,8 @@ export function columnExpr( literal: string, ): Code { const entry = names?.resolved.fields[fieldName]; - return names !== undefined && entry !== undefined + // A flattened field has no `column` (its members do), so it takes the literal arm. + return names !== undefined && entry?.column !== undefined ? code`${names.symbol}.fields.${fieldName}.column` : code`${JSON.stringify(literal)}`; } diff --git a/server/typescript/packages/codegen-ts/src/prisma/build-prisma-schema.ts b/server/typescript/packages/codegen-ts/src/prisma/build-prisma-schema.ts index e0a4104c7d..9314c9af94 100644 --- a/server/typescript/packages/codegen-ts/src/prisma/build-prisma-schema.ts +++ b/server/typescript/packages/codegen-ts/src/prisma/build-prisma-schema.ts @@ -19,6 +19,7 @@ import { FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DOUBLE, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_INET, @@ -37,6 +38,7 @@ import { pluralize, refTarget, resolveColumnName, + resolveFlattenedColumnName, resolveTableSchema, type MetaField, type MetaObject, @@ -44,6 +46,7 @@ import { } from "@metaobjectsdev/metadata"; import { buildExpectedSchemaWithProvenance, + type SchemaBuildSettings, qualifiedDbName, type CheckDescriptor, type ColumnDescriptor, @@ -113,6 +116,8 @@ export interface BuildPrismaSchemaOptions { * them out, which is the only way to take a model with checks to SQLite. */ readonly checks?: "file" | "omit"; + /** The project's schema conventions and CHECK settings (`GenContext.schemaSettings`). */ + readonly schemaSettings?: SchemaBuildSettings; } /** Postgres' default schema: a table in it carries no `@@schema`. */ @@ -213,8 +218,9 @@ function refuseSchema(obj: MetaObject, schema: string, provider: PrismaProvider) /** * The fields whose column lands in the table: the declaring object's, then each TPH subtype's. A - * `field.object @storage: flattened` holds one column per field of its value object, - * `_`, which is Prisma field ``. + * `field.object @storage: flattened` holds one column per field of its value object, named by + * core naming's `resolveFlattenedColumnName` (the field's `@flattenedColumnPrefix`, else its column + * and `_`, then the member's column), which is Prisma field ``. */ function fieldsByColumn(obj: MetaObject, all: Iterable, strategy: ReturnType["columns"]): Map { const members = [obj, ...[...all].filter((o) => o !== obj && tphStorageObject(o) === obj)]; @@ -229,8 +235,7 @@ function fieldsByColumn(obj: MetaObject, all: Iterable, strategy: Re if (field.subType === FIELD_SUBTYPE_OBJECT && field.attr(FIELD_ATTR_STORAGE) === STORAGE_FLATTENED) { const value = refTarget(field, FIELD_ATTR_OBJECT_REF); if (value === undefined || !isMetaObject(value)) continue; - const prefix = `${resolveColumnName(field, strategy)}_`; - for (const nested of value.fields()) add(prefix + resolveColumnName(nested, strategy), nested, field.name + upperFirst(nested.name)); + for (const nested of value.fields()) add(resolveFlattenedColumnName(field, nested, strategy), nested, field.name + upperFirst(nested.name)); continue; } add(resolveColumnName(field, strategy), field, field.name); @@ -274,8 +279,16 @@ function pgColumnType(t: SqlType, field: MetaField): ColumnType { return { scalar: "String", list: false, native: "@db.Uuid" }; case "inet": return { scalar: "String", list: false, native: "@db.Inet" }; + case "interval": + // field.duration. Prisma has no interval type, scalar or native, so the column is declared + // `Unsupported("interval")`: `prisma db push` / `migrate` create the interval column + // `meta verify --db` expects, and Prisma Client leaves the field out of its model — read and + // write it with `$queryRaw` (bind the Postgres spelling, `01:30:00`, not ISO-8601). + return { scalar: 'Unsupported("interval")', list: false }; case "array": { if (t.element.kind === "array") throw unreachableType("postgresql", field, "a nested array"); + // An array of an unsupported type is one unsupported type, never a list of one. + if (t.element.kind === "interval") return { scalar: 'Unsupported("interval[]")', list: false }; const element = pgColumnType(t.element, field); return { ...element, list: true }; } @@ -327,6 +340,9 @@ function mysqlColumnType(t: SqlType, field: MetaField, needsLength: boolean): Co return varchar(36); case FIELD_SUBTYPE_INET: return varchar(45); + // A BIGINT count of microseconds, the column the Drizzle generator's MySQL arm writes. + case FIELD_SUBTYPE_DURATION: + return { scalar: "BigInt", list: false }; case FIELD_SUBTYPE_TIMESTAMP: return { scalar: "DateTime", list: false, native: "@db.DateTime(3)" }; case FIELD_SUBTYPE_TIME: @@ -337,6 +353,9 @@ function mysqlColumnType(t: SqlType, field: MetaField, needsLength: boolean): Co return { scalar: "Float", list: false, native: "@db.Float" }; case FIELD_SUBTYPE_ENUM: { if (t.kind === "integer") return { scalar: "Int", list: false }; + // A length the expected schema carries (resolveEnumMaxLength: @maxLength, else the + // project's enums.maxLength) is the column; with none, the longest member sizes it. + if (t.kind === "text" && t.maxLength !== undefined) return varchar(t.maxLength); // ADR-0039: resolving — @values may be inherited. const raw = field.attr(FIELD_ATTR_VALUES); const values = Array.isArray(raw) ? raw.map(String) : []; @@ -681,7 +700,10 @@ export function buildPrismaSchema(root: MetaRoot, options: BuildPrismaSchemaOpti if (schema !== undefined) throw refuseSchema(obj, schema, provider); } } - const { snapshot, provenance } = buildExpectedSchemaWithProvenance(root, dialect === undefined ? {} : { dialect }); + const { snapshot, provenance } = buildExpectedSchemaWithProvenance(root, { + ...options.schemaSettings, + ...(dialect === undefined ? {} : { dialect }), + }); const strategy = physicalNamingOf(root).columns; const entries: TableEntry[] = []; diff --git a/server/typescript/packages/codegen-ts/src/reference/prisma-schema.ts b/server/typescript/packages/codegen-ts/src/reference/prisma-schema.ts index d0458b1c39..873223eb7c 100644 --- a/server/typescript/packages/codegen-ts/src/reference/prisma-schema.ts +++ b/server/typescript/packages/codegen-ts/src/reference/prisma-schema.ts @@ -125,6 +125,7 @@ function generatePrismaFiles(ctx: GenContext, options: PrismaSchemaFileOptions): // The generator's own filter, never the run's selection: see the header. matches: ctx.matches, checks: options.checks ?? "file", + ...(ctx.schemaSettings !== undefined && { schemaSettings: ctx.schemaSettings }), }); const fileName = options.fileName ?? DEFAULT_SCHEMA_FILE; const checksFile = spec.checks.length > 0 ? prismaChecksFileName(fileName) : undefined; diff --git a/server/typescript/packages/codegen-ts/src/runner.ts b/server/typescript/packages/codegen-ts/src/runner.ts index 19dae55244..d7c658499f 100644 --- a/server/typescript/packages/codegen-ts/src/runner.ts +++ b/server/typescript/packages/codegen-ts/src/runner.ts @@ -1,3 +1,4 @@ +import type { SchemaBuildSettings } from "@metaobjectsdev/migrate-ts"; import { join, relative, resolve, isAbsolute, dirname, sep } from "node:path"; import { warnMissingPromptGenerators } from "./prompt-generator-gate.js"; import { runEmitsEntityModule, runEmitsHonoRoutes, runEmitsQueries, runEmitsRestRoutes, runEmitsUiTier, runRoutesExpose, warnUnmarkedRoutesGenerators, warnUnmarkedUiGenerators } from "./ui-tier-gate.js"; @@ -158,6 +159,8 @@ export interface RunGenOpts { * asks (`ctx.docs.agentSchema`). Absent: `agentDocsFile()` writes no schema page. */ agentSchema?: () => AgentSchemaInput | undefined; + /** Threaded onto `GenContext.schemaSettings`; see there. */ + schemaSettings?: SchemaBuildSettings; /** * Narrow this run to the declared generators with these stable names (`meta gen * --generators docs,api-docs`). Only the generators named run; every run-level fact (the @@ -967,6 +970,7 @@ export async function runGen(opts: RunGenOpts): Promise { ...(opts.sourceRoots !== undefined && { sourceRoots: opts.sourceRoots }), ...(config.providers !== undefined && { providers: config.providers }), docs: docsFactsFor(selfTarget.name), + ...(opts.schemaSettings !== undefined && { schemaSettings: opts.schemaSettings }), ...(selectedManifests !== undefined && { libraries: selectedManifests }), ...(generator.readsRunOutput === true && { runOutput: runOutputFiles }), ...(requirementLedger !== undefined && { requirementLedger }), diff --git a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts index 7568548d9c..0431389d79 100644 --- a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts +++ b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts @@ -18,7 +18,7 @@ import { import type { RenderContext } from "../render-context.js"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "../import-path.js"; import { - mapColumnType, PG_IDENTITY_CAPABLE_FNS, + mapColumnType, PG_IDENTITY_CAPABLE_FNS, DURATION_COLUMN_FN, type ColumnSpec, type EnumIntCustomType, } from "../column-mapper.js"; import { tableNameFromEntity } from "../naming.js"; @@ -28,6 +28,7 @@ import { } from "../names.js"; import { resolveTableSchema } from "@metaobjectsdev/metadata"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; import { collectTphSubtypeFields, tphConcreteSubtypes } from "./tph-discriminator.js"; @@ -46,7 +47,8 @@ export function renderDrizzleSchema( obj: MetaObject, ctx: RenderContext, /** - * OUT param: the int-backed-enum codec const names this table DECLARES. A caller that + * OUT param: the local column-codec const names this table DECLARES (int-backed enums, and + * the duration codec). A caller that * emits another artifact referencing the same codecs into the SAME module — entity-file's * write-through replica view — passes a set here and hands it to that emitter, which then * references the consts instead of re-declaring them. A duplicate module-scope `const` is @@ -211,6 +213,8 @@ export function renderDrizzleSchema( // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by // const name so a shared enum used by two fields of the SAME entity emits once. const enumIntTypes = new Map(); + // The field.duration codec: one declaration serves every duration column of the table. + let usesDurationCodec = false; for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -225,6 +229,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, columnNameExpr(child, spec.dbName), child, ctx, isPk, pkGeneration, fkInfo, isComposite, pkg, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -244,6 +249,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, columnNameExpr(child, spec.dbName), child, ctx, false, undefined, fkMap.get(child.name), isComposite, pkg, obj.name, true, @@ -403,9 +409,11 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} .map((t) => renderEnumIntCustomType(t, importModule)); if (declaredEnumIntCodecs !== undefined) { for (const name of enumIntTypes.keys()) declaredEnumIntCodecs.add(name); + if (usesDurationCodec) declaredEnumIntCodecs.add(DURATION_COLUMN_FN); } - const blocks: Code[] = [...enumIntBlocks, tableBlock]; + const durationBlocks = usesDurationCodec ? [renderDurationCustomType(dialect, importModule)] : []; + const blocks: Code[] = [...durationBlocks, ...enumIntBlocks, tableBlock]; if (relationsBlock !== null) blocks.push(relationsBlock); return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); } @@ -518,7 +526,9 @@ function renderColumn( const fnSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); let baseCall: Code; if (spec.fnOptions !== undefined && Object.keys(spec.fnOptions).length > 0) { diff --git a/server/typescript/packages/codegen-ts/src/templates/duration-wire-helper.ts b/server/typescript/packages/codegen-ts/src/templates/duration-wire-helper.ts new file mode 100644 index 0000000000..7ddd1d38c7 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/templates/duration-wire-helper.ts @@ -0,0 +1,213 @@ +// The wire rule for a field.duration, as text a generated model module carries as local code. +// Generated model code imports nothing from MetaObjects, so the rule is emitted, not imported; +// metadata/src/core/field/duration-wire.ts holds the same rule and both are held to +// fixtures/duration-wire/vectors.json (test/duration-wire-helper.test.ts evaluates the text +// emitted here against them). +// +// Accepted: optional weeks (exactly 7 days) and days, then hours, minutes and seconds; the seconds +// fraction may have any number of digits. Canonical: `[-]PT[H][M][[.]S]` — hours never +// folded into days, zero components left out, zero is `PT0S`, one leading `-`. A span is counted in +// microseconds, so the fraction keeps six digits and the rest is truncated toward zero. +// +// The emitted code counts a span in microseconds as a `bigint`, so it has no limit of its own: a +// span is as long as the stack it is stored in allows. The only limit here is a column's — the +// 64-bit integer a SQLite, D1 or MySQL column holds (refused past it, never wrapped). Postgres's +// interval has its own range, which Postgres enforces. The emitted code calls `BigInt(...)` and +// writes no bigint literal, so it needs the ES2020 `BigInt` global and nothing newer. + +import { code, imp, type Code } from "ts-poet"; +import { DURATION_PATTERN } from "@metaobjectsdev/metadata"; +import type { Dialect } from "../metaobjects-config.js"; +import { DURATION_COLUMN_FN } from "../column-mapper.js"; + +/** The accepted ISO-8601 duration form as a JS regex literal — the metadata's own pattern. */ +export const DURATION_REGEX_LITERAL = `/${DURATION_PATTERN}/`; + +/** The Zod expression that accepts a field.duration as sent (and as read back): the accepted form. */ +export const ZOD_DURATION_EXPR = `z.string().regex(${DURATION_REGEX_LITERAL})`; + +export const DURATION_WRITE_HELPER = "durationWrite"; + +/** The Zod expression of a duration WRITE field: the accepted form, rewritten to the canonical one, + * so equal spans are equal strings on the wire and in a jsonb value object. */ +export const ZOD_DURATION_WRITE_EXPR = `${ZOD_DURATION_EXPR}.transform(${DURATION_WRITE_HELPER})`; + +/** The accepted form with its components captured: sign, weeks, days, hours, minutes, seconds, + * fraction. Equivalent to DURATION_PATTERN (the same lookaheads refuse `P`, `PT` and `P1DT`). */ +const ISO_PARTS_SOURCE = String.raw`/^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/`; + +/** The default (`IntervalStyle = postgres`) text of a Postgres interval: signed year / mon / day + * parts, then an optionally signed `H:MM:SS[.ffffff]`. */ +const PG_INTERVAL_SOURCE = String.raw`/^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/`; + +/** Indent every line of `block` by `n` spaces (blank lines stay blank). */ +function indent(block: string, n: number): string { + const pad = " ".repeat(n); + return block.split("\n").map((l) => (l === "" ? l : pad + l)).join("\n"); +} + +/** Microsecond unit constants, as `BigInt(...)` calls (no bigint literal). */ +const UNIT_LINES = [ + "const ZERO = BigInt(0);", + "const SECOND = BigInt(1000000);", + "const MINUTE = BigInt(60000000);", + "const HOUR = BigInt(3600000000);", + "const DAY = BigInt(86400000000);", + "const WEEK = BigInt(604800000000);", +].join("\n"); + +/** `parseIso(text)`: the microseconds of an accepted ISO-8601 duration, or undefined. */ +const PARSE_ISO_FN = [ + "const parseIso = (text: string): bigint | undefined => {", + ` const p = ${ISO_PARTS_SOURCE}.exec(text);`, + " if (p === null) return undefined;", + " const mag = BigInt(p[2] ?? 0) * WEEK + BigInt(p[3] ?? 0) * DAY + BigInt(p[4] ?? 0) * HOUR", + ' + BigInt(p[5] ?? 0) * MINUTE + BigInt(p[6] ?? 0) * SECOND + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6));', + ' return p[1] === "-" ? -mag : mag;', + "};", +].join("\n"); + +/** `isoText(micros)`: the canonical wire form. */ +const ISO_TEXT_FN = [ + "const isoText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + " const s = rest / SECOND;", + " const f = rest % SECOND;", + ' let out = "PT";', + ' if (h > ZERO) out += String(h) + "H";', + ' if (m > ZERO) out += String(m) + "M";', + ' if (s > ZERO || f > ZERO) out += String(s) + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + "S";', + ' if (out === "PT") return "PT0S";', + ' return neg ? "-" + out : out;', + "};", +].join("\n"); + +export const DURATION_WRITE_HELPER_DECL = [ + "/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten", + " * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one", + " * leading \"-\" for a negative span. Counted in bigint microseconds, so any length is exact and a", + " * fraction past six digits is truncated toward zero. Generated. */", + `function ${DURATION_WRITE_HELPER}(text: string): string {`, + indent(UNIT_LINES, 2), + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + " const micros = parseIso(text);", + " return micros === undefined ? text : isoText(micros);", + "}", +].join("\n"); + +/** The physical column type a duration column's `customType` declares, per dialect: the + * Postgres-native interval; a 64-bit integer count of microseconds on SQLite / D1 (INTEGER) and + * MySQL (BIGINT) — the persistence corpus's canonical schemas, and what `meta migrate` writes. */ +export function durationColumnDataType(dialect: Dialect): string { + return dialect === "postgres" ? "interval" : dialect === "mysql" ? "bigint" : "integer"; +} + +/** + * The body of the duration column's declaration, around the `customType` reference (which the + * caller supplies as an import). `[before, after]`: `before + customType + after` is the whole + * declaration. Exported so a test can evaluate the exact text a generated module carries. + * + * - `toDriver` binds what the column holds: on Postgres the interval spelling `[-]HH:MM:SS[.f]` + * (Postgres refuses a signed ISO-8601 duration, and returns a value written this way in exactly + * this spelling); on SQLite, D1 and MySQL the microsecond count — a number when it is a safe + * integer, else a bigint — refused past the 64-bit range, which is that column's limit. It takes + * the value as an ISO-8601 duration (what the generated write schema hands it), or as what + * runtime-ts already binds a filter operand in (the Postgres text; the microsecond integer), and + * throws for anything else. + * - `fromDriver` reads what the driver returns — the microsecond integer (a number, a bigint or a + * digit string), ISO-8601 text, Postgres interval text in the default `IntervalStyle`, or a + * node-postgres interval object — as the canonical ISO-8601 form. An interval with a year or + * month part is not a fixed-length span: reading one throws. + */ +export function durationColumnDeclParts(dialect: Dialect): [string, string] { + const postgres = dialect === "postgres"; + const before = [ + "/** A field.duration column. Binds " + (postgres + ? "the Postgres interval spelling [-]HH:MM:SS[.f] (Postgres refuses a signed ISO-8601 duration);" + : "the span as a 64-bit integer count of microseconds (refused past that range);"), + " * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month", + " * part. Counted in bigint microseconds, so the column is the only limit. Generated. */", + `const ${DURATION_COLUMN_FN} = (() => {`, + indent(UNIT_LINES, 2), + ` const INT64_MAX = BigInt("9223372036854775807");`, + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + indent([ + "const pgText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + ' const two = (n: bigint): string => String(n).padStart(2, "0");', + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + ' let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND);', + " const f = rest % SECOND;", + ' if (f > ZERO) out += "." + String(f).padStart(6, "0").replace(/0+$/, "");', + ' return neg ? "-" + out : out;', + "};", + "const stored = (micros: bigint): number | bigint => {", + " if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) {", + ' throw new Error("the span " + isoText(micros) + " does not fit the 64-bit microsecond column a field.duration is stored in");', + " }", + " const n = Number(micros);", + " return Number.isSafeInteger(n) ? n : micros;", + "};", + "const notFixed = (v: unknown): Error =>", + ' new Error("the interval " + JSON.stringify(v) + " has a year or month part, so it is not a fixed-length field.duration");', + "const read = (v: unknown): bigint => {", + ' if (typeof v === "bigint") return v;', + ' if (typeof v === "number" && Number.isInteger(v)) return BigInt(v);', + ' if (typeof v === "string") {', + " if (/^-?\\d+$/.test(v)) return BigInt(v);", + " const iso = parseIso(v);", + " if (iso !== undefined) return iso;", + ` const p = v.length === 0 ? null : ${PG_INTERVAL_SOURCE}.exec(v);`, + ' if (p === null) throw new Error("not a field.duration value: " + JSON.stringify(v));', + " if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) throw notFixed(v);", + " let time = ZERO;", + " if (p[5] !== undefined) {", + ' time = BigInt(p[5]) * HOUR + BigInt(p[6] ?? 0) * MINUTE + BigInt(p[7] ?? 0) * SECOND + BigInt((p[8] ?? "").padEnd(6, "0"));', + ' if (p[4] === "-") time = -time;', + " }", + " return BigInt(p[3] ?? 0) * DAY + time;", + " }", + ' if (v !== null && typeof v === "object") {', + " const o = v as Record;", + " if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) throw notFixed(v);", + " const whole = (n: number | undefined): bigint => BigInt(Math.trunc(n ?? 0));", + " // postgres-interval carries microseconds as a fractional millisecond count.", + " return whole(o.days) * DAY + whole(o.hours) * HOUR + whole(o.minutes) * MINUTE + whole(o.seconds) * SECOND", + " + BigInt(Math.round((o.milliseconds ?? 0) * 1000));", + " }", + ' throw new Error("not a field.duration value: " + JSON.stringify(v));', + "};", + ].join("\n"), 2), + " return ", + ].join("\n"); + const after = [ + `<{ data: string; driverData: ${postgres ? "string" : "number | bigint | string"} }>({`, + ` dataType: () => ${JSON.stringify(durationColumnDataType(dialect))},`, + ` toDriver: (value) => ${postgres ? "pgText" : "stored"}(read(value)),`, + " fromDriver: (value) => isoText(read(value)),", + " });", + "})();", + ].join("\n"); + return [before, after]; +} + +/** A duration column's Drizzle `customType` declaration, emitted ahead of the table or view that + * uses it (once per module — see enum-int-codec.ts for why the order and the once matter). */ +export function renderDurationCustomType(dialect: Dialect, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const [before, after] = durationColumnDeclParts(dialect); + return code` +${before}${customTypeSym}${after} +`; +} diff --git a/server/typescript/packages/codegen-ts/src/templates/entity-ui-descriptor.ts b/server/typescript/packages/codegen-ts/src/templates/entity-ui-descriptor.ts index 1d30363650..4fb4ced2e4 100644 --- a/server/typescript/packages/codegen-ts/src/templates/entity-ui-descriptor.ts +++ b/server/typescript/packages/codegen-ts/src/templates/entity-ui-descriptor.ts @@ -39,6 +39,7 @@ import { VIEW_SUBTYPE_HIDDEN, VIEW_SUBTYPE_DROPDOWN, VIEW_SUBTYPE_RADIO, + FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, VALIDATOR_SUBTYPE_REQUIRED, @@ -268,9 +269,10 @@ function buildFieldRules(field: MetaField): UiRule[] { rules.push({ kind: "required", message: `${humanize(field.name)} is required` }); } - // Field-level @maxLength attr (if not already covered). + // Field-level @maxLength attr (if not already covered). On a field.enum it is the column + // length, not an input rule: the member list is the rule, and every member fits. const maxLenAttr = field.attr(FIELD_ATTR_MAX_LENGTH); - if (!hasMaxLength && typeof maxLenAttr === "number") { + if (!hasMaxLength && typeof maxLenAttr === "number" && field.subType !== FIELD_SUBTYPE_ENUM) { rules.push({ kind: "maxLength", value: maxLenAttr, diff --git a/server/typescript/packages/codegen-ts/src/templates/field-meta.ts b/server/typescript/packages/codegen-ts/src/templates/field-meta.ts index 921296073b..44e77153fd 100644 --- a/server/typescript/packages/codegen-ts/src/templates/field-meta.ts +++ b/server/typescript/packages/codegen-ts/src/templates/field-meta.ts @@ -22,6 +22,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STRING_FORMAT, @@ -43,6 +44,7 @@ import { enumValues } from "../enum-meta.js"; import { zodEnumExpr } from "./zod-enum.js"; import { viewForContext } from "../view-context.js"; import { ZOD_INET_EXPR } from "./net-regex.js"; +import { ZOD_DURATION_EXPR } from "./duration-wire-helper.js"; // --------------------------------------------------------------------------- // inferViewKind @@ -140,6 +142,9 @@ export function zodTypeFor(field: MetaField, timestampMode: "date" | "string" = // (v4 or v6 literal). Zod-version-agnostic regex union (Zod 4 removed // `z.string().ip()`); see net-regex.ts. @lenient → a plain string. return field.attr(FIELD_ATTR_LENIENT) === true ? "z.string()" : ZOD_INET_EXPR; + case FIELD_SUBTYPE_DURATION: + // An ISO-8601 duration string; the column codec reads every row in the canonical form. + return ZOD_DURATION_EXPR; case FIELD_SUBTYPE_BOOLEAN: return "z.boolean()"; case FIELD_SUBTYPE_DATE: diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts index 649bf11e73..c103ed601b 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts @@ -13,6 +13,7 @@ import { FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_ATTR_LOCAL_TIME, opsForField, @@ -70,6 +71,9 @@ const VALUE_FORMAT_SUBTYPES = new Set([ FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_DECIMAL, + // An ISO-8601 duration: runtime-ts refuses a value that is not one, and the column codec binds + // the dialect's spelling of it. + FIELD_SUBTYPE_DURATION, ]); /** Subtypes whose values are whole numbers — `field.currency` is integer minor units. */ diff --git a/server/typescript/packages/codegen-ts/src/templates/fr010-field-mapping.ts b/server/typescript/packages/codegen-ts/src/templates/fr010-field-mapping.ts index 14c43e9992..d5fd010644 100644 --- a/server/typescript/packages/codegen-ts/src/templates/fr010-field-mapping.ts +++ b/server/typescript/packages/codegen-ts/src/templates/fr010-field-mapping.ts @@ -16,6 +16,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, @@ -46,6 +47,8 @@ export function scalarKind(subType: string): string | null { // ADR-0036/0037 Wave 3: uri/inet are string scalars on the wire. case FIELD_SUBTYPE_URI: case FIELD_SUBTYPE_INET: + // field.duration is an ISO-8601 duration string on the wire. + case FIELD_SUBTYPE_DURATION: case FIELD_SUBTYPE_DATE: case FIELD_SUBTYPE_TIME: case FIELD_SUBTYPE_TIMESTAMP: diff --git a/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts b/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts index 1ba0acd7f4..d49ccc03cb 100644 --- a/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts +++ b/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts @@ -27,6 +27,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_REQUIRED, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_VALUE_TYPE, @@ -215,6 +216,8 @@ const SCALAR_TS_BY_SUBTYPE: Record = { // type, same as uuid). Other ports bind to their native URI/IP type. [FIELD_SUBTYPE_URI]: "string", [FIELD_SUBTYPE_INET]: "string", + // field.duration is an ISO-8601 duration string: TS has no native duration type. + [FIELD_SUBTYPE_DURATION]: "string", [FIELD_SUBTYPE_INT]: "number", [FIELD_SUBTYPE_LONG]: "number", [FIELD_SUBTYPE_DOUBLE]: "number", diff --git a/server/typescript/packages/codegen-ts/src/templates/names-decl.ts b/server/typescript/packages/codegen-ts/src/templates/names-decl.ts index 97e192394a..5a23a808aa 100644 --- a/server/typescript/packages/codegen-ts/src/templates/names-decl.ts +++ b/server/typescript/packages/codegen-ts/src/templates/names-decl.ts @@ -109,7 +109,13 @@ export function renderNamesDecl( }; const fieldsBlock = collection("fields", n.fields, n.ownFields, - (f: FieldNames) => `{ name: ${q(f.name)}, column: ${q(f.column)} }`); + (f: FieldNames) => f.members === undefined + ? `{ name: ${q(f.name)}, column: ${q(f.column)} }` + // A flattened value object: no column of its own, one per member of the value object. + : `{\n name: ${q(f.name)},\n` + + ` members: { ${Object.keys(f.members).sort() + .map((m) => `${key(m)}: { name: ${q(f.members![m]!.name)}, column: ${q(f.members![m]!.column)} }`) + .join(", ")} },\n }`); // `type` and `subType` on every entry, because the artifact mirrors the metadata tree // and because on these two collections the subType is the only thing that says whether diff --git a/server/typescript/packages/codegen-ts/src/templates/output-parser.ts b/server/typescript/packages/codegen-ts/src/templates/output-parser.ts index 1ce72468cd..24fef87d45 100644 --- a/server/typescript/packages/codegen-ts/src/templates/output-parser.ts +++ b/server/typescript/packages/codegen-ts/src/templates/output-parser.ts @@ -26,6 +26,7 @@ import { import type { MetaField } from "@metaobjectsdev/metadata"; import { enumValues } from "../enum-meta.js"; import { zodEnumExpr } from "./zod-enum.js"; +import { ZOD_DURATION_EXPR } from "./duration-wire-helper.js"; import { templateSymbolBase } from "../naming.js"; import { responseShape } from "./find-inbound.js"; import { isRequired } from "./fr010-field-mapping.js"; @@ -85,6 +86,8 @@ const SCALAR_ZOD: Record = { currency: "z.number().int()", uri: "z.string()", inet: "z.string()", + // An ISO-8601 duration string, in the accepted form (no years or months). + duration: ZOD_DURATION_EXPR, }; // ADR-0039: resolving — root has no super (children()==ownChildren()); a top-level object/template may itself extend, so resolve rather than work-by-accident. diff --git a/server/typescript/packages/codegen-ts/src/templates/projection-decl.ts b/server/typescript/packages/codegen-ts/src/templates/projection-decl.ts index 8041c3f4dd..add0e961d7 100644 --- a/server/typescript/packages/codegen-ts/src/templates/projection-decl.ts +++ b/server/typescript/packages/codegen-ts/src/templates/projection-decl.ts @@ -21,7 +21,7 @@ import type { ColumnNamingStrategy } from "../metaobjects-config.js"; import type { RenderContext } from "../render-context.js"; import { valueObjectModuleSpecifier } from "../import-path.js"; import type { Dialect } from "../metaobjects-config.js"; -import { physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "../names.js"; +import { columnExpr, physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "../names.js"; import { renderFilterAllowlist, renderSortAllowlist } from "./filter-allowlist.js"; import { renderFilterType } from "./filter-type.js"; import { buildUiFieldDescriptor, resourcePath } from "./entity-ui-descriptor.js"; @@ -156,11 +156,8 @@ export function renderProjectionDecl( // A6 — reference the constant whenever the artifact is in the run AND carries this // field. A lookup MISS (a TPH subtype field the base's names artifact never saw) is // normal, not a divergence — fall back to the literal, same as every other §A6 site. - const namesEntry = names?.resolved.fields[f.name]; - const dbColExpr: Code = - names !== undefined && namesEntry !== undefined - ? code`${names.symbol}.fields.${f.name}.column` - : code`${JSON.stringify(dbCol)}`; + // A flattened field has no `column` of its own (its members do), so it takes the literal. + const dbColExpr: Code = columnExpr(names, f.name, dbCol); // #356 — a projection's descriptor is the same shape the entity descriptor emits, // and now literally the SAME derivation: `buildUiFieldDescriptor` is what // `renderEntityConstants` reads, so the view kind, the label and the currency meta diff --git a/server/typescript/packages/codegen-ts/src/templates/view-decl.ts b/server/typescript/packages/codegen-ts/src/templates/view-decl.ts index 5880589ebe..7ef123bc1a 100644 --- a/server/typescript/packages/codegen-ts/src/templates/view-decl.ts +++ b/server/typescript/packages/codegen-ts/src/templates/view-decl.ts @@ -14,8 +14,9 @@ import { type MetaField, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, } from "@metaobjectsdev/metadata"; import type { ColumnNamingStrategy } from "../metaobjects-config.js"; -import { mapColumnType, type EnumIntCustomType } from "../column-mapper.js"; +import { mapColumnType, DURATION_COLUMN_FN, type EnumIntCustomType } from "../column-mapper.js"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { zodTypeFor } from "./field-meta.js"; import { columnExpr, type ObjectNames } from "../names.js"; @@ -25,7 +26,8 @@ export interface ViewDeclOpts { /** Drives the timestamp column TS type (Date vs string) in the view declaration. */ readonly timestampMode: "date" | "string"; /** - * Int-backed-enum codec const names ALREADY declared in the module this view lands in. + * Local column-codec const names (int-backed enums, the duration codec) ALREADY declared in + * the module this view lands in. * A write-through entity's table and its replica view share one file, and both reference * the same codecs — so the view references these by name and declares only what the table * did not (a derived field's codec, which the write table omits). Absent = the view owns @@ -88,6 +90,8 @@ function viewColumnLine( opts: ViewDeclOpts, /** Codecs this view needs, collected for the caller to DECLARE — see below. */ enumIntTypes: Map, + /** The duration codec's name, when a column needs it — declared by the caller, as above. */ + usesDuration: Set, ): Code { const { dialect, columnNamingStrategy, timestampMode } = opts; const spec = mapColumnType(f, dialect, columnNamingStrategy, timestampMode); @@ -99,10 +103,14 @@ function viewColumnLine( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + // The field.duration codec is a local const too; collected under its name, declared once. + if (spec.durationCustomType !== undefined) usesDuration.add(spec.durationCustomType.fnConstName); const colSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const optsArg = spec.fnOptions && Object.keys(spec.fnOptions).length > 0 ? `, ${JSON.stringify(spec.fnOptions)}` @@ -164,7 +172,8 @@ export function renderExistingViewDecl( const { viewFn, core: viewModule } = dialectModule(opts.dialect); const viewSym = imp(`${viewFn}@${viewModule}`); const enumIntTypes = new Map(); - const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes)); + const usesDuration = new Set(); + const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes, usesDuration)); const viewNameExpr = typeof viewName === "string" ? code`${JSON.stringify(viewName)}` : viewName; // @schema — a view lands in a schema exactly as a table does, and migrate qualifies the @@ -188,6 +197,9 @@ export function renderExistingViewDecl( .filter((t) => alreadyDeclared === undefined || !alreadyDeclared.has(t.fnConstName)) .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) .map((t) => renderEnumIntCustomType(t, viewModule)); + if (usesDuration.has(DURATION_COLUMN_FN) && (alreadyDeclared === undefined || !alreadyDeclared.has(DURATION_COLUMN_FN))) { + enumIntBlocks.unshift(renderDurationCustomType(opts.dialect, viewModule)); + } const viewBlock = code` // View declaration — Drizzle uses this for typed SELECT queries. diff --git a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts index 3cb10e53e9..87edb972d2 100644 --- a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts +++ b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts @@ -17,6 +17,7 @@ import { FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DECIMAL, FIELD_ATTR_PRECISION, FIELD_ATTR_SCALE, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_ARRAY, IDENTITY_ATTR_FIELDS, IDENTITY_ATTR_GENERATION, @@ -28,7 +29,9 @@ import { AUTO_SET_ON_CREATE, AUTO_SET_ON_UPDATE, VALIDATOR_ATTR_MAX, VALIDATOR_ATTR_MIN, VALIDATOR_ATTR_PATTERN, VALIDATOR_SUBTYPE_COMPARISON, VALIDATOR_SUBTYPE_REQUIRED_WHEN, - VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_SUBTYPE_EXACTLY_ONE, + VALIDATOR_ATTR_CONDITION, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, EXPR_FN_COALESCE, + FILTER_OP_IS_NULL, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, parseDurationLiteral, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, GENERATION_INCREMENT, GENERATION_UUID, @@ -48,6 +51,7 @@ import { } from "./date-time-regex.js"; import { renderDocsFor } from "./jsdoc.js"; import { DECIMAL_WRITE_HELPER_DECL, zodDecimalWriteExpr } from "./decimal-wire-helper.js"; +import { DURATION_WRITE_HELPER_DECL, ZOD_DURATION_EXPR, ZOD_DURATION_WRITE_EXPR } from "./duration-wire-helper.js"; import { sharedEnumForField } from "../enum-shared.js"; import { sharedEnumImportSpecifier } from "../enum-import.js"; import { sharedEnumZodConstName } from "./enums-file.js"; @@ -86,6 +90,13 @@ function decimalHelperSince(since: number): string { return decimalWriteUses > since ? `${DECIMAL_WRITE_HELPER_DECL}\n\n` : ""; } +// The same device for the duration write rule (duration-wire-helper.ts). +let durationWriteUses = 0; + +function durationHelperSince(since: number): string { + return durationWriteUses > since ? `${DURATION_WRITE_HELPER_DECL}\n\n` : ""; +} + /** True when this object declares at least one @autoSet timestamp field * (onCreate or onUpdate). Drives whether codegen emits the #203 @@ -260,13 +271,16 @@ function isInsertRequiredPk(field: MetaField, assignedPk: Set): boolean } // ── Entity-scoped cross-field validators on the full (insert) record ────────── -// validator.comparison / requiredWhen / presentIff / atLeastOne are enforced on the -// INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged +// validator.comparison / requiredWhen / presentIff / atLeastOne / exactlyOne are enforced on +// the INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged // row, so the Update schema is deliberately left alone (the DB CHECK stays the guard // there). Semantics mirror the migrate-ts CHECK derivation: absent / null / undefined is // SQL NULL, "" is NOT null; a comparison with a NULL side passes; requiredWhen is violated // only when the gate EQUALS the literal and the target is null; presentIff requires -// (target != null) === (gate equals literal), a null gate being "not equal". +// (target != null) === (gate equals literal), a null gate being "not equal"; exactlyOne +// requires exactly one non-null field. A rule with an @condition is checked only when the +// condition is TRUE: the CHECK is NOT (condition) OR (rule), so a condition that is false or +// NULL (three-valued, as SQL evaluates it) lets the row through. const CROSS_FIELD_OPS: Record = { gt: ">", gte: ">=", lt: "<", lte: "<=", ne: "!==", eq: "===", @@ -278,7 +292,7 @@ const NUMERIC_SUBTYPES: ReadonlySet = new Set([ ]); const ORDERED_SUBTYPES: ReadonlySet = new Set([ - FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, + FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_DURATION, ]); /** Subtypes with equality but no ordering: a pair of the same one may be tested with eq / ne. */ @@ -287,7 +301,7 @@ const EQUALITY_SUBTYPES: ReadonlySet = new Set([ ]); /** Whether a comparison between these two fields has a sound meaning, as in every port: - * numerics compare with each other, strings and temporals only with their own subtype and + * numerics compare with each other, strings, temporals and durations only with their own subtype and * @localTime flag (a date against a timestamp, or an instant against a wall clock, compares * two differently shaped strings, which is meaningless), * and boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, never rejected. */ @@ -329,10 +343,47 @@ function crossFieldLiteral(raw: unknown, when: MetaField): { lit: string; numeri * subtype also exports `InsertBaseSchema` (unrefined) and `InsertBodySchema` * (refined, pinned discriminator omitted). */ export function hasInsertRefinements(obj: MetaObject): boolean { - return crossFieldRefinementBody(obj).length > 0; + return crossFieldRefinement(obj).blocks.length > 0; } -function crossFieldRefinementBody(obj: MetaObject): string[] { +/** + * How a value is compared, from the field it is or is compared with: numerically, as a + * duration's microseconds (the canonical string does not order), or as written (strings, + * dates and times in their ISO form, booleans, enum members). + */ +type CompareKind = "number" | "duration" | "plain"; + +function compareKind(f: MetaField | undefined): CompareKind { + if (f === undefined) return "plain"; + if (NUMERIC_SUBTYPES.has(f.subType)) return "number"; + if (f.subType === FIELD_SUBTYPE_DURATION) return "duration"; + return "plain"; +} + +/** The local helpers a refinement body uses, declared once at its top when needed. */ +const HELPER_CMP = ` const cmp = (a: unknown, b: unknown): number | null => { + if (a == null || b == null) return null; + if (typeof a === "number" || typeof b === "number") return Number(a) - Number(b); + if (a instanceof Date || b instanceof Date) return new Date(a as string).getTime() - new Date(b as string).getTime(); + return String(a) < String(b) ? -1 : String(a) > String(b) ? 1 : 0; + };`; +const HELPER_TRI = ` const all = (...v: Array): boolean | null => v.includes(false) ? false : v.includes(null) ? null : true; + const any = (...v: Array): boolean | null => v.includes(true) ? true : v.includes(null) ? null : false; + const not = (v: boolean | null): boolean | null => v === null ? null : !v;`; +// An ISO-8601 duration (weeks, days, hours, minutes, seconds) as microseconds, or null when the +// text is not one (the accepted form: `P`, `PT` and `P1DT` are not durations). +const HELPER_MICROS = ` const micros = (d: unknown): number | null => { + if (d == null) return null; + const m = /^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$/.exec(String(d)); + if (m === null) return null; + const hours = (Number(m[2] ?? 0) * 7 + Number(m[3] ?? 0)) * 24 + Number(m[4] ?? 0); + const us = ((hours * 60 + Number(m[5] ?? 0)) * 60 + Number(m[6] ?? 0)) * 1e6 + Number((m[7] ?? "").padEnd(6, "0").slice(0, 6)); + return m[1] === "-" ? -us : us; + };`; + +interface RefinementBuild { blocks: string[]; helpers: Set } + +function crossFieldRefinement(obj: MetaObject): RefinementBuild { const inSchema = insertSchemaFieldNameSet(obj); const byName = new Map(obj.fields().map((f) => [f.name, f] as const)); // A reference must name a field that exists AND is in the insert schema; otherwise skip @@ -350,9 +401,116 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { : `row[${JSON.stringify(f.name)}]`; const issue = (path: string, msg: string) => `ctx.addIssue({ code: "custom", path: [${path}], message: ${JSON.stringify(msg)} });`; + // The helpers the rule being built uses; merged into `helpers` only when the rule is kept, + // so a rule left to the database declares nothing that nothing reads. + const helpers = new Set(); + let ruleHelpers = new Set(); + /** A value compared as `kind`. */ + const asKind = (expr: string, kind: CompareKind): string => { + if (kind === "number") return `(${expr} == null ? null : Number(${expr}))`; + if (kind === "duration") { ruleHelpers.add(HELPER_MICROS); return `micros(${expr})`; } + return expr; + }; + + /** The field a node compares as: itself, or a coalesce's first field. */ + const nodeField = (node: unknown): MetaField | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") return byName.get(n["field"]); + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + for (const a of n["args"] as unknown[]) { const f = nodeField(a); if (f !== undefined) return f; } + } + return undefined; + }; + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + const nodeFields = (node: unknown): MetaField[] => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { const f = byName.get(n["field"]); return f === undefined ? [] : [f]; } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) return (n["args"] as unknown[]).flatMap(nodeFields); + return []; + }; + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + const conditionComparable = (left: unknown, right: unknown, opName: string): boolean => { + const fields = [...nodeFields(left), ...nodeFields(right)]; + return fields.every((a, i) => fields.slice(i + 1).every((b) => crossComparable(a, b, opName))); + }; + /** A value node (field, literal, coalesce) as a TS expression compared as `kind`, or undefined. */ + const valueTs = (node: unknown, kind: CompareKind): string | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { + const f = ref(n["field"]); + return f === undefined ? undefined : asKind(acc(f), kind); + } + if ("value" in n) { + const value = n["value"]; + if (kind === "duration" && typeof value === "string") { + const us = parseDurationLiteral(value); + return us === undefined ? undefined : String(Number(us)); + } + return JSON.stringify(value); + } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map((a) => valueTs(a, kind)); + if (args.some((a) => a === undefined)) return undefined; + return `([${args.join(", ")}].find((x) => x != null) ?? null)`; + } + return undefined; + }; + /** A condition node as a TS expression of type `boolean | null`, or undefined when it names + * a field the body does not carry (the rule is then left to the database). */ + const condTs = (node: unknown): string | undefined => { + const n = (node ?? {}) as Record; + const op = n["op"]; + if (typeof op === "string" && CROSS_FIELD_OPS[op] !== undefined) { + if (!conditionComparable(n["left"], n["right"], op)) return undefined; + const kind = compareKind(nodeField(n["left"]) ?? nodeField(n["right"])); + const l = valueTs(n["left"], kind); + const r = valueTs(n["right"], kind); + if (l === undefined || r === undefined) return undefined; + ruleHelpers.add(HELPER_CMP); + return `((c) => c === null ? null : c ${CROSS_FIELD_OPS[op]} 0)(cmp(${l}, ${r}))`; + } + if (op === FILTER_OP_IS_NULL || op === EXPR_OP_IS_NOT_NULL) { + const a = valueTs(n["arg"], "plain"); + if (a === undefined) return undefined; + return op === FILTER_OP_IS_NULL ? `(${a} == null)` : `(${a} != null)`; + } + if (op === EXPR_OP_NOT) { + const a = condTs(n["arg"]); + if (a === undefined) return undefined; + ruleHelpers.add(HELPER_TRI); + return `not(${a})`; + } + if ((op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map(condTs); + if (args.some((a) => a === undefined)) return undefined; + ruleHelpers.add(HELPER_TRI); + return `${op === FILTER_COMPOSE_AND ? "all" : "any"}(${args.join(", ")})`; + } + return undefined; + }; + const blocks: string[] = []; + /** Add a rule's block, guarded by its @condition when it has one. */ + const push = (v: { attr(name: string): unknown }, block: string): void => { + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + const guard = condition === undefined ? undefined : condTs(condition); + if (condition !== undefined && guard === undefined) return; + for (const h of ruleHelpers) helpers.add(h); + if (guard === undefined) { blocks.push(block); return; } + const inner = block.split("\n").map((l) => (l === "" ? l : ` ${l}`)).join("\n"); + blocks.push(` if (${guard} === true) {\n${inner}\n }`); + }; + /** The @fields of an atLeastOne / exactlyOne, each in the body, or undefined. */ + const fieldList = (raw: unknown): MetaField[] | undefined => { + const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); + const refs = names.map(ref); + if (refs.length === 0 || refs.some((r) => r === undefined)) return undefined; + return refs as MetaField[]; + }; // ADR-0039: entity.validators() is the resolving accessor (never ownValidators). for (const v of obj.validators()) { + ruleHelpers = new Set(); switch (v.subType) { case VALIDATOR_SUBTYPE_COMPARISON: { const left = ref(v.attr(VALIDATOR_ATTR_LEFT)); @@ -361,12 +519,15 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const op = CROSS_FIELD_OPS[opName]; if (!left || !right || !op || left.resolvedIsArray() || right.resolvedIsArray()) break; if (!crossComparable(left, right, opName)) break; - const numeric = NUMERIC_SUBTYPES.has(left.subType); - const c = numeric + const kind = compareKind(left); + const c = kind === "number" || kind === "duration" ? "Number(l) - Number(r)" : "l instanceof Date && r instanceof Date ? l.getTime() - r.getTime() : String(l) < String(r) ? -1 : String(l) > String(r) ? 1 : 0"; - blocks.push( - ` {\n const l: unknown = ${acc(left)};\n const r: unknown = ${acc(right)};\n` + + // A duration side is read as its microseconds; text that is not a duration reads as + // null, so the rule is not checked (the field's own schema refuses that text). + const readKind: CompareKind = kind === "duration" ? "duration" : "plain"; + push(v, + ` {\n const l: unknown = ${asKind(acc(left), readKind)};\n const r: unknown = ${asKind(acc(right), readKind)};\n` + ` if (l != null && r != null) {\n const c = ${c};\n` + ` if (!(c ${op} 0)) ${issue(JSON.stringify(left.name), `${left.name} must be ${opName} ${right.name}`)}\n }\n }`, ); @@ -382,35 +543,41 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const path = JSON.stringify(target.name); const decl = ` const g: unknown = ${acc(when)};\n const t: unknown = ${acc(target)};\n const open = ${gate};\n`; if (v.subType === VALIDATOR_SUBTYPE_REQUIRED_WHEN) { - blocks.push(` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } else { - blocks.push(` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } break; } case VALIDATOR_SUBTYPE_AT_LEAST_ONE: { - const raw = v.attr(VALIDATOR_ATTR_FIELDS); - const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); - const refs = names.map(ref); - if (refs.length === 0 || refs.some((r) => r === undefined)) break; - const fields = refs as MetaField[]; + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; const anyPresent = fields.map((f) => `${acc(f)} != null`).join(" || "); - blocks.push(` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + push(v, ` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + break; + } + case VALIDATOR_SUBTYPE_EXACTLY_ONE: { + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; + const present = fields.map((f) => `${acc(f)}`).join(", "); + push(v, ` if ([${present}].filter((x) => x != null).length !== 1) ${issue(JSON.stringify(fields[0]!.name), `exactly one of ${fields.map((f) => f.name).join(", ")} is required`)}`); break; } default: break; } } - return blocks; + return { blocks, helpers }; } /** `.superRefine(...)` suffix for an insert-shaped schema, or "" when the object has no * enforceable cross-field validator (output is then byte-identical to before). */ function insertRefinementSuffix(obj: MetaObject): string { - const blocks = crossFieldRefinementBody(obj); + const { blocks, helpers } = crossFieldRefinement(obj); if (blocks.length === 0) return ""; - return `.superRefine((row, ctx) => {\n${blocks.join("\n")}\n})`; + // Declared in a fixed order, so the output does not depend on which rule needed one first. + const decls = [HELPER_CMP, HELPER_TRI, HELPER_MICROS].filter((h) => helpers.has(h)); + return `.superRefine((row, ctx) => {\n${[...decls, ...blocks].join("\n")}\n})`; } /** @@ -426,6 +593,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); @@ -477,7 +645,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const docsPrefix = docs ? `${docs}\n` : ""; return code` -${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ +${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${durationHelperSince(durationUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ ${joinCode(insertFieldLines, { on: ",\n" })} })${insertRefinementSuffix(obj)}; `; @@ -586,6 +754,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const pkFields = primaryKeyFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); @@ -728,7 +897,7 @@ export type ${objName}CreatePreserving = ${z}.input { test("string → text(name)", () => { @@ -299,6 +300,34 @@ describe("mapColumnType — field.uuid (R6 Plan 2a)", () => { expect(mapColumnType(metaField(FIELD_SUBTYPE_INET, "ip"), "sqlite").fnName).toBe("text"); }); + // field.duration — a local customType codec on every dialect (interval / text / varchar(64)), + // because Drizzle's pg-core interval() reads Postgres's text spelling, not the ISO wire form. + test("duration → the local duration codec on every dialect", () => { + for (const dialect of ["postgres", "sqlite", "mysql"] as const) { + const spec = mapColumnType(metaField(FIELD_SUBTYPE_DURATION, "holdFor"), dialect); + expect(spec.fnName).toBe(DURATION_COLUMN_FN); + expect(spec.durationCustomType).toEqual({ fnConstName: DURATION_COLUMN_FN, dialect }); + expect(spec.fnOptions).toBeUndefined(); + } + }); + + test("duration isArray → interval[] on Postgres; a JSON string array on SQLite and MySQL", () => { + const laps = (): ReturnType => { + const f = metaField(FIELD_SUBTYPE_DURATION, "laps"); + f.setIsArray(true); + return f; + }; + const pg = mapColumnType(laps(), "postgres"); + expect(pg.fnName).toBe(DURATION_COLUMN_FN); + expect(pg.modifiers).toContain(".array()"); + const lite = mapColumnType(laps(), "sqlite"); + expect(lite.fnName).toBe("text"); + expect(lite.durationCustomType).toBeUndefined(); + expect(lite.dollarTypeRef).toEqual({ kind: "scalar", tsType: "string", array: true }); + const my = mapColumnType(laps(), "mysql"); + expect(my.fnName).toBe("json"); + }); + // Native arrays are DERIVED from isArray (dbColumnType slim-and-derive Phase 1): // no @dbColumnType:uuid_array / text_array escape hatch — the array-ness lives on // the field's isArray axis. diff --git a/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts b/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts index 310aa57c52..7a8f6ff382 100644 --- a/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts +++ b/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts @@ -86,7 +86,7 @@ function sampleRow(): Record { id: "6f1c2b1e-0d7a-4d55-9c1e-0a4e8f1d2c3b", title: "A title", email: null, host: null, quantity: 3, views: null, ratio: null, score: null, amount: null, active: null, startsOn: null, startsAt: null, publishedAt: null, wallClock: null, priceCents: null, status: "draft", - stage: null, homepage: null, looseLink: null, address: null, shipTo: null, stops: null, tags: null, + stage: null, homepage: null, looseLink: null, address: null, holdFor: null, shipTo: null, stops: null, tags: null, internalNote: null, issuedBy: null, syncedBy: null, createdAt: null, updatedAt: null, }; } @@ -174,6 +174,23 @@ describe("types the JSON Schema formats do not state", () => { expect(valid({ ...sampleRow(), startsAt: "9am" })).toBe(false); }); + test("with wire.durations iso, a duration is an ISO-8601 duration of weeks, days, hours, minutes and seconds — never a calendar span", async () => { + const valid = validatorOf(await contractTree(corpusCase("all-fields-wire-string-omit")), `${ALL_FIELDS}.schema.json`); + for (const ok of ["PT1H30M", "P1DT2H", "-PT1M30S", "PT0.5S", "PT0S", "P2W", "P1W1DT1H", "PT1.1234567890S"]) { + expect(valid({ ...sampleRow(), holdFor: ok }), ok).toBe(true); + } + // ajv-formats' `duration` alone takes years and months; the pattern refuses them. + for (const bad of ["P1M", "P1Y", "P1D1W", "1:30:00", "PT1.5H", "pt1h", "PT"]) { + expect(valid({ ...sampleRow(), holdFor: bad }), bad).toBe(false); + } + }); + + test("under the default, a duration is the serving language's own form, so any string", async () => { + const valid = validatorOf(await contractTree(corpusCase("all-fields")), `${ALL_FIELDS}.schema.json`); + for (const ok of ["PT1H30M", "01:30:00", "1.02:00:00"]) expect(valid({ ...sampleRow(), holdFor: ok }), ok).toBe(true); + expect(valid({ ...sampleRow(), holdFor: 90 })).toBe(false); + }); + test("a local timestamp has no offset, and a zoned one must", async () => { const valid = validatorOf(await contractTree(corpusCase("all-fields")), `${ALL_FIELDS}.schema.json`); expect(valid({ ...sampleRow(), wallClock: "2026-10-30T09:30:00" })).toBe(true); diff --git a/server/typescript/packages/codegen-ts/test/contract/duration-json-schema-pattern.test.ts b/server/typescript/packages/codegen-ts/test/contract/duration-json-schema-pattern.test.ts new file mode 100644 index 0000000000..2b08dd0e64 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/contract/duration-json-schema-pattern.test.ts @@ -0,0 +1,51 @@ +// The JSON Schema `pattern` of a field.duration is DURATION_PATTERN spelled without lookaround (not +// I-Regexp; RE2-based validators refuse it). The two must give the same verdict on every duration +// vector, and on shapes around each lookahead the plain pattern replaces. + +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { DURATION_PATTERN } from "@metaobjectsdev/metadata"; +import { DURATION_JSON_SCHEMA_PATTERN } from "../../src/contract/field-schema.js"; + +interface Vectors { + wire: { + parse: Array<{ in: string; out: string }>; + precision: Array<{ in: string; out: Record }>; + refused: string[]; + }; +} +const { wire } = JSON.parse( + readFileSync(join(import.meta.dir, "..", "..", "..", "..", "..", "..", "fixtures", "duration-wire", "vectors.json"), "utf8"), +) as Vectors; +/** Every duration the wire accepts: the parse ins and the precision ins (a fraction of any length). */ +const accepted = [...wire.parse.map((p) => p.in), ...wire.precision.map((p) => p.in)]; + +const lookahead = new RegExp(DURATION_PATTERN); +const plain = new RegExp(DURATION_JSON_SCHEMA_PATTERN); + +/** Shapes the vectors do not spell, around each lookahead the plain pattern replaces. */ +const EXTRA = [ + "P1D", "P1DT1H", "P1DT1M", "P1DT1S", "P1DT1H1M1S", "PT1H1M", "PT1M1.5S", "PT1H1.000000001S", + "P1DT", "PT", "P", "PT1", "P1", "P1DT1", "PT1H1", "PT1HM", "PTH", "P1D1H", "-P", "+PT", "PT1H1M1S1", + "P1W1D", "P1WT1H", "P1W1DT1M1.5S", "P1WT", "P1W1DT", "P1WD", "PW", "P1W1W", "P1W1", "-P1W1DT1S", +]; + +describe("the JSON Schema duration pattern", () => { + test("has no lookaround, no \\d and no non-capturing group", () => { + expect(DURATION_JSON_SCHEMA_PATTERN).not.toMatch(/\(\?/); + expect(DURATION_JSON_SCHEMA_PATTERN).not.toContain("\\d"); + }); + + for (const v of [...accepted, ...wire.refused, ...EXTRA]) { + test(`${JSON.stringify(v)}: the same verdict as DURATION_PATTERN`, () => { + expect(plain.test(v)).toBe(lookahead.test(v)); + }); + } + for (const v of accepted) { + test(`accepts ${v}`, () => expect(plain.test(v)).toBe(true)); + } + for (const v of wire.refused) { + test(`refuses ${JSON.stringify(v)}`, () => expect(plain.test(v)).toBe(false)); + } +}); diff --git a/server/typescript/packages/codegen-ts/test/cube/build-cube-model.test.ts b/server/typescript/packages/codegen-ts/test/cube/build-cube-model.test.ts index e431c038dc..2aa37d6a30 100644 --- a/server/typescript/packages/codegen-ts/test/cube/build-cube-model.test.ts +++ b/server/typescript/packages/codegen-ts/test/cube/build-cube-model.test.ts @@ -284,6 +284,15 @@ describe("Table C — dimension types", () => { expect(await dimensionOf(field)).toEqual({ name: "d", sql: '{CUBE}."x"', type: "string" }); }); + test("duration → number, its span in seconds (exact), so it sorts and filters by length", async () => { + const field = { "field.duration": { name: "x" } }; + expect(await dimensionOf(field)).toEqual({ name: "d", sql: 'EXTRACT(EPOCH FROM {CUBE}."x")', type: "number" }); + // MySQL holds a BIGINT of microseconds; widening the dividend keeps all six fractional digits. + expect(await dimensionOf(field, {}, "mysql")).toEqual({ + name: "d", sql: "CAST({CUBE}.`x` AS DECIMAL(65, 6)) / 1000000", type: "number", + }); + }); + test("enum with @intValueMap → string, a CASE that carries the member symbol", async () => { expect( await dimensionOf({ "field.enum": { name: "x", "@values": ["A", "B"], "@intValueMap": { A: 1, B: 2 } } }), diff --git a/server/typescript/packages/codegen-ts/test/drizzle-duration-column.test.ts b/server/typescript/packages/codegen-ts/test/drizzle-duration-column.test.ts new file mode 100644 index 0000000000..f7a0bd9ecd --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/drizzle-duration-column.test.ts @@ -0,0 +1,184 @@ +// field.duration in the generated Drizzle modules: the column is the module-local duration codec +// (a `customType`, never an import from drizzle core), declared ONCE per module — including the +// write-through entity whose table and replica view share one file — and the module compiles on +// every dialect. The codec's behaviour is held to fixtures/duration-wire/vectors.json by +// duration-wire-helper.test.ts; this file runs it once more through a real SQLite round trip. + +import { describe, test, expect } from "bun:test"; +import { Database } from "bun:sqlite"; +import { mkdtempSync, writeFileSync, rmSync } from "node:fs"; +import { join } from "node:path"; +import { pathToFileURL } from "node:url"; +import ts from "typescript"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { entityFile } from "../src/generators/entity-file.js"; +import { makeRenderContext } from "../src/render-context.js"; +import { buildPkMap } from "../src/pk-resolver.js"; +import { buildRelationMap } from "../src/relation-resolver.js"; +import type { GenContext } from "../src/generator.js"; +import type { Dialect } from "../src/metaobjects-config.js"; + +async function loadRoot() { + const json = JSON.stringify({ + "metadata.root": { + package: "test", + children: [ + { + "object.entity": { + name: "Hold", + children: [ + { "source.rdb": { "@table": "hold" } }, + { "field.int": { name: "id" } }, + { "field.duration": { name: "holdFor", "@required": true } }, + { "field.duration": { name: "grace" } }, + { "field.duration": { name: "laps", isArray: true } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ], + }, + }, + { + "object.entity": { + name: "Timer", + children: [ + // Write-through: the table and its replica view land in one module. + { "source.rdb": { "@kind": "table", "@table": "timer", "@role": "primary" } }, + { "source.rdb": { "@kind": "view", "@view": "v_timer", "@role": "replica" } }, + { "field.int": { name: "id" } }, + { "field.duration": { name: "period", "@required": true } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ], + }, + }, + { + "object.projection": { + name: "HoldView", + children: [ + { "source.rdb": { "@kind": "view", "@view": "v_hold" } }, + { "field.int": { name: "id", extends: "Hold.id", "@required": true } }, + { "field.duration": { name: "holdFor", extends: "Hold.holdFor", "@required": true } }, + { "identity.primary": { name: "pk", extends: "Hold.pk" } }, + ], + }, + }, + ], + }, + }); + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`); + } + return result.root; +} + +function compile(dir: string, files: string[]): readonly ts.Diagnostic[] { + const program = ts.createProgram(files.map((f) => join(dir, f)), { + strict: true, + noEmit: true, + noUncheckedIndexedAccess: true, + target: ts.ScriptTarget.ES2022, + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + skipLibCheck: true, + }); + return ts.getPreEmitDiagnostics(program); +} + +async function generate(dialect: Dialect, dir: string) { + const root = await loadRoot(); + const renderContext = makeRenderContext({ + dialect, + loadedRoot: root, + outDir: dir, + dbImport: "~/db", + pkMap: buildPkMap(root), + relationMap: buildRelationMap(root), + }); + const ctx: GenContext = { + entities: root.objects(), + loadedRoot: root, + matches: () => true, + projectRoot: dir, + config: { outDir: dir, extStyle: "none", dbImport: "~/db", dialect } as never, + renderContext, + warn: () => {}, + }; + return await entityFile({ allowlists: false }).generate(ctx); +} + +const occurrences = (text: string, needle: string): number => text.split(needle).length - 1; + +describe("generated field.duration columns", () => { + for (const dialect of ["sqlite", "postgres", "mysql"] as const) { + test(`${dialect}: the codec is local, declared once per module, and every module compiles`, async () => { + const dir = mkdtempSync(join(import.meta.dir, `tmp-duration-${dialect}-`)); + try { + const files = await generate(dialect, dir); + for (const f of files) writeFileSync(join(dir, f.path), f.content); + const core = `drizzle-orm/${dialect === "postgres" ? "pg" : dialect}-core`; + + for (const name of ["Hold", "Timer", "HoldView"]) { + const file = files.find((f) => f.path === `${name}.ts`); + expect(file, `${name}.ts was generated`).toBeDefined(); + const content = file!.content; + expect(occurrences(content, "const durationColumn =")).toBe(1); + expect(content).not.toMatch(new RegExp(`import\\s*\\{[^}]*durationColumn[^}]*\\}\\s*from\\s*"${core}"`)); + expect(content).toContain(`durationColumn(`); + } + const hold = files.find((f) => f.path === "Hold.ts")!.content; + expect(hold).toContain(dialect === "postgres" ? 'dataType: () => "interval"' : dialect === "mysql" ? 'dataType: () => "bigint"' : 'dataType: () => "integer"'); + // The write schemas carry the canonicalizing helper, once. + expect(occurrences(hold, "function durationWrite(")).toBe(1); + if (dialect === "postgres") expect(hold).toMatch(/laps: durationColumn\([^)]*\)\.array\(\)/); + + const diagnostics = compile(dir, files.map((f) => f.path)); + expect(diagnostics.map((d) => ts.flattenDiagnosticMessageText(d.messageText, "\n"))).toEqual([]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + test("sqlite: a duration is stored as integer microseconds, reads back canonical, and orders as a span", async () => { + const dir = mkdtempSync(join(import.meta.dir, "tmp-duration-sqlite-run-")); + try { + const files = await generate("sqlite", dir); + for (const f of files) writeFileSync(join(dir, f.path), f.content); + // The generated module imports `~/db` only from the query files; the table module needs + // drizzle alone, which resolves from this package. + const mod = (await import(pathToFileURL(join(dir, "Hold.ts")).href)) as Record; + const { drizzle } = await import("drizzle-orm/bun-sqlite"); + const { asc, eq, gt, lt } = await import("drizzle-orm"); + const sqlite = new Database(":memory:"); + sqlite.run("CREATE TABLE hold (id INTEGER PRIMARY KEY, hold_for INTEGER NOT NULL, grace INTEGER, laps TEXT)"); + const db = drizzle(sqlite); + // biome-ignore lint/suspicious/noExplicitAny: the generated table, imported dynamically + const table = mod.holds as any; + const insert = (mod.HoldInsertSchema as { parse(v: unknown): Record }).parse({ + id: 1, holdFor: "P1DT0.5S", grace: "-PT90S", laps: ["PT90M"], + }); + await db.insert(table).values(insert); + expect(sqlite.query("SELECT hold_for, grace FROM hold").get()).toEqual({ hold_for: 86400500000, grace: -90000000 }); + // A filter value binds through the codec too: a spelling of the same span matches. + const rows = await db.select().from(table).where(eq(table.holdFor, "P1DT0.5S")); + expect(rows).toEqual([{ id: 1, holdFor: "PT24H0.5S", grace: "-PT1M30S", laps: ["PT1H30M"] }]); + // Ordering is the span's, not the text's: PT9S < PT10S < PT1H (as text, "PT10S" < "PT1H" < "PT9S"). + const parse = (mod.HoldInsertSchema as { parse(v: unknown): Record }).parse; + for (const [id, holdFor] of [[2, "PT10S"], [3, "PT9S"], [4, "PT1H"], [5, "-PT1M"]] as const) { + await db.insert(table).values(parse({ id, holdFor })); + } + const sorted = await db.select({ holdFor: table.holdFor }).from(table).orderBy(asc(table.holdFor)); + expect(sorted.map((r: { holdFor: string }) => r.holdFor)).toEqual(["-PT1M", "PT9S", "PT10S", "PT1H", "PT24H0.5S"]); + const between = await db.select({ id: table.id }).from(table) + .where(gt(table.holdFor, "PT9S")).orderBy(asc(table.holdFor)); + expect(between.map((r: { id: number }) => r.id)).toEqual([2, 4, 1]); + const under = await db.select({ id: table.id }).from(table).where(lt(table.holdFor, "PT10S")).orderBy(asc(table.id)); + expect(under.map((r: { id: number }) => r.id)).toEqual([3, 5]); + // runtime-ts binds a SQLite operand as the microsecond integer; the codec passes it through. + const byMicros = await db.select({ id: table.id }).from(table).where(eq(table.holdFor, 10000000 as never)); + expect(byMicros).toEqual([{ id: 2 }]); + sqlite.close(); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/drizzle-enum-max-length.test.ts b/server/typescript/packages/codegen-ts/test/drizzle-enum-max-length.test.ts new file mode 100644 index 0000000000..c932d7bd6a --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/drizzle-enum-max-length.test.ts @@ -0,0 +1,57 @@ +// A string-stored enum with a column length (its @maxLength, else the project's +// enums.maxLength) binds a varchar of that length, the column `meta migrate` creates; +// with none it stays text on Postgres and sizes to the longest member on MySQL, as before. +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { renderDrizzleSchema } from "../src/templates/drizzle-schema.js"; +import { makeRenderContext } from "../src/render-context.js"; +import { buildPkMap } from "../src/pk-resolver.js"; +import { buildRelationMap } from "../src/relation-resolver.js"; +import type { Dialect } from "../src/metaobjects-config.js"; + +async function emit(statusDecl: Record, dialect: Dialect, maxLength?: number) { + const json = JSON.stringify({ "metadata.root": { children: [ + { "object.entity": { name: "Order", children: [ + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", ...statusDecl } }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ] } }, + ] } }); + const res = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(res.errors).toEqual([]); + if (maxLength !== undefined) res.root.enumDefaults = { maxLength }; + const ctx = makeRenderContext({ + dialect, loadedRoot: res.root, outDir: "/x", dbImport: "~/db", + pkMap: buildPkMap(res.root), relationMap: buildRelationMap(res.root), + }); + return renderDrizzleSchema(res.root.findObject("Order")!, ctx).toString(); +} + +const VALUES = ["DRAFT", "PUBLISHED"]; + +describe("Drizzle: a string-stored enum's column length", () => { + test("Postgres: @maxLength → varchar of that length", async () => { + expect(await emit({ "@values": VALUES, "@maxLength": 16 }, "postgres")).toContain('varchar("status", { length: 16, enum:'); + }); + + test("Postgres: the project default sizes an enum with no @maxLength", async () => { + expect(await emit({ "@values": VALUES }, "postgres", 32)).toContain('varchar("status", { length: 32, enum:'); + }); + + test("Postgres: no length anywhere → text, as before", async () => { + expect(await emit({ "@values": VALUES }, "postgres")).toContain('text("status", { enum:'); + }); + + test("MySQL: @maxLength beats the longest member", async () => { + expect(await emit({ "@values": VALUES, "@maxLength": 16 }, "mysql")).toContain('varchar("status", { length: 16, enum:'); + }); + + test("Postgres: an array enum stays text[] — migrate's column — whatever the length", async () => { + const own = await emit({ "@values": VALUES, "@maxLength": 16, isArray: true }, "postgres"); + expect(own).not.toContain("varchar("); + // A member longer than the project default does not refuse an array enum: it has no length. + const long = await emit({ "@values": ["A_VERY_LONG_MEMBER"], isArray: true }, "postgres", 4); + expect(long).not.toContain("varchar("); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/duration-wire-helper.test.ts b/server/typescript/packages/codegen-ts/test/duration-wire-helper.test.ts new file mode 100644 index 0000000000..f9e6f399c9 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/duration-wire-helper.test.ts @@ -0,0 +1,170 @@ +// The field.duration code a generated model module carries (templates/duration-wire-helper.ts): +// the Zod write helper and the Drizzle column codec. The EMITTED TEXT is transpiled and evaluated +// here against fixtures/duration-wire/vectors.json, so the vectors hold what an adopter runs, not +// a re-implementation of it. The emitted code counts microseconds (precision "6") and has the +// Postgres text codec (it binds and reads interval text) and the stored-micros codec (SQLite, D1, +// MySQL), so it passes every `codecs` group too. + +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { z } from "zod"; +import { DURATION_PATTERN, canonicalDuration } from "@metaobjectsdev/metadata"; +import { DURATION_COLUMN_FN } from "../src/column-mapper.js"; +import { + DURATION_REGEX_LITERAL, + DURATION_WRITE_HELPER_DECL, + ZOD_DURATION_WRITE_EXPR, + durationColumnDeclParts, +} from "../src/templates/duration-wire-helper.js"; +import type { Dialect } from "../src/metaobjects-config.js"; + +interface Vectors { + wire: { + parse: Array<{ in: string; out: string }>; + precision: Array<{ in: string; out: Record }>; + refused: string[]; + }; + codecs: { + postgres: Array<{ in: string; out: string }>; + postgresRefused: string[]; + postgresText: Array<{ in: string; out: string }>; + storedMicros: Array<{ in: string; out: string }>; + }; +} +const { wire, codecs } = JSON.parse( + readFileSync(join(import.meta.dir, "..", "..", "..", "..", "..", "fixtures", "duration-wire", "vectors.json"), "utf8"), +) as Vectors; +/** The emitted code holds a duration as microseconds. */ +const PRECISION = "6"; +/** The parse vectors, then the precision vectors at this precision. */ +const parsed = [...wire.parse, ...wire.precision.map((p) => ({ in: p.in, out: p.out[PRECISION]! }))]; + +const transpiler = new Bun.Transpiler({ loader: "ts" }); + +/** The write schema exactly as a generated module spells it: the helper declaration, then the expression. */ +function writeSchema(): z.ZodType { + const js = transpiler.transformSync(`${DURATION_WRITE_HELPER_DECL}\nexport const schema = ${ZOD_DURATION_WRITE_EXPR};`); + const body = js.replace(/export const schema =/, "return"); + return new Function("z", body)(z) as z.ZodType; +} + +interface Codec { dataType(): string; toDriver(v: string): string | number | bigint; fromDriver(v: unknown): string } + +/** The column codec exactly as a generated module spells it; `customType` hands back its config. */ +function columnCodec(dialect: Dialect): Codec { + const [before, after] = durationColumnDeclParts(dialect); + const js = transpiler.transformSync(`${before}customType${after}\nexport const codec = ${DURATION_COLUMN_FN};`); + const body = js.replace(/export const codec =/, "return"); + return new Function("customType", body)((cfg: Codec) => cfg) as Codec; +} + +describe("generated duration write schema", () => { + const schema = writeSchema(); + + test("the Zod regex is the metadata's accepted pattern", () => { + expect(DURATION_REGEX_LITERAL).toBe(`/${DURATION_PATTERN}/`); + expect(ZOD_DURATION_WRITE_EXPR).toBe(`z.string().regex(/${DURATION_PATTERN}/).transform(durationWrite)`); + }); + for (const v of parsed) { + test(`parse ${v.in} -> ${v.out}`, () => { + expect(schema.parse(v.in)).toBe(v.out); + // the canonical form is itself accepted, and is a fixed point + expect(schema.parse(v.out)).toBe(v.out); + }); + } + for (const v of wire.refused) { + test(`refused ${JSON.stringify(v)}`, () => { + expect(schema.safeParse(v).success).toBe(false); + }); + } + test("no limit of its own: a span past every 64-bit or TimeSpan range is exact, never in exponent notation", () => { + expect(schema.parse("PT256204779H")).toBe("PT256204779H"); + expect(schema.parse("P99999999999999999999999D")).toBe("PT2399999999999999999999976H"); + expect(schema.parse("PT1" + "0".repeat(40) + ".000001S")).toBe( + "PT2777777777777777777777777777777777777H46M40.000001S", + ); + }); + test("a non-string is refused", () => { + expect(schema.safeParse(90).success).toBe(false); + expect(schema.safeParse(null).success).toBe(false); + }); +}); + +describe("generated duration column codec", () => { + const pg = columnCodec("postgres"); + const sqlite = columnCodec("sqlite"); + const mysql = columnCodec("mysql"); + + test("the physical column type per dialect: interval; a 64-bit integer of microseconds elsewhere", () => { + expect(pg.dataType()).toBe("interval"); + expect(sqlite.dataType()).toBe("integer"); + expect(mysql.dataType()).toBe("bigint"); + }); + + for (const v of codecs.postgresText) { + test(`Postgres binds ${v.in} as ${v.out}, and reads it back`, () => { + expect(pg.toDriver(v.in)).toBe(v.out); + expect(pg.fromDriver(v.out)).toBe(v.in); + }); + } + for (const v of codecs.storedMicros) { + test(`SQLite / MySQL store ${v.in} as ${v.out} microseconds, and read it back`, () => { + // Read back, it is the canonical form (sub-microsecond digits already truncated). + const canonical = canonicalDuration(v.in)!; + for (const codec of [sqlite, mysql]) { + expect(String(codec.toDriver(v.in))).toBe(v.out); + // A driver hands the integer back as a number, a bigint or a digit string. + expect(codec.fromDriver(Number(v.out))).toBe(canonical); + expect(codec.fromDriver(BigInt(v.out))).toBe(canonical); + expect(codec.fromDriver(v.out)).toBe(canonical); + } + }); + } + for (const v of parsed) { + test(`${v.in} round-trips as ${v.out} on every dialect`, () => { + for (const codec of [pg, sqlite, mysql]) expect(codec.fromDriver(codec.toDriver(v.in))).toBe(v.out); + }); + } + test("a safe microsecond count binds as a number; past 2^53 as a bigint; past the 64-bit range it is refused", () => { + expect(sqlite.toDriver("PT876000H")).toBe(3153600000000000); + expect(sqlite.toDriver("PT2562047788H")).toBe(BigInt("9223372036800000000")); + expect(sqlite.fromDriver(BigInt("9223372036800000000"))).toBe("PT2562047788H"); + expect(() => sqlite.toDriver("PT2562047789H")).toThrow(/64-bit/); + expect(() => mysql.toDriver("-PT2562047789H")).toThrow(/64-bit/); + // Postgres has no such limit in the codec: its interval range is Postgres's to enforce. + expect(pg.toDriver("PT2562047789H")).toBe("2562047789:00:00"); + }); + test("runtime-ts's filter operands bind through the codec unchanged: the micros on SQLite / MySQL, the interval text on Postgres", () => { + expect(sqlite.toDriver(5400000000 as never)).toBe(5400000000); + expect(mysql.toDriver(BigInt("-90000000") as never)).toBe(-90000000); + expect(pg.toDriver("01:30:00")).toBe("01:30:00"); + }); + for (const v of wire.refused) { + test(`binding ${JSON.stringify(v)} throws`, () => { + // The codec takes the Postgres spelling too (runtime-ts binds a Postgres operand in it), and + // `1:30:00` is that spelling — one hour thirty — so on Postgres it binds. + if (v !== "1:30:00") expect(() => pg.toDriver(v)).toThrow(); + expect(() => sqlite.toDriver(v === "1:30:00" ? "1:30" : v)).toThrow(); + }); + } + for (const v of codecs.postgres) { + test(`Postgres text ${v.in} reads as ${v.out}`, () => { + expect(pg.fromDriver(v.in)).toBe(v.out); + }); + } + for (const v of codecs.postgresRefused) { + test(`Postgres text ${v} is refused on read and on bind`, () => { + expect(() => pg.fromDriver(v)).toThrow(); + expect(() => pg.toDriver(v)).toThrow(); + }); + } + test("a node-postgres interval object reads as the canonical form; a month part is refused", () => { + expect(pg.fromDriver({ days: 1, hours: 2 })).toBe("PT26H"); + expect(pg.fromDriver({ minutes: -1, seconds: -30 })).toBe("-PT1M30S"); + expect(pg.fromDriver({ seconds: 1, milliseconds: 500.25 })).toBe("PT1.50025S"); + expect(() => pg.fromDriver({ months: 1 })).toThrow(/year or month/); + expect(() => pg.fromDriver(1.5)).toThrow(); + expect(() => pg.fromDriver(true)).toThrow(); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/enum-max-length-ui.test.ts b/server/typescript/packages/codegen-ts/test/enum-max-length-ui.test.ts new file mode 100644 index 0000000000..61184bc37d --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/enum-max-length-ui.test.ts @@ -0,0 +1,26 @@ +// An enum's @maxLength is its column length, not an input rule: the UI descriptor emits no +// maxLength rule for it (the member list is the rule), while a string's @maxLength still is one. +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource, type MetaObject } from "@metaobjectsdev/metadata"; +import { buildUiFieldDescriptor } from "../src/templates/entity-ui-descriptor.js"; + +describe("UI descriptor — @maxLength", () => { + test("an enum's column length is not a form rule; a string's maxLength still is", async () => { + const json = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", "@values": ["OPEN", "CLOSED"], "@maxLength": 16 } }, + { "field.string": { name: "title", "@maxLength": 40 } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + ] } }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(errors).toEqual([]); + const ticket = root.findObject("Ticket") as MetaObject; + const rules = (name: string) => + JSON.stringify(buildUiFieldDescriptor(ticket.fields().find((f) => f.name === name)!, root)); + expect(rules("status")).not.toContain("maxLength"); + expect(rules("title")).toContain("maxLength"); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/names-flattened.test.ts b/server/typescript/packages/codegen-ts/test/names-flattened.test.ts new file mode 100644 index 0000000000..d6d0b6cf2a --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/names-flattened.test.ts @@ -0,0 +1,209 @@ +// A flattened value object has no column of its own: each member is a column of the owning +// table, named by core naming (`resolveFlattenedColumnName`). The names artifact carries +// those columns under the field's `members`, so code that needs a flattened column reads a +// constant instead of spelling it. The field's entry has no `column`: it would name a column +// the table does not have, so reading it is a compile error. +// +// The corpus is the cross-port naming fixture: every `..` key in it is +// a flattened column, and the artifact must name each one exactly as `meta migrate` does, +// under every naming rule. + +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import ts from "typescript"; +import { FileSource } from "@metaobjectsdev/metadata/core"; +import { + MetaDataLoader, + InMemoryStringSource, + isMetaRoot, + type ColumnNamingStrategy, + type MetaObject, + type MetaRoot, + type TableNamingRule, +} from "@metaobjectsdev/metadata"; +import { resolveObjectNames, type FieldNames } from "../src/names.js"; +import { renderNamesDecl } from "../src/templates/names-decl.js"; +import { renderProjectionDecl } from "../src/templates/projection-decl.js"; +import { code } from "ts-poet"; + +const CORPUS = resolve(import.meta.dir, "../../../../../fixtures/naming-conformance/physical-names"); + +interface NamingCase { + name: string; + naming: { columns?: ColumnNamingStrategy; tables?: TableNamingRule } | null; + columns: Record; +} +const corpus = JSON.parse(readFileSync(join(CORPUS, "cases.json"), "utf8")) as { cases: NamingCase[] }; + +async function loadCorpusModel(c: NamingCase): Promise { + const result = await new MetaDataLoader().load([new FileSource(join(CORPUS, "meta.naming.json"))]); + expect(result.errors).toEqual([]); + if (!isMetaRoot(result.root)) throw new Error("not a root"); + result.root.physicalNaming = { + columns: c.naming?.columns ?? "snake_case", + tables: c.naming?.tables ?? "snake_case", + }; + return result.root; +} + +/** The column the artifact names for a corpus key, or undefined when it names none. */ +function artifactColumn(root: MetaRoot, key: string): string | undefined { + const [objectName, fieldName, memberName] = key.split(".") as [string, string, string | undefined]; + const names = resolveObjectNames(root.findObject(objectName) as MetaObject); + const entry: FieldNames | undefined = names?.fields[fieldName]; + if (entry === undefined) return undefined; + if (memberName === undefined) return entry.column; + return entry.members?.[memberName]?.column; +} + +describe("names artifact — flattened columns (cross-port naming corpus)", () => { + for (const c of corpus.cases) { + test(`${c.name}: every column in the corpus, flattened members included`, async () => { + const root = await loadCorpusModel(c); + const actual: Record = {}; + for (const key of Object.keys(c.columns)) actual[key] = artifactColumn(root, key); + expect(actual).toEqual(c.columns); + }); + } + + test("a flattened field has members and no column of its own", async () => { + const root = await loadCorpusModel(corpus.cases[0]!); + const names = resolveObjectNames(root.findObject("Shipment") as MetaObject); + expect(names?.fields.shipTo?.column).toBeUndefined(); + expect(Object.keys(names?.fields.shipTo?.members ?? {}).sort()).toEqual(["country", "postalCode", "street"]); + expect(names?.fields.id?.members).toBeUndefined(); + }); +}); + +describe("names artifact — rendered flattened members", () => { + const DOC = { + "metadata.root": { + package: "acme", + children: [ + { "object.value": { name: "Address", children: [ + { "field.string": { name: "street" } }, + { "field.string": { name: "postalCode", "@column": "ZIP" } }, + ] } }, + { "object.entity": { name: "Shipment", children: [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { name: "id" } }, + { "field.object": { name: "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "field.object": { name: "payload", "@objectRef": "Address" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + ], + }, + }; + + test("members, each with its column, and no column for the field; a jsonb object is unchanged", async () => { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toEqual([]); + const out = renderNamesDecl(root.findObject("Shipment") as MetaObject, { strategy: "snake_case" }); + + expect(out).toContain([ + " shipTo: {", + ' name: "shipTo",', + ' members: { postalCode: { name: "postalCode", column: "ship_to_ZIP" }, street: { name: "street", column: "ship_to_street" } },', + " },", + ].join("\n")); + expect(out).toContain('payload: { name: "payload", column: "payload" }'); + }); +}); + +describe("names artifact — a flattened field's column does not exist", () => { + /** Strict-mode errors for in-memory files. */ + function diagnose(files: Record): string[] { + const names = Object.keys(files); + const options: ts.CompilerOptions = { strict: true, noEmit: true, target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext, moduleResolution: ts.ModuleResolutionKind.Bundler }; + const svc = ts.createLanguageService({ + getScriptFileNames: () => names, + getScriptVersion: () => "1", + getScriptSnapshot: (f) => { + const text = files[f] ?? (ts.sys.fileExists(f) ? ts.sys.readFile(f) : undefined); + return text === undefined ? undefined : ts.ScriptSnapshot.fromString(text); + }, + getCurrentDirectory: () => "/", + getCompilationSettings: () => options, + getDefaultLibFileName: ts.getDefaultLibFilePath, + fileExists: (f) => f in files || ts.sys.fileExists(f), + readFile: (f) => files[f] ?? ts.sys.readFile(f), + }, ts.createDocumentRegistry()); + const errors: string[] = []; + for (const f of names) { + for (const d of [...svc.getSyntacticDiagnostics(f), ...svc.getSemanticDiagnostics(f)]) { + const text = files[f]!; + const lineStart = text.lastIndexOf("\n", d.start!) + 1; + const lineEnd = text.indexOf("\n", d.start!); + errors.push(`${f}: ${text.slice(lineStart, lineEnd === -1 ? undefined : lineEnd)}`); + } + } + return errors; + } + + const DOC = { + "metadata.root": { + package: "test", + children: [ + { "object.value": { name: "Address", children: [{ "field.string": { name: "street" } }] } }, + { "object.entity": { name: "Shipment", children: [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { name: "id" } }, + { "field.object": { name: "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + ], + }, + }; + + test("reading fields..column fails to compile; a member column and a plain column compile", async () => { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toEqual([]); + const artifact = renderNamesDecl(root.findObject("Shipment") as MetaObject, { strategy: "snake_case" }); + expect(diagnose({ "/Shipment.names.ts": artifact })).toEqual([]); + + const use = [ + 'import { ShipmentNames } from "./Shipment.names";', + "export const own = ShipmentNames.fields.shipTo.column;", + "export const member = ShipmentNames.fields.shipTo.members.street.column;", + "export const id = ShipmentNames.fields.id.column;", + "", + ].join("\n"); + expect(diagnose({ "/Shipment.names.ts": artifact, "/use.ts": use })) + .toEqual(["/use.ts: export const own = ShipmentNames.fields.shipTo.column;"]); + }); +}); + +describe("names artifact — a projection over a flattened field", () => { + const DOC = { + "metadata.root": { + package: "acme", + children: [ + { "object.value": { name: "Address", children: [{ "field.string": { name: "street" } }] } }, + { "object.entity": { name: "Shipment", children: [ + { "source.rdb": { "@table": "shipments" } }, + { "field.long": { name: "id" } }, + { "field.object": { name: "shipTo", "@objectRef": "Address", "@storage": "flattened" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + { "object.projection": { name: "ShipmentView", children: [ + { "source.rdb": { "@kind": "view", "@view": "v_shipments" } }, + { "field.long": { name: "id", extends: "Shipment.id" } }, + { "field.object": { name: "shipTo", extends: "Shipment.shipTo" } }, + { "identity.primary": { name: "pk", extends: "Shipment.pk" } }, + ] } }, + ], + }, + }; + + test("the flattened field's dbCol is a literal, never names.fields..column (which does not exist)", async () => { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toEqual([]); + const names = { resolved: resolveObjectNames(root.findObject("Shipment") as MetaObject)!, symbol: code`ShipmentNames` }; + const out = renderProjectionDecl(root.findObject("ShipmentView") as MetaObject, root, { + columnNamingStrategy: "snake_case", dialect: "postgres", names, + }); + expect(out).toContain("ShipmentNames.fields.id.column"); + expect(out).not.toContain("ShipmentNames.fields.shipTo.column"); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/output-parser-subtype-coverage.test.ts b/server/typescript/packages/codegen-ts/test/output-parser-subtype-coverage.test.ts index 4894f22c5d..f4eca06e52 100644 --- a/server/typescript/packages/codegen-ts/test/output-parser-subtype-coverage.test.ts +++ b/server/typescript/packages/codegen-ts/test/output-parser-subtype-coverage.test.ts @@ -59,6 +59,7 @@ const WIDE = [ { "field.currency": { name: "cur", "@required": true, "@currency": "USD" } }, { "field.uri": { name: "url", "@required": true } }, { "field.inet": { name: "ip", "@required": true } }, + { "field.duration": { name: "wait", "@required": true } }, ], }, }, @@ -94,6 +95,8 @@ describe("A5 — strict response parser subtype coverage", () => { // Temporal values are ISO strings in raw JSON, the same reasoning zod-validators.ts // records for z.coerce.date(). expect(src).toContain("ts: z.string()"); + // A duration is an ISO-8601 string in the accepted form — never a calendar span. + expect(src).toContain("wait: z.string().regex(/^[-+]?P(?!$)(\\d+W)?(\\d+D)?"); }); test("an enum inheriting its @values through extends still resolves (ADR-0039 resolving)", async () => { @@ -142,7 +145,7 @@ describe("A5 — strict response parser subtype coverage", () => { s: "x", i: 1, l: 2, dbl: 1.5, b: true, e: "A", u: "3f2504e0-4f89-11d3-9a0c-0305e82c3301", d: "2026-01-01", t: "10:00:00", ts: "2026-01-01T10:00:00Z", - dec: "1.25", cur: 199, url: "https://example.test", ip: "10.0.0.1", + dec: "1.25", cur: 199, url: "https://example.test", ip: "10.0.0.1", wait: "PT1H30M", }; expect(schema.safeParse(ok).success).toBe(true); @@ -152,5 +155,7 @@ describe("A5 — strict response parser subtype coverage", () => { expect(schema.safeParse({ ...ok, e: null }).success).toBe(false); // And a float where integer minor units are required. expect(schema.safeParse({ ...ok, cur: 1.99 }).success).toBe(false); + // A calendar span is not a duration. + expect(schema.safeParse({ ...ok, wait: "P1M" }).success).toBe(false); }); }); diff --git a/server/typescript/packages/codegen-ts/test/prisma/prisma-schema.test.ts b/server/typescript/packages/codegen-ts/test/prisma/prisma-schema.test.ts index 9f71701c83..d125a43561 100644 --- a/server/typescript/packages/codegen-ts/test/prisma/prisma-schema.test.ts +++ b/server/typescript/packages/codegen-ts/test/prisma/prisma-schema.test.ts @@ -12,6 +12,7 @@ import { InMemoryStringSource, MetaDataLoader, type MetaRoot } from "@metaobject import { defineConfig, runGen, type Generator } from "../../src/index.js"; import { prismaSchemaFile, type PrismaSchemaFileOptions } from "../../src/generators/prisma-schema-file.js"; import { buildPrismaSchema } from "../../src/prisma/build-prisma-schema.js"; +import { efcoreConventions } from "@metaobjectsdev/migrate-ts"; import { ERR_PRISMA_NAME_COLLISION, ERR_PRISMA_READ_ONLY_SOURCE, @@ -125,6 +126,34 @@ describe("prisma-schema — what the schema says", () => { expect(spec.checks.map((c) => c.name)).toContain("customers_tier_chk"); }); + test("a string-stored enum with a length is a VarChar of it, on Postgres and MySQL", async () => { + const root = await loadJson(model([entity("Ticket", [ + { "source.rdb": { "@table": "tickets" } }, longId, + { "field.enum": { name: "status", "@values": ["OPEN", "CLOSED"], "@maxLength": 16 } }, pk, + ])])); + for (const provider of ["postgresql", "mysql"] as const) { + const status = buildPrismaSchema(root, { provider }).models.find((m) => m.name === "Ticket")! + .fields.find((f) => f.name === "status")!; + expect(status.attributes).toContain("@db.VarChar(16)"); + } + }); + + test("a field.duration: Unsupported(\"interval\") on Postgres, BigInt microseconds on MySQL and SQLite", async () => { + const root = await loadJson(model([entity("Hold", [ + { "source.rdb": { "@table": "holds" } }, longId, + { "field.duration": { name: "holdFor", "@required": true } }, + { "field.duration": { name: "laps", isArray: true } }, pk, + ])])); + const fieldsOf = (provider: "postgresql" | "mysql" | "sqlite") => + buildPrismaSchema(root, { provider }).models.find((m) => m.name === "Hold")!.fields; + const pg = fieldsOf("postgresql"); + expect(pg.find((f) => f.name === "holdFor")).toMatchObject({ type: 'Unsupported("interval")', list: false }); + expect(pg.find((f) => f.name === "laps")).toMatchObject({ type: 'Unsupported("interval[]")', list: false }); + // SQLite and MySQL store a 64-bit integer count of microseconds. + expect(fieldsOf("mysql").find((f) => f.name === "holdFor")).toMatchObject({ type: "BigInt", list: false }); + expect(fieldsOf("sqlite").find((f) => f.name === "holdFor")).toMatchObject({ type: "BigInt", list: false }); + }); + test("a self relation is named, so its two ends pair", async () => { const spec = buildPrismaSchema(await loadShop(), { provider: "postgresql" }); const employee = spec.models.find((m) => m.name === "Employee")!; @@ -134,6 +163,27 @@ describe("prisma-schema — what the schema says", () => { expect(back).toMatchObject({ type: "Employee", list: true, attributes: ['@relation("employees_manager_id_fk")'] }); }); + test("the project's schema conventions name the keys, as they do for meta migrate", async () => { + const root = await loadShop(); + const managerOf = (spec: ReturnType) => + spec.models.find((m) => m.name === "Employee")!.fields.find((f) => f.name === "manager")!.attributes[0]; + expect(managerOf(buildPrismaSchema(root, { provider: "postgresql" }))).toStartWith('@relation("employees_manager_id_fk", '); + expect(managerOf(buildPrismaSchema(root, { provider: "postgresql", schemaSettings: { conventions: efcoreConventions } }))) + .toStartWith('@relation("FK_employees_employees_manager_id", '); + }); + + test("the generator reads the settings from the run (GenContext.schemaSettings)", async () => { + await runGen({ + config: defineConfig({ outDir: tmp, dialect: "postgres", generators: [prismaSchemaFile({ provider: "postgresql" })] }), + metadata: await loadShop(), + projectRoot: tmp, + schemaSettings: { conventions: efcoreConventions }, + }); + const schema = readFileSync(join(tmp, "schema.prisma"), "utf8"); + expect(schema).toContain('@relation("FK_employees_employees_manager_id"'); + expect(schema).not.toContain("employees_manager_id_fk"); + }); + test("a one-to-one reference gets an optional singular back relation", async () => { const root = await loadJson(model([ entity("User", [{ "source.rdb": {} }, longId, pk]), @@ -201,6 +251,23 @@ describe("prisma-schema — what the schema says", () => { expect(pgPostal.attributes).toContain("@db.VarChar(10)"); }); + test("a flattened field.object's @flattenedColumnPrefix names its columns, as meta migrate creates them", async () => { + const root = await loadJson(model([ + { "object.value": { name: "Address", children: [{ "field.string": { name: "street" } }] } }, + entity("Customer", [ + { "source.rdb": {} }, + longId, + { "field.object": { name: "homeAddress", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "" } }, + { "field.object": { name: "workAddress", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "office_" } }, + pk, + ]), + ])); + const fields = buildPrismaSchema(root, { provider: "postgresql" }).models.find((m) => m.name === "Customer")!.fields; + expect(fields.map((f) => f.name)).toEqual(["id", "homeAddressStreet", "workAddressStreet"]); + expect(fields[1]!.attributes).toContain('@map("street")'); + expect(fields[2]!.attributes).toContain('@map("office_street")'); + }); + test("a stored procedure is not a model, and not refused: meta migrate does not create it either", async () => { const root = await loadJson(model([ entity("Fresh", [{ "source.rdb": {} }, longId, pk]), diff --git a/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts b/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts index 438403a266..6ee797c377 100644 --- a/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts +++ b/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts @@ -208,7 +208,7 @@ describe("sharedModelFile()", () => { schema_version: 1, name: "acme-common", version: "1.0.0", - metamodelVersion: "1.2", + metamodelVersion: "1.3", artifact: "acme-common.metaobjects.json", integrity: PINNED_HASH, packages: ["acme::common"], diff --git a/server/typescript/packages/codegen-ts/test/templates/filter-allowlist.test.ts b/server/typescript/packages/codegen-ts/test/templates/filter-allowlist.test.ts index 055a278a1a..c12514556a 100644 --- a/server/typescript/packages/codegen-ts/test/templates/filter-allowlist.test.ts +++ b/server/typescript/packages/codegen-ts/test/templates/filter-allowlist.test.ts @@ -31,6 +31,23 @@ describe("renderFilterAllowlist", () => { expect(out).toMatch(/createdAt:\s*\{[^}]*"gte"/); }); + test("a field.duration carries the ordered band and its wire format, and is sortable", async () => { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify({ + "metadata.root": { package: "t", children: [{ "object.entity": { name: "Hold", children: [ + { "source.rdb": { "@table": "hold" } }, + { "field.long": { name: "id" } }, + { "field.duration": { name: "holdFor", "@filterable": true } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }] }, + }))]); + expect(errors).toEqual([]); + const hold = root.objects().find((c) => c.name === "Hold")!; + const out = renderFilterAllowlist(hold).toString(); + expect(out).toMatch(/holdFor:\s*\{[^}]*ops: \["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"\]/); + expect(out).toMatch(/holdFor:\s*\{[^}]*format: "duration"/); + expect(renderSortAllowlist(hold).toString()).toContain("holdFor"); + }); + test("only @filterable fields appear in allowlist", async () => { const entity = await loadEntity("Subscriber"); const out = renderFilterAllowlist(entity).toString(); diff --git a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts index 5b237c0213..0082254cd4 100644 --- a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts +++ b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts @@ -1,7 +1,7 @@ import { describe, test, expect } from "bun:test"; import { TypeId, TYPE_IDENTITY, TYPE_VALIDATOR, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, - FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_SUBTYPE_DURATION, DURATION_PATTERN, FIELD_ATTR_STRING_FORMAT, STRING_FORMAT_EMAIL, STRING_FORMAT_HOSTNAME, IDENTITY_SUBTYPE_PRIMARY, OBJECT_SUBTYPE_ENTITY, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_LENGTH, @@ -391,6 +391,22 @@ describe("renderZodValidators", () => { expect(compact).toContain("z.string().regex(/^(?:(?:25[0-5]"); // the IPv4 octet regex }); + test("field.duration → the accepted-form regex, canonicalized by the module-local durationWrite (declared once)", () => { + const hold = metaObject(OBJECT_SUBTYPE_ENTITY, "Hold"); + hold.addChild(metaField(FIELD_SUBTYPE_LONG, "id")); + hold.addChild(metaField(FIELD_SUBTYPE_DURATION, "holdFor")); + hold.addChild(metaField(FIELD_SUBTYPE_DURATION, "grace")); + const primary = meta(new TypeId(TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY), "primary"); + primary.setAttr("fields", ["id"]); + primary.setAttr("generation", "increment"); + hold.addChild(primary); + + const out = renderZodValidators(hold).toString(); + // Whitespace and the formatter's trailing comma (`transform(durationWrite,)` on a wrapped line) aside. + expect(out.replace(/\s+/g, "").replace(/,\)/g, ")")).toContain(`holdFor:z.string().regex(/${DURATION_PATTERN}/).transform(durationWrite).optional()`); + expect(out.split("function durationWrite(").length - 1).toBe(1); + }); + // #234 — @lenient degrades field.uri / field.inet to a plain string (no validator). test("field.uri @lenient / field.inet @lenient → plain z.string() (no url/ip validator)", () => { const ep = metaObject(OBJECT_SUBTYPE_ENTITY, "Endpoint"); diff --git a/server/typescript/packages/codegen-ts/test/wire-format-e2e.test.ts b/server/typescript/packages/codegen-ts/test/wire-format-e2e.test.ts index 5332cc8d53..ef82032db6 100644 --- a/server/typescript/packages/codegen-ts/test/wire-format-e2e.test.ts +++ b/server/typescript/packages/codegen-ts/test/wire-format-e2e.test.ts @@ -122,7 +122,7 @@ describe("wire.decimals", () => { }); test("string: the canonical form, exactly as 1.1.1", async () => { - const { app, sqlite, options } = await serverFor({ decimals: "string", nullMembers: "native" }); + const { app, sqlite, options } = await serverFor({ decimals: "string", nullMembers: "native", durations: "native" }); expect(options).toContain('decimalWire: "string"'); seed(sqlite); const row = await get(app); @@ -157,7 +157,7 @@ describe("wire.nullMembers", () => { }); test("omit: null members are dropped from a value object, an array of them and a map of them", async () => { - const { app, sqlite, options } = await serverFor({ decimals: "native", nullMembers: "omit" }); + const { app, sqlite, options } = await serverFor({ decimals: "native", nullMembers: "omit", durations: "native" }); expect(options).toContain('nullMembers: "omit"'); seed(sqlite); const row = await get(app); @@ -167,7 +167,7 @@ describe("wire.nullMembers", () => { }); test("emit: every declared member is present, null when unset", async () => { - const { app, sqlite, options } = await serverFor({ decimals: "native", nullMembers: "emit" }); + const { app, sqlite, options } = await serverFor({ decimals: "native", nullMembers: "emit", durations: "native" }); expect(options).toContain('nullMembers: "emit"'); seed(sqlite); const row = await get(app); @@ -179,7 +179,7 @@ describe("wire.nullMembers", () => { test("a top-level null field is unchanged in every mode", async () => { for (const nullMembers of ["native", "emit", "omit"] as const) { - const { app, sqlite } = await serverFor({ decimals: "native", nullMembers }); + const { app, sqlite } = await serverFor({ decimals: "native", nullMembers, durations: "native" }); sqlite.run(`INSERT INTO entries (amount) VALUES (NULL)`); const row = await get(app); expect(row["amount"]).toBeNull(); diff --git a/server/typescript/packages/codegen-ts/test/zod-cross-field.test.ts b/server/typescript/packages/codegen-ts/test/zod-cross-field.test.ts index 90b9f643a8..20227b5d99 100644 --- a/server/typescript/packages/codegen-ts/test/zod-cross-field.test.ts +++ b/server/typescript/packages/codegen-ts/test/zod-cross-field.test.ts @@ -14,6 +14,7 @@ import { VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, + VALIDATOR_ATTR_CONDITION, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_DURATION, } from "@metaobjectsdev/metadata"; import type { AttrValue, MetaField, MetaObject } from "@metaobjectsdev/metadata"; import { meta, metaObject, metaField } from "./_meta-build.js"; @@ -117,6 +118,47 @@ describe("cross-field validators on the insert schema", () => { expect(mod.ArrInsertSchema.safeParse({ nums: [1], b: 0 }).success).toBe(true); }); + test("a rule left to the database declares no helper: nothing unused reaches the schema", async () => { + // Each rule below asks for a helper and is then dropped: its condition compares a date with a + // timestamp, which has no order. The one rule kept needs no helper. + const crossKind = { op: "gt", left: { field: "d" }, right: { field: "t" } }; + const e = entityWith("Kept", [ + f(FIELD_SUBTYPE_INT, "a"), f(FIELD_SUBTYPE_INT, "b"), f(FIELD_SUBTYPE_DATE, "d"), + f(FIELD_SUBTYPE_TIMESTAMP, "t"), f(FIELD_SUBTYPE_DURATION, "p"), f(FIELD_SUBTYPE_DURATION, "q"), + f(FIELD_SUBTYPE_STRING, "kind"), f(FIELD_SUBTYPE_STRING, "note"), + ], [ + validator(VALIDATOR_SUBTYPE_AT_LEAST_ONE, "and", { + [VALIDATOR_ATTR_FIELDS]: ["note"], + [VALIDATOR_ATTR_CONDITION]: { op: "and", args: [{ op: "gt", left: { field: "a" }, right: { field: "b" } }, crossKind] }, + }), + validator(VALIDATOR_SUBTYPE_COMPARISON, "span", { + [VALIDATOR_ATTR_LEFT]: "p", [VALIDATOR_ATTR_OP]: "lte", [VALIDATOR_ATTR_RIGHT]: "q", [VALIDATOR_ATTR_CONDITION]: crossKind, + }), + validator(VALIDATOR_SUBTYPE_REQUIRED_WHEN, "rw", { [VALIDATOR_ATTR_FIELD]: "note", [VALIDATOR_ATTR_WHEN]: "kind", [VALIDATOR_ATTR_EQUALS]: "X" }), + ]); + const out = renderZodValidators(e).toString(); + expect(out).toContain(".superRefine("); + expect(out).not.toContain("const cmp = "); + expect(out).not.toContain("const all = "); + expect(out).not.toContain("const micros = (d"); + }); + + test("two durations compare by length, and text that is no duration is unchecked like a null side", async () => { + const e = entityWith("Span", [f(FIELD_SUBTYPE_DURATION, "p"), f(FIELD_SUBTYPE_DURATION, "q")], [ + validator(VALIDATOR_SUBTYPE_COMPARISON, "span", { [VALIDATOR_ATTR_LEFT]: "p", [VALIDATOR_ATTR_OP]: "lte", [VALIDATOR_ATTR_RIGHT]: "q" }), + ]); + const out = renderZodValidators(e).toString(); + // The refinement's parser accepts only what the field's own schema accepts: no bare `P`, `PT` + // or `P1DT`, so text that is no duration reads as null (unchecked) rather than as a zero span. + const parser = out.slice(out.indexOf("const micros = (d")); + expect(parser).toContain("P(?!$)"); + expect(parser).toContain("T(?=\\d)"); + expect(parser).not.toContain("?? 0) - (micros("); + const mod = await executeGenerated(out); + expect(mod.SpanInsertSchema.safeParse({ p: "PT1H30M", q: "PT1H" }).success).toBe(false); + expect(mod.SpanInsertSchema.safeParse({ p: "PT45M", q: "PT1H" }).success).toBe(true); + }); + test("numeric @equals compares numerically against the gate field", async () => { const e = entityWith("Lv", [f(FIELD_SUBTYPE_INT, "level"), f(FIELD_SUBTYPE_STRING, "note")], [ validator(VALIDATOR_SUBTYPE_REQUIRED_WHEN, "rw", { [VALIDATOR_ATTR_FIELD]: "note", [VALIDATOR_ATTR_WHEN]: "level", [VALIDATOR_ATTR_EQUALS]: "3" }), diff --git a/server/typescript/packages/codegen-ts/test/zod-validators-execution.test.ts b/server/typescript/packages/codegen-ts/test/zod-validators-execution.test.ts index 8446301d77..e5fc9d66ef 100644 --- a/server/typescript/packages/codegen-ts/test/zod-validators-execution.test.ts +++ b/server/typescript/packages/codegen-ts/test/zod-validators-execution.test.ts @@ -37,7 +37,7 @@ import { TypeId, TYPE_IDENTITY, TYPE_VALIDATOR, IDENTITY_SUBTYPE_PRIMARY, OBJECT_SUBTYPE_ENTITY, OBJECT_SUBTYPE_VALUE, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_INT, FIELD_SUBTYPE_STRING, - FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_ATTR_REQUIRED, FIELD_ATTR_DEFAULT, FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_VALUES, FIELD_ATTR_LENIENT, FIELD_ATTR_STRING_FORMAT, FIELD_ATTR_LOCAL_TIME, @@ -254,6 +254,35 @@ describe("field.inet — the 0.20.6 regression class, executed", () => { }); }); +describe("field.duration — the accepted ISO-8601 form, stored canonically", () => { + test("the insert schema accepts an ISO-8601 duration and answers its canonical form", async () => { + const holdFor = metaField(FIELD_SUBTYPE_DURATION, "holdFor"); + holdFor.setAttr(FIELD_ATTR_REQUIRED, true); + const schema = await insertSchemaOf(entityWith("Hold", holdFor)); + + expect(schema.parse({ holdFor: "PT90M" }).holdFor).toBe("PT1H30M"); + expect(schema.parse({ holdFor: "P1DT2H" }).holdFor).toBe("PT26H"); + expect(schema.parse({ holdFor: "-PT90S" }).holdFor).toBe("-PT1M30S"); + expect(schema.parse({ holdFor: "PT1.123456789S" }).holdFor).toBe("PT1.123456S"); + // A week is exactly 7 days, before the days. + expect(schema.parse({ holdFor: "P1W1DT1H" }).holdFor).toBe("PT193H"); + // No cross-port bound: the helper counts in bigint, so a span past any 64-bit range is exact. + expect(schema.parse({ holdFor: "PT256204779H" }).holdFor).toBe("PT256204779H"); + for (const bad of ["P1M", "P1Y", "P1D1W", "1:30:00", "PT1.5H", "pt1h", "PT", ""]) { + expect(accepts(schema, { holdFor: bad })).toBe(false); + } + }); + + test("the update schema canonicalizes too, and an optional duration may be null there", async () => { + const holdFor = metaField(FIELD_SUBTYPE_DURATION, "holdFor"); + const schema = await updateSchemaOf(entityWith("Hold", holdFor)); + + expect(schema.parse({ holdFor: "PT5400S" }).holdFor).toBe("PT1H30M"); + expect(schema.parse({ holdFor: null }).holdFor).toBeNull(); + expect(accepts(schema, { holdFor: "P1M" })).toBe(false); + }); +}); + describe("field.uri — absolute, scheme-bearing only", () => { test("accepts absolute URIs (incl. urn:/mailto:) and rejects relative ones", async () => { const link = metaField(FIELD_SUBTYPE_URI, "link"); diff --git a/server/typescript/packages/docs-site/src/builders/object-data.ts b/server/typescript/packages/docs-site/src/builders/object-data.ts index 2181a0dd51..0ebe0d6b45 100644 --- a/server/typescript/packages/docs-site/src/builders/object-data.ts +++ b/server/typescript/packages/docs-site/src/builders/object-data.ts @@ -1,4 +1,8 @@ import type { MetaData } from "@metaobjectsdev/metadata"; +import { + EXPR_FN_COALESCE, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, + FILTER_OP_EQ, FILTER_OP_GT, FILTER_OP_GTE, FILTER_OP_IS_NULL, FILTER_OP_LT, FILTER_OP_LTE, FILTER_OP_NE, +} from "@metaobjectsdev/metadata"; import { LinkGraph, fqnOf, type Ref } from "../link-graph.js"; import type { CoverageTracker } from "../coverage.js"; import { esc, badge } from "../badges.js"; @@ -253,8 +257,14 @@ export function buildObjectPage(fqn: string, g: LinkGraph, cov: CoverageTracker) if (v.subType === "comparison") rule = `${esc(String(at("left") ?? ""))} ${esc(String(at("op") ?? ""))} ${esc(String(at("right") ?? ""))}`; else if (v.subType === "requiredWhen") rule = `${esc(String(at("field") ?? ""))} required when ${esc(String(at("when") ?? ""))} = ${esc(String(at("equals") ?? ""))}`; else if (v.subType === "presentIff") rule = `${esc(String(at("field") ?? ""))} present iff ${esc(String(at("when") ?? ""))} = ${esc(String(at("equals") ?? ""))}`; - else if (v.subType === "atLeastOne") { const fs = at("fields"); rule = `at least one of ${esc(Array.isArray(fs) ? fs.map(String).join(", ") : String(fs ?? ""))}`; } + else if (v.subType === "atLeastOne" || v.subType === "exactlyOne") { + const fs = at("fields"); + const list = esc(Array.isArray(fs) ? fs.map(String).join(", ") : String(fs ?? "")); + rule = `${v.subType === "exactlyOne" ? "exactly one" : "at least one"} of ${list}`; + } else rule = esc(v.subType); + const condition = at("condition"); + if (condition !== undefined && rule !== esc(v.subType)) rule += `, when ${esc(expressionText(condition))}`; validators.push({ scope: "object", subject: v.name || v.subType, rule }); } validators.sort((a, b) => a.subject.localeCompare(b.subject)); @@ -366,3 +376,23 @@ export function buildObjectPage(fqn: string, g: LinkGraph, cov: CoverageTracker) report, reporting, }; } + +const EXPR_TEXT_OP: Readonly> = { + [FILTER_OP_EQ]: "=", [FILTER_OP_NE]: "≠", [FILTER_OP_GT]: ">", [FILTER_OP_GTE]: "≥", [FILTER_OP_LT]: "<", [FILTER_OP_LTE]: "≤", +}; + +/** An attr.expression as one line of reading text (a cross-field validator's @condition). */ +function expressionText(node: unknown): string { + if (typeof node !== "object" || node === null) return String(node); + const n = node as Record; + if (typeof n["field"] === "string") return n["field"]; + if ("value" in n) return JSON.stringify(n["value"]); + const op = n["op"]; + if (typeof op === "string" && EXPR_TEXT_OP[op] !== undefined) return `${expressionText(n["left"])} ${EXPR_TEXT_OP[op]} ${expressionText(n["right"])}`; + if (op === FILTER_OP_IS_NULL) return `${expressionText(n["arg"])} is empty`; + if (op === EXPR_OP_IS_NOT_NULL) return `${expressionText(n["arg"])} is set`; + if (op === EXPR_OP_NOT) return `not (${expressionText(n["arg"])})`; + if ((op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) && Array.isArray(n["args"])) return n["args"].map((a) => `(${expressionText(a)})`).join(` ${op} `); + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) return `coalesce(${n["args"].map(expressionText).join(", ")})`; + return JSON.stringify(node); +} diff --git a/server/typescript/packages/integration-tests/package.json b/server/typescript/packages/integration-tests/package.json index 122ce91c74..0a0a53eadf 100644 --- a/server/typescript/packages/integration-tests/package.json +++ b/server/typescript/packages/integration-tests/package.json @@ -10,6 +10,7 @@ "gen:schema": "bun run src/gen-canonical-schema.ts", "gen:report-api-schema": "bun run src/gen-api-contract-report-schema.ts", "gen:decimal-api-schema": "bun run src/gen-api-contract-decimal-schema.ts", + "gen:duration-api-schema": "bun run src/gen-api-contract-duration-schema.ts", "gen:report-shapes": "bun run src/gen-report-shapes.ts", "oracle": "bun run src/oracle-cli.ts" }, diff --git a/server/typescript/packages/integration-tests/src/api-contract-decimal-server.ts b/server/typescript/packages/integration-tests/src/api-contract-decimal-server.ts index 50d42e322d..87b827af60 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-decimal-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-decimal-server.ts @@ -30,12 +30,22 @@ import { import { splitStatements } from "./sql-script.ts"; export type DecimalEngine = "postgres" | "mysql" | "sqlite" | "d1"; +/** The engines a generated-lane sub-corpus served by this module runs on. */ +export type CorpusEngine = DecimalEngine; -/** The served objects: the emitted routes file and the route the corpus calls. */ -const SERVED = ["Entry", "EntryTotals", "Sample"] as const; - -/** The tables the seed fills (and resets), in the order they are emptied. */ -const TABLES = ["entries", "samples"] as const; +/** What differs between the sub-corpora this module serves (decimal/, duration/). */ +export interface GeneratedCorpusSpec { + /** A short name for temp directories and globals. */ + name: string; + /** The served objects: each has an emitted .routes.ts the server mounts. */ + served: readonly string[]; + /** The tables the seed fills (and resets), in the order they are emptied. */ + tables: readonly string[]; + /** Views a previous server on a shared MySQL database may have left behind. */ + mysqlViews: readonly string[]; + /** The committed schema artifact of each engine. */ + schemaPath: Record; +} export interface DecimalServerHandle { baseUrl: string; @@ -51,23 +61,42 @@ interface EngineClient { close(): Promise; } -const SCHEMA_PATH: Record = { - postgres: API_CONTRACT_DECIMAL_SCHEMA_SQL_PATH, - mysql: API_CONTRACT_DECIMAL_MYSQL_SCHEMA_SQL_PATH, - sqlite: API_CONTRACT_DECIMAL_SQLITE_SCHEMA_SQL_PATH, - d1: API_CONTRACT_DECIMAL_SQLITE_SCHEMA_SQL_PATH, +const DECIMAL: GeneratedCorpusSpec = { + name: "decimal", + served: ["Entry", "EntryTotals", "Sample"], + tables: ["entries", "samples"], + mysqlViews: ["v_entry_totals"], + schemaPath: { + postgres: API_CONTRACT_DECIMAL_SCHEMA_SQL_PATH, + mysql: API_CONTRACT_DECIMAL_MYSQL_SCHEMA_SQL_PATH, + sqlite: API_CONTRACT_DECIMAL_SQLITE_SCHEMA_SQL_PATH, + d1: API_CONTRACT_DECIMAL_SQLITE_SCHEMA_SQL_PATH, + }, }; -export async function startDecimalServer( +export function startDecimalServer( engine: DecimalEngine, metaPath: string, /** The Postgres or MySQL connection URL; unused by SQLite and D1. */ connectionUrl?: string, ): Promise { + return startGeneratedCorpusServer(DECIMAL, engine, metaPath, connectionUrl); +} + +/** Generate, provision and mount one generated-lane sub-corpus on `engine`. */ +export async function startGeneratedCorpusServer( + spec: GeneratedCorpusSpec, + engine: CorpusEngine, + metaPath: string, + /** The Postgres or MySQL connection URL; unused by SQLite and D1. */ + connectionUrl?: string, +): Promise { + const SERVED = spec.served; + const TABLES = spec.tables; const here = dirname(fileURLToPath(import.meta.url)); const genTmpRoot = join(here, "..", ".gen-tmp"); mkdirSync(genTmpRoot, { recursive: true }); - const tmp = mkdtempSync(join(genTmpRoot, `api-contract-decimal-${engine}-`)); + const tmp = mkdtempSync(join(genTmpRoot, `api-contract-${spec.name}-${engine}-`)); const fail = (message: string): never => { rmSync(tmp, { recursive: true, force: true }); throw new Error(message); @@ -90,8 +119,8 @@ export async function startDecimalServer( for (const name of SERVED) if (!existsSync(join(tmp, `${name}.routes.ts`))) fail(`codegen emitted no ${name}.routes.ts`); if (lr.warnings.length > 0) fail(`codegen produced warnings: ${lr.warnings.join("; ")}`); - const client = await openEngine(engine, tmp, connectionUrl); - for (const stmt of splitStatements(readFileSync(SCHEMA_PATH[engine], "utf8"))) await client.run(stmt); + const client = await openEngine(spec, engine, tmp, connectionUrl); + for (const stmt of splitStatements(readFileSync(spec.schemaPath[engine], "utf8"))) await client.run(stmt); const fastify = Fastify(); for (const name of SERVED) { @@ -128,7 +157,8 @@ export async function startDecimalServer( } /** Write the `db` module the emitted routes import and open the engine behind it. */ -async function openEngine(engine: DecimalEngine, tmp: string, url: string | undefined): Promise { +async function openEngine(spec: GeneratedCorpusSpec, engine: CorpusEngine, tmp: string, url: string | undefined): Promise { + const TABLES = spec.tables; if (engine === "postgres") { if (url === undefined) throw new Error("postgres needs a connection URL"); const bigintTypes = pathToFileURL(join(dirname(fileURLToPath(import.meta.url)), "pg-bigint-number-types.ts")).href; @@ -166,17 +196,17 @@ export const db = drizzle(pool); const mod = (await import(pathToFileURL(join(tmp, "db.ts")).href)) as { pool: { end(cb: () => void): void } }; const admin = await mysql.createConnection({ uri: url, timezone: "Z" }); // A previous server on this database may have left the objects behind. - await admin.query("DROP VIEW IF EXISTS `v_entry_totals`"); - await admin.query("DROP TABLE IF EXISTS `entries`"); - await admin.query("DROP TABLE IF EXISTS `samples`"); + const dropAll = async () => { + for (const view of spec.mysqlViews) await admin.query(`DROP VIEW IF EXISTS \`${view}\``); + for (const table of TABLES) await admin.query(`DROP TABLE IF EXISTS \`${table}\``); + }; + await dropAll(); return { run: async (sql, params = []) => { await admin.query(sql, params); }, afterSeed: async () => {}, close: async () => { await new Promise((done) => mod.pool.end(done)); - await admin.query("DROP VIEW IF EXISTS `v_entry_totals`"); - await admin.query("DROP TABLE IF EXISTS `entries`"); - await admin.query("DROP TABLE IF EXISTS `samples`"); + await dropAll(); await admin.end(); }, }; @@ -184,7 +214,7 @@ export const db = drizzle(pool); if (engine === "d1") { const local = await startLocalD1(); // One key per server, so two servers in flight never read each other's binding. - const key = `__decimalD1_${randomUUID().replaceAll("-", "")}`; + const key = `__${spec.name}D1_${randomUUID().replaceAll("-", "")}`; (globalThis as Record)[key] = local.d1; writeFileSync(join(tmp, "db.ts"), ` import { drizzle } from "drizzle-orm/d1"; diff --git a/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts b/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts index 47ed26875f..85e0896ed4 100644 --- a/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts +++ b/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts @@ -118,6 +118,7 @@ const TABLES = [ \`uriVal\` VARCHAR(255) NOT NULL, \`inetVal\` VARCHAR(45) NOT NULL, \`inet6Val\` VARCHAR(45) NOT NULL, + \`durationVal\` BIGINT, \`settings\` JSON, \`labels\` JSON )`, diff --git a/server/typescript/packages/integration-tests/src/gen-api-contract-duration-schema.ts b/server/typescript/packages/integration-tests/src/gen-api-contract-duration-schema.ts new file mode 100644 index 0000000000..9bde2117ac --- /dev/null +++ b/server/typescript/packages/integration-tests/src/gen-api-contract-duration-schema.ts @@ -0,0 +1,56 @@ +// gen-api-contract-duration-schema.ts — (re)generate the committed duration sub-corpus schemas. +// +// Run: `bun run gen:duration-api-schema` (from this package). Pure metadata→SQL — no DB. +// +// Postgres and SQLite: `meta migrate`'s own output. MySQL: the adopter's hand-written table +// (ADR-0015), a duration held as a BIGINT count of microseconds as on SQLite. + +import { join } from "node:path"; + +import type { MetaRoot } from "@metaobjectsdev/metadata"; +import { generateCanonicalSchemaSql, writeArtifact } from "./canonical-schema.ts"; +import { renderMysqlSchemaArtifact } from "./canonical-schema-mysql.ts"; +import { loadMetadataFile } from "./load-metadata.ts"; +import { + API_CONTRACT_DURATION_DIR, + API_CONTRACT_DURATION_MYSQL_SCHEMA_SQL_PATH, + API_CONTRACT_DURATION_SCHEMA_SQL_PATH, + API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH, +} from "./paths.ts"; + +type Dialect = "postgres" | "sqlite" | "mysql"; +const LABEL: Record = { postgres: "Postgres", sqlite: "SQLite", mysql: "MySQL 8.4" }; + +function header(dialect: Dialect): string { + return [ + "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", + `-- api-contract duration sub-corpus schema (${LABEL[dialect]}, literal column naming).`, + "-- Regenerate: `bun run gen:duration-api-schema` in server/typescript/packages/integration-tests.", + ].join("\n"); +} + +const MYSQL_TABLES = [ + `CREATE TABLE \`timers\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`label\` VARCHAR(16) NOT NULL, + \`timeout\` BIGINT +)`, +]; + +async function schemaSql(root: MetaRoot, dialect: Dialect): Promise { + if (dialect === "mysql") return renderMysqlSchemaArtifact(header(dialect), MYSQL_TABLES, root); + return generateCanonicalSchemaSql(root, { header: header(dialect), dialect }); +} + +async function main(): Promise { + const root = await loadMetadataFile(join(API_CONTRACT_DURATION_DIR, "meta.json")); + writeArtifact(API_CONTRACT_DURATION_SCHEMA_SQL_PATH, await schemaSql(root, "postgres")); + writeArtifact(API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH, await schemaSql(root, "sqlite")); + writeArtifact(API_CONTRACT_DURATION_MYSQL_SCHEMA_SQL_PATH, await schemaSql(root, "mysql")); +} + +main().catch((err: unknown) => { + // eslint-disable-next-line no-console + console.error(err); + process.exit(1); +}); diff --git a/server/typescript/packages/integration-tests/src/load-metadata.ts b/server/typescript/packages/integration-tests/src/load-metadata.ts index cb4d6c4ecb..488602d90b 100644 --- a/server/typescript/packages/integration-tests/src/load-metadata.ts +++ b/server/typescript/packages/integration-tests/src/load-metadata.ts @@ -6,10 +6,10 @@ import { existsSync, readFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { pathToFileURL } from "node:url"; import { DEFAULT_WIRE_FORMAT, loadDirectory, loadUris, type MetaRoot, type WireFormat } from "@metaobjectsdev/metadata"; -import { ConfigSchema, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "@metaobjectsdev/sdk"; +import { ConfigSchema, enumDefaultsFromConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "@metaobjectsdev/sdk"; /** - * ADR-0058 — set `root`'s physical naming and wire format from the nearest `.metaobjects/config.json` at or + * ADR-0058 — set `root`'s physical naming, wire format and enum defaults from the nearest `.metaobjects/config.json` at or * above `startDir`, as a project-aware load does. A corpus that pins a committed schema * declares the naming that schema uses there, and every port's runner reads that one file. * No file up to the repository root leaves the default. @@ -21,6 +21,8 @@ export function applyNearestNaming(root: MetaRoot, startDir: string): MetaRoot { // ADR-0019 Amendment 2: the corpus pins its wire form in the same file, so generated // routes are emitted with it and no lane carries a literal. root.wireFormat = wireFormatFromConfig(config); + // `enums.maxLength` sizes an enum column, so a corpus that sets it gets the schema it pins. + root.enumDefaults = enumDefaultsFromConfig(config); return root; } diff --git a/server/typescript/packages/integration-tests/src/paths.ts b/server/typescript/packages/integration-tests/src/paths.ts index 890784201c..f6f43572f8 100644 --- a/server/typescript/packages/integration-tests/src/paths.ts +++ b/server/typescript/packages/integration-tests/src/paths.ts @@ -65,3 +65,11 @@ export const API_CONTRACT_DECIMAL_SCENARIOS_DIR = resolve(API_CONTRACT_DECIMAL_D export const API_CONTRACT_DECIMAL_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DECIMAL_DIR, "schema.postgres.sql"); export const API_CONTRACT_DECIMAL_SQLITE_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DECIMAL_DIR, "schema.sqlite.sql"); export const API_CONTRACT_DECIMAL_MYSQL_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DECIMAL_DIR, "schema.mysql.sql"); + +// fixtures/api-contract-conformance/duration/ — a field.duration is an ISO-8601 span on the wire and +// filters and sorts as a span (read, write, filter, sort) over HTTP, generated lane only. +export const API_CONTRACT_DURATION_DIR = resolve(API_CONTRACT_DIR, "duration"); +export const API_CONTRACT_DURATION_SCENARIOS_DIR = resolve(API_CONTRACT_DURATION_DIR, "scenarios"); +export const API_CONTRACT_DURATION_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DURATION_DIR, "schema.postgres.sql"); +export const API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DURATION_DIR, "schema.sqlite.sql"); +export const API_CONTRACT_DURATION_MYSQL_SCHEMA_SQL_PATH = resolve(API_CONTRACT_DURATION_DIR, "schema.mysql.sql"); diff --git a/server/typescript/packages/integration-tests/test/api-contract-duration.test.ts b/server/typescript/packages/integration-tests/test/api-contract-duration.test.ts new file mode 100644 index 0000000000..c22213b66e --- /dev/null +++ b/server/typescript/packages/integration-tests/test/api-contract-duration.test.ts @@ -0,0 +1,71 @@ +// api-contract conformance: a field.duration is an ISO-8601 span on the wire, GENERATED lane. +// +// Drives fixtures/api-contract-conformance/duration/ over HTTP against the EMITTED routes of one +// writable entity, on Postgres (a native interval), MySQL 8.4, SQLite and Cloudflare D1's local +// runtime (a BIGINT count of microseconds). The corpus pins "wire": { "durations": "iso" } in its +// own config. Requires Docker for Postgres and MySQL. + +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { + API_CONTRACT_DURATION_DIR, + API_CONTRACT_DURATION_MYSQL_SCHEMA_SQL_PATH, + API_CONTRACT_DURATION_SCENARIOS_DIR, + API_CONTRACT_DURATION_SCHEMA_SQL_PATH, + API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH, +} from "../src/paths.ts"; +import { loadScenarios, runScenario } from "../src/api-contract-scenario.ts"; +import { startGeneratedCorpusServer, type CorpusEngine, type GeneratedCorpusSpec } from "../src/api-contract-decimal-server.ts"; +import type { ReportSeed } from "../src/api-contract-report-generated-server.ts"; +import { startMysql, type MysqlContainerHandle } from "../src/mysql-container.ts"; +import { startPostgres } from "../src/postgres-container.ts"; + +const SEED = JSON.parse(readFileSync(join(API_CONTRACT_DURATION_DIR, "seed.json"), "utf8")) as ReportSeed; +const META_PATH = join(API_CONTRACT_DURATION_DIR, "meta.json"); +const ENGINES: readonly CorpusEngine[] = ["postgres", "mysql", "sqlite", "d1"]; + +const DURATION: GeneratedCorpusSpec = { + name: "duration", + served: ["Timer"], + tables: ["timers"], + mysqlViews: [], + schemaPath: { + postgres: API_CONTRACT_DURATION_SCHEMA_SQL_PATH, + mysql: API_CONTRACT_DURATION_MYSQL_SCHEMA_SQL_PATH, + sqlite: API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH, + d1: API_CONTRACT_DURATION_SQLITE_SCHEMA_SQL_PATH, + }, +}; + +let mysqlContainer: MysqlContainerHandle | undefined; +beforeAll(async () => { mysqlContainer = await startMysql(); }, 240_000); +afterAll(() => { mysqlContainer?.stop(); }, 60_000); + +async function withServer(engine: CorpusEngine, fn: (baseUrl: string) => Promise): Promise { + if (engine === "postgres") { + const pg = await startPostgres(); + try { + const server = await startGeneratedCorpusServer(DURATION, engine, META_PATH, pg.connectionUri); + try { await server.applySeed(SEED); await fn(server.baseUrl); } finally { await server.close(); } + } finally { + await pg.stop(); + } + return; + } + const server = await startGeneratedCorpusServer(DURATION, engine, META_PATH, engine === "mysql" ? mysqlContainer!.url : undefined); + try { await server.applySeed(SEED); await fn(server.baseUrl); } finally { await server.close(); } +} + +const scenarios = loadScenarios(API_CONTRACT_DURATION_SCENARIOS_DIR); + +test("the duration corpus has scenarios", () => { expect(scenarios.length).toBeGreaterThan(0); }); + +for (const engine of ENGINES) describe(`api contract duration — GENERATED routes on ${engine}`, () => { + for (const scenario of scenarios) { + if (scenario.engines !== undefined && !scenario.engines.includes(engine)) continue; + test(scenario.name, async () => { + await withServer(engine, (baseUrl) => runScenario(scenario, baseUrl)); + }, { timeout: 120_000 }); + } +}); diff --git a/server/typescript/packages/integration-tests/test/load-metadata-config.test.ts b/server/typescript/packages/integration-tests/test/load-metadata-config.test.ts new file mode 100644 index 0000000000..3e9b9f6c8c --- /dev/null +++ b/server/typescript/packages/integration-tests/test/load-metadata-config.test.ts @@ -0,0 +1,30 @@ +// A corpus's `.metaobjects/config.json` reaches the loaded root the way a project load does: +// naming, wire format and enum defaults, so the schema a lane builds is the one the corpus pins. +import { test, expect, afterAll } from "bun:test"; +import { mkdtemp, rm, mkdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { resolveEnumMaxLength } from "@metaobjectsdev/metadata"; +import { loadMetadataDir } from "../src/load-metadata.ts"; + +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); + +test("enums.maxLength from the corpus config sizes an enum without its own @maxLength", async () => { + const dir = await mkdtemp(join(tmpdir(), "it-load-config-")); + dirs.push(dir); + await mkdir(join(dir, ".metaobjects")); + await writeFile(join(dir, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, enums: { maxLength: 40 } })); + await mkdir(join(dir, "meta")); + await writeFile(join(dir, "meta", "meta.json"), JSON.stringify({ "metadata.root": { package: "p", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "state", "@values": ["OPEN", "CLOSED"] } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ] } }, + ] } })); + const root = await loadMetadataDir(join(dir, "meta")); + const state = root.findObject("Ticket")!.fields().find((f) => f.name === "state")!; + expect(resolveEnumMaxLength(state)).toBe(40); +}); diff --git a/server/typescript/packages/integration-tests/test/validation-conformance-cross-field.test.ts b/server/typescript/packages/integration-tests/test/validation-conformance-cross-field.test.ts index ad505b9bba..d295e4f96c 100644 --- a/server/typescript/packages/integration-tests/test/validation-conformance-cross-field.test.ts +++ b/server/typescript/packages/integration-tests/test/validation-conformance-cross-field.test.ts @@ -1,7 +1,8 @@ // Cross-field validation-conformance, TS (Zod) reference runner. // -// Renders the GENERATED `EventInsertSchema` (validator.comparison / requiredWhen / -// presentIff / atLeastOne enforced as a refinement) through the real codegen template +// Renders the GENERATED `EventInsertSchema` and `ShiftInsertSchema` (validator.comparison / +// requiredWhen / presentIff / atLeastOne / exactlyOne, each with or without an @condition, +// enforced as a refinement) through the real codegen template // into a temp module, then safeParses every cross-field-cases.json payload. import { describe, test, expect, beforeAll, afterAll } from "bun:test"; @@ -17,7 +18,7 @@ import { loadCrossFieldCases } from "../src/validation-cases.ts"; type ParseFn = { safeParse(value: unknown): { success: boolean } }; -const ENTITY_NAMES = ["Event"] as const; +const ENTITY_NAMES = ["Event", "Shift"] as const; const insertSchemas = new Map(); let updateSchema: ParseFn; let tmpDir: string; @@ -40,7 +41,7 @@ beforeAll(async () => { const schema = mod[`${name}InsertSchema`]; if (!schema) throw new Error(`generated module did not export ${name}InsertSchema`); insertSchemas.set(name, schema as unknown as ParseFn); - updateSchema = mod[`${name}UpdateSchema`] as unknown as ParseFn; + if (name === "Event") updateSchema = mod[`${name}UpdateSchema`] as unknown as ParseFn; } }); diff --git a/server/typescript/packages/metadata/src/core/field/duration-wire.ts b/server/typescript/packages/metadata/src/core/field/duration-wire.ts new file mode 100644 index 0000000000..3970d7d085 --- /dev/null +++ b/server/typescript/packages/metadata/src/core/field/duration-wire.ts @@ -0,0 +1,182 @@ +// The field.duration wire rule: a fixed-length span of time on the wire is an ISO-8601 +// duration, and every port answers the same canonical form for it. The rule and its +// cross-port vectors are in fixtures/duration-wire/. Shared by runtime-ts (reads, writes, +// filters, validation) and migrate-ts (a declared @default); generated code carries its own +// copy, held to the same vectors. +// +// Canonical: `[-]PT[H][M][[.]S]` — hours never folded into days, zero components +// left out, zero is `PT0S`, one leading `-` for a negative span. TypeScript holds a value as a +// bigint count of microseconds, so a span of any length is exact and a longer fraction is +// truncated toward zero to six digits (the `precision` vectors' "6" entries). + +/** The accepted ISO-8601 duration form: an optional sign, `P`, optional weeks (7 days) and days, + * then an optional `T` with hours, minutes and seconds in that order; the seconds may have a + * fraction of any length. At least one component; no years or months (a calendar span has no + * fixed length). */ +export const DURATION_PATTERN = "^[-+]?P(?!$)(\\d+W)?(\\d+D)?(T(?=\\d)(\\d+H)?(\\d+M)?(\\d+(\\.\\d+)?S)?)?$"; +export const DURATION_RE = new RegExp(DURATION_PATTERN); + +const DURATION_PARTS_RE = /^([-+])?P(?:(\d+)W)?(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/; + +// The default (`IntervalStyle = postgres`) text of an interval: signed year / mon / day parts, +// then an optionally signed `H:MM:SS[.ffffff]`. Years and months are matched so they can be +// refused by name rather than as unreadable text. +const PG_INTERVAL_RE = + /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/; + +const MICROS_PER_SECOND = 1_000_000n; +const MICROS_PER_MINUTE = 60n * MICROS_PER_SECOND; +const MICROS_PER_HOUR = 60n * MICROS_PER_MINUTE; +const MICROS_PER_DAY = 24n * MICROS_PER_HOUR; + +/** A 64-bit integer column (SQLite, D1, MySQL `BIGINT`) stores a duration as microseconds; it holds + * this range either way. A database's own limit, so it is checked only when binding to one. */ +export const DURATION_MICROS_MIN = -9223372036854775808n; +export const DURATION_MICROS_MAX = 9223372036854775807n; + +/** Fractional-second digits as microseconds, truncated toward zero. */ +function fractionMicros(digits: string | undefined): bigint { + if (digits === undefined || digits.length === 0) return 0n; + return BigInt(digits.padEnd(6, "0").slice(0, 6)); +} + +/** The microseconds an accepted ISO-8601 duration spans, or undefined when `text` is not one. */ +export function parseDurationLiteral(text: string): bigint | undefined { + if (!DURATION_RE.test(text)) return undefined; + const m = DURATION_PARTS_RE.exec(text); + if (m === null) return undefined; + const [, sign, w, d, h, min, s, f] = m; + const magnitude = + BigInt(w ?? "0") * 7n * MICROS_PER_DAY + + BigInt(d ?? "0") * MICROS_PER_DAY + + BigInt(h ?? "0") * MICROS_PER_HOUR + + BigInt(min ?? "0") * MICROS_PER_MINUTE + + BigInt(s ?? "0") * MICROS_PER_SECOND + + fractionMicros(f); + return sign === "-" ? -magnitude : magnitude; +} + +/** The magnitude of a span split into hours, minutes, seconds and microseconds of a second. */ +function splitMicros(micros: bigint): { negative: boolean; h: bigint; m: bigint; s: bigint; frac: bigint } { + const negative = micros < 0n; + let rest = negative ? -micros : micros; + const h = rest / MICROS_PER_HOUR; + rest %= MICROS_PER_HOUR; + const m = rest / MICROS_PER_MINUTE; + rest %= MICROS_PER_MINUTE; + const s = rest / MICROS_PER_SECOND; + return { negative, h, m, s, frac: rest % MICROS_PER_SECOND }; +} + +/** `.` and the microseconds with trailing zeros trimmed, or `""` when there is no fraction. */ +function fractionSuffix(frac: bigint): string { + return frac > 0n ? "." + String(frac).padStart(6, "0").replace(/0+$/, "") : ""; +} + +/** The canonical wire form of a span of `micros` microseconds. */ +export function formatDuration(micros: bigint): string { + const { negative, h, m, s, frac } = splitMicros(micros); + let out = "PT"; + if (h > 0n) out += `${h}H`; + if (m > 0n) out += `${m}M`; + if (s > 0n || frac > 0n) out += `${s}${fractionSuffix(frac)}S`; + if (out === "PT") return "PT0S"; + return negative ? "-" + out : out; +} + +/** + * A span as the text Postgres reads as interval input AND writes back for it: `[-]HH:MM:SS[.f]`, + * hours unbounded, no day part. Postgres refuses an ISO-8601 duration with a leading sign + * (`-PT1M30S`), so a value bound as text, and a declared column default, use this form; an + * interval written this way reads back in this exact spelling, so a default does not drift. + */ +export function formatPostgresInterval(micros: bigint): string { + const { negative, h, m, s, frac } = splitMicros(micros); + const two = (n: bigint): string => String(n).padStart(2, "0"); + const out = `${two(h)}:${two(m)}:${two(s)}${fractionSuffix(frac)}`; + return negative && micros !== 0n ? "-" + out : out; +} + +/** The canonical form of an accepted ISO-8601 duration, or undefined when `text` is not one. */ +export function canonicalDuration(text: string): string | undefined { + const micros = parseDurationLiteral(text); + return micros === undefined ? undefined : formatDuration(micros); +} + +/** Write side: a caller's value as the canonical string to bind, or undefined when it is not a + * duration. Postgres reads the canonical ISO form as interval input; a text column stores it, so + * equal spans are equal strings. */ +export function parseDurationInput(value: unknown): string | undefined { + return typeof value === "string" ? canonicalDuration(value) : undefined; +} + +/** The microseconds of the default-style text of a Postgres interval. Throws for an interval + * with a year or month part (not a fixed length) and for text that is not an interval. */ +export function parsePostgresInterval(text: string): bigint { + const m = text.length === 0 ? null : PG_INTERVAL_RE.exec(text); + if (m === null) throw new Error(`not a Postgres interval: ${JSON.stringify(text)}`); + const [, years, months, days, timeSign, h, min, s, f] = m; + if ((years !== undefined && BigInt(years) !== 0n) || (months !== undefined && BigInt(months) !== 0n)) { + throw new Error(`the interval ${JSON.stringify(text)} has a year or month part, so it is not a fixed-length field.duration`); + } + let time = 0n; + if (h !== undefined) { + time = BigInt(h) * MICROS_PER_HOUR + BigInt(min!) * MICROS_PER_MINUTE + BigInt(s!) * MICROS_PER_SECOND + fractionMicros(f); + if (timeSign === "-") time = -time; + } + return BigInt(days ?? "0") * MICROS_PER_DAY + time; +} + +/** The fields node-postgres's default interval parser (`postgres-interval`) hands back. */ +interface PostgresIntervalLike { + years?: number; months?: number; days?: number; + hours?: number; minutes?: number; seconds?: number; milliseconds?: number; +} + +function isIntervalObject(value: unknown): value is PostgresIntervalLike { + if (value === null || typeof value !== "object" || Array.isArray(value)) return false; + return ["days", "hours", "minutes", "seconds", "milliseconds", "years", "months"].some((k) => k in value) + || typeof (value as { toPostgres?: unknown }).toPostgres === "function"; +} + +function intervalObjectMicros(v: PostgresIntervalLike): bigint { + if ((v.years ?? 0) !== 0 || (v.months ?? 0) !== 0) { + throw new Error(`the interval ${JSON.stringify(v)} has a year or month part, so it is not a fixed-length field.duration`); + } + const whole = (n: number | undefined): bigint => BigInt(Math.trunc(n ?? 0)); + // postgres-interval carries microseconds as a fractional millisecond count. + const micros = BigInt(Math.round((v.milliseconds ?? 0) * 1000)); + return whole(v.days) * MICROS_PER_DAY + whole(v.hours) * MICROS_PER_HOUR + + whole(v.minutes) * MICROS_PER_MINUTE + whole(v.seconds) * MICROS_PER_SECOND + micros; +} + +/** + * Write side, for a 64-bit integer column (SQLite, D1, MySQL): the span as the microsecond count the + * column stores — a number when it is a safe integer, else a bigint. Throws past the 64-bit range, + * which is that column's limit and no other database's. + */ +export function durationToStoredMicros(micros: bigint): number | bigint { + if (micros > DURATION_MICROS_MAX || micros < DURATION_MICROS_MIN) { + throw new Error(`a span of ${formatDuration(micros)} does not fit the 64-bit microsecond column this database stores a field.duration in`); + } + const n = Number(micros); + return Number.isSafeInteger(n) ? n : micros; +} + +/** Read side: what a driver hands back for a duration column — the microsecond integer of a SQLite, + * D1 or MySQL column, an ISO string (Postgres under `IntervalStyle = iso_8601`), the default + * Postgres interval text, or a node-postgres interval object — as the canonical wire string. + * Throws for anything else. */ +export function durationToWire(value: unknown): string { + // A 64-bit integer column (SQLite, D1, MySQL) holds microseconds; a driver may return it as a + // number, a bigint or a digit string. + if (typeof value === "bigint") return formatDuration(value); + if (typeof value === "number" && Number.isInteger(value)) return formatDuration(BigInt(value)); + if (typeof value === "string" && /^-?\d+$/.test(value)) return formatDuration(BigInt(value)); + if (typeof value === "string") { + const iso = canonicalDuration(value); + return iso ?? formatDuration(parsePostgresInterval(value)); + } + if (isIntervalObject(value)) return formatDuration(intervalObjectMicros(value)); + throw new Error(`not a field.duration value: ${JSON.stringify(value)}`); +} diff --git a/server/typescript/packages/metadata/src/core/field/field-constants.ts b/server/typescript/packages/metadata/src/core/field/field-constants.ts index 1e470bf7ee..0c340412a1 100644 --- a/server/typescript/packages/metadata/src/core/field/field-constants.ts +++ b/server/typescript/packages/metadata/src/core/field/field-constants.ts @@ -34,6 +34,12 @@ export const FIELD_SUBTYPE_URI = "uri"; * IP (InetAddress / IPAddress / ipaddress). DB column is Postgres-native `inet`. * Codegen emits IP validation accepting v4+v6 (Zod .ip()). */ export const FIELD_SUBTYPE_INET = "inet"; +/** A fixed-length span of time (days, hours, minutes, seconds). Concept with a + * native type + behavior → a subtype. TS binds to `string` (an ISO-8601 + * duration; TS has no native duration type); other ports bind to native + * (TimeSpan / java.time.Duration / timedelta). DB column is Postgres-native + * `interval`, a 64-bit microsecond integer elsewhere. Years, months and weeks are refused. */ +export const FIELD_SUBTYPE_DURATION = "duration"; export const FIELD_SUBTYPES = [ SUBTYPE_BASE, @@ -54,6 +60,7 @@ export const FIELD_SUBTYPES = [ FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, ] as const; export type FieldSubType = (typeof FIELD_SUBTYPES)[number]; diff --git a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts index 75209e3a82..d3eb2bf19b 100644 --- a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts @@ -285,6 +285,14 @@ export const FIELD_DEFINITION: ProviderDefinition = { "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { + "type": "attr", + "subType": "int", + "name": "maxLength", + "min": 0, + "max": 1, + "description": "Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1." + }, { "type": "attr", "subType": "intMap", @@ -338,6 +346,14 @@ export const FIELD_DEFINITION: ProviderDefinition = { "description": "#234: opt this field out of strict well-formedness enforcement. When true, codegen binds a plain string (no URL/IP validator, no native URI/InetAddress type; field.inet uses a text column, not the native inet type) so a not-strictly-valid value round-trips unchanged. Default (absent/false) is strict: an absolute-scheme URI / an IPv4-or-IPv6 literal." } ] + }, + { + "type": "field", + "subType": "duration", + "dataType": "string", + "extendsBase": true, + "description": "A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.", + "whenToUse": "A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span (\"every 3 months\") is not a duration: model it as an int and a unit." } ] }; diff --git a/server/typescript/packages/metadata/src/core/field/meta-field.ts b/server/typescript/packages/metadata/src/core/field/meta-field.ts index 7bdb2ea246..971f9ab26e 100644 --- a/server/typescript/packages/metadata/src/core/field/meta-field.ts +++ b/server/typescript/packages/metadata/src/core/field/meta-field.ts @@ -34,6 +34,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_REQUIRED, FIELD_ATTR_UNIQUE, FIELD_ATTR_DEFAULT, @@ -59,6 +60,9 @@ const FIELD_DATA_TYPE: Readonly> = { // URI/IP type in TS). Other ports bind to their native URI/IP type. [FIELD_SUBTYPE_URI]: DATA_TYPE_STRING, [FIELD_SUBTYPE_INET]: DATA_TYPE_STRING, + // field.duration binds to TS `string` (an ISO-8601 duration); other ports bind + // to their native duration type. + [FIELD_SUBTYPE_DURATION]: DATA_TYPE_STRING, [FIELD_SUBTYPE_INT]: DATA_TYPE_INT, [FIELD_SUBTYPE_LONG]: DATA_TYPE_LONG, [FIELD_SUBTYPE_CURRENCY]: DATA_TYPE_LONG, diff --git a/server/typescript/packages/metadata/src/core/query/query-constants.ts b/server/typescript/packages/metadata/src/core/query/query-constants.ts index 8eba954b2c..df65704691 100644 --- a/server/typescript/packages/metadata/src/core/query/query-constants.ts +++ b/server/typescript/packages/metadata/src/core/query/query-constants.ts @@ -1,4 +1,4 @@ -import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_ATTR_INT_VALUE_MAP } from "../field/field-constants.js"; +import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_SUBTYPE_DURATION, FIELD_ATTR_INT_VALUE_MAP } from "../field/field-constants.js"; // Query concern constants — filter operators, sort order values. // @@ -61,6 +61,9 @@ export const OPS_BY_SUBTYPE: Readonly> = { // inet: an IP address — identity-comparison ops only (no `like`; ordering // gt/lt is not exposed). Same op band as uuid. [FIELD_SUBTYPE_INET]: [FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL], + // duration: ordered on every database — a Postgres interval, and the 64-bit + // microsecond integer SQLite / D1 / MySQL store it as. + [FIELD_SUBTYPE_DURATION]: [FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_GT, FILTER_OP_GTE, FILTER_OP_LT, FILTER_OP_LTE, FILTER_OP_IN, FILTER_OP_IS_NULL], } as const; export function opsForSubType(subType: string): readonly FilterOp[] { diff --git a/server/typescript/packages/metadata/src/core/validator/validator-constants.ts b/server/typescript/packages/metadata/src/core/validator/validator-constants.ts index 719828e1f9..acbbbd0f74 100644 --- a/server/typescript/packages/metadata/src/core/validator/validator-constants.ts +++ b/server/typescript/packages/metadata/src/core/validator/validator-constants.ts @@ -16,6 +16,7 @@ export const VALIDATOR_SUBTYPE_COMPARISON = "comparison"; export const VALIDATOR_SUBTYPE_REQUIRED_WHEN = "requiredWhen"; export const VALIDATOR_SUBTYPE_PRESENT_IFF = "presentIff"; export const VALIDATOR_SUBTYPE_AT_LEAST_ONE = "atLeastOne"; +export const VALIDATOR_SUBTYPE_EXACTLY_ONE = "exactlyOne"; export const VALIDATOR_SUBTYPES = [ SUBTYPE_BASE, @@ -28,6 +29,7 @@ export const VALIDATOR_SUBTYPES = [ VALIDATOR_SUBTYPE_REQUIRED_WHEN, VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_EXACTLY_ONE, ] as const; export type ValidatorSubType = (typeof VALIDATOR_SUBTYPES)[number]; @@ -50,3 +52,16 @@ export const VALIDATOR_ATTR_FIELDS = "fields"; // Comparison operators (@op allowed values) → relational operator. export const VALIDATOR_COMPARISON_OPS = ["gt", "gte", "lt", "lte", "ne", "eq"] as const; export type ComparisonOp = (typeof VALIDATOR_COMPARISON_OPS)[number]; + +/** comparison / requiredWhen / presentIff / atLeastOne / exactlyOne: the case the rule applies in (an attr.expression). */ +export const VALIDATOR_ATTR_CONDITION = "condition"; +/** Every validator (db provider): the database name of the CHECK it generates. */ +export const VALIDATOR_ATTR_CONSTRAINT_NAME = "constraintName"; +/** The cross-field validators: the database enforces them, nothing else evaluates them. */ +export const CROSS_FIELD_VALIDATOR_SUBTYPES: readonly string[] = [ + VALIDATOR_SUBTYPE_COMPARISON, + VALIDATOR_SUBTYPE_REQUIRED_WHEN, + VALIDATOR_SUBTYPE_PRESENT_IFF, + VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_EXACTLY_ONE, +]; diff --git a/server/typescript/packages/metadata/src/core/validator/validator-definition.embedded.ts b/server/typescript/packages/metadata/src/core/validator/validator-definition.embedded.ts index c8f5c6a83d..1786cca426 100644 --- a/server/typescript/packages/metadata/src/core/validator/validator-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/validator/validator-definition.embedded.ts @@ -180,6 +180,14 @@ export const VALIDATOR_DEFINITION: ProviderDefinition = { "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." + }, + { + "type": "attr", + "subType": "expression", + "name": "condition", + "min": 0, + "max": 1, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, @@ -213,6 +221,14 @@ export const VALIDATOR_DEFINITION: ProviderDefinition = { "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." + }, + { + "type": "attr", + "subType": "expression", + "name": "condition", + "min": 0, + "max": 1, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, @@ -246,6 +262,14 @@ export const VALIDATOR_DEFINITION: ProviderDefinition = { "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." + }, + { + "type": "attr", + "subType": "expression", + "name": "condition", + "min": 0, + "max": 1, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, @@ -264,6 +288,40 @@ export const VALIDATOR_DEFINITION: ProviderDefinition = { "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." + }, + { + "type": "attr", + "subType": "expression", + "name": "condition", + "min": 0, + "max": 1, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." + } + ] + }, + { + "type": "validator", + "subType": "exactlyOne", + "description": "Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.", + "whenToUse": "A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.", + "rules": "@fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.", + "children": [ + { + "type": "attr", + "subType": "string", + "name": "fields", + "isArray": true, + "min": 1, + "max": 1, + "description": "Names of the alternative fields; exactly one must be present." + }, + { + "type": "attr", + "subType": "expression", + "name": "condition", + "min": 0, + "max": 1, + "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] } diff --git a/server/typescript/packages/metadata/src/enum-length.ts b/server/typescript/packages/metadata/src/enum-length.ts new file mode 100644 index 0000000000..af3d3e5a90 --- /dev/null +++ b/server/typescript/packages/metadata/src/enum-length.ts @@ -0,0 +1,66 @@ +// The column length of a string-stored field.enum: the field's @maxLength, else the +// project's default (`enums.maxLength` in .metaobjects/config.json, carried on the loaded +// root beside the naming rules), else none — an unbounded text column, as before. + +import type { MetaData } from "./shared/meta-data.js"; +import { MetaModelError } from "./errors.js"; +import { + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_VALUES, + FIELD_SUBTYPE_ENUM, +} from "./core/field/field-constants.js"; +import { TYPE_OBJECT } from "./shared/base-types.js"; + +/** The `enums` block in `.metaobjects/config.json`. */ +export const ENUMS_KEY_MAX_LENGTH = "maxLength"; + +/** The project's enum storage defaults: `enums` in `.metaobjects/config.json`. */ +export interface EnumDefaults { + /** The column length of a string-stored enum that declares no @maxLength. */ + readonly maxLength?: number | undefined; +} + +/** No project default: an enum with no @maxLength is an unbounded text column. */ +export const DEFAULT_ENUM_DEFAULTS: EnumDefaults = Object.freeze({}); + +/** The enum defaults for the tree `node` belongs to: what its root carries, else none. */ +export function enumDefaultsOf(node: MetaData): EnumDefaults { + const defaults = (node.root() as { enumDefaults?: EnumDefaults }).enumDefaults; + return defaults ?? DEFAULT_ENUM_DEFAULTS; +} + +/** + * The column length of `field`, a string-stored `field.enum`: its @maxLength (ADR-0039 — + * resolving, it may be inherited), else the project default, else undefined. Undefined for + * an @intValueMap enum, whose column is an integer, for an array enum, a text[] column whose + * elements carry no length, and for any other field. + * + * The loader refuses an @maxLength a member does not fit; it does not see the project + * config, so a member longer than the project DEFAULT is refused here, by every consumer + * that sizes the column (`meta migrate`, the generated bindings). + */ +export function resolveEnumMaxLength(field: MetaData): number | undefined { + if (field.subType !== FIELD_SUBTYPE_ENUM || field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined) return undefined; + // An array enum is a text[] column, whose elements carry no length (ADR-0039: resolving). + if ((field as { resolvedIsArray?: () => boolean }).resolvedIsArray?.() === true) return undefined; + const own = field.attr(FIELD_ATTR_MAX_LENGTH); + if (typeof own === "number") return own; + const fallback = enumDefaultsOf(field).maxLength; + if (fallback === undefined) return undefined; + const values = field.attr(FIELD_ATTR_VALUES); + const longest = (Array.isArray(values) ? values : []) + .filter((v): v is string => typeof v === "string") + .reduce((a, v) => (a === undefined || v.length > a.length ? v : a), undefined); + if (longest !== undefined && longest.length > fallback) { + const owner = field.parent; + const label = owner !== undefined && owner.type === TYPE_OBJECT ? `${owner.name}.${field.name}` : field.name; + throw new MetaModelError( + `field.enum "${label}" has the member "${longest}" (${longest.length} characters), longer than ` + + `the project's enums.maxLength ${fallback}. Give the field its own @maxLength of at least ` + + `${longest.length}, raise enums.maxLength, or shorten the member.`, + { code: "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH" }, + ); + } + return fallback; +} diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index e9616a15a8..4f8b3fac21 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -153,6 +153,14 @@ export const ERROR_CODES = [ "ERR_TEMPLATE_ENGINE_UNSUPPORTED", "ERR_BAD_ATTR_FILTER", "ERR_STORAGE_FLATTENED_ARRAY", + "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE", + // field.enum @maxLength: a member longer than the column, or a length on an int-stored enum. + "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", + "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP", + // identity.primary @sequenceStart / @sequenceName without @generation "increment". + "ERR_SEQUENCE_WITHOUT_INCREMENT", + // identity.primary @sequenceStart below 1, or past what its key field can hold. + "ERR_SEQUENCE_START_OUT_OF_RANGE", "ERR_STORAGE_WITHOUT_OBJECT_REF", // ADR-0013 — a field.object REQUIRES @objectRef (open/untyped JSON uses // the physical @dbColumnType: jsonb escape hatch on field.string instead). diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index 6ddc3032ef..73f4655a6b 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -27,6 +27,7 @@ export * from "./shared/base-types.js"; export * from "./shared/structural.js"; export * from "./core/object/object-constants.js"; export * from "./core/field/field-constants.js"; +export * from "./core/field/duration-wire.js"; // FR-037 R1 — THE mutability accessors. Every consumer deciding "may this be // written, and when?" must go through these rather than reading the attr, so the // absent-means-readWrite default lives in exactly one place per port. @@ -94,6 +95,7 @@ export { } from "./core/requirement/resolve-claim.js"; export * from "./core/relationship/relationship-constants.js"; export * from "./core/query/query-constants.js"; +export { EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, EXPR_FN_COALESCE, type ExprNode } from "./core/attr/meta-attr-expression.js"; export * from "./persistence/source/source-constants.js"; export * from "./persistence/origin/origin-constants.js"; export * from "./template/template-constants.js"; @@ -368,25 +370,29 @@ export { // Naming — hoisted from runtime-ts in v0.2.3 so multiple consumers (runtime-ts, migrate-ts, codegen-ts) // share identical name resolution. See spec §4.1. export { - toSnakeCase, toKebabCase, pluralize, pluralizeTableNameLegacy, + toSnakeCase, toKebabCase, toUpperFirstColumn, pluralize, pluralizeTableNameLegacy, applyColumnNamingStrategy, DEFAULT_COLUMN_NAMING_STRATEGY, COLUMN_NAMING_STRATEGIES, TABLE_NAMING_RULES, DEFAULT_PHYSICAL_NAMING, physicalNamingOf, applyTableNamingRule, PHYSICAL_NAMING_MIGRATION_GUIDE, PHYSICAL_NAMING_MIGRATION_GUIDE_PATH, physicalNamingRemovedMessage, - resolveTableName, resolveColumnName, resolveTableSchema, resolveIndexName, + resolveTableName, resolveColumnName, resolveFlattenedColumnPrefix, resolveFlattenedColumnName, + resolveTableSchema, resolveIndexName, primaryRdbSource, sourceAddressKey, buildNameMap, stripPackage, packageOfResolutionKey, } from "./naming.js"; export type { EntityNameMap, ColumnNamingStrategy, PhysicalNaming, TableNamingRule } from "./naming.js"; +export { resolveEnumMaxLength, enumDefaultsOf, DEFAULT_ENUM_DEFAULTS, ENUMS_KEY_MAX_LENGTH } from "./enum-length.js"; +export type { EnumDefaults } from "./enum-length.js"; // Wire format (ADR-0019 Amendment 2): the `wire` block of .metaobjects/config.json. export { - WIRE_DECIMALS, WIRE_NULL_MEMBERS, WIRE_KEY_DECIMALS, WIRE_KEY_NULL_MEMBERS, + WIRE_DECIMALS, WIRE_NULL_MEMBERS, WIRE_DURATIONS, WIRE_KEY_DECIMALS, WIRE_KEY_NULL_MEMBERS, WIRE_KEY_DURATIONS, WIRE_DECIMALS_NATIVE, WIRE_DECIMALS_STRING, WIRE_NULL_MEMBERS_NATIVE, WIRE_NULL_MEMBERS_EMIT, WIRE_NULL_MEMBERS_OMIT, + WIRE_DURATIONS_NATIVE, WIRE_DURATIONS_ISO, DEFAULT_WIRE_FORMAT, wireFormatOf, } from "./wire.js"; -export type { WireFormat, WireDecimals, WireNullMembers } from "./wire.js"; +export type { WireFormat, WireDecimals, WireNullMembers, WireDurations } from "./wire.js"; // Retired vocabulary: the map the loader reads to explain a retirement, and the raw-document // rewriter `meta upgrade` drives from the SAME entries — so the error message and the fix diff --git a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts index f939431360..726c1671b3 100644 --- a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts +++ b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts @@ -25,7 +25,8 @@ import { ParseError } from "../errors.js"; import type { LoaderWarning } from "../source.js"; import { codeSource, resolvedSource } from "../source.js"; import { parseJson } from "../parser-json.js"; -import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateSortableHasSupportedSubtype, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateOneSideReferenceResolution, validateM2MJunctionPairing, validateIndexLookupFields, validateProjectionFilter, validateRetiredRequirementLinks, validateColumnNameCollisions } from "./validation-passes.js"; +import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateSortableHasSupportedSubtype, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, + validateEnumMaxLength, validateIdentitySequence, validateCrossFieldValidators, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateOneSideReferenceResolution, validateM2MJunctionPairing, validateIndexLookupFields, validateProjectionFilter, validateRetiredRequirementLinks, validateColumnNameCollisions } from "./validation-passes.js"; import { validateReporting } from "./reporting-validation.js"; import { runRegisteredValidation } from "./validation-registry.js"; import { validateSourceRoles } from "../persistence/source/validate-source-roles.js"; @@ -724,6 +725,15 @@ export class MetaDataLoader { // @valueType (when set) must name a known scalar subtype. errors.push(...validateFieldMap(root)); + // field.enum @maxLength — every member fits; refused with @intValueMap. + errors.push(...validateEnumMaxLength(root)); + + // identity.primary @sequenceStart / @sequenceName need @generation "increment". + errors.push(...validateIdentitySequence(root)); + + // Cross-field validators: exactlyOne names two or more fields; @condition names real fields. + errors.push(...validateCrossFieldValidators(root)); + // Tenth pass: one-primary multi-source rule — if an object has ≥1 source, // exactly one must carry role "primary" (ERR_SOURCE_NO_PRIMARY / // ERR_SOURCE_MULTIPLE_PRIMARY). diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 3d72cc712b..b11b5dbf30 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -10,6 +10,18 @@ // Private: _findObject, _findField, _findRelationship, // _validateFromPath, _validateViaPath (helpers, not exported). +import { TYPE_VALIDATOR } from "../shared/base-types.js"; +import { + CROSS_FIELD_VALIDATOR_SUBTYPES, + VALIDATOR_ATTR_CONDITION, + VALIDATOR_ATTR_EQUALS, + VALIDATOR_ATTR_FIELDS, + VALIDATOR_ATTR_WHEN, + VALIDATOR_SUBTYPE_EXACTLY_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, + VALIDATOR_SUBTYPE_REQUIRED_WHEN, +} from "../core/validator/validator-constants.js"; +import { parseDurationLiteral } from "../core/field/duration-wire.js"; import type { MetaData } from "../shared/meta-data.js"; import type { MetaObject } from "../core/object/meta-object.js"; import type { MetaReferenceIdentity } from "../core/identity/meta-identity.js"; @@ -33,7 +45,7 @@ import { INDEX_SUBTYPE_LOOKUP, INDEX_ATTR_FIELDS, } from "../core/index/index-constants.js"; -import { IDENTITY_SUBTYPE_SECONDARY } from "../core/identity/identity-constants.js"; +import { IDENTITY_SUBTYPE_SECONDARY, IDENTITY_SUBTYPE_PRIMARY, IDENTITY_ATTR_GENERATION, GENERATION_INCREMENT } from "../core/identity/identity-constants.js"; import type { MetaIdentity } from "../core/identity/meta-identity.js"; import { MetaIndex } from "../core/index/meta-index.js"; import { @@ -55,8 +67,8 @@ import { OBJECT_PROJECTION_ATTR_FILTER, } from "../core/object/object-constants.js"; import { MetaSource } from "../persistence/source/meta-source.js"; -import { resolveColumnName } from "../naming.js"; -import { isMetaField } from "../shared/node-guards.js"; +import { resolveColumnName, resolveFlattenedColumnName } from "../naming.js"; +import { isMetaField, isMetaObject } from "../shared/node-guards.js"; import { LAYOUT_SUBTYPE_DATA_GRID, LAYOUT_DATA_GRID_ATTR_DEFAULT_SORT_FIELD, @@ -77,6 +89,9 @@ import { FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_ENUM, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_VALUES, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_ATTR_REQUIRED, @@ -86,8 +101,9 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, } from "../core/field/field-constants.js"; -import { FIELD_ATTR_COLUMN, FIELD_ATTR_DB_INDEXED, IDENTITY_ATTR_EXPR } from "../persistence/db/db-constants.js"; +import { FIELD_ATTR_COLUMN, FIELD_ATTR_FLATTENED_COLUMN_PREFIX, FIELD_ATTR_DB_INDEXED, IDENTITY_ATTR_EXPR, IDENTITY_ATTR_SEQUENCE_NAME, IDENTITY_ATTR_SEQUENCE_START } from "../persistence/db/db-constants.js"; import { SOURCE_KIND_VIEW } from "../persistence/source/source-constants.js"; import { IDENTITY_ATTR_FIELDS, @@ -1677,6 +1693,244 @@ export function validateDerivedFieldProvidability(root: MetaData): ParseError[] return errors; } +// --------------------------------------------------------------------------- +// Cross-field validators (comparison / requiredWhen / presentIff / atLeastOne / exactlyOne), +// which the database enforces as CHECKs: +// - exactlyOne @fields names two or more fields (ERR_BAD_ATTR_VALUE): one is a required field; +// - every field a @condition expression names is a field of the owning entity +// (ERR_BAD_ATTR_VALUE), so the CHECK it renders to references a real column; +// - a requiredWhen / presentIff @equals gating a field.duration is an ISO-8601 duration +// (ERR_BAD_ATTR_VALUE), the form its CHECK literal is built from. +// Read RESOLVING (ADR-0039): an entity's validators and fields include inherited ones. +// --------------------------------------------------------------------------- + +/** The field names an attr.expression tree references. */ +function expressionFieldRefs(node: unknown, out: string[] = []): string[] { + if (typeof node !== "object" || node === null) return out; + const n = node as Record; + if (typeof n["field"] === "string") out.push(n["field"]); + for (const key of ["left", "right", "arg"]) expressionFieldRefs(n[key], out); + for (const key of ["args"]) if (Array.isArray(n[key])) for (const a of n[key] as unknown[]) expressionFieldRefs(a, out); + return out; +} + +/** What a literal compared with a field.duration must be. */ +const DURATION_LITERAL_FORM = "an ISO-8601 duration (such as PT1H30M; no years or months)"; + +/** Field subtypes whose values a condition compares with a number literal. */ +const _CONDITION_NUMBER_SUBTYPES = new Set([ + FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, + FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_CURRENCY, +]); + +/** + * Why a cross-field validator's `@condition` cannot become a CHECK guard, or undefined when it + * can. Its names are already known to be fields. The rule, the same in every port: + * - it types cleanly (the origin.computed inference) and its root is boolean, since a CHECK + * negates it; + * - a literal compared with a field is of that field's kind (a boolean for field.boolean, a + * number for a numeric field, a string otherwise) and, for field.enum, one of its @values, + * for field.duration, an ISO-8601 duration; + * - no comparison has a null literal: `x = NULL` is never true, so use isNull / isNotNull. + */ +function conditionProblem(condition: unknown, obj: MetaObject): string | undefined { + const fields = new Map(obj.fields().map((f) => [f.name, f] as const)); + const inferred = inferExprType(condition as ExprNode, (name) => fields.get(name)?.subType); + if (inferred.errors.length > 0) return `is not a valid expression: ${inferred.errors.join("; ")}.`; + if (inferred.type !== FIELD_SUBTYPE_BOOLEAN) { + return `must be a boolean expression (a comparison, a null test, or and / or / not of them), not a field.${String(inferred.type)}.`; + } + let problem: string | undefined; + const visit = (node: unknown): void => { + if (problem !== undefined || typeof node !== "object" || node === null) return; + const n = node as Record; + if (n["left"] !== undefined && n["right"] !== undefined) { + for (const [a, b] of [[n["left"], n["right"]], [n["right"], n["left"]]] as const) { + const lit = a as Record; + const ref = b as Record; + if (!("value" in lit)) continue; + if (lit["value"] === null) { + problem = `compares with null, which is never true: use isNull or isNotNull.`; + return; + } + const field = typeof ref["field"] === "string" ? fields.get(ref["field"]) : undefined; + if (field === undefined) continue; + const value = lit["value"]; + const kind = field.subType === FIELD_SUBTYPE_BOOLEAN ? "boolean" + : _CONDITION_NUMBER_SUBTYPES.has(field.subType) ? "number" : "string"; + if (typeof value !== kind) { + problem = `compares "${field.name}" (field.${field.subType}) with ${JSON.stringify(value)}; it needs a ${kind}.`; + return; + } + const members = field.subType === FIELD_SUBTYPE_ENUM ? field.attr(FIELD_ATTR_VALUES) : undefined; + if (Array.isArray(members) && !members.map(String).includes(String(value))) { + problem = `compares "${field.name}" with ${JSON.stringify(value)}, which is not one of its @values (${members.join(", ")}).`; + return; + } + if (field.subType === FIELD_SUBTYPE_DURATION && parseDurationLiteral(value as string) === undefined) { + problem = `compares "${field.name}" with ${JSON.stringify(value)}, which is not ${DURATION_LITERAL_FORM}.`; + return; + } + } + } + for (const key of ["left", "right", "arg"]) visit(n[key]); + if (Array.isArray(n["args"])) for (const a of n["args"]) visit(a); + }; + visit(condition); + return problem; +} + +export function validateCrossFieldValidators(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + const fieldNames = new Set((obj as MetaObject).fields().map((f) => f.name)); + // ADR-0039: own — each validator is checked where it is declared. + for (const v of obj.ownChildren()) { + if (v.type !== TYPE_VALIDATOR || !CROSS_FIELD_VALIDATOR_SUBTYPES.includes(v.subType)) continue; + if (v.subType === VALIDATOR_SUBTYPE_EXACTLY_ONE) { + const fields = [v.attr(VALIDATOR_ATTR_FIELDS)].flat().filter((f) => typeof f === "string"); + if (fields.length < 2) { + errors.push(new ParseError( + `validator.exactlyOne "${obj.name}.${v.name}" @fields names ${fields.length} field${fields.length === 1 ? "" : "s"}; ` + + `it needs two or more. For one field, make that field required.`, + { code: "ERR_BAD_ATTR_VALUE", source: v.source }, + )); + continue; + } + } + if (v.subType === VALIDATOR_SUBTYPE_REQUIRED_WHEN || v.subType === VALIDATOR_SUBTYPE_PRESENT_IFF) { + const when = (obj as MetaObject).fields().find((f) => f.name === v.attr(VALIDATOR_ATTR_WHEN)); + const equals = v.attr(VALIDATOR_ATTR_EQUALS); + if (when?.subType === FIELD_SUBTYPE_DURATION && (typeof equals !== "string" || parseDurationLiteral(equals) === undefined)) { + errors.push(new ParseError( + `validator.${v.subType} "${obj.name}.${v.name}" @equals ${JSON.stringify(equals)} gates field.duration ` + + `"${when.name}"; it needs ${DURATION_LITERAL_FORM}.`, + { code: "ERR_BAD_ATTR_VALUE", source: v.source }, + )); + continue; + } + } + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + if (condition !== undefined) { + const label = `validator.${v.subType} "${obj.name}.${v.name}" @condition`; + const unknown = [...new Set(expressionFieldRefs(condition))].filter((f) => !fieldNames.has(f)).sort(); + if (unknown.length > 0) { + errors.push(new ParseError( + `${label} names ${unknown.map((f) => `"${f}"`).join(", ")}, ` + + `which ${unknown.length === 1 ? "is not a field" : "are not fields"} of "${obj.name}".`, + { code: "ERR_BAD_ATTR_VALUE", source: v.source }, + )); + continue; + } + const problem = conditionProblem(condition, obj as MetaObject); + if (problem !== undefined) { + errors.push(new ParseError(`${label} ${problem}`, { code: "ERR_BAD_ATTR_VALUE", source: v.source })); + } + } + } + } + return errors; +} + +// --------------------------------------------------------------------------- +// identity.primary @sequenceStart / @sequenceName describe the sequence an @generation +// "increment" key draws from. On any other generation there is no sequence, so either one +// is refused (ERR_SEQUENCE_WITHOUT_INCREMENT) rather than silently ignored. Read RESOLVING +// (ADR-0039), reported on the identity that declares one of them. +// --------------------------------------------------------------------------- + +/** The largest value a field.int key (a 32-bit integer column) can hold. */ +const SEQUENCE_INT_MAX = 2147483647; + +export function validateIdentitySequence(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + for (const id of obj.ownChildren()) { + if (id.type !== TYPE_IDENTITY || id.subType !== IDENTITY_SUBTYPE_PRIMARY) continue; + const declared = [IDENTITY_ATTR_SEQUENCE_START, IDENTITY_ATTR_SEQUENCE_NAME].filter((a) => id.ownAttr(a) !== undefined); + if (declared.length === 0) continue; + if (id.attr(IDENTITY_ATTR_GENERATION) === GENERATION_INCREMENT) { + const start = id.attr(IDENTITY_ATTR_SEQUENCE_START); + if (typeof start === "number") { + // The key field's own range bounds the start (Postgres refuses the sequence + // otherwise, at apply time): 1 to the largest value of a field.int / field.long. + const keyName = [id.attr(IDENTITY_ATTR_FIELDS)].flat()[0]; + const key = (obj as MetaObject).fields().find((f) => f.name === keyName); + const max = key?.subType === FIELD_SUBTYPE_INT ? SEQUENCE_INT_MAX : undefined; + if (start < 1 || (max !== undefined && start > max)) { + errors.push(new ParseError( + `identity.primary "${obj.name}.${id.name}" @sequenceStart ${start} is out of range for its key ` + + `"${String(keyName)}"${key !== undefined ? ` (field.${key.subType})` : ""}: it must be at least 1` + + `${max !== undefined ? ` and at most ${max}` : ""}.`, + { code: "ERR_SEQUENCE_START_OUT_OF_RANGE", source: id.source }, + )); + } + } + continue; + } + const generation = id.attr(IDENTITY_ATTR_GENERATION); + const shown = typeof generation === "string" ? `"${generation}"` : "not set"; + errors.push(new ParseError( + `identity.primary "${obj.name}.${id.name}" sets ${declared.map((a) => `@${a}`).join(" and ")}, but its @generation is ${shown}; ` + + `only an @generation "increment" key draws from a sequence. Set @generation "increment", or remove ${declared.map((a) => `@${a}`).join(" and ")}.`, + { code: "ERR_SEQUENCE_WITHOUT_INCREMENT", source: id.source }, + )); + } + } + return errors; +} + +// --------------------------------------------------------------------------- +// field.enum @maxLength — the column length of a string-stored enum: +// 1. every member of @values fits in it (ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH), the +// longest offender named; +// 2. it is refused beside @intValueMap (ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP): an +// int-stored enum's column is an integer, so a length would change nothing. +// Read RESOLVING (ADR-0039: all three may be inherited through extends), and reported on +// the node that DECLARES one of them, so an enum that only inherits them is not +// reported a second time. Every field.enum in the tree is checked, root-level abstract +// enums included. The project default (enums.maxLength) is not this pass's: the loader +// does not see the project config; resolveEnumMaxLength refuses an overflow of it. +// --------------------------------------------------------------------------- + +export function validateEnumMaxLength(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + const visit = (node: MetaData): void => { + // ADR-0039: own children — every node is visited once, where it is declared. + for (const child of node.ownChildren()) { + if (child.type === TYPE_FIELD && child.subType === FIELD_SUBTYPE_ENUM) checkEnum(child); + if (child.type !== TYPE_FIELD) visit(child); + } + }; + const checkEnum = (field: MetaData): void => { + const declares = [FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP] + .some((a) => field.ownAttr(a) !== undefined); + const maxLength = field.attr(FIELD_ATTR_MAX_LENGTH); + if (!declares || typeof maxLength !== "number") return; + const owner = field.parent; + const label = owner !== undefined && owner.type === TYPE_OBJECT ? `${owner.name}.${field.name}` : field.name; + if (field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined) { + errors.push(new ParseError( + `field.enum "${label}" sets @maxLength ${maxLength} and @intValueMap; an @intValueMap enum is stored as an integer, so a column length does not apply. Remove @maxLength.`, + { code: "ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP", source: field.source }, + )); + return; + } + const values = field.attr(FIELD_ATTR_VALUES); + const tooLong = (Array.isArray(values) ? values : []) + .filter((v): v is string => typeof v === "string" && v.length > maxLength) + .sort((a, b) => b.length - a.length); + if (tooLong.length > 0) { + errors.push(new ParseError( + `field.enum "${label}" @maxLength ${maxLength} is shorter than its member "${tooLong[0]}" (${tooLong[0]!.length} characters). Raise @maxLength to at least ${tooLong[0]!.length}, or shorten the member.`, + { code: "ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH", source: field.source }, + )); + } + }; + visit(root); + return errors; +} + // --------------------------------------------------------------------------- // field.object + @storage cross-attribute validation // @@ -1696,6 +1950,8 @@ export function validateDerivedFieldProvidability(root: MetaData): ParseError[] // node: when @objectRef is absent we skip the flattened/array check below.) // 2. @storage "flattened" requires isArray to be absent or false (cannot // flatten a variable-length array into a fixed column set). +// 3. @flattenedColumnPrefix requires @storage "flattened" (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE): +// no other storage gives the members columns of their own to prefix. // --------------------------------------------------------------------------- export function validateFieldObjectStorage(root: MetaData): ParseError[] { @@ -1724,6 +1980,17 @@ export function validateFieldObjectStorage(root: MetaData): ParseError[] { // ADR-0039: resolving — @storage and array-ness may be inherited via extends. const storage = field.attr(FIELD_ATTR_STORAGE); + // @flattenedColumnPrefix names the columns a flattened value object adds to the owning table. + // Under any other storage (absent is jsonb) there are no such columns, so the prefix + // would change nothing: refuse it rather than ignore it silently. + if (field.attr(FIELD_ATTR_FLATTENED_COLUMN_PREFIX) !== undefined && storage !== STORAGE_FLATTENED) { + errors.push( + new ParseError( + `field.object "${obj.name}.${field.name}" sets @flattenedColumnPrefix, but its @storage is ${storage === undefined || storage === null ? "not set (jsonb)" : `"${String(storage)}"`}; only a @storage "flattened" value object puts its members' columns in the owning table. Set @storage "flattened", or remove @flattenedColumnPrefix.`, + { code: "ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE", source: field.source }, + ), + ); + } if (storage === undefined || storage === null) continue; if (storage === STORAGE_FLATTENED && field.resolvedIsArray()) { errors.push( @@ -2775,8 +3042,12 @@ export function validateRetiredRequirementLinks(root: MetaData): ParseError[] { // is compared, through the single door `resolveColumnName`, so an explicit @column // override, an inherited field and the project's column strategy are all honoured. // Scope: a concrete object that declares at least one source (a source-less -// object generates nothing); derived fields (origin.*) and field.object (which may -// flatten to several columns) are not columns of this table and are skipped. +// object generates nothing); derived fields (origin.*) are not columns of this table +// and are skipped. A `field.object @storage "flattened"` has no column of its own: each +// member of its value object is a column, named by `resolveFlattenedColumnName` exactly +// as migrate creates it, so two embeddings with one `@flattenedColumnPrefix` (or a `""` +// prefix beside a field of the member's name) collide here. Any other field.object is +// one jsonb column under its own resolved name. // ADR-0039: effective fields, so an inherited field collides like an own one. // Exempt: an object whose every source is read-only and at least one is a // `source.rdb @kind: view` with an `@sql` body. The author's SQL defines that view's @@ -2799,31 +3070,66 @@ function hasExplicitColumn(field: MetaData): boolean { return typeof col === "string" && col !== ""; } +/** One physical column a field contributes to its object's table. */ +interface PhysicalColumn { + readonly column: string; + /** `street`, or `shipTo.street` for a member of a flattened value object. */ + readonly label: string; + /** The field that names it: the field itself, or the flattened value object's member. */ + readonly named: MetaData; + /** Set for a flattened member: the embedding field.object. */ + readonly embedding?: MetaData; +} + +function physicalColumnsOf(field: MetaData): PhysicalColumn[] { + // ADR-0039: resolving — @storage and @objectRef may be inherited via extends. + if (field.subType === FIELD_SUBTYPE_OBJECT && field.attr(FIELD_ATTR_STORAGE) === STORAGE_FLATTENED) { + const target = refTarget(field, FIELD_ATTR_OBJECT_REF); + if (target === undefined || !isMetaObject(target)) return []; + // Every member, as migrate's flattenObjectField expands it. + return target.fields().map((member) => ({ + column: resolveFlattenedColumnName(field, member), + label: `${field.name}.${member.name}`, + named: member, + embedding: field, + })); + } + return [{ column: resolveColumnName(field), label: field.name, named: field }]; +} + +function collisionHint(a: PhysicalColumn, b: PhysicalColumn): string { + if (a.embedding !== undefined || b.embedding !== undefined) { + return "Give one flattened field a different @flattenedColumnPrefix, or rename one field."; + } + return hasExplicitColumn(a.named) && hasExplicitColumn(b.named) + ? "Rename one field, or point one @column at a different column." + : "Rename one field, or give one an explicit @column so the two columns differ."; +} + export function validateColumnNameCollisions(root: MetaData): ParseError[] { const errors: ParseError[] = []; for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { if (obj.isAbstract) continue; if (!obj.children().some((c) => c.type === TYPE_SOURCE)) continue; if (readsThroughSqlView(obj)) continue; - const seen = new Map(); + const seen = new Map(); for (const field of obj.children()) { - if (!isMetaField(field) || field.subType === FIELD_SUBTYPE_OBJECT || field.isDerived()) continue; - const column = resolveColumnName(field); - const first = seen.get(column); - if (first === undefined) { - seen.set(column, field); - continue; + if (!isMetaField(field) || field.isDerived()) continue; + for (const col of physicalColumnsOf(field)) { + const first = seen.get(col.column); + if (first === undefined) { + seen.set(col.column, col); + continue; + } + errors.push( + new ParseError( + `fields "${first.label}" and "${col.label}" on "${obj.name}" both resolve to physical column ` + + `"${col.column}". ` + + collisionHint(first, col), + { code: "ERR_COLUMN_NAME_COLLISION", source: field.source }, + ), + ); } - errors.push( - new ParseError( - `fields "${first.name}" and "${field.name}" on "${obj.name}" both resolve to physical column ` + - `"${column}". ` + - (hasExplicitColumn(first) && hasExplicitColumn(field) - ? "Rename one field, or point one @column at a different column." - : "Rename one field, or give one an explicit @column so the two columns differ."), - { code: "ERR_COLUMN_NAME_COLLISION", source: field.source }, - ), - ); } } return errors; diff --git a/server/typescript/packages/metadata/src/naming.ts b/server/typescript/packages/metadata/src/naming.ts index 794335465f..6e1f7c2e25 100644 --- a/server/typescript/packages/metadata/src/naming.ts +++ b/server/typescript/packages/metadata/src/naming.ts @@ -2,7 +2,7 @@ import { docsBlobUrl } from "./docs-ref.js"; import type { MetaData } from "./shared/meta-data.js"; import { TYPE_FIELD } from "./shared/base-types.js"; import { PACKAGE_SEPARATOR } from "./shared/structural.js"; -import { FIELD_ATTR_COLUMN } from "./persistence/db/db-constants.js"; +import { FIELD_ATTR_COLUMN, FIELD_ATTR_FLATTENED_COLUMN_PREFIX } from "./persistence/db/db-constants.js"; import { SOURCE_ATTR_SCHEMA, SOURCE_ROLE_PRIMARY, @@ -53,6 +53,22 @@ export function toKebabCase(s: string): string { return toSnakeCase(s).replace(/_/g, "-"); } +/** + * The `upperFirst` column rule: the first letter upper-cased, nothing else changed + * (`createdAt` → `CreatedAt`, `requestURL` → `RequestURL`, `address_line1` → + * `Address_line1`). Not a word split: every other character keeps the case it was written + * in, so a model maps onto a schema whose columns are its field names with a capital first + * letter (EF Core's default, for one) with no `@column`. + */ +export function toUpperFirstColumn(s: string): string { + // One UTF-16 unit in, one out — as C#'s char.ToUpperInvariant and Java's + // Character.toUpperCase do — so `ß` (whose JavaScript upper case is "SS") stays `ß` in + // every port. + const first = s.charAt(0); + const upper = first.toUpperCase(); + return (upper.length === 1 ? upper : first) + s.slice(1); +} + /** * How a field with no `@column` becomes a column name. One project setting, * `naming.columns` in `.metaobjects/config.json`, read by every port (ADR-0058): the @@ -60,10 +76,10 @@ export function toKebabCase(s: string): string { * and verify resolve the same name. Not a metadata attr: `@column` is the per-field * override, this is the rule for the rest. */ -export type ColumnNamingStrategy = "snake_case" | "literal" | "kebab-case"; +export type ColumnNamingStrategy = "snake_case" | "literal" | "kebab-case" | "upperFirst"; /** Every column-naming rule, in the order the conformance corpus lists them. */ -export const COLUMN_NAMING_STRATEGIES: readonly ColumnNamingStrategy[] = ["snake_case", "literal", "kebab-case"]; +export const COLUMN_NAMING_STRATEGIES: readonly ColumnNamingStrategy[] = ["snake_case", "literal", "kebab-case", "upperFirst"]; /** The default in EVERY port: what `meta migrate` has always created. */ export const DEFAULT_COLUMN_NAMING_STRATEGY: ColumnNamingStrategy = "snake_case"; @@ -134,6 +150,7 @@ export function applyColumnNamingStrategy(name: string, strategy: ColumnNamingSt case "literal": return name; case "kebab-case": return toKebabCase(name); case "snake_case": return toSnakeCase(name); + case "upperFirst": return toUpperFirstColumn(name); } } @@ -316,6 +333,40 @@ export function resolveColumnName( return applyColumnNamingStrategy(field.name, strategy); } +/** + * The text before each member's column when `field` (a `field.object @storage "flattened"`) + * expands its value object into the owning table: its `@flattenedColumnPrefix` verbatim (`""` allowed), + * else its own resolved column and `_`. With `@column: "bill"` the members are + * `bill_`; under the `upperFirst` rule an `address` field gives `Address_`, + * which is EF Core's owned-type default. + * + * Core naming, not migrate's: generated ORM mappings (the Prisma schema, the C# EF mapping, + * the Kotlin Exposed tables) address these columns too, and must spell them exactly as the + * migration creates them. + * ADR-0039: resolving — `@flattenedColumnPrefix` and `@column` may be inherited through `extends`. + */ +export function resolveFlattenedColumnPrefix( + field: MetaData, + strategy: ColumnNamingStrategy = physicalNamingOf(field).columns, +): string { + const prefix = field.attr(FIELD_ATTR_FLATTENED_COLUMN_PREFIX); + if (typeof prefix === "string") return prefix; + return resolveColumnName(field, strategy) + "_"; +} + +/** + * The column a member of a flattened value object gets in the owning table: the embedding + * field's {@link resolveFlattenedColumnPrefix} followed by the member's own resolved column + * (its `@column`, else its name under the rule). + */ +export function resolveFlattenedColumnName( + field: MetaData, + member: MetaData, + strategy: ColumnNamingStrategy = physicalNamingOf(field).columns, +): string { + return resolveFlattenedColumnPrefix(field, strategy) + resolveColumnName(member, strategy); +} + /** * An index's DATABASE name — for an `identity.secondary` (a unique alternate key) or an * `index.lookup` (a non-unique retrieval index). diff --git a/server/typescript/packages/metadata/src/parser-core.ts b/server/typescript/packages/metadata/src/parser-core.ts index fc3798e939..fa9af783cc 100644 --- a/server/typescript/packages/metadata/src/parser-core.ts +++ b/server/typescript/packages/metadata/src/parser-core.ts @@ -65,6 +65,7 @@ import { PACKAGE_SEPARATOR, } from "./shared/structural.js"; import { ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_STRINGARRAY } from "./core/attr/attr-constants.js"; +import { FIELD_ATTR_FLATTENED_COLUMN_PREFIX } from "./persistence/db/db-constants.js"; import { attrClassFor } from "./attr-class-map.js"; import type { AttrValue } from "./shared/meta-data.js"; @@ -1041,6 +1042,9 @@ function parseNodeInto( } } +/** Attrs whose empty string is a meaningful value rather than "not set" (FR5c). */ +const EMPTY_STRING_IS_A_VALUE: ReadonlySet = new Set([FIELD_ATTR_FLATTENED_COLUMN_PREFIX]); + /** FR5c — for every @-prefixed inline attr in `nodeData`, check whether the * target already declares the same attr (in `preAttrs`) with a different * non-empty value. If so, emit ERR_MERGE_CONFLICT with a `format: "merged"` @@ -1060,9 +1064,11 @@ function detectAttrMergeConflicts( const existingFiles = "files" in target.source ? [...target.source.files] : []; - function isEmptyValue(v: unknown): boolean { + function isEmptyValue(v: unknown, attrName: string): boolean { if (v === undefined || v === null) return true; - if (typeof v === "string" && v === "") return true; + // `@flattenedColumnPrefix: ""` is a value (keep the members' own columns), not "unset", + // so `""` in one file and `"home_"` in another is a conflict, not last-loaded-wins. + if (typeof v === "string" && v === "") return !EMPTY_STRING_IS_A_VALUE.has(attrName); if (Array.isArray(v) && v.length === 0) return true; return false; } @@ -1112,7 +1118,7 @@ function detectAttrMergeConflicts( const existingVal = preAttrs.get(attrName); if (existingVal === undefined) continue; - if (isEmptyValue(newValRaw) || isEmptyValue(existingVal)) continue; + if (isEmptyValue(newValRaw, attrName) || isEmptyValue(existingVal, attrName)) continue; // Both sides set a non-empty value — compare. If equal, last-writer-wins // is a no-op; if different, that's a conflict. if (attrValuesEqual(existingVal, newValRaw as AttrValue)) continue; diff --git a/server/typescript/packages/metadata/src/persistence/db/db-constants.ts b/server/typescript/packages/metadata/src/persistence/db/db-constants.ts index 1f2563f5ba..3ec3dfcdf5 100644 --- a/server/typescript/packages/metadata/src/persistence/db/db-constants.ts +++ b/server/typescript/packages/metadata/src/persistence/db/db-constants.ts @@ -6,6 +6,12 @@ import { /** Column name override on a field (maps to @column in metadata). */ export const FIELD_ATTR_COLUMN = "column"; +/** + * `@flattenedColumnPrefix` on a `field.object @storage "flattened"`: the text before each member's + * column, verbatim (`""` allowed). Absent → the field's resolved column and `_` + * ({@link resolveFlattenedColumnPrefix} in naming.ts). + */ +export const FIELD_ATTR_FLATTENED_COLUMN_PREFIX = "flattenedColumnPrefix"; /** When true, suppress the @filterable-without-index Loader warning (Project D drift check). */ export const FIELD_ATTR_DB_INDEXED = "db.indexed"; @@ -24,6 +30,10 @@ export const IDENTITY_ATTR_EXPR = "expr"; export const IDENTITY_ATTR_USING = "using"; /** identity.reference: physical FK constraint-name override. Absent → the auto-derived `
__fk`. */ export const IDENTITY_ATTR_CONSTRAINT_NAME = "constraintName"; +/** identity.primary (@generation "increment"): the sequence's first value. Absent → 1. */ +export const IDENTITY_ATTR_SEQUENCE_START = "sequenceStart"; +/** identity.primary (@generation "increment"): the sequence's physical name. Absent → the database's default. */ +export const IDENTITY_ATTR_SEQUENCE_NAME = "sequenceName"; /** * R6 Plan 2b: `@dbColumnType` — physical DB column-type override on a field. diff --git a/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts b/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts index 2b6f014b24..91d4fe284f 100644 --- a/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts @@ -61,6 +61,14 @@ export const DB_DEFINITION: ProviderDefinition = { "type": "field", "subType": "object", "children": [ + { + "type": "attr", + "subType": "string", + "name": "flattenedColumnPrefix", + "min": 0, + "max": 1, + "description": "On a field.object with @storage \"flattened\": the text put before each member's column, verbatim (\"\" allowed, so the members keep their own column names). Absent: this field's resolved column name and \"_\" (with @column \"bill\", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1." + }, { "type": "attr", "subType": "string", @@ -246,6 +254,65 @@ export const DB_DEFINITION: ProviderDefinition = { } ] }, + { + "type": "field", + "subType": "enum", + "children": [ + { + "type": "attr", + "subType": "string", + "name": "constraintName", + "min": 0, + "max": 1, + "description": "The database name of the CHECK constraint this enum's member list generates. Absent:
__chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + } + ] + }, + { + "type": "validator", + "subType": [ + "numeric", + "length", + "regex", + "comparison", + "requiredWhen", + "presentIff", + "atLeastOne", + "exactlyOne" + ], + "children": [ + { + "type": "attr", + "subType": "string", + "name": "constraintName", + "min": 0, + "max": 1, + "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + } + ] + }, + { + "type": "identity", + "subType": "primary", + "children": [ + { + "type": "attr", + "subType": "long", + "name": "sequenceStart", + "min": 0, + "max": 1, + "description": "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + }, + { + "type": "attr", + "subType": "string", + "name": "sequenceName", + "min": 0, + "max": 1, + "description": "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
__seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." + } + ] + }, { "type": "identity", "subType": "reference", diff --git a/server/typescript/packages/metadata/src/registry-manifest.ts b/server/typescript/packages/metadata/src/registry-manifest.ts index 3f1acdfe96..4fa5345bce 100644 --- a/server/typescript/packages/metadata/src/registry-manifest.ts +++ b/server/typescript/packages/metadata/src/registry-manifest.ts @@ -112,7 +112,7 @@ export interface ManifestType { * constant read `"0.10"`). Bump with that script — never by hand — so the manifest and * all four port constants move together. */ -export const METAMODEL_VERSION = "1.2"; +export const METAMODEL_VERSION = "1.3"; /** The full canonical manifest. All collections are sorted for byte-stability. */ interface RegistryManifest { diff --git a/server/typescript/packages/metadata/src/shared/meta-root.ts b/server/typescript/packages/metadata/src/shared/meta-root.ts index 3fd83218a8..feaabdcfe6 100644 --- a/server/typescript/packages/metadata/src/shared/meta-root.ts +++ b/server/typescript/packages/metadata/src/shared/meta-root.ts @@ -9,6 +9,7 @@ import type { MetaObject } from "../core/object/meta-object.js"; import type { MetaField } from "../core/field/meta-field.js"; import { DEFAULT_PHYSICAL_NAMING, type PhysicalNaming } from "../naming.js"; import { DEFAULT_WIRE_FORMAT, type WireFormat } from "../wire.js"; +import { DEFAULT_ENUM_DEFAULTS, type EnumDefaults } from "../enum-length.js"; import { PACKAGE_SEPARATOR } from "./structural.js"; import { lookupCanonical } from "../naming-refs.js"; @@ -26,6 +27,12 @@ export class MetaRoot extends MetaData { */ wireFormat: WireFormat = DEFAULT_WIRE_FORMAT; + /** + * The project's enum storage defaults (`enums` in `.metaobjects/config.json`), set by the + * project-aware load beside `physicalNaming` and read through `enumDefaultsOf`. + */ + enumDefaults: EnumDefaults = DEFAULT_ENUM_DEFAULTS; + /** Object entities defined at this root level. */ objects(): MetaObject[] { return this.cached("objects", () => diff --git a/server/typescript/packages/metadata/src/wire.ts b/server/typescript/packages/metadata/src/wire.ts index 1b7d8bbdac..d107f39ba7 100644 --- a/server/typescript/packages/metadata/src/wire.ts +++ b/server/typescript/packages/metadata/src/wire.ts @@ -15,25 +15,39 @@ export type WireDecimals = (typeof WIRE_DECIMALS)[number]; export const WIRE_NULL_MEMBERS = ["native", "emit", "omit"] as const; export type WireNullMembers = (typeof WIRE_NULL_MEMBERS)[number]; +/** + * `wire.durations`: how a `field.duration` is written and read. `native` = the language's own + * serializer for its duration type (System.Text.Json's `01:30:00` for a C# `TimeSpan`); `iso` = + * canonical ISO-8601 (`PT1H30M`) in every port. TypeScript has no duration type, so it writes + * ISO-8601 either way. + */ +export const WIRE_DURATIONS = ["native", "iso"] as const; +export type WireDurations = (typeof WIRE_DURATIONS)[number]; + /** The keys of the `wire` block. */ export const WIRE_KEY_DECIMALS = "decimals"; export const WIRE_KEY_NULL_MEMBERS = "nullMembers"; +export const WIRE_KEY_DURATIONS = "durations"; export const WIRE_DECIMALS_NATIVE: WireDecimals = "native"; export const WIRE_DECIMALS_STRING: WireDecimals = "string"; export const WIRE_NULL_MEMBERS_NATIVE: WireNullMembers = "native"; export const WIRE_NULL_MEMBERS_EMIT: WireNullMembers = "emit"; export const WIRE_NULL_MEMBERS_OMIT: WireNullMembers = "omit"; +export const WIRE_DURATIONS_NATIVE: WireDurations = "native"; +export const WIRE_DURATIONS_ISO: WireDurations = "iso"; export interface WireFormat { readonly decimals: WireDecimals; readonly nullMembers: WireNullMembers; + readonly durations: WireDurations; } /** The default in every port: each language's own standard output. */ export const DEFAULT_WIRE_FORMAT: WireFormat = Object.freeze({ decimals: WIRE_DECIMALS_NATIVE, nullMembers: WIRE_NULL_MEMBERS_NATIVE, + durations: WIRE_DURATIONS_NATIVE, }); /** The wire format for the tree `node` belongs to: what its root carries, else the default. */ diff --git a/server/typescript/packages/metadata/test/enum-max-length.test.ts b/server/typescript/packages/metadata/test/enum-max-length.test.ts new file mode 100644 index 0000000000..0145c4dd2e --- /dev/null +++ b/server/typescript/packages/metadata/test/enum-max-length.test.ts @@ -0,0 +1,70 @@ +// A string-stored field.enum's column length: its @maxLength, else the project's +// enums.maxLength (carried on the loaded root, like the naming rules), else none (an +// unbounded text column, as before). Every member must fit the length that applies; the +// loader checks an explicit @maxLength, and the resolver refuses a project default that a +// member overflows, so every consumer that sizes the column refuses the same model. + +import { describe, expect, test } from "bun:test"; +import { + MetaDataLoader, + InMemoryStringSource, + MetaModelError, + resolveEnumMaxLength, + type MetaField, + type MetaObject, + type MetaRoot, +} from "../src/index.js"; + +async function load(statusDecl: Record): Promise { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Order", children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", ...statusDecl } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + ] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toEqual([]); + return root as MetaRoot; +} + +const status = (root: MetaRoot): MetaField => + (root.findObject("Order") as MetaObject).fields().find((f) => f.name === "status")!; + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; + +describe("resolveEnumMaxLength", () => { + test("@maxLength is the column length", async () => { + expect(resolveEnumMaxLength(status(await load({ "@values": VALUES, "@maxLength": 20 })))).toBe(20); + }); + + test("no @maxLength and no project default: none", async () => { + expect(resolveEnumMaxLength(status(await load({ "@values": VALUES })))).toBeUndefined(); + }); + + test("the project default applies when the field declares none", async () => { + const root = await load({ "@values": VALUES }); + root.enumDefaults = { maxLength: 32 }; + expect(resolveEnumMaxLength(status(root))).toBe(32); + }); + + test("@maxLength beats the project default, either way round", async () => { + const root = await load({ "@values": VALUES, "@maxLength": 9 }); + root.enumDefaults = { maxLength: 32 }; + expect(resolveEnumMaxLength(status(root))).toBe(9); + }); + + test("an @intValueMap enum has no length, whatever the default", async () => { + const root = await load({ "@values": VALUES, "@intValueMap": { DRAFT: 0, PUBLISHED: 1, ARCHIVED: 2 } }); + root.enumDefaults = { maxLength: 4 }; + expect(resolveEnumMaxLength(status(root))).toBeUndefined(); + }); + + test("a member longer than the project default is refused, naming the field and the member", async () => { + const root = await load({ "@values": ["DRAFT", "AWAITING_REVIEW"] }); + root.enumDefaults = { maxLength: 9 }; + expect(() => resolveEnumMaxLength(status(root))).toThrow(MetaModelError); + expect(() => resolveEnumMaxLength(status(root))).toThrow(/"Order\.status".*"AWAITING_REVIEW".*enums\.maxLength 9/); + }); +}); diff --git a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts index 3c6a86f7fa..7d95baeaaf 100644 --- a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts +++ b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts @@ -12,7 +12,7 @@ // decimal → @precision, @scale // object → @objectRef // currency → @currency -// enum → @values, @provided, @intValueMap +// enum → @values, @provided, @intValueMap, @maxLength // int/long/double/float/boolean/date/time/timestamp/uuid → none // - the "any attr" wildcard is gone; a misplaced attr is now ERR_UNKNOWN_ATTR. // @@ -104,11 +104,19 @@ const ENUM_CORE_EXTRA: Record = { values: { valueType: "string", required: true }, provided: { valueType: "boolean", required: false }, intValueMap: { valueType: "intMap", required: false }, + // The column length of a string-stored enum (refused beside @intValueMap at load). + maxLength: { valueType: "int", required: false }, +}; + +// db provider: the name of the CHECK an enum's member list generates. +const ENUM_DB_EXTRA: Record = { + constraintName: { valueType: "string", required: false }, }; /** Scoped concern attrs (db / prompt). */ const STORAGE_EXTRA: Record = { storage: { valueType: "string", required: false }, + flattenedColumnPrefix: { valueType: "string", required: false }, }; const AUTO_SET_EXTRA: Record = { autoSet: { valueType: "string", required: false }, @@ -154,6 +162,8 @@ const EXPECTED_DATA_TYPE: Record = { // ADR-0036/0037 Wave 3 — uri/inet bind to TS string. uri: "string", inet: "string", + // a fixed-length duration binds to TS string (an ISO-8601 duration). + duration: "string", }; function expectedAttrsFor(subType: string): Record { @@ -163,7 +173,7 @@ function expectedAttrsFor(subType: string): Record { if (subType === FIELD_SUBTYPE_OBJECT) Object.assign(exp, OBJECT_CORE_EXTRA, STORAGE_EXTRA); if (subType === FIELD_SUBTYPE_MAP) Object.assign(exp, MAP_EXTRA); if (subType === FIELD_SUBTYPE_CURRENCY) Object.assign(exp, CURRENCY_EXTRA); - if (subType === FIELD_SUBTYPE_ENUM) Object.assign(exp, ENUM_CORE_EXTRA, ENUM_PROMPT_EXTRA); + if (subType === FIELD_SUBTYPE_ENUM) Object.assign(exp, ENUM_CORE_EXTRA, ENUM_PROMPT_EXTRA, ENUM_DB_EXTRA); if (TEMPORAL.has(subType)) Object.assign(exp, AUTO_SET_EXTRA); if (subType === FIELD_SUBTYPE_TIMESTAMP) Object.assign(exp, LOCAL_TIME_EXTRA); if (subType === "uri" || subType === "inet") Object.assign(exp, LENIENT_EXTRA); @@ -171,7 +181,7 @@ function expectedAttrsFor(subType: string): Record { } describe("field provider externalization — strict per-subtype completeness", () => { - test("registers all 18 field subtypes", () => { + test("registers all 19 field subtypes", () => { const registered = registry.allSubTypesOf(TYPE_FIELD).sort(); expect(registered).toEqual([...FIELD_SUBTYPES].sort()); }); diff --git a/server/typescript/packages/metadata/test/naming.test.ts b/server/typescript/packages/metadata/test/naming.test.ts index 73bda32885..565b772afd 100644 --- a/server/typescript/packages/metadata/test/naming.test.ts +++ b/server/typescript/packages/metadata/test/naming.test.ts @@ -1,4 +1,5 @@ import { describe, it, expect } from "bun:test"; +import { toUpperFirstColumn } from "../src/naming.js"; import { readFileSync } from "node:fs"; import { dirname, resolve } from "node:path"; import { fileURLToPath } from "node:url"; @@ -371,3 +372,17 @@ describe("fixtures/naming-conformance/already-plural-pluralize.json", () => { }, ); }); + +describe("toUpperFirstColumn", () => { + it("upper-cases the first letter only", () => { + expect(toUpperFirstColumn("requestURL")).toBe("RequestURL"); + expect(toUpperFirstColumn("")).toBe(""); + }); + + // C#'s char.ToUpperInvariant and Java's Character.toUpperCase map one character to one + // character, so `ß` stays `ß` there; JavaScript's toUpperCase gives "SS". Every port keeps a + // first character whose upper case is not a single character, so all five name it alike. + it("keeps a first character whose upper case is not one character", () => { + expect(toUpperFirstColumn("ßeta")).toBe("ßeta"); + }); +}); diff --git a/server/typescript/packages/metadata/test/validator-definition-completeness.test.ts b/server/typescript/packages/metadata/test/validator-definition-completeness.test.ts index 0e2c247254..dd339e1ff8 100644 --- a/server/typescript/packages/metadata/test/validator-definition-completeness.test.ts +++ b/server/typescript/packages/metadata/test/validator-definition-completeness.test.ts @@ -41,19 +41,27 @@ const EXPECTED: Record "b" OR …`, not `NOT ("a" = "b") OR …`. +// +// The negation is exact in SQL's three-valued logic: a flipped comparison is NULL exactly when +// the original is, isNull/isNotNull swap, and/or follow De Morgan. A field resolves to its +// physical column through the caller's lookup; a field the lookup does not know refuses the +// render (the loader refuses that model first, ERR_BAD_ATTR_VALUE). + +import { + EXPR_FN_COALESCE, + EXPR_OP_IS_NOT_NULL, + EXPR_OP_NOT, + FILTER_COMPOSE_AND, + FILTER_COMPOSE_OR, + FILTER_OP_EQ, + FILTER_OP_GT, + FILTER_OP_GTE, + FILTER_OP_IS_NULL, + FILTER_OP_LT, + FILTER_OP_LTE, + FILTER_OP_NE, + type ExprNode, +} from "@metaobjectsdev/metadata"; +import { isSqliteFamily, type Dialect } from "./types.js"; + +const SQL_OP: Readonly> = { + [FILTER_OP_EQ]: "=", [FILTER_OP_NE]: "<>", [FILTER_OP_GT]: ">", + [FILTER_OP_GTE]: ">=", [FILTER_OP_LT]: "<", [FILTER_OP_LTE]: "<=", +}; + +const NEGATED_OP: Readonly> = { + [FILTER_OP_EQ]: FILTER_OP_NE, [FILTER_OP_NE]: FILTER_OP_EQ, + [FILTER_OP_GT]: FILTER_OP_LTE, [FILTER_OP_LTE]: FILTER_OP_GT, + [FILTER_OP_LT]: FILTER_OP_GTE, [FILTER_OP_GTE]: FILTER_OP_LT, + [FILTER_OP_IS_NULL]: EXPR_OP_IS_NOT_NULL, [EXPR_OP_IS_NOT_NULL]: FILTER_OP_IS_NULL, +}; + +type Node = Record; + +/** The logical negation of `node`, pushed down to its comparisons. */ +export function negateExpression(node: ExprNode): ExprNode { + const n = node as Node; + const op = n["op"]; + if (op === EXPR_OP_NOT) return n["arg"] as ExprNode; + if (op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) { + return { + op: op === FILTER_COMPOSE_AND ? FILTER_COMPOSE_OR : FILTER_COMPOSE_AND, + args: (n["args"] as ExprNode[]).map(negateExpression), + } as ExprNode; + } + if (typeof op === "string" && NEGATED_OP[op] !== undefined) { + return { ...n, op: NEGATED_OP[op] } as ExprNode; + } + return { op: EXPR_OP_NOT, arg: node } as ExprNode; +} + +/** + * How a literal compared with a field is stored in that field's column, when that is not the + * literal itself: an int-backed enum (`@intValueMap`) stores a member as its integer, and a + * duration is an interval or a microsecond integer. Answers the + * SQL for the stored value, or undefined to render the literal as written. + */ +export type StoredLiteral = (field: string, value: unknown) => string | undefined; + +/** A node SQL can use as a comparison operand without parentheses. */ +function isAtom(x: unknown): boolean { + const n = x as Node; + return typeof n["field"] === "string" || "value" in n || n["fn"] !== undefined; +} + +/** + * `node` as SQL. `column(field)` is the quoted physical column for a field name, or undefined + * when the entity has no such field. A compound operand of a comparison or a null test is + * parenthesised: `IS` binds looser than `=`, and `AND` / `OR` looser than both. + */ +export function renderExpressionSql( + node: ExprNode, column: (field: string) => string | undefined, dialect: Dialect | undefined, + stored?: StoredLiteral, +): string { + const n = node as Node; + if (typeof n["field"] === "string") { + const col = column(n["field"]); + if (col === undefined) throw new Error(`@condition names "${n["field"]}", which is not a field of the entity`); + return col; + } + if ("value" in n) return renderLiteral(n["value"], dialect); + const op = n["op"]; + const sub = (x: unknown): string => renderExpressionSql(x as ExprNode, column, dialect, stored); + const operand = (x: unknown, other: unknown): string => { + const field = storedFieldOf(other); + if (stored !== undefined && "value" in (x as Node) && field !== undefined) { + const sql = stored(field, (x as Node)["value"]); + if (sql !== undefined) return sql; + } + return isAtom(x) ? sub(x) : `(${sub(x)})`; + }; + if (typeof op === "string" && SQL_OP[op] !== undefined) { + return `${operand(n["left"], n["right"])} ${SQL_OP[op]} ${operand(n["right"], n["left"])}`; + } + if (op === FILTER_OP_IS_NULL) return `${operand(n["arg"], undefined)} IS NULL`; + if (op === EXPR_OP_IS_NOT_NULL) return `${operand(n["arg"], undefined)} IS NOT NULL`; + if (op === EXPR_OP_NOT) return `NOT (${sub(n["arg"])})`; + if (op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) { + return (n["args"] as unknown[]).map((a) => `(${sub(a)})`).join(op === FILTER_COMPOSE_AND ? " AND " : " OR "); + } + if (n["fn"] === EXPR_FN_COALESCE) return `COALESCE(${(n["args"] as unknown[]).map(sub).join(", ")})`; + throw new Error(`@condition: an expression node migrate cannot render: ${JSON.stringify(node)}`); +} + +/** + * The field whose column form a literal compared with `node` takes: the field itself, or for a + * `coalesce` the first field among its arguments (a coalesce of fields has their column type). + */ +function storedFieldOf(node: unknown): string | undefined { + const n = (node ?? {}) as Node; + if (typeof n["field"] === "string") return n["field"]; + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + for (const a of n["args"] as unknown[]) { + const f = storedFieldOf(a); + if (f !== undefined) return f; + } + } + return undefined; +} + +function renderLiteral(v: unknown, dialect: Dialect | undefined): string { + if (v === null) return "NULL"; + if (typeof v === "boolean") return isSqliteFamily(dialect) ? (v ? "1" : "0") : (v ? "TRUE" : "FALSE"); + if (typeof v === "number") return String(v); + return `'${String(v).replace(/'/g, "''")}'`; +} diff --git a/server/typescript/packages/migrate-ts/src/diff/index.ts b/server/typescript/packages/migrate-ts/src/diff/index.ts index ef9640c4b8..bdd995519d 100644 --- a/server/typescript/packages/migrate-ts/src/diff/index.ts +++ b/server/typescript/packages/migrate-ts/src/diff/index.ts @@ -4,7 +4,9 @@ import type { DependentRelation, Change, ChangeStatus, DiffResult, AllowOptions, AmbiguousCallback, Dialect, CheckDescriptor, ColumnDefault, NameChange, DeclaredRename, ExprCanonPath, + SchemaAdvisory, } from "../types.js"; +import { PG_IDENTIFIER_MAX_BYTES } from "../types.js"; import type { SqlType } from "../sql-type.js"; import { sqlTypeEquals } from "../sql-type.js"; import { applyStatus } from "./status.js"; @@ -281,12 +283,50 @@ export async function diff( const { renamed, predecessors } = diffRenamedTables(changes, expectedTables, actualTables, args.dialect, canon); // Pass 2: tables in both → compare columns/indexes/FKs + const demotedKeys: { table: TableDescriptor; columns: readonly string[] }[] = []; + const renamedConstraints = new Map>(); for (const [id, expectedTable] of expectedTables) { const actualTable = actualTables.get(id); if (!actualTable) continue; // An FK onto a renamed table follows the rename in the engine — compare it as such. - diffTablePair(expectedTable, withRenamedRefTables(actualTable, renamed), changes, args.dialect, canon); + if (args.dialect !== "postgres") { + diffTablePair(expectedTable, withRenamedRefTables(actualTable, renamed), changes, args.dialect, canon); + continue; + } + // Postgres: a constraint or index whose only change is its name is renamed in place, and so + // is a primary key. A unique key that changed only between index and constraint is promoted + // in place, or demoted by drop and re-create. + const tableChanges: Change[] = []; + diffTablePair(expectedTable, withRenamedRefTables(actualTable, renamed), tableChanges, args.dialect, canon); + const { constraintRenames, indexRenames, rest } = pairNameOnlyChanges(tableChanges, textColumnsOf(actualTable), canon); + const pk = primaryKeyRename(expectedTable, actualTable, actualTable.name); + if (pk !== undefined) constraintRenames.push(pk); + // A renamed key is compared for its kind under the name it is renamed to: a live UNIQUE + // constraint renamed onto a key the conventions want as a plain index is renamed AND + // demoted, in one migration. The rename runs first, so the demotion uses the new name. + const { promotions, demotions } = uniqueKindChanges( + expectedTable, withRenamedIndexes(actualTable, [...constraintRenames, ...indexRenames]), canon); + if (constraintRenames.length > 0) renamedConstraints.set(id, new Map(constraintRenames.map((r) => [r.from, r.to]))); + if (constraintRenames.length > 0 || indexRenames.length > 0 || promotions.length > 0) { + changes.push({ + kind: "rename-constraints", table: expectedTable.name, ...schemaSpread(expectedTable.schema), + ...(constraintRenames.length > 0 ? { constraintRenames } : {}), + ...(indexRenames.length > 0 ? { indexRenames } : {}), + ...(promotions.length > 0 ? { uniquePromotions: promotions } : {}), + status: ALLOWED, + }); + } + changes.push(...rest); + for (const d of demotions) { + const sx = schemaSpread(expectedTable.schema); + changes.push({ kind: "drop-index", table: expectedTable.name, ...sx, index: d.live.name, restore: d.live, status: ALLOWED }); + changes.push({ kind: "add-index", table: expectedTable.name, ...sx, index: d.want, status: ALLOWED }); + demotedKeys.push({ table: expectedTable, columns: d.live.columns }); + } } + // A demoted UNIQUE constraint is dropped, and Postgres refuses that while a foreign key + // relies on it, so each such FK is dropped first and added back once the index exists. + if (demotedKeys.length > 0) refitForeignKeysOnto(demotedKeys, actualTables, expectedTables, renamedConstraints, changes); // Pass 2b: views. Identity is (schema, name). How "changed" is decided is // DIALECT-DEPENDENT — see diffViews. @@ -364,6 +404,10 @@ export async function diff( changes, blocked: changes.filter((c) => c.status.state === "blocked"), hazards: changes.flatMap((c) => dataHazardOf(c) ?? []), + advisories: [ + ...intervalAsStringAdvisories(expectedTables, (id) => actualTables.get(id) ?? predecessors.get(id)), + ...(args.dialect === "sqlite" || args.dialect === "d1" ? sequenceNotAppliedAdvisories(changes) : []), + ], ...(args.dialect === "postgres" ? { exprCanonPath } : {}), ...(shadowAnswer !== undefined && "unavailable" in shadowAnswer ? { shadowUnavailable: shadowAnswer.unavailable } : {}), }; @@ -422,7 +466,11 @@ function diffRenamedTables( const carried = withRenamedRefTables({ ...actual, name: r.to }, renamed); const tableChanges: Change[] = []; diffTablePair(expected, carried, tableChanges, dialect, canon); - changes.push(...(dialect === "postgres" ? foldNameOnlyChanges(r, tableChanges, textColumnsOf(actual), canon) : tableChanges)); + if (dialect !== "postgres") { changes.push(...tableChanges); continue; } + changes.push(...foldNameOnlyChanges(r, tableChanges, textColumnsOf(actual), canon)); + // `ALTER TABLE … RENAME` keeps the old primary-key name. + const pk = primaryKeyRename(expected, actual, r.from); + if (pk !== undefined) (r.constraintRenames ??= []).push(pk); } return { renamed, predecessors }; } @@ -546,6 +594,21 @@ function foldNameOnlyChanges( textColumns: ReadonlySet, canon: ExprCanon, ): Change[] { + const { constraintRenames, indexRenames, rest } = pairNameOnlyChanges(tableChanges, textColumns, canon); + if (constraintRenames.length > 0) r.constraintRenames = constraintRenames; + if (indexRenames.length > 0) r.indexRenames = indexRenames; + return rest; +} + +/** + * Pair each drop of a constraint or index with an add of the SAME definition under another + * name. The pairs are name changes; `rest` is every other change, in order. + */ +function pairNameOnlyChanges( + tableChanges: Change[], + textColumns: ReadonlySet, + canon: ExprCanon, +): { constraintRenames: NameChange[]; indexRenames: NameChange[]; rest: Change[] } { const constraintRenames: NameChange[] = []; const indexRenames: NameChange[] = []; const folded = new Set(); @@ -573,10 +636,7 @@ function foldNameOnlyChanges( } } } - if (folded.size === 0) return tableChanges; - if (constraintRenames.length > 0) r.constraintRenames = constraintRenames; - if (indexRenames.length > 0) r.indexRenames = indexRenames; - return tableChanges.filter((c) => !folded.has(c)); + return { constraintRenames, indexRenames, rest: folded.size === 0 ? tableChanges : tableChanges.filter((c) => !folded.has(c)) }; } function idOfChange(c: Change): string | undefined { @@ -705,6 +765,94 @@ async function carryThroughColumnRenames( } } +/** Postgres keeps the first 63 bytes of an identifier, so a longer name is compared as it is stored. */ +function pgStoredName(name: string): string { + if (Buffer.byteLength(name, "utf8") <= PG_IDENTIFIER_MAX_BYTES) return name; + let out = ""; + for (const ch of name) { + if (Buffer.byteLength(out + ch, "utf8") > PG_IDENTIFIER_MAX_BYTES) break; + out += ch; + } + return out; +} + +/** + * The rename a live primary key needs to carry the name the schema conventions give it, or + * undefined when the names agree or either side has no key (pg-mem reports none). An absent + * name is `
_pkey`; `liveTable` is the table's name as the database knows it. + */ +function primaryKeyRename(expected: TableDescriptor, actual: TableDescriptor, liveTable: string): NameChange | undefined { + if (expected.primaryKey.length === 0 || actual.primaryKey.length === 0) return undefined; + const from = pgStoredName(actual.primaryKeyName ?? `${liveTable}_pkey`); + const to = pgStoredName(expected.primaryKeyName ?? `${expected.name}_pkey`); + return from === to ? undefined : { from, to }; +} + +/** The live table with each renamed index (or the index backing a renamed constraint) under its new name. */ +function withRenamedIndexes(actual: TableDescriptor, renames: readonly NameChange[]): TableDescriptor { + if (renames.length === 0) return actual; + const to = new Map(renames.map((r) => [r.from, r.to])); + return { ...actual, indexes: actual.indexes.map((i) => (to.has(i.name) ? { ...i, name: to.get(i.name)! } : i)) }; +} + +/** + * Unique keys whose definition and name match but which differ only in being a plain index or + * a UNIQUE constraint. `promotions` are the expected constraints a live index becomes; + * `demotions` pair a live constraint with the plain index the conventions ask for (`plainIndex`); + * under conventions with no preference a live constraint stays. + */ +function uniqueKindChanges( + expected: TableDescriptor, + actual: TableDescriptor, + canon: ExprCanon, +): { promotions: IndexDescriptor[]; demotions: { live: IndexDescriptor; want: IndexDescriptor }[] } { + const promotions: IndexDescriptor[] = []; + const demotions: { live: IndexDescriptor; want: IndexDescriptor }[] = []; + const live = new Map(actual.indexes.map((i) => [i.name, i])); + for (const want of expected.indexes) { + const a = live.get(want.name); + if (a === undefined || !want.unique || !a.unique || !indexEquals(want, a, canon)) continue; + // Only a plain index and a UNIQUE constraint trade places; a PK or EXCLUDE index is not a unique key. + if ((want.constraint ?? "unique") !== "unique" || (a.constraint ?? "unique") !== "unique") continue; + if (want.constraint === "unique" && a.constraint === undefined) promotions.push(want); + else if (want.plainIndex === true && a.constraint === "unique") demotions.push({ live: a, want }); + } + return { promotions, demotions }; +} + +/** + * Drop and re-add every live foreign key that targets a demoted key's columns, unless the + * diff already drops it. An FK the same migration renames is dropped under its new name, since + * the rename runs first. The re-added FK is the expected one when the model still declares + * it, else the live one. + */ +function refitForeignKeysOnto( + keys: readonly { table: TableDescriptor; columns: readonly string[] }[], + actualTables: ReadonlyMap, + expectedTables: ReadonlyMap, + renamedConstraints: ReadonlyMap>, + changes: Change[], +): void { + const dropped = new Set(changes.flatMap((c) => + c.kind === "drop-fk" ? [`${tableIdentity({ name: c.table, ...schemaSpread(c.schema) })}\u0000${c.fk}`] : [])); + for (const [id, t] of actualTables) { + for (const fk of t.foreignKeys) { + // An FK names its target table in the referencing table's schema. + const target = keys.find((k) => fk.refTable === k.table.name && t.schema === k.table.schema + && fk.refColumns.length === k.columns.length && fk.refColumns.every((c, i) => c === k.columns[i])); + if (target === undefined || dropped.has(`${id}\u0000${fk.name}`)) continue; + const sx = schemaSpread(t.schema); + const name = renamedConstraints.get(id)?.get(fk.name) ?? fk.name; + const live = name === fk.name ? fk : { ...fk, name }; + const want = expectedTables.get(id)?.foreignKeys.find((f) => f.name === name) ?? live; + changes.push({ kind: "drop-fk", table: t.name, ...sx, fk: name, restore: live, status: ALLOWED }); + if (!changes.some((c) => c.kind === "add-fk" && idOfChange(c) === id && c.fk.name === want.name)) { + changes.push({ kind: "add-fk", table: t.name, ...sx, fk: want, status: ALLOWED }); + } + } + } +} + /** * #258 — throw {@link PrimaryKeyChangeError} when a table's live PRIMARY KEY differs from * the metadata identity. Live PK column names are first mapped through any detected @@ -760,10 +908,71 @@ function canonTypeForDialect(t: SqlType, dialect: Dialect | undefined): SqlType return t; } +/** + * Before field.duration, introspection read a Postgres `interval` column as `text`, and a model + * could only describe one as a string. So a live interval column the model declares as text is + * read as text, exactly as it was, and does not turn into drift (an `interval-as-string` + * advisory says so). An expected interval compares as one, and any other expected type is a + * real type change from the interval. + */ +function asPreDurationType(actual: SqlType, expected: SqlType): SqlType { + // Only where the model expects text (a field.string): a live interval modelled as anything + // else is a real type change, reported as interval → that type. + if (actual.kind === "interval" && expected.kind === "text") return { kind: "text" }; + if (actual.kind === "array" && expected.kind === "array") { + const element = asPreDurationType(actual.element, expected.element); + return element === actual.element ? actual : { kind: "array", element }; + } + return actual; +} + +/** Every column a live `interval` (or array of them) the model does not declare as a duration: + * `asPreDurationType` reads it as text so it is not drift, and this says so. */ +function intervalAsStringAdvisories( + expectedTables: ReadonlyMap, liveOf: (id: string) => TableDescriptor | undefined, +): SchemaAdvisory[] { + const out: SchemaAdvisory[] = []; + for (const [id, expected] of expectedTables) { + const live = liveOf(id); + if (live === undefined) continue; + const liveCols = new Map(live.columns.map((c) => [c.name, c])); + for (const ec of expected.columns) { + const lc = liveCols.get(ec.name); + if (lc === undefined || asPreDurationType(lc.sqlType, ec.sqlType) === lc.sqlType) continue; + out.push({ kind: "interval-as-string", table: expected.name, ...schemaSpread(expected.schema), column: ec.name }); + } + } + return out; +} + +/** A column this diff creates on SQLite or D1 with a declared sequence start or name, which + * that database cannot hold (it has no sequence). */ +function sequenceNotAppliedAdvisories(changes: readonly Change[]): SchemaAdvisory[] { + const out: SchemaAdvisory[] = []; + const at = (table: string, schema: string | undefined, column: ColumnDescriptor): void => { + if (column.sequence !== undefined) out.push({ kind: "sequence-not-applied", table, ...schemaSpread(schema), column: column.name }); + }; + for (const c of changes) { + if (c.kind === "create-table") for (const col of c.table.columns) at(c.table.name, c.table.schema, col); + if (c.kind === "add-column") at(c.table, c.schema, c.column); + } + return out; +} + function sqlTypeEqualsForDialect(a: SqlType, b: SqlType, dialect: Dialect | undefined): boolean { return sqlTypeEquals(canonTypeForDialect(a, dialect), canonTypeForDialect(b, dialect)); } +/** + * The name Postgres gives an identity or serial column's sequence when none is declared: + * `
__seq`. Undefined past the 63-byte identifier limit, where Postgres + * truncates by its own rule; such a name is then never compared offline. + */ +function defaultSequenceName(table: string, column: string): string | undefined { + const name = `${table}_${column}_seq`; + return Buffer.byteLength(name, "utf8") <= PG_IDENTIFIER_MAX_BYTES ? name : undefined; +} + function diffTableColumns( expected: TableDescriptor, actual: TableDescriptor, @@ -777,11 +986,13 @@ function diffTableColumns( const actualCols = new Map(actual.columns.map((c) => [c.name, c])); for (const [name, ec] of expectedCols) { - const ac = actualCols.get(name); - if (!ac) { + const live = actualCols.get(name); + if (!live) { changes.push({ kind: "add-column", table, ...sx, column: ec, status: ALLOWED }); continue; } + const preDuration = asPreDurationType(live.sqlType, ec.sqlType); + const ac = preDuration === live.sqlType ? live : { ...live, sqlType: preDuration }; // Compare type, nullable, default — emit per-aspect change. Type equality is // dialect-aware: SQLite/D1 cannot physically represent a VARCHAR length or a // native json type, so those distinctions must not read as drift (see @@ -846,6 +1057,26 @@ function diffTableColumns( }; changes.push(change); } + // The declared start and name of an increment key's sequence (Postgres only; SQLite has + // none). Only what the model DECLARES is compared, so an undeclared one never produces a + // change. The other side is the live sequence when introspected (`liveSequence`), else — + // a committed snapshot — what the previous model declared, with Postgres's defaults (start + // 1, `
__seq`) for what it left undeclared. + if (dialect === "postgres" && ec.identity === "increment" && ac.identity === "increment" && ec.sequence !== undefined) { + const live = ac.liveSequence; + const actualName = live?.name ?? ac.sequence?.name ?? defaultSequenceName(table, name); + const actualStart = live?.start ?? ac.sequence?.start ?? 1; + const startDiffers = ec.sequence.start !== undefined && ec.sequence.start !== actualStart; + const nameDiffers = ec.sequence.name !== undefined && actualName !== undefined && ec.sequence.name !== actualName; + if ((startDiffers || nameDiffers) && actualName !== undefined) { + changes.push({ + kind: "alter-sequence", table, ...sx, column: name, sequence: actualName, status: ALLOWED, + ...(live?.schema !== undefined ? { sequenceSchema: live.schema } : {}), + ...(startDiffers ? { fromStart: actualStart, toStart: ec.sequence.start! } : {}), + ...(nameDiffers ? { toName: ec.sequence.name! } : {}), + }); + } + } if (canon.catalog === true) { // Both sides read `increment` off a GENERATED identity (attidentity) or a legacy serial's // nextval default; a serial losing its sequence is the default change above, not this. @@ -855,6 +1086,7 @@ function diffTableColumns( changes.push({ kind: "change-column-identity", table, ...sx, column: name, status: ALLOWED, ...(from !== undefined ? { from } : {}), ...(to !== undefined ? { to } : {}), + ...(to !== undefined && ec.sequence !== undefined ? { sequence: ec.sequence } : {}), }); } if (ec.description !== ac.description) { diff --git a/server/typescript/packages/migrate-ts/src/diff/status.ts b/server/typescript/packages/migrate-ts/src/diff/status.ts index a777fc390f..d6aff0974d 100644 --- a/server/typescript/packages/migrate-ts/src/diff/status.ts +++ b/server/typescript/packages/migrate-ts/src/diff/status.ts @@ -180,6 +180,10 @@ function blockFor( + `the column starts failing). Declare @generation: increment on the identity to keep it, or pass ` + `--allow drop-identity-default if removing it is intentional`); + // Altering a sequence's start or name keeps every value it has handed out. + case "alter-sequence": + return null; + case "change-comment": if (c.from === undefined || c.to !== undefined) return null; return gate(allow, "dropComment", `${c.column !== undefined ? `column "${c.table}"."${c.column}"` : `table "${c.table}"`} ` @@ -189,6 +193,7 @@ function blockFor( // Always-allowed kinds case "create-table": case "rename-table": + case "rename-constraints": case "add-column": case "rename-column": case "add-index": diff --git a/server/typescript/packages/migrate-ts/src/drift/drift.ts b/server/typescript/packages/migrate-ts/src/drift/drift.ts index 6c76415350..2462473011 100644 --- a/server/typescript/packages/migrate-ts/src/drift/drift.ts +++ b/server/typescript/packages/migrate-ts/src/drift/drift.ts @@ -15,7 +15,7 @@ import type { Kysely } from "kysely"; import type { MetaRoot } from "@metaobjectsdev/metadata"; -import { buildExpectedSchemaWithProvenance } from "../expected-schema.js"; +import { buildExpectedSchemaWithProvenance, type SchemaBuildSettings } from "../expected-schema.js"; import { introspect } from "../introspect/index.js"; import { diff } from "../diff/index.js"; import { collectUnmanagedNames } from "../unmanaged.js"; @@ -27,8 +27,11 @@ import type { ViewReplaceProbe } from "../view-replace.js"; import { postgresExprCanonProbe } from "../expr-canon-probe.js"; import type { ExprCanonProbe } from "../expr-canon.js"; import type { ShadowSchemaSource } from "../shadow-expected.js"; +import type { FkTargetViolation } from "../errors.js"; -export interface ComputeDriftOptions { +/** `conventions` and `skipRedundantChecks` are the project's schema settings, passed to the + * expected-schema build (see `BuildExpectedSchemaOptions`); absent = migrate's defaults. */ +export interface ComputeDriftOptions extends SchemaBuildSettings { /** * Destructive-change permissions. Mirrors `diff`'s `allow` — a drift that is * "allowed" still appears in `changes` (it's still drift), so the gate fails @@ -111,6 +114,12 @@ export interface DriftResult extends DiffResult { * come to disagree with the one this comparison actually made. */ importedOutOfScope: readonly string[] | undefined; + /** + * FKs whose target is not a unique key, on a dialect that does not refuse them at + * CREATE TABLE (SQLite and D1). Not drift and not a refusal: the caller reports them, + * as `meta migrate` does. + */ + unenforcedFkTargetViolations?: readonly FkTargetViolation[]; } /** @@ -136,6 +145,8 @@ export async function computeDriftFromActual( buildExpectedSchemaWithProvenance(metadata, { dialect, ...(opts?.views !== undefined ? { views: opts.views } : {}), + conventions: opts?.conventions, + skipRedundantChecks: opts?.skipRedundantChecks, }), opts?.inScope, opts?.imported !== undefined ? { imported: opts.imported } : undefined, @@ -163,6 +174,7 @@ export async function computeDriftFromActual( outOfScope: scoped.outOfScope, declaredSchemas: scoped.declaredSchemas, importedOutOfScope: scoped.importedOutOfScope, + ...(scoped.unenforcedFkTargetViolations !== undefined ? { unenforcedFkTargetViolations: scoped.unenforcedFkTargetViolations } : {}), }; } diff --git a/server/typescript/packages/migrate-ts/src/emit/postgres.ts b/server/typescript/packages/migrate-ts/src/emit/postgres.ts index 4f367f9344..46d75b6786 100644 --- a/server/typescript/packages/migrate-ts/src/emit/postgres.ts +++ b/server/typescript/packages/migrate-ts/src/emit/postgres.ts @@ -1,3 +1,4 @@ +import { canBeUniqueConstraint } from "../schema-conventions.js"; import type { Change, EmitResult, ColumnDescriptor, IndexDescriptor, FkDescriptor, TableDescriptor, ViewDescriptor, ColumnDefault, FkAction, NameChange, DataHazard, @@ -37,12 +38,16 @@ import { dataHazardOf } from "../hazards.js"; const STAGE_ORDER: Record = { "drop-view": 0, "rename-table": 0.5, + // After a table rename (its new name is the one this change addresses), before every drop/add. + "rename-constraints": 0.6, "drop-fk": 1, "drop-check": 1, "create-table": 1, "drop-index": 1.5, "add-column": 2, "drop-column": 2, "change-column-type": 2, "change-column-nullable": 2, "change-column-default": 2, "change-column-identity": 2, + // After column changes (the identity exists), before anything that could read the name. + "alter-sequence": 2.5, "rename-column": 3, "change-comment": 3.5, "add-index": 4, @@ -230,6 +235,7 @@ function renderUp(c: Change, recreated: ReadonlySet = new Set()): string // so a no-op down would still record the rollback as done. case "drop-table": return `DROP TABLE IF EXISTS ${quoteQualified(c.table, c.schema)};`; case "rename-table": return renderRenameTable(c, "up"); + case "rename-constraints": return renderRenameConstraints(c, "up"); case "add-column": { const base = `ALTER TABLE ${quoteQualified(c.table, c.schema)} ADD COLUMN ${renderColumn(c.column)};`; if (!c.column.description) return base; @@ -246,7 +252,8 @@ function renderUp(c: Change, recreated: ReadonlySet = new Set()): string return c.to !== undefined ? `ALTER TABLE ${quoteQualified(c.table, c.schema)} ALTER COLUMN ${quote(c.column)} SET DEFAULT ${renderDefault(c.to)};` : `ALTER TABLE ${quoteQualified(c.table, c.schema)} ALTER COLUMN ${quote(c.column)} DROP DEFAULT;`; - case "change-column-identity": return renderIdentity(c.table, c.schema, c.column, c.to); + case "change-column-identity": return renderIdentity(c.table, c.schema, c.column, c.to, c.sequence); + case "alter-sequence": return renderAlterSequence(c.sequenceSchema ?? c.schema, c.sequence, c.toStart, c.toName); case "change-comment": return renderComment(c.table, c.schema, c.column, c.to); case "add-index": return renderCreateIndex(c.table, c.schema, c.index); // #285: an index OWNED by a UNIQUE / PRIMARY KEY / EXCLUDE constraint cannot be @@ -339,6 +346,34 @@ function renderRenameTable(c: Extract, dir: "u return (up ? lines : [...lines].reverse()).join("\n\n"); } +/** Constraints and indexes renamed in place on a table that keeps its name; the down renames them back. */ +function renderRenameConstraints(c: Extract, dir: "up" | "down"): string { + const up = dir === "up"; + const table = quoteQualified(c.table, c.schema); + const flip = (r: NameChange): NameChange => (up ? r : { from: r.to, to: r.from }); + // A promotion's down drops the constraint, which drops its index, then re-creates the + // plain index. Postgres refuses that drop while a foreign key relies on the key; the FK + // must be dropped first. + const promotions = (c.uniquePromotions ?? []).map((ix) => up + ? `ALTER TABLE ${table} ADD CONSTRAINT ${quote(ix.name)} UNIQUE USING INDEX ${quote(ix.name)};` + : [`ALTER TABLE ${table} DROP CONSTRAINT ${quote(ix.name)};`, + renderCreateIndex(c.table, c.schema, withoutConstraint(ix))].join("\n")); + const lines = [ + ...(c.constraintRenames ?? []).map(flip).map((r) => + `ALTER TABLE ${table} RENAME CONSTRAINT ${quote(r.from)} TO ${quote(r.to)};`), + ...(c.indexRenames ?? []).map(flip).map((r) => + `ALTER INDEX ${quoteIndexQualified(r.from, c.schema)} RENAME TO ${quote(r.to)};`), + ...promotions, + ]; + return (up ? lines : [...lines].reverse()).join("\n\n"); +} + +/** The index with no owning constraint: what a promotion started from. */ +function withoutConstraint(ix: IndexDescriptor): IndexDescriptor { + const { constraint: _owned, ...plain } = ix; + return plain; +} + function renderDown(c: Change, recreated: ReadonlySet = new Set()): string { switch (c.kind) { case "create-table": return `DROP TABLE ${quoteQualified(c.table.name, c.table.schema)};`; @@ -360,6 +395,7 @@ function renderDown(c: Change, recreated: ReadonlySet = new Set()): stri return parts.join("\n"); } case "rename-table": return renderRenameTable(c, "down"); + case "rename-constraints": return renderRenameConstraints(c, "down"); case "add-column": return `ALTER TABLE ${quoteQualified(c.table, c.schema)} DROP COLUMN ${quote(c.column.name)};`; case "drop-column": return c.restore @@ -376,8 +412,19 @@ function renderDown(c: Change, recreated: ReadonlySet = new Set()): stri ? `ALTER TABLE ${quoteQualified(c.table, c.schema)} ALTER COLUMN ${quote(c.column)} SET DEFAULT ${renderDefault(c.from)};` : `ALTER TABLE ${quoteQualified(c.table, c.schema)} ALTER COLUMN ${quote(c.column)} DROP DEFAULT;`; case "change-column-identity": return renderIdentity(c.table, c.schema, c.column, c.from); + // The down addresses the sequence by the name the up gave it, and runs in reverse order: + // rename back first, then restore the start. + case "alter-sequence": + // The counter is not moved back: values it has handed out stay handed out. + return [ + ...(c.toName !== undefined ? [`ALTER SEQUENCE ${quoteQualified(c.toName, c.sequenceSchema ?? c.schema)} RENAME TO ${quote(c.sequence)};`] : []), + ...(c.fromStart !== undefined ? [`ALTER SEQUENCE ${quoteQualified(c.sequence, c.sequenceSchema ?? c.schema)} START WITH ${c.fromStart};`] : []), + ].join("\n"); case "change-comment": return renderComment(c.table, c.schema, c.column, c.from); - case "add-index": return `DROP INDEX ${quoteIndexQualified(c.index.name, c.schema)};`; + case "add-index": + return isUniqueConstraint(c.index) + ? `ALTER TABLE ${quoteQualified(c.table, c.schema)} DROP CONSTRAINT ${quote(c.index.name)};` + : `DROP INDEX ${quoteIndexQualified(c.index.name, c.schema)};`; case "drop-index": // #285: the up dropped a CONSTRAINT, so the down must add one back — recreating a // bare unique index would leave the database in a different shape than it started @@ -432,7 +479,7 @@ function renderDown(c: Change, recreated: ReadonlySet = new Set()): stri function renderCreateTable(t: TableDescriptor): string { const colDefs = t.columns.map((c) => ` ${renderColumn(c)}`); if (t.primaryKey.length > 0) { - colDefs.push(` CONSTRAINT ${quote(t.name + "_pkey")} PRIMARY KEY (${t.primaryKey.map(quote).join(", ")})`); + colDefs.push(` CONSTRAINT ${quote(t.primaryKeyName ?? t.name + "_pkey")} PRIMARY KEY (${t.primaryKey.map(quote).join(", ")})`); } for (const chk of t.checks ?? []) { colDefs.push(` CONSTRAINT ${quote(chk.name)} CHECK (${chk.expression})`); @@ -455,9 +502,36 @@ function renderTableComments(t: TableDescriptor): string[] { return out; } -function renderIdentity(table: string, schema: string | undefined, column: string, to: "increment" | undefined): string { +/** + * Start first, under the live name; then the rename. `START WITH` alone only records the + * restart point, so the counter is moved up to the new start — never down, which could hand + * out an id already in use — by `setval(..., false)` when it has not reached it. + */ +function renderAlterSequence(schema: string | undefined, sequence: string, toStart: number | undefined, toName: string | undefined): string { + const seq = quoteQualified(sequence, schema); + const regclass = `'${seq.replace(/'/g, "''")}'`; + return [ + ...(toStart !== undefined + ? [ + `ALTER SEQUENCE ${seq} START WITH ${toStart};`, + `SELECT setval(${regclass}, ${toStart}, false) FROM ${seq} WHERE last_value < ${toStart};`, + ] + : []), + ...(toName !== undefined ? [`ALTER SEQUENCE ${seq} RENAME TO ${quote(toName)};`] : []), + ].join("\n"); +} + +function renderIdentity( + table: string, + schema: string | undefined, + column: string, + to: "increment" | undefined, + sequence?: { name?: string; start?: number }, +): string { const alter = `ALTER TABLE ${quoteQualified(table, schema)} ALTER COLUMN ${quote(column)}`; - return to === "increment" ? `${alter} ADD GENERATED BY DEFAULT AS IDENTITY;` : `${alter} DROP IDENTITY IF EXISTS;`; + return to === "increment" + ? `${alter} ADD GENERATED BY DEFAULT AS IDENTITY${identityOptions(sequence !== undefined ? { sequence } : {})};` + : `${alter} DROP IDENTITY IF EXISTS;`; } function renderComment(table: string, schema: string | undefined, column: string | undefined, to: string | undefined): string { @@ -502,7 +576,7 @@ function hazardComment(h: DataHazard): string { function renderColumn(c: ColumnDescriptor): string { let s = `${quote(c.name)} ${pgType(c.sqlType)}`; - if (c.identity === "increment") s += " GENERATED BY DEFAULT AS IDENTITY"; + if (c.identity === "increment") s += " GENERATED BY DEFAULT AS IDENTITY" + identityOptions(c); if (c.identity === "uuid") s += " DEFAULT gen_random_uuid()"; s += c.nullable ? "" : " NOT NULL"; if (c.default !== undefined && c.identity === undefined) { @@ -514,6 +588,15 @@ function renderColumn(c: ColumnDescriptor): string { return s; } +/** ` (SEQUENCE NAME "x" START WITH n)` for what the model declares, else nothing. */ +function identityOptions(c: Pick): string { + const opts = [ + ...(c.sequence?.name !== undefined ? [`SEQUENCE NAME ${quote(c.sequence.name)}`] : []), + ...(c.sequence?.start !== undefined ? [`START WITH ${c.sequence.start}`] : []), + ]; + return opts.length === 0 ? "" : ` (${opts.join(" ")})`; +} + export function pgType(t: SqlType): string { switch (t.kind) { case "text": return t.maxLength !== undefined ? `VARCHAR(${t.maxLength})` : "TEXT"; @@ -533,6 +616,7 @@ export function pgType(t: SqlType): string { case "blob": return "BYTEA"; case "uuid": return "UUID"; case "inet": return "INET"; + case "interval": return "INTERVAL"; case "array": return `${pgType(t.element)}[]`; } } @@ -543,7 +627,15 @@ function renderDefault(d: ColumnDefault): string { return `'${pgEscape(d.value)}'`; } +/** A unique key the schema conventions create as a constraint (only a plain column key can be one). */ +function isUniqueConstraint(ix: IndexDescriptor): boolean { + return ix.constraint === "unique" && canBeUniqueConstraint(ix); +} + function renderCreateIndex(table: string, schema: string | undefined, ix: IndexDescriptor): string { + if (isUniqueConstraint(ix)) { + return `ALTER TABLE ${quoteQualified(table, schema)} ADD CONSTRAINT ${quote(ix.name)} UNIQUE (${ix.columns.map(quote).join(", ")});`; + } const u = ix.unique ? "UNIQUE " : ""; // Key list: a raw expression (functional/expression index) takes precedence over // the per-column list. The expression is emitted verbatim (already valid SQL). diff --git a/server/typescript/packages/migrate-ts/src/emit/sqlite.ts b/server/typescript/packages/migrate-ts/src/emit/sqlite.ts index 9689dafbf0..19ce6422dd 100644 --- a/server/typescript/packages/migrate-ts/src/emit/sqlite.ts +++ b/server/typescript/packages/migrate-ts/src/emit/sqlite.ts @@ -41,12 +41,13 @@ const STAGE_ORDER: Record = { // rename-table runs before every other table change (mirrors postgres): changes on a // renamed table — including its recreate-and-copy — are keyed by the NEW name. "rename-table": 0.5, + "rename-constraints": 0.6, "drop-fk": 1, "drop-check": 1, "create-table": 1, "drop-index": 1.5, "add-column": 2, "drop-column": 2, "change-column-type": 2, "change-column-nullable": 2, "change-column-default": 2, - "change-column-identity": 2, "change-comment": 2, + "change-column-identity": 2, "change-comment": 2, "alter-sequence": 2, "rename-column": 3, "add-index": 4, "add-fk": 5, @@ -151,6 +152,7 @@ export function changeTable(c: Change): string | undefined { case "create-table": return c.table.name; case "drop-table": return c.table; case "rename-table": return c.from; // applies to source table + case "rename-constraints": return c.table; case "add-column": case "drop-column": case "rename-column": @@ -390,6 +392,8 @@ function renderUpNative(c: Change): string { assertNoCarriedRenames(c); return `ALTER TABLE ${quote(c.from)} RENAME TO ${quote(c.to)};`; } + case "rename-constraints": + throw new Error("rename-constraints is postgres-only — the sqlite/d1 diff never produces it (sqlite cannot rename a constraint)"); case "add-column": return `ALTER TABLE ${quote(c.table)} ADD COLUMN ${renderColumnInline(c.column)};`; case "drop-column": return `ALTER TABLE ${quote(c.table)} DROP COLUMN ${quote(c.column)};`; case "rename-column": { @@ -411,7 +415,8 @@ function renderUpNative(c: Change): string { throw new Error(`renderUpNative: ${c.kind} should have been handled by recreate bundler`); case "change-column-identity": case "change-comment": - // Planned only on the Postgres shadow path; SQLite has neither GENERATED identity nor COMMENT. + case "alter-sequence": + // Planned only on Postgres; SQLite has no GENERATED identity, COMMENT or sequence. throw new Error(`renderUpNative: ${c.kind} is a Postgres-only change`); // SQLite has no schema namespacing for views and no CREATE OR REPLACE VIEW; // a replace is DROP + CREATE. The view body lives in ViewDescriptor.sql. @@ -465,6 +470,8 @@ function renderDownNative(c: Change): string { assertNoCarriedRenames(c); return `ALTER TABLE ${quote(c.to)} RENAME TO ${quote(c.from)};`; } + case "rename-constraints": + throw new Error("rename-constraints is postgres-only — the sqlite/d1 diff never produces it (sqlite cannot rename a constraint)"); case "add-column": return `ALTER TABLE ${quote(c.table)} DROP COLUMN ${quote(c.column.name)};`; case "drop-column": return renderDropColumnDown(c); case "rename-column": { @@ -486,7 +493,8 @@ function renderDownNative(c: Change): string { throw new Error(`renderDownNative: ${c.kind} should have been handled by recreate bundler`); case "change-column-identity": case "change-comment": - // Planned only on the Postgres shadow path; SQLite has neither GENERATED identity nor COMMENT. + case "alter-sequence": + // Planned only on Postgres; SQLite has no GENERATED identity, COMMENT or sequence. throw new Error(`renderDownNative: ${c.kind} is a Postgres-only change`); case "create-view": return `DROP VIEW IF EXISTS ${quote(c.view.name)};`; // The view as the DB held it (`restore`) is a valid restore payload: a view dropped @@ -584,6 +592,7 @@ export function sqliteType(t: SqlType, identity: ColumnDescriptor["identity"]): case "blob": return "BLOB"; case "uuid": return "TEXT"; case "inet": return "TEXT"; // SQLite has no inet type; store as text + case "interval": return "INTEGER"; // the duration's microseconds (orders as the span does) case "array": return "TEXT"; // SQLite has no array type; store as JSON text } } diff --git a/server/typescript/packages/migrate-ts/src/errors.ts b/server/typescript/packages/migrate-ts/src/errors.ts index 06d51a947a..f9cba66068 100644 --- a/server/typescript/packages/migrate-ts/src/errors.ts +++ b/server/typescript/packages/migrate-ts/src/errors.ts @@ -35,6 +35,47 @@ export class SetNullNotNullableError extends Error { } } +/** One identity.reference whose target fields are not a unique key on the table it references. */ +export interface FkTargetViolation { + /** The holder's table, qualified (`.
`): what `scopeExpectedSchema` governs by. */ + table: string; + entityName: string; + referenceName: string; + targetEntityName: string; + targetFields: string[]; + /** Set when the target is a TPH subtype: the entity whose table it is stored in. */ + storedIn?: string; +} + +/** + * Thrown by buildExpectedSchema() when one or more identity.references target fields the + * target table does not hold unique: not its primary key, not a plain identity.secondary, + * not a single @unique field. A database refuses such an FK at apply time, after earlier + * statements in the migration already ran; refusing here names every offending reference + * at once, so a model with several is fixed in one pass rather than one run per reference. + */ +export class FkTargetNotUniqueError extends Error { + override readonly name = "FkTargetNotUniqueError"; + readonly violations: FkTargetViolation[]; + + constructor(violations: FkTargetViolation[]) { + const lines = violations.map((v) => { + const fieldList = v.targetFields.join(", "); + const head = `Entity "${v.entityName}": reference "${v.referenceName}" targets [${fieldList}] on "${v.targetEntityName}", ` + + `which is not a unique key of its table (its primary key, an identity.secondary without @where/@expr, or a @unique field). `; + return v.storedIn !== undefined + ? head + `"${v.targetEntityName}" is stored in the table of "${v.storedIn}", and a key declared on a TPH subtype is not emitted: ` + + `declare it on "${v.storedIn}" (an identity.secondary with @fields [${fieldList}]).` + : head + `Fix: declare an identity.secondary on "${v.targetEntityName}" with @fields [${fieldList}], or reference its whole primary key.`; + }); + const head = violations.length === 1 + ? "A database refuses this foreign key." + : `A database refuses these ${violations.length} foreign keys.`; + super(`${head}\n${lines.map((l) => ` - ${l}`).join("\n")}`); + this.violations = violations; + } +} + const ALLOW_OPTION_BY_KIND: Partial> = { "drop-column": "dropColumn", "drop-table": "dropTable", @@ -75,6 +116,7 @@ function changeLocator(c: Change): string { case "change-column-nullable": case "change-column-default": case "change-column-identity": + case "alter-sequence": return `${c.table}.${"column" in c ? c.column : c.from}`; case "change-comment": return c.column !== undefined ? `${c.table}.${c.column}` : c.table; @@ -83,6 +125,8 @@ function changeLocator(c: Change): string { case "drop-table": case "rename-table": return c.kind === "drop-table" ? c.table : `${c.from}→${c.to}`; + case "rename-constraints": + return `${c.table}: ${[...(c.constraintRenames ?? []), ...(c.indexRenames ?? [])].map((r) => `${r.from}→${r.to}`).join(", ")}`; case "create-table": return c.table.name; case "add-index": @@ -243,6 +287,13 @@ export class DuplicateSqlNameError extends Error { "(e.g. `ownersByEmail`).", ); } + if (collisions.some((c) => c.owners.some((o) => o.includes("@sequenceName")))) { + fixes.push( + "A declared sequence name (identity.primary @sequenceName) is a relation in the same namespace: give each " + + "table's sequence its own name. An @sequenceName on an abstract base's identity is inherited by every " + + "entity that extends it, so declare it on each concrete entity instead.", + ); + } if (anyOther) { fixes.push("Rename one of the tables/views (entity source `@table`, projection `@kind view @view`)."); } @@ -282,6 +333,73 @@ export function isDuplicateSqlNameError(err: unknown): err is DuplicateSqlNameEr return err instanceof Error && err.name === "DuplicateSqlNameError"; } +/** One problem with a declared CHECK name (`@constraintName` on a validator or a field.enum). */ +export interface CheckNameProblem { + readonly table: string; + readonly name: string; + /** Another constraint on the same table already has the name. */ + readonly duplicateOf?: string; + /** The name is longer than Postgres's 63-byte identifier limit, in bytes. */ + readonly bytes?: number; +} + +/** + * A declared CHECK name that cannot be applied as written. Two constraints on one table with + * one name fail at apply on Postgres ("already exists") and, on SQLite, collapse in a + * name-keyed diff so every run proposes the CHECK again. A name over 63 bytes is truncated by + * Postgres when it is created, so the declared name never matches the database and the CHECK + * drifts forever. Either way the fix is to the metadata, so it is refused while the expected + * schema is built. + */ +export class InvalidCheckNameError extends Error { + override readonly name = "InvalidCheckNameError"; + readonly problems: readonly CheckNameProblem[]; + + constructor(problems: readonly CheckNameProblem[]) { + const lines = problems.map((p) => p.duplicateOf !== undefined + ? ` CHECK "${p.name}" on table "${p.table}" has the same name as ${p.duplicateOf}` + : ` CHECK "${p.name}" on table "${p.table}" is ${p.bytes} bytes; Postgres keeps only the first 63`); + super( + `ERR_CHECK_NAME: a declared @constraintName cannot be applied as written.\n${lines.join("\n")}\n` + + "Give each constraint on a table its own @constraintName, at most 63 bytes.", + ); + this.problems = problems; + } +} + +/** Is `err` the {@link InvalidCheckNameError} from a declared CHECK name that cannot be applied? */ +export function isInvalidCheckNameError(err: unknown): err is InvalidCheckNameError { + return err instanceof Error && err.name === "InvalidCheckNameError"; +} + +/** `migrate.conventions` names a schema-conventions preset that does not exist. */ +export class UnknownSchemaConventionsError extends Error { + override readonly name = "UnknownSchemaConventionsError"; + constructor(readonly selected: string, readonly presets: readonly string[]) { + super(`migrate.conventions "${selected}" is not a schema-conventions preset: expected one of ${presets.join(", ")}`); + } +} + +/** A project-supplied schema-conventions object lacks members this version calls. */ +export class InvalidSchemaConventionsError extends Error { + override readonly name = "InvalidSchemaConventionsError"; + constructor(readonly id: string, readonly missing: readonly string[]) { + super(`the schema conventions "${id}" are missing ${missing.join(", ")} — start from a preset ` + + `({ ...efcoreConventions, id: "${id}", … }) so every member this version calls is present`); + } +} + +/** Is `err` the {@link UnknownSchemaConventionsError}? */ +export function isUnknownSchemaConventionsError(err: unknown): err is UnknownSchemaConventionsError | InvalidSchemaConventionsError { + // Both are a project's conventions selection the tool cannot use: a config error. + return err instanceof Error && (err.name === "UnknownSchemaConventionsError" || err.name === "InvalidSchemaConventionsError"); +} + +/** Is `err` the {@link FkTargetNotUniqueError} from FKs whose targets are not unique keys? */ +export function isFkTargetNotUniqueError(err: unknown): err is FkTargetNotUniqueError { + return err instanceof Error && err.name === "FkTargetNotUniqueError"; +} + /** Is `err` the {@link DeclaredRenameError} from a declared rename that does not apply? */ export function isDeclaredRenameError(err: unknown): err is DeclaredRenameError { return err instanceof Error && err.name === "DeclaredRenameError"; diff --git a/server/typescript/packages/migrate-ts/src/expected-schema.ts b/server/typescript/packages/migrate-ts/src/expected-schema.ts index 6f41e955f7..e375f11d49 100644 --- a/server/typescript/packages/migrate-ts/src/expected-schema.ts +++ b/server/typescript/packages/migrate-ts/src/expected-schema.ts @@ -1,11 +1,15 @@ -import { DuplicateSqlNameError } from "./errors.js"; +import { + DuplicateSqlNameError, FkTargetNotUniqueError, InvalidCheckNameError, type CheckNameProblem, type FkTargetViolation, +} from "./errors.js"; +import { negateExpression, renderExpressionSql } from "./check-expression.js"; +import { canBeUniqueConstraint, resolveSchemaConventions, type SchemaConventions } from "./schema-conventions.js"; import type { ColumnNamingStrategy, MetaData, MetaField, MetaObject, MetaReferenceIdentity, MetaRoot, MetaValidator, } from "@metaobjectsdev/metadata"; import { VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_COMPARISON, VALIDATOR_SUBTYPE_REQUIRED_WHEN, - VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_SUBTYPE_EXACTLY_ONE, VALIDATOR_ATTR_PATTERN, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, @@ -46,6 +50,10 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, + formatPostgresInterval, + parseDurationLiteral, + parsePostgresInterval, FIELD_SUBTYPE_ENUM, FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP, @@ -59,7 +67,13 @@ import { STORAGE_FLATTENED, DOC_ATTR_DESCRIPTION, applyColumnNamingStrategy, physicalNamingOf, - resolveTableName, resolveColumnName, resolveTableSchema, resolveIndexName, + resolveTableName, resolveColumnName, resolveFlattenedColumnName, resolveTableSchema, resolveIndexName, + resolveEnumMaxLength, + IDENTITY_ATTR_SEQUENCE_NAME, + IDENTITY_ATTR_SEQUENCE_START, + VALIDATOR_ATTR_CONDITION, + VALIDATOR_ATTR_CONSTRAINT_NAME, + type ExprNode, } from "@metaobjectsdev/metadata"; import type { SqlType } from "./sql-type.js"; import type { @@ -69,6 +83,7 @@ import type { import { qualifiedDbName } from "./qualified-name.js"; import { SQLITE_ISO_NOW } from "./column-default.js"; import { viewFingerprint } from "./view-fingerprint.js"; +import { DIALECT_SQLITE, PG_IDENTIFIER_MAX_BYTES, isSqliteFamily } from "./types.js"; import { resolveReferentialActions, findReferentialActionConflict, @@ -106,8 +121,29 @@ export interface BuildExpectedSchemaOptions { * Advisory: the schema built is the same with or without it. */ onReferentialActionConflict?: (conflict: ReferentialActionConflict) => void; + /** + * Names and physical choices for the objects no code addresses (PK and FK constraint + * names, FK indexes, unique key kind). Absent = `defaultConventions`, what migrate has + * always emitted. Resolved by the caller (the CLI reads `migrate.conventions` and + * `metaobjects.config.ts`); this build reads no project setting off the model. + */ + conventions?: SchemaConventions | undefined; + /** + * Leave out a validator CHECK the column already guarantees: a `validator.length` maximum + * the column's VARCHAR(n) enforces (never on SQLite / D1, which do not enforce a declared + * length) and a length minimum of 0 or less. A validator with a declared `@constraintName` + * keeps its CHECK regardless. `migrate.checks.skipRedundant` in `.metaobjects/config.json`; + * absent = false, every CHECK emitted. + */ + skipRedundantChecks?: boolean | undefined; } +/** + * The project settings an expected-schema build takes, which every path that builds one + * (migrate, drift, the offline plan, a baseline) passes through unchanged so they agree. + */ +export type SchemaBuildSettings = Pick; + /** * A view as the caller (codegen-ts `buildProjectionViews`) produces it — structurally * an `ExpectedView`. It becomes a full `ViewDescriptor` in Pass 4, which is where the @@ -144,6 +180,19 @@ export type SchemaProvenance = ReadonlyMap; export interface ExpectedSchemaWithProvenance { snapshot: SchemaSnapshot; provenance: SchemaProvenance; + /** + * Foreign keys whose target columns are not a unique key of the table they reference. + * Not thrown here: `scopeExpectedSchema` refuses the ones on governed tables and returns + * the rest (an imported or out-of-scope holder is not this project's to fix), and + * {@link buildExpectedSchema} refuses them all. Absent when there are none. + */ + fkTargetViolations?: FkTargetViolation[]; + /** + * Set on SQLite (not D1) when there are violations: SQLite checks foreign keys only with + * `PRAGMA foreign_keys` on, which is off by default, so the `CREATE TABLE` succeeds and only + * a write with enforcement on fails. The violations are then reported, never refused. + */ + fkTargetsUnenforced?: true; } /** @@ -154,7 +203,12 @@ export function buildExpectedSchema( root: MetaData, opts?: BuildExpectedSchemaOptions, ): SchemaSnapshot { - return buildExpectedSchemaWithProvenance(root, opts).snapshot; + const built = buildExpectedSchemaWithProvenance(root, opts); + // No scope here: every FK whose target is not a unique key is refused. + if (built.fkTargetViolations !== undefined && built.fkTargetsUnenforced === undefined) { + throw new FkTargetNotUniqueError(built.fkTargetViolations); + } + return built.snapshot; } /** @@ -255,11 +309,16 @@ export function buildExpectedSchemaWithProvenance( const provenance = new Map(); // Pass 2: build full descriptors with FK resolution. + // The caller's conventions (checked for every member), else migrate's default. + const conventions = resolveSchemaConventions(opts?.conventions); + const skipRedundantChecks = opts?.skipRedundantChecks === true; + const fkCtx: ForeignKeyBuildContext = { conventions, dialect, origins: new Map() }; // Schema is resolved here (not stored in Pass 1) to avoid exactOptionalPropertyTypes // issues with `string | undefined` vs `schema?: string`. const tables: TableDescriptor[] = entities.map(({ entity, tableName }) => { const t = buildTable( - entity, tableName, resolveTargetTable, root as MetaRoot, strategy, dialect, opts?.onReferentialActionConflict, + entity, tableName, resolveTargetTable, root as MetaRoot, strategy, dialect, fkCtx, skipRedundantChecks, + opts?.onReferentialActionConflict, ); const schema = resolveTableSchema(entity); if (schema !== undefined) t.schema = schema; @@ -267,6 +326,12 @@ export function buildExpectedSchemaWithProvenance( return t; }); + // Checked against the BUILT tables, before the conventions add FK indexes: a key the model + // declares but never emits (on a TPH subtype) is no key a database can reference. + const fkTargetViolations = collectFkTargetViolations(tables, fkCtx.origins, provenance); + applySchemaConventions(tables, conventions, dialect); + checkDeclaredCheckNames(tables, dialect); + // Pass 3: dialect-specific SqlType normalization. if (dialect === "sqlite") { for (const table of tables) { @@ -289,6 +354,11 @@ export function buildExpectedSchemaWithProvenance( const normalized = normalizeBooleanLiteralForSqlite(col.default.value); if (normalized !== undefined) col.default = { kind: "literal", value: normalized }; } + // A duration's default is built in the Postgres interval spelling; the SQLite column holds + // the microsecond integer, which is what the runtime writes there. + if (kindBefore === "interval" && col.default?.kind === "literal") { + col.default = { kind: "literal", value: String(parsePostgresInterval(col.default.value)) }; + } // `now()` is Postgres-only; SQLite has no such function, so a // `DEFAULT now()` (the @autoSet insert-time default, and any authored // @default "now()") makes the emitted CREATE TABLE un-appliable @@ -399,6 +469,16 @@ export function buildExpectedSchemaWithProvenance( addOwner(t.schema, idx.name, label); } } + // A declared sequence name is a relation too (Postgres `pg_class`, per schema): two tables + // declaring one, or one naming a table, view or index, would fail at apply time — an + // identity.primary @sequenceName on an abstract base is inherited by every entity extending it. + for (const t of tables) { + for (const col of t.columns) { + if (col.sequence?.name !== undefined) { + addOwner(t.schema, col.sequence.name, `identity.primary @sequenceName "${col.sequence.name}" on table "${t.name}"`); + } + } + } const collisions = [...sqlNameOwners.entries()] .filter(([, owners]) => owners.length > 1) .map(([key, owners]) => ({ @@ -408,7 +488,13 @@ export function buildExpectedSchemaWithProvenance( })); if (collisions.length > 0) throw new DuplicateSqlNameError(collisions); - return { snapshot: { tables, views }, provenance }; + return { + snapshot: { tables, views }, + provenance, + ...(fkTargetViolations.length > 0 ? { fkTargetViolations } : {}), + // D1 always enforces foreign keys; plain SQLite only with the pragma on. + ...(fkTargetViolations.length > 0 && opts?.dialect === DIALECT_SQLITE ? { fkTargetsUnenforced: true as const } : {}), + }; } /** @@ -443,6 +529,9 @@ function normalizeForSqlite(sqlType: SqlType): SqlType { case "date": case "time": return { kind: "text" }; + case "interval": + // A duration is the 64-bit microsecond count: an integer orders and compares as the span. + return { kind: "integer", bits: 64 }; case "integer": // SQLite stores every INTEGER as a 64-bit value and Drizzle's int() emits // plain "INTEGER" regardless of source bit-width. Collapse 32 → 64 so the @@ -518,6 +607,8 @@ function buildTable( root: MetaRoot, strategy: ColumnNamingStrategy, dialect: Dialect | undefined, + fkCtx: ForeignKeyBuildContext, + skipRedundantChecks: boolean, onConflict?: (conflict: ReferentialActionConflict) => void, ): TableDescriptor { // Use effective accessors so inherited fields/identities (from `extends:` / @@ -553,7 +644,9 @@ function buildTable( // The parent field.object itself does NOT produce its own column. columns.push(...flattenObjectField(field, strategy)); } else { - columns.push(buildColumn(field, isPk, isPk ? pkGeneration : undefined, strategy)); + const col = buildColumn(field, isPk, isPk ? pkGeneration : undefined, strategy); + if (isPk && pkGeneration === "increment" && pkIdentity !== undefined) applyDeclaredSequence(col, pkIdentity); + columns.push(col); } } @@ -601,8 +694,8 @@ function buildTable( name: tableName, columns, indexes: buildSecondaryIndexes(entity, tableName, strategy), - foreignKeys: buildForeignKeys(entity, tableName, resolveTargetTable, root, strategy, onConflict), - checks: buildChecks(entity, tableName, strategy, dialect), + foreignKeys: buildForeignKeys(entity, tableName, resolveTargetTable, root, strategy, fkCtx, onConflict), + checks: buildChecks(entity, tableName, strategy, dialect, skipRedundantChecks), primaryKey, }; const entityDesc = readDescription(entity); @@ -750,7 +843,13 @@ function buildSecondaryIndexes( * column name verbatim (matching the enum-check convention). */ function validatorCheck( - v: MetaValidator, qcol: string, tableName: string, col: string, dialect: Dialect | undefined, + v: MetaValidator, + qcol: string, + tableName: string, + col: string, + dialect: Dialect | undefined, + skipRedundant = false, + enforcedMax?: number, ): CheckDescriptor | null { switch (v.subType) { case VALIDATOR_SUBTYPE_NUMERIC: { @@ -762,8 +861,10 @@ function validatorCheck( } case VALIDATOR_SUBTYPE_LENGTH: { const parts: string[] = []; - if (v.min !== undefined) parts.push(`length(${qcol}) >= ${v.min}`); - if (v.max !== undefined) parts.push(`length(${qcol}) <= ${v.max}`); + // Under skipRedundant, a minimum of 0 is always true and a maximum the column's + // VARCHAR(n) already enforces (n <= max) adds nothing. + if (v.min !== undefined && !(skipRedundant && v.min <= 0)) parts.push(`length(${qcol}) >= ${v.min}`); + if (v.max !== undefined && !(enforcedMax !== undefined && enforcedMax <= v.max)) parts.push(`length(${qcol}) <= ${v.max}`); if (parts.length === 0) return null; return { name: `${tableName}_${col}_length_chk`, expression: parts.join(" AND ") }; } @@ -797,7 +898,11 @@ function quoteCheckCol(col: string): string { } function buildChecks( - entity: MetaObject, tableName: string, strategy: ColumnNamingStrategy, dialect: Dialect | undefined, + entity: MetaObject, + tableName: string, + strategy: ColumnNamingStrategy, + dialect: Dialect | undefined, + skipRedundant: boolean, ): CheckDescriptor[] { const checks: CheckDescriptor[] = []; for (const field of entity.fields()) { @@ -840,19 +945,30 @@ function buildChecks( } else { expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; } - checks.push({ name: `${tableName}_${col}_chk`, expression }); + const enumCheck = { name: `${tableName}_${col}_chk`, expression }; + checks.push(withDeclaredName(enumCheck, field)); } } // Validator-derived checks. + // The VARCHAR(n) the column already is, when the project skips what it enforces. Not on + // SQLite / D1: they accept a declared length and store longer values anyway. + const columnType = subtypeToSqlType(field); + const enforcesLength = !isSqliteFamily(dialect); + const enforcedMax = skipRedundant && enforcesLength && columnType.kind === "text" ? columnType.maxLength : undefined; for (const v of field.validators()) { - const check = validatorCheck(v, qcol, tableName, col, dialect); - if (check) checks.push(check); + // A declared @constraintName asks for the CHECK by name, so it is kept even where the + // project's `migrate.checks.skipRedundant` would leave it out: a declaration beats a default. + const keep = declaredCheckName(v) !== undefined; + const check = keep + ? validatorCheck(v, qcol, tableName, col, dialect) + : validatorCheck(v, qcol, tableName, col, dialect, skipRedundant, enforcedMax); + if (check) checks.push(withDeclaredName(check, v)); } } // Entity-scoped cross-field validators (comparison / requiredWhen / presentIff / atLeastOne). for (const v of entity.validators()) { const check = crossFieldCheck(v, entity, tableName, strategy, dialect); - if (check) checks.push(check); + if (check) checks.push(withDeclaredName(withCondition(check, v, entity, strategy, dialect), v)); } return checks; } @@ -888,14 +1004,33 @@ function resolveRef( return { qcol: quoteCheckCol(resolveColumnName(field, strategy)), field }; } +/** + * A literal compared with a `field.duration` column, in the form the column stores: the + * interval spelling on Postgres (which refuses a signed ISO-8601 duration), the microsecond + * integer on SQLite and D1. A literal that is not a duration is refused here, where the CHECK is + * built, rather than by the database when it is applied. + */ +function durationLiteralSql(field: MetaData, raw: unknown, dialect: Dialect | undefined, where: string): string { + const micros = typeof raw === "string" ? parseDurationLiteral(raw) : undefined; + if (micros === undefined) { + throw new Error( + `${where} compares field.duration '${field.name}' with ${JSON.stringify(raw)}, which is not an ISO-8601 ` + + `duration (such as PT1H30M; no years or months) — cannot build the CHECK.`, + ); + } + if (dialect === "sqlite" || dialect === "d1") return String(micros); + return `'${formatPostgresInterval(micros)}'`; +} + /** * Render an @equals gating value as a SQL literal, typed by the gating field's * subtype: boolean → TRUE/FALSE (1/0 on sqlite/d1), numeric → bare number, - * everything else → quoted string. + * a duration → the form its column stores, everything else → quoted string. */ function renderEquals( raw: unknown, whenField: ReturnType[number], dialect: Dialect | undefined, ): string { + if (whenField.subType === FIELD_SUBTYPE_DURATION) return durationLiteralSql(whenField, raw, dialect, "@equals"); const s = String(raw); if (whenField.subType === FIELD_SUBTYPE_BOOLEAN) { const truthy = s === "true" || s === "1" || s === "TRUE"; @@ -909,6 +1044,86 @@ function renderEquals( return `'${s.replace(/'/g, "''")}'`; } +/** A validator's or field.enum's declared CHECK name (`@constraintName`; ADR-0039 resolving), if any. */ +function declaredCheckName(node: MetaData): string | undefined { + const name = node.attr(VALIDATOR_ATTR_CONSTRAINT_NAME); + return typeof name === "string" && name.length > 0 ? name : undefined; +} + +/** The CHECKs whose name was declared (`@constraintName`), so the name checks below apply only + * to names an author wrote: a derived name keeps the behaviour it always had. */ +const DECLARED_CHECKS = new WeakSet(); + +function withDeclaredName(check: CheckDescriptor, node: MetaData): CheckDescriptor { + const name = declaredCheckName(node); + if (name === undefined) return check; + const named = { ...check, name }; + DECLARED_CHECKS.add(named); + return named; +} + +/** + * Refuse a declared CHECK name another constraint on the same table already has, or (on + * Postgres) one over 63 bytes. Runs after the conventions have named primary keys, foreign + * keys and unique constraints. A collision among derived names is left as it always was. + */ +function checkDeclaredCheckNames(tables: readonly TableDescriptor[], dialect: Dialect | undefined): void { + const problems: CheckNameProblem[] = []; + const limited = !isSqliteFamily(dialect); + for (const t of tables) { + const declared = new Set(t.checks.filter((c) => DECLARED_CHECKS.has(c)).map((c) => c.name)); + if (declared.size === 0) continue; + const owners = new Map(); + const reported = new Set(); + const claim = (name: string, label: string): void => { + const prior = owners.get(name); + if (prior === undefined) { + owners.set(name, label); + } else if (declared.has(name) && !reported.has(name)) { + reported.add(name); + problems.push({ table: t.name, name, duplicateOf: prior }); + } + }; + if (t.primaryKey.length > 0) claim(t.primaryKeyName ?? `${t.name}_pkey`, "the primary key"); + for (const fk of t.foreignKeys) claim(fk.name, `the foreign key "${fk.name}"`); + for (const ix of t.indexes) if (ix.constraint === "unique") claim(ix.name, `the unique constraint "${ix.name}"`); + for (const c of t.checks) claim(c.name, `another CHECK named "${c.name}"`); + if (!limited) continue; + for (const name of declared) { + const bytes = Buffer.byteLength(name, "utf8"); + if (bytes > PG_IDENTIFIER_MAX_BYTES) problems.push({ table: t.name, name, bytes }); + } + } + if (problems.length > 0) throw new InvalidCheckNameError(problems); +} + +/** + * A cross-field validator's `@condition` (an attr.expression): the rule applies only when it + * holds, so the CHECK is the negated condition OR the rule. A row the condition is NULL for is + * not checked, as for any CHECK. + */ +function withCondition( + check: CheckDescriptor, v: MetaValidator, entity: MetaObject, strategy: ColumnNamingStrategy, dialect: Dialect | undefined, +): CheckDescriptor { + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + if (condition === undefined || condition === null || typeof condition !== "object") return check; + const column = (name: string): string | undefined => { + const field = entity.fields().find((f) => f.name === name); + return field === undefined ? undefined : quoteCheckCol(resolveColumnName(field, strategy)); + }; + // An int-backed enum (@intValueMap) stores a member as its integer, so a member compared + // with it is rendered as that integer; a duration is rendered in the form its column stores. + const stored = (name: string, value: unknown): string | undefined => { + const field = entity.fields().find((f) => f.name === name); + if (field?.subType === FIELD_SUBTYPE_DURATION) return durationLiteralSql(field, value, dialect, "@condition"); + const intMap = field === undefined ? undefined : intValueMapOf(field); + const n = intMap === undefined || typeof value !== "string" ? undefined : intMap[value]; + return typeof n === "number" ? String(n) : undefined; + }; + const unless = renderExpressionSql(negateExpression(condition as ExprNode), column, dialect, stored); + return { ...check, expression: `(${unless}) OR (${check.expression})` }; +} + /** * Derive a CHECK from an entity-scoped cross-field validator. Every reference is * resolved to its physical column by name; nothing raw is read from metadata. @@ -964,6 +1179,20 @@ function crossFieldCheck( expression: refs.map((r) => `${r!.qcol} IS NOT NULL`).join(" OR "), }; } + case VALIDATOR_SUBTYPE_EXACTLY_ONE: { + // ADR-0039: effective attr — @fields may be inherited via the validator's extends. + const raw = v.attr(VALIDATOR_ATTR_FIELDS); + const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); + const refs = names.map((n) => resolveRef(entity, n, strategy)); + if (refs.length === 0 || refs.some((r) => r === null)) return null; + const firstCol = resolveColumnName(refs[0]!.field, strategy); + // Postgres counts the present fields with num_nonnulls; SQLite and D1 add the presence + // tests, which they evaluate as 0 / 1. + const count = isSqliteFamily(dialect) + ? `(${refs.map((r) => `(${r!.qcol} IS NOT NULL)`).join(" + ")})` + : `num_nonnulls(${refs.map((r) => r!.qcol).join(", ")})`; + return { name: `${tableName}_${firstCol}_exactlyone_chk`, expression: `${count} = 1` }; + } default: return null; } @@ -975,6 +1204,7 @@ function buildForeignKeys( resolveTargetTable: (target: MetaObject) => string | undefined, root: MetaRoot, strategy: ColumnNamingStrategy, + fkCtx: ForeignKeyBuildContext, onConflict?: (conflict: ReferentialActionConflict) => void, ): FkDescriptor[] { const fks: FkDescriptor[] = []; @@ -1042,7 +1272,7 @@ function buildForeignKeys( const constraintName = typeof constraintNameOverride === "string" && constraintNameOverride.length > 0 ? constraintNameOverride - : `${tableName}_${fkCols[0]}_fk`; + : fkCtx.conventions.foreignKeyName({ table: tableName, columns: fkCols, refTable, refColumns, dialect: fkCtx.dialect }); // Guard: ON DELETE SET NULL requires nullable FK columns. validateSetNullNullability(holder, refChild, onDelete, constraintName); @@ -1058,17 +1288,128 @@ function buildForeignKeys( // Two subtypes may declare the same FK column; it folds to ONE column, so ONE constraint. if (fks.some((existing) => existing.name === fk.name)) continue; fks.push(fk); + fkCtx.origins.set(fk, { + entityName: holder.name, referenceName: refChild.name, targetEntityName: targetObj.name, + targetKey: targetObj.resolutionKey(), targetFields: targetFieldNames, + }); } return fks; } +/** What building a table's foreign keys needs beyond the table: the conventions, and where refusals go. */ +interface ForeignKeyBuildContext { + conventions: SchemaConventions; + dialect: Dialect | undefined; + /** Where each built FK came from, for the target check that runs once every table is built. */ + origins: Map; +} + +interface FkOrigin { + entityName: string; + referenceName: string; + targetEntityName: string; + /** The target entity's resolution key, to tell a TPH subtype from the entity owning the table. */ + targetKey: string; + targetFields: string[]; +} + +/** + * Every built FK whose referenced columns are not a unique key of the BUILT target table: its + * primary key, or a unique index with no predicate or expression, as a column set (Postgres + * accepts any order). A target this model does not build — an @unmanaged table, another + * owner's — is skipped: its keys are not known here, and the database is the judge. + */ +function collectFkTargetViolations( + tables: TableDescriptor[], + origins: ReadonlyMap, + provenance: ReadonlyMap, +): FkTargetViolation[] { + const setOf = (columns: readonly string[]) => [...columns].sort().join(","); + const violations: FkTargetViolation[] = []; + for (const t of tables) { + for (const fk of t.foreignKeys) { + // The emitter assumes the referenced table lives in the holder's schema. + const target = tables.find((x) => x.name === fk.refTable && x.schema === t.schema); + if (target === undefined) continue; + const wanted = setOf(fk.refColumns); + const keys = [ + target.primaryKey, + ...target.indexes.filter((ix) => ix.unique && ix.where === undefined && ix.expr === undefined).map((ix) => ix.columns), + ]; + if (keys.some((k) => k.length > 0 && setOf(k) === wanted)) continue; + const origin = origins.get(fk); + if (origin === undefined) continue; + const owner = provenance.get(qualifiedDbName(target)); + const storedIn = owner !== undefined && owner !== origin.targetKey ? owner.split("::").pop() : undefined; + violations.push({ + table: qualifiedDbName(t), + entityName: origin.entityName, + referenceName: origin.referenceName, + targetEntityName: origin.targetEntityName, + targetFields: origin.targetFields, + ...(storedIn !== undefined ? { storedIn } : {}), + }); + } + } + return violations; +} + +/** + * The schema conventions' second pass, over every built table: the primary key's name, the + * kind of each unique key (an FK anywhere in the model may reference it), and the index each + * foreign key gets. FK names were decided while the FKs were built, where a declared + * `@constraintName` is known. The default conventions change nothing here. + */ +function applySchemaConventions(tables: TableDescriptor[], conventions: SchemaConventions, dialect: Dialect | undefined): void { + // An FK references a table in its own schema (the emitter's assumption), so the key is schema-qualified. + const keyOf = (schema: string | undefined, table: string, columns: readonly string[]) => + `${schema ?? ""}.${table}|${[...columns].sort().join(",")}`; + const referenced = new Set(tables.flatMap((t) => t.foreignKeys.map((fk) => keyOf(t.schema, fk.refTable, fk.refColumns)))); + for (const t of tables) { + if (t.primaryKey.length > 0) { + const pkName = conventions.primaryKeyName({ table: t.name, columns: t.primaryKey, dialect }); + if (pkName !== `${t.name}_pkey`) t.primaryKeyName = pkName; + } + for (const ix of t.indexes) { + // Only a plain key can be a UNIQUE constraint: one with a predicate, an expression, a DESC + // column or a non-btree method stays an index whatever the conventions prefer. + if (!canBeUniqueConstraint(ix)) continue; + const kind = conventions.uniqueKeyKind({ table: t.name, columns: ix.columns, dialect, referencedByForeignKey: referenced.has(keyOf(t.schema, t.name, ix.columns)) }); + if (kind === "constraint") ix.constraint = "unique"; + else if (kind === "index") ix.plainIndex = true; + } + const fkIndexes = conventions.foreignKeyIndexes({ + table: t.name, + dialect, + primaryKey: t.primaryKey, + indexes: t.indexes, + foreignKeys: t.foreignKeys.map((fk) => ({ table: t.name, columns: fk.columns, refTable: fk.refTable, refColumns: fk.refColumns, dialect })), + }); + for (const index of fkIndexes) { + if (t.indexes.some((ix) => ix.name === index.name)) { + throw new DuplicateSqlNameError([{ + name: index.name, + owners: [ + `the ${conventions.id} conventions' foreign-key index on [${index.columns.join(", ")}] of table "${t.name}"`, + `an index already named "${index.name}" on table "${t.name}"`, + ], + involvesIndex: true, + }]); + } + t.indexes.push({ name: index.name, columns: [...index.columns], unique: false }); + } + } +} + + /** * Expand a `field.object @storage "flattened"` into one ColumnDescriptor per - * nested field of the referenced value-object, prefixed by the parent field's - * resolved column name + underscore. + * nested field of the referenced value-object, each named by core naming's + * `resolveFlattenedColumnName` (the field's `@flattenedColumnPrefix`, else its resolved + * column and `_`, then the member's own column), so every port spells these + * columns the way this migration creates them. * - * EF OwnsOne pattern: no JSON column for the parent itself; each nested field - * becomes `_` in the owning entity's table. + * EF OwnsOne pattern: no JSON column for the parent itself. */ function flattenObjectField( field: MetaData, strategy: ColumnNamingStrategy, @@ -1076,13 +1417,10 @@ function flattenObjectField( // ADR-0039: resolving — @objectRef may be inherited via extends. const targetObject = refTarget(field, FIELD_ATTR_OBJECT_REF); if (targetObject === undefined || !isMetaObject(targetObject)) return []; - const prefix = resolveColumnName(field, strategy) + "_"; - const cols: ColumnDescriptor[] = []; - for (const nested of targetObject.fields()) { - const inner = buildColumn(nested, /* isPk */ false, /* pkGeneration */ undefined, strategy); - cols.push({ ...inner, name: prefix + inner.name }); - } - return cols; + return targetObject.fields().map((nested) => ({ + ...buildColumn(nested, /* isPk */ false, /* pkGeneration */ undefined, strategy), + name: resolveFlattenedColumnName(field, nested, strategy), + })); } /** @@ -1100,6 +1438,21 @@ const EXPR_DEFAULT_PATTERNS = [ /\(\)/, // anything function-like ]; +/** + * The declared start and name of an increment key's sequence (identity.primary + * @sequenceStart / @sequenceName; ADR-0039: resolving). Only what is declared is set, so only + * that is compared against a live sequence. + */ +function applyDeclaredSequence(col: ColumnDescriptor, pkIdentity: MetaData): void { + const start = pkIdentity.attr(IDENTITY_ATTR_SEQUENCE_START); + const name = pkIdentity.attr(IDENTITY_ATTR_SEQUENCE_NAME); + if (typeof start !== "number" && typeof name !== "string") return; + col.sequence = { + ...(typeof name === "string" ? { name } : {}), + ...(typeof start === "number" ? { start } : {}), + }; +} + function buildColumn( field: MetaData, isPk: boolean, @@ -1124,7 +1477,18 @@ function buildColumn( // against @values by the loader (FR-011 Check 5), so a miss here is unreachable; // throw rather than silently emit the symbol, which would fail only at apply time. const enumIntMap = field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined; - if (enumIntMap !== undefined) { + if (field.subType === FIELD_SUBTYPE_DURATION && col.sqlType.kind === "interval") { + // Postgres refuses a signed ISO-8601 duration as interval input and stores any default in + // its own spelling, so the default is written in that spelling: it reads back unchanged. + const micros = parseDurationLiteral(defaultRaw); + if (micros === undefined) { + throw new Error( + `field.duration '${field.name}' @default '${defaultRaw}' is not an ISO-8601 duration ` + + `(such as PT1H30M; no years or months) — cannot build the column default for '${col.name}'.`, + ); + } + col.default = { kind: "literal", value: formatPostgresInterval(micros) }; + } else if (enumIntMap !== undefined) { const mapped = enumIntMap[defaultRaw]; if (typeof mapped !== "number") { throw new Error( @@ -1165,7 +1529,6 @@ function buildColumn( if (isPk && (pkGeneration === "increment" || pkGeneration === "uuid")) { col.identity = pkGeneration; } - const fieldDesc = readDescription(field); if (fieldDesc !== undefined) col.description = fieldDesc; @@ -1213,6 +1576,7 @@ function arrayElementSqlType(field: MetaData): SqlType | undefined { // ADR-0039: resolving — @localTime may be inherited via extends. return { kind: "timestamp", withTimezone: field.attr(FIELD_ATTR_LOCAL_TIME) !== true }; case FIELD_SUBTYPE_INET: return { kind: "inet" }; + case FIELD_SUBTYPE_DURATION: return { kind: "interval" }; default: return undefined; // object/map → single jsonb column } } @@ -1296,10 +1660,18 @@ function subtypeToSqlType(field: MetaData): SqlType { // stores as text (the native inet column would reject a not-strictly-valid // value at INSERT). ADR-0039: resolving — @lenient may be inherited via extends. case FIELD_SUBTYPE_INET: return field.attr(FIELD_ATTR_LENIENT) === true ? { kind: "text" } : { kind: "inet" }; + case FIELD_SUBTYPE_DURATION: return { kind: "interval" }; // Postgres native INTERVAL; a BIGINT count of microseconds on SQLite // A string-backed field.enum is a text column with a membership CHECK; an // INT-BACKED one (@intValueMap, design D5) stores the mapped integer instead. // The TS/wire type is the member string either way — only the column differs. - case FIELD_SUBTYPE_ENUM: return isIntBackedEnum(field) ? { kind: "integer", bits: 32 } : { kind: "text" }; + // A string-backed one with a length (its @maxLength, else the project's + // enums.maxLength — core's resolveEnumMaxLength, which refuses a member that does not + // fit) is VARCHAR(n); with none it stays unbounded text. + case FIELD_SUBTYPE_ENUM: { + if (isIntBackedEnum(field)) return { kind: "integer", bits: 32 }; + const length = resolveEnumMaxLength(field); + return length === undefined ? { kind: "text" } : { kind: "text", maxLength: length }; + } default: return { kind: "text" }; // unknown → text fallback } } diff --git a/server/typescript/packages/migrate-ts/src/expected.ts b/server/typescript/packages/migrate-ts/src/expected.ts index 2be7b18c73..a759a1a737 100644 --- a/server/typescript/packages/migrate-ts/src/expected.ts +++ b/server/typescript/packages/migrate-ts/src/expected.ts @@ -7,7 +7,7 @@ // connection; keep it that way. export { buildExpectedSchema, buildExpectedSchemaWithProvenance } from "./expected-schema.js"; -export type { BuildExpectedSchemaOptions, ExpectedSchemaWithProvenance, SchemaProvenance } from "./expected-schema.js"; +export type { BuildExpectedSchemaOptions, SchemaBuildSettings, ExpectedSchemaWithProvenance, SchemaProvenance } from "./expected-schema.js"; export { qualifiedDbName } from "./qualified-name.js"; export type { SqlType } from "./sql-type.js"; export type { diff --git a/server/typescript/packages/migrate-ts/src/index.ts b/server/typescript/packages/migrate-ts/src/index.ts index 8104ab2dce..b30ea8d700 100644 --- a/server/typescript/packages/migrate-ts/src/index.ts +++ b/server/typescript/packages/migrate-ts/src/index.ts @@ -14,9 +14,13 @@ export { describeReferentialActionConflict, type ReferentialActionConflict, } from "./referential-actions.js"; -export type { ExpectedSchemaWithProvenance, SchemaProvenance } from "./expected-schema.js"; +export type { ExpectedSchemaWithProvenance, SchemaProvenance, SchemaBuildSettings } from "./expected-schema.js"; export { introspect, introspectPostgres, introspectSqlite } from "./introspect/index.js"; export { diff } from "./diff/index.js"; +export { + defaultConventions, efcoreConventions, resolveSchemaConventions, SCHEMA_CONVENTION_PRESETS, + type SchemaConventions, type KeyContext, type ForeignKeyContext, type TableKeys, +} from "./schema-conventions.js"; export { collectUnmanagedNames } from "./unmanaged.js"; // Per-command scope (`migrate.scope`) — see scope.ts for why the suppression is // two-sided and why the pattern engine stays in @metaobjectsdev/sdk. @@ -57,8 +61,16 @@ export { DuplicateSqlNameError, allowOptionFor, isBlockedChangesError, isPrimaryKeyChangeError, isDeclaredRenameError, isDuplicateSqlNameError, + InvalidCheckNameError, + isInvalidCheckNameError, + isFkTargetNotUniqueError, + isUnknownSchemaConventionsError, + UnknownSchemaConventionsError, + InvalidSchemaConventionsError, + FkTargetNotUniqueError, + type FkTargetViolation, } from "./errors.js"; -export type { SqlNameCollision } from "./errors.js"; +export type { SqlNameCollision, CheckNameProblem } from "./errors.js"; // Drop+add pairs in one table that may be a rename — the CLI's hint text reads it so the // declared rename is offered ahead of the data-losing `--allow drop-column`. @@ -76,8 +88,9 @@ export type { ViewDescriptor, ViewChangeReason, FkAction, NameChange, Change, ChangeKind, ChangeStatus, AllowOptions, AmbiguousChange, AmbiguousResolution, AmbiguousCallback, - DiffResult, DataHazard, DeclaredRename, EmitResult, Dialect, ExprCanonPath, + DiffResult, DataHazard, DeclaredRename, EmitResult, Dialect, ExprCanonPath, SchemaAdvisory, } from "./types.js"; +export { DIALECT_POSTGRES, DIALECT_SQLITE, DIALECT_D1, DIALECTS, isSqliteFamily } from "./types.js"; export type { DiffArgs } from "./diff/index.js"; export type { EmitOptions } from "./emit/index.js"; export type { WriteMigrationOptions, WriteMigrationResult } from "./write-migration.js"; diff --git a/server/typescript/packages/migrate-ts/src/introspect/postgres.ts b/server/typescript/packages/migrate-ts/src/introspect/postgres.ts index 16008bfd43..dcc050c280 100644 --- a/server/typescript/packages/migrate-ts/src/introspect/postgres.ts +++ b/server/typescript/packages/migrate-ts/src/introspect/postgres.ts @@ -50,8 +50,9 @@ export async function introspectPostgres(db: Kysely>): P for (const { schema, name } of tableRefs) { const columns = await readColumns(k, schema, name); - const primaryKey = await readPrimaryKey(k, schema, name); + const { columns: primaryKey, name: primaryKeyName } = await readPrimaryKey(k, schema, name); const description = await readPgCatalogColumnFacts(k, schema, name, columns); + await readPgColumnSequences(k, schema, name, columns); tables.push({ name, schema, @@ -60,6 +61,9 @@ export async function introspectPostgres(db: Kysely>): P foreignKeys: await readPgForeignKeys(k, schema, name), checks: await readPgChecks(k, schema, name), primaryKey, + // Only a name other than Postgres's default `
_pkey`, as the expected side + // records it — a database built without schema conventions reads back unchanged. + ...(primaryKeyName !== undefined && primaryKeyName !== `${name}_pkey` ? { primaryKeyName } : {}), ...(description !== undefined ? { description } : {}), }); } @@ -184,6 +188,7 @@ export function pgTypeToSqlType( // INET (ADR-0036/0037 Wave 3 — field.inet) if (dt === "inet") return { kind: "inet" }; + if (dt === "interval") return { kind: "interval" }; // Unknown types (user-defined enums, citext, ltree, etc.) fall back to text // so we don't blow up on unrecognised types. @@ -529,16 +534,66 @@ async function readPgCatalogColumnFacts( } } +/** + * The sequence behind each auto-increment column — a GENERATED identity's or a legacy + * serial's — found through `pg_depend` (the column OWNS it), with its name and start value + * from `pg_sequence`. Sets `liveSequence` on `columns` in place (never `sequence`, which is + * the model's declaration). Compared only against what the model declares. + */ +async function readPgColumnSequences( + k: RawKysely, schema: string, tableName: string, columns: ColumnDescriptor[], +): Promise { + // pg-mem gap: pg_depend / pg_sequence do not exist there — the query throws and the + // columns carry no sequence, so nothing about one is compared. Only that error is + // swallowed: on a real server any other failure (a permission error) must surface, or + // the diff would fall back to the default sequence name and could rename a sequence that + // does not exist. + try { + const rows = await sql<{ column_name: string; sequence_schema: string; sequence_name: string; seqstart: string | number }>` + SELECT a.attname AS column_name, sn.nspname AS sequence_schema, seq.relname AS sequence_name, ps.seqstart + FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped + JOIN pg_depend d ON d.refobjid = c.oid AND d.refobjsubid = a.attnum + AND d.classid = 'pg_class'::regclass AND d.refclassid = 'pg_class'::regclass + AND d.deptype IN ('a', 'i') + JOIN pg_class seq ON seq.oid = d.objid AND seq.relkind = 'S' + JOIN pg_namespace sn ON sn.oid = seq.relnamespace + JOIN pg_sequence ps ON ps.seqrelid = seq.oid + WHERE n.nspname = ${schema} + AND c.relname = ${tableName} + `.execute(k); + const byName = new Map(columns.map((c) => [c.name, c] as const)); + for (const r of rows.rows) { + const col = byName.get(r.column_name); + if (col === undefined) continue; + col.liveSequence = { + ...(r.sequence_schema !== schema ? { schema: r.sequence_schema } : {}), + name: r.sequence_name, + start: Number(r.seqstart), + }; + } + } catch (err) { + if (!isMissingCatalogRelation(err, ["pg_depend", "pg_sequence"])) throw err; + } +} + +/** pg-mem's error for a catalog relation it does not implement. */ +function isMissingCatalogRelation(err: unknown, relations: readonly string[]): boolean { + const message = err instanceof Error ? err.message : ""; + return relations.some((r) => message.includes(`relation "${r}" does not exist`)); +} + // Typed alias for raw Kysely — avoids per-call `as any` casts in the helpers below. // eslint-disable-next-line @typescript-eslint/no-explicit-any type RawKysely = Kysely; // eslint-disable-next-line @typescript-eslint/no-explicit-any -async function readPrimaryKey(k: Kysely, schema: string, tableName: string): Promise { +async function readPrimaryKey(k: Kysely, schema: string, tableName: string): Promise<{ columns: string[]; name: string | undefined }> { // Uses information_schema only — avoids pg_attribute / pg_constraint joins // that are either missing or return empty in pg-mem. - const rows = await sql<{ column_name: string; ordinal_position: number }>` - SELECT kcu.column_name, kcu.ordinal_position + const rows = await sql<{ column_name: string; ordinal_position: number; constraint_name: string }>` + SELECT kcu.column_name, kcu.ordinal_position, tc.constraint_name FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON kcu.constraint_name = tc.constraint_name @@ -549,7 +604,7 @@ async function readPrimaryKey(k: Kysely, schema: string, tableName: string) AND tc.table_name = ${tableName} ORDER BY kcu.ordinal_position `.execute(k); - return rows.rows.map((r) => r.column_name); + return { columns: rows.rows.map((r) => r.column_name), name: rows.rows[0]?.constraint_name }; } async function readPgIndexes(k: RawKysely, schema: string, table: string): Promise { @@ -703,7 +758,7 @@ export function indexDefKeyExpr(indexdef: string): string { } async function readPgForeignKeys(k: RawKysely, schema: string, table: string): Promise { - // pg-mem gap: information_schema.referential_constraints is not supported — + // pg-mem gap: pg_constraint (and the information_schema views before it) is not supported — // the query returns empty rows or throws. We catch and return [] so other // tests still pass on pg-mem. Real PG (Postgres 16) handles this correctly. let rows: { rows: Array<{ fk_name: string; column_name: string; ref_table: string; ref_column: string; update_rule: string; delete_rule: string; ordinal: number }> }; @@ -717,27 +772,34 @@ async function readPgForeignKeys(k: RawKysely, schema: string, table: string): P delete_rule: string; ordinal: number; }>` - SELECT tc.constraint_name AS fk_name, - kcu.column_name, - ccu.table_name AS ref_table, - ccu.column_name AS ref_column, - rc.update_rule, - rc.delete_rule, - kcu.ordinal_position AS ordinal - FROM information_schema.table_constraints tc - JOIN information_schema.key_column_usage kcu - ON kcu.constraint_name = tc.constraint_name - AND kcu.table_schema = tc.table_schema - JOIN information_schema.referential_constraints rc - ON rc.constraint_name = tc.constraint_name - AND rc.constraint_schema = tc.table_schema - JOIN information_schema.constraint_column_usage ccu - ON ccu.constraint_name = tc.constraint_name - AND ccu.table_schema = tc.table_schema - WHERE tc.constraint_type = 'FOREIGN KEY' - AND tc.table_schema = ${schema} - AND tc.table_name = ${table} - ORDER BY tc.constraint_name, kcu.ordinal_position + -- pg_constraint, not information_schema: constraint_column_usage has no column + -- position, so joining it to key_column_usage paired every FK column with every + -- referenced column (a two-column FK read back as four). conkey and confkey are + -- parallel arrays; unnesting them together keeps each pair and the FK's order. + SELECT con.conname AS fk_name, + a.attname AS column_name, + rt.relname AS ref_table, + ra.attname AS ref_column, + CASE con.confupdtype WHEN 'c' THEN 'CASCADE' WHEN 'n' THEN 'SET NULL' WHEN 'r' THEN 'RESTRICT' + WHEN 'd' THEN 'SET DEFAULT' ELSE 'NO ACTION' END AS update_rule, + CASE con.confdeltype WHEN 'c' THEN 'CASCADE' WHEN 'n' THEN 'SET NULL' WHEN 'r' THEN 'RESTRICT' + WHEN 'd' THEN 'SET DEFAULT' ELSE 'NO ACTION' END AS delete_rule, + key.ord AS ordinal + FROM pg_constraint con + JOIN pg_class t ON t.oid = con.conrelid + JOIN pg_namespace n ON n.oid = t.relnamespace + JOIN pg_class rt ON rt.oid = con.confrelid + CROSS JOIN LATERAL unnest(con.conkey, con.confkey) WITH ORDINALITY AS key(attnum, refattnum, ord) + JOIN pg_attribute a ON a.attrelid = con.conrelid AND a.attnum = key.attnum + JOIN pg_attribute ra ON ra.attrelid = con.confrelid AND ra.attnum = key.refattnum + WHERE con.contype = 'f' + AND n.nspname = ${schema} + AND t.relname = ${table} + -- Same-schema FKs only: FkDescriptor carries no referenced schema and the emitter + -- assumes the holder's, so a cross-schema FK would read as one to a same-named + -- table here. (The information_schema join this replaced excluded them too.) + AND rt.relnamespace = t.relnamespace + ORDER BY con.conname, key.ord `.execute(k); } catch { // pg-mem: referential_constraints not supported — return empty FK list. diff --git a/server/typescript/packages/migrate-ts/src/schema-conventions.ts b/server/typescript/packages/migrate-ts/src/schema-conventions.ts new file mode 100644 index 0000000000..3972191234 --- /dev/null +++ b/server/typescript/packages/migrate-ts/src/schema-conventions.ts @@ -0,0 +1,145 @@ +/** + * Schema conventions: the names and physical choices for database objects that NO CODE + * ADDRESSES — primary-key and foreign-key constraint names, whether a foreign key gets an + * index and what it is called, and whether a unique key is created as an index or a + * constraint. + * + * Names that code does address — tables, views, columns, and any constraint or index a + * program compares against — are not decided here. They resolve in core naming + * (`naming` in `.metaobjects/config.json`) and from metadata overrides, the same in every + * port. A metadata override always beats a convention: `@constraintName` on an + * `identity.reference`, and an index node's own name. + * + * Selected with `migrate.conventions` in `.metaobjects/config.json` (a preset name), or + * supplied as an object by the project (`schemaConventions` in `metaobjects.config.ts`). + * `default` is what migrate has always emitted, so a project that sets nothing sees no + * change. `efcore` is a reference preset: it reproduces one ORM's documented naming (EF + * Core's, for a database first built by EF Core migrations). The mechanism is general; a + * project with another tool's conventions supplies its own object. + * + * Which CHECKs exist is not a convention: that is the project's `migrate.checks` setting, + * passed to the expected-schema build as `skipRedundantChecks`. + */ + +import { InvalidSchemaConventionsError, UnknownSchemaConventionsError } from "./errors.js"; +import { PG_IDENTIFIER_MAX_BYTES, isSqliteFamily, type Dialect, type IndexDescriptor } from "./types.js"; + +/** A key on one table: the table's database name and the key's columns, in key order, and the + * database the schema is built for (undefined is Postgres, as elsewhere in migrate). */ +export interface KeyContext { + table: string; + columns: readonly string[]; + dialect?: Dialect | undefined; +} + +/** A foreign key: its own columns, and the table and columns it references. */ +export interface ForeignKeyContext extends KeyContext { + refTable: string; + refColumns: readonly string[]; +} + +/** A table as the FK-index decision sees it: its keys, its declared indexes, its foreign keys. */ +export interface TableKeys { + table: string; + dialect?: Dialect | undefined; + primaryKey: readonly string[]; + /** The indexes the model declares (unique keys included); the conventions add to these. */ + indexes: readonly { columns: readonly string[]; where?: string; expr?: string }[]; + foreignKeys: readonly ForeignKeyContext[]; +} + +export interface SchemaConventions { + /** Shown in diagnostics; a preset's name, or the project's own. */ + readonly id: string; + primaryKeyName(key: KeyContext): string; + /** Used when the identity.reference declares no `@constraintName`. */ + foreignKeyName(fk: ForeignKeyContext): string; + /** The indexes the conventions add for a table's foreign keys (none, for most conventions). */ + foreignKeyIndexes(table: TableKeys): { name: string; columns: string[] }[]; + /** + * How a unique key (`identity.secondary`) is created. A partial or expression key is always an index. + * `either` creates a plain index and accepts a live key of either kind, so a database whose keys + * are UNIQUE constraints (Drizzle's `unique()`) is left as it is. `index` and `constraint` are a + * preference: a live key of the other kind is converted. + */ + uniqueKeyKind(key: KeyContext & { referencedByForeignKey: boolean }): "index" | "constraint" | "either"; +} + +/** + * EF Core's truncation: a name over the provider's identifier limit keeps one character fewer + * than the limit and ends in `~`. Only Postgres has a limit here; EF Core's SQLite provider + * truncates nothing, so neither does this on SQLite or D1. + */ +function efTruncate(name: string, dialect: Dialect | undefined): string { + if (isSqliteFamily(dialect)) return name; + return name.length <= PG_IDENTIFIER_MAX_BYTES ? name : `${name.slice(0, PG_IDENTIFIER_MAX_BYTES - 1)}~`; +} + +/** True when `columns` are the leading columns of `key`, in order. */ +function startsWith(key: readonly string[], columns: readonly string[]): boolean { + return columns.length > 0 && columns.length <= key.length && columns.every((c, i) => key[i] === c); +} + +export const defaultConventions: SchemaConventions = { + id: "default", + primaryKeyName: ({ table }) => `${table}_pkey`, + foreignKeyName: ({ table, columns }) => `${table}_${columns[0]}_fk`, + foreignKeyIndexes: () => [], + uniqueKeyKind: () => "either", +}; + +export const efcoreConventions: SchemaConventions = { + id: "efcore", + primaryKeyName: ({ table, dialect }) => efTruncate(`PK_${table}`, dialect), + foreignKeyName: ({ table, columns, refTable, dialect }) => efTruncate(`FK_${table}_${refTable}_${columns.join("_")}`, dialect), + // EF Core's ForeignKeyIndexConvention: every foreign key gets an index unless a key or + // another index already starts with its columns. Its AreIndexedBy compares property lists + // only — a filter is a relational annotation the convention never sees, so a partial index + // covers — and an index added later removes a convention index it covers, so the result + // does not depend on the order the foreign keys are declared in. + foreignKeyIndexes: (t) => { + const candidates: string[][] = []; + for (const fk of t.foreignKeys) { + const columns = [...fk.columns]; + if (startsWith(t.primaryKey, columns)) continue; + if (t.indexes.some((ix) => ix.expr === undefined && startsWith(ix.columns, columns))) continue; + if (candidates.some((c) => c.length === columns.length && startsWith(c, columns))) continue; + candidates.push(columns); + } + return candidates + .filter((c) => !candidates.some((o) => o.length > c.length && startsWith(o, c))) + .map((columns) => ({ name: efTruncate(`IX_${t.table}_${columns.join("_")}`, t.dialect), columns })); + }, + // An alternate key (one an FK references) is a UNIQUE constraint; any other unique key is an index. + uniqueKeyKind: ({ referencedByForeignKey }) => (referencedByForeignKey ? "constraint" : "index"), +}; + +const PRESETS: Readonly> = { + [defaultConventions.id]: defaultConventions, + [efcoreConventions.id]: efcoreConventions, +}; + +/** The preset names `migrate.conventions` accepts in `.metaobjects/config.json`. */ +export const SCHEMA_CONVENTION_PRESETS = Object.keys(PRESETS) as readonly string[]; + +/** A preset by name, a project-supplied object as itself, and nothing as `default`. */ +export function resolveSchemaConventions(selected: string | SchemaConventions | undefined): SchemaConventions { + if (selected === undefined) return defaultConventions; + if (typeof selected !== "string") { + const members = ["primaryKeyName", "foreignKeyName", "foreignKeyIndexes", "uniqueKeyKind"] as const; + const missing = members.filter((m) => typeof (selected as unknown as Record)[m] !== "function"); + const id = String((selected as { id?: unknown }).id ?? "(no id)"); + if (missing.length > 0) throw new InvalidSchemaConventionsError(id, missing); + return selected; + } + const preset = PRESETS[selected]; + if (preset === undefined) throw new UnknownSchemaConventionsError(selected, SCHEMA_CONVENTION_PRESETS); + return preset; +} + +/** A unique key a `UNIQUE` constraint can express: plain ascending btree columns, no predicate. */ +export function canBeUniqueConstraint(ix: IndexDescriptor): boolean { + return ix.unique && ix.where === undefined && ix.expr === undefined + && (ix.using === undefined || ix.using === "btree") + && (ix.orders === undefined || ix.orders.every((o) => o === "asc")); +} diff --git a/server/typescript/packages/migrate-ts/src/scope.ts b/server/typescript/packages/migrate-ts/src/scope.ts index b6f4dbb786..6b5f716453 100644 --- a/server/typescript/packages/migrate-ts/src/scope.ts +++ b/server/typescript/packages/migrate-ts/src/scope.ts @@ -52,6 +52,7 @@ import { DEFAULT_DB_SCHEMA_POSTGRES } from "@metaobjectsdev/metadata"; import type { DiffArgs } from "./diff/index.js"; import type { ExpectedSchemaWithProvenance } from "./expected-schema.js"; +import { FkTargetNotUniqueError, type FkTargetViolation } from "./errors.js"; import { qualifiedDbName } from "./qualified-name.js"; import type { SchemaSnapshot } from "./types.js"; @@ -96,6 +97,17 @@ export interface ScopedExpectedSchema { * declares no dependencies. */ importedOutOfScope?: string[]; + /** + * FKs whose target is not a unique key, on tables this run does NOT govern (imported, or + * outside `migrate.scope`): never emitted by this project, so reported rather than + * refused. The ones on governed tables are refused by `scopeExpectedSchema` itself. + */ + ungovernedFkTargetViolations?: FkTargetViolation[]; + /** + * SQLite only: FKs whose target is not a unique key, on any table. SQLite accepts the DDL + * and checks foreign keys only with `PRAGMA foreign_keys` on, so they are reported, not refused. + */ + unenforcedFkTargetViolations?: FkTargetViolation[]; } /** @@ -277,7 +289,7 @@ export function scopeExpectedSchema( ): ScopedExpectedSchema { const imported = opts?.imported; if (inScope === undefined && imported === undefined) { - return { snapshot: built.snapshot, outOfScope: [] }; + return withFkTargetVerdict(built, { snapshot: built.snapshot, outOfScope: [] }); } // PASS 1 — the import partition. An imported object survives only when the @@ -321,7 +333,7 @@ export function scopeExpectedSchema( return false; }; - return { + return withFkTargetVerdict(built, { snapshot: { ...remainder, tables: remainder.tables.filter(governed), @@ -332,5 +344,19 @@ export function scopeExpectedSchema( outOfScope: [...importedOutOfScope, ...scopeOutOfScope], ...(declared.length > 0 ? { declaredSchemas: declared } : {}), ...(imported !== undefined ? { importedOutOfScope } : {}), - }; + }); +} + +/** + * Refuse every FK-target violation on a table the scoped run governs (its FK would be + * emitted and fail at apply), and return the others for the caller to report. + */ +function withFkTargetVerdict(built: ExpectedSchemaWithProvenance, scoped: ScopedExpectedSchema): ScopedExpectedSchema { + const violations = built.fkTargetViolations ?? []; + if (violations.length === 0) return scoped; + if (built.fkTargetsUnenforced !== undefined) return { ...scoped, unenforcedFkTargetViolations: violations }; + const governedTables = new Set(scoped.snapshot.tables.map(qualifiedDbName)); + const governed = violations.filter((v) => governedTables.has(v.table)); + if (governed.length > 0) throw new FkTargetNotUniqueError(governed); + return { ...scoped, ungovernedFkTargetViolations: violations }; } diff --git a/server/typescript/packages/migrate-ts/src/shadow-expected.ts b/server/typescript/packages/migrate-ts/src/shadow-expected.ts index b08ad3f746..2b0bb0a3cd 100644 --- a/server/typescript/packages/migrate-ts/src/shadow-expected.ts +++ b/server/typescript/packages/migrate-ts/src/shadow-expected.ts @@ -21,10 +21,30 @@ export type ShadowCatalog = { catalog: SchemaSnapshot } | { unavailable: string /** Applies `expected` to a scratch database and reads it back (see shadow-canon.ts). */ export type ShadowSchemaSource = (expected: SchemaSnapshot) => Promise; -/** `expected` re-read from `shadow`: each model table replaced by the shadow's table of the same name. */ +/** + * `expected` re-read from `shadow`: each model table replaced by the shadow's table of the same + * name. A column's DECLARED sequence (`sequence`) is the model's, carried across, and the + * shadow's own read of it (`liveSequence`) is dropped: the shadow database names every + * sequence, declared or not, and only what the model declares may be compared or emitted. + */ export function expectedFromShadow(shadow: SchemaSnapshot, expected: SchemaSnapshot): SchemaSnapshot { const byId = new Map(shadow.tables.map((t) => [qualifiedDbName(t), t] as const)); - return { ...expected, tables: expected.tables.map((model) => byId.get(qualifiedDbName(model)) ?? model) }; + return { + ...expected, + tables: expected.tables.map((model) => { + const read = byId.get(qualifiedDbName(model)); + if (read === undefined) return model; + const declared = new Map(model.columns.map((c) => [c.name, c.sequence] as const)); + return { + ...read, + columns: read.columns.map((c) => { + const { liveSequence: _live, sequence: _read, ...rest } = c; + const seq = declared.get(c.name); + return seq === undefined ? rest : { ...rest, sequence: seq }; + }), + }; + }), + }; } /** diff --git a/server/typescript/packages/migrate-ts/src/snapshot/plan.ts b/server/typescript/packages/migrate-ts/src/snapshot/plan.ts index c1c0b7ca22..a96e613189 100644 --- a/server/typescript/packages/migrate-ts/src/snapshot/plan.ts +++ b/server/typescript/packages/migrate-ts/src/snapshot/plan.ts @@ -5,9 +5,10 @@ import { diff, type DiffArgs } from "../diff/index.js"; import { collectUnmanagedNames } from "../unmanaged.js"; import { carryForwardOutOfScope, scopeExpectedSchema, scopedDiffInputs, type ObjectScopePredicate } from "../scope.js"; import type { Dialect, DiffResult, SchemaSnapshot } from "../types.js"; -import type { ExpectedViewInput } from "../expected-schema.js"; +import type { ExpectedViewInput, SchemaBuildSettings } from "../expected-schema.js"; -export interface PlanOfflineArgs extends Pick { +/** `conventions` and `skipRedundantChecks`: the project's schema settings, passed to the build. */ +export interface PlanOfflineArgs extends Pick, SchemaBuildSettings { metadata: MetaData; dialect: Dialect; /** The stored reference snapshot (the "from" side). Use `{ tables: [], views: [] }` for a fresh project. */ @@ -67,6 +68,8 @@ export async function planOffline(args: PlanOfflineArgs): Promise ({ + ...t, + columns: t.columns.map(({ liveSequence: _live, ...c }) => c), + })), + }; +} + export function serializeSnapshot(snapshot: SchemaSnapshot): string { const file: SnapshotFile = { formatVersion: SNAPSHOT_FORMAT_VERSION, - snapshot: canonicalize(snapshot), + snapshot: canonicalize(withoutLiveSequences(snapshot)), }; return stableStringify(file) + "\n"; } diff --git a/server/typescript/packages/migrate-ts/src/sql-type.ts b/server/typescript/packages/migrate-ts/src/sql-type.ts index 63bdfd6e68..82ec80c76f 100644 --- a/server/typescript/packages/migrate-ts/src/sql-type.ts +++ b/server/typescript/packages/migrate-ts/src/sql-type.ts @@ -19,6 +19,7 @@ export type SqlType = | { kind: "blob" } | { kind: "uuid" } | { kind: "inet" } // Postgres-native inet (IPv4/IPv6) — field.inet (ADR-0036/0037 Wave 3) + | { kind: "interval" } // Postgres-native interval — field.duration (a BIGINT count of microseconds elsewhere) | { kind: "array"; element: SqlType }; // native SQL array (e.g. uuid[], text[]) /** @@ -64,6 +65,7 @@ export function sqlTypeEquals(a: SqlType, b: SqlType): boolean { case "blob": case "uuid": case "inet": + case "interval": return true; } } @@ -115,6 +117,7 @@ export function isWidening(from: SqlType, to: SqlType): boolean { case "blob": case "uuid": case "inet": + case "interval": case "timestamp": case "array": // Array element-type changes are not auto-widened — require explicit allow. diff --git a/server/typescript/packages/migrate-ts/src/types.ts b/server/typescript/packages/migrate-ts/src/types.ts index a7527d5ce8..15a6bdeb0d 100644 --- a/server/typescript/packages/migrate-ts/src/types.ts +++ b/server/typescript/packages/migrate-ts/src/types.ts @@ -39,6 +39,13 @@ export interface TableDescriptor { foreignKeys: FkDescriptor[]; checks: CheckDescriptor[]; primaryKey: string[]; // column names; [] if none + /** + * The primary-key constraint's name, when the project's schema conventions name it + * something other than `
_pkey` (absent = `
_pkey`, which keeps every + * existing snapshot byte-identical). Rendered at create; on Postgres the diff compares it + * and renames a live key whose name differs (`rename-constraints`). + */ + primaryKeyName?: string; /** * Human-readable description threaded from entity `@description`. * Postgres: emitted as `COMMENT ON TABLE … IS '…';` after CREATE TABLE. @@ -61,6 +68,20 @@ export interface ColumnDescriptor { udtName?: string; default?: ColumnDefault; identity?: "increment" | "uuid"; + /** + * The sequence an `increment` identity draws from, as the MODEL declares it: only what the + * identity.primary declares (`@sequenceStart` → `start`, `@sequenceName` → `name`). The only + * sequence fact the emitter renders and a snapshot records, and the only one compared, so an + * undeclared start or name never produces a change or a different byte of SQL. + */ + sequence?: { name?: string; start?: number }; + /** + * Postgres introspection only: the sequence the live column actually owns (`pg_depend`), its + * schema, name and start (`pg_sequence`). Read by the diff to compare against `sequence` and + * to address an `alter-sequence`; never emitted, never written to a snapshot, so it changes + * nothing for a model that declares no sequence. + */ + liveSequence?: { schema?: string; name: string; start: number }; /** * Human-readable description threaded from field `@description`. * Postgres: emitted as `COMMENT ON COLUMN … IS '…';` after CREATE TABLE or ADD COLUMN. @@ -112,16 +133,23 @@ export interface IndexDescriptor { * refuses `DROP INDEX` on such an index ("cannot drop index X because constraint * X on table Y requires it"), so the emitter must drop the CONSTRAINT instead. * - * Never authored in metadata and never present on the expected side — it describes - * how the database happens to hold an index, not what the model asks for. It is - * therefore deliberately NOT compared by `indexEquals`: a unique index and a unique - * constraint over the same columns are the same model-level thing. + * Never authored in metadata. On the expected side it is set only by the project's + * schema conventions (`uniqueKeyKind`), so the emitter creates the key as a UNIQUE + * constraint (EF Core's alternate keys are constraints). It is deliberately NOT + * compared by `indexEquals`: a unique index and a unique constraint over the same + * columns are the same model-level thing, so an existing database never churns. * * This matters far beyond an edge case: Drizzle's `unique()` produces constraints, * so every adopter migrating from a Drizzle-managed schema has constraint-backed * unique indexes (#285). */ constraint?: "unique" | "primary" | "exclude"; + /** + * EXPECTED-ONLY. Set when the schema conventions want this unique key as a plain index + * (`uniqueKeyKind` answers `index`), so a live UNIQUE constraint over it is converted. Absent, + * a live key of either kind satisfies the model. + */ + plainIndex?: true; } export interface CheckDescriptor { @@ -251,6 +279,25 @@ export type Change = indexRenames?: NameChange[]; status: ChangeStatus; } + | { + kind: "rename-constraints"; table: string; schema?: string; + /** + * Constraints (FK / CHECK / constraint-backed index) on a table that is NOT renamed, + * whose definition is unchanged and whose name differs — e.g. after the project's + * schema conventions changed. Renamed in place, never dropped and re-created. + * Postgres only; sqlite/d1 cannot rename a constraint and keep the drop+add. + */ + constraintRenames?: NameChange[]; + /** Plain indexes on that table whose definition is unchanged and whose name differs. Postgres only. */ + indexRenames?: NameChange[]; + /** + * Plain unique indexes the schema conventions now want as UNIQUE constraints, with the + * same name and definition. Promoted in place (`ADD CONSTRAINT … UNIQUE USING INDEX`), so + * a foreign key that relies on the index keeps it. Postgres only. + */ + uniquePromotions?: IndexDescriptor[]; + status: ChangeStatus; + } | { kind: "add-column"; table: string; schema?: string; column: ColumnDescriptor; status: ChangeStatus } | { kind: "drop-column"; table: string; schema?: string; column: string; restore?: ColumnDescriptor; status: ChangeStatus; @@ -290,7 +337,17 @@ export type Change = from?: ColumnDefault; to?: ColumnDefault; status: ChangeStatus } /** Postgres, shadow path only: an `increment` identity (`GENERATED BY DEFAULT AS IDENTITY`) added or dropped. */ | { kind: "change-column-identity"; table: string; schema?: string; column: string; - from?: "increment"; to?: "increment"; status: ChangeStatus } + from?: "increment"; to?: "increment"; + /** The model's declared sequence for an identity being ADDED (rendered as its options). */ + sequence?: { name?: string; start?: number }; + status: ChangeStatus } + /** + * Postgres: an increment identity's sequence whose declared start or name differs from the + * live one, altered in place (`ALTER SEQUENCE … START WITH` / `RENAME TO`). `sequence` is the + * live name the change addresses; each pair is set only when it differs. + */ + | { kind: "alter-sequence"; table: string; schema?: string; column: string; sequence: string; + sequenceSchema?: string; fromStart?: number; toStart?: number; toName?: string; status: ChangeStatus } /** * Postgres, shadow path only: a table comment (no `column`) or column comment set, changed or * cleared. Clearing one the model does not declare is gated behind `allow.dropComment`. @@ -569,8 +626,24 @@ export interface DiffResult { exprCanonPath?: ExprCanonPath; /** Postgres: why no scratch (shadow) database was used, when one was asked for and could not be. */ shadowUnavailable?: string; + /** Things that are not drift but deserve a word: see {@link SchemaAdvisory}. Empty when none. */ + advisories: SchemaAdvisory[]; } +/** + * Not drift, but a better model exists. + * - `interval-as-string`: a live Postgres `interval` column the model declares as something + * other than a `field.duration` (a `field.string`, the only way to model one before + * `field.duration` existed). It is compared as text, so it does not drift; modelling it as + * `field.duration` gives the native type and span ordering. + * - `sequence-not-applied`: a column created on SQLite or D1 whose primary key declares + * `@sequenceStart` or `@sequenceName`. SQLite has no sequence, so neither is applied; the + * key counts from 1. Reported when the column is created, not on every run. + */ +export type SchemaAdvisory = + | { kind: "interval-as-string"; table: string; schema?: string; column: string } + | { kind: "sequence-not-applied"; table: string; schema?: string; column: string }; + /** * Where Postgres's own spelling of an expected expression came from. * - `shadow`: the expected schema applied to a scratch database (shadow-canon.ts). @@ -628,4 +701,16 @@ export interface EmitResult { notices?: readonly string[]; } -export type Dialect = "postgres" | "sqlite" | "d1"; +export const DIALECT_POSTGRES = "postgres"; +export const DIALECT_SQLITE = "sqlite"; +export const DIALECT_D1 = "d1"; +export const DIALECTS = [DIALECT_POSTGRES, DIALECT_SQLITE, DIALECT_D1] as const; +export type Dialect = (typeof DIALECTS)[number]; + +/** SQLite and D1 (which is SQLite) share one SQL dialect. */ +export function isSqliteFamily(dialect: Dialect | undefined): boolean { + return dialect === DIALECT_SQLITE || dialect === DIALECT_D1; +} + +/** Postgres's identifier limit, in bytes (NAMEDATALEN - 1). The server truncates a longer name. */ +export const PG_IDENTIFIER_MAX_BYTES = 63; diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-field-object-storage.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-field-object-storage.test.ts index 06d16e9746..9324b15aca 100644 --- a/server/typescript/packages/migrate-ts/test/expected-schema-field-object-storage.test.ts +++ b/server/typescript/packages/migrate-ts/test/expected-schema-field-object-storage.test.ts @@ -39,6 +39,31 @@ describe("buildExpectedSchema — field.object @storage", () => { expect(cols).not.toContain("shipping_address"); }); + test("@flattenedColumnPrefix replaces the default prefix, verbatim — \"\" keeps the members' own columns", async () => { + const root = await loadDoc({ + "metadata.root": { + package: "test", + children: [ + { "object.value": { name: "Address", children: [ + { "field.string": { name: "street" } }, + { "field.string": { name: "postalCode", "@column": "ZIP" } }, + ]}}, + { "object.entity": { name: "Customer", children: [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { name: "id" } }, + { "field.object": { name: "home", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "" } }, + { "field.object": { name: "work", "@objectRef": "Address", "@storage": "flattened", "@flattenedColumnPrefix": "Office" } }, + { "field.object": { name: "billing", "@objectRef": "Address", "@storage": "flattened", "@column": "bill" } }, + { "identity.primary": { "name": "id", "@fields": "id" } }, + ]}}, + ], + }, + }); + const snap = buildExpectedSchema(root, { dialect: "postgres" }); + const cols = snap.tables.find((t) => t.name === "customers")!.columns.map((c) => c.name); + expect(cols).toEqual(["id", "street", "ZIP", "Officestreet", "OfficeZIP", "bill_street", "bill_ZIP"]); + }); + test("@storage \"flattened\" preserves nested @required per column", async () => { const root = await loadDoc({ "metadata.root": { diff --git a/server/typescript/packages/migrate-ts/test/integration/pg-constraint-backed-index-285.test.ts b/server/typescript/packages/migrate-ts/test/integration/pg-constraint-backed-index-285.test.ts index 09c1af2cdb..d450eea642 100644 --- a/server/typescript/packages/migrate-ts/test/integration/pg-constraint-backed-index-285.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/pg-constraint-backed-index-285.test.ts @@ -19,7 +19,10 @@ * The gate is a REAL engine, per this package's doctrine — emitted-SQL inspection is * exactly what missed the bug, since the SQL looked perfectly reasonable: * 1. Build the live shape the way Drizzle does (a UNIQUE *constraint*). - * 2. Diff against metadata that names the key differently → drop-index + add-index. + * 2. Diff against metadata whose key is DEFINED differently (here: partial) → drop-index + * + add-index. (A key that differs only in NAME is renamed in place and never dropped; + * one the conventions also want as a plain index is renamed, then demoted. The last + * three tests below gate those.) * 3. APPLY to the real engine. Pre-fix this throws; that is the regression. * 4. Re-introspect and re-diff → MUST be empty (convergence, not just "it ran"). * 5. Prove uniqueness is still enforced afterwards — a dropped constraint that took @@ -33,6 +36,7 @@ import { Pool } from "pg"; import { Kysely, PostgresDialect, sql } from "kysely"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { defaultConventions } from "../../src/schema-conventions.js"; import { introspectPostgres } from "../../src/introspect/postgres.js"; import { diff } from "../../src/diff/index.js"; import { emit } from "../../src/emit/index.js"; @@ -40,8 +44,9 @@ import { emit } from "../../src/emit/index.js"; const PG_URL = process.env["MIGRATE_TS_PG_URL"]; // `identity.secondary` names the unique key `byMessageId`, while the live constraint is -// called `work_item_message_id_unique` — the rename that triggers drop+add. -const META = JSON.stringify({ +// called `work_item_message_id_unique`. META_PARTIAL also makes the key partial — a change +// of DEFINITION, which is what drops it; META alone is a pure rename. +const meta = (secondary: Record) => JSON.stringify({ "metadata.root": { package: "acme", children: [ @@ -53,13 +58,15 @@ const META = JSON.stringify({ { "field.long": { name: "id" } }, { "field.string": { name: "messageId", "@column": "message_id", "@required": true } }, { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, - { "identity.secondary": { name: "byMessageId", "@fields": ["messageId"] } }, + { "identity.secondary": { name: "byMessageId", "@fields": ["messageId"], ...secondary } }, ], }, }, ], }, }); +const META = meta({}); +const META_PARTIAL = meta({ "@where": "message_id <> ''" }); async function applyRaw(k: Kysely>, sqlText: string): Promise { for (const stmt of sqlText.split(";").map((s) => s.trim()).filter((s) => s.length > 0)) { @@ -115,7 +122,7 @@ describe("#285 — constraint-backed index drops as a constraint (PG)", () => { expect(bare?.constraint).toBeUndefined(); }); - test("REGRESSION: the rename applies against a real engine, converges, and keeps uniqueness", async () => { + test("REGRESSION: a redefined key is dropped as a constraint, applies, converges, and keeps uniqueness", async () => { await sql.raw("DROP TABLE IF EXISTS work_item CASCADE").execute(kysely); await sql .raw( @@ -127,14 +134,14 @@ describe("#285 — constraint-backed index drops as a constraint (PG)", () => { ) .execute(kysely); - const loaded = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + const loaded = await new MetaDataLoader().load([new InMemoryStringSource(META_PARTIAL)]); expect(loaded.errors).toEqual([]); const expectedSchema = buildExpectedSchema(loaded.root, { dialect: "postgres" }); const actual = await introspectPostgres(kysely); const d = await diff({ expected: expectedSchema, actual, dialect: "postgres", allow: { dropIndex: true } }); - // The rename really does present as drop+add — the precondition for the bug. + // The redefinition really does present as drop+add — the precondition for the bug. expect(d.changes.some((c) => c.kind === "drop-index")).toBe(true); const sqlText = emit(d.changes, { dialect: "postgres" }).up; @@ -163,4 +170,100 @@ describe("#285 — constraint-backed index drops as a constraint (PG)", () => { sql.raw(`INSERT INTO work_item (message_id) VALUES ('m-1')`).execute(kysely), ).rejects.toThrow(/unique|duplicate key/i); }); + + test("a key that differs only in NAME is renamed in place: ALTER INDEX RENAME, converges, still unique", async () => { + await sql.raw("DROP TABLE IF EXISTS work_item CASCADE").execute(kysely); + await sql + .raw( + `CREATE TABLE work_item (id serial PRIMARY KEY, message_id text NOT NULL); + CREATE UNIQUE INDEX work_item_message_id_unique ON work_item (message_id)`, + ) + .execute(kysely); + const loaded = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + expect(loaded.errors).toEqual([]); + const expectedSchema = buildExpectedSchema(loaded.root, { dialect: "postgres" }); + + const d = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres" }); + // (The fixture's `serial` id against the model's long also yields a column-type change.) + expect(d.changes.filter((c) => c.kind === "rename-constraints")).toHaveLength(1); + expect(d.changes.some((c) => c.kind === "drop-index" || c.kind === "add-index")).toBe(false); + const sqlText = emit(d.changes, { dialect: "postgres" }).up; + expect(sqlText).toContain('RENAME TO "byMessageId"'); + expect(sqlText).not.toMatch(/DROP (INDEX|CONSTRAINT)/); + + await applyRaw(kysely, sqlText); + const reDiff = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres" }); + expect(reDiff.changes.map((c) => c.kind)).toEqual([]); + await sql.raw(`INSERT INTO work_item (message_id) VALUES ('m-1')`).execute(kysely); + await expect( + sql.raw(`INSERT INTO work_item (message_id) VALUES ('m-1')`).execute(kysely), + ).rejects.toThrow(/unique|duplicate key/i); + }); + + test("a UNIQUE constraint under another name, under the default conventions: renamed and kept a constraint", async () => { + await sql.raw("DROP TABLE IF EXISTS work_item CASCADE").execute(kysely); + await sql + .raw( + `CREATE TABLE work_item ( + id serial PRIMARY KEY, + message_id text NOT NULL, + CONSTRAINT work_item_message_id_unique UNIQUE (message_id) + )`, + ) + .execute(kysely); + const loaded = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + expect(loaded.errors).toEqual([]); + const expectedSchema = buildExpectedSchema(loaded.root, { dialect: "postgres" }); + + // Drizzle's unique() makes constraints; the default conventions accept either kind. + const d = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres" }); + expect(d.changes.some((c) => c.kind === "drop-index" || c.kind === "add-index")).toBe(false); + const sqlText = emit(d.changes, { dialect: "postgres" }).up; + expect(sqlText).toContain('RENAME CONSTRAINT "work_item_message_id_unique" TO "byMessageId"'); + expect(sqlText).not.toMatch(/DROP (INDEX|CONSTRAINT)/); + + await applyRaw(kysely, sqlText); + const live = await introspectPostgres(kysely); + expect(live.tables.find((t) => t.name === "work_item")!.indexes.find((i) => i.name === "byMessageId")?.constraint).toBe("unique"); + const reDiff = await diff({ expected: expectedSchema, actual: live, dialect: "postgres" }); + expect(reDiff.changes.map((c) => c.kind)).toEqual([]); + }); + + test("a UNIQUE constraint under another name, wanted as a plain index: renamed AND demoted in one migration", async () => { + await sql.raw("DROP TABLE IF EXISTS work_item CASCADE").execute(kysely); + await sql + .raw( + `CREATE TABLE work_item ( + id serial PRIMARY KEY, + message_id text NOT NULL, + CONSTRAINT work_item_message_id_unique UNIQUE (message_id) + )`, + ) + .execute(kysely); + const loaded = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + expect(loaded.errors).toEqual([]); + const expectedSchema = buildExpectedSchema(loaded.root, { + dialect: "postgres", + conventions: { ...defaultConventions, id: "plain-unique-index", uniqueKeyKind: () => "index" }, + }); + + // The demotion drops the constraint, so it is gated like any drop-index. + const gated = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres" }); + expect(gated.changes.find((c) => c.kind === "drop-index")?.status.state).toBe("blocked"); + + const d = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres", allow: { dropIndex: true } }); + const sqlText = emit(d.changes, { dialect: "postgres" }).up; + // Renamed first, then dropped as a constraint under its new name and re-created as an index. + expect(sqlText).toContain('RENAME CONSTRAINT "work_item_message_id_unique" TO "byMessageId"'); + expect(sqlText).toMatch(/DROP CONSTRAINT IF EXISTS "byMessageId"/); + expect(sqlText.indexOf("RENAME CONSTRAINT")).toBeLessThan(sqlText.indexOf("DROP CONSTRAINT")); + + await applyRaw(kysely, sqlText); + const reDiff = await diff({ expected: expectedSchema, actual: await introspectPostgres(kysely), dialect: "postgres" }); + expect(reDiff.changes.map((c) => c.kind)).toEqual([]); + await sql.raw(`INSERT INTO work_item (message_id) VALUES ('m-1')`).execute(kysely); + await expect( + sql.raw(`INSERT INTO work_item (message_id) VALUES ('m-1')`).execute(kysely), + ).rejects.toThrow(/unique|duplicate key/i); + }); }); diff --git a/server/typescript/packages/migrate-ts/test/integration/pg-cross-field-checks-roundtrip.test.ts b/server/typescript/packages/migrate-ts/test/integration/pg-cross-field-checks-roundtrip.test.ts new file mode 100644 index 0000000000..12e473fc22 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/integration/pg-cross-field-checks-roundtrip.test.ts @@ -0,0 +1,103 @@ +/** + * Real Postgres: the CHECKs the cross-field validator extensions emit (exactlyOne, a + * @condition with every node kind, an int-backed enum member in a condition) apply, and a + * second diff proposes nothing. The diff here runs WITHOUT the expression-canonicalisation + * probe, so this is the fallback normaliser a read-only `verify --db` role gets: Postgres + * rewrites a BETWEEN and re-parenthesises every comparison, and a spelling the normaliser + * cannot match would drift on every run. + * + * Gated on MIGRATE_TS_PG_URL; skips cleanly when unset. + */ + +import { test, expect, beforeAll, afterAll, describe } from "bun:test"; +import { Pool } from "pg"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { introspectPostgres } from "../../src/introspect/postgres.js"; +import { diff } from "../../src/diff/index.js"; +import { emit } from "../../src/emit/index.js"; + +const PG_URL = process.env["MIGRATE_TS_PG_URL"]; +const realDescribe = PG_URL ? describe : describe.skip; +const TABLE = "xfield_checks_rt"; + +const META = JSON.stringify({ "metadata.root": { package: "acme", children: [{ "object.entity": { name: "Note", children: [ + { "source.rdb": { "@table": TABLE } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "a" } }, { "field.long": { name: "b" } }, { "field.long": { name: "c" } }, + { "field.boolean": { name: "flag" } }, + { "field.string": { name: "label" } }, + { "field.enum": { name: "kind", "@values": ["Owned", "Shared"], "@intValueMap": { Owned: 1, Shared: 2 } } }, + { "field.duration": { name: "wait" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "validator.atLeastOne": { name: "few", "@fields": ["a", "b", "c"], "@constraintName": "CK_Note_Few" } }, + { "validator.exactlyOne": { name: "one", "@fields": ["a", "b"], "@constraintName": "CK_Note_One", + "@condition": { op: "eq", left: { field: "kind" }, right: { value: "Owned" } } } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", "@constraintName": "CK_Note_Cmp", + "@condition": { op: "and", args: [ + { op: "isNotNull", arg: { field: "label" } }, + { op: "not", arg: { op: "eq", left: { field: "flag" }, right: { value: true } } }, + { op: "eq", left: { op: "isNull", arg: { field: "c" } }, right: { field: "flag" } }, + ] } } }, + { "validator.requiredWhen": { name: "rw", "@field": "c", "@when": "label", "@equals": "x", + "@condition": { op: "gte", left: { fn: "coalesce", args: [{ field: "a" }, { field: "b" }] }, right: { value: 10 } } } }, + // A duration literal is rendered in the interval spelling: Postgres refuses a signed ISO-8601 one. + { "validator.comparison": { name: "waitCmp", "@left": "a", "@op": "lt", "@right": "b", "@constraintName": "CK_Note_Wait", + "@condition": { op: "gt", left: { field: "wait" }, right: { value: "-PT90S" } } } }, + { "validator.requiredWhen": { name: "hourly", "@field": "label", "@when": "wait", "@equals": "PT1H", "@constraintName": "CK_Note_Hourly" } }, +] } }] } }); + +let k: Kysely>; +let pool: Pool; + +if (PG_URL) { + beforeAll(() => { + pool = new Pool({ connectionString: PG_URL }); + k = new Kysely>({ dialect: new PostgresDialect({ pool }) }); + }); + afterAll(async () => { + await sql.raw(`DROP TABLE IF EXISTS "${TABLE}" CASCADE`).execute(k); + await k.destroy(); + }); +} + +const ours = (changes: readonly unknown[]) => + changes.filter((c) => JSON.stringify(c).includes(TABLE)); + +realDescribe("cross-field CHECK extensions — apply, then re-diff with no probe", () => { + test("every emitted CHECK reads back as itself", async () => { + await sql.raw(`DROP TABLE IF EXISTS "${TABLE}" CASCADE`).execute(k); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + expect(errors.map((e) => e.message)).toEqual([]); + const expected = buildExpectedSchema(root, { dialect: "postgres" }); + const first = await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres" }); + const { up } = emit(ours(first.changes) as typeof first.changes, { dialect: "postgres", expectedSchema: expected }); + for (const stmt of up.split(";").map((s) => s.trim()).filter(Boolean)) await sql.raw(stmt).execute(k); + + const followup = ours((await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres" })).changes); + if (followup.length > 0) for (const c of followup) console.error(" -", JSON.stringify(c)); + expect(followup).toEqual([]); + }); + + test("the guarded rule is enforced only when the condition holds", async () => { + await sql.raw(`DELETE FROM "${TABLE}"`).execute(k); + // kind = Shared (2): the exactly-one rule does not apply, so two present is fine. + await sql.raw(`INSERT INTO "${TABLE}" ("a", "b", "kind") VALUES (1, 2, 2)`).execute(k); + // kind = Owned (1): exactly one of a, b. + await expect(sql.raw(`INSERT INTO "${TABLE}" ("a", "b", "kind") VALUES (1, 2, 1)`).execute(k)).rejects.toThrow(/CK_Note_One/); + await sql.raw(`INSERT INTO "${TABLE}" ("a", "kind") VALUES (1, 1)`).execute(k); + // atLeastOne: none of a, b, c present is refused. + await expect(sql.raw(`INSERT INTO "${TABLE}" ("kind") VALUES (2)`).execute(k)).rejects.toThrow(/CK_Note_Few/); + }); + + test("a duration literal in a condition or @equals compares as the span", async () => { + await sql.raw(`DELETE FROM "${TABLE}"`).execute(k); + // wait <= -PT90S: the a < b rule does not apply. + await sql.raw(`INSERT INTO "${TABLE}" ("a", "b", "wait") VALUES (2, 1, '-00:05:00')`).execute(k); + await expect(sql.raw(`INSERT INTO "${TABLE}" ("a", "b", "wait") VALUES (2, 1, '00:00:10')`).execute(k)).rejects.toThrow(/CK_Note_Wait/); + // label is required when wait is exactly an hour, however it is spelled. + await expect(sql.raw(`INSERT INTO "${TABLE}" ("a", "wait") VALUES (1, '60 minutes')`).execute(k)).rejects.toThrow(/CK_Note_Hourly/); + await sql.raw(`INSERT INTO "${TABLE}" ("a", "wait", "label") VALUES (1, '01:00:00', 'set')`).execute(k); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/integration/pg-duration-roundtrip.test.ts b/server/typescript/packages/migrate-ts/test/integration/pg-duration-roundtrip.test.ts new file mode 100644 index 0000000000..47b832a045 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/integration/pg-duration-roundtrip.test.ts @@ -0,0 +1,110 @@ +/** + * Real Postgres: a field.duration column is a native INTERVAL. Its table applies, a value in the + * canonical ISO-8601 form is stored as the span it names, and a second diff proposes nothing — + * including for a declared @default, which Postgres stores in its own interval spelling. + * + * Gated on MIGRATE_TS_PG_URL; skips cleanly when unset. + */ + +import { test, expect, beforeAll, afterAll, describe } from "bun:test"; +import { Pool } from "pg"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import { MetaDataLoader, InMemoryStringSource, formatPostgresInterval, parseDurationLiteral } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { introspectPostgres } from "../../src/introspect/postgres.js"; +import { diff } from "../../src/diff/index.js"; +import { emit } from "../../src/emit/index.js"; + +const PG_URL = process.env["MIGRATE_TS_PG_URL"]; +const realDescribe = PG_URL ? describe : describe.skip; +const TABLE = "duration_rt"; + +const META = JSON.stringify({ "metadata.root": { package: "acme", children: [{ "object.entity": { name: "Timer", children: [ + { "source.rdb": { "@table": TABLE } }, + { "field.long": { name: "id" } }, + { "field.duration": { name: "timeout", "@required": true } }, + { "field.duration": { name: "grace", "@default": "PT1H30M" } }, + { "field.duration": { name: "backoff", isArray: true } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, +] } }] } }); + +let k: Kysely>; + +if (PG_URL) { + beforeAll(() => { + k = new Kysely>({ dialect: new PostgresDialect({ pool: new Pool({ connectionString: PG_URL }) }) }); + }); + afterAll(async () => { + await sql.raw(`DROP TABLE IF EXISTS "${TABLE}" CASCADE`).execute(k); + await k.destroy(); + }); +} + +const ours = (changes: readonly unknown[]) => changes.filter((c) => JSON.stringify(c).includes(TABLE)); + +realDescribe("field.duration on Postgres", () => { + test("the table applies as INTERVAL columns and re-diffs empty", async () => { + await sql.raw(`DROP TABLE IF EXISTS "${TABLE}" CASCADE`).execute(k); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(META)]); + expect(errors.map((e) => e.message)).toEqual([]); + const expected = buildExpectedSchema(root, { dialect: "postgres" }); + const first = await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres" }); + const { up } = emit(ours(first.changes) as typeof first.changes, { dialect: "postgres", expectedSchema: expected }); + expect(up).toContain('"timeout" INTERVAL NOT NULL'); + for (const stmt of up.split(";").map((s) => s.trim()).filter(Boolean)) await sql.raw(stmt).execute(k); + + const types = (await sql.raw( + `SELECT column_name, data_type FROM information_schema.columns WHERE table_name = '${TABLE}' ORDER BY column_name`, + ).execute(k)).rows as Array<{ column_name: string; data_type: string }>; + expect(Object.fromEntries(types.map((r) => [r.column_name, r.data_type]))).toEqual({ + backoff: "ARRAY", grace: "interval", id: "bigint", timeout: "interval", + }); + + const followup = ours((await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres" })).changes); + if (followup.length > 0) for (const c of followup) console.error(" -", JSON.stringify(c)); + expect(followup).toEqual([]); + }); + + test("a value bound in the Postgres spelling is stored as the span it names; the default applies", async () => { + // Postgres reads a non-negative ISO-8601 duration as interval input but refuses a signed one + // (`-PT1M30S`), which is why a string is bound in its own spelling (formatPostgresInterval). + await expect(sql.raw(`SELECT '-PT1M30S'::interval`).execute(k)).rejects.toThrow(/invalid input syntax/); + const bound = [parseDurationLiteral("PT26H0.5S")!, parseDurationLiteral("PT1S")!, parseDurationLiteral("-PT1M30S")!].map(formatPostgresInterval); + await sql.raw(`INSERT INTO "${TABLE}" ("timeout", "backoff") VALUES ('${bound[0]}', ARRAY['${bound[1]}','${bound[2]}']::interval[])`).execute(k); + const row = (await sql.raw( + `SELECT EXTRACT(EPOCH FROM "timeout")::text AS t, EXTRACT(EPOCH FROM "grace")::text AS g, "backoff"::text AS b FROM "${TABLE}"`, + ).execute(k)).rows[0] as Record; + expect(Number(row["t"])).toBe(26 * 3600 + 0.5); + expect(Number(row["g"])).toBe(5400); + expect(row["b"]).toBe('{00:00:01,-00:01:30}'); + // Read back, each element is exactly the spelling it was bound in. + const back = (await sql.raw(`SELECT "timeout"::text AS t FROM "${TABLE}"`).execute(k)).rows[0] as Record; + expect(back["t"]).toBe(bound[0]); + }); + + test("an existing interval column modelled as a string, as it had to be before field.duration, is not drift", async () => { + const legacy = "duration_legacy_rt"; + await sql.raw(`DROP TABLE IF EXISTS "${legacy}" CASCADE`).execute(k); + await sql.raw(`CREATE TABLE "${legacy}" ("id" bigint PRIMARY KEY, "wait" interval, "waits" interval[])`).execute(k); + try { + const doc = JSON.stringify({ "metadata.root": { package: "acme", children: [{ "object.entity": { name: "Legacy", children: [ + { "source.rdb": { "@table": legacy } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "wait" } }, + { "field.string": { name: "waits", isArray: true } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }] } }); + const { root } = await new MetaDataLoader().load([new InMemoryStringSource(doc)]); + const expected = buildExpectedSchema(root, { dialect: "postgres" }); + const result = await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres" }); + expect(result.changes.filter((c) => JSON.stringify(c).includes(legacy))).toEqual([]); + // Not drift, but said: each such column is an advisory pointing at field.duration. + expect(result.advisories.filter((a) => a.table === legacy)).toEqual([ + { kind: "interval-as-string", table: legacy, column: "wait" }, + { kind: "interval-as-string", table: legacy, column: "waits" }, + ]); + } finally { + await sql.raw(`DROP TABLE IF EXISTS "${legacy}" CASCADE`).execute(k); + } + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/integration/postgres-composite-fk-introspect.test.ts b/server/typescript/packages/migrate-ts/test/integration/postgres-composite-fk-introspect.test.ts new file mode 100644 index 0000000000..a62dbd27a0 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/integration/postgres-composite-fk-introspect.test.ts @@ -0,0 +1,87 @@ +/** + * Composite foreign keys introspect as the columns they have, paired in order. + * + * information_schema.constraint_column_usage carries no column position, so joining it + * to key_column_usage pairs every FK column with every referenced column: a two-column + * FK read back as four columns, a three-column one as nine. `meta migrate` reads its + * expected side back from a shadow database, so the cross product reached the emitted + * DDL ("foreign key referenced-columns list must not contain duplicates"). The FK below + * also lists its columns in a different order from the target key, so a reader that + * pairs by position in the target rather than in the FK fails too. + */ +import { describe, test, expect } from "bun:test"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import { Pool } from "pg"; +import { introspectPostgres } from "../../src/introspect/postgres.js"; + +const PG_URL = process.env.MIGRATE_TS_PG_URL; +const d = PG_URL ? describe : describe.skip; + +d("postgres composite FK introspection (real PG)", () => { + test("reads each composite FK column once, paired with its referenced column", async () => { + const k = new Kysely>({ dialect: new PostgresDialect({ pool: new Pool({ connectionString: PG_URL }) }) }); + const p = "cfk_parent_" + Math.random().toString(36).slice(2, 8); + const c = "cfk_child_" + Math.random().toString(36).slice(2, 8); + try { + await sql.raw(`CREATE TABLE ${p} ( id INTEGER PRIMARY KEY, region INTEGER NOT NULL, kind INTEGER NOT NULL )`).execute(k); + await sql.raw(`CREATE UNIQUE INDEX ${p}_ak ON ${p} (id, region, kind)`).execute(k); + await sql.raw( + `CREATE TABLE ${c} ( id INTEGER PRIMARY KEY, p_kind INTEGER, p_id INTEGER, p_region INTEGER, ` + + `CONSTRAINT ${c}_fk FOREIGN KEY (p_kind, p_id, p_region) REFERENCES ${p} (kind, id, region) ON DELETE RESTRICT )`, + ).execute(k); + + const snap = await introspectPostgres(k); + const fk = snap.tables.find((x) => x.name === c)!.foreignKeys.find((f) => f.name === `${c}_fk`)!; + + expect(fk.columns).toEqual(["p_kind", "p_id", "p_region"]); + expect(fk.refColumns).toEqual(["kind", "id", "region"]); + expect(fk.refTable).toBe(p); + expect(fk.onDelete).toBe("restrict"); + } finally { + await sql.raw(`DROP TABLE IF EXISTS ${c}`).execute(k); + await sql.raw(`DROP TABLE IF EXISTS ${p}`).execute(k); + await k.destroy(); + } + }); + + test("reads a primary key's constraint name when it is not
_pkey", async () => { + const k = new Kysely>({ dialect: new PostgresDialect({ pool: new Pool({ connectionString: PG_URL }) }) }); + const named = "pkn_" + Math.random().toString(36).slice(2, 8); + const plain = "pkp_" + Math.random().toString(36).slice(2, 8); + try { + await sql.raw(`CREATE TABLE ${named} ( id INTEGER, CONSTRAINT "PK_${named}" PRIMARY KEY (id) )`).execute(k); + await sql.raw(`CREATE TABLE ${plain} ( id INTEGER PRIMARY KEY )`).execute(k); + + const snap = await introspectPostgres(k); + + expect(snap.tables.find((x) => x.name === named)!.primaryKeyName).toBe(`PK_${named}`); + // the default name is left absent, so a snapshot without conventions is unchanged + expect(snap.tables.find((x) => x.name === plain)!.primaryKeyName).toBeUndefined(); + } finally { + await sql.raw(`DROP TABLE IF EXISTS ${named}`).execute(k); + await sql.raw(`DROP TABLE IF EXISTS ${plain}`).execute(k); + await k.destroy(); + } + }); + + test("an FK into another schema is not read (FkDescriptor has no referenced schema; as before)", async () => { + const k = new Kysely>({ dialect: new PostgresDialect({ pool: new Pool({ connectionString: PG_URL }) }) }); + const a = "xsa_" + Math.random().toString(36).slice(2, 8); + const b = "xsb_" + Math.random().toString(36).slice(2, 8); + try { + await sql.raw(`CREATE SCHEMA ${a}; CREATE SCHEMA ${b}`).execute(k); + await sql.raw(`CREATE TABLE ${b}.users ( id INTEGER PRIMARY KEY )`).execute(k); + await sql.raw(`CREATE TABLE ${a}.users ( id INTEGER PRIMARY KEY )`).execute(k); + await sql.raw(`CREATE TABLE ${a}.profiles ( id INTEGER PRIMARY KEY, user_id INTEGER REFERENCES ${b}.users (id), local_id INTEGER REFERENCES ${a}.users (id) )`).execute(k); + + const snap = await introspectPostgres(k); + const profiles = snap.tables.find((x) => x.name === "profiles" && x.schema === a)!; + + // Only the same-schema FK: the cross-schema one would read as an FK to a.users. + expect(profiles.foreignKeys.map((f) => f.columns)).toEqual([["local_id"]]); + } finally { + await sql.raw(`DROP SCHEMA IF EXISTS ${a} CASCADE; DROP SCHEMA IF EXISTS ${b} CASCADE`).execute(k); + await k.destroy(); + } + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/check-redundant.test.ts b/server/typescript/packages/migrate-ts/test/unit/check-redundant.test.ts new file mode 100644 index 0000000000..2b295be031 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/check-redundant.test.ts @@ -0,0 +1,92 @@ +/** + * CHECK constraints the schema already guarantees. + * + * - `skipRedundantChecks` on the build (`migrate.checks.skipRedundant` in the project config, + * off by default so nothing changes for an existing project): a `validator.length` bound the + * column's VARCHAR(n) already enforces, and a minimum of 0 (always true), emit nothing; the + * rest of the check stands. + * - It is not a schema convention: the efcore conventions alone emit every CHECK. + * - Not on SQLite / D1, which accept a declared length and store longer values anyway: there + * the length CHECK is the only thing enforcing it. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { defaultConventions, efcoreConventions } from "../../src/schema-conventions.js"; +import { baselineFromMetadata, planOffline } from "../../src/snapshot/plan.js"; +import { computeDriftFromActual } from "../../src/drift/drift.js"; + +async function root(lengthValidator: Record, maxLength?: number) { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "code", ...(maxLength !== undefined ? { "@maxLength": maxLength } : {}), children: [ + { "validator.length": { name: "len", ...lengthValidator } }, + ] } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + ] } }, + ] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toHaveLength(0); + return root; +} + +const checksOf = (s: ReturnType) => + s.tables.find((t) => t.name === "tickets")!.checks.map((c) => c.expression); + +const skip = { skipRedundantChecks: true } as const; + +describe("redundant CHECKs", () => { + test("by default: unchanged — the length check is emitted whole", async () => { + const r = await root({ "@min": 0, "@max": 3 }, 3); + expect(checksOf(buildExpectedSchema(r))).toEqual(['length("code") >= 0 AND length("code") <= 3']); + expect(checksOf(buildExpectedSchema(r, { skipRedundantChecks: false }))).toEqual(['length("code") >= 0 AND length("code") <= 3']); + }); + + test("the efcore conventions alone skip nothing", async () => { + const r = await root({ "@min": 0, "@max": 3 }, 3); + expect(checksOf(buildExpectedSchema(r, { conventions: efcoreConventions }))).toEqual(['length("code") >= 0 AND length("code") <= 3']); + expect(checksOf(buildExpectedSchema(r, { conventions: defaultConventions, ...skip }))).toEqual([]); + }); + + test("a max the VARCHAR(n) enforces and a min of 0 emit no check", async () => { + const r = await root({ "@min": 0, "@max": 3 }, 3); + expect(checksOf(buildExpectedSchema(r, skip))).toEqual([]); + }); + + test("a max looser than the VARCHAR(n) is redundant too; a real min stays", async () => { + const r = await root({ "@min": 2, "@max": 10 }, 3); + expect(checksOf(buildExpectedSchema(r, skip))).toEqual(['length("code") >= 2']); + }); + + test("a max tighter than the column, or on an unbounded column, stays", async () => { + expect(checksOf(buildExpectedSchema(await root({ "@max": 2 }, 3), skip))).toEqual(['length("code") <= 2']); + expect(checksOf(buildExpectedSchema(await root({ "@max": 2 }), skip))).toEqual(['length("code") <= 2']); + }); + + test("on SQLite the length check stays, since SQLite does not enforce VARCHAR(n)", async () => { + const r = await root({ "@min": 0, "@max": 3 }, 3); + expect(checksOf(buildExpectedSchema(r, { ...skip, dialect: "sqlite" }))).toEqual(['length("code") <= 3']); + }); +}); + +describe("skipRedundantChecks reaches every build", () => { + test("the offline plan, the baseline and the drift computation pass it to the build", async () => { + const r = await root({ "@min": 0, "@max": 3 }, 3); + const empty = { tables: [], views: [] }; + const tableChecks = (s: { tables: { name: string; checks: unknown[] }[] }) => s.tables.find((t) => t.name === "tickets")!.checks; + + expect(tableChecks((await planOffline({ metadata: r, dialect: "postgres", snapshot: empty, ...skip })).expected)).toEqual([]); + expect(tableChecks((await planOffline({ metadata: r, dialect: "postgres", snapshot: empty })).expected)).toHaveLength(1); + expect(tableChecks(baselineFromMetadata(r, "postgres", undefined, skip))).toEqual([]); + expect(tableChecks(baselineFromMetadata(r, "postgres"))).toHaveLength(1); + + // The database has the CHECK; skipping it makes it extra on the live side, not missing. + const live = buildExpectedSchema(r, { dialect: "postgres" }); + const kinds = async (opts: { skipRedundantChecks?: boolean }) => + (await computeDriftFromActual(live, "postgres", r, opts)).changes.map((c) => c.kind); + expect(await kinds({})).toEqual([]); + expect(await kinds(skip)).toEqual(["drop-check"]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/cross-field-check-extensions.test.ts b/server/typescript/packages/migrate-ts/test/unit/cross-field-check-extensions.test.ts new file mode 100644 index 0000000000..18cf951438 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/cross-field-check-extensions.test.ts @@ -0,0 +1,255 @@ +/** + * CHECKs a database names by hand, an exactly-one presence rule, and a rule that applies in + * one case only: + * - `@constraintName` on a validator or a field.enum is the CHECK's database name; + * - `validator.exactlyOne` is exactly one of its fields present; + * - `@condition` on a cross-field validator: the CHECK is the negated condition OR the rule, + * the negation pushed into the comparison so it reads as written by hand. + */ +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { efcoreConventions } from "../../src/schema-conventions.js"; +import { isInvalidCheckNameError } from "../../src/errors.js"; + +const FIXTURE = join(import.meta.dir, "../../../../../../fixtures/conformance/validator-constraint-name-condition-exactly-one/input/meta.checks.json"); + +async function checks(dialect: "postgres" | "sqlite" = "postgres") { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(readFileSync(FIXTURE, "utf8"))]); + expect(errors).toHaveLength(0); + const t = buildExpectedSchema(root, { dialect }).tables.find((x) => x.name === "StoreHours")!; + return Object.fromEntries(t.checks.map((c) => [c.name, c.expression])); +} + +describe("cross-field CHECK extensions", () => { + test("@constraintName names the enum, field-validator (length, regex, numeric) and cross-field CHECKs", async () => { + expect(Object.keys(await checks()).sort()).toEqual([ + "CK_StoreHours_AnyTime", + "CK_StoreHours_Code", + "CK_StoreHours_CodeShape", + "CK_StoreHours_ExactlyOneLocation", + "CK_StoreHours_Kind", + "CK_StoreHours_SameDayTimes", + "CK_StoreHours_StartDay", + ]); + }); + + test("exactlyOne is exactly one present (Postgres num_nonnulls), under its @condition", async () => { + expect((await checks())["CK_StoreHours_ExactlyOneLocation"]) + .toBe(`("kind" <> 'InStore') OR (num_nonnulls("store_id", "kiosk_id") = 1)`); + }); + + test("exactlyOne on SQLite counts the present fields, under its @condition", async () => { + expect((await checks("sqlite"))["CK_StoreHours_ExactlyOneLocation"]) + .toBe(`("kind" <> 'InStore') OR ((("store_id" IS NOT NULL) + ("kiosk_id" IS NOT NULL)) = 1)`); + }); + + test("atLeastOne takes @constraintName like the other cross-field validators", async () => { + expect((await checks())["CK_StoreHours_AnyTime"]).toBe('"start_time" IS NOT NULL OR "end_time" IS NOT NULL'); + }); + + test("@condition: the negated condition, then the rule", async () => { + expect((await checks())["CK_StoreHours_SameDayTimes"]).toBe('("start_day" <> "end_day") OR ("end_time" > "start_time")'); + }); + + test("exactlyOne with no @constraintName:
__exactlyone_chk", async () => { + const doc = { "metadata.root": { package: "acme", children: [{ "object.entity": { name: "T", children: [ + { "source.rdb": { "@table": "t" } }, { "field.long": { name: "id" } }, + { "field.long": { name: "a" } }, { "field.long": { name: "b" } }, { "field.long": { name: "c" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "validator.exactlyOne": { name: "one", "@fields": ["a", "b", "c"] } }, + ] } }] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toHaveLength(0); + const t = buildExpectedSchema(root, { dialect: "postgres" }).tables[0]!; + expect(t.checks).toEqual([{ name: "t_a_exactlyone_chk", expression: 'num_nonnulls("a", "b", "c") = 1' }]); + }); + + test("exactlyOne over one field is refused at load: that is a required field", async () => { + const doc = { "metadata.root": { package: "acme", children: [{ "object.entity": { name: "T", children: [ + { "source.rdb": { "@table": "t" } }, { "field.long": { name: "id" } }, { "field.long": { name: "a" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "validator.exactlyOne": { name: "one", "@fields": ["a"] } }, + ] } }] } }; + const { errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors.map((e) => (e as { code?: string }).code)).toEqual(["ERR_BAD_ATTR_VALUE"]); + expect(errors[0]!.message).toContain("make that field required"); + }); + + test("no @condition, no @constraintName: unchanged", async () => { + const doc = { "metadata.root": { package: "acme", children: [{ "object.entity": { name: "T", children: [ + { "source.rdb": { "@table": "t" } }, { "field.long": { name: "id" } }, + { "field.long": { name: "a" } }, { "field.long": { name: "b" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"] } }, + ] } }] } }; + const { root } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + const t = buildExpectedSchema(root, { dialect: "postgres" }).tables[0]!; + expect(t.checks).toEqual([{ name: "t_a_atleastone_chk", expression: '"a" IS NOT NULL OR "b" IS NOT NULL' }]); + }); + +}); + +/** An entity "T" (table "t") with an id key and the given children, loaded with no errors. */ +async function load(children: unknown[]) { + const doc = { "metadata.root": { package: "acme", children: [{ "object.entity": { name: "T", children: [ + { "source.rdb": { "@table": "t" } }, { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ...children, + ] } }] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors.map((e) => e.message)).toEqual([]); + return root; +} + +async function checksOf(children: unknown[], dialect: "postgres" | "sqlite" = "postgres") { + const t = buildExpectedSchema(await load(children), { dialect }).tables[0]!; + return t.checks.map((c) => c.expression); +} + +const AB = [{ "field.long": { name: "a" } }, { "field.long": { name: "b" } }]; + +describe("@condition rendering", () => { + test("and / or / not / isNull / isNotNull negate by De Morgan, down to the comparisons", async () => { + const condition = { op: "and", args: [{ op: "isNull", arg: { field: "a" } }, { op: "not", arg: { op: "eq", left: { field: "b" }, right: { value: 1 } } }] }; + expect(await checksOf([...AB, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }])) + .toEqual(['(("a" IS NOT NULL) OR ("b" = 1)) OR ("a" IS NOT NULL OR "b" IS NOT NULL)']); + }); + + test("exactlyOne takes a @condition", async () => { + const condition = { op: "isNotNull", arg: { field: "a" } }; + expect(await checksOf([...AB, { "validator.exactlyOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }])) + .toEqual(['("a" IS NULL) OR (num_nonnulls("a", "b") = 1)']); + }); + + test("an isNotNull condition negates to isNull; or negates to and", async () => { + const condition = { op: "or", args: [{ op: "isNotNull", arg: { field: "a" } }, { op: "gt", left: { field: "b" }, right: { value: 2 } }] }; + expect(await checksOf([...AB, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }])) + .toEqual(['(("a" IS NULL) AND ("b" <= 2)) OR ("a" IS NOT NULL OR "b" IS NOT NULL)']); + }); + + test("coalesce is an operand, not a compound", async () => { + const condition = { op: "eq", left: { fn: "coalesce", args: [{ field: "a" }, { field: "b" }] }, right: { value: 1 } }; + expect(await checksOf([...AB, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }])) + .toEqual(['(COALESCE("a", "b") <> 1) OR ("a" IS NOT NULL OR "b" IS NOT NULL)']); + }); + + test("a compound comparison operand is parenthesised (IS binds looser than =)", async () => { + const flag = { "field.boolean": { name: "flag" } }; + const condition = { op: "eq", left: { op: "isNull", arg: { field: "a" } }, right: { field: "flag" } }; + expect(await checksOf([...AB, flag, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }])) + .toEqual(['(("a" IS NULL) <> "flag") OR ("a" IS NOT NULL OR "b" IS NOT NULL)']); + }); + + test("a boolean literal is 1 / 0 on SQLite", async () => { + const flag = { "field.boolean": { name: "flag" } }; + const condition = { op: "eq", left: { field: "flag" }, right: { value: true } }; + expect(await checksOf([...AB, flag, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }], "sqlite")) + .toEqual(['("flag" <> 1) OR ("a" IS NOT NULL OR "b" IS NOT NULL)']); + }); + + test("an int-backed enum member is compared as its stored integer", async () => { + const kind = { "field.enum": { name: "kind", "@values": ["Owned", "Shared"], "@intValueMap": { Owned: 1, Shared: 2 } } }; + const condition = { op: "eq", left: { field: "kind" }, right: { value: "Owned" } }; + const out = await checksOf([...AB, kind, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@condition": condition } }]); + expect(out).toContain('("kind" <> 1) OR ("a" IS NOT NULL OR "b" IS NOT NULL)'); + }); + + test("requiredWhen and presentIff take a @condition too", async () => { + const kind = { "field.string": { name: "kind" } }; + const condition = { op: "eq", left: { field: "kind" }, right: { value: "x" } }; + const out = await checksOf([...AB, kind, + { "validator.requiredWhen": { name: "rw", "@field": "a", "@when": "b", "@equals": "1", "@condition": condition } }, + { "validator.presentIff": { name: "pi", "@field": "b", "@when": "a", "@equals": "2", "@condition": condition } }, + ]); + expect(out).toEqual([ + `("kind" <> 'x') OR (("b" IS DISTINCT FROM 1) OR ("a" IS NOT NULL))`, + `("kind" <> 'x') OR (("b" IS NOT NULL) = ("a" IS NOT DISTINCT FROM 2))`, + ]); + }); +}); + +describe("declared CHECK names", () => { + const len = (constraintName: string) => ({ "field.string": { name: "code", "@maxLength": 10, children: [ + { "validator.length": { name: "len", "@max": 10, "@constraintName": constraintName } }, + ] } }); + + test("two CHECKs on one table with one declared name are refused", async () => { + const root = await load([...AB, + { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@constraintName": "CK_T_X" } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", "@constraintName": "CK_T_X" } }, + ]); + let err: unknown; + try { buildExpectedSchema(root, { dialect: "postgres" }); } catch (e) { err = e; } + expect(isInvalidCheckNameError(err)).toBe(true); + expect((err as Error).message).toContain('CHECK "CK_T_X" on table "t" has the same name as another CHECK named "CK_T_X"'); + }); + + test("a declared name equal to the primary key's is refused", async () => { + const root = await load([...AB, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@constraintName": "t_pkey" } }]); + expect(() => buildExpectedSchema(root, { dialect: "postgres" })).toThrow(/has the same name as the primary key/); + }); + + test("a declared name over 63 bytes is refused on Postgres, accepted on SQLite", async () => { + const long = "CK_" + "x".repeat(61); + const root = await load([...AB, { "validator.atLeastOne": { name: "one", "@fields": ["a", "b"], "@constraintName": long } }]); + expect(() => buildExpectedSchema(root, { dialect: "postgres" })).toThrow(/is 64 bytes; Postgres keeps only the first 63/); + expect(buildExpectedSchema(root, { dialect: "sqlite" }).tables[0]!.checks.map((c) => c.name)).toEqual([long]); + }); + + test("a declared name keeps a length CHECK skipRedundantChecks would leave out", async () => { + const opts = { dialect: "postgres", conventions: efcoreConventions, skipRedundantChecks: true } as const; + const t = buildExpectedSchema(await load([len("CK_T_Code")]), opts).tables[0]!; + expect(t.checks).toEqual([{ name: "CK_T_Code", expression: 'length("code") <= 10' }]); + const plain = buildExpectedSchema(await load([{ "field.string": { name: "code", "@maxLength": 10, children: [ + { "validator.length": { name: "len", "@max": 10 } }, + ] } }]), opts).tables[0]!; + expect(plain.checks).toEqual([]); + }); +}); + +describe("a duration literal in a CHECK", () => { + const model = [...AB, { "field.duration": { name: "wait" } }, { "field.string": { name: "label" } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", + "@condition": { op: "gt", left: { field: "wait" }, right: { value: "-PT90S" } } } }, + { "validator.requiredWhen": { name: "rw", "@field": "label", "@when": "wait", "@equals": "PT1H" } }, + ]; + + test("is the interval spelling on Postgres and the microsecond integer on SQLite", async () => { + const root = await load(model); + const on = (dialect: "postgres" | "sqlite") => buildExpectedSchema(root, { dialect }).tables[0]!.checks.map((c) => c.expression); + expect(on("postgres")).toEqual([ + `("wait" <= '-00:01:30') OR ("a" < "b")`, + `("wait" IS DISTINCT FROM '01:00:00') OR ("label" IS NOT NULL)`, + ]); + expect(on("sqlite")).toEqual([ + `("wait" <= -90000000) OR ("a" < "b")`, + `("wait" IS NOT 3600000000) OR ("label" IS NOT NULL)`, + ]); + }); + + test("a literal compared with a coalesce of durations takes the column's form too", async () => { + const root = await load([...AB, { "field.duration": { name: "wait" } }, { "field.duration": { name: "hold" } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", + "@condition": { op: "gt", left: { fn: "coalesce", args: [{ field: "wait" }, { field: "hold" }] }, right: { value: "-PT1M" } } } }]); + expect(buildExpectedSchema(root, { dialect: "sqlite" }).tables[0]!.checks.map((c) => c.expression)) + .toEqual([`(COALESCE("wait", "hold") <= -60000000) OR ("a" < "b")`]); + expect(buildExpectedSchema(root, { dialect: "postgres" }).tables[0]!.checks.map((c) => c.expression)) + .toEqual([`(COALESCE("wait", "hold") <= '-00:01:00') OR ("a" < "b")`]); + }); + + test("a literal that is not a duration is refused at load, before any CHECK is built", async () => { + const doc = { "metadata.root": { package: "acme", children: [{ "object.entity": { name: "T", children: [ + { "source.rdb": { "@table": "t" } }, { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ...AB, { "field.duration": { name: "wait" } }, + { "validator.comparison": { name: "cmp", "@left": "a", "@op": "lt", "@right": "b", + "@condition": { op: "gt", left: { field: "wait" }, right: { value: "P1M" } } } }, + ] } }] } }; + const { errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors.map((e) => [(e as { code?: string }).code, e.message])).toEqual([["ERR_BAD_ATTR_VALUE", + 'validator.comparison "T.cmp" @condition compares "wait" with "P1M", which is not an ISO-8601 duration (such as PT1H30M; no years or months).']]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/diff-rename-constraints.test.ts b/server/typescript/packages/migrate-ts/test/unit/diff-rename-constraints.test.ts new file mode 100644 index 0000000000..31659d3e8c --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/diff-rename-constraints.test.ts @@ -0,0 +1,115 @@ +/** + * A constraint or index whose definition is unchanged and whose NAME differs is renamed, + * not dropped and re-created — on a table that is not itself renamed, too. + * + * Switching a project's schema conventions (default → efcore) renames every FK. Before + * this, each one became a gated `drop-fk` plus an `add-fk`: the migration was blocked + * behind `--allow drop-fk`, and allowing it dropped and rebuilt every foreign key on a + * live database. On Postgres the pair is one `ALTER TABLE … RENAME CONSTRAINT`, the same + * fold a table rename already gets. SQLite cannot rename a constraint, so it keeps the + * rebuild there. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { defaultConventions, efcoreConventions } from "../../src/schema-conventions.js"; +import { diff } from "../../src/diff/index.js"; +import { emit } from "../../src/emit/index.js"; + +const DOC = { + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Order", children: [ + { "source.rdb": { "@table": "Orders" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + { "object.entity": { name: "OrderLine", children: [ + { "source.rdb": { "@table": "OrderLines" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.long": { name: "orderId", "@column": "OrderId" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "index.lookup": { name: "IX_OrderLines_OrderId", "@fields": ["orderId"] } }, + { "identity.reference": { name: "order", "@fields": ["orderId"], "@references": "Order" } }, + ] } }, + ], + }, +}; + +async function schemas() { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toHaveLength(0); + return { + live: buildExpectedSchema(root, { conventions: defaultConventions }), + wanted: buildExpectedSchema(root, { conventions: efcoreConventions }), + }; +} + +describe("an FK whose only change is its name", () => { + test("Postgres: one rename-constraints change per table, no drop-fk / add-fk", async () => { + const { live, wanted } = await schemas(); + const { changes } = await diff(wanted, live, { dialect: "postgres" }); + + expect(changes.filter((c) => c.kind === "drop-fk" || c.kind === "add-fk")).toEqual([]); + const renames = changes.filter((c) => c.kind === "rename-constraints"); + expect(renames.map((r) => r.table).sort()).toEqual(["OrderLines", "Orders"]); + expect(renames.find((r) => r.table === "OrderLines")).toMatchObject({ + constraintRenames: [ + { from: "OrderLines_OrderId_fk", to: "FK_OrderLines_Orders_OrderId" }, + { from: "OrderLines_pkey", to: "PK_OrderLines" }, + ], + }); + expect(renames.find((r) => r.table === "Orders")).toMatchObject({ + constraintRenames: [{ from: "Orders_pkey", to: "PK_Orders" }], + }); + }); + + test("Postgres: rendered as RENAME CONSTRAINT, and the down renames it back", async () => { + const { live, wanted } = await schemas(); + const { changes } = await diff(wanted, live, { dialect: "postgres" }); + const { up, down } = emit(changes, { dialect: "postgres" }); + + expect(up).toContain('ALTER TABLE "OrderLines" RENAME CONSTRAINT "OrderLines_OrderId_fk" TO "FK_OrderLines_Orders_OrderId";'); + expect(down).toContain('ALTER TABLE "OrderLines" RENAME CONSTRAINT "FK_OrderLines_Orders_OrderId" TO "OrderLines_OrderId_fk";'); + expect(up).toContain('ALTER TABLE "Orders" RENAME CONSTRAINT "Orders_pkey" TO "PK_Orders";'); + expect(down).toContain('ALTER TABLE "Orders" RENAME CONSTRAINT "PK_Orders" TO "Orders_pkey";'); + expect(up).not.toContain("DROP CONSTRAINT"); + }); + + test("after the rename, the database matches and nothing is proposed", async () => { + const { wanted } = await schemas(); + const { changes } = await diff(wanted, structuredClone(wanted), { dialect: "postgres" }); + expect(changes).toEqual([]); + }); + + test("a primary key the database truncated is compared as stored", async () => { + const { live } = await schemas(); + const long = "T".repeat(60); + const want = structuredClone(live); + want.tables.find((t) => t.name === "Orders")!.name = long; + const actual = structuredClone(want); + actual.tables.find((t) => t.name === long)!.primaryKeyName = `${long}_pk`; + const { changes } = await diff(want, actual, { dialect: "postgres" }); + expect(changes.filter((c) => c.kind === "rename-constraints")).toEqual([]); + }); + + test("an FK whose definition also changed is still dropped and re-added", async () => { + const { live, wanted } = await schemas(); + const actual = structuredClone(live); + actual.tables.find((t) => t.name === "OrderLines")!.foreignKeys[0]!.onDelete = "cascade"; + const { changes } = await diff(wanted, actual, { dialect: "postgres" }); + + expect(changes.some((c) => c.kind === "drop-fk")).toBe(true); + // Only the primary keys are renamed; the changed FK is not folded into a rename. + const renamed = changes.flatMap((c) => (c.kind === "rename-constraints" ? c.constraintRenames ?? [] : [])); + expect(renamed.map((r) => r.to).sort()).toEqual(["PK_OrderLines", "PK_Orders"]); + }); + + test("SQLite does not fold (it cannot rename a constraint)", async () => { + const { live, wanted } = await schemas(); + const { changes } = await diff(wanted, live, { dialect: "sqlite" }); + + expect(changes.some((c) => c.kind === "rename-constraints")).toBe(false); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/diff-unique-key-kind.test.ts b/server/typescript/packages/migrate-ts/test/unit/diff-unique-key-kind.test.ts new file mode 100644 index 0000000000..624b2acda8 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/diff-unique-key-kind.test.ts @@ -0,0 +1,123 @@ +/** + * A unique key whose definition and name are unchanged but which the schema conventions now + * want as the other kind (plain unique index vs UNIQUE constraint). Postgres only. Conventions + * with no preference (the default) accept either kind, so nothing is converted. + * + * Index to constraint is done in place (`ADD CONSTRAINT … UNIQUE USING INDEX`), so a foreign + * key that relies on the index keeps working. Constraint to index has no in-place form: the + * constraint is dropped and the index re-created, and every foreign key onto the key is + * dropped first and added back after, because Postgres refuses to drop a key an FK relies on. + * Those drops are gated like any other. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { defaultConventions, efcoreConventions } from "../../src/schema-conventions.js"; +import { diff } from "../../src/diff/index.js"; +import { emit } from "../../src/emit/index.js"; +import type { SchemaSnapshot } from "../../src/types.js"; + +const DOC = { + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Account", children: [ + { "source.rdb": { "@table": "Accounts" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "code", "@column": "Code", "@required": true } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.secondary": { name: "IX_Accounts_Code", "@fields": ["code"] } }, + ] } }, + { "object.entity": { name: "Ledger", children: [ + { "source.rdb": { "@table": "Ledgers" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "accountCode", "@column": "AccountCode" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.reference": { name: "account", "@fields": ["accountCode"], "@references": "Account.code" } }, + ] } }, + ], + }, +}; + +/** + * The same model with the key as a constraint (efcore), as an index the conventions ask for, and + * as the default conventions leave it (no preference), with every other name held to the wanted one. + */ +async function schemas(): Promise<{ asIndex: SchemaSnapshot; asConstraint: SchemaSnapshot; asEither: SchemaSnapshot }> { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toHaveLength(0); + const asConstraint = buildExpectedSchema(root, { conventions: efcoreConventions }); + const plain = buildExpectedSchema(root, { conventions: defaultConventions }); + const asIndex = structuredClone(asConstraint); + for (const t of asIndex.tables) { + for (const ix of t.indexes) { + const p = plain.tables.find((x) => x.name === t.name)!.indexes.find((x) => x.name === ix.name || x.columns.join() === ix.columns.join()); + if (p !== undefined && p.constraint === undefined) delete ix.constraint; + } + } + const asEither = structuredClone(asIndex); + for (const t of asIndex.tables) for (const ix of t.indexes) if (ix.unique && ix.constraint === undefined) ix.plainIndex = true; + return { asIndex, asConstraint, asEither }; +} + +describe("a unique key that changes only between index and constraint", () => { + test("the fixture really differs only in the key's kind", async () => { + const { asIndex, asConstraint } = await schemas(); + const key = (s: SchemaSnapshot) => s.tables.find((t) => t.name === "Accounts")!.indexes.find((i) => i.unique)!; + expect(key(asConstraint).constraint).toBe("unique"); + expect(key(asIndex).constraint).toBeUndefined(); + }); + + test("index to constraint: promoted in place, and the down demotes it", async () => { + const { asIndex, asConstraint } = await schemas(); + const { changes } = await diff(asConstraint, asIndex, { dialect: "postgres" }); + expect(changes.filter((c) => c.kind !== "rename-constraints")).toEqual([]); + const name = asConstraint.tables.find((t) => t.name === "Accounts")!.indexes.find((i) => i.unique)!.name; + const { up, down } = emit(changes, { dialect: "postgres" }); + expect(up).toContain(`ALTER TABLE "Accounts" ADD CONSTRAINT "${name}" UNIQUE USING INDEX "${name}";`); + expect(down).toContain(`ALTER TABLE "Accounts" DROP CONSTRAINT "${name}";`); + expect(down).toContain(`CREATE UNIQUE INDEX "${name}" ON "Accounts"`); + }); + + test("constraint to index: the FK onto it is dropped first and added back after", async () => { + const { asIndex, asConstraint } = await schemas(); + // Both drops are gated like any other: the key and the FK are briefly gone. + const gated = await diff(asIndex, asConstraint, { dialect: "postgres" }); + expect(gated.changes.filter((c) => c.status.state !== "allowed").map((c) => c.kind).sort()).toEqual(["drop-fk", "drop-index"]); + const { changes } = await diff(asIndex, asConstraint, { dialect: "postgres", allow: { dropIndex: true, dropFk: true } }); + expect(changes.map((c) => c.kind).sort()).toEqual(["add-fk", "add-index", "drop-fk", "drop-index"]); + const { up } = emit(changes, { dialect: "postgres" }); + const at = (s: string) => up.indexOf(s); + expect(at("DROP CONSTRAINT IF EXISTS \"FK_")).toBeLessThan(at("CREATE UNIQUE INDEX")); + expect(at("CREATE UNIQUE INDEX")).toBeLessThan(at("ADD CONSTRAINT \"FK_")); + }); + + test("constraint to index while the FK onto it is renamed: the refit drops and re-adds it under the new name", async () => { + const { asIndex, asConstraint } = await schemas(); + const ledgers = (s: SchemaSnapshot) => s.tables.find((t) => t.name === "Ledgers")!; + const wanted = ledgers(asIndex).foreignKeys[0]!.name; + ledgers(asConstraint).foreignKeys[0]!.name = "ledgers_account_fkey"; + const { changes } = await diff(asIndex, asConstraint, { dialect: "postgres", allow: { dropIndex: true, dropFk: true } }); + const rename = changes.find((c) => c.kind === "rename-constraints" && c.table === "Ledgers"); + expect(rename?.kind === "rename-constraints" ? rename.constraintRenames : undefined) + .toEqual([{ from: "ledgers_account_fkey", to: wanted }]); + expect(changes.flatMap((c) => (c.kind === "drop-fk" ? [c.fk] : []))).toEqual([wanted]); + expect(changes.flatMap((c) => (c.kind === "add-fk" ? [c.fk.name] : []))).toEqual([wanted]); + const { up, down } = emit(changes, { dialect: "postgres" }); + expect(up).not.toContain("DROP CONSTRAINT IF EXISTS \"ledgers_account_fkey\""); + expect(up.indexOf("RENAME CONSTRAINT \"ledgers_account_fkey\"")).toBeLessThan(up.indexOf(`DROP CONSTRAINT IF EXISTS "${wanted}"`)); + expect(down.indexOf(`ADD CONSTRAINT "${wanted}"`)).toBeLessThan(down.indexOf(`RENAME CONSTRAINT "${wanted}" TO "ledgers_account_fkey"`)); + }); + + test("no preference (the default conventions): a live constraint is left as it is", async () => { + const { asEither, asConstraint } = await schemas(); + const { changes } = await diff(asEither, asConstraint, { dialect: "postgres" }); + expect(changes).toEqual([]); + }); + + test("SQLite has no such distinction to compare", async () => { + const { asIndex, asConstraint } = await schemas(); + const { changes } = await diff(asConstraint, asIndex, { dialect: "sqlite" }); + expect(changes).toEqual([]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/duration-column.test.ts b/server/typescript/packages/migrate-ts/test/unit/duration-column.test.ts new file mode 100644 index 0000000000..7275b933a3 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/duration-column.test.ts @@ -0,0 +1,59 @@ +/** + * field.duration's column: a native INTERVAL on Postgres (a @default in Postgres's own interval + * spelling, which it reads back unchanged), and on SQLite the 64-bit microsecond INTEGER the + * runtimes store, so it orders and compares as the span does (a @default is that integer too). + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { emit } from "../../src/emit/index.js"; +import { diff } from "../../src/diff/index.js"; + +const DOC = JSON.stringify({ "metadata.root": { package: "acme", children: [{ "object.entity": { name: "Timer", children: [ + { "source.rdb": { "@table": "timers" } }, + { "field.long": { name: "id" } }, + { "field.duration": { name: "grace", "@default": "-PT90M" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, +] } }] } }); + +async function column(dialect: "postgres" | "sqlite") { + const { root } = await new MetaDataLoader().load([new InMemoryStringSource(DOC)]); + const schema = buildExpectedSchema(root, { dialect }); + return { schema, col: schema.tables[0]!.columns.find((c) => c.name === "grace")! }; +} + +describe("field.duration column", () => { + test("Postgres: INTERVAL, default in the interval spelling", async () => { + const { col } = await column("postgres"); + expect(col.sqlType).toEqual({ kind: "interval" }); + expect(col.default).toEqual({ kind: "literal", value: "-01:30:00" }); + }); + + test("SQLite: a 64-bit INTEGER of microseconds, default included", async () => { + const { schema, col } = await column("sqlite"); + expect(col.sqlType).toEqual({ kind: "integer", bits: 64 }); + expect(col.default).toEqual({ kind: "literal", value: "-5400000000" }); + const { changes } = await diff({ expected: schema, actual: { tables: [], views: [] }, dialect: "sqlite" }); + expect(emit(changes, { dialect: "sqlite", expectedSchema: schema }).up).toContain('"grace" INTEGER DEFAULT -5400000000'); + }); + + test("a live interval modelled as text is read as text with an advisory; modelled as anything else, it is a type change", async () => { + const live = (expected: Awaited>["schema"]) => ({ + ...expected, + tables: expected.tables.map((t) => ({ ...t, columns: t.columns.map((c) => c.name === "grace" ? { ...c, sqlType: { kind: "interval" as const } } : c) })), + }); + const as = async (subType: string) => { + const doc = DOC.replace('"field.duration":{"name":"grace","@default":"-PT90M"}', `"field.${subType}":{"name":"grace"}`); + expect(doc).not.toBe(DOC); + const { root } = await new MetaDataLoader().load([new InMemoryStringSource(doc)]); + const expected = buildExpectedSchema(root, { dialect: "postgres" }); + return diff({ expected, actual: live(expected), dialect: "postgres" }); + }; + const asString = await as("string"); + expect(asString.changes.filter((c) => c.kind === "change-column-type")).toEqual([]); + expect(asString.advisories).toEqual([{ kind: "interval-as-string", table: "timers", column: "grace" }]); + const asInt = await as("int"); + expect(asInt.changes.filter((c) => c.kind === "change-column-type").map((c) => (c as { from: unknown }).from)).toEqual([{ kind: "interval" }]); + expect(asInt.advisories).toEqual([]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/enum-max-length.test.ts b/server/typescript/packages/migrate-ts/test/unit/enum-max-length.test.ts new file mode 100644 index 0000000000..823e69257f --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/enum-max-length.test.ts @@ -0,0 +1,89 @@ +/** + * A string-stored field.enum's column is VARCHAR(n) when it has a length (its @maxLength, + * else the project's enums.maxLength, resolveEnumMaxLength), and unbounded text otherwise, + * as before. An @intValueMap enum stays an integer. A member longer than the project + * default refuses the build before any SQL is emitted. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource, MetaModelError } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { emit } from "../../src/emit/index.js"; +import { diff } from "../../src/diff/index.js"; + +async function schema(enumDecls: Record[], maxLength?: number) { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "tickets" } }, + { "field.long": { name: "id" } }, + ...enumDecls.map((d) => ({ "field.enum": d })), + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + ] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toHaveLength(0); + if (maxLength !== undefined) root.enumDefaults = { maxLength }; + return () => buildExpectedSchema(root); +} + +const column = (s: ReturnType, name: string) => + s.tables.find((t) => t.name === "tickets")!.columns.find((c) => c.name === name)!; + +describe("a string-stored enum's column length", () => { + test("@maxLength → VARCHAR(n) in the CREATE TABLE", async () => { + const s = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"], "@maxLength": 16 }]))(); + expect(column(s, "status").sqlType).toEqual({ kind: "text", maxLength: 16 }); + const { changes } = await diff(s, { tables: [], views: [] } as never, { dialect: "postgres" }); + expect(emit(changes, { dialect: "postgres" }).up).toContain('"status" VARCHAR(16)'); + }); + + test("the project default sizes an enum with no @maxLength", async () => { + const s = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }], 32))(); + expect(column(s, "status").sqlType).toEqual({ kind: "text", maxLength: 32 }); + }); + + test("no length anywhere → unbounded text, as before", async () => { + const s = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }]))(); + expect(column(s, "status").sqlType).toEqual({ kind: "text" }); + }); + + test("an @intValueMap enum stays an integer under a project default", async () => { + const s = (await schema([{ name: "rank", "@values": ["A", "B"], "@intValueMap": { A: 1, B: 2 } }], 32))(); + expect(column(s, "rank").sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("a member longer than the project default refuses the build", async () => { + const build = await schema([{ name: "status", "@values": ["OPEN", "IN_PROGRESS"] }], 10); + expect(build).toThrow(MetaModelError); + expect(build).toThrow(/IN_PROGRESS/); + }); +}); + +describe("turning a length on for an enum column that already exists", () => { + // text → VARCHAR(n) fails on any existing row longer than n, and the member list says + // nothing about rows written before the enum was narrowed, so it is a lossy type change: + // gated behind --allow type-change like any other narrowing. + test("text → VARCHAR(n) is a change-column-type gated as lossy", async () => { + const before = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }]))(); + const after = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }], 32))(); + const { changes } = await diff(after, before, { dialect: "postgres" }); + const change = changes.find((c) => c.kind === "change-column-type"); + expect(change).toMatchObject({ column: "status", from: { kind: "text" }, to: { kind: "text", maxLength: 32 } }); + expect(change?.status).toMatchObject({ state: "blocked" }); + const allowed = await diff(after, before, { dialect: "postgres", allow: { typeChange: true } }); + expect(emit(allowed.changes, { dialect: "postgres" }).up).toContain('ALTER COLUMN "status" TYPE VARCHAR(32)'); + }); + + test("turning it off again (VARCHAR(n) → text) widens, so it is allowed", async () => { + const before = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }], 32))(); + const after = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }]))(); + const { changes } = await diff(after, before, { dialect: "postgres" }); + expect(changes.find((c) => c.kind === "change-column-type")?.status).toEqual({ state: "allowed" }); + }); + + test("SQLite does not enforce a VARCHAR length, so there is nothing to change", async () => { + const before = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }]))(); + const after = (await schema([{ name: "status", "@values": ["OPEN", "CLOSED"] }], 32))(); + const { changes } = await diff(after, before, { dialect: "sqlite" }); + expect(changes.filter((c) => c.kind === "change-column-type")).toEqual([]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/expected-schema-conventions.test.ts b/server/typescript/packages/migrate-ts/test/unit/expected-schema-conventions.test.ts new file mode 100644 index 0000000000..526f74d2f2 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/expected-schema-conventions.test.ts @@ -0,0 +1,171 @@ +/** + * buildExpectedSchema applies the project's schema conventions to the objects no code + * addresses, and the Postgres emitter renders what they decide. A metadata override + * (`@constraintName`, an index node's name) still wins, and the `default` conventions + * leave the expected schema exactly as it has always been. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { efcoreConventions, type SchemaConventions } from "../../src/schema-conventions.js"; +import { InvalidSchemaConventionsError } from "../../src/errors.js"; +import { emit } from "../../src/emit/index.js"; +import { diff } from "../../src/diff/index.js"; + +const DOC = { + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Order", children: [ + { "source.rdb": { "@table": "Orders" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ] } }, + { "object.entity": { name: "OrderLine", children: [ + { "source.rdb": { "@table": "OrderLines" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.long": { name: "orderId", "@column": "OrderId" } }, + { "field.long": { name: "replacesOrderId", "@column": "ReplacesOrderId" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.reference": { name: "order", "@fields": ["orderId"], "@references": "Order" } }, + { "identity.reference": { name: "replaces", "@fields": ["replacesOrderId"], "@references": "Order", "@constraintName": "FK_Legacy_Name" } }, + ] } }, + { "object.entity": { name: "Region", children: [ + { "source.rdb": { "@table": "Regions" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "code", "@column": "Code" } }, + { "field.string": { name: "name", "@column": "Name" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.secondary": { name: "AK_Regions_Code", "@fields": ["code"] } }, + { "identity.secondary": { name: "IX_Regions_Name", "@fields": ["name"] } }, + ] } }, + { "object.entity": { name: "Store", children: [ + { "source.rdb": { "@table": "Stores" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "regionCode", "@column": "RegionCode" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "index.lookup": { name: "IX_Stores_RegionCode_Id", "@fields": ["regionCode", "id"] } }, + { "identity.reference": { name: "region", "@fields": ["regionCode"], "@references": "Region.code" } }, + ] } }, + ], + }, +}; + +async function load() { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DOC))]); + expect(errors).toHaveLength(0); + return root; +} + +const table = (s: ReturnType, name: string) => s.tables.find((t) => t.name === name)!; + +describe("buildExpectedSchema with the efcore conventions", () => { + test("PK and FK names follow EF Core; a declared @constraintName still wins", async () => { + const s = buildExpectedSchema(await load(), { conventions: efcoreConventions }); + + expect(table(s, "Orders").primaryKeyName).toBe("PK_Orders"); + expect(table(s, "OrderLines").foreignKeys.map((f) => f.name).sort()).toEqual(["FK_Legacy_Name", "FK_OrderLines_Orders_OrderId"]); + }); + + test("an FK gets an IX_ index unless an existing index already starts with its columns", async () => { + const s = buildExpectedSchema(await load(), { conventions: efcoreConventions }); + + expect(table(s, "OrderLines").indexes.map((i) => [i.name, i.columns, i.unique])).toEqual([ + ["IX_OrderLines_OrderId", ["OrderId"], false], + ["IX_OrderLines_ReplacesOrderId", ["ReplacesOrderId"], false], + ]); + // IX_Stores_RegionCode_Id already leads with RegionCode + expect(table(s, "Stores").indexes.map((i) => i.name)).toEqual(["IX_Stores_RegionCode_Id"]); + }); + + test("a unique key an FK references is a constraint; another stays an index", async () => { + const s = buildExpectedSchema(await load(), { conventions: efcoreConventions }); + const kinds = Object.fromEntries(table(s, "Regions").indexes.map((i) => [i.name, i.constraint ?? "index"])); + + expect(kinds).toEqual({ AK_Regions_Code: "unique", IX_Regions_Name: "index" }); + }); + + test("the Postgres emitter renders the PK name and the unique constraint", async () => { + const s = buildExpectedSchema(await load(), { conventions: efcoreConventions }); + const { changes } = await diff(s, { tables: [], views: [] }, { dialect: "postgres" }); + const { up, down } = emit(changes, { dialect: "postgres" }); + + expect(up).toContain('CONSTRAINT "PK_Orders" PRIMARY KEY ("Id")'); + expect(up).toContain('ALTER TABLE "Regions" ADD CONSTRAINT "AK_Regions_Code" UNIQUE ("Code");'); + expect(up).toContain('CREATE UNIQUE INDEX "IX_Regions_Name" ON "Regions" ("Name");'); + expect(up).toContain('CREATE INDEX "IX_OrderLines_OrderId" ON "OrderLines" ("OrderId");'); + expect(down).toContain('ALTER TABLE "Regions" DROP CONSTRAINT "AK_Regions_Code";'); + }); +}); + +describe("buildExpectedSchema with the default conventions", () => { + test("is exactly what it was without any conventions", async () => { + const root = await load(); + + expect(buildExpectedSchema(root, { conventions: undefined })).toEqual(buildExpectedSchema(root)); + const s = buildExpectedSchema(root); + expect(table(s, "Orders").primaryKeyName).toBeUndefined(); + expect(table(s, "OrderLines").foreignKeys.map((f) => f.name).sort()).toEqual(["FK_Legacy_Name", "OrderLines_OrderId_fk"]); + expect(table(s, "OrderLines").indexes).toEqual([]); + expect(table(s, "Regions").indexes.every((i) => i.constraint === undefined)).toBe(true); + }); +}); + +describe("conventions come only from the caller", () => { + test("a model never supplies them: no option is the default conventions", async () => { + const root = await load(); + + expect(table(buildExpectedSchema(root), "Orders").primaryKeyName).toBeUndefined(); + expect(table(buildExpectedSchema(root, { conventions: efcoreConventions }), "Orders").primaryKeyName).toBe("PK_Orders"); + }); + + test("a conventions object missing a member is refused by name", async () => { + const root = await load(); + const partial = { id: "acme", primaryKeyName: () => "pk" } as unknown as SchemaConventions; + + expect(() => buildExpectedSchema(root, { conventions: partial })).toThrow(InvalidSchemaConventionsError); + }); +}); + +describe("efcore unique keys that must stay indexes", () => { + const doc = (secondary: Record, extra: unknown[] = []) => ({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Region", children: [ + { "source.rdb": { "@table": "Regions", "@schema": "a" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "code", "@column": "Code" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.secondary": { name: "AK_Regions_Code", "@fields": ["code"], ...secondary } }, + ] } }, + { "object.entity": { name: "Store", children: [ + { "source.rdb": { "@table": "Stores", "@schema": "a" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "regionCode", "@column": "RegionCode" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.reference": { name: "region", "@fields": ["regionCode"], "@references": "Region.code" } }, + ] } }, + ...extra, + ] } }); + const kind = async (d: unknown, schema: string) => { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(d))]); + expect(errors).toHaveLength(0); + const t = buildExpectedSchema(root, { conventions: efcoreConventions }).tables.find((x) => x.name === "Regions" && x.schema === schema)!; + return t.indexes.find((i) => i.name.startsWith("AK_"))!.constraint ?? "index"; + }; + + test("a DESC key or a non-btree method stays an index (a UNIQUE constraint carries neither)", async () => { + expect(await kind(doc({}), "a")).toBe("unique"); + expect(await kind(doc({ "@orders": ["desc"] }), "a")).toBe("index"); + expect(await kind(doc({ "@using": "hash" }), "a")).toBe("index"); + }); + + test("an FK in one schema does not make a same-named key in another schema a constraint", async () => { + const other = { "object.entity": { name: "OtherRegion", children: [ + { "source.rdb": { "@table": "Regions", "@schema": "b" } }, + { "field.long": { name: "id", "@column": "Id" } }, + { "field.string": { name: "code", "@column": "Code" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.secondary": { name: "AK_OtherRegions_Code", "@fields": ["code"] } }, + ] } }; + expect(await kind(doc({}, [other]), "b")).toBe("index"); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/fk-target-unique.test.ts b/server/typescript/packages/migrate-ts/test/unit/fk-target-unique.test.ts new file mode 100644 index 0000000000..1159cf4ac5 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/fk-target-unique.test.ts @@ -0,0 +1,266 @@ +/** + * Guard: a foreign key must reference columns the target table holds unique. + * + * A reference whose dotted @references names target fields other than the target's + * primary key, with nothing declaring those fields unique, used to pass load and emit + * an FK Postgres refuses at apply time ("there is no unique constraint matching given + * keys for referenced table"). buildExpectedSchema() must refuse it before any SQL is + * emitted, naming the reference and the fix. + * + * Unique means: exactly the primary key's fields, exactly the fields of an + * identity.secondary that is a plain unique key (no @where, no @expr), or a single + * field carrying @unique. Field order does not matter; a subset or superset does. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { FkTargetNotUniqueError } from "../../src/errors.js"; +import { buildExpectedSchemaWithProvenance } from "../../src/expected-schema.js"; +import { scopeExpectedSchema } from "../../src/scope.js"; +import { emit } from "../../src/emit/index.js"; +import { diff } from "../../src/diff/index.js"; +import { Database } from "bun:sqlite"; + +async function loadDoc(doc: unknown) { + const { root, errors } = await new MetaDataLoader().load([ + new InMemoryStringSource(JSON.stringify(doc)), + ]); + expect(errors).toHaveLength(0); + return root; +} + +/** A target entity (Person: id, region, code) and a holder (Visit) referencing it. */ +function makeDoc(options: { + references: string; + holderFields: string[]; + targetExtra?: unknown[]; + codeUnique?: boolean; +}) { + return { + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Person", + children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.long": { name: "region" } }, + { "field.string": { name: "code", ...(options.codeUnique ? { "@unique": true } : {}) } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ...(options.targetExtra ?? []), + ], + }, + }, + { + "object.entity": { + name: "Visit", + children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.long": { name: "personId" } }, + { "field.long": { name: "personRegion" } }, + { "field.string": { name: "personCode" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { + "identity.reference": { + name: "ref_person", + "@fields": options.holderFields, + "@references": options.references, + }, + }, + ], + }, + }, + ], + }, + }; +} + +async function build(options: Parameters[0]) { + const root = await loadDoc(makeDoc(options)); + return () => buildExpectedSchema(root); +} + +describe("buildExpectedSchema — an FK must reference a unique key", () => { + test("a non-key target field with nothing unique on it → FkTargetNotUniqueError naming the fix", async () => { + const run = await build({ references: "Person.code", holderFields: ["personCode"] }); + + expect(run).toThrow(FkTargetNotUniqueError); + expect(run).toThrow(/Entity "Visit": reference "ref_person" targets \[code\] on "Person"/); + expect(run).toThrow(/identity\.secondary/); + }); + + test("every offending reference, across tables, in ONE error — not one per run", async () => { + const doc = makeDoc({ references: "Person.code", holderFields: ["personCode"] }); + const children = (doc["metadata.root"] as { children: unknown[] }).children; + children.push({ + "object.entity": { + name: "Badge", + children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.long": { name: "ownerRegion" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "identity.reference": { name: "ref_owner", "@fields": ["ownerRegion"], "@references": "Person.region" } }, + ], + }, + }); + const root = await loadDoc(doc); + + let caught: unknown; + try { buildExpectedSchema(root); } catch (e) { caught = e; } + + expect(caught).toBeInstanceOf(FkTargetNotUniqueError); + const err = caught as FkTargetNotUniqueError; + expect(err.violations.map((v) => `${v.entityName}.${v.referenceName}`).sort()).toEqual(["Badge.ref_owner", "Visit.ref_person"]); + expect(err.message).toContain('reference "ref_person" targets [code] on "Person"'); + expect(err.message).toContain('reference "ref_owner" targets [region] on "Person"'); + }); + + test("a composite target that is not a declared key → throws", async () => { + const run = await build({ references: "Person.id,region", holderFields: ["personId", "personRegion"] }); + + expect(run).toThrow(FkTargetNotUniqueError); + }); + + test("a partial unique index is not a key an FK can reference → throws", async () => { + const run = await build({ + references: "Person.code", + holderFields: ["personCode"], + targetExtra: [{ "identity.secondary": { name: "ux_person_code", "@fields": ["code"], "@where": "\"region\" = 1" } }], + }); + + expect(run).toThrow(FkTargetNotUniqueError); + }); + + test("an identity.secondary on exactly the target fields, in any order → no throw", async () => { + const run = await build({ + references: "Person.id,region", + holderFields: ["personId", "personRegion"], + targetExtra: [{ "identity.secondary": { name: "ak_person_region_id", "@fields": ["region", "id"] } }], + }); + + expect(run).not.toThrow(); + }); + + test("a single target field with @unique → no throw", async () => { + const run = await build({ references: "Person.code", holderFields: ["personCode"], codeUnique: true }); + + expect(run).not.toThrow(); + }); + + test("the target's primary key, bare or dotted → no throw", async () => { + expect(await build({ references: "Person", holderFields: ["personId"] })).not.toThrow(); + expect(await build({ references: "Person.id", holderFields: ["personId"] })).not.toThrow(); + }); + + test("a TPH subtype's unique key is not emitted, so it does not count — the hint names the base", async () => { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Party", "@discriminator": "kind", children: [ + { "source.rdb": { "@table": "parties" } }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "kind", "@values": ["Carrier", "Broker"] } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ] } }, + { "object.entity": { name: "Carrier", extends: "Party", "@discriminatorValue": "Carrier", children: [ + { "field.string": { name: "code" } }, + { "identity.secondary": { name: "ux_carrier_code", "@fields": ["code"] } }, + ] } }, + { "object.entity": { name: "Policy", children: [ + { "source.rdb": { "@table": "policies" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "carrierCode" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "identity.reference": { name: "ref_carrier", "@fields": ["carrierCode"], "@references": "Carrier.code" } }, + ] } }, + ] } }; + const root = await loadDoc(doc); + + expect(() => buildExpectedSchema(root)).toThrow(FkTargetNotUniqueError); + expect(() => buildExpectedSchema(root)).toThrow(/stored in the table of "Party".*declare it on "Party"/); + }); + + test("a bare reference into a composite primary key with one FK field is refused", async () => { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Line", children: [ + { "source.rdb": {} }, + { "field.long": { name: "orderId" } }, + { "field.int": { name: "lineNo" } }, + { "identity.primary": { name: "pk", "@fields": ["orderId", "lineNo"] } }, + ] } }, + { "object.entity": { name: "Note", children: [ + { "source.rdb": {} }, + { "field.long": { name: "id" } }, + { "field.long": { name: "lineOrderId" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + { "identity.reference": { name: "ref_line", "@fields": ["lineOrderId"], "@references": "Line" } }, + ] } }, + ] } }; + + const root = await loadDoc(doc); + + expect(() => buildExpectedSchema(root)).toThrow(FkTargetNotUniqueError); + expect(() => buildExpectedSchema(root)).toThrow(/reference "ref_line" targets \[orderId\] on "Line"/); + }); + + test("a target this model does not build (@unmanaged) is not refused: its keys are not known here", async () => { + const doc = makeDoc({ references: "Person.code", holderFields: ["personCode"] }); + const person = (doc["metadata.root"].children[0] as { "object.entity": { children: unknown[] } })["object.entity"]; + person.children[0] = { "source.rdb": { "@unmanaged": true } }; + const root = await loadDoc(doc); + + expect(() => buildExpectedSchema(root)).not.toThrow(); + }); + + test("through scopeExpectedSchema only a governed holder is refused; the others are returned", async () => { + const root = await loadDoc(makeDoc({ references: "Person.code", holderFields: ["personCode"] })); + const built = buildExpectedSchemaWithProvenance(root); + + expect(built.fkTargetViolations?.map((v) => v.referenceName)).toEqual(["ref_person"]); + expect(() => scopeExpectedSchema(built, undefined)).toThrow(FkTargetNotUniqueError); + const outside = scopeExpectedSchema(built, (fqn) => !fqn.endsWith("Visit")); + expect(outside.ungovernedFkTargetViolations?.map((v) => v.referenceName)).toEqual(["ref_person"]); + }); +}); + +describe("on SQLite an FK onto a non-unique target is reported, not refused", () => { + // SQLite checks foreign keys only with PRAGMA foreign_keys on (off by default), so the + // DDL is accepted and only an enforced write fails. D1 always enforces them, and Postgres + // refuses the DDL itself, so both keep the refusal. + test("buildExpectedSchema on SQLite does not throw; on Postgres and D1 it does", async () => { + const root = await loadDoc(makeDoc({ references: "Person.code", holderFields: ["personCode"] })); + expect(() => buildExpectedSchema(root, { dialect: "sqlite" })).not.toThrow(); + expect(() => buildExpectedSchema(root, { dialect: "d1" })).toThrow(FkTargetNotUniqueError); + expect(() => buildExpectedSchema(root, { dialect: "postgres" })).toThrow(FkTargetNotUniqueError); + }); + + test("scopeExpectedSchema returns them as unenforced, governed or not", async () => { + const root = await loadDoc(makeDoc({ references: "Person.code", holderFields: ["personCode"] })); + const scoped = scopeExpectedSchema(buildExpectedSchemaWithProvenance(root, { dialect: "sqlite" }), undefined); + expect(scoped.unenforcedFkTargetViolations?.map((v) => v.referenceName)).toEqual(["ref_person"]); + expect(scoped.ungovernedFkTargetViolations).toBeUndefined(); + }); + + test("the SQLite DDL applies; a write fails only once foreign keys are enforced", async () => { + const root = await loadDoc(makeDoc({ references: "Person.code", holderFields: ["personCode"] })); + const want = buildExpectedSchema(root, { dialect: "sqlite" }); + const { changes } = await diff(want, { tables: [], views: [] }, { dialect: "sqlite" }); + const { up } = emit(changes, { dialect: "sqlite" }); + const db = new Database(":memory:"); + try { + db.exec(up); + const person = want.tables.find((t) => t.name !== "visit" && t.columns.some((c) => c.name === "code"))!; + const visit = want.tables.find((t) => t !== person)!; + const col = (t: typeof person, n: string) => t.columns.find((c) => c.name.replace(/_/g, "").toLowerCase() === n.toLowerCase())!.name; + db.exec(`INSERT INTO "${person.name}" ("${col(person, "id")}", "${col(person, "region")}", "${col(person, "code")}") VALUES (1, 1, 'a')`); + const insertVisit = `INSERT INTO "${visit.name}" ("${col(visit, "id")}", "${col(visit, "personId")}", "${col(visit, "personRegion")}", "${col(visit, "personCode")}") VALUES (1, 1, 1, 'a')`; + db.exec(insertVisit); + db.exec("PRAGMA foreign_keys = ON"); + expect(() => db.exec(insertVisit.replace("VALUES (1,", "VALUES (2,"))).toThrow(/foreign key mismatch/); + } finally { + db.close(); + } + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/identity-sequence.test.ts b/server/typescript/packages/migrate-ts/test/unit/identity-sequence.test.ts new file mode 100644 index 0000000000..0267d331e9 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/identity-sequence.test.ts @@ -0,0 +1,158 @@ +/** + * identity.primary @sequenceStart / @sequenceName: the sequence an @generation "increment" + * key draws from. Postgres creates it with GENERATED BY DEFAULT AS IDENTITY + * (SEQUENCE NAME … START WITH …); a live sequence whose declared start or name differs is + * altered in place (ALTER SEQUENCE … START WITH / RENAME TO), never dropped. Only what the + * model DECLARES is compared, so a database whose sequence kept an old name after a table + * rename sees no new change until the model names it. SQLite has no sequences to name. + */ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { diff } from "../../src/diff/index.js"; +import { emit } from "../../src/emit/index.js"; +import type { SchemaSnapshot as DatabaseSchema } from "../../src/types.js"; +import { expectedFromShadow } from "../../src/shadow-expected.js"; +import { serializeSnapshot } from "../../src/snapshot/serialize.js"; +import { DuplicateSqlNameError } from "../../src/errors.js"; + +async function schema(pk: Record) { + const doc = { "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Ticket", children: [ + { "source.rdb": { "@table": "Ticket" } }, + { "field.int": { name: "id", "@column": "Id" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment", ...pk } }, + ] } }, + ] } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toHaveLength(0); + return buildExpectedSchema(root); +} + +const EMPTY = { tables: [], views: [] } as unknown as DatabaseSchema; +const idColumn = (s: DatabaseSchema) => s.tables.find((t) => t.name === "Ticket")!.columns.find((c) => c.name === "Id")!; + +/** The live side: the same table, its sequence as Postgres reports it (introspection only). */ +function live(expected: DatabaseSchema, sequence: { name: string; start: number }): DatabaseSchema { + const copy = structuredClone(expected); + delete idColumn(copy).sequence; + idColumn(copy).liveSequence = sequence; + return copy; +} + +describe("identity.primary sequence start and name", () => { + test("the expected column carries only what the model declares", async () => { + expect(idColumn(await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" })).sequence) + .toEqual({ start: 1000000000, name: "Old_Id_seq" }); + expect(idColumn(await schema({})).sequence).toBeUndefined(); + }); + + test("Postgres CREATE names the sequence and its start", async () => { + const s = await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" }); + const { changes } = await diff(s, EMPTY, { dialect: "postgres" }); + expect(emit(changes, { dialect: "postgres" }).up) + .toContain('"Id" INTEGER GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME "Old_Id_seq" START WITH 1000000000)'); + }); + + test("a live sequence with another start and name is altered in place, and the down reverses it", async () => { + const s = await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" }); + const { changes } = await diff(s, live(s, { name: "Ticket_Id_seq", start: 1 }), { dialect: "postgres" }); + expect(changes.filter((c) => c.kind === "alter-sequence")).toHaveLength(1); + const { up, down } = emit(changes, { dialect: "postgres" }); + expect(up).toContain('ALTER SEQUENCE "Ticket_Id_seq" START WITH 1000000000;'); + // START WITH alone only records a restart point: the counter is moved up to it, never down. + expect(up).toContain(`SELECT setval('"Ticket_Id_seq"', 1000000000, false) FROM "Ticket_Id_seq" WHERE last_value < 1000000000;`); + expect(up).toContain('ALTER SEQUENCE "Ticket_Id_seq" RENAME TO "Old_Id_seq";'); + expect(down).toContain('ALTER SEQUENCE "Old_Id_seq" RENAME TO "Ticket_Id_seq";'); + expect(down).toContain('ALTER SEQUENCE "Ticket_Id_seq" START WITH 1;'); + expect(up).not.toMatch(/DROP|IDENTITY/); + }); + + test("a live sequence matching the declaration is no change", async () => { + const s = await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" }); + const { changes } = await diff(s, live(s, { name: "Old_Id_seq", start: 1000000000 }), { dialect: "postgres" }); + expect(changes.filter((c) => c.kind === "alter-sequence")).toEqual([]); + }); + + test("undeclared start and name are not compared, whatever the live sequence says", async () => { + const s = await schema({}); + const { changes } = await diff(s, live(s, { name: "Renamed_Id_seq", start: 500 }), { dialect: "postgres" }); + expect(changes.filter((c) => c.kind === "alter-sequence")).toEqual([]); + }); + + test("SQLite has no sequence to name: no clause, no change, and an advisory when the key is created", async () => { + const s = await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" }); + const { changes, advisories } = await diff(s, EMPTY, { dialect: "sqlite" }); + const { up } = emit(changes, { dialect: "sqlite" }); + expect(up).not.toContain("SEQUENCE"); + expect(up).not.toContain("1000000000"); + expect(advisories).toEqual([{ kind: "sequence-not-applied", table: "Ticket", column: "Id" }]); + // Once the table exists there is nothing to create, so nothing to say on every later run. + expect((await diff(s, s, { dialect: "sqlite" })).advisories).toEqual([]); + // Postgres applies both, so it says nothing. + expect((await diff(s, EMPTY, { dialect: "postgres" })).advisories).toEqual([]); + }); + + test("shadow path: a model that declares nothing gets no sequence clause and no change", async () => { + const model = await schema({}); + // The shadow database's own read of the model's tables: introspection fills liveSequence. + const shadow = live(model, { name: "Ticket_Id_seq", start: 1 }); + const expected = expectedFromShadow(shadow, model); + expect(idColumn(expected).sequence).toBeUndefined(); + // A live sequence left behind by a table rename, with a non-default start: not compared. + const { changes } = await diff(expected, live(model, { name: "OldTicket_Id_seq", start: 1000 }), { dialect: "postgres" }); + expect(changes.filter((c) => c.kind === "alter-sequence")).toEqual([]); + // CREATE from the shadow-read expected side: plain IDENTITY, as before. + const created = await diff(expected, EMPTY, { dialect: "postgres" }); + expect(emit(created.changes, { dialect: "postgres" }).up).not.toContain("SEQUENCE NAME"); + }); + + test("shadow path: what the model declares survives the shadow read", async () => { + const model = await schema({ "@sequenceStart": 1000000000, "@sequenceName": "Old_Id_seq" }); + const expected = expectedFromShadow(live(model, { name: "Old_Id_seq", start: 1000000000 }), model); + expect(idColumn(expected).sequence).toEqual({ start: 1000000000, name: "Old_Id_seq" }); + }); + + test("offline (committed snapshot): a changed declared start is a change, addressed by the default name", async () => { + const before = await schema({ "@sequenceStart": 1000 }); + const after = await schema({ "@sequenceStart": 2000 }); + const { changes } = await diff(after, before, { dialect: "postgres" }); + const alter = changes.filter((c) => c.kind === "alter-sequence"); + expect(alter).toHaveLength(1); + expect(emit(changes, { dialect: "postgres" }).up).toContain('ALTER SEQUENCE "Ticket_Id_seq" START WITH 2000;'); + }); + + test("adding the identity to an existing column carries the declared options", () => { + const { up } = emit([{ + kind: "change-column-identity", table: "Ticket", column: "Id", to: "increment", + sequence: { name: "Old_Id_seq", start: 1000000000 }, status: "allowed", + } as never], { dialect: "postgres" }); + expect(up).toContain('ALTER TABLE "Ticket" ALTER COLUMN "Id" ADD GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME "Old_Id_seq" START WITH 1000000000);'); + }); + + test("a dropped table's down rebuilds it as before: the live sequence is never emitted", async () => { + const s = await schema({}); + const { changes } = await diff(EMPTY, live(s, { name: "OldTicket_Id_seq", start: 1000 }), { dialect: "postgres", allow: { dropTable: true } } as never); + expect(emit(changes, { dialect: "postgres" }).down).not.toContain("SEQUENCE NAME"); + }); + + test("a snapshot never records the live sequence", async () => { + const s = await schema({}); + const text = serializeSnapshot(live(s, { name: "OldTicket_Id_seq", start: 1000 })); + expect(text).not.toContain("OldTicket_Id_seq"); + expect(text).not.toContain("liveSequence"); + }); + + test("two tables declaring one sequence name are refused before any SQL is emitted", async () => { + const doc = { "metadata.root": { package: "acme", children: ["A", "B"].map((n) => ( + { "object.entity": { name: n, children: [ + { "source.rdb": { "@table": n } }, + { "field.int": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment", "@sequenceName": "shared_seq" } }, + ] } })) } }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(doc))]); + expect(errors).toHaveLength(0); + expect(() => buildExpectedSchema(root)).toThrow(DuplicateSqlNameError); + expect(() => buildExpectedSchema(root)).toThrow(/shared_seq/); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/unit/schema-conventions.test.ts b/server/typescript/packages/migrate-ts/test/unit/schema-conventions.test.ts new file mode 100644 index 0000000000..5980574104 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/unit/schema-conventions.test.ts @@ -0,0 +1,109 @@ +/** + * Schema conventions: the names and physical choices for database objects that no code + * addresses — primary-key and foreign-key constraint names, whether a foreign key gets an + * index (and its name), and whether a unique key is an index or a constraint. + * + * `default` is what migrate has always emitted, so a project that sets nothing sees no + * change. `efcore` reproduces EF Core's own conventions, so a database first built by EF + * migrations and one built by `meta migrate` have the same catalog. + */ +import { describe, test, expect } from "bun:test"; +import { + defaultConventions, efcoreConventions, resolveSchemaConventions, SCHEMA_CONVENTION_PRESETS, +} from "../../src/schema-conventions.js"; + +const fk = { table: "OrderLines", columns: ["OrderId"], refTable: "Orders", refColumns: ["Id"] }; + +describe("efcore conventions", () => { + test("PK_{table}, FK_{table}_{principal}_{columns}, IX_{table}_{columns}", () => { + expect(efcoreConventions.primaryKeyName({ table: "Orders", columns: ["Id"] })).toBe("PK_Orders"); + expect(efcoreConventions.foreignKeyName(fk)).toBe("FK_OrderLines_Orders_OrderId"); + expect(efcoreConventions.foreignKeyName({ ...fk, columns: ["OrderId", "Region"], refColumns: ["Id", "Region"] })) + .toBe("FK_OrderLines_Orders_OrderId_Region"); + expect(efcoreConventions.foreignKeyIndexes({ table: "OrderLines", primaryKey: ["Id"], indexes: [], foreignKeys: [fk] })) + .toEqual([{ name: "IX_OrderLines_OrderId", columns: ["OrderId"] }]); + }); + + test("on Postgres a name over 63 characters is cut to 62 and marked with ~, as EF Core does", () => { + const long = { table: "WarehouseShippingPreferences", columns: ["ShippingPreferenceId"], refTable: "ShippingPreferences", refColumns: ["Id"] }; + const name = efcoreConventions.foreignKeyName(long); + + expect(name).toBe("FK_WarehouseShippingPreferences_ShippingPreferences_ShippingPr~"); + expect(name.length).toBe(63); + expect(efcoreConventions.foreignKeyName({ ...long, dialect: "postgres" })).toBe(name); + expect(efcoreConventions.primaryKeyName({ table: "A".repeat(61), columns: ["Id"] })).toBe(`PK_${"A".repeat(59)}~`); + }); + + test("on SQLite and D1 nothing is truncated: EF Core's SQLite provider has no identifier limit", () => { + const long = { table: "WarehouseShippingPreferences", columns: ["ShippingPreferenceId"], refTable: "ShippingPreferences", refColumns: ["Id"] }; + for (const dialect of ["sqlite", "d1"] as const) { + expect(efcoreConventions.foreignKeyName({ ...long, dialect })) + .toBe("FK_WarehouseShippingPreferences_ShippingPreferences_ShippingPreferenceId"); + expect(efcoreConventions.primaryKeyName({ table: "A".repeat(61), columns: ["Id"], dialect })).toBe(`PK_${"A".repeat(61)}`); + expect(efcoreConventions.foreignKeyIndexes({ table: "A".repeat(61), dialect, primaryKey: ["Id"], indexes: [], + foreignKeys: [{ table: "A".repeat(61), columns: ["BId"], refTable: "B", refColumns: ["Id"], dialect }] })[0]!.name) + .toBe(`IX_${"A".repeat(61)}_BId`); + } + }); + + test("no FK index when the primary key or an existing index already starts with the FK's columns", () => { + const ix = (primaryKey: string[], indexes: { columns: string[]; where?: string; expr?: string }[]) => + efcoreConventions.foreignKeyIndexes({ table: "OrderLines", primaryKey, indexes, foreignKeys: [fk] }).map((i) => i.name); + + expect(ix(["OrderId", "LineNo"], [])).toEqual([]); + expect(ix(["Id"], [{ columns: ["OrderId", "Sku"] }])).toEqual([]); + // an index that merely CONTAINS the columns later does not cover them + expect(ix(["Id"], [{ columns: ["Sku", "OrderId"] }])).toEqual(["IX_OrderLines_OrderId"]); + // a PARTIAL index covers them: EF Core's AreIndexedBy compares properties, and a filter is + // a relational annotation that convention never sees + expect(ix(["Id"], [{ columns: ["OrderId"], where: "x" }])).toEqual([]); + // an expression index has no columns to lead with + expect(ix(["Id"], [{ columns: [], expr: "lower(x)" }])).toEqual(["IX_OrderLines_OrderId"]); + }); + + test("FK indexes do not depend on FK order: one an FK index covers is dropped, as EF Core does", () => { + const a = { table: "hs", columns: ["a"], refTable: "x", refColumns: ["id"] }; + const ab = { table: "hs", columns: ["a", "b"], refTable: "y", refColumns: ["id", "k"] }; + const names = (foreignKeys: typeof a[]) => + efcoreConventions.foreignKeyIndexes({ table: "hs", primaryKey: ["id"], indexes: [], foreignKeys }).map((i) => i.name); + + expect(names([a, ab])).toEqual(["IX_hs_a_b"]); + expect(names([ab, a])).toEqual(["IX_hs_a_b"]); + // two FKs on the same columns get one index + expect(names([a, { ...a, refTable: "z" }])).toEqual(["IX_hs_a"]); + }); + + test("a unique key an FK references is a constraint; any other is an index", () => { + expect(efcoreConventions.uniqueKeyKind({ table: "Orders", columns: ["Code"], referencedByForeignKey: true })).toBe("constraint"); + expect(efcoreConventions.uniqueKeyKind({ table: "Orders", columns: ["Code"], referencedByForeignKey: false })).toBe("index"); + }); +}); + +describe("default conventions (what migrate has always emitted)", () => { + test("{table}_pkey, {table}_{firstColumn}_fk, no FK index, a unique key of either kind", () => { + expect(defaultConventions.primaryKeyName({ table: "orders", columns: ["id"] })).toBe("orders_pkey"); + expect(defaultConventions.foreignKeyName({ ...fk, table: "order_lines", columns: ["order_id", "region"] })).toBe("order_lines_order_id_fk"); + expect(defaultConventions.foreignKeyIndexes({ table: "OrderLines", primaryKey: ["Id"], indexes: [], foreignKeys: [fk] })).toEqual([]); + expect(defaultConventions.uniqueKeyKind({ table: "orders", columns: ["code"], referencedByForeignKey: true })).toBe("either"); + }); +}); + +describe("resolveSchemaConventions", () => { + test("a preset by name, a supplied object as itself, nothing as default", () => { + expect(resolveSchemaConventions("efcore")).toBe(efcoreConventions); + expect(resolveSchemaConventions(undefined)).toBe(defaultConventions); + const custom = { ...defaultConventions, id: "acme" }; + expect(resolveSchemaConventions(custom)).toBe(custom); + }); + + test("an unknown preset is refused, naming the ones that exist", () => { + expect(() => resolveSchemaConventions("hibernate")).toThrow(/migrate\.conventions "hibernate".*default, efcore/); + expect(SCHEMA_CONVENTION_PRESETS).toEqual(["default", "efcore"]); + }); + + test("a supplied object missing a member is refused by name (e.g. written for another version)", () => { + const partial = { id: "acme", primaryKeyName: () => "pk" } as unknown as Parameters[0]; + expect(() => resolveSchemaConventions(partial)) + .toThrow(/schema conventions "acme" are missing foreignKeyName, foreignKeyIndexes, uniqueKeyKind/); + }); +}); diff --git a/server/typescript/packages/runtime-ts/src/drivers/in-memory-driver.ts b/server/typescript/packages/runtime-ts/src/drivers/in-memory-driver.ts index df04c02f1d..9c1d4ac35a 100644 --- a/server/typescript/packages/runtime-ts/src/drivers/in-memory-driver.ts +++ b/server/typescript/packages/runtime-ts/src/drivers/in-memory-driver.ts @@ -3,9 +3,12 @@ import type { UpdateSpec, UpdateManySpec, DeleteSpec, DeleteManySpec, WhereClause, Row, } from "../persistence-driver.js"; import { ConstraintViolationError } from "../errors.js"; +import { parseDurationLiteral } from "../duration-wire.js"; export interface InMemoryDriverOptions { - /** Initial table data: { tableName: Row[] }. */ + /** Initial table data: { tableName: Row[] }, in the stored form the driver writes (a + * `field.duration` is a bigint microsecond count; a wire-form string such as `"PT1H"` is + * compared with a stored value as a span, but two seeded strings compare as text). */ seed?: Record; /** Per-table PK field names (used to detect collisions + auto-increment). Default: ["id"]. */ pkFields?: Record; @@ -195,9 +198,9 @@ function matchesWhere(row: Row, where?: WhereClause): boolean { if (!where) return true; switch (where.kind) { case "eq": - return row[where.column] === where.value; + return sameValue(row[where.column], where.value); case "ne": - return row[where.column] !== where.value; + return !sameValue(row[where.column], where.value); case "gt": return compareValues(row[where.column], where.value) > 0; case "gte": @@ -207,7 +210,7 @@ function matchesWhere(row: Row, where?: WhereClause): boolean { case "lte": return compareValues(row[where.column], where.value) <= 0; case "in": - return where.values.some((v) => row[where.column] === v); + return where.values.some((v) => sameValue(row[where.column], v)); case "like": return likeMatch(row[where.column], where.pattern); case "isNull": { @@ -233,11 +236,32 @@ function compareForSort(a: unknown, b: unknown): number { return compareValues(a, b); } -function compareValues(a: unknown, b: unknown): number { +/** + * A duration is written as a bigint microsecond count, but a row seeded through + * `InMemoryDriverOptions.seed` may hold it in its wire form (`"PT1H"`). When one side is a bigint + * and the other a duration string, compare the string's microseconds. + */ +function asComparable(a: unknown, b: unknown): [unknown, unknown] { + if (typeof a === "bigint" && typeof b === "string") return [a, parseDurationLiteral(b) ?? b]; + if (typeof b === "bigint" && typeof a === "string") return [parseDurationLiteral(a) ?? a, b]; + return [a, b]; +} + +function sameValue(a: unknown, b: unknown): boolean { + const [x, y] = asComparable(a, b); + return x === y; +} + +function compareValues(left: unknown, right: unknown): number { + const [a, b] = asComparable(left, right); if (a === b) return 0; if (a === null || a === undefined) return -1; if (b === null || b === undefined) return 1; if (typeof a === "number" && typeof b === "number") return a - b; + // A duration is held as a bigint microsecond count; compare it as a number, never as text. + if ((typeof a === "bigint" || typeof a === "number") && (typeof b === "bigint" || typeof b === "number")) { + return a < b ? -1 : a > b ? 1 : 0; + } return String(a).localeCompare(String(b)); } diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-allowlist.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-allowlist.ts index 386b10e1e4..e349d9cf03 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-allowlist.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-allowlist.ts @@ -15,6 +15,7 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, type FilterOp, } from "@metaobjectsdev/metadata"; export { FILTER_OPS, OPS_BY_SUBTYPE, opsForSubType, type FilterOp }; @@ -33,6 +34,7 @@ export const FILTER_VALUE_FORMATS = [ FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, ] as const; export type FilterValueFormat = (typeof FILTER_VALUE_FORMATS)[number]; diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts index c4b3066f5d..644e9a9616 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts @@ -15,8 +15,11 @@ import { ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT, looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant, } from "./filter-value-format.js"; -import { FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_TIMESTAMP } from "@metaobjectsdev/metadata"; +import { FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_TIMESTAMP } from "@metaobjectsdev/metadata"; import { canonicalDecimal } from "../decimal-wire.js"; +import { + DURATION_MICROS_MAX, DURATION_MICROS_MIN, durationToStoredMicros, formatPostgresInterval, parseDurationLiteral, +} from "../duration-wire.js"; // biome-ignore lint/suspicious/noExplicitAny: dynamic dispatch over user's Drizzle table type AnyTable = any; @@ -344,6 +347,16 @@ function coerce(value: unknown, rule: FilterFieldRule, col: unknown, field: stri if (dialect === "mysql") return sql`CAST(${canonical} AS DECIMAL(65,30))`; return dialect === "sqlite" ? Number(canonical) : canonical; } + // A duration bound is spelled the way the column stores it: the microsecond integer on SQLite / + // D1 / MySQL, the interval spelling on Postgres (which refuses a signed ISO-8601 duration). Both + // compare and order as the span does. + if (rule.format === FIELD_SUBTYPE_DURATION) { + const micros = parseDurationLiteral(s) as bigint; + // A well-formed span past 64 bits of microseconds is no stored value on any database, so it is + // a bad operand (400), not an error out of the binder. + if (micros > DURATION_MICROS_MAX || micros < DURATION_MICROS_MIN) throw invalidValue(field, op, s, DURATION_IN_RANGE); + return dialect === "sqlite" || dialect === "mysql" ? durationToStoredMicros(micros) : formatPostgresInterval(micros); + } // A timestamp bound is compared in the same UTC spelling the generated schemas store: // on SQLite/D1 the column is TEXT and compares as text. An instant field (`instant`, i.e. // not @localTime) reads a zoneless or date-only bound as UTC, as its write schema does — @@ -364,6 +377,7 @@ function coerce(value: unknown, rule: FilterFieldRule, col: unknown, field: stri /** `expected` for a non-whole bound on an integer-valued field (`integer: true`). */ const INTEGER_EXPECTED = "integer (a whole number)"; +const DURATION_IN_RANGE = "duration within the 64-bit microsecond range"; function coerceAs(value: unknown, subType: string, field: string, op: string, dateValues?: boolean): unknown { if (value === null || value === undefined) return null; diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-value-format.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-value-format.ts index 5def0974f6..cf8a3f1097 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-value-format.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-value-format.ts @@ -17,9 +17,11 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, } from "@metaobjectsdev/metadata"; import type { FilterValueFormat } from "./filter-allowlist.js"; import { canonicalDecimal } from "../decimal-wire.js"; +import { parseDurationLiteral } from "../duration-wire.js"; const DATE_RE = /^(\d{4})-(\d{2})-(\d{2})$/; const TIME_RE = /^(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d{1,9})?)?$/; @@ -48,6 +50,7 @@ const CHECKS: Readonly boolean>> = { [FIELD_SUBTYPE_TIMESTAMP]: isTimestamp, [FIELD_SUBTYPE_UUID]: (s) => UUID_RE.test(s), [FIELD_SUBTYPE_DECIMAL]: (s) => canonicalDecimal(s) !== undefined, + [FIELD_SUBTYPE_DURATION]: (s) => parseDurationLiteral(s) !== undefined, }; /** The `expected` member of the `invalid_filter_value` envelope, per format. */ @@ -57,6 +60,7 @@ export const FORMAT_EXPECTED: Readonly> = { [FIELD_SUBTYPE_TIMESTAMP]: "timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", [FIELD_SUBTYPE_UUID]: "uuid (8-4-4-4-12 hex)", [FIELD_SUBTYPE_DECIMAL]: "decimal number (digits, optional fraction, optional exponent)", + [FIELD_SUBTYPE_DURATION]: "duration (ISO 8601, e.g. PT1H30M; no years or months)", }; /** `expected` for a temporal rule that does not say WHICH temporal format it is. */ diff --git a/server/typescript/packages/runtime-ts/src/duration-wire.ts b/server/typescript/packages/runtime-ts/src/duration-wire.ts new file mode 100644 index 0000000000..732659f9b0 --- /dev/null +++ b/server/typescript/packages/runtime-ts/src/duration-wire.ts @@ -0,0 +1,7 @@ +// The field.duration wire rule lives in @metaobjectsdev/metadata (shared with migrate-ts); +// re-exported here so the runtime's callers import it beside the decimal wire helpers. +export { + DURATION_PATTERN, DURATION_RE, parseDurationLiteral, formatDuration, formatPostgresInterval, + canonicalDuration, parseDurationInput, parsePostgresInterval, durationToWire, durationToStoredMicros, + DURATION_MICROS_MIN, DURATION_MICROS_MAX, +} from "@metaobjectsdev/metadata"; diff --git a/server/typescript/packages/runtime-ts/src/n2m-resolver.ts b/server/typescript/packages/runtime-ts/src/n2m-resolver.ts index 25d1cf4124..eb957f7491 100644 --- a/server/typescript/packages/runtime-ts/src/n2m-resolver.ts +++ b/server/typescript/packages/runtime-ts/src/n2m-resolver.ts @@ -31,7 +31,7 @@ import { import type { MetaRelationship } from "@metaobjectsdev/metadata"; import { MetadataError } from "./errors.js"; import { buildSelectSpec, resolvePkFields } from "./query-builder.js"; -import type { SelectSpec, WhereClause, PrimitiveValue, Row } from "./persistence-driver.js"; +import type { Dialect, SelectSpec, WhereClause, PrimitiveValue, Row } from "./persistence-driver.js"; export interface N2mDescriptor { /** Entity that declares the relationship (source of the lookup; its PK feeds sourceField). */ @@ -108,8 +108,9 @@ export function buildN2mLazySpecs( sourceRecord: Row, root: MetaData, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): N2mLazyOutput { - return buildSpecs(desc, sourceRecord, root, strategy); + return buildSpecs(desc, sourceRecord, root, strategy, dialect); } export function buildN2mBatchSpecs( @@ -117,8 +118,9 @@ export function buildN2mBatchSpecs( sourceRecords: Row[], root: MetaData, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): N2mBatchOutput { - return buildSpecs(desc, sourceRecords, root, strategy); + return buildSpecs(desc, sourceRecords, root, strategy, dialect); } // Single + batch share one code path: a single record is the one-element case. @@ -127,6 +129,7 @@ function buildSpecs( source: Row | Row[], root: MetaData, strategy: ColumnNamingStrategy, + dialect: Dialect | undefined, ): N2mLazyOutput { const joinEntity = mustGetEntity(root, desc.joinEntityName); const targetEntity = mustGetEntity(root, desc.targetEntityName); @@ -143,7 +146,7 @@ function buildSpecs( // buildSelectSpec compiles a filter on the entity's fields; for symmetric we // need an OR across two columns, which the filter DSL can't express. So we // build the join spec directly off buildSelectSpec then swap in the where. - const joinSpec = buildSelectSpec(joinEntity, undefined, {}, undefined, strategy); + const joinSpec = buildSelectSpec(joinEntity, undefined, {}, undefined, strategy, dialect); joinSpec.where = desc.symmetric ? { kind: "or", clauses: [inOrEq(sourceCol, sourceIds), inOrEq(targetCol, sourceIds)] } : inOrEq(sourceCol, sourceIds); @@ -156,7 +159,7 @@ function buildSpecs( const targetPkField = resolvePkFields(targetEntity)[0]!; // PK values are always string|number; the IN filter type excludes boolean. const ids = targetIds.filter((v): v is string | number => typeof v !== "boolean"); - return buildSelectSpec(targetEntity, { [targetPkField]: ids }, {}, undefined, strategy); + return buildSelectSpec(targetEntity, { [targetPkField]: ids }, {}, undefined, strategy, dialect); }; return { joinSpec, makeTargetSpec }; diff --git a/server/typescript/packages/runtime-ts/src/object-manager.ts b/server/typescript/packages/runtime-ts/src/object-manager.ts index 8437a1068e..0e6b670702 100644 --- a/server/typescript/packages/runtime-ts/src/object-manager.ts +++ b/server/typescript/packages/runtime-ts/src/object-manager.ts @@ -8,7 +8,7 @@ import { FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DECIMAL, } from "@metaobjectsdev/metadata"; import type { - PersistenceDriver, Row, WhereClause, + Dialect, PersistenceDriver, Row, WhereClause, InsertManySpec, UpdateManySpec, DeleteManySpec, } from "./persistence-driver.js"; import { @@ -125,7 +125,7 @@ export class ObjectManager { const entity = this.requireEntity(entityName); const driver = opts.tx ?? this.driver; const scoped = this.scopeFilter(filter, tphSubtypeOf(entity)); - const spec = buildSelectSpec(entity, scoped, { ...opts, limit: 1 }, undefined, this.columnNamingStrategy); + const spec = buildSelectSpec(entity, scoped, { ...opts, limit: 1 }, undefined, this.columnNamingStrategy, driver.dialect); const row = await driver.selectOne(spec); if (row === null) return null; const jsRow = this.toJsRow(entity, row); @@ -139,7 +139,7 @@ export class ObjectManager { const entity = this.requireEntity(entityName); const driver = opts.tx ?? this.driver; const scoped = this.scopeFilter(filter, tphSubtypeOf(entity)); - const spec = buildSelectSpec(entity, scoped, opts, undefined, this.columnNamingStrategy); + const spec = buildSelectSpec(entity, scoped, opts, undefined, this.columnNamingStrategy, driver.dialect); const rows = (await driver.selectMany(spec)).map((r) => this.toJsRow(entity, r)); if (opts.include && opts.include.length > 0) { await this.attachIncludes(entity, rows, opts.include, driver); @@ -151,7 +151,7 @@ export class ObjectManager { const entity = this.requireEntity(entityName); const driver = opts.tx ?? this.driver; const scoped = this.scopeFilter(filter, tphSubtypeOf(entity)); - return driver.count(buildCountSpec(entity, scoped, this.columnNamingStrategy)); + return driver.count(buildCountSpec(entity, scoped, this.columnNamingStrategy, driver.dialect)); } async load(refString: string): Promise { @@ -291,7 +291,7 @@ export class ObjectManager { const spec: UpdateManySpec = { table: resolveTableName(entity), values: this.toDbRow(entity, coerced), - where: requireNonEmptyFilter(entity, filter, "updateMany", this.columnNamingStrategy), + where: requireNonEmptyFilter(entity, filter, "updateMany", this.columnNamingStrategy, driver.dialect), }; return driver.updateMany(spec); } @@ -301,7 +301,7 @@ export class ObjectManager { const driver = opts.tx ?? this.driver; const spec: DeleteManySpec = { table: resolveTableName(entity), - where: requireNonEmptyFilter(entity, filter, "deleteMany", this.columnNamingStrategy), + where: requireNonEmptyFilter(entity, filter, "deleteMany", this.columnNamingStrategy, driver.dialect), }; return driver.deleteMany(spec); } @@ -323,7 +323,7 @@ export class ObjectManager { const n2m = resolveN2mDescriptor(entity, relationName, this.metadata); if (n2m !== null) { const target = this.requireEntity(n2m.targetEntityName); - const { joinSpec, makeTargetSpec } = buildN2mLazySpecs(n2m, record, this.metadata, this.columnNamingStrategy); + const { joinSpec, makeTargetSpec } = buildN2mLazySpecs(n2m, record, this.metadata, this.columnNamingStrategy, driver.dialect); const joinRows = await driver.selectMany(joinSpec); const targetSpec = makeTargetSpec(joinRows); if (targetSpec === null) return []; @@ -333,7 +333,7 @@ export class ObjectManager { const desc = resolveRelationDescriptor(entity, relationName, this.metadata); const target = this.requireEntity(desc.targetEntityName); - const spec = buildLazyRelateSpec(desc, record, this.metadata, this.columnNamingStrategy); + const spec = buildLazyRelateSpec(desc, record, this.metadata, this.columnNamingStrategy, driver.dialect); if (spec === null) return desc.cardinality === "one" ? null : []; if (desc.cardinality === "one") { const row = await driver.selectOne(spec); @@ -376,7 +376,7 @@ export class ObjectManager { const n2m = resolveN2mDescriptor(entity, inc, this.metadata); if (n2m !== null) { const target = this.requireEntity(n2m.targetEntityName); - const { joinSpec, makeTargetSpec } = buildN2mBatchSpecs(n2m, records, this.metadata, this.columnNamingStrategy); + const { joinSpec, makeTargetSpec } = buildN2mBatchSpecs(n2m, records, this.metadata, this.columnNamingStrategy, driver.dialect); const joinRows = await driver.selectMany(joinSpec); const targetSpec = makeTargetSpec(joinRows); const targetRows = targetSpec === null ? [] : (await driver.selectMany(targetSpec)).map((r) => this.toJsRow(target, r)); @@ -418,7 +418,7 @@ export class ObjectManager { const desc = resolveRelationDescriptor(entity, inc, this.metadata); const target = this.requireEntity(desc.targetEntityName); - const spec = buildIncludeBatchSpec(desc, records, this.metadata, this.columnNamingStrategy); + const spec = buildIncludeBatchSpec(desc, records, this.metadata, this.columnNamingStrategy, driver.dialect); const targetRows = spec === null ? [] : (await driver.selectMany(spec)).map((r) => this.toJsRow(target, r)); if (desc.cardinality === "one") { @@ -553,9 +553,9 @@ export class ObjectManager { // updateMany / deleteMany with an empty filter would silently affect every row. // Force callers to be explicit (use $or-style or a tautology if they really mean "all"). function requireNonEmptyFilter( - entity: MetaData, filter: Filter, op: string, strategy: ColumnNamingStrategy, + entity: MetaData, filter: Filter, op: string, strategy: ColumnNamingStrategy, dialect?: Dialect, ): WhereClause { - const where = compileFilter(entity, filter, strategy); + const where = compileFilter(entity, filter, strategy, dialect); if (where === null) { throw new MetadataError( `${op} on '${entity.name}' requires a non-empty filter — pass an explicit condition or use a per-row loop`, diff --git a/server/typescript/packages/runtime-ts/src/query-builder.ts b/server/typescript/packages/runtime-ts/src/query-builder.ts index ef8ec00798..4a14751af0 100644 --- a/server/typescript/packages/runtime-ts/src/query-builder.ts +++ b/server/typescript/packages/runtime-ts/src/query-builder.ts @@ -5,12 +5,14 @@ import { IDENTITY_ATTR_FIELDS, OBJECT_SUBTYPE_REPORT, DEFAULT_COLUMN_NAMING_STRATEGY, + FIELD_SUBTYPE_DURATION, resolveTableName, resolveColumnName, + durationToStoredMicros, formatPostgresInterval, parseDurationLiteral, } from "@metaobjectsdev/metadata"; import { MetadataError } from "./errors.js"; import { intValueMapOf } from "./type-coercer.js"; import type { - WhereClause, OrderBy, PrimitiveValue, Row, + Dialect, WhereClause, OrderBy, PrimitiveValue, Row, SelectSpec, InsertSpec, UpdateSpec, DeleteSpec, CountSpec, } from "./persistence-driver.js"; @@ -92,13 +94,14 @@ export function compileFilter( entity: MetaData, filter: Filter, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): WhereClause | null { if ("$and" in filter && Array.isArray(filter.$and)) { // TS can't narrow $and away from the Record branch, so the runtime check above // proves it's a Filter[] but the element type still needs a bridge cast. const andFilters = filter.$and as Filter[]; const clauses = andFilters - .map((f) => compileFilter(entity, f, strategy)) + .map((f) => compileFilter(entity, f, strategy, dialect)) .filter((c): c is WhereClause => c !== null); if (clauses.length === 0) return null; return clauses.length === 1 ? clauses[0]! : { kind: "and", clauses }; @@ -106,16 +109,16 @@ export function compileFilter( const entries = Object.entries(filter); if (entries.length === 0) return null; if (entries.length === 1) { - return compileEntry(entity, entries[0]![0], entries[0]![1] as FilterValue, strategy); + return compileEntry(entity, entries[0]![0], entries[0]![1] as FilterValue, strategy, dialect); } return { kind: "and", - clauses: entries.map(([k, v]) => compileEntry(entity, k, v as FilterValue, strategy)), + clauses: entries.map(([k, v]) => compileEntry(entity, k, v as FilterValue, strategy, dialect)), }; } function compileEntry( - entity: MetaData, fieldName: string, value: FilterValue, strategy: ColumnNamingStrategy, + entity: MetaData, fieldName: string, value: FilterValue, strategy: ColumnNamingStrategy, dialect?: Dialect, ): WhereClause { const field = getField(entity, fieldName); const column = resolveColumnName(field, strategy); @@ -126,7 +129,7 @@ function compileEntry( // for type integer"). Applied at this ONE seam because every operator arrives here // with the field in hand; `like` is not reachable for such a field (the loader's // field-level operator band drops it), and `isNull` carries no value to encode. - const enc = (v: T): T => encodeFilterValue(field, v); + const enc = (v: T): T => encodeFilterValue(field, v, dialect); if (value === null) return { kind: "isNull", column, not: false }; if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") { @@ -167,7 +170,22 @@ function compileEntry( * is the honest answer for a member that does not exist, and the loader already * pins `@intValueMap`'s key set to `@values`. */ -function encodeFilterValue(field: MetaData, value: T): T { +function encodeFilterValue(field: MetaData, value: T, dialect?: Dialect): T { + // A duration is compared in the form its column stores: the microsecond integer on SQLite / D1 / + // MySQL, the interval spelling on Postgres (which refuses a signed ISO-8601 duration). A value + // that is not a duration is refused, not bound. + if (field.subType === FIELD_SUBTYPE_DURATION && typeof value === "string") { + const micros = parseDurationLiteral(value); + if (micros === undefined) { + throw new MetadataError(`filter on field.duration '${field.name}': ${JSON.stringify(value)} is not an ISO-8601 duration`); + } + if (dialect === "postgres") return formatPostgresInterval(micros) as T; + if (dialect === "sqlite" || dialect === "mysql") return durationToStoredMicros(micros) as T; + if (dialect === "memory") return micros as T; + // With no dialect there is no stored form to compare against: an ISO string would match + // nothing on a microsecond column and fail on an interval one. + throw new MetadataError(`filter on field.duration '${field.name}' needs the driver's dialect to encode ${JSON.stringify(value)}`); + } const intMap = intValueMapOf(field); if (intMap === undefined || typeof value !== "string") return value; const stored = intMap[value]; @@ -192,6 +210,7 @@ export function buildSelectSpec( opts: QueryOpts, projectedFields?: string[], strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): SelectSpec { const allFields = projectedFields ?? listFieldNames(entity); // A report's read model (FR-044) has no identity, so there is no key to add to the @@ -217,7 +236,7 @@ export function buildSelectSpec( table: resolveTableName(entity), columns, }; - const where = filter !== undefined ? compileFilter(entity, filter, strategy) : null; + const where = filter !== undefined ? compileFilter(entity, filter, strategy, dialect) : null; if (where !== null) spec.where = where; if (orderBy !== undefined) spec.orderBy = orderBy; if (opts.limit !== undefined) spec.limit = opts.limit; @@ -228,9 +247,10 @@ export function buildSelectSpec( export function buildCountSpec( entity: MetaData, filter: Filter | undefined, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): CountSpec { const spec: CountSpec = { table: resolveTableName(entity) }; - const where = filter !== undefined ? compileFilter(entity, filter, strategy) : null; + const where = filter !== undefined ? compileFilter(entity, filter, strategy, dialect) : null; if (where !== null) spec.where = where; return spec; } diff --git a/server/typescript/packages/runtime-ts/src/relation-resolver.ts b/server/typescript/packages/runtime-ts/src/relation-resolver.ts index 2953bb8836..7284d7d79d 100644 --- a/server/typescript/packages/runtime-ts/src/relation-resolver.ts +++ b/server/typescript/packages/runtime-ts/src/relation-resolver.ts @@ -12,7 +12,7 @@ import { MetadataError } from "./errors.js"; import { buildSelectSpec, resolvePkFields, } from "./query-builder.js"; -import type { SelectSpec, PrimitiveValue, Row } from "./persistence-driver.js"; +import type { Dialect, SelectSpec, PrimitiveValue, Row } from "./persistence-driver.js"; export interface RelationDescriptor { /** "one" = source holds the FK pointing at target. "many" = target holds the FK pointing back at source. */ @@ -148,11 +148,12 @@ export function buildLazyRelateSpec( sourceRecord: Row, root: MetaData, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): SelectSpec | null { const lookup = sourceRecord[desc.sourceField]; if (lookup === null || lookup === undefined) return null; const target = mustGetEntity(root, desc.targetEntityName); - return buildSelectSpec(target, { [desc.targetField]: lookup as PrimitiveValue }, {}, undefined, strategy); + return buildSelectSpec(target, { [desc.targetField]: lookup as PrimitiveValue }, {}, undefined, strategy, dialect); } /** Builds one batched IN(...) lookup. Returns null when there are no non-null source values. */ @@ -161,6 +162,7 @@ export function buildIncludeBatchSpec( sourceRecords: Row[], root: MetaData, strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, + dialect?: Dialect, ): SelectSpec | null { const seen = new Set(); for (const rec of sourceRecords) { @@ -170,7 +172,7 @@ export function buildIncludeBatchSpec( } if (seen.size === 0) return null; const target = mustGetEntity(root, desc.targetEntityName); - return buildSelectSpec(target, { [desc.targetField]: [...seen] as (string | number)[] }, {}, undefined, strategy); + return buildSelectSpec(target, { [desc.targetField]: [...seen] as (string | number)[] }, {}, undefined, strategy, dialect); } function mustGetEntity(root: MetaData, name: string): MetaData { diff --git a/server/typescript/packages/runtime-ts/src/type-coercer.ts b/server/typescript/packages/runtime-ts/src/type-coercer.ts index 12fcc17811..c8f4735dd2 100644 --- a/server/typescript/packages/runtime-ts/src/type-coercer.ts +++ b/server/typescript/packages/runtime-ts/src/type-coercer.ts @@ -27,6 +27,7 @@ import { FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_TIMESTAMP, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_STORAGE, FIELD_ATTR_OBJECT_REF, OBJECT_SUBTYPE_VALUE, @@ -40,10 +41,13 @@ import { } from "@metaobjectsdev/metadata"; import type { Dialect, Row } from "./persistence-driver.js"; import { decimalFits, decimalToWire, parseDecimalInput } from "./decimal-wire.js"; +import { + durationToStoredMicros, durationToWire, formatDuration, formatPostgresInterval, parseDurationLiteral, +} from "./duration-wire.js"; export function coerceRowOnRead(entity: MetaData, row: Row, dialect: Dialect): Row { const hydrated = parseJsonTextArrays(entity, deserializeJsonbObjectFields(entity, row)); - const decoded = canonicalizeDecimals(entity, decodeIntBackedEnums(entity, hydrated)); + const decoded = canonicalizeDurations(entity, canonicalizeDecimals(entity, decodeIntBackedEnums(entity, hydrated))); if (dialect !== "sqlite" && dialect !== "mysql") return decoded; return mapBooleansFromInt(entity, decoded); } @@ -141,7 +145,7 @@ function encodeIntBackedEnums(entity: MetaData, row: Row): Row { } export function coerceRowOnWrite(entity: MetaData, row: Row, dialect: Dialect): Row { - const jsonbColumned = serializeJsonbColumns(entity, bindDecimals(entity, row)); + const jsonbColumned = serializeJsonbColumns(entity, bindDurations(entity, bindDecimals(entity, row), dialect)); // Dialect-independent: an int-backed enum's column is INTEGER on every dialect // (SQLite has one integer storage class), so the symbol→int encode is not gated // on the dialect the way the boolean mapping below is. @@ -390,3 +394,84 @@ function bindDecimals(entity: MetaData, row: Row): Row { return bind(v); }); } + +/** + * Apply `convert` to every `field.duration` value of `owner` present in `row`, an `isArray` one + * element by element, and recurse into a `field.object` / `field.map` whose value object declares + * durations (as `mapDecimals` does). `top` says whether `owner` is the stored row itself: a + * duration inside a jsonb value object is JSON, so it is always the canonical ISO string, while a + * top-level duration column takes the dialect's spelling. A row is copied only when something + * changed. + */ +function mapDurations( + owner: MetaData, row: Row, convert: (field: MetaData, value: unknown, top: boolean) => unknown, top = true, +): Row { + let out: Row | null = null; + // ADR-0039: resolving — a duration may be inherited via extends. + for (const child of owner.children()) { + if (child.type !== TYPE_FIELD) continue; + if (!(child.name in row)) continue; + const v = row[child.name]; + if (v === null || v === undefined) continue; + let next: unknown = v; + if (child.subType === FIELD_SUBTYPE_DURATION) { + next = Array.isArray(v) + ? v.map((el) => (el === null || el === undefined ? el : convert(child, el, top))) + : convert(child, v, top); + } else { + const vo = valueObjectOf(child); + if (vo === undefined) continue; + const one = (el: unknown): unknown => + el !== null && typeof el === "object" && !Array.isArray(el) ? mapDurations(vo, el as Row, convert, false) : el; + if (child.subType === FIELD_SUBTYPE_MAP) { + if (typeof v === "object" && !Array.isArray(v)) { + let mapped: Row | null = null; + for (const [key, el] of Object.entries(v as Row)) { + const m = one(el); + if (m !== el) (mapped ??= { ...(v as Row) })[key] = m; + } + if (mapped !== null) next = mapped; + } + } else if (Array.isArray(v)) { + const mapped = v.map(one); + if (mapped.some((el, i) => el !== v[i])) next = mapped; + } else { + next = one(v); + } + } + if (next === v) continue; + out ??= { ...row }; + out[child.name] = next; + } + return out ?? row; +} + +/** Read side: a duration column's driver value (an interval's text or object, or the stored ISO + * string), and a duration inside a jsonb value object, become the canonical wire string. An + * interval with a year or month part throws. */ +function canonicalizeDurations(entity: MetaData, row: Row): Row { + return mapDurations(entity, row, (_field, v) => durationToWire(v)); +} + +/** Write side: a duration column is bound as the 64-bit microsecond integer SQLite, D1 and MySQL + * store it as, in the Postgres interval spelling on Postgres, which refuses a signed ISO-8601 + * duration, and as a bigint microsecond count in the in-memory driver. A duration inside a jsonb value object, + * or an element of an `isArray` duration on SQLite / D1 / MySQL (a JSON array column), is JSON, so it is the + * canonical ISO string. A value that is not a duration is refused here, not by the database. */ +function bindDurations(entity: MetaData, row: Row, dialect: Dialect): Row { + return mapDurations(entity, row, (child, v, top) => { + const micros = typeof v === "string" ? parseDurationLiteral(v) : undefined; + if (micros === undefined) { + throw new Error(`field.duration '${child.name}' cannot be written: ${JSON.stringify(v)} is not an ISO-8601 duration (such as PT1H30M; no years or months)`); + } + if (!top) return formatDuration(micros); + if (dialect === "postgres") return formatPostgresInterval(micros); + // ADR-0039: resolving. An array has no 64-bit column on SQLite or MySQL: it is a JSON array + // in one text/json column, holding the canonical ISO strings, as the generated schema writes it. + if ((dialect === "sqlite" || dialect === "mysql") && child.resolvedIsArray()) return formatDuration(micros); + if (dialect === "sqlite" || dialect === "mysql") return durationToStoredMicros(micros); + // The in-memory driver holds the exact microsecond count, so it compares and sorts as the + // span does; it has no column, so no 64-bit limit. + return micros; + }); +} diff --git a/server/typescript/packages/runtime-ts/src/validator-runner.ts b/server/typescript/packages/runtime-ts/src/validator-runner.ts index 84d149fa15..0d4fcf79a0 100644 --- a/server/typescript/packages/runtime-ts/src/validator-runner.ts +++ b/server/typescript/packages/runtime-ts/src/validator-runner.ts @@ -14,12 +14,13 @@ import { OBJECT_SUBTYPE_VALUE, VALIDATOR_ATTR_MIN, VALIDATOR_ATTR_MAX, VALIDATOR_ATTR_PATTERN, VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_ARRAY, - FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_ATTR_LENIENT, + FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_ATTR_LENIENT, FIELD_SUBTYPE_DURATION, IDENTITY_ATTR_FIELDS, IDENTITY_ATTR_GENERATION, GENERATION_INCREMENT, GENERATION_UUID, } from "@metaobjectsdev/metadata"; import type { ValidationFailure } from "./errors.js"; import { decimalFits, parseDecimalInput } from "./decimal-wire.js"; import { isAbsoluteUri, isInetLiteral } from "./net-format.js"; +import { parseDurationLiteral } from "./duration-wire.js"; export type ValidationResult = | { ok: true } @@ -319,7 +320,8 @@ function comparable(value: unknown): number | bigint | undefined { function checkType(subType: string, value: unknown): string | null { if (subType === FIELD_SUBTYPE_STRING || subType === FIELD_SUBTYPE_UUID - || subType === FIELD_SUBTYPE_URI || subType === FIELD_SUBTYPE_INET) { + || subType === FIELD_SUBTYPE_URI || subType === FIELD_SUBTYPE_INET + || subType === FIELD_SUBTYPE_DURATION) { if (typeof value !== "string") return `expected string`; } else if (NUMERIC_FIELD_SUBTYPES.has(subType)) { if (typeof value !== "number") return `expected number`; @@ -479,5 +481,15 @@ function scalarErrors(field: MetaData, value: unknown, required: boolean, label: }); } } + + // field.duration — an ISO-8601 duration with no years or months (no @lenient: the + // column is an interval on Postgres, so a value that is not a duration cannot be stored). + if (typeof value === "string" && field.subType === FIELD_SUBTYPE_DURATION && parseDurationLiteral(value) === undefined) { + errors.push({ + field: label, rule: "format", + message: `'${label}' must be an ISO-8601 duration such as PT1H30M (no years or months)`, + expected: "duration", received: value, + }); + } return errors; } diff --git a/server/typescript/packages/runtime-ts/test/drizzle-fastify/filter-duration-range.test.ts b/server/typescript/packages/runtime-ts/test/drizzle-fastify/filter-duration-range.test.ts new file mode 100644 index 0000000000..e7837ea9d1 --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/drizzle-fastify/filter-duration-range.test.ts @@ -0,0 +1,50 @@ +// A duration operand that parses but is past 64 bits of microseconds is a bad filter value: a +// 400 `invalid_filter_value`, never an error thrown out of the binder (a 500). + +import { describe, test, expect, beforeAll } from "bun:test"; +import Fastify, { type FastifyInstance } from "fastify"; +import { createClient } from "@libsql/client"; +import { drizzle } from "drizzle-orm/libsql"; +import { sqliteTable, integer, text } from "drizzle-orm/sqlite-core"; +import { z } from "zod"; +import { mountCrudRoutes } from "../../src/drizzle-fastify/index.js"; +import type { FilterAllowlist } from "../../src/drizzle-fastify/filter-allowlist.js"; + +const timers = sqliteTable("timers", { + id: integer("id").primaryKey({ autoIncrement: true }), + label: text("label").notNull(), + timeout: integer("timeout").notNull(), +}); +const InsertSchema = z.object({ label: z.string(), timeout: z.number().int() }); +const DURATION_OPS = ["eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull"] as const; +const filterAllowlist: FilterAllowlist = { + timeout: { ops: DURATION_OPS, subType: "string", leadingWildcard: false, format: "duration" }, +}; + +let app: FastifyInstance; +beforeAll(async () => { + const client = createClient({ url: ":memory:" }); + await client.execute("CREATE TABLE timers (id INTEGER PRIMARY KEY AUTOINCREMENT, label TEXT NOT NULL, timeout INTEGER NOT NULL)"); + await client.execute("INSERT INTO timers (label, timeout) VALUES ('hour', 3600000000)"); + app = Fastify(); + mountCrudRoutes({ + fastify: app, path: "/timers", db: drizzle(client), table: timers, + insertSchema: InsertSchema, updateSchema: InsertSchema.partial(), + filterAllowlist, sortAllowlist: {}, dialect: "sqlite", + }); + await app.ready(); +}); + +describe("a duration filter operand past the 64-bit microsecond range", () => { + test("in range: compared as the span", async () => { + const r = await app.inject({ method: "GET", url: "/timers?filter[timeout][gt]=PT30M" }); + expect(r.statusCode).toBe(200); + expect((JSON.parse(r.body) as Array<{ label: string }>).map((x) => x.label)).toEqual(["hour"]); + }); + + test("out of range: 400 invalid_filter_value, not a 500", async () => { + const r = await app.inject({ method: "GET", url: "/timers?filter[timeout][gt]=PT9999999999999H" }); + expect(r.statusCode).toBe(400); + expect(r.body).toContain("invalid_filter_value"); + }); +}); diff --git a/server/typescript/packages/runtime-ts/test/duration-wire-vectors.test.ts b/server/typescript/packages/runtime-ts/test/duration-wire-vectors.test.ts new file mode 100644 index 0000000000..fcf6a47a88 --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/duration-wire-vectors.test.ts @@ -0,0 +1,111 @@ +// The field.duration wire rule: fixtures/duration-wire/vectors.json against the runtime's +// PRODUCTION helpers, the ones the type coercer, the validator runner and the filter parser call. +// TypeScript holds microseconds (precision 6) and has every codec: it reads and binds Postgres +// interval text and stores microseconds on SQLite, D1 and MySQL. No database. + +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { + canonicalDuration, durationToWire, formatDuration, formatPostgresInterval, parseDurationInput, + parseDurationLiteral, parsePostgresInterval, durationToStoredMicros, DURATION_RE, +} from "../src/duration-wire.js"; + +interface Vectors { + wire: { + parse: Array<{ in: string; out: string }>; + precision: Array<{ in: string; out: Record }>; + refused: string[]; + }; + codecs: { + postgres: Array<{ in: string; out: string }>; + postgresRefused: string[]; + postgresText: Array<{ in: string; out: string }>; + storedMicros: Array<{ in: string; out: string }>; + }; +} +const { wire, codecs } = JSON.parse( + readFileSync(join(import.meta.dir, "..", "..", "..", "..", "..", "fixtures", "duration-wire", "vectors.json"), "utf8"), +) as Vectors; +/** TypeScript holds a duration as microseconds. */ +const PRECISION = "6"; + +describe("duration wire vectors (production helpers)", () => { + for (const v of [...wire.parse, ...wire.precision.map((p) => ({ in: p.in, out: p.out[PRECISION]! }))]) { + test(`parse ${v.in} -> ${v.out}`, () => { + expect(DURATION_RE.test(v.in)).toBe(true); + expect(canonicalDuration(v.in)).toBe(v.out); + expect(parseDurationInput(v.in)).toBe(v.out); + expect(durationToWire(v.in)).toBe(v.out); + // the canonical form is itself accepted, and is a fixed point + expect(canonicalDuration(v.out)).toBe(v.out); + }); + } + for (const text of wire.refused) { + test(`refused ${JSON.stringify(text)}`, () => { + expect(DURATION_RE.test(text)).toBe(false); + expect(parseDurationLiteral(text)).toBeUndefined(); + expect(canonicalDuration(text)).toBeUndefined(); + expect(parseDurationInput(text)).toBeUndefined(); + }); + } + for (const v of codecs.postgres) { + test(`postgres ${JSON.stringify(v.in)} -> ${v.out}`, () => { + expect(formatDuration(parsePostgresInterval(v.in))).toBe(v.out); + expect(durationToWire(v.in)).toBe(v.out); + }); + } + for (const text of codecs.postgresRefused) { + test(`postgres refused ${JSON.stringify(text)}`, () => { + expect(() => durationToWire(text)).toThrow(); + }); + } + + for (const v of codecs.postgresText) { + test(`postgres text of ${v.in} -> ${JSON.stringify(v.out)}`, () => { + expect(formatPostgresInterval(parseDurationLiteral(v.in)!)).toBe(v.out); + // and reading that text back is the canonical form again + expect(durationToWire(v.out)).toBe(canonicalDuration(v.in) as string); + }); + } + + for (const v of codecs.storedMicros) { + test(`stored micros of ${v.in} -> ${v.out}`, () => { + expect(String(durationToStoredMicros(parseDurationLiteral(v.in)!))).toBe(v.out); + // a driver may return the integer as a number, a bigint or a digit string + const canonical = canonicalDuration(v.in)!; + expect(durationToWire(BigInt(v.out))).toBe(canonical); + expect(durationToWire(v.out)).toBe(canonical); + if (Number.isSafeInteger(Number(v.out))) expect(durationToWire(Number(v.out))).toBe(canonical); + }); + } + + test("past the 64-bit microsecond column, binding to it is refused (and only there)", () => { + const huge = parseDurationLiteral("PT2562047789H")!; + expect(() => durationToStoredMicros(huge)).toThrow(/64-bit microsecond column/); + expect(formatDuration(huge)).toBe("PT2562047789H"); + }); + + test("a Postgres session not on the default IntervalStyle is refused, never misread", () => { + // iso_8601 (a negative span's per-component signs), sql_standard and postgres_verbose + for (const text of ["PT-1M-30S", "1 2:00:00", "@ 1 day 2 hours", "1-2"]) { + expect(() => durationToWire(text)).toThrow(); + } + // an unsigned iso_8601 value is plain ISO-8601, so it reads as the span it is + expect(durationToWire("P1DT2H")).toBe("PT26H"); + }); + + test("a node-postgres interval object reads as its canonical form", () => { + expect(durationToWire({ hours: 1, minutes: 30 })).toBe("PT1H30M"); + expect(durationToWire({ days: 1, hours: 2, seconds: 1, milliseconds: 500.001 })).toBe("PT26H1.500001S"); + expect(durationToWire({ hours: -1, minutes: -30 })).toBe("-PT1H30M"); + expect(() => durationToWire({ months: 1 })).toThrow(/year or month/); + }); + + test("a number is not a duration a caller can send; read back from an integer column it is microseconds", () => { + expect(parseDurationInput(90)).toBeUndefined(); + expect(durationToWire(90)).toBe("PT0.00009S"); + expect(() => durationToWire(1.5)).toThrow(); + expect(() => durationToWire(true)).toThrow(); + }); +}); diff --git a/server/typescript/packages/runtime-ts/test/object-manager-duration.test.ts b/server/typescript/packages/runtime-ts/test/object-manager-duration.test.ts new file mode 100644 index 0000000000..8693fdb4c1 --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/object-manager-duration.test.ts @@ -0,0 +1,134 @@ +// field.duration through the ObjectManager: a value is read in the canonical ISO-8601 form +// (fixtures/duration-wire); on SQLite the column stores the microsecond integer, so sorting, range +// filters and equality follow the span; a duration inside a jsonb value object is the canonical +// string; and a filter operand is in the form the column stores. Postgres (which refuses a signed ISO-8601 duration as interval input) is +// covered live by the persistence-conformance roundtrip-all-types scenario. +import { test, expect } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { Kysely, sql } from "kysely"; +import { LibsqlDialect } from "@libsql/kysely-libsql"; +import { MetaDataLoader, InMemoryStringSource, type MetaRoot } from "@metaobjectsdev/metadata"; +import { ObjectManager, type Row } from "../src/index.js"; +import { kyselyDriver, inMemoryDriver } from "../src/drivers/index.js"; +import { buildSelectSpec } from "../src/query-builder.js"; + +const DURATION_MODEL = { "metadata.root": { package: "p", children: [ + { "object.value": { name: "Policy", children: [ + { "field.duration": { name: "timeout" } }, + { "field.duration": { name: "steps", isArray: true } }, + ] } }, + { "object.entity": { name: "Job", children: [ + { "source.rdb": { "@table": "jobs" } }, + { "field.long": { name: "id" } }, + { "field.duration": { name: "wait" } }, + { "field.object": { name: "policy", "@objectRef": "Policy", "@storage": "jsonb" } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, +] } }; + +async function durationRoot(): Promise { + const r = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(DURATION_MODEL))]); + expect(r.errors).toEqual([]); + return r.root; +} + +async function withSqlite(fn: (om: ObjectManager, db: Kysely>) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "om-duration-")); + const db = new Kysely>({ dialect: new LibsqlDialect({ url: `file:${join(dir, "t.db")}` }) }); + try { + await sql.raw("CREATE TABLE jobs (id INTEGER PRIMARY KEY AUTOINCREMENT, wait INTEGER, policy TEXT)").execute(db); + await fn(new ObjectManager({ metadata: await durationRoot(), driver: kyselyDriver({ db, dialect: "sqlite" }) }), db); + } finally { + await db.destroy(); + rmSync(dir, { recursive: true, force: true }); + } +} + +test("SQLite: a duration is stored as microseconds; one inside a jsonb value object as the canonical string", async () => { + await withSqlite(async (om, db) => { + const created = await om.create("Job", { wait: "PT90M", policy: { timeout: "P1D", steps: ["PT60S", "-PT90S"] } }); + expect(created.wait).toBe("PT1H30M"); + expect(created.policy).toEqual({ timeout: "PT24H", steps: ["PT1M", "-PT1M30S"] }); + const stored = (await sql.raw("SELECT wait, policy FROM jobs").execute(db)).rows[0] as Record; + expect(Number(stored["wait"])).toBe(5_400_000_000); + expect(JSON.parse(stored["policy"]!)).toEqual({ timeout: "PT24H", steps: ["PT1M", "-PT1M30S"] }); + }); +}); + +test("SQLite: equality, range filters and sort follow the span", async () => { + await withSqlite(async (om) => { + await om.create("Job", { wait: "PT1H30M" }); + await om.create("Job", { wait: "-PT1M30S" }); + expect((await om.findMany("Job", { wait: "PT90M" })).map((r) => r.wait)).toEqual(["PT1H30M"]); + expect((await om.findMany("Job", { wait: { $in: ["-PT90S", "PT5400S"] } })).length).toBe(2); + expect(await om.count("Job", { wait: { $ne: "P0D" } })).toBe(2); + await om.create("Job", { wait: "PT9S" }); + await om.create("Job", { wait: "PT10S" }); + expect((await om.findMany("Job", { wait: { $gt: "PT9S" } }, { orderBy: ["wait", "asc"] })).map((r) => r.wait)) + .toEqual(["PT10S", "PT1H30M"]); + expect((await om.findMany("Job", {}, { orderBy: ["wait", "asc"] })).map((r) => r.wait)) + .toEqual(["-PT1M30S", "PT9S", "PT10S", "PT1H30M"]); + await expect(om.findMany("Job", { wait: "P1M" })).rejects.toThrow(/not an ISO-8601 duration/); + }); +}); + +test("in memory: durations compare and sort as spans, not as text", async () => { + const om = new ObjectManager({ metadata: await durationRoot(), driver: inMemoryDriver() }); + for (const wait of ["PT10H", "PT9H", "-PT1M", "P1W"]) await om.create("Job", { wait }); + expect((await om.findMany("Job", {}, { orderBy: ["wait", "asc"] })).map((r) => r.wait)) + .toEqual(["-PT1M", "PT9H", "PT10H", "PT168H"]); + expect((await om.findMany("Job", { wait: { $gt: "PT9H" } }, { orderBy: ["wait", "asc"] })).map((r) => r.wait)) + .toEqual(["PT10H", "PT168H"]); + expect((await om.findMany("Job", { wait: "PT600M" })).map((r) => r.wait)).toEqual(["PT10H"]); +}); + +test("in memory: a duration seeded in its wire form still filters and sorts against stored ones", async () => { + // The seed's stored form is the bigint microsecond count; a wire-form string is compared with it as a span. + const om = new ObjectManager({ metadata: await durationRoot(), driver: inMemoryDriver({ + seed: { jobs: [{ id: 1, wait: "PT10H" }, { id: 2, wait: 32_400_000_000n }] }, + }) }); + await om.create("Job", { wait: "PT1H" }); + expect((await om.findMany("Job", { wait: "PT600M" })).map((r) => r.id)).toEqual([1]); + expect((await om.findMany("Job", { wait: { $in: ["PT9H", "PT1H"] } })).length).toBe(2); + expect((await om.findMany("Job", { wait: { $gt: "PT2H" } }, { orderBy: ["wait", "asc"] })).map((r) => r.wait)) + .toEqual(["PT9H", "PT10H"]); + expect((await om.findMany("Job", {}, { orderBy: ["wait", "asc"] })).map((r) => r.wait)).toEqual(["PT1H", "PT9H", "PT10H"]); +}); + +test("a filter operand on Postgres is the interval spelling, never a signed ISO literal", async () => { + const root = await durationRoot(); + const spec = buildSelectSpec(root.findObject("Job")!, { wait: "-PT90S" }, {}, undefined, undefined, "postgres"); + expect(spec.where).toEqual({ kind: "eq", column: "wait", value: "-00:01:30" }); + const sqlite = buildSelectSpec(root.findObject("Job")!, { wait: "-PT90S" }, {}, undefined, undefined, "sqlite"); + expect(sqlite.where).toEqual({ kind: "eq", column: "wait", value: -90_000_000 }); + // With no dialect there is no stored form, so the filter is refused rather than bound as text. + expect(() => buildSelectSpec(root.findObject("Job")!, { wait: "-PT90S" }, {})).toThrow(/needs the driver's dialect/); +}); + +test("SQLite: an isArray duration column is a JSON array of canonical ISO strings, as the generated schema writes it", async () => { + const model = { "metadata.root": { package: "p", children: [ + { "object.entity": { name: "Run", children: [ + { "source.rdb": { "@table": "runs" } }, + { "field.long": { name: "id" } }, + { "field.duration": { name: "laps", isArray: true } }, + { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, + ] } }, + ] } }; + const r = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(model))]); + expect(r.errors).toEqual([]); + const dir = mkdtempSync(join(tmpdir(), "om-duration-array-")); + const db = new Kysely>({ dialect: new LibsqlDialect({ url: `file:${join(dir, "t.db")}` }) }); + try { + await sql.raw("CREATE TABLE runs (id INTEGER PRIMARY KEY AUTOINCREMENT, laps TEXT)").execute(db); + const om = new ObjectManager({ metadata: r.root, driver: kyselyDriver({ db, dialect: "sqlite" }) }); + const created = await om.create("Run", { laps: ["PT60S", "-PT90S"] }); + expect(created.laps).toEqual(["PT1M", "-PT1M30S"]); + const stored = (await sql.raw("SELECT laps FROM runs").execute(db)).rows[0] as Record; + expect(JSON.parse(stored["laps"]!)).toEqual(["PT1M", "-PT1M30S"]); + } finally { + await db.destroy(); + rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/server/typescript/packages/sdk/src/collection.ts b/server/typescript/packages/sdk/src/collection.ts index b0f10e0ce0..98803015db 100644 --- a/server/typescript/packages/sdk/src/collection.ts +++ b/server/typescript/packages/sdk/src/collection.ts @@ -22,8 +22,10 @@ import { type PhysicalNaming, type WireFormat, DEFAULT_WIRE_FORMAT, + type EnumDefaults, + DEFAULT_ENUM_DEFAULTS, } from "@metaobjectsdev/metadata"; -import { CONFIG_FILE, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; +import { CONFIG_FILE, enumDefaultsFromConfig, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; import { discoverCollectionRoot, exists, isDir } from "./discovery.js"; import { compileScope, matchesScope, type Scope } from "./scope.js"; import { DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, isMetadataFile } from "./metadata-files.js"; @@ -85,6 +87,14 @@ export interface Collection { readonly physicalNaming: PhysicalNaming; /** The `wire` block of the project config (ADR-0019 Amendment 2), default when absent. */ readonly wireFormat: WireFormat; + /** The `enums` block of the project config (string-stored enum defaults), none when absent. + * Optional so a hand-built Collection from before the block existed still type-checks. */ + readonly enumDefaults?: EnumDefaults | undefined; + /** `migrate.conventions` of the project config — a schema-conventions preset name — when set. */ + readonly schemaConventions: string | undefined; + /** `migrate.checks.skipRedundant` of the project config: leave out the validator CHECKs a + * column type already enforces. False when absent. */ + readonly skipRedundantChecks: boolean; /** FR-023 — the `FileSource` id each file loads under, for the dependency * ARTIFACTS only (`dep:/`); own files are absent and keep the * default `basename(path)`. Threaded to `loadMemory` so every node an artifact @@ -272,6 +282,9 @@ export async function resolveCollection( let libraries: readonly string[] = []; let physicalNaming: PhysicalNaming = DEFAULT_PHYSICAL_NAMING; let wireFormat: WireFormat = DEFAULT_WIRE_FORMAT; + let enumDefaults: EnumDefaults = DEFAULT_ENUM_DEFAULTS; + let schemaConventions: string | undefined; + let skipRedundantChecks = false; if (hasConfig) { // No try/catch here: a config.json that EXISTS but fails to load @@ -288,6 +301,9 @@ export async function resolveCollection( libraries = cfg.libraries; physicalNaming = physicalNamingFromConfig(cfg); wireFormat = wireFormatFromConfig(cfg); + enumDefaults = enumDefaultsFromConfig(cfg); + schemaConventions = cfg.migrate?.conventions; + skipRedundantChecks = cfg.migrate?.checks?.skipRedundant === true; // An unknown token is a hard config error naming the valid ones, while // `librarySources` keeps skipping one silently for a programmatic caller. The two // are deliberately different: an API caller asking for a library this version does @@ -402,6 +418,9 @@ export async function resolveCollection( libraries, physicalNaming, wireFormat, + enumDefaults, + schemaConventions, + skipRedundantChecks, fileIds: new Map(dependencies.map((d) => [d.artifactPath, d.sourceId])), importedPackages: [...importedPackages].sort(), importedNodes, diff --git a/server/typescript/packages/sdk/src/config.ts b/server/typescript/packages/sdk/src/config.ts index ae2e385592..714862c6dd 100644 --- a/server/typescript/packages/sdk/src/config.ts +++ b/server/typescript/packages/sdk/src/config.ts @@ -8,10 +8,15 @@ import { DEFAULT_PHYSICAL_NAMING, TABLE_NAMING_RULES, DEFAULT_WIRE_FORMAT, + DEFAULT_ENUM_DEFAULTS, + type EnumDefaults, WIRE_DECIMALS, WIRE_NULL_MEMBERS, + WIRE_DURATIONS, WIRE_KEY_DECIMALS, WIRE_KEY_NULL_MEMBERS, + WIRE_KEY_DURATIONS, + type WireDurations, type WireDecimals, type WireFormat, type WireNullMembers, @@ -81,6 +86,14 @@ const MigrateBlock = z.object({ * Include-only — there's no `migrate.scope.exclude`, since a migration run * is scoped to what it's touching, not filtered down from "everything". */ scope: z.array(z.string().min(1)), + /** Schema conventions for the database objects no code addresses: a preset name + * (`default`, `efcore`); meta migrate refuses a name it does not know. A project's own + * conventions object is supplied from metaobjects.config.ts instead. */ + conventions: z.string().min(1), + /** Which validator CHECKs the expected schema carries. `skipRedundant` (default false) leaves + * out a CHECK the column type already enforces, for a database whose CHECKs were written by + * hand only where the type does not; a CHECK with a declared `@constraintName` is kept. */ + checks: z.object({ skipRedundant: z.boolean() }).partial().strict(), // .strict() for the same reason as the top level and the source arms below, and // with sharper teeth here: `{ migrate: { scopee: [...], dialect: "postgres" } }` // used to parse to `{ dialect: "postgres" }`, so a typo'd `scope` key meant @@ -214,7 +227,9 @@ export const ConfigSchema = z.object({ * (default; the string the in-process type holds) or `string` (canonical W1 string). * `nullMembers`: `native` (default; as stored), `emit` (every declared member, `null` when * unset) or `omit` (drop null members) for a value object's members. Input rules do not - * follow it. Absent means the default for both. + * follow those two. `durations`: `native` (default; the language's own serializer for its + * duration type, both ways) or `iso` (canonical ISO-8601 in every port, both ways). Absent + * means the default for each. */ wire: z .strictObject({ @@ -224,12 +239,26 @@ export const ConfigSchema = z.object({ nullMembers: z.enum(WIRE_NULL_MEMBERS as unknown as [WireNullMembers, ...WireNullMembers[]], { message: `"wire.nullMembers" must be one of ${WIRE_NULL_MEMBERS.map((v) => `"${v}"`).join(", ")}`, }).optional(), + durations: z.enum(WIRE_DURATIONS as unknown as [WireDurations, ...WireDurations[]], { + message: `"wire.durations" must be one of ${WIRE_DURATIONS.map((v) => `"${v}"`).join(", ")}`, + }).optional(), }, { error: (iss) => iss.code === "unrecognized_keys" - ? `"wire" has an unknown key "${iss.keys[0]}" (expected "${WIRE_KEY_DECIMALS}" and/or "${WIRE_KEY_NULL_MEMBERS}")` + ? `"wire" has an unknown key "${iss.keys[0]}" (expected "${WIRE_KEY_DECIMALS}", "${WIRE_KEY_NULL_MEMBERS}" and/or "${WIRE_KEY_DURATIONS}")` : iss.code === "invalid_type" ? `"wire" must be an object` : undefined, }) .optional(), + /** + * How a string-stored `field.enum` is stored. `maxLength` is the column length (VARCHAR(n)) + * of an enum that declares no `@maxLength` of its own; every member must fit. Absent means + * no default: such an enum is an unbounded text column, as before. An `@intValueMap` enum is + * an integer column and takes no length. Read by every port's config reader. + */ + enums: z + .strictObject({ + maxLength: z.number().int().positive().optional(), + }) + .optional(), /** * C#-scoped settings. `outDir` is where `dotnet meta gen` writes and a bare `dotnet meta * verify` diffs, relative to the directory holding `.metaobjects/`; `namespace` is the C# @@ -262,11 +291,17 @@ export function physicalNamingFromConfig(cfg: Pick): PhysicalN }; } +/** The enum storage defaults a config declares (`enums`); none when absent. */ +export function enumDefaultsFromConfig(cfg: Pick): EnumDefaults { + return cfg.enums?.maxLength === undefined ? DEFAULT_ENUM_DEFAULTS : { maxLength: cfg.enums.maxLength }; +} + /** The wire format a config declares, each absent key at its default (ADR-0019 Amendment 2). */ export function wireFormatFromConfig(cfg: Pick): WireFormat { return { decimals: cfg.wire?.decimals ?? DEFAULT_WIRE_FORMAT.decimals, nullMembers: cfg.wire?.nullMembers ?? DEFAULT_WIRE_FORMAT.nullMembers, + durations: cfg.wire?.durations ?? DEFAULT_WIRE_FORMAT.durations, }; } diff --git a/server/typescript/packages/sdk/src/index.ts b/server/typescript/packages/sdk/src/index.ts index 6a8373cd30..fdd0ea82cd 100644 --- a/server/typescript/packages/sdk/src/index.ts +++ b/server/typescript/packages/sdk/src/index.ts @@ -16,7 +16,7 @@ // shows `resolveCollection`, the documented single authority for locating a project. // Config -export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig } from "./config.js"; +export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig, enumDefaultsFromConfig } from "./config.js"; export type { Config } from "./config.js"; // Metadata dependencies (FR-023) — the `dependencies` config key's schema, diff --git a/server/typescript/packages/sdk/src/memory.ts b/server/typescript/packages/sdk/src/memory.ts index 8facb118ed..32155f6ab5 100644 --- a/server/typescript/packages/sdk/src/memory.ts +++ b/server/typescript/packages/sdk/src/memory.ts @@ -15,6 +15,7 @@ import { type MetaRoot, type PhysicalNaming, type WireFormat, + type EnumDefaults, } from "@metaobjectsdev/metadata"; import { FileSource } from "@metaobjectsdev/metadata/core"; // FR-043 — a node-only subpath (it reaches the filesystem), which is why it is imported @@ -114,6 +115,8 @@ export interface LoadMemoryOptions { physicalNaming?: PhysicalNaming; /** The project wire format; defaults to the collection's (`resolveCollection(...).wireFormat`) when `files` is not passed. */ wireFormat?: WireFormat; + /** The project's enum storage defaults; defaults to the collection's (`resolveCollection(...).enumDefaults`) when `files` is not passed. */ + enumDefaults?: EnumDefaults; } /** Default provider bundle threaded by {@link loadMemory} when no options @@ -169,7 +172,7 @@ export async function loadMemory( repoRoot: string, options?: LoadMemoryOptions, ): Promise { - const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat } = + const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults } = await loadCollectionRaw(repoRoot, options, options?.strict === true); if (result.errors.length > 0) { @@ -190,6 +193,7 @@ export async function loadMemory( if (physicalNaming !== undefined) result.root.physicalNaming = physicalNaming; if (wireFormat !== undefined) result.root.wireFormat = wireFormat; + if (enumDefaults !== undefined) result.root.enumDefaults = enumDefaults; return result.root; } @@ -264,6 +268,7 @@ async function loadCollectionRaw( libraries: readonly string[] | undefined; physicalNaming: PhysicalNaming | undefined; wireFormat: WireFormat | undefined; + enumDefaults: EnumDefaults | undefined; }> { const extra = options?.providers ?? []; let providers: readonly MetaDataTypeProvider[]; @@ -292,6 +297,7 @@ async function loadCollectionRaw( let libraries: readonly string[] | undefined = options?.libraries; let physicalNaming: PhysicalNaming | undefined = options?.physicalNaming; let wireFormat: WireFormat | undefined = options?.wireFormat; + let enumDefaults: EnumDefaults | undefined = options?.enumDefaults; if (options?.files !== undefined) { paths = [...options.files]; fileIds = options.fileIds; @@ -312,6 +318,7 @@ async function loadCollectionRaw( libraries ??= collection.libraries; physicalNaming ??= collection.physicalNaming; wireFormat ??= collection.wireFormat; + enumDefaults ??= collection.enumDefaults; } const loader = new MetaDataLoader({ @@ -337,7 +344,7 @@ async function loadCollectionRaw( return id === undefined ? new FileSource(p) : new FileSource(p, { id }); }), ]); - return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat }; + return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults }; } /** Every source file that contributed to a node, across the envelope variants that diff --git a/server/typescript/packages/sdk/test/enum-length-conformance.test.ts b/server/typescript/packages/sdk/test/enum-length-conformance.test.ts new file mode 100644 index 0000000000..0e4886a713 --- /dev/null +++ b/server/typescript/packages/sdk/test/enum-length-conformance.test.ts @@ -0,0 +1,81 @@ +// The cross-port enum-length corpus, run through TypeScript's own config reader and +// resolver: the `enums` block parses with the schema `.metaobjects/config.json` is read with, +// the loaded root carries it, and every enum's column length resolves through +// resolveEnumMaxLength, the function `meta migrate` and the generated bindings call. + +import { describe, expect, test } from "bun:test"; +import { copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { FileSource } from "@metaobjectsdev/metadata/core"; +import { MetaDataLoader, MetaModelError, isMetaRoot, resolveEnumMaxLength, type MetaObject, type MetaRoot } from "@metaobjectsdev/metadata"; +import { ConfigSchema, enumDefaultsFromConfig } from "../src/config.js"; +import { loadMemory } from "../src/memory.js"; + +const CORPUS = resolve(import.meta.dir, "../../../../../fixtures/enum-length-conformance"); + +interface Case { name: string; enums: unknown; lengths: Record } +interface Overflow { name: string; enums: unknown; field: string; member: string } +const corpus = JSON.parse(readFileSync(join(CORPUS, "cases.json"), "utf8")) as { + cases: Case[]; + overflow: Overflow[]; + invalid: { name: string; enums: unknown }[]; +}; + +const configFor = (enums: unknown): Record => + enums === null ? { schema_version: 1 } : { schema_version: 1, enums }; + +async function loadCorpusModel(enums: unknown): Promise { + const result = await new MetaDataLoader().load([new FileSource(join(CORPUS, "meta.enums.json"))]); + expect(result.errors).toEqual([]); + if (!isMetaRoot(result.root)) throw new Error("not a root"); + result.root.enumDefaults = enumDefaultsFromConfig(ConfigSchema.parse(configFor(enums))); + return result.root; +} + +const ticketField = (root: MetaRoot, name: string) => + (root.findObject("Ticket") as MetaObject).fields().find((f) => f.name === name)!; + +describe("enum-length conformance", () => { + for (const c of corpus.cases) { + test(`${c.name}: every enum's column length`, async () => { + const root = await loadCorpusModel(c.enums); + const actual: Record = {}; + for (const name of Object.keys(c.lengths)) actual[name] = resolveEnumMaxLength(ticketField(root, name)) ?? null; + expect(actual).toEqual(c.lengths); + }); + } + + for (const o of corpus.overflow) { + test(`${o.name}: refused, naming the member`, async () => { + const root = await loadCorpusModel(o.enums); + let caught: unknown; + try { resolveEnumMaxLength(ticketField(root, o.field)); } catch (e) { caught = e; } + expect(caught).toBeInstanceOf(MetaModelError); + expect((caught as MetaModelError).code).toBe("ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH"); + expect((caught as Error).message).toContain(o.member); + }); + } + + for (const c of corpus.invalid) { + test(`${c.name}: the config is refused when read`, () => { + expect(ConfigSchema.safeParse(configFor(c.enums)).success).toBe(false); + }); + } + + test("the project-aware load sets the root from .metaobjects/config.json", async () => { + const withDefault = corpus.cases.find((c) => c.enums !== null && Object.keys(c.enums as object).length > 0)!; + const dir = mkdtempSync(join(tmpdir(), "mo-enums-")); + try { + mkdirSync(join(dir, ".metaobjects")); + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, ".metaobjects", "config.json"), JSON.stringify(configFor(withDefault.enums))); + copyFileSync(join(CORPUS, "meta.enums.json"), join(dir, "metaobjects", "meta.enums.json")); + const root = await loadMemory(dir); + expect(root.enumDefaults).toEqual(withDefault.enums as object); + expect(resolveEnumMaxLength(ticketField(root, "status")) ?? null).toBe(withDefault.lengths.status ?? null); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/sdk/test/physical-naming-conformance.test.ts b/server/typescript/packages/sdk/test/physical-naming-conformance.test.ts index ad8281861f..c2afeec46f 100644 --- a/server/typescript/packages/sdk/test/physical-naming-conformance.test.ts +++ b/server/typescript/packages/sdk/test/physical-naming-conformance.test.ts @@ -11,8 +11,12 @@ import { FileSource } from "@metaobjectsdev/metadata/core"; import { MetaDataLoader, isMetaRoot, + isMetaObject, + refTarget, resolveColumnName, + resolveFlattenedColumnName, resolveTableName, + FIELD_ATTR_OBJECT_REF, type MetaRoot, } from "@metaobjectsdev/metadata"; import { ConfigSchema, physicalNamingFromConfig } from "../src/config.js"; @@ -49,9 +53,17 @@ function assertNames(root: MetaRoot, c: NamingCase): void { expect(tables).toEqual(c.tables); const columns: Record = {}; for (const key of Object.keys(c.columns)) { - const [objectName, fieldName] = key.split(".") as [string, string]; + const [objectName, fieldName, memberName] = key.split(".") as [string, string, string | undefined]; const field = root.findObject(objectName)!.fields().find((f) => f.name === fieldName)!; - columns[key] = resolveColumnName(field); + if (memberName === undefined) { + columns[key] = resolveColumnName(field); + continue; + } + // `..`: a member of the flattened value object the field embeds. + const target = refTarget(field, FIELD_ATTR_OBJECT_REF); + if (target === undefined || !isMetaObject(target)) throw new Error(`${key}: @objectRef does not resolve`); + const member = target.fields().find((f) => f.name === memberName)!; + columns[key] = resolveFlattenedColumnName(field, member); } expect(columns).toEqual(c.columns); } diff --git a/server/typescript/packages/sdk/test/schema-conventions-config.test.ts b/server/typescript/packages/sdk/test/schema-conventions-config.test.ts new file mode 100644 index 0000000000..2938f21f3e --- /dev/null +++ b/server/typescript/packages/sdk/test/schema-conventions-config.test.ts @@ -0,0 +1,80 @@ +// `migrate.conventions` and `migrate.checks` in .metaobjects/config.json: the schema +// conventions meta migrate names database-only objects with, and which validator CHECKs it +// emits. Both are migrate settings, so the resolved collection carries them and the loaded +// model does not; the CLI resolves them once and hands them to every expected-schema build. + +import { describe, expect, test } from "bun:test"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ConfigSchema, loadConfig } from "../src/config.js"; +import { resolveCollection } from "../src/collection.js"; +import { loadMemory } from "../src/memory.js"; + +const MODEL = { "metadata.root": { package: "w", children: [ + { "object.entity": { name: "Thing", children: [{ "field.string": { name: "id" } }] } }, +] } }; + +async function withProject(config: Record, fn: (dir: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "mo-conventions-")); + try { + mkdirSync(join(dir, ".metaobjects")); + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, ".metaobjects", "config.json"), JSON.stringify({ schema_version: 1, ...config })); + writeFileSync(join(dir, "metaobjects", "meta.w.json"), JSON.stringify(MODEL)); + return await fn(dir); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("migrate.conventions", () => { + test("is a string in the config schema", () => { + expect(ConfigSchema.safeParse({ schema_version: 1, migrate: { conventions: "efcore" } }).success).toBe(true); + expect(ConfigSchema.safeParse({ schema_version: 1, migrate: { conventions: 3 } }).success).toBe(false); + expect(ConfigSchema.safeParse({ schema_version: 1, migrate: { conventions: "" } }).success).toBe(false); + }); + + test("the resolved collection carries it", async () => { + await withProject({ migrate: { conventions: "efcore" } }, async (dir) => { + expect((await resolveCollection(dir)).schemaConventions).toBe("efcore"); + }); + await withProject({}, async (dir) => { + expect((await resolveCollection(dir)).schemaConventions).toBeUndefined(); + }); + }); + + test("the loaded model does not: it is migrate's setting, not the model's", async () => { + await withProject({ migrate: { conventions: "efcore" } }, async (dir) => { + expect("schemaConventions" in (await loadMemory(dir))).toBe(false); + }); + }); +}); + +describe("migrate.checks", () => { + test("skipRedundant is carried by the collection, false when absent", async () => { + await withProject({ migrate: { checks: { skipRedundant: true } } }, async (dir) => { + expect((await resolveCollection(dir)).skipRedundantChecks).toBe(true); + }); + await withProject({ migrate: { checks: {} } }, async (dir) => { + expect((await resolveCollection(dir)).skipRedundantChecks).toBe(false); + }); + await withProject({}, async (dir) => { + expect((await resolveCollection(dir)).skipRedundantChecks).toBe(false); + }); + }); + + test("a bad checks block is refused, naming the key", async () => { + const refusal = (checks: unknown) => withProject({ migrate: { checks } }, async (dir) => { + try { + await loadConfig(join(dir, ".metaobjects")); + } catch (err) { + return (err as Error).message; + } + return "accepted"; + }); + expect(await refusal(true)).toMatch(/migrate\.checks: .*expected object/); + expect(await refusal({ skipRedundnat: true })).toMatch(/migrate\.checks: .*skipRedundnat/); + expect(await refusal({ skipRedundant: "yes" })).toMatch(/migrate\.checks\.skipRedundant: .*expected boolean/); + }); +}); diff --git a/server/typescript/packages/sdk/test/wire-conformance.test.ts b/server/typescript/packages/sdk/test/wire-conformance.test.ts index 366bdc7646..a25bf0f279 100644 --- a/server/typescript/packages/sdk/test/wire-conformance.test.ts +++ b/server/typescript/packages/sdk/test/wire-conformance.test.ts @@ -6,13 +6,13 @@ import { describe, expect, test } from "bun:test"; import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; -import { DEFAULT_WIRE_FORMAT, wireFormatOf, type WireDecimals, type WireNullMembers } from "@metaobjectsdev/metadata"; +import { DEFAULT_WIRE_FORMAT, wireFormatOf, type WireDecimals, type WireDurations, type WireNullMembers } from "@metaobjectsdev/metadata"; import { ConfigSchema, wireFormatFromConfig } from "../src/config.js"; import { loadMemory } from "../src/memory.js"; const CORPUS = resolve(import.meta.dir, "../../../../../fixtures/wire-conformance/config-blocks.json"); -interface WireCase { name: string; wire: unknown; decimals: WireDecimals; nullMembers: WireNullMembers } +interface WireCase { name: string; wire: unknown; decimals: WireDecimals; nullMembers: WireNullMembers; durations: WireDurations } const corpus = JSON.parse(readFileSync(CORPUS, "utf8")) as { cases: WireCase[]; invalid: { name: string; wire: unknown }[]; @@ -29,9 +29,9 @@ const MODEL = { "metadata.root": { package: "w", children: [ describe("wire conformance (ADR-0019 Amendment 2)", () => { for (const c of corpus.cases) { - test(`${c.name}: resolves to decimals=${c.decimals} nullMembers=${c.nullMembers}`, () => { + test(`${c.name}: resolves to decimals=${c.decimals} nullMembers=${c.nullMembers} durations=${c.durations}`, () => { const wire = wireFormatFromConfig(ConfigSchema.parse(configFor(c.wire))); - expect(wire).toEqual({ decimals: c.decimals, nullMembers: c.nullMembers }); + expect(wire).toEqual({ decimals: c.decimals, nullMembers: c.nullMembers, durations: c.durations }); }); } @@ -50,7 +50,7 @@ describe("wire conformance (ADR-0019 Amendment 2)", () => { test("an unknown key and a non-object block are refused by name", () => { const unknown = ConfigSchema.safeParse(configFor({ dates: "iso" })); expect(unknown.error?.issues.map((i) => i.message)).toContain( - '"wire" has an unknown key "dates" (expected "decimals" and/or "nullMembers")'); + '"wire" has an unknown key "dates" (expected "decimals", "nullMembers" and/or "durations")'); const scalar = ConfigSchema.safeParse(configFor("string")); expect(scalar.error?.issues.map((i) => i.message)).toContain('"wire" must be an object'); }); @@ -64,8 +64,8 @@ describe("wire conformance (ADR-0019 Amendment 2)", () => { writeFileSync(join(dir, ".metaobjects", "config.json"), JSON.stringify(configFor(portable.wire))); writeFileSync(join(dir, "metaobjects", "meta.w.json"), JSON.stringify(MODEL)); const root = await loadMemory(dir); - expect(root.wireFormat).toEqual({ decimals: "string", nullMembers: "omit" }); - expect(wireFormatOf(root.findObject("Thing")!)).toEqual({ decimals: "string", nullMembers: "omit" }); + expect(root.wireFormat).toEqual({ decimals: "string", nullMembers: "omit", durations: "iso" }); + expect(wireFormatOf(root.findObject("Thing")!)).toEqual({ decimals: "string", nullMembers: "omit", durations: "iso" }); } finally { rmSync(dir, { recursive: true, force: true }); } diff --git a/server/typescript/packages/test-generators/src/templates/drizzle-schema.ts b/server/typescript/packages/test-generators/src/templates/drizzle-schema.ts index 93e85d56a0..74df06cb0d 100644 --- a/server/typescript/packages/test-generators/src/templates/drizzle-schema.ts +++ b/server/typescript/packages/test-generators/src/templates/drizzle-schema.ts @@ -18,7 +18,7 @@ import { import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { - mapColumnType, PG_IDENTITY_CAPABLE_FNS, + mapColumnType, PG_IDENTITY_CAPABLE_FNS, DURATION_COLUMN_FN, type ColumnSpec, type EnumIntCustomType, } from "@metaobjectsdev/codegen-ts/engine"; import { tableNameFromEntity } from "@metaobjectsdev/codegen-ts/engine"; @@ -28,6 +28,7 @@ import { } from "@metaobjectsdev/codegen-ts/engine"; import { resolveTableSchema } from "@metaobjectsdev/metadata"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; import { collectTphSubtypeFields, tphConcreteSubtypes } from "./tph-discriminator.js"; @@ -46,7 +47,8 @@ export function renderDrizzleSchema( obj: MetaObject, ctx: RenderContext, /** - * OUT param: the int-backed-enum codec const names this table DECLARES. A caller that + * OUT param: the local column-codec const names this table DECLARES (int-backed enums, and + * the duration codec). A caller that * emits another artifact referencing the same codecs into the SAME module — entity-file's * write-through replica view — passes a set here and hands it to that emitter, which then * references the consts instead of re-declaring them. A duplicate module-scope `const` is @@ -211,6 +213,8 @@ export function renderDrizzleSchema( // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by // const name so a shared enum used by two fields of the SAME entity emits once. const enumIntTypes = new Map(); + // The field.duration codec: one declaration serves every duration column of the table. + let usesDurationCodec = false; for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -225,6 +229,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, columnNameExpr(child, spec.dbName), child, ctx, isPk, pkGeneration, fkInfo, isComposite, pkg, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -244,6 +249,7 @@ export function renderDrizzleSchema( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + if (spec.durationCustomType !== undefined) usesDurationCodec = true; const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, columnNameExpr(child, spec.dbName), child, ctx, false, undefined, fkMap.get(child.name), isComposite, pkg, obj.name, true, @@ -403,9 +409,11 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} .map((t) => renderEnumIntCustomType(t, importModule)); if (declaredEnumIntCodecs !== undefined) { for (const name of enumIntTypes.keys()) declaredEnumIntCodecs.add(name); + if (usesDurationCodec) declaredEnumIntCodecs.add(DURATION_COLUMN_FN); } - const blocks: Code[] = [...enumIntBlocks, tableBlock]; + const durationBlocks = usesDurationCodec ? [renderDurationCustomType(dialect, importModule)] : []; + const blocks: Code[] = [...durationBlocks, ...enumIntBlocks, tableBlock]; if (relationsBlock !== null) blocks.push(relationsBlock); return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); } @@ -518,7 +526,9 @@ function renderColumn( const fnSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); let baseCall: Code; if (spec.fnOptions !== undefined && Object.keys(spec.fnOptions).length > 0) { diff --git a/server/typescript/packages/test-generators/src/templates/duration-wire-helper.ts b/server/typescript/packages/test-generators/src/templates/duration-wire-helper.ts new file mode 100644 index 0000000000..81aeaac06e --- /dev/null +++ b/server/typescript/packages/test-generators/src/templates/duration-wire-helper.ts @@ -0,0 +1,213 @@ +// The wire rule for a field.duration, as text a generated model module carries as local code. +// Generated model code imports nothing from MetaObjects, so the rule is emitted, not imported; +// metadata/src/core/field/duration-wire.ts holds the same rule and both are held to +// fixtures/duration-wire/vectors.json (test/duration-wire-helper.test.ts evaluates the text +// emitted here against them). +// +// Accepted: optional weeks (exactly 7 days) and days, then hours, minutes and seconds; the seconds +// fraction may have any number of digits. Canonical: `[-]PT[H][M][[.]S]` — hours never +// folded into days, zero components left out, zero is `PT0S`, one leading `-`. A span is counted in +// microseconds, so the fraction keeps six digits and the rest is truncated toward zero. +// +// The emitted code counts a span in microseconds as a `bigint`, so it has no limit of its own: a +// span is as long as the stack it is stored in allows. The only limit here is a column's — the +// 64-bit integer a SQLite, D1 or MySQL column holds (refused past it, never wrapped). Postgres's +// interval has its own range, which Postgres enforces. The emitted code calls `BigInt(...)` and +// writes no bigint literal, so it needs the ES2020 `BigInt` global and nothing newer. + +import { code, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_PATTERN } from "@metaobjectsdev/metadata"; +import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; +import { DURATION_COLUMN_FN } from "@metaobjectsdev/codegen-ts/engine"; + +/** The accepted ISO-8601 duration form as a JS regex literal — the metadata's own pattern. */ +export const DURATION_REGEX_LITERAL = `/${DURATION_PATTERN}/`; + +/** The Zod expression that accepts a field.duration as sent (and as read back): the accepted form. */ +export const ZOD_DURATION_EXPR = `z.string().regex(${DURATION_REGEX_LITERAL})`; + +export const DURATION_WRITE_HELPER = "durationWrite"; + +/** The Zod expression of a duration WRITE field: the accepted form, rewritten to the canonical one, + * so equal spans are equal strings on the wire and in a jsonb value object. */ +export const ZOD_DURATION_WRITE_EXPR = `${ZOD_DURATION_EXPR}.transform(${DURATION_WRITE_HELPER})`; + +/** The accepted form with its components captured: sign, weeks, days, hours, minutes, seconds, + * fraction. Equivalent to DURATION_PATTERN (the same lookaheads refuse `P`, `PT` and `P1DT`). */ +const ISO_PARTS_SOURCE = String.raw`/^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/`; + +/** The default (`IntervalStyle = postgres`) text of a Postgres interval: signed year / mon / day + * parts, then an optionally signed `H:MM:SS[.ffffff]`. */ +const PG_INTERVAL_SOURCE = String.raw`/^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/`; + +/** Indent every line of `block` by `n` spaces (blank lines stay blank). */ +function indent(block: string, n: number): string { + const pad = " ".repeat(n); + return block.split("\n").map((l) => (l === "" ? l : pad + l)).join("\n"); +} + +/** Microsecond unit constants, as `BigInt(...)` calls (no bigint literal). */ +const UNIT_LINES = [ + "const ZERO = BigInt(0);", + "const SECOND = BigInt(1000000);", + "const MINUTE = BigInt(60000000);", + "const HOUR = BigInt(3600000000);", + "const DAY = BigInt(86400000000);", + "const WEEK = BigInt(604800000000);", +].join("\n"); + +/** `parseIso(text)`: the microseconds of an accepted ISO-8601 duration, or undefined. */ +const PARSE_ISO_FN = [ + "const parseIso = (text: string): bigint | undefined => {", + ` const p = ${ISO_PARTS_SOURCE}.exec(text);`, + " if (p === null) return undefined;", + " const mag = BigInt(p[2] ?? 0) * WEEK + BigInt(p[3] ?? 0) * DAY + BigInt(p[4] ?? 0) * HOUR", + ' + BigInt(p[5] ?? 0) * MINUTE + BigInt(p[6] ?? 0) * SECOND + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6));', + ' return p[1] === "-" ? -mag : mag;', + "};", +].join("\n"); + +/** `isoText(micros)`: the canonical wire form. */ +const ISO_TEXT_FN = [ + "const isoText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + " const s = rest / SECOND;", + " const f = rest % SECOND;", + ' let out = "PT";', + ' if (h > ZERO) out += String(h) + "H";', + ' if (m > ZERO) out += String(m) + "M";', + ' if (s > ZERO || f > ZERO) out += String(s) + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + "S";', + ' if (out === "PT") return "PT0S";', + ' return neg ? "-" + out : out;', + "};", +].join("\n"); + +export const DURATION_WRITE_HELPER_DECL = [ + "/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten", + " * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one", + " * leading \"-\" for a negative span. Counted in bigint microseconds, so any length is exact and a", + " * fraction past six digits is truncated toward zero. Generated. */", + `function ${DURATION_WRITE_HELPER}(text: string): string {`, + indent(UNIT_LINES, 2), + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + " const micros = parseIso(text);", + " return micros === undefined ? text : isoText(micros);", + "}", +].join("\n"); + +/** The physical column type a duration column's `customType` declares, per dialect: the + * Postgres-native interval; a 64-bit integer count of microseconds on SQLite / D1 (INTEGER) and + * MySQL (BIGINT) — the persistence corpus's canonical schemas, and what `meta migrate` writes. */ +export function durationColumnDataType(dialect: Dialect): string { + return dialect === "postgres" ? "interval" : dialect === "mysql" ? "bigint" : "integer"; +} + +/** + * The body of the duration column's declaration, around the `customType` reference (which the + * caller supplies as an import). `[before, after]`: `before + customType + after` is the whole + * declaration. Exported so a test can evaluate the exact text a generated module carries. + * + * - `toDriver` binds what the column holds: on Postgres the interval spelling `[-]HH:MM:SS[.f]` + * (Postgres refuses a signed ISO-8601 duration, and returns a value written this way in exactly + * this spelling); on SQLite, D1 and MySQL the microsecond count — a number when it is a safe + * integer, else a bigint — refused past the 64-bit range, which is that column's limit. It takes + * the value as an ISO-8601 duration (what the generated write schema hands it), or as what + * runtime-ts already binds a filter operand in (the Postgres text; the microsecond integer), and + * throws for anything else. + * - `fromDriver` reads what the driver returns — the microsecond integer (a number, a bigint or a + * digit string), ISO-8601 text, Postgres interval text in the default `IntervalStyle`, or a + * node-postgres interval object — as the canonical ISO-8601 form. An interval with a year or + * month part is not a fixed-length span: reading one throws. + */ +export function durationColumnDeclParts(dialect: Dialect): [string, string] { + const postgres = dialect === "postgres"; + const before = [ + "/** A field.duration column. Binds " + (postgres + ? "the Postgres interval spelling [-]HH:MM:SS[.f] (Postgres refuses a signed ISO-8601 duration);" + : "the span as a 64-bit integer count of microseconds (refused past that range);"), + " * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month", + " * part. Counted in bigint microseconds, so the column is the only limit. Generated. */", + `const ${DURATION_COLUMN_FN} = (() => {`, + indent(UNIT_LINES, 2), + ` const INT64_MAX = BigInt("9223372036854775807");`, + indent(PARSE_ISO_FN, 2), + indent(ISO_TEXT_FN, 2), + indent([ + "const pgText = (micros: bigint): string => {", + " const neg = micros < ZERO;", + " let rest = neg ? -micros : micros;", + ' const two = (n: bigint): string => String(n).padStart(2, "0");', + " const h = rest / HOUR;", + " rest %= HOUR;", + " const m = rest / MINUTE;", + " rest %= MINUTE;", + ' let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND);', + " const f = rest % SECOND;", + ' if (f > ZERO) out += "." + String(f).padStart(6, "0").replace(/0+$/, "");', + ' return neg ? "-" + out : out;', + "};", + "const stored = (micros: bigint): number | bigint => {", + " if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) {", + ' throw new Error("the span " + isoText(micros) + " does not fit the 64-bit microsecond column a field.duration is stored in");', + " }", + " const n = Number(micros);", + " return Number.isSafeInteger(n) ? n : micros;", + "};", + "const notFixed = (v: unknown): Error =>", + ' new Error("the interval " + JSON.stringify(v) + " has a year or month part, so it is not a fixed-length field.duration");', + "const read = (v: unknown): bigint => {", + ' if (typeof v === "bigint") return v;', + ' if (typeof v === "number" && Number.isInteger(v)) return BigInt(v);', + ' if (typeof v === "string") {', + " if (/^-?\\d+$/.test(v)) return BigInt(v);", + " const iso = parseIso(v);", + " if (iso !== undefined) return iso;", + ` const p = v.length === 0 ? null : ${PG_INTERVAL_SOURCE}.exec(v);`, + ' if (p === null) throw new Error("not a field.duration value: " + JSON.stringify(v));', + " if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) throw notFixed(v);", + " let time = ZERO;", + " if (p[5] !== undefined) {", + ' time = BigInt(p[5]) * HOUR + BigInt(p[6] ?? 0) * MINUTE + BigInt(p[7] ?? 0) * SECOND + BigInt((p[8] ?? "").padEnd(6, "0"));', + ' if (p[4] === "-") time = -time;', + " }", + " return BigInt(p[3] ?? 0) * DAY + time;", + " }", + ' if (v !== null && typeof v === "object") {', + " const o = v as Record;", + " if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) throw notFixed(v);", + " const whole = (n: number | undefined): bigint => BigInt(Math.trunc(n ?? 0));", + " // postgres-interval carries microseconds as a fractional millisecond count.", + " return whole(o.days) * DAY + whole(o.hours) * HOUR + whole(o.minutes) * MINUTE + whole(o.seconds) * SECOND", + " + BigInt(Math.round((o.milliseconds ?? 0) * 1000));", + " }", + ' throw new Error("not a field.duration value: " + JSON.stringify(v));', + "};", + ].join("\n"), 2), + " return ", + ].join("\n"); + const after = [ + `<{ data: string; driverData: ${postgres ? "string" : "number | bigint | string"} }>({`, + ` dataType: () => ${JSON.stringify(durationColumnDataType(dialect))},`, + ` toDriver: (value) => ${postgres ? "pgText" : "stored"}(read(value)),`, + " fromDriver: (value) => isoText(read(value)),", + " });", + "})();", + ].join("\n"); + return [before, after]; +} + +/** A duration column's Drizzle `customType` declaration, emitted ahead of the table or view that + * uses it (once per module — see enum-int-codec.ts for why the order and the once matter). */ +export function renderDurationCustomType(dialect: Dialect, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const [before, after] = durationColumnDeclParts(dialect); + return code` +${before}${customTypeSym}${after} +`; +} diff --git a/server/typescript/packages/test-generators/src/templates/field-meta.ts b/server/typescript/packages/test-generators/src/templates/field-meta.ts index 6182a31306..72549fd876 100644 --- a/server/typescript/packages/test-generators/src/templates/field-meta.ts +++ b/server/typescript/packages/test-generators/src/templates/field-meta.ts @@ -22,6 +22,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STRING_FORMAT, @@ -43,6 +44,7 @@ import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; import { zodEnumExpr } from "./zod-enum.js"; import { viewForContext } from "@metaobjectsdev/codegen-ts/engine"; import { ZOD_INET_EXPR } from "./net-regex.js"; +import { ZOD_DURATION_EXPR } from "./duration-wire-helper.js"; // --------------------------------------------------------------------------- // inferViewKind @@ -140,6 +142,9 @@ export function zodTypeFor(field: MetaField, timestampMode: "date" | "string" = // (v4 or v6 literal). Zod-version-agnostic regex union (Zod 4 removed // `z.string().ip()`); see net-regex.ts. @lenient → a plain string. return field.attr(FIELD_ATTR_LENIENT) === true ? "z.string()" : ZOD_INET_EXPR; + case FIELD_SUBTYPE_DURATION: + // An ISO-8601 duration string; the column codec reads every row in the canonical form. + return ZOD_DURATION_EXPR; case FIELD_SUBTYPE_BOOLEAN: return "z.boolean()"; case FIELD_SUBTYPE_DATE: diff --git a/server/typescript/packages/test-generators/src/templates/filter-allowlist.ts b/server/typescript/packages/test-generators/src/templates/filter-allowlist.ts index 4a0cd087fd..ec7c195b7c 100644 --- a/server/typescript/packages/test-generators/src/templates/filter-allowlist.ts +++ b/server/typescript/packages/test-generators/src/templates/filter-allowlist.ts @@ -13,6 +13,7 @@ import { FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_UUID, + FIELD_SUBTYPE_DURATION, FIELD_SUBTYPE_ENUM, FIELD_ATTR_LOCAL_TIME, opsForField, @@ -70,6 +71,9 @@ const VALUE_FORMAT_SUBTYPES = new Set([ FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_DECIMAL, + // An ISO-8601 duration: runtime-ts refuses a value that is not one, and the column codec binds + // the dialect's spelling of it. + FIELD_SUBTYPE_DURATION, ]); /** Subtypes whose values are whole numbers — `field.currency` is integer minor units. */ diff --git a/server/typescript/packages/test-generators/src/templates/inferred-types.ts b/server/typescript/packages/test-generators/src/templates/inferred-types.ts index 1eec56f318..4b87f2f9bc 100644 --- a/server/typescript/packages/test-generators/src/templates/inferred-types.ts +++ b/server/typescript/packages/test-generators/src/templates/inferred-types.ts @@ -27,6 +27,7 @@ import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, FIELD_ATTR_REQUIRED, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_VALUE_TYPE, @@ -215,6 +216,8 @@ const SCALAR_TS_BY_SUBTYPE: Record = { // type, same as uuid). Other ports bind to their native URI/IP type. [FIELD_SUBTYPE_URI]: "string", [FIELD_SUBTYPE_INET]: "string", + // field.duration is an ISO-8601 duration string: TS has no native duration type. + [FIELD_SUBTYPE_DURATION]: "string", [FIELD_SUBTYPE_INT]: "number", [FIELD_SUBTYPE_LONG]: "number", [FIELD_SUBTYPE_DOUBLE]: "number", diff --git a/server/typescript/packages/test-generators/src/templates/projection-decl.ts b/server/typescript/packages/test-generators/src/templates/projection-decl.ts index e9ab1e45d2..a7ccbf0f93 100644 --- a/server/typescript/packages/test-generators/src/templates/projection-decl.ts +++ b/server/typescript/packages/test-generators/src/templates/projection-decl.ts @@ -21,7 +21,7 @@ import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { valueObjectModuleSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { Dialect } from "@metaobjectsdev/codegen-ts/engine"; -import { physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; +import { columnExpr, physicalNameExpr, sourceSchemaExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; import { renderFilterAllowlist, renderSortAllowlist } from "./filter-allowlist.js"; import { renderFilterType } from "./filter-type.js"; import { buildUiFieldDescriptor, resourcePath } from "@metaobjectsdev/codegen-ts/engine"; @@ -156,11 +156,8 @@ export function renderProjectionDecl( // A6 — reference the constant whenever the artifact is in the run AND carries this // field. A lookup MISS (a TPH subtype field the base's names artifact never saw) is // normal, not a divergence — fall back to the literal, same as every other §A6 site. - const namesEntry = names?.resolved.fields[f.name]; - const dbColExpr: Code = - names !== undefined && namesEntry !== undefined - ? code`${names.symbol}.fields.${f.name}.column` - : code`${JSON.stringify(dbCol)}`; + // A flattened field has no `column` of its own (its members do), so it takes the literal. + const dbColExpr: Code = columnExpr(names, f.name, dbCol); // #356 — a projection's descriptor is the same shape the entity descriptor emits, // and now literally the SAME derivation: `buildUiFieldDescriptor` is what // `renderEntityConstants` reads, so the view kind, the label and the currency meta diff --git a/server/typescript/packages/test-generators/src/templates/view-decl.ts b/server/typescript/packages/test-generators/src/templates/view-decl.ts index da4a523058..eec5e9ed44 100644 --- a/server/typescript/packages/test-generators/src/templates/view-decl.ts +++ b/server/typescript/packages/test-generators/src/templates/view-decl.ts @@ -14,8 +14,9 @@ import { type MetaField, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_OBJECT_REF, } from "@metaobjectsdev/metadata"; import type { ColumnNamingStrategy } from "@metaobjectsdev/codegen-ts/engine"; -import { mapColumnType, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; +import { mapColumnType, DURATION_COLUMN_FN, type EnumIntCustomType } from "@metaobjectsdev/codegen-ts/engine"; import { renderEnumIntCustomType } from "./enum-int-codec.js"; +import { renderDurationCustomType } from "./duration-wire-helper.js"; import { zodTypeFor } from "./field-meta.js"; import { columnExpr, type ObjectNames } from "@metaobjectsdev/codegen-ts/engine"; @@ -25,7 +26,8 @@ export interface ViewDeclOpts { /** Drives the timestamp column TS type (Date vs string) in the view declaration. */ readonly timestampMode: "date" | "string"; /** - * Int-backed-enum codec const names ALREADY declared in the module this view lands in. + * Local column-codec const names (int-backed enums, the duration codec) ALREADY declared in + * the module this view lands in. * A write-through entity's table and its replica view share one file, and both reference * the same codecs — so the view references these by name and declares only what the table * did not (a derived field's codec, which the write table omits). Absent = the view owns @@ -88,6 +90,8 @@ function viewColumnLine( opts: ViewDeclOpts, /** Codecs this view needs, collected for the caller to DECLARE — see below. */ enumIntTypes: Map, + /** The duration codec's name, when a column needs it — declared by the caller, as above. */ + usesDuration: Set, ): Code { const { dialect, columnNamingStrategy, timestampMode } = opts; const spec = mapColumnType(f, dialect, columnNamingStrategy, timestampMode); @@ -99,10 +103,14 @@ function viewColumnLine( if (spec.enumIntCustomType !== undefined) { enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); } + // The field.duration codec is a local const too; collected under its name, declared once. + if (spec.durationCustomType !== undefined) usesDuration.add(spec.durationCustomType.fnConstName); const colSym = spec.enumIntCustomType !== undefined ? spec.enumIntCustomType.fnConstName - : imp(`${spec.fnName}@${spec.importModule}`); + : spec.durationCustomType !== undefined + ? spec.durationCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const optsArg = spec.fnOptions && Object.keys(spec.fnOptions).length > 0 ? `, ${JSON.stringify(spec.fnOptions)}` @@ -164,7 +172,8 @@ export function renderExistingViewDecl( const { viewFn, core: viewModule } = dialectModule(opts.dialect); const viewSym = imp(`${viewFn}@${viewModule}`); const enumIntTypes = new Map(); - const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes)); + const usesDuration = new Set(); + const viewColumnLines = fields.map((f) => viewColumnLine(f, opts, enumIntTypes, usesDuration)); const viewNameExpr = typeof viewName === "string" ? code`${JSON.stringify(viewName)}` : viewName; // @schema — a view lands in a schema exactly as a table does, and migrate qualifies the @@ -188,6 +197,9 @@ export function renderExistingViewDecl( .filter((t) => alreadyDeclared === undefined || !alreadyDeclared.has(t.fnConstName)) .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) .map((t) => renderEnumIntCustomType(t, viewModule)); + if (usesDuration.has(DURATION_COLUMN_FN) && (alreadyDeclared === undefined || !alreadyDeclared.has(DURATION_COLUMN_FN))) { + enumIntBlocks.unshift(renderDurationCustomType(opts.dialect, viewModule)); + } const viewBlock = code` // View declaration — Drizzle uses this for typed SELECT queries. diff --git a/server/typescript/packages/test-generators/src/templates/zod-validators.ts b/server/typescript/packages/test-generators/src/templates/zod-validators.ts index 917a215da5..195fddfacb 100644 --- a/server/typescript/packages/test-generators/src/templates/zod-validators.ts +++ b/server/typescript/packages/test-generators/src/templates/zod-validators.ts @@ -17,6 +17,7 @@ import { FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DECIMAL, FIELD_ATTR_PRECISION, FIELD_ATTR_SCALE, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, + FIELD_SUBTYPE_DURATION, VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, VALIDATOR_SUBTYPE_NUMERIC, VALIDATOR_SUBTYPE_ARRAY, IDENTITY_ATTR_FIELDS, IDENTITY_ATTR_GENERATION, @@ -28,7 +29,9 @@ import { AUTO_SET_ON_CREATE, AUTO_SET_ON_UPDATE, VALIDATOR_ATTR_MAX, VALIDATOR_ATTR_MIN, VALIDATOR_ATTR_PATTERN, VALIDATOR_SUBTYPE_COMPARISON, VALIDATOR_SUBTYPE_REQUIRED_WHEN, - VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, + VALIDATOR_SUBTYPE_PRESENT_IFF, VALIDATOR_SUBTYPE_AT_LEAST_ONE, VALIDATOR_SUBTYPE_EXACTLY_ONE, + VALIDATOR_ATTR_CONDITION, EXPR_OP_IS_NOT_NULL, EXPR_OP_NOT, EXPR_FN_COALESCE, + FILTER_OP_IS_NULL, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, parseDurationLiteral, VALIDATOR_ATTR_LEFT, VALIDATOR_ATTR_OP, VALIDATOR_ATTR_RIGHT, VALIDATOR_ATTR_FIELD, VALIDATOR_ATTR_WHEN, VALIDATOR_ATTR_EQUALS, VALIDATOR_ATTR_FIELDS, GENERATION_INCREMENT, GENERATION_UUID, @@ -48,6 +51,7 @@ import { } from "./date-time-regex.js"; import { renderDocsFor } from "./jsdoc.js"; import { DECIMAL_WRITE_HELPER_DECL, zodDecimalWriteExpr } from "./decimal-wire-helper.js"; +import { DURATION_WRITE_HELPER_DECL, ZOD_DURATION_EXPR, ZOD_DURATION_WRITE_EXPR } from "./duration-wire-helper.js"; import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumZodConstName } from "./enums-file.js"; @@ -86,6 +90,13 @@ function decimalHelperSince(since: number): string { return decimalWriteUses > since ? `${DECIMAL_WRITE_HELPER_DECL}\n\n` : ""; } +// The same device for the duration write rule (duration-wire-helper.ts). +let durationWriteUses = 0; + +function durationHelperSince(since: number): string { + return durationWriteUses > since ? `${DURATION_WRITE_HELPER_DECL}\n\n` : ""; +} + /** True when this object declares at least one @autoSet timestamp field * (onCreate or onUpdate). Drives whether codegen emits the #203 @@ -260,13 +271,16 @@ function isInsertRequiredPk(field: MetaField, assignedPk: Set): boolean } // ── Entity-scoped cross-field validators on the full (insert) record ────────── -// validator.comparison / requiredWhen / presentIff / atLeastOne are enforced on the -// INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged +// validator.comparison / requiredWhen / presentIff / atLeastOne / exactlyOne are enforced on +// the INSERT-shaped schemas only, as one `.superRefine`. A PATCH body cannot see the merged // row, so the Update schema is deliberately left alone (the DB CHECK stays the guard // there). Semantics mirror the migrate-ts CHECK derivation: absent / null / undefined is // SQL NULL, "" is NOT null; a comparison with a NULL side passes; requiredWhen is violated // only when the gate EQUALS the literal and the target is null; presentIff requires -// (target != null) === (gate equals literal), a null gate being "not equal". +// (target != null) === (gate equals literal), a null gate being "not equal"; exactlyOne +// requires exactly one non-null field. A rule with an @condition is checked only when the +// condition is TRUE: the CHECK is NOT (condition) OR (rule), so a condition that is false or +// NULL (three-valued, as SQL evaluates it) lets the row through. const CROSS_FIELD_OPS: Record = { gt: ">", gte: ">=", lt: "<", lte: "<=", ne: "!==", eq: "===", @@ -278,7 +292,7 @@ const NUMERIC_SUBTYPES: ReadonlySet = new Set([ ]); const ORDERED_SUBTYPES: ReadonlySet = new Set([ - FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, + FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_DATE, FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_DURATION, ]); /** Subtypes with equality but no ordering: a pair of the same one may be tested with eq / ne. */ @@ -287,7 +301,7 @@ const EQUALITY_SUBTYPES: ReadonlySet = new Set([ ]); /** Whether a comparison between these two fields has a sound meaning, as in every port: - * numerics compare with each other, strings and temporals only with their own subtype and + * numerics compare with each other, strings, temporals and durations only with their own subtype and * @localTime flag (a date against a timestamp, or an instant against a wall clock, compares * two differently shaped strings, which is meaningless), * and boolean, enum, uuid, uri and inet fields only test equality. Anything else is skipped, never rejected. */ @@ -329,10 +343,47 @@ function crossFieldLiteral(raw: unknown, when: MetaField): { lit: string; numeri * subtype also exports `InsertBaseSchema` (unrefined) and `InsertBodySchema` * (refined, pinned discriminator omitted). */ export function hasInsertRefinements(obj: MetaObject): boolean { - return crossFieldRefinementBody(obj).length > 0; + return crossFieldRefinement(obj).blocks.length > 0; } -function crossFieldRefinementBody(obj: MetaObject): string[] { +/** + * How a value is compared, from the field it is or is compared with: numerically, as a + * duration's microseconds (the canonical string does not order), or as written (strings, + * dates and times in their ISO form, booleans, enum members). + */ +type CompareKind = "number" | "duration" | "plain"; + +function compareKind(f: MetaField | undefined): CompareKind { + if (f === undefined) return "plain"; + if (NUMERIC_SUBTYPES.has(f.subType)) return "number"; + if (f.subType === FIELD_SUBTYPE_DURATION) return "duration"; + return "plain"; +} + +/** The local helpers a refinement body uses, declared once at its top when needed. */ +const HELPER_CMP = ` const cmp = (a: unknown, b: unknown): number | null => { + if (a == null || b == null) return null; + if (typeof a === "number" || typeof b === "number") return Number(a) - Number(b); + if (a instanceof Date || b instanceof Date) return new Date(a as string).getTime() - new Date(b as string).getTime(); + return String(a) < String(b) ? -1 : String(a) > String(b) ? 1 : 0; + };`; +const HELPER_TRI = ` const all = (...v: Array): boolean | null => v.includes(false) ? false : v.includes(null) ? null : true; + const any = (...v: Array): boolean | null => v.includes(true) ? true : v.includes(null) ? null : false; + const not = (v: boolean | null): boolean | null => v === null ? null : !v;`; +// An ISO-8601 duration (weeks, days, hours, minutes, seconds) as microseconds, or null when the +// text is not one (the accepted form: `P`, `PT` and `P1DT` are not durations). +const HELPER_MICROS = ` const micros = (d: unknown): number | null => { + if (d == null) return null; + const m = /^([-+])?P(?!$)(?:(\\d+)W)?(?:(\\d+)D)?(?:T(?=\\d)(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+)(?:\\.(\\d+))?S)?)?$/.exec(String(d)); + if (m === null) return null; + const hours = (Number(m[2] ?? 0) * 7 + Number(m[3] ?? 0)) * 24 + Number(m[4] ?? 0); + const us = ((hours * 60 + Number(m[5] ?? 0)) * 60 + Number(m[6] ?? 0)) * 1e6 + Number((m[7] ?? "").padEnd(6, "0").slice(0, 6)); + return m[1] === "-" ? -us : us; + };`; + +interface RefinementBuild { blocks: string[]; helpers: Set } + +function crossFieldRefinement(obj: MetaObject): RefinementBuild { const inSchema = insertSchemaFieldNameSet(obj); const byName = new Map(obj.fields().map((f) => [f.name, f] as const)); // A reference must name a field that exists AND is in the insert schema; otherwise skip @@ -350,9 +401,116 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { : `row[${JSON.stringify(f.name)}]`; const issue = (path: string, msg: string) => `ctx.addIssue({ code: "custom", path: [${path}], message: ${JSON.stringify(msg)} });`; + // The helpers the rule being built uses; merged into `helpers` only when the rule is kept, + // so a rule left to the database declares nothing that nothing reads. + const helpers = new Set(); + let ruleHelpers = new Set(); + /** A value compared as `kind`. */ + const asKind = (expr: string, kind: CompareKind): string => { + if (kind === "number") return `(${expr} == null ? null : Number(${expr}))`; + if (kind === "duration") { ruleHelpers.add(HELPER_MICROS); return `micros(${expr})`; } + return expr; + }; + + /** The field a node compares as: itself, or a coalesce's first field. */ + const nodeField = (node: unknown): MetaField | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") return byName.get(n["field"]); + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + for (const a of n["args"] as unknown[]) { const f = nodeField(a); if (f !== undefined) return f; } + } + return undefined; + }; + /** Every field a value node reads: the field itself, or each field among a coalesce's arguments. */ + const nodeFields = (node: unknown): MetaField[] => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { const f = byName.get(n["field"]); return f === undefined ? [] : [f]; } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) return (n["args"] as unknown[]).flatMap(nodeFields); + return []; + }; + /** A comparison inside a condition follows the validator rule: every pair of fields it reads + * must be comparable, or the whole rule is left to the database. */ + const conditionComparable = (left: unknown, right: unknown, opName: string): boolean => { + const fields = [...nodeFields(left), ...nodeFields(right)]; + return fields.every((a, i) => fields.slice(i + 1).every((b) => crossComparable(a, b, opName))); + }; + /** A value node (field, literal, coalesce) as a TS expression compared as `kind`, or undefined. */ + const valueTs = (node: unknown, kind: CompareKind): string | undefined => { + const n = (node ?? {}) as Record; + if (typeof n["field"] === "string") { + const f = ref(n["field"]); + return f === undefined ? undefined : asKind(acc(f), kind); + } + if ("value" in n) { + const value = n["value"]; + if (kind === "duration" && typeof value === "string") { + const us = parseDurationLiteral(value); + return us === undefined ? undefined : String(Number(us)); + } + return JSON.stringify(value); + } + if (n["fn"] === EXPR_FN_COALESCE && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map((a) => valueTs(a, kind)); + if (args.some((a) => a === undefined)) return undefined; + return `([${args.join(", ")}].find((x) => x != null) ?? null)`; + } + return undefined; + }; + /** A condition node as a TS expression of type `boolean | null`, or undefined when it names + * a field the body does not carry (the rule is then left to the database). */ + const condTs = (node: unknown): string | undefined => { + const n = (node ?? {}) as Record; + const op = n["op"]; + if (typeof op === "string" && CROSS_FIELD_OPS[op] !== undefined) { + if (!conditionComparable(n["left"], n["right"], op)) return undefined; + const kind = compareKind(nodeField(n["left"]) ?? nodeField(n["right"])); + const l = valueTs(n["left"], kind); + const r = valueTs(n["right"], kind); + if (l === undefined || r === undefined) return undefined; + ruleHelpers.add(HELPER_CMP); + return `((c) => c === null ? null : c ${CROSS_FIELD_OPS[op]} 0)(cmp(${l}, ${r}))`; + } + if (op === FILTER_OP_IS_NULL || op === EXPR_OP_IS_NOT_NULL) { + const a = valueTs(n["arg"], "plain"); + if (a === undefined) return undefined; + return op === FILTER_OP_IS_NULL ? `(${a} == null)` : `(${a} != null)`; + } + if (op === EXPR_OP_NOT) { + const a = condTs(n["arg"]); + if (a === undefined) return undefined; + ruleHelpers.add(HELPER_TRI); + return `not(${a})`; + } + if ((op === FILTER_COMPOSE_AND || op === FILTER_COMPOSE_OR) && Array.isArray(n["args"])) { + const args = (n["args"] as unknown[]).map(condTs); + if (args.some((a) => a === undefined)) return undefined; + ruleHelpers.add(HELPER_TRI); + return `${op === FILTER_COMPOSE_AND ? "all" : "any"}(${args.join(", ")})`; + } + return undefined; + }; + const blocks: string[] = []; + /** Add a rule's block, guarded by its @condition when it has one. */ + const push = (v: { attr(name: string): unknown }, block: string): void => { + const condition = v.attr(VALIDATOR_ATTR_CONDITION); + const guard = condition === undefined ? undefined : condTs(condition); + if (condition !== undefined && guard === undefined) return; + for (const h of ruleHelpers) helpers.add(h); + if (guard === undefined) { blocks.push(block); return; } + const inner = block.split("\n").map((l) => (l === "" ? l : ` ${l}`)).join("\n"); + blocks.push(` if (${guard} === true) {\n${inner}\n }`); + }; + /** The @fields of an atLeastOne / exactlyOne, each in the body, or undefined. */ + const fieldList = (raw: unknown): MetaField[] | undefined => { + const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); + const refs = names.map(ref); + if (refs.length === 0 || refs.some((r) => r === undefined)) return undefined; + return refs as MetaField[]; + }; // ADR-0039: entity.validators() is the resolving accessor (never ownValidators). for (const v of obj.validators()) { + ruleHelpers = new Set(); switch (v.subType) { case VALIDATOR_SUBTYPE_COMPARISON: { const left = ref(v.attr(VALIDATOR_ATTR_LEFT)); @@ -361,12 +519,15 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const op = CROSS_FIELD_OPS[opName]; if (!left || !right || !op || left.resolvedIsArray() || right.resolvedIsArray()) break; if (!crossComparable(left, right, opName)) break; - const numeric = NUMERIC_SUBTYPES.has(left.subType); - const c = numeric + const kind = compareKind(left); + const c = kind === "number" || kind === "duration" ? "Number(l) - Number(r)" : "l instanceof Date && r instanceof Date ? l.getTime() - r.getTime() : String(l) < String(r) ? -1 : String(l) > String(r) ? 1 : 0"; - blocks.push( - ` {\n const l: unknown = ${acc(left)};\n const r: unknown = ${acc(right)};\n` + + // A duration side is read as its microseconds; text that is not a duration reads as + // null, so the rule is not checked (the field's own schema refuses that text). + const readKind: CompareKind = kind === "duration" ? "duration" : "plain"; + push(v, + ` {\n const l: unknown = ${asKind(acc(left), readKind)};\n const r: unknown = ${asKind(acc(right), readKind)};\n` + ` if (l != null && r != null) {\n const c = ${c};\n` + ` if (!(c ${op} 0)) ${issue(JSON.stringify(left.name), `${left.name} must be ${opName} ${right.name}`)}\n }\n }`, ); @@ -382,35 +543,41 @@ function crossFieldRefinementBody(obj: MetaObject): string[] { const path = JSON.stringify(target.name); const decl = ` const g: unknown = ${acc(when)};\n const t: unknown = ${acc(target)};\n const open = ${gate};\n`; if (v.subType === VALIDATOR_SUBTYPE_REQUIRED_WHEN) { - blocks.push(` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if (open && t == null) ${issue(path, `${target.name} is required when ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } else { - blocks.push(` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); + push(v, ` {\n${decl} if ((t != null) !== open) ${issue(path, `${target.name} must be present if and only if ${when.name} is ${String(v.attr(VALIDATOR_ATTR_EQUALS))}`)}\n }`); } break; } case VALIDATOR_SUBTYPE_AT_LEAST_ONE: { - const raw = v.attr(VALIDATOR_ATTR_FIELDS); - const names = Array.isArray(raw) ? raw : (typeof raw === "string" ? [raw] : []); - const refs = names.map(ref); - if (refs.length === 0 || refs.some((r) => r === undefined)) break; - const fields = refs as MetaField[]; + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; const anyPresent = fields.map((f) => `${acc(f)} != null`).join(" || "); - blocks.push(` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + push(v, ` if (!(${anyPresent})) ${issue(JSON.stringify(fields[0]!.name), `at least one of ${fields.map((f) => f.name).join(", ")} is required`)}`); + break; + } + case VALIDATOR_SUBTYPE_EXACTLY_ONE: { + const fields = fieldList(v.attr(VALIDATOR_ATTR_FIELDS)); + if (fields === undefined) break; + const present = fields.map((f) => `${acc(f)}`).join(", "); + push(v, ` if ([${present}].filter((x) => x != null).length !== 1) ${issue(JSON.stringify(fields[0]!.name), `exactly one of ${fields.map((f) => f.name).join(", ")} is required`)}`); break; } default: break; } } - return blocks; + return { blocks, helpers }; } /** `.superRefine(...)` suffix for an insert-shaped schema, or "" when the object has no * enforceable cross-field validator (output is then byte-identical to before). */ function insertRefinementSuffix(obj: MetaObject): string { - const blocks = crossFieldRefinementBody(obj); + const { blocks, helpers } = crossFieldRefinement(obj); if (blocks.length === 0) return ""; - return `.superRefine((row, ctx) => {\n${blocks.join("\n")}\n})`; + // Declared in a fixed order, so the output does not depend on which rule needed one first. + const decls = [HELPER_CMP, HELPER_TRI, HELPER_MICROS].filter((h) => helpers.has(h)); + return `.superRefine((row, ctx) => {\n${[...decls, ...blocks].join("\n")}\n})`; } /** @@ -426,6 +593,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); @@ -477,7 +645,7 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co const docsPrefix = docs ? `${docs}\n` : ""; return code` -${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ +${utcTimestampHelperSince(utcUsesBefore)}${decimalHelperSince(decimalUsesBefore)}${durationHelperSince(durationUsesBefore)}${docsPrefix}export const ${insertSchemaName} = ${z}.object({ ${joinCode(insertFieldLines, { on: ",\n" })} })${insertRefinementSuffix(obj)}; `; @@ -586,6 +754,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const z = imp("z@zod"); const utcUsesBefore = utcTimestampUses; const decimalUsesBefore = decimalWriteUses; + const durationUsesBefore = durationWriteUses; const autoGenPkFields = autoGenPkFieldNames(obj); const pkFields = primaryKeyFieldNames(obj); const assignedPkFields = assignedPkFieldNames(obj); @@ -728,7 +897,7 @@ export type ${objName}CreatePreserving = ${z}.inputtypes/field.md#fielddouble + + + + + @@ -396,6 +401,11 @@ + + + + + diff --git a/site-reference/providers.html b/site-reference/providers.html index 6668e2ed3b..47ac13f1c5 100644 --- a/site-reference/providers.html +++ b/site-reference/providers.html @@ -51,13 +51,13 @@ INDEX.md lists by type.

metaobjects-core-types

Core metaobjects metamodel types and subtypes.

-

Owns (registers): attr.base, attr.boolean, attr.class, attr.double, attr.expression, attr.filter, attr.int, attr.intMap, attr.long, attr.properties, attr.string, field.base, field.boolean, field.currency, field.date, field.decimal, field.double, field.enum, field.float, field.inet, field.int, field.long, field.map, field.object, field.string, field.time, field.timestamp, field.uri, field.uuid, identity.primary, identity.reference, identity.secondary, layout.base, layout.dataGrid, object.base, object.entity, object.projection, object.report, object.value, origin.aggregate, origin.base, origin.computed, origin.first, origin.passthrough, relationship.aggregation, relationship.association, relationship.base, relationship.composition, source.base, source.rdb, template.base, template.output, template.prompt, template.toolcall, validator.array, validator.atLeastOne, validator.base, validator.comparison, validator.length, validator.numeric, validator.presentIff, validator.regex, validator.required, validator.requiredWhen, view.base, view.currency

+

Owns (registers): attr.base, attr.boolean, attr.class, attr.double, attr.expression, attr.filter, attr.int, attr.intMap, attr.long, attr.properties, attr.string, field.base, field.boolean, field.currency, field.date, field.decimal, field.double, field.duration, field.enum, field.float, field.inet, field.int, field.long, field.map, field.object, field.string, field.time, field.timestamp, field.uri, field.uuid, identity.primary, identity.reference, identity.secondary, layout.base, layout.dataGrid, object.base, object.entity, object.projection, object.report, object.value, origin.aggregate, origin.base, origin.computed, origin.first, origin.passthrough, relationship.aggregation, relationship.association, relationship.base, relationship.composition, source.base, source.rdb, template.base, template.output, template.prompt, template.toolcall, validator.array, validator.atLeastOne, validator.base, validator.comparison, validator.exactlyOne, validator.length, validator.numeric, validator.presentIff, validator.regex, validator.required, validator.requiredWhen, view.base, view.currency

Contributes attributes:

metaobjects-db

DB-domain attributes — @column / @db.indexed / @dbColumnType on every field, @storage on field.object, @autoSet on temporal fields, @table/@kind/@role/@schema on source.rdb.

@@ -100,22 +101,32 @@

metaobjects-db

  • field.date: @autoSet, @column, @db.indexed, @dbColumnType
  • field.decimal: @column, @db.indexed, @dbColumnType
  • field.double: @column, @db.indexed, @dbColumnType
    • -
  • field.enum: @column, @db.indexed, @dbColumnType
    • +
  • field.duration: @column, @db.indexed, @dbColumnType
    • +
  • field.enum: @column, @constraintName, @db.indexed, @dbColumnType
  • field.float: @column, @db.indexed, @dbColumnType
  • field.inet: @column, @db.indexed, @dbColumnType
  • field.int: @column, @db.indexed, @dbColumnType
  • field.long: @column, @db.indexed, @dbColumnType
  • field.map: @column, @db.indexed, @dbColumnType
    • -
  • field.object: @column, @db.indexed, @dbColumnType, @storage
    • +
  • field.object: @column, @db.indexed, @dbColumnType, @flattenedColumnPrefix, @storage
  • field.string: @column, @db.indexed, @dbColumnType
  • field.time: @autoSet, @column, @db.indexed, @dbColumnType
  • field.timestamp: @autoSet, @column, @db.indexed, @dbColumnType, @localTime
  • field.uri: @column, @db.indexed, @dbColumnType
  • field.uuid: @column, @db.indexed, @dbColumnType
    • +
  • identity.primary: @sequenceName, @sequenceStart
  • identity.reference: @constraintName, @onDelete, @onUpdate
  • identity.secondary: @expr, @orders, @using, @where
  • index.lookup: @expr, @orders, @using, @where
  • source.rdb: @function, @kind, @materializedView, @parameterRef, @proc, @role, @schema, @sql, @table, @unmanaged, @view
    • +
  • validator.atLeastOne: @constraintName
    • +
  • validator.comparison: @constraintName
    • +
  • validator.exactlyOne: @constraintName
    • +
  • validator.length: @constraintName
    • +
  • validator.numeric: @constraintName
    • +
  • validator.presentIff: @constraintName
    • +
  • validator.regex: @constraintName
    • +
  • validator.requiredWhen: @constraintName

    • metaobjects-documentation

    Universal documentation common attrs (description / title / notes / deprecated / replacedBy / seeAlso / aliases) accepted on every metatype.

    @@ -130,6 +141,7 @@

    metaobjects-prompt

  • field.date: @example, @instruction, @xmlText
  • field.decimal: @example, @instruction, @xmlText
  • field.double: @example, @instruction, @xmlText
    • +
  • field.duration: @example, @instruction, @xmlText
  • field.enum: @coerceDefault, @enumAlias, @enumDoc, @example, @instruction, @normalize, @xmlText
  • field.float: @example, @instruction, @xmlText
  • field.inet: @example, @instruction, @xmlText
    • @@ -154,6 +166,7 @@

    metaobjects-ui

  • field.date: @filterable, @formExclude, @sortable, @sortableDefaultOrder
  • field.decimal: @filterable, @formExclude, @sortable, @sortableDefaultOrder
  • field.double: @filterable, @formExclude, @sortable, @sortableDefaultOrder
    • +
  • field.duration: @filterable, @formExclude, @sortable, @sortableDefaultOrder
  • field.enum: @filterable, @formExclude, @sortable, @sortableDefaultOrder
  • field.float: @filterable, @formExclude, @sortable, @sortableDefaultOrder
  • field.inet: @filterable, @formExclude, @sortable, @sortableDefaultOrder
    • diff --git a/site-reference/types/field.html b/site-reference/types/field.html index 0b53d4a050..e8ff866b9f 100644 --- a/site-reference/types/field.html +++ b/site-reference/types/field.html @@ -987,6 +987,156 @@

    field.double

  • validator.* — 0..*
  • view.* — 0..*
    • +

    field.duration

    +

    A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.

    +

    Owning provider: metaobjects-core-types

    +

    When to use: A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span ("every 3 months") is not a duration: model it as an int and a unit.

    +

    Attributes

    +
    field.durationA fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.types/field.md#fieldduration
    field.enum String-backed enumeration constrained to a closed set of member symbols (@values). Each member is its own stored string with no name/value divergence. types/field.md#fieldenumtypes/validator.md#validatorcomparison
    validator.exactlyOneCardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.types/validator.md#validatorexactlyone
    validator.length Bounds string length / collection size via @min/@max. types/validator.md#validatorlength
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributeTypeRequiredDefaultAllowed valuesProviderDescription
    @columnstringnometaobjects-dbPhysical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the "naming" setting in .metaobjects/config.json).
    @db.indexedbooleannometaobjects-dbWhen true, suppress the @filterable-without-index Loader warning (the field is indexed by other means).
    @dbColumnTypestringnouuid, jsonbmetaobjects-dbPhysical DB column-type override (ADR-0013 escape hatch). Legal values are uuid | jsonb, both on field.string (uuid = native Postgres uuid column over a string-typed field; jsonb = genuinely-open JSON column). The logical field type and its native binding are unchanged. Native SQL arrays (uuid[]/text[]) are NOT declared here — they are derived from a field subtype + isArray (ADR-0036 Wave 1). The retired timestamp_with_tz value is gone — timezone-awareness lives in field.timestamp (instant by default) + @localTime (the naive opt-out), per ADR-0036 Wave 2.
    @defaultanynometaobjects-core-typesDefault value applied to the column when no value is supplied. Its type follows the field's own subtype (string / boolean / number / ...). Converted at consumption time via MetaField.defaultValue().
    @examplestringnometaobjects-promptFR-010: an example value for this field, shown in the generated output-format prompt fragment.
    @filterablebooleannometaobjects-uiWhen true, the field is exposed in generated CRUD filter allowlists (Project D filter layer).
    @formExcludebooleannometaobjects-uiWhen true, the field is omitted from generated forms, and the generated create, PATCH and PUT routes ignore a value sent for it (reads of it are unchanged); inert on fields for which no form is generated (e.g. projection/derived fields).
    @instructionstringnometaobjects-promptFR-010: a short instruction for this field, shown in the generated output-format prompt fragment.
    @mutabilitystringnoreadWrite, writeOnce, readOnlymetaobjects-core-typesFR-037 R1: who may write this field, and when. 'readWrite' (the default when absent) — the caller may set it on create and change it on update. 'writeOnce' — the caller sets it on create; it is excluded from the update shape thereafter, so a value presented on PATCH is ignored rather than rejected. 'readOnly' — nobody writes it: codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas omit it from input variants; the value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. The three are mutually exclusive modes of ONE axis — who may write, and when — so the illegal pair is unrepresentable and inheritance has a total order: a subtype may TIGHTEN an inherited mode (readWrite < writeOnce < readOnly) and never loosen it (ERR_MUTABILITY_DOWNGRADE). Pairing a non-readWrite mode with @autoSet is ERR_MUTABILITY_AUTOSET_CONFLICT: @autoSet already says the SERVER supplies the value, which is a different axis from who may write it.
    @requiredbooleannometaobjects-core-typesWhen true, the field is NOT NULL. Equivalent to attaching a validator.required child. On a non-array string, generated wire-tier input validation (create/patch) additionally rejects the empty string by default — whitespace is accepted — unless an explicit validator.length @min: 0 opts back to presence-only. In-process read models never enforce this at construction.
    @sortablebooleannometaobjects-uiWhen true, the field is exposed in generated CRUD sort allowlists. Inherits from @filterable by default; set false to opt out.
    @sortableDefaultOrderstringnoasc, descmetaobjects-uiDirection a sort takes when it names this field but omits the order: a ?sort= with no :order, and a layout.dataGrid whose @defaultSortField names it with no @defaultSortOrder. Defaults to 'asc'.
    @uniquebooleannometaobjects-core-typesWhen true, the field gets a column-level UNIQUE constraint.
    @xmlTextbooleannometaobjects-promptWhen true, this field receives its element's XML TEXT CONTENT during tolerant extract (JAXB @XmlValue / Jackson @JacksonXmlText / .NET [XmlText]) instead of a same-named child. No effect when the eliciting prompt's @responseFormat is json.
    +

    Allowed children

    +

    field.enum

    String-backed enumeration constrained to a closed set of member symbols (@values). Each member is its own stored string with no name/value divergence.

    Owning provider: metaobjects-core-types

    @@ -1024,6 +1174,15 @@

    field.enum

    Physical column name for this field on an rdb source. Defaults to the field name under the project's naming.columns rule (the "naming" setting in .metaobjects/config.json). +@constraintName +string +no + + +metaobjects-db +The database name of the CHECK constraint this enum's member list generates. Absent: __chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -1114,6 +1273,15 @@

    field.enum

    + + + + + + + + + @@ -2051,6 +2219,15 @@

    field.object

    + + + + + + + + + diff --git a/site-reference/types/identity.html b/site-reference/types/identity.html index 7f0fd6698c..1b2266494c 100644 --- a/site-reference/types/identity.html +++ b/site-reference/types/identity.html @@ -85,6 +85,24 @@

    identity.primary

    + + + + + + + + + + + + + + + + + + + + + + + +
    @db.indexed boolean no Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language.
    @maxLengthintnometaobjects-core-typesColumn length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1.
    @mutability string no When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer).
    @flattenedColumnPrefixstringnometaobjects-dbOn a field.object with @storage "flattened": the text put before each member's column, verbatim ("" allowed, so the members keep their own column names). Absent: this field's resolved column name and "" (with @column "bill", the members are bill). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1.
    @formExclude boolean no metaobjects-core-types Primary-key value generation strategy: 'increment' (auto-increment), 'uuid', or 'assigned' (caller-supplied).
    @sequenceNamestringnometaobjects-dbThe physical name of an @generation "increment" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default, __seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + + + + + + + + +
    @sequenceStartlongnometaobjects-dbThe first value an @generation "increment" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation "increment" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1.

    Allowed children

    No structural children.

    diff --git a/site-reference/types/validator.html b/site-reference/types/validator.html index 50f22a081d..8f9383a3a8 100644 --- a/site-reference/types/validator.html +++ b/site-reference/types/validator.html @@ -107,6 +107,24 @@

    validator.atLeastOne

    @conditionexpressionnometaobjects-core-typesThe case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1.
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -174,6 +192,24 @@

    validator.comparison

    + + + + + + + + + + + + + + + +
    @fields string[] yes
    @conditionexpressionnometaobjects-core-typesThe case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1.
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -203,6 +239,54 @@

    validator.comparison

    @left string yes

    Allowed children

    No structural children.

    +

    validator.exactlyOne

    +

    Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.

    +

    Owning provider: metaobjects-core-types

    +

    Rules: @fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.

    +

    When to use: A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.

    +

    Attributes

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributeTypeRequiredDefaultAllowed valuesProviderDescription
    @conditionexpressionnometaobjects-core-typesThe case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1.
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + + + + + + + + + +
    @fieldsstring[]yesmetaobjects-core-typesNames of the alternative fields; exactly one must be present.
    +

    Allowed children

    +

    No structural children.

    validator.length

    Bounds string length / collection size via @min/@max.

    Owning provider: metaobjects-core-types

    @@ -221,6 +305,15 @@

    validator.length

    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -259,6 +352,15 @@

    validator.numeric

    + + + + + + +
    @max int no
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -298,6 +400,24 @@

    validator.presentIff

    + + + + + + + + + + + + + + + +
    @max int no
    @conditionexpressionnometaobjects-core-typesThe case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1.
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -345,6 +465,15 @@

    validator.regex

    + + + + + + +
    @equals string yes
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + @@ -401,6 +530,24 @@

    validator.requiredWhen

    + + + + + + + + + + + + + + + +
    @max int no
    @conditionexpressionnometaobjects-core-typesThe case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1.
    @constraintNamestringnometaobjects-dbThe database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived _chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1. + + diff --git a/spec/decisions/ADR-0019-runtime-return-type-contract.md b/spec/decisions/ADR-0019-runtime-return-type-contract.md index 5a843dd511..83e5d74b7c 100644 --- a/spec/decisions/ADR-0019-runtime-return-type-contract.md +++ b/spec/decisions/ADR-0019-runtime-return-type-contract.md @@ -194,3 +194,14 @@ spelling (`"12.50"`, not `"12.5"`). A project that wants 1.1.1's output in every `JSON.parse` turns a number into a double before any user code sees it (Amendment 1, "Why a string"). That is a choice the project makes; it is no longer the default. - No vocabulary changes, so `metamodelVersion` does not move. The fix ships in 1.1.2 as a bug fix. + +**Addendum (2026-11-04): `wire.durations`.** `field.duration` (1.3) follows this rule rather than +answering it. `wire.durations` is `"native"` (default) or `"iso"`. `native` writes and reads what +the port's standard serializer does for its duration type: System.Text.Json's `TimeSpan` form in +C# (`"01:30:00"`), `Duration.toString()` / `Duration.parse` in Java and Kotlin as Spring Boot +configures Jackson, Pydantic v2's `timedelta` form in Python (ISO-8601 that folds 24 hours into a +day), and the canonical string in TypeScript, whose in-process type is a `string`. `iso` is the +canonical ISO-8601 form of `fixtures/duration-wire/` in every port, and the api-contract and +persistence corpora pin it, as they pin `decimals: "string"`. A filter operand is an ISO-8601 +duration in both modes, since it is query grammar, not the body serializer. The shared config +cases are in `fixtures/wire-conformance/config-blocks.json`. diff --git a/spec/decisions/ADR-0023-strict-metadata-provenance.md b/spec/decisions/ADR-0023-strict-metadata-provenance.md index 43c8172e3c..bf4b55718b 100644 --- a/spec/decisions/ADR-0023-strict-metadata-provenance.md +++ b/spec/decisions/ADR-0023-strict-metadata-provenance.md @@ -50,3 +50,47 @@ Downstream apps may register their own providers (extending the registry before - **Runtime codegen-read instrumentation** (record every attr a generator queries, fail undeclared). Rejected: strict-load + sealed-registry makes undeclared attrs unreadable by construction, so instrumentation is unnecessary. - **Seal the global singleton as-is** (JVM). Rejected: it would freeze the *polluted* vocabulary; the loader must default to the defined-provider set instead. - **Static attr-name-constant provenance scan.** Useful as a lint but fragile (misses dynamic reads); the structural approach is stronger. + +## Amendment 1 (2026-11-04): the 1.3 schema vocabulary, justified + +Section 3 asks for a written case, per attribute, that its value cannot be computed from the +metadata already there. The 1.3 release adds the vocabulary below. Each one states a fact about a +database or a rule that the model has no other way to know; each derived default stays as it was, +so a model that declares none of them emits what it always did. + +- **`@flattenedColumnPrefix`** (on a `field.object` with `@storage: flattened`). The default is + computed: the field's resolved column and `_`. What cannot be computed is an existing table that + used another prefix, or none at all, and a value object embedded twice on one table with a + different prefix each time. `@column` on the field cannot say it: `""` is not a column name, and + the prefix belongs to the embedding, not to the value object, which other tables embed too. +- **`@sequenceStart` and `@sequenceName`** (on an `@generation: increment` `identity.primary`). + The first value a sequence hands out and its physical name are facts of a database that already + exists: a low id range kept for seeded rows, a sequence that kept its name across a table + rename. Nothing in the model implies either. They are not `@dbColumnType`, which spells a + column's type and is own-only: these are options of the identity's sequence, compared and + altered in place by `meta migrate`, and they keep their meaning when the column type changes. + Postgres only; SQLite and D1 have no sequence, and migrate says so when it creates such a key. +- **`@constraintName`** (on a validator that generates a CHECK, and on a string-stored + `field.enum`). The derived name (`
    @equals string yes
    ___chk`) stays the default. The declared + one is a name an adopter chose: a database whose CHECKs were named by hand, or a program that + compares a violation against the constraint name. A chosen name cannot be derived. +- **`@condition`** (on the cross-field validators: `comparison`, `requiredWhen`, `presentIff`, + `atLeastOne`, `exactlyOne`). A rule that holds only in some case + ("a refund needs a reason") is a guard the author states; no other metadata says when a rule + applies. It uses the existing `attr.expression` grammar, so no new expression language comes + with it. +- **`validator.exactlyOne`** (a subtype, per ADR-0037). "Exactly one of these fields is set" has + its own SQL form (`num_nonnulls(...) = 1`) and its own record-model check, so it is a thing with + its own behaviour, not a variant of `atLeastOne`. The first draft spelled it `atLeastOne` with + `@max: 1`, which overloaded `@max` (numeric everywhere else) and read against the validator's + name; that never shipped. It cannot be computed: nothing else in the model bounds how many of a + set of fields may be set. +- **`@maxLength` on `field.enum`**. The longest member could size the column, but then adding a + longer member would silently become a column type change, and the width an existing database + already uses (a `VARCHAR(32)` sized for growth) is a physical fact the member list does not + carry. The project-wide default (`enums.maxLength`) is configuration, not vocabulary. Absent + both, the column stays unbounded text, as before. +- **`field.duration`** (a subtype, per ADR-0037). A fixed-length span has its own native type in + four ports (`TimeSpan`, `java.time.Duration`, `timedelta`) and in Postgres (`interval`), its own + wire grammar and its own ordering. A `field.string` or a `field.long` holding a count cannot carry + the unit or the native type, so it is a subtype, not an attribute. diff --git a/spec/decisions/ADR-0058-one-physical-naming-setting.md b/spec/decisions/ADR-0058-one-physical-naming-setting.md index fdae02d7fc..822e674752 100644 --- a/spec/decisions/ADR-0058-one-physical-naming-setting.md +++ b/spec/decisions/ADR-0058-one-physical-naming-setting.md @@ -42,7 +42,9 @@ every port already reads for `sources`, `dependencies` and `libraries`: { "schema_version": 1, "naming": { "columns": "snake_case", "tables": "snake_case" } } ``` -- `columns`: `snake_case` | `literal` | `kebab-case`. Applies to a field with no `@column`. +- `columns`: `snake_case` | `literal` | `kebab-case` | `upperFirst`. Applies to a field with no + `@column`. (`upperFirst` was added after the decision: the first letter upper-cased, the rest verbatim. + It is a column rule only, like `kebab-case`.) - `tables`: `snake_case` | `literal`. Applies to a source with no physical-name attribute: its structural `name` is cased by the rule; with no name either, the owning object's name is cased by the rule and then pluralized by the frozen legacy pluralizer, under every rule. diff --git a/spec/metamodel/db.json b/spec/metamodel/db.json index 44c1048d73..4e087e72f0 100644 --- a/spec/metamodel/db.json +++ b/spec/metamodel/db.json @@ -21,6 +21,7 @@ "type": "field", "subType": "object", "children": [ + { "type": "attr", "subType": "string", "name": "flattenedColumnPrefix", "min": 0, "max": 1, "description": "On a field.object with @storage \"flattened\": the text put before each member's column, verbatim (\"\" allowed, so the members keep their own column names). Absent: this field's resolved column name and \"_\" (with @column \"bill\", the members are bill_). A fact about the owning table, so it lives on the embedding field, never on the value object. Refused on any other storage (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE). Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "string", "name": "storage", "min": 0, "max": 1, "allowedValues": ["flattened", "jsonb", "subdocument"], "description": "Storage strategy for an object-typed field (set with @objectRef). \"flattened\" expands the nested value into prefixed columns on the parent table. \"jsonb\" stores the structured value in a single jsonb column (supports isArray=true for arrays of values). \"subdocument\" is a hint for document-store codegen targets and emits no Postgres column." } ] }, @@ -58,6 +59,28 @@ { "type": "attr", "subType": "string", "name": "using", "min": 0, "max": 1, "description": "Index access method (e.g. \"gin\", \"gist\", \"hash\"); default \"btree\" (not rendered). Pair with @expr for e.g. a GIN index over an array/jsonb expression. RDB-physical — contributed by the db provider." } ] }, + { + "type": "field", + "subType": "enum", + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this enum's member list generates. Absent:
    __chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": ["numeric", "length", "regex", "comparison", "requiredWhen", "presentIff", "atLeastOne", "exactlyOne"], + "children": [ + { "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "The database name of the CHECK constraint this validator generates (numeric, length, regex and the cross-field validators; the others generate no CHECK). Absent: the derived
    ___chk. Lets a model adopt a database whose CHECKs were named by hand. RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "identity", + "subType": "primary", + "children": [ + { "type": "attr", "subType": "long", "name": "sequenceStart", "min": 0, "max": 1, "description": "The first value an @generation \"increment\" key's sequence hands out (Postgres: GENERATED BY DEFAULT AS IDENTITY (START WITH n)). Absent: 1. Lets a model reproduce a table that keeps a low id range for seeded rows. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." }, + { "type": "attr", "subType": "string", "name": "sequenceName", "min": 0, "max": 1, "description": "The physical name of an @generation \"increment\" key's sequence (Postgres: GENERATED BY DEFAULT AS IDENTITY (SEQUENCE NAME ...)). Absent: the database's own default,
    __seq on Postgres. Lets a model adopt a sequence that kept its name across a table rename. Refused without @generation \"increment\" (ERR_SEQUENCE_WITHOUT_INCREMENT). RDB-physical — contributed by the db provider. Justified in ADR-0023 Amendment 1." } + ] + }, { "type": "identity", "subType": "reference", diff --git a/spec/metamodel/field.json b/spec/metamodel/field.json index b054bda765..3b4a4ce9fc 100644 --- a/spec/metamodel/field.json +++ b/spec/metamodel/field.json @@ -151,6 +151,7 @@ "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "int", "name": "maxLength", "min": 0, "max": 1, "description": "Column length of a string-stored enum (drives VARCHAR(n)); every member must fit. Overrides the project's enums.maxLength. Refused with @intValueMap, whose column is an integer. Justified in ADR-0023 Amendment 1." }, { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, @@ -183,6 +184,14 @@ "children": [ { "type": "attr", "subType": "boolean", "name": "lenient", "min": 0, "max": 1, "description": "#234: opt this field out of strict well-formedness enforcement. When true, codegen binds a plain string (no URL/IP validator, no native URI/InetAddress type; field.inet uses a text column, not the native inet type) so a not-strictly-valid value round-trips unchanged. Default (absent/false) is strict: an absolute-scheme URI / an IPv4-or-IPv6 literal." } ] + }, + { + "type": "field", + "subType": "duration", + "dataType": "string", + "extendsBase": true, + "description": "A fixed-length span of time: days, hours, minutes and seconds (ADR-0036/0037 — a concept with a native type + behavior). Binds to TS string (TS has no native duration type); other ports bind to their native type (C# System.TimeSpan, Java and Kotlin java.time.Duration, Python datetime.timedelta). DB column is the Postgres-native interval type; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, which orders and compares as the span does. The wire form is an ISO-8601 duration: every port writes [-]PT[nH][nM][n[.f]S] (hours never folded into days, as many fractional digits as its native type holds, zero is PT0S) and reads that form plus leading week and day counts (nW, exactly 7 days; nD, exactly 24 hours) and a fraction of any length, truncated toward zero. Years and months are refused: a calendar span is not a fixed length. Justified in ADR-0023 Amendment 1.", + "whenToUse": "A field holds an elapsed time, a timeout or a delay. Use it for a native Postgres interval column and a native duration type instead of a string or a number of seconds. A calendar span (\"every 3 months\") is not a duration: model it as an int and a unit." } ] } diff --git a/spec/metamodel/validator.json b/spec/metamodel/validator.json index 9520605bd7..4aba88dbd3 100644 --- a/spec/metamodel/validator.json +++ b/spec/metamodel/validator.json @@ -66,7 +66,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "left", "min": 1, "max": 1, "description": "Name of the left-hand field of the owning entity." }, { "type": "attr", "subType": "string", "name": "op", "min": 1, "max": 1, "allowedValues": ["gt", "gte", "lt", "lte", "ne", "eq"], "description": "Relational operator: gt (>), gte (>=), lt (<), lte (<=), ne (<>), eq (=)." }, - { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." } + { "type": "attr", "subType": "string", "name": "right", "min": 1, "max": 1, "description": "Name of the right-hand field of the owning entity." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -78,7 +79,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field that becomes required when the condition holds." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field whose value triggers the requirement." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; when @when equals this, @field must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -90,7 +92,8 @@ "children": [ { "type": "attr", "subType": "string", "name": "field", "min": 1, "max": 1, "description": "Name of the field whose presence is governed by the condition." }, { "type": "attr", "subType": "string", "name": "when", "min": 1, "max": 1, "description": "Name of the gating field." }, - { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." } + { "type": "attr", "subType": "string", "name": "equals", "min": 1, "max": 1, "description": "The gating value; @field is present exactly when @when equals this." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] }, { @@ -100,7 +103,19 @@ "whenToUse": "At least one of several fields must be present (email or phone). Declare it instead of hand-written OR-presence checks.", "rules": "@fields names two or more fields of the owning entity. Satisfied when any one of them is non-null.", "children": [ - { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." } + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the candidate fields; at least one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } + ] + }, + { + "type": "validator", + "subType": "exactlyOne", + "description": "Cardinality of presence: exactly one of the named fields (@fields) is present (NOT NULL), and the others are null. Entity-scoped; references fields by name (same @fields-by-name pattern as identity.*). Justified in ADR-0023 Amendment 1.", + "whenToUse": "A row belongs to exactly one of several alternatives (a store or a kiosk, never both, never neither). Declare it instead of hand-written count-of-non-nulls checks.", + "rules": "@fields names two or more fields of the owning entity (ERR_BAD_ATTR_VALUE): one field would just be a required field. Satisfied when exactly one of them is non-null.", + "children": [ + { "type": "attr", "subType": "string", "name": "fields", "isArray": true, "min": 1, "max": 1, "description": "Names of the alternative fields; exactly one must be present." }, + { "type": "attr", "subType": "expression", "name": "condition", "min": 0, "max": 1, "description": "The case this rule applies in: an expression over the owning entity's fields (the attr.expression grammar). Absent: always. The database CHECK is NOT (condition) OR (rule), so a row the condition does not hold for, or holds NULL for, is not checked. Enforced like the rule itself: by the database CHECK and, on create, by the generated record model in every port. Justified in ADR-0023 Amendment 1." } ] } ] diff --git a/tools/prerelease/prerelease-link.sh b/tools/prerelease/prerelease-link.sh index 9e19558d37..a6e74a61a1 100755 --- a/tools/prerelease/prerelease-link.sh +++ b/tools/prerelease/prerelease-link.sh @@ -176,11 +176,19 @@ ends_in_newline() { # ends_in_newline # it afterward (a plain `&&`/`||` can't survive being stored and read back later). had_newline() { ends_in_newline "$1" && echo 1 || echo 0; } +# In-place extended-regex sed that works on GNU and BSD (macOS) sed alike. Both accept the +# backup suffix only ATTACHED to -i (-i.suffix); a bare `-i -E` makes BSD sed take "-E" as the +# suffix, leaving -E behind and running the expression as a basic regex, whose ( ) groups +# do not capture. Neither sed adds a missing trailing newline here (verified on both). +sed_inplace() { # sed_inplace + sed -E -i.mo-sed-bak "$1" "$2" && rm -f "$2.mo-sed-bak" +} + # Every in-place rewrite in this script writes to a tmp file and moves it over the original. # awk's `print` unconditionally terminates the last line with a newline, even when the file # being rewritten had none — so a no-op "link, then unlink" round trip left every pom.xml that # lacked a trailing newline with one, which `git status` then reported as modified. sed -# does not have this problem (verified: `sed -i` preserves a missing trailing newline whether +# does not have this problem (verified: `sed_inplace` above preserves a missing trailing newline whether # or not anything matched), so only the awk call sites route through this. is # `had_newline` captured on immediately before the awk call that produced ; a # caller that already knows nothing needs preserving can pass "1", which is a no-op here. @@ -443,7 +451,7 @@ repin_py() { # repin_py while IFS= read -r root; do for f in "$root/pyproject.toml" "$root"/requirements*.txt "$root"/constraints*.txt; do [ -f "$f" ] || continue - sed -i -E "s/(${PY_DIST}[[:space:]]*)(==|>=|~=)[[:space:]]*[0-9][^\"',[:space:]]*/\\1==${pv}/g" "$f" + sed_inplace "s/(${PY_DIST}[[:space:]]*)(==|>=|~=)[[:space:]]*[0-9][^\"',[:space:]]*/\\1==${pv}/g" "$f" done done < <( { echo "$PROJECT"; py_roots; } | sort -u) say "repinned $PY_DIST to $pv" @@ -452,7 +460,7 @@ repin_py() { # repin_py repin_nuget() { # repin_nuget local v="$1"; [ -n "$v" ] || return 0 while IFS= read -r f; do - sed -i -E "s|(]*Include=\"${NUGET_PREFIX}[^\"]*\"[^>]*Version=\")[^\"]*(\")|\\1${v}\\2|g" "$f" + sed_inplace "s|(]*Include=\"${NUGET_PREFIX}[^\"]*\"[^>]*Version=\")[^\"]*(\")|\\1${v}\\2|g" "$f" done < <(project_find \( -name '*.csproj' -o -name '*.fsproj' -o -name 'Directory.Packages.props' \) \ -not -path '*/obj/*' -not -path '*/bin/*') say "repinned ${NUGET_PREFIX}* package references to $v" @@ -486,7 +494,7 @@ repin_pom() { # repin_pom if grep -q '' "$pom"; then local before after before="$(cat "$pom")" - sed -i -E "s|()[^<]*()|\\1${mv}\\2|g" "$pom" + sed_inplace "s|()[^<]*()|\\1${mv}\\2|g" "$pom" after="$(cat "$pom")" # Only when the property's value actually changed — a second `link` at the same version # (or a pom already pinned to $mv) must not claim a repin that did not happen. diff --git a/tools/prerelease/test-prerelease-link.sh b/tools/prerelease/test-prerelease-link.sh index 040b5dbfbc..c0578b6cc0 100755 --- a/tools/prerelease/test-prerelease-link.sh +++ b/tools/prerelease/test-prerelease-link.sh @@ -329,6 +329,8 @@ check "a literal dependency version is repinned" 'grep -q "8.1.0-rc.90.61.0" "$poly/ports/kotlin/pom.xml" && grep -q "2.0.21" "$poly/ports/kotlin/pom.xml"' check "every metaobjects version in the pom is repinned, a plugin's after its configuration included" '[ "$(grep -c "8.1.0-rc.9" "$poly/ports/kotlin/pom.xml")" = 3 ] && ! grep -q "8.0.9" "$poly/ports/kotlin/pom.xml"' check "the package reference is repinned" 'grep -q "Version=\"1.1.0-rc.9\"" "$poly/ports/csharp/App.csproj"' +check "the in-place repins leave no sed backup files (BSD sed reads a bare -i -E as suffix -E)" \ + '[ -z "$(find "$poly" \( -name "*-E" -o -name "*.mo-sed-bak" \) -print)" ]' check "the local tool manifest is repinned" 'grep -q "\"metaobjects.cli\": { \"version\": \"1.1.0-rc.9\"" "$poly/.config/dotnet-tools.json"' check "another tool in the manifest is left alone" 'grep -q "\"dotnet-ef\": { \"version\": \"8.0.0\"" "$poly/.config/dotnet-tools.json"' # R3: a second `link` at the same version changes no pom, so it reports no repin, a pom pinned @@ -372,7 +374,7 @@ printf '%s' ' ' > "$mvnproj/app/pom.xml" check "the fixture poms have no trailing newline" \ - '[ "$(tail -c1 "$mvnproj/pom.xml" | wc -l)" = 0 ] && [ "$(tail -c1 "$mvnproj/app/pom.xml" | wc -l)" = 0 ]' + '[ "$(tail -c1 "$mvnproj/pom.xml" | wc -l | tr -d " ")" = 0 ] && [ "$(tail -c1 "$mvnproj/app/pom.xml" | wc -l | tr -d " ")" = 0 ]' git init -q "$mvnproj" git -C "$mvnproj" add -A && git -C "$mvnproj" commit -q -m init cp "$mvnproj/pom.xml" "$WORK/pom.xml.orig"