Skip to content

Commit f8a1dc5

Browse files
authored
feat: bring every port to parity with the 1.3.0 codegen and migration work (#567)
* feat: bring every port to parity with the rc19 work Codegen and migrations, across the five ports: - @sequenceStart is honoured forward-only: a changed start sets START WITH and moves the counter to GREATEST(n, max(id)+1) only when it is below, never back (TypeScript migrate, the Drizzle identity column, C# design-time services, Kotlin Exposed). Java and Python write no DDL. - Output paths keyed by metadata package: Python's package layout, C#'s opt-in csharp.outputLayout / csharp.packageNamespaces, and a Java/Kotlin refusal when two metadata packages map to one directory. meta init, metaobjects init and dotnet meta init scaffold the package layout; engine defaults stay flat. - A value object is never renamed. One claim registry per run in TypeScript, C# and Python refuses a real clash with ERR_NAME_COLLISION; a collection-name clash is ERR_COLLECTION_NAME_COLLISION in every port. - @collectionName and the naming.collections pluralizer drive REST segments in Java, Kotlin and Python too. - TypeScript view lowering: every join LEFT OUTER, @schema-only qualification, declared-type casts, multi-operator origin filters, projection-extends- projection, filter paths, multi-hop origin.first, TPH scope, and the view vocabulary (entity-derived fields, origin @segment, base-row operands, relative dates, case). - C#: the EF context declares meta migrate's CHECKs (new check-constraint corpus), JSON-column comments, entity-derived fields are no column, and framework references are global::-qualified under package namespaces. - Gates run in both output layouts; ADR-0015, ADR-0044 and ADR-0056 amended. * fix: close the pre-gate review findings C# design-time HasDifferences counts a JSON-column comment change; the framework qualifier leaves global using and preprocessor lines alone. TypeScript and Python refuse two shared enums with one name instead of dropping one; Python refuses a package tree it cannot import. An inlined origin.first with no outer alias is refused, short-name counts are cached per root, and the init scaffold says a barrel still claims every name. * refactor(csharp): reuse the companion and global:: helpers in CSharpNaming * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): read NeutralConfig once per gen/verify invocation * no-mistakes(review): Restore C# duplicate physical-address guard * no-mistakes(review): Restore obsolete C# ValueObjectNames.Names shim * no-mistakes(review): Advance new identity sequence past max(id) * no-mistakes(review): merge CHANGELOG headings; fix using-statement lexer * no-mistakes(review): Verify round-7 fixes; no further changes needed * no-mistakes(review): apply reviewer inline fixes * no-mistakes(test): apply changes made during test validation * no-mistakes(document): Docs: LEFT OUTER joins, layout and naming parity * no-mistakes(document): Docs: claim-registry guidance in generator recipe
1 parent b061181 commit f8a1dc5

327 files changed

Lines changed: 11802 additions & 3137 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
7171
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.
7272

7373
**Cross-port conformance corpora** (every port runs the shared corpus):
74-
- Metamodel: `fixtures/conformance/` (510 fixtures; 38 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
74+
- Metamodel: `fixtures/conformance/` (510 fixtures; 39 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
7575
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
7676
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
7777
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.

‎CHANGELOG.md‎

Lines changed: 193 additions & 45 deletions
Large diffs are not rendered by default.

‎agent-context/skills/metaobjects-codegen/references/typescript-document-store.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ What the repository does, built on the generated contract:
5555
// its wire contract (<E>InsertSchema, <E>UpdateSchema, <E>FilterAllowlist); this emits the
5656
// data access over it. The collection name comes from an author-supplied `@mongo` property
5757
// bag on the entity: `"@mongo": { "collection": "orders" }`.
58-
import { isSourcelessEntity, perEntity, type Generator } from "@metaobjectsdev/codegen-ts";
58+
import { effectivePackage, entityOutputPath, isSourcelessEntity, perEntity, type Generator } from "@metaobjectsdev/codegen-ts";
5959
import type { MetaObject } from "@metaobjectsdev/metadata";
6060

6161
function collectionOf(obj: MetaObject): string {
@@ -155,7 +155,11 @@ export function mongoRepositoryGenerator(): Generator {
155155
return {
156156
name: "mongo-repository",
157157
filter: isSourcelessEntity,
158-
generate: perEntity((obj) => ({ path: `${obj.name}.repository.ts`, content: render(obj) })),
158+
// Beside the entity file it imports (`./<Entity>.js`), in either output layout.
159+
generate: perEntity((obj, ctx) => ({
160+
path: entityOutputPath(ctx.renderContext?.outputLayout ?? "flat", effectivePackage(obj), `${obj.name}.repository.ts`),
161+
content: render(obj),
162+
})),
159163
};
160164
}
161165
```

‎docs/CONFORMANCE.md‎

Lines changed: 33 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Conformance coverage
22

3-
The MetaObjects standard ships **38 shared conformance corpora** under
3+
The MetaObjects standard ships **39 shared conformance corpora** under
44
[`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to
55
it* and asserts the same expected behaviour against the same fixtures. **This page
66
is the inverse index**: fixture → feature doc + per-port pass status, and it is the
@@ -57,9 +57,10 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
5757
| [`fixtures/field-lint-conformance/`](../fixtures/field-lint-conformance/) (the `verify` field authoring lint) | 15 | ✓ (reference) | ✓ | inherits via Java | ✓ | ✓ |
5858
| [`fixtures/requirement-check-conformance/`](../fixtures/requirement-check-conformance/) (the `verify` requirement gate, ADR-0057) | 43 cases | ✓ (reference) | ✓ | inherits via Java (one Maven `metaobjects:verify` goal) | ✓ | ✓ |
5959
| [`fixtures/requirement-test-identity-conformance/`](../fixtures/requirement-test-identity-conformance/) (the identity, skip state and digest of each generated requirement test, and the filter seam, ADR-0057) | 26 cases | ✓ (reference) | ✓ | identity function inherits via Java; emitted names asserted by its own generator test | ✓ | ✓ |
60-
| [`fixtures/naming-conformance/`](../fixtures/naming-conformance/) | 8 cases | ✓ | ✓ | inherits via Java (`RouteNaming.pluralize`) | ✓ | ✓ |
60+
| [`fixtures/naming-conformance/`](../fixtures/naming-conformance/) | 8 cases + 23 collection segments (`@collectionName`, `naming.collections`) | ✓ | ✓ | inherits via Java (`RouteNaming.pluralize`, `RouteNaming.collectionSegment`) | ✓ | ✓ |
6161
| [`fixtures/wire-conformance/`](../fixtures/wire-conformance/) (the `wire` setting, ADR-0019 Amendment 2) | 6 cases + 8 refusals | ✓ | ✓ | inherits via Java (one config reader) | ✓ | ✓ |
6262
| [`fixtures/enum-length-conformance/`](../fixtures/enum-length-conformance/) (a string-stored enum's column length: `@maxLength`, else `enums.maxLength`) | 5 cases + 1 overflow + 5 refusals; run by a port that sizes the column | ✓ | ✓ | inherits via Java (one config reader, one resolver) | ✓ | — (sizes no column: emits no DDL or column type; the config reader accepts and shape-checks the `enums` block) |
63+
| [`fixtures/check-constraint-conformance/`](../fixtures/check-constraint-conformance/) (the CHECK constraints `meta migrate` derives: name and expression text per dialect) | 1 model, 5 expected files (postgres, sqlite, d1, and both skip-redundant variants) | ✓ (reference, `migrate-ts`) | — (no CHECK in a generated model) | — (same) | ✓ (the EF context declares them; live-Postgres `EnsureCreated` lane) | — (same) |
6364
| [`fixtures/codegen-noop/`](../fixtures/codegen-noop/) (FR-044 — reporting vocabulary: what is lowered, what stays inert) | 1 model pair (`reporting/with` vs `reporting/without`) | ✓ (codegen + migrate) | ✓ | ✓ | ✓ | ✓ |
6465
| [`fixtures/cube-model/`](../fixtures/cube-model/) (FR-044 Plan 4 — the `cube-model` generator's golden Cube files, plus the canonical model's golden) | 46 cases (36 trees + 10 errors) + 1 canonical golden | ✓ (the exporter is TypeScript-only, spec R6; its `cube` lane also loads the files into a real Cube) | — | — | — | — |
6566
| [`fixtures/contract-conformance/`](../fixtures/contract-conformance/) (FR-022 — the `contract-schema` and `openapi` generators' golden trees and refusals) | 19 cases (14 trees + 5 errors) | ✓ (one neutral engine, ADR-0034 Amendment 10; the generated API lanes also hold real traffic to the documents) | — | — | — | — |
@@ -111,27 +112,29 @@ corpora, or between its *vocabulary* and its *behaviour* — never gets a row of
111112
own, and its boundary ends up discoverable only from the feature doc. The split ones
112113
are recorded here instead.
113114

114-
**`identity.primary @sequenceStart`** (one semantic in every port): on a fresh
115+
**`identity.primary @sequenceStart`** (one semantic in every port that writes DDL): on a fresh
115116
database the identity or sequence begins at `@sequenceStart`; when the start changes on an
116-
existing database, the change sets `START WITH` only. It never `RESTART`s, and in 1.3 it does
117-
not move the current value, because live rows already hold the ids it has handed out (a later
118-
release is planned to move a counter below the start forward, never backward). The port TODOs below
119-
are tracked in [#544](https://github.com/metaobjectsdev/metaobjects/issues/544).
120-
117+
existing database, the change sets `START WITH n` and moves the counter FORWARD only, to
118+
`GREATEST(n, max(id) + 1)` when it sits below that. It never `RESTART`s and never moves a counter
119+
back, because live rows already hold the ids it handed out. Tracked in
120+
[#544](https://github.com/metaobjectsdev/metaobjects/issues/544).
121+
122+
- *Honoured and pinned:* TypeScript (`meta migrate`). A fresh table is created `START WITH n`, and
123+
the Drizzle schema template carries the start. A changed start writes `ALTER SEQUENCE … START
124+
WITH n` plus an advance-only `setval`. The persistence migration scenarios
125+
`sequence-start-change-moves-counter-forward` and `sequence-start-change-below-max-id` pin on
126+
live Postgres that the next id is the start, or the highest id plus one when a row already
127+
holds an id past the start.
121128
- *Honoured and pinned:* C#. The generated context carries the start on Npgsql, and the generated
122-
rewrite (`UseIdentityStartMigrations()`) writes every start-only change — added, changed or
123-
removed — as `ALTER TABLE … ALTER COLUMN … SET START WITH n`, whether the migration is scaffolded
124-
through it or its SQL is generated through it (Npgsql alone writes `RESTART WITH n`, or
125-
`RESTART WITH 1` for a removed start). `EfModelGapsTests` pins all three transitions on both
126-
paths, and on live Postgres that the next id after the change follows the last row.
127-
- *Honoured and pinned:* TypeScript (`meta migrate`). A fresh table is created `START WITH n`;
128-
a changed start writes `ALTER SEQUENCE … START WITH n` only. The persistence migration
129-
scenario `sequence-start-change-keeps-counter` pins on live Postgres that the next id after
130-
the change follows the last row.
131-
- *TODO, Java (OMDB) and Kotlin (Exposed):* the vocabulary loads; neither emits or honours the
132-
start in its DDL or mapping. Add it with the same semantic and the same pin.
133-
- *TODO, Python:* the vocabulary loads; the runtime has no DDL path. If it gains one, the same
134-
semantic and pin apply.
129+
`<Context>DesignTimeServices` makes `dotnet ef migrations add` scaffold a start-only change as
130+
`SET START WITH n` plus the same advance-only move (Npgsql alone writes `RESTART WITH n`).
131+
`EfModelGapsTests` pins the transitions, and on live Postgres the next id after the change.
132+
- *Honoured and pinned:* Kotlin (Exposed 0.x and 1.x). A declared start or name generates
133+
`autoIncrement(Sequence("<name>", startWith = n))`; a live-Postgres test pins that a fresh
134+
table's first id is the start. Exposed generated code has no migration path, so a changed start
135+
is `meta migrate`'s job.
136+
- *Not applicable:* Java (OMDB is pure data access with no DDL, ADR-0015) and Python (no DDL
137+
path). The vocabulary loads in both; schema comes from `meta migrate`.
135138

136139
**Cross-port vocabulary implemented in some ports so far (`implementedIn`).** Each entry
137140
below is in the shared registry and the registry manifest, marked `implementedIn`. A port that
@@ -148,12 +151,6 @@ canonical (`fixtures/registry-conformance/expected-registry.json`).
148151
- `view.image`, and its `@accept`, `@aspectRatio`, `@maxBytes`, `@maxEdge`, `@store`
149152
- *Implemented in Java only* (`implementedIn: ["java"]`): `object.* @object` (ADR-0001), the
150153
native class an object binds to. Java's runtime acts on it; Java's own behaviour is unchanged.
151-
- *Implemented in C# and TypeScript* (`implementedIn: ["csharp", "typescript"]`): `object.*
152-
@collectionName`, an object's collection name: the REST collection segment and reverse-finder
153-
names in both, the DbSet in C#, the list function, list hook and Drizzle table variable in
154-
TypeScript. Both apply the `naming.collections` pluralizer to an object that declares none
155-
(`fixtures/naming-conformance/collection-segments.json`); every port registers and type-checks
156-
the attr and the key.
157154
- *Loads strictly in every port.* Each port registers the `metaobjects-ui-web` provider
158155
(`view.textarea @rows`, the `view.image` attrs) and `@object` / `@objectAdapter` on
159156
`object.*`, loads all three fixtures strictly, and emits `implementedIn` in its manifest, which
@@ -286,6 +283,10 @@ manifest contains without moving these boundaries.
286283
classpath and stubbing them drowns the signal; that tier is compiled and booted over
287284
real HTTP in the api-contract integration lane instead. The boundary is one
288285
cross-port rule, not four local concessions.
286+
- *The gate and the generated-server api-contract lanes run in both output layouts* —
287+
the engine default `flat` and `outputLayout: "package"` (which `meta init` writes) —
288+
so a defect only the non-default layout shows (a relative import one directory short,
289+
two packages landing in one directory) fails a gate rather than an adopter's build.
289290
- *The prompt tier is out of scope here* because the fitness corpus declares no
290291
`template.*` nodes. Including those generators would emit nothing and read as
291292
coverage that is not there.
@@ -610,7 +611,11 @@ only `KotlinSharedModelGeneratorTest`. **C# runs all 15**, in `SharedModelConfor
610611
> recorded here rather than in a ledger, because ledgering a fixture the design intends to pass
611612
> is how a port quietly stays behind.
612613
613-
### `fixtures/naming-conformance/` (8 cases)
614+
### `fixtures/naming-conformance/` (8 cases + 23 collection segments)
615+
616+
`collection-segments.json` (23 cases) pins the REST collection segment from an object's
617+
`@collectionName` and the project's `naming.collections` pluralizer; every port runs it (see the
618+
corpus README). The pluralize cases:
614619

615620
All 8 cases → the already-plural pluralize fix (CHANGELOG `## [Unreleased]`):
616621
an entity whose name is already a plural noun (`Stats`, `Settings`, `Series`)

‎docs/features/api-contract.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,10 @@ every field. The TS mounts can restrict the update to one verb with
9191

9292
`<entity>` is the **entity name** `snake_case`d and then pluralized. One rule, the
9393
same in all five ports, and derived from the NAME — never from the physical
94-
`@table`:
94+
`@table`. An object that declares `@collectionName` (or a project that configures
95+
`naming.collections`) names its segment instead, in every port; the rule and what
96+
each port names with it live in
97+
[physical-naming.md](migrations/physical-naming.md#collection-names).
9598

9699
| Name | Segment | Why |
97100
|---|---|---|

‎docs/features/cube-export.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -568,7 +568,7 @@ there.
568568

569569
| Difference | Why | Effect |
570570
|---|---|---|
571-
| Joins | Cube compiles every join to a `LEFT JOIN`. The view joins a required belongs-to reference `INNER`. | A fact row whose reference matches no row forms a null group in Cube and is dropped by the view. A foreign key constraint on the column makes such a row impossible. |
571+
| Joins | Cube compiles every join to a `LEFT JOIN`. The view's joins are `LEFT OUTER` too, everywhere (they inner-joined a required reference before). | A fact row whose reference matches no row forms a null group in Cube and, through the outer join, in the view as well. A foreign key constraint on the column makes such a row impossible on the conformance data. |
572572
| Query time zone | A Cube query may pass `timezone`. The view is UTC only. | With a zone other than `UTC`, Cube re-buckets instants, which is its feature, and also shifts naive timestamps and dates, which the view buckets as stored. |
573573
| Grains | Cube offers every granularity on a time dimension. | `@grains` is carried as `meta.grains`, not enforced. |
574574
| Ad-hoc queries | Cube lets a caller pick any members. | Nothing here checks the numbers for a combination no report declares. |

‎docs/features/field-types.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,6 +301,16 @@ Two rules the loader/codegen enforce:
301301
abstract enum is inline as well. Declaring the vocabulary at package level is how
302302
you ask for one type.
303303

304+
**Two package-level shared enums with one name are refused.** TypeScript and Python
305+
generate every shared enum into one shared module under its own name, so a second
306+
`Tier` in another package would keep the first and silently type its fields with the
307+
first's members; the pair is a codegen error, `ERR_NAME_COLLISION`, naming both
308+
declarations. Place them apart: the package output layout does not split the shared
309+
module (rename one). C# puts each in its package's own namespace when
310+
`csharp.packageNamespaces` is on; without it both would declare one type in one
311+
namespace, and the pair is refused the same way. Java and Kotlin write each in its
312+
package's own directory, so the pair never meets.
313+
304314
## Embedded value objects — `field.object` + `@storage`
305315

306316
`field.object` declares an embedded structured value backed by another `object`

‎docs/features/migrations-and-drift.md‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -305,12 +305,12 @@ existing column with the same options), and `meta verify --db` reads the live se
305305
from the catalog. A live sequence whose start or name differs from the declaration is altered
306306
in place, never dropped:
307307

308-
- a different start records it (`ALTER SEQUENCE … START WITH n`) and nothing else. It never
309-
restarts the sequence, and in this release it does not move its current value, because live
310-
rows already hold the ids it handed out: `@sequenceStart` is the first value of a fresh
311-
sequence, the same in every port. The down migration restores the old start value, again
312-
without touching the counter. A later release is planned to move a counter that sits below
313-
the declared start forward to it (never backward), so do not rely on the counter staying put.
308+
- a different start records it (`ALTER SEQUENCE … START WITH n`) and moves the counter forward
309+
only: when the next value would be below `GREATEST(n, max(id) + 1)`, a `setval` moves it there.
310+
It never restarts the sequence and never moves a counter back, because live rows already hold
311+
the ids it handed out. The down migration restores the old start value and leaves the counter
312+
where it is. Every port that writes DDL holds the same rule (C# through its generated design-time
313+
services, Kotlin's Exposed table on a fresh database).
314314
- a different name renames it (`ALTER SEQUENCE … RENAME TO …`); the down renames it back.
315315

316316
Only what the model declares is compared, on every path (a live database, a shadow database,
@@ -415,7 +415,10 @@ the re-diff is empty afterwards. Postgres keeps `IS [NOT] DISTINCT FROM`.
415415

416416
### CHECKs from validators: names, conditions, how many
417417

418-
A `validator.*` node and a `field.enum`'s member list each become a CHECK constraint. Three
418+
A `validator.*` node and a `field.enum`'s member list each become a CHECK constraint. A validator
419+
on a derived field — one carrying an `origin.*`, or a cross-field validator naming one — makes
420+
none: the field is no column of the table, so a CHECK on it would fail `CREATE TABLE`; the C#
421+
context skips the same validators. Three
419422
attributes shape it, all database-only (no generated code reads them):
420423

421424
- **`@constraintName`** names the CHECK, for a database whose CHECKs were named by hand. On the

0 commit comments

Comments
 (0)