Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ name: conformance
# gates — fixture-lint, doc-template drift, embedded-library drift, leak scan and
# the other offline repo gates (`--only gates`)
# ts-fast — workspace build + typecheck, TS conformance, completeness (mutation) gate
# (the FULL pass here: MO_CI_MUTATION_SCOPE=full on that job)
# csharp / java-fast (java + kotlin) / python — each port's conformance corpora
# java-reactor — full-reactor `mvn clean install`, tests on (`--only java-slow`)
# The Java lanes point the script's Maven repository at ~/.m2/repository so setup-java's
Expand Down Expand Up @@ -129,6 +130,9 @@ jobs:
- name: Conformance (${{ matrix.lane }})
env:
LANE: ${{ matrix.lane }}
# Tag, nightly and dispatch runs mutate EVERY stryker.conf.mjs file, not only the
# changed ones ts-fast mutates elsewhere (see gate_completeness_diff).
MO_CI_MUTATION_SCOPE: full
run: |
METAOBJECTS_CI_M2_REPO="$HOME/.m2/repository" \
scripts/ci-local.sh --only "$LANE" --no-integration --strict-toolchains
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/local-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,9 @@ jobs:
# gate runs only named conformance FILES, so the rest of those suites ran nowhere
# and a regression could sit red on `main` under a green ts lane.
# Deliberately separate from ts-fast rather than folded into it: ts-fast carries
# the 30-min mutation gate, which flakes under runner contention, and a mutation
# flake must not mask a unit-test failure. The suites total ~5s, so this lane is
# the mutation gate (diff-scoped: on a push to main it mutates nothing new and
# SKIPs; the full pass runs in conformance.yml), which flaked under runner
# contention, and a mutation flake must not mask a unit-test failure. The suites total ~5s, so this lane is
# nearly free and gives an independent, fast signal.
ts-unit:
needs: detect
Expand Down
8 changes: 6 additions & 2 deletions .no-mistakes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,12 @@
# along the script's own `--only` section boundaries so neither step repeats the other.
# `scripts/ci-local.sh --help` enumerates what each section contains, and
# `MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only <section>` prints its exact step list.
# Measured green on a warm tree 2026-09-18: test 256s (it carries the mutation gate and is
# almost all of the cost), lint 23s.
# Measured green on a warm tree 2026-09-18: test 256s, lint 23s. The test step's mutation
# gate is DIFF-SCOPED (gate_completeness_diff in the script): it mutates only the
# stryker.conf.mjs `mutate` files the branch changed against its base, and SKIPs when it
# changed none. Before 2026-11-14 it ran the full pass every round, 10-45 min under load
# and most of this step's wall time. The full pass is `--only mutation`, the flagless
# run's mutation lane, and conformance.yml (tags, nightly, dispatch).
#
# ORDER IS LOAD-BEARING, and it is why the work is split this way round rather than one
# `--quick` in `test`. The pipeline runs test BEFORE lint. The `gates` section contains
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.

**Cross-port conformance corpora** (every port runs the shared corpus):
- Metamodel: `fixtures/conformance/` (421 fixtures; 34 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
- Metamodel: `fixtures/conformance/` (421 fixtures; 36 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.
Expand Down
132 changes: 132 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,138 @@ here.**

## [Unreleased]

### Added

- **A code writer for generators: `CodeWriter` and `TsWriter` in `@metaobjectsdev/codegen-ts`**
(ADR-0063). A generator says what a line contains, and the writer decides where it sits. It
builds the file in memory with no formatter. It collects imports while the body is written,
treats blank lines as requests that collapse, and prints a list on one line when the line fits
120 columns, otherwise one item per line. `TsWriter` accepts ts-poet `Code` fragments and
collects their `imp()` imports, so a generator can move to it one function at a time. The
writer is exported from the package root and from `@metaobjectsdev/codegen-ts/engine`, so an
owned (ejected) generator can use it too. Both classes are built to be extended: every
method is public or protected, so a subclass can override one step, such as import order,
and keep the rest. ADR-0063 also rules that reference generators will be extended the same
way, one overridable method per piece of output, without promising those methods are stable
between minor versions. The contract is `spec/code-writer.md`, pinned by
the new `fixtures/writer-conformance/` corpus. No shipped generator uses the writer yet, so
generated output does not change in this release.

### Changed

- **`gen` skips rendering when nothing it is built from changed, in every port (ADR-0062).**
The skip key is computed after the metadata loads and before anything renders. Its model half
hashes each loaded root-level node in its canonical effective form (`extends` and overlays
applied), rolled up per package and for the model, so reformatting or splitting a metadata file
changes nothing while an edit to an inherited parent changes every child. Its generator half
hashes the built engine code, the codegen config and every owned generator, the template
roots, declared extra inputs, the project config and the output-changing flags. `gen` skips
when the key matches the last successful run and every generated file still has the bytes
that run left; a skipped run renders and writes nothing. The key is stored, uncommitted, in
`.metaobjects/.gen-state/.fingerprint-<port>.json`. `--force` (`meta gen`, `metaobjects gen`,
`dotnet meta gen`; `-Dmetaobjects.force=true` on Maven) renders everything, and a generator
that cannot declare what it reads opts out with `cache: false` (or its port's equivalent).
- **`verify --codegen` may answer locally without rendering, and never writes a temp tree.** In
CI (`CI` set) it always renders and compares everything. Locally it answers "no drift" when the
key matches and every generated file still hashes to its record; `--full`
(`-Dmetaobjects.verify.full=true` on Maven) renders. When it renders, every port compares the
rendered bytes with disk in memory.
- **The hash manifest is written once per run, and only when an entry changed** (TypeScript,
Python and C#, matching Java and Kotlin). A file whose fresh bytes equal what is on disk is never
rewritten in any port, so its timestamp stays. Python `gen` no longer reports unchanged files
as written.
- **TypeScript runs `git merge-file` only for a file edited by hand since it was generated**, and
deletes the merge's temporary files in every case, under `$DEV_SCRATCH` when set. An untouched
file is overwritten directly.

## [1.2.2] — 2026-11-13

_npm `1.2.2` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates)._

_Proposed as npm `1.2.2` (all 14 `@metaobjectsdev/*` packages), NuGet `1.2.2` and Maven Central
`8.2.2`; `1.2.1` was spent by the Python-only release. PyPI stays at `1.2.1`: no Python product
file changes. No vocabulary or `metamodelVersion` change._

### Security

- **Generated create routes in Java, Kotlin and C#, and TypeScript updates, let a client write
fields it must never set.** The same class of hole the Python 1.2.1 patch closed, found by an
audit of the other four ports after it. Every adopter serving these generated routes should
upgrade and regenerate; an ejected (owned) generator needs the same change made by hand.

- **Java (`codegen-spring`).** The create route passed the bound DTO to the repository as
sent, so a client `id` on a generated key and a `@mutability: readOnly` or derived field
reached it. On a TPH per-subtype route the body's discriminator and a sibling subtype's
columns did too. The route now rebuilds the DTO with each of those components set to null
before the repository call; the route's subtype still comes from the URL. A `@required`
`readOnly` or derived value-object column no longer fails create validation once nulled.
- **Kotlin (`codegen-kotlin`).** The create route wrote `readOnly` and derived columns from the
body, and a TPH per-subtype route wrote a sibling subtype's columns. It now writes only the
route subtype's own settable columns.
- **C# (`MetaObjects.Codegen`).** The create route bound the EF Core entity from the body, so a
client `id` on a generated key was inserted, a TPH create stored the body's discriminator (a
POST to the Bridge route could create a Copay row), and a TPH subtype's `readOnly` field and
an M:N navigation graph were written. The route now resets the generated key and the
discriminator to their defaults and empties the M:N navigations before `Add`, and a
`readOnly` property of every shape, a TPH subtype's included, has a private setter, which
System.Text.Json does not bind. A `@required` `readOnly` field is therefore neither demanded
of the create body nor held to its `[Required]` there, so such an entity stays creatable.
- **TypeScript (`codegen-ts`).** The `<Entity>UpdateSchema` kept an assigned (non-generated)
primary key, so a PUT or PATCH could re-key the row. The update schema now leaves out every
primary-key field; the insert schema still takes an assigned key, which is the caller's on
create.

Each of these values is dropped silently, never answered with a 400. A key the model does not
declare is dropped silently in TypeScript, C# and Python. In Java and Kotlin the answer to an
undeclared key is the host's Jackson `ObjectMapper`'s: Spring Boot's default ignores it, and a
plain `ObjectMapper` (`FAIL_ON_UNKNOWN_PROPERTIES` on) answers 400. This patch leaves that as
it was.

Gated by the shared api-contract scenarios `input-server-owned-dropped`,
`tph-input-server-owned-dropped` and `tph-input-foreign-discriminator-dropped`, and a `readOnly`
`role` on the `secret-fields` corpus, in every port's generated lane and in the reference lanes.
Python's generated TPH routes answer a foreign body discriminator with a 400 rather than
dropping it (nothing is stored); its runner skips that one scenario, recorded in the Python
codegen `KNOWN_GAPS.md`.

### Fixed

- **C#: concurrent reads of a loaded model no longer corrupt its cache.** `MetaData.Cached`
memoized derived reads (`Validators()`, `ResolvedIsArray()`, `OwnAttrs()`, …) in a plain
`Dictionary`, so two threads touching a frozen node for the first time could corrupt it, and
every later read in the process then failed with "Operations that change non-concurrent
collections must have exclusive access". A host that loads metadata once and serves requests
from it in parallel, or a parallel test run, hit this. The cache is now a
`ConcurrentDictionary`.

## [1.2.1] — Python only

_PyPI `1.2.1` only. npm, NuGet and Maven Central stay at `1.2.0` (`8.2.0`): none of them has a
product change. No vocabulary or `metamodelVersion` change._

### Security

- **The generated Python routes passed raw request input to the repository.** The FastAPI
create, PATCH and PUT handlers validated the body against `<Entity>Create` / `<Entity>Patch`
and then handed the repository the raw body, not what was validated. Pydantic ignores a key a
model does not declare, so a client could write fields it must never touch: a client-sent `id`
on create (or on update, changing the row's identity), a `@mutability: readOnly` field, and any
made-up field such as `isAdmin` all reached the repository. Found by an external review of
1.2.0. Every Python adopter serving the generated routes should upgrade and regenerate; an
ejected (owned) router needs the same change made by hand.

The repository now receives only the validated input, as the TypeScript routes already do.
Each handler validates the body, then keeps only the keys the input model validated, at every
depth (a nested value object included), before any repository call, in the plain and the TPH
per-subtype handlers. A field the entity does not declare, a server-generated id, a `readOnly`
or derived field on create, and the id, `readOnly` and `writeOnce` fields on update are all
dropped silently. None of them is answered with a 400, so a client that sends an extra harmless
field keeps working; a malformed value on a field the client may set is still
`400 {"error": "validation"}`. An assigned key is still the caller's on create. Values are the
body's own, so a repository receives the same value types as in 1.2.0.

Gated by `server/python/tests/integration/test_generated_router_validated_input.py`.

## [1.2.0] — 2026-10-30

_npm `1.2.0` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates)._
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ dotnet meta gen metaobjects --out Generated --namespace Acme.Generated
dotnet meta gen --list # list all registered generators
dotnet meta gen metaobjects --out Generated --generators entity # narrow to declared entries

dotnet meta verify --codegen # regenerate to temp, diff vs committed output
dotnet meta verify --codegen # regenerate in memory, diff vs committed output
dotnet meta verify --templates # check {{field}} refs vs payload VO (bare default)
```

Expand Down
2 changes: 1 addition & 1 deletion agent-context/skills/metaobjects-audit/references/java.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ goal bound to `generate-sources` in `pom.xml`.
```bash
mvn metaobjects:generate # codegen goal directly
mvn compile # also runs it (bound to generate-sources)
mvn metaobjects:verify -Dmeta.verify.mode=codegen # regen to temp, diff vs committed output
mvn metaobjects:verify -Dmeta.verify.mode=codegen # regen in memory, diff vs committed output
mvn metaobjects:verify -Dmeta.verify.mode=templates # {{field}} refs vs payload VO
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ committed — confirm the plugin is wired and the goal bound to `generate-source
```bash
mvn metaobjects:generate # codegen goal directly
mvn compile # also runs it (bound to generate-sources)
mvn metaobjects:verify -Dmeta.verify.mode=codegen # regen to temp, diff vs committed output
mvn metaobjects:verify -Dmeta.verify.mode=codegen # regen in memory, diff vs committed output
mvn metaobjects:verify -Dmeta.verify.mode=templates # {{field}} refs vs payload VO
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ directory structure beyond the target's `outDir`.
```bash
metaobjects gen # run every target's generators (metaobjects.config.yaml)
metaobjects gen --generators entity # narrow to declared entries
metaobjects verify --codegen # regenerate to temp, diff vs each committed outDir (bare default)
metaobjects verify --codegen # regenerate in memory, diff vs each committed outDir (bare default)
metaobjects verify --templates # {{field}} refs vs payload VO
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ the default single-target config, generated files land at the one configured
meta gen # render → 3-way merge → write (overwrites @generated files only)
meta gen --dry-run # preview without writing
meta gen Author Post # scope to named entities
meta verify --codegen # regenerate to temp, diff vs committed output
meta verify --codegen # regenerate in memory, diff vs committed output
meta verify --templates # check {{field}} refs vs payload VO (a bare `meta verify` runs this and `--codegen`)
meta verify --db # schema drift — live DB vs metadata (requires DB connection)
meta verify --codegen --templates --db # all three drift checks in one pass
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ the config's `templates` is refused.
```bash
metaobjects gen # runs every target's generators into its outDir
metaobjects gen --generators entity # narrow to declared entries (never adds one)
metaobjects verify --codegen # regenerate the same list into a temp dir + diff
metaobjects verify --codegen # regenerate the same list in memory + diff
```

With no `metaobjects.config.yaml`, `gen` and `verify --codegen` exit 2 and write nothing.
Expand Down
Loading
Loading