Skip to content

Commit d199122

Browse files
committed
Merge remote-tracking branch 'origin/main' into fm/mo-codegen-loader-selection
# Conflicts: # CHANGELOG.md # server/python/src/metaobjects/cli.py
2 parents 041c103 + 88a5520 commit d199122

78 files changed

Lines changed: 1943 additions & 312 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/conformance.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ name: conformance
1818
# gates — fixture-lint, doc-template drift, embedded-library drift, leak scan and
1919
# the other offline repo gates (`--only gates`)
2020
# ts-fast — workspace build + typecheck, TS conformance, completeness (mutation) gate
21+
# (the FULL pass here: MO_CI_MUTATION_SCOPE=full on that job)
2122
# csharp / java-fast (java + kotlin) / python — each port's conformance corpora
2223
# java-reactor — full-reactor `mvn clean install`, tests on (`--only java-slow`)
2324
# The Java lanes point the script's Maven repository at ~/.m2/repository so setup-java's
@@ -129,6 +130,9 @@ jobs:
129130
- name: Conformance (${{ matrix.lane }})
130131
env:
131132
LANE: ${{ matrix.lane }}
133+
# Tag, nightly and dispatch runs mutate EVERY stryker.conf.mjs file, not only the
134+
# changed ones ts-fast mutates elsewhere (see gate_completeness_diff).
135+
MO_CI_MUTATION_SCOPE: full
132136
run: |
133137
METAOBJECTS_CI_M2_REPO="$HOME/.m2/repository" \
134138
scripts/ci-local.sh --only "$LANE" --no-integration --strict-toolchains

‎.github/workflows/integration-tests.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,10 @@ jobs:
6666
POSTGRES_DB: metaobjects_test
6767
ports:
6868
- 5432:5432
69+
# The data directory (the image's VOLUME) on a size-capped tmpfs: a throwaway database
70+
# never touches the runner's drives.
6971
options: >-
72+
--tmpfs /var/lib/postgresql/data:rw,size=2g
7073
--health-cmd "pg_isready -U metaobjects -d metaobjects_test"
7174
--health-interval 10s
7275
--health-timeout 5s
@@ -75,6 +78,16 @@ jobs:
7578
- uses: actions/checkout@v4
7679
with:
7780
persist-credentials: false
81+
- name: Postgres durability off (a throwaway database)
82+
# A service container takes no server flags, so set them on the running server. Its data
83+
# directory is already a tmpfs (options above); nothing in it needs to survive a crash.
84+
run: >-
85+
docker exec ${{ job.services.postgres.id }}
86+
psql -U metaobjects -d metaobjects_test
87+
-c "ALTER SYSTEM SET fsync = off"
88+
-c "ALTER SYSTEM SET synchronous_commit = off"
89+
-c "ALTER SYSTEM SET full_page_writes = off"
90+
-c "SELECT pg_reload_conf()"
7891
7992
- name: Set up Bun (TS lanes)
8093
if: matrix.lane == 'ts-slow' || matrix.lane == 'cube'

‎.github/workflows/local-ci.yml‎

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,8 +116,9 @@ jobs:
116116
# gate runs only named conformance FILES, so the rest of those suites ran nowhere
117117
# and a regression could sit red on `main` under a green ts lane.
118118
# Deliberately separate from ts-fast rather than folded into it: ts-fast carries
119-
# the 30-min mutation gate, which flakes under runner contention, and a mutation
120-
# flake must not mask a unit-test failure. The suites total ~5s, so this lane is
119+
# the mutation gate (diff-scoped: on a push to main it mutates nothing new and
120+
# SKIPs; the full pass runs in conformance.yml), which flaked under runner
121+
# contention, and a mutation flake must not mask a unit-test failure. The suites total ~5s, so this lane is
121122
# nearly free and gives an independent, fast signal.
122123
ts-unit:
123124
needs: detect
@@ -146,7 +147,10 @@ jobs:
146147
POSTGRES_DB: metaobjects_test
147148
ports:
148149
- 5432 # dynamic host port — no fixed-port collision
150+
# The data directory (the image's VOLUME) on a size-capped tmpfs: a throwaway database
151+
# never touches the runner's drives.
149152
options: >-
153+
--tmpfs /var/lib/postgresql/data:rw,size=2g
150154
--health-cmd "pg_isready -U metaobjects -d metaobjects_test"
151155
--health-interval 10s
152156
--health-timeout 5s
@@ -155,6 +159,16 @@ jobs:
155159
- uses: actions/checkout@v4
156160
with:
157161
persist-credentials: false
162+
- name: Postgres durability off (a throwaway database)
163+
# A service container takes no server flags, so set them on the running server. Its data
164+
# directory is already a tmpfs (options above); nothing in it needs to survive a crash.
165+
run: >-
166+
docker exec ${{ job.services.postgres.id }}
167+
psql -U metaobjects -d metaobjects_test
168+
-c "ALTER SYSTEM SET fsync = off"
169+
-c "ALTER SYSTEM SET synchronous_commit = off"
170+
-c "ALTER SYSTEM SET full_page_writes = off"
171+
-c "SELECT pg_reload_conf()"
158172
- env:
159173
METAOBJECTS_TEST_PG_URL: postgres://metaobjects:metaobjects@localhost:${{ job.services.postgres.ports['5432'] }}/metaobjects_test
160174
# Arms migrate-ts's real-Postgres suites (apply / lifecycle / rollback /
@@ -205,7 +219,10 @@ jobs:
205219
POSTGRES_DB: metaobjects_test
206220
ports:
207221
- 5432
222+
# The data directory (the image's VOLUME) on a size-capped tmpfs: a throwaway database
223+
# never touches the runner's drives.
208224
options: >-
225+
--tmpfs /var/lib/postgresql/data:rw,size=2g
209226
--health-cmd "pg_isready -U metaobjects -d metaobjects_test"
210227
--health-interval 10s
211228
--health-timeout 5s
@@ -214,6 +231,16 @@ jobs:
214231
- uses: actions/checkout@v4
215232
with:
216233
persist-credentials: false
234+
- name: Postgres durability off (a throwaway database)
235+
# A service container takes no server flags, so set them on the running server. Its data
236+
# directory is already a tmpfs (options above); nothing in it needs to survive a crash.
237+
run: >-
238+
docker exec ${{ job.services.postgres.id }}
239+
psql -U metaobjects -d metaobjects_test
240+
-c "ALTER SYSTEM SET fsync = off"
241+
-c "ALTER SYSTEM SET synchronous_commit = off"
242+
-c "ALTER SYSTEM SET full_page_writes = off"
243+
-c "SELECT pg_reload_conf()"
217244
- env:
218245
METAOBJECTS_TEST_PG_URL: postgres://metaobjects:metaobjects@localhost:${{ job.services.postgres.ports['5432'] }}/metaobjects_test
219246
run: scripts/ci-local.sh --only java-slow --strict-toolchains
@@ -232,7 +259,10 @@ jobs:
232259
POSTGRES_DB: metaobjects_test
233260
ports:
234261
- 5432
262+
# The data directory (the image's VOLUME) on a size-capped tmpfs: a throwaway database
263+
# never touches the runner's drives.
235264
options: >-
265+
--tmpfs /var/lib/postgresql/data:rw,size=2g
236266
--health-cmd "pg_isready -U metaobjects -d metaobjects_test"
237267
--health-interval 10s
238268
--health-timeout 5s
@@ -241,6 +271,16 @@ jobs:
241271
- uses: actions/checkout@v4
242272
with:
243273
persist-credentials: false
274+
- name: Postgres durability off (a throwaway database)
275+
# A service container takes no server flags, so set them on the running server. Its data
276+
# directory is already a tmpfs (options above); nothing in it needs to survive a crash.
277+
run: >-
278+
docker exec ${{ job.services.postgres.id }}
279+
psql -U metaobjects -d metaobjects_test
280+
-c "ALTER SYSTEM SET fsync = off"
281+
-c "ALTER SYSTEM SET synchronous_commit = off"
282+
-c "ALTER SYSTEM SET full_page_writes = off"
283+
-c "SELECT pg_reload_conf()"
244284
- env:
245285
METAOBJECTS_TEST_PG_URL: postgres://metaobjects:metaobjects@localhost:${{ job.services.postgres.ports['5432'] }}/metaobjects_test
246286
run: scripts/ci-local.sh --only python --strict-toolchains
@@ -259,7 +299,10 @@ jobs:
259299
POSTGRES_DB: metaobjects_test
260300
ports:
261301
- 5432
302+
# The data directory (the image's VOLUME) on a size-capped tmpfs: a throwaway database
303+
# never touches the runner's drives.
262304
options: >-
305+
--tmpfs /var/lib/postgresql/data:rw,size=2g
263306
--health-cmd "pg_isready -U metaobjects -d metaobjects_test"
264307
--health-interval 10s
265308
--health-timeout 5s
@@ -268,6 +311,16 @@ jobs:
268311
- uses: actions/checkout@v4
269312
with:
270313
persist-credentials: false
314+
- name: Postgres durability off (a throwaway database)
315+
# A service container takes no server flags, so set them on the running server. Its data
316+
# directory is already a tmpfs (options above); nothing in it needs to survive a crash.
317+
run: >-
318+
docker exec ${{ job.services.postgres.id }}
319+
psql -U metaobjects -d metaobjects_test
320+
-c "ALTER SYSTEM SET fsync = off"
321+
-c "ALTER SYSTEM SET synchronous_commit = off"
322+
-c "ALTER SYSTEM SET full_page_writes = off"
323+
-c "SELECT pg_reload_conf()"
271324
- env:
272325
METAOBJECTS_TEST_PG_URL: postgres://metaobjects:metaobjects@localhost:${{ job.services.postgres.ports['5432'] }}/metaobjects_test
273326
run: scripts/ci-local.sh --only csharp --strict-toolchains

‎.no-mistakes.yaml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,12 @@
1111
# along the script's own `--only` section boundaries so neither step repeats the other.
1212
# `scripts/ci-local.sh --help` enumerates what each section contains, and
1313
# `MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only <section>` prints its exact step list.
14-
# Measured green on a warm tree 2026-09-18: test 256s (it carries the mutation gate and is
15-
# almost all of the cost), lint 23s.
14+
# Measured green on a warm tree 2026-09-18: test 256s, lint 23s. The test step's mutation
15+
# gate is DIFF-SCOPED (gate_completeness_diff in the script): it mutates only the
16+
# stryker.conf.mjs `mutate` files the branch changed against its base, and SKIPs when it
17+
# changed none. Before 2026-11-14 it ran the full pass every round, 10-45 min under load
18+
# and most of this step's wall time. The full pass is `--only mutation`, the flagless
19+
# run's mutation lane, and conformance.yml (tags, nightly, dispatch).
1620
#
1721
# ORDER IS LOAD-BEARING, and it is why the work is split this way round rather than one
1822
# `--quick` in `test`. The pipeline runs test BEFORE lint. The `gates` section contains

‎AGENTS.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -499,3 +499,27 @@ inverse of metadata-is-the-spine. *OMDB engine debt* — FR-003 Plan 4 closed th
499499
anti-patterns; the Spring Boot 3 starter and OMDB autoconfiguration shipped. *WARN
500500
envelope-shape on `expected-warnings.json`* — every runner asserts it; the legacy string-list
501501
path is retired.
502+
503+
## Disk I/O: keep throwaway data off the drives
504+
505+
Test databases, sandboxes and temp files must not wear out the developer's drives.
506+
507+
- **Throwaway data goes in RAM.** Put scratch files, mutation-testing sandboxes (Stryker
508+
`tempDirName`) and test-only temp directories under `$DEV_SCRATCH` (fall back to the system temp
509+
directory when it is unset). Delete what you create when the run ends.
510+
- **Test databases run on a capped tmpfs at the image's `VOLUME` path** (`postgres:18*`:
511+
`/var/lib/postgresql`; `postgres:16/17` and `pgvector:*-pg17`: `/var/lib/postgresql/data`;
512+
`mysql:8*`: `/var/lib/mysql`; check others with `docker image inspect -f '{{json .Config.Volumes}}'`),
513+
`size=2g`, with Postgres durability off: `-c fsync=off -c synchronous_commit=off -c full_page_writes=off`.
514+
- **Every container you start, you remove.** Use `docker run --rm`, Testcontainers with Ryuk enabled,
515+
or explicit teardown in a `finally`/`trap`. Never leave a hand-started database running after your
516+
task. A database you mean to keep gets the label `dev.keep=true`.
517+
- **Generators and tools never rewrite a file whose bytes did not change**, write any manifest once per
518+
run (not once per file), and verify drift in memory rather than by rendering everything to a temp
519+
directory.
520+
- **Run the narrowest tests that prove the change** while iterating (one module, the affected test
521+
classes); run the full suite once, before the PR.
522+
- **No background processes outlive the task.** Kill test watchers, dev servers and sandboxes you
523+
started before you finish.
524+
525+
Project specifics: test databases start in the TypeScript `integration-tests` package (`src/postgres-container.ts`, `src/mysql-container.ts`, `src/cube-container.ts`, sharing `src/throwaway.ts`), the Python `tests/integration/` launchers (`postgres_container.py`, `mysql_container.py`, `throwaway.py`), the C# `MetaObjects.IntegrationTests/Runner/PostgresContainer.cs`, the Java and Kotlin Testcontainers `PostgresContainer` classes under `server/java/integration-tests*`, and the services and sidecars in `.github/workflows/integration-tests.yml` and `local-ci.yml`.

‎CHANGELOG.md‎

Lines changed: 38 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,16 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi
307307
`Cannot find module './dprint-node.linux-x64-gnu.node'` on every command that loads the
308308
codegen engine, `meta migrate` and `meta verify --db` included. Those schema commands, the
309309
binary's documented scope, now run; `meta gen` is still not supported inside the binary.
310+
- **Test databases stay off the drives.** Every Docker-backed database launcher in every port
311+
(the TypeScript `integration-tests` Postgres, MySQL and Cube containers; the Python MySQL and
312+
Postgres launchers; the C# MySQL and Postgres runners; the Java and Kotlin Testcontainers
313+
classes; the services and sidecars in the integration-test and local-CI workflows and
314+
`scripts/ci-local.sh`) mounts a 2 GiB tmpfs at the image's `VOLUME` path, and Postgres runs
315+
with `fsync`, `synchronous_commit` and `full_page_writes` off. Each container is removed when
316+
its run ends: by `--rm`, by Ryuk when a killed process never reaches its own cleanup, and by an
317+
exit or SIGTERM handler that also removes anonymous volumes. The Stryker mutation gate for the
318+
metadata package builds its sandbox under `$DEV_SCRATCH` (the system temp directory when unset)
319+
and stops its test runners with it. Contributor steps are in [CONTRIBUTING.md](CONTRIBUTING.md).
310320

311321
### Deprecated
312322

@@ -322,10 +332,34 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi
322332
the per-type name index instead of a scan of the root. Twenty copies of the canonical
323333
conformance model (1,160 objects) load in about 2 seconds of CPU, and `LargeModelLoadTest`
324334
holds that to a budget. Java and Kotlin.
325-
- **Python `metaobjects gen` lists only the files it changed.** It printed every file it emitted
326-
and counted them all as written, unchanged ones included. It now lists the files it wrote and
327-
counts the rest as unchanged (`wrote 2 file(s), 40 unchanged, across 1 target(s).`), as the
328-
other ports report.
335+
- **`gen <Entity>` no longer shrinks the files rendered from the whole model.** Narrowing a run
336+
to some entities (`meta gen Post`, Python `metaobjects gen --entities Post`) handed every
337+
generator only the named entities, so a whole-model file (the TypeScript barrel and Cube
338+
model, Python's shared enums module, duration codec and report writers, and the object index
339+
Python's entity and route generators resolve relations through) was rendered from just them. A narrowed run now writes only the named entities' own files,
340+
and every whole-model file it writes is byte-identical to a full run's. A generator sees the
341+
whole model in `ctx.entities` and the narrowing as `ctx.selected`; `perEntity` (`per_entity`)
342+
applies it. A Python target's declared `entities` still bounds that target's model. The C# and
343+
Maven entity selection added in this release follows the same rule. Two consequences to expect: a narrowed run
344+
checks the whole model as a full run does (a sourced object with no `dialect`, colliding
345+
collection names), because the files it writes are rendered from it; and an owned generator
346+
that writes one file per object by looping over `ctx.entities` itself, instead of through
347+
`perEntity`, now sees every object, so it should set each file's `sourceNode` (the runner
348+
then drops the unnamed objects' files) or apply `ctx.selected`.
349+
- **`gen` writes `.hashes.json` once per run, and not at all when nothing changed.** TypeScript,
350+
Python and C# re-read and re-wrote the whole manifest after every generated file, so a run of
351+
N files wrote it N times; they now hold it in memory for the run and write it once, as Java
352+
already did. Python's `metaobjects gen` output changes, which matters to a script that parses it:
353+
it now lists only the files it wrote, and its summary line counts the files it left as they
354+
were (unchanged or skipped) separately instead of reporting them as written. A run that
355+
fails still records the files it wrote before failing, and a failed manifest write then no
356+
longer hides the error that stopped the run.
357+
- **`meta gen` loads the metadata once.** The unknown-attribute warning re-loaded the project
358+
strictly on every run; it now reuses the generation load and parses strictly again only when
359+
the lenient load reported an unknown bare key. `meta gen` also no longer imports a database
360+
driver: the migration advice it prints reads `@metaobjectsdev/migrate-ts/offline`, a new
361+
subpath with the schema builder and diff and none of the introspection or apply code.
362+
329363
- **A projection view keeps the columns a reverse `@via` hop feeds.** An origin `@via` that walks
330364
an `identity.reference` backwards (`Project.timeEntries`, #560) loaded, but `meta migrate` left
331365
its `origin.aggregate` / `origin.first` columns out of `CREATE VIEW` without a word. The view

‎CONTRIBUTING.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,14 @@ cd server/python && pytest # Python (in its .venv)
7878
Cross-language persistence / api-contract corpora (Docker + Testcontainers) run via
7979
`scripts/integration-test.sh`.
8080

81+
Throwaway test databases stay off the drives. The Docker-backed launchers (Postgres, MySQL, the
82+
Cube stack and the `scripts/ci-local.sh` Postgres sidecar) keep each database's data directory on
83+
a size-capped tmpfs with Postgres durability off, and remove the container when the run ends.
84+
The Stryker mutation gate (`bun run conformance:mutation` in
85+
`server/typescript/packages/metadata`, via `scripts/run-stryker.sh`) builds its sandbox under
86+
`$DEV_SCRATCH` when that is set, else the system temp directory, and removes it on exit. Point
87+
`DEV_SCRATCH` at a RAM-backed directory to keep that scratch off disk.
88+
8189
### Local CI (the checks GitHub runs — run it before opening a PR)
8290

8391
GitHub Actions is enabled on this repository. The check workflows in

‎docs/features/cli.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -468,7 +468,8 @@ targets:
468468
- **`--config <path>`** picks the config file explicitly on either command; without it both
469469
look for `./metaobjects.config.yaml` in the current directory. **No config is an error**:
470470
`gen` and `verify --codegen` print `no metaobjects.config.yaml`, exit 2 and write nothing.
471-
- **Run options still apply to the list:** `--entities` (narrows each target's entities),
471+
- **Run options still apply to the list:** `--entities` (narrows the entities each target writes; a
472+
target's own `entities` still bounds its model),
472473
`--provider` (added to `providers`), `--templates` (`--prompts` on `verify`), `--baseline`.
473474
Column naming is not a run option: it is `naming` in `.metaobjects/config.json`, which every
474475
port reads, and `--column-naming` is refused

‎docs/features/codegen-concepts.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,15 @@ custom iteration:
227227
is kept as a soft-deprecated alias — "run" is ambiguous under multi-target output;
228228
`perModel` names the data scope.)
229229

230+
**`meta gen <Entity>` narrows what is written, never what a whole-model file sees.** `ctx.entities`
231+
is the whole model in scope (after the project's `scope`), even when the run names some entities.
232+
`ctx.selected` is the predicate for the named ones; it is absent when every object is selected.
233+
`perEntity` applies it. A file rendered from the whole model (a barrel, an index, a shared schema)
234+
reads `ctx.entities`, so it is the same on a narrowed run as on a full one. A generator that loops
235+
over `ctx.entities` itself must apply `ctx.selected` (TypeScript: `ctx.selected?.(o) ?? true`;
236+
Python: `ctx.is_selected(o)`), or set `sourceNode` on each per-object file, which lets the runner
237+
drop the files of objects the run did not name.
238+
230239
A generator picks its scope by which helper it composes — the same way it picks
231240
object vs model today. Reference templates and the guidance call out the scope each
232241
starting point is for, so "I need a per-package service layer" maps to a clear

0 commit comments

Comments
 (0)