Skip to content

fix(codegen-ts): keep whole-model output whole and write the manifest once per gen run - #584

Merged
dmealing merged 10 commits into
mainfrom
fm/mo-codegen-run-core
Nov 15, 2026
Merged

dmealing merged 10 commits into
mainfrom
fm/mo-codegen-run-core

Conversation

@dmealing

@dmealing dmealing commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

What Changed

  • gen <Entity> now narrows only the files it writes. Every generator sees the whole model in ctx.entities and the selection as ctx.selected (selectedEntities / perEntity apply it), so whole-model files such as the TypeScript barrel and Cube model, and Python's shared enums and object index, are byte-identical to a full run. This is implemented in the TypeScript runner and the Python runner.
  • .hashes.json is held in memory for the duration of a run and written once at its end, and not at all when nothing changed. This applies to the TypeScript, Python and C# runners (withManifestBatch, manifest_batch, HashManifest.Batch). Python's gen output now lists only the files it wrote, and its summary counts unchanged or skipped files separately. A failed run still records the files it wrote before failing.
  • meta gen loads the metadata once, reusing the generation load for the unknown-attribute check, and no longer imports a database driver: migration advice points to the new @metaobjectsdev/migrate-ts/offline subpath. The Docker test databases in every port now run on a size-capped tmpfs with Postgres durability off and are removed when each run ends, and the Stryker sandbox lives under $DEV_SCRATCH (scripts/run-stryker.sh).
Validation

Risk Assessment

⚠️ Medium: The manifest batching, whole-model ctx.entities with ctx.selected, single metadata load and RAM-backed test databases all trace correctly across the TS, Python and C# runners, but the change is large and cross-port. The Python CLI output change (it lists only the files it wrote and counts the rest separately) was reviewed and kept by the task owner; the CHANGELOG records it.

Testing

The baseline configured command (ci-local ts-fast + ts-unit) was already green on this tree. I then drove eleven live scenarios: in a disposable RAM-scratch project the real Node meta CLI showed narrowed runs keeping whole-model files byte-identical (barrel md5-verified), jsonSchemaFile writing only the named entity's schemas, the manifest written exactly once on changed runs and zero times on unchanged runs (strace-counted opens), a failed run recording its 7 partial writes and surfacing the refusal with exit 1, zero database-driver loads under a refusing Node resolve hook (with live controls), and both unknown-attribute warning paths from the single load with correct strict behavior; the real Python metaobjects CLI showed the same narrowing, write-once and verify --codegen in-sync behaviors; the C# lane ran the real dotnet meta gen + verify --codegen and all 5,428 of its tests green including OverwriteHashManifestTests; and both lanes' integration runs proved the throwaway test databases mount size-capped tmpfs with durability off and are removed after the run. Full regression beyond the baseline covered the python and csharp lanes (both LOCAL CI PASSED); Java/Kotlin lanes were not run within budget — the diff's only Java/Kotlin delta is test-container/reaper infrastructure exercised by those ports' integration lanes, with no Java/Kotlin product code in the change. One limitation: this sandbox could not simulate an unwritable manifest for the failed-manifest-write-preserves-the-run's-error path live, so that sub-behavior rests on its unit tests (green in ts-unit and the C# lane). No failures; no action needed.

  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
TS narrowed meta gen Post keeps whole-model files whole: deleted barrel regenerated byte-identical with both entities' exports; unselected entity's files untouched ✅ pass live ts-narrowed-run.txt — barrel md5 1ded10ec80314740d34ce58e3de878a6 identical on full and narrowed runs, exporting ./blog/content/Author.js and Post.js; only Post's 7 own files + barrel considered
TS jsonSchemaFile() on a narrowed run writes only the named entity's schema files (deleted Author schemas are NOT recreated) ✅ pass live ts-narrowed-run.txt — after deleting Author.schema.json/-create/-update, meta gen Post recreated 0 of them
TS gen writes .hashes.json once per run and not at all when nothing changed ✅ pass live ts-gen-strace.txt (changed 19-file run: exactly 1 O_WRONLY open of .hashes.json), ts-gen-unchanged-strace.txt (0) and mtime unchanged
TS failed run keeps the files it wrote recorded and surfaces its own failure: hand-edit refusal exits 1, 7 other files written and recorded, manifest written once ✅ pass live ts-gen-refusal.txt + ts-gen-refusal-strace.txt — src/generated/blog/content/Post.ts,refused, exit 1, hand edit intact on disk, 1 manifest write, Author.website written
TS meta gen loads no database driver: full gen completes under a Node resolve hook refusing kysely/pg/mysql2/better-sqlite3/@libsql, with live controls ✅ pass live ts-gen-no-driver.txt (exit 0, zero FORBIDDEN refusals, Post files rewritten with a new field); controls: importing @metaobjectsdev/migrate-ts root refused with 'FORBIDDEN DB DRIVER IMPORT REACHED: kys…
TS unknown-attribute advisories come from the single load: lenient gen warns (attr-typo and bare-key paths) and exits 0; --strict fails before writing ✅ pass live ts-gen-unknown-attr.txt, ts-gen-bare-key.txt (warnings naming attribute, node, file+jsonPath), ts-gen-strict.txt (exit 1, load error, nothing written)
Python metaobjects gen --entities Post keeps whole-model files whole: shared enums.py regenerated byte-identical (AuthorRank + PostState); Author.py untouched; unchanged files reported separately ✅ pass live py-gen-narrowed.txt — cmp: regenerated enums.py BYTE-IDENTICAL to the full run; Author.py byte-identical; "wrote 1 file(s)... 5 file(s) left as they were"
Python target-declared entities still bounds that target's model: scoped out dir gets only Post's files and enums.py holds only PostState ✅ pass live fresh src/generated/scoped listing (6 files, Post only) and scoped enums.py containing only class PostState; metaobjects verify --codegen in sync (10 file(s)) afterwards
Python gen writes .hashes.json once per run and not at all when nothing changed ✅ pass live py-gen-changed-strace.txt (1 O_WRONLY open), py-gen-unchanged-strace.txt (0), py-gen-unchanged.txt summary line
C# dotnet meta gen writes the manifest once per run: real CLI drives gen (3 file(s) written) and verify --codegen reports in sync; OverwriteHashManifestTests green ✅ pass live ci-lane-results.txt — the csharp lane's live CLI drive plus MetaObjects.Codegen.Tests 1590 passed / 0 failed (includes ARun_WritesTheManifestOnce_AtTheEnd, AFailedRun_KeepsItsOwnError_WhenTheManifestW…
Drive-wear: throwaway test databases mount size-capped tmpfs at the image VOLUME path with Postgres durability off, and are removed when the run ends ✅ pass live throwaway-db-tmpfs.txt — docker inspect of the lanes' containers: tmpfs {"/var/lib/postgresql":"size=1g"/"size=2g"}, cmd carries -c fsync=off -c synchronous_commit=off -c full_page_writes=off; lane si…
Evidence: TS narrowed run: barrel byte-identical, Author schemas not recreated

meta gen Post after deleting index.ts + Author schemas: "1 written, 7 unchanged"; barrel md5 identical before/after, exports both Author.js and Post.js; Author schema files recreated: 0

## TS narrowed run: whole-model barrel stays whole; jsonSchemaFile writes only the named entity's schemas

\### full run barrel (md5 1ded10ec80314740d34ce58e3de878a6):
// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts.
export * from "./blog/content/Author.js";
export * from "./blog/content/Post.js";

\### meta gen Post (after deleting index.ts and all Author schema files):
meta: Skipped orphan cleanup: this run generated only Post, so it cannot tell a file belonging to an unselected entity from one that is no longer generated. Run 'meta gen' with no entity filter to reconcile deletions.
gen[9]{file,status}:
  src/generated/blog/content/Post.ts,unchanged
  src/generated/blog/content/Post.queries.ts,unchanged
  src/generated/blog/content/Post.routes.ts,unchanged
  src/generated/blog/content/Post.names.ts,unchanged
  src/generated/schemas/blog/content/Post.schema.json,unchanged
  src/generated/schemas/blog/content/Post.create.schema.json,unchanged
  src/generated/schemas/blog/content/Post.update.schema.json,unchanged
  src/generated/index.ts,new
  src/generated/flaky.json,unchanged
summary: "1 written, 8 unchanged"
warnings[1]: "Skipped orphan cleanup: this run generated only Post, so it cannot tell a file belonging to an unselected entity from one that is no longer generated. Run 'meta gen' with no entity filter to reconcile deletions."

\### barrel after narrowed run (md5 1ded10ec80314740d34ce58e3de878a6):
// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts.
export * from "./blog/content/Author.js";
export * from "./blog/content/Post.js";

\### Author schema files recreated by the narrowed run:
0
Evidence: strace: changed 19-file TS run writes .hashes.json once; unchanged run zero times

An strace -f -e openat of the real meta gen on a 19-file project showed .gen-state/.hashes.json opened for writing exactly once on a run that changed files, and zero times on an unchanged re-run (its mtime was untouched). The raw trace is omitted here: it lists local scratch paths, including six three-way-merge temp files under the system temp directory.

Evidence: TS hand-edit refusal: exit 1, partial writes recorded, one manifest write

summary: "7 written, 8 unchanged, 1 refused"; Refused to overwrite .../Post.ts ... NOT overwritten; manifest writes during failed run: 1

meta: Refused to overwrite <scratch-tsproj>/src/generated/blog/content/Post.ts: this file has been edited since it was generated, and there is no .gen-state snapshot body on this machine to merge against — it was NOT overwritten, and your version is intact on disk. Recovery: docs/features/own-your-codegen.md.
meta: To KEEP your version of the file(s) above, the edit must be committed first — then: 'meta gen --baseline=fresh' (writes fresh output over them and seeds the missing .gen-state snapshot), 'git checkout -- <paths>' to bring your version back, then 'meta gen' again — the snapshot now exists, so the edit merges. To DISCARD your version instead, '--baseline=fresh' on its own is the whole answer. Moving the edit into a non-generated file works only where the edit can live outside the generated one; a requirementTests() stub's body cannot, since the test name is its link to the requirement.
gen[16]{file,status}:
  src/generated/blog/content/Author.ts,overwrite
  src/generated/blog/content/Post.ts,refused
  src/generated/blog/content/Author.queries.ts,unchanged
  src/generated/blog/content/Post.queries.ts,unchanged
  src/generated/blog/content/Author.routes.ts,overwrite
  src/generated/blog/content/Post.routes.ts,unchanged
  src/generated/blog/content/Author.names.ts,overwrite
  src/generated/blog/content/Post.names.ts,unchanged
  src/generated/schemas/blog/content/Author.schema.json,overwrite
  src/generated/schemas/blog/content/Author.create.schema.json,overwrite
  src/generated/schemas/blog/content/Author.update.schema.json,overwrite
  src/generated/schemas/blog/content/Post.schema.json,unchanged
  src/generated/schemas/blog/content/Post.create.schema.json,unchanged
  src/generated/schemas/blog/content/Post.update.schema.json,unchanged
  src/generated/index.ts,unchanged
  src/generated/flaky.json,overwrite
summary: "7 written, 8 unchanged, 1 refused"
help[2]: typecheck the generated code with `npx tsc`,create your database tables with `meta migrate --from-db --db <url> --dialect sqlite --slug init --apply`
antiPatterns:
  status: ran
  total: 0
  rows: []
warnings[2]: "Refused to overwrite <scratch-tsproj>/src/generated/blog/content/Post.ts: this file has been edited since it was generated, and there is no .gen-state snapshot body on this machine to merge against — it was NOT overwritten, and your version is intact on disk. Recovery: docs/features/own-your-codegen.md.","To KEEP your version of the file(s) above, the edit must be committed first — then: 'meta gen --baseline=fresh' (writes fresh output over them and seeds the missing .gen-state snapshot), 'git checkout -- <paths>' to bring your version back, then 'meta gen' again — the snapshot now exists, so the edit merges. To DISCARD your version instead, '--baseline=fresh' on its own is the whole answer. Moving the edit into a non-generated file works only where the edit can live outside the generated one; a requirementTests() stub's body cannot, since the test name is its link to the requirement."
Evidence: TS gen under no-driver hook: exit 0, zero FORBIDDEN refusals, real writes
gen[15]{file,status}:
  src/generated/blog/content/Author.ts,unchanged
  src/generated/blog/content/Post.ts,overwrite
  src/generated/blog/content/Author.queries.ts,unchanged
  src/generated/blog/content/Post.queries.ts,unchanged
  src/generated/blog/content/Author.routes.ts,unchanged
  src/generated/blog/content/Post.routes.ts,overwrite
  src/generated/blog/content/Author.names.ts,unchanged
  src/generated/blog/content/Post.names.ts,overwrite
  src/generated/schemas/blog/content/Author.schema.json,unchanged
  src/generated/schemas/blog/content/Author.create.schema.json,unchanged
  src/generated/schemas/blog/content/Author.update.schema.json,unchanged
  src/generated/schemas/blog/content/Post.schema.json,overwrite
  src/generated/schemas/blog/content/Post.create.schema.json,overwrite
  src/generated/schemas/blog/content/Post.update.schema.json,overwrite
  src/generated/index.ts,unchanged
summary: "6 written, 9 unchanged"
help[2]: typecheck the generated code with `npx tsc`,create your database tables with `meta migrate --from-db --db <url> --dialect sqlite --slug init --apply`
antiPatterns:
  status: ran
  total: 0
  rows: []
Evidence: TS unknown-attr + bare-key warnings; strict fails
meta: warning: ERR_UNKNOWN_ATTR: Unknown attribute '@maximalLength' on field.string 'email' — not declared by any registered provider for field.string
  in meta.blog.json at $['metadata.root'].children[0]['object.entity'].children[3]['field.string']
meta: warning: meta gen loads leniently and generated anyway, but `meta verify` rejects this metadata (ADR-0023): fix or remove the attribute — a typo'd one (`isAbstrakt`, `@requird`) silently changes what is generated. `meta gen --strict` stops the run instead.
gen[15]{file,status}:
  src/generated/blog/content/Author.ts,unchanged
  src/generated/blog/content/Post.ts,unchanged
  src/generated/blog/content/Author.queries.ts,unchanged
  src/generated/blog/content/Post.queries.ts,unchanged
  src/generated/blog/content/Author.routes.ts,unchanged
  src/generated/blog/content/Post.routes.ts,unchanged
  src/generated/blog/content/Author.names.ts,unchanged
  src/generated/blog/content/Post.names.ts,unchanged
  src/generated/schemas/blog/content/Author.schema.json,unchanged
  src/generated/schemas/blog/content/Author.create.schema.json,unchanged
  src/generated/schemas/blog/content/Author.update.schema.json,unchanged
  src/generated/schemas/blog/content/Post.schema.json,unchanged
  src/generated/schemas/blog/content/Post.create.schema.json,unchanged
  src/generated/schemas/blog/content/Post.update.schema.json,unchanged
  src/generated/index.ts,unchanged
summary: 15 unchanged
help[1]: typecheck the generated code with `npx tsc`
antiPatterns:
  status: ran
  total: 0
  rows: []
warnings[2]: "ERR_UNKNOWN_ATTR: Unknown attribute '@maximalLength' on field.string 'email' — not declared by any registered provider for field.string\n  in meta.blog.json at $['metadata.root'].children[0]['object.entity'].children[3]['field.string']","meta gen loads leniently and generated anyway, but `meta verify` rejects this metadata (ADR-0023): fix or remove the attribute — a typo'd one (`isAbstrakt`, `@requird`) silently changes what is generated. `meta gen --strict` stops the run instead."
Evidence: TS bare-key typo warning path
meta: warning: ERR_UNKNOWN_ATTR: Unknown key 'isAbstrakt' on object.entity 'Author' at metadata.root.children[0].object.entity (must be reserved or @-prefixed)
  in meta.blog.json at $['metadata.root'].children[0]['object.entity']
meta: warning: ERR_UNKNOWN_ATTR: Unknown attribute '@maximalLength' on field.string 'email' — not declared by any registered provider for field.string
  in meta.blog.json at $['metadata.root'].children[0]['object.entity'].children[3]['field.string']
meta: warning: meta gen loads leniently and generated anyway, but `meta verify` rejects this metadata (ADR-0023): fix or remove the attribute — a typo'd one (`isAbstrakt`, `@requird`) silently changes what is generated. `meta gen --strict` stops the run instead.
gen[15]{file,status}:
  src/generated/blog/content/Author.ts,unchanged
  src/generated/blog/content/Post.ts,unchanged
  src/generated/blog/content/Author.queries.ts,unchanged
  src/generated/blog/content/Post.queries.ts,unchanged
  src/generated/blog/content/Author.routes.ts,unchanged
  src/generated/blog/content/Post.routes.ts,unchanged
  src/generated/blog/content/Author.names.ts,unchanged
  src/generated/blog/content/Post.names.ts,unchanged
  src/generated/schemas/blog/content/Author.schema.json,unchanged
  src/generated/schemas/blog/content/Author.create.schema.json,unchanged
  src/generated/schemas/blog/content/Author.update.schema.json,unchanged
  src/generated/schemas/blog/content/Post.schema.json,unchanged
  src/generated/schemas/blog/content/Post.create.schema.json,unchanged
  src/generated/schemas/blog/content/Post.update.schema.json,unchanged
  src/generated/index.ts,unchanged
summary: 15 unchanged
help[1]: typecheck the generated code with `npx tsc`
antiPatterns:
  status: ran
  total: 0
  rows: []
warnings[3]: "ERR_UNKNOWN_ATTR: Unknown key 'isAbstrakt' on object.entity 'Author' at metadata.root.children[0].object.entity (must be reserved or @-prefixed)\n  in meta.blog.json at $['metadata.root'].children[0]['object.entity']","ERR_UNKNOWN_ATTR: Unknown attribute '@maximalLength' on field.string 'email' — not declared by any registered provider for field.string\n  in meta.blog.json at $['metadata.root'].children[0]['object.entity'].children[3]['field.string']","meta gen loads leniently and generated anyway, but `meta verify` rejects this metadata (ADR-0023): fix or remove the attribute — a typo'd one (`isAbstrakt`, `@requird`) silently changes what is generated. `meta gen --strict` stops the run instead."
Evidence: TS strict gen fails the load (exit 1)
meta: failed to load metadata: ERR_UNKNOWN_ATTR: Unknown attribute '@maximalLength' on field.string 'email' — not declared by any registered provider for field.string
  in meta.blog.json at $['metadata.root'].children[0]['object.entity'].children[3]['field.string']
meta: meta gen --strict loads strictly, as `meta verify` does (ADR-0023): every authored @attr must be declared, and nothing was written. Fix: register the attr on a metadata provider, OR move arbitrary author-supplied properties into an `attr.properties` bag, OR drop `--strict`.
  • Evidence: strace: TS unchanged re-run writes manifest zero times (raw trace kept with the local validation run, not published)
Evidence: Python unchanged re-run output

metaobjects gen: wrote 0 file(s) across 1 target(s). 9 file(s) left as they were.

metaobjects gen: wrote 0 file(s) across 1 target(s). 9 file(s) left as they were.
Evidence: Python narrowed run: enums.py byte-identical, Author untouched

wrote 1 file(s) across 1 target(s). 5 file(s) left as they were. — regenerated enums.py holds class AuthorRank AND class PostState; cmp: BYTE-IDENTICAL to full run

<scratch-pyproj>/src/generated/api/Post.py
<scratch-pyproj>/src/generated/api/post_router.py
<scratch-pyproj>/src/generated/api/__init__.py
warning: Skipped orphan cleanup: this run was narrowed by --generators or --entities, so it cannot tell a file the rest of the list writes from one no generator emits any more. Run `metaobjects gen` without them to delete generated files that are no longer generated.
metaobjects gen: wrote 3 file(s) across 1 target(s). 2 file(s) left as they were.
  • Evidence: strace: Python changed run writes .hashes.json once (raw trace kept with the local validation run, not published)
  • Evidence: strace: Python unchanged run writes .hashes.json zero times (raw trace kept with the local validation run, not published)
Evidence: CI lane results (python + csharp, both PASSED; C# test suites incl. OverwriteHashManifestTests; live dotnet meta gen)
## scripts/ci-local.sh --only python --strict-toolchains
══════════════════════ SUMMARY ══════════════════════
  ✓ conformance: python  (2m13s)
  ✓ integration-tests (python)  (38s)
── slowest steps ──
     2m13s  conformance: python
       38s  integration-tests (python)
  total wall time: 2m55s

## scripts/ci-local.sh --only csharp --strict-toolchains
══════════════════════ SUMMARY ══════════════════════
  ✓ conformance: csharp  (1m33s)
  ✓ integration-tests (csharp)  (4m12s)
── slowest steps ──
     4m12s  integration-tests (csharp)
     1m33s  conformance: csharp
  total wall time: 5m49s
══════════════════════════════════════════════════════
LOCAL CI PASSED

## C# unit suites (includes OverwriteHashManifestTests):
Passed!  - Failed:     0, Passed:  2593, Skipped:     0, Total:  2593, Duration: 3 s - MetaObjects.Conformance.Tests.dll (net8.0)
Passed!  - Failed:     0, Passed:   364, Skipped:     0, Total:   364, Duration: 59 ms - MetaObjects.Render.Tests.dll (net8.0)
Passed!  - Failed:     0, Passed:  1590, Skipped:     1, Total:  1591, Duration: 43 s - MetaObjects.Codegen.Tests.dll (net8.0)
Passed!  - Failed:     0, Passed:   385, Skipped:     0, Total:   385, Duration: 27 s - MetaObjects.Cli.Tests.dll (net8.0)
Passed!  - Failed:     0, Passed:   496, Skipped:     0, Total:   496, Duration: 4 m 6 s - MetaObjects.IntegrationTests.dll (net8.0)

## the C# lane's live CLI drive (real dotnet meta binary):
Test run for {run-worktree}/server/csharp/MetaObjects.Codegen.Tests/bin/Debug/net8.0/MetaObjects.Codegen.Tests.dll (.NETCoreApp,Version=v8.0)
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  written: schemas/Address.schema.json
  written: schemas/Customer.schema.json
  written: openapi.json
dotnet meta gen: 3 file(s) written
Evidence: Throwaway test DBs on tmpfs with durability off; run's containers removed

musing_lalande: tmpfs {"/var/lib/postgresql":"size=1g"}, -c fsync=off -c synchronous_commit=off -c full_page_writes=off; jolly_neumann: size=2g, same flags

## Throwaway test databases: RAM-backed, size-capped tmpfs + Postgres durability off

== musing_lalande (started by this run's CI lanes)
image: postgres:18-alpine
tmpfs mounts: {"/var/lib/postgresql":"size=1g"}
-c fsync=off -c synchronous_commit=off -c full_page_writes=off

== jolly_neumann (started by this run's CI lanes)
image: postgres:18-alpine
tmpfs mounts: {"/var/lib/postgresql":"rw,size=2g"}
-c fsync=off -c synchronous_commit=off -c full_page_writes=off

## containers NOT started by this run (predate it) - left untouched:
  mystifying_northcutt 2026-11-14T23:02:41.741556394Z postgres:18-alpine
  boring_merkle 2026-11-14T23:56:22.737863225Z postgres:18-alpine
  romantic_villani 2026-11-14T23:56:23.171726396Z redis:7-alpine
  • Outcome: ⚠️ 1 info across 1 run (43m45s)

D5: integration suites and container cleanup (run after the push, no new commit)

Each port's integration suite was run on the pushed head 96f52076, one at a time, against Docker. After each one I compared docker ps -a with a snapshot taken before the first.

Suite Result Wall-clock
TypeScript integration-tests (bun test, serial) 1266 pass, 0 fail 43m36s
Python pytest tests/integration -n 8 482 passed, 15 skipped, 0 failed 39s
C# MetaObjects.IntegrationTests 496 passed, 0 failed 3m24s
Java integration-tests 415 run, 0 failures, 0 errors, 3 skipped 1m50s
Kotlin integration-tests-kotlin + codegen-kotlin-exposed1x-check 450 + 5 run, 0 failures, 0 errors, 7 skipped 7m02s

The four non-TS suites were run through scripts/integration-test.sh <lane> with a private Maven repository and TMPDIR on RAM scratch. Both were deleted afterwards.

docker ps -a after each suite: clean. None of the Postgres, MySQL or Ryuk containers these suites started is still present. The TS reaper exited on its own about 15 seconds after its suite finished. While the Kotlin suite was running, another worktree started its own TypeScript suite. That run's metaobjects-mysql-<id> container was running when I checked, and I left it alone: it carries the TS/Java name pattern, not Kotlin's metaobjects-mysql-kt-*. This also covers the Java and Kotlin lanes, which the pipeline's Test step did not run.

Two exited metaobjects-mysql-py-* containers were left by Python runs that happened before this task started, so they predate the change. They were left untouched, as were the unrelated containers of other projects.

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 warning
  • ⚠️ server/python/src/metaobjects/cli.py:1850 - Python metaobjects gen now prints only the files it changed and splits the summary line into 'wrote N file(s)' and 'M file(s) left as they were'. That is not needed to write the manifest once, which is the stated item 1. It changes the output of a command that scripts may parse (unchanged_out, STATUS_UNCHANGED/STATUS_SKIPPED, the changed list at cli.py:1850, the summary at cli.py:1861). TypeScript, C# and Java do not do this. This was raised in review round 1; the config-rewrite half was restored, but this half is still present and the CHANGELOG records it as deliberate. The narrower form that satisfies the intent is to keep printing every emitted path and the old count, and keep only the manifest batching and the _record early return. Removing it would also drop the test edit in test_cli_verify_requirements.py and the test_an_unchanged_file_is_not_reported_as_written test. Keep it only with the author's explicit sign-off.

Outcome: kept. The task owner confirmed the output change; the CHANGELOG [Unreleased] entry names it for script authors.

⚠️ **Test** - 1 info
  • ℹ️ The failed-manifest-write-must-not-hide-the-run's-error path (withManifestBatch's catch in TypeScript, BatchScope.Dispose in C#, manifest_batch in Python) could not be driven live: this environment's sandbox grants the product's own write syscalls privileges that shell-level probes cannot replicate, so no chmod/unwritable-directory setup reliably failed a manifest write. The behavior is covered by green unit tests in each port (manifest-write-once.test.ts 'a run that throws keeps its own error when the manifest write fails too', OverwriteHashManifestTests.AFailedRun_KeepsItsOwnError_WhenTheManifestWriteFailsToo) and the live hand-edit-refusal run verified the adjacent half (a failing run still records the files it wrote, manifest written once, run's own error surfaced with exit 1).
  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
TS narrowed meta gen Post keeps whole-model files whole: deleted barrel regenerated byte-identical with both entities' exports; unselected entity's files untouched ✅ pass live ts-narrowed-run.txt — barrel md5 1ded10ec80314740d34ce58e3de878a6 identical on full and narrowed runs, exporting ./blog/content/Author.js and Post.js; only Post's 7 own files + barrel considered
TS jsonSchemaFile() on a narrowed run writes only the named entity's schema files (deleted Author schemas are NOT recreated) ✅ pass live ts-narrowed-run.txt — after deleting Author.schema.json/-create/-update, meta gen Post recreated 0 of them
TS gen writes .hashes.json once per run and not at all when nothing changed ✅ pass live ts-gen-strace.txt (changed 19-file run: exactly 1 O_WRONLY open of .hashes.json), ts-gen-unchanged-strace.txt (0) and mtime unchanged
TS failed run keeps the files it wrote recorded and surfaces its own failure: hand-edit refusal exits 1, 7 other files written and recorded, manifest written once ✅ pass live ts-gen-refusal.txt + ts-gen-refusal-strace.txt — src/generated/blog/content/Post.ts,refused, exit 1, hand edit intact on disk, 1 manifest write, Author.website written
TS meta gen loads no database driver: full gen completes under a Node resolve hook refusing kysely/pg/mysql2/better-sqlite3/@libsql, with live controls ✅ pass live ts-gen-no-driver.txt (exit 0, zero FORBIDDEN refusals, Post files rewritten with a new field); controls: importing @metaobjectsdev/migrate-ts root refused with 'FORBIDDEN DB DRIVER IMPORT REACHED: kys…
TS unknown-attribute advisories come from the single load: lenient gen warns (attr-typo and bare-key paths) and exits 0; --strict fails before writing ✅ pass live ts-gen-unknown-attr.txt, ts-gen-bare-key.txt (warnings naming attribute, node, file+jsonPath), ts-gen-strict.txt (exit 1, load error, nothing written)
Python metaobjects gen --entities Post keeps whole-model files whole: shared enums.py regenerated byte-identical (AuthorRank + PostState); Author.py untouched; unchanged files reported separately ✅ pass live py-gen-narrowed.txt — cmp: regenerated enums.py BYTE-IDENTICAL to the full run; Author.py byte-identical; "wrote 1 file(s)... 5 file(s) left as they were"
Python target-declared entities still bounds that target's model: scoped out dir gets only Post's files and enums.py holds only PostState ✅ pass live fresh src/generated/scoped listing (6 files, Post only) and scoped enums.py containing only class PostState; metaobjects verify --codegen in sync (10 file(s)) afterwards
Python gen writes .hashes.json once per run and not at all when nothing changed ✅ pass live py-gen-changed-strace.txt (1 O_WRONLY open), py-gen-unchanged-strace.txt (0), py-gen-unchanged.txt summary line
C# dotnet meta gen writes the manifest once per run: real CLI drives gen (3 file(s) written) and verify --codegen reports in sync; OverwriteHashManifestTests green ✅ pass live ci-lane-results.txt — the csharp lane's live CLI drive plus MetaObjects.Codegen.Tests 1590 passed / 0 failed (includes ARun_WritesTheManifestOnce_AtTheEnd, AFailedRun_KeepsItsOwnError_WhenTheManifestW…
Drive-wear: throwaway test databases mount size-capped tmpfs at the image VOLUME path with Postgres durability off, and are removed when the run ends ✅ pass live throwaway-db-tmpfs.txt — docker inspect of the lanes' containers: tmpfs {"/var/lib/postgresql":"size=1g"/"size=2g"}, cmd carries -c fsync=off -c synchronous_commit=off -c full_page_writes=off; lane si…
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • baseline (pre-run, green): scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • S1: full meta gen then meta gen Post with the barrel deleted — barrel regenerated byte-identical (md5 1ded10ec80314740d34ce58e3de878a6 both runs) exporting Author AND Post; only Post's 7 own files + barrel considered
  • S2: meta gen Post after deleting all Author schema files — 0 recreated (jsonSchemaFile narrowed fix)
  • S3: strace openat on the real meta CLI — changed 19-file run opens .hashes.json for write exactly once; unchanged re-run opens it zero times and its mtime is untouched
  • S4: hand-edit refusal run (snapshot removed, file edited, unrelated entity changed) — exit 1, Post.ts refused by name with the hand edit intact, 7 other files written and recorded, manifest written exactly once
  • S5: Node module resolve hook refusing kysely/pg/mysql2/better-sqlite3/@libsql — controls prove it live (migrate-ts root import refused on kysely; /offline subpath loads clean); real meta gen with a real content change completes exit 0 with zero refusals
  • S6: unknown @maximalLength and bare-key isAbstrakt typos — lenient gen warns naming attribute, node, file+jsonPath and exits 0; --strict exits 1 before writing (both load paths of loadMemoryWithFindings)
  • S7: metaobjects gen full → 9 files; re-run → "wrote 0 file(s)... 9 file(s) left as they were"; --entities Post after deleting enums.py + Post files — enums.py regenerated byte-identical (AuthorRank + PostState), Author.py byte-identical
  • S8: Python target entities: [Post] — fresh out dir gets only Post files, scoped enums.py holds only PostState; metaobjects verify --codegen reports in sync (10 file(s))
  • S9: strace on the real metaobjects CLI — changed run writes .hashes.json once, unchanged run zero times
  • S10: C# lane drove the real dotnet meta gen (3 file(s) written) + dotnet meta verify --codegen OK; 1590 MetaObjects.Codegen.Tests passed incl. OverwriteHashManifestTests
  • S11: drive-wear — integration lanes ran the throwaway containers; docker inspect shows tmpfs on /var/lib/postgresql (size=1g / 2g) with fsync/synchronous_commit/full_page_writes off; lane sidecars reaped, my containers removed at teardown
  • regression: scripts/ci-local.sh --only python --strict-toolchains (PASSED) and --only csharp --strict-toolchains (PASSED)
🔧 **Document** - 2 issues found → auto-fixed (2) ✅
  • ✅ server/typescript/packages/codegen-ts/src/index.ts:10: selectedEntities is now exported beside perEntity/perPackage/perModel, so owned generators outside the package can import the predicate the docs tell them to apply.
  • ✅ The drive-wear work is documented: AGENTS.md gains the "Disk I/O: keep throwaway data off the drives" section, naming where this repository starts its test databases, CONTRIBUTING.md gains a test-storage note, and the CHANGELOG [Unreleased] section has an entry for the tmpfs, durability-off, container-removal and Stryker-sandbox changes.
  • ℹ️ Narrowed-run behaviour (ctx.selected, selectedEntities) is documented for generator authors.

✅ Re-checked - no issues remain.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…load metadata once

- TypeScript, Python and C# hold .gen-state/.hashes.json in memory for a gen run and
  write it once at the end (Java already did); an unchanged run writes nothing.
  Python gen no longer lists unchanged files as written.
- A run narrowed to some entities (meta gen <Entity>, metaobjects gen --entities)
  writes only those entities' own files; every generator still sees the whole model
  (ctx.entities), with the narrowing as ctx.selected. Whole-model files are now
  byte-identical to a full run's. A Python target's declared entities still bound
  its model.
- meta gen reuses its generation load for the unknown-attribute warning, re-parsing
  strictly only when a bare unknown key was reported.
- meta gen no longer imports a database driver: CLI modules on the gen path read the
  new @metaobjectsdev/migrate-ts/offline subpath. Gated by an import-graph test.
- Under Node the config file loads from its own path, so jiti's cache hits instead of
  filling with randomly named entries.
… ends

Database test containers mount a size-capped tmpfs at the image's VOLUME path, Postgres runs
with fsync, synchronous_commit and full_page_writes off, and every docker-CLI launcher also
removes its containers on exit and SIGTERM (covering a disabled Ryuk). The CI sidecar and
workflow services get the same treatment. The Stryker sandbox moves to a RAM scratch dir and
its runners stop with it. Work in progress: AGENTS.md section, CHANGELOG and suite proof follow.
…ails too

A run that throws still writes the manifest for the files it wrote, but a
failed write no longer replaces the run's error (TypeScript, Python, C#).
Also: Python's gen summary counts skip-existing files as left as they
were, not unchanged; a Postgres image tag that names no major takes the
Postgres 18 data directory; CHANGELOG notes what a narrowed run now
checks and what an owned per-object generator must do.
@dmealing
dmealing merged commit 88a5520 into main Nov 15, 2026
9 checks passed
@dmealing
dmealing deleted the fm/mo-codegen-run-core branch November 15, 2026 01:42
dmealing added a commit that referenced this pull request Nov 15, 2026
…anged (ADR-0064) (#590)

* feat(codegen): skip gen when nothing changed, in every port (ADR-0064)

Forward-port of #585 from release/1.2 (b9bc9c7) onto main, merged with the
manifest written once per run (#584) and the uniform entity selection (#588).
The skip ADR is numbered ADR-0064 here: ADR-0062 and ADR-0063 are taken on main.

- gen builds a key from the loaded model (each root node in canonical effective
  form), the generators' identity (built engine code, config, owned generators,
  templates, declared extra inputs, project config) and the output-changing
  flags, before anything renders. A match with the last successful run, with
  every generated file still holding that run's bytes, renders and writes
  nothing. --force (Maven: -Dmetaobjects.force=true) turns it off; a generator
  that cannot declare its inputs opts out.
- A run that names entities (positional, --entities, -Dmeta.entities) forces
  exactly those: it never skips, never stores a key and leaves the stored
  full-run key alone, in every port.
- verify --codegen may answer "no drift" locally from the key; CI and --full
  always render, and every port compares in memory.
- The hash manifest stays loaded once and written once, and only when an entry
  changed; identical bytes are never rewritten; TypeScript runs git merge-file
  only for a hand-edited file.
- fixtures/codegen-skip-conformance/cases.json: the shared case table.

Hand-resolved: TypeScript runner, overwrite-policy, gen/verify/args/index and
codegen-drift; Python cli, runner, overwrite_policy; C# CodegenRunner,
CodegenCli, CodegenDrift, Generator, Program; Java generate and verify mojos;
docs, changelog and the ADR index.

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): warn unknown attrs before gen skip; add regression test

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): Carry warnings in skipped gen json/toon document

* no-mistakes(test): Align C# agent-context fixture wording; sdk suite green

* no-mistakes(document): align codegen-skip docs with ADR-0064
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant