Skip to content

feat(codegen): uniform entity selection across ports and a linear Java metadata loader - #588

Merged
dmealing merged 8 commits into
mainfrom
fm/mo-codegen-loader-selection
Nov 15, 2026
Merged

dmealing merged 8 commits into
mainfrom
fm/mo-codegen-loader-selection

Conversation

@dmealing

@dmealing dmealing commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

What Changed

  • Entity selection now works the same way in every port. C# gains dotnet meta gen --entities A,B (and verify --codegen --entities), Java/Kotlin gain mvn metaobjects:generate -Dmeta.entities=A,B, matching TypeScript's positional names and Python's --entities: only the named objects' files are regenerated, whole-model files (AppDbContext, the Java/Kotlin validator registry, report export writer, binding provider) still render from the whole model, a narrowed run never deletes or forgets unselected files, and a blank selection or a name matching no object is refused (exit 2 in C#, a failed Maven goal). TypeScript's meta gen <Name>... now refuses an unknown name with exit 2 instead of dropping it silently, and the option is documented on every port page.
  • Java/Kotlin generators honor <filters>. codegen-spring and codegen-kotlin generators previously ignored a generator's <filters> and wrote every object; they now respect them, as codegen-base generators always did.
  • The Java metadata loader handles large models. Quadratic reference resolution (linear candidate scan per child and per reference) is replaced with a per-type name index and set lookups, so a ~1,160-object model loads in ~2 seconds instead of blowing the timeout — held by a new LargeModelLoadTest budget. The fixed 30-second init timeout is now a documented setting (LoaderOptions.setInitTimeoutMs, the metaobjects.loader.initTimeoutMs system property, or Maven loaderTimeoutMs), defaulting to the previous 30 seconds.

Validation took four no-mistakes runs: the first was stopped, the second cancelled after a minute, the third was lost when the machine rebooted, and the fourth passed. After that, main was merged in (only CHANGELOG.md conflicted, and both sides were kept). That merge was published without a fifth run: the gate's local suite (scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains, then --only gates) passed on the merge commit before it was pushed.

Validation

Risk Assessment

✅ Low: Both prior review findings are fixed with gating tests, the loader's linearization preserves resolution semantics (key precedence and declaration-order tie-break verified), and the cross-port selection refusals/whole-model invariants hold by construction in each port's mechanism; only an informational note on the C# probe's double-generation cost remains.

Testing

Drove the real codegen entry point of every port live — TS meta CLI, Java and Kotlin via the Maven plugin built from the branch, C# dotnet meta CLI with its init-scaffolded owned codegen project against locally packed 1.3.0 packages, and the Python metaobjects console script via uv — each against disposable projects in RAM scratch. Narrowing behaved identically everywhere: only selected per-object files rewritten, unselected files left untouched, orphan cleanup skipped, whole-model files (TS barrel, Java/Kotlin ReportExportWriter, C# AppDbContext + ReportExportWriter, Python enums.py) recreated byte-identical to the unfiltered run, and unknown/blank selections refused with matching per-port messages and exit 2. The Java loader work was driven live too: every documented timeout path (-Dmeta.loaderTimeoutMs, the loader arg, -Dmetaobjects.loader.initTimeoutMs) takes effect with clear failures for timeouts and invalid values, and a synthetic 1,201-object model loads and fully generates in 6.4s under the unchanged 30s default — the scale that previously blew the fixed timeout. No failures and no source changes, so the configured test command's passing baseline stands as broad regression; all scratch projects, local feeds, build servers, and background builds were torn down and the worktree is clean. Evidence transcripts are in the evidence directory.

  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
TS: meta gen Program narrows per-object output while the whole-model barrel still renders whole (picks up a newly added Order export without generating Order.ts) ✅ pass live typescript-python-selection-transcript.txt; run of the branch-built meta CLI against a disposable init-scaffolded project
TS: unknown positional beside/without a known name is refused exit 2 with the shared wording, before anything is written ✅ pass live typescript-python-selection-transcript.txt (meta gen: names &#39;Ordr&#39;, which matches no object in the metadata., exit 2, no files created)
Java (Maven plugin): -Dmeta.entities=Program rewrites only selected per-object files, keeps whole-model ReportExportWriter whole, narrows a per-entity template scope, and skips the orphan sweep with… ✅ pass live java-narrowed-run-transcript.txt (corrupted WeekRepository left stale, deleted ReportExportWriter recreated, 57→56 template files)
Java: unknown and blank meta.entities selections fail the goal with the shared refusal messages ✅ pass live java-narrowed-run-transcript.txt (BUILD FAILURE, 'names Zzz, which matches no object…' / 'names no object.')
Java: loader timeout is a documented setting — -Dmeta.loaderTimeoutMs and -Dmetaobjects.loader.initTimeoutMs both take effect, invalid values are refused, default stays 30s ✅ pass live java-loader-timeout-and-scale.txt (1ms timeouts fail with the guidance message; 'abc' refused)
Java loader scale: a ~1,200-object model loads and fully generates well inside the default timeout (quadratic step gone) ✅ pass live java-loader-timeout-and-scale.txt (1,201-object synthetic model, 6,000 generated files, 6.4s wall for the whole Maven process under the 30s default)
Java: a generator that cannot narrow (MustacheTemplateGenerator) logs the review-1 warning instead of silently narrowing nothing ✅ pass live java-narrowed-run-transcript.txt ([WARNING] … cannot narrow to meta.entities; it renders every object.)
Kotlin: -Dmeta.entities=Program narrows Kotlin generators' per-object output and keeps the whole-model ReportExportWriter.kt byte-identical ✅ pass live kotlin-narrowed-run.txt (sha256 -c OK against full-run output; Week.kt left stale)
C#: dotnet meta gen --entities Product narrows per-object output; whole-model AppDbContext.g.cs and shared ReportExportWriter.g.cs are recreated byte-identical even though the selection touches neit… ✅ pass live csharp-narrowed-run-transcript.txt (sha256 -c OK for both whole-model files; Supplier.g.cs left 'STALE'; Widget.g.cs left absent)
C#: --entities naming an unknown object or nothing is refused exit 2 with the shared messages ✅ pass live csharp-narrowed-run-transcript.txt (both refusals, exit 2)
Python: metaobjects gen --entities Product (pre-existing surface, parity requirement) narrows per-object output, keeps whole-model enums.py byte-identical, skips orphan cleanup with a warning, and r… ✅ pass live typescript-python-selection-transcript.txt (enums.py sha256 OK; 'names Zzz…' and 'names no object.' both exit 2)
Evidence: Java/Kotlin narrowed-run transcript (selection, refusals, mustache warning)
# Java/Kotlin (Maven plugin 8.3.0, local install of branch build) — mvn metaobjects:generate -Dmeta.entities=Program
# Model: fixtures/persistence-conformance/canonical/meta.fitness.json (116 objects incl. served reports)

$ mvn metaobjects:generate -Dmeta.entities=Program
[INFO] MetaData Mojo > Generating only Program (meta.entities); files built from the whole model still see every object.
[INFO] MetaData Mojo > Deleted no orphaned file: meta.entities narrowed this build.
[INFO] BUILD SUCCESS

# State driven before the run: deleted whole-model ReportExportWriter.java + selected ProgramRepository.java;
# corrupted WeekRepository.java ("STALE"); deleted tpl/Week.txt (per-entity template scope).
# After the run:
ReportExportWriter.java   -> RECREATED (whole-model file survives a selection containing no report)
ProgramRepository.java    -> RECREATED (selected object)
WeekRepository.java       -> still "STALE" (unselected object not rewritten)
tpl/Week.txt              -> still absent (per-entity template scope narrowed, 56 of 57 files)
tpl/Program.txt           -> rewritten

$ mvn metaobjects:generate -Dmeta.entities=Program,Zzz
[ERROR] ... failed: mvn metaobjects:generate: meta.entities names 'Zzz', which matches no object in the metadata.
[INFO] BUILD FAILURE

$ mvn metaobjects:generate -Dmeta.entities=          # blank (unset shell variable)
[ERROR] ... failed: mvn metaobjects:generate: meta.entities names no object.
[INFO] BUILD FAILURE

# Generators that cannot narrow (review-1 fix):
$ mvn metaobjects:generate -Dmeta.entities=Program   # with MustacheTemplateGenerator in <generators>
[WARNING] MetaData Mojo > com.metaobjects.generator.mustache.MustacheTemplateGenerator cannot narrow to meta.entities; it renders every object.
Evidence: Java loader timeout knobs + 1,201-object scale run
# Loader timeout (plan item 6) driven through the real mojo, branch build 8.3.0

$ mvn metaobjects:generate -Dmeta.loaderTimeoutMs=1
[ERROR] ... failed: Loader initialization timeout after 1ms: MetaDataLoader:manual:null:1.
A large model can need longer: raise LoaderOptions.setInitTimeoutMs, the loader argument
'initTimeoutMs' (Maven: -Dmeta.loaderTimeoutMs), or -Dmetaobjects.loader.initTimeoutMs.

$ mvn metaobjects:generate -Dmetaobjects.loader.initTimeoutMs=1     # JVM system-property path
[ERROR] ... failed: Loader initialization timeout after 1ms ... (same message)
[INFO] BUILD FAILURE

$ mvn metaobjects:generate -Dmetaobjects.loader.initTimeoutMs=abc   # invalid value
[ERROR] ... failed: -Dmetaobjects.loader.initTimeoutMs must be a positive number of milliseconds, got "abc"

# Scale (quadratic step removed): a synthetic 1,201-object model (1,200 entities extending one
# abstract base, each with fields/enum/decimal/source/identity/index) —
$ mvn -o -q metaobjects:generate    # full run, DEFAULT timeout (30 s)
BUILD SUCCESS — wall clock 6.4 s for the whole Maven process, load + generation of 6,000 Java files.
(The reported failure this fixes: ~1,160 objects previously could not load inside the fixed 30 s timeout.)
Evidence: Kotlin narrowed run with byte-identical whole-model writer
# Kotlin generators via the same mojo (com.metaobjects.generator.kotlin.*), fitness model
# Before: deleted Program.kt + ProgramTable.kt (selected), corrupted Week.kt ("STALE"),
#         deleted whole-model fitness/ReportExportWriter.kt (sha256 recorded).

$ mvn -o -q metaobjects:generate -Dmeta.entities=Program
# (no errors)

Program.kt / ProgramTable.kt -> RECREATED ("package fitness")
Week.kt                      -> still "STALE"
ReportExportWriter.kt        -> RECREATED, sha256 IDENTICAL to the full-run output:
  $ sha256sum -c kt-writer.sha
  target/generated-kotlin/fitness/ReportExportWriter.kt: OK
Evidence: C# --entities narrowed run and refusals (whole-model files sha256-identical)
# C# port: dotnet meta gen --entities (CLI built from the branch; owned codegen/ project scaffolded
# by `dotnet meta init`, pinned to locally packed MetaObjects.Codegen 1.3.0).
# Model: Product/Supplier/Widget/Gadget(+shared abstract enum) + reporting fixture
# (Member/Project/TimeEntry + 2 served reports). Baseline full gen: 33 files.

$ dotnet meta gen ... --entities Product        # after deleting AppDbContext.g.cs,
  written: acme/Product.g.cs                     #   ReportExportWriter.g.cs, Product.g.cs,
  written: AppDbContext.g.cs                     #   Widget.g.cs; corrupting Supplier.g.cs
  written: ReportExportWriter.g.cs
  dotnet meta gen: 3 file(s) written

# Whole-model files byte-identical to the unfiltered run (sha256 -c against baseline):
generated/AppDbContext.g.cs: OK          # still contains all 9 DbSets
generated/ReportExportWriter.g.cs: OK    # review-2 case: selection contains no report, file still whole
acme/Supplier.g.cs -> still "STALE"      # unselected, not rewritten, not swept
acme/Widget.g.cs   -> still absent

$ dotnet meta gen ... --entities Product,Zzz
error: dotnet meta gen: --entities names 'Zzz', which matches no object in the metadata.   # exit 2
$ dotnet meta gen ... --entities " "
error: dotnet meta gen: --entities names no object.                                        # exit 2
Evidence: TypeScript + Python selection and refusal transcripts
# TypeScript (meta CLI built from the branch, disposable project scaffolded by `meta init`)
$ meta gen Program                    # model: Program, Week (+ Order added mid-test)
  summary: "1 written, 3 unchanged"   # barrel index.ts REWRITTEN to export the new Order
                                       # while Order.ts / Week.ts were NOT generated (per-object narrowing)
$ meta gen Program Ardr
  meta: meta gen: names 'Ordr', which matches no object in the metadata.   # exit 2, nothing written
$ meta gen Zzz
  meta: meta gen: names 'Zzz', which matches no object in the metadata.    # exit 2

# Python (package from the branch via uv): whole-model enums.py + refusals
$ metaobjects gen --entities Product
  metaobjects gen: wrote 2 file(s) across 1 target(s). 4 file(s) left as they were.
  warning: Skipped orphan cleanup: this run was narrowed by --generators or --entities ...
  generated/enums.py -> recreated, sha256 IDENTICAL to the full run (whole-model stays whole)
  generated/acme/Supplier.py -> still "STALE" (unselected)
$ metaobjects gen --entities Product,Zzz   # error: ... names 'Zzz', which matches no object ...  exit 2
$ metaobjects gen --entities ""            # error: metaobjects gen: --entities names no object.  exit 2

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ℹ️ server/csharp/MetaObjects.Codegen/CodegenRunner.cs:323 - Informational, no defect: under --entities, Emit runs every non-WholeModel generator twice (whole-model pass + narrowed probe), so a narrowed C# run costs roughly a full pass plus a small extra pass rather than a fraction of one. The plan's stated aim is a fast develop loop, so it is worth knowing that in C# --entities is a safety/parity feature, not a speedup — unlike TypeScript, where ctx.selected narrows without a second pass. The tradeoff is documented in the method's own comment and is the price of the 'whole-model files render whole' invariant held by construction (content always comes from the whole run), so no change is requested.
✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
TS: meta gen Program narrows per-object output while the whole-model barrel still renders whole (picks up a newly added Order export without generating Order.ts) ✅ pass live typescript-python-selection-transcript.txt; run of the branch-built meta CLI against a disposable init-scaffolded project
TS: unknown positional beside/without a known name is refused exit 2 with the shared wording, before anything is written ✅ pass live typescript-python-selection-transcript.txt (meta gen: names &#39;Ordr&#39;, which matches no object in the metadata., exit 2, no files created)
Java (Maven plugin): -Dmeta.entities=Program rewrites only selected per-object files, keeps whole-model ReportExportWriter whole, narrows a per-entity template scope, and skips the orphan sweep with… ✅ pass live java-narrowed-run-transcript.txt (corrupted WeekRepository left stale, deleted ReportExportWriter recreated, 57→56 template files)
Java: unknown and blank meta.entities selections fail the goal with the shared refusal messages ✅ pass live java-narrowed-run-transcript.txt (BUILD FAILURE, 'names Zzz, which matches no object…' / 'names no object.')
Java: loader timeout is a documented setting — -Dmeta.loaderTimeoutMs and -Dmetaobjects.loader.initTimeoutMs both take effect, invalid values are refused, default stays 30s ✅ pass live java-loader-timeout-and-scale.txt (1ms timeouts fail with the guidance message; 'abc' refused)
Java loader scale: a ~1,200-object model loads and fully generates well inside the default timeout (quadratic step gone) ✅ pass live java-loader-timeout-and-scale.txt (1,201-object synthetic model, 6,000 generated files, 6.4s wall for the whole Maven process under the 30s default)
Java: a generator that cannot narrow (MustacheTemplateGenerator) logs the review-1 warning instead of silently narrowing nothing ✅ pass live java-narrowed-run-transcript.txt ([WARNING] … cannot narrow to meta.entities; it renders every object.)
Kotlin: -Dmeta.entities=Program narrows Kotlin generators' per-object output and keeps the whole-model ReportExportWriter.kt byte-identical ✅ pass live kotlin-narrowed-run.txt (sha256 -c OK against full-run output; Week.kt left stale)
C#: dotnet meta gen --entities Product narrows per-object output; whole-model AppDbContext.g.cs and shared ReportExportWriter.g.cs are recreated byte-identical even though the selection touches neit… ✅ pass live csharp-narrowed-run-transcript.txt (sha256 -c OK for both whole-model files; Supplier.g.cs left 'STALE'; Widget.g.cs left absent)
C#: --entities naming an unknown object or nothing is refused exit 2 with the shared messages ✅ pass live csharp-narrowed-run-transcript.txt (both refusals, exit 2)
Python: metaobjects gen --entities Product (pre-existing surface, parity requirement) narrows per-object output, keeps whole-model enums.py byte-identical, skips orphan cleanup with a warning, and r… ✅ pass live typescript-python-selection-transcript.txt (enums.py sha256 OK; 'names Zzz…' and 'names no object.' both exit 2)
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • Baseline broad regression: scripts/ci_local not applicable — configured command scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (passed at target commit before this step; no source files were changed in this turn)
  • TS: bun <branch cli>/dist/bin/meta.js init/gen in a disposable project — full baseline, narrowed gen Program with a mid-test new entity (barrel rewritten whole, per-object files not generated), gen Program Ordr and gen Zzz exit-2 refusals with nothing written
  • Java: mvn -DskipTests install of the branch reactor, then mvn metaobjects:generate over the 116-object fitness canonical model — baseline full run; narrowed -Dmeta.entities=Program against deliberately deleted/corrupted files (whole-model ReportExportWriter recreated, unselected files untouched, per-entity template scope narrowed, orphan sweep skipped with log line); unknown and blank selection refusals; mustache cannot-narrow warning
  • Java loader: -Dmeta.loaderTimeoutMs=1 and -Dmetaobjects.loader.initTimeoutMs=1 timeout failures, =abc refusal, and a synthetic 1,201-object model fully generating in 6.4s under the default 30s timeout
  • Kotlin: mvn metaobjects:generate with KotlinEntity/ExposedTable/Names/FilterAllowlist/Relations/SpringController/ReportExport generators — narrowed run recreates selected files, leaves Week.kt stale, recreates ReportExportWriter.kt sha256-identical
  • C#: dotnet build + pack of MetaObjects, MetaObjects.Render, MetaObjects.Codegen, MetaObjects.Cli 1.3.0 into a local feed; dotnet meta init scaffold; full gen baseline; --entities Product run with sha256 checks of AppDbContext.g.cs and ReportExportWriter.g.cs against the full run; --entities Product,Zzz and blank refusals (exit 2)
  • Python: uv run of the branch package against a disposable project — full gen, --entities Product (whole-model enums.py sha256-identical, orphan-cleanup warning), --entities Product,Zzz and --entities "" refusals (exit 2)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…oss ports

Java loader (also Kotlin):
- Remove the quadratic child-key check (list -> set) and resolve references
  through the per-type name index. 20 copies of the canonical model
  (1,160 objects) load in about 2 s CPU; it previously exceeded the 30 s timeout.
- The 30 s init timeout is now a setting: LoaderOptions.setInitTimeoutMs, loader
  arg initTimeoutMs, -Dmeta.loaderTimeoutMs on the Maven plugin, or
  -Dmetaobjects.loader.initTimeoutMs. LargeModelLoadTest holds a CPU budget.

Selection parity:
- codegen-spring and codegen-kotlin generators now honour <filters>
  (confirmed by running that they ignored them before).
- mvn metaobjects:generate -Dmeta.entities=A,B narrows per-object output for
  Java and Kotlin; whole-model files still render from the whole model, a
  narrowed run sweeps no orphans, and blank or unknown names fail the goal.
- C# dotnet meta gen/verify --codegen --entities, with the same semantics.
- Python gen lists only the files it changed and counts the rest unchanged.
- Selection documented on every port page; CHANGELOG under [Unreleased].
Every other port refuses an unknown --entities / meta.entities name; TypeScript
dropped it silently when another name matched and only warned when none did.
Now exit 2 with the same wording, before anything is written.
…selection

# Conflicts:
#	CHANGELOG.md
#	server/python/src/metaobjects/cli.py
@dmealing
dmealing merged commit 56a4d89 into main Nov 15, 2026
9 checks passed
@dmealing
dmealing deleted the fm/mo-codegen-loader-selection branch November 15, 2026 15:07
dmealing added a commit that referenced this pull request Nov 15, 2026
…anged (ADR-0064) (#590)

* feat(codegen): skip gen when nothing changed, in every port (ADR-0064)

Forward-port of #585 from release/1.2 (b9bc9c7) onto main, merged with the
manifest written once per run (#584) and the uniform entity selection (#588).
The skip ADR is numbered ADR-0064 here: ADR-0062 and ADR-0063 are taken on main.

- gen builds a key from the loaded model (each root node in canonical effective
  form), the generators' identity (built engine code, config, owned generators,
  templates, declared extra inputs, project config) and the output-changing
  flags, before anything renders. A match with the last successful run, with
  every generated file still holding that run's bytes, renders and writes
  nothing. --force (Maven: -Dmetaobjects.force=true) turns it off; a generator
  that cannot declare its inputs opts out.
- A run that names entities (positional, --entities, -Dmeta.entities) forces
  exactly those: it never skips, never stores a key and leaves the stored
  full-run key alone, in every port.
- verify --codegen may answer "no drift" locally from the key; CI and --full
  always render, and every port compares in memory.
- The hash manifest stays loaded once and written once, and only when an entry
  changed; identical bytes are never rewritten; TypeScript runs git merge-file
  only for a hand-edited file.
- fixtures/codegen-skip-conformance/cases.json: the shared case table.

Hand-resolved: TypeScript runner, overwrite-policy, gen/verify/args/index and
codegen-drift; Python cli, runner, overwrite_policy; C# CodegenRunner,
CodegenCli, CodegenDrift, Generator, Program; Java generate and verify mojos;
docs, changelog and the ADR index.

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): warn unknown attrs before gen skip; add regression test

* no-mistakes(review): apply reviewer inline fixes

* no-mistakes(review): Carry warnings in skipped gen json/toon document

* no-mistakes(test): Align C# agent-context fixture wording; sdk suite green

* no-mistakes(document): align codegen-skip docs with ADR-0064
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant