Skip to content

Commit 8e52192

Browse files
authored
feat(codegen-ts): add CodeWriter and TsWriter layout foundation (ADR-0063) (#587)
* feat(codegen-ts): code writer, writer contract and corpus, layout ratchet (ADR-0063) Generators own their layout; formatters become test oracles only (ADR-0063). This change lands the foundation and converts no generator, so generated output is unchanged. - spec/code-writer.md: the shared writer contract (lines and blocks, blank-line requests, snippets, fit-or-break lists, output form, and the TypeScript dialect's import collection). - fixtures/writer-conformance/: 40 cases (30 core, 10 TypeScript), the 35th shared corpus. - CodeWriter and TsWriter (codegen-ts/src/writer/), at 120 columns, exported from the package root and the engine entry and listed in eject.engineSurface. TsWriter accepts ts-poet Code fragments and collects their imports, so generators can convert one function at a time. - Layout ratchet: an AST walk (ts.createSourceFile) over generator source that flags newline and indentation literals, newline joins and repeat-built indentation, resolving same-file constants. It has its own catch/pass cases. The allowlist names all 74 unconverted files with reasons, fails on stale entries and may only shrink against the merge base. * feat(codegen-ts): writer and reference generators are extended by subclassing (ADR-0063) CodeWriter and TsWriter keep no private members: their steps are protected methods (piece, breakAt, fits, writeHeader, writeImports, writeImport, orderModules), so a subclass overrides one and keeps the rest. ADR-0063 item 7 and spec/code-writer.md state that reference generators take the same shape, one overridable method per piece of output, and that these methods are not a stability contract. A test shows a subclass overriding orderModules. * no-mistakes(review): refuse trailing newline in writer line() * fix(codegen-ts): adapt the writer foundation to main (vendored poet, ratchet allowlist, counts) TsWriter reads Code from src/poet.ts (the ts-poet copy main vendored in #579), so a generator's imp() imports are collected again. The layout ratchet's allowlist gains the six main files that type layout (one unconverted helper, four JSON document writers and the vendored printer, exempt). Corpus count 41. * no-mistakes(document): docs: name CodeWriter/TsWriter in engine surface
1 parent 88a5520 commit 8e52192

61 files changed

Lines changed: 3041 additions & 7 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
7171
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.
7272

7373
**Cross-port conformance corpora** (every port runs the shared corpus):
74-
- Metamodel: `fixtures/conformance/` (510 fixtures; 40 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
74+
- Metamodel: `fixtures/conformance/` (510 fixtures; 41 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
7575
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
7676
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
7777
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.

‎CHANGELOG.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,21 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi
7777

7878
### Added
7979

80+
- **A code writer for generators: `CodeWriter` and `TsWriter` in `@metaobjectsdev/codegen-ts`**
81+
(ADR-0063). A generator says what a line contains, and the writer decides where it sits. It
82+
builds the file in memory with no formatter. It collects imports while the body is written,
83+
treats blank lines as requests that collapse, and prints a list on one line when the line fits
84+
120 columns, otherwise one item per line. `TsWriter` accepts ts-poet `Code` fragments and
85+
collects their `imp()` imports, so a generator can move to it one function at a time. The
86+
writer is exported from the package root and from `@metaobjectsdev/codegen-ts/engine`, so an
87+
owned (ejected) generator can use it too. Both classes are built to be extended: every
88+
method is public or protected, so a subclass can override one step, such as import order,
89+
and keep the rest. ADR-0063 also rules that reference generators will be extended the same
90+
way, one overridable method per piece of output, without promising those methods are stable
91+
between minor versions. The contract is `spec/code-writer.md`, pinned by
92+
the new `fixtures/writer-conformance/` corpus. No shipped generator uses the writer yet, so
93+
generated output does not change in this release.
94+
8095
- **A page size for generated list routes: `api.pagination`, in every port.** An external review
8196
of 1.2.0 found that list endpoints had no hard cap: the TypeScript and C# routes returned the
8297
whole table when `limit` was omitted, and `?limit=100000` was accepted in every port. The new

‎docs/CONFORMANCE.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Conformance coverage
22

3-
The MetaObjects standard ships **40 shared conformance corpora** under
3+
The MetaObjects standard ships **41 shared conformance corpora** under
44
[`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to
55
it* and asserts the same expected behaviour against the same fixtures. **This page
66
is the inverse index**: fixture → feature doc + per-port pass status, and it is the
@@ -64,6 +64,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
6464
| [`fixtures/foreign-key-index-conformance/`](../fixtures/foreign-key-index-conformance/) (the indexes `migrate.foreignKeyIndexes` adds: which foreign keys are uncovered, and each index's name per dialect, a name over 63 bytes cut on Postgres; and the name each declared `identity.secondary` / `index.lookup` is created under) | 1 model, 4 expected files (FK and declared, postgres and sqlite) | ✓ (reference, `migrate-ts`; the Drizzle tables declare the declared names) | — (no index in a generated model) | ✓ (the Exposed tables declare them) | ✓ (the EF context declares them) | — (same as Java) |
6565
| [`fixtures/codegen-noop/`](../fixtures/codegen-noop/) (FR-044 — reporting vocabulary: what is lowered, what stays inert) | 1 model pair (`reporting/with` vs `reporting/without`) | ✓ (codegen + migrate) | ✓ | ✓ | ✓ | ✓ |
6666
| [`fixtures/cube-model/`](../fixtures/cube-model/) (FR-044 Plan 4 — the `cube-model` generator's golden Cube files, plus the canonical model's golden) | 46 cases (36 trees + 10 errors) + 1 canonical golden | ✓ (the exporter is TypeScript-only, spec R6; its `cube` lane also loads the files into a real Cube) | — | — | — | — |
67+
| [`fixtures/writer-conformance/`](../fixtures/writer-conformance/) (ADR-0063 — the code writer generators write through) | 41 cases (31 core + 10 TypeScript) | ✓ (the first writer; core and TypeScript dialect) | — (writer not built yet) | — (writer not built yet) | — (writer not built yet) | — (writer not built yet) |
6768
| [`fixtures/contract-conformance/`](../fixtures/contract-conformance/) (FR-022 — the `contract-schema` and `openapi` generators' golden trees and refusals) | 19 cases (14 trees + 5 errors) | ✓ (one neutral engine, ADR-0034 Amendment 10; the generated API lanes also hold real traffic to the documents) | — | — | — | — |
6869
| [`fixtures/explorer-conformance/`](../fixtures/explorer-conformance/) (the `explorer.json` data spine: `meta export --effective` + `meta verify --format json` + the generated-file manifest in, one explorer document out) | 1 case (`shop`) | ✓ (the builder is TypeScript, and reads only port-neutral inputs) | — | — | — | — |
6970

@@ -97,7 +98,8 @@ the corpora above do two different jobs. Only the first is a promise to adopters
9798
A red cell here is a MetaObjects bug.
9899
- **Template quality checks — not a promise.** The generated lane of
99100
`api-contract-conformance/`, `generator-registry-conformance/` (stable generator names),
100-
`codegen-noop/` (vocabulary with no lowering yet emits nothing), `contract-conformance/` (the JSON Schema and OpenAPI output), `cube-model/` (the Cube
101+
`codegen-noop/` (vocabulary with no lowering yet emits nothing), `writer-conformance/` (the code
102+
writer the reference generators write through, ADR-0063), `contract-conformance/` (the JSON Schema and OpenAPI output), `cube-model/` (the Cube
101103
exporter's output, a TypeScript reference helper),
102104
`requirement-test-identity-conformance/` (which tests the `requirement-tests` reference
103105
generator yields from a ledger and what each is called; the text it writes is not

‎docs/features/own-your-codegen.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,9 @@ holds), the naming tables and the type-mapping tables (`names.ts`, `naming.ts`,
109109
projection and report lowering that `meta migrate` and `meta verify` run. An owned file
110110
imports the engine from the package's engine entry, `@metaobjectsdev/codegen-ts/engine`,
111111
which also exports the code printer's `code`, `imp` and `joinCode` (`poet.ts`, the package's
112-
vendored copy of `ts-poet`). The list is `ENGINE_MODULES`, exported from that entry; a module
112+
vendored copy of `ts-poet`) and the code writer (`CodeWriter`, `TsWriter`, ADR-0063),
113+
which lays a file out but decides nothing about what the file contains, so like the
114+
printer it stays with the engine. The list is `ENGINE_MODULES`, exported from that entry; a module
113115
that is not on it and that a generator reaches is copied. An owned generator therefore
114116
imports no packaged module that returns emitted text, and a test ejects each generator alone
115117
into an empty project to prove it: it compiles as your `tsconfig.codegen.json` checks it and

‎examples/showcase/site-payload.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
},
99
"counts": {
1010
"fixtures": 510,
11-
"corpora": 40,
11+
"corpora": 41,
1212
"baseTypes": 17
1313
},
1414
"snippets": {

‎fixtures/generator-registry-conformance/registry.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -396,7 +396,9 @@
396396
"codegen-ts/src/templates/find-inbound.ts",
397397
"codegen-ts/src/templates/find-templates.ts",
398398
"codegen-ts/src/tph.ts",
399-
"codegen-ts/src/view-context.ts"
399+
"codegen-ts/src/view-context.ts",
400+
"codegen-ts/src/writer/code-writer.ts",
401+
"codegen-ts/src/writer/ts-writer.ts"
400402
]
401403
},
402404
"csharp": {
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Writer conformance
2+
3+
The shared cases for the code writer every port's generators write through
4+
([ADR-0063](../../spec/decisions/ADR-0063-generators-own-layout.md)). The contract and the case
5+
format are in [`spec/code-writer.md`](../../spec/code-writer.md).
6+
7+
- `cases/*.json` with `"dialect": "core"` pin the language-neutral core: indentation, blocks,
8+
blank-line requests, snippets, fit-or-break lists, line endings and the error codes. Every
9+
port's writer runs them with the options the case states.
10+
- Cases with a language dialect (`"dialect": "typescript"`) pin that language's writer: import
11+
collection and ordering, doc comments and its defaults.
12+
13+
Runner: TypeScript, `server/typescript/packages/codegen-ts/test/writer-conformance.test.ts`.
14+
The other ports run the `core` cases when their writers land, and add cases for their own
15+
dialect beside these.
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
{
2+
"description": "No blank line directly after the line that opens a block.",
3+
"dialect": "core",
4+
"options": {
5+
"indent": " ",
6+
"width": 40,
7+
"trailingComma": true
8+
},
9+
"ops": [
10+
[
11+
"open",
12+
"f() {"
13+
],
14+
[
15+
"blank"
16+
],
17+
[
18+
"line",
19+
"a;"
20+
],
21+
[
22+
"close",
23+
"}"
24+
]
25+
],
26+
"expected": [
27+
"f() {",
28+
" a;",
29+
"}"
30+
]
31+
}
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"description": "A blank-line request after the last line writes nothing.",
3+
"dialect": "core",
4+
"options": {
5+
"indent": " ",
6+
"width": 40,
7+
"trailingComma": true
8+
},
9+
"ops": [
10+
[
11+
"line",
12+
"a;"
13+
],
14+
[
15+
"blank"
16+
]
17+
],
18+
"expected": [
19+
"a;"
20+
]
21+
}
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"description": "A blank-line request before the first line writes nothing.",
3+
"dialect": "core",
4+
"options": {
5+
"indent": " ",
6+
"width": 40,
7+
"trailingComma": true
8+
},
9+
"ops": [
10+
[
11+
"blank"
12+
],
13+
[
14+
"line",
15+
"a;"
16+
]
17+
],
18+
"expected": [
19+
"a;"
20+
]
21+
}

0 commit comments

Comments
 (0)