diff --git a/AGENTS.md b/AGENTS.md index 5d9f43db6..8c2df5a5d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (510 fixtures; 40 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (510 fixtures; 41 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived. diff --git a/CHANGELOG.md b/CHANGELOG.md index b857c4dbf..e97919b68 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,21 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi ### Added +- **A code writer for generators: `CodeWriter` and `TsWriter` in `@metaobjectsdev/codegen-ts`** + (ADR-0063). A generator says what a line contains, and the writer decides where it sits. It + builds the file in memory with no formatter. It collects imports while the body is written, + treats blank lines as requests that collapse, and prints a list on one line when the line fits + 120 columns, otherwise one item per line. `TsWriter` accepts ts-poet `Code` fragments and + collects their `imp()` imports, so a generator can move to it one function at a time. The + writer is exported from the package root and from `@metaobjectsdev/codegen-ts/engine`, so an + owned (ejected) generator can use it too. Both classes are built to be extended: every + method is public or protected, so a subclass can override one step, such as import order, + and keep the rest. ADR-0063 also rules that reference generators will be extended the same + way, one overridable method per piece of output, without promising those methods are stable + between minor versions. The contract is `spec/code-writer.md`, pinned by + the new `fixtures/writer-conformance/` corpus. No shipped generator uses the writer yet, so + generated output does not change in this release. + - **A page size for generated list routes: `api.pagination`, in every port.** An external review of 1.2.0 found that list endpoints had no hard cap: the TypeScript and C# routes returned the whole table when `limit` was omitted, and `?limit=100000` was accepted in every port. The new diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 7cfecafe7..a3f6a72db 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -1,6 +1,6 @@ # Conformance coverage -The MetaObjects standard ships **40 shared conformance corpora** under +The MetaObjects standard ships **41 shared conformance corpora** under [`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to it* and asserts the same expected behaviour against the same fixtures. **This page is the inverse index**: fixture → feature doc + per-port pass status, and it is the @@ -64,6 +64,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/foreign-key-index-conformance/`](../fixtures/foreign-key-index-conformance/) (the indexes `migrate.foreignKeyIndexes` adds: which foreign keys are uncovered, and each index's name per dialect, a name over 63 bytes cut on Postgres; and the name each declared `identity.secondary` / `index.lookup` is created under) | 1 model, 4 expected files (FK and declared, postgres and sqlite) | ✓ (reference, `migrate-ts`; the Drizzle tables declare the declared names) | — (no index in a generated model) | ✓ (the Exposed tables declare them) | ✓ (the EF context declares them) | — (same as Java) | | [`fixtures/codegen-noop/`](../fixtures/codegen-noop/) (FR-044 — reporting vocabulary: what is lowered, what stays inert) | 1 model pair (`reporting/with` vs `reporting/without`) | ✓ (codegen + migrate) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/cube-model/`](../fixtures/cube-model/) (FR-044 Plan 4 — the `cube-model` generator's golden Cube files, plus the canonical model's golden) | 46 cases (36 trees + 10 errors) + 1 canonical golden | ✓ (the exporter is TypeScript-only, spec R6; its `cube` lane also loads the files into a real Cube) | — | — | — | — | +| [`fixtures/writer-conformance/`](../fixtures/writer-conformance/) (ADR-0063 — the code writer generators write through) | 41 cases (31 core + 10 TypeScript) | ✓ (the first writer; core and TypeScript dialect) | — (writer not built yet) | — (writer not built yet) | — (writer not built yet) | — (writer not built yet) | | [`fixtures/contract-conformance/`](../fixtures/contract-conformance/) (FR-022 — the `contract-schema` and `openapi` generators' golden trees and refusals) | 19 cases (14 trees + 5 errors) | ✓ (one neutral engine, ADR-0034 Amendment 10; the generated API lanes also hold real traffic to the documents) | — | — | — | — | | [`fixtures/explorer-conformance/`](../fixtures/explorer-conformance/) (the `explorer.json` data spine: `meta export --effective` + `meta verify --format json` + the generated-file manifest in, one explorer document out) | 1 case (`shop`) | ✓ (the builder is TypeScript, and reads only port-neutral inputs) | — | — | — | — | @@ -97,7 +98,8 @@ the corpora above do two different jobs. Only the first is a promise to adopters A red cell here is a MetaObjects bug. - **Template quality checks — not a promise.** The generated lane of `api-contract-conformance/`, `generator-registry-conformance/` (stable generator names), - `codegen-noop/` (vocabulary with no lowering yet emits nothing), `contract-conformance/` (the JSON Schema and OpenAPI output), `cube-model/` (the Cube + `codegen-noop/` (vocabulary with no lowering yet emits nothing), `writer-conformance/` (the code + writer the reference generators write through, ADR-0063), `contract-conformance/` (the JSON Schema and OpenAPI output), `cube-model/` (the Cube exporter's output, a TypeScript reference helper), `requirement-test-identity-conformance/` (which tests the `requirement-tests` reference generator yields from a ledger and what each is called; the text it writes is not diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 381ed3588..858eea380 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -109,7 +109,9 @@ holds), the naming tables and the type-mapping tables (`names.ts`, `naming.ts`, projection and report lowering that `meta migrate` and `meta verify` run. An owned file imports the engine from the package's engine entry, `@metaobjectsdev/codegen-ts/engine`, which also exports the code printer's `code`, `imp` and `joinCode` (`poet.ts`, the package's -vendored copy of `ts-poet`). The list is `ENGINE_MODULES`, exported from that entry; a module +vendored copy of `ts-poet`) and the code writer (`CodeWriter`, `TsWriter`, ADR-0063), +which lays a file out but decides nothing about what the file contains, so like the +printer it stays with the engine. The list is `ENGINE_MODULES`, exported from that entry; a module that is not on it and that a generator reaches is copied. An owned generator therefore imports no packaged module that returns emitted text, and a test ejects each generator alone into an empty project to prove it: it compiles as your `tsconfig.codegen.json` checks it and diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index 77dc60265..3bdbbe135 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -8,7 +8,7 @@ }, "counts": { "fixtures": 510, - "corpora": 40, + "corpora": 41, "baseTypes": 17 }, "snippets": { diff --git a/fixtures/generator-registry-conformance/registry.json b/fixtures/generator-registry-conformance/registry.json index d536a236c..fa315e29c 100644 --- a/fixtures/generator-registry-conformance/registry.json +++ b/fixtures/generator-registry-conformance/registry.json @@ -396,7 +396,9 @@ "codegen-ts/src/templates/find-inbound.ts", "codegen-ts/src/templates/find-templates.ts", "codegen-ts/src/tph.ts", - "codegen-ts/src/view-context.ts" + "codegen-ts/src/view-context.ts", + "codegen-ts/src/writer/code-writer.ts", + "codegen-ts/src/writer/ts-writer.ts" ] }, "csharp": { diff --git a/fixtures/writer-conformance/README.md b/fixtures/writer-conformance/README.md new file mode 100644 index 000000000..e6fbe73d4 --- /dev/null +++ b/fixtures/writer-conformance/README.md @@ -0,0 +1,15 @@ +# Writer conformance + +The shared cases for the code writer every port's generators write through +([ADR-0063](../../spec/decisions/ADR-0063-generators-own-layout.md)). The contract and the case +format are in [`spec/code-writer.md`](../../spec/code-writer.md). + +- `cases/*.json` with `"dialect": "core"` pin the language-neutral core: indentation, blocks, + blank-line requests, snippets, fit-or-break lists, line endings and the error codes. Every + port's writer runs them with the options the case states. +- Cases with a language dialect (`"dialect": "typescript"`) pin that language's writer: import + collection and ordering, doc comments and its defaults. + +Runner: TypeScript, `server/typescript/packages/codegen-ts/test/writer-conformance.test.ts`. +The other ports run the `core` cases when their writers land, and add cases for their own +dialect beside these. diff --git a/fixtures/writer-conformance/cases/blank-after-open-dropped.json b/fixtures/writer-conformance/cases/blank-after-open-dropped.json new file mode 100644 index 000000000..b13f5ee0f --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-after-open-dropped.json @@ -0,0 +1,31 @@ +{ + "description": "No blank line directly after the line that opens a block.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "f() {" + ], + [ + "blank" + ], + [ + "line", + "a;" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "f() {", + " a;", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/blank-at-file-end-dropped.json b/fixtures/writer-conformance/cases/blank-at-file-end-dropped.json new file mode 100644 index 000000000..075061367 --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-at-file-end-dropped.json @@ -0,0 +1,21 @@ +{ + "description": "A blank-line request after the last line writes nothing.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;" + ], + [ + "blank" + ] + ], + "expected": [ + "a;" + ] +} diff --git a/fixtures/writer-conformance/cases/blank-at-file-start-dropped.json b/fixtures/writer-conformance/cases/blank-at-file-start-dropped.json new file mode 100644 index 000000000..a7850f018 --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-at-file-start-dropped.json @@ -0,0 +1,21 @@ +{ + "description": "A blank-line request before the first line writes nothing.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "blank" + ], + [ + "line", + "a;" + ] + ], + "expected": [ + "a;" + ] +} diff --git a/fixtures/writer-conformance/cases/blank-before-close-dropped.json b/fixtures/writer-conformance/cases/blank-before-close-dropped.json new file mode 100644 index 000000000..a3392eafd --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-before-close-dropped.json @@ -0,0 +1,31 @@ +{ + "description": "No blank line directly before the line that closes a block.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "f() {" + ], + [ + "line", + "a;" + ], + [ + "blank" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "f() {", + " a;", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/blank-between-blocks-kept.json b/fixtures/writer-conformance/cases/blank-between-blocks-kept.json new file mode 100644 index 000000000..1879b8c93 --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-between-blocks-kept.json @@ -0,0 +1,37 @@ +{ + "description": "A request between two blocks gives one blank line between them.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "a() {" + ], + [ + "close", + "}" + ], + [ + "blank" + ], + [ + "open", + "b() {" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "a() {", + "}", + "", + "b() {", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/blank-requests-collapse.json b/fixtures/writer-conformance/cases/blank-requests-collapse.json new file mode 100644 index 000000000..56b3f6f88 --- /dev/null +++ b/fixtures/writer-conformance/cases/blank-requests-collapse.json @@ -0,0 +1,30 @@ +{ + "description": "Two blank-line requests in a row give one blank line.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;" + ], + [ + "blank" + ], + [ + "blank" + ], + [ + "line", + "b;" + ] + ], + "expected": [ + "a;", + "", + "b;" + ] +} diff --git a/fixtures/writer-conformance/cases/empty-file.json b/fixtures/writer-conformance/cases/empty-file.json new file mode 100644 index 000000000..5d565a949 --- /dev/null +++ b/fixtures/writer-conformance/cases/empty-file.json @@ -0,0 +1,15 @@ +{ + "description": "A writer with nothing written produces the empty string, not a lone newline.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "blank" + ] + ], + "expected": [] +} diff --git a/fixtures/writer-conformance/cases/empty-line-is-blank-request.json b/fixtures/writer-conformance/cases/empty-line-is-blank-request.json new file mode 100644 index 000000000..67e46c1ab --- /dev/null +++ b/fixtures/writer-conformance/cases/empty-line-is-blank-request.json @@ -0,0 +1,36 @@ +{ + "description": "A line whose text is empty is a blank-line request, so it collapses and is dropped like one.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;" + ], + [ + "line", + "" + ], + [ + "line", + "" + ], + [ + "line", + "b;" + ], + [ + "line", + "" + ] + ], + "expected": [ + "a;", + "", + "b;" + ] +} diff --git a/fixtures/writer-conformance/cases/error-close-without-open.json b/fixtures/writer-conformance/cases/error-close-without-open.json new file mode 100644 index 000000000..b0ab38fe7 --- /dev/null +++ b/fixtures/writer-conformance/cases/error-close-without-open.json @@ -0,0 +1,20 @@ +{ + "description": "close() with no block open is an error.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;" + ], + [ + "close", + "}" + ] + ], + "error": "ERR_WRITER_UNBALANCED_CLOSE" +} diff --git a/fixtures/writer-conformance/cases/error-newline-in-line.json b/fixtures/writer-conformance/cases/error-newline-in-line.json new file mode 100644 index 000000000..61f3cad6c --- /dev/null +++ b/fixtures/writer-conformance/cases/error-newline-in-line.json @@ -0,0 +1,16 @@ +{ + "description": "line() refuses text containing a line break; each line is written separately.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;\nb;" + ] + ], + "error": "ERR_WRITER_NEWLINE_IN_LINE" +} diff --git a/fixtures/writer-conformance/cases/error-trailing-newline-in-line.json b/fixtures/writer-conformance/cases/error-trailing-newline-in-line.json new file mode 100644 index 000000000..90fc02213 --- /dev/null +++ b/fixtures/writer-conformance/cases/error-trailing-newline-in-line.json @@ -0,0 +1,16 @@ +{ + "description": "line() refuses text ending in a line break; the break is not trimmed away first.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a;\n" + ] + ], + "error": "ERR_WRITER_NEWLINE_IN_LINE" +} diff --git a/fixtures/writer-conformance/cases/error-unclosed-block.json b/fixtures/writer-conformance/cases/error-unclosed-block.json new file mode 100644 index 000000000..babc073fd --- /dev/null +++ b/fixtures/writer-conformance/cases/error-unclosed-block.json @@ -0,0 +1,29 @@ +{ + "description": "Finishing the file with a block open is an error whose message quotes the line that opened it.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "function outer() {" + ], + [ + "open", + "if (ready) {" + ], + [ + "line", + "go();" + ], + [ + "close", + "}" + ] + ], + "error": "ERR_WRITER_UNCLOSED_BLOCK", + "errorQuotes": "function outer() {" +} diff --git a/fixtures/writer-conformance/cases/indent-nested-blocks.json b/fixtures/writer-conformance/cases/indent-nested-blocks.json new file mode 100644 index 000000000..0d6da624e --- /dev/null +++ b/fixtures/writer-conformance/cases/indent-nested-blocks.json @@ -0,0 +1,38 @@ +{ + "description": "open() writes its line then indents; close() dedents then writes its line. Depth comes from nesting only.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "class A {" + ], + [ + "open", + "run() {" + ], + [ + "line", + "go();" + ], + [ + "close", + "}" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "class A {", + " run() {", + " go();", + " }", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/indent-unit-tab.json b/fixtures/writer-conformance/cases/indent-unit-tab.json new file mode 100644 index 000000000..6d5c1295d --- /dev/null +++ b/fixtures/writer-conformance/cases/indent-unit-tab.json @@ -0,0 +1,28 @@ +{ + "description": "The indent unit is an option; a tab is one unit like any other.", + "dialect": "core", + "options": { + "indent": "\t", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "func() {" + ], + [ + "line", + "x()" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "func() {", + "\tx()", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/line-endings-normalized.json b/fixtures/writer-conformance/cases/line-endings-normalized.json new file mode 100644 index 000000000..4db6490e1 --- /dev/null +++ b/fixtures/writer-conformance/cases/line-endings-normalized.json @@ -0,0 +1,20 @@ +{ + "description": "Output is always \\n. A snippet written with \\r\\n or \\r line breaks is split on them.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "text", + "a;\r\nb;\rc;" + ] + ], + "expected": [ + "a;", + "b;", + "c;" + ] +} diff --git a/fixtures/writer-conformance/cases/lines-at-root.json b/fixtures/writer-conformance/cases/lines-at-root.json new file mode 100644 index 000000000..d6687646f --- /dev/null +++ b/fixtures/writer-conformance/cases/lines-at-root.json @@ -0,0 +1,23 @@ +{ + "description": "Each line() is one line at the current indent, which starts at zero.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "const a = 1;" + ], + [ + "line", + "const b = 2;" + ] + ], + "expected": [ + "const a = 1;", + "const b = 2;" + ] +} diff --git a/fixtures/writer-conformance/cases/list-breaks-no-trailing-comma.json b/fixtures/writer-conformance/cases/list-breaks-no-trailing-comma.json new file mode 100644 index 000000000..636038c4e --- /dev/null +++ b/fixtures/writer-conformance/cases/list-breaks-no-trailing-comma.json @@ -0,0 +1,34 @@ +{ + "description": "With trailingComma off, a broken list puts no comma after its last item.", + "dialect": "core", + "options": { + "indent": " ", + "width": 30, + "trailingComma": false + }, + "ops": [ + [ + "line", + "call", + { + "list": { + "open": "(", + "items": [ + "alphaAlpha", + "betaBeta", + "gammaGamma" + ], + "close": ")" + } + }, + ";" + ] + ], + "expected": [ + "call(", + " alphaAlpha,", + " betaBeta,", + " gammaGamma", + ");" + ] +} diff --git a/fixtures/writer-conformance/cases/list-breaks-one-per-line.json b/fixtures/writer-conformance/cases/list-breaks-one-per-line.json new file mode 100644 index 000000000..6246a6a60 --- /dev/null +++ b/fixtures/writer-conformance/cases/list-breaks-one-per-line.json @@ -0,0 +1,34 @@ +{ + "description": "When the line does not fit, the list prints opener, one item per line with a comma, closer.", + "dialect": "core", + "options": { + "indent": " ", + "width": 30, + "trailingComma": true + }, + "ops": [ + [ + "line", + "call", + { + "list": { + "open": "(", + "items": [ + "alphaAlpha", + "betaBeta", + "gammaGamma" + ], + "close": ")" + } + }, + ";" + ] + ], + "expected": [ + "call(", + " alphaAlpha,", + " betaBeta,", + " gammaGamma,", + ");" + ] +} diff --git a/fixtures/writer-conformance/cases/list-empty-never-breaks.json b/fixtures/writer-conformance/cases/list-empty-never-breaks.json new file mode 100644 index 000000000..2be540eaa --- /dev/null +++ b/fixtures/writer-conformance/cases/list-empty-never-breaks.json @@ -0,0 +1,26 @@ +{ + "description": "An empty list prints as its brackets and never breaks, so a line with only empty lists stays long.", + "dialect": "core", + "options": { + "indent": " ", + "width": 30, + "trailingComma": true + }, + "ops": [ + [ + "line", + "const somethingQuiteLongIndeed = new Thing", + { + "list": { + "open": "(", + "items": [], + "close": ")" + } + }, + ";" + ] + ], + "expected": [ + "const somethingQuiteLongIndeed = new Thing();" + ] +} diff --git a/fixtures/writer-conformance/cases/list-exact-width-fits.json b/fixtures/writer-conformance/cases/list-exact-width-fits.json new file mode 100644 index 000000000..3819a7af6 --- /dev/null +++ b/fixtures/writer-conformance/cases/list-exact-width-fits.json @@ -0,0 +1,30 @@ +{ + "description": "A line exactly as long as the width fits (the width is inclusive).", + "dialect": "core", + "options": { + "indent": " ", + "width": 36, + "trailingComma": true + }, + "ops": [ + [ + "line", + "x = ", + { + "list": { + "open": "[", + "items": [ + "aaaaaaaaaa", + "bbbbbbbbbb", + "ccccc" + ], + "close": "]" + } + }, + ";" + ] + ], + "expected": [ + "x = [aaaaaaaaaa, bbbbbbbbbb, ccccc];" + ] +} diff --git a/fixtures/writer-conformance/cases/list-fits-one-line.json b/fixtures/writer-conformance/cases/list-fits-one-line.json new file mode 100644 index 000000000..3cea9ef10 --- /dev/null +++ b/fixtures/writer-conformance/cases/list-fits-one-line.json @@ -0,0 +1,30 @@ +{ + "description": "A list prints on one line, items separated by \", \", when the whole line fits the width.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "f", + { + "list": { + "open": "(", + "items": [ + "a", + "b", + "c" + ], + "close": ")" + } + }, + ";" + ] + ], + "expected": [ + "f(a, b, c);" + ] +} diff --git a/fixtures/writer-conformance/cases/list-last-list-breaks.json b/fixtures/writer-conformance/cases/list-last-list-breaks.json new file mode 100644 index 000000000..95fe911b5 --- /dev/null +++ b/fixtures/writer-conformance/cases/list-last-list-breaks.json @@ -0,0 +1,45 @@ +{ + "description": "Only the LAST non-empty list on the line breaks; earlier lists stay on the opener line.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "f", + { + "list": { + "open": "(", + "items": [ + "a", + "b" + ], + "close": ")" + } + }, + ".then", + { + "list": { + "open": "(", + "items": [ + "resultOne", + "resultTwo", + "resultThree" + ], + "close": ")" + } + }, + ";" + ] + ], + "expected": [ + "f(a, b).then(", + " resultOne,", + " resultTwo,", + " resultThree,", + ");" + ] +} diff --git a/fixtures/writer-conformance/cases/list-nested-breaks-recursively.json b/fixtures/writer-conformance/cases/list-nested-breaks-recursively.json new file mode 100644 index 000000000..a5129506e --- /dev/null +++ b/fixtures/writer-conformance/cases/list-nested-breaks-recursively.json @@ -0,0 +1,50 @@ +{ + "description": "An item that still does not fit after its list broke breaks its own last list.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "const rule = ", + { + "list": { + "open": "{", + "items": [ + "short: 1", + [ + "ops: ", + { + "list": { + "open": "[", + "items": [ + "\"equalsEquals\"", + "\"notEqualsNot\"", + "\"greaterThan\"" + ], + "close": "]" + } + } + ] + ], + "close": "}", + "pad": true + } + }, + ";" + ] + ], + "expected": [ + "const rule = {", + " short: 1,", + " ops: [", + " \"equalsEquals\",", + " \"notEqualsNot\",", + " \"greaterThan\",", + " ],", + "};" + ] +} diff --git a/fixtures/writer-conformance/cases/list-one-over-width-breaks.json b/fixtures/writer-conformance/cases/list-one-over-width-breaks.json new file mode 100644 index 000000000..34d45cc25 --- /dev/null +++ b/fixtures/writer-conformance/cases/list-one-over-width-breaks.json @@ -0,0 +1,34 @@ +{ + "description": "One column over the width breaks the list.", + "dialect": "core", + "options": { + "indent": " ", + "width": 35, + "trailingComma": true + }, + "ops": [ + [ + "line", + "x = ", + { + "list": { + "open": "[", + "items": [ + "aaaaaaaaaa", + "bbbbbbbbbb", + "ccccc" + ], + "close": "]" + } + }, + ";" + ] + ], + "expected": [ + "x = [", + " aaaaaaaaaa,", + " bbbbbbbbbb,", + " ccccc,", + "];" + ] +} diff --git a/fixtures/writer-conformance/cases/list-padded.json b/fixtures/writer-conformance/cases/list-padded.json new file mode 100644 index 000000000..22e8ffc1c --- /dev/null +++ b/fixtures/writer-conformance/cases/list-padded.json @@ -0,0 +1,44 @@ +{ + "description": "A padded list prints a space inside its brackets on one line; an empty padded list prints as its brackets.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "const a = ", + { + "list": { + "open": "{", + "items": [ + "x", + "y" + ], + "close": "}", + "pad": true + } + }, + ";" + ], + [ + "line", + "const b = ", + { + "list": { + "open": "{", + "items": [], + "close": "}", + "pad": true + } + }, + ";" + ] + ], + "expected": [ + "const a = { x, y };", + "const b = {};" + ] +} diff --git a/fixtures/writer-conformance/cases/overlong-line-without-list-kept.json b/fixtures/writer-conformance/cases/overlong-line-without-list-kept.json new file mode 100644 index 000000000..d8014b315 --- /dev/null +++ b/fixtures/writer-conformance/cases/overlong-line-without-list-kept.json @@ -0,0 +1,18 @@ +{ + "description": "A line with no list is written as given even when it exceeds the width.", + "dialect": "core", + "options": { + "indent": " ", + "width": 20, + "trailingComma": true + }, + "ops": [ + [ + "line", + "const message = \"this line is longer than the width\";" + ] + ], + "expected": [ + "const message = \"this line is longer than the width\";" + ] +} diff --git a/fixtures/writer-conformance/cases/text-blank-lines-are-requests.json b/fixtures/writer-conformance/cases/text-blank-lines-are-requests.json new file mode 100644 index 000000000..10d64c6a6 --- /dev/null +++ b/fixtures/writer-conformance/cases/text-blank-lines-are-requests.json @@ -0,0 +1,30 @@ +{ + "description": "A blank line inside a snippet is a blank-line request: runs collapse, and one at a block edge is dropped.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "{" + ], + [ + "text", + "\na;\n\n\nb;\n\n" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "{", + " a;", + "", + " b;", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/text-snippet-reindented.json b/fixtures/writer-conformance/cases/text-snippet-reindented.json new file mode 100644 index 000000000..6e6fd5427 --- /dev/null +++ b/fixtures/writer-conformance/cases/text-snippet-reindented.json @@ -0,0 +1,30 @@ +{ + "description": "text() removes a snippet's common margin, drops its first and last empty lines, and re-indents it to the current level.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "open", + "class A {" + ], + [ + "text", + "\n get x() {\n return 1;\n }\n " + ], + [ + "close", + "}" + ] + ], + "expected": [ + "class A {", + " get x() {", + " return 1;", + " }", + "}" + ] +} diff --git a/fixtures/writer-conformance/cases/trailing-whitespace-stripped.json b/fixtures/writer-conformance/cases/trailing-whitespace-stripped.json new file mode 100644 index 000000000..bbf5279f3 --- /dev/null +++ b/fixtures/writer-conformance/cases/trailing-whitespace-stripped.json @@ -0,0 +1,23 @@ +{ + "description": "Trailing spaces and tabs are never written.", + "dialect": "core", + "options": { + "indent": " ", + "width": 40, + "trailingComma": true + }, + "ops": [ + [ + "line", + "a; " + ], + [ + "line", + "b;\t" + ] + ], + "expected": [ + "a;", + "b;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-defaults.json b/fixtures/writer-conformance/cases/ts-defaults.json new file mode 100644 index 000000000..95d9504b2 --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-defaults.json @@ -0,0 +1,63 @@ +{ + "description": "The TypeScript dialect indents two spaces, breaks at 120 columns and puts a comma after the last item of a broken list.", + "dialect": "typescript", + "ops": [ + [ + "line", + "export const names = ", + { + "list": { + "open": "[", + "items": [ + "\"n00\"", + "\"n01\"", + "\"n02\"", + "\"n03\"", + "\"n04\"", + "\"n05\"", + "\"n06\"", + "\"n07\"", + "\"n08\"", + "\"n09\"", + "\"n10\"", + "\"n11\"", + "\"n12\"", + "\"n13\"", + "\"n14\"", + "\"n15\"", + "\"n16\"", + "\"n17\"", + "\"n18\"", + "\"n19\"" + ], + "close": "]" + } + }, + ";" + ] + ], + "expected": [ + "export const names = [", + " \"n00\",", + " \"n01\",", + " \"n02\",", + " \"n03\",", + " \"n04\",", + " \"n05\",", + " \"n06\",", + " \"n07\",", + " \"n08\",", + " \"n09\",", + " \"n10\",", + " \"n11\",", + " \"n12\",", + " \"n13\",", + " \"n14\",", + " \"n15\",", + " \"n16\",", + " \"n17\",", + " \"n18\",", + " \"n19\",", + "];" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-doc-comment.json b/fixtures/writer-conformance/cases/ts-doc-comment.json new file mode 100644 index 000000000..f99b457cc --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-doc-comment.json @@ -0,0 +1,34 @@ +{ + "description": "A one-entry doc comment that fits is one line; more entries, or one that does not fit, make a block. An empty entry is a bare ` *` line.", + "dialect": "typescript", + "ops": [ + [ + "doc", + "The user." + ], + [ + "line", + "a;" + ], + [ + "doc", + "First line.", + "", + "@see other" + ], + [ + "line", + "b;" + ] + ], + "expected": [ + "/** The user. */", + "a;", + "/**", + " * First line.", + " *", + " * @see other", + " */", + "b;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-header-imports-body.json b/fixtures/writer-conformance/cases/ts-header-imports-body.json new file mode 100644 index 000000000..8c872798d --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-header-imports-body.json @@ -0,0 +1,28 @@ +{ + "description": "Header lines print first, then a blank line, the imports, a blank line and the body. A file with no imports has no import block.", + "dialect": "typescript", + "ops": [ + [ + "header", + "// Generated by MetaObjects. Do not edit.", + "\"use client\";" + ], + [ + "use", + "x", + "./x" + ], + [ + "line", + "export const y = x;" + ] + ], + "expected": [ + "// Generated by MetaObjects. Do not edit.", + "\"use client\";", + "", + "import { x } from \"./x\";", + "", + "export const y = x;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-collected-and-sorted.json b/fixtures/writer-conformance/cases/ts-imports-collected-and-sorted.json new file mode 100644 index 000000000..e8f22c5fe --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-collected-and-sorted.json @@ -0,0 +1,43 @@ +{ + "description": "Imports used anywhere in the body print first: package modules, then relative ones, each in code-unit order; names in code-unit order.", + "dialect": "typescript", + "ops": [ + [ + "line", + "const x = 1;" + ], + [ + "use", + "eq", + "drizzle-orm" + ], + [ + "use", + "and", + "drizzle-orm" + ], + [ + "use", + "z", + "zod" + ], + [ + "use", + "helper", + "./helper" + ], + [ + "use", + "Alpha", + "../alpha" + ] + ], + "expected": [ + "import { and, eq } from \"drizzle-orm\";", + "import { z } from \"zod\";", + "import { Alpha } from \"../alpha\";", + "import { helper } from \"./helper\";", + "", + "const x = 1;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-conflict.json b/fixtures/writer-conformance/cases/ts-imports-conflict.json new file mode 100644 index 000000000..d3f6daa6f --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-conflict.json @@ -0,0 +1,17 @@ +{ + "description": "One local name imported from two modules is an error, not a silent shadow.", + "dialect": "typescript", + "ops": [ + [ + "use", + "Row", + "./a" + ], + [ + "use", + "Row", + "./b" + ] + ], + "error": "ERR_WRITER_IMPORT_CONFLICT" +} diff --git a/fixtures/writer-conformance/cases/ts-imports-default-namespace-side-effect.json b/fixtures/writer-conformance/cases/ts-imports-default-namespace-side-effect.json new file mode 100644 index 000000000..198f311e3 --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-default-namespace-side-effect.json @@ -0,0 +1,40 @@ +{ + "description": "Default and named imports from one module share a line; a namespace import is its own line; a side-effect import prints only when nothing else comes from that module.", + "dialect": "typescript", + "ops": [ + [ + "line", + "x;" + ], + [ + "useDefault", + "React", + "react" + ], + [ + "use", + "useState", + "react" + ], + [ + "useNamespace", + "path", + "node:path" + ], + [ + "useSideEffect", + "./polyfill" + ], + [ + "useSideEffect", + "react" + ] + ], + "expected": [ + "import * as path from \"node:path\";", + "import React, { useState } from \"react\";", + "import \"./polyfill\";", + "", + "x;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-long-list-breaks.json b/fixtures/writer-conformance/cases/ts-imports-long-list-breaks.json new file mode 100644 index 000000000..cdf9f4f02 --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-long-list-breaks.json @@ -0,0 +1,46 @@ +{ + "description": "An import whose name list does not fit the width breaks one name per line.", + "dialect": "typescript", + "ops": [ + [ + "line", + "x;" + ], + [ + "use", + "parseFilterParams", + "@metaobjectsdev/runtime-ts/drizzle-fastify" + ], + [ + "use", + "parseSortParams", + "@metaobjectsdev/runtime-ts/drizzle-fastify" + ], + [ + "use", + "buildListResponse", + "@metaobjectsdev/runtime-ts/drizzle-fastify" + ], + [ + "use", + "notFound", + "@metaobjectsdev/runtime-ts/drizzle-fastify" + ], + [ + "use", + "methodNotAllowed", + "@metaobjectsdev/runtime-ts/drizzle-fastify" + ] + ], + "expected": [ + "import {", + " buildListResponse,", + " methodNotAllowed,", + " notFound,", + " parseFilterParams,", + " parseSortParams,", + "} from \"@metaobjectsdev/runtime-ts/drizzle-fastify\";", + "", + "x;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-single-name-never-breaks.json b/fixtures/writer-conformance/cases/ts-imports-single-name-never-breaks.json new file mode 100644 index 000000000..5217513a5 --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-single-name-never-breaks.json @@ -0,0 +1,20 @@ +{ + "description": "An import of a single name stays on one line however long the module path is.", + "dialect": "typescript", + "ops": [ + [ + "line", + "x;" + ], + [ + "use", + "VeryLongExportedSymbolNameForTesting", + "@metaobjectsdev/some-really-long-package-name/with/a/deep/sub/path/module" + ] + ], + "expected": [ + "import { VeryLongExportedSymbolNameForTesting } from \"@metaobjectsdev/some-really-long-package-name/with/a/deep/sub/path/module\";", + "", + "x;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-type-only.json b/fixtures/writer-conformance/cases/ts-imports-type-only.json new file mode 100644 index 000000000..4694a225d --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-type-only.json @@ -0,0 +1,36 @@ +{ + "description": "A module whose every name is type-only prints `import type`; a module mixing both marks the type names inline.", + "dialect": "typescript", + "ops": [ + [ + "line", + "x;" + ], + [ + "useType", + "Row", + "./row" + ], + [ + "useType", + "Filter", + "./row" + ], + [ + "use", + "table", + "./table" + ], + [ + "useType", + "Cols", + "./table" + ] + ], + "expected": [ + "import type { Filter, Row } from \"./row\";", + "import { type Cols, table } from \"./table\";", + "", + "x;" + ] +} diff --git a/fixtures/writer-conformance/cases/ts-imports-value-wins.json b/fixtures/writer-conformance/cases/ts-imports-value-wins.json new file mode 100644 index 000000000..188079d4d --- /dev/null +++ b/fixtures/writer-conformance/cases/ts-imports-value-wins.json @@ -0,0 +1,25 @@ +{ + "description": "A name used both as a type and as a value is imported as a value.", + "dialect": "typescript", + "ops": [ + [ + "line", + "x;" + ], + [ + "useType", + "User", + "./user" + ], + [ + "use", + "User", + "./user" + ] + ], + "expected": [ + "import { User } from \"./user\";", + "", + "x;" + ] +} diff --git a/fixtures/writer-conformance/cases/width-counts-code-points.json b/fixtures/writer-conformance/cases/width-counts-code-points.json new file mode 100644 index 000000000..8c31c0cce --- /dev/null +++ b/fixtures/writer-conformance/cases/width-counts-code-points.json @@ -0,0 +1,29 @@ +{ + "description": "Width is counted in Unicode code points: an emoji is one, though it is two UTF-16 units.", + "dialect": "core", + "options": { + "indent": " ", + "width": 15, + "trailingComma": true + }, + "ops": [ + [ + "line", + "s = ", + { + "list": { + "open": "[", + "items": [ + "\"😀\"", + "\"b\"" + ], + "close": "]" + } + }, + ";" + ] + ], + "expected": [ + "s = [\"😀\", \"b\"];" + ] +} diff --git a/fixtures/writer-conformance/cases/width-counts-indent.json b/fixtures/writer-conformance/cases/width-counts-indent.json new file mode 100644 index 000000000..d245014cc --- /dev/null +++ b/fixtures/writer-conformance/cases/width-counts-indent.json @@ -0,0 +1,58 @@ +{ + "description": "The indent counts toward the width: the same line fits at the root and breaks inside a block.", + "dialect": "core", + "options": { + "indent": " ", + "width": 19, + "trailingComma": true + }, + "ops": [ + [ + "line", + "f", + { + "list": { + "open": "(", + "items": [ + "aaaaaa", + "bbbbbb" + ], + "close": ")" + } + }, + ";" + ], + [ + "open", + "{" + ], + [ + "line", + "f", + { + "list": { + "open": "(", + "items": [ + "aaaaaa", + "bbbbbb" + ], + "close": ")" + } + }, + ";" + ], + [ + "close", + "}" + ] + ], + "expected": [ + "f(aaaaaa, bbbbbb);", + "{", + " f(", + " aaaaaa,", + " bbbbbb,", + " );", + "}" + ] +} diff --git a/server/typescript/packages/codegen-ts/src/engine-surface.ts b/server/typescript/packages/codegen-ts/src/engine-surface.ts index 6811e137c..179308b28 100644 --- a/server/typescript/packages/codegen-ts/src/engine-surface.ts +++ b/server/typescript/packages/codegen-ts/src/engine-surface.ts @@ -9,7 +9,9 @@ // route an entity is served on, which requirements a model holds), // - the naming tables and the type-mapping tables (`names.ts`, `naming.ts`, // `column-mapper.ts`) — the ruling in Amendment 8 that an adopter keeps importing them, -// - the projection/report lowering `meta migrate` and `meta verify` also run. +// - the projection/report lowering `meta migrate` and `meta verify` also run, +// - the code writer (`CodeWriter`, `TsWriter`, ADR-0063): it lays a file out and decides +// nothing about what the file contains, so an owned generator keeps importing it. // // An owned copy imports these from the package, by its public entry. Every other module // the owned generator reaches is copied into the project, under `codegen/templates/`. @@ -69,4 +71,6 @@ export const ENGINE_MODULES: readonly string[] = [ "codegen-ts/src/templates/find-templates.ts", "codegen-ts/src/tph.ts", "codegen-ts/src/view-context.ts", + "codegen-ts/src/writer/code-writer.ts", + "codegen-ts/src/writer/ts-writer.ts", ]; diff --git a/server/typescript/packages/codegen-ts/src/engine.ts b/server/typescript/packages/codegen-ts/src/engine.ts index 39f424239..f79f498d4 100644 --- a/server/typescript/packages/codegen-ts/src/engine.ts +++ b/server/typescript/packages/codegen-ts/src/engine.ts @@ -59,4 +59,6 @@ export * from "./templates/find-inbound.js"; export * from "./templates/find-templates.js"; export * from "./tph.js"; export * from "./view-context.js"; +export * from "./writer/code-writer.js"; +export * from "./writer/ts-writer.js"; export { ENGINE_MODULES } from "./engine-surface.js"; diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index 5c837892e..0ccdd011b 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -207,6 +207,33 @@ export type { ReferenceGeneratorName } from "./reference-templates.js"; export { code, imp, joinCode } from "./poet.js"; export type { Code } from "./poet.js"; +// The code writer (ADR-0063): generators say what a line contains, the writer decides where +// it sits. No formatter runs over its output. `TsWriter` accepts the `Code` fragments above, +// so a generator can move to it one function at a time. +export { + CodeWriter, + flat, + Group, + list, + WRITER_ERRORS, + WriterError, + type CodeWriterOptions, + type Piece, + type WriterErrorCode, +} from "./writer/code-writer.js"; +export { + arr, + call, + key, + type ModuleImports, + type NamedImport, + Props, + props, + str, + TsWriter, + type TsWriterOptions, +} from "./writer/ts-writer.js"; + export { renderFindByIdFn, renderListFn, diff --git a/server/typescript/packages/codegen-ts/src/writer/code-writer.ts b/server/typescript/packages/codegen-ts/src/writer/code-writer.ts new file mode 100644 index 000000000..ff8b03c4d --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/writer/code-writer.ts @@ -0,0 +1,274 @@ +// The language-neutral code writer (ADR-0063; contract: spec/code-writer.md). +// +// A generator says WHAT a line contains; the writer decides WHERE it sits. It builds the +// file in memory and returns a string, so nothing here touches the disk and no formatter +// ever runs over the result. It never parses the text it is handed: it lays out the +// structure the generator gave it — lines, blocks, blank-line requests and breakable lists. +// +// This module and the dialects beside it are the ONLY codegen source allowed to type +// layout (newlines, indentation). The layout ratchet test (test/layout-ratchet/) holds +// every other generator module to that. + +/** Error codes a writer throws. Shared by every port's writer (spec/code-writer.md). */ +export const WRITER_ERRORS = { + /** `close()` with no block open. */ + unbalancedClose: "ERR_WRITER_UNBALANCED_CLOSE", + /** The file was finished with a block still open; the message quotes the line that opened it. */ + unclosedBlock: "ERR_WRITER_UNCLOSED_BLOCK", + /** `line()` was handed text containing a line break. */ + newlineInLine: "ERR_WRITER_NEWLINE_IN_LINE", + /** One local name imported from two different places. */ + importConflict: "ERR_WRITER_IMPORT_CONFLICT", + /** A piece the writer does not know how to print. */ + unknownPiece: "ERR_WRITER_UNKNOWN_PIECE", +} as const; +export type WriterErrorCode = (typeof WRITER_ERRORS)[keyof typeof WRITER_ERRORS]; + +export class WriterError extends Error { + constructor( + readonly code: WriterErrorCode, + message: string, + ) { + super(`${code}: ${message}`); + this.name = "WriterError"; + } +} + +/** + * A bracketed, comma-separated list. Printed on one line when the line fits the width; + * otherwise opener, one item per line (each followed by a comma) and closer. + */ +export class Group { + constructor( + readonly open: string, + readonly items: readonly Piece[], + readonly close: string, + /** `{ a, b }` (true) versus `[a, b]` or `f(a, b)` (false) when printed on one line. */ + readonly pad = false, + ) {} +} + +/** An expression fragment: text, a breakable list, or a concatenation of fragments. */ +export type Piece = string | Group | readonly Piece[]; + +/** A list with no padding inside its brackets: `[a, b]`, `f(a, b)`, ``. */ +export const list = (open: string, items: readonly Piece[], close: string): Group => new Group(open, items, close); + +export interface CodeWriterOptions { + /** One level of indentation. Default two spaces. */ + readonly indent?: string; + /** The column a line may reach before its last list breaks. Default 120. */ + readonly width?: number; + /** Whether a broken list puts a comma after its last item too. Default true. */ + readonly trailingComma?: boolean; +} + +const NEWLINE = "\n"; +const ANY_LINE_BREAK = /\r\n?|\n/; + +/** Character width of `s`, in Unicode code points. */ +function widthOf(s: string): number { + // A fast path: generated code is almost always ASCII, where length is the answer. + return /^[\x00-\x7f]*$/.test(s) ? s.length : [...s].length; +} + +/** + * The core writer. Built to be extended (ADR-0063): every method is public or protected, so a + * dialect or an adopter's subclass overrides the one step it needs to change, such as where a + * line breaks (`breakAt`) or what a part becomes (`piece`). These methods are not a stability + * contract and may change between minor versions. + */ +export class CodeWriter { + readonly indent: string; + readonly width: number; + readonly trailingComma: boolean; + + protected readonly out: string[] = []; + /** The text of each line that opened a block still open, outermost first. */ + protected readonly openers: string[] = []; + protected readonly prefixes: string[] = [""]; + protected wantBlank = false; + /** True at the start of the file and right after a block opens: no blank line goes here. */ + protected atBlockStart = true; + + constructor(options: CodeWriterOptions = {}) { + this.indent = options.indent ?? " "; + this.width = options.width ?? 120; + this.trailingComma = options.trailingComma ?? true; + } + + /** + * One line at the current indent. The parts are concatenated; when the line does not fit + * the width, the LAST non-empty list in it is printed one item per line. A line whose + * text is empty is a blank-line request. + */ + line(...parts: readonly unknown[]): this { + return this.put(this.piece(parts)); + } + + /** Ask for one blank line before whatever is written next. Requests never stack. */ + blank(): this { + this.wantBlank = true; + return this; + } + + /** Write the line that opens a block, then indent. */ + open(...parts: readonly unknown[]): this { + return this.openPiece(this.piece(parts)); + } + + /** Dedent, then write the line that closes the block. */ + close(...parts: readonly unknown[]): this { + return this.closePiece(this.piece(parts)); + } + + /** `open(head)`, the body, then `close(tail)`: the indent cannot be left unbalanced. */ + block(head: unknown, body: () => void, tail: unknown = "}"): this { + this.open(head); + body(); + return this.close(tail); + } + + /** + * A multi-line snippet written as it should read. Its common left margin is removed and + * it is re-indented to the current level; a blank line in it is a blank-line request. + * Every line is re-indented, including lines inside a multi-line string literal. + */ + text(snippet: string): this { + const lines = snippet.split(ANY_LINE_BREAK); + if (lines.length > 1 && lines[0]!.trim() === "") lines.shift(); + if (lines.length > 0 && lines[lines.length - 1]!.trim() === "") lines.pop(); + let margin = Number.POSITIVE_INFINITY; + for (const l of lines) { + if (l.trim() !== "") margin = Math.min(margin, l.length - l.trimStart().length); + } + for (const l of lines) { + if (l.trim() === "") this.blank(); + else this.put(l.slice(margin)); + } + return this; + } + + /** The body text: every line joined, with no final newline. Throws if a block is still open. */ + protected body(): string { + if (this.openers.length > 0) { + throw new WriterError(WRITER_ERRORS.unclosedBlock, `unclosed block opened by: ${this.openers[this.openers.length - 1]!}`); + } + return this.out.join(NEWLINE); + } + + /** The finished file: `\n` line endings, no trailing whitespace, one final newline (none when empty). */ + toString(): string { + const body = this.body(); + return body === "" ? "" : body + NEWLINE; + } + + /** + * Turn one argument of `line()` into a writer piece. A dialect overrides this to accept + * its own fragment types (TsWriter accepts ts-poet `Code`), and calls `super` for the rest. + */ + protected piece(p: unknown): Piece { + if (typeof p === "string") return p; + if (p instanceof Group) { + const items = p.items.map((i) => this.piece(i)); + return items.every((i, n) => i === p.items[n]) ? p : new Group(p.open, items, p.close, p.pad); + } + if (Array.isArray(p)) return p.map((x) => this.piece(x)); + throw new WriterError(WRITER_ERRORS.unknownPiece, `cannot write ${describe(p)}`); + } + + /** Whether `text` fits on one line at the current indent. */ + protected fits(text: string): boolean { + return widthOf(this.prefix()) + widthOf(text) <= this.width; + } + + /** + * Where a line that does not fit breaks: [text before, the list that breaks, text after], or + * undefined to print the line as it is. The default is the line's LAST non-empty list. + */ + protected breakAt(piece: Piece): [string, Group, string] | undefined { + return lastGroup(piece); + } + + protected openPiece(piece: Piece): this { + this.put(piece); + this.openers.push(flat(piece).trim()); + this.atBlockStart = true; + return this; + } + + protected closePiece(piece: Piece): this { + if (this.openers.length === 0) { + throw new WriterError(WRITER_ERRORS.unbalancedClose, `close(${JSON.stringify(flat(piece))}) with no block open`); + } + this.openers.pop(); + this.wantBlank = false; + return this.put(piece); + } + + protected prefix(): string { + const depth = this.openers.length; + while (this.prefixes.length <= depth) this.prefixes.push(this.prefixes[this.prefixes.length - 1]! + this.indent); + return this.prefixes[depth]!; + } + + protected put(piece: Piece): this { + const raw = flat(piece); + if (ANY_LINE_BREAK.test(raw)) { + throw new WriterError( + WRITER_ERRORS.newlineInLine, + `line text contains a line break; write each line separately or use text(): ${JSON.stringify(raw)}`, + ); + } + const text = raw.trimEnd(); + if (text === "") return this.blank(); + if (this.wantBlank && !this.atBlockStart) this.out.push(""); + this.wantBlank = false; + this.atBlockStart = false; + const pre = this.prefix(); + const split = this.fits(text) ? undefined : this.breakAt(piece); + if (split === undefined) { + this.out.push(pre + text); + return this; + } + const [before, group, after] = split; + this.openPiece(before + group.open); + group.items.forEach((item, n) => { + const comma = n < group.items.length - 1 || this.trailingComma ? "," : ""; + this.put([item, comma]); + }); + return this.closePiece(group.close + after); + } +} + +/** A piece printed on one line. */ +export function flat(p: Piece): string { + if (typeof p === "string") return p; + if (p instanceof Group) { + if (p.items.length === 0) return p.open + p.close; + const inner = p.items.map(flat).join(", "); + return p.pad ? `${p.open} ${inner} ${p.close}` : p.open + inner + p.close; + } + let s = ""; + for (const x of p) s += flat(x); + return s; +} + +/** Split a piece around its LAST non-empty list: [text before, the list, text after]. */ +function lastGroup(p: Piece): [string, Group, string] | undefined { + if (typeof p === "string") return undefined; + if (p instanceof Group) return p.items.length === 0 ? undefined : ["", p, ""]; + for (let i = p.length - 1; i >= 0; i--) { + const hit = lastGroup(p[i]!); + if (hit !== undefined) { + return [flat(p.slice(0, i)) + hit[0], hit[1], hit[2] + flat(p.slice(i + 1))]; + } + } + return undefined; +} + +function describe(p: unknown): string { + if (p === null || p === undefined) return String(p); + if (typeof p === "object") return `an object (${p.constructor?.name ?? "no constructor"})`; + return `a ${typeof p}`; +} diff --git a/server/typescript/packages/codegen-ts/src/writer/ts-writer.ts b/server/typescript/packages/codegen-ts/src/writer/ts-writer.ts new file mode 100644 index 000000000..972b7455f --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/writer/ts-writer.ts @@ -0,0 +1,258 @@ +// The TypeScript dialect of the code writer (ADR-0063; contract: spec/code-writer.md). +// +// Adds what is TypeScript's own: imports collected while the body is written and printed +// first, string quoting, object literals and doc comments. It also accepts `Code` fragments +// (src/poet.ts, the vendored ts-poet printer), harvesting their `imp()` imports, so a generator +// can move to the writer one function at a time while the rest of it still returns `Code`. + +import { Code, Import, ImportsAll, ImportsDefault, ImportsName, Node, SideEffect } from "../poet.js"; +import { CodeWriter, type CodeWriterOptions, flat, Group, type Piece, WRITER_ERRORS, WriterError } from "./code-writer.js"; + +/** A correctly quoted TypeScript string literal. */ +export const str = (value: string): string => JSON.stringify(value); + +/** An array literal: `[a, b]`. */ +export const arr = (items: readonly unknown[]): Group => new Group("[", items as Piece[], "]"); + +/** A call: `fn(a, b)`. */ +export const call = (fn: string, args: readonly unknown[]): Group => new Group(`${fn}(`, args as Piece[], ")"); + +const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/; + +/** An object key: bare when it is an identifier, quoted otherwise. */ +export const key = (name: string): string => (IDENTIFIER.test(name) ? name : str(name)); + +/** An object literal under construction; members print in the order they are set. */ +export class Props extends Group { + protected readonly entries: Piece[]; + constructor() { + const entries: Piece[] = []; + super("{", entries, "}", true); + this.entries = entries; + } + /** `name: value`. The name is written as given; use `key()` for one that may need quotes. */ + set(name: string, value: unknown): this { + this.entries.push([name, ": ", value as Piece]); + return this; + } + /** A member written whole: a shorthand `name`, a spread `...rest` or a method. */ + add(member: unknown): this { + this.entries.push(member as Piece); + return this; + } +} + +/** A new, empty object literal. */ +export const props = (): Props => new Props(); + +/** One named import, keyed by its local name. */ +export interface NamedImport { + readonly exported: string; + typeOnly: boolean; +} + +/** Everything imported from one module. */ +export interface ModuleImports { + readonly named: Map; + defaultName?: string; + readonly namespaces: Set; + sideEffect: boolean; +} + +export interface TsWriterOptions extends CodeWriterOptions {} + +/** + * The TypeScript writer. Like `CodeWriter` it is built to be extended: each section of the + * file (`writeHeader`, `writeImports`, `writeImport`, `orderModules`) is its own overridable + * method, and none of them is a stability contract. + */ +export class TsWriter extends CodeWriter { + protected readonly modules = new Map(); + /** Local name -> the module it was imported from, so one name cannot come from two. */ + protected readonly locals = new Map(); + protected readonly headLines: string[] = []; + + constructor(options: TsWriterOptions = {}) { + super({ indent: " ", width: 120, trailingComma: true, ...options }); + } + + /** Reference a value import; returns the name to write. */ + use(name: string, from: string): string { + return this.addNamed(name, name, from, false); + } + + /** Reference a type-only import; returns the name to write. */ + useType(name: string, from: string): string { + return this.addNamed(name, name, from, true); + } + + /** Reference a module's default export under `name`; returns the name to write. */ + useDefault(name: string, from: string): string { + this.claim(name, from); + const m = this.module(from); + if (m.defaultName !== undefined && m.defaultName !== name) { + throw new WriterError(WRITER_ERRORS.importConflict, `default of ${str(from)} imported as both ${m.defaultName} and ${name}`); + } + m.defaultName = name; + return name; + } + + /** Reference a whole module as `import * as name`; returns the name to write. */ + useNamespace(name: string, from: string): string { + this.claim(name, from); + this.module(from).namespaces.add(name); + return name; + } + + /** Import a module for its side effects only: `import "from";`. */ + useSideEffect(from: string): void { + this.module(from).sideEffect = true; + } + + /** Lines printed at the very top, before the imports: a banner comment or a `"use client";` directive. */ + header(...lines: readonly string[]): this { + this.headLines.push(...lines); + return this; + } + + /** A doc comment: one line when it fits, otherwise a block with one ` * ` line per entry. */ + doc(...lines: readonly string[]): this { + if (lines.length === 0) return this; + const safe = lines.map((l) => l.replaceAll("*/", "*\\/")); + const one = `/** ${safe[0]} */`; + if (safe.length === 1 && this.fits(one)) return this.line(one); + this.line("/**"); + for (const l of safe) this.line(l === "" ? " *" : ` * ${l}`); + return this.line(" */"); + } + + /** A ts-poet `Code` fragment, possibly many lines: its imports are collected, its text re-indented. */ + code(fragment: Code): this { + return this.text(fragment instanceof Code ? this.harvest(fragment) : flat(this.piece(fragment))); + } + + override toString(): string { + const head = this.sectionWriter(); + this.writeHeader(head); + head.blank(); + this.writeImports(head); + const top = head.toString(); + const body = super.toString(); + if (top === "") return body; + if (body === "") return top; + return `${top}\n${body}`; + } + + /** The writer the header and import sections are laid out with: same options, no dialect. */ + protected sectionWriter(): CodeWriter { + return new CodeWriter({ indent: this.indent, width: this.width, trailingComma: this.trailingComma }); + } + + /** The `header(...)` lines, as given. */ + protected writeHeader(w: CodeWriter): void { + for (const l of this.headLines) w.line(l); + } + + /** Every collected import, one module at a time in `orderModules` order. */ + protected writeImports(w: CodeWriter): void { + for (const from of this.orderModules([...this.modules.keys()])) this.writeImport(w, from, this.modules.get(from)!); + } + + /** Package imports first, then relative ones; each group in code-unit order. */ + protected orderModules(modules: readonly string[]): string[] { + const relative = (m: string) => m.startsWith("."); + return [...modules.filter((m) => !relative(m)).sort(byCodeUnit), ...modules.filter(relative).sort(byCodeUnit)]; + } + + protected override piece(p: unknown): Piece { + if (p instanceof Code) return this.harvest(p); + if (p instanceof Import) { + this.record(p); + return p.toCodeString(); + } + if (looksLikeForeignPoet(p)) { + // Two physical copies of the code primitives in one process (a linked or global CLI next + // to a project dependency, or the old ts-poet package): the fragment's imports cannot be + // read, so refuse rather than drop them. + throw new WriterError( + WRITER_ERRORS.unknownPiece, + "a Code or Import built by a different copy of the code primitives; import code and imp from @metaobjectsdev/codegen-ts", + ); + } + return super.piece(p); + } + + protected harvest(fragment: Code): string { + const todo: unknown[] = [fragment]; + for (let i = 0; i < todo.length; i++) { + const n = todo[i]; + if (n instanceof Import) this.record(n); + else if (n instanceof Node) todo.push(...n.childNodes); + else if (Array.isArray(n)) todo.push(...n); + } + return fragment.toCodeString(); + } + + protected record(i: Import): void { + if (i instanceof ImportsName) this.addNamed(i.sourceSymbol ?? i.symbol, i.symbol, i.source, i.typeImport === true); + else if (i instanceof ImportsDefault) this.useDefault(i.symbol, i.source); + else if (i instanceof ImportsAll) this.useNamespace(i.symbol, i.source); + else if (i instanceof SideEffect) this.useSideEffect(i.source); + // An implicit import (`imp("Foo")` with no module) names a global and imports nothing. + } + + protected addNamed(exported: string, local: string, from: string, typeOnly: boolean): string { + this.claim(local, from); + const named = this.module(from).named; + const seen = named.get(local); + if (seen === undefined) named.set(local, { exported, typeOnly }); + else if (seen.exported !== exported) { + throw new WriterError(WRITER_ERRORS.importConflict, `${local} from ${str(from)} names both ${seen.exported} and ${exported}`); + } else seen.typeOnly &&= typeOnly; + return local; + } + + protected claim(local: string, from: string): void { + const owner = this.locals.get(local); + if (owner !== undefined && owner !== from) { + throw new WriterError(WRITER_ERRORS.importConflict, `${local} is imported from both ${str(owner)} and ${str(from)}`); + } + this.locals.set(local, from); + } + + protected module(from: string): ModuleImports { + let m = this.modules.get(from); + if (m === undefined) { + m = { named: new Map(), namespaces: new Set(), sideEffect: false }; + this.modules.set(from, m); + } + return m; + } + + protected writeImport(w: CodeWriter, from: string, m: ModuleImports): void { + const tail = ` from ${str(from)};`; + for (const ns of [...m.namespaces].sort(byCodeUnit)) w.line(`import * as ${ns}${tail}`); + const names = [...m.named].sort(([a], [b]) => byCodeUnit(a, b)); + const typeOnly = m.defaultName === undefined && names.length > 0 && names.every(([, n]) => n.typeOnly); + const items = names.map(([local, n]) => { + const spec = n.exported === local ? local : `${n.exported} as ${local}`; + return n.typeOnly && !typeOnly ? `type ${spec}` : spec; + }); + const lead = typeOnly ? "import type " : "import "; + if (items.length > 0) { + // One name never breaks: `import { X } from "a/long/module"` stays on one line. + const braces: Piece = items.length === 1 ? `{ ${items[0]} }` : new Group("{", items, "}", true); + w.line(lead, m.defaultName === undefined ? "" : `${m.defaultName}, `, braces, tail); + } else if (m.defaultName !== undefined) { + w.line(`import ${m.defaultName}${tail}`); + } else if (m.namespaces.size === 0 && m.sideEffect) { + w.line(`import ${str(from)};`); + } + } +} + +function looksLikeForeignPoet(p: unknown): boolean { + return typeof p === "object" && p !== null && "toCodeString" in p && "childNodes" in p; +} + +const byCodeUnit = (a: string, b: string): number => (a < b ? -1 : a > b ? 1 : 0); diff --git a/server/typescript/packages/codegen-ts/test/layout-ratchet.test.ts b/server/typescript/packages/codegen-ts/test/layout-ratchet.test.ts new file mode 100644 index 000000000..e32dea921 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/layout-ratchet.test.ts @@ -0,0 +1,276 @@ +// The layout ratchet (ADR-0063): generator source never types layout; the writer does. +// +// Three parts: +// 1. the analyzer's own cases — layout it must catch, and writer-based code it must pass; +// 2. the scope rule — which files are generator source, by rule, not by list; +// 3. the ratchet over the real packages — every file with hand-typed layout is on the +// allowlist with a reason, every allowlisted file still needs its entry, and the +// allowlist only shrinks. + +import { describe, expect, test } from "bun:test"; +import { existsSync, readdirSync, readFileSync, statSync } from "node:fs"; +import { join, relative, resolve } from "node:path"; +import { analyzeSource, type FindingKind, isGeneratorSource, SCANNED_PACKAGES } from "./layout-ratchet/analyze.js"; + +const kinds = (source: string, file = "fixture.ts"): FindingKind[] => analyzeSource(file, source).map((f) => f.kind); + +// ── 1. the analyzer ────────────────────────────────────────────────────────────────────── + +describe("the analyzer catches hand-typed layout", () => { + const caught: [string, string, FindingKind[]][] = [ + ["a newline escape in a string", `const s = "a\\nb";`, ["newline-literal"]], + ["a hex newline escape", `const s = "a\\x0ab";`, ["newline-literal"]], + ["a carriage return", `const s = "a\\r";`, ["newline-literal"]], + ["a line break inside a template literal", "const s = `a\nb`;", ["newline-literal"]], + ["a line break in a ts-poet code template", "const c = code`\nexport const ${name} = 1;\n`;", ["newline-literal", "newline-literal"]], + ["a line break after a substitution", "const s = `${a}\n${b}`;", ["newline-literal"]], + ["a string that starts with two spaces", `rows.push(" " + name);`, ["indent-literal"]], + ["a string that starts with a tab", `rows.push("\\tx");`, ["indent-literal"]], + ["a template that starts with an indentation run", "rows.push(` ${name}: string;`);", ["indent-literal"]], + ["a join on a newline literal", `out.join("\\n");`, ["newline-join", "newline-literal"]], + ["a join on a comma-newline literal", `rows.join(",\\n");`, ["newline-join", "newline-literal"]], + ["a joinCode with a newline separator", `joinCode(parts, { on: "\\n" });`, ["newline-join", "newline-literal"]], + ["an indentation repeat", `const pad = " ".repeat(depth);`, ["indent-repeat", "indent-literal"]], + ["a single-space repeat", `const pad = " ".repeat(depth * 2);`, ["indent-repeat"]], + ]; + for (const [name, source, expected] of caught) { + test(name, () => expect(kinds(source).sort()).toEqual([...expected].sort())); + } + + // A constant cannot hide a line break: the declaration is a finding, AND so is every use. + test("a newline hidden behind a same-file constant is caught where it is used", () => { + const source = ["const NL = \"\\n\";", "export const out = lines.join(NL);"].join("\n"); + const found = analyzeSource("fixture.ts", source); + expect(found.map((f) => [f.kind, f.line])).toEqual([ + ["newline-literal", 1], + ["newline-join", 2], + ]); + }); + + test("the constant may be built from other constants or held in an object", () => { + const source = [ + "const NL = \"\\n\";", + "const SEP = \",\" + NL;", + "const LAYOUT = { rowSep: `${SEP}` } as const;", + "rows.join(SEP);", + "rows.join(LAYOUT.rowSep);", + "joinCode(chunks, { on: NL });", + ].join("\n"); + expect(analyzeSource("fixture.ts", source).filter((f) => f.kind === "newline-join").map((f) => f.line)).toEqual([4, 5, 6]); + }); + + test("an indentation unit hidden behind a constant is caught at the repeat", () => { + const source = ["const UNIT = \" \";", "const pad = UNIT.repeat(4);"].join("\n"); + expect(kinds(source)).toEqual(["indent-repeat"]); + }); + + test("either branch of a conditional separator counts", () => { + expect(kinds(`rows.join(compact ? ", " : ",\\n");`).sort()).toEqual(["newline-join", "newline-literal"]); + }); + + test("findings carry the line and column of the node", () => { + const [f] = analyzeSource("fixture.ts", ["const a = 1;", " const s = `x", "y`;"].join("\n")); + expect([f!.line, f!.column]).toEqual([2, 13]); + }); +}); + +describe("the analyzer passes code that leaves layout to the writer", () => { + test("a generator written on TsWriter has no findings", () => { + const source = ` + import { props, arr, str, TsWriter } from "@metaobjectsdev/codegen-ts/engine"; + + // Comments may say anything, including "\\n" and + // indented text: a comment is never a node. + /* + * a block comment, indented + */ + export function renderAllowlist(entity: MetaObject): string { + const w = new TsWriter(); + const FilterAllowlist = w.useType("FilterAllowlist", "@metaobjectsdev/runtime-ts"); + w.block(\`export const \${entity.name}FilterAllowlist = {\`, () => { + for (const f of entity.fields()) { + w.line(\`\${f.name}: \`, props().set("ops", [arr(ops(f).map(str)), " as const"]), ","); + } + }, \`} as const satisfies \${FilterAllowlist};\`); + w.blank(); + w.text(\` + export function check() { + return true; + } + \`); + return w.toString(); + } + `; + expect(analyzeSource("fixture.ts", source)).toEqual([]); + }); + + test("a line break used as a search pattern is not output", () => { + const source = [ + "const parts = description.split(\"\\n\");", + "if (body.includes(\"\\n\")) flag();", + "const flat = body.replace(\"\\r\\n\", \" \").replaceAll(\"\\n\", \" \");", + "const at = body.indexOf(\" \");", + ].join("\n"); + expect(analyzeSource("fixture.ts", source)).toEqual([]); + }); + + test("a replacement that writes a line break is still output", () => { + expect(kinds(`const s = body.replace(";", ";\\n");`)).toEqual(["newline-literal"]); + }); + + test("type positions are not values", () => { + expect(kinds(`type Sep = "\\n" | " ";\nlet s: "\\n";`)).toEqual([]); + }); + + test("single spaces, joins on a space and spacing after a substitution are not layout", () => { + const source = ["const a = names.join(\", \");", "const b = `${left} ${right}`;", "const c = \" \" + x;"].join("\n"); + expect(analyzeSource("fixture.ts", source)).toEqual([]); + }); + + test("a parameter shadowing a layout constant is not resolved to it", () => { + const source = ["function f(SEP: string) {", " return rows.join(SEP);", "}", "const SEP = \", \";"].join("\n"); + expect(analyzeSource("fixture.ts", source)).toEqual([]); + }); +}); + +// ── 2. the scope rule ──────────────────────────────────────────────────────────────────── + +describe("which files are generator source", () => { + const cases: [string, boolean][] = [ + ["codegen-ts/src/templates/routes-file.ts", true], + ["codegen-ts-react/src/templates/form-file.ts", true], + ["codegen-ts-angular/src/templates/grid-file.ts", true], + ["codegen-ts/src/reference/routes.ts", true], + ["codegen-ts/src/writer/code-writer.ts", false], + ["codegen-ts/src/writer/ts-writer.ts", false], + ["codegen-ts/test/routes.test.ts", false], + ["codegen-ts/src/templates/routes.test.ts", false], + ["codegen-ts/src/test/helper.ts", false], + ["codegen-ts/src/render-engine/embedded-templates.generated.ts", false], + ["codegen-ts/src/index.d.ts", false], + ["runtime-ts/src/filter.ts", false], + ["codegen-ts/README.md", false], + ]; + for (const [path, scanned] of cases) { + test(`${path} is ${scanned ? "" : "not "}scanned`, () => expect(isGeneratorSource(path)).toBe(scanned)); + } +}); + +// ── 3. the ratchet over the real packages ──────────────────────────────────────────────── + +const PACKAGES = resolve(import.meta.dir, "../.."); +const REPO = resolve(PACKAGES, "../../.."); +const ALLOWLIST_PATH = resolve(import.meta.dir, "layout-ratchet/allowlist.json"); +const ALLOWLIST_REL = relative(REPO, ALLOWLIST_PATH).replaceAll("\\", "/"); + +interface Allowlist { + unconverted: Record; + exempt: Record; +} + +const allowlist = JSON.parse(readFileSync(ALLOWLIST_PATH, "utf8")) as Allowlist; +const entries = { ...allowlist.unconverted, ...allowlist.exempt }; + +function walk(dir: string): string[] { + const out: string[] = []; + for (const name of readdirSync(dir)) { + if (name === "node_modules" || name === "dist") continue; + const full = join(dir, name); + if (statSync(full).isDirectory()) out.push(...walk(full)); + else out.push(full); + } + return out; +} + +const scanned = SCANNED_PACKAGES.flatMap((p) => walk(join(PACKAGES, p, "src"))) + .map((f) => relative(PACKAGES, f).replaceAll("\\", "/")) + .filter(isGeneratorSource) + .sort(); +const findings = new Map(scanned.map((f) => [f, analyzeSource(f, readFileSync(join(PACKAGES, f), "utf8"))])); + +describe("the layout ratchet", () => { + test("it reads every scanned package", () => { + for (const p of SCANNED_PACKAGES) expect(scanned.some((f) => f.startsWith(`${p}/`)), p).toBe(true); + }); + + test("no generator source types layout unless its file is allowlisted", () => { + const offenders = [...findings] + .filter(([file, found]) => found.length > 0 && !(file in entries)) + .map(([file, found]) => [file, ...found.map((f) => ` ${f.line}:${f.column} ${f.kind} ${f.excerpt}`)].join("\n")); + expect( + offenders, + "Write these lines through TsWriter (src/writer/) instead of typing line breaks or indentation. " + + "A file is never added to the allowlist: it only shrinks.", + ).toEqual([]); + }); + + test("every allowlisted file still needs its entry", () => { + const stale = Object.keys(entries).filter((file) => (findings.get(file)?.length ?? 0) === 0); + expect(stale, "these files no longer type layout (or no longer exist): remove their allowlist entries").toEqual([]); + }); + + test("every allowlisted path is generator source that exists", () => { + for (const file of Object.keys(entries)) { + expect(isGeneratorSource(file), `${file} is not in the ratchet's scope`).toBe(true); + expect(existsSync(join(PACKAGES, file)), `${file} does not exist`).toBe(true); + } + }); + + test("every entry carries a written reason", () => { + for (const [file, reason] of Object.entries(entries)) expect(reason.length, `${file} needs a reason`).toBeGreaterThan(40); + }); + + test("a file is either unconverted or exempt, never both, and each list is sorted", () => { + expect(Object.keys(allowlist.unconverted).filter((f) => f in allowlist.exempt)).toEqual([]); + for (const list of [allowlist.unconverted, allowlist.exempt]) expect(Object.keys(list)).toEqual(Object.keys(list).sort()); + }); + + // The allowlist may only shrink. Compared with the copy at the merge base of every + // integration branch this checkout knows: an entry that is not there was added here. + const bases = mergeBaseAllowlists(); + test.skipIf(bases === undefined)("the allowlist only shrinks", () => { + for (const { ref, list } of bases!) { + expect(growth(list, allowlist), `the allowlist grew since the merge base with ${ref}; convert the file to the writer instead`).toEqual([]); + } + }); + + test("growth is an added entry, or an unconverted file relabelled exempt; removal is not", () => { + const before: Allowlist = { unconverted: { "a.ts": "r", "b.ts": "r" }, exempt: { "c.ts": "r" } }; + expect(growth(before, { unconverted: { "a.ts": "r" }, exempt: {} })).toEqual([]); + expect(growth(before, { unconverted: { "a.ts": "r", "d.ts": "r" }, exempt: { "c.ts": "r" } })).toEqual(["added d.ts"]); + expect(growth(before, { unconverted: { "a.ts": "r" }, exempt: { "b.ts": "r", "c.ts": "r" } })).toEqual(["relabelled b.ts as exempt"]); + }); +}); + +/** How `now` grew relative to `before`: entries that are new, and unconverted files relabelled exempt. */ +function growth(before: Allowlist, now: Allowlist): string[] { + const had = new Set([...Object.keys(before.unconverted), ...Object.keys(before.exempt ?? {})]); + return [ + ...Object.keys({ ...now.unconverted, ...now.exempt }).filter((f) => !had.has(f)).map((f) => `added ${f}`), + ...Object.keys(now.exempt).filter((f) => f in before.unconverted).map((f) => `relabelled ${f} as exempt`), + ]; +} + +/** + * The allowlist as it stood at the merge base of HEAD with each integration branch + * (`origin/main`, `origin/release/*`). `undefined` when git or those refs are unavailable + * (an export or a shallow clone), in which case the shrink check is skipped and says so. + */ +function mergeBaseAllowlists(): { ref: string; list: Allowlist }[] | undefined { + const git = (...args: string[]): string | undefined => { + const r = Bun.spawnSync(["git", "-C", REPO, ...args], { stdout: "pipe", stderr: "ignore" }); + return r.exitCode === 0 ? r.stdout.toString().trim() : undefined; + }; + const refs = git("for-each-ref", "--format=%(refname:short)", "refs/remotes/origin/main", "refs/remotes/origin/release/"); + if (refs === undefined || refs === "") return undefined; + const out: { ref: string; list: Allowlist }[] = []; + let reachable = false; + for (const ref of refs.split(/\s+/)) { + const base = git("merge-base", "HEAD", ref); + if (base === undefined) continue; + reachable = true; + const text = git("show", `${base}:${ALLOWLIST_REL}`); + if (text !== undefined) out.push({ ref, list: JSON.parse(text) as Allowlist }); + } + return reachable ? out : undefined; +} diff --git a/server/typescript/packages/codegen-ts/test/layout-ratchet/allowlist.json b/server/typescript/packages/codegen-ts/test/layout-ratchet/allowlist.json new file mode 100644 index 000000000..d4db4dd03 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/layout-ratchet/allowlist.json @@ -0,0 +1,88 @@ +{ + "$comment": "The layout ratchet's allowlist (ADR-0063). `unconverted`: generator source that still types its own layout; each entry names what it emits and the writer PR that converts it, and is removed when that PR lands (the test fails while a converted file keeps its entry). `exempt`: source that writes text which is not generated code; each says why. Neither list may grow: test/layout-ratchet.test.ts compares it with the copy at the branch's merge base.", + "unconverted": { + "codegen-ts-angular/src/generators/barrel.ts": "Emits the Angular barrel; template strings with hand-typed line breaks and indentation. Converts with the TanStack and Angular writer PR.", + "codegen-ts-angular/src/templates/form-file.ts": "Emits the Angular form component; template strings with hand-typed line breaks and indentation. Converts with the TanStack and Angular writer PR.", + "codegen-ts-angular/src/templates/grid-file.ts": "Emits the Angular grid component; template strings with hand-typed line breaks and indentation. Converts with the TanStack and Angular writer PR.", + "codegen-ts-angular/src/templates/service-file.ts": "Emits the Angular data service; template strings with hand-typed line breaks and indentation. Converts with the TanStack and Angular writer PR.", + "codegen-ts-react/src/templates/form-file.ts": "Emits the React form (.tsx) with JSX typed at fixed depths inside strings. Converts with the React form writer PR (JSX).", + "codegen-ts-tanstack/src/templates/columns-file.ts": "Emits the TanStack grid column definitions; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the TanStack and Angular writer PR.", + "codegen-ts-tanstack/src/templates/grid-hook-file.ts": "Emits the TanStack grid hook; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the TanStack and Angular writer PR.", + "codegen-ts-tanstack/src/templates/hooks-file.ts": "Emits the TanStack Query hooks; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the TanStack and Angular writer PR.", + "codegen-ts/src/api-surface.ts": "Builds the writeDeniedColumns and API-surface lines of the generated routes with an indent string passed in by the caller. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/client-directive.ts": "Prepends the \"use client\" directive to client artifacts by string concatenation. Converts with the TanStack and Angular writer PR, when client artifacts use TsWriter.header().", + "codegen-ts/src/constants.ts": "Builds the sidecar comment line of generated files with a typed line break. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/cube/cube-yaml.ts": "Emits the Cube data-model YAML with hand-typed indentation. Converts with the data-format emitters (Cube, Prisma) writer PR.", + "codegen-ts/src/generators/agent-requirements-page.ts": "Emits the agent requirements markdown page; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/agent-schema-page.ts": "Emits the agent schema markdown page; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/agent-ui-page.ts": "Emits the agent UI markdown page; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/api-doc-render.ts": "Emits the API documentation markdown; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/docs-data-builder.ts": "Builds markdown fragments for the entity docs page payload with typed line breaks. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/docs-file.ts": "Emits the per-entity docs page; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/prompt-render-file.ts": "Emits the prompt render module; template strings with hand-typed line breaks and indentation. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/generators/report-doc.ts": "Emits the report documentation markdown; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/requirements-markdown.ts": "Emits the requirements ledger markdown; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/template-doc-builder.ts": "Builds the template documentation markdown; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/generators/template-source-render.ts": "Renders template source into documentation blocks; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/prisma/render-prisma-schema.ts": "Emits the Prisma schema with hand-typed indentation and newline joins. Converts with the data-format emitters (Cube, Prisma) writer PR.", + "codegen-ts/src/projection/report-ddl-emit.ts": "Emits the report view DDL (SQL) with hand-typed indentation. Converts with the SQL emitters' writer PR (view and report DDL).", + "codegen-ts/src/projection/view-ddl-emit.ts": "Emits the projection view DDL (SQL) with hand-typed indentation. Converts with the SQL emitters' writer PR (view and report DDL).", + "codegen-ts/src/reference/barrel.ts": "Reference copy of the barrel generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/docs.ts": "Reference copy of the docs generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/entity.ts": "Reference copy of the entity generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/prompt-render.ts": "Reference copy of the prompt render generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/queries.ts": "Reference copy of the queries generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/report-export.ts": "Reference copy of the report export generator, embedding its own renderer; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts). Its embedded renderer duplicates templates/report-export-file.ts and is deleted in the routes PR.", + "codegen-ts/src/reference/requirement-tests.ts": "Reference copy of the requirement-tests generator; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts).", + "codegen-ts/src/reference/routes-hono.ts": "Reference copy of the Hono routes generator, embedding its own renderer; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts). Its embedded renderer duplicates templates/routes-file-hono.ts and is deleted in the routes PR.", + "codegen-ts/src/reference/routes.ts": "Reference copy of the Fastify routes generator, embedding its own renderer; template strings with hand-typed line breaks and indentation. An ejectable reference copy of a shipped generator; it converts with that generator and stays byte-identical to it (test/reference-byte-identical.test.ts). Its embedded renderer duplicates templates/routes-file.ts and is deleted in the routes PR.", + "codegen-ts/src/requirement-walk.ts": "Builds the requirement test scaffold text with typed line breaks. Converts with the requirement-tests generator in the docs and markdown writer PR.", + "codegen-ts/src/routes-expose.ts": "Builds the auth guidance comment block of the generated routes with typed line breaks and ` * ` prefixes. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/shared-model/build-shared-model.ts": "Emits the shared-model artifact text with typed line breaks. Converts with the docs and markdown writer PR (it is published data, not code an adopter edits).", + "codegen-ts/src/templates/barrel.ts": "Emits the generated barrel (index.ts); ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/callable-file.ts": "Emits the stored-procedure callable module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/date-time-regex.ts": "Emits the Zod date and time patterns as fixed text with hand-typed indentation. Converts with the entity tier B writer PR (Zod validators, field meta, UI descriptor).", + "codegen-ts/src/templates/decimal-wire-helper.ts": "Emits the decimal wire helper function as fixed text, each line typed with its indentation. Converts with the entity tier B writer PR (Zod validators, field meta, UI descriptor). Its fixed text becomes one TsWriter.text() snippet.", + "codegen-ts/src/templates/drizzle-schema.ts": "Emits the Drizzle table and column definitions; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/duration-wire-helper.ts": "Emits the field.duration wire helper that a generated model module carries as local code; lines joined with newlines and hand-typed indentation. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/entity-constants.ts": "Emits the entity constants block; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/entity-file.ts": "Composes the entity module from its sections with newline joins; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/entity-meta-file.ts": "Emits the entity field-meta module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier B writer PR (Zod validators, field meta, UI descriptor).", + "codegen-ts/src/templates/enum-int-codec.ts": "Emits the int-backed enum codec; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/enums-file.ts": "Emits the shared enums module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/extract-delegate-emitter.ts": "Emits the extractor delegate; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/extractor.ts": "Emits the FR-010 extractor module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/filter-allowlist.ts": "Emits the filter allowlist with two typed spaces per row and a \",\\n\" join, and types its import as text. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/filter-type.ts": "Emits the client filter type; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/fr010-field-mapping.ts": "Emits the FR-010 field accessor lines with hand-typed indentation. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/inferred-types.ts": "Emits the inferred Row, Insert and Update types; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/jsdoc.ts": "Builds JSDoc comment blocks with a newline join. Converts with the entity tier A writer PR, replaced by TsWriter.doc().", + "codegen-ts/src/templates/mermaid-er.ts": "Emits the Mermaid ER diagram with hand-typed indentation. Converts with the docs and markdown writer PR; its output is documentation, not code an adopter edits, so it goes after the code generators.", + "codegen-ts/src/templates/names-decl.ts": "Emits the physical-name constants; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/output-parser.ts": "Emits the output parser, building indentation with repeat(). Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/output-prompt.ts": "Emits the output-format prompt module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/projection-decl.ts": "Emits the projection read model; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/queries-file.ts": "Emits the per-entity queries module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/queries.ts": "Emits the query functions (find, list, create, update, delete); ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/relations-block.ts": "Emits the Drizzle relations block; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/render-helper.ts": "Emits the typed prompt render helper; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/report-export-file.ts": "Emits the report export module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/requirement-test.ts": "Emits one requirement test scaffold; template strings with hand-typed line breaks and indentation. Converts with the docs and markdown writer PR, beside requirement-walk.ts.", + "codegen-ts/src/templates/routes-file-hono.ts": "Emits the Hono routes module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/routes-file.ts": "Emits the Fastify routes module, threading an indent string through its helpers; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/routes-index.ts": "Emits the routes index module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the queries, routes, report-export, callable and barrel writer PR.", + "codegen-ts/src/templates/tph-discriminator.ts": "Emits the TPH discriminator helpers; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/trace-helper.ts": "Emits the trace helper module as fixed lines, each typed with its indentation. Converts with the prompt-tier writer PR (output parser, prompt, extractor, render and trace helpers).", + "codegen-ts/src/templates/value-object-file.ts": "Emits the value-object module; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/view-decl.ts": "Emits the view-backed read model; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier A writer PR (schema, relations, constants, enums, value objects, filters, inferred types).", + "codegen-ts/src/templates/zod-validators.ts": "Emits the Zod validators; ts-poet `code` templates with hand-typed indentation and newline joins. Converts with the entity tier B writer PR (Zod validators, field meta, UI descriptor)." + }, + "exempt": { + "codegen-ts/src/generators/contract-schema-file.ts": "Writes the contract-schema generator's JSON Schema documents as JSON.stringify(value, null, 2) plus a final newline. The layout is the JSON serializer's, so there is no code to lay out and no writer to move to.", + "codegen-ts/src/generators/openapi-file.ts": "Writes the openapi generator's OpenAPI document as JSON.stringify(value, null, 2) plus a final newline. The layout is the JSON serializer's, so there is no code to lay out and no writer to move to.", + "codegen-ts/src/overwrite-policy.ts": "Writes the codegen tool-state files (.metaobjects/.gen-state/.hashes.json and the engine stamp) as JSON.stringify output plus a final newline. That is a data file the runner owns, not generated code, so it has no writer to move to.", + "codegen-ts/src/poet.ts": "The vendored ts-poet printer, the engine the writer replaces: its import emitter prints the import lines of a Code file. It is retired when the last generator moves to the writer, not converted to it.", + "codegen-ts/src/reference/contract-schema.ts": "The reference (ejectable) copy of the contract-schema generator; writes JSON Schema documents as JSON.stringify(value, null, 2) plus a final newline. The layout is the JSON serializer's, so there is no code to lay out and no writer to move to.", + "codegen-ts/src/reference/openapi.ts": "The reference (ejectable) copy of the openapi generator; writes the OpenAPI document as JSON.stringify(value, null, 2) plus a final newline. The layout is the JSON serializer's, so there is no code to lay out and no writer to move to." + } +} diff --git a/server/typescript/packages/codegen-ts/test/layout-ratchet/analyze.ts b/server/typescript/packages/codegen-ts/test/layout-ratchet/analyze.ts new file mode 100644 index 000000000..c2fb8af1a --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/layout-ratchet/analyze.ts @@ -0,0 +1,247 @@ +// The layout ratchet's analyzer (ADR-0063): finds layout a generator typed by hand. +// +// Generators say what a line contains and the writer decides where it sits, so generator +// source should never spell a line break or an indentation run. This reads a source file +// with the TypeScript compiler's parser (`ts.createSourceFile`) and walks the syntax tree. +// Working on the tree, not the text, is what makes the rules exact: +// +// - comments are never nodes, so a comment is never a finding; +// - a string's value is the parser's cooked value, so `"\n"`, `"\x0a"` and a literal line +// break inside a template are the same finding; +// - a constant is followed to its declaration, so `const NL = "\n"; xs.join(NL)` is a +// finding at the join as well as at the declaration. +// +// Four rules: +// +// newline-literal a string or template literal part whose value contains a line break. +// indent-literal a string or template literal that starts a line of output (a whole +// string, or the head of a template) and begins with an indentation run: +// two or more spaces, or a tab. +// newline-join `xs.join(sep)` or `joinCode(xs, { on: sep })` whose separator contains a +// line break, the separator resolved through same-file constants. +// indent-repeat `s.repeat(n)` where `s` resolves to whitespace: building indentation. +// +// Two places a literal is not output, and so is not layout, are ruled out by position: +// +// - a search pattern: the first argument of split, indexOf, lastIndexOf, includes, +// startsWith, endsWith, replace or replaceAll (the text being looked for, not written); +// - a writer snippet: the argument of a `.text(...)` call, the writer's entry for fixed +// code written as it should read (it removes the margin and re-indents). +// +// Type positions (`type Sep = "\n"`) and module specifiers are not values and are skipped. + +import ts from "typescript"; + +export type FindingKind = "newline-literal" | "indent-literal" | "newline-join" | "indent-repeat"; + +export interface Finding { + readonly kind: FindingKind; + /** 1-based line and column of the node in the source. */ + readonly line: number; + readonly column: number; + /** The source text of the offending node, shortened. */ + readonly excerpt: string; +} + +const LINE_BREAK = /[\r\n]/; +const INDENT_RUN = /^(?: {2,}|\t)/; +const WHITESPACE = /^[ \t\r\n]+$/; +const SEARCH_METHODS = new Set(["split", "indexOf", "lastIndexOf", "includes", "startsWith", "endsWith", "replace", "replaceAll"]); +const SNIPPET_METHOD = "text"; +const JOIN_METHOD = "join"; +const JOIN_CODE = "joinCode"; +const JOIN_CODE_SEPARATOR = "on"; +const REPEAT_METHOD = "repeat"; +const MAX_RESOLVE_DEPTH = 8; + +/** What a string expression is known to contain. `complete` is false when part of it is unknown. */ +interface Resolved { + readonly text: string; + readonly complete: boolean; +} + +export function analyzeSource(fileName: string, source: string): Finding[] { + const kind = fileName.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS; + const sf = ts.createSourceFile(fileName, source, ts.ScriptTarget.Latest, true, kind); + const consts = collectConsts(sf); + const findings: Finding[] = []; + + const report = (k: FindingKind, node: ts.Node): void => { + const { line, character } = sf.getLineAndCharacterOfPosition(node.getStart(sf)); + const raw = node.getText(sf).replace(/\s+/g, " "); + findings.push({ kind: k, line: line + 1, column: character + 1, excerpt: raw.length > 60 ? `${raw.slice(0, 57)}...` : raw }); + }; + + const resolve = (expr: ts.Expression, depth = 0): Resolved | undefined => { + if (depth > MAX_RESOLVE_DEPTH) return undefined; + if (ts.isStringLiteral(expr) || ts.isNoSubstitutionTemplateLiteral(expr)) return { text: expr.text, complete: true }; + if (ts.isParenthesizedExpression(expr) || ts.isAsExpression(expr) || ts.isSatisfiesExpression(expr) || ts.isNonNullExpression(expr) || ts.isTypeAssertionExpression(expr)) { + return resolve(expr.expression, depth + 1); + } + if (ts.isBinaryExpression(expr) && expr.operatorToken.kind === ts.SyntaxKind.PlusToken) { + const l = resolve(expr.left, depth + 1); + const r = resolve(expr.right, depth + 1); + if (l === undefined && r === undefined) return undefined; + return { text: (l?.text ?? "") + (r?.text ?? ""), complete: l?.complete === true && r?.complete === true }; + } + if (ts.isTemplateExpression(expr)) { + let text = expr.head.text; + let complete = true; + for (const span of expr.templateSpans) { + const sub = resolve(span.expression, depth + 1); + text += (sub?.text ?? "") + span.literal.text; + complete &&= sub?.complete === true; + } + return { text, complete }; + } + if (ts.isConditionalExpression(expr)) { + // Either branch may be written; for "does it contain a line break" both count. + const a = resolve(expr.whenTrue, depth + 1); + const b = resolve(expr.whenFalse, depth + 1); + if (a === undefined && b === undefined) return undefined; + return { text: (a?.text ?? "") + (b?.text ?? ""), complete: false }; + } + if (ts.isIdentifier(expr)) { + const init = consts.get(expr.text); + return init === undefined ? undefined : resolve(init, depth + 1); + } + if (ts.isPropertyAccessExpression(expr) && ts.isIdentifier(expr.expression)) { + const obj = consts.get(expr.expression.text); + const target = obj === undefined ? undefined : unwrap(obj); + if (target !== undefined && ts.isObjectLiteralExpression(target)) { + const prop = findProperty(target, expr.name.text); + return prop === undefined ? undefined : resolve(prop, depth + 1); + } + } + return undefined; + }; + + const visit = (node: ts.Node): void => { + if (ts.isTypeNode(node) || ts.isImportDeclaration(node) || ts.isExportDeclaration(node) || ts.isExternalModuleReference(node)) return; + if (ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node) || ts.isTemplateHead(node) || ts.isTemplateMiddle(node) || ts.isTemplateTail(node)) { + checkLiteral(node); + } else if (ts.isCallExpression(node)) { + checkCall(node); + } + ts.forEachChild(node, visit); + }; + + const checkLiteral = (lit: ts.StringLiteral | ts.NoSubstitutionTemplateLiteral | ts.TemplateLiteralLikeNode): void => { + const whole = literalExpression(lit); + if (isSearchPattern(whole) || isSnippet(whole)) return; + const value = lit.text; + if (LINE_BREAK.test(value)) report("newline-literal", lit); + else if (startsALine(lit) && INDENT_RUN.test(value)) report("indent-literal", lit); + }; + + const checkCall = (call: ts.CallExpression): void => { + const method = calledName(call); + if (method === JOIN_METHOD && ts.isPropertyAccessExpression(call.expression) && call.arguments.length > 0) { + const sep = resolve(call.arguments[0]!); + if (sep !== undefined && LINE_BREAK.test(sep.text)) report("newline-join", call); + } else if (method === JOIN_CODE && call.arguments.length > 1) { + const opts = unwrap(call.arguments[1]!); + const on = opts !== undefined && ts.isObjectLiteralExpression(opts) ? findProperty(opts, JOIN_CODE_SEPARATOR) : undefined; + const sep = on === undefined ? undefined : resolve(on); + if (sep !== undefined && LINE_BREAK.test(sep.text)) report("newline-join", call); + } else if (method === REPEAT_METHOD && ts.isPropertyAccessExpression(call.expression)) { + const unit = resolve(call.expression.expression); + if (unit !== undefined && unit.complete && WHITESPACE.test(unit.text)) report("indent-repeat", call); + } + }; + + visit(sf); + return findings; +} + +/** Every `const name = ` in the file. A name declared more than once is left out: it is shadowed somewhere. */ +function collectConsts(sf: ts.SourceFile): Map { + const seen = new Map(); + const visit = (node: ts.Node): void => { + if (ts.isVariableDeclaration(node) && ts.isIdentifier(node.name)) { + const list = node.parent; + const isConst = ts.isVariableDeclarationList(list) && (list.flags & ts.NodeFlags.Const) !== 0; + const name = node.name.text; + seen.set(name, seen.has(name) || !isConst || node.initializer === undefined ? null : node.initializer); + } else if ((ts.isParameter(node) || ts.isBindingElement(node)) && ts.isIdentifier(node.name)) { + // A parameter or destructured name shadows any constant of the same name. + seen.set(node.name.text, null); + } + ts.forEachChild(node, visit); + }; + visit(sf); + const out = new Map(); + for (const [name, init] of seen) if (init !== null) out.set(name, init); + return out; +} + +function unwrap(expr: ts.Expression): ts.Expression | undefined { + let e: ts.Expression = expr; + while (ts.isParenthesizedExpression(e) || ts.isAsExpression(e) || ts.isSatisfiesExpression(e)) e = e.expression; + return e; +} + +function findProperty(obj: ts.ObjectLiteralExpression, name: string): ts.Expression | undefined { + for (const p of obj.properties) { + if (ts.isPropertyAssignment(p) && (ts.isIdentifier(p.name) || ts.isStringLiteral(p.name)) && p.name.text === name) return p.initializer; + } + return undefined; +} + +/** The method or function name a call invokes: `x.join(...)` is "join", `joinCode(...)` is "joinCode". */ +function calledName(call: ts.CallExpression): string | undefined { + const callee = call.expression; + if (ts.isPropertyAccessExpression(callee)) return callee.name.text; + if (ts.isIdentifier(callee)) return callee.text; + return undefined; +} + +/** The whole expression a literal part belongs to: the literal itself, or its template. */ +function literalExpression(lit: ts.Node): ts.Node { + if (ts.isTemplateHead(lit)) return lit.parent; + if (ts.isTemplateMiddle(lit) || ts.isTemplateTail(lit)) return lit.parent.parent; + return lit; +} + +/** A string or template head starts its own text; a middle or tail follows a substitution. */ +function startsALine(lit: ts.Node): boolean { + return !(ts.isTemplateMiddle(lit) || ts.isTemplateTail(lit)); +} + +function isArgument(expr: ts.Node, method: (name: string) => boolean, index: number): boolean { + const call = expr.parent; + if (call === undefined || !ts.isCallExpression(call) || !ts.isPropertyAccessExpression(call.expression)) return false; + return method(call.expression.name.text) && call.arguments[index] === expr; +} + +function isSearchPattern(expr: ts.Node): boolean { + return isArgument(expr, (n) => SEARCH_METHODS.has(n), 0); +} + +function isSnippet(expr: ts.Node): boolean { + return isArgument(expr, (n) => n === SNIPPET_METHOD, 0); +} + +// ── which files the ratchet reads ──────────────────────────────────────────────────────── + +/** The packages whose `src/` holds TypeScript generator source, relative to `server/typescript/packages/`. */ +export const SCANNED_PACKAGES = ["codegen-ts", "codegen-ts-react", "codegen-ts-tanstack", "codegen-ts-angular"] as const; + +/** The writer itself: the one module family that lays text out, so the one allowed to type layout. */ +const WRITER_DIR = "codegen-ts/src/writer/"; + +/** + * Whether a file (relative to `server/typescript/packages/`) is generator source the ratchet + * reads. Excluded by rule, never by listing: anything outside a scanned package's `src/`, + * tests, declaration files, the writer, and files a build script generated (`*.generated.ts`, + * which hold data such as embedded template bodies and are rewritten by that script). + */ +export function isGeneratorSource(rel: string): boolean { + const path = rel.replaceAll("\\", "/"); + if (!SCANNED_PACKAGES.some((p) => path.startsWith(`${p}/src/`))) return false; + if (!/\.tsx?$/.test(path) || path.endsWith(".d.ts")) return false; + if (/(?:^|\/)(?:test|tests|__tests__)\//.test(path) || /\.(?:test|spec)\.tsx?$/.test(path)) return false; + if (path.startsWith(WRITER_DIR)) return false; + if (/\.generated\.tsx?$/.test(path)) return false; + return true; +} diff --git a/server/typescript/packages/codegen-ts/test/writer-conformance.test.ts b/server/typescript/packages/codegen-ts/test/writer-conformance.test.ts new file mode 100644 index 000000000..c0345fc18 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/writer-conformance.test.ts @@ -0,0 +1,88 @@ +// Runs fixtures/writer-conformance/ against the TypeScript writer (spec/code-writer.md). +// `core` cases run on CodeWriter with the options the case states; `typescript` cases run on +// TsWriter with its own defaults unless the case overrides them. + +import { describe, expect, test } from "bun:test"; +import { readdirSync, readFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { CodeWriter, type CodeWriterOptions, Group, type Piece, WriterError } from "../src/writer/code-writer.js"; +import { TsWriter } from "../src/writer/ts-writer.js"; + +const CASES = resolve(import.meta.dir, "../../../../../fixtures/writer-conformance/cases"); + +type JsonPiece = string | { list: { open: string; items: JsonPiece[]; close: string; pad?: boolean } } | JsonPiece[]; +type Op = [string, ...unknown[]]; + +interface WriterCase { + description: string; + dialect: "core" | "typescript"; + options?: CodeWriterOptions; + ops: Op[]; + expected?: string[]; + error?: string; + errorQuotes?: string; +} + +function toPiece(p: JsonPiece): Piece { + if (typeof p === "string") return p; + if (Array.isArray(p)) return p.map(toPiece); + return new Group(p.list.open, p.list.items.map(toPiece), p.list.close, p.list.pad === true); +} + +function run(c: WriterCase): string { + const w = c.dialect === "core" ? new CodeWriter(c.options) : new TsWriter(c.options); + for (const [op, ...args] of c.ops) { + const pieces = (args as JsonPiece[]).map(toPiece); + const s = args as string[]; + switch (op) { + case "line": w.line(...pieces); break; + case "open": w.open(...pieces); break; + case "close": w.close(...pieces); break; + case "blank": w.blank(); break; + case "text": w.text(s[0]!); break; + default: { + if (!(w instanceof TsWriter)) throw new Error(`op ${op} is not a core op`); + switch (op) { + case "use": w.use(s[0]!, s[1]!); break; + case "useType": w.useType(s[0]!, s[1]!); break; + case "useDefault": w.useDefault(s[0]!, s[1]!); break; + case "useNamespace": w.useNamespace(s[0]!, s[1]!); break; + case "useSideEffect": w.useSideEffect(s[0]!); break; + case "header": w.header(...s); break; + case "doc": w.doc(...s); break; + default: throw new Error(`unknown op ${op}`); + } + } + } + } + return w.toString(); +} + +const files = readdirSync(CASES).filter((f) => f.endsWith(".json")).sort(); + +describe("writer conformance", () => { + test("the corpus is present", () => { + expect(files.length).toBeGreaterThanOrEqual(25); + }); + + for (const file of files) { + const c = JSON.parse(readFileSync(join(CASES, file), "utf8")) as WriterCase; + test(`${file.replace(/\.json$/, "")}: ${c.description}`, () => { + if (c.dialect === "core") expect(c.options, "a core case states its options").toBeDefined(); + if (c.error !== undefined) { + let thrown: unknown; + try { + run(c); + } catch (e) { + thrown = e; + } + expect(thrown).toBeInstanceOf(WriterError); + expect((thrown as WriterError).code).toBe(c.error as WriterError["code"]); + if (c.errorQuotes !== undefined) expect((thrown as WriterError).message).toContain(c.errorQuotes); + return; + } + const lines = c.expected!; + expect(run(c)).toBe(lines.length === 0 ? "" : `${lines.join("\n")}\n`); + }); + } +}); diff --git a/server/typescript/packages/codegen-ts/test/writer-ts.test.ts b/server/typescript/packages/codegen-ts/test/writer-ts.test.ts new file mode 100644 index 000000000..81cee2969 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/writer-ts.test.ts @@ -0,0 +1,155 @@ +// TsWriter beyond the shared corpus: ts-poet `Code` interop (so a generator can convert one +// function at a time), the object-literal helpers, and the refusals. + +import { describe, expect, test } from "bun:test"; +import { OBJECT_SUBTYPE_ENTITY } from "@metaobjectsdev/metadata"; +import { code, Import, imp, joinCode } from "../src/poet.js"; +import { renderInferredTypes } from "../src/templates/inferred-types.js"; +import { WRITER_ERRORS, WriterError } from "../src/writer/code-writer.js"; +import { arr, call, key, props, str, TsWriter } from "../src/writer/ts-writer.js"; +import { metaObject } from "./_meta-build.js"; + +function errorCode(fn: () => unknown): string | undefined { + try { + fn(); + } catch (e) { + return e instanceof WriterError ? e.code : `not a WriterError: ${String(e)}`; + } + return undefined; +} + +describe("TsWriter accepts ts-poet Code", () => { + test("an inline fragment is written as its text and its imports are collected", () => { + const w = new TsWriter(); + w.line("export const a = ", code`${imp("eq@drizzle-orm")}(x, y)`, ";"); + w.line("export type T = ", imp("t:Row@./row"), ";"); + expect(w.toString()).toBe( + 'import { eq } from "drizzle-orm";\nimport type { Row } from "./row";\n\nexport const a = eq(x, y);\nexport type T = Row;\n', + ); + }); + + test("a multi-line fragment is re-indented to the current level and its blank-line runs collapse", () => { + const w = new TsWriter(); + const body = code` +const ${imp("z@zod")}Schema = z.object({}); + + +export default zSchema; +`; + w.block("namespace N {", () => w.code(body)); + expect(w.toString()).toBe( + 'import { z } from "zod";\n\nnamespace N {\n const zSchema = z.object({});\n\n export default zSchema;\n}\n', + ); + }); + + test("default, namespace, side-effect and renamed imports carry over", () => { + const w = new TsWriter(); + w.line(code`${imp("React=react")}; ${imp("path*node:path")}; ${imp("Poly+./polyfill")};`); + w.line(code`${Import.importsName("Local", "./mod", false, "Exported")};`); + expect(w.toString()).toBe( + [ + 'import * as path from "node:path";', + 'import React from "react";', + 'import { Exported as Local } from "./mod";', + 'import "./polyfill";', + "", + "React; path; Poly;", + "Local;", + "", + ].join("\n"), + ); + }); + + test("nested and joined fragments are walked for imports", () => { + const parts = [code`${imp("and@drizzle-orm")}(a)`, code`${imp("or@drizzle-orm")}(b)`]; + const w = new TsWriter(); + w.line("const c = [", joinCode(parts, { on: ", " }), "];"); + expect(w.toString()).toBe('import { and, or } from "drizzle-orm";\n\nconst c = [and(a), or(b)];\n'); + }); + + test("a real generator section moves into a writer-built file with its imports hoisted", () => { + const section = renderInferredTypes(metaObject(OBJECT_SUBTYPE_ENTITY, "Post")); + const w = new TsWriter(); + w.line("export const posts = table;"); + w.blank(); + w.code(section); + expect(w.toString()).toBe( + [ + 'import type { InferInsertModel, InferSelectModel } from "drizzle-orm";', + "", + "export const posts = table;", + "", + "export type Post = InferSelectModel;", + "export type PostInsert = InferInsertModel;", + "export type PostUpdate = Partial;", + "", + ].join("\n"), + ); + }); + + test("a fragment from a different ts-poet copy is refused, not written without its imports", () => { + const foreign = { toCodeString: () => "x", childNodes: [] }; + expect(errorCode(() => new TsWriter().line(foreign))).toBe(WRITER_ERRORS.unknownPiece); + }); + + test("a multi-line fragment passed to line() is refused", () => { + expect(errorCode(() => new TsWriter().line(code`a;\nb;`))).toBe(WRITER_ERRORS.newlineInLine); + }); +}); + +describe("TsWriter expression helpers", () => { + test("props, arr, call, str and key build literals that fit or break", () => { + const rule = props() + .set("ops", [arr(["eq", "ne"].map(str)), " as const"]) + .set(key("leading-wildcard"), "false") + .add("...rest"); + const w = new TsWriter({ width: 90 }); + w.line("export const a = ", rule, ";"); + w.line("export const b = ", call("define", [str("a value that is long enough"), rule]), ";"); + expect(w.toString()).toBe( + [ + 'export const a = { ops: ["eq", "ne"] as const, "leading-wildcard": false, ...rest };', + "export const b = define(", + ' "a value that is long enough",', + ' { ops: ["eq", "ne"] as const, "leading-wildcard": false, ...rest },', + ");", + "", + ].join("\n"), + ); + }); + + test("a doc comment escapes a closing delimiter in its text", () => { + const w = new TsWriter(); + w.doc("Ends a comment with */ in its text."); + expect(w.toString()).toBe("/** Ends a comment with *\\/ in its text. */\n"); + }); + + test("a doc comment too long for one line becomes a block", () => { + const w = new TsWriter({ width: 20 }); + w.doc("A sentence that does not fit."); + expect(w.toString()).toBe("/**\n * A sentence that does not fit.\n */\n"); + }); + + test("the same default imported under two names is refused", () => { + const w = new TsWriter(); + w.useDefault("React", "react"); + expect(errorCode(() => w.useDefault("R", "react"))).toBe(WRITER_ERRORS.importConflict); + }); +}); + +describe("the writer is extended by subclassing (ADR-0063)", () => { + test("a subclass overrides one method and keeps everything else", () => { + // An adopter who wants relative imports first changes only the ordering step. + class RelativeFirst extends TsWriter { + protected override orderModules(modules: readonly string[]): string[] { + const base = super.orderModules(modules); + return [...base.filter((m) => m.startsWith(".")), ...base.filter((m) => !m.startsWith("."))]; + } + } + const w = new RelativeFirst(); + w.line("const x = ", w.use("x", "zod"), "(", w.use("y", "./local"), ");"); + expect(w.toString()).toBe( + ['import { y } from "./local";', 'import { x } from "zod";', "", "const x = x(y);", ""].join("\n"), + ); + }); +}); diff --git a/spec/code-writer.md b/spec/code-writer.md new file mode 100644 index 000000000..35a94973e --- /dev/null +++ b/spec/code-writer.md @@ -0,0 +1,170 @@ +# The code writer contract + +Every port's generators write text through a code writer +([ADR-0063](decisions/ADR-0063-generators-own-layout.md)). The generator says what a line +contains, and the writer decides where it sits. This document is the contract the writers +share. The cases in [`fixtures/writer-conformance/`](../fixtures/writer-conformance/) pin it. + +A writer has two parts: + +- **The core**, which is language-neutral: lines, indentation, blocks, blank-line requests, + snippets, fit-or-break lists and the output form. Every port's core runs the `core` cases. +- **A dialect** for the language being written: import collection and ordering, string + quoting, doc comments and expression helpers. A dialect runs the cases for its language, + such as `typescript`. + +The writer never parses text. It lays out the structure the generator handed it. When a shape +comes out badly, the generator breaks it explicitly with a block. The writer is not made +smarter. + +## Core + +### Options + +| Option | Meaning | TypeScript default | +|---|---|---| +| `indent` | One level of indentation | two spaces | +| `width` | The last column a line may reach before its last list breaks | 120 | +| `trailingComma` | Whether a broken list puts a comma after its last item too | true | + +A dialect sets the defaults for its language. A `core` case always states all three, so it +means the same thing in every port. + +### Lines and blocks + +- **`line(parts...)`** writes one line at the current indent. Its parts are concatenated. There + is no unindented variant. +- **`open(parts...)`** writes the line, then indents one level. **`close(parts...)`** dedents + one level, then writes the line. **`block(head, body, tail)`** is `open(head)`, then the body, + then `close(tail)`. A port gives `block` its idiomatic scoped form: a callback in + TypeScript, `try (var b = ...)` in Java, `with` in Python, `using` in C#. +- Indentation comes only from open blocks, so a line's indent is the depth of the blocks around + it times `indent`. +- `close` with no block open is **`ERR_WRITER_UNBALANCED_CLOSE`**. Finishing the file with a + block still open is **`ERR_WRITER_UNCLOSED_BLOCK`**, and the message quotes the line that + opened the innermost open block. +- Line text containing a line break (`\n`, `\r\n` or `\r`) is **`ERR_WRITER_NEWLINE_IN_LINE`**. + Write each line separately, or use `text`. + +### Blank lines are requests + +- **`blank()`** asks for one blank line before whatever is written next. +- Two or more requests in a row give one blank line. +- A request gives nothing at the start of the file, directly after a line that opens a block, + directly before a line that closes a block, or at the end of the file. +- A line whose text is empty is a blank-line request, not a written line. + +### Snippets + +**`text(snippet)`** writes a block of fixed code as it should read. The writer splits it on +line breaks (`\n`, `\r\n` and `\r`). It drops the first line when that line is blank, and the +last line when that line is blank. It removes the common left margin of the non-blank lines and +writes each line at the current indent, keeping the line's indentation beyond the margin. A +blank line inside the snippet is a blank-line request. Every line is re-indented, including a +line inside a multi-line string literal in the snippet. + +### Lists fit or break + +A **list** is an opener, items and a closer. On one line its items are separated by `, `. A +**padded** list puts a space inside its brackets (`{ a, b }`); an unpadded one does not +(`[a, b]`, `f(a, b)`). An empty list prints as its opener and closer (`{}`, `[]`, `()`) and +never breaks. + +When a line's indent plus its text is at most `width`, the line prints as it is. When it is +longer: + +1. The **last** non-empty list on the line breaks. Everything before it, flattened, plus its + opener, is a line that opens a block. Each item is a line followed by a comma, except the + last item when `trailingComma` is false. The closer plus everything after it closes the + block. +2. Each item line is held to the same rule, so an item that still does not fit breaks its own + last list. +3. A line with no non-empty list prints as it is, however long. + +Width is counted in Unicode code points, so an emoji counts as one even though it is two UTF-16 +units. The comparison includes the indent, and a line exactly `width` long fits. + +### Output form + +- Lines end with `\n` on every platform. +- No line has trailing whitespace. +- A file with any content ends with exactly one `\n`. A file with no content is the empty + string. +- Text is UTF-8 when it is written to disk. + +## TypeScript dialect (`TsWriter`) + +- **Imports are collected while the body is written.** `use(name, from)` records a value + import and returns the name to write. `useType` records a type-only import. `useDefault`, + `useNamespace` and `useSideEffect` record the other forms. Imports print before the body, + after any `header(...)` lines, with one blank line between each of the three sections. +- **Order.** Package modules come first, then relative ones (those starting with `.`). Each + group is in code-unit order, with no blank line between the groups. Within a module, names are + in code-unit order. +- **One line per module.** `import * as ns from "m";` prints on its own line. Default and named + imports share a line: `import D, { a, type B } from "m";`. A module whose every named import + is type-only, with no default, prints as `import type { A, B } from "m";`. A name used both as + a type and as a value is imported as a value. A side-effect import (`import "m";`) prints only + when nothing else is imported from that module. +- The named list is a padded list and fits or breaks like any other. A single name never + breaks. +- One local name imported from two modules, or naming two different exports, is + **`ERR_WRITER_IMPORT_CONFLICT`**. +- **`doc(lines...)`** writes `/** text */` when there is one entry and it fits. Otherwise it + writes a block of ` * ` lines, with a bare ` *` line for an empty entry. A `*/` inside the + text is written as `*\/`. +- **`Code` interop.** Any part may be a `Code` or `Import` from the package's code primitives + (`src/poet.ts`, the vendored ts-poet printer). Its text is written, and its imports are + collected. `code(fragment)` writes a multi-line fragment as a snippet. A fragment from a + different physical copy of those primitives (or from the ts-poet package itself) is refused + with **`ERR_WRITER_UNKNOWN_PIECE`**, because its imports cannot be read. +- Helpers: `str` (a quoted string literal), `arr`, `call`, `props()` (an object literal built + with `.set(name, value)` and `.add(member)`) and `key` (an object key, quoted only when it is + not an identifier). + +## Extension by subclassing + +A writer and the generators built on it are meant to be extended in place +([ADR-0063](decisions/ADR-0063-generators-own-layout.md), item 7): + +- A writer class is not final. Every method is public or protected, never private, so a + subclass can override one step and keep the rest. In TypeScript the core's steps include + `piece` (what a part becomes), `breakAt` (where a line that does not fit breaks) and `fits`. + `TsWriter` adds `writeHeader`, `writeImports`, `writeImport` and `orderModules`. +- A reference generator is a class whose output comes from one overridable method per + meaningful piece: the file header, the imports, and each section or member. An adopter + overrides the method for the piece it wants changed, and passes its subclass where the + reference generator was configured. +- Overridable methods are not a stability contract. They may change between minor versions. + The conformance cases pin the output of the methods as shipped, not their names. + +## Fixture format + +Each case in `fixtures/writer-conformance/cases/` is one JSON file: + +```jsonc +{ + "description": "What the case pins, in one sentence.", + "dialect": "core", // or a language: "typescript" + "options": { "indent": " ", "width": 40, "trailingComma": true }, // required for core + "ops": [ + ["open", "class A {"], + ["line", "f", { "list": { "open": "(", "items": ["a", "b"], "close": ")" } }, ";"], + ["blank"], + ["text", "fixed\ncode"], + ["close", "}"] + ], + "expected": ["class A {", " f(a, b);", " fixed", " code", "}"] +} +``` + +- An op is an array: its name, then its arguments. The core ops are `line`, `open` and `close` + (whose arguments are parts), `blank` and `text`. The TypeScript ops are `use`, `useType`, + `useDefault`, `useNamespace` and `useSideEffect` (name, then module; `useSideEffect` takes + only the module), `header` and `doc` (lines). +- A part is a string, a list (`{"list": {"open", "items", "close", "pad"?}}`, where items are + parts) or an array of parts, which are concatenated. +- `expected` is the output's lines. The file is those lines joined with `\n` plus a final `\n`, + or the empty string when the array is empty. +- A case that must fail has `"error"` (the error code) in place of `expected`. It may also have + `"errorQuotes"`, text the error message must contain. diff --git a/spec/decisions/ADR-0063-generators-own-layout.md b/spec/decisions/ADR-0063-generators-own-layout.md new file mode 100644 index 000000000..759c5ca63 --- /dev/null +++ b/spec/decisions/ADR-0063-generators-own-layout.md @@ -0,0 +1,129 @@ +# ADR-0063: Generators own their layout; formatters are test oracles only + +## Status + +**Accepted** (2026-11-14). Registers no vocabulary: `metamodelVersion` does not move. It sets +how every port's generators produce text. This decision ships the TypeScript writer, the shared +writer contract and corpus, and the test that holds generator source to the rule. It converts no +generator and changes no generated output. Each generator is converted in its own later change. + +## Context + +The Java `codegen-base` direct family (`FileIndentor` and `FileDirectWriter`, about 130 lines) +gives layout that is correct by construction. A generator writes one line per statement, and +the writer puts it at the current indent. There is no later pass that can fail, be skipped, or +change with a tool version. Generators written since then use weaker techniques: + +- **TypeScript** builds files from ts-poet `code` templates with hand-typed indentation and + `",\n"` joins. It runs dprint (120 columns) over the result, and then Biome (80 columns) over + most `.ts` files. Formatting is about 97% of per-file generation time. Both formatters swallow + a failure and write the unformatted text. Neither can fix structure: imports typed as text end + up in the middle of a file, and blank lines depend on each template's literal edges. +- **Python** runs two `ruff` subprocesses per file and returns the text unformatted when `ruff` + is not installed. The bytes an adopter gets therefore depend on whether `ruff` is installed, + and on which version. +- **C#, Java Spring and Kotlin** build strings with indentation typed inside the literals and + import lists written by hand. + +The Java design lost ground because it stopped at indentation. It had no imports, no long-list +wrapping and no optional members. Once a generator types its commas by hand, it may as well type +its spaces too, and nothing stopped that. + +## Decision + +1. **The generator owns layout.** A generator says what a line contains, and a writer in its own + port decides where the line sits. The writer builds the file in memory and returns a string. + It never parses text: it lays out the structure the generator handed it (lines, blocks, + blank-line requests and breakable lists). + +2. **No formatter runs in the generation path, in any port.** No formatter runs in `meta gen`, + `dotnet meta gen`, `mvn metaobjects:generate` or `metaobjects gen`, and none runs in a + reference generator. Generated output therefore depends only on the generator and the + model. The existing formatter entry points (`Code.toString()` with dprint, `formatTs` with + Biome, Python `ruff_format`) stay exported for adopter-owned generators, because an owned + copy calls them itself (ADR-0034). Retiring them is a separate decision. + +3. **Formatters are test oracles only.** A formatter may run in a test to check output, never + in a step an adopter's generation depends on. + +4. **Output aims for clean, simple code, not parity with today's formatting.** A conversion + does not have to reproduce the bytes the formatters produce now. When the simple writer + rules give a different but clean result, the writer's result is right. A conversion is + checked in one of two ways: + - format the old output and the new output with the same oracle formatter and compare the + results; or + - build the generated code and run its tests (the codegen-compile gate, the api-contract + corpus and the port's goldens). + + The goldens move when a conversion lands, and the change lists what moved and why. + +5. **One writer contract, one writer per port.** Each port has a language-neutral core (lines, + indentation, blocks, blank-line requests, fit-or-break lists, one output form) and a dialect + for the language it writes (import collection and ordering, string quoting, doc comments). + The contract is [`spec/code-writer.md`](../code-writer.md). The shared cases live in + `fixtures/writer-conformance/`: `core` cases run on every port's writer, and dialect cases + on the writer for that language. A language's own facility may be the base of its writer + where one fits (C#'s `IndentedTextWriter` for indentation, Java text blocks and C# raw + strings as input to the snippet call). It is not adopted when it would make generator source + stop reading like the file it writes; builder APIs such as JavaPoet and Roslyn + `SyntaxFactory` are not adopted for that reason. + +6. **Each port has a ratchet against hand-typed layout.** A test reads generator source and + fails on layout typed by hand: a string literal holding a line break, a string that starts a + line with an indentation run, a join on a newline, or an indentation built with `repeat`. + The test parses the source with the language's own parser and walks the syntax tree. It does + not match patterns over the text, so comments are never findings and a constant cannot hide + a newline. Test files, the writer itself and build-script output are excluded by rule. An + allowlist names every unconverted file, each with a written reason, and only shrinks. A file + that stops typing layout must drop its entry. TypeScript's ratchet is + `server/typescript/packages/codegen-ts/test/layout-ratchet.test.ts`, and the other ports add + theirs with their writers. + +7. **Reference generators are extended by subclassing.** Ejecting a generator and maintaining + the copy (ADR-0034) stays available, but it is not the only way to change one. An adopter may + extend the reference generator's class and override the few methods it needs, and keep + taking every other change the reference gets. So: + - each reference generator is a class, and each meaningful piece of its output (the file + header, the imports, each section or member) comes from its own overridable method; + - nothing an adopter might want to change is `final`, `private`, `sealed`, `#private` or a + module-private helper only; it is `protected` or public; + - the generator config or registry accepts an adopter's subclass wherever it accepts the + reference generator (in TypeScript it already does, since `generators` takes any + `Generator` object); + - `CodeWriter` and `TsWriter` are not final, and their methods are overridable, so a + subclass can change one layout step, such as import order or where a line breaks. + + **This is not a stability contract.** Overridable methods may be renamed, split or have + their signatures changed between minor versions, and the CHANGELOG notes it when they do. + An adopter who needs a contract ejects. The ratchet in item 6 does not enforce this rule; + review does. Existing generators take this shape as each one is converted to the writer. + +## The TypeScript writer + +`CodeWriter` and `TsWriter` (`server/typescript/packages/codegen-ts/src/writer/`) are exported +from the package root and from `@metaobjectsdev/codegen-ts/engine`, and both are listed in +`eject.engineSurface`, so an owned generator imports them from the package. TypeScript lays out +at **120 columns** with a two-space indent and a trailing comma in broken lists. + +`TsWriter` accepts `Code` fragments from the package's code primitives (`src/poet.ts`, the +vendored ts-poet printer). A fragment is written as its text, its `imp()` +symbols join the file's import block, and a multi-line fragment is re-indented by +`w.code(fragment)`. A generator can therefore move to the writer one function at a time while the +sections it has not converted still return `Code`. A `Code` built by a different physical copy of +those primitives cannot be read, so it is refused rather than written without its imports. + +## Consequences + +- Generation gets faster and becomes deterministic. Once a generator is converted, no formatter + version, configuration file or missing binary can change its bytes. +- Imports are collected while the body is written, so structural defects that a formatter could + never fix (mid-file imports, duplicate imports) go away with the conversion. +- Generated files change once per port as generators convert. A hand-edited generated file gets + the usual three-way merge against the new base. +- The writer stays small because it never parses text. A shape the fit-or-break rule lays out + badly is broken explicitly by the generator with a block, rather than by making the writer + smarter. +- Adopters can change one piece of a reference generator without owning all of it. The price + is that an override can break on a minor upgrade, which is the trade item 7 accepts. +- Five writers must agree, which the shared corpus checks, as the repository already does for + everything else that must match across ports. diff --git a/spec/decisions/README.md b/spec/decisions/README.md index 6e0de2234..b2d88f319 100644 --- a/spec/decisions/README.md +++ b/spec/decisions/README.md @@ -59,6 +59,7 @@ Write one when a decision is **cross-cutting** (affects more than one language p - [ADR-0060 — Java and Kotlin codegen decide overwrites from the committed hash manifest](ADR-0060-jvm-codegen-hash-manifest.md) — *Accepted* (the JVM plugin records each file it writes in `.metaobjects/.gen-state/.hashes.json`, refuses an edited one, and deletes an untouched orphan, owned per generator and output directory) - [ADR-0061 — A template body may use a per-port engine, chosen by its file](ADR-0061-per-port-template-engines.md) — *Accepted* (Python and TypeScript render and check Jinja bodies — `.jinja` / `.j2` / `.jinja2` — against the same payload check; the metadata never names an engine; Java, Kotlin and C# refuse Jinja with `ERR_TEMPLATE_ENGINE_UNSUPPORTED`) - [ADR-0062 — Two providers may not claim one attribute name on a node type](ADR-0062-provider-attribute-ownership.md) — *Accepted* (the registry records which provider registered each attribute and refuses a second provider's claim on the same name on the same slot, the shared root or a descendant subtype: `ERR_PROVIDER_ATTR_CONFLICT` for the same slot, new `ERR_PROVIDER_ATTR_SHADOW` for the related slots; each provider keeps its own idiomatic physical-name word, a store without a provider uses `attr.properties`, no wire-name attribute is added) +- [ADR-0063 — Generators own their layout; formatters are test oracles only](ADR-0063-generators-own-layout.md) — *Accepted* (a writer in each port lays generated text out and no formatter runs in generation; output aims for clean code, not parity with today's formatting; a syntax-tree ratchet holds generator source to it, with an allowlist that only shrinks) > **Index gap — ADR-0031 through ADR-0051 are on disk but not listed above.** The index stopped > being maintained after ADR-0030; the files are authoritative, this list is not. Read