diff --git a/AGENTS.md b/AGENTS.md index 254c95dc2..aad82fd87 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (428 fixtures; 35 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (428 fixtures; 36 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived. diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d09f7917..42ce430fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -112,6 +112,24 @@ here.** and the `wire` setting, and a project on any port runs them through the Node `meta` CLI. They are reference helpers: opt-in, listed by `meta gen --list`, ejectable and gated by `verify --codegen` (ADR-0034 Amendment 10). Protobuf is not built. See `docs/features/contract-emitters.md`. +- **The data spine for a model explorer.** These changes are data only. No screen reads the new + data yet. + - `meta export --effective` writes the effective canonical JSON. Every field, child and attribute + a node inherits through `extends` is written onto that node. + - `meta verify --format json` adds `requirementLedger`, one row per requirement. A row carries its + digest, each `@implementedBy` link with the node the gate resolved it to, the findings against + it, and a `clean`, `warn` or `error` verdict. Text output is unchanged. + - `meta gen` records, in `.hashes.json`, the node each per-entity file was rendered from: + `{ "hash", "generatedBy", "sourceNode" }`. `EmittedFile.sourceNode` lets an owned generator set + it, and `perEntity` sets it automatically. A file rendered from the whole model keeps the plain + hash entry. The field is not called `generator`, because on the JVM that word claims ownership. + - `@metaobjectsdev/docs-site` exports `buildExplorer` and `serializeExplorer`. Together they build a + deterministic `explorer.json` from those three inputs. The file holds nodes, typed edges, the + requirement ledger, the generated files for each node, and derived slices for each package, + requirement subtree and prompt. + + Gated by the new `fixtures/explorer-conformance/` corpus. See `docs/features/explorer-data.md`. + No metamodel vocabulary changes, and `metamodelVersion` does not move. - **A recipe for checking an API change against the last release.** Emit the document at the last tag and at `HEAD`, then run oasdiff. See `docs/recipes/check-api-against-last-release.md`. - **A `pydantic` extra on the Python package.** `pip install "metaobjects[pydantic]"` installs diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 0addaf83d..f7200848b 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -1,6 +1,6 @@ # Conformance coverage -The MetaObjects standard ships **35 shared conformance corpora** under +The MetaObjects standard ships **36 shared conformance corpora** under [`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to it* and asserts the same expected behaviour against the same fixtures. **This page is the inverse index**: fixture → feature doc + per-port pass status, and it is the @@ -61,6 +61,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/codegen-noop/`](../fixtures/codegen-noop/) (FR-044 — reporting vocabulary: what is lowered, what stays inert) | 1 model pair (`reporting/with` vs `reporting/without`) | ✓ (codegen + migrate) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/cube-model/`](../fixtures/cube-model/) (FR-044 Plan 4 — the `cube-model` generator's golden Cube files, plus the canonical model's golden) | 46 cases (36 trees + 10 errors) + 1 canonical golden | ✓ (the exporter is TypeScript-only, spec R6; its `cube` lane also loads the files into a real Cube) | — | — | — | — | | [`fixtures/contract-conformance/`](../fixtures/contract-conformance/) (FR-022 — the `contract-schema` and `openapi` generators' golden trees and refusals) | 19 cases (14 trees + 5 errors) | ✓ (one neutral engine, ADR-0034 Amendment 10; the generated API lanes also hold real traffic to the documents) | — | — | — | — | +| [`fixtures/explorer-conformance/`](../fixtures/explorer-conformance/) (the `explorer.json` data spine: `meta export --effective` + `meta verify --format json` + the generated-file manifest in, one explorer document out) | 1 case (`shop`) | ✓ (the builder is TypeScript, and reads only port-neutral inputs) | — | — | — | — | A ✓ means the port runs the corpus green; an explicit `n / m` is used where a port carries a ledgered divergence. The two ledgered YAML fixtures are documented @@ -610,12 +611,28 @@ message. The case list, with the rule each pins, is the (`server/typescript/packages/integration-tests/cube-live/cube-model.live.ts`) loads the same files into a real Cube. +### `fixtures/explorer-conformance/` (1 case) + +The `shop` case → [features/explorer-data.md](features/explorer-data.md). Its `project/` is a +small model with inheritance, a value object, a reference, a prompt with a payload and a reply, +and a five-level requirement tree that has a resolved member claim, a dangling planned claim and a +live requirement nothing implements. `effective.json` and `verify.json` are what +`meta export --effective` and `meta verify --format json` write for it, `hashes.json` is a +hand-written manifest with node provenance, and `expected.json` is the explorer document built +from the three, compared byte for byte. + +**One port runs it:** TypeScript, in +`server/typescript/packages/docs-site/test/explorer-build.test.ts` (the builder) and +`server/typescript/packages/cli/test/explorer-fixture.test.ts` (the two committed inputs are +what the CLI writes today). The builder reads no loader, so a port that writes the same effective +JSON and verify payload feeds it unchanged. + ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 428 + yaml 16 + verify 45 + render 18 + persistence 59 + api-contract 119 + source-resolution 25 + scope 10 + -dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the -`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining +dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the +`cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider composition, agent context, docs emit) rather than user-facing metamodel behaviour, so they have no feature-doc row. diff --git a/docs/README.md b/docs/README.md index 181636b8d..b71020904 100644 --- a/docs/README.md +++ b/docs/README.md @@ -47,6 +47,7 @@ docs/ │ ├── reporting.md # dimensions, measures, segments, reports: vocabulary and load rules (FR-044) │ ├── contract-emitters.md # the contract-schema and openapi generators: JSON Schema and OpenAPI for the REST contract (FR-022) │ ├── cube-export.md # the cube-model generator: reporting vocabulary as Cube data-model files (FR-044) +│ ├── explorer-data.md # explorer.json and its inputs: export --effective, the verify ledger rows, manifest provenance │ └── own-your-codegen.md # scaffold-and-own generator ownership (ADR-0034) └── ports/ # one file per language/framework port ├── typescript.md @@ -78,6 +79,7 @@ this tree is documentation, not the source of truth. | Keep Prisma as the ORM, with `schema.prisma` generated from the model and gated by `verify` (TypeScript) | [`recipes/prisma.md`](recipes/prisma.md) | | Publish the API as JSON Schema or OpenAPI, and check a change against the last release | [`features/contract-emitters.md`](features/contract-emitters.md), [`recipes/check-api-against-last-release.md`](recipes/check-api-against-last-release.md) | | Export the reporting vocabulary to Cube (`cube-model`: cubes, joins, rollups; the `cube` lane that checks it against a real Cube) | [`features/cube-export.md`](features/cube-export.md) | +| Build the data a model explorer reads (`explorer.json` from `meta export --effective`, `meta verify --format json` and the generated-file manifest) | [`features/explorer-data.md`](features/explorer-data.md) | | Wire prompt construction (FR-004) | [`features/templates-and-payloads.md`](features/templates-and-payloads.md) | | Share a metadata shape across multiple instances (abstracts, `extends:`) | [`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md) | | Add a custom metamodel subtype or attribute to a downstream project | [`features/extending-with-providers.md`](features/extending-with-providers.md) + [`recipes/extending-metaobjects-with-providers.md`](recipes/extending-metaobjects-with-providers.md) | diff --git a/docs/features/cli.md b/docs/features/cli.md index b098c2c50..0e6d1de71 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -32,6 +32,7 @@ command surface splits in two: | **Codegen drift** (`verify --codegen`) | **Node `meta`** | `meta verify --codegen` | TS reference (ADR-0021 D2) — regen-to-temp + diff committed output | | **Template/prompt drift** (`verify --templates`) | **Node `meta`** | `meta verify --templates` | TS reference (ADR-0021 D2) — `{{field}}`↔payload; part of the bare-`verify` default | | **Vocabulary upgrade** (`upgrade`) | **Node `meta`** | `meta upgrade [--to ] [--apply]` | **any backend** — rewrites RETIRED metadata vocabulary (`@violation` → `@counterexample`, `@readOnly` → `@mutability`, dropping `@verifiedBy`) and resolves ATTRIBUTE CONTRADICTIONS (`@fields` beside `@expr` on an index key). Node-only because it edits the metadata documents themselves, which every port shares; a non-TS project runs `npx meta upgrade` against its own `metaobjects/`. **Canonical JSON and YAML alike.** Previews by default. Retirements needing a human decision are refused and the run exits non-zero, so CI cannot record a partial migration as finished | +| **Model export** (`export`) | **Node `meta`** | `meta export [--effective] [--out ]` | the loaded model, overlays merged, as one canonical JSON document. `--effective` also writes every inherited field, child and attribute onto the node that inherits it, which is the input the [explorer builder](explorer-data.md) reads. The output is for reading: loading an effective document back would declare each inherited member twice | | **Vocabulary search** (`types`) | **Node `meta`** | `meta types [query]` | **any backend** — apropos/`kubectl explain` over the live metamodel registry (names + descriptions + when-to-use); the vocabulary is cross-port identical (registry-conformance) | | **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency (`path`, or an installed `npm` / `python` package) into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript, Python and Java then LOAD that snapshot at every rung of their own source ladder (Kotlin loads through Java's loader and Maven plugin); C# loads the snapshot through `dotnet meta` (Phase 2 S6). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) | | **Canonical format** (`fmt`) | every port | `meta fmt` / `dotnet meta fmt` / `mvn metaobjects:fmt` / `metaobjects fmt` | (#304) ships in **every** port — the canonical serializer already exists in all five, so formatting is local, not Node-only. `--check` lists non-canonical files and exits non-zero without changing anything. Formats each file STANDALONE — own-mode, declared-here layer only (ADR-0039); an `extends` onto another file's base is preserved as-is, never resolved or erred on; a file declaring `overlay: true` anywhere — whether or not a same-file base exists — is reported as skipped rather than merged (merging would drop the overlay marker, a structural change fmt never makes). A YAML file is always skipped — no canonical YAML emitter exists (ADR-0006). Before writing, every port reloads the whole project with the candidate swapped in and refuses the write unless the reloaded model's canonical form is byte-identical to the untouched baseline — fmt never changes meaning. See [Fixture corpus](#meta-fmt--canonical-formatting-304) below and `fixtures/fmt-conformance/README.md` | diff --git a/docs/features/explorer-data.md b/docs/features/explorer-data.md new file mode 100644 index 000000000..fd55f77fa --- /dev/null +++ b/docs/features/explorer-data.md @@ -0,0 +1,228 @@ +# Explorer data: `explorer.json` and the three inputs it reads + +A model explorer shows one project so a reader can see what it does and how it was designed: +the objects and how they connect, the prompts and what they send and get back, the requirement +tree and whether each claim holds, and which generated files each node produced. All of that +is read from one data file, `explorer.json`. This page documents that file and the three +inputs it is built from. + +This is the data layer only. No screen reads it yet. + +## The inputs are port-neutral + +The builder (`buildExplorer` in `@metaobjectsdev/docs-site`) never loads metadata itself. It +reads three documents: + +| Input | Produced by | What it contributes | +| --- | --- | --- | +| Effective canonical JSON | `meta export --effective` | nodes, inheritance, every model edge | +| The verify payload | `meta verify --format json` | the requirement ledger, and the `implementedBy` / `supersededBy` edges | +| The generated-file manifest | `.metaobjects/.gen-state/.hashes.json`, written by `meta gen` | which generated files each node produced | + +Only the first is required. Without the verify payload there is no `ledger` and no requirement +claim edge; without the manifest there is no `generated` section. + +Because the builder reads no loader and no provider, it does not depend on which port wrote +its inputs. Every port has a canonical serializer and writes the same manifest file. + +### `meta export --effective` + +``` +meta export --effective [--out ] +``` + +`meta export` writes the loaded model as one canonical JSON document, with overlays merged. +By default each node holds only what it declares. With `--effective`, every field, child and +attribute a node inherits through `extends` is written onto that node as well. An entity that +extends `BaseEntity` then lists `id` and `createdAt` itself, and `extends` is still written, +as the canonical fully-qualified name. Reference attributes (`@references`, `@objectRef`, +`@payloadRef`, `@responseRef`) are written in their canonical form in both modes. + +The effective document is for reading, not for loading back. A loader reading it would see +each inherited member twice, once on the base and once on the subtype. + +### `meta verify --format json`: `requirementLedger` + +When the model declares a requirement, the structured verify payload carries one +`requirementLedger` row per `requirement.*` node, in document order, beside `requirementCounts`. +The text output is unchanged. Each row states what the gate decided for that entry, so a reader +does not have to infer "checked and clean" from a path that no diagnostic mentions. + +```json +{ + "path": "Shop.Checkout.Orders.Recorded.Quotable", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "subType": "functional", + "level": 5, + "status": "live", + "digest": "<64 hex characters>", + "implementedBy": [ + { "ref": "Order.reference", "resolves": true, "target": "acme::shop::Order.reference" } + ], + "diagnostics": [], + "verdict": "clean" +} +``` + +| Field | Meaning | +| --- | --- | +| `path` | The dotted child-name address every diagnostic prints. | +| `package` | The requirement's effective package, `""` when it has none. | +| `qualifiedPath` | `::`. It is also the prefix of each generated requirement test's id, ` []`. | +| `level`, `status`, `disposition` | As declared, and absent when not declared. | +| `digest` | The same SHA-256 over the claim that each generated requirement test carries ([requirements.md](requirements.md)). | +| `implementedBy` | Each link as written (`ref`), whether it resolves, and the node it resolves to (`target`): the object's resolution key, followed by `.` for a member. The gate's own resolver produces these values. | +| `supersededBy` | The same, for `@supersededBy`; its `target` is the replacement's `qualifiedPath`. | +| `diagnostics` | Every finding reported against this requirement, as `{ code, severity, source }`. `source` is `gate` when the finding can fail the build and `lint` when it cannot. | +| `verdict` | From gate findings only: `error`, `warn`, or `clean`. | + +A finding that is not about a single requirement, such as an entity nobody claims, appears +in `requirements.rows` and in no ledger row. The structured payload is TypeScript-only, as +the rest of `--format json` is. + +### Node provenance in `.hashes.json` + +Each generated file now records the metadata node it was rendered from. A generator built with +`perEntity` sets this automatically. An `EmittedFile` can also name its node directly with +`sourceNode`, which is a node resolution key such as `acme::shop::Order`. `meta gen` then +writes that file's manifest entry in object form: + +```json +{ + "src/generated/acme/shop/Order.ts": { + "hash": "", + "generatedBy": "entity", + "sourceNode": "acme::shop::Order" + }, + "src/generated/index.ts": "" +} +``` + +A file rendered from the whole model or a whole package keeps the plain `"": ""` +form, so a project that writes only such files has an unchanged manifest. Every port's +manifest reader takes `hash` from either form. + +The provenance field is named `generatedBy`, not `generator`. In a Java or Kotlin module's +manifest, `generator` means that generator owns the file (ADR-0060), and the orphan sweep acts +on that ownership. TypeScript does not make that claim. TypeScript is the only port that writes +provenance today, and the JVM rewrites its manifest without these fields. + +## The `explorer.json` shape + +```ts +interface ExplorerJson { + format: "metaobjects-explorer/1"; + nodes: ExplorerNode[]; + edges: ExplorerEdge[]; + ledger?: ExplorerLedger; // absent without a verify payload with requirements + generated?: ExplorerGenerated[]; // absent without a manifest + slices: ExplorerSlice[]; +} +``` + +The types are exported from `@metaobjectsdev/docs-site`. The `format` value changes when a field +changes meaning or is removed. Adding a field does not change it. + +### `nodes` + +The output has one node for each top-level node (object, prompt, requirement) and one for +each node nested inside one (field, identity, source, view, index, nested requirement). + +| Field | Meaning | +| --- | --- | +| `id` | A top-level node's id is `::`. A nested node's id is its parent's id followed by `.`, for example `acme::shop::Order.total`. A nested requirement's id equals its ledger row's `qualifiedPath`. A nested node with no name gets `#` in place of `.`. When siblings of different types share a name, the first one in the effective JSON keeps `.`, because that is the node a dotted `@implementedBy` reference such as `Order.id` resolves to. Each later one adds its type, for example `acme::shop::AuditEntry.id[identity.primary]`. | +| `type`, `subType`, `name`, `package` | As in the model. `package` is the package of the top-level ancestor. | +| `parent` | The id of the node it is nested in. A top-level node has no `parent`. | +| `extends`, `abstract`, `isArray` | As in the effective JSON. | +| `inheritedFrom` | Set on a member that came from a super. It holds the id of the object the member is declared on, meaning the topmost ancestor that has a member with the same type, name and body. A member redeclared with a different body belongs to the node itself. | +| `attrs` | Every `@` attribute, without the `@`, with keys sorted. | + +Two nodes that would still share an id are an error: two top-level nodes with one name in one package, or two siblings with the same type and name. The builder throws and names the id. + +### `edges` + +Each edge is typed. An edge is emitted only when both its ends are nodes in the file. + +| `kind` | From → to | Source | +| --- | --- | --- | +| `extends` | node → its super | `extends` | +| `references` | object → referenced object, `via` the `identity.reference` | `@references` | +| `objectRef` | object → value or entity, `via` the `field.object` | `@objectRef` on a field | +| `association` | object → target, `via` the `relationship.*` | `@objectRef` on a relationship | +| `payload` | prompt → payload object | `@payloadRef` | +| `reply` | prompt → reply object | `@responseRef` | +| `contains` | requirement → nested requirement | nesting | +| `implementedBy` | requirement → object or member | a ledger row's resolved `target` | +| `supersededBy` | requirement → replacing requirement | a ledger row's resolved `target` | + +An `implementedBy` edge exists only when the verify payload resolved that link. The builder +never matches names itself. A planned requirement that names a node that does not exist yet +has a ledger row whose link is `"resolves": false`, and it has no edge. + +### `ledger` + +`counts` is `requirementCounts`, copied as is. `rows` holds the `requirementLedger` rows, each +with a `node` id added. `diagnostics` is `requirements.rows`, copied as is: every gate and lint +finding, including those about something other than a single requirement. + +### `generated` + +The manifest entries that carry `sourceNode`, grouped by node: `{ node, files: [{ path, +generatedBy }] }`, sorted by node and then by path. A plain entry has no node and is left out. + +### `slices` + +Slices are named node sets that a view can open on. They are all derived from the model, so +none needs to be written by hand: + +- `package:`: the top-level nodes of one package. +- `requirement:`: one for every requirement that has nested requirements. It holds the + subtree, every node the subtree claims, and the object that owns each claimed member. +- `prompt:`: one for every prompt. It holds the prompt, its payload and its reply. + +The design report also proposes slices saved by hand. Those belong in tool configuration, +never in metadata, and are not built yet. + +### Determinism + +The same three inputs always produce the same bytes. Nodes are sorted by id. Edges are sorted +by kind, then `from`, then `to`, then `via`, with duplicates removed. Slices are sorted by id, +and ledger rows stay in the verify payload's order. `serializeExplorer` writes JSON indented +two spaces, with one trailing newline. That makes the file safe to commit and diff. + +## Building it + +```ts +import { readFileSync, writeFileSync } from "node:fs"; +import { buildExplorer, serializeExplorer } from "@metaobjectsdev/docs-site"; + +const read = (f: string) => JSON.parse(readFileSync(f, "utf8")); +writeFileSync("explorer.json", serializeExplorer(buildExplorer({ + model: read("effective.json"), // meta export --effective --out effective.json + verify: read("verify.json"), // meta verify --format json > verify.json + manifest: read(".metaobjects/.gen-state/.hashes.json"), +}))); +``` + +A `verify` or `manifest` that is missing or malformed leaves its section out. A `model` that is +not canonical metadata JSON makes the builder throw. + +## Not built yet + +- **`usage`**: which hand-written modules import a generated module. The design marks it + optional, and nothing builds it yet. +- **Routes per node** in `generated`: the manifest records files, not the routes inside them. +- **Provenance from the JVM, Python and C# generators.** Their manifests have no `sourceNode` + yet, so for those ports `generated` is empty. + +## Verified by + +- [`fixtures/explorer-conformance/shop/`](../../fixtures/explorer-conformance/shop/): a small + model with inheritance, a prompt, and a five-level requirement tree. Its `expected.json` is + checked byte for byte. `server/typescript/packages/cli/test/explorer-fixture.test.ts` checks + that the CLI still writes the committed `effective.json` and `verify.json`, and checks the + `requirementLedger` rows. +- `server/typescript/packages/codegen-ts/test/generated-provenance.test.ts`: covers how + `perEntity` stamps each file, the object and plain manifest forms, and that JVM owner fields + are preserved. diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 8edc20fae..0371bd297 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -511,6 +511,11 @@ overwritten; an edited one is refused by name (`keep`, th replaced (`overwrite`). There is **no three-way merge** (TypeScript-only). **Commit the manifest.** +A TypeScript entry can also carry the node the file was rendered from, +`{ "hash", "generatedBy", "sourceNode" }`. That is provenance, not ownership, which is why it +does not use the JVM's `generator` field. Every reader takes `hash` from either form. See +[explorer-data.md](explorer-data.md#node-provenance-in-hashesjson). + The `GENERATED` token in the header still matters at one moment: a file at a generated path that the manifest has no record of. A file that carries the token is taken as generated output (**adopted**: written fresh and recorded); one that does not is refused, whatever diff --git a/docs/features/requirements.md b/docs/features/requirements.md index 721c2ed79..c7b7db9d3 100644 --- a/docs/features/requirements.md +++ b/docs/features/requirements.md @@ -397,7 +397,10 @@ A status with no entry is left out. When coverage is not measured the line ends counts the recorded gaps with no `@disposition`, when there are any. Then come the diagnostics, and when any is an error, ` error(s).` and a non-zero exit. TypeScript caps the warnings it prints at `--limit` (20 by default); the other ports print -every one. The structured `--format json|toon` payload is TypeScript-only. +every one. The structured `--format json|toon` payload is TypeScript-only. It carries one +`requirementLedger` row per requirement: its digest, each `@implementedBy` link with the node +it resolved to, the findings against it and a `clean` / `warn` / `error` verdict. See +[explorer-data.md](explorer-data.md#meta-verify---format-json-requirementledger). **Metadata that does not load** is not a model with no requirement: `verify` fails on it in every port. The load errors are printed once. Where the drift gate that ran loaded the model, diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index 0dc75bd7a..7a98e03d8 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -8,7 +8,7 @@ }, "counts": { "fixtures": 428, - "corpora": 35, + "corpora": 36, "baseTypes": 17 }, "snippets": { diff --git a/fixtures/explorer-conformance/README.md b/fixtures/explorer-conformance/README.md new file mode 100644 index 000000000..2dc475a8e --- /dev/null +++ b/fixtures/explorer-conformance/README.md @@ -0,0 +1,49 @@ +# explorer-conformance + +The `explorer.json` data spine: three port-neutral inputs in, one explorer document out, +compared byte for byte. The shape is documented in +[docs/features/explorer-data.md](../../docs/features/explorer-data.md). + +## `shop/` + +| File | What it is | How it is produced | +| --- | --- | --- | +| `project/` | A small project: an abstract `BaseEntity` two entities extend, a value object held by a `field.object`, an `identity.reference`, an index, an entity no requirement claims, a `template.prompt` with a payload and a reply, and a requirement tree five levels deep. | Hand-written. | +| `effective.json` | The effective canonical JSON. | `meta export --effective` from `project/`. | +| `verify.json` | The verify payload, with one `requirementLedger` row per requirement. | `meta verify --format json` from `project/`. | +| `hashes.json` | A generated-file manifest: three entries carry node provenance, one is a plain hash. | Hand-written (the project wires no generators). | +| `expected.json` | The explorer document. | `buildExplorer` over the three files above. | + +What the requirement tree exercises: + +- `Shop.Checkout.Orders.Recorded` claims two objects, and its child `Quotable` claims a member + (`Order.reference`), so the slice for `Recorded` also holds `Order`, the member's owner. +- `Shop.Checkout.Orders.Refunded` is `planned` and names `Refund`, which does not exist: the + ledger row says the link does not resolve, and no edge is drawn. +- `Shop.Checkout.Shipping` is `live` and nothing under it implements anything: its row's verdict + is `warn` (`WARN_REQUIREMENT_NOTHING_IMPLEMENTS`). +- `AuditEntry` is claimed by nothing, which the gate reports with no requirement path; it lands + in `ledger.diagnostics` and in no row. +- `AuditEntry` names its field and its primary identity both `id`. The field comes first and + keeps the id `acme::shop::AuditEntry.id`; the identity is `acme::shop::AuditEntry.id[identity.primary]`. + +## Regenerating + +From the repository root, after a deliberate change to either command's output or to the builder: + +``` +( cd fixtures/explorer-conformance/shop/project && \ + bun ../../../../server/typescript/packages/cli/bin/meta.ts export --effective --out ../effective.json && \ + bun ../../../../server/typescript/packages/cli/bin/meta.ts verify --format json > ../verify.json ) +``` + +Then rebuild `expected.json` with `buildExplorer` and `serializeExplorer` from +`@metaobjectsdev/docs-site`, and review the diff: every changed line is a change to what an +explorer shows. + +## Who runs it + +- `server/typescript/packages/docs-site/test/explorer-build.test.ts`: the builder reproduces + `expected.json` from the committed inputs. +- `server/typescript/packages/cli/test/explorer-fixture.test.ts`: the CLI still writes + `effective.json` and `verify.json` exactly, and the fresh inputs still build `expected.json`. diff --git a/fixtures/explorer-conformance/shop/effective.json b/fixtures/explorer-conformance/shop/effective.json new file mode 100644 index 000000000..2b3a0fb71 --- /dev/null +++ b/fixtures/explorer-conformance/shop/effective.json @@ -0,0 +1,348 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "BaseEntity", + "abstract": true, + "children": [ + { + "field.uuid": { + "name": "id" + } + }, + { + "field.timestamp": { + "name": "createdAt" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.value": { + "name": "Address", + "children": [ + { + "field.string": { + "name": "street" + } + }, + { + "field.string": { + "name": "city" + } + } + ] + } + }, + { + "object.entity": { + "name": "Customer", + "extends": "acme::shop::BaseEntity", + "children": [ + { + "field.uuid": { + "name": "id" + } + }, + { + "field.timestamp": { + "name": "createdAt" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "field.string": { + "name": "email" + } + }, + { + "field.object": { + "name": "shippingAddress", + "@objectRef": "acme::shop::Address" + } + }, + { + "source.rdb": { + "name": "db", + "@table": "customers" + } + } + ] + } + }, + { + "object.entity": { + "name": "Order", + "extends": "acme::shop::BaseEntity", + "children": [ + { + "field.uuid": { + "name": "id" + } + }, + { + "field.timestamp": { + "name": "createdAt" + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + }, + { + "field.uuid": { + "name": "customerId" + } + }, + { + "field.string": { + "name": "reference" + } + }, + { + "field.decimal": { + "name": "total" + } + }, + { + "field.enum": { + "name": "state", + "@values": [ + "OPEN", + "PAID", + "SHIPPED" + ] + } + }, + { + "identity.reference": { + "name": "customerRef", + "@fields": [ + "customerId" + ], + "@references": "acme::shop::Customer" + } + }, + { + "index.lookup": { + "name": "orderByCustomerId", + "@fields": [ + "customerId" + ] + } + }, + { + "source.rdb": { + "name": "db", + "@table": "orders" + } + } + ] + } + }, + { + "object.entity": { + "name": "AuditEntry", + "children": [ + { + "field.uuid": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "field.string": { + "name": "message" + } + }, + { + "source.rdb": { + "name": "db", + "@table": "audit_entries" + } + } + ] + } + }, + { + "object.value": { + "name": "OrderNoteRequest", + "children": [ + { + "field.string": { + "name": "reference", + "@required": true + } + } + ] + } + }, + { + "object.value": { + "name": "OrderNote", + "children": [ + { + "field.string": { + "name": "text", + "@required": true + } + } + ] + } + }, + { + "template.prompt": { + "name": "OrderNotePrompt", + "@format": "text", + "@payloadRef": "acme::shop::OrderNoteRequest", + "@responseFormat": "json", + "@responseRef": "acme::shop::OrderNote", + "@textRef": "shop/order-note" + } + }, + { + "requirement.architectural": { + "name": "Addressable", + "@counterexample": "A row nobody can point at.", + "@implementedBy": [ + "BaseEntity" + ], + "@statement": "Every stored row has a stable id.", + "@status": "live" + } + }, + { + "requirement.functional": { + "name": "Shop", + "@counterexample": "A customer who cannot place an order.", + "@level": 1, + "@statement": "A customer can buy from the shop.", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "Checkout", + "@counterexample": "An order taken outside checkout.", + "@disposition": "deferred", + "@level": 2, + "@statement": "Orders are taken at checkout.", + "@status": "partial", + "@trackedBy": [ + "https://example.invalid/issues/1" + ], + "children": [ + { + "requirement.functional": { + "name": "Orders", + "@counterexample": "A placed order that is lost.", + "@level": 3, + "@statement": "Every placed order is kept.", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "Recorded", + "@counterexample": "A placed order has no row.", + "@implementedBy": [ + "Order", + "Customer" + ], + "@level": 4, + "@statement": "An order is recorded when it is placed.", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "Quotable", + "@counterexample": "An order nobody can quote back.", + "@implementedBy": [ + "Order.reference" + ], + "@level": 5, + "@statement": "An order carries a reference the customer can quote.", + "@status": "live" + } + } + ] + } + }, + { + "requirement.functional": { + "name": "Noted", + "@counterexample": "An order with no note.", + "@implementedBy": [ + "OrderNotePrompt" + ], + "@level": 4, + "@statement": "A placed order gets a short note for the customer.", + "@status": "live" + } + }, + { + "requirement.functional": { + "name": "Refunded", + "@counterexample": "A paid order that cannot be refunded.", + "@implementedBy": [ + "Refund" + ], + "@level": 4, + "@statement": "A paid order can be refunded.", + "@status": "planned" + } + } + ] + } + }, + { + "requirement.functional": { + "name": "Shipping", + "@counterexample": "A paid order that never ships.", + "@level": 3, + "@statement": "A paid order is shipped.", + "@status": "live" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/explorer-conformance/shop/expected.json b/fixtures/explorer-conformance/shop/expected.json new file mode 100644 index 000000000..d28845d4f --- /dev/null +++ b/fixtures/explorer-conformance/shop/expected.json @@ -0,0 +1,936 @@ +{ + "format": "metaobjects-explorer/1", + "nodes": [ + { + "id": "acme::shop::Address", + "type": "object", + "subType": "value", + "name": "Address", + "package": "acme::shop", + "attrs": {} + }, + { + "id": "acme::shop::Address.city", + "type": "field", + "subType": "string", + "name": "city", + "package": "acme::shop", + "parent": "acme::shop::Address", + "attrs": {} + }, + { + "id": "acme::shop::Address.street", + "type": "field", + "subType": "string", + "name": "street", + "package": "acme::shop", + "parent": "acme::shop::Address", + "attrs": {} + }, + { + "id": "acme::shop::Addressable", + "type": "requirement", + "subType": "architectural", + "name": "Addressable", + "package": "acme::shop", + "attrs": { + "counterexample": "A row nobody can point at.", + "implementedBy": [ + "BaseEntity" + ], + "statement": "Every stored row has a stable id.", + "status": "live" + } + }, + { + "id": "acme::shop::AuditEntry", + "type": "object", + "subType": "entity", + "name": "AuditEntry", + "package": "acme::shop", + "attrs": {} + }, + { + "id": "acme::shop::AuditEntry.db", + "type": "source", + "subType": "rdb", + "name": "db", + "package": "acme::shop", + "parent": "acme::shop::AuditEntry", + "attrs": { + "table": "audit_entries" + } + }, + { + "id": "acme::shop::AuditEntry.id", + "type": "field", + "subType": "uuid", + "name": "id", + "package": "acme::shop", + "parent": "acme::shop::AuditEntry", + "attrs": {} + }, + { + "id": "acme::shop::AuditEntry.id[identity.primary]", + "type": "identity", + "subType": "primary", + "name": "id", + "package": "acme::shop", + "parent": "acme::shop::AuditEntry", + "attrs": { + "fields": [ + "id" + ] + } + }, + { + "id": "acme::shop::AuditEntry.message", + "type": "field", + "subType": "string", + "name": "message", + "package": "acme::shop", + "parent": "acme::shop::AuditEntry", + "attrs": {} + }, + { + "id": "acme::shop::BaseEntity", + "type": "object", + "subType": "entity", + "name": "BaseEntity", + "package": "acme::shop", + "abstract": true, + "attrs": {} + }, + { + "id": "acme::shop::BaseEntity.createdAt", + "type": "field", + "subType": "timestamp", + "name": "createdAt", + "package": "acme::shop", + "parent": "acme::shop::BaseEntity", + "attrs": {} + }, + { + "id": "acme::shop::BaseEntity.id", + "type": "field", + "subType": "uuid", + "name": "id", + "package": "acme::shop", + "parent": "acme::shop::BaseEntity", + "attrs": {} + }, + { + "id": "acme::shop::BaseEntity.pk", + "type": "identity", + "subType": "primary", + "name": "pk", + "package": "acme::shop", + "parent": "acme::shop::BaseEntity", + "attrs": { + "fields": [ + "id" + ] + } + }, + { + "id": "acme::shop::Customer", + "type": "object", + "subType": "entity", + "name": "Customer", + "package": "acme::shop", + "extends": "acme::shop::BaseEntity", + "attrs": {} + }, + { + "id": "acme::shop::Customer.createdAt", + "type": "field", + "subType": "timestamp", + "name": "createdAt", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": {}, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Customer.db", + "type": "source", + "subType": "rdb", + "name": "db", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": { + "table": "customers" + } + }, + { + "id": "acme::shop::Customer.email", + "type": "field", + "subType": "string", + "name": "email", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": {} + }, + { + "id": "acme::shop::Customer.id", + "type": "field", + "subType": "uuid", + "name": "id", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": {}, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Customer.name", + "type": "field", + "subType": "string", + "name": "name", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": { + "required": true + } + }, + { + "id": "acme::shop::Customer.pk", + "type": "identity", + "subType": "primary", + "name": "pk", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": { + "fields": [ + "id" + ] + }, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Customer.shippingAddress", + "type": "field", + "subType": "object", + "name": "shippingAddress", + "package": "acme::shop", + "parent": "acme::shop::Customer", + "attrs": { + "objectRef": "acme::shop::Address" + } + }, + { + "id": "acme::shop::Order", + "type": "object", + "subType": "entity", + "name": "Order", + "package": "acme::shop", + "extends": "acme::shop::BaseEntity", + "attrs": {} + }, + { + "id": "acme::shop::Order.createdAt", + "type": "field", + "subType": "timestamp", + "name": "createdAt", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": {}, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Order.customerId", + "type": "field", + "subType": "uuid", + "name": "customerId", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": {} + }, + { + "id": "acme::shop::Order.customerRef", + "type": "identity", + "subType": "reference", + "name": "customerRef", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": { + "fields": [ + "customerId" + ], + "references": "acme::shop::Customer" + } + }, + { + "id": "acme::shop::Order.db", + "type": "source", + "subType": "rdb", + "name": "db", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": { + "table": "orders" + } + }, + { + "id": "acme::shop::Order.id", + "type": "field", + "subType": "uuid", + "name": "id", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": {}, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Order.orderByCustomerId", + "type": "index", + "subType": "lookup", + "name": "orderByCustomerId", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": { + "fields": [ + "customerId" + ] + } + }, + { + "id": "acme::shop::Order.pk", + "type": "identity", + "subType": "primary", + "name": "pk", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": { + "fields": [ + "id" + ] + }, + "inheritedFrom": "acme::shop::BaseEntity" + }, + { + "id": "acme::shop::Order.reference", + "type": "field", + "subType": "string", + "name": "reference", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": {} + }, + { + "id": "acme::shop::Order.state", + "type": "field", + "subType": "enum", + "name": "state", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": { + "values": [ + "OPEN", + "PAID", + "SHIPPED" + ] + } + }, + { + "id": "acme::shop::Order.total", + "type": "field", + "subType": "decimal", + "name": "total", + "package": "acme::shop", + "parent": "acme::shop::Order", + "attrs": {} + }, + { + "id": "acme::shop::OrderNote", + "type": "object", + "subType": "value", + "name": "OrderNote", + "package": "acme::shop", + "attrs": {} + }, + { + "id": "acme::shop::OrderNote.text", + "type": "field", + "subType": "string", + "name": "text", + "package": "acme::shop", + "parent": "acme::shop::OrderNote", + "attrs": { + "required": true + } + }, + { + "id": "acme::shop::OrderNotePrompt", + "type": "template", + "subType": "prompt", + "name": "OrderNotePrompt", + "package": "acme::shop", + "attrs": { + "format": "text", + "payloadRef": "acme::shop::OrderNoteRequest", + "responseFormat": "json", + "responseRef": "acme::shop::OrderNote", + "textRef": "shop/order-note" + } + }, + { + "id": "acme::shop::OrderNoteRequest", + "type": "object", + "subType": "value", + "name": "OrderNoteRequest", + "package": "acme::shop", + "attrs": {} + }, + { + "id": "acme::shop::OrderNoteRequest.reference", + "type": "field", + "subType": "string", + "name": "reference", + "package": "acme::shop", + "parent": "acme::shop::OrderNoteRequest", + "attrs": { + "required": true + } + }, + { + "id": "acme::shop::Shop", + "type": "requirement", + "subType": "functional", + "name": "Shop", + "package": "acme::shop", + "attrs": { + "counterexample": "A customer who cannot place an order.", + "level": 1, + "statement": "A customer can buy from the shop.", + "status": "live" + } + }, + { + "id": "acme::shop::Shop.Checkout", + "type": "requirement", + "subType": "functional", + "name": "Checkout", + "package": "acme::shop", + "parent": "acme::shop::Shop", + "attrs": { + "counterexample": "An order taken outside checkout.", + "disposition": "deferred", + "level": 2, + "statement": "Orders are taken at checkout.", + "status": "partial", + "trackedBy": [ + "https://example.invalid/issues/1" + ] + } + }, + { + "id": "acme::shop::Shop.Checkout.Orders", + "type": "requirement", + "subType": "functional", + "name": "Orders", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout", + "attrs": { + "counterexample": "A placed order that is lost.", + "level": 3, + "statement": "Every placed order is kept.", + "status": "live" + } + }, + { + "id": "acme::shop::Shop.Checkout.Orders.Noted", + "type": "requirement", + "subType": "functional", + "name": "Noted", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout.Orders", + "attrs": { + "counterexample": "An order with no note.", + "implementedBy": [ + "OrderNotePrompt" + ], + "level": 4, + "statement": "A placed order gets a short note for the customer.", + "status": "live" + } + }, + { + "id": "acme::shop::Shop.Checkout.Orders.Recorded", + "type": "requirement", + "subType": "functional", + "name": "Recorded", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout.Orders", + "attrs": { + "counterexample": "A placed order has no row.", + "implementedBy": [ + "Order", + "Customer" + ], + "level": 4, + "statement": "An order is recorded when it is placed.", + "status": "live" + } + }, + { + "id": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "type": "requirement", + "subType": "functional", + "name": "Quotable", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout.Orders.Recorded", + "attrs": { + "counterexample": "An order nobody can quote back.", + "implementedBy": [ + "Order.reference" + ], + "level": 5, + "statement": "An order carries a reference the customer can quote.", + "status": "live" + } + }, + { + "id": "acme::shop::Shop.Checkout.Orders.Refunded", + "type": "requirement", + "subType": "functional", + "name": "Refunded", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout.Orders", + "attrs": { + "counterexample": "A paid order that cannot be refunded.", + "implementedBy": [ + "Refund" + ], + "level": 4, + "statement": "A paid order can be refunded.", + "status": "planned" + } + }, + { + "id": "acme::shop::Shop.Checkout.Shipping", + "type": "requirement", + "subType": "functional", + "name": "Shipping", + "package": "acme::shop", + "parent": "acme::shop::Shop.Checkout", + "attrs": { + "counterexample": "A paid order that never ships.", + "level": 3, + "statement": "A paid order is shipped.", + "status": "live" + } + } + ], + "edges": [ + { + "kind": "contains", + "from": "acme::shop::Shop", + "to": "acme::shop::Shop.Checkout" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout", + "to": "acme::shop::Shop.Checkout.Orders" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout", + "to": "acme::shop::Shop.Checkout.Shipping" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout.Orders", + "to": "acme::shop::Shop.Checkout.Orders.Noted" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout.Orders", + "to": "acme::shop::Shop.Checkout.Orders.Recorded" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout.Orders", + "to": "acme::shop::Shop.Checkout.Orders.Refunded" + }, + { + "kind": "contains", + "from": "acme::shop::Shop.Checkout.Orders.Recorded", + "to": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable" + }, + { + "kind": "extends", + "from": "acme::shop::Customer", + "to": "acme::shop::BaseEntity" + }, + { + "kind": "extends", + "from": "acme::shop::Order", + "to": "acme::shop::BaseEntity" + }, + { + "kind": "implementedBy", + "from": "acme::shop::Addressable", + "to": "acme::shop::BaseEntity" + }, + { + "kind": "implementedBy", + "from": "acme::shop::Shop.Checkout.Orders.Noted", + "to": "acme::shop::OrderNotePrompt" + }, + { + "kind": "implementedBy", + "from": "acme::shop::Shop.Checkout.Orders.Recorded", + "to": "acme::shop::Customer" + }, + { + "kind": "implementedBy", + "from": "acme::shop::Shop.Checkout.Orders.Recorded", + "to": "acme::shop::Order" + }, + { + "kind": "implementedBy", + "from": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "to": "acme::shop::Order.reference" + }, + { + "kind": "objectRef", + "from": "acme::shop::Customer", + "to": "acme::shop::Address", + "via": "acme::shop::Customer.shippingAddress" + }, + { + "kind": "payload", + "from": "acme::shop::OrderNotePrompt", + "to": "acme::shop::OrderNoteRequest" + }, + { + "kind": "references", + "from": "acme::shop::Order", + "to": "acme::shop::Customer", + "via": "acme::shop::Order.customerRef" + }, + { + "kind": "reply", + "from": "acme::shop::OrderNotePrompt", + "to": "acme::shop::OrderNote" + } + ], + "ledger": { + "counts": { + "total": 9, + "functional": 8, + "architectural": 1, + "byStatus": [ + { + "status": "planned", + "count": 1 + }, + { + "status": "live", + "count": 7 + }, + { + "status": "partial", + "count": 1 + } + ], + "undecided": 1, + "entitiesClaimed": 2, + "entitiesTotal": 3, + "metadataFiles": 1 + }, + "rows": [ + { + "node": "acme::shop::Addressable", + "path": "Addressable", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Addressable", + "subType": "architectural", + "status": "live", + "digest": "c22897fd7559820a2a7a69bae54353900122be23f3d0058e9ec2b6195ae840e1", + "implementedBy": [ + { + "ref": "BaseEntity", + "resolves": true, + "target": "acme::shop::BaseEntity" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop", + "path": "Shop", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop", + "subType": "functional", + "level": 1, + "status": "live", + "digest": "edc8a9b8159eb03a08d020427683253d6d1471032e0cf48115956362fc121653", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout", + "path": "Shop.Checkout", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout", + "subType": "functional", + "level": 2, + "status": "partial", + "disposition": "deferred", + "digest": "111b8c39a19c717933fdd529af808540a27c0592406d0a792c524960ac775a2d", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Orders", + "path": "Shop.Checkout.Orders", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders", + "subType": "functional", + "level": 3, + "status": "live", + "digest": "e84c69fda54cb9f04f3949af069c2c5d56027e008be3198a08c4ee4fbcad8be6", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Orders.Recorded", + "path": "Shop.Checkout.Orders.Recorded", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Recorded", + "subType": "functional", + "level": 4, + "status": "live", + "digest": "6feebe1e415dda976cc6843c4546c1541b25e8ee44b85316811bfdb177f77635", + "implementedBy": [ + { + "ref": "Order", + "resolves": true, + "target": "acme::shop::Order" + }, + { + "ref": "Customer", + "resolves": true, + "target": "acme::shop::Customer" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "path": "Shop.Checkout.Orders.Recorded.Quotable", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "subType": "functional", + "level": 5, + "status": "live", + "digest": "b5d7fb7a9657bfda7b30e2629617bf5c56e4337ef7a06abfe55715613ecdda63", + "implementedBy": [ + { + "ref": "Order.reference", + "resolves": true, + "target": "acme::shop::Order.reference" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Orders.Noted", + "path": "Shop.Checkout.Orders.Noted", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Noted", + "subType": "functional", + "level": 4, + "status": "live", + "digest": "45ed514b57928304f2d8b7c69e4cc3e5196cd8131ea57bc5ececc8056f52b995", + "implementedBy": [ + { + "ref": "OrderNotePrompt", + "resolves": true, + "target": "acme::shop::OrderNotePrompt" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Orders.Refunded", + "path": "Shop.Checkout.Orders.Refunded", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Refunded", + "subType": "functional", + "level": 4, + "status": "planned", + "digest": "dcaacae2ee305ce97e804148559dfe32c13dc69083ff736f09cab1b60a8d7a74", + "implementedBy": [ + { + "ref": "Refund", + "resolves": false + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "node": "acme::shop::Shop.Checkout.Shipping", + "path": "Shop.Checkout.Shipping", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Shipping", + "subType": "functional", + "level": 3, + "status": "live", + "digest": "d7d26a3dd5661647327cec62f969b61c74346e1df1e954de4d8392705bdecf31", + "implementedBy": [], + "diagnostics": [ + { + "code": "WARN_REQUIREMENT_NOTHING_IMPLEMENTS", + "severity": "warn", + "source": "gate" + } + ], + "verdict": "warn" + } + ], + "diagnostics": [ + { + "code": "WARN_REQUIREMENT_NOTHING_IMPLEMENTS", + "path": "Shop.Checkout.Shipping", + "severity": "warn", + "source": "gate", + "message": "is 'live' but neither it nor anything nested under it names an implementing node. A functional requirement's check is existence — a subtree that claims nothing is a capability nobody built." + }, + { + "code": "WARN_REQUIREMENT_OBJECT_UNCLAIMED", + "path": "", + "severity": "warn", + "source": "gate", + "message": "no requirement claims 'acme::shop::AuditEntry'. Add it to an L4 requirement's 'implementedBy'." + } + ] + }, + "generated": [ + { + "node": "acme::shop::Customer", + "files": [ + { + "path": "src/generated/acme/shop/Customer.ts", + "generatedBy": "entity" + } + ] + }, + { + "node": "acme::shop::Order", + "files": [ + { + "path": "src/generated/acme/shop/Order.queries.ts", + "generatedBy": "queries" + }, + { + "path": "src/generated/acme/shop/Order.ts", + "generatedBy": "entity" + } + ] + } + ], + "slices": [ + { + "id": "package:acme::shop", + "kind": "package", + "anchor": "acme::shop", + "nodes": [ + "acme::shop::Address", + "acme::shop::Addressable", + "acme::shop::AuditEntry", + "acme::shop::BaseEntity", + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::OrderNote", + "acme::shop::OrderNotePrompt", + "acme::shop::OrderNoteRequest", + "acme::shop::Shop" + ] + }, + { + "id": "prompt:acme::shop::OrderNotePrompt", + "kind": "prompt", + "anchor": "acme::shop::OrderNotePrompt", + "nodes": [ + "acme::shop::OrderNote", + "acme::shop::OrderNotePrompt", + "acme::shop::OrderNoteRequest" + ] + }, + { + "id": "requirement:acme::shop::Shop", + "kind": "requirement", + "anchor": "acme::shop::Shop", + "nodes": [ + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::Order.reference", + "acme::shop::OrderNotePrompt", + "acme::shop::Shop", + "acme::shop::Shop.Checkout", + "acme::shop::Shop.Checkout.Orders", + "acme::shop::Shop.Checkout.Orders.Noted", + "acme::shop::Shop.Checkout.Orders.Recorded", + "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "acme::shop::Shop.Checkout.Orders.Refunded", + "acme::shop::Shop.Checkout.Shipping" + ] + }, + { + "id": "requirement:acme::shop::Shop.Checkout", + "kind": "requirement", + "anchor": "acme::shop::Shop.Checkout", + "nodes": [ + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::Order.reference", + "acme::shop::OrderNotePrompt", + "acme::shop::Shop.Checkout", + "acme::shop::Shop.Checkout.Orders", + "acme::shop::Shop.Checkout.Orders.Noted", + "acme::shop::Shop.Checkout.Orders.Recorded", + "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "acme::shop::Shop.Checkout.Orders.Refunded", + "acme::shop::Shop.Checkout.Shipping" + ] + }, + { + "id": "requirement:acme::shop::Shop.Checkout.Orders", + "kind": "requirement", + "anchor": "acme::shop::Shop.Checkout.Orders", + "nodes": [ + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::Order.reference", + "acme::shop::OrderNotePrompt", + "acme::shop::Shop.Checkout.Orders", + "acme::shop::Shop.Checkout.Orders.Noted", + "acme::shop::Shop.Checkout.Orders.Recorded", + "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "acme::shop::Shop.Checkout.Orders.Refunded" + ] + }, + { + "id": "requirement:acme::shop::Shop.Checkout.Orders.Recorded", + "kind": "requirement", + "anchor": "acme::shop::Shop.Checkout.Orders.Recorded", + "nodes": [ + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::Order.reference", + "acme::shop::Shop.Checkout.Orders.Recorded", + "acme::shop::Shop.Checkout.Orders.Recorded.Quotable" + ] + } + ] +} diff --git a/fixtures/explorer-conformance/shop/hashes.json b/fixtures/explorer-conformance/shop/hashes.json new file mode 100644 index 000000000..a07fb4fff --- /dev/null +++ b/fixtures/explorer-conformance/shop/hashes.json @@ -0,0 +1,18 @@ +{ + "src/generated/acme/shop/Customer.ts": { + "hash": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "generatedBy": "entity", + "sourceNode": "acme::shop::Customer" + }, + "src/generated/acme/shop/Order.queries.ts": { + "hash": "sha256:2222222222222222222222222222222222222222222222222222222222222222", + "generatedBy": "queries", + "sourceNode": "acme::shop::Order" + }, + "src/generated/acme/shop/Order.ts": { + "hash": "sha256:3333333333333333333333333333333333333333333333333333333333333333", + "generatedBy": "entity", + "sourceNode": "acme::shop::Order" + }, + "src/generated/index.ts": "sha256:4444444444444444444444444444444444444444444444444444444444444444" +} diff --git a/fixtures/explorer-conformance/shop/project/.metaobjects/config.json b/fixtures/explorer-conformance/shop/project/.metaobjects/config.json new file mode 100644 index 000000000..dea4b1e4a --- /dev/null +++ b/fixtures/explorer-conformance/shop/project/.metaobjects/config.json @@ -0,0 +1,4 @@ +{ + "schema_version": 1, + "sources": [] +} diff --git a/fixtures/explorer-conformance/shop/project/metaobjects/meta.shop.json b/fixtures/explorer-conformance/shop/project/metaobjects/meta.shop.json new file mode 100644 index 000000000..ce876bc67 --- /dev/null +++ b/fixtures/explorer-conformance/shop/project/metaobjects/meta.shop.json @@ -0,0 +1,186 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "BaseEntity", + "abstract": true, + "children": [ + { "field.uuid": { "name": "id" } }, + { "field.timestamp": { "name": "createdAt" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] + } + }, + { + "object.value": { + "name": "Address", + "children": [ + { "field.string": { "name": "street" } }, + { "field.string": { "name": "city" } } + ] + } + }, + { + "object.entity": { + "name": "Customer", + "extends": "BaseEntity", + "children": [ + { "field.string": { "name": "name", "@required": true } }, + { "field.string": { "name": "email" } }, + { "field.object": { "name": "shippingAddress", "@objectRef": "Address" } }, + { "source.rdb": { "name": "db", "@table": "customers" } } + ] + } + }, + { + "object.entity": { + "name": "Order", + "extends": "BaseEntity", + "children": [ + { "field.uuid": { "name": "customerId" } }, + { "field.string": { "name": "reference" } }, + { "field.decimal": { "name": "total" } }, + { "field.enum": { "name": "state", "@values": ["OPEN", "PAID", "SHIPPED"] } }, + { "identity.reference": { "name": "customerRef", "@fields": ["customerId"], "@references": "Customer" } }, + { "index.lookup": { "name": "orderByCustomerId", "@fields": ["customerId"] } }, + { "source.rdb": { "name": "db", "@table": "orders" } } + ] + } + }, + { + "object.entity": { + "name": "AuditEntry", + "children": [ + { "field.uuid": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "field.string": { "name": "message" } }, + { "source.rdb": { "name": "db", "@table": "audit_entries" } } + ] + } + }, + { + "object.value": { + "name": "OrderNoteRequest", + "children": [ + { "field.string": { "name": "reference", "@required": true } } + ] + } + }, + { + "object.value": { + "name": "OrderNote", + "children": [ + { "field.string": { "name": "text", "@required": true } } + ] + } + }, + { + "template.prompt": { + "name": "OrderNotePrompt", + "@payloadRef": "OrderNoteRequest", + "@responseRef": "OrderNote", + "@textRef": "shop/order-note", + "@format": "text", + "@responseFormat": "json" + } + }, + { + "requirement.architectural": { + "name": "Addressable", + "@status": "live", + "@statement": "Every stored row has a stable id.", + "@counterexample": "A row nobody can point at.", + "@implementedBy": ["BaseEntity"] + } + }, + { + "requirement.functional": { + "name": "Shop", + "@level": 1, + "@status": "live", + "@statement": "A customer can buy from the shop.", + "@counterexample": "A customer who cannot place an order.", + "children": [ + { + "requirement.functional": { + "name": "Checkout", + "@level": 2, + "@status": "partial", + "@disposition": "deferred", + "@trackedBy": ["https://example.invalid/issues/1"], + "@statement": "Orders are taken at checkout.", + "@counterexample": "An order taken outside checkout.", + "children": [ + { + "requirement.functional": { + "name": "Orders", + "@level": 3, + "@status": "live", + "@statement": "Every placed order is kept.", + "@counterexample": "A placed order that is lost.", + "children": [ + { + "requirement.functional": { + "name": "Recorded", + "@level": 4, + "@status": "live", + "@statement": "An order is recorded when it is placed.", + "@counterexample": "A placed order has no row.", + "@implementedBy": ["Order", "Customer"], + "children": [ + { + "requirement.functional": { + "name": "Quotable", + "@level": 5, + "@status": "live", + "@statement": "An order carries a reference the customer can quote.", + "@counterexample": "An order nobody can quote back.", + "@implementedBy": ["Order.reference"] + } + } + ] + } + }, + { + "requirement.functional": { + "name": "Noted", + "@level": 4, + "@status": "live", + "@statement": "A placed order gets a short note for the customer.", + "@counterexample": "An order with no note.", + "@implementedBy": ["OrderNotePrompt"] + } + }, + { + "requirement.functional": { + "name": "Refunded", + "@level": 4, + "@status": "planned", + "@statement": "A paid order can be refunded.", + "@counterexample": "A paid order that cannot be refunded.", + "@implementedBy": ["Refund"] + } + } + ] + } + }, + { + "requirement.functional": { + "name": "Shipping", + "@level": 3, + "@status": "live", + "@statement": "A paid order is shipped.", + "@counterexample": "A paid order that never ships." + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/explorer-conformance/shop/project/prompts/shop/order-note.mustache b/fixtures/explorer-conformance/shop/project/prompts/shop/order-note.mustache new file mode 100644 index 000000000..62faf4546 --- /dev/null +++ b/fixtures/explorer-conformance/shop/project/prompts/shop/order-note.mustache @@ -0,0 +1 @@ +Write a short note for order {{reference}}. diff --git a/fixtures/explorer-conformance/shop/verify.json b/fixtures/explorer-conformance/shop/verify.json new file mode 100644 index 000000000..54702a591 --- /dev/null +++ b/fixtures/explorer-conformance/shop/verify.json @@ -0,0 +1,259 @@ +{ + "verify": [ + { + "gate": "templates", + "ran": true, + "ok": true + }, + { + "gate": "schema", + "ran": false, + "ok": true + }, + { + "gate": "codegen", + "ran": false, + "ok": true + }, + { + "gate": "deps", + "ran": false, + "ok": true + }, + { + "gate": "requirements", + "ran": true, + "ok": true + }, + { + "gate": "replay", + "ran": false, + "ok": true + } + ], + "exitCode": 0, + "summary": "2 gate(s) ran, all clean; 2 requirement diagnostic(s)", + "help": [], + "antiPatterns": { + "status": "ran", + "total": 0, + "rows": [] + }, + "requirements": { + "status": "ran", + "total": 2, + "rows": [ + { + "code": "WARN_REQUIREMENT_NOTHING_IMPLEMENTS", + "path": "Shop.Checkout.Shipping", + "severity": "warn", + "source": "gate", + "message": "is 'live' but neither it nor anything nested under it names an implementing node. A functional requirement's check is existence — a subtree that claims nothing is a capability nobody built." + }, + { + "code": "WARN_REQUIREMENT_OBJECT_UNCLAIMED", + "path": "", + "severity": "warn", + "source": "gate", + "message": "no requirement claims 'acme::shop::AuditEntry'. Add it to an L4 requirement's 'implementedBy'." + } + ] + }, + "overlays": { + "status": "ran", + "total": 0, + "rows": [] + }, + "names": { + "status": "ran", + "total": 0, + "rows": [] + }, + "deprecations": { + "status": "ran", + "total": 0, + "rows": [] + }, + "fields": { + "status": "ran", + "total": 0, + "rows": [] + }, + "requirementCounts": { + "total": 9, + "functional": 8, + "architectural": 1, + "byStatus": [ + { + "status": "planned", + "count": 1 + }, + { + "status": "live", + "count": 7 + }, + { + "status": "partial", + "count": 1 + } + ], + "undecided": 1, + "entitiesClaimed": 2, + "entitiesTotal": 3, + "metadataFiles": 1 + }, + "requirementLedger": [ + { + "path": "Addressable", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Addressable", + "subType": "architectural", + "status": "live", + "digest": "c22897fd7559820a2a7a69bae54353900122be23f3d0058e9ec2b6195ae840e1", + "implementedBy": [ + { + "ref": "BaseEntity", + "resolves": true, + "target": "acme::shop::BaseEntity" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop", + "subType": "functional", + "level": 1, + "status": "live", + "digest": "edc8a9b8159eb03a08d020427683253d6d1471032e0cf48115956362fc121653", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout", + "subType": "functional", + "level": 2, + "status": "partial", + "disposition": "deferred", + "digest": "111b8c39a19c717933fdd529af808540a27c0592406d0a792c524960ac775a2d", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Orders", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders", + "subType": "functional", + "level": 3, + "status": "live", + "digest": "e84c69fda54cb9f04f3949af069c2c5d56027e008be3198a08c4ee4fbcad8be6", + "implementedBy": [], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Orders.Recorded", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Recorded", + "subType": "functional", + "level": 4, + "status": "live", + "digest": "6feebe1e415dda976cc6843c4546c1541b25e8ee44b85316811bfdb177f77635", + "implementedBy": [ + { + "ref": "Order", + "resolves": true, + "target": "acme::shop::Order" + }, + { + "ref": "Customer", + "resolves": true, + "target": "acme::shop::Customer" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Orders.Recorded.Quotable", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + "subType": "functional", + "level": 5, + "status": "live", + "digest": "b5d7fb7a9657bfda7b30e2629617bf5c56e4337ef7a06abfe55715613ecdda63", + "implementedBy": [ + { + "ref": "Order.reference", + "resolves": true, + "target": "acme::shop::Order.reference" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Orders.Noted", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Noted", + "subType": "functional", + "level": 4, + "status": "live", + "digest": "45ed514b57928304f2d8b7c69e4cc3e5196cd8131ea57bc5ececc8056f52b995", + "implementedBy": [ + { + "ref": "OrderNotePrompt", + "resolves": true, + "target": "acme::shop::OrderNotePrompt" + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Orders.Refunded", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Orders.Refunded", + "subType": "functional", + "level": 4, + "status": "planned", + "digest": "dcaacae2ee305ce97e804148559dfe32c13dc69083ff736f09cab1b60a8d7a74", + "implementedBy": [ + { + "ref": "Refund", + "resolves": false + } + ], + "diagnostics": [], + "verdict": "clean" + }, + { + "path": "Shop.Checkout.Shipping", + "package": "acme::shop", + "qualifiedPath": "acme::shop::Shop.Checkout.Shipping", + "subType": "functional", + "level": 3, + "status": "live", + "digest": "d7d26a3dd5661647327cec62f969b61c74346e1df1e954de4d8392705bdecf31", + "implementedBy": [], + "diagnostics": [ + { + "code": "WARN_REQUIREMENT_NOTHING_IMPLEMENTS", + "severity": "warn", + "source": "gate" + } + ], + "verdict": "warn" + } + ], + "notRepresented": [ + "per-gate drift detail for every gate but schema (which template variable drifted, which generated file differs, which migration failed to replay) — printed as text on stderr; this payload carries each gate's pass/fail verdict, and the schema gate's differences in schemaDrift", + "the loader's own warnings and the agent-context/manifest advisories — printed as text on stderr" + ] +} diff --git a/server/typescript/packages/cli/src/commands/export.ts b/server/typescript/packages/cli/src/commands/export.ts index 7db145be2..4b6aa66d6 100644 --- a/server/typescript/packages/cli/src/commands/export.ts +++ b/server/typescript/packages/cli/src/commands/export.ts @@ -3,7 +3,7 @@ import { writeFile } from "node:fs/promises"; import { parseExportArgs } from "../lib/args.js"; import { log } from "../lib/log.js"; import { FileSource } from "@metaobjectsdev/metadata/core"; -import { composeRegistry, MetaDataLoader, canonicalSerialize } from "@metaobjectsdev/metadata"; +import { composeRegistry, MetaDataLoader, canonicalSerialize, canonicalSerializeEffective } from "@metaobjectsdev/metadata"; import { defaultLoadMemoryProviders, forgeTypesProvider, resolveCollection } from "@metaobjectsdev/sdk"; import { loadProjectProviders } from "../lib/load-metaobjects-config.js"; import { collectionLoadOptions } from "../lib/collection-load-options.js"; @@ -67,7 +67,10 @@ export async function exportCommand(args: string[], cwd: string): Promise w.message), }; diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index 4ed889e74..4eac6c743 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -11,6 +11,7 @@ import { existsSync } from "node:fs"; import { join, relative, resolve as resolvePath } from "node:path"; import { parseVerifyArgs, type MigrateFlags } from "../lib/args.js"; import { log } from "../lib/log.js"; +import { requirementLedgerRows, type RequirementLedgerRow } from "../lib/requirement-ledger.js"; import { emitStructured, type OutputFormat } from "../lib/format.js"; import { antiPatternRows, missingBaseUrlRows, removedPropRows, ranSection, skippedSection, warnCapped, @@ -432,6 +433,10 @@ export async function verifyCommand( // then omits the block rather than reporting zeroes that would read as an empty // ledger instead of no ledger. let requirementCounts: RequirementCounts | undefined; + // One row per requirement: its claim digest, each link as resolved, and the codes + // reported against its path. Undefined exactly when `requirementCounts` is — same + // opt-in-by-declaration rule. + let requirementLedger: RequirementLedgerRow[] | undefined; // Whether the schema gate is selected. `runSchemaVerify` makes this decision // internally (it is selected by --db or --dialect d1, not by a subverb flag); @@ -526,6 +531,7 @@ export async function verifyCommand( exitCode, requirements: requirementSection, requirementCounts, + requirementLedger, antiPatterns: antiPatternSection, overlays: overlaySection, names: nameSection, @@ -866,6 +872,11 @@ export async function verifyCommand( ...diags.map((d) => toDiagnosticRow(d, "gate")), ...lint.map((d) => toDiagnosticRow(d, "lint")), ]); + // The same findings per ENTRY, beside what each entry declares — so a reader learns an + // entry was checked and found clean from a row saying so, not from its path's absence. + if (s !== undefined && structured) { + requirementLedger = requirementLedgerRows(root, scan, diags, lint); + } if (errors.length > 0) { log.error(`meta verify — requirements: ${errors.length} error(s).`); @@ -2081,6 +2092,7 @@ function buildVerifyPayload(input: { exitCode: number; requirements: AdvisorySection; requirementCounts: RequirementCounts | undefined; + requirementLedger?: RequirementLedgerRow[] | undefined; antiPatterns: AdvisorySection; overlays: AdvisorySection; names: AdvisorySection; @@ -2186,6 +2198,7 @@ function buildVerifyPayload(input: { deprecations: input.deprecations, fields: input.fields, ...(input.requirementCounts !== undefined ? { requirementCounts: input.requirementCounts } : {}), + ...(input.requirementLedger !== undefined ? { requirementLedger: input.requirementLedger } : {}), ...(input.schemaDrift !== undefined ? { schemaDrift: input.schemaDrift } : {}), // The honest boundary. Everything named here is REACHABLE — it is printed as // text on stderr — but it is not in this document, and a reader must not have diff --git a/server/typescript/packages/cli/src/index.ts b/server/typescript/packages/cli/src/index.ts index d63cc6858..e501e8bdc 100644 --- a/server/typescript/packages/cli/src/index.ts +++ b/server/typescript/packages/cli/src/index.ts @@ -453,6 +453,9 @@ USAGE: FLAGS: --out Write output to a file (default: stdout) + --effective Write the effective tree: every inherited field, child and + attribute materialized on the node that inherits it (overlays + are merged either way). The input the explorer builder reads. --help, -h Print this help `, fmt: `meta fmt — rewrite metadata files into canonical form diff --git a/server/typescript/packages/cli/src/lib/args.ts b/server/typescript/packages/cli/src/lib/args.ts index 715527104..5e1452f58 100644 --- a/server/typescript/packages/cli/src/lib/args.ts +++ b/server/typescript/packages/cli/src/lib/args.ts @@ -199,11 +199,14 @@ function parseGeneratorNames(raw: string): string[] { export interface ExportFlags { out: string | undefined; + /** Emit the EFFECTIVE tree: inherited members and attributes materialized, overlays merged. */ + effective: boolean; } /** The flag table `parseExportArgs` parses. Exported so the help text can be gated against it. */ export const EXPORT_OPTIONS = { out: { type: "string" }, + effective: { type: "boolean", default: false }, } as const; export function parseExportArgs(argv: string[]): ExportFlags { @@ -215,6 +218,7 @@ export function parseExportArgs(argv: string[]): ExportFlags { }); return { out: values.out, + effective: values.effective, }; } diff --git a/server/typescript/packages/cli/src/lib/requirement-check.ts b/server/typescript/packages/cli/src/lib/requirement-check.ts index 117e379ec..afcff955b 100644 --- a/server/typescript/packages/cli/src/lib/requirement-check.ts +++ b/server/typescript/packages/cli/src/lib/requirement-check.ts @@ -63,6 +63,11 @@ export interface Diagnostic { message: string; } +/** The requirement node a gate or lint diagnostic is about. Two packages may each declare a + * requirement at the same path, so the path alone cannot tell their findings apart. Kept + * beside the diagnostic, not on it, so a diagnostic's shape is only its four fields. */ +export const diagnosticRequirement = new WeakMap(); + export const ERR_REQUIREMENT_LINK_ABOVE_FLOOR = "ERR_REQUIREMENT_LINK_ABOVE_FLOOR"; export const ERR_REQUIREMENT_DANGLING_REF = "ERR_REQUIREMENT_DANGLING_REF"; export const ERR_REQUIREMENT_BAD_LEVEL = "ERR_REQUIREMENT_BAD_LEVEL"; @@ -426,6 +431,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe if (addressed.length === 0) return out; // opt-in by declaration — no requirements, nothing to say for (const { node: req, path: reqPath } of addressed) { + const report = (d: Diagnostic): void => { diagnosticRequirement.set(d, req); out.push(d); }; const architectural = req.subType === REQUIREMENT_SUBTYPE_ARCHITECTURAL; const level = req.level(); const refs = req.implementedBy(); @@ -443,7 +449,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe if (!architectural || levelled) { if (!levelled || !Number.isInteger(level) || level < REQUIREMENT_MIN_LEVEL || level > REQUIREMENT_MAX_LEVEL) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_BAD_LEVEL, path: reqPath, message: `level must be an integer ${REQUIREMENT_MIN_LEVEL}-${REQUIREMENT_MAX_LEVEL} (got ${String(level)}). ` + `L1 solution, L2 segment (app/library), L3 service, L4 object, L5 member.` + @@ -455,7 +461,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe if (parent !== undefined && parent.type === TYPE_REQUIREMENT) { const pl = (parent as MetaRequirement).level(); if (pl !== undefined && level !== undefined && level <= pl) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_LEVEL_NESTING, path: reqPath, message: `nested under "${parent.name}" (level ${pl}) but declares level ${level}. ` + `Nesting is the hierarchy — a child sits strictly below its parent.`, @@ -466,7 +472,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // -- the link boundary ---------------------------------------------------- if (refs.length > 0 && !req.mayReferenceModel()) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_LINK_ABOVE_FLOOR, path: reqPath, message: `'implementedBy' is legal at L${REQUIREMENT_LINK_FLOOR_LEVEL} (object) and ` + `L${REQUIREMENT_MAX_LEVEL} (member) only. L1-L3 are organisational and never reference ` + @@ -495,7 +501,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // existing one — unlike the level checks above, which `@level` already // promised. if (!architectural && level === REQUIREMENT_LINK_FLOOR_LEVEL && !isObjectRef) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_L4_NOT_OBJECT, path: reqPath, message: `L${REQUIREMENT_LINK_FLOOR_LEVEL} references an object; '${ref}' names a member. ` + `Move it to a nested L${REQUIREMENT_LEVEL_MEMBER} child, or reference the object itself.`, @@ -503,7 +509,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe continue; } if (!architectural && level === REQUIREMENT_LEVEL_MEMBER && isObjectRef) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_L5_NOT_MEMBER, path: reqPath, message: `L${REQUIREMENT_LEVEL_MEMBER} references a member (field, view or identity); ` + `'${ref}' names an object. Move it to its L${REQUIREMENT_LINK_FLOOR_LEVEL} parent.`, @@ -521,7 +527,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // resolved and only the member is gone, it listed that same object back and said // to qualify it: name the member instead. const hint = node === undefined ? didYouMeanHint(root, owner) : missingMemberHint(node, path); - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_DANGLING_REF, path: reqPath, message: `'${ref}' does not resolve in the loaded model (status '${String(req.status())}' — ` + `the model moved and the requirement is stale).` + hint, @@ -547,7 +553,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe if (superseded !== undefined) { const referrerPkg = effectiveRequirementPackage(req); if (resolveRequirementRef(addressed, superseded, referrerPkg) === undefined) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_DANGLING_REF, path: reqPath, message: `@supersededBy '${superseded}' does not name a requirement in the loaded ` + `ledger. It must name the requirement that REPLACED this one — if nothing did, ` + @@ -577,7 +583,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // all": true for a flat policy (the original form), false for L1-L3 of a // levelled one, true again at the link floor. if (architectural && live && refs.length === 0 && req.mayReferenceModel()) { - out.push({ + report({ severity: "error", code: ERR_REQUIREMENT_ARCH_NO_IMPLEMENTERS, path: reqPath, message: `architectural requirement is '${String(status)}' but nothing implements it. ` + `Its check is universality — a claim set of zero means the policy is declared and unapplied.`, @@ -587,7 +593,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // -- disposition: the decision, not the state ----------------------------- const disposition = req.disposition(); if (disposition !== undefined && !req.hasOutstandingWork()) { - out.push({ + report({ severity: "warn", code: WARN_REQUIREMENT_DISPOSITION_NOT_APPLICABLE, path: reqPath, message: `carries @disposition '${disposition}' but its status is '${String(status)}', which has no ` + `outstanding work to decide about. A disposition is meaningful on 'planned' and 'partial' only — ` + @@ -608,7 +614,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // keeps its `WARN_` name on purpose: it identifies the finding, and one finding // under two codes would split every suppression and corpus case that keys on it. if (!architectural && live && !subtreeClaimsAnything(req)) { - out.push({ + report({ severity: scan.requireImplementers ? "error" : "warn", code: WARN_REQUIREMENT_NOTHING_IMPLEMENTS, path: reqPath, message: `is '${String(status)}' but neither it nor anything nested under it names an ` + @@ -618,7 +624,7 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe } if (disposition === REQUIREMENT_DISPOSITION_DEFERRED && req.trackedBy().length === 0) { - out.push({ + report({ severity: "warn", code: WARN_REQUIREMENT_DEFERRED_UNTRACKED, path: reqPath, message: `is deferred but names no @trackedBy issue. Deferring without a ticket is how a known gap ` + `becomes an unknown one — nothing will raise it again.`, diff --git a/server/typescript/packages/cli/src/lib/requirement-ledger.ts b/server/typescript/packages/cli/src/lib/requirement-ledger.ts new file mode 100644 index 000000000..9a1896f9f --- /dev/null +++ b/server/typescript/packages/cli/src/lib/requirement-ledger.ts @@ -0,0 +1,152 @@ +// `meta verify --format json` — the per-requirement ledger rows. +// +// The requirement gate and its counts say how the ledger fares as a WHOLE; a reader +// (the explorer builder, an agent, a dashboard) also needs each entry's own verdict, and +// before this it had to infer "checked and clean" from a path's absence in the +// diagnostics. One row per `requirement.*` node states it outright: what the entry +// declares, the digest of its claim, whether each `@implementedBy` link resolves and to +// WHAT, and the codes the gate and the authoring lint reported against it. +// +// Every value here comes from the SAME scan, resolver and diagnostics the gate used on +// this run. Nothing is re-derived: a row that disagreed with the gate beside it would be +// a second answer to one question. + +import { + effectiveRequirementPackage, + resolveClaimTarget, + resolveMember, + type MetaRequirement, + type MetaData, +} from "@metaobjectsdev/metadata"; +import { requirementDigest } from "@metaobjectsdev/codegen-ts"; +import { + diagnosticRequirement, + resolveRequirementRef, + splitMemberRef, + type Diagnostic, + type RequirementScan, +} from "./requirement-check.js"; + +/** One `@implementedBy` (or `@supersededBy`) link, as the gate resolved it. */ +export interface RequirementLinkRow { + /** The link exactly as authored. */ + ref: string; + resolves: boolean; + /** The node it binds to — `` or `.` + * for `@implementedBy`, `::` for `@supersededBy`. Absent when it does not + * resolve. */ + target?: string; +} + +export interface RequirementDiagnosticRef { + code: string; + severity: string; + /** "gate" can fail the build; "lint" never can. */ + source: "gate" | "lint"; +} + +/** One `requirement.*` node, as this run of `meta verify` judged it. */ +export interface RequirementLedgerRow { + /** The dotted child-name address every diagnostic prints. */ + path: string; + /** The requirement's effective package, "" when it has none. */ + package: string; + /** `::`, or `` with no package — the prefix of every generated + * requirement test's id (` []`). */ + qualifiedPath: string; + subType: string; + level?: number; + status?: string; + disposition?: string; + /** SHA-256 over the claim (subtype, level, status, statement, counterexample, + * `@implementedBy`) — the digest every port's requirement tests carry. */ + digest: string; + implementedBy: RequirementLinkRow[]; + supersededBy?: RequirementLinkRow; + /** Every finding this run reported against this requirement, gate and lint, in report order. */ + diagnostics: RequirementDiagnosticRef[]; + /** From the GATE findings only: "error" when one can fail the build, "warn" when the + * worst is a warning, "clean" when the gate reported nothing against this entry. */ + verdict: "clean" | "warn" | "error"; +} + +function implementedByRows(root: MetaData, req: MetaRequirement): RequirementLinkRow[] { + // referrerPkg is the requirement's own effective package — the binding the gate uses + // (ADR-0042), so a row resolves exactly the links the gate resolved. + const referrerPkg = effectiveRequirementPackage(req); + return req.implementedBy().map((ref) => { + const { owner, path } = splitMemberRef(ref); + const node = resolveClaimTarget(root, owner, referrerPkg); + if (node === undefined) return { ref, resolves: false }; + if (path.length > 0 && resolveMember(node, path) === undefined) return { ref, resolves: false }; + const key = node.resolutionKey(); + return { ref, resolves: true, target: path.length === 0 ? key : `${key}.${path.join(".")}` }; + }); +} + +/** + * One row per requirement in the scan, in the scan's (document) order. + * + * `diags` are this run's gate diagnostics and `lint` its authoring findings. Each is matched + * to a row by the requirement node it names, and by path only when it names none. + */ +export function requirementLedgerRows( + root: MetaData, + scan: RequirementScan, + diags: readonly Diagnostic[], + lint: readonly Diagnostic[], +): RequirementLedgerRow[] { + const byNode = new Map(); + const byPath = new Map(); + const note = (d: Diagnostic, source: "gate" | "lint"): void => { + const ref = { code: d.code, severity: d.severity, source }; + const requirement = diagnosticRequirement.get(d); + if (requirement !== undefined) { + byNode.set(requirement, [...(byNode.get(requirement) ?? []), ref]); + } else if (d.path !== undefined) { + byPath.set(d.path, [...(byPath.get(d.path) ?? []), ref]); + } + }; + for (const d of diags) note(d, "gate"); + for (const d of lint) note(d, "lint"); + + return scan.addressed.map(({ node: req, path }) => { + const pkg = effectiveRequirementPackage(req); + const diagnostics = [...(byNode.get(req) ?? []), ...(byPath.get(path) ?? [])]; + const gate = diagnostics.filter((d) => d.source === "gate"); + const verdict = gate.some((d) => d.severity === "error") + ? "error" + : gate.length > 0 ? "warn" : "clean"; + const level = req.level(); + const status = req.status(); + const disposition = req.disposition(); + const superseded = req.supersededBy(); + let supersededBy: RequirementLinkRow | undefined; + if (superseded !== undefined) { + const target = resolveRequirementRef(scan.addressed, superseded, pkg); + const hit = target === undefined ? undefined : scan.addressed.find((r) => r.node === target); + supersededBy = hit === undefined + ? { ref: superseded, resolves: false } + : { ref: superseded, resolves: true, target: qualify(effectiveRequirementPackage(hit.node), hit.path) }; + } + return { + path, + package: pkg, + qualifiedPath: qualify(pkg, path), + subType: req.subType, + // `exactOptionalPropertyTypes` — an omitted key, never an explicit `undefined`. + ...(level !== undefined ? { level } : {}), + ...(status !== undefined ? { status } : {}), + ...(disposition !== undefined ? { disposition } : {}), + digest: requirementDigest(req), + implementedBy: implementedByRows(root, req), + ...(supersededBy !== undefined ? { supersededBy } : {}), + diagnostics, + verdict, + }; + }); +} + +function qualify(pkg: string, path: string): string { + return pkg === "" ? path : `${pkg}::${path}`; +} diff --git a/server/typescript/packages/cli/src/lib/requirement-lint.ts b/server/typescript/packages/cli/src/lib/requirement-lint.ts index cc1e324ad..7f1820e42 100644 --- a/server/typescript/packages/cli/src/lib/requirement-lint.ts +++ b/server/typescript/packages/cli/src/lib/requirement-lint.ts @@ -56,7 +56,7 @@ import { // compares this path set against `walkRequirements()`, the generator's own walk, // rather than leaving the two to agree by inspection.) import { - collectAddressedRequirements, type AddressedRequirement, type Diagnostic, + collectAddressedRequirements, diagnosticRequirement, type AddressedRequirement, type Diagnostic, } from "./requirement-check.js"; export const WARN_REQUIREMENT_NAME_NOT_ADDRESSABLE = "WARN_REQUIREMENT_NAME_NOT_ADDRESSABLE"; @@ -236,8 +236,10 @@ function excerpt(value: string): string { return JSON.stringify(flat.length > LIMIT ? `${flat.slice(0, LIMIT - 1)}…` : flat); } -function warn(path: string, code: string, message: string): Diagnostic { - return { severity: "warn", code, path, message }; +function warn(at: AddressedRequirement, code: string, message: string): Diagnostic { + const d: Diagnostic = { severity: "warn", code, path: at.path, message }; + diagnosticRequirement.set(d, at.node); + return d; } /** @@ -263,7 +265,8 @@ export function lintRequirements( ): Diagnostic[] { const out: Diagnostic[] = []; - for (const { node, path } of addressed) { + for (const at of addressed) { + const { node, path } = at; const name = node.name; // -- the name is an address, not prose ------------------------------------ @@ -279,7 +282,7 @@ export function lintRequirements( if (name.trim() === "") problems.push("is blank"); else if (name.trim() !== name) problems.push("has leading or trailing whitespace"); if (problems.length > 0) { - out.push(warn(path, WARN_REQUIREMENT_NAME_NOT_ADDRESSABLE, + out.push(warn(at, WARN_REQUIREMENT_NAME_NOT_ADDRESSABLE, `name ${JSON.stringify(name)} ${problems.join(", and ")}. A requirement's name is its ` + `address — it is the segment of the dotted path '${path}' and the filename of its generated ` + `test stub — so this either collides with nesting or produces a path the stub cannot be ` + @@ -293,11 +296,11 @@ export function lintRequirements( // the statement, "rename it" is the wrong instruction. The instruction is "you // have written the claim twice; @statement is the one that is read". if (statementKey !== "" && normalise(toSnakeCase(name)) === statementKey) { - out.push(warn(path, WARN_REQUIREMENT_NAME_RESTATES_STATEMENT, + out.push(warn(at, WARN_REQUIREMENT_NAME_RESTATES_STATEMENT, `name '${name}' says the same thing as @statement. The claim belongs in @statement, which every ` + `surface reads; the name is an address and is better as a short identifier.`)); } else if (name.trim().split(/\s+/).filter((w) => w !== "").length >= PROSE_WORD_COUNT) { - out.push(warn(path, WARN_REQUIREMENT_NAME_READS_AS_PROSE, + out.push(warn(at, WARN_REQUIREMENT_NAME_READS_AS_PROSE, `name '${name}' reads as a sentence. The name is an address — the dotted path other entries and ` + `the generated stub filename are built from — so a short identifier keeps both legible. Put the ` + `prose in @statement.`)); @@ -310,7 +313,7 @@ export function lintRequirements( // OWN-ONLY: the fix is deleting or filling one declaration. See readOwnSlot. const value = readOwnSlot(node, slot); if (value !== undefined && value.trim() === "") { - out.push(warn(path, WARN_REQUIREMENT_PROSE_EMPTY, + out.push(warn(at, WARN_REQUIREMENT_PROSE_EMPTY, `@${slot} is declared but empty. The loader requires the attribute to be present, not to say ` + `anything — an empty one passes every check while recording nothing.`)); } @@ -329,12 +332,12 @@ export function lintRequirements( const description = readSlot(node, DOC_ATTR_DESCRIPTION); if (description !== undefined) { if (normalise(description) === statementKey) { - out.push(warn(path, WARN_REQUIREMENT_PROSE_DUPLICATED, + out.push(warn(at, WARN_REQUIREMENT_PROSE_DUPLICATED, `description repeats @statement verbatim. @statement already IS the description of what the ` + `requirement is; description holds the SCOPE — what the claim covers, what it deliberately ` + `does not, which sibling entry owns the rest. If the scope is obvious, leave it off.`)); } else if (leadingSentenceMatches(description, statementKey)) { - out.push(warn(path, WARN_REQUIREMENT_PROSE_DUPLICATED, + out.push(warn(at, WARN_REQUIREMENT_PROSE_DUPLICATED, `description opens by repeating @statement verbatim, then continues. Drop the first sentence — ` + `it is already read from @statement, and description is only the scope that follows it.`)); } @@ -342,7 +345,7 @@ export function lintRequirements( const counterexample = readSlot(node, REQUIREMENT_ATTR_COUNTEREXAMPLE); if (counterexample !== undefined && normalise(counterexample) === statementKey) { - out.push(warn(path, WARN_REQUIREMENT_PROSE_DUPLICATED, + out.push(warn(at, WARN_REQUIREMENT_PROSE_DUPLICATED, `@counterexample repeats @statement verbatim. It must describe what BREAKING the claim looks ` + `like — the thing you could point at to falsify it — which is what makes the claim checkable.`)); } @@ -353,7 +356,7 @@ export function lintRequirements( // OWN-ONLY: the fix is deleting one attribute. See readOwnSlot. const value = readOwnSlot(node, slot); if (value === undefined || value.trim() === "") continue; - out.push(warn(path, WARN_REQUIREMENT_INERT_DOC_SLOT, + out.push(warn(at, WARN_REQUIREMENT_INERT_DOC_SLOT, `@${slot} says nothing here that @${REQUIREMENT_ATTR_STATEMENT} does not. A requirement's ` + `statement is REQUIRED and is already the one-line sentence, so a summary beside it can only ` + `repeat it — and no requirement surface reads it. Its content is invisible: ${excerpt(value)}. ` + @@ -364,7 +367,7 @@ export function lintRequirements( // -- an id is not a label ------------------------------------------------- const title = readOwnSlot(node, DOC_ATTR_TITLE); if (title !== undefined && TITLE_ID_ONLY.test(title.trim())) { - out.push(warn(path, WARN_REQUIREMENT_TITLE_IS_AN_ID, + out.push(warn(at, WARN_REQUIREMENT_TITLE_IS_AN_ID, `@title holds only a catalogue or ticket id: ${excerpt(title)}. A title is the entry's ` + `LABEL and an id is not a name, so this slot says nothing a reader can use. Put the id in ` + `@${REQUIREMENT_ATTR_TRACKED_BY}, which is the free-form reference slot and IS read, and ` + diff --git a/server/typescript/packages/cli/test/explorer-fixture.test.ts b/server/typescript/packages/cli/test/explorer-fixture.test.ts new file mode 100644 index 000000000..d01388949 --- /dev/null +++ b/server/typescript/packages/cli/test/explorer-fixture.test.ts @@ -0,0 +1,149 @@ +// The explorer data spine, end to end, over `fixtures/explorer-conformance/shop/`: +// +// `meta export --effective` → effective.json (committed) +// `meta verify --format json` → verify.json (committed; carries `requirementLedger`) +// buildExplorer(both + the manifest) → expected.json (committed, byte-checked in docs-site) +// +// This file keeps the committed INPUTS honest: regenerating them from `project/` must give +// the same bytes, so a change to either command's output shows up here as a fixture diff +// rather than as an explorer that quietly reads stale data. +import { describe, test, expect, beforeEach, afterEach, afterAll } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { buildExplorer, serializeExplorer } from "@metaobjectsdev/docs-site"; +import { run } from "../src/index.js"; + +const FIXTURE = resolve(import.meta.dir, "../../../../../fixtures/explorer-conformance/shop"); +const PROJECT = join(FIXTURE, "project"); +const committed = (name: string): string => readFileSync(join(FIXTURE, name), "utf8"); + +const dirs: string[] = []; +afterAll(() => { + for (const d of dirs) rmSync(d, { recursive: true, force: true }); +}); + +let out: string[]; +let origLog: typeof console.log; +let origErr: typeof console.error; +beforeEach(() => { + out = []; + origLog = console.log; + origErr = console.error; + console.log = (...a: unknown[]) => { out.push(a.map(String).join(" ")); }; + console.error = () => {}; +}); +afterEach(() => { + console.log = origLog; + console.error = origErr; +}); + +async function exportJson(effective: boolean): Promise { + const dir = mkdtempSync(join(tmpdir(), "explorer-export-")); + dirs.push(dir); + const file = join(dir, "model.json"); + const exit = await run(["export", ...(effective ? ["--effective"] : []), "--out", file, "--cwd", PROJECT]); + expect(exit).toBe(0); + return readFileSync(file, "utf8"); +} + +async function verifyJson(): Promise { + const exit = await run(["verify", "--format", "json", "--cwd", PROJECT]); + expect(exit).toBe(0); + return out.join("\n") + "\n"; +} + +interface LedgerRow { + path: string; + qualifiedPath: string; + verdict: string; + digest: string; + implementedBy: { ref: string; resolves: boolean; target?: string }[]; + diagnostics: { code: string; severity: string; source: string }[]; +} + +describe("explorer fixture — the committed inputs are what the CLI writes today", () => { + test("`meta export --effective` reproduces effective.json", async () => { + expect(await exportJson(true)).toBe(committed("effective.json")); + }); + + test("`meta verify --format json` reproduces verify.json", async () => { + expect(await verifyJson()).toBe(committed("verify.json")); + }); + + test("the fresh inputs build the committed expected.json", async () => { + const model = JSON.parse(await exportJson(true)); + out = []; + const verify = JSON.parse(await verifyJson()); + const manifest = JSON.parse(committed("hashes.json")); + expect(serializeExplorer(buildExplorer({ model, verify, manifest }))).toBe(committed("expected.json")); + }); +}); + +describe("meta export --effective", () => { + test("materializes inherited members; the plain export does not", async () => { + const own = JSON.parse(await exportJson(false)); + const eff = JSON.parse(await exportJson(true)); + const fieldNames = (doc: unknown, object: string): string[] => { + const root = (doc as { "metadata.root": { children: Record[] }>[] } })["metadata.root"]; + const node = root.children.map((c) => Object.values(c)[0]).find((b) => b?.name === object); + return (node?.children ?? []).map((c) => Object.values(c)[0]?.name ?? ""); + }; + expect(fieldNames(own, "Customer")).not.toContain("id"); + expect(fieldNames(eff, "Customer")).toContain("id"); + expect(fieldNames(eff, "Customer")).toContain("createdAt"); + }); +}); + +describe("meta verify --format json — requirementLedger rows", () => { + async function rows(): Promise> { + const payload = JSON.parse(await verifyJson()) as { requirementLedger: LedgerRow[] }; + return new Map(payload.requirementLedger.map((r) => [r.path, r])); + } + + test("one row per requirement, in document order, each with a sha-256 digest", async () => { + const r = await rows(); + expect([...r.keys()]).toEqual([ + "Addressable", + "Shop", + "Shop.Checkout", + "Shop.Checkout.Orders", + "Shop.Checkout.Orders.Recorded", + "Shop.Checkout.Orders.Recorded.Quotable", + "Shop.Checkout.Orders.Noted", + "Shop.Checkout.Orders.Refunded", + "Shop.Checkout.Shipping", + ]); + for (const row of r.values()) expect(row.digest).toMatch(/^[0-9a-f]{64}$/); + expect(r.get("Shop.Checkout")?.qualifiedPath).toBe("acme::shop::Shop.Checkout"); + }); + + test("each @implementedBy link carries the node the gate resolved it to", async () => { + const r = await rows(); + expect(r.get("Shop.Checkout.Orders.Recorded")?.implementedBy).toEqual([ + { ref: "Order", resolves: true, target: "acme::shop::Order" }, + { ref: "Customer", resolves: true, target: "acme::shop::Customer" }, + ]); + expect(r.get("Shop.Checkout.Orders.Recorded.Quotable")?.implementedBy).toEqual([ + { ref: "Order.reference", resolves: true, target: "acme::shop::Order.reference" }, + ]); + // A planned requirement may name a node that does not exist yet: the row says so. + expect(r.get("Shop.Checkout.Orders.Refunded")?.implementedBy).toEqual([{ ref: "Refund", resolves: false }]); + }); + + test("the verdict comes from the gate's findings against that path", async () => { + const r = await rows(); + const shipping = r.get("Shop.Checkout.Shipping"); + expect(shipping?.verdict).toBe("warn"); + expect(shipping?.diagnostics).toEqual([ + { code: "WARN_REQUIREMENT_NOTHING_IMPLEMENTS", severity: "warn", source: "gate" }, + ]); + expect(r.get("Shop.Checkout.Orders.Recorded")?.verdict).toBe("clean"); + }); + + test("text output carries no ledger rows", async () => { + const exit = await run(["verify", "--format", "text", "--cwd", PROJECT]); + expect(exit).toBe(0); + expect(out.join("\n")).not.toContain("requirementLedger"); + }); +}); diff --git a/server/typescript/packages/cli/test/unit/args-export.test.ts b/server/typescript/packages/cli/test/unit/args-export.test.ts index d5b889464..c3ad749b2 100644 --- a/server/typescript/packages/cli/test/unit/args-export.test.ts +++ b/server/typescript/packages/cli/test/unit/args-export.test.ts @@ -2,19 +2,25 @@ import { describe, test, expect } from "bun:test"; import { parseExportArgs } from "../../src/lib/args.js"; describe("parseExportArgs", () => { - test("default flags — out is undefined", () => { - expect(parseExportArgs([])).toEqual({ out: undefined }); + test("default flags — out is undefined, effective is off", () => { + expect(parseExportArgs([])).toEqual({ out: undefined, effective: false }); + }); + + test("--effective selects the effective tree", () => { + expect(parseExportArgs(["--effective"])).toEqual({ out: undefined, effective: true }); }); test("--out sets the output path", () => { expect(parseExportArgs(["--out", "dist/metadata.json"])).toEqual({ out: "dist/metadata.json", + effective: false, }); }); test("--out with absolute path", () => { expect(parseExportArgs(["--out", "/tmp/metadata.json"])).toEqual({ out: "/tmp/metadata.json", + effective: false, }); }); diff --git a/server/typescript/packages/cli/test/unit/requirement-ledger.test.ts b/server/typescript/packages/cli/test/unit/requirement-ledger.test.ts new file mode 100644 index 000000000..a135dce8f --- /dev/null +++ b/server/typescript/packages/cli/test/unit/requirement-ledger.test.ts @@ -0,0 +1,67 @@ +// `requirementLedgerRows` — the per-requirement rows `meta verify --format json` carries. +// +// A requirement's dotted path carries no package, so two packages may each declare one at +// the same path. Each row must hold only the findings against its own node: a finding +// against one package's `Billing` is not a finding against the other's. +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadMemory } from "@metaobjectsdev/sdk"; +import { checkRequirements, scanRequirements } from "../../src/lib/requirement-check.js"; +import { lintRequirements } from "../../src/lib/requirement-lint.js"; +import { requirementLedgerRows } from "../../src/lib/requirement-ledger.js"; + +const pkg = (name: string, claim: string): string => ` +metadata: + package: acme::${name} + children: + - object.entity: + name: Invoice + children: + - source.rdb: { table: ${name}_invoices } + - field.uuid: { name: id } + - identity.primary: { name: pk, fields: [id] } + - requirement.functional: + name: Billing + level: 1 + status: live + statement: "Invoices are issued" + counterexample: "A sale with no invoice" + children: + - requirement.functional: + name: Invoicing + level: 4 + status: live + statement: "Each invoice is stored" + counterexample: "An invoice that cannot be found again" + implementedBy: ["${claim}"] +`; + +describe("requirementLedgerRows", () => { + test("two packages' requirements at one path keep their own findings", async () => { + const dir = mkdtempSync(join(tmpdir(), "req-ledger-")); + try { + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, "metaobjects/meta.a.yaml"), pkg("a", "acme::a::Invoice")); + writeFileSync(join(dir, "metaobjects/meta.b.yaml"), pkg("b", "acme::b::Receipt")); + const root = await loadMemory(dir, { strict: true }); + const scan = scanRequirements(root); + const gate = checkRequirements(root, scan); + expect(gate.length).toBeGreaterThan(0); + for (const d of [...gate, ...lintRequirements(root, scan.addressed)]) { + expect(Object.keys(d).filter((k) => !["code", "message", "path", "severity"].includes(k))).toEqual([]); + } + const rows = requirementLedgerRows(root, scan, gate, lintRequirements(root, scan.addressed)); + const verdicts = rows.map((r) => [r.qualifiedPath, r.verdict, r.diagnostics.map((d) => d.code)]); + expect(verdicts).toEqual([ + ["acme::a::Billing", "clean", []], + ["acme::a::Billing.Invoicing", "clean", []], + ["acme::b::Billing", "clean", []], + ["acme::b::Billing.Invoicing", "error", ["ERR_REQUIREMENT_DANGLING_REF"]], + ]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index 93c86ad4e..d6caa1339 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -17,6 +17,12 @@ export interface EmittedFile { content: string; /** Set by the runner from generator.name — generators should not set this. */ generatedBy?: string; + /** The resolution key of the metadata node this file was rendered from + * (`acme::shop::Order`), recorded beside the file's hash in `.gen-state/.hashes.json`. + * `perEntity` sets it for you; a generator rendering one file from one node by other + * means sets it itself. Leave it unset for a file rendered from the whole model or a + * whole package (a barrel, a docs index). */ + sourceNode?: string; } export interface GenContext { @@ -229,8 +235,15 @@ export function perEntity( ): (ctx: GenContext) => Promise { return async (ctx) => { const matched = ctx.entities.filter(ctx.matches); - const results = await Promise.all(matched.map((e) => fn(e, ctx))); - return results.flatMap((r) => (Array.isArray(r) ? r : [r])); + const results = await Promise.all(matched.map(async (e) => { + const r = await fn(e, ctx); + // Each file names the object it was rendered from, unless the generator said + // otherwise — so every per-entity generator, owned copies included, records its + // provenance without a line of its own. + const key = e.resolutionKey(); + return (Array.isArray(r) ? r : [r]).map((f) => (f.sourceNode !== undefined ? f : { ...f, sourceNode: key })); + })); + return results.flat(); }; } diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index b696f9f0e..57464372d 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -95,7 +95,8 @@ export { decideAndWrite, GitMissingError, WRITE_STATUSES } from "./overwrite-pol // `listGeneratedPaths` answers the sibling question — WHICH paths we have ever // written — which is how the gate scopes itself to its own output instead of // convicting every stranger's file that happens to sit in the same directory. -export { contentHash, readGeneratedHash, listGeneratedPaths } from "./overwrite-policy.js"; +export { contentHash, readGeneratedHash, listGeneratedPaths, recordGeneratedProvenance } from "./overwrite-policy.js"; +export type { GeneratedProvenance } from "./overwrite-policy.js"; export { CodegenError } from "./errors.js"; export { GENERATED_HEADER, GENERATED_EDIT_NOTE, NAMES_FILE_SUFFIX, DEFAULT_OUT_DIR, RETIRED_CODEGEN_ATTRS, sidecarLine, type RetiredCodegenAttr } from "./constants.js"; diff --git a/server/typescript/packages/codegen-ts/src/overwrite-policy.ts b/server/typescript/packages/codegen-ts/src/overwrite-policy.ts index a1b0d45b5..70311f155 100644 --- a/server/typescript/packages/codegen-ts/src/overwrite-policy.ts +++ b/server/typescript/packages/codegen-ts/src/overwrite-policy.ts @@ -184,18 +184,80 @@ function loadHashes(genStateDir: string): HashesFile { } function saveHashes(genStateDir: string, hashes: HashesFile): void { - mkdirSync(genStateDir, { recursive: true }); const raw = readRawManifest(genStateDir); + const next: Record = {}; + for (const [key, hash] of Object.entries(hashes)) { + const existing = raw[key]; + next[key] = isRecord(existing) ? { ...existing, hash } : hash; + } + writeRawManifest(genStateDir, next); +} + +function writeRawManifest(genStateDir: string, entries: Record): void { + mkdirSync(genStateDir, { recursive: true }); // Keys SORTED, because this file is committed. Insertion order would make the // diff — and any merge conflict between two people who both regenerated — // depend on which entity happened to generate first, which is noise nobody can // review. const sorted: Record = {}; - for (const key of Object.keys(hashes).sort()) { + for (const key of Object.keys(entries).sort()) sorted[key] = entries[key]; + writeFileSync(join(genStateDir, HASHES_FILE), JSON.stringify(sorted, null, 2) + "\n"); +} + +/** What produced one generated file: the generator, and the metadata node it was rendered + * from when it was rendered from one. */ +export interface GeneratedProvenance { + generatedBy: string; + /** The resolution key of the node (`acme::shop::Order`). Absent for a file rendered from + * the whole model or a whole package — a barrel, a docs index. */ + sourceNode?: string; +} + +/** The manifest fields provenance owns. Every other field on an entry — `hash`, and the + * JVM's `generator` / `outputDir` owner (ADR-0060) — is kept as it is. */ +const PROVENANCE_FIELDS = ["generatedBy", "sourceNode"] as const; + +/** + * Record, on each recorded path's manifest entry, the node it was generated from. + * + * A path with a source node becomes `{ "hash", "generatedBy", "sourceNode" }`; one without + * keeps (or returns to) the plain `""` form, so a file rendered from the whole model + * adds nothing to the committed manifest. A path with no recorded hash is left alone — this + * describes what a write recorded, it never records a write. + * + * `generatedBy` is deliberately not the JVM's `generator` field: there `generator` is an + * OWNERSHIP claim the orphan sweep acts on, and a TypeScript name in it would read as a + * generator no Maven execution configures. + * + * Batched: one read and at most one write for the whole run. Nothing is written when no + * entry changes, so a regeneration that changes nothing leaves the file's bytes alone. + */ +export function recordGeneratedProvenance( + genStateDir: string, + provenance: ReadonlyMap, +): void { + if (!existsSync(join(genStateDir, HASHES_FILE))) return; + const raw = readRawManifest(genStateDir); + let changed = false; + for (const [key, prov] of provenance) { const existing = raw[key]; - sorted[key] = isRecord(existing) ? { ...existing, hash: hashes[key] } : hashes[key]; + const hash = entryHash(existing); + if (hash === undefined) continue; + const rest: Record = {}; + if (isRecord(existing)) { + for (const [k, v] of Object.entries(existing)) { + if (k !== "hash" && !(PROVENANCE_FIELDS as readonly string[]).includes(k)) rest[k] = v; + } + } + const next: unknown = prov.sourceNode !== undefined + ? { hash, ...rest, generatedBy: prov.generatedBy, sourceNode: prov.sourceNode } + : Object.keys(rest).length > 0 ? { hash, ...rest } : hash; + if (JSON.stringify(next) !== JSON.stringify(existing)) { + raw[key] = next; + changed = true; + } } - writeFileSync(join(genStateDir, HASHES_FILE), JSON.stringify(sorted, null, 2) + "\n"); + if (changed) writeRawManifest(genStateDir, raw); } // --------------------------------------------------------------------------- diff --git a/server/typescript/packages/codegen-ts/src/runner.ts b/server/typescript/packages/codegen-ts/src/runner.ts index 78e84b97e..f8bc70baa 100644 --- a/server/typescript/packages/codegen-ts/src/runner.ts +++ b/server/typescript/packages/codegen-ts/src/runner.ts @@ -33,6 +33,8 @@ import { listGeneratedPaths, loadEngineVersion, saveEngineVersion, + recordGeneratedProvenance, + type GeneratedProvenance, type WriteResult, type MergeStrategy, type BaselineMode, @@ -791,7 +793,13 @@ export async function runGen(opts: RunGenOpts): Promise { } // 4. Run each generator with a per-target render context; collect with full path. - const emitted: { fullPath: string; content: string; generatedBy: string; handEdits?: HandEdits }[] = []; + const emitted: { + fullPath: string; + content: string; + generatedBy: string; + handEdits?: HandEdits; + sourceNode?: string; + }[] = []; // FR-038 §8 — generators that opted into orphan reconciliation, paired with the // directory their policy's relative paths are measured from. Collected here // because `writeOutDir` is resolved per generator inside this loop. @@ -972,6 +980,7 @@ export async function runGen(opts: RunGenOpts): Promise { content: file.content, generatedBy: generator.name, ...(generator.handEdits !== undefined && { handEdits: generator.handEdits }), + ...(file.sourceNode !== undefined && { sourceNode: file.sourceNode }), }); } } @@ -1277,6 +1286,21 @@ export async function runGen(opts: RunGenOpts): Promise { noteUntracked(); reportAdoptions(); + // G3 — which node each recorded file came from, beside its hash. Only with a real + // projectRoot: without one the manifest is keyed by path hashes, not paths. + if (projectRoot !== undefined) { + const provenance = new Map(); + for (const file of emitted) { + if (outsideProject(file.fullPath)) continue; + // The same key policyFor gives the write, so the two always name one entry. + provenance.set(relative(projectRoot, file.fullPath), { + generatedBy: file.generatedBy, + ...(file.sourceNode !== undefined && { sourceNode: file.sourceNode }), + }); + } + recordGeneratedProvenance(genStateDir, provenance); + } + // Sweep AFTER the writes: writing is the primary job, and a deletion that runs // first would be unrecoverable if a later write threw. Ordering cannot change // the decision — a path this run emitted is never an orphan either way. diff --git a/server/typescript/packages/codegen-ts/test/generated-provenance.test.ts b/server/typescript/packages/codegen-ts/test/generated-provenance.test.ts new file mode 100644 index 000000000..e1c027716 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/generated-provenance.test.ts @@ -0,0 +1,88 @@ +// Node provenance in the generated-file manifest: `perEntity` stamps each file with the node +// it rendered, and the runner records that on the file's `.hashes.json` entry, so a reader +// (the explorer builder) can say which files a node produced without matching names. +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { MetaDataLoader } from "@metaobjectsdev/metadata"; +import { FileSource } from "@metaobjectsdev/metadata/core"; +import { runGen } from "../src/runner.js"; +import { defineConfig } from "../src/metaobjects-config.js"; +import { perEntity, perModel, type Generator } from "../src/generator.js"; +import { recordGeneratedProvenance } from "../src/overwrite-policy.js"; + +const FIXTURE = resolve(import.meta.dir, "fixtures", "single-entity.json"); + +let tmp: string; +beforeEach(() => { tmp = mkdtempSync(join(tmpdir(), "codegen-provenance-")); }); +afterEach(() => { rmSync(tmp, { recursive: true, force: true }); }); + +const manifest = (dir: string): Record => + JSON.parse(readFileSync(join(dir, ".metaobjects", ".gen-state", ".hashes.json"), "utf8")); + +describe("perEntity stamps the source node", () => { + test("each file carries its entity's resolution key unless the generator set one", async () => { + const { root } = await new MetaDataLoader().load([new FileSource(FIXTURE)]); + const plain = perEntity((e) => ({ path: `${e.name}.ts`, content: "" })); + const preset = perEntity((e) => ({ path: `${e.name}.ts`, content: "", sourceNode: "kept" })); + const ctx = { entities: root.objects(), root, matches: () => true } as unknown as Parameters[0]; + expect((await plain(ctx)).map((f) => f.sourceNode)).toEqual(["demo::Post"]); + expect((await preset(ctx)).map((f) => f.sourceNode)).toEqual(["kept"]); + }); +}); + +describe("runGen records provenance in .hashes.json", () => { + test("a per-entity file gets { hash, generatedBy, sourceNode }; a whole-model file stays plain", async () => { + const { root } = await new MetaDataLoader().load([new FileSource(FIXTURE)]); + const entity: Generator = { name: "alpha", generate: perEntity((e) => ({ path: `${e.name}.alpha.ts`, content: "// a\n" })) }; + const barrel: Generator = { name: "barrel", generate: perModel(() => ({ path: "index.ts", content: "// i\n" })) }; + await runGen({ + config: defineConfig({ outDir: join(tmp, "gen"), extStyle: "none", dbImport: "../db", dialect: "sqlite", generators: [entity, barrel] }), + metadata: root, + projectRoot: tmp, + }); + const m = manifest(tmp); + expect(m["gen/Post.alpha.ts"]).toEqual({ + hash: expect.any(String), + generatedBy: "alpha", + sourceNode: "demo::Post", + }); + expect(typeof m["gen/index.ts"]).toBe("string"); + }); +}); + +describe("recordGeneratedProvenance", () => { + function seed(entries: Record): string { + const dir = join(tmp, ".gen-state"); + mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, ".hashes.json"), JSON.stringify(entries)); + return dir; + } + const read = (dir: string): Record => JSON.parse(readFileSync(join(dir, ".hashes.json"), "utf8")); + + test("keeps the JVM owner fields beside the provenance", () => { + const dir = seed({ "a.kt": { hash: "h1", generator: "kotlin-entity", outputDir: "src" } }); + recordGeneratedProvenance(dir, new Map([["a.kt", { generatedBy: "entity", sourceNode: "x::A" }]])); + expect(read(dir)["a.kt"]).toEqual({ hash: "h1", generator: "kotlin-entity", outputDir: "src", generatedBy: "entity", sourceNode: "x::A" }); + }); + + test("a file with no source node returns to the plain form", () => { + const dir = seed({ "i.ts": { hash: "h2", generatedBy: "barrel", sourceNode: "gone::X" } }); + recordGeneratedProvenance(dir, new Map([["i.ts", { generatedBy: "barrel" }]])); + expect(read(dir)["i.ts"]).toBe("h2"); + }); + + test("never records a path the manifest has no hash for, and writes nothing when nothing changes", () => { + const dir = seed({ "a.ts": "h3" }); + const before = readFileSync(join(dir, ".hashes.json"), "utf8"); + recordGeneratedProvenance(dir, new Map([["b.ts", { generatedBy: "g", sourceNode: "x::B" }], ["a.ts", { generatedBy: "g" }]])); + expect(readFileSync(join(dir, ".hashes.json"), "utf8")).toBe(before); + }); + + test("no manifest, no file", () => { + const dir = join(tmp, "none"); + recordGeneratedProvenance(dir, new Map([["a.ts", { generatedBy: "g", sourceNode: "x::A" }]])); + expect(() => readFileSync(join(dir, ".hashes.json"))).toThrow(); + }); +}); diff --git a/server/typescript/packages/docs-site/src/explorer/build.ts b/server/typescript/packages/docs-site/src/explorer/build.ts new file mode 100644 index 000000000..f7918becf --- /dev/null +++ b/server/typescript/packages/docs-site/src/explorer/build.ts @@ -0,0 +1,404 @@ +// The `explorer.json` builder — the data spine every explorer page reads. +// +// PORT-NEUTRAL INPUT, on purpose. It reads three documents and never the loader: +// +// model the EFFECTIVE canonical JSON (`meta export --effective`): inheritance and +// overlays already resolved, references already in their canonical spelling. +// verify the `meta verify --format json` payload: the requirement ledger rows, each +// link as the gate resolved it, the counts and the findings. +// manifest `.metaobjects/.gen-state/.hashes.json`: which generated file came from which +// node, where the port records it. +// +// Every port already has a canonical serializer and writes the same manifest, so a Java, C# +// or Python project can feed this without the Node CLI knowing its providers. The builder +// therefore resolves nothing the inputs have not resolved: an `@implementedBy` link becomes +// an edge only because the verify row names its target, never by a name match here. +// +// DETERMINISTIC: the same three inputs give the same bytes (`serializeExplorer`), so the +// file can sit under a drift gate. + +import { + ATTR_PREFIX, + FIELD_ATTR_OBJECT_REF, + IDENTITY_REFERENCE_ATTR_REFERENCES, + IDENTITY_SUBTYPE_REFERENCE, + PACKAGE_SEPARATOR, + CHILD_REF_SEPARATOR, + RELATIONSHIP_ATTR_OBJECT_REF, + RESERVED_KEY_ABSTRACT, + RESERVED_KEY_CHILDREN, + RESERVED_KEY_EXTENDS, + RESERVED_KEY_IS_ARRAY, + RESERVED_KEY_NAME, + RESERVED_KEY_PACKAGE, + SUBTYPE_ROOT, + TEMPLATE_ATTR_PAYLOAD_REF, + TEMPLATE_ATTR_RESPONSE_REF, + TYPE_FIELD, + TYPE_IDENTITY, + TYPE_METADATA, + TYPE_RELATIONSHIP, + TYPE_REQUIREMENT, + TYPE_SUBTYPE_SEPARATOR, + TYPE_TEMPLATE, +} from "@metaobjectsdev/metadata/constants"; +import { + EXPLORER_FORMAT, + type ExplorerEdge, + type ExplorerEdgeKind, + type ExplorerGenerated, + type ExplorerJson, + type ExplorerLedger, + type ExplorerLedgerRow, + type ExplorerNode, + type ExplorerSlice, +} from "./types.js"; + +export interface ExplorerInput { + /** Parsed `meta export --effective` output (or any port's effective canonical JSON). */ + model: unknown; + /** Parsed `meta verify --format json` output. Without it there is no ledger and no + * `implementedBy` / `supersededBy` edge. */ + verify?: unknown; + /** Parsed `.metaobjects/.gen-state/.hashes.json`. Without it there is no `generated`. */ + manifest?: unknown; +} + +const ROOT_KEY = `${TYPE_METADATA}${TYPE_SUBTYPE_SEPARATOR}${SUBTYPE_ROOT}`; + +/** Code-unit order: the same in every runtime and locale, unlike `localeCompare`. */ +const byCodeUnit = (a: string, b: string): number => (a < b ? -1 : a > b ? 1 : 0); + +function isRecord(v: unknown): v is Record { + return typeof v === "object" && v !== null && !Array.isArray(v); +} + +/** One `{ ".": body }` element of a `children` array. */ +interface RawChild { + type: string; + subType: string; + body: Record; +} + +function rawChildren(body: Record): RawChild[] { + const list = body[RESERVED_KEY_CHILDREN]; + if (!Array.isArray(list)) return []; + const out: RawChild[] = []; + for (const el of list) { + if (!isRecord(el)) continue; + const keys = Object.keys(el); + if (keys.length !== 1) continue; + const key = keys[0] as string; + const inner = el[key]; + if (!isRecord(inner)) continue; + const sep = key.indexOf(TYPE_SUBTYPE_SEPARATOR); + out.push({ + type: sep === -1 ? key : key.slice(0, sep), + subType: sep === -1 ? "" : key.slice(sep + 1), + body: inner, + }); + } + return out; +} + +function attrsOf(body: Record): Record { + const out: Record = {}; + for (const key of Object.keys(body).sort(byCodeUnit)) { + if (key.startsWith(ATTR_PREFIX)) out[key.slice(ATTR_PREFIX.length)] = body[key]; + } + return out; +} + +function stringAttr(body: Record, name: string): string | undefined { + const v = body[`${ATTR_PREFIX}${name}`]; + return typeof v === "string" && v !== "" ? v : undefined; +} + +/** A node as walked, before it is written out: its id, its raw body and its top-level owner. */ +interface Walked { + node: ExplorerNode; + body: Record; + /** The id of the top-level node this one sits under (itself, at the top). */ + owner: string; +} + +function walkModel(model: unknown): Walked[] { + if (!isRecord(model) || !isRecord(model[ROOT_KEY])) { + throw new Error(`explorer: the model is not canonical metadata JSON — expected one "${ROOT_KEY}" object`); + } + const root = model[ROOT_KEY] as Record; + const rootPkg = typeof root[RESERVED_KEY_PACKAGE] === "string" ? (root[RESERVED_KEY_PACKAGE] as string) : ""; + const out: Walked[] = []; + const seen = new Set(); + + const visit = (child: RawChild, index: number, parent: Walked | undefined): void => { + const name = typeof child.body[RESERVED_KEY_NAME] === "string" ? (child.body[RESERVED_KEY_NAME] as string) : ""; + let pkg: string; + let id: string; + if (parent === undefined) { + const own = child.body[RESERVED_KEY_PACKAGE]; + pkg = typeof own === "string" ? own : rootPkg; + id = pkg === "" ? name : `${pkg}${PACKAGE_SEPARATOR}${name}`; + } else { + pkg = parent.node.package; + id = name === "" ? `${parent.node.id}#${index}` : `${parent.node.id}${CHILD_REF_SEPARATOR}${name}`; + // The first sibling of a name keeps the plain id: it is the one a dotted reference + // resolves to. A later sibling of another type sharing the name is qualified by type. + if (name !== "" && seen.has(id)) id = `${id}[${child.type}${TYPE_SUBTYPE_SEPARATOR}${child.subType}]`; + } + if (seen.has(id)) { + throw new Error(`explorer: two nodes share the id "${id}" — the model names two siblings alike`); + } + seen.add(id); + const ext = child.body[RESERVED_KEY_EXTENDS]; + const node: ExplorerNode = { + id, + type: child.type, + subType: child.subType, + name, + package: pkg, + ...(parent !== undefined ? { parent: parent.node.id } : {}), + ...(typeof ext === "string" && ext !== "" ? { extends: nodeIdOf(ext) } : {}), + ...(child.body[RESERVED_KEY_ABSTRACT] === true ? { abstract: true as const } : {}), + ...(child.body[RESERVED_KEY_IS_ARRAY] === true ? { isArray: true as const } : {}), + attrs: attrsOf(child.body), + }; + const walked: Walked = { node, body: child.body, owner: parent === undefined ? id : parent.owner }; + out.push(walked); + rawChildren(child.body).forEach((c, i) => visit(c, i, walked)); + }; + + rawChildren(root).forEach((c, i) => visit(c, i, undefined)); + return out; +} + +/** A canonical reference spells a root-level target `::Name`; the node's id is `Name`. */ +function nodeIdOf(ref: string): string { + return ref.startsWith(PACKAGE_SEPARATOR) ? ref.slice(PACKAGE_SEPARATOR.length) : ref; +} + +/** + * Mark the members a top-level node's effective tree inherited: a direct child whose type, + * name and body equal a direct child of an ancestor named through `extends`. The id recorded + * is the TOPMOST such ancestor — where the member is declared. + */ +function markInherited(walked: Walked[], byId: Map): void { + const childKey = (w: Walked): string => `${w.node.type}${TYPE_SUBTYPE_SEPARATOR}${w.node.subType}:${w.node.name}`; + const directChildren = new Map>(); + for (const w of walked) { + if (w.node.parent === undefined || w.node.name === "") continue; + const m = directChildren.get(w.node.parent) ?? new Map(); + m.set(childKey(w), w); + directChildren.set(w.node.parent, m); + } + for (const w of walked) { + if (w.node.parent !== undefined || w.node.extends === undefined) continue; + const members = directChildren.get(w.node.id); + if (members === undefined) continue; + for (const [key, member] of members) { + const body = JSON.stringify(member.body); + let declaredOn: string | undefined; + const visited = new Set([w.node.id]); + let cur = byId.get(w.node.extends); + while (cur !== undefined && !visited.has(cur.node.id)) { + visited.add(cur.node.id); + const same = directChildren.get(cur.node.id)?.get(key); + if (same === undefined || JSON.stringify(same.body) !== body) break; + declaredOn = cur.node.id; + cur = cur.node.extends === undefined ? undefined : byId.get(cur.node.extends); + } + if (declaredOn !== undefined) member.node.inheritedFrom = declaredOn; + } + } +} + +function modelEdges(walked: Walked[], byId: Map): ExplorerEdge[] { + const edges: ExplorerEdge[] = []; + const add = (kind: ExplorerEdgeKind, from: string, to: string | undefined, via?: string): void => { + if (to === undefined) return; + const target = nodeIdOf(to); + if (!byId.has(from) || !byId.has(target)) return; + edges.push({ kind, from, to: target, ...(via !== undefined && via !== from ? { via } : {}) }); + }; + for (const w of walked) { + const { node, body, owner } = w; + if (node.extends !== undefined) add("extends", node.id, node.extends); + if (node.type === TYPE_IDENTITY && node.subType === IDENTITY_SUBTYPE_REFERENCE) { + add("references", owner, stringAttr(body, IDENTITY_REFERENCE_ATTR_REFERENCES), node.id); + } + if (node.type === TYPE_FIELD) add("objectRef", owner, stringAttr(body, FIELD_ATTR_OBJECT_REF), node.id); + if (node.type === TYPE_RELATIONSHIP) add("association", owner, stringAttr(body, RELATIONSHIP_ATTR_OBJECT_REF), node.id); + if (node.type === TYPE_TEMPLATE) { + add("payload", node.id, stringAttr(body, TEMPLATE_ATTR_PAYLOAD_REF)); + add("reply", node.id, stringAttr(body, TEMPLATE_ATTR_RESPONSE_REF)); + } + if (node.type === TYPE_REQUIREMENT && node.parent !== undefined && byId.get(node.parent)?.node.type === TYPE_REQUIREMENT) { + add("contains", node.parent, node.id); + } + } + return edges; +} + +function asLinks(v: unknown): { ref: string; resolves: boolean; target?: string }[] { + if (!Array.isArray(v)) return []; + return v.filter(isRecord).map(asLink).filter((l): l is NonNullable => l !== undefined); +} + +function asLink(v: unknown): { ref: string; resolves: boolean; target?: string } | undefined { + if (!isRecord(v) || typeof v.ref !== "string") return undefined; + return { + ref: v.ref, + resolves: v.resolves === true, + ...(typeof v.target === "string" ? { target: v.target } : {}), + }; +} + +function ledgerFrom(verify: unknown, byId: Map, edges: ExplorerEdge[]): ExplorerLedger | undefined { + if (!isRecord(verify) || !Array.isArray(verify.requirementLedger)) return undefined; + const rows: ExplorerLedgerRow[] = []; + for (const raw of verify.requirementLedger) { + if (!isRecord(raw) || typeof raw.qualifiedPath !== "string" || typeof raw.path !== "string") continue; + const node = raw.qualifiedPath; + const implementedBy = asLinks(raw.implementedBy); + const supersededBy = asLink(raw.supersededBy); + const diagnostics = Array.isArray(raw.diagnostics) + ? raw.diagnostics.filter(isRecord).map((d) => ({ + code: String(d.code ?? ""), + severity: String(d.severity ?? ""), + source: String(d.source ?? ""), + })) + : []; + rows.push({ + node, + path: raw.path, + package: typeof raw.package === "string" ? raw.package : "", + qualifiedPath: raw.qualifiedPath, + subType: typeof raw.subType === "string" ? raw.subType : "", + ...(typeof raw.level === "number" ? { level: raw.level } : {}), + ...(typeof raw.status === "string" ? { status: raw.status } : {}), + ...(typeof raw.disposition === "string" ? { disposition: raw.disposition } : {}), + digest: typeof raw.digest === "string" ? raw.digest : "", + implementedBy, + ...(supersededBy !== undefined ? { supersededBy } : {}), + diagnostics, + verdict: typeof raw.verdict === "string" ? raw.verdict : "", + }); + if (byId.has(node)) { + for (const link of implementedBy) { + if (link.target !== undefined && byId.has(link.target)) edges.push({ kind: "implementedBy", from: node, to: link.target }); + } + if (supersededBy?.target !== undefined && byId.has(supersededBy.target)) { + edges.push({ kind: "supersededBy", from: node, to: supersededBy.target }); + } + } + } + const requirements = verify.requirements; + return { + counts: isRecord(verify.requirementCounts) ? verify.requirementCounts : {}, + rows, + diagnostics: isRecord(requirements) && Array.isArray(requirements.rows) ? requirements.rows : [], + }; +} + +function generatedFrom(manifest: unknown): ExplorerGenerated[] | undefined { + if (!isRecord(manifest)) return undefined; + const byNode = new Map(); + for (const [path, entry] of Object.entries(manifest)) { + if (!isRecord(entry) || typeof entry.sourceNode !== "string") continue; + const by = typeof entry.generatedBy === "string" ? entry.generatedBy : ""; + const list = byNode.get(entry.sourceNode) ?? []; + list.push({ path, generatedBy: by }); + byNode.set(entry.sourceNode, list); + } + return [...byNode.keys()].sort(byCodeUnit).map((node) => ({ + node, + files: (byNode.get(node) ?? []).sort((a, b) => byCodeUnit(a.path, b.path)), + })); +} + +function derivedSlices(walked: Walked[], byId: Map, edges: ExplorerEdge[]): ExplorerSlice[] { + const slices: ExplorerSlice[] = []; + const ownerOf = (id: string): string => byId.get(id)?.owner ?? id; + const sorted = (s: Set): string[] => [...s].sort(byCodeUnit); + + // Per package: its top-level nodes. + const packages = new Map>(); + for (const w of walked) { + if (w.node.parent !== undefined) continue; + const set = packages.get(w.node.package) ?? new Set(); + set.add(w.node.id); + packages.set(w.node.package, set); + } + for (const [pkg, ids] of packages) slices.push({ id: `package:${pkg}`, kind: "package", anchor: pkg, nodes: sorted(ids) }); + + // Per requirement subtree, for every requirement with nested requirements: the subtree, + // each node it claims, and the object owning a claimed member. + const containsFrom = new Map(); + const claimsFrom = new Map(); + for (const e of edges) { + const target = e.kind === "contains" ? containsFrom : e.kind === "implementedBy" ? claimsFrom : undefined; + if (target === undefined) continue; + const list = target.get(e.from) ?? []; + list.push(e.to); + target.set(e.from, list); + } + for (const w of walked) { + if (w.node.type !== TYPE_REQUIREMENT || !containsFrom.has(w.node.id)) continue; + const set = new Set(); + const stack = [w.node.id]; + while (stack.length > 0) { + const id = stack.pop() as string; + if (set.has(id)) continue; + set.add(id); + for (const t of claimsFrom.get(id) ?? []) { + set.add(t); + set.add(ownerOf(t)); + } + stack.push(...(containsFrom.get(id) ?? [])); + } + slices.push({ id: `requirement:${w.node.id}`, kind: "requirement", anchor: w.node.id, nodes: sorted(set) }); + } + + // Per prompt: the prompt, its payload and its reply. + for (const w of walked) { + if (w.node.type !== TYPE_TEMPLATE || w.node.parent !== undefined) continue; + const set = new Set([w.node.id]); + for (const e of edges) if (e.from === w.node.id && (e.kind === "payload" || e.kind === "reply")) set.add(e.to); + slices.push({ id: `prompt:${w.node.id}`, kind: "prompt", anchor: w.node.id, nodes: sorted(set) }); + } + + return slices.sort((a, b) => byCodeUnit(a.id, b.id)); +} + +const edgeKey = (e: ExplorerEdge): string => `${e.kind}\u0000${e.from}\u0000${e.to}\u0000${e.via ?? ""}`; + +/** Build `explorer.json` from the port-neutral inputs. Throws when `model` is not canonical + * metadata JSON; a missing or malformed `verify` / `manifest` only leaves its part out. */ +export function buildExplorer(input: ExplorerInput): ExplorerJson { + const walked = walkModel(input.model); + const byId = new Map(walked.map((w) => [w.node.id, w] as const)); + markInherited(walked, byId); + const edges = modelEdges(walked, byId); + const ledger = input.verify !== undefined ? ledgerFrom(input.verify, byId, edges) : undefined; + const generated = input.manifest !== undefined ? generatedFrom(input.manifest) : undefined; + + // One edge per (kind, from, to, via): an object can reach one target twice through two + // members, which is two edges, but never the same edge twice. + const unique = new Map(); + for (const e of edges) unique.set(edgeKey(e), e); + const sortedEdges = [...unique.values()].sort((a, b) => byCodeUnit(edgeKey(a), edgeKey(b))); + + return { + format: EXPLORER_FORMAT, + nodes: walked.map((w) => w.node).sort((a, b) => byCodeUnit(a.id, b.id)), + edges: sortedEdges, + ...(ledger !== undefined ? { ledger } : {}), + ...(generated !== undefined ? { generated } : {}), + slices: derivedSlices(walked, byId, sortedEdges), + }; +} + +/** The file's bytes: two-space JSON and one trailing newline. */ +export function serializeExplorer(explorer: ExplorerJson): string { + return JSON.stringify(explorer, null, 2) + "\n"; +} diff --git a/server/typescript/packages/docs-site/src/explorer/types.ts b/server/typescript/packages/docs-site/src/explorer/types.ts new file mode 100644 index 000000000..9fc5a69ce --- /dev/null +++ b/server/typescript/packages/docs-site/src/explorer/types.ts @@ -0,0 +1,121 @@ +// The `explorer.json` shape: the one data file every explorer view reads. +// +// It is BUILT from port-neutral inputs only (see build.ts), so its fields hold what those +// inputs state and nothing a loader would have to recompute. + +/** Bumped when a field changes meaning or is removed; adding a field does not bump it. */ +export const EXPLORER_FORMAT = "metaobjects-explorer/1"; + +/** One metadata node: a top-level node (object, prompt, requirement, …) or any node nested + * in one (field, identity, source, view, nested requirement, …). */ +export interface ExplorerNode { + /** `::` for a top-level node (`` with no package), then `.` + * per nesting level: `acme::shop::Order`, `acme::shop::Order.total`, + * `acme::shop::Shop.Checkout`. A nested node with no name gets `#` instead. */ + id: string; + type: string; + subType: string; + /** "" for a nested node with no name. */ + name: string; + /** The package of the node's top-level ancestor, "" when it has none. */ + package: string; + /** The id of the node this one is nested in; absent on a top-level node. */ + parent?: string; + /** The id `extends` names, as the effective tree spells it. */ + extends?: string; + abstract?: true; + isArray?: true; + /** On a member the effective tree materialized from a super: the id of the object it is + * declared on. Read as "same type, name and body as that ancestor's member", so a member + * redeclared with a different body is this node's own. */ + inheritedFrom?: string; + /** Every `@` attribute on the effective node, without the `@`, keys sorted. */ + attrs: Record; +} + +export const EXPLORER_EDGE_KINDS = [ + "extends", + "references", + "association", + "objectRef", + "payload", + "reply", + "contains", + "implementedBy", + "supersededBy", +] as const; +export type ExplorerEdgeKind = (typeof EXPLORER_EDGE_KINDS)[number]; + +/** A typed link between two nodes. Only links whose both ends are nodes are drawn. */ +export interface ExplorerEdge { + kind: ExplorerEdgeKind; + from: string; + to: string; + /** The member that carries the link, when it is not `from` itself: the + * `identity.reference`, the `field.object`, the `relationship.*`. */ + via?: string; +} + +/** One requirement as `meta verify --format json` judged it (`requirementLedger[]`), with + * the node id it has here. */ +export interface ExplorerLedgerRow { + node: string; + path: string; + package: string; + qualifiedPath: string; + subType: string; + level?: number; + status?: string; + disposition?: string; + digest: string; + implementedBy: { ref: string; resolves: boolean; target?: string }[]; + supersededBy?: { ref: string; resolves: boolean; target?: string }; + diagnostics: { code: string; severity: string; source: string }[]; + verdict: string; +} + +export interface ExplorerLedger { + /** `requirementCounts` from the verify payload, verbatim. */ + counts: Record; + /** One row per requirement, in the verify payload's (document) order. */ + rows: ExplorerLedgerRow[]; + /** `requirements.rows` from the verify payload, verbatim: every gate and lint finding, + * including the ones whose subject is not a requirement (an unclaimed entity). */ + diagnostics: unknown[]; +} + +export interface ExplorerGeneratedFile { + /** Project-relative, as the manifest keys it. */ + path: string; + generatedBy: string; +} + +/** The generated files the manifest attributes to one node. */ +export interface ExplorerGenerated { + node: string; + files: ExplorerGeneratedFile[]; +} + +export type ExplorerSliceKind = "requirement" | "package" | "prompt"; + +/** A named node set a view opens on, derived from the model with no authoring. */ +export interface ExplorerSlice { + /** `:`. */ + id: string; + kind: ExplorerSliceKind; + /** The requirement, package or prompt the slice is derived from ("" for the root package). */ + anchor: string; + /** Node ids, sorted. */ + nodes: string[]; +} + +export interface ExplorerJson { + format: typeof EXPLORER_FORMAT; + nodes: ExplorerNode[]; + edges: ExplorerEdge[]; + /** Absent without a verify payload, or when the model declares no requirement. */ + ledger?: ExplorerLedger; + /** Absent without a manifest. */ + generated?: ExplorerGenerated[]; + slices: ExplorerSlice[]; +} diff --git a/server/typescript/packages/docs-site/src/index.ts b/server/typescript/packages/docs-site/src/index.ts index 47ab07b96..6fdf286c9 100644 --- a/server/typescript/packages/docs-site/src/index.ts +++ b/server/typescript/packages/docs-site/src/index.ts @@ -2,3 +2,18 @@ export { generateSite, DEFAULT_ASSET_BASE } from "./site.js"; export type { SiteOptions, SiteResult } from "./site.js"; export { SITE_TEMPLATE_NAMES, SITE_ASSET_NAMES, readSiteFile } from "./scaffold.js"; export type { MetaDataTypeProvider } from "@metaobjectsdev/metadata"; +export { buildExplorer, serializeExplorer } from "./explorer/build.js"; +export type { ExplorerInput } from "./explorer/build.js"; +export { EXPLORER_FORMAT, EXPLORER_EDGE_KINDS } from "./explorer/types.js"; +export type { + ExplorerJson, + ExplorerNode, + ExplorerEdge, + ExplorerEdgeKind, + ExplorerLedger, + ExplorerLedgerRow, + ExplorerGenerated, + ExplorerGeneratedFile, + ExplorerSlice, + ExplorerSliceKind, +} from "./explorer/types.js"; diff --git a/server/typescript/packages/docs-site/test/explorer-build.test.ts b/server/typescript/packages/docs-site/test/explorer-build.test.ts new file mode 100644 index 000000000..00718de1e --- /dev/null +++ b/server/typescript/packages/docs-site/test/explorer-build.test.ts @@ -0,0 +1,125 @@ +// `buildExplorer` — the explorer data spine — over the committed conformance fixture +// (`fixtures/explorer-conformance/shop/`), checked byte for byte, plus the behaviour a +// byte check alone would not explain when it breaks. +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { buildExplorer, serializeExplorer, type ExplorerJson } from "../src/index.js"; + +const FIXTURE = resolve(import.meta.dir, "../../../../../fixtures/explorer-conformance/shop"); +const text = (name: string): string => readFileSync(join(FIXTURE, name), "utf8"); +const json = (name: string): unknown => JSON.parse(text(name)); + +function build(parts: { verify?: boolean; manifest?: boolean } = { verify: true, manifest: true }): ExplorerJson { + return buildExplorer({ + model: json("effective.json"), + ...(parts.verify ? { verify: json("verify.json") } : {}), + ...(parts.manifest ? { manifest: json("hashes.json") } : {}), + }); +} + +describe("explorer-conformance/shop", () => { + test("builds expected.json byte for byte", () => { + expect(serializeExplorer(build())).toBe(text("expected.json")); + }); + + test("object key order in the model and the manifest does not change the bytes", () => { + const reversed = (v: unknown): unknown => + Array.isArray(v) ? v.map(reversed) + : typeof v === "object" && v !== null + ? Object.fromEntries(Object.entries(v).reverse().map(([k, x]) => [k, reversed(x)])) + : v; + const explorer = buildExplorer({ + model: reversed(json("effective.json")), + verify: json("verify.json"), + manifest: reversed(json("hashes.json")), + }); + expect(serializeExplorer(explorer)).toBe(text("expected.json")); + }); +}); + +describe("buildExplorer", () => { + test("marks inherited members with the object that declares them", () => { + const nodes = new Map(build().nodes.map((n) => [n.id, n])); + expect(nodes.get("acme::shop::Order.id")?.inheritedFrom).toBe("acme::shop::BaseEntity"); + expect(nodes.get("acme::shop::Order.total")?.inheritedFrom).toBeUndefined(); + // AuditEntry declares its own `id` and does not extend BaseEntity. + expect(nodes.get("acme::shop::AuditEntry.id")?.inheritedFrom).toBeUndefined(); + }); + + test("an implementedBy edge exists only where the verify ledger resolved the link", () => { + const withVerify = build().edges.filter((e) => e.kind === "implementedBy"); + expect(withVerify).toContainEqual({ kind: "implementedBy", from: "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", to: "acme::shop::Order.reference" }); + // `Refunded` names `Refund`, which does not exist: a ledger row, no edge. + expect(withVerify.some((e) => e.from.endsWith("Refunded"))).toBe(false); + expect(build({ manifest: true }).edges.some((e) => e.kind === "implementedBy")).toBe(false); + expect(build({ manifest: true }).ledger).toBeUndefined(); + }); + + test("generated groups the manifest's attributed files by node; plain entries are skipped", () => { + expect(build().generated?.map((g) => g.node)).toEqual(["acme::shop::Customer", "acme::shop::Order"]); + expect(build({ verify: true }).generated).toBeUndefined(); + }); + + test("a requirement slice holds the subtree, its targets and the owner of a claimed member", () => { + const slice = build().slices.find((s) => s.id === "requirement:acme::shop::Shop.Checkout.Orders.Recorded"); + expect(slice?.nodes).toEqual([ + "acme::shop::Customer", + "acme::shop::Order", + "acme::shop::Order.reference", + "acme::shop::Shop.Checkout.Orders.Recorded", + "acme::shop::Shop.Checkout.Orders.Recorded.Quotable", + ]); + }); + + test("rejects input that is not canonical metadata JSON", () => { + expect(() => buildExplorer({ model: { nodes: [] } })).toThrow(/not canonical metadata JSON/); + }); + + test("a sibling sharing a member's name gets a type-qualified id; the member keeps the plain one", () => { + const nodes = new Map(build().nodes.map((n) => [n.id, n])); + expect(nodes.get("acme::shop::AuditEntry.id")?.type).toBe("field"); + expect(nodes.get("acme::shop::AuditEntry.id[identity.primary]")?.type).toBe("identity"); + }); + + test("rejects two top-level nodes sharing an id", () => { + const model = { + "metadata.root": { + package: "p", + children: [ + { "object.entity": { name: "A" } }, + { "object.value": { name: "A" } }, + ], + }, + }; + expect(() => buildExplorer({ model })).toThrow(/two nodes share the id "p::A"/); + }); + + test("a root-level target spelled `::Name` still gets its edges and inheritedFrom", () => { + const idField = { name: "id", type: "long" }; + const model = { + "metadata.root": { + children: [ + { "object.entity": { name: "BaseEntity", abstract: true, children: [{ "field.long": idField }] } }, + { + "object.entity": { + name: "Order", + package: "acme::shop", + extends: "::BaseEntity", + children: [ + { "field.long": idField }, + { "relationship.association": { name: "owner", "@objectRef": "::BaseEntity" } }, + ], + }, + }, + ], + }, + }; + const built = buildExplorer({ model }); + const nodes = new Map(built.nodes.map((n) => [n.id, n])); + expect(nodes.get("acme::shop::Order")?.extends).toBe("BaseEntity"); + expect(nodes.get("acme::shop::Order.id")?.inheritedFrom).toBe("BaseEntity"); + expect(built.edges).toContainEqual({ kind: "extends", from: "acme::shop::Order", to: "BaseEntity" }); + expect(built.edges).toContainEqual({ kind: "association", from: "acme::shop::Order", to: "BaseEntity", via: "acme::shop::Order.owner" }); + }); +});