Skip to content

Commit daf64ef

Browse files
committed
Merge remote-tracking branch 'origin/main' into fm/mo-deps-p2-s6
2 parents 0254dc7 + 2433746 commit daf64ef

126 files changed

Lines changed: 6076 additions & 332 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/rules/cross-language-porting.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ Preserve the following contracts exactly across all language ports:
4848
- Overlay/merge: same `package` + same object `name` across multiple files → merged. Last-writer-wins on attr conflicts; structural children accumulate.
4949
- Default scan path: `metaobjects/**/*.json` (recursive).
5050

51-
**D1 is TS-only.** Cloudflare D1 is a peer of `sqlite`/`postgres` in TS's `dialect` vocabulary. It is SQLite at the SQL level — Java/Python/C# don't have an analogue (Cloudflare Workers run JS). When adding cross-language vocabulary, D1 doesn't constrain anything: its uniqueness is wrangler-CLI transport + Wrangler-native file layout (`migrations/<seq>_<slug>.sql`), both of which are TS-only concerns.
51+
**D1 is TS-only.** Cloudflare D1 is a peer of `sqlite`/`postgres` in `meta migrate`'s `dialect` vocabulary (`migrate.dialect`, `--dialect d1`); codegen's `dialect` is `sqlite`/`postgres`/`mysql` and D1 uses `sqlite` (`"d1"` there is an error). It is SQLite at the SQL level — Java/Python/C# don't have an analogue (Cloudflare Workers run JS). When adding cross-language vocabulary, D1 doesn't constrain anything: its uniqueness is wrangler-CLI transport + Wrangler-native file layout (`migrations/<seq>_<slug>.sql`), both of which are TS-only concerns.
5252

5353
**Constants discipline:**
5454
- TS: named constants, imported from `@metaobjectsdev/metadata/constants` (defined in 16 per-concern `*-constants.ts` modules under `packages/metadata/src/`, e.g. `core/field/field-constants.ts`, `persistence/db/db-constants.ts`). Never inline metamodel strings as literals in code.

‎CHANGELOG.md‎

Lines changed: 61 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,34 @@ here.**
205205

206206
### Changed
207207

208+
- **On Postgres, `meta verify --db` and `meta migrate` ask Postgres whether a CHECK or an index
209+
expression is the one stored, instead of comparing rewritten text.** For each table the plan
210+
creates a temporary table with the live columns inside a transaction it rolls back (a savepoint
211+
when the connection is already in one), adds the expected CHECK to it, creates each expected
212+
expression-index key and partial-index predicate as a temporary index, reads back
213+
`pg_get_constraintdef`, `pg_get_indexdef` and `pg_get_expr`, and compares them with the live
214+
definitions as plain strings. Every rewrite Postgres makes, including ones nobody has hand-coded,
215+
now compares equal, and the string normaliser stops growing Postgres-shaped special cases. It
216+
also ends the normaliser's two verified false negatives: a precedence change (`(a OR b) AND c`
217+
against `a OR (b AND c)`) and a regrouped `NOT` (`NOT (a AND b)` against `NOT a AND b`) now report
218+
as drift, as does the same change in an index `@where`. A database that held such a change and
219+
verified clean before now reports it once: run `meta migrate` to bring it back.
220+
The string normaliser stays, and decides, on SQLite and D1, in the offline `meta migrate`
221+
(which has no database), on a read-only connection or hot standby, for a role without the `TEMP`
222+
privilege, and for an expression that does not compile against the live columns. A probe that
223+
cannot answer never reports drift or fails the command. An expression holding a `;` anywhere, or
224+
an index `@using` that is not a plain access-method name, is never sent to the database. A refusal
225+
for one table (such as no `SELECT` on it) gives that table no answer and does not stop the other
226+
tables being probed. `meta migrate baseline --from-db` now records the generated text, not
227+
Postgres's spelling, for a CHECK or an index expression the live database proves equal to the
228+
metadata's, so the offline plan after a baseline compares like with like; it is best effort and
229+
writes the introspected snapshot as before if the metadata does not load.
230+
`meta verify --db` also uses the probe for its committed-snapshot check. New exports from
231+
`@metaobjectsdev/migrate-ts`: `postgresExprCanonProbe`, `adoptGeneratedSpelling` and the
232+
`ExprCanonProbe` types, with a `DiffArgs.exprCanonProbe` option. Schema comparison is
233+
TypeScript-owned, so no other port carries this comparison. MySQL, SQL Server and Oracle are not
234+
migrate dialects, so they have no probe.
235+
208236
- **A bare `verify` runs the same default gate set in every port: codegen drift, prompt
209237
templates and the requirement ledger.** Until now a bare run covered one gate: `meta verify`
210238
and `dotnet meta verify` ran only templates, so "run `verify` in CI" did not catch generated
@@ -248,6 +276,29 @@ here.**
248276
the command and rolls the sync back. A stale `docs/CONFORMANCE.md` cell that claimed Python passed
249277
only 2 of the dependency corpus's cases is corrected.
250278

279+
- **Generated routes no longer accept or return fields they must not (#484).** A field marked
280+
`@formExclude`, or a credential (a field that declares a `view.password`), is now **ignored** by
281+
the generated create, PATCH and PUT in every port: the value is dropped, not rejected, so
282+
an edit form that submits every field still saves, and a present `null` no longer clears it. A credential is
283+
also never returned: not by list, get, or the echo of a write, or by a served report that groups by
284+
it or takes its `min` or `max` (a report column is derived from its `@of` field, and it keeps the
285+
credential mark; a report export names no column for it, and that column is not filterable or sortable). Before this, `@formExclude` kept
286+
a field out of the generated form while the route still wrote it, and a generated read returned password
287+
hashes. Generated forms (React, Angular) and the api docs omit credentials too. Your own code is unaffected:
288+
the entity, DTO and generated queries still carry the field, so you keep writing a hash or token yourself.
289+
A project with neither mark generates byte-identical output. If you relied on a route writing a
290+
`@formExclude` field, write it from your own code. A field that is `@required` and also
291+
`@formExclude` or a credential now makes the generated create route fail, because the route
292+
can no longer supply its value: make the field optional, or create that entity from your own
293+
code (see "Credentials and excluded fields" in `docs/features/api-contract.md`). Gated by the new `secret-fields/` sub-corpus
294+
of the api-contract conformance corpus, in the generated lane of all five ports.
295+
296+
- **`meta gen` and `meta verify --codegen` refuse an unknown codegen `dialect`** instead of silently
297+
generating Postgres code. `dialect: "d1"` gets a message that says to use `"sqlite"` (D1 is
298+
SQLite; `"d1"` names only the `meta migrate` transport). `meta init --d1` now scaffolds
299+
`dialect: "sqlite"` in `metaobjects.config.ts`; it used to write `"d1"`. If your config has `dialect: "d1"`,
300+
change it to `"sqlite"`.
301+
251302
- **The "Model / metadata" link in the Python, C#, Java and Kotlin API docs no longer points at
252303
a page that does not exist.** Those ports write no model pages, so the relative link was dead.
253304
It is now written only when `modelBaseUrl` is set.
@@ -286,13 +337,19 @@ here.**
286337
`meta migrate` to bring it back. Separately, `char(n)` is no longer treated as a text column
287338
for the `(col)::text` fold: its own regex operator sees the blank padding the cast trims, and
288339
Postgres never adds the cast there, so an explicit `(col)::text ~ '…'` on a `char(n)` column
289-
now reads as drift. A precedence change (`(a OR b) AND c` against `a OR (b AND c)`) is still
290-
not detected; that needs Postgres to canonicalise the expected side. A new Postgres test
291-
migrates, applies and verifies every CHECK shape `migrate` generates. Schema comparison is
292-
TypeScript-owned, so no other port carries this comparison.
340+
now reads as drift. A precedence change (`(a OR b) AND c` against `a OR (b AND c)`) is not
341+
detected by the string comparison; on Postgres Postgres now answers that (see "Changed", the
342+
CHECK probe). A new Postgres test migrates, applies and verifies every CHECK shape `migrate`
343+
generates. Schema comparison is TypeScript-owned, so no other port carries this comparison.
293344

294345
### Documentation
295346

347+
- **New recipes and a real D1 example.** `docs/recipes/` gains keeping drizzle-kit migrations
348+
(`drizzle-kit-migrations.md`), guarding generated routes with your own auth
349+
(`guarding-generated-routes.md`) and proving your `meta verify` gate can fail
350+
(`prove-your-gate-fails.md`). The Cloudflare Workers recipe now has a per-request `drizzle(env.DB)`
351+
example, and the `drizzle({} as never)` placeholder and the wrong `"d1"` codegen-dialect statements are corrected.
352+
296353
- **Two doc errors corrected against the code.** `docs/recipes/mysql.md` said Python ships a
297354
Postgres driver only; it ships SQLite and MySQL drivers beside it. `docs/features/api-contract.md`
298355
said only TypeScript enforces the `in`-list cap of 100; every port does.

‎agent-context/skills/metaobjects-audit/references/capability-checklist.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -198,8 +198,10 @@ soft-delete / status / type view it models.
198198
and are NOT in the cross-port registry — `view.text`, `view.textarea`, `view.date`,
199199
`view.month`, `view.hotlink`, `view.image`, `view.dropdown`, `view.radio`, `view.checkbox`,
200200
`view.number`, `view.password`, `view.hidden`, `view.web`. Every port LOADS them, so shared metadata that
201-
carries them is valid in a backend port; nothing outside TS/web reads them. **Audit these
202-
only for TS adopters.** Only
201+
carries them is valid in a backend port. The one exception is `view.password`: every port's generated
202+
routes read it as a credential, so it is never returned and never written (see
203+
"Credentials and excluded fields" in `docs/features/api-contract.md`).
204+
Audit the other widgets only for TS adopters. Only
203205
`view.base` / `view.currency` are cross-port-gated.
204206

205207
## Template — `template.*` (prompt pillar)

‎agent-context/skills/metaobjects-audit/references/typescript.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -199,4 +199,4 @@ TypeScript is the reference implementation — it ships the full feature set:
199199
- Output-parser codegen: **complete** (`outputParser()` generator; `parse*`/`safeParse*`/`extract*` emitted as `<Name>.response.ts` per **responding `template.prompt`** — one carrying `@responseRef`; ADR-0052).
200200
- ObjectManager: **complete** (`@metaobjectsdev/runtime-ts`; `kyselyDriver` / `inMemoryDriver`).
201201
- Schema migrate / `meta verify --db`: **complete** (TS owns the shared migrate engine — ADR-0015).
202-
- D1 dialect: **TS-only by design** — `dialect: "d1"` in `metaobjects.config.ts` targets Cloudflare D1 via the Wrangler CLI layout. Do NOT recommend porting D1 support to other languages.
202+
- D1 dialect: **TS-only by design** — `migrate.dialect: "d1"` (and `meta migrate --dialect d1`) targets Cloudflare D1 via the Wrangler CLI layout; codegen's `dialect` in `metaobjects.config.ts` is `"sqlite"` for D1 and `"d1"` there is refused. Do NOT recommend porting D1 support to other languages.

‎docs/CONFORMANCE.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
3636
| [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ |
3737
| [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ |
3838
| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) |
39-
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 117 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
39+
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 119 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
4040
| [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) |
4141
| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) |
4242
| [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ |
@@ -365,9 +365,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc
365365
`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the
366366
write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates).
367367

368-
### `fixtures/api-contract-conformance/` (117)
368+
### `fixtures/api-contract-conformance/` (119)
369369

370-
All 117 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
370+
All 119 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
371371
REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD
372372
routes answer identically over HTTP — list / get / create / patch+put / delete,
373373
plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1`
@@ -378,11 +378,11 @@ The corpus also covers the 9 cross-port filter operators (`eq`, `ne`, `gt`,
378378
`gte`, `lt`, `lte`, `in`, `like`, `isNull`) plus the implicit-AND combinator
379379
and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the
380380
URL grammar `?filter[<field>][<op>]=<value>` (FR-009). On top of the 31 core
381-
scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table
381+
scenarios the corpus carries eight sub-corpora — `tph/` (10, single-table
382382
inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination
383383
each corpus alone could not reach, and 1 pinning the collection-URL spelling),
384384
`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
385-
`write-through/` (2, table-write + view-read entities), `projection/` (12, a
385+
`write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, and a credential is never returned), `projection/` (12, a
386386
read-only view answers reads and refuses writes with 405; a projection with no declared
387387
identity has no item route; a projection keyed on a field not named `id` is addressed by
388388
it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one
@@ -392,7 +392,7 @@ its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` re
392392
a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java,
393393
Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and
394394
the port's own GENERATED API artifact booted over HTTP. `write-through/`,
395-
`projection/`, `decimal/` and `report/` run the generated lane only, on all five ports: what
395+
`secret-fields/`, `projection/`, `decimal/` and `report/` run the generated lane only, on all five ports: what
396396
they test is whether a port's generator emits the routes. The `report/` sub-corpus is the one
397397
that reads real views, so it runs full stack on Postgres, SQLite and MySQL
398398
(`report/schema.sqlite.sql`, `report/schema.mysql.sql`, both TypeScript-produced) in every
@@ -610,7 +610,7 @@ files into a real Cube.
610610
## Orphaned fixtures (tested but not yet documented)
611611

612612
The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
613-
+ render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 +
613+
+ render 16 + persistence 59 + api-contract 119 + source-resolution 25 + scope 10 +
614614
dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the
615615
`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining
616616
corpora in the totals table gate tooling contracts (registry manifests, provider

‎docs/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ this tree is documentation, not the source of truth.
8484
| See a worked, non-toy model end to end (projections, value objects, TPH, prompt payloads on one runnable spine) | [`../examples/advanced-modeling/`](../examples/advanced-modeling/) |
8585
| Read the canonical spec (target-agnostic) | [`../spec/`](../spec/) |
8686
| Find a release recipe (Cloudflare D1, etc.) | [`recipes/`](recipes/) |
87+
| Keep drizzle-kit migrations while MetaObjects owns the model | [`recipes/drizzle-kit-migrations.md`](recipes/drizzle-kit-migrations.md) |
88+
| Put generated routes behind your own auth (Fastify, Hono, `expose`) | [`recipes/guarding-generated-routes.md`](recipes/guarding-generated-routes.md) |
89+
| Show that your `meta verify` CI gate actually fails on drift | [`recipes/prove-your-gate-fails.md`](recipes/prove-your-gate-fails.md) |
8790
| See the deeper design rationale for a feature | [`superpowers/specs/`](superpowers/specs/) |
8891

8992
## Conventions used in these docs

0 commit comments

Comments
 (0)