Skip to content

Commit 1dfb18a

Browse files
authored
fix: keep credential and excluded fields out of generated routes, forms and reports (#503)
* fix(codegen): generated routes ignore @formExclude and credential fields on write, never return credentials (#484) A field marked @formExclude, or a credential (a field declaring a view.password child), is now ignored by the generated create, PATCH and PUT in every port and a credential is never returned by list, get, write echoes, M:N targets or TPH rows. Generated forms and api docs omit credentials too. Pinned by the new api-contract secret-fields/ sub-corpus (generated lane, all five ports). Also: meta gen / meta verify --codegen refuse an unknown codegen dialect ("d1" points at "sqlite"; meta init --d1 scaffolds "sqlite"), and five docs recipes (drizzle-kit migrations, Workers + D1 per-request example, guarding generated routes, proving a gate fails). Fixes #484 * fix(codegen): hide subtype credentials on an M:N onto a TPH base; Python redaction covers dataclass and mapping rows (#484) * refactor: simplify the #484 changes * fix(codegen): redact Java read-only mounts and Kotlin at the controller boundary, prove denied writes on stored rows (#484) * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Trim codegen-ts barrel, note required-denied create, behavioral projection test * no-mistakes(review): Add behavioral credential checks for TS/Python projection and TPH M:N * no-mistakes(review): Count TPH credential and denied marks on every subtype * no-mistakes(review): Deny hierarchy-marked TPH columns on every subtype PATCH * no-mistakes(review): Fix stale KDoc on Kotlin M:N credential fields * no-mistakes(review): Widen TS and C# TPH subtype marks to hierarchy * no-mistakes(test): Import TphPlan in ejectable SpringGeneratorSupport, widen discriminatorRoot * no-mistakes(document): Docs current; fixed capability checklist claim that view.password is TS-only * fix(codegen): a served report hides a credential column in every port (#484) A report column is derived from its @Of field and lost its view.password, so a report that groups by a credential, or takes its min or max, served it from the list route and the export. A derived field over a credential is now a credential in every port: the list route omits the key and an export names no column for it. Also documents it in the CHANGELOG entry for #484, the api-contract and reporting guides. * fix(codegen): a credential report column is neither filterable nor sortable (#484) A served report marks every derived field filterable and sortable, so a column hidden from the response could still be probed with a like-prefix filter or a sort, recovering the secret one prefix at a time. A derived field over a credential now carries no filter or sort entry in every port. Also documents it in the CHANGELOG, api-contract and reporting guides. * no-mistakes(review): Test C# report JSON export omits credential keys
1 parent a50b9f2 commit 1dfb18a

111 files changed

Lines changed: 5135 additions & 281 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/rules/cross-language-porting.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ Preserve the following contracts exactly across all language ports:
4848
- Overlay/merge: same `package` + same object `name` across multiple files → merged. Last-writer-wins on attr conflicts; structural children accumulate.
4949
- Default scan path: `metaobjects/**/*.json` (recursive).
5050

51-
**D1 is TS-only.** Cloudflare D1 is a peer of `sqlite`/`postgres` in TS's `dialect` vocabulary. It is SQLite at the SQL level — Java/Python/C# don't have an analogue (Cloudflare Workers run JS). When adding cross-language vocabulary, D1 doesn't constrain anything: its uniqueness is wrangler-CLI transport + Wrangler-native file layout (`migrations/<seq>_<slug>.sql`), both of which are TS-only concerns.
51+
**D1 is TS-only.** Cloudflare D1 is a peer of `sqlite`/`postgres` in `meta migrate`'s `dialect` vocabulary (`migrate.dialect`, `--dialect d1`); codegen's `dialect` is `sqlite`/`postgres`/`mysql` and D1 uses `sqlite` (`"d1"` there is an error). It is SQLite at the SQL level — Java/Python/C# don't have an analogue (Cloudflare Workers run JS). When adding cross-language vocabulary, D1 doesn't constrain anything: its uniqueness is wrangler-CLI transport + Wrangler-native file layout (`migrations/<seq>_<slug>.sql`), both of which are TS-only concerns.
5252

5353
**Constants discipline:**
5454
- TS: named constants, imported from `@metaobjectsdev/metadata/constants` (defined in 16 per-concern `*-constants.ts` modules under `packages/metadata/src/`, e.g. `core/field/field-constants.ts`, `persistence/db/db-constants.ts`). Never inline metamodel strings as literals in code.

‎CHANGELOG.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,29 @@ here.**
235235
the command and rolls the sync back. A stale `docs/CONFORMANCE.md` cell that claimed Python passed
236236
only 2 of the dependency corpus's cases is corrected.
237237

238+
- **Generated routes no longer accept or return fields they must not (#484).** A field marked
239+
`@formExclude`, or a credential (a field that declares a `view.password`), is now **ignored** by
240+
the generated create, PATCH and PUT in every port: the value is dropped, not rejected, so
241+
an edit form that submits every field still saves, and a present `null` no longer clears it. A credential is
242+
also never returned: not by list, get, or the echo of a write, or by a served report that groups by
243+
it or takes its `min` or `max` (a report column is derived from its `@of` field, and it keeps the
244+
credential mark; a report export names no column for it, and that column is not filterable or sortable). Before this, `@formExclude` kept
245+
a field out of the generated form while the route still wrote it, and a generated read returned password
246+
hashes. Generated forms (React, Angular) and the api docs omit credentials too. Your own code is unaffected:
247+
the entity, DTO and generated queries still carry the field, so you keep writing a hash or token yourself.
248+
A project with neither mark generates byte-identical output. If you relied on a route writing a
249+
`@formExclude` field, write it from your own code. A field that is `@required` and also
250+
`@formExclude` or a credential now makes the generated create route fail, because the route
251+
can no longer supply its value: make the field optional, or create that entity from your own
252+
code (see "Credentials and excluded fields" in `docs/features/api-contract.md`). Gated by the new `secret-fields/` sub-corpus
253+
of the api-contract conformance corpus, in the generated lane of all five ports.
254+
255+
- **`meta gen` and `meta verify --codegen` refuse an unknown codegen `dialect`** instead of silently
256+
generating Postgres code. `dialect: "d1"` gets a message that says to use `"sqlite"` (D1 is
257+
SQLite; `"d1"` names only the `meta migrate` transport). `meta init --d1` now scaffolds
258+
`dialect: "sqlite"` in `metaobjects.config.ts`; it used to write `"d1"`. If your config has `dialect: "d1"`,
259+
change it to `"sqlite"`.
260+
238261
- **The "Model / metadata" link in the Python, C#, Java and Kotlin API docs no longer points at
239262
a page that does not exist.** Those ports write no model pages, so the relative link was dead.
240263
It is now written only when `modelBaseUrl` is set.
@@ -280,6 +303,12 @@ here.**
280303

281304
### Documentation
282305

306+
- **New recipes and a real D1 example.** `docs/recipes/` gains keeping drizzle-kit migrations
307+
(`drizzle-kit-migrations.md`), guarding generated routes with your own auth
308+
(`guarding-generated-routes.md`) and proving your `meta verify` gate can fail
309+
(`prove-your-gate-fails.md`). The Cloudflare Workers recipe now has a per-request `drizzle(env.DB)`
310+
example, and the `drizzle({} as never)` placeholder and the wrong `"d1"` codegen-dialect statements are corrected.
311+
283312
- **Two doc errors corrected against the code.** `docs/recipes/mysql.md` said Python ships a
284313
Postgres driver only; it ships SQLite and MySQL drivers beside it. `docs/features/api-contract.md`
285314
said only TypeScript enforces the `in`-list cap of 100; every port does.

‎agent-context/skills/metaobjects-audit/references/capability-checklist.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -198,8 +198,10 @@ soft-delete / status / type view it models.
198198
and are NOT in the cross-port registry — `view.text`, `view.textarea`, `view.date`,
199199
`view.month`, `view.hotlink`, `view.image`, `view.dropdown`, `view.radio`, `view.checkbox`,
200200
`view.number`, `view.password`, `view.hidden`, `view.web`. Every port LOADS them, so shared metadata that
201-
carries them is valid in a backend port; nothing outside TS/web reads them. **Audit these
202-
only for TS adopters.** Only
201+
carries them is valid in a backend port. The one exception is `view.password`: every port's generated
202+
routes read it as a credential, so it is never returned and never written (see
203+
"Credentials and excluded fields" in `docs/features/api-contract.md`).
204+
Audit the other widgets only for TS adopters. Only
203205
`view.base` / `view.currency` are cross-port-gated.
204206

205207
## Template — `template.*` (prompt pillar)

‎agent-context/skills/metaobjects-audit/references/typescript.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -199,4 +199,4 @@ TypeScript is the reference implementation — it ships the full feature set:
199199
- Output-parser codegen: **complete** (`outputParser()` generator; `parse*`/`safeParse*`/`extract*` emitted as `<Name>.response.ts` per **responding `template.prompt`** — one carrying `@responseRef`; ADR-0052).
200200
- ObjectManager: **complete** (`@metaobjectsdev/runtime-ts`; `kyselyDriver` / `inMemoryDriver`).
201201
- Schema migrate / `meta verify --db`: **complete** (TS owns the shared migrate engine — ADR-0015).
202-
- D1 dialect: **TS-only by design** — `dialect: "d1"` in `metaobjects.config.ts` targets Cloudflare D1 via the Wrangler CLI layout. Do NOT recommend porting D1 support to other languages.
202+
- D1 dialect: **TS-only by design** — `migrate.dialect: "d1"` (and `meta migrate --dialect d1`) targets Cloudflare D1 via the Wrangler CLI layout; codegen's `dialect` in `metaobjects.config.ts` is `"sqlite"` for D1 and `"d1"` there is refused. Do NOT recommend porting D1 support to other languages.

‎docs/CONFORMANCE.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
3636
| [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ |
3737
| [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ |
3838
| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) |
39-
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 117 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
39+
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 119 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
4040
| [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) |
4141
| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) |
4242
| [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ |
@@ -365,9 +365,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc
365365
`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the
366366
write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates).
367367

368-
### `fixtures/api-contract-conformance/` (117)
368+
### `fixtures/api-contract-conformance/` (119)
369369

370-
All 117 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
370+
All 119 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
371371
REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD
372372
routes answer identically over HTTP — list / get / create / patch+put / delete,
373373
plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1`
@@ -378,11 +378,11 @@ The corpus also covers the 9 cross-port filter operators (`eq`, `ne`, `gt`,
378378
`gte`, `lt`, `lte`, `in`, `like`, `isNull`) plus the implicit-AND combinator
379379
and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the
380380
URL grammar `?filter[<field>][<op>]=<value>` (FR-009). On top of the 31 core
381-
scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table
381+
scenarios the corpus carries eight sub-corpora — `tph/` (10, single-table
382382
inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination
383383
each corpus alone could not reach, and 1 pinning the collection-URL spelling),
384384
`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
385-
`write-through/` (2, table-write + view-read entities), `projection/` (12, a
385+
`write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, and a credential is never returned), `projection/` (12, a
386386
read-only view answers reads and refuses writes with 405; a projection with no declared
387387
identity has no item route; a projection keyed on a field not named `id` is addressed by
388388
it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one
@@ -392,7 +392,7 @@ its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` re
392392
a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java,
393393
Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and
394394
the port's own GENERATED API artifact booted over HTTP. `write-through/`,
395-
`projection/`, `decimal/` and `report/` run the generated lane only, on all five ports: what
395+
`secret-fields/`, `projection/`, `decimal/` and `report/` run the generated lane only, on all five ports: what
396396
they test is whether a port's generator emits the routes. The `report/` sub-corpus is the one
397397
that reads real views, so it runs full stack on Postgres, SQLite and MySQL
398398
(`report/schema.sqlite.sql`, `report/schema.mysql.sql`, both TypeScript-produced) in every
@@ -606,7 +606,7 @@ files into a real Cube.
606606
## Orphaned fixtures (tested but not yet documented)
607607

608608
The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
609-
+ render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 +
609+
+ render 16 + persistence 59 + api-contract 119 + source-resolution 25 + scope 10 +
610610
dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the
611611
`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining
612612
corpora in the totals table gate tooling contracts (registry manifests, provider

‎docs/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ this tree is documentation, not the source of truth.
8484
| See a worked, non-toy model end to end (projections, value objects, TPH, prompt payloads on one runnable spine) | [`../examples/advanced-modeling/`](../examples/advanced-modeling/) |
8585
| Read the canonical spec (target-agnostic) | [`../spec/`](../spec/) |
8686
| Find a release recipe (Cloudflare D1, etc.) | [`recipes/`](recipes/) |
87+
| Keep drizzle-kit migrations while MetaObjects owns the model | [`recipes/drizzle-kit-migrations.md`](recipes/drizzle-kit-migrations.md) |
88+
| Put generated routes behind your own auth (Fastify, Hono, `expose`) | [`recipes/guarding-generated-routes.md`](recipes/guarding-generated-routes.md) |
89+
| Show that your `meta verify` CI gate actually fails on drift | [`recipes/prove-your-gate-fails.md`](recipes/prove-your-gate-fails.md) |
8790
| See the deeper design rationale for a feature | [`superpowers/specs/`](superpowers/specs/) |
8891

8992
## Conventions used in these docs

‎docs/features/api-contract.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -529,6 +529,34 @@ log-only.
529529
| Kotlin | shipped — `metaobjects-codegen-kotlin` `KotlinSpringControllerGenerator` → Spring `@RestController` | One controller per writable entity (`source.rdb @kind="table"`) plus a READ-ONLY one per view-kind `object.projection` (F22, see "Read-only projections" below); 5 CRUD endpoints (GET list / GET by id / POST / PATCH+PUT / DELETE); `?sort`, `?limit/?offset`, `?withCount=1` envelope, 404 + 400 envelopes per the contract. Filter operators ship via the generated `<Entity>FilterAllowlist` (`KotlinFilterAllowlistGenerator`) + an inline `parse<Entity>Filter` helper emitted in the controller. |
530530
| Python | shipped — `metaobjects.codegen.generators.router_generator` → FastAPI `APIRouter` | One router per writable entity (`source.rdb @kind="table"`) plus a READ-ONLY one per view-kind `object.projection` (F22, see "Read-only projections" below); 5 CRUD endpoints (GET list / GET by id / POST / PATCH+PUT / DELETE); `?sort`, `?limit/?offset`, `?withCount=1` envelope, 404 + 400 envelopes per the contract. Consumer wires the repository via FastAPI `app.dependency_overrides`; the generator emits a `Protocol` interface so the persistence layer (SQLAlchemy / asyncpg / etc.) is the consumer's choice. Filter operators ship via the generated `<entity>_filter_allowlist.py` (`filter_allowlist_generator.py`) + the shared `filter_parser` helper, wired into the list handler. |
531531

532+
## Credentials and excluded fields
533+
534+
Two marks narrow what the generated routes accept and send, in every port (#484):
535+
536+
| Mark | Create, PATCH, PUT | List, get, and the echo of a write |
537+
|---|---|---|
538+
| `@formExclude: true` | the field is **ignored** | returned |
539+
| a child `view.password` (a credential) | the field is **ignored** | **never returned** (the key is absent, not `null`) |
540+
541+
A value for an ignored field is dropped, never rejected: the request answers `200` or `201` as if the key
542+
were absent, and a present `null` does not clear it. That is the rule for every key outside the settable
543+
set (`@mutability: writeOnce` on PATCH, the TPH discriminator, `@autoSet: onCreate`), and it is load-bearing:
544+
a generated edit form submits every field it was given, so a `400` would fail every save. A required field
545+
that is also ignored can never be created through the API, so mark such a field optional.
546+
547+
Only the generated HTTP boundary changes. The entity, its DTO and the generated queries still carry the
548+
field, so your own code writes a hash or token and reads it back. Allowlists are unchanged: a field is
549+
filterable or sortable only if you wrote `@filterable` or `@sortable` on it, so do not put those on a credential
550+
(a filter on a secret answers whether a guess is right). To return a credential from a route, hand-write that
551+
route; the generated one will not. Generated forms and the api docs omit both kinds of field as well.
552+
553+
A served report is covered too. A report column is derived from its `@of` field, and a derived column keeps
554+
that field's credential mark, so a report that groups by a credential (or takes its `min` or `max`) never
555+
returns the column, its `report-export` output has no column for it, and it is neither filterable nor sortable
556+
(a report's other columns all are, so this one is taken out). `count` over a credential is a number and is returned.
557+
558+
`fixtures/api-contract-conformance/secret-fields/` pins this in the generated lane of every port.
559+
532560
## Read-only projections
533561

534562
**All five ports serve an `object.projection` whose only source is a read-only

‎docs/features/reporting.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -299,7 +299,9 @@ A derived column carries the type-shaping attributes of its `@of` field where th
299299
(`@currency`, `@values`, `@intValueMap`, `@maxLength`, `@precision`, `@scale`, `@localTime`,
300300
`@objectRef`, `@storage`, `@dbColumnType`, `isArray`) and nothing else: no `@column`, no
301301
`@required` beyond the rules above, no `@default` (a measure's `@default` is applied in the
302-
view, not carried), no validators.
302+
view, not carried), no validators. The one view it does carry is a credential: a column whose
303+
`@of` field declares or inherits `view.password` is itself a credential, so the served list route
304+
omits it, an export has no column for it, and it is not filterable or sortable (#484).
303305

304306
### Measures
305307

0 commit comments

Comments
 (0)