From 783ac219e2159129a452d477e47d5ed20c61d7cc Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 10:17:23 -0500 Subject: [PATCH 01/31] fix(python): pass only validated input to the repository in generated routes (#566) * fix(python): generated routes hand the repository only the validated input (1.2.1) The generated FastAPI create, PATCH and PUT handlers validated the body against Create / Patch and then passed the RAW body to the repository. Pydantic ignores undeclared keys and the input models leave out server-owned fields, so a client-sent id, a readOnly field and a made-up field such as isAdmin all reached the write. Security fix. Each handler (plain and TPH per-subtype) now rebinds dto to the keys the input model validated, at every depth, before any repository call, as the TypeScript routes do. Undeclared and non-settable keys are dropped silently, never 400'd, so a client sending an extra harmless field keeps working; values stay the body's own, so repositories see the same value types as in 1.2.0. Python package version 1.2.1 only; CHANGELOG entry under Security. Regression gate: tests/integration/test_generated_router_validated_input.py. * no-mistakes(document): python routes pass only validated input * no-mistakes(lint): regenerate showcase site payload for PyPI 1.2.1 --- CHANGELOG.md | 28 ++ docs/features/generated-mutations.md | 12 +- .../.metaobjects/.gen-state/.hashes.json | 2 +- .../generated/python/subscriber_router.py | 32 ++- examples/showcase/site-payload.json | 2 +- server/python/pyproject.toml | 2 +- .../src/metaobjects/codegen/KNOWN_GAPS.md | 6 +- .../codegen/generators/router_generator.py | 98 +++++-- .../test_generated_router_validated_input.py | 272 ++++++++++++++++++ server/python/uv.lock | 2 +- 10 files changed, 414 insertions(+), 42 deletions(-) create mode 100644 server/python/tests/integration/test_generated_router_validated_input.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 79042d8184..9fe7ecc774 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,34 @@ here.** ## [Unreleased] +## [1.2.1] — Python only + +_PyPI `1.2.1` only. npm, NuGet and Maven Central stay at `1.2.0` (`8.2.0`): none of them has a +product change. No vocabulary or `metamodelVersion` change._ + +### Security + +- **The generated Python routes passed raw request input to the repository.** The FastAPI + create, PATCH and PUT handlers validated the body against `Create` / `Patch` + and then handed the repository the raw body, not what was validated. Pydantic ignores a key a + model does not declare, so a client could write fields it must never touch: a client-sent `id` + on create (or on update, changing the row's identity), a `@mutability: readOnly` field, and any + made-up field such as `isAdmin` all reached the repository. Found by an external review of + 1.2.0. Every Python adopter serving the generated routes should upgrade and regenerate; an + ejected (owned) router needs the same change made by hand. + + The repository now receives only the validated input, as the TypeScript routes already do. + Each handler validates the body, then keeps only the keys the input model validated, at every + depth (a nested value object included), before any repository call, in the plain and the TPH + per-subtype handlers. A field the entity does not declare, a server-generated id, a `readOnly` + or derived field on create, and the id, `readOnly` and `writeOnce` fields on update are all + dropped silently. None of them is answered with a 400, so a client that sends an extra harmless + field keeps working; a malformed value on a field the client may set is still + `400 {"error": "validation"}`. An assigned key is still the caller's on create. Values are the + body's own, so a repository receives the same value types as in 1.2.0. + + Gated by `server/python/tests/integration/test_generated_router_validated_input.py`. + ## [1.2.0] — 2026-10-30 _npm `1.2.0` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates)._ diff --git a/docs/features/generated-mutations.md b/docs/features/generated-mutations.md index 57aa6f409a..3a367bbb6f 100644 --- a/docs/features/generated-mutations.md +++ b/docs/features/generated-mutations.md @@ -41,7 +41,9 @@ but the semantics are one contract — call them **PATCH-1..7** (FR-035): simply omits every non-settable field, and Zod drops unknown keys — so `safeParse` succeeds and the key never reaches the `SET`. The TPH discriminator is additionally deleted by hand after parse, the same convention. A malformed - *value* on a settable key is still `400 {"error":"validation"}`. + *value* on a settable key is still `400 {"error":"validation"}`. A key the entity + does not declare at all is dropped the same way. The Python routes hand the + repository only the keys their input model validated. This matters for `writeOnce` specifically: the generated edit form submits EVERY registered field (`handleSubmit` passes all values, and 0.19.2 switched the resolver to `UpdateSchema` on edit rather than diff-and-omit), so rejecting a @@ -109,10 +111,10 @@ repo.patch(id) { it[Authors.bio] = "…" } // touches only what you set the controller binds the partial JSON body via `Patch.fromJson` (presence-tracked) and delegates to `repository.patch`, so an omitted key is never written. -**Python** — the generated FastAPI route binds the raw JSON body (the set of -*present* keys — the tristate signal) and validates the present values against the -generated `Patch` model before delegating to the runtime `update`, which -`SET`s only those keys. +**Python** — the generated FastAPI route binds the JSON body (the set of +*present* keys — the tristate signal), validates the present values against the +generated `Patch` model, and hands the repository only the keys that model +validated. The runtime `update` then `SET`s only those keys. **C#** — the generated route runs a per-field merge loop over the present JSON keys (the app's configured `JsonSerializerOptions`), writing only the columns the body diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index 8a7a4f6bb9..131a0265b9 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -10,7 +10,7 @@ "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", "generated/python/subscriber_filter_allowlist.py": "b0414cc95ff3982c80a0e1a06839f090c0b423e41f5a94275df93426cea09df5", "generated/python/subscriber_names.py": "269637a02bf9b46f55658b58cb42f83196c03a877e1e2101337847fd406c1561", - "generated/python/subscriber_router.py": "5a354e50b1aab48239a4249b670a6d2351c1816cbca6d710ea688f252e9612e5", + "generated/python/subscriber_router.py": "f3c81be3c07a10a94afb6a6a0f87eb84b9bca40ead03dd989ebd019f71b97dfc", "generated/ts/Subscriber.queries.ts": "9f1a459c57fafc98bf074b4443b506d4a1e7023ed188f293021ae89c298d4562", "generated/ts/Subscriber.routes.ts": "147d32ae658f985e024c3de6e8894ca7359f767eeaa765cec70756802e3bd300", "generated/ts/Subscriber.ts": "a78c17f915222ee108290c28468313d77f77b5cff438f485324409302e389a96", diff --git a/examples/showcase/generated/python/subscriber_router.py b/examples/showcase/generated/python/subscriber_router.py index 47aaaab7ce..fe148c5e3e 100644 --- a/examples/showcase/generated/python/subscriber_router.py +++ b/examples/showcase/generated/python/subscriber_router.py @@ -91,6 +91,26 @@ async def _json_body(request: Request) -> dict[str, Any] | None: return None +def _validated_input(valid: BaseModel, raw: dict[str, Any]) -> dict[str, Any]: + """GENERATED — *raw* cut down to the keys *valid* validated, at every depth. A key the + input model does not declare (an undeclared name, a server-generated id, a readOnly field) + is dropped silently and never reaches the repository; values stay the body's own.""" + return { + _k: _validated_value(getattr(valid, _k), raw[_k]) + for _k in valid.model_fields_set + } + + +def _validated_value(valid: Any, raw: Any) -> Any: + if isinstance(valid, BaseModel) and isinstance(raw, dict): + return _validated_input(valid, raw) + if isinstance(valid, list) and isinstance(raw, list): + return [_validated_value(v, r) for v, r in zip(valid, raw)] + if isinstance(valid, dict) and isinstance(raw, dict): + return {k: _validated_value(v, raw[k]) for k, v in valid.items() if k in raw} + return raw + + _FROZEN_FIELDS: frozenset[str] = frozenset() @@ -198,9 +218,13 @@ def create_subscriber( # FR-036: run the generated field constraints (length / pattern / # numeric bounds) before persisting; a violation is the cross-port 400. try: - SubscriberCreate(**dto) + _valid = SubscriberCreate(**dto) except ValidationError: return JSONResponse(status_code=400, content={"error": "validation"}) + # SECURITY (1.2.1): only what the input model validated reaches the repository. A key + # it does not declare (an undeclared name, a server-generated id, a readOnly or derived + # field, a frozen one on update) is dropped silently, never 400'd. + dto = _validated_input(_valid, dto) # #203/ADR-0045: stamp @autoSet columns (server-owned; caller ignored). _asnow = _dt.datetime.now(_dt.timezone.utc) dto["createdAt"] = _asnow @@ -239,9 +263,13 @@ def update_subscriber( # FR-036: validate PRESENT, non-null values with the create rules (the # all-optional patch model never fires required-checks on ABSENT fields). try: - SubscriberPatch(**dto) + _valid = SubscriberPatch(**dto) except ValidationError: return JSONResponse(status_code=400, content={"error": "validation"}) + # SECURITY (1.2.1): only what the input model validated reaches the repository. A key + # it does not declare (an undeclared name, a server-generated id, a readOnly or derived + # field, a frozen one on update) is dropped silently, never 400'd. + dto = _validated_input(_valid, dto) dto.pop("createdAt", None) # onCreate @autoSet is write-once (server-owned) try: saved = repo.update(subscriber_id, dto) diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index cc09d97e22..237e16a881 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -1,7 +1,7 @@ { "registries": { "npm": "1.2.0", - "pypi": "1.2.0", + "pypi": "1.2.1", "nuget": "1.2.0", "maven": "8.2.0", "metamodel": "1.2" diff --git a/server/python/pyproject.toml b/server/python/pyproject.toml index 2854f66cad..2366d3f4bc 100644 --- a/server/python/pyproject.toml +++ b/server/python/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "metaobjects" -version = "1.2.0" +version = "1.2.1" description = "Cross-language metadata standard: declare typed entities once, generate idiomatic drift-checked code across languages — Python port." readme = "README.md" requires-python = ">=3.11" diff --git a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md index 2cb454b8d7..1b300bfa6f 100644 --- a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md +++ b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md @@ -36,9 +36,9 @@ The generated router takes `dto: dict[str, Any]` for `POST` / `PATCH` / (`from . import , Patch`) and constructs them to enforce field constraints over HTTP — POST validates `(**dto)`, PATCH validates the all-optional `Patch(**dto)` (present values only) → `400 {"error":"validation"}`. -The repository seam still receives the original `dict` (preserving the FR-035 -present-key tristate + wire values), so the strong-typing simplification below -remains for the repo/response types; only the validation import was added. +The repository seam receives only the keys the input model validated, as a `dict` +(the FR-035 present-key tristate + wire values are kept), so the strong-typing +simplification below remains for the repo/response types. This keeps the router module decoupled from sibling generated files (the entity-model generator emits one `@dataclass`/Pydantic model per diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 227346832c..9e0586bcdb 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -434,10 +434,9 @@ def _frozen_names(entity: MetaObject) -> list[str]: ADR-0045: the OUTERMOST generated write artifact enforces the mode, and for FastAPI that is the ROUTER, not the ``Patch`` model — the handler binds - ``dto`` (the raw body as a dict) and passes THAT to the repository, so a - field merely absent from the patch model still reaches the row. @autoSet gets - away with it because the router OVERWRITES its key server-side; a writeOnce - field has no server value to overwrite, so it must be popped explicitly.""" + ``dto`` as a dict. Since 1.2.1 the router hands the repository only the keys the + patch model validated (:func:`_validated_input_lines`), which already drops these; + the explicit pop stays as the FR-037 seam an owned router edits.""" return [ f.name for f in entity.fields() @@ -461,6 +460,49 @@ def _frozen_strip_lines(entity: MetaObject, indent: str = " ") -> list[str]: ] +#: 1.2.1 security fix. The request body cut down to what the input model validated, emitted +#: once per writable router. 1.2.0 validated the body and then handed the RAW body to the +#: repository, so a client-sent server-generated id, a ``readOnly`` field or a name the entity +#: does not declare (an ``isAdmin``) reached the write. Pydantic ignores a key a model does not +#: declare, and the input models leave out every field a client may not set, so keeping only +#: the keys the model validated (at every depth: a nested value object included) drops all of +#: them silently, as the TypeScript routes do. The values stay the body's own (decimal-exact) +#: values, so a repository receives the same value types as in 1.2.0. +_VALIDATED_INPUT_LINES: list[str] = [ + "def _validated_input(valid: BaseModel, raw: dict[str, Any]) -> dict[str, Any]:", + ' """GENERATED — *raw* cut down to the keys *valid* validated, at every depth. A key the', + " input model does not declare (an undeclared name, a server-generated id, a readOnly field)", + ' is dropped silently and never reaches the repository; values stay the body\'s own."""', + " return {_k: _validated_value(getattr(valid, _k), raw[_k]) for _k in valid.model_fields_set}", + "", + "", + "def _validated_value(valid: Any, raw: Any) -> Any:", + " if isinstance(valid, BaseModel) and isinstance(raw, dict):", + " return _validated_input(valid, raw)", + " if isinstance(valid, list) and isinstance(raw, list):", + " return [_validated_value(v, r) for v, r in zip(valid, raw)]", + " if isinstance(valid, dict) and isinstance(raw, dict):", + " return {k: _validated_value(v, raw[k]) for k, v in valid.items() if k in raw}", + " return raw", +] + + +def _validated_input_lines(model: str, *comment: str) -> list[str]: + """Validate ``dto`` against the input *model*, then rebind it to what the model validated + (see :data:`_VALIDATED_INPUT_LINES`): the repository never sees any other key.""" + return [ + *(f" # {line}" for line in comment), + " try:", + f" _valid = {model}(**dto)", + " except ValidationError:", + ' return JSONResponse(status_code=400, content={"error": "validation"})', + " # SECURITY (1.2.1): only what the input model validated reaches the repository. A key", + " # it does not declare (an undeclared name, a server-generated id, a readOnly or derived", + " # field, a frozen one on update) is dropped silently, never 400'd.", + " dto = _validated_input(_valid, dto)", + ] + + def _auto_set_split(entity: MetaObject) -> tuple[list[MetaField], list[MetaField]]: """The entity's ``@autoSet`` temporal fields split into ``(onCreate, onUpdate)``. @@ -1083,12 +1125,11 @@ def _emit_route_handler( ") -> Any:", *_DECIMAL_COERCE, *denied_strip, - " # FR-036: run the generated field constraints (length / pattern /", - " # numeric bounds) before persisting; a violation is the cross-port 400.", - " try:", - f" {model_name}(**dto)", - " except ValidationError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + *_validated_input_lines( + model_name, + "FR-036: run the generated field constraints (length / pattern /", + "numeric bounds) before persisting; a violation is the cross-port 400.", + ), # #203/ADR-0045: the generated router (the API surface) stamps @autoSet columns # ABOVE the consumer repo seam — onCreate + onUpdate both stamped from one instant # (a fresh row's createdAt == updatedAt), ignoring any caller-supplied value. @@ -1114,12 +1155,11 @@ def _emit_route_handler( " for _k in _REQUIRED_FIELDS:", " if _k in dto and dto[_k] is None:", ' return JSONResponse(status_code=400, content={"error": "validation"})', - " # FR-036: validate PRESENT, non-null values with the create rules (the", - " # all-optional patch model never fires required-checks on ABSENT fields).", - " try:", - f" {patch_model}(**dto)", - " except ValidationError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + *_validated_input_lines( + patch_model, + "FR-036: validate PRESENT, non-null values with the create rules (the", + "all-optional patch model never fires required-checks on ABSENT fields).", + ), # #203/ADR-0045: bump onUpdate @autoSet columns (server-owned) on EVERY patch, # above the repo seam — a server-inserted present key (compatible with the PATCH # present-key tristate); onCreate columns are never touched here. @@ -1468,6 +1508,9 @@ def _render_tph_router( parts.extend(_JSON_BODY_LINES) parts.append("") parts.append("") + parts.extend(_VALIDATED_INPUT_LINES) + parts.append("") + parts.append("") # FR-037 R1 — the names a PATCH must never write. Emitted even when EMPTY so # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). @@ -1547,11 +1590,9 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(") -> Any:") parts.extend(_DECIMAL_COERCE) parts.extend(denied_strip_tph) - parts.append(" # FR-036: run the subtype's field constraints before persisting.") - parts.append(" try:") - parts.append(f" {sub}Create(**dto)") - parts.append(" except ValidationError:") - parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') + parts.extend(_validated_input_lines( + f"{sub}Create", "FR-036: run the subtype's field constraints before persisting." + )) # #203/ADR-0045: stamp @autoSet columns ABOVE the repo seam (empty for a # non-@autoSet hierarchy — byte-identical output). parts.extend(create_autoset) @@ -1584,11 +1625,9 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(" for _k in _REQUIRED_FIELDS:") parts.append(" if _k in dto and dto[_k] is None:") parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') - parts.append(" # FR-036: validate PRESENT, non-null values with the subtype's create rules.") - parts.append(" try:") - parts.append(f" {sub}Patch(**dto)") - parts.append(" except ValidationError:") - parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') + parts.extend(_validated_input_lines( + f"{sub}Patch", "FR-036: validate PRESENT, non-null values with the subtype's create rules." + )) # #203/ADR-0045: pop the write-once onCreate column(s), bump onUpdate — ABOVE # the repo seam (empty for a non-@autoSet hierarchy — byte-identical output). parts.extend(update_autoset) @@ -1857,6 +1896,9 @@ def render_router( parts.extend(_JSON_BODY_LINES) parts.append("") parts.append("") + parts.extend(_VALIDATED_INPUT_LINES) + parts.append("") + parts.append("") # FR-037 R1 — the names a PATCH must never write. Emitted even when EMPTY so # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). @@ -1892,8 +1934,8 @@ def render_router( create_autoset=create_autoset, update_autoset=update_autoset, # FR-037 R1 — ADR-0045: the ROUTER is the outermost write artifact for - # FastAPI (the handler binds the RAW body dict and hands THAT to the - # repository), so the strip has to live here, not in Patch. + # FastAPI (the handler binds the body as a dict), so the strip lives here, + # not in Patch. frozen_strip=_frozen_strip_lines(entity), denied_strip=_denied_strip_lines(denied_names), ) diff --git a/server/python/tests/integration/test_generated_router_validated_input.py b/server/python/tests/integration/test_generated_router_validated_input.py new file mode 100644 index 0000000000..b43ed21c08 --- /dev/null +++ b/server/python/tests/integration/test_generated_router_validated_input.py @@ -0,0 +1,272 @@ +"""1.2.1 security fix: a generated create / update hands the repository ONLY the validated input. + +The 1.2.0 router validated the request body against ``Create`` / ``Patch`` and +then passed the RAW body to the repository. Pydantic ignores a key a model does not declare, and +the input models leave out the server-owned fields, so a client-sent ``id``, a ``readOnly`` field +and a made-up ``isAdmin`` all reached the write. Now the repository receives only what the input +model validated, at every depth, as with the TypeScript routes: a name the entity does not +declare, a server-generated id, a ``readOnly`` field and the id on update are all dropped +silently. None of them is a 400: a client that sends an extra harmless field keeps working. + +The model is generated with the real Python codegen (``run_gen``: entity + filter-allowlist + +routes), the emitted package is imported unmodified, and the only hand-written piece is a +repository that records what it was handed. No database: the router is the artifact, as in the +other generated-router lanes. +""" +from __future__ import annotations + +import importlib +import json +import sys +import tempfile +import uuid +from pathlib import Path +from typing import Any + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient + +from metaobjects import MetaDataLoader +from metaobjects.codegen.config import GenConfig +from metaobjects.codegen.generators.entity_model import entity_model +from metaobjects.codegen.generators.filter_allowlist_generator import filter_allowlist_generator +from metaobjects.codegen.generators.router_generator import router_generator +from metaobjects.codegen.runner import run_gen + +from .generated_tph_app import build_generated_tph_app + +_META = { + "metadata.root": { + "package": "acme::secfix", + "children": [ + {"object.value": { + "name": "Address", + "children": [ + {"field.string": {"name": "city", "@maxLength": 40}}, + ], + }}, + {"object.entity": { + "name": "Member", + "children": [ + {"source.rdb": {"@table": "members"}}, + {"field.long": {"name": "id"}}, + {"field.string": {"name": "name", "@required": True, "@maxLength": 80}}, + {"field.string": {"name": "role", "@maxLength": 20, "@mutability": "readOnly"}}, + {"field.string": {"name": "tenant", "@maxLength": 20, "@mutability": "writeOnce"}}, + {"field.object": {"name": "address", "@objectRef": "Address", "@storage": "jsonb"}}, + {"field.timestamp": {"name": "createdAt", "@autoSet": "onCreate"}}, + {"identity.primary": {"name": "id", "@fields": "id", "@generation": "increment"}}, + ], + }}, + {"object.entity": { + "name": "Account", + "children": [ + {"source.rdb": {"@table": "accounts"}}, + {"field.string": {"name": "code", "@maxLength": 20}}, + {"field.string": {"name": "label", "@maxLength": 80}}, + {"identity.primary": {"name": "code", "@fields": "code"}}, + ], + }}, + ], + } +} + + +class _RecordingRepo: + """The generated repository seam, recording every write it is handed.""" + + def __init__(self, pk: str) -> None: + self.pk = pk + self.writes: list[tuple[str, dict[str, Any]]] = [] + + def create(self, dto: Any) -> Any: + self.writes.append(("create", dict(dto))) + return {self.pk: 1, **dto} + + def update(self, id: Any, dto: Any) -> Any | None: + self.writes.append(("update", dict(dto))) + return {self.pk: id, **dto} + + +def _generate_package() -> str: + """Run the real Python codegen for ``_META`` into a temp package; return its name.""" + meta_dir = Path(tempfile.mkdtemp(prefix="secfix-meta-")) + (meta_dir / "meta.json").write_text(json.dumps(_META)) + loaded = MetaDataLoader.from_directory(str(meta_dir)) + assert not loaded.errors, "; ".join(f"{e.code}: {e.message}" for e in loaded.errors) + pkg = f"secfix_{uuid.uuid4().hex[:8]}" + root = Path(tempfile.mkdtemp(prefix="secfix-gen-")) + run_gen( + GenConfig(out_dir=str(root / pkg)), + loaded.root, + generators=[entity_model(), filter_allowlist_generator(), router_generator()], + ) + sys.path.insert(0, str(root)) + return pkg + + +_PKG = _generate_package() + + +def _client(router_module: str, pk: str) -> tuple[TestClient, _RecordingRepo, str]: + mod = importlib.import_module(f"{_PKG}.{router_module}") + repo = _RecordingRepo(pk) + app = FastAPI() + app.include_router(mod.router) + app.dependency_overrides[mod.get_repository] = lambda: repo + return TestClient(app), repo, mod.router.prefix + + +@pytest.fixture() +def member() -> tuple[TestClient, _RecordingRepo, str]: + return _client("member_router", "id") + + +@pytest.fixture() +def account() -> tuple[TestClient, _RecordingRepo, str]: + return _client("account_router", "code") + + +# --- the reported defect: a client-sent id and a made-up isAdmin reach the repository ------- + + +def test_create_never_hands_the_repository_a_client_id_or_an_undeclared_field(member) -> None: + client, repo, base = member + resp = client.post(base, json={"id": 999, "name": "Mallory", "isAdmin": True}) + assert resp.status_code == 201, resp.text + [(op, written)] = repo.writes + assert op == "create" + assert set(written) == {"name", "createdAt"} # createdAt is the router's own stamp + assert written["name"] == "Mallory" + + +@pytest.mark.parametrize("method", ["PATCH", "PUT"]) +def test_update_never_hands_the_repository_an_undeclared_field(member, method: str) -> None: + client, repo, base = member + resp = client.request(method, f"{base}/1", json={"name": "M", "isAdmin": True}) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"name": "M"})] + + +def test_an_undeclared_key_inside_a_value_object_never_reaches_the_repository(member) -> None: + client, repo, base = member + resp = client.post(base, json={"name": "Mallory", "address": {"city": "Oslo", "isAdmin": True}}) + assert resp.status_code == 201, resp.text + [(_op, written)] = repo.writes + assert written["address"] == {"city": "Oslo"} + + +def test_a_malformed_value_on_a_settable_field_is_still_a_400(member) -> None: + client, repo, base = member + resp = client.post(base, json={"name": "x" * 81, "isAdmin": True}) + assert resp.status_code == 400, resp.text + assert resp.json() == {"error": "validation"} + assert repo.writes == [] + + +# --- declared but not settable: dropped, never 400'd, and never handed to the repository ---- + + +def test_create_strips_the_client_id_and_a_read_only_field(member) -> None: + client, repo, base = member + resp = client.post( + base, + json={"id": 999, "name": "Mallory", "role": "admin", "tenant": "t1", + "address": {"city": "Oslo"}}, + ) + assert resp.status_code == 201, resp.text + [(op, written)] = repo.writes + assert op == "create" + # The server generates the id and owns `role`; `createdAt` is the router's own stamp. + assert set(written) == {"name", "tenant", "address", "createdAt"} + assert written["name"] == "Mallory" + assert written["tenant"] == "t1" # writeOnce is settable on create + assert written["address"] == {"city": "Oslo"} + + +@pytest.mark.parametrize("method", ["PATCH", "PUT"]) +def test_update_never_hands_the_repository_the_id_or_a_frozen_field(member, method: str) -> None: + client, repo, base = member + resp = client.request( + method, + f"{base}/1", + json={"id": 999, "name": "M2", "role": "admin", "tenant": "t2", "createdAt": "2000-01-01T00:00:00Z"}, + ) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"name": "M2"})] + + +def test_update_passes_a_present_null_through_as_a_clear(member) -> None: + """The PATCH present-key tristate is unchanged: an explicit null on a nullable field clears it.""" + client, repo, base = member + resp = client.patch(f"{base}/1", json={"address": None}) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"address": None})] + + +def test_an_assigned_key_is_still_the_callers_on_create_and_route_owned_on_update(account) -> None: + client, repo, base = account + created = client.post(base, json={"code": "ACME", "label": "Acme"}) + assert created.status_code == 201, created.text + updated = client.patch(f"{base}/ACME", json={"code": "EVIL", "label": "Acme 2"}) + assert updated.status_code == 200, updated.text + assert repo.writes == [ + ("create", {"code": "ACME", "label": "Acme"}), + ("update", {"label": "Acme 2"}), + ] + + +# --- TPH: the per-subtype create / update handlers are a separate code path ----------------- + + +def _find_tph_dir() -> Path: + cur = Path(__file__).resolve() + while cur != cur.parent: + candidate = cur / "fixtures" / "api-contract-conformance" / "tph" + if candidate.is_dir(): + return candidate + cur = cur.parent + raise RuntimeError("fixtures/api-contract-conformance/tph not found") + + +@pytest.fixture() +def tph() -> tuple[TestClient, list[tuple[str, str, dict[str, Any]]]]: + app, repo = build_generated_tph_app(_find_tph_dir()) + writes: list[tuple[str, str, dict[str, Any]]] = [] + create, update = repo.create, repo.update + + def recording_create(subtype: str, dto: Any) -> Any: + writes.append(("create", subtype, dict(dto))) + return create(subtype, dto) + + def recording_update(subtype: str, id: int, dto: Any) -> Any: + writes.append(("update", subtype, dict(dto))) + return update(subtype, id, dto) + + repo.create, repo.update = recording_create, recording_update + return TestClient(app), writes + + +def test_tph_subtype_create_drops_the_client_id_and_an_undeclared_field(tph) -> None: + client, writes = tph + resp = client.post( + "/api/auths/bridge", json={"id": 999, "reference": "R-1", "quantity": 2, "isAdmin": True} + ) + assert resp.status_code == 201, resp.text + [(op, subtype, written)] = writes + assert (op, subtype) == ("create", "Bridge") + assert "id" not in written + assert "isAdmin" not in written + assert {"reference", "quantity"} <= set(written) + + +def test_tph_subtype_update_drops_the_id_and_an_undeclared_field(tph) -> None: + client, writes = tph + # The store is unseeded, so the answer is a 404; what matters is what the write was handed. + client.patch("/api/auths/bridge/1", json={"id": 999, "quantity": 3, "isAdmin": True}) + [(op, subtype, written)] = writes + assert (op, subtype) == ("update", "Bridge") + assert "id" not in written + assert "isAdmin" not in written + assert written["quantity"] == 3 diff --git a/server/python/uv.lock b/server/python/uv.lock index 2999705817..a3b21881ca 100644 --- a/server/python/uv.lock +++ b/server/python/uv.lock @@ -399,7 +399,7 @@ wheels = [ [[package]] name = "metaobjects" -version = "1.2.0" +version = "1.2.1" source = { editable = "." } dependencies = [ { name = "pyyaml" }, From dbd3ba03fed3bc9c4c8644c08eb3f3e94e06ce54 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 11:51:35 -0500 Subject: [PATCH 02/31] fix(codegen): generated routes drop server-owned and undeclared input in every port An audit after the Python 1.2.1 fix found the same request-input hole in the other ports' generated create handlers. Each port now drops, silently and before the write, a server-generated key, readOnly and derived fields, and on a TPH per-subtype route the discriminator and sibling-subtype columns; undeclared keys are dropped everywhere. - Python: forward-port of the 1.2.1 fix; TPH create drops a body discriminator. - Java: create nulls the dropped DTO components; DTOs ignore unknown properties. - Kotlin: create writes only settable columns; entity classes ignore unknown keys. - C#: create resets the generated key, discriminator and M:N navigations; every readOnly property gets a private setter. - TypeScript: an assigned primary key leaves the update schema. Shared api-contract scenarios input-server-owned-dropped (core) and tph-input-server-owned-dropped, a readOnly column in secret-fields, and an `absent` assertion in every runner gate it. --- CHANGELOG.md | 37 +++ docs/features/api-contract.md | 23 ++ docs/features/generated-mutations.md | 12 +- .../codegen/templates/zod-validators.ts | 8 + .../.metaobjects/.gen-state/.hashes.json | 28 +- .../codegen/templates/zod-validators.ts | 8 + .../generated/csharp/SubscriberRoutes.g.cs | 2 + .../generated/java/acme/SubscriberDto.java | 1 + .../generated/kotlin/acme/Subscriber.kt | 2 + .../kotlin/acme/SubscriberBlurbPayload.kt | 2 + .../generated/python/subscriber_router.py | 32 ++- examples/showcase/site-payload.json | 12 +- fixtures/api-contract-conformance/README.md | 10 +- .../scenarios/input-server-owned-dropped.yaml | 86 ++++++ .../secret-fields/README.md | 5 +- .../secret-fields/meta.json | 1 + .../scenarios/read-omits-credential.yaml | 3 + .../scenarios/write-ignores-excluded.yaml | 15 +- .../secret-fields/seed.json | 4 +- .../api-contract-conformance/tph/README.md | 3 +- .../tph-input-server-owned-dropped.yaml | 64 +++++ .../m2m/expected/openapi.json | 77 +---- .../acme/social/AccountTag.update.schema.json | 11 +- .../acme/social/Follow.update.schema.json | 11 +- .../acme/social/Friendship.update.schema.json | 11 +- .../MemberAccountTag.update.schema.json | 11 +- .../social/PostReviewer.update.schema.json | 11 +- .../acme/social/PostTag.update.schema.json | 11 +- .../ScopedAccountTag.update.schema.json | 11 +- .../secret-fields/expected/openapi.json | 10 +- .../schemas/acme/secrets/Account.schema.json | 10 +- .../ServerOwnedCreateInputTests.cs | 134 +++++++++ .../Generators/EntityGenerator.cs | 26 +- .../Generators/RoutesGenerator.cs | 50 +++- .../Api/ApiContractAssertions.cs | 12 + .../ApiContractSecretFieldsConformanceTest.cs | 5 +- .../Api/SecretFieldsFixture.cs | 3 +- .../generator/kotlin/KotlinEntityGenerator.kt | 7 + .../kotlin/KotlinSpringControllerGenerator.kt | 10 +- .../kotlin/KotlinServerOwnedInputTest.kt | 91 ++++++ .../acme/blog/Author.kt | 2 + .../acme/blog/Post.kt | 2 + .../acme/blog/Author.kt | 2 + .../acme/catalog/Product.kt | 2 + .../acme/catalog/Product.kt | 2 + .../entity-with-enum/acme/game/Player.kt | 2 + .../entity-with-fk/acme/blog/Author.kt | 2 + .../entity-with-fk/acme/blog/Post.kt | 2 + .../acme/commerce/Address.kt | 2 + .../acme/commerce/User.kt | 2 + .../acme/commerce/UserMetadata.kt | 2 + .../acme/commerce/Address.kt | 2 + .../entity-with-storage/acme/commerce/User.kt | 2 + .../acme/commerce/UserMetadata.kt | 2 + .../acme/ops/OrderReport.kt | 2 + .../acme/ops/OrderReportArgs.kt | 2 + .../entity-with-tph/acme/auth/Auth.kt | 2 + .../acme/auth/AuthController.kt | 6 - .../entity-with-tph/acme/auth/AuthLine.kt | 2 + .../acme/report/SalesReport.kt | 2 + .../payload-with-origins/acme/ai/Author.kt | 2 + .../acme/ai/AuthorReport.kt | 2 + .../payload-with-origins/acme/ai/Post.kt | 2 + .../acme/demo/Author.kt | 2 + .../acme/ai/OpinionOutputPayload.kt | 2 + .../acme/ai/Greeting.kt | 2 + .../template-prompt-simple/acme/ai/Author.kt | 2 + .../spring/SpringControllerGenerator.java | 57 +++- .../generator/spring/SpringDtoGenerator.java | 3 + .../spring/SpringSecretFieldsCodegenTest.java | 7 +- .../SpringServerOwnedInputCodegenTest.java | 109 +++++++ .../expected/acme/shipping/CrateDto.java | 1 + .../expected/acme/shipping/ParcelDto.java | 1 + .../expected/acme/shipping/ShipmentDto.java | 1 + .../acme/shipping/ShipmentSummaryDto.java | 1 + .../kotlin/api/ApiContractAssertions.kt | 9 + .../GeneratedSecretFieldsControllerHarness.kt | 6 +- .../api/ApiContractAssertions.java | 11 + .../InMemoryAccountRepositorySource.java | 11 +- .../InMemoryAuthorRepositorySource.java | 6 +- .../InMemoryAuthRepositorySource.java | 11 +- .../src/metaobjects/codegen/KNOWN_GAPS.md | 6 +- .../codegen/generators/router_generator.py | 102 +++++-- .../integration/api_contract_assertions.py | 13 + .../generated_secret_fields_app.py | 2 +- .../test_generated_router_validated_input.py | 272 ++++++++++++++++++ .../src/templates/zod-validators.ts | 8 + .../__snapshots__/reporting-docs.test.ts.snap | 66 ++--- .../test/contract/contract-behaviours.test.ts | 2 +- .../test/templates/zod-validators.test.ts | 16 ++ .../src/api-contract-scenario.ts | 11 + ...contract-secret-fields-generated-server.ts | 7 +- .../src/api-contract-server.ts | 19 +- .../src/templates/zod-validators.ts | 8 + 94 files changed, 1400 insertions(+), 294 deletions(-) create mode 100644 fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml create mode 100644 fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml create mode 100644 server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt create mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java create mode 100644 server/python/tests/integration/test_generated_router_validated_input.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 933b82d2c9..60079995e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,43 @@ here.** **New metamodel vocabulary: `measure.derived` and its `@expr`.** It is additive: nothing that loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersion` 1.3 if it merges before the 1.3.0 publish; if it lands after, the next release moves `metamodelVersion`. +### Security + +- **Generated create routes wrote fields a client must never set, in Java, Kotlin, C# and + Python; the TypeScript update shape let a client re-key a row.** An external review of 1.2.0 + found that the Python routes passed raw request input to the repository (fixed for Python in + 1.2.1). An audit of the other ports found the same class of hole on create, in the plain and + the TPH per-subtype handlers: + - **Java** (`codegen-spring`): create forwarded a client-sent generated `id`, `readOnly` and + derived fields to `repository.create`, and in TPH also the discriminator and a sibling + subtype's columns to `createWithType`. + - **Kotlin** (`codegen-kotlin`): create wrote `readOnly` columns, and TPH create wrote every + column of the union table, a sibling subtype's included. + - **C#** (`MetaObjects.Codegen`): create bound the request to the entity and added it as + sent, so a client-sent generated `id` was inserted, a POST to one subtype's route with + another subtype's `type` created a row of that other subtype, M:N navigation collections in + the body were inserted with the row, and a `readOnly` field other than a plain scalar (an + enum, array, value object or map, or any TPH subtype field) had a public setter. A `readOnly` + property now has a private setter on every shape. + - **TypeScript** (`codegen-ts`): an ASSIGNED primary key stayed in `UpdateSchema`, so + PATCH or PUT could change the row's key. A generated key was already refused. + + Every port now drops these keys silently before the write, as the TypeScript create already + did: a server-generated key, a `readOnly` or derived field, and in TPH the discriminator (the + route's subtype is its only source) and every column the route's subtype does not declare. An + assigned key is still the caller's on create. No port answers them with a 400. A key the + entity does not declare is also dropped, never rejected, in every port: Java DTOs and Kotlin + entity classes now carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer no + longer depends on the host's `ObjectMapper` (a plain one answered 400). The Python 1.2.1 fix + is carried forward to this line, and the Python TPH create now drops a body's discriminator + instead of answering 400 for a foreign one. + + Gated by two new shared api-contract scenarios, `input-server-owned-dropped` (generated and + reference lanes) and `tph/tph-input-server-owned-dropped` (both TPH lanes), which send a + client `id`, a foreign discriminator, a `writeOnce` change and a made-up `isAdmin`, and by a + `readOnly` `role` column in the `secret-fields` corpus. Scenarios gain an `absent: [keys]` + assertion. Adopters with ejected (owned) generators need the same change made by hand. + ### Added - **Report measures over Money and other value-object members (FR-044 R12).** A dimension's or diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 461dacf19b..9a516d1b7f 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -559,6 +559,29 @@ set (`@mutability: writeOnce` on PATCH, the TPH discriminator, `@autoSet: onCrea a generated edit form submits every field it was given, so a `400` would fail every save. A required field that is also ignored can never be created through the API, so mark such a field optional. +The same rule covers every key a client never sets, in every port, on the plain and the TPH per-subtype +routes: + +| Key in the body | Create | PATCH, PUT | +|---|---|---| +| a server-generated primary key (`@generation`) | dropped | dropped (the URL names the row) | +| an assigned primary key | **kept** (the caller chooses it) | dropped (the URL names the row) | +| `@mutability: readOnly`, a derived field | dropped | dropped | +| `@mutability: writeOnce` | kept | dropped | +| the TPH discriminator | dropped (the route's subtype is its only source) | dropped | +| a column only a sibling TPH subtype declares | dropped | dropped | +| a key the entity does not declare | dropped | dropped | + +None of these is a `400`, and none reaches the repository. Each port does it in its own idiom: TypeScript +parses with the Zod input schema (which drops unknown keys) and writes the parsed value; Python keeps only +the keys its input model validated; Java nulls the dropped components of the bound DTO before +`repository.create` and its DTOs carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer does not +depend on the host's `ObjectMapper`; Kotlin writes only the route's settable columns; C# resets the bound +entity's generated key, TPH discriminator and M:N navigations before `Add`, and gives a `readOnly` property +a private setter, which System.Text.Json never binds. The shared scenarios `input-server-owned-dropped` and +`tph/tph-input-server-owned-dropped` pin it, and `secret-fields/` pins a `readOnly` column beside the +credential. + Only the generated HTTP boundary changes. The entity, its DTO and the generated queries still carry the field, so your own code writes a hash or token and reads it back. Allowlists are unchanged: a field is filterable or sortable only if you wrote `@filterable` or `@sortable` on it, so do not put those on a credential diff --git a/docs/features/generated-mutations.md b/docs/features/generated-mutations.md index 57aa6f409a..3a367bbb6f 100644 --- a/docs/features/generated-mutations.md +++ b/docs/features/generated-mutations.md @@ -41,7 +41,9 @@ but the semantics are one contract — call them **PATCH-1..7** (FR-035): simply omits every non-settable field, and Zod drops unknown keys — so `safeParse` succeeds and the key never reaches the `SET`. The TPH discriminator is additionally deleted by hand after parse, the same convention. A malformed - *value* on a settable key is still `400 {"error":"validation"}`. + *value* on a settable key is still `400 {"error":"validation"}`. A key the entity + does not declare at all is dropped the same way. The Python routes hand the + repository only the keys their input model validated. This matters for `writeOnce` specifically: the generated edit form submits EVERY registered field (`handleSubmit` passes all values, and 0.19.2 switched the resolver to `UpdateSchema` on edit rather than diff-and-omit), so rejecting a @@ -109,10 +111,10 @@ repo.patch(id) { it[Authors.bio] = "…" } // touches only what you set the controller binds the partial JSON body via `Patch.fromJson` (presence-tracked) and delegates to `repository.patch`, so an omitted key is never written. -**Python** — the generated FastAPI route binds the raw JSON body (the set of -*present* keys — the tristate signal) and validates the present values against the -generated `Patch` model before delegating to the runtime `update`, which -`SET`s only those keys. +**Python** — the generated FastAPI route binds the JSON body (the set of +*present* keys — the tristate signal), validates the present values against the +generated `Patch` model, and hands the repository only the keys that model +validated. The runtime `update` then `SET`s only those keys. **C#** — the generated route runs a per-field merge loop over the present JSON keys (the app's configured `JsonSerializerOptions`), writing only the columns the body diff --git a/examples/advanced-modeling/codegen/templates/zod-validators.ts b/examples/advanced-modeling/codegen/templates/zod-validators.ts index 2e421a0819..3d0a9e52fd 100644 --- a/examples/advanced-modeling/codegen/templates/zod-validators.ts +++ b/examples/advanced-modeling/codegen/templates/zod-validators.ts @@ -719,6 +719,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of flatSurfaceFields(obj)) { @@ -734,6 +735,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // A primary-key field is the row's identity, which the URL names: an ASSIGNED key is + // omitted from the update shape exactly like a generated one, so a client-sent id is + // stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -818,6 +823,9 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // The row's identity is the URL's: an ASSIGNED primary key leaves the update shape too + // (a generated one never entered it), so a client-sent id is stripped, never a re-key. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index 1d4aa83726..3f9fca7aab 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -4,17 +4,33 @@ "generated/csharp/SubscriberBlurbPayload.g.cs": "29eb3a6ce64496ffe734cba55d0213994316b4281d3fa6791917393bae36c9ed", "generated/csharp/SubscriberFilterAllowlist.g.cs": "3839b04884e4e393d4f7d7bf4c7d4fe5b3852bd3a92b1ecb282cb8935a3c98e5", "generated/csharp/SubscriberNames.g.cs": "c2cf0248c2ecde155ec2623604771e39bed23783603d6f18a2ed82967227a732", - "generated/csharp/SubscriberRoutes.g.cs": "15450e230d396a739d789340c42488da354c22265193b5ac61b074b0ba6fb1ff", + "generated/csharp/SubscriberRoutes.g.cs": "d05912d0a76b52334857796a42b2919be8709997d6c073c11d2c44590fe686a9", "generated/python/Subscriber.py": "9dd4c0acea47f99d1daf79930f9de2e400f30780ebce6c63f023be528a72b762", "generated/python/SubscriberBlurbPayload.py": "a5ef8a08e63efbe0f73e39a21a368cff5150c68cc8beeea6e8aab0c755f8bbdd", "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", "generated/python/subscriber_filter_allowlist.py": "b0414cc95ff3982c80a0e1a06839f090c0b423e41f5a94275df93426cea09df5", "generated/python/subscriber_names.py": "269637a02bf9b46f55658b58cb42f83196c03a877e1e2101337847fd406c1561", - "generated/python/subscriber_router.py": "5f219b6e46c913ae30bb46ca80f68713a427c954139a950400eb4d88a776b614", - "generated/ts/Subscriber.queries.ts": "9f1a459c57fafc98bf074b4443b506d4a1e7023ed188f293021ae89c298d4562", - "generated/ts/Subscriber.routes.ts": "147d32ae658f985e024c3de6e8894ca7359f767eeaa765cec70756802e3bd300", - "generated/ts/Subscriber.ts": "a78c17f915222ee108290c28468313d77f77b5cff438f485324409302e389a96", - "generated/ts/SubscriberBlurbPayload.ts": "83272130c428ad0cc68190cb63e32b418a0bcfd05345804def9bf22e79b55fcc", + "generated/python/subscriber_router.py": "021768a4ed1fe24c2432087bba8b350579f60595728a95c6d43a528c3fdcb1be", + "generated/ts/Subscriber.queries.ts": { + "hash": "9f1a459c57fafc98bf074b4443b506d4a1e7023ed188f293021ae89c298d4562", + "generatedBy": "queries-file", + "sourceNode": "acme::Subscriber" + }, + "generated/ts/Subscriber.routes.ts": { + "hash": "147d32ae658f985e024c3de6e8894ca7359f767eeaa765cec70756802e3bd300", + "generatedBy": "routes-file", + "sourceNode": "acme::Subscriber" + }, + "generated/ts/Subscriber.ts": { + "hash": "a78c17f915222ee108290c28468313d77f77b5cff438f485324409302e389a96", + "generatedBy": "entity-file", + "sourceNode": "acme::Subscriber" + }, + "generated/ts/SubscriberBlurbPayload.ts": { + "hash": "83272130c428ad0cc68190cb63e32b418a0bcfd05345804def9bf22e79b55fcc", + "generatedBy": "entity-file", + "sourceNode": "acme::SubscriberBlurbPayload" + }, "generated/ts/index.ts": "2c915565fd8711503dbcd0b0139c657bb927c7f0f502d8d2a8b51e026b257694", "generated/ts/prompts.ts": "9eec821ec8cb6063f98b52b78ae1758cfd22c81a71cbb90b0555dee347a687af", "generated/ts/requirements/subscriberCanBePausedWithoutErasingHistory.field.enum.test.ts": "a5d2494ce5340673da43f2ee3ef13d7505ee349724e8d85f377a84eafeea50ae" diff --git a/examples/showcase/codegen/templates/zod-validators.ts b/examples/showcase/codegen/templates/zod-validators.ts index 2e421a0819..3d0a9e52fd 100644 --- a/examples/showcase/codegen/templates/zod-validators.ts +++ b/examples/showcase/codegen/templates/zod-validators.ts @@ -719,6 +719,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of flatSurfaceFields(obj)) { @@ -734,6 +735,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // A primary-key field is the row's identity, which the URL names: an ASSIGNED key is + // omitted from the update shape exactly like a generated one, so a client-sent id is + // stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -818,6 +823,9 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // The row's identity is the URL's: an ASSIGNED primary key leaves the update shape too + // (a generated one never entered it), so a client-sent id is stripped, never a re-key. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { diff --git a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs index ffb1ed369c..9987f12b3c 100644 --- a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs +++ b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs @@ -100,6 +100,8 @@ await db.Subscribers.FindAsync(id) is { } found try { input = System.Text.Json.JsonSerializer.Deserialize(__body.RootElement.GetRawText(), jsonOpts); } catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = "validation" }); } if (input is null) return Results.BadRequest(new { error = "validation" }); + // Server-owned: a generated key, the TPH discriminator and M:N navigations are not settable by the client; the value is ignored. + input.Id = default!; var __vr = new System.Collections.Generic.List(); if (!System.ComponentModel.DataAnnotations.Validator.TryValidateObject( input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true)) diff --git a/examples/showcase/generated/java/acme/SubscriberDto.java b/examples/showcase/generated/java/acme/SubscriberDto.java index f77826cf41..8304cf25aa 100644 --- a/examples/showcase/generated/java/acme/SubscriberDto.java +++ b/examples/showcase/generated/java/acme/SubscriberDto.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for Subscriber. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record SubscriberDto( Long id, @NotNull @Size(min = 1, max = 320) String email, diff --git a/examples/showcase/generated/kotlin/acme/Subscriber.kt b/examples/showcase/generated/kotlin/acme/Subscriber.kt index e91e6d0f21..e8c40e6d0d 100644 --- a/examples/showcase/generated/kotlin/acme/Subscriber.kt +++ b/examples/showcase/generated/kotlin/acme/Subscriber.kt @@ -1,5 +1,6 @@ package acme +import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.NotNull import jakarta.validation.constraints.Size import java.time.Instant @@ -10,6 +11,7 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ +@JsonIgnoreProperties(ignoreUnknown = true) public data class Subscriber( public val id: Long? = null, @field:NotNull diff --git a/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt b/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt index acb1c79b0a..fc048fbf6b 100644 --- a/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt +++ b/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt @@ -1,5 +1,6 @@ package acme +import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.NotNull import kotlin.String import kotlin.jvm.JvmStatic @@ -7,6 +8,7 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ +@JsonIgnoreProperties(ignoreUnknown = true) public data class SubscriberBlurbPayload( public val name: String? = null, @field:NotNull diff --git a/examples/showcase/generated/python/subscriber_router.py b/examples/showcase/generated/python/subscriber_router.py index f5deb0e491..9db0b29b63 100644 --- a/examples/showcase/generated/python/subscriber_router.py +++ b/examples/showcase/generated/python/subscriber_router.py @@ -92,6 +92,26 @@ async def _json_body(request: Request) -> dict[str, Any] | None: return None +def _validated_input(valid: BaseModel, raw: dict[str, Any]) -> dict[str, Any]: + """GENERATED — *raw* cut down to the keys *valid* validated, at every depth. A key the + input model does not declare (an undeclared name, a server-generated id, a readOnly field) + is dropped silently and never reaches the repository; values stay the body's own.""" + return { + _k: _validated_value(getattr(valid, _k), raw[_k]) + for _k in valid.model_fields_set + } + + +def _validated_value(valid: Any, raw: Any) -> Any: + if isinstance(valid, BaseModel) and isinstance(raw, dict): + return _validated_input(valid, raw) + if isinstance(valid, list) and isinstance(raw, list): + return [_validated_value(v, r) for v, r in zip(valid, raw)] + if isinstance(valid, dict) and isinstance(raw, dict): + return {k: _validated_value(v, raw[k]) for k, v in valid.items() if k in raw} + return raw + + _FROZEN_FIELDS: frozenset[str] = frozenset() @@ -199,9 +219,13 @@ def create_subscriber( # FR-036: run the generated field constraints (length / pattern / # numeric bounds) before persisting; a violation is the cross-port 400. try: - SubscriberCreate(**dto) + _valid = SubscriberCreate(**dto) except ValidationError: return JSONResponse(status_code=400, content={"error": "validation"}) + # SECURITY (1.2.1): only what the input model validated reaches the repository. A key + # it does not declare (an undeclared name, a server-generated id, a readOnly or derived + # field, a frozen one on update) is dropped silently, never 400'd. + dto = _validated_input(_valid, dto) # #203/ADR-0045: stamp @autoSet columns (server-owned; caller ignored). _asnow = _dt.datetime.now(_dt.timezone.utc) dto["createdAt"] = _asnow @@ -240,9 +264,13 @@ def update_subscriber( # FR-036: validate PRESENT, non-null values with the create rules (the # all-optional patch model never fires required-checks on ABSENT fields). try: - SubscriberPatch(**dto) + _valid = SubscriberPatch(**dto) except ValidationError: return JSONResponse(status_code=400, content={"error": "validation"}) + # SECURITY (1.2.1): only what the input model validated reaches the repository. A key + # it does not declare (an undeclared name, a server-generated id, a readOnly or derived + # field, a frozen one on update) is dropped silently, never 400'd. + dto = _validated_input(_valid, dto) dto.pop("createdAt", None) # onCreate @autoSet is write-once (server-owned) try: saved = repo.update(subscriber_id, dto) diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index c716780b2b..93a11d0e23 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -38,15 +38,15 @@ }, "java-dto": { "lang": "java", - "inline": "…\n/** GENERATED — wire DTO for Subscriber. Do not hand-edit; regenerated from metadata. */\npublic record SubscriberDto(\n Long id,\n @NotNull @Size(min = 1, max = 320) String email,\n String name,\n @NotNull SubscriberStatus status,\n java.time.Instant createdAt\n) {\n public enum SubscriberStatus { active, paused, cancelled }\n\n /** Issue #203 @autoSet: a copy with every @autoSet column (onCreate AND onUpdate) stamped to now() on insert (the model value is ignored; a fresh row's updated_at equals its created_at). */\n public static SubscriberDto stampForInsert(SubscriberDto dto) {\n java.time.Instant __nowInstant = java.time.Instant.now();\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n __nowInstant\n );\n }\n…\n}\n…", - "full": "package acme;\n\nimport jakarta.validation.constraints.*;\n\n/** GENERATED — wire DTO for Subscriber. Do not hand-edit; regenerated from metadata. */\npublic record SubscriberDto(\n Long id,\n @NotNull @Size(min = 1, max = 320) String email,\n String name,\n @NotNull SubscriberStatus status,\n java.time.Instant createdAt\n) {\n public enum SubscriberStatus { active, paused, cancelled }\n\n /** Issue #203 @autoSet: a copy with every @autoSet column (onCreate AND onUpdate) stamped to now() on insert (the model value is ignored; a fresh row's updated_at equals its created_at). */\n public static SubscriberDto stampForInsert(SubscriberDto dto) {\n java.time.Instant __nowInstant = java.time.Instant.now();\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n __nowInstant\n );\n }\n\n /** Issue #203 @autoSet: a copy with @autoSet onUpdate columns stamped to now(); onCreate columns are preserved (never rewrites created_at — the full-DTO update path). */\n public static SubscriberDto stampForUpdate(SubscriberDto dto) {\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n dto.createdAt()\n );\n }\n\n /** Issue #203 @autoSet escape hatch: the DTO written VERBATIM (import/restore/replication). */\n public static SubscriberDto insertPreserving(SubscriberDto dto) { return dto; }\n\n /** A fluent builder, so a caller can set a subset of the components. */\n public static Builder builder() { return new Builder(); }\n\n /** Builds a {@link SubscriberDto}; an unset component stays null. */\n public static final class Builder {\n private Long id;\n private String email;\n private String name;\n private SubscriberStatus status;\n private java.time.Instant createdAt;\n\n private Builder() {}\n\n public Builder id(Long v) { this.id = v; return this; }\n public Builder email(String v) { this.email = v; return this; }\n public Builder name(String v) { this.name = v; return this; }\n public Builder status(SubscriberStatus v) { this.status = v; return this; }\n public Builder createdAt(java.time.Instant v) { this.createdAt = v; return this; }\n\n public SubscriberDto build() { return new SubscriberDto(id, email, name, status, createdAt); }\n }\n}\n", - "lineCount": 62 + "inline": "…\n/** GENERATED — wire DTO for Subscriber. Do not hand-edit; regenerated from metadata. */\n…\npublic record SubscriberDto(\n Long id,\n @NotNull @Size(min = 1, max = 320) String email,\n String name,\n @NotNull SubscriberStatus status,\n java.time.Instant createdAt\n) {\n public enum SubscriberStatus { active, paused, cancelled }\n\n /** Issue #203 @autoSet: a copy with every @autoSet column (onCreate AND onUpdate) stamped to now() on insert (the model value is ignored; a fresh row's updated_at equals its created_at). */\n public static SubscriberDto stampForInsert(SubscriberDto dto) {\n java.time.Instant __nowInstant = java.time.Instant.now();\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n __nowInstant\n );\n }\n…\n}\n…", + "full": "package acme;\n\nimport jakarta.validation.constraints.*;\n\n/** GENERATED — wire DTO for Subscriber. Do not hand-edit; regenerated from metadata. */\n@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\npublic record SubscriberDto(\n Long id,\n @NotNull @Size(min = 1, max = 320) String email,\n String name,\n @NotNull SubscriberStatus status,\n java.time.Instant createdAt\n) {\n public enum SubscriberStatus { active, paused, cancelled }\n\n /** Issue #203 @autoSet: a copy with every @autoSet column (onCreate AND onUpdate) stamped to now() on insert (the model value is ignored; a fresh row's updated_at equals its created_at). */\n public static SubscriberDto stampForInsert(SubscriberDto dto) {\n java.time.Instant __nowInstant = java.time.Instant.now();\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n __nowInstant\n );\n }\n\n /** Issue #203 @autoSet: a copy with @autoSet onUpdate columns stamped to now(); onCreate columns are preserved (never rewrites created_at — the full-DTO update path). */\n public static SubscriberDto stampForUpdate(SubscriberDto dto) {\n return new SubscriberDto(\n dto.id(),\n dto.email(),\n dto.name(),\n dto.status(),\n dto.createdAt()\n );\n }\n\n /** Issue #203 @autoSet escape hatch: the DTO written VERBATIM (import/restore/replication). */\n public static SubscriberDto insertPreserving(SubscriberDto dto) { return dto; }\n\n /** A fluent builder, so a caller can set a subset of the components. */\n public static Builder builder() { return new Builder(); }\n\n /** Builds a {@link SubscriberDto}; an unset component stays null. */\n public static final class Builder {\n private Long id;\n private String email;\n private String name;\n private SubscriberStatus status;\n private java.time.Instant createdAt;\n\n private Builder() {}\n\n public Builder id(Long v) { this.id = v; return this; }\n public Builder email(String v) { this.email = v; return this; }\n public Builder name(String v) { this.name = v; return this; }\n public Builder status(SubscriberStatus v) { this.status = v; return this; }\n public Builder createdAt(java.time.Instant v) { this.createdAt = v; return this; }\n\n public SubscriberDto build() { return new SubscriberDto(id, email, name, status, createdAt); }\n }\n}\n", + "lineCount": 63 }, "kotlin-entity": { "lang": "kotlin", - "inline": "…\n/**\n * GENERATED — do not hand-edit. Regenerated from metadata.\n */\npublic data class Subscriber(\n public val id: Long? = null,\n @field:NotNull\n @field:Size(min = 1, max = 320)\n public val email: String,\n public val name: String? = null,\n @field:NotNull\n public val status: SubscriberStatus,\n public val createdAt: Instant? = null,\n…\n)\n…", - "full": "package acme\n\nimport jakarta.validation.constraints.NotNull\nimport jakarta.validation.constraints.Size\nimport java.time.Instant\nimport kotlin.Long\nimport kotlin.String\nimport kotlin.jvm.JvmStatic\n\n/**\n * GENERATED — do not hand-edit. Regenerated from metadata.\n */\npublic data class Subscriber(\n public val id: Long? = null,\n @field:NotNull\n @field:Size(min = 1, max = 320)\n public val email: String,\n public val name: String? = null,\n @field:NotNull\n public val status: SubscriberStatus,\n public val createdAt: Instant? = null,\n) {\n /**\n * Fluent builder — lets a JAVA caller set a subset of properties.\n */\n public class Builder {\n private var id: Long? = null\n\n private var email: String? = null\n\n private var name: String? = null\n\n private var status: SubscriberStatus? = null\n\n private var createdAt: Instant? = null\n\n public fun id(v: Long?): Builder {\n this.id = v\n return this\n }\n\n public fun email(v: String?): Builder {\n this.email = v\n return this\n }\n\n public fun name(v: String?): Builder {\n this.name = v\n return this\n }\n\n public fun status(v: SubscriberStatus?): Builder {\n this.status = v\n return this\n }\n\n public fun createdAt(v: Instant?): Builder {\n this.createdAt = v\n return this\n }\n\n public fun build(): Subscriber = Subscriber(\n id = id,\n email = requireNotNull(email) { "email is required" },\n name = name,\n status = requireNotNull(status) { "status is required" },\n createdAt = createdAt\n )\n }\n\n public companion object {\n @JvmStatic\n public fun builder(): Builder = Builder()\n }\n}\n", - "lineCount": 75 + "inline": "…\n/**\n * GENERATED — do not hand-edit. Regenerated from metadata.\n */\n…\npublic data class Subscriber(\n public val id: Long? = null,\n @field:NotNull\n @field:Size(min = 1, max = 320)\n public val email: String,\n public val name: String? = null,\n @field:NotNull\n public val status: SubscriberStatus,\n public val createdAt: Instant? = null,\n…\n)\n…", + "full": "package acme\n\nimport com.fasterxml.jackson.`annotation`.JsonIgnoreProperties\nimport jakarta.validation.constraints.NotNull\nimport jakarta.validation.constraints.Size\nimport java.time.Instant\nimport kotlin.Long\nimport kotlin.String\nimport kotlin.jvm.JvmStatic\n\n/**\n * GENERATED — do not hand-edit. Regenerated from metadata.\n */\n@JsonIgnoreProperties(ignoreUnknown = true)\npublic data class Subscriber(\n public val id: Long? = null,\n @field:NotNull\n @field:Size(min = 1, max = 320)\n public val email: String,\n public val name: String? = null,\n @field:NotNull\n public val status: SubscriberStatus,\n public val createdAt: Instant? = null,\n) {\n /**\n * Fluent builder — lets a JAVA caller set a subset of properties.\n */\n public class Builder {\n private var id: Long? = null\n\n private var email: String? = null\n\n private var name: String? = null\n\n private var status: SubscriberStatus? = null\n\n private var createdAt: Instant? = null\n\n public fun id(v: Long?): Builder {\n this.id = v\n return this\n }\n\n public fun email(v: String?): Builder {\n this.email = v\n return this\n }\n\n public fun name(v: String?): Builder {\n this.name = v\n return this\n }\n\n public fun status(v: SubscriberStatus?): Builder {\n this.status = v\n return this\n }\n\n public fun createdAt(v: Instant?): Builder {\n this.createdAt = v\n return this\n }\n\n public fun build(): Subscriber = Subscriber(\n id = id,\n email = requireNotNull(email) { "email is required" },\n name = name,\n status = requireNotNull(status) { "status is required" },\n createdAt = createdAt\n )\n }\n\n public companion object {\n @JvmStatic\n public fun builder(): Builder = Builder()\n }\n}\n", + "lineCount": 77 }, "sql-migration": { "lang": "sql", diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index b012a9ed08..452f71d00a 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -27,6 +27,7 @@ fixtures/api-contract-conformance/ ├── get-by-id.yaml ├── get-by-id-not-found.yaml ├── create-201.yaml + ├── input-server-owned-dropped.yaml # a client id, a made-up key and a writeOnce change are dropped, never 400 ├── create-constraint-violation-400.yaml # FR-036 POST field-constraint → 400 {error:"validation"} ├── update-patch-and-put.yaml ├── update-constraint-violation-400.yaml # FR-036 PATCH present-value constraint → 400 @@ -52,17 +53,17 @@ fixtures/api-contract-conformance/ └── filter-invalid-value.yaml # uncoercible value → 400; the fourth envelope, previously ungated ``` -Beside the 31 core scenarios above sit ten **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (92 scenarios in all, 123 with the +Beside the 32 core scenarios above sit ten **sub-corpora**, each a directory with its +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (93 scenarios in all, 125 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| -| `tph/` | 10 | single-table inheritance | both | +| `tph/` | 11 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | | `jsonb/` | 5 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | -| `secret-fields/` | 2 | #484: a `@formExclude` or credential (`view.password`) field is ignored on create, PATCH and PUT, and a credential is never returned | generated only | +| `secret-fields/` | 2 | #484: a `@formExclude` or credential (`view.password`) field is ignored on create, PATCH and PUT, and a credential is never returned; a `readOnly` field is ignored the same way and still returned | generated only | | `enum-default/` | 2 | a create that omits a `field.enum` carrying `@default` stores the default (echo and GET by id), and a supplied member is kept | generated only | | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | @@ -121,6 +122,7 @@ requests: | `error` | the response body has `error: ""` | | `field` | **only meaningful alongside `error`** — the envelope's `field` names the rejected filter/sort field. Declared as a sibling of `error` and checked inside that branch, because the `error` branch returns; a runner that checked it as an independent key would skip it whenever `error` matched first. | | `empty` | the response body is empty / null (204 No Content) | +| `absent` | the response body is an object; none of the listed keys is present (not even as `null`). Pins that a key a client sent but may not set never comes back. | | `fieldsEqual`| the response body is an object; assert the listed field names are all equal to **each other** (raw, field-vs-field — no literal, so timestamp non-determinism is a non-issue). Used by the `@autoSet` gate (#203/ADR-0045). | | `fieldsNotEqual`| the response body is an object; assert the two listed fields **differ** from each other. Used by the `@autoSet` gate (e.g. after a PATCH, `autoCreatedAt` (onCreate, preserved) != `autoUpdatedAt` (onUpdate, bumped)). | diff --git a/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml new file mode 100644 index 0000000000..0e71b5fa3a --- /dev/null +++ b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml @@ -0,0 +1,86 @@ +name: input-server-owned-dropped +description: > + A write route hands storage only what the client may set. A client-sent `id` on a + server-generated key, a key the entity does not declare (`isAdmin`) and, on update, a + `writeOnce` field are all DROPPED SILENTLY: the request answers as if the key were absent + (201/200, never 400), and none of them reaches storage. This is the rule TypeScript has + always followed and Python adopted in 1.2.1, where the generated routes handed the + repository the raw request body. + + Each body carries an ordinary settable field DELIBERATELY: it is applied, and that pairing + is what makes the assertion about the server-owned keys and not about the request being + ignored whole. `absent` asserts the undeclared key is not echoed back (a stored key would + be), and the GETs of the client's ids prove no row was written or re-keyed under them. +requests: + - id: r1 + method: POST + path: /api/authors + body: + id: 999 + name: "Barbara Liskov" + createdAt: "2026-02-01T10:00:00" + isAdmin: true + expect: + status: 201 + body: + row: + name: "Barbara Liskov" + hasId: true + absent: [isAdmin] + - id: r2 + method: GET + path: /api/authors/999 + expect: + status: 404 + - id: r3 + method: PATCH + path: /api/authors/1 + body: + id: 998 + name: "Ada L. (input probe)" + isAdmin: true + expect: + status: 200 + body: + row: + id: 1 + name: "Ada L. (input probe)" + absent: [isAdmin] + - id: r4 + method: PUT + path: /api/authors/2 + body: + id: 997 + name: "Alan T. (input probe)" + createdAt: "2026-01-02T10:00:00" + issuedCurrency: "JPY" + isAdmin: true + expect: + status: 200 + body: + row: + id: 2 + name: "Alan T. (input probe)" + # writeOnce: frozen after create, so the PUT's value never lands. + issuedCurrency: "GBP" + absent: [isAdmin] + - id: r5 + method: GET + path: /api/authors/998 + expect: + status: 404 + - id: r6 + method: GET + path: /api/authors/997 + expect: + status: 404 + - id: r7 + method: GET + path: /api/authors/1 + expect: + status: 200 + body: + row: + id: 1 + name: "Ada L. (input probe)" + absent: [isAdmin] diff --git a/fixtures/api-contract-conformance/secret-fields/README.md b/fixtures/api-contract-conformance/secret-fields/README.md index 29bc632038..b7108083c7 100644 --- a/fixtures/api-contract-conformance/secret-fields/README.md +++ b/fixtures/api-contract-conformance/secret-fields/README.md @@ -11,6 +11,7 @@ The model is one `Account`: | `name` | (nothing) | accepted | returned | | `tokenHash` | `@formExclude: true` | **ignored** | returned | | `pinHash` | child `view.password` | **ignored** | **never returned** | +| `role` | `@mutability: readOnly` | **ignored** | returned | ## The contract @@ -22,6 +23,8 @@ The model is one `Account`: - **A present `null` is a write**, so it does not reach a denied column either. - **A credential is never returned**: not by list, not by get, not in the echo of a create or an update. The key is absent from the object, not `null`. +- **A `readOnly` field is ignored on every write the same way**, and returned like any other + field: the server owns its value (here, the seeded `role`), so a create leaves it `null`. - **`@formExclude` alone does not hide a field.** It says how the field is written, and the field is still part of the resource. - The application still writes these fields through its own code (the generated queries, the @@ -43,7 +46,7 @@ secret-fields/ ├── seed.json # two rows, both secrets set └── scenarios/ ├── read-omits-credential.yaml # GET by id and list: pinHash absent, tokenHash present - └── write-ignores-excluded.yaml # POST / PATCH / PUT / present-null leave tokenHash alone + └── write-ignores-excluded.yaml # POST / PATCH / PUT / present-null leave tokenHash and the readOnly role alone ``` Generated lane only: what is under test is whether each port's GENERATOR emits routes that diff --git a/fixtures/api-contract-conformance/secret-fields/meta.json b/fixtures/api-contract-conformance/secret-fields/meta.json index d47906d8ad..d000a949a6 100644 --- a/fixtures/api-contract-conformance/secret-fields/meta.json +++ b/fixtures/api-contract-conformance/secret-fields/meta.json @@ -10,6 +10,7 @@ { "field.string": { "name": "name", "@required": true, "@maxLength": 100, "@filterable": true, "@sortable": true } }, { "field.string": { "name": "tokenHash", "@maxLength": 100, "@formExclude": true } }, { "field.string": { "name": "pinHash", "@maxLength": 100, "children": [ { "view.password": {} } ] } }, + { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } ] }} diff --git a/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml b/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml index 92f48eb50f..f62e03a4dc 100644 --- a/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml +++ b/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml @@ -18,6 +18,7 @@ requests: id: 1 name: "Ada Lovelace" tokenHash: "token-hash-1" + role: "member" - id: r2 method: GET path: /api/accounts?sort=id @@ -28,6 +29,8 @@ requests: - id: 1 name: "Ada Lovelace" tokenHash: "token-hash-1" + role: "member" - id: 2 name: "Alan Turing" tokenHash: "token-hash-2" + role: "owner" diff --git a/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml b/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml index 05f5e36306..214947fbd0 100644 --- a/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml +++ b/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml @@ -1,8 +1,8 @@ name: write-ignores-excluded description: > - A field excluded from forms (`@formExclude`) or marked as a credential (`view.password`) - is not writable through the generated routes: create, PATCH and PUT all leave it as it - was. The value is IGNORED, never rejected (HTTP 200/201, not 400), the same rule as + A field excluded from forms (`@formExclude`), marked as a credential (`view.password`) or + declared `@mutability: readOnly` is not writable through the generated routes: create, + PATCH and PUT all leave it as it was. The value is IGNORED, never rejected (HTTP 200/201, not 400), the same rule as `@mutability: writeOnce` on PATCH (`writeonce-patch-stripped`), because a generated edit form submits every field it was given. @@ -19,6 +19,7 @@ requests: name: "Ada L. (probe)" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 200 body: @@ -26,6 +27,7 @@ requests: id: 1 name: "Ada L. (probe)" tokenHash: "token-hash-1" + role: "member" - id: r2 method: PUT path: /api/accounts/2 @@ -33,6 +35,7 @@ requests: name: "Alan T. (probe)" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 200 body: @@ -40,6 +43,7 @@ requests: id: 2 name: "Alan T. (probe)" tokenHash: "token-hash-2" + role: "owner" - id: r3 method: POST path: /api/accounts @@ -47,6 +51,7 @@ requests: name: "Grace Hopper" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 201 body: @@ -54,6 +59,8 @@ requests: name: "Grace Hopper" # Never written: the column stays null, and `forged` never landed. tokenHash: null + # readOnly is server-owned: a create cannot set it either. + role: null hasId: true - id: r4 method: PATCH @@ -63,6 +70,7 @@ requests: name: "Ada L. (probe 2)" tokenHash: null pinHash: null + role: null expect: status: 200 body: @@ -70,3 +78,4 @@ requests: id: 1 name: "Ada L. (probe 2)" tokenHash: "token-hash-1" + role: "member" diff --git a/fixtures/api-contract-conformance/secret-fields/seed.json b/fixtures/api-contract-conformance/secret-fields/seed.json index 6697ba1fcb..13b7225941 100644 --- a/fixtures/api-contract-conformance/secret-fields/seed.json +++ b/fixtures/api-contract-conformance/secret-fields/seed.json @@ -1,6 +1,6 @@ { "rows": [ - { "id": 1, "name": "Ada Lovelace", "tokenHash": "token-hash-1", "pinHash": "pin-hash-1" }, - { "id": 2, "name": "Alan Turing", "tokenHash": "token-hash-2", "pinHash": "pin-hash-2" } + { "id": 1, "name": "Ada Lovelace", "tokenHash": "token-hash-1", "pinHash": "pin-hash-1", "role": "member" }, + { "id": 2, "name": "Alan Turing", "tokenHash": "token-hash-2", "pinHash": "pin-hash-2", "role": "owner" } ] } diff --git a/fixtures/api-contract-conformance/tph/README.md b/fixtures/api-contract-conformance/tph/README.md index 5c45a2231f..d058058cf4 100644 --- a/fixtures/api-contract-conformance/tph/README.md +++ b/fixtures/api-contract-conformance/tph/README.md @@ -49,7 +49,8 @@ tph/ ├── tph-writeonce-patch-stripped.yaml # FR-037: inherited @mutability "writeOnce" stripped on the per-subtype PATCH ├── tph-update-explicit-null-clears.yaml # FR-035 tristate: present null clears; omitted @required untouched; null on @required → 400 ├── tph-create-constraint-violation.yaml # FR-036: per-subtype CREATE validates against the annotated Dto - └── tph-update-constraint-violation.yaml # FR-036: per-subtype PATCH validates each PRESENT value + ├── tph-update-constraint-violation.yaml # FR-036: per-subtype PATCH validates each PRESENT value + └── tph-input-server-owned-dropped.yaml # a client id, a foreign `type`, a writeOnce change and a made-up key are dropped, never 400 ``` `meta.json` declares the `acme::auth` package: diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml new file mode 100644 index 0000000000..f340d90be7 --- /dev/null +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml @@ -0,0 +1,64 @@ +name: tph-input-server-owned-dropped +description: > + The per-subtype write routes hand storage only what the client may set, the same rule as + `input-server-owned-dropped` in the core corpus. A client-sent `id` on the generated key, a + key the hierarchy does not declare (`isAdmin`), the discriminator (`type`) and, on update, a + `writeOnce` field are all DROPPED SILENTLY (201/200, never 400): the row keeps its + server-assigned id, its subtype comes from the URL, and nothing undeclared is echoed back. +requests: + - id: r1 + method: POST + path: /api/auths/bridge + body: + id: 999 + type: "Copay" + reference: "REF-INPUT" + quantity: 3 + isAdmin: true + expect: + status: 201 + body: + row: { type: "Bridge", reference: "REF-INPUT", quantity: 3 } + hasId: true + absent: [isAdmin] + - id: r2 + method: GET + path: /api/auths/999 + expect: + status: 404 + - id: r3 + method: PATCH + path: /api/auths/bridge/1 + body: + id: 998 + type: "Copay" + reference: "REF-1-INPUT" + issuedCurrency: "JPY" + isAdmin: true + expect: + status: 200 + body: + row: { id: 1, type: "Bridge", reference: "REF-1-INPUT", issuedCurrency: "USD" } + absent: [isAdmin] + - id: r4 + method: PUT + path: /api/auths/copay/2 + body: + id: 997 + reference: "REF-2-INPUT" + isAdmin: true + expect: + status: 200 + body: + row: { id: 2, type: "Copay", reference: "REF-2-INPUT" } + absent: [isAdmin] + - id: r5 + method: GET + path: /api/auths/998 + expect: + status: 404 + - id: r6 + method: GET + path: /api/auths/997 + expect: + status: 404 diff --git a/fixtures/contract-conformance/m2m/expected/openapi.json b/fixtures/contract-conformance/m2m/expected/openapi.json index 28d59f6255..813d6036ee 100644 --- a/fixtures/contract-conformance/m2m/expected/openapi.json +++ b/fixtures/contract-conformance/m2m/expected/openapi.json @@ -5030,16 +5030,7 @@ "AccountTagUpdate": { "title": "AccountTagUpdate", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "ApiError": { "title": "ApiError", @@ -5126,16 +5117,7 @@ "FollowUpdate": { "title": "FollowUpdate", "type": "object", - "properties": { - "followerId": { - "type": "integer", - "format": "int64" - }, - "followeeId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "Friendship": { "title": "Friendship", @@ -5202,16 +5184,7 @@ "FriendshipUpdate": { "title": "FriendshipUpdate", "type": "object", - "properties": { - "personAId": { - "type": "integer", - "format": "int64" - }, - "personBId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "GuestAccount": { "title": "GuestAccount", @@ -5444,16 +5417,7 @@ "MemberAccountTagUpdate": { "title": "MemberAccountTagUpdate", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "MemberAccountUpdate": { "title": "MemberAccountUpdate", @@ -5734,16 +5698,7 @@ "PostReviewerUpdate": { "title": "PostReviewerUpdate", "type": "object", - "properties": { - "postId": { - "type": "integer", - "format": "int64" - }, - "accountId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "PostTag": { "title": "PostTag", @@ -5810,16 +5765,7 @@ "PostTagUpdate": { "title": "PostTagUpdate", "type": "object", - "properties": { - "postId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "PostUpdate": { "title": "PostUpdate", @@ -5897,16 +5843,7 @@ "ScopedAccountTagUpdate": { "title": "ScopedAccountTagUpdate", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} }, "Tag": { "title": "Tag", diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/AccountTag.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/AccountTag.update.schema.json index 8b7cec3207..b1176800e4 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/AccountTag.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/AccountTag.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/AccountTag.update.schema.json", "title": "AccountTag.update", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Follow.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Follow.update.schema.json index bafa370de5..301ee27b86 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Follow.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Follow.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/Follow.update.schema.json", "title": "Follow.update", "type": "object", - "properties": { - "followerId": { - "type": "integer", - "format": "int64" - }, - "followeeId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Friendship.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Friendship.update.schema.json index b7afbad306..f3fce52a55 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Friendship.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/Friendship.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/Friendship.update.schema.json", "title": "Friendship.update", "type": "object", - "properties": { - "personAId": { - "type": "integer", - "format": "int64" - }, - "personBId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/MemberAccountTag.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/MemberAccountTag.update.schema.json index 3ab529a36e..9f43d28345 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/MemberAccountTag.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/MemberAccountTag.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/MemberAccountTag.update.schema.json", "title": "MemberAccountTag.update", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostReviewer.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostReviewer.update.schema.json index 851406cef3..962ddd08a5 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostReviewer.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostReviewer.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/PostReviewer.update.schema.json", "title": "PostReviewer.update", "type": "object", - "properties": { - "postId": { - "type": "integer", - "format": "int64" - }, - "accountId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostTag.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostTag.update.schema.json index 1eaac8f6fc..7387ae2891 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostTag.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/PostTag.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/PostTag.update.schema.json", "title": "PostTag.update", "type": "object", - "properties": { - "postId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/ScopedAccountTag.update.schema.json b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/ScopedAccountTag.update.schema.json index 28422b417b..fe3409645e 100644 --- a/fixtures/contract-conformance/m2m/expected/schemas/acme/social/ScopedAccountTag.update.schema.json +++ b/fixtures/contract-conformance/m2m/expected/schemas/acme/social/ScopedAccountTag.update.schema.json @@ -3,14 +3,5 @@ "$id": "schemas/acme/social/ScopedAccountTag.update.schema.json", "title": "ScopedAccountTag.update", "type": "object", - "properties": { - "accountId": { - "type": "integer", - "format": "int64" - }, - "tagId": { - "type": "integer", - "format": "int64" - } - } + "properties": {} } diff --git a/fixtures/contract-conformance/secret-fields/expected/openapi.json b/fixtures/contract-conformance/secret-fields/expected/openapi.json index b2cd08f362..5be082d05d 100644 --- a/fixtures/contract-conformance/secret-fields/expected/openapi.json +++ b/fixtures/contract-conformance/secret-fields/expected/openapi.json @@ -459,12 +459,20 @@ "null" ], "maxLength": 100 + }, + "role": { + "type": [ + "string", + "null" + ], + "maxLength": 20 } }, "required": [ "id", "name", - "tokenHash" + "tokenHash", + "role" ] }, "AccountCreate": { diff --git a/fixtures/contract-conformance/secret-fields/expected/schemas/acme/secrets/Account.schema.json b/fixtures/contract-conformance/secret-fields/expected/schemas/acme/secrets/Account.schema.json index e66a8f1d5b..0250ddfb7b 100644 --- a/fixtures/contract-conformance/secret-fields/expected/schemas/acme/secrets/Account.schema.json +++ b/fixtures/contract-conformance/secret-fields/expected/schemas/acme/secrets/Account.schema.json @@ -18,11 +18,19 @@ "null" ], "maxLength": 100 + }, + "role": { + "type": [ + "string", + "null" + ], + "maxLength": 20 } }, "required": [ "id", "name", - "tokenHash" + "tokenHash", + "role" ] } diff --git a/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs new file mode 100644 index 0000000000..4fb3855300 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs @@ -0,0 +1,134 @@ +// ServerOwnedCreateInputTests — a generated create hands storage ONLY what the client may set. +// The HTTP behaviour is gated by the api-contract corpus (`input-server-owned-dropped`, +// `tph-input-server-owned-dropped`, secret-fields); this pins the emission: the generated key, +// the TPH discriminator and the M:N navigations are reset before `Add`, and a readOnly field of +// EVERY shape (including a TPH subtype's own) is declared with a private setter. + +using MetaObjects.Codegen; +using MetaObjects.Codegen.Generators; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ServerOwnedCreateInputTests +{ + private const string PlainModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Post", "children": [ + { "source.rdb": { "@table": "posts" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@maxLength": 100 } }, + { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, + { "field.enum": { "name": "stage", "@values": ["Draft", "Live"], "@mutability": "readOnly" } }, + { "relationship.association": { "name": "tags", "@cardinality": "many", "@objectRef": "Tag", "@through": "PostTag" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "Tag", "children": [ + { "source.rdb": { "@table": "tags" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "PostTag", "children": [ + { "source.rdb": { "@table": "post_tags" } }, + { "field.long": { "name": "postId" } }, + { "field.long": { "name": "tagId" } }, + { "identity.primary": { "@fields": ["postId", "tagId"] } }, + { "identity.reference": { "name": "fkPost", "@fields": "postId", "@references": "Post" } }, + { "identity.reference": { "name": "fkTag", "@fields": "tagId", "@references": "Tag" } } + ]}} + ]}} + """; + + private const string AssignedKeyModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Code", "children": [ + { "source.rdb": { "@table": "codes" } }, + { "field.string": { "name": "id", "@maxLength": 20 } }, + { "identity.primary": { "@fields": "id", "@generation": "assigned" } } + ]}} + ]}} + """; + + private const string TphModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Thing", "@discriminator": "kind", "children": [ + { "source.rdb": { "@table": "things" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "kind", "@values": ["A", "B"] } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "AThing", "extends": "Thing", "@discriminatorValue": "A", "children": [ + { "field.string": { "name": "note", "@maxLength": 20 } }, + { "field.string": { "name": "grade", "@maxLength": 20, "@mutability": "readOnly" } } + ]}}, + { "object.entity": { "name": "BThing", "extends": "Thing", "@discriminatorValue": "B", "children": [ + { "field.int": { "name": "qty" } } + ]}} + ]}} + """; + + private static string Emit(string model, string generator, string file) + { + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "input.json")]); + Assert.Empty(r.Errors); + var ctx = new GenContext + { + Entities = r.Root.Objects(), + Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = ColumnNamingStrategy.Literal }, + }; + IGenerator g = generator == "routes" ? new RoutesGenerator() : new EntityGenerator(); + return g.Generate(ctx).Single(f => f.Path == file).Content; + } + + [Fact] + public void Create_resets_a_generated_key_and_empties_navigations_before_Add() + { + var src = Emit(PlainModel, "routes", "PostRoutes.g.cs"); + var post = src[src.IndexOf("app.MapPost(", StringComparison.Ordinal)..]; + var reset = post.IndexOf("input.Id = default!;", StringComparison.Ordinal); + var tags = post.IndexOf("input.Tags = new System.Collections.Generic.List();", StringComparison.Ordinal); + var add = post.IndexOf("db.Posts.Add(input);", StringComparison.Ordinal); + Assert.True(reset >= 0 && tags >= 0 && add > reset && add > tags, post); + } + + [Fact] + public void An_assigned_key_is_the_callers_and_is_not_reset() + { + var src = Emit(AssignedKeyModel, "routes", "CodeRoutes.g.cs"); + Assert.DoesNotContain("input.Id = default!;", src); + Assert.DoesNotContain("Server-owned", src); + } + + [Fact] + public void A_readOnly_field_of_every_shape_has_a_private_setter() + { + var src = Emit(PlainModel, "entity", "Post.g.cs"); + Assert.Contains("Role { get; private set; }", src); + Assert.Contains("Stage { get; private set; }", src); + Assert.Contains("Title { get; set; }", src); + } + + [Fact] + public void A_tph_subtype_readOnly_field_has_a_private_setter() + { + var src = Emit(TphModel, "entity", "AThing.g.cs"); + Assert.Contains("Grade { get; private set; }", src); + Assert.Contains("Note { get; set; }", src); + } + + [Fact] + public void Tph_subtype_create_resets_the_key_and_pins_the_discriminator_to_the_route_subtype() + { + var src = Emit(TphModel, "routes", "ThingRoutes.g.cs"); + var start = src.IndexOf("app.MapPost(prefix + \"/things/a\"", StringComparison.Ordinal); + var end = src.IndexOf("app.MapPost(prefix + \"/things/b\"", StringComparison.Ordinal); + Assert.True(start >= 0 && end > start, src); + var a = src[start..end]; + Assert.Contains("input.Id = default!;", a); + Assert.Contains("input.Kind = default!;", a); + Assert.True(a.IndexOf("input.Kind = default!;", StringComparison.Ordinal) < a.IndexOf(".Add(input);", StringComparison.Ordinal), a); + } +} diff --git a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs index c6a59fcd7f..4e298e5958 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs @@ -603,6 +603,16 @@ protected virtual void EmitClassDeclarationLine( private static string MappedBaseClause(MetaObject entity, GenContext ctx) => DirectMappedParent(entity, ctx.Config) is { } parent ? $" : {CSharpNaming.Pascal(parent.Name)}" : ""; + // FR-037 R1 / input hardening — the accessor pair of a mapped property. A `readOnly` field takes a + // PRIVATE setter on EVERY property shape (scalar, enum, array, value object, map; plain entity + // or TPH subtype): EF materializes it on read and application code cannot write it, so the + // generated create route's body binding (System.Text.Json skips a non-public setter) can never + // hand a client-supplied value to storage. Chosen over resetting the member in each route + // because it holds for every writer of the class, not just the routes, and needs no per-shape + // reset code. `writeOnce` keeps a PUBLIC setter on purpose (the caller supplies it on insert). + private static string SetterFor(MetaField field) => + field.IsReadOnlyMutability ? "get; private set;" : "get; set;"; + // A M:N navigation collection property on the source entity. The collection is // the relationship name PascalCased; its element type is the target entity. // @@ -749,7 +759,7 @@ protected virtual string TphSubtypeScalarProperty( AppendDurationConverterAttribute(arr, field, _wire); // A NON-required decimal array is nullable (no initializer) so PATCH null / an omitted // POST value stays a NULL column instead of failing as a required collection. - arr.Append(PropertyDeclaration(entity, field, propName, "get; set;")); + arr.Append(PropertyDeclaration(entity, field, propName, SetterFor(field))); return arr.ToString(); } @@ -772,7 +782,7 @@ protected virtual string TphSubtypeScalarProperty( // life at the CLR default (false/0/null) on `new Sub()` instead of the declared // @default value — every OTHER scalar-property path in this generator already // applies DefaultInitializer; this one was the one gap. - sb.Append(PropertyDeclaration(entity, field, propName, "get; set;")); + sb.Append(PropertyDeclaration(entity, field, propName, SetterFor(field))); return sb.ToString(); } @@ -783,7 +793,7 @@ protected virtual string TphSubtypeEnumProperty(MetaObject entity, MetaField fie _site = new PropertySite(ValueObject: false, TphSubtype: true, null); var propName = PropertyName(field); var colAttr = ColumnAttributeLine(entity, field, strategy, config.IncludeNames) is { } column ? column + "\n" : ""; - return colAttr + PropertyDeclaration(entity, field, propName, "get; set;"); + return colAttr + PropertyDeclaration(entity, field, propName, SetterFor(field)); } // An enum ARRAY property is IList with an empty-list initializer, except a @@ -2623,7 +2633,7 @@ protected virtual string EnumProperty( var colAttr = withAttributes && ColumnAttributeLine(entity, field, strategy, config.IncludeNames) is { } column ? column + "\n" : ""; - return colAttr + PropertyDeclaration(entity, field, propName, "get; set;"); + return colAttr + PropertyDeclaration(entity, field, propName, SetterFor(field)); } // A scalar property. withAttributes adds the EF-mapping annotations ([Key] for a @@ -2683,7 +2693,7 @@ protected virtual string ScalarProperty( // POST value stays a NULL column instead of failing as a required collection; a // NON-required value-object array is nullable so a null member is omitted on the wire. // (That rule is the default of the PropertyClrType / PropertyInitializer hooks.) - arr.Append(PropertyDeclaration(owner, field, propName, "get; set;")); + arr.Append(PropertyDeclaration(owner, field, propName, SetterFor(field))); return arr.ToString(); } @@ -2752,7 +2762,7 @@ protected virtual string ScalarProperty( // the column from UPDATE). `writeOnce` keeps a PUBLIC setter on purpose — the caller // must be able to supply it on insert; freezing it afterwards is the DbContext's // SetAfterSaveBehavior(Ignore), not the property's job. - var setter = field.IsReadOnlyMutability ? "get; private set;" : "get; set;"; + var setter = SetterFor(field); sb.Append(PropertyDeclaration(owner, field, propName, setter)); return sb.ToString(); } @@ -3021,7 +3031,7 @@ protected static string EscapeAttrString(string s) // single-VO nullability below. A non-null empty-list default would erase that // null-vs-empty distinction (EF re-serializes the empty list to `[]` on write). // (The type and initializer are the PropertyClrType / PropertyInitializer defaults.) - return PropertyDeclaration(owner, field, propName, "get; set;", ctx); + return PropertyDeclaration(owner, field, propName, SetterFor(field), ctx); } // A property for a field.map (an open-keyed map). Emits a single @@ -3051,7 +3061,7 @@ protected string MapProperty(MetaObject owner, MetaField field, GenContext ctx, && ColumnAttributeLine(owner, field, ctx.Config.ColumnNamingStrategy, ctx.Config.IncludeNames, ctx) is { } column ? column + "\n" : ""; - return colAttr + PropertyDeclaration(owner, field, propName, "get; set;", ctx); + return colAttr + PropertyDeclaration(owner, field, propName, SetterFor(field), ctx); } // The CLR scalar type for a projection field whose value is a MIN/MAX aggregate: diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 6d439938a3..53ff2ce597 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -429,7 +429,8 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte AppendCreateHandler(sb, "/" + route, cls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + route + "\"", "input", credentials), voFields, mapVoFields, autoSetFields, writeThrough ? (readDbSet, pkProp!, route) : null, parseBody: HasDecimalField(entity) || HasIsoDurationField(entity, ctx.Root), - deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials); + deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials, + ServerOwnedCreateResets(entity, ctx.Root, _ => { })); // PATCH + PUT share the same handler — TS reference exposes both verbs, and both // return the updated row (HTTP 200), matching the cross-port api-contract. @@ -999,7 +1000,8 @@ private static void AppendCreateHandler( (string ReadDbSet, string PkProp, string Route)? reRead = null, bool parseBody = false, IReadOnlyList? deniedNavs = null, - IReadOnlyList? credentials = null) + IReadOnlyList? credentials = null, + IReadOnlyList? serverOwned = null) { // A decimal field can refuse its value while the body is bound (out of declared range, not a // number). The framework's own binding answers that with a bare 400 and no body, so an entity @@ -1008,6 +1010,7 @@ private static void AppendCreateHandler( { sb.AppendLine($" app.MapPost(prefix + \"{mapPath}\", async ({cls} input, AppDbContext db) =>"); sb.AppendLine(" {"); + AppendCreateServerOwned(sb, serverOwned); AppendCreateIgnoreDenied(sb, deniedNavs); AppendCreateValidation(sb); AppendCreateVoValidation(sb, voFields, mapVoFields); @@ -1044,6 +1047,7 @@ private static void AppendCreateHandler( sb.AppendLine(" try { input = System.Text.Json.JsonSerializer.Deserialize<" + cls + ">(__body.RootElement.GetRawText(), jsonOpts); }"); sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); sb.AppendLine(" if (input is null) return Results.BadRequest(new { error = \"validation\" });"); + AppendCreateServerOwned(sb, serverOwned); AppendCreateIgnoreDenied(sb, deniedNavs); AppendCreateValidation(sb); AppendCreateVoValidation(sb, voFields, mapVoFields); @@ -1078,6 +1082,45 @@ private static void AppendCreatePersist( } } + // Input hardening — a generated create hands storage ONLY what the client may set. The body is + // bound straight onto the EF entity, so every member the server owns is reset to its CLR default + // before validation and `Add`, and the request answers as if the key were absent (never a 400): + // - a server-generated primary key (@generation increment|uuid): default, so EF generates it; + // - the TPH discriminator on a per-subtype route: default, so EF's discriminator generator + // stamps the subtype's own @discriminatorValue (the route's subtype is its only source); + // - M:N navigation collections: emptied, else `Add` would insert a client-sent related graph. + // @readOnly fields need no line here: the entity declares them with a private setter, which + // System.Text.Json never binds (see EntityGenerator.SetterFor). Each statement is a complete line. + private static void AppendCreateServerOwned(StringBuilder sb, IReadOnlyList? statements) + { + if (statements is null || statements.Count == 0) return; + sb.AppendLine(" // Server-owned: a generated key, the TPH discriminator and M:N navigations are not settable by the client; the value is ignored."); + foreach (var st in statements) sb.AppendLine(" " + st); + } + + // The reset statements AppendCreateServerOwned emits for one entity (empty when it has none, which + // keeps the output byte-identical for an entity without a generated key, navigation or discriminator). + private static List ServerOwnedCreateResets(MetaObject entity, MetaRoot root, Action onWarn, string? discProp = null, MetaObject? navOwner = null) + { + var list = new List(); + if (entity.PrimaryIdentity() is MetaPrimaryIdentity + { + Generation: MetaObjects.Core.Identity.IdentityConstants.GENERATION_INCREMENT + or MetaObjects.Core.Identity.IdentityConstants.GENERATION_UUID, + Fields: [var keyName], + } && entity.Fields().FirstOrDefault(f => f.Name == keyName) is { } keyField) + list.Add($"input.{CSharpNaming.Pascal(keyField.Name)} = default!;"); + if (discProp is not null) list.Add($"input.{discProp} = default!;"); + // A TPH subtype class declares no M:N property of its own: the navigations that exist are those of the + // discriminator base class (navOwner), so a subtype route resets exactly those. + foreach (var nav in M2MNavigationBuilder.For(navOwner ?? entity, root, onWarn)) + { + var target = CSharpNaming.Pascal(nav.Target.Name); + list.Add($"input.{CSharpNaming.Pascal(nav.Name)} = new System.Collections.Generic.List<{target}>();"); + } + return list; + } + // #484 — a write-denied key (@formExclude or credential) on a create body is IGNORED, never // rejected: the bound value is reset to the CLR default (unset / null) before validation. private static void AppendCreateIgnoreDenied(StringBuilder sb, IReadOnlyList? deniedNavs) @@ -1493,7 +1536,8 @@ private static void AppendTphSubtypeRoutes( AppendCreateHandler(sb, "/" + subRoute, subCls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + subRoute + "/\" + input." + pkProp, "input", credentials), [], mapVoFields, autoSetFields, parseBody: HasDecimalField(st.Entity) || HasIsoDurationField(st.Entity, root), - deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials); + deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials, + serverOwned: ServerOwnedCreateResets(st.Entity, root, _ => { }, discProp, hierarchy[0])); // Per-subtype update (PATCH + PUT): partial merge of the JSON body onto the scoped // entity, NEVER touching the PK or the discriminator. Cross-subtype id → 404. diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs index e6bc127b54..2e8ee64f3d 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs @@ -151,6 +151,18 @@ public static void AssertResponse(string scenarioName, ApiRequest request, int s $"{scenarioName} / {request.Id}: expected row.{kvp.Key}={Render(kvp.Value)}, got {Render(actual)}"); } } + // The listed keys must not be on the object at all (not even as null): a key the client + // may not set never reached storage and so never comes back in the echo. + if (want.TryGetValue("absent", out var absentObj) && absentObj is List absentKeys) + { + if (body is not Dictionary absentRow) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {request.Id}: expected object for absent, got: {Render(body)}"); + var present = absentKeys.Select(k => k?.ToString() ?? "").Where(absentRow.ContainsKey).ToList(); + if (present.Count > 0) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {request.Id}: expected no {string.Join(", ", present)} in body, got: {Render(body)}"); + } if (want.TryGetValue("hasId", out var hasIdObj) && hasIdObj is bool hi && hi) { if (body is not Dictionary row) diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs index 9314afaa29..09066f09e7 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs @@ -37,7 +37,7 @@ private static async Task AssertStoredSecretsUntouchedAsync(string connString) await using var c = new Npgsql.NpgsqlConnection(connString); await c.OpenAsync(); await using var cmd = c.CreateCommand(); - cmd.CommandText = "SELECT \"id\", \"tokenHash\", \"pinHash\" FROM \"accounts\" ORDER BY \"id\""; + cmd.CommandText = "SELECT \"id\", \"tokenHash\", \"pinHash\", \"role\" FROM \"accounts\" ORDER BY \"id\""; await using var reader = await cmd.ExecuteReaderAsync(); var seen = new HashSet(); while (await reader.ReadAsync()) @@ -45,16 +45,19 @@ private static async Task AssertStoredSecretsUntouchedAsync(string connString) long id = reader.GetInt64(0); string? token = reader.IsDBNull(1) ? null : reader.GetString(1); string? pin = reader.IsDBNull(2) ? null : reader.GetString(2); + string? role = reader.IsDBNull(3) ? null : reader.GetString(3); seen.Add(id); if (seed.TryGetValue(id, out var row)) { Assert.Equal(row["tokenHash"]!.GetValue(), token); Assert.Equal(row["pinHash"]!.GetValue(), pin); + Assert.Equal(row["role"]!.GetValue(), role); // readOnly: no write path reaches it } else { Assert.Null(token); Assert.Null(pin); + Assert.Null(role); } } Assert.True(seed.Keys.All(seen.Contains), "a seeded row went missing"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs index 2aaf7d81ca..7818486b16 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs @@ -12,6 +12,7 @@ internal static class SecretFieldsFixture "name" VARCHAR(100) NOT NULL, "tokenHash" VARCHAR(100) NULL, "pinHash" VARCHAR(100) NULL, + "role" VARCHAR(20) NULL, CONSTRAINT "accounts_pkey" PRIMARY KEY ("id") ); """; @@ -29,7 +30,7 @@ public static async Task ApplySeedAsync(string connString, string seedPath) { var root = JsonNode.Parse(File.ReadAllText(seedPath)) as JsonObject ?? throw new InvalidOperationException($"{seedPath}: top-level must be an object"); - var cols = new[] { "id", "name", "tokenHash", "pinHash" }; + var cols = new[] { "id", "name", "tokenHash", "pinHash", "role" }; await using var c = new NpgsqlConnection(connString); await c.OpenAsync(); foreach (var rowNode in root["rows"]!.AsArray()) diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt index 810cf28e5b..aa06c02a31 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt @@ -240,6 +240,13 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { .addModifiers(KModifier.DATA) .addKdoc("GENERATED — do not hand-edit. Regenerated from metadata.\n") .apply { + // The class IS the create request body: an undeclared key is DROPPED, never rejected, + // whatever FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries. + addAnnotation( + AnnotationSpec.builder(ClassName("com.fasterxml.jackson.annotation", "JsonIgnoreProperties")) + .addMember("ignoreUnknown = true") + .build(), + ) // `wire.nullMembers: omit` (ADR-0019 Amendment 2): a value object's null members are left // out. Under native/emit Jackson writes them as null with the key present. if (obj.subType == MetaObject.SUBTYPE_VALUE && wire.omitNullMembers()) { diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 312382c169..6d9828e6fa 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -542,6 +542,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase serverOwnedOnCreate(MetaObject entity) { // #484: a write-denied field is IGNORED on create, so a @NotNull on it describes the // response shape and must not reject a body that (necessarily) omits it. if (pkFields.contains(f.getName()) || AutoSetSupport.isAutoSet(f) - || denied.contains(f.getName())) { + || denied.contains(f.getName()) || isNonSettableOnCreate(f)) { String component = SpringNaming.recordComponentName(f.getName()); if (!out.contains(component)) out.add(component); } @@ -1748,6 +1753,52 @@ private static void appendFillDefaults(StringBuilder src, String dtoName, List generatedPkNames(MetaObject entity) { + com.metaobjects.identity.PrimaryIdentity pk = entity.getPrimaryIdentity(); + if (pk == null || !pk.isAutoGenerated()) return Set.of(); + return new java.util.LinkedHashSet<>(pk.getFields()); + } + + /** + * The components a plain create handler drops from the bound body before it reaches the + * repository: write-denied ({@code @formExclude} / credential), a generated primary key (an + * assigned key is the caller's), {@code readOnly} and derived fields. + */ + private static Set createDeniedNames(MetaObject entity, List components) { + Set out = new java.util.LinkedHashSet<>(SpringGeneratorSupport.writeDeniedNames(components)); + Set pk = generatedPkNames(entity); + for (MetaField f : components) { + if (pk.contains(f.getName()) || isNonSettableOnCreate(f)) out.add(f.getName()); + } + return out; + } + + /** + * The union-DTO components a TPH per-subtype create drops: everything in + * {@link #createDeniedNames}, the discriminator, and every column the route's subtype does not + * itself declare (a sibling subtype's, never this route's to write). + */ + private static Set tphCreateDeniedNames(MetaObject subtype, List union, + Set writeDenied, String discField) { + Set out = new java.util.LinkedHashSet<>(writeDenied); + Set pk = generatedPkNames(subtype); + java.util.Map own = new java.util.HashMap<>(); + for (MetaField f : SpringGeneratorSupport.scalarFields(subtype)) own.put(f.getName(), f); + for (MetaField u : union) { + MetaField f = own.get(u.getName()); + if (f == null || pk.contains(u.getName()) || u.getName().equals(discField) || isNonSettableOnCreate(f)) { + out.add(u.getName()); + } + } + return out; + } + private static void appendStripWriteDenied(StringBuilder src, String dtoName, List components, Set writeDenied) { if (components.stream().noneMatch(f -> writeDenied.contains(f.getName()))) return; diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java index f0d5e80bde..8d266b9448 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java @@ -646,6 +646,9 @@ private void emitRecord(MetaObject entity, Path outRoot, List fields, } src.append("/** GENERATED — wire DTO for ").append(shortName) .append(". Do not hand-edit; regenerated from metadata. */\n"); + // The DTO is the create request body: an undeclared key is DROPPED, never rejected, whatever + // FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries (a plain ObjectMapper has it on). + src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); src.append("public record ").append(recordName).append("(\n"); // Filter ObjectField — see class javadoc. Same reason as the Kotlin port's diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java index 4856d29870..0e49a1e098 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java @@ -63,7 +63,7 @@ private static void run(com.metaobjects.generator.direct.MultiFileDirectGenerato public void createIgnoresBothDeniedFields_andPatchCarriesNeither() throws Exception { String controller = generate("marked", MARKED, "AccountController.java", true); assertTrue("create rebuilds the DTO without tokenHash and pinHash; saw:\n" + controller, - controller.contains("dto = new AccountDto(dto.id(), dto.name(), null, null);")); + controller.contains("dto = new AccountDto(null, dto.name(), null, null);")); Path gen = tmp.getRoot().toPath().resolve("gen-marked"); String patch = Files.readString(gen.resolve("acme/secrets/AccountPatch.java")); @@ -89,7 +89,8 @@ public void onlyTheCredentialIsHiddenOnRead() throws Exception { public void unmarkedModelEmitsNoRedactionAndNoStrip() throws Exception { String controller = generate("plain", PLAIN, "AccountController.java", false); assertFalse(controller.contains("redact")); - assertFalse(controller.contains("dto = new AccountDto(")); + // only the generated key is dropped: nothing here is a credential or @formExclude + assertTrue(controller.contains("dto = new AccountDto(null, dto.name(), dto.tokenHash(), dto.pinHash());")); } /** @@ -123,7 +124,7 @@ public void tphSharedColumnMarkedOnALaterSubtypeIsHiddenAndIgnored() throws Exce assertTrue("the union list hides the credential; saw:\n" + controller, controller.contains("redactAll(rows, \"token\")")); assertTrue("every subtype create ignores both marked columns; saw:\n" + controller, - controller.contains("dto = new ThingDto(dto.id(), dto.kind(), null, null);")); + controller.contains("dto = new ThingDto(null, null, null, null);")); assertFalse("memo (@formExclude only) is still returned", controller.contains("\"memo\")")); } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java new file mode 100644 index 0000000000..0af89692c4 --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java @@ -0,0 +1,109 @@ +package com.metaobjects.generator.spring; + +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * A generated create route hands the repository only what a client may set: a server-generated + * key, a {@code readOnly} field and (TPH) the discriminator and a sibling subtype's column are + * dropped from the bound body, and the request type ignores an undeclared key itself. + */ +public class SpringServerOwnedInputCodegenTest extends SharedRegistryTestBase { + + @Rule + public TemporaryFolder tmp = new TemporaryFolder(); + + private static final String PLAIN = """ + { + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Doc", "children": [ + { "source.rdb": { "@table": "docs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "field.string": { "name": "role", "@mutability": "readOnly" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "Tag", "children": [ + { "source.rdb": { "@table": "tags" } }, + { "field.string": { "name": "code" } }, + { "field.string": { "name": "label" } }, + { "identity.primary": { "name": "pk", "@fields": ["code"] } } + ] } } + ] } + } + """; + + private static final String TPH = """ + { + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Prior"] } }, + { "field.string": { "name": "reference" } }, + { "field.string": { "name": "grade", "@mutability": "readOnly" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", "children": [ + { "field.int": { "name": "quantity" } } + ] } }, + { "object.entity": { "name": "PriorAuth", "extends": "Auth", "@discriminatorValue": "Prior", "children": [ + { "field.string": { "name": "approver" } } + ] } } + ] } + } + """; + + private Path generate(String name, String fixture) throws Exception { + Path gen = tmp.newFolder("gen-" + name).toPath(); + MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder("ws-" + name).toPath(), name, fixture); + for (var g : new com.metaobjects.generator.direct.MultiFileDirectGeneratorBase[] { + new SpringControllerGenerator(), new SpringDtoGenerator() }) { + Map args = new HashMap<>(); + args.put("outputDir", gen.toString()); + g.setArgs(args); + g.execute(loader); + } + return gen; + } + + @Test + public void plainCreateDropsTheGeneratedKeyAndReadOnlyField() throws Exception { + Path gen = generate("plain", PLAIN); + String controller = Files.readString(gen.resolve("acme/own/DocController.java")); + assertTrue("generated id and readOnly role are nulled; saw:\n" + controller, + controller.contains("dto = new DocDto(null, dto.title(), null);")); + // an assigned key (no @generation) is the caller's on create + String tag = Files.readString(gen.resolve("acme/own/TagController.java")); + assertFalse("assigned key must be kept; saw:\n" + tag, tag.contains("dto = new TagDto(")); + } + + @Test + public void requestTypeIgnoresUnknownProperties() throws Exception { + Path gen = generate("unknown", PLAIN); + String dto = Files.readString(gen.resolve("acme/own/DocDto.java")); + assertTrue(dto.contains("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\npublic record DocDto(")); + } + + @Test + public void tphCreateDropsKeyDiscriminatorReadOnlyAndSiblingColumns() throws Exception { + Path gen = generate("tph", TPH); + String controller = Files.readString(gen.resolve("acme/own/AuthController.java")); + // union components: id, type, reference, grade, quantity, approver + assertTrue("Bridge route keeps reference and quantity only; saw:\n" + controller, + controller.contains("dto = new AuthDto(null, null, dto.reference(), null, dto.quantity(), null);")); + assertTrue("Prior route keeps reference and approver only; saw:\n" + controller, + controller.contains("dto = new AuthDto(null, null, dto.reference(), null, null, dto.approver());")); + } +} diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CrateDto.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CrateDto.java index 36bb6b96dd..55e45df9e9 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CrateDto.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CrateDto.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for Crate. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record CrateDto( @NotNull Long id, @NotNull Status stage diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelDto.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelDto.java index 3fd14af431..29060b4e86 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelDto.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelDto.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for Parcel. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ParcelDto( @NotNull Long id, @NotNull Status phase diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentDto.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentDto.java index 071d54e9f5..fba5bf9e66 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentDto.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentDto.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for Shipment. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ShipmentDto( @NotNull Long id, @NotNull ShipmentStatus status, diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java index bdfadca7f1..a2459cdb36 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for ShipmentSummary. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ShipmentSummaryDto( @NotNull Long id, @NotNull Status status diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt index 193959935d..73ae81ee31 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt @@ -152,6 +152,15 @@ object ApiContractAssertions { throw AssertionError("$scenarioName / ${request.id}: expected row.$k=$v, got $actual") } } + // The listed keys must not be on the object at all (not even as null): a key the client + // may not set never reached storage and so never comes back in the echo. + (want["absent"] as? List<*>)?.let { keys -> + val row = body as? Map<*, *> + ?: throw AssertionError("$scenarioName / ${request.id}: expected object for absent, got: $body") + val present = keys.filter { row.containsKey(it) } + if (present.isNotEmpty()) + throw AssertionError("$scenarioName / ${request.id}: expected no $present in body, got: $body") + } if (want["hasId"] == true) { val row = body as? Map<*, *> ?: throw AssertionError("$scenarioName / ${request.id}: expected object, got: $body") diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt index a0492ebdd7..719fdb3929 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt @@ -112,7 +112,7 @@ class GeneratedSecretFieldsControllerHarness( // Ids are assigned by the identity column in seed order (1, 2), so a later POST does not collide. // Mixed-case columns are quoted (naming.columns: literal), as in the write-through harness. for (r in rows) { - exec("INSERT INTO accounts (\"name\", \"tokenHash\", \"pinHash\") VALUES (${sqlStr(r["name"])}, ${sqlStr(r["tokenHash"])}, ${sqlStr(r["pinHash"])})") + exec("INSERT INTO accounts (\"name\", \"tokenHash\", \"pinHash\", \"role\") VALUES (${sqlStr(r["name"])}, ${sqlStr(r["tokenHash"])}, ${sqlStr(r["pinHash"])}, ${sqlStr(r["role"])})") } } @@ -128,9 +128,9 @@ class GeneratedSecretFieldsControllerHarness( val database = db ?: error("reset() must be called before storedRows()") return transaction(database) { val out = mutableListOf>() - exec("SELECT id, \"tokenHash\", \"pinHash\" FROM accounts ORDER BY id") { rs -> + exec("SELECT id, \"tokenHash\", \"pinHash\", \"role\" FROM accounts ORDER BY id") { rs -> while (rs.next()) { - out += mapOf("id" to rs.getLong(1), "tokenHash" to rs.getString(2), "pinHash" to rs.getString(3)) + out += mapOf("id" to rs.getLong(1), "tokenHash" to rs.getString(2), "pinHash" to rs.getString(3), "role" to rs.getString(4)) } } out diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java index b7ddbff70d..6c6348a0c7 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java @@ -177,6 +177,17 @@ static void assertResponse(String scenarioName, ApiRequest request, int status, + ": expected row." + e.getKey() + "=" + e.getValue() + ", got " + actual); } } + // The listed keys must not be on the object at all (not even as null): a key the client + // may not set never reached storage and so never comes back in the echo. + if (want.get("absent") instanceof List absentKeys) { + if (!(body instanceof Map row)) + throw new AssertionError(scenarioName + " / " + request.id() + + ": expected object for absent, got: " + body); + List present = absentKeys.stream().filter(row::containsKey).toList(); + if (!present.isEmpty()) + throw new AssertionError(scenarioName + " / " + request.id() + + ": expected no " + present + " in body, got: " + body); + } if (Boolean.TRUE.equals(want.get("hasId"))) { if (!(body instanceof Map row)) throw new AssertionError(scenarioName + " / " + request.id() diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java index 773413c696..57dfd7d252 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java @@ -68,7 +68,10 @@ public Optional findById(Long id) { @Override public AccountDto create(AccountDto dto) { - AccountDto saved = new AccountDto(nextId.getAndIncrement(), dto.name(), dto.tokenHash(), dto.pinHash()); + // Naive adopter repository: trusts the id and every column it is handed, so a forwarded + // client id or readOnly `role` would be stored and visible to the scenarios. + Long id = dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()); + AccountDto saved = new AccountDto(id, dto.name(), dto.tokenHash(), dto.pinHash(), dto.role()); rows.add(saved); return saved; } @@ -81,7 +84,8 @@ public Optional update(Long id, AccountDto dto) { AccountDto merged = new AccountDto(id, dto.name() != null ? dto.name() : cur.name(), dto.tokenHash() != null ? dto.tokenHash() : cur.tokenHash(), - dto.pinHash() != null ? dto.pinHash() : cur.pinHash()); + dto.pinHash() != null ? dto.pinHash() : cur.pinHash(), + dto.role() != null ? dto.role() : cur.role()); rows.set(i, merged); return Optional.of(merged); } @@ -98,7 +102,8 @@ public Optional patch(Long id, AccountPatch patch) { AccountDto merged = new AccountDto(id, assigned.containsKey("name") ? (String) assigned.get("name") : cur.name(), assigned.containsKey("tokenHash") ? (String) assigned.get("tokenHash") : cur.tokenHash(), - assigned.containsKey("pinHash") ? (String) assigned.get("pinHash") : cur.pinHash()); + assigned.containsKey("pinHash") ? (String) assigned.get("pinHash") : cur.pinHash(), + assigned.containsKey("role") ? (String) assigned.get("role") : cur.role()); rows.set(i, merged); return Optional.of(merged); } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java index 7027381627..a4c01db44c 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java @@ -100,7 +100,11 @@ public AuthorDto create(AuthorDto dto) { // Issue #203 / ADR-0045: the generated controller already stamped @autoSet via // AuthorDto.stampForInsert(dto) before calling create — the repo just persists // the incoming (already-stamped) values verbatim. - AuthorDto saved = new AuthorDto(nextId.getAndIncrement(), dto.name(), dto.bio(), + // A naive adopter repository: it trusts the id it is handed and mints one only when + // absent, so a client id the controller forwarded would be stored (and the shared + // input-server-owned-dropped scenario would see it). + Long id = dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()); + AuthorDto saved = new AuthorDto(id, dto.name(), dto.bio(), dto.createdAt(), dto.issuedCurrency(), dto.autoCreatedAt(), dto.autoUpdatedAt()); rows.add(saved); return saved; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java index 4f540110eb..f8343cbb49 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java @@ -15,8 +15,8 @@ * *

The base {@code AuthDto} is the UNION of all subtype columns (the TPH-base DTO shape), so a * single record type backs both polymorphic and per-subtype rows. Discriminator scoping is a plain - * {@code type.equals(discriminator)} filter; a create injects the URL-supplied discriminator and - * never reads {@code dto.type()}.

+ * {@code type.equals(discriminator)} filter; a create stores whatever id/discriminator the DTO carries, + * falling back to the URL's discriminator.

*/ final class InMemoryAuthRepositorySource { @@ -100,7 +100,9 @@ public Optional findByIdAndType(Long id, String discriminator) { @Override public AuthDto createWithType(String discriminator, AuthDto dto) { - // Discriminator injected from the URL — dto.type() is intentionally ignored. + // A naive adopter repository: it trusts what it is handed (the id and the DTO's own + // discriminator when present, falling back to the URL's), so a value the controller + // forwarded from the request body is stored and the shared scenarios can see it. // `type` is a value-constrained enum (field.enum discriminator), so coerce the // String URL discriminator into the generated AuthType (its members ARE the // discriminator values). @@ -109,7 +111,8 @@ public AuthDto createWithType(String discriminator, AuthDto dto) { // AuthDto.stampForInsert(dto) before calling createWithType — the repo just // persists the incoming (already-stamped) autoCreatedAt/autoUpdatedAt verbatim. AuthDto saved = new AuthDto( - nextId.getAndIncrement(), AuthDto.AuthType.valueOf(discriminator), dto.reference(), + dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()), + dto.type() != null ? dto.type() : AuthDto.AuthType.valueOf(discriminator), dto.reference(), // FR-037 R1: writeOnce IS bound on create — settable exactly once. dto.issuedCurrency(), dto.autoCreatedAt(), dto.autoUpdatedAt(), diff --git a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md index 2cb454b8d7..1b300bfa6f 100644 --- a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md +++ b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md @@ -36,9 +36,9 @@ The generated router takes `dto: dict[str, Any]` for `POST` / `PATCH` / (`from . import , Patch`) and constructs them to enforce field constraints over HTTP — POST validates `(**dto)`, PATCH validates the all-optional `Patch(**dto)` (present values only) → `400 {"error":"validation"}`. -The repository seam still receives the original `dict` (preserving the FR-035 -present-key tristate + wire values), so the strong-typing simplification below -remains for the repo/response types; only the validation import was added. +The repository seam receives only the keys the input model validated, as a `dict` +(the FR-035 present-key tristate + wire values are kept), so the strong-typing +simplification below remains for the repo/response types. This keeps the router module decoupled from sibling generated files (the entity-model generator emits one `@dataclass`/Pydantic model per diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 175d3beff3..6323db2384 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -486,10 +486,9 @@ def _frozen_names(entity: MetaObject) -> list[str]: ADR-0045: the OUTERMOST generated write artifact enforces the mode, and for FastAPI that is the ROUTER, not the ``Patch`` model — the handler binds - ``dto`` (the raw body as a dict) and passes THAT to the repository, so a - field merely absent from the patch model still reaches the row. @autoSet gets - away with it because the router OVERWRITES its key server-side; a writeOnce - field has no server value to overwrite, so it must be popped explicitly.""" + ``dto`` as a dict. Since 1.2.1 the router hands the repository only the keys the + patch model validated (:func:`_validated_input_lines`), which already drops these; + the explicit pop stays as the FR-037 seam an owned router edits.""" return [ f.name for f in entity.fields() @@ -513,6 +512,49 @@ def _frozen_strip_lines(entity: MetaObject, indent: str = " ") -> list[str]: ] +#: 1.2.1 security fix. The request body cut down to what the input model validated, emitted +#: once per writable router. 1.2.0 validated the body and then handed the RAW body to the +#: repository, so a client-sent server-generated id, a ``readOnly`` field or a name the entity +#: does not declare (an ``isAdmin``) reached the write. Pydantic ignores a key a model does not +#: declare, and the input models leave out every field a client may not set, so keeping only +#: the keys the model validated (at every depth: a nested value object included) drops all of +#: them silently, as the TypeScript routes do. The values stay the body's own (decimal-exact) +#: values, so a repository receives the same value types as in 1.2.0. +_VALIDATED_INPUT_LINES: list[str] = [ + "def _validated_input(valid: BaseModel, raw: dict[str, Any]) -> dict[str, Any]:", + ' """GENERATED — *raw* cut down to the keys *valid* validated, at every depth. A key the', + " input model does not declare (an undeclared name, a server-generated id, a readOnly field)", + ' is dropped silently and never reaches the repository; values stay the body\'s own."""', + " return {_k: _validated_value(getattr(valid, _k), raw[_k]) for _k in valid.model_fields_set}", + "", + "", + "def _validated_value(valid: Any, raw: Any) -> Any:", + " if isinstance(valid, BaseModel) and isinstance(raw, dict):", + " return _validated_input(valid, raw)", + " if isinstance(valid, list) and isinstance(raw, list):", + " return [_validated_value(v, r) for v, r in zip(valid, raw)]", + " if isinstance(valid, dict) and isinstance(raw, dict):", + " return {k: _validated_value(v, raw[k]) for k, v in valid.items() if k in raw}", + " return raw", +] + + +def _validated_input_lines(model: str, *comment: str) -> list[str]: + """Validate ``dto`` against the input *model*, then rebind it to what the model validated + (see :data:`_VALIDATED_INPUT_LINES`): the repository never sees any other key.""" + return [ + *(f" # {line}" for line in comment), + " try:", + f" _valid = {model}(**dto)", + " except ValidationError:", + ' return JSONResponse(status_code=400, content={"error": "validation"})', + " # SECURITY (1.2.1): only what the input model validated reaches the repository. A key", + " # it does not declare (an undeclared name, a server-generated id, a readOnly or derived", + " # field, a frozen one on update) is dropped silently, never 400'd.", + " dto = _validated_input(_valid, dto)", + ] + + def _auto_set_split(entity: MetaObject) -> tuple[list[MetaField], list[MetaField]]: """The entity's ``@autoSet`` temporal fields split into ``(onCreate, onUpdate)``. @@ -1135,12 +1177,11 @@ def _emit_route_handler( ") -> Any:", *_DECIMAL_COERCE, *denied_strip, - " # FR-036: run the generated field constraints (length / pattern /", - " # numeric bounds) before persisting; a violation is the cross-port 400.", - " try:", - f" {model_name}(**dto)", - " except ValidationError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + *_validated_input_lines( + model_name, + "FR-036: run the generated field constraints (length / pattern /", + "numeric bounds) before persisting; a violation is the cross-port 400.", + ), # #203/ADR-0045: the generated router (the API surface) stamps @autoSet columns # ABOVE the consumer repo seam — onCreate + onUpdate both stamped from one instant # (a fresh row's createdAt == updatedAt), ignoring any caller-supplied value. @@ -1166,12 +1207,11 @@ def _emit_route_handler( " for _k in _REQUIRED_FIELDS:", " if _k in dto and dto[_k] is None:", ' return JSONResponse(status_code=400, content={"error": "validation"})', - " # FR-036: validate PRESENT, non-null values with the create rules (the", - " # all-optional patch model never fires required-checks on ABSENT fields).", - " try:", - f" {patch_model}(**dto)", - " except ValidationError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + *_validated_input_lines( + patch_model, + "FR-036: validate PRESENT, non-null values with the create rules (the", + "all-optional patch model never fires required-checks on ABSENT fields).", + ), # #203/ADR-0045: bump onUpdate @autoSet columns (server-owned) on EVERY patch, # above the repo seam — a server-inserted present key (compatible with the PATCH # present-key tristate); onCreate columns are never touched here. @@ -1521,6 +1561,9 @@ def _render_tph_router( parts.extend(_JSON_BODY_LINES) parts.append("") parts.append("") + parts.extend(_VALIDATED_INPUT_LINES) + parts.append("") + parts.append("") # FR-037 R1 — the names a PATCH must never write. Emitted even when EMPTY so # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). @@ -1600,11 +1643,13 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(") -> Any:") parts.extend(_DECIMAL_COERCE) parts.extend(denied_strip_tph) - parts.append(" # FR-036: run the subtype's field constraints before persisting.") - parts.append(" try:") - parts.append(f" {sub}Create(**dto)") - parts.append(" except ValidationError:") - parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') + # The route's subtype is the only source of the discriminator: a body value is + # dropped silently before validation (as the TypeScript route's `.omit` does), so + # a foreign tag is never a 400 and never reaches the repository. + parts.append(f' dto.pop("{plan.discriminator_field}", None)') + parts.extend(_validated_input_lines( + f"{sub}Create", "FR-036: run the subtype's field constraints before persisting." + )) # #203/ADR-0045: stamp @autoSet columns ABOVE the repo seam (empty for a # non-@autoSet hierarchy — byte-identical output). parts.extend(create_autoset) @@ -1637,11 +1682,9 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(" for _k in _REQUIRED_FIELDS:") parts.append(" if _k in dto and dto[_k] is None:") parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') - parts.append(" # FR-036: validate PRESENT, non-null values with the subtype's create rules.") - parts.append(" try:") - parts.append(f" {sub}Patch(**dto)") - parts.append(" except ValidationError:") - parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') + parts.extend(_validated_input_lines( + f"{sub}Patch", "FR-036: validate PRESENT, non-null values with the subtype's create rules." + )) # #203/ADR-0045: pop the write-once onCreate column(s), bump onUpdate — ABOVE # the repo seam (empty for a non-@autoSet hierarchy — byte-identical output). parts.extend(update_autoset) @@ -1911,6 +1954,9 @@ def render_router( parts.extend(_JSON_BODY_LINES) parts.append("") parts.append("") + parts.extend(_VALIDATED_INPUT_LINES) + parts.append("") + parts.append("") # FR-037 R1 — the names a PATCH must never write. Emitted even when EMPTY so # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). @@ -1946,8 +1992,8 @@ def render_router( create_autoset=create_autoset, update_autoset=update_autoset, # FR-037 R1 — ADR-0045: the ROUTER is the outermost write artifact for - # FastAPI (the handler binds the RAW body dict and hands THAT to the - # repository), so the strip has to live here, not in Patch. + # FastAPI (the handler binds the body as a dict), so the strip lives here, + # not in Patch. frozen_strip=_frozen_strip_lines(entity), denied_strip=_denied_strip_lines(denied_names), ) diff --git a/server/python/tests/integration/api_contract_assertions.py b/server/python/tests/integration/api_contract_assertions.py index 11a6f33445..4d700250b9 100644 --- a/server/python/tests/integration/api_contract_assertions.py +++ b/server/python/tests/integration/api_contract_assertions.py @@ -202,6 +202,19 @@ def assert_response( f"but both = {a_value!r}" ) + if "absent" in expect_body: + # The listed keys must not be on the object at all (not even as null): a key the + # client may not set never reached storage and so never comes back in the echo. + if not isinstance(body, dict): + raise AssertionError( + f"{scenario_name} / {request_id}: expected object for absent, got: {body!r}" + ) + present = [k for k in expect_body["absent"] if k in body] + if present: + raise AssertionError( + f"{scenario_name} / {request_id}: expected no {present!r} in body, got: {body!r}" + ) + if expect_body.get("hasId") is True: if not isinstance(body, dict): raise AssertionError( diff --git a/server/python/tests/integration/generated_secret_fields_app.py b/server/python/tests/integration/generated_secret_fields_app.py index 61d9eb9a4e..3a6a1f2f1c 100644 --- a/server/python/tests/integration/generated_secret_fields_app.py +++ b/server/python/tests/integration/generated_secret_fields_app.py @@ -28,7 +28,7 @@ from .corpus_config import attach_corpus_wire from .generated_router_app import InMemoryAuthorRepository -COLUMNS = ("id", "name", "tokenHash", "pinHash") +COLUMNS = ("id", "name", "tokenHash", "pinHash", "role") def load_account(meta_json: Path) -> MetaObject: diff --git a/server/python/tests/integration/test_generated_router_validated_input.py b/server/python/tests/integration/test_generated_router_validated_input.py new file mode 100644 index 0000000000..b43ed21c08 --- /dev/null +++ b/server/python/tests/integration/test_generated_router_validated_input.py @@ -0,0 +1,272 @@ +"""1.2.1 security fix: a generated create / update hands the repository ONLY the validated input. + +The 1.2.0 router validated the request body against ``Create`` / ``Patch`` and +then passed the RAW body to the repository. Pydantic ignores a key a model does not declare, and +the input models leave out the server-owned fields, so a client-sent ``id``, a ``readOnly`` field +and a made-up ``isAdmin`` all reached the write. Now the repository receives only what the input +model validated, at every depth, as with the TypeScript routes: a name the entity does not +declare, a server-generated id, a ``readOnly`` field and the id on update are all dropped +silently. None of them is a 400: a client that sends an extra harmless field keeps working. + +The model is generated with the real Python codegen (``run_gen``: entity + filter-allowlist + +routes), the emitted package is imported unmodified, and the only hand-written piece is a +repository that records what it was handed. No database: the router is the artifact, as in the +other generated-router lanes. +""" +from __future__ import annotations + +import importlib +import json +import sys +import tempfile +import uuid +from pathlib import Path +from typing import Any + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient + +from metaobjects import MetaDataLoader +from metaobjects.codegen.config import GenConfig +from metaobjects.codegen.generators.entity_model import entity_model +from metaobjects.codegen.generators.filter_allowlist_generator import filter_allowlist_generator +from metaobjects.codegen.generators.router_generator import router_generator +from metaobjects.codegen.runner import run_gen + +from .generated_tph_app import build_generated_tph_app + +_META = { + "metadata.root": { + "package": "acme::secfix", + "children": [ + {"object.value": { + "name": "Address", + "children": [ + {"field.string": {"name": "city", "@maxLength": 40}}, + ], + }}, + {"object.entity": { + "name": "Member", + "children": [ + {"source.rdb": {"@table": "members"}}, + {"field.long": {"name": "id"}}, + {"field.string": {"name": "name", "@required": True, "@maxLength": 80}}, + {"field.string": {"name": "role", "@maxLength": 20, "@mutability": "readOnly"}}, + {"field.string": {"name": "tenant", "@maxLength": 20, "@mutability": "writeOnce"}}, + {"field.object": {"name": "address", "@objectRef": "Address", "@storage": "jsonb"}}, + {"field.timestamp": {"name": "createdAt", "@autoSet": "onCreate"}}, + {"identity.primary": {"name": "id", "@fields": "id", "@generation": "increment"}}, + ], + }}, + {"object.entity": { + "name": "Account", + "children": [ + {"source.rdb": {"@table": "accounts"}}, + {"field.string": {"name": "code", "@maxLength": 20}}, + {"field.string": {"name": "label", "@maxLength": 80}}, + {"identity.primary": {"name": "code", "@fields": "code"}}, + ], + }}, + ], + } +} + + +class _RecordingRepo: + """The generated repository seam, recording every write it is handed.""" + + def __init__(self, pk: str) -> None: + self.pk = pk + self.writes: list[tuple[str, dict[str, Any]]] = [] + + def create(self, dto: Any) -> Any: + self.writes.append(("create", dict(dto))) + return {self.pk: 1, **dto} + + def update(self, id: Any, dto: Any) -> Any | None: + self.writes.append(("update", dict(dto))) + return {self.pk: id, **dto} + + +def _generate_package() -> str: + """Run the real Python codegen for ``_META`` into a temp package; return its name.""" + meta_dir = Path(tempfile.mkdtemp(prefix="secfix-meta-")) + (meta_dir / "meta.json").write_text(json.dumps(_META)) + loaded = MetaDataLoader.from_directory(str(meta_dir)) + assert not loaded.errors, "; ".join(f"{e.code}: {e.message}" for e in loaded.errors) + pkg = f"secfix_{uuid.uuid4().hex[:8]}" + root = Path(tempfile.mkdtemp(prefix="secfix-gen-")) + run_gen( + GenConfig(out_dir=str(root / pkg)), + loaded.root, + generators=[entity_model(), filter_allowlist_generator(), router_generator()], + ) + sys.path.insert(0, str(root)) + return pkg + + +_PKG = _generate_package() + + +def _client(router_module: str, pk: str) -> tuple[TestClient, _RecordingRepo, str]: + mod = importlib.import_module(f"{_PKG}.{router_module}") + repo = _RecordingRepo(pk) + app = FastAPI() + app.include_router(mod.router) + app.dependency_overrides[mod.get_repository] = lambda: repo + return TestClient(app), repo, mod.router.prefix + + +@pytest.fixture() +def member() -> tuple[TestClient, _RecordingRepo, str]: + return _client("member_router", "id") + + +@pytest.fixture() +def account() -> tuple[TestClient, _RecordingRepo, str]: + return _client("account_router", "code") + + +# --- the reported defect: a client-sent id and a made-up isAdmin reach the repository ------- + + +def test_create_never_hands_the_repository_a_client_id_or_an_undeclared_field(member) -> None: + client, repo, base = member + resp = client.post(base, json={"id": 999, "name": "Mallory", "isAdmin": True}) + assert resp.status_code == 201, resp.text + [(op, written)] = repo.writes + assert op == "create" + assert set(written) == {"name", "createdAt"} # createdAt is the router's own stamp + assert written["name"] == "Mallory" + + +@pytest.mark.parametrize("method", ["PATCH", "PUT"]) +def test_update_never_hands_the_repository_an_undeclared_field(member, method: str) -> None: + client, repo, base = member + resp = client.request(method, f"{base}/1", json={"name": "M", "isAdmin": True}) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"name": "M"})] + + +def test_an_undeclared_key_inside_a_value_object_never_reaches_the_repository(member) -> None: + client, repo, base = member + resp = client.post(base, json={"name": "Mallory", "address": {"city": "Oslo", "isAdmin": True}}) + assert resp.status_code == 201, resp.text + [(_op, written)] = repo.writes + assert written["address"] == {"city": "Oslo"} + + +def test_a_malformed_value_on_a_settable_field_is_still_a_400(member) -> None: + client, repo, base = member + resp = client.post(base, json={"name": "x" * 81, "isAdmin": True}) + assert resp.status_code == 400, resp.text + assert resp.json() == {"error": "validation"} + assert repo.writes == [] + + +# --- declared but not settable: dropped, never 400'd, and never handed to the repository ---- + + +def test_create_strips_the_client_id_and_a_read_only_field(member) -> None: + client, repo, base = member + resp = client.post( + base, + json={"id": 999, "name": "Mallory", "role": "admin", "tenant": "t1", + "address": {"city": "Oslo"}}, + ) + assert resp.status_code == 201, resp.text + [(op, written)] = repo.writes + assert op == "create" + # The server generates the id and owns `role`; `createdAt` is the router's own stamp. + assert set(written) == {"name", "tenant", "address", "createdAt"} + assert written["name"] == "Mallory" + assert written["tenant"] == "t1" # writeOnce is settable on create + assert written["address"] == {"city": "Oslo"} + + +@pytest.mark.parametrize("method", ["PATCH", "PUT"]) +def test_update_never_hands_the_repository_the_id_or_a_frozen_field(member, method: str) -> None: + client, repo, base = member + resp = client.request( + method, + f"{base}/1", + json={"id": 999, "name": "M2", "role": "admin", "tenant": "t2", "createdAt": "2000-01-01T00:00:00Z"}, + ) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"name": "M2"})] + + +def test_update_passes_a_present_null_through_as_a_clear(member) -> None: + """The PATCH present-key tristate is unchanged: an explicit null on a nullable field clears it.""" + client, repo, base = member + resp = client.patch(f"{base}/1", json={"address": None}) + assert resp.status_code == 200, resp.text + assert repo.writes == [("update", {"address": None})] + + +def test_an_assigned_key_is_still_the_callers_on_create_and_route_owned_on_update(account) -> None: + client, repo, base = account + created = client.post(base, json={"code": "ACME", "label": "Acme"}) + assert created.status_code == 201, created.text + updated = client.patch(f"{base}/ACME", json={"code": "EVIL", "label": "Acme 2"}) + assert updated.status_code == 200, updated.text + assert repo.writes == [ + ("create", {"code": "ACME", "label": "Acme"}), + ("update", {"label": "Acme 2"}), + ] + + +# --- TPH: the per-subtype create / update handlers are a separate code path ----------------- + + +def _find_tph_dir() -> Path: + cur = Path(__file__).resolve() + while cur != cur.parent: + candidate = cur / "fixtures" / "api-contract-conformance" / "tph" + if candidate.is_dir(): + return candidate + cur = cur.parent + raise RuntimeError("fixtures/api-contract-conformance/tph not found") + + +@pytest.fixture() +def tph() -> tuple[TestClient, list[tuple[str, str, dict[str, Any]]]]: + app, repo = build_generated_tph_app(_find_tph_dir()) + writes: list[tuple[str, str, dict[str, Any]]] = [] + create, update = repo.create, repo.update + + def recording_create(subtype: str, dto: Any) -> Any: + writes.append(("create", subtype, dict(dto))) + return create(subtype, dto) + + def recording_update(subtype: str, id: int, dto: Any) -> Any: + writes.append(("update", subtype, dict(dto))) + return update(subtype, id, dto) + + repo.create, repo.update = recording_create, recording_update + return TestClient(app), writes + + +def test_tph_subtype_create_drops_the_client_id_and_an_undeclared_field(tph) -> None: + client, writes = tph + resp = client.post( + "/api/auths/bridge", json={"id": 999, "reference": "R-1", "quantity": 2, "isAdmin": True} + ) + assert resp.status_code == 201, resp.text + [(op, subtype, written)] = writes + assert (op, subtype) == ("create", "Bridge") + assert "id" not in written + assert "isAdmin" not in written + assert {"reference", "quantity"} <= set(written) + + +def test_tph_subtype_update_drops_the_id_and_an_undeclared_field(tph) -> None: + client, writes = tph + # The store is unseeded, so the answer is a 404; what matters is what the write was handed. + client.patch("/api/auths/bridge/1", json={"id": 999, "quantity": 3, "isAdmin": True}) + [(op, subtype, written)] = writes + assert (op, subtype) == ("update", "Bridge") + assert "id" not in written + assert "isAdmin" not in written + assert written["quantity"] == 3 diff --git a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts index 714f11d277..4035e3f9c4 100644 --- a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts +++ b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts @@ -719,6 +719,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of flatSurfaceFields(obj)) { @@ -734,6 +735,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // A primary-key field is the row's identity, which the URL names: an ASSIGNED key is + // omitted from the update shape exactly like a generated one, so a client-sent id is + // stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -818,6 +823,9 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // The row's identity is the URL's: an ASSIGNED primary key leaves the update shape too + // (a generated one never entered it), so a client-sent id is stripped, never a re-key. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { diff --git a/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap b/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap index 52ec2cdb9c..2ed61838bc 100644 --- a/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap +++ b/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap @@ -205,36 +205,36 @@ Generated API reference for this project; call these exactly as written. Imports \`import { Program, ProgramInsertSchema, ProgramUpdateSchema } from "./Program"\` - \`interface Program { id: number; title?: string; createdAt?: string }\` — The typed shape of a Program row, generated from its metadata. - \`ProgramInsertSchema: ZodType<{ id: number; title?: string; createdAt?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`ProgramUpdateSchema: ZodType<{ id?: number; title?: string; createdAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`ProgramUpdateSchema: ZodType<{ title?: string; createdAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). \`import { findProgramById, listPrograms, createProgram, updateProgram, deleteProgramById } from "./Program.queries"\` - \`findProgramById(db: Db, id: number): Promise\` — Fetch a single Program by its primary key; null when not found. - \`listPrograms(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List Program rows with optional limit/offset paging. - \`createProgram(db: Db, data: { id: number; title?: string; createdAt?: string }): Promise\` — Validate (via ProgramInsertSchema) and insert a new Program. [throws: ZodError when data fails ProgramInsertSchema validation.] -- \`updateProgram(db: Db, id: number, patch: { id?: number; title?: string; createdAt?: string }): Promise\` — Partially update a Program by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails ProgramUpdateSchema validation.] +- \`updateProgram(db: Db, id: number, patch: { title?: string; createdAt?: string }): Promise\` — Partially update a Program by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails ProgramUpdateSchema validation.] - \`deleteProgramById(db: Db, id: number): Promise\` — Delete a Program by primary key; true when a row was removed. \`import { programRoutes } from "./Program.routes"\` - \`GET /api/programs -> { id: number; title?: string; createdAt?: string }\` — List Program (supports filter/sort/paging query params). - \`GET /api/programs/:id -> { id: number; title?: string; createdAt?: string }\` — Fetch a single Program by id (404 when not found). - \`POST /api/programs body: { id: number; title?: string; createdAt?: string }\` — Create a Program (body validated by ProgramInsertSchema). -- \`PATCH /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — Partially update a Program by id (body validated by ProgramUpdateSchema). -- \`PUT /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). +- \`PATCH /api/programs/:id body: { title?: string; createdAt?: string }\` — Partially update a Program by id (body validated by ProgramUpdateSchema). +- \`PUT /api/programs/:id body: { title?: string; createdAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). - \`DELETE /api/programs/:id\` — Delete a Program by id. \`import { registerProgramRoutes } from "./Program.routes.hono"\` - \`GET /api/programs -> { id: number; title?: string; createdAt?: string }\` — [Hono] List Program (filter/sort/paging query params). - \`GET /api/programs/:id -> { id: number; title?: string; createdAt?: string }\` — [Hono] Fetch a single Program by id (404 when not found). - \`POST /api/programs body: { id: number; title?: string; createdAt?: string }\` — [Hono] Create a Program (body validated by ProgramInsertSchema). -- \`PATCH /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — [Hono] Partially update a Program by id (body validated by ProgramUpdateSchema). -- \`PUT /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). +- \`PATCH /api/programs/:id body: { title?: string; createdAt?: string }\` — [Hono] Partially update a Program by id (body validated by ProgramUpdateSchema). +- \`PUT /api/programs/:id body: { title?: string; createdAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). - \`DELETE /api/programs/:id\` — [Hono] Delete a Program by id. Example: \`\`\`ts const created = await createProgram(db, { id: 1 }); const found = await findProgramById(db, created.id); -const updated = await updateProgram(db, created.id, { id: 1 }); +const updated = await updateProgram(db, created.id, { title: "…" }); const removed = await deleteProgramById(db, created.id); \`\`\` @@ -243,37 +243,37 @@ const removed = await deleteProgramById(db, created.id); \`import { Purchase, PurchaseInsertSchema, PurchaseUpdateSchema, purchasesRelations } from "./Purchase"\` - \`interface Purchase { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — The typed shape of a Purchase row, generated from its metadata. - \`PurchaseInsertSchema: ZodType<{ id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`PurchaseUpdateSchema: ZodType<{ id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`PurchaseUpdateSchema: ZodType<{ programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). - \`const purchasesRelations { program?: Program (1:1 / N:1) }\` — Drizzle relations() for Purchase — register it with your schema, then traverse via the relational query API (db.query.purchases.findMany({ with: { … } })). \`import { findPurchaseById, listPurchases, createPurchase, updatePurchase, deletePurchaseById } from "./Purchase.queries"\` - \`findPurchaseById(db: Db, id: number): Promise\` — Fetch a single Purchase by its primary key; null when not found. - \`listPurchases(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List Purchase rows with optional limit/offset paging. - \`createPurchase(db: Db, data: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Validate (via PurchaseInsertSchema) and insert a new Purchase. [throws: ZodError when data fails PurchaseInsertSchema validation.] -- \`updatePurchase(db: Db, id: number, patch: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Partially update a Purchase by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails PurchaseUpdateSchema validation.] +- \`updatePurchase(db: Db, id: number, patch: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Partially update a Purchase by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails PurchaseUpdateSchema validation.] - \`deletePurchaseById(db: Db, id: number): Promise\` — Delete a Purchase by primary key; true when a row was removed. \`import { purchaseRoutes } from "./Purchase.routes"\` - \`GET /api/purchases -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — List Purchase (supports filter/sort/paging query params). - \`GET /api/purchases/:id -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Fetch a single Purchase by id (404 when not found). - \`POST /api/purchases body: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Create a Purchase (body validated by PurchaseInsertSchema). -- \`PATCH /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Partially update a Purchase by id (body validated by PurchaseUpdateSchema). -- \`PUT /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). +- \`PATCH /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Partially update a Purchase by id (body validated by PurchaseUpdateSchema). +- \`PUT /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). - \`DELETE /api/purchases/:id\` — Delete a Purchase by id. \`import { registerPurchaseRoutes } from "./Purchase.routes.hono"\` - \`GET /api/purchases -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] List Purchase (filter/sort/paging query params). - \`GET /api/purchases/:id -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Fetch a single Purchase by id (404 when not found). - \`POST /api/purchases body: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Create a Purchase (body validated by PurchaseInsertSchema). -- \`PATCH /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Partially update a Purchase by id (body validated by PurchaseUpdateSchema). -- \`PUT /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). +- \`PATCH /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Partially update a Purchase by id (body validated by PurchaseUpdateSchema). +- \`PUT /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). - \`DELETE /api/purchases/:id\` — [Hono] Delete a Purchase by id. Example: \`\`\`ts const created = await createPurchase(db, { id: 1 }); const found = await findPurchaseById(db, created.id); -const updated = await updatePurchase(db, created.id, { id: 1 }); +const updated = await updatePurchase(db, created.id, { programId: 1 }); const removed = await deletePurchaseById(db, created.id); \`\`\` @@ -282,36 +282,36 @@ const removed = await deletePurchaseById(db, created.id); \`import { WorkoutEvent, WorkoutEventInsertSchema, WorkoutEventUpdateSchema } from "./WorkoutEvent"\` - \`interface WorkoutEvent { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — The typed shape of a WorkoutEvent row, generated from its metadata. - \`WorkoutEventInsertSchema: ZodType<{ id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`WorkoutEventUpdateSchema: ZodType<{ id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`WorkoutEventUpdateSchema: ZodType<{ programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). \`import { findWorkoutEventById, listWorkoutEvents, createWorkoutEvent, updateWorkoutEvent, deleteWorkoutEventById } from "./WorkoutEvent.queries"\` - \`findWorkoutEventById(db: Db, id: number): Promise\` — Fetch a single WorkoutEvent by its primary key; null when not found. - \`listWorkoutEvents(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List WorkoutEvent rows with optional limit/offset paging. - \`createWorkoutEvent(db: Db, data: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Validate (via WorkoutEventInsertSchema) and insert a new WorkoutEvent. [throws: ZodError when data fails WorkoutEventInsertSchema validation.] -- \`updateWorkoutEvent(db: Db, id: number, patch: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Partially update a WorkoutEvent by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails WorkoutEventUpdateSchema validation.] +- \`updateWorkoutEvent(db: Db, id: number, patch: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Partially update a WorkoutEvent by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails WorkoutEventUpdateSchema validation.] - \`deleteWorkoutEventById(db: Db, id: number): Promise\` — Delete a WorkoutEvent by primary key; true when a row was removed. \`import { workoutEventRoutes } from "./WorkoutEvent.routes"\` - \`GET /api/workout_events -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — List WorkoutEvent (supports filter/sort/paging query params). - \`GET /api/workout_events/:id -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Fetch a single WorkoutEvent by id (404 when not found). - \`POST /api/workout_events body: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Create a WorkoutEvent (body validated by WorkoutEventInsertSchema). -- \`PATCH /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). -- \`PUT /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). +- \`PATCH /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). +- \`PUT /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). - \`DELETE /api/workout_events/:id\` — Delete a WorkoutEvent by id. \`import { registerWorkoutEventRoutes } from "./WorkoutEvent.routes.hono"\` - \`GET /api/workout_events -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] List WorkoutEvent (filter/sort/paging query params). - \`GET /api/workout_events/:id -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Fetch a single WorkoutEvent by id (404 when not found). - \`POST /api/workout_events body: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Create a WorkoutEvent (body validated by WorkoutEventInsertSchema). -- \`PATCH /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). -- \`PUT /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). +- \`PATCH /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). +- \`PUT /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). - \`DELETE /api/workout_events/:id\` — [Hono] Delete a WorkoutEvent by id. Example: \`\`\`ts const created = await createWorkoutEvent(db, { id: 1 }); const found = await findWorkoutEventById(db, created.id); -const updated = await updateWorkoutEvent(db, created.id, { id: 1 }); +const updated = await updateWorkoutEvent(db, created.id, { programId: 1 }); const removed = await deleteWorkoutEventById(db, created.id); \`\`\` " @@ -344,7 +344,7 @@ import { createProgram, findProgramById, updateProgram, deleteProgramById } from const created = await createProgram(db, { id: 1 }); const found = await findProgramById(db, created.id); -const updated = await updateProgram(db, created.id, { id: 1 }); +const updated = await updateProgram(db, created.id, { title: "…" }); const removed = await deleteProgramById(db, created.id); \`\`\` @@ -414,7 +414,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -496,7 +495,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -514,7 +512,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -598,7 +595,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -616,7 +612,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -662,7 +657,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | " @@ -695,7 +689,7 @@ import { createPurchase, findPurchaseById, updatePurchase, deletePurchaseById } const created = await createPurchase(db, { id: 1 }); const found = await findPurchaseById(db, created.id); -const updated = await updatePurchase(db, created.id, { id: 1 }); +const updated = await updatePurchase(db, created.id, { programId: 1 }); const removed = await deletePurchaseById(db, created.id); \`\`\` @@ -791,7 +785,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -893,7 +886,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -916,7 +908,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1020,7 +1011,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1043,7 +1033,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1099,7 +1088,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1151,7 +1139,7 @@ import { createWorkoutEvent, findWorkoutEventById, updateWorkoutEvent, deleteWor const created = await createWorkoutEvent(db, { id: 1 }); const found = await findWorkoutEventById(db, created.id); -const updated = await updateWorkoutEvent(db, created.id, { id: 1 }); +const updated = await updateWorkoutEvent(db, created.id, { programId: 1 }); const removed = await deleteWorkoutEventById(db, created.id); \`\`\` @@ -1229,7 +1217,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1327,7 +1314,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1349,7 +1335,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1449,7 +1434,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1471,7 +1455,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1525,7 +1508,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | diff --git a/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts b/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts index 7a8f6ff382..3fd6e9be60 100644 --- a/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts +++ b/server/typescript/packages/codegen-ts/test/contract/contract-behaviours.test.ts @@ -255,7 +255,7 @@ describe("the three shapes", () => { const tree = await contractTree(corpusCase("secret-fields")); const keysOf = (path: string): string[] => Object.keys(jsonAt(tree, path)["properties"] as JsonDoc); const account = "schemas/acme/secrets/Account"; - expect(keysOf(`${account}.schema.json`)).toEqual(["id", "name", "tokenHash"]); + expect(keysOf(`${account}.schema.json`)).toEqual(["id", "name", "tokenHash", "role"]); expect(keysOf(`${account}.create.schema.json`)).toEqual(["name"]); expect(keysOf(`${account}.update.schema.json`)).toEqual(["name"]); expect(tree.get("openapi.json")).not.toContain("pinHash"); diff --git a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts index 0082254cd4..4ad3748b3e 100644 --- a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts +++ b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts @@ -356,6 +356,22 @@ describe("renderZodValidators", () => { expect(update).toContain("name"); }); + test("an ASSIGNED primary key is in the Insert schema and OUT of the Update schema", () => { + // The URL names the row on PUT/PATCH. An assigned key (no @generation) is the caller's on + // create, but leaving it in the update shape let a client-sent `code` re-key the row. + const coupon = metaObject(OBJECT_SUBTYPE_ENTITY, "Coupon"); + coupon.addChild(metaField(FIELD_SUBTYPE_STRING, "code")); + coupon.addChild(metaField(FIELD_SUBTYPE_STRING, "label")); + const primary = meta(new TypeId(TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY), "primary"); + primary.setAttr("fields", ["code"]); + coupon.addChild(primary); + const out = renderZodValidators(coupon).toString(); + expect(schemaBody(out, "CouponInsertSchema")).toContain("code"); + const update = schemaBody(out, "CouponUpdateSchema"); + expect(update).not.toContain("code"); + expect(update).toContain("label"); + }); + test("FR-037 R1: no @mutability (⇒ readWrite) is in BOTH schemas — the no-churn default", () => { const out = customerWithMutability(undefined, "lifetimeValue"); expect(schemaBody(out, "CustomerInsertSchema")).toContain("lifetimeValue"); diff --git a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts index ba4019d47d..9c65eb2be0 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts @@ -26,6 +26,10 @@ export interface ExpectBody { namesUnordered?: string[]; row?: Record; hasId?: boolean; + /** The response body is an object and NONE of the listed keys is present on it (not even as + * `null`). Used by the input scenarios to prove a key the client may not set never reached + * storage and came back in the echo. */ + absent?: string[]; /** Assert the listed RESPONSE fields all equal EACH OTHER (never a literal) — * format-agnostic. Used by @autoSet scenarios to prove two stamped columns * hold the same value without pinning the exact timestamp. */ @@ -288,6 +292,13 @@ export function assertResponse( if (typeof row["id"] !== "number") throw new Error(`${scenarioName} / ${request.id}: expected numeric id in body, got: ${stringify(body)}`); } + if (want.absent) { + if (body === null || typeof body !== "object" || Array.isArray(body)) + throw new Error(`${scenarioName} / ${request.id}: expected object for absent, got: ${stringify(body)}`); + const present = want.absent.filter((k) => Object.hasOwn(body, k)); + if (present.length > 0) + throw new Error(`${scenarioName} / ${request.id}: expected no ${present.join(", ")} in body, got: ${stringify(body)}`); + } if (want.fieldsEqual) { const row = body as Record; // Compare the raw response fields to EACH OTHER (format-agnostic) — every diff --git a/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts index 59885dde10..dbf642ed62 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts @@ -65,7 +65,7 @@ const READ_PATHS = { }; export interface SecretFieldsSeed { - rows: Array<{ id: number; name: string; tokenHash: string; pinHash: string }>; + rows: Array<{ id: number; name: string; tokenHash: string; pinHash: string; role: string }>; } export interface GeneratedSecretFieldsServerHandle { @@ -124,7 +124,8 @@ export const db = drizzle(pool); "id" bigserial PRIMARY KEY, "name" varchar(100) NOT NULL, "token_hash" varchar(100), - "pin_hash" varchar(100) + "pin_hash" varchar(100), + "role" varchar(20) ); CREATE OR REPLACE VIEW "v_account" AS SELECT "id", "name", "pin_hash" FROM "accounts"; CREATE TABLE IF NOT EXISTS "payers" ("id" bigserial PRIMARY KEY); @@ -165,7 +166,7 @@ export const db = drizzle(pool); for (const r of seed.rows) await executeSql( connectionUri, - `INSERT INTO "accounts" ("id","name","token_hash","pin_hash") VALUES (${r.id}, ${str(r.name)}, ${str(r.tokenHash)}, ${str(r.pinHash)})`, + `INSERT INTO "accounts" ("id","name","token_hash","pin_hash","role") VALUES (${r.id}, ${str(r.name)}, ${str(r.tokenHash)}, ${str(r.pinHash)}, ${str(r.role)})`, ); // The seed names ids explicitly; keep the sequence ahead of them for the POST scenarios. await executeSql(connectionUri, `SELECT setval(pg_get_serial_sequence('accounts','id'), (SELECT MAX("id") FROM "accounts"));`); diff --git a/server/typescript/packages/integration-tests/src/api-contract-server.ts b/server/typescript/packages/integration-tests/src/api-contract-server.ts index 32cc2b3661..4342322804 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-server.ts @@ -188,6 +188,16 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis return reply; }); + // The keys a client may write, as the generated routes' Insert/Update schemas have them: the + // server-generated `id`, the @autoSet columns and any key the entity does not declare are + // DROPPED silently (never 400'd), and `issuedCurrency` (writeOnce) is settable on create only. + const CREATE_KEYS = ["name", "bio", "createdAt", "issuedCurrency"] as const; + const UPDATE_KEYS = ["name", "bio", "createdAt"] as const; + const pick = (body: unknown, keys: readonly string[]): Record => { + const src = (body ?? {}) as Record; + return Object.fromEntries(keys.filter((k) => Object.hasOwn(src, k)).map((k) => [k, src[k]])); + }; + // POST /api/authors fastify.post(ROUTE_BASE, async (req, reply) => { // #203 / ADR-0045 — the TS runtime ObjectManager does NOT stamp @autoSet @@ -195,7 +205,7 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // the generated lane's InsertSchema. Both onCreate + onUpdate columns get the // SAME captured now() on create; any caller-supplied value is ignored. const now = new Date().toISOString(); - const body = { ...(req.body as Record), autoCreatedAt: now, autoUpdatedAt: now }; + const body = { ...pick(req.body, CREATE_KEYS), autoCreatedAt: now, autoUpdatedAt: now }; const row = await om.create(ENTITY, body); reply.code(201).send(row); return reply; @@ -215,12 +225,7 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // plain z.object() discards unknown keys). Dropping it also covers the // present-NULL arm for free: clearing is a write, so an explicit null must // not reach the column either. - const { - autoCreatedAt: _dropOnCreate, - issuedCurrency: _dropWriteOnce, - ...rest - } = (req.body as Record) ?? {}; - const body = { ...rest, autoUpdatedAt: new Date().toISOString() }; + const body = { ...pick(req.body, UPDATE_KEYS), autoUpdatedAt: new Date().toISOString() }; // A ValidationError (e.g. FR-035 present-null on a @required field) propagates // to the shared setErrorHandler above → 400 {error:"validation"}. const row = await om.update(ENTITY, Number(id), body, { ifMissing: "ignore" }); diff --git a/server/typescript/packages/test-generators/src/templates/zod-validators.ts b/server/typescript/packages/test-generators/src/templates/zod-validators.ts index 2e421a0819..3d0a9e52fd 100644 --- a/server/typescript/packages/test-generators/src/templates/zod-validators.ts +++ b/server/typescript/packages/test-generators/src/templates/zod-validators.ts @@ -719,6 +719,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of flatSurfaceFields(obj)) { @@ -734,6 +735,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // A primary-key field is the row's identity, which the URL names: an ASSIGNED key is + // omitted from the update shape exactly like a generated one, so a client-sent id is + // stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -818,6 +823,9 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // The row's identity is the URL's: an ASSIGNED primary key leaves the update shape too + // (a generated one never entered it), so a client-sent id is stripped, never a re-key. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { From fc9145aeb8f898e231688c0979bb2d6373e27d17 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 13:34:13 -0500 Subject: [PATCH 03/31] feat(codegen): a validation 400 lists every issue in every port The same model answered a failed create or update differently by language: TypeScript listed each problem, while C#, Java, Kotlin and Python returned a bare {"error":"validation"}. Every port's generated routes and its hand-rolled reference servers now answer 400 {"error":"validation","issues":[{code,path, message}]} with every failure, codes in Zod's vocabulary and path as an array of wire names. - C#: ValidationIssues runtime file (ejected with routes); create reads the body itself so an unbindable body lists its issue; PATCH collects instead of returning on the first failure. - Java: ValidationIssues runtime helper; a Patch binds every field before it throws. Kotlin: per-package MetaValidationIssues.kt; create lists every absent required value. - Python: every Pydantic error mapped in constraint_errors; decimal failures carry their wire path. Shared scenarios validation-lists-issues and tph-validation-lists-issues, with a new `issues` assertion in every runner, gate it in both lanes. --- docs/features/field-types.md | 2 +- docs/features/generated-mutations.md | 2 +- docs/features/own-your-codegen.md | 10 +- .../.metaobjects/.gen-state/.hashes.json | 4 +- .../generated/csharp/SubscriberRoutes.g.cs | 36 +-- .../generated/java/acme/SubscriberPatch.java | 10 +- .../generated/python/subscriber_router.py | 47 ++- examples/showcase/site-payload.json | 2 +- .../scenarios/validation-lists-issues.yaml | 51 ++++ .../tph-validation-lists-issues.yaml | 47 +++ .../EntityGeneratorTests.cs | 6 +- .../MapFieldCodegenTests.cs | 8 +- .../ValidationIssuesTests.cs | 105 +++++++ .../ValueObjectValidationCodegenTests.cs | 4 +- .../Generators/KNOWN_GAPS.md | 2 +- .../Generators/RoutesGenerator.cs | 134 +++++---- .../MetaObjects.Codegen/HelperRuntime.cs | 1 + .../MetaObjects.Codegen.csproj | 1 + .../Runtime/ValidationIssues.cs | 279 ++++++++++++++++++ .../Runtime/ValueObjectValidator.cs | 38 ++- .../Api/ApiContractAssertions.cs | 37 ++- .../Api/ApiContractRequiredValueTypeTest.cs | 12 + .../Api/AuthorApiServer.cs | 35 ++- .../Api/JsonbReferenceServer.cs | 27 +- .../Api/ReferenceValidation.cs | 40 +++ .../Api/TphReferenceServer.cs | 67 +++-- server/java/codegen-kotlin/pom.xml | 1 + .../generator/kotlin/GeneratorRegistry.kt | 5 +- .../generator/kotlin/KNOWN_GAPS.md | 3 +- .../kotlin/KotlinDecimalWireEmitter.kt | 47 ++- .../kotlin/KotlinSpringControllerGenerator.kt | 161 ++++++---- .../kotlin/KotlinValidationIssuesEmitter.kt | 135 +++++++++ .../kotlin/KotlinAutoSetStampingTest.kt | 6 +- .../generator/kotlin/KotlinDecimalWireTest.kt | 38 ++- .../kotlin/KotlinSecretFieldsTest.kt | 2 +- .../kotlin/KotlinServerOwnedInputTest.kt | 28 +- .../kotlin/KotlinValidationIssuesTest.kt | 95 ++++++ .../acme/blog/AuthorController.kt | 35 ++- .../acme/blog/MetaDecimalWire.kt | 47 ++- .../acme/blog/MetaValidationIssues.kt | 105 +++++++ .../acme/catalog/MetaDecimalWire.kt | 47 ++- .../acme/catalog/MetaValidationIssues.kt | 105 +++++++ .../acme/catalog/ProductController.kt | 60 ++-- .../acme/catalog/MetaDecimalWire.kt | 47 ++- .../acme/catalog/MetaValidationIssues.kt | 105 +++++++ .../acme/catalog/ProductController.kt | 60 ++-- .../acme/auth/AuthController.kt | 82 ++--- .../acme/auth/AuthLineController.kt | 31 +- .../acme/auth/MetaDecimalWire.kt | 47 ++- .../acme/auth/MetaValidationIssues.kt | 105 +++++++ .../acme/report/MetaDecimalWire.kt | 47 ++- .../acme/report/MetaValidationIssues.kt | 105 +++++++ server/java/codegen-spring/pom.xml | 11 + .../generator/GeneratorRegistry.java | 2 +- .../generator/spring/KNOWN_GAPS.md | 3 +- .../spring/SpringControllerGenerator.java | 205 ++++++------- .../generator/spring/SpringDtoGenerator.java | 29 +- .../spring/runtime/CanonicalDecimal.java | 19 +- .../runtime/PatchValidationException.java | 103 ++++++- .../spring/runtime/RecordComponentNames.java | 12 + .../spring/runtime/UnreadableBody.java | 32 +- .../spring/runtime/ValidationIssues.java | 184 ++++++++++++ .../spring/SpringControllerGeneratorTest.java | 2 +- .../spring/SpringMapFieldCodegenTest.java | 12 +- .../spring/runtime/ValidationIssuesTest.java | 158 ++++++++++ .../expected/acme/shipping/CratePatch.java | 6 +- .../expected/acme/shipping/ParcelPatch.java | 6 +- .../expected/acme/shipping/ShipmentPatch.java | 8 +- .../acme/shipping/ShipmentSummaryPatch.java | 6 +- .../kotlin/api/ApiContractAssertions.kt | 25 ++ .../integration/kotlin/api/AuthorApiServer.kt | 32 +- .../kotlin/api/jsonb/DocumentApiServer.kt | 28 +- .../api/ApiContractAssertions.java | 33 +++ .../integration/api/AuthorApiServer.java | 50 +++- .../integration/api/JsonbReferenceServer.java | 16 +- .../mojo/EjectRuntimeRoundTripTest.java | 10 +- .../src/metaobjects/codegen/KNOWN_GAPS.md | 4 +- .../codegen/generators/router_generator.py | 68 +++-- .../codegen/runtime/constraint_errors.py | 122 +++++++- .../codegen/runtime/decimal_wire.py | 44 ++- .../tests/codegen/test_router_generator.py | 15 + .../integration/api_contract_assertions.py | 29 ++ .../tests/integration/api_contract_server.py | 45 ++- .../test_generated_decimal_wire.py | 29 +- .../test_generated_router_validated_input.py | 15 +- .../tests/runtime/test_validation_issues.py | 137 +++++++++ .../src/api-contract-scenario.ts | 36 +++ .../src/api-contract-server.ts | 33 ++- 88 files changed, 3482 insertions(+), 643 deletions(-) create mode 100644 fixtures/api-contract-conformance/scenarios/validation-lists-issues.yaml create mode 100644 fixtures/api-contract-conformance/tph/scenarios/tph-validation-lists-issues.yaml create mode 100644 server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs create mode 100644 server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs create mode 100644 server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesEmitter.kt create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesTest.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaValidationIssues.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaValidationIssues.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaValidationIssues.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaValidationIssues.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaValidationIssues.kt create mode 100644 server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/ValidationIssues.java create mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java create mode 100644 server/python/tests/runtime/test_validation_issues.py diff --git a/docs/features/field-types.md b/docs/features/field-types.md index 4cd13888bd..b140fe95bd 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -84,7 +84,7 @@ Five rows need a footnote: **Nested map values are validated on every write path, on every port.** A posted map value that violates the referenced `object.value`'s own constraints is rejected with the cross-port - `{"error":"validation"}` envelope before the row is written. Scalar-valued maps + `{"error":"validation", "issues": [...]}` envelope before the row is written. Scalar-valued maps (`@valueType`) carry no nested bean, so nothing extra runs for them. > This was a real hole until 1.0.1, and it is worth knowing what it was, because the failure diff --git a/docs/features/generated-mutations.md b/docs/features/generated-mutations.md index 3a367bbb6f..1adb3f2234 100644 --- a/docs/features/generated-mutations.md +++ b/docs/features/generated-mutations.md @@ -41,7 +41,7 @@ but the semantics are one contract — call them **PATCH-1..7** (FR-035): simply omits every non-settable field, and Zod drops unknown keys — so `safeParse` succeeds and the key never reaches the `SET`. The TPH discriminator is additionally deleted by hand after parse, the same convention. A malformed - *value* on a settable key is still `400 {"error":"validation"}`. A key the entity + *value* on a settable key is still `400 {"error":"validation", "issues": [...]}`. A key the entity does not declare at all is dropped the same way. The Python routes hand the repository only the keys their input model validated. This matters for `writeOnce` specifically: the generated edit form submits EVERY diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 9f2ef99357..8a94d88f0e 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -1412,11 +1412,11 @@ on are yours too. Eject copies two kinds of source with a Java generator: | Ejecting | Shared emit rules, beside the generator | Helper runtime its output imports | |---|---|---| -| `routes` | `SpringGeneratorSupport` | `CanonicalDecimal`, `ConstraintErrors`, `FilterParseResult`, `FilterParser`, `FilterPredicate`, `PatchValidationException`, `RecordComponentNames`, `UnreadableBody` | -| `dto` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `DeclaredDecimal`, `PatchValidationException` | +| `routes` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDuration`, `ConstraintErrors`, `FilterParseResult`, `FilterParser`, `FilterPredicate`, `PatchValidationException`, `RecordComponentNames`, `ReportExport`, `UnreadableBody`, `ValidationIssues` | +| `dto` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `CanonicalDuration`, `CanonicalDurationDeserializer`, `CanonicalDurationSerializer`, `DeclaredDecimal`, `PatchValidationException` | | `repository` | `SpringGeneratorSupport` | `FilterPredicate` | -| `value-object` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `DeclaredDecimal` | -| any other Java generator, every Kotlin generator | nothing | nothing: its output imports only core (Kotlin writes its decimal wire into the output as `MetaDecimalWire.kt`) | +| `value-object` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `CanonicalDuration`, `CanonicalDurationDeserializer`, `CanonicalDurationSerializer`, `DeclaredDecimal` | +| any other Java generator, every Kotlin generator | nothing | nothing: its output imports only core (Kotlin writes its decimal wire and its validation-issue list into the output, as `MetaDecimalWire.kt` and `MetaValidationIssues.kt`) | **The shared emit rules.** `SpringGeneratorSupport` holds the rules those four generators apply together: which fields a DTO and a patch carry, which of them are stamped for @@ -1656,7 +1656,7 @@ Otherwise a bug in the filter parser would still mean waiting for a MetaObjects `dotnet meta eject routes` therefore also writes their source to `codegen/runtime/`: `FilterParser.cs`, `FilterParseResult.cs`, `FilterPredicate.cs`, -`EfCoreFilterDispatch.cs`, `ValueObjectValidator.cs`, `ConstraintErrors.cs` and +`EfCoreFilterDispatch.cs`, `ValueObjectValidator.cs`, `ValidationIssues.cs`, `ConstraintErrors.cs` and `Iso8601TimestampConverter.cs`, the converter your host registers for the api contract's timestamp spelling. Each file is the packaged source with one change: its namespace becomes `Codegen.Runtime`, matching the folder the way `codegen/generators/` matches diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index 3f9fca7aab..14608b3548 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -4,13 +4,13 @@ "generated/csharp/SubscriberBlurbPayload.g.cs": "29eb3a6ce64496ffe734cba55d0213994316b4281d3fa6791917393bae36c9ed", "generated/csharp/SubscriberFilterAllowlist.g.cs": "3839b04884e4e393d4f7d7bf4c7d4fe5b3852bd3a92b1ecb282cb8935a3c98e5", "generated/csharp/SubscriberNames.g.cs": "c2cf0248c2ecde155ec2623604771e39bed23783603d6f18a2ed82967227a732", - "generated/csharp/SubscriberRoutes.g.cs": "d05912d0a76b52334857796a42b2919be8709997d6c073c11d2c44590fe686a9", + "generated/csharp/SubscriberRoutes.g.cs": "85969bbaf6a58ca2946c595d520e7ec48dcc5b5e21123e0340c5cc08aa8b9e04", "generated/python/Subscriber.py": "9dd4c0acea47f99d1daf79930f9de2e400f30780ebce6c63f023be528a72b762", "generated/python/SubscriberBlurbPayload.py": "a5ef8a08e63efbe0f73e39a21a368cff5150c68cc8beeea6e8aab0c755f8bbdd", "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", "generated/python/subscriber_filter_allowlist.py": "b0414cc95ff3982c80a0e1a06839f090c0b423e41f5a94275df93426cea09df5", "generated/python/subscriber_names.py": "269637a02bf9b46f55658b58cb42f83196c03a877e1e2101337847fd406c1561", - "generated/python/subscriber_router.py": "021768a4ed1fe24c2432087bba8b350579f60595728a95c6d43a528c3fdcb1be", + "generated/python/subscriber_router.py": "9bb26702fc7ad041cf4bb52c5e588226a315a2b0a858555666060863795dcfc8", "generated/ts/Subscriber.queries.ts": { "hash": "9f1a459c57fafc98bf074b4443b506d4a1e7023ed188f293021ae89c298d4562", "generatedBy": "queries-file", diff --git a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs index 9987f12b3c..efdd5400ca 100644 --- a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs +++ b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs @@ -87,25 +87,24 @@ await db.Subscribers.FindAsync(id) is { } found .Value.SerializerOptions; System.Text.Json.JsonDocument __body; try { __body = await System.Text.Json.JsonDocument.ParseAsync(http.Request.Body); } - catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = "validation" }); } + catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); } using var __bodyScope = __body; if (__body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) - return Results.BadRequest(new { error = "validation" }); + return ValidationIssues.Unreadable(); + var __issues = new ValidationIssues(); var __present = new System.Collections.Generic.HashSet(System.StringComparer.OrdinalIgnoreCase); foreach (var __p in __body.RootElement.EnumerateObject()) if (__p.Value.ValueKind != System.Text.Json.JsonValueKind.Null) __present.Add(__p.Name); foreach (var __req in new[] { "email", "status" }) - if (!__present.Contains(__req)) return Results.BadRequest(new { error = "validation" }); - Subscriber? input; + if (!__present.Contains(__req)) __issues.AddRequired(__req); + Subscriber? input = null; try { input = System.Text.Json.JsonSerializer.Deserialize(__body.RootElement.GetRawText(), jsonOpts); } - catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = "validation" }); } - if (input is null) return Results.BadRequest(new { error = "validation" }); + catch (System.Exception __e) { __issues.AddReadFailure(__e, typeof(Subscriber)); } + if (input is null) return __issues.Any ? __issues.ToResult() : ValidationIssues.Unreadable(); // Server-owned: a generated key, the TPH discriminator and M:N navigations are not settable by the client; the value is ignored. input.Id = default!; - var __vr = new System.Collections.Generic.List(); - if (!System.ComponentModel.DataAnnotations.Validator.TryValidateObject( - input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true)) - return Results.BadRequest(new { error = "validation" }); + __issues.ValidateObject(input, jsonOpts); + if (__issues.Any) return __issues.ToResult(); input.CreatedAt = System.DateTimeOffset.UtcNow; db.Subscribers.Add(input); try { await db.SaveChangesAsync(); } @@ -126,8 +125,9 @@ async System.Threading.Tasks.Task UpdateSubscriber(long id, HttpContext if (existing is null) return Results.NotFound(new { error = "not_found" }); System.Text.Json.JsonDocument body; try { body = await System.Text.Json.JsonDocument.ParseAsync(http.Request.Body); } - catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = "validation" }); } + catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); } using var bodyScope = body; + if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable(); // Deserialize each value with the app's CONFIGURED JSON options, not defaults — // a field.timestamp binds to DateTimeOffset via a custom converter registered on // ConfigureHttpJsonOptions, and default options cannot read the wire format, throwing 500. @@ -135,32 +135,34 @@ async System.Threading.Tasks.Task UpdateSubscriber(long id, HttpContext http.RequestServices.GetService(typeof(Microsoft.Extensions.Options.IOptions))!) .Value.SerializerOptions; var entry = db.Entry(existing); + var __issues = new ValidationIssues(); foreach (var prop in body.RootElement.EnumerateObject()) { var target = entry.Metadata.FindProperty(prop.Name) ?? entry.Metadata.GetProperties().FirstOrDefault(p => string.Equals(p.Name, prop.Name, System.StringComparison.OrdinalIgnoreCase)); if (target is null) continue; + var __wire = ValidationIssues.WireName(existing.GetType(), target.Name, jsonOpts); if (target.IsPrimaryKey()) continue; // PK is route-authoritative if (string.Equals(target.Name, "CreatedAt", System.StringComparison.Ordinal)) continue; // @autoSet: server-owned if (prop.Value.ValueKind == System.Text.Json.JsonValueKind.Null) { // FR-035 PATCH-2: explicit null clears a nullable column; on a // NOT-NULL (@required) column it is a 400 (matching the cross-port contract). - if (!target.IsNullable) return Results.BadRequest(new { error = "validation" }); + if (!target.IsNullable) { __issues.AddRequired(__wire); continue; } entry.CurrentValues[target] = null; continue; } var clr = System.Nullable.GetUnderlyingType(target.ClrType) ?? target.ClrType; - var __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts); + object? __val; + try { __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts); } + catch (System.Exception __e) { __issues.AddReadFailure(__e, clr, __wire); continue; } // FR-036 A2 — validate the PRESENT, non-null value with the same per-field // DataAnnotations as create. This validates ONLY the present value (never the // whole row) so an ABSENT @required field is untouched, not a 400. - var __pr = new System.Collections.Generic.List(); - if (!System.ComponentModel.DataAnnotations.Validator.TryValidateProperty(__val, - new System.ComponentModel.DataAnnotations.ValidationContext(existing) { MemberName = target.Name }, __pr)) - return Results.BadRequest(new { error = "validation" }); + if (!__issues.ValidateProperty(existing, target.Name, __val, jsonOpts)) continue; entry.CurrentValues[target] = __val; } + if (__issues.Any) return __issues.ToResult(); try { await db.SaveChangesAsync(); } catch (System.Exception __saveError) { diff --git a/examples/showcase/generated/java/acme/SubscriberPatch.java b/examples/showcase/generated/java/acme/SubscriberPatch.java index 03b31a618b..aeeb00156f 100644 --- a/examples/showcase/generated/java/acme/SubscriberPatch.java +++ b/examples/showcase/generated/java/acme/SubscriberPatch.java @@ -38,9 +38,11 @@ public final class SubscriberPatch { public static SubscriberPatch fromJson(JsonNode body, ObjectMapper mapper) { if (!body.isObject()) throw new PatchValidationException("(request body)"); Map assigned = new LinkedHashMap<>(); - bind(body, mapper, assigned, "email", new TypeReference() {}, true); - bind(body, mapper, assigned, "name", new TypeReference() {}, false); - bind(body, mapper, assigned, "status", new TypeReference() {}, true); + java.util.List problems = PatchValidationException.newProblems(); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "email", new TypeReference() {}, true)); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "name", new TypeReference() {}, false)); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "status", new TypeReference() {}, true)); + if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned); return new SubscriberPatch(assigned); } @@ -56,7 +58,7 @@ private static void bind(JsonNode body, ObjectMapper mapper, Map try { assigned.put(field, mapper.treeToValue(node, type)); } catch (JsonProcessingException e) { - throw new PatchValidationException(field); + throw new PatchValidationException(field, e); } } } diff --git a/examples/showcase/generated/python/subscriber_router.py b/examples/showcase/generated/python/subscriber_router.py index 9db0b29b63..0823ed7798 100644 --- a/examples/showcase/generated/python/subscriber_router.py +++ b/examples/showcase/generated/python/subscriber_router.py @@ -21,6 +21,11 @@ from fastapi.responses import JSONResponse, Response from metaobjects.codegen.runtime.constraint_errors import ( classify_constraint_error, + pydantic_issues, + required_null_issues, + unreadable_body_issues, + validation_envelope, + value_error_issues, ) from metaobjects.codegen.runtime.decimal_wire import ( WireOutput, @@ -209,19 +214,25 @@ def create_subscriber( if ( dto is None ): # the body was not a JSON object: the cross-port 400, not FastAPI's 422 - return JSONResponse(status_code=400, content={"error": "validation"}) + return JSONResponse( + status_code=400, content=validation_envelope(unreadable_body_issues()) + ) try: dto = coerce_decimal_fields( dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS ) - except ValueError: - return JSONResponse(status_code=400, content={"error": "validation"}) + except ValueError as _exc: + return JSONResponse( + status_code=400, content=validation_envelope(value_error_issues(_exc)) + ) # FR-036: run the generated field constraints (length / pattern / # numeric bounds) before persisting; a violation is the cross-port 400. try: _valid = SubscriberCreate(**dto) - except ValidationError: - return JSONResponse(status_code=400, content={"error": "validation"}) + except ValidationError as _exc: + return JSONResponse( + status_code=400, content=validation_envelope(pydantic_issues(_exc.errors())) + ) # SECURITY (1.2.1): only what the input model validated reaches the repository. A key # it does not declare (an undeclared name, a server-generated id, a readOnly or derived # field, a frozen one on update) is dropped silently, never 400'd. @@ -248,25 +259,29 @@ def update_subscriber( if ( dto is None ): # the body was not a JSON object: the cross-port 400, not FastAPI's 422 - return JSONResponse(status_code=400, content={"error": "validation"}) + return JSONResponse( + status_code=400, content=validation_envelope(unreadable_body_issues()) + ) try: dto = coerce_decimal_fields( dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS ) - except ValueError: - return JSONResponse(status_code=400, content={"error": "validation"}) - # FR-035 PATCH-2: an explicit null on a @required field is a 400 — - # a present null on a NON-required field falls through and clears it, - # and an OMITTED required field is untouched (never a 400). - for _k in _REQUIRED_FIELDS: - if _k in dto and dto[_k] is None: - return JSONResponse(status_code=400, content={"error": "validation"}) + except ValueError as _exc: + return JSONResponse( + status_code=400, content=validation_envelope(value_error_issues(_exc)) + ) + # FR-035 PATCH-2: an explicit null on a @required field is a 400 — a present + # null on a NON-required field falls through and clears it, and an OMITTED + # required field is untouched (never a 400). # FR-036: validate PRESENT, non-null values with the create rules (the # all-optional patch model never fires required-checks on ABSENT fields). + _issues = required_null_issues(dto, _REQUIRED_FIELDS) try: _valid = SubscriberPatch(**dto) - except ValidationError: - return JSONResponse(status_code=400, content={"error": "validation"}) + except ValidationError as _exc: + _issues += pydantic_issues(_exc.errors()) + if _issues: + return JSONResponse(status_code=400, content=validation_envelope(_issues)) # SECURITY (1.2.1): only what the input model validated reaches the repository. A key # it does not declare (an undeclared name, a server-generated id, a readOnly or derived # field, a frozen one on update) is dropped silently, never 400'd. diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index 93a11d0e23..ab9e1aa587 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -74,7 +74,7 @@ }, "verify-transcript": { "lang": "console", - "inline": "meta verify — requirements: 1 entries (1 functional, 0 architectural) — 1 live; 1/1 entities claimed, counted over 1 metadata file(s).\nmeta verify — not run: schema (--db <url>), codegen (--codegen), deps (--deps), replay (--replay)\nmeta: [subscriberBlurb] (prompt) ERR_VAR_NOT_ON_PAYLOAD: name (templates/subscriber/blurb.mustache:1)\nmeta: meta verify — 1 drift error(s) across 1 template(s).", + "inline": "meta verify — requirements: 1 entries (1 functional, 0 architectural) — 1 live; 1/1 entities claimed, counted over 1 metadata file(s).\nmeta verify — not run: schema (--db <url>), codegen (--codegen), deps (--deps), replay (--replay)\nmeta: [subscriberBlurb] (prompt) ERR_VAR_NOT_ON_PAYLOAD: name (templates/subscriber/blurb.mustache:1)\nmeta: meta verify — 1 drift error(s) across 1 template(s).\nmeta: meta verify — generated list routes have no page size (advisory — does not fail the build):\nmeta: .metaobjects/config.json: routes-file is wired and the project sets no api.pagination, so a generated list route with no ?limit= returns the whole table and accepts any ?limit=. Add \"api\": { \"pagination\": { \"defaultLimit\": 50, \"maxLimit\": 1000 } } to give every list a default page size and answer a larger ?limit= with 400 limit_too_large.", "full": null, "lineCount": null }, diff --git a/fixtures/api-contract-conformance/scenarios/validation-lists-issues.yaml b/fixtures/api-contract-conformance/scenarios/validation-lists-issues.yaml new file mode 100644 index 0000000000..17da0bbf70 --- /dev/null +++ b/fixtures/api-contract-conformance/scenarios/validation-lists-issues.yaml @@ -0,0 +1,51 @@ +name: validation-lists-issues +description: > + A body that fails validation is answered 400 with every issue listed, in the shape the + TypeScript routes already use: {"error": "validation", "issues": [...]}, where each issue + carries a `code` (Zod's names: invalid_type, too_small, too_big, invalid_format, + invalid_value, custom), a `path` (the field, as an array) and a `message` (the port's own + wording, never asserted). The `issues` assertion names the issues that must be present. + + r1 breaks two rules at once (a name over its @maxLength and a missing @required createdAt), so + a port that stops at the first issue fails it. r3 sends a present null on a @required field. + r2 and r4 prove that nothing was written. +requests: + - id: r1 + method: POST + path: /api/authors + body: + name: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + expect: + status: 400 + body: + error: "validation" + issues: + - { path: [name], code: too_big } + - { path: [createdAt], code: invalid_type } + - id: r2 + method: GET + path: /api/authors + expect: + status: 200 + body: + length: 5 + - id: r3 + method: PATCH + path: /api/authors/1 + body: + name: null + expect: + status: 400 + body: + error: "validation" + issues: + - { path: [name], code: invalid_type } + - id: r4 + method: GET + path: /api/authors/1 + expect: + status: 200 + body: + row: + id: 1 + name: "Ada Lovelace" diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-validation-lists-issues.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-validation-lists-issues.yaml new file mode 100644 index 0000000000..cafe01af40 --- /dev/null +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-validation-lists-issues.yaml @@ -0,0 +1,47 @@ +name: tph-validation-lists-issues +description: > + The per-subtype routes list every validation issue, in the shape of the core + `validation-lists-issues` scenario: {"error": "validation", "issues": [{code, path, message}]}. + r1 sends a reference over its @maxLength and omits the BridgeAuth subtype's @required + quantity. r2 nulls the base's @required reference and sends a PriorAuth approver over its + @maxLength. r3 and r4 prove that nothing was written. +requests: + - id: r1 + method: POST + path: /api/auths/bridge + body: + reference: "RXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" + expect: + status: 400 + body: + error: "validation" + issues: + - { path: [reference], code: too_big } + - { path: [quantity], code: invalid_type } + - id: r2 + method: PATCH + path: /api/auths/priorauth/3 + body: + reference: null + approver: "AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" + expect: + status: 400 + body: + error: "validation" + issues: + - { path: [reference], code: invalid_type } + - { path: [approver], code: too_big } + - id: r3 + method: GET + path: /api/auths + expect: + status: 200 + body: + length: 3 + - id: r4 + method: GET + path: /api/auths/3 + expect: + status: 200 + body: + row: { id: 3, reference: "REF-3", approver: "Dr. Smith" } diff --git a/server/csharp/MetaObjects.Codegen.Tests/EntityGeneratorTests.cs b/server/csharp/MetaObjects.Codegen.Tests/EntityGeneratorTests.cs index 119b12f826..eb97630dff 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/EntityGeneratorTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/EntityGeneratorTests.cs @@ -125,10 +125,10 @@ public void Routes_generator_emits_crud_endpoints() // not an unhandled 500) + DataAnnotations-validates. Assert.Contains("app.MapPost(prefix + \"/subscribers\", async (HttpContext http, AppDbContext db) =>", src); Assert.Contains("foreach (var __req in new[] { \"email\" })", src); - Assert.Contains("if (!__present.Contains(__req)) return Results.BadRequest(new { error = \"validation\" });", src); - Assert.Contains("Subscriber? input;", src); + Assert.Contains("if (!__present.Contains(__req)) __issues.AddRequired(__req);", src); + Assert.Contains("Subscriber? input = null;", src); Assert.Contains("try { input = System.Text.Json.JsonSerializer.Deserialize(__body.RootElement.GetRawText(), jsonOpts); }", src); - Assert.Contains("catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }", src); + Assert.Contains("catch (System.Exception __e) { __issues.AddReadFailure(__e, typeof(Subscriber)); }", src); // PATCH + PUT share the same update handler (matches TS reference). It takes the raw // HttpContext (not a bound DTO) and PARTIAL-merges only the properties present in the // JSON body — a bound DTO would fill omitted fields with CLR defaults and clobber them. diff --git a/server/csharp/MetaObjects.Codegen.Tests/MapFieldCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/MapFieldCodegenTests.cs index 576c0ad06c..998d2c3c44 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/MapFieldCodegenTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/MapFieldCodegenTests.cs @@ -315,15 +315,15 @@ public void Both_write_paths_validate_the_values_of_a_value_object_map() .Single(f => f.Path == "CustomerRoutes.g.cs").Content; // Create: the map column joins the field.object columns in the validation preamble. - Assert.Contains("ValueObjectValidator.Validate(input.Addresses)", routes); + Assert.Contains("ValueObjectValidator.Collect(input.Addresses, __issues, jsonOpts, \"addresses\")", routes); // PATCH: a typed arm ahead of the generic path, so a present key is validated // before assignment rather than after persistence (or never). - Assert.Contains("if (!ValueObjectValidator.Validate(__map0)) return Results.BadRequest", routes); + Assert.Contains("if (!ValueObjectValidator.Collect(__map0, __issues, jsonOpts, \"addresses\")) continue;", routes); // The control: a scalar-valued map has no nested bean and must reach NEITHER path. // Without this, an implementation that validated every map would pass the two // assertions above while doing something quite different. - Assert.DoesNotContain("ValueObjectValidator.Validate(input.Labels)", routes); - Assert.DoesNotContain("ValueObjectValidator.Validate(input.Channels)", routes); + Assert.DoesNotContain("ValueObjectValidator.Collect(input.Labels", routes); + Assert.DoesNotContain("ValueObjectValidator.Collect(input.Channels", routes); } } diff --git a/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs new file mode 100644 index 0000000000..a3a00e2b8e --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs @@ -0,0 +1,105 @@ +// ValidationIssues — the Zod-named code mapping of the 400 validation envelope, per DataAnnotations attribute. + +using System.ComponentModel.DataAnnotations; +using System.Text.Json; +using System.Text.Json.Serialization; +using MetaObjects.Codegen.Runtime; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ValidationIssuesTests +{ + public enum Colour { Red, Green } + + public sealed class Doc + { + [Required, MaxLength(5)] public string? Name { get; set; } + [StringLength(8, MinimumLength = 3)] public string? Code { get; set; } + [MinLength(2)] public string? Tag { get; set; } + [Range(1, 10)] public int Rank { get; set; } + [RegularExpression("^[a-z]+$")] public string? Slug { get; set; } + [EmailAddress] public string? Email { get; set; } + [JsonPropertyName("wireOnly"), MaxLength(1)] public string? Renamed { get; set; } + public Colour Shade { get; set; } + public Guid Ref { get; set; } + public decimal Amount { get; set; } + } + + private static readonly JsonSerializerOptions Web = new(JsonSerializerDefaults.Web); + + private static List<(string Code, string Path)> Run(Doc doc) + { + var issues = new ValidationIssues(); + issues.ValidateObject(doc, Web); + return issues.Issues.Select(i => ((string)i["code"]!, string.Join(".", (List)i["path"]!))).ToList(); + } + + [Fact] + public void Each_attribute_maps_to_its_zod_code_at_its_wire_path() + { + var found = Run(new Doc + { + Name = "toolong", Code = "ab", Tag = "x", Rank = 11, Slug = "NOPE", Email = "nope", Renamed = "xx", + }); + Assert.Contains(("too_big", "name"), found); + Assert.Contains(("too_small", "code"), found); + Assert.Contains(("too_small", "tag"), found); + Assert.Contains(("too_big", "rank"), found); + Assert.Contains(("invalid_format", "slug"), found); + Assert.Contains(("invalid_format", "email"), found); + Assert.Contains(("too_big", "wireOnly"), found); + } + + [Fact] + public void Range_below_the_minimum_is_too_small_and_a_long_string_is_too_big() + { + Assert.Contains(("too_small", "rank"), Run(new Doc { Name = "a", Rank = 0 })); + Assert.Contains(("too_big", "code"), Run(new Doc { Name = "a", Rank = 1, Code = "123456789" })); + } + + [Fact] + public void A_missing_required_value_is_invalid_type_and_is_listed_with_every_other_issue() + { + var found = Run(new Doc { Name = null, Rank = 1, Tag = "x" }); + Assert.Contains(("invalid_type", "name"), found); + Assert.Contains(("too_small", "tag"), found); + } + + [Fact] + public void A_valid_object_records_nothing() => Assert.Empty(Run(new Doc { Name = "ok", Rank = 5 })); + + [Fact] + public void The_same_code_at_the_same_path_is_recorded_once() + { + var issues = new ValidationIssues(); + issues.AddRequired("name"); + issues.AddRequired("name"); + Assert.Equal(1, issues.Count); + } + + private static (string Code, string Path) Json(string body) + { + var issues = new ValidationIssues(); + try { JsonSerializer.Deserialize(body, Web); Assert.Fail("expected a JsonException"); } + catch (JsonException e) { issues.AddJson(e, typeof(Doc)); } + var i = Assert.Single(issues.Issues); + return ((string)i["code"]!, string.Join(".", (List)i["path"]!)); + } + + [Fact] + public void A_value_that_cannot_be_read_is_listed_at_its_wire_path() + { + Assert.Equal(("invalid_type", "rank"), Json("{\"rank\":\"abc\"}")); + Assert.Equal(("invalid_format", "ref"), Json("{\"ref\":\"not-a-guid\"}")); + Assert.Equal(("invalid_value", "shade"), Json("{\"shade\":\"Blue\"}")); + } + + [Fact] + public void An_unreadable_body_has_an_empty_path() + { + var issues = new ValidationIssues(); + issues.Add("invalid_type", "x", []); + Assert.Empty((List)issues.Issues[0]["path"]!); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/ValueObjectValidationCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ValueObjectValidationCodegenTests.cs index ee8ecb8334..b226657e95 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ValueObjectValidationCodegenTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ValueObjectValidationCodegenTests.cs @@ -80,8 +80,8 @@ public void Routes_emit_typed_vo_merge_arms_and_post_validation() Assert.Contains("Deserialize", routes); // single VO Assert.Contains("Deserialize>", routes); // array VO // Recursive VO validation on BOTH PATCH and POST. - Assert.Contains("ValueObjectValidator.Validate", routes); - Assert.Contains("ValueObjectValidator.Validate(input.PrimaryMarker)", routes); + Assert.Contains("ValueObjectValidator.Collect", routes); + Assert.Contains("ValueObjectValidator.Collect(input.PrimaryMarker, __issues, jsonOpts, \"primaryMarker\")", routes); } [Fact] diff --git a/server/csharp/MetaObjects.Codegen/Generators/KNOWN_GAPS.md b/server/csharp/MetaObjects.Codegen/Generators/KNOWN_GAPS.md index 589112585f..79f2a1fcb8 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/KNOWN_GAPS.md +++ b/server/csharp/MetaObjects.Codegen/Generators/KNOWN_GAPS.md @@ -35,7 +35,7 @@ Update when a gap closes or a new one surfaces. **Contract.** The exact `error` code vocabulary is NOT a hard Tier-1 invariant; consumers should treat any 4xx as user-facing and 5xx as retryable. Reference: [`api-contract.md` "Error response"](../../../../docs/features/api-contract.md). -**Today.** We emit `{ "error": "not_found" }` for 404 and `{ "error": "validation", "message": "..." }` for the sort-validation 400. **FR-036 wired explicit body validation with the cross-port `{ "error": "validation" }` envelope:** a POST runs `Validator.TryValidateObject(input, ...)` and a PATCH runs `Validator.TryValidateProperty(value, ...)` per present value → 400 `{ "error": "validation" }` (ASP.NET minimal-API never runs DataAnnotations on its own, so the annotations were previously decorative at the wire tier). The `issues` array is still TS-only idiomatic (Tier 2). +**Today.** We emit `{ "error": "not_found" }` for 404 and `{ "error": "validation", "message": "..." }` for the sort-validation 400. **FR-036 wired explicit body validation with the cross-port `{ "error": "validation" }` envelope:** a POST runs `Validator.TryValidateObject(input, ...)` and a PATCH runs `Validator.TryValidateProperty(value, ...)` per present value → 400 `{ "error": "validation", "issues": [...] }` (ASP.NET minimal-API never runs DataAnnotations on its own, so the annotations were previously decorative at the wire tier). Since 1.3.0 the handler collects every failure (each DataAnnotations attribute run on its own, value-object members, JSON read failures) through `ValidationIssues` and lists them all in the cross-port shape, `{code, path, message}` with Zod's code names; before that the 400 was bare. ### G7 — Value-object columns on TPH entities are still skipped on PATCH diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 53ff2ce597..27177afb38 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -428,7 +428,7 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte // body carries the derived (view-computed) fields (read-your-writes). AppendCreateHandler(sb, "/" + route, cls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + route + "\"", "input", credentials), voFields, mapVoFields, autoSetFields, - writeThrough ? (readDbSet, pkProp!, route) : null, parseBody: HasDecimalField(entity) || HasIsoDurationField(entity, ctx.Root), + writeThrough ? (readDbSet, pkProp!, route) : null, deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials, ServerOwnedCreateResets(entity, ctx.Root, _ => { })); @@ -455,8 +455,9 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte sb.AppendLine(" if (existing is null) return Results.NotFound(new { error = \"not_found\" });"); sb.AppendLine(" System.Text.Json.JsonDocument body;"); sb.AppendLine(" try { body = await System.Text.Json.JsonDocument.ParseAsync(http.Request.Body); }"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); + sb.AppendLine(" catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); }"); sb.AppendLine(" using var bodyScope = body;"); + sb.AppendLine(" if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable();"); sb.AppendLine(" // Deserialize each value with the app's CONFIGURED JSON options, not defaults —"); sb.AppendLine(" // a field.timestamp binds to DateTimeOffset via a custom converter registered on"); sb.AppendLine(" // ConfigureHttpJsonOptions, and default options cannot read the wire format, throwing 500."); @@ -998,59 +999,47 @@ private static void AppendCreateHandler( IReadOnlyList mapVoFields, IReadOnlyList autoSetFields, (string ReadDbSet, string PkProp, string Route)? reRead = null, - bool parseBody = false, IReadOnlyList? deniedNavs = null, IReadOnlyList? credentials = null, IReadOnlyList? serverOwned = null) { - // A decimal field can refuse its value while the body is bound (out of declared range, not a - // number). The framework's own binding answers that with a bare 400 and no body, so an entity - // carrying one parses the body itself and returns the cross-port {error:"validation"} envelope. - if (requiredKeys.Count == 0 && !parseBody) - { - sb.AppendLine($" app.MapPost(prefix + \"{mapPath}\", async ({cls} input, AppDbContext db) =>"); - sb.AppendLine(" {"); - AppendCreateServerOwned(sb, serverOwned); - AppendCreateIgnoreDenied(sb, deniedNavs); - AppendCreateValidation(sb); - AppendCreateVoValidation(sb, voFields, mapVoFields); - AppendCreateAutoSet(sb, autoSetFields); - AppendCreatePersist(sb, dbSet, createdExpr, reRead, credentials ?? []); - sb.AppendLine(" });"); - return; - } - + // The body is always read here, not model-bound: the framework answers a body it cannot bind with a + // bare 400 and no body, while every validation failure must list its issues (the cross-port + // {error:"validation", issues:[...]} envelope), so the handler reads the body itself and collects them. sb.AppendLine($" app.MapPost(prefix + \"{mapPath}\", async (HttpContext http, AppDbContext db) =>"); sb.AppendLine(" {"); AppendJsonOptsLocal(sb); sb.AppendLine(" System.Text.Json.JsonDocument __body;"); sb.AppendLine(" try { __body = await System.Text.Json.JsonDocument.ParseAsync(http.Request.Body); }"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); + sb.AppendLine(" catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); }"); sb.AppendLine(" using var __bodyScope = __body;"); sb.AppendLine(" if (__body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object)"); - sb.AppendLine(" return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" return ValidationIssues.Unreadable();"); + sb.AppendLine(" var __issues = new ValidationIssues();"); // FR-036 #4 — @required-key presence: an omitted key OR an explicit null on a @required // field → 400, matching the JVM/Python "missing @required → 400" contract. Case-insensitive // so a camelCase wire key ("createdAt") matches its PascalCase property ("CreatedAt"). - sb.AppendLine(" var __present = new System.Collections.Generic.HashSet(System.StringComparer.OrdinalIgnoreCase);"); - sb.AppendLine(" foreach (var __p in __body.RootElement.EnumerateObject())"); - sb.AppendLine(" if (__p.Value.ValueKind != System.Text.Json.JsonValueKind.Null) __present.Add(__p.Name);"); - sb.Append(requiredKeys.Count == 0 ? " foreach (var __req in new string[] { " : " foreach (var __req in new[] { "); - sb.Append(string.Join(", ", requiredKeys.Select(k => "\"" + k + "\""))); - sb.AppendLine(" })"); - sb.AppendLine(" if (!__present.Contains(__req)) return Results.BadRequest(new { error = \"validation\" });"); - // A malformed body (e.g. a JSON array for a single-VO / scalar for an object column) - // throws JsonException — return the cross-port 400 {error:"validation"} rather than an - // unhandled 500 (matches the no-required-keys path's automatic model-binding 400, and - // every other port). Program D — the VO columns make wrong-shaped bodies reachable. - sb.AppendLine($" {cls}? input;"); + if (requiredKeys.Count > 0) + { + sb.AppendLine(" var __present = new System.Collections.Generic.HashSet(System.StringComparer.OrdinalIgnoreCase);"); + sb.AppendLine(" foreach (var __p in __body.RootElement.EnumerateObject())"); + sb.AppendLine(" if (__p.Value.ValueKind != System.Text.Json.JsonValueKind.Null) __present.Add(__p.Name);"); + sb.Append(" foreach (var __req in new[] { "); + sb.Append(string.Join(", ", requiredKeys.Select(k => "\"" + k + "\""))); + sb.AppendLine(" })"); + sb.AppendLine(" if (!__present.Contains(__req)) __issues.AddRequired(__req);"); + } + // A value that cannot be read (a wrong-shaped VO, a decimal out of range, a malformed date) throws + // JsonException: it is listed at its path, and the body cannot be checked further. + sb.AppendLine($" {cls}? input = null;"); sb.AppendLine(" try { input = System.Text.Json.JsonSerializer.Deserialize<" + cls + ">(__body.RootElement.GetRawText(), jsonOpts); }"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); - sb.AppendLine(" if (input is null) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" catch (System.Exception __e) { __issues.AddReadFailure(__e, typeof(" + cls + ")); }"); + sb.AppendLine(" if (input is null) return __issues.Any ? __issues.ToResult() : ValidationIssues.Unreadable();"); AppendCreateServerOwned(sb, serverOwned); AppendCreateIgnoreDenied(sb, deniedNavs); AppendCreateValidation(sb); AppendCreateVoValidation(sb, voFields, mapVoFields); + sb.AppendLine(" if (__issues.Any) return __issues.ToResult();"); AppendCreateAutoSet(sb, autoSetFields); AppendCreatePersist(sb, dbSet, createdExpr, reRead, credentials ?? []); sb.AppendLine(" });"); @@ -1213,14 +1202,14 @@ private static void AppendCreateVoValidation( { foreach (var vf in voFields) sb.AppendLine( - $" if (!ValueObjectValidator.Validate(input.{vf.Nav})) return Results.BadRequest(new {{ error = \"validation\" }});"); + $" ValueObjectValidator.Collect(input.{vf.Nav}, __issues, jsonOpts, \"{vf.WireName}\");"); // #362 — a field.map @objectRef's values are value objects and get the same graph // validation. ValueObjectValidator unwraps an IDictionary to its Values; before that // arm existed a Dictionary hit the IEnumerable arm, yielded KeyValuePair structs, and // validated vacuously — so this line and that arm are one fix, not two. foreach (var mf in mapVoFields) sb.AppendLine( - $" if (!ValueObjectValidator.Validate(input.{mf.Nav})) return Results.BadRequest(new {{ error = \"validation\" }});"); + $" ValueObjectValidator.Collect(input.{mf.Nav}, __issues, jsonOpts, \"{mf.WireName}\");"); } // The configured System.Text.Json options local (`jsonOpts`) — the app's @@ -1279,10 +1268,7 @@ private static void AppendJsonOptsLocal(StringBuilder sb) // DataAnnotations on its own). Assumes the bound local `input` is in scope. private static void AppendCreateValidation(StringBuilder sb) { - sb.AppendLine(" var __vr = new System.Collections.Generic.List();"); - sb.AppendLine(" if (!System.ComponentModel.DataAnnotations.Validator.TryValidateObject("); - sb.AppendLine(" input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true))"); - sb.AppendLine(" return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" __issues.ValidateObject(input, jsonOpts);"); } // FR-035 PATCH-2 — the shared present-key merge loop for the generated update @@ -1306,6 +1292,8 @@ private static void AppendPartialMergeLoop( if (voFields[i].IsArray && !voFields[i].Required) sb.AppendLine($" var __clearVo{i} = false;"); + // Every violation is collected (never the first alone) and answered together after the loop. + sb.AppendLine(" var __issues = new ValidationIssues();"); sb.AppendLine(" foreach (var prop in body.RootElement.EnumerateObject())"); sb.AppendLine(" {"); // #484 — @formExclude / credential keys are IGNORED (never a 400), ahead of every arm below @@ -1327,7 +1315,10 @@ private static void AppendPartialMergeLoop( sb.AppendLine(" if (prop.Value.ValueKind == System.Text.Json.JsonValueKind.Null)"); sb.AppendLine(" {"); if (vf.Required) - sb.AppendLine(" return Results.BadRequest(new { error = \"validation\" }); // @required VO column: present-null is a 400"); + { + sb.AppendLine($" __issues.AddRequired(\"{vf.WireName}\"); // @required VO column: present-null is a 400"); + sb.AppendLine(" continue;"); + } else { sb.AppendLine($" existing.{vf.Nav} = null;"); @@ -1341,8 +1332,8 @@ private static void AppendPartialMergeLoop( // unhandled 500 (matches every other port + this port's own reference lane). sb.AppendLine($" {vf.DeserType} {v};"); sb.AppendLine($" try {{ {v} = System.Text.Json.JsonSerializer.Deserialize<{vf.DeserType}>(prop.Value.GetRawText(), jsonOpts); }}"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); - sb.AppendLine($" if (!ValueObjectValidator.Validate({v})) return Results.BadRequest(new {{ error = \"validation\" }});"); + sb.AppendLine($" catch (System.Text.Json.JsonException __e) {{ __issues.AddJson(__e, null, \"{vf.WireName}\"); continue; }}"); + sb.AppendLine($" if (!ValueObjectValidator.Collect({v}, __issues, jsonOpts, \"{vf.WireName}\")) continue;"); sb.AppendLine($" existing.{vf.Nav} = {v}{(vf.Required ? "!" : "")};"); sb.AppendLine(" continue;"); sb.AppendLine(" }"); @@ -1362,8 +1353,8 @@ private static void AppendPartialMergeLoop( sb.AppendLine($" {{ existing.{mf.Nav} = null; continue; }}"); sb.AppendLine($" {mf.DeserType}? {mv};"); sb.AppendLine($" try {{ {mv} = System.Text.Json.JsonSerializer.Deserialize<{mf.DeserType}>(prop.Value.GetRawText(), jsonOpts); }}"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); - sb.AppendLine($" if (!ValueObjectValidator.Validate({mv})) return Results.BadRequest(new {{ error = \"validation\" }});"); + sb.AppendLine($" catch (System.Text.Json.JsonException __e) {{ __issues.AddJson(__e, null, \"{mf.WireName}\"); continue; }}"); + sb.AppendLine($" if (!ValueObjectValidator.Collect({mv}, __issues, jsonOpts, \"{mf.WireName}\")) continue;"); sb.AppendLine($" existing.{mf.Nav} = {mv};"); sb.AppendLine(" continue;"); sb.AppendLine(" }"); @@ -1371,6 +1362,7 @@ private static void AppendPartialMergeLoop( sb.AppendLine(" var target = entry.Metadata.FindProperty(prop.Name)"); sb.AppendLine(" ?? entry.Metadata.GetProperties().FirstOrDefault(p => string.Equals(p.Name, prop.Name, System.StringComparison.OrdinalIgnoreCase));"); sb.AppendLine(" if (target is null) continue;"); + sb.AppendLine(" var __wire = ValidationIssues.WireName(existing.GetType(), target.Name, jsonOpts);"); sb.AppendLine(" if (target.IsPrimaryKey()) continue; // PK is route-authoritative"); if (discProp is not null) { @@ -1391,7 +1383,7 @@ private static void AppendPartialMergeLoop( sb.AppendLine(" {"); sb.AppendLine(" // FR-035 PATCH-2: explicit null clears a nullable column; on a"); sb.AppendLine(" // NOT-NULL (@required) column it is a 400 (matching the cross-port contract)."); - sb.AppendLine(" if (!target.IsNullable) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (!target.IsNullable) { __issues.AddRequired(__wire); continue; }"); sb.AppendLine(" entry.CurrentValues[target] = null;"); sb.AppendLine(" continue;"); sb.AppendLine(" }"); @@ -1409,15 +1401,22 @@ private static void AppendPartialMergeLoop( { sb.AppendLine(" if (clr == typeof(decimal))"); sb.AppendLine(" {"); - sb.AppendLine(" if (!DecimalWire.TryParseElement(prop.Value, out var __dec) || !DecimalWire.FitsDeclared(existing.GetType(), target.Name, __dec)) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (!DecimalWire.TryParseElement(prop.Value, out var __dec)) { __issues.Add(\"invalid_type\", __wire + \" is not a decimal\", __wire); continue; }"); + sb.AppendLine(" if (!DecimalWire.FitsDeclared(existing.GetType(), target.Name, __dec)) { __issues.Add(\"too_big\", __wire + \" exceeds its declared precision and scale\", __wire); continue; }"); sb.AppendLine(" __val = __dec;"); sb.AppendLine(" }"); sb.AppendLine(" else if (clr == typeof(System.Collections.Generic.IList))"); sb.AppendLine(" {"); - sb.AppendLine(" if (prop.Value.ValueKind != System.Text.Json.JsonValueKind.Array) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (prop.Value.ValueKind != System.Text.Json.JsonValueKind.Array) { __issues.Add(\"invalid_type\", __wire + \" is not an array\", __wire); continue; }"); sb.AppendLine(" var __decs = new System.Collections.Generic.List();"); + sb.AppendLine(" var __decBad = false;"); sb.AppendLine(" foreach (var __el in prop.Value.EnumerateArray())"); - sb.AppendLine(" if (DecimalWire.TryParseElement(__el, out var __d1) && DecimalWire.FitsDeclared(existing.GetType(), target.Name, __d1)) __decs.Add(__d1); else return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" {"); + sb.AppendLine(" if (!DecimalWire.TryParseElement(__el, out var __d1)) { __issues.Add(\"invalid_type\", __wire + \" holds a value that is not a decimal\", __wire, __decs.Count); __decBad = true; break; }"); + sb.AppendLine(" if (!DecimalWire.FitsDeclared(existing.GetType(), target.Name, __d1)) { __issues.Add(\"too_big\", __wire + \" holds a value over its declared precision and scale\", __wire, __decs.Count); __decBad = true; break; }"); + sb.AppendLine(" __decs.Add(__d1);"); + sb.AppendLine(" }"); + sb.AppendLine(" if (__decBad) continue;"); sb.AppendLine(" __val = __decs;"); sb.AppendLine(" }"); arm = "else if"; @@ -1430,32 +1429,42 @@ private static void AppendPartialMergeLoop( { sb.AppendLine($" {arm} (clr == typeof(System.TimeSpan))"); sb.AppendLine(" {"); - sb.AppendLine(" if (!DurationWire.TryParseElement(prop.Value, out var __dur)) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (!DurationWire.TryParseElement(prop.Value, out var __dur)) { __issues.Add(\"invalid_type\", __wire + \" is not an ISO-8601 duration\", __wire); continue; }"); sb.AppendLine(" __val = __dur;"); sb.AppendLine(" }"); sb.AppendLine(" else if (clr == typeof(System.Collections.Generic.IList))"); sb.AppendLine(" {"); - sb.AppendLine(" if (prop.Value.ValueKind != System.Text.Json.JsonValueKind.Array) return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (prop.Value.ValueKind != System.Text.Json.JsonValueKind.Array) { __issues.Add(\"invalid_type\", __wire + \" is not an array\", __wire); continue; }"); sb.AppendLine(" var __durs = new System.Collections.Generic.List();"); + sb.AppendLine(" var __durBad = false;"); sb.AppendLine(" foreach (var __el in prop.Value.EnumerateArray())"); - sb.AppendLine(" if (DurationWire.TryParseElement(__el, out var __u1)) __durs.Add(__u1); else return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" {"); + sb.AppendLine(" if (!DurationWire.TryParseElement(__el, out var __u1)) { __issues.Add(\"invalid_type\", __wire + \" holds a value that is not an ISO-8601 duration\", __wire, __durs.Count); __durBad = true; break; }"); + sb.AppendLine(" __durs.Add(__u1);"); + sb.AppendLine(" }"); + sb.AppendLine(" if (__durBad) continue;"); sb.AppendLine(" __val = __durs;"); sb.AppendLine(" }"); } sb.AppendLine(" else"); - sb.AppendLine(" __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts);"); + sb.AppendLine(" {"); + sb.AppendLine(" try { __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts); }"); + sb.AppendLine(" catch (System.Exception __e) { __issues.AddReadFailure(__e, clr, __wire); continue; }"); + sb.AppendLine(" }"); } else - sb.AppendLine(" var __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts);"); + { + sb.AppendLine(" object? __val;"); + sb.AppendLine(" try { __val = System.Text.Json.JsonSerializer.Deserialize(prop.Value.GetRawText(), clr, jsonOpts); }"); + sb.AppendLine(" catch (System.Exception __e) { __issues.AddReadFailure(__e, clr, __wire); continue; }"); + } sb.AppendLine(" // FR-036 A2 — validate the PRESENT, non-null value with the same per-field"); sb.AppendLine(" // DataAnnotations as create. This validates ONLY the present value (never the"); sb.AppendLine(" // whole row) so an ABSENT @required field is untouched, not a 400."); - sb.AppendLine(" var __pr = new System.Collections.Generic.List();"); - sb.AppendLine(" if (!System.ComponentModel.DataAnnotations.Validator.TryValidateProperty(__val,"); - sb.AppendLine(" new System.ComponentModel.DataAnnotations.ValidationContext(existing) { MemberName = target.Name }, __pr))"); - sb.AppendLine(" return Results.BadRequest(new { error = \"validation\" });"); + sb.AppendLine(" if (!__issues.ValidateProperty(existing, target.Name, __val, jsonOpts)) continue;"); sb.AppendLine(" entry.CurrentValues[target] = __val;"); sb.AppendLine(" }"); + sb.AppendLine(" if (__issues.Any) return __issues.ToResult();"); } // Emit the per-subtype CRUD route set under //. Scoped to the subtype @@ -1535,7 +1544,6 @@ private static void AppendTphSubtypeRoutes( // VO columns on a TPH subtype are out of scope (Program D §6) — no typed arms. AppendCreateHandler(sb, "/" + subRoute, subCls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + subRoute + "/\" + input." + pkProp, "input", credentials), [], mapVoFields, autoSetFields, - parseBody: HasDecimalField(st.Entity) || HasIsoDurationField(st.Entity, root), deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials, serverOwned: ServerOwnedCreateResets(st.Entity, root, _ => { }, discProp, hierarchy[0])); @@ -1548,7 +1556,9 @@ private static void AppendTphSubtypeRoutes( sb.AppendLine(" if (existing is null) return Results.NotFound(new { error = \"not_found\" });"); sb.AppendLine(" System.Text.Json.JsonDocument body;"); sb.AppendLine(" try { body = await System.Text.Json.JsonDocument.ParseAsync(http.Request.Body); }"); - sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); + sb.AppendLine(" catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); }"); + sb.AppendLine(" using var bodyScope = body;"); + sb.AppendLine(" if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable();"); sb.AppendLine(" // Configured JSON options (not defaults) so a custom converter — e.g. the"); sb.AppendLine(" // field.timestamp DateTimeOffset converter — is honored; defaults throw 500 on it."); sb.AppendLine(" var jsonOpts = ((Microsoft.Extensions.Options.IOptions)"); diff --git a/server/csharp/MetaObjects.Codegen/HelperRuntime.cs b/server/csharp/MetaObjects.Codegen/HelperRuntime.cs index e8c26ed74d..b2ed40726d 100644 --- a/server/csharp/MetaObjects.Codegen/HelperRuntime.cs +++ b/server/csharp/MetaObjects.Codegen/HelperRuntime.cs @@ -69,6 +69,7 @@ public static class HelperRuntime "EfCoreFilterDispatch.cs", "ValueObjectValidator.cs", "ConstraintErrors.cs", + "ValidationIssues.cs", "Iso8601TimestampConverter.cs", "DecimalWire.cs", "DurationWire.cs", diff --git a/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj b/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj index aa0b82d43f..d9b2c3965b 100644 --- a/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj +++ b/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj @@ -86,6 +86,7 @@ + diff --git a/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs b/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs new file mode 100644 index 0000000000..cd73ad3f93 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs @@ -0,0 +1,279 @@ +// ValidationIssues — the 400 validation envelope a generated route answers with. +// +// { "error": "validation", "issues": [ { "code": "too_big", "path": ["name"], "message": "..." } ] } +// +// The shape the TypeScript routes already use. Every issue is listed, not just the first. +// code Zod's names: invalid_type (missing or null required key, wrong JSON type, an unparseable +// value), too_small, too_big (over a max length/value, a decimal over its precision/scale), +// invalid_format (a regex or format, a malformed date or guid), invalid_value (an enum value +// that is not a member), custom (anything else). +// path the field's WIRE name (not the C# property name), nested value-object members and array +// indexes as further elements; [] when the body as a whole cannot be read. +// message the port's own wording, never asserted across ports. +// +// DataAnnotations' ValidationResult does not say WHICH attribute failed, so each property's attributes +// are evaluated one by one here and the attribute type (and, for a length or range, which bound the +// value crossed) decides the code. + +using System.Collections; +using System.ComponentModel.DataAnnotations; +using System.Globalization; +using System.Reflection; +using System.Text.Json; +using System.Text.Json.Serialization; +using Microsoft.AspNetCore.Http; + +namespace MetaObjects.Codegen.Runtime; + +/// An ordered, de-duplicated list of validation issues, and the 400 response that carries it. +public sealed class ValidationIssues +{ + private readonly List> _issues = []; + private readonly HashSet _seen = []; + + /// How many distinct issues have been recorded. + public int Count => _issues.Count; + + /// True once any issue has been recorded. + public bool Any => _issues.Count > 0; + + /// The recorded issues, as the wire objects. + public IReadOnlyList> Issues => _issues; + + /// The 400 response listing every recorded issue. + public IResult ToResult() => Results.BadRequest(new { error = "validation", issues = _issues }); + + /// The 400 for a body that cannot be read as a whole (not JSON, or not an object): one issue, empty path. + public static IResult Unreadable(string message = "the request body is not a JSON object") + { + var issues = new ValidationIssues(); + issues.Add("invalid_type", message, []); + return issues.ToResult(); + } + + /// Record an issue; the same code at the same path is recorded once. + public void Add(string code, string message, params object[] path) + { + if (!_seen.Add(code + "|" + JsonSerializer.Serialize(path))) return; + _issues.Add(new Dictionary { ["code"] = code, ["path"] = path.ToList(), ["message"] = message }); + } + + /// A @required key that is absent or null. + public void AddRequired(string wireName) => Add("invalid_type", $"{wireName} is required", wireName); + + /// + /// Record a failure to read a value. The exception's own path locates it below ; + /// (the CLR type at the prefix) lets an enum member that does not exist read as + /// invalid_value and a value that cannot be read as the target type as invalid_type. + /// + public void AddJson(JsonException error, Type? type, params object[] prefix) + { + var path = prefix.ToList(); + path.AddRange(ParseJsonPath(error.Path)); + var message = error.Message; + var cut = message.IndexOf(" Path: ", StringComparison.Ordinal); + if (cut > 0) message = message[..cut]; + Add(CodeFor(error, ResolveType(type, path.Skip(prefix.Length))), message, [.. path]); + } + + /// + /// Record a failure to read a value from the body: a JsonException is listed at its path, and so is any other + /// reader failure (a host converter that cannot take a null or a wrongly typed token may throw anything, such + /// as an ArgumentNullException or FormatException) unless a more specific issue is already recorded. + /// + public void AddReadFailure(Exception error, Type? type, params object[] prefix) + { + if (error is JsonException json) AddJson(json, type, prefix); + else if (!Any) Add("invalid_type", error.Message, prefix); + } + + private static string CodeFor(JsonException error, Type? target) + { + // A decimal the declared precision and scale cannot hold, or that no decimal can hold. + if (error.Message.Contains("exceeds the declared precision", StringComparison.Ordinal) + || error.Message.Contains("out of range", StringComparison.Ordinal)) return "too_big"; + var underlying = target is null ? null : Nullable.GetUnderlyingType(target) ?? target; + if (underlying is not null && underlying.IsEnum) return "invalid_value"; + // A number or boolean given as the wrong JSON type is a type error, not a format error. + if (underlying is not null && (underlying.IsPrimitive || underlying == typeof(decimal))) return "invalid_type"; + // A string that is not a date, a guid, a time of day...: System.Text.Json reports the parse failure + // as the inner FormatException. + for (Exception? e = error; e is not null; e = e.InnerException) + if (e is FormatException) return "invalid_format"; + return "invalid_type"; + } + + // "$.items[2].name" -> items, 2, name ; "$" -> nothing. + private static List ParseJsonPath(string? path) + { + var parts = new List(); + if (string.IsNullOrEmpty(path)) return parts; + var i = path.StartsWith('$') ? 1 : 0; + while (i < path.Length) + { + if (path[i] == '.') { i++; continue; } + if (path[i] == '[') + { + var end = path.IndexOf(']', i); + if (end < 0) break; + var inner = path[(i + 1)..end]; + parts.Add(int.TryParse(inner, NumberStyles.None, CultureInfo.InvariantCulture, out var ix) ? ix : inner.Trim('\'', '"')); + i = end + 1; + continue; + } + var next = i; + while (next < path.Length && path[next] != '.' && path[next] != '[') next++; + parts.Add(path[i..next]); + i = next; + } + return parts; + } + + // The CLR type at the end of a wire path below ; null once it cannot be followed. + private static Type? ResolveType(Type? root, IEnumerable path) + { + var type = root; + foreach (var step in path) + { + if (type is null) return null; + type = Nullable.GetUnderlyingType(type) ?? type; + if (step is int) + { + type = ElementType(type); + continue; + } + if (typeof(IDictionary).IsAssignableFrom(type)) { type = type.IsGenericType ? type.GetGenericArguments()[^1] : null; continue; } + var name = (string)step; + type = type.GetProperties(BindingFlags.Public | BindingFlags.Instance) + .FirstOrDefault(p => string.Equals(WireName(p, null), name, StringComparison.OrdinalIgnoreCase) + || string.Equals(p.Name, name, StringComparison.OrdinalIgnoreCase))?.PropertyType; + } + return type; + } + + private static Type? ElementType(Type type) => + type.IsArray ? type.GetElementType() + : type.IsGenericType ? type.GetGenericArguments()[0] + : null; + + /// The wire name of a property: its [JsonPropertyName], else the options' naming policy, else the CLR name. + public static string WireName(PropertyInfo property, JsonSerializerOptions? options) + { + if (property.GetCustomAttribute() is { } named) return named.Name; + return (options is null ? JsonNamingPolicy.CamelCase : options.PropertyNamingPolicy)?.ConvertName(property.Name) ?? property.Name; + } + + /// The wire name of the CLR property of . + public static string WireName(Type owner, string clrName, JsonSerializerOptions? options) => + owner.GetProperty(clrName) is { } property ? WireName(property, options) : clrName; + + /// + /// Evaluate every DataAnnotations attribute of every property of , recording one + /// issue per failing attribute at prefix + wireName. Class-level attributes and IValidatableObject + /// are honoured as a fallback when no property failed. Returns true when nothing failed. + /// + public bool ValidateObject(object instance, JsonSerializerOptions? options, params object[] prefix) + { + var before = Count; + foreach (var property in instance.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) continue; + ValidateProperty(instance, property, property.GetValue(instance), WireName(property, options), prefix); + } + if (Count != before) return false; + var results = new List(); + if (Validator.TryValidateObject(instance, new ValidationContext(instance), results, validateAllProperties: true)) return true; + foreach (var result in results) + Add("custom", result.ErrorMessage ?? "invalid", [.. prefix, .. ResultPath(instance, result, options)]); + return false; + } + + private static IEnumerable ResultPath(object instance, ValidationResult result, JsonSerializerOptions? options) + { + var member = result.MemberNames.FirstOrDefault(m => !string.IsNullOrEmpty(m)); + if (member is not null) yield return WireName(instance.GetType(), member, options); + } + + /// + /// Evaluate the DataAnnotations attributes of the property of + /// against (a PATCH validates only the present value, + /// never the whole row), recording issues at prefix + wireName. Returns true when it is valid. + /// + public bool ValidateProperty(object owner, string clrName, object? value, JsonSerializerOptions? options, params object[] prefix) + { + var property = owner.GetType().GetProperty(clrName); + if (property is null) return true; + var before = Count; + ValidateProperty(owner, property, value, WireName(property, options), prefix); + return Count == before; + } + + private void ValidateProperty(object owner, PropertyInfo property, object? value, string wire, object[] prefix) + { + foreach (var attribute in property.GetCustomAttributes(inherit: true)) + { + var context = new ValidationContext(owner) { MemberName = property.Name, DisplayName = wire }; + var result = attribute.GetValidationResult(value, context); + if (result is null || result == ValidationResult.Success) continue; + Add(CodeFor(attribute, value), result.ErrorMessage ?? $"{wire} is invalid", [.. prefix, wire]); + } + } + + /// The Zod code for a DataAnnotations attribute that rejected . + public static string CodeFor(ValidationAttribute attribute, object? value) + { + switch (attribute) + { + case RequiredAttribute: + return "invalid_type"; + case MaxLengthAttribute: + return "too_big"; + case MinLengthAttribute: + return "too_small"; + case StringLengthAttribute sl: + return Length(value) is { } n && n < sl.MinimumLength ? "too_small" : "too_big"; + case RangeAttribute range: + return RangeSide(range, value); + case EnumDataTypeAttribute: + return "invalid_value"; + case RegularExpressionAttribute: + case DataTypeAttribute: + return "invalid_format"; + default: + return "custom"; + } + } + + private static int? Length(object? value) => value switch + { + string s => s.Length, + Array a => a.Length, + ICollection c => c.Count, + _ => null, + }; + + // Which bound a value crossed: below the minimum is too_small, above the maximum too_big. + private static string RangeSide(RangeAttribute range, object? value) + { + try + { + var converter = System.ComponentModel.TypeDescriptor.GetConverter(range.OperandType); + object Operand(object bound) => bound is string s + ? converter.ConvertFromString(null, CultureInfo.InvariantCulture, s)! + : Convert.ChangeType(bound, range.OperandType, CultureInfo.InvariantCulture); + var v = value is string text + ? converter.ConvertFromString(null, CultureInfo.InvariantCulture, text) + : Convert.ChangeType(value, range.OperandType, CultureInfo.InvariantCulture); + if (v is IComparable comparable) + { + if (comparable.CompareTo(Operand(range.Minimum)) < 0) return "too_small"; + if (comparable.CompareTo(Operand(range.Maximum)) > 0) return "too_big"; + } + } + catch (Exception e) when (e is FormatException or InvalidCastException or OverflowException or NotSupportedException or ArgumentException) + { + // not comparable: fall through + } + return "custom"; + } +} diff --git a/server/csharp/MetaObjects.Codegen/Runtime/ValueObjectValidator.cs b/server/csharp/MetaObjects.Codegen/Runtime/ValueObjectValidator.cs index ff1f3d7434..5796838a9e 100644 --- a/server/csharp/MetaObjects.Codegen/Runtime/ValueObjectValidator.cs +++ b/server/csharp/MetaObjects.Codegen/Runtime/ValueObjectValidator.cs @@ -35,10 +35,20 @@ public static class ValueObjectValidator /// is non-recursive). Returns true when the whole graph is valid; a null / empty /// graph is vacuously valid (a required-null column is enforced by the caller, not here). /// - public static bool Validate(object? value) + public static bool Validate(object? value) => Collect(value, new ValidationIssues(), null); + + /// + /// Like , but records each violation in at its wire path + /// ( first, then each nested member, array index or map key). Returns true when the + /// whole graph is valid. + /// + public static bool Collect(object? value, ValidationIssues issues, System.Text.Json.JsonSerializerOptions? options, params object[] path) { var voAssembly = ValueObjectAssembly(value); - return voAssembly is null || ValidateGraph(value, voAssembly); + if (voAssembly is null) return true; + var before = issues.Count; + CollectGraph(value, voAssembly, issues, options, path); + return issues.Count == before; } // The assembly the value objects in this graph live in — used to scope recursion to @@ -70,41 +80,39 @@ public static bool Validate(object? value) } } - private static bool ValidateGraph(object? value, Assembly voAssembly) + private static void CollectGraph(object? value, Assembly voAssembly, ValidationIssues issues, System.Text.Json.JsonSerializerOptions? options, object[] path) { - if (value is null || value is string) return true; + if (value is null || value is string) return; // #362 — must precede the IEnumerable arm (every IDictionary is one). Without it a // Dictionary validated its KeyValuePair structs, which carry no // DataAnnotations and live in CoreLib, so every value passed unchecked. This arm also // covers a map nested INSIDE a value object, reached by the property walk below. if (value is IDictionary dict) { - foreach (var v in dict.Values) - if (!ValidateGraph(v, voAssembly)) return false; - return true; + foreach (DictionaryEntry entry in dict) + CollectGraph(entry.Value, voAssembly, issues, options, [.. path, entry.Key?.ToString() ?? ""]); + return; } if (value is IEnumerable seq) { + var index = 0; foreach (var item in seq) - if (!ValidateGraph(item, voAssembly)) return false; - return true; + CollectGraph(item, voAssembly, issues, options, [.. path, index++]); + return; } var type = value.GetType(); // Only validate + recurse into value-object instances (declared in the generated // assembly). A framework/primitive member is covered by its owner's attributes; // skipping it also prevents recursing (and cycling) through a framework graph. - if (type.Assembly != voAssembly) return true; + if (type.Assembly != voAssembly) return; - var results = new List(); - if (!Validator.TryValidateObject(value, new ValidationContext(value), results, validateAllProperties: true)) - return false; + issues.ValidateObject(value, options, path); // Non-recursive TryValidateObject — descend into nested-VO members explicitly. foreach (var prop in type.GetProperties(BindingFlags.Public | BindingFlags.Instance)) { if (prop.GetIndexParameters().Length != 0 || !prop.CanRead) continue; - if (!ValidateGraph(prop.GetValue(value), voAssembly)) return false; + CollectGraph(prop.GetValue(value), voAssembly, issues, options, [.. path, ValidationIssues.WireName(prop, options)]); } - return true; } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs index 2e8ee64f3d..62632e011c 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs @@ -1,6 +1,6 @@ // ApiContractAssertions — assertion vocabulary for api-contract-conformance. // The recognized keys (equals / length / ids / names / row / hasId / envelope / -// error / empty / fieldsEqual / fieldsNotEqual) are the cross-port contract — +// error / issues / empty / fieldsEqual / fieldsNotEqual) are the cross-port contract — // every per-port runner must implement them identically. See // fixtures/api-contract-conformance/README.md. // @@ -84,6 +84,8 @@ public static void AssertResponse(string scenarioName, ApiRequest request, int s throw new Xunit.Sdk.XunitException( $"{scenarioName} / {request.Id}: expected field=\"{wantField}\", got: {Render(body)}"); } + if (want.TryGetValue("issues", out var issuesObj) && issuesObj is List wantIssues) + AssertIssues(scenarioName, request.Id, body, wantIssues); return; } if (want.TryGetValue("length", out var lenObj) && lenObj is not null) @@ -213,6 +215,39 @@ public static void AssertResponse(string scenarioName, ApiRequest request, int s } } + // The validation envelope lists its issues: `issues` is an array, every issue carries a string + // `code`, an array `path` and a string `message`, and each expected { path, code } is among them. + private static void AssertIssues(string scenarioName, string requestId, object? body, List expected) + { + if (body is not Dictionary bm || !bm.TryGetValue("issues", out var io) || io is not List issues) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {requestId}: expected an issues array, got: {Render(body)}"); + foreach (var issue in issues) + { + if (issue is not Dictionary im + || !im.TryGetValue("code", out var c) || c is not string + || !im.TryGetValue("path", out var p) || p is not List + || !im.TryGetValue("message", out var m) || m is not string) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {requestId}: an issue lacks code, path or message: {Render(issue)}"); + } + foreach (var want in expected) + { + if (want is not Dictionary wm) + throw new Xunit.Sdk.XunitException($"{scenarioName} / {requestId}: malformed expected issue: {Render(want)}"); + wm.TryGetValue("code", out var wantCode); + wm.TryGetValue("path", out var wantPath); + var found = issues.Any(issue => + { + var im = (Dictionary)issue!; + return StructuralEquals(im["code"], wantCode) && StructuralEquals(im["path"], wantPath); + }); + if (!found) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {requestId}: expected an issue {Render(want)}, got: {Render(issues)}"); + } + } + /// /// Structural equality that normalizes numeric subtypes (int/long/etc are equal /// when their long value matches), recurses into maps + lists, and treats diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractRequiredValueTypeTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractRequiredValueTypeTest.cs index 1a349a2ec9..b35242b456 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractRequiredValueTypeTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractRequiredValueTypeTest.cs @@ -41,6 +41,7 @@ public async Task Post_omitting_required_value_type_field_is_rejected_400() var missingCreatedAt = await Post(client, new { name = "Ada Lovelace", bio = "Analytical Engine" }); Assert.Equal(400, (int)missingCreatedAt.Status); Assert.Equal("validation", ErrorOf(missingCreatedAt.Body)); + Assert.Contains("createdAt", IssuePaths(missingCreatedAt.Body)); // (2) A valid, complete body still creates (201) — the presence check does not over-reject. var full = await Post(client, new @@ -55,11 +56,13 @@ public async Task Post_omitting_required_value_type_field_is_rejected_400() var missingName = await Post(client, new { bio = "no name", createdAt = "2026-02-01T10:00:00" }); Assert.Equal(400, (int)missingName.Status); Assert.Equal("validation", ErrorOf(missingName.Body)); + Assert.Contains("name", IssuePaths(missingName.Body)); // (4) An explicit null on a @required value-type field is also rejected (present-null → 400). var nullCreatedAt = await Post(client, new { name = "Grace Hopper", createdAt = (string?)null }); Assert.Equal(400, (int)nullCreatedAt.Status); Assert.Equal("validation", ErrorOf(nullCreatedAt.Body)); + Assert.Contains("createdAt", IssuePaths(nullCreatedAt.Body)); } private static async Task<(System.Net.HttpStatusCode Status, string Body)> Post(HttpClient client, object body) @@ -73,6 +76,15 @@ public async Task Post_omitting_required_value_type_field_is_rejected_400() return (resp.StatusCode, await resp.Content.ReadAsStringAsync()); } + // The first path element of each listed issue: the wire name of the field that failed. + private static List IssuePaths(string body) + { + using var doc = JsonDocument.Parse(body); + return doc.RootElement.GetProperty("issues").EnumerateArray() + .Select(i => i.GetProperty("path").EnumerateArray().FirstOrDefault().GetString()) + .ToList(); + } + private static string? ErrorOf(string body) { if (string.IsNullOrEmpty(body)) return null; diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs index 0d81694254..08a0cc4b13 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs @@ -508,24 +508,31 @@ private async Task GetAuthorAsync(HttpContext ctx, string idStr) // FR-036 — the corpus Author model's @maxLength bounds (name 100, bio 1000). Only a // PRESENT, non-null string is measured; an absent field is ignored (untouched on PATCH; // required-presence is enforced separately). - private static bool ExceedsAuthorMaxLength(Dictionary body) => - (body.GetValueOrDefault("name") is string n && n.Length > 100) - || (body.GetValueOrDefault("bio") is string b && b.Length > 1000); + private static void AddAuthorMaxLengthIssues(Dictionary body, List> issues) + { + if (body.GetValueOrDefault("name") is string n && n.Length > 100) issues.Add(ReferenceValidation.TooLong("name", 100)); + if (body.GetValueOrDefault("bio") is string b && b.Length > 1000) issues.Add(ReferenceValidation.TooLong("bio", 1000)); + } private async Task CreateAuthorAsync(HttpContext ctx) { var parsed = await ReadJsonBodyAsync(ctx); if (parsed is not Dictionary body) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Unreadable()); return; } // FR-036 — mirror the corpus Author @maxLength bounds so this lane rejects an // over-length string with the same {error:"validation"} envelope the generated // lane's [MaxLength] produces (otherwise a too-long name would 500 on the PG write). - if (ExceedsAuthorMaxLength(body)) + // Every issue is listed, not just the first: a missing @required key, then each over-length string. + var createIssues = new List>(); + foreach (var reqField in new[] { "name", "createdAt" }) + if (body.GetValueOrDefault(reqField) is null) createIssues.Add(ReferenceValidation.Required(reqField)); + AddAuthorMaxLengthIssues(body, createIssues); + if (createIssues.Count > 0) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Envelope(createIssues)); return; } long newId; @@ -569,25 +576,21 @@ private async Task UpdateAuthorAsync(HttpContext ctx, string idStr) var parsed = await ReadJsonBodyAsync(ctx); if (parsed is not Dictionary body) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Unreadable()); return; } // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an // omitted required field is untouched (never a 400). + var patchIssues = new List>(); foreach (var reqField in new[] { "name", "createdAt" }) - { - if (body.TryGetValue(reqField, out var rv) && rv is null) - { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); - return; - } - } + if (body.TryGetValue(reqField, out var rv) && rv is null) patchIssues.Add(ReferenceValidation.Required(reqField)); // FR-036 — a PRESENT, non-null value that exceeds @maxLength is a 400 (same envelope); // an absent field is untouched (never checked). Mirrors the generated-lane [MaxLength]. - if (ExceedsAuthorMaxLength(body)) + AddAuthorMaxLengthIssues(body, patchIssues); + if (patchIssues.Count > 0) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Envelope(patchIssues)); return; } var sets = new List(); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs index b7dbc4abc7..3866214834 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs @@ -133,12 +133,12 @@ private async Task CreateDocumentAsync(HttpContext ctx) // Required keys: title + primaryMarker present + non-null (mirrors the generated // create handler's @required-key presence check). - if (!PresentNonNull(body, "title")) { await BadValidation(ctx); return; } - if (!PresentNonNull(body, "primaryMarker") || !ValidMarker(body["primaryMarker"])) { await BadValidation(ctx); return; } + if (!PresentNonNull(body, "title")) { await BadValidation(ctx, "title"); return; } + if (!PresentNonNull(body, "primaryMarker") || !ValidMarker(body["primaryMarker"])) { await BadValidation(ctx, "primaryMarker"); return; } // Optional VO columns: when present + non-null, validate nested constraints. - if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx); return; } - if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx); return; } - if (!CanonicalizeMeasures(body)) { await BadValidation(ctx); return; } + if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx, "optionalMarker"); return; } + if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx, "markers"); return; } + if (!CanonicalizeMeasures(body)) { await BadValidation(ctx, "measures"); return; } long newId; await using var c = new NpgsqlConnection(_pg.ConnectionString); @@ -175,14 +175,14 @@ private async Task UpdateDocumentAsync(HttpContext ctx, string idStr) // primaryMarker (required) — present-null → 400; present value must be a valid VO. // optionalMarker (nullable) — present-null clears; present value must be a valid VO. // markers (nullable array) — present-null clears; present [] persists; present value valid. - if (body.ContainsKey("title") && !PresentNonNull(body, "title")) { await BadValidation(ctx); return; } + if (body.ContainsKey("title") && !PresentNonNull(body, "title")) { await BadValidation(ctx, "title"); return; } if (body.ContainsKey("primaryMarker")) { - if (!PresentNonNull(body, "primaryMarker") || !ValidMarker(body["primaryMarker"])) { await BadValidation(ctx); return; } + if (!PresentNonNull(body, "primaryMarker") || !ValidMarker(body["primaryMarker"])) { await BadValidation(ctx, "primaryMarker"); return; } } - if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx); return; } - if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx); return; } - if (!CanonicalizeMeasures(body)) { await BadValidation(ctx); return; } + if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx, "optionalMarker"); return; } + if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx, "markers"); return; } + if (!CanonicalizeMeasures(body)) { await BadValidation(ctx, "measures"); return; } await using var c = new NpgsqlConnection(_pg.ConnectionString); await c.OpenAsync(); @@ -350,8 +350,11 @@ private static bool PresentNonNull(JsonObject body, string key) => body.TryGetPropertyValue(key, out var node) && node is not null && node.GetValueKind() != JsonValueKind.Null; - private Task BadValidation(HttpContext ctx) => - SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + // Lists the offending field (its wire name) as the one issue; an unreadable body has an empty path. + private Task BadValidation(HttpContext ctx, string? field = null) => + SendJsonAsync(ctx, 400, field is null + ? ReferenceValidation.Unreadable() + : ReferenceValidation.Envelope([ReferenceValidation.Issue("invalid_type", field, field + " is invalid")])); // ----- data / helpers ----- diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs new file mode 100644 index 0000000000..37baf6d90e --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs @@ -0,0 +1,40 @@ +// ReferenceValidation — the validation envelope the hand-rolled reference servers answer with: +// { "error": "validation", "issues": [ { "code", "path", "message" } ] }, the shape the TypeScript +// routes use (Zod's issue codes; `path` is the wire field name). + +namespace MetaObjects.IntegrationTests.Api; + +internal static class ReferenceValidation +{ + public static Dictionary Issue(string code, string field, string message) => new() + { + ["code"] = code, + ["path"] = new List { field }, + ["message"] = message, + }; + + public static Dictionary Envelope(IEnumerable> issues) => new() + { + ["error"] = "validation", + ["issues"] = issues.Cast().ToList(), + }; + + /// The body as a whole cannot be read: one issue with an empty path. + public static Dictionary Unreadable() => Envelope( + [ + new Dictionary + { + ["code"] = "invalid_type", + ["path"] = new List(), + ["message"] = "the request body is not a JSON object", + }, + ]); + + /// A required key that is absent or null. + public static Dictionary Required(string field) => + Issue("invalid_type", field, $"{field} is required"); + + /// A present string over its max length. + public static Dictionary TooLong(string field, int max) => + Issue("too_big", field, $"{field} must be at most {max} characters"); +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs index 1d5955b386..615580a864 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs @@ -140,21 +140,18 @@ private async Task DispatchAsync(HttpContext ctx) if (method == "POST") { var body = await ReadBodyAsync(ctx); - // ADR-0019 Amendment 1 (W3): copayAmount is decimal(10,2); a value it would - // round or overflow is a 400 validation, never stored rounded. - if (!CopayAmountFits(body)) + // FR-036 — the per-subtype CREATE validates every constraint, matching the generated + // controller's Dto DataAnnotations: the @required keys (the base `reference`, the + // BridgeAuth `quantity`), the @maxLength:80 columns and copayAmount's decimal(10,2). + // Every issue is listed, not just the first. + var createIssues = new List>(); + if (!PresentNonNull(body, "reference")) createIssues.Add(ReferenceValidation.Required("reference")); + if (disc == "Bridge" && !PresentNonNull(body, "quantity")) createIssues.Add(ReferenceValidation.Required("quantity")); + AddMaxLengthIssues(body, createIssues); + AddCopayIssues(body, createIssues); + if (createIssues.Count > 0) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); - return; - } - // FR-036 — the per-subtype CREATE validates the present-value field - // constraints, matching the generated controller's Dto DataAnnotations - // (and the vanilla POST + the per-subtype PATCH). `reference` is the base - // @required @maxLength:80 column shared by every subtype; an over-length - // value is a 400 {error:"validation"}, not a 500 on the PG write. - if (ExceedsMaxLength(body)) - { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Envelope(createIssues)); return; } var created = await CreateAsync(disc, body); @@ -178,20 +175,15 @@ private async Task DispatchAsync(HttpContext ctx) // not-null merge guard. In the single TPH table only base-required // columns are NOT NULL; subtype columns (quantity/copayAmount/approver) // are nullable, so a present null there clears them (FR-035 tristate). - if (body.ContainsKey("reference")) - { - var refVal = body["reference"]; - if (refVal is null || refVal.GetValueKind() == JsonValueKind.Null) - { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); - return; - } - } - // FR-036 — a present value exceeding @maxLength:80 (reference/approver) is a - // 400, matching the generated controller's per-present-value PATCH validation. - if (ExceedsMaxLength(body) || !CopayAmountFits(body)) + // A present value exceeding @maxLength:80 (reference/approver) is a 400 too. + var patchIssues = new List>(); + if (body.ContainsKey("reference") && !PresentNonNull(body, "reference")) + patchIssues.Add(ReferenceValidation.Required("reference")); + AddMaxLengthIssues(body, patchIssues); + AddCopayIssues(body, patchIssues); + if (patchIssues.Count > 0) { - await SendJsonAsync(ctx, 400, new Dictionary { ["error"] = "validation" }); + await SendJsonAsync(ctx, 400, ReferenceValidation.Envelope(patchIssues)); return; } var updated = await UpdateAsync(disc, subId, body); @@ -426,13 +418,19 @@ private async Task DeleteAsync(string discriminator, long id) // (mirrors the generated Dto's [MaxLength(80)] + the vanilla ExceedsAuthorMaxLength // guard), rejected before the PG write (which would otherwise 500 with 22001) — on the // per-subtype create AND on a present PATCH value. - private static bool ExceedsMaxLength(JsonObject body) => - IsOverLength(body, "reference") || IsOverLength(body, "approver"); + private static void AddMaxLengthIssues(JsonObject body, List> issues) + { + foreach (var col in new[] { "reference", "approver" }) + if (IsOverLength(body, col)) issues.Add(ReferenceValidation.TooLong(col, 80)); + } private static bool IsOverLength(JsonObject body, string col) => body.TryGetPropertyValue(col, out var node) && node is JsonValue jv && jv.TryGetValue(out var s) && s.Length > 80; + private static bool PresentNonNull(JsonObject body, string key) => + body.TryGetPropertyValue(key, out var node) && node is not null && node.GetValueKind() != JsonValueKind.Null; + private static object CoerceColumn(string col, JsonNode v) => col switch { "quantity" => v.GetValue(), @@ -450,11 +448,16 @@ private static bool IsOverLength(JsonObject body, string col) => // W3: copayAmount is decimal(10,2); a value the column would round or overflow is refused // rather than stored rounded. Absent or null is the caller's business, not this check's. - private static bool CopayAmountFits(JsonObject body) + // ADR-0019 Amendment 1 (W3): copayAmount is decimal(10,2); a value it would round or overflow is a + // 400 validation (too_big), never stored rounded; a value that is no decimal at all is invalid_type. + private static void AddCopayIssues(JsonObject body, List> issues) { if (!body.TryGetPropertyValue("copayAmount", out var node) || node is null - || node.GetValueKind() == JsonValueKind.Null) return true; - return ReadDecimal(node) is { } d && DecimalWire.Fits(d, 10, 2); + || node.GetValueKind() == JsonValueKind.Null) return; + if (ReadDecimal(node) is not { } d) + issues.Add(ReferenceValidation.Issue("invalid_type", "copayAmount", "copayAmount is not a decimal")); + else if (!DecimalWire.Fits(d, 10, 2)) + issues.Add(ReferenceValidation.Issue("too_big", "copayAmount", "copayAmount exceeds decimal(10,2)")); } // Map a DB row into the wire dictionary. `type` is the discriminator value (text); diff --git a/server/java/codegen-kotlin/pom.xml b/server/java/codegen-kotlin/pom.xml index 661e860642..3c7f2aecda 100644 --- a/server/java/codegen-kotlin/pom.xml +++ b/server/java/codegen-kotlin/pom.xml @@ -259,6 +259,7 @@ KotlinEnumEmitter.kt KotlinDecimalWireEmitter.kt KotlinDurationWireEmitter.kt + KotlinValidationIssuesEmitter.kt KotlinExtractSchemaEmitter.kt KotlinOutputFormatSpecEmitter.kt KotlinCrossFieldRules.kt diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt index 992e727929..41b77ed4fb 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt @@ -135,6 +135,9 @@ private val CROSS_FIELD_EMIT: List = listOf(KotlinCrossFieldRules::class /** The duration wire file `entity`, `routes` and `exposed-table` each write beside their own output. */ private val DURATION_WIRE_EMIT: List = listOf(KotlinDurationWireEmitter::class.java.name) +/** The validation envelope file `routes` writes beside its controllers. */ +private val VALIDATION_ISSUES_EMIT: List = listOf(KotlinValidationIssuesEmitter::class.java.name) + /** The rest of `api-docs`: the page IR (one file, `KotlinApiModel.kt`, which also holds the unit, symbol, field * and kind types), the builder that fills it, the renderer and the page paths. */ private val API_DOCS_EMIT: List = listOf( @@ -174,7 +177,7 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( layer = GeneratorLayer.API, factory = ::KotlinSpringControllerGenerator, ejectResourcePath = ejectPath("KotlinSpringControllerGenerator"), - ejectWith = DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT + DURATION_WIRE_EMIT, + ejectWith = DECIMAL_WIRE_EMIT + CROSS_FIELD_EMIT + DURATION_WIRE_EMIT + VALIDATION_ISSUES_EMIT, requires = listOf("entity", "exposed-table", "filter-allowlist", "relations"), ), "repository" to GeneratorInfo( diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md index 9459e77488..6f44fd03d5 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md @@ -56,7 +56,8 @@ value → set). Notes: - **Present-value constraint validation (RESOLVED by FR-036).** The handler now injects a `jakarta.validation.Validator` and runs `validateValue(::class.java, - field, value)` on each present value → 400 `{"error":"validation"}`, on both the + field, value)` on each present value → 400 `{"error":"validation", "issues": [...]}` (every + violation listed through the generated `MetaValidationIssues`, since 1.3.0), on both the vanilla and (FR-036 Program B) the TPH per-subtype update paths. (The prior claim that "TS and Python DO validate present values on PATCH" was wrong — before FR-036 only TS's vanilla path did.) One TPH caveat: the TPH union data class carries no diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireEmitter.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireEmitter.kt index 56c2cdcefa..3c3f13449d 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireEmitter.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireEmitter.kt @@ -66,6 +66,7 @@ object KotlinDecimalWireEmitter { |import com.fasterxml.jackson.databind.SerializerProvider |import com.fasterxml.jackson.databind.deser.ContextualDeserializer |import com.fasterxml.jackson.databind.deser.std.StdDeserializer + |import com.fasterxml.jackson.databind.exc.InvalidFormatException |import com.fasterxml.jackson.databind.ser.std.StdSerializer |import java.math.BigDecimal |import java.math.MathContext @@ -141,18 +142,54 @@ object KotlinDecimalWireEmitter { | return value | } | - | /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + | /** + | * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + | * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + | * wrong JSON type, an unparseable value) invalid_type. + | */ + | fun failureCode(failure: Throwable): String { + | var t: Throwable? = failure + | while (t != null) { + | val message = t.message + | if (t is InvalidFormatException) { + | val target = t.targetType + | if (target != null) { + | if (target.isEnum) return "invalid_value" + | val n = target.name + | if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + | n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + | } + | if (message != null && message.contains(": decimal out of range: ")) return "too_big" + | } + | if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + | t = if (t.cause === t) null else t.cause + | } + | return "invalid_type" + | } + | + | /** + | * The validation envelope for a request body Jackson could not read: one issue whose path is every property + | * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + | */ | fun unreadableBody(failure: Throwable): Map { - | val out = LinkedHashMap() - | out["error"] = "validation" + | val path = ArrayList() | var t: Throwable? = failure | while (t != null) { | if (t is JsonMappingException && t.path.isNotEmpty()) { - | val name = t.path[t.path.size - 1].fieldName - | if (name != null) { out["field"] = name; break } + | for (ref in t.path) { + | if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + | } + | break | } | t = if (t.cause === t) null else t.cause | } + | val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + | val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + | val out = LinkedHashMap() + | out["error"] = "validation" + | out["issues"] = listOf(issue) + | val field = path.lastOrNull { it is String } + | if (field != null) out["field"] = field | return out | } | diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 6d9828e6fa..068fb092f5 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -222,6 +222,8 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") - append(" if (validator.validate(dto).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + val patchKtType = { field: MetaField<*> -> + val elem: String = if (field is ObjectField) { + voElementTypeFqn(field, loader) + } else if (field is MapField) { + "kotlin.collections.Map" + } else { + (KotlinTypeMapper.enumTypeName(field, entity, providedEnums) ?: KotlinTypeMapper.kotlinTypeName(field)).toString() + } + if (field.isArrayType) "kotlin.collections.List<$elem>" else elem + } + if (hasPatchFields) { + // Bind the raw body so a body Jackson cannot bind as a whole (a required value missing) still + // gets EVERY issue listed: on a failed bind each present field is bound and validated on its own, + // and the failure itself is added at the field it names. + append(" fun create(@RequestBody body: JsonNode): ResponseEntity = transaction {\n") + append(" val issues = MetaValidationIssues.Collector()\n") + append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + append(" val dto: ${shortName} = try {\n") + append(" objectMapper.treeToValue(body, ${shortName}::class.java)\n") + append(" } catch (e: Exception) {\n") + // A required value with no default that is absent or null cannot bind: list EACH one here, since + // Jackson names only the first it meets. + for (field in patchSettableFields) { + if (KotlinGenUtil.isRequiredField(field) && !field.hasMetaAttr(MetaField.ATTR_DEFAULT) && + !KotlinGenUtil.isServerOwnedOnCreate(field, entity) && !KotlinGenUtil.isNullableShapeProperty(field) && + !KotlinGenUtil.isDerivedField(field) + ) { + append(" if (!body.has(\"${field.name}\") || body.get(\"${field.name}\").isNull) issues.add(\"invalid_type\", listOf(\"${field.name}\"), \"field '${field.name}' is required\")\n") + } + } + for (field in patchSettableFields) { + val fn = field.name + val cap = capitalizeFirst(fn) + val kt = patchKtType(field) + append(" if (body.has(\"$fn\") && !body.get(\"$fn\").isNull) {\n") + append(" val v$cap: $kt? = issues.bind<$kt>(\"$fn\") { ${patchBindExpr(field, fn, kt)} }\n") + append(" if (v$cap != null) issues.addAt(validator.validateValue(${shortName}::class.java, \"$fn\", v$cap), \"$fn\")\n") + appendDecimalRangeCheck(" ", field, "v$cap") + append(" }\n") + } + append(" issues.addFailure(e)\n") + append(" return@transaction ResponseEntity.badRequest().body(issues.body() as Any)\n") + append(" }\n") + append(" issues.addViolations(validator.validate(dto))\n") + } else { + append(" fun create(@RequestBody dto: ${shortName}): ResponseEntity = transaction {\n") + append(" val issues = MetaValidationIssues.Collector().addViolations(validator.validate(dto))\n") + } for (field in entity.metaFields) appendDecimalRangeCheck(" ", field, "dto.${field.name}") + // Every issue is listed (never just the first): collected above, answered once. + append(" if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any)\n") // #203/ADR-0045: capture one now() per temporal type so createdAt == updatedAt exactly. for ((expr, valName) in insertNowVal) append(" val $valName = $expr\n") append(" val newId = ${tableObjectName}.insert {\n") @@ -519,7 +569,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase) and the Exposed Column codec @@ -537,7 +587,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") - append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - for (field in patchSettableFields) { - if (KotlinGenUtil.isRequiredField(field)) { - append(" if (body.has(\"${field.name}\") && body.get(\"${field.name}\").isNull) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } - } + appendPatchHead(this, patchSettableFields) val mustStampOnUpdate = onUpdateAutoSetFields.isNotEmpty() if (hasPatchFields) { val settableNamesList = patchSettableFields.joinToString(", ") { "\"${it.name}\"" } @@ -592,7 +637,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase" else elem val fn = field.name val cap = capitalizeFirst(fn) - append(" val has$cap = body.has(\"$fn\")\n") - if (KotlinGenUtil.isRequiredField(field)) { - // required: present ⇒ non-null (present-null was 400'd above). - append(" val v$cap: $ktType? = if (has$cap) ${patchBindExpr(field, fn, ktType)} else null\n") - append(" if (has$cap && validator.validateValue(${shortName}::class.java, \"$fn\", v$cap).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } else { - append(" val null$cap = has$cap && body.get(\"$fn\").isNull\n") - append(" val v$cap: $ktType? = if (has$cap && !null$cap) ${patchBindExpr(field, fn, ktType)} else null\n") - append(" if (has$cap && !null$cap && validator.validateValue(${shortName}::class.java, \"$fn\", v$cap).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } - appendDecimalRangeCheck(" ", field, "v$cap") + appendPatchFieldBinding(this, field, ktType, patchBindExpr(field, fn, ktType), "${shortName}::class.java") // Program D (spec §0): jakarta validateValue does NOT cascade @Valid into a nested // value-object's constraints, so a present VO is validated EXPLICITLY — // validator.validate(voElement) per element (single VO whole; array VO per element), // 400 on any violation. (A present-null / cleared value is already handled above.) if (field is MapField) { // Each map VALUE is a value object validated on its own (the keys are arbitrary text). - append(" if (v$cap != null && v$cap.values.any { validator.validate(it).isNotEmpty() }) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + append(" if (v$cap != null) for ((k, el) in v$cap) issues.addViolations(validator.validate(el), \"$fn\", k)\n") } if (field is ObjectField) { if (field.isArrayType) { - append(" if (v$cap != null && v$cap.any { validator.validate(it).isNotEmpty() }) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + append(" if (v$cap != null) v$cap.forEachIndexed { i, el -> issues.addViolations(validator.validate(el), \"$fn\", i) }\n") } else { - append(" if (v$cap != null && validator.validate(v$cap).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + append(" if (v$cap != null) issues.addViolations(validator.validate(v$cap), \"$fn\")\n") } } } + // Every issue found above is answered at once, before anything is written. + append(" if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any)\n") // Apply the already-bound + validated values. Columns are qualified (`Table.col`) // so a field named like the `body`/`id` params can't shadow them. append(" ${tableObjectName}.update({ ${tableObjectName}.${pkFieldName} eq id }) {\n") @@ -662,8 +699,6 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + append(" val issues = MetaValidationIssues.Collector()\n") for (field in stPatch) { - append(" if (validator.validateValue($validationRef::class.java, \"${field.name}\", dto.${field.name}).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + append(" issues.addAt(validator.validateValue($validationRef::class.java, \"${field.name}\", dto.${field.name}), \"${field.name}\")\n") appendDecimalRangeCheck(" ", field, "dto.${field.name}") } for ((_, rule) in crossFieldRules) { - append(" if (!($rule)) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") + append(" if (!($rule)) issues.add(\"custom\", emptyList(), \"a cross-field rule failed\")\n") } + append(" if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any)\n") // #203/ADR-0045: capture one now() per temporal type so createdAt == updatedAt exactly. for ((expr, valName) in insertNowVal) append(" val $valName = $expr\n") append(" val newId = $table.insert {\n") @@ -1169,12 +1206,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") - append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - for (field in stPatch) { - if (KotlinGenUtil.isRequiredField(field)) { - append(" if (body.has(\"${field.name}\") && body.get(\"${field.name}\").isNull) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } - } + appendPatchHead(this, stPatch) // #203/ADR-0045: mirrors the vanilla update handler's mustStampOnUpdate control flow — // an onUpdate @autoSet column must bump on EVERY patch, so the "any present" guard is // dropped when one exists (else the guard would keep a no-settable-field patch from @@ -1183,7 +1215,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase" else "$elem" val fn = field.name val cap = capitalizeFirst(fn) - append(" val has$cap = body.has(\"$fn\")\n") - if (KotlinGenUtil.isRequiredField(field)) { - // required: present ⇒ non-null (present-null was 400'd above). - append(" val v$cap: $ktType? = if (has$cap) ${patchBindExpr(field, fn, ktType)} else null\n") - append(" if (has$cap && validator.validateValue($validationRef::class.java, \"$fn\", v$cap).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } else { - append(" val null$cap = has$cap && body.get(\"$fn\").isNull\n") - append(" val v$cap: $ktType? = if (has$cap && !null$cap) ${patchBindExpr(field, fn, ktType)} else null\n") - append(" if (has$cap && !null$cap && validator.validateValue($validationRef::class.java, \"$fn\", v$cap).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\") as Any)\n") - } - appendDecimalRangeCheck(" ", field, "v$cap") + appendPatchFieldBinding(this, field, ktType, patchBindExpr(field, fn, ktType), "$validationRef::class.java") } + append(" if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any)\n") // Apply the already-bound + validated values, scoped to (pk eq id) and (disc eq value). // Columns are qualified ($table.col) so a field named like the body/id params can't shadow. append(" $table.update({ ($table.$pkFieldName eq id) and ($table.${plan.discriminatorField} eq $disc) }) {\n") @@ -1223,8 +1246,6 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase, fn: String, ktType: String): String = if (field is DurationField && durationsIso) { // field.duration: the accepted ISO-8601 form only, never the ObjectMapper's Duration.parse. - val read = if (field.isArrayType) { + if (field.isArrayType) { "body.get(\"$fn\").also { require(it.isArray) }.map { MetaDurationWire.fromNode(it) }" } else { "MetaDurationWire.fromNode(body.get(\"$fn\"))" } - "runCatching { $read }" + - ".getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\", \"field\" to \"$fn\") as Any) }" } else if (field is DecimalField) { - val read = if (field.isArrayType) { + if (field.isArrayType) { "body.get(\"$fn\").also { require(it.isArray) }.map { MetaDecimalWire.fromNode(it) }" } else { "MetaDecimalWire.fromNode(body.get(\"$fn\"))" } - "runCatching { $read }" + - ".getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"validation\", \"field\" to \"$fn\") as Any) }" } else { "objectMapper.treeToValue(body.get(\"$fn\"), object : TypeReference<$ktType>() {})" } + /** + * The head of a PATCH handler: the issue collector, a non-object body (an issue with an empty path), and an + * explicit null on a @required field (an issue at that field). Every issue is collected and answered together. + */ + private fun appendPatchHead(out: StringBuilder, fields: List>) { + out.append(" val issues = MetaValidationIssues.Collector()\n") + out.append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + for (field in fields) { + if (KotlinGenUtil.isRequiredField(field)) { + out.append(" if (body.has(\"${field.name}\") && body.get(\"${field.name}\").isNull) issues.add(\"invalid_type\", listOf(\"${field.name}\"), \"field '${field.name}' cannot be null\")\n") + } + } + } + + /** + * Bind one PRESENT PATCH value into the typed local `v` (a bind failure becomes an issue and leaves it null) + * and validate it against [validationClass] when non-null, then check a decimal against its declared range. + * `has` and `null` are the presence flags the update block reads. + */ + private fun appendPatchFieldBinding(out: StringBuilder, field: MetaField<*>, ktType: String, bindExpr: String, validationClass: String) { + val fn = field.name + val cap = capitalizeFirst(fn) + out.append(" val has$cap = body.has(\"$fn\")\n") + out.append(" val null$cap = has$cap && body.get(\"$fn\").isNull\n") + out.append(" val v$cap: $ktType? = if (has$cap && !null$cap) issues.bind<$ktType>(\"$fn\") { $bindExpr } else null\n") + out.append(" if (v$cap != null) issues.addAt(validator.validateValue($validationClass, \"$fn\", v$cap), \"$fn\")\n") + out.appendDecimalRangeCheck(" ", field, "v$cap") + } + /** Capitalize the first char (method-name suffix from a discriminator value). */ private fun capitalizeFirst(s: String): String = if (s.isEmpty()) s else s[0].uppercaseChar() + s.substring(1) diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesEmitter.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesEmitter.kt new file mode 100644 index 0000000000..6c6e83915d --- /dev/null +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesEmitter.kt @@ -0,0 +1,135 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.generator.util.GeneratedFileWriter +import java.nio.file.Path + +/** + * Writes the per-package `MetaValidationIssues.kt` support file: the cross-port validation answer + * `400 {"error": "validation", "issues": [{"code", "path", "message"}]}` a generated controller + * collects into. `code` uses Zod's names (`invalid_type`, `too_small`, `too_big`, `invalid_format`, + * `invalid_value`, `custom`), `path` is the field's wire name (nested members and indexes as further + * elements, empty when the body as a whole cannot be read), `message` is this port's own wording. + * + * Generated source, so the generated controller imports nothing from MetaObjects at runtime; the + * only dependencies are the jakarta validation API the controller already uses and the package's + * generated `MetaDecimalWire` (which classifies a binding failure). Written by the controller + * generator into every package it emits a controller into; the content is fixed. + */ +object KotlinValidationIssuesEmitter { + + /** File name (sans `.kt`) and the name of the `internal object` it declares. */ + const val FILE_NAME = "MetaValidationIssues" + + /** Write `MetaValidationIssues.kt` into [pkg] under [outRoot]. */ + fun write(pkg: String, outRoot: Path) { + val header = if (pkg.isEmpty()) "" else "package $pkg\n\n" + val outFile = outRoot.resolve(pkg.replace('.', '/')).resolve("$FILE_NAME.kt") + GeneratedFileWriter.write(outFile, header + BODY + "\n") + } + + /** The fixed body. No Kotlin string templates, so this raw string needs no `$` escaping. */ + val BODY: String = """ + |import com.fasterxml.jackson.databind.JsonMappingException + |import jakarta.validation.ConstraintViolation + |import jakarta.validation.Path + | + |/** + | * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + | * before answering `400 {"error": "validation", "issues": [...]}`. + | */ + |internal object MetaValidationIssues { + | /** Collects issues; [body] is the response body. */ + | class Collector { + | private val issues = ArrayList>() + | + | fun isEmpty(): Boolean = issues.isEmpty() + | + | fun add(code: String, path: List, message: String): Collector { + | issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + | return this + | } + | + | /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + | fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + | for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + | return this + | } + | + | /** Every violation of a single-property validateValue, at exactly [path]. */ + | fun addAt(violations: Collection>, vararg path: Any): Collector { + | for (v in violations) add(codeOf(v), path.toList(), v.message) + | return this + | } + | + | /** Run [read]; a failure becomes an issue at [field] and yields null. */ + | fun bind(field: String, read: () -> T): T? = + | try { + | read() + | } catch (e: Exception) { + | add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + | null + | } + | + | /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + | fun addFailure(failure: Throwable): Collector { + | val path = ArrayList() + | var t: Throwable? = failure + | while (t != null) { + | if (t is JsonMappingException && t.path.isNotEmpty()) { + | for (ref in t.path) { + | if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + | } + | break + | } + | t = if (t.cause === t) null else t.cause + | } + | if (issues.any { it["path"] == path }) return this + | val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + | return add(MetaDecimalWire.failureCode(failure), path, message) + | } + | + | fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + | } + | + | /** A path as wire elements: property names, with array indexes and map keys. */ + | fun pathOf(path: Path): List { + | val out = ArrayList() + | for (node in path) { + | if (node.isInIterable) { + | val index = node.index + | val key = node.key + | if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + | } + | val name = node.name + | if (name != null) out.add(name) + | } + | return out + | } + | + | /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + | fun codeOf(v: ConstraintViolation<*>): String { + | val descriptor = v.constraintDescriptor + | val value = v.invalidValue + | val attrs = descriptor.attributes + | return when (descriptor.annotation.annotationClass.java.simpleName) { + | "NotNull" -> "invalid_type" + | "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + | "Size", "Length" -> { + | val max = attrs["max"] as? Number + | val length: Long? = when (value) { + | is CharSequence -> value.length.toLong() + | is Collection<*> -> value.size.toLong() + | is Map<*, *> -> value.size.toLong() + | else -> null + | } + | if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + | } + | "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + | "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + | "Pattern", "Email", "URL" -> "invalid_format" + | else -> "custom" + | } + | } + |} + |""".trimMargin() +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt index 69730eeaf9..9ce41c4af6 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt @@ -93,7 +93,7 @@ class KotlinAutoSetStampingTest { // owned by the stamping helper. Only the regular column (name) is a caller-value write. assertTrue("it[EventTable.name] = dto.name" in src, "insert should still write the regular caller-supplied column; saw:\n$src") - assertTrue("it[EventTable.createdAt]" !in src, + assertTrue("it[EventTable.createdAt]" !in src.substringAfter("fun update("), "@autoSet createdAt must NEVER be a caller-value (it[...]) write; saw:\n$src") assertTrue("it[EventTable.updatedAt]" !in src, "@autoSet updatedAt must NEVER be a caller-value (it[...]) write; saw:\n$src") @@ -193,7 +193,7 @@ class KotlinAutoSetStampingTest { // ONE captured now() val, assigned to BOTH columns → a fresh row's createdAt == updatedAt exactly. assertTrue("val autoSetNow0 = java.time.Instant.now()" in src, "insert must capture one now() val per temporal type; saw:\n$src") - assertTrue("it[createdAt] = autoSetNow0" in src && "it[updatedAt] = autoSetNow0" in src, + assertTrue("it[EventTable.createdAt] = autoSetNow0" in src && "it[EventTable.updatedAt] = autoSetNow0" in src, "insert must stamp BOTH @autoSet columns from the captured val; saw:\n$src") assertTrue("dto.createdAt" !in src && "dto.updatedAt" !in src, "an @autoSet column must never be bound from the caller dto; saw:\n$src") @@ -203,7 +203,7 @@ class KotlinAutoSetStampingTest { val src = generateController("autoset-event-ctl2", autoSetFixture, "acme/events/EventController.kt") assertTrue("it[EventTable.updatedAt] = java.time.Instant.now()" in src, "patch must bump the onUpdate column; saw:\n$src") - assertTrue("it[EventTable.createdAt]" !in src, + assertTrue("it[EventTable.createdAt]" !in src.substringAfter("fun update("), "patch/update must never touch the onCreate column createdAt; saw:\n$src") // onUpdate present ⇒ the update runs on EVERY patch (no "any present" guard). assertTrue(".any { body.has(it) }" !in src, diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt index d32e964ab0..fd26146e6a 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt @@ -104,7 +104,7 @@ class KotlinDecimalWireTest { fun `a PATCH decimal binds through the same reader as a create body and a bad one is a 400 naming the field`() { val src = emit(model, all()).getValue("acme/shop/ItemController.kt") assertTrue("MetaDecimalWire.fromNode(body.get(\"price\"))" in src, src) - assertTrue("mapOf(\"error\" to \"validation\", \"field\" to \"price\")" in src, src) + assertTrue("issues.bind<" in src && "(\"price\") { MetaDecimalWire.fromNode(body.get(\"price\")) }" in src, src) } @Test @@ -130,10 +130,10 @@ class KotlinDecimalWireTest { val src = emit(model, all()).getValue("acme/shop/ItemController.kt") // @precision 12, @scale 2: at most 2 fraction digits and 10 integer digits. val fits = "MetaDecimalWire.fits(" ; val bounds = ", 12, 2)" - assertEquals(2, Regex(Regex.escape(fits) + "[^\\n]*?" + Regex.escape(bounds)).findAll(src).count(), src) + assertEquals(3, Regex(Regex.escape(fits) + "[^\\n]*?" + Regex.escape(bounds)).findAll(src).count(), src) assertTrue("if (!MetaDecimalWire.fits(dto.price, 12, 2))" in src, src) assertTrue("if (!MetaDecimalWire.fits(vPrice, 12, 2))" in src, src) - assertTrue("\"field\" to \"price\"" in src, src) + assertTrue("issues.add(\"too_big\", listOf(\"price\")" in src, src) } @Test @@ -162,19 +162,23 @@ class KotlinDecimalWireTest { object Harness { @JvmStatic fun create(dto: Dto): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() ${line("if (dto.readings")} + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body()) ResponseEntity(201, dto.readings?.map { MetaDecimalWire.format(it) }) } @JvmStatic fun patch(body: JsonNode): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() ${line("val hasReadings")} ${line("val nullReadings")} ${line("val vReadings")} - ${line("if (vReadings")} + ${line("if (vReadings?.any")} + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body()) ResponseEntity(200, vReadings?.map { MetaDecimalWire.format(it) }) } } """.trimIndent() - val sources = files.filterKeys { it.endsWith("MetaDecimalWire.kt") }.map { (path, src) -> + val sources = files.filterKeys { it.endsWith("MetaDecimalWire.kt") || it.endsWith("MetaValidationIssues.kt") }.map { (path, src) -> com.tschuchort.compiletesting.SourceFile.kotlin(path.substringAfterLast('/'), src) } + com.tschuchort.compiletesting.SourceFile.kotlin("Harness.kt", harness) val result = com.tschuchort.compiletesting.KotlinCompilation().apply { @@ -196,17 +200,27 @@ class KotlinDecimalWireTest { .newInstance(readings.map { java.math.BigDecimal(it) }) return outcome(h.getMethod("create", dto.javaClass).invoke(null, dto)) } - val refused = mapOf("error" to "validation", "field" to "readings") + // The 400 body is the validation envelope: the issue is at the decimal array, too_big for a value past the + // declared range and invalid_type for one that is not a decimal, and `status` stays 400. + fun refused(code: String): Pair = 400 to mapOf( + "error" to "validation", + "issues" to listOf(mapOf("code" to code, "path" to listOf("readings"), "message" to "*")), + ) + fun withoutMessage(r: Pair): Pair { + val body = r.second as? Map<*, *> ?: return r + val issues = (body["issues"] as? List<*>)?.map { (it as Map<*, *>) + ("message" to "*") } ?: return r + return r.first to (body + ("issues" to issues)) + } // Oracle: ADR-0019 Amendment 1 — canonical strings (W1), exact input (W2), decimal(9, 4) range (W3). assertEquals(200 to listOf("100.5", "0.3", "15"), patch("""{"readings": ["100.5000", 0.3, "1.5e1"]}""")) - assertEquals(400 to refused, patch("""{"readings": ["1", "0.12345"]}""")) - assertEquals(400 to refused, patch("""{"readings": ["100000"]}""")) - assertEquals(400 to refused, patch("""{"readings": ["12abc"]}""")) - assertEquals(400 to refused, patch("""{"readings": "100.5"}""")) + assertEquals(refused("too_big"), withoutMessage(patch("""{"readings": ["1", "0.12345"]}"""))) + assertEquals(refused("too_big"), withoutMessage(patch("""{"readings": ["100000"]}"""))) + assertEquals(refused("invalid_type"), withoutMessage(patch("""{"readings": ["12abc"]}"""))) + assertEquals(refused("invalid_type"), withoutMessage(patch("""{"readings": "100.5"}"""))) assertEquals(201 to listOf("100.5", "0"), create("100.5000", "0.0000")) - assertEquals(400 to refused, create("1", "0.12345")) - assertEquals(400 to refused, create("100000")) + assertEquals(refused("too_big"), withoutMessage(create("1", "0.12345"))) + assertEquals(refused("too_big"), withoutMessage(create("100000"))) } @Test diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinSecretFieldsTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinSecretFieldsTest.kt index f0190864ec..017e2825dc 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinSecretFieldsTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinSecretFieldsTest.kt @@ -116,7 +116,7 @@ class KotlinSecretFieldsTest { fun `a model with neither mark generates the plain write and read paths`() { val (controller, entity) = controllerAndEntity(marked = false) for (f in listOf("tokenHash", "pinHash")) { - assertTrue("it[$f] = dto.$f" in controller, controller) + assertTrue("it[AccountTable.$f] = dto.$f" in controller, controller) assertTrue("$f = row[AccountTable.$f]," in controller, controller) } assertTrue("listOf(\"name\", \"tokenHash\", \"pinHash\").any { body.has(it) }" in controller, controller) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt index 44195040ee..70fabd09f3 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt @@ -59,9 +59,9 @@ class KotlinServerOwnedInputTest { try { val controller = Files.readString(out.resolve("acme/own/DocController.kt")) val create = controller.substringAfter("fun create(").substringBefore("fun update(") - assertTrue("it[title] = dto.title" in create, create) - assertFalse("it[role]" in create, create) - assertFalse("it[id] = dto.id" in create, create) + assertTrue("it[DocTable.title] = dto.title" in create, create) + assertFalse("it[DocTable.role]" in create, create) + assertFalse("it[DocTable.id] = dto.id" in create, create) val entity = Files.readString(out.resolve("acme/own/Doc.kt")) assertTrue("@JsonIgnoreProperties(ignoreUnknown = true)" in entity, entity) } finally { @@ -69,6 +69,28 @@ class KotlinServerOwnedInputTest { } } + @Test + fun `create lists every absent required value, not only the first the bind meets`() { + val json = plain.replace( + "{ \"field.string\": { \"name\": \"title\", \"@maxLength\": 100 } },", + "{ \"field.string\": { \"name\": \"title\", \"@required\": true, \"@maxLength\": 100 } }," + + " { \"field.string\": { \"name\": \"subtitle\", \"@required\": true, \"@maxLength\": 100 } }," + + " { \"field.string\": { \"name\": \"kind\", \"@required\": true, \"@default\": \"x\", \"@maxLength\": 10 } },", + ) + val out = gen("owned-required", json) + try { + val controller = Files.readString(out.resolve("acme/own/DocController.kt")) + val create = controller.substringAfter("fun create(").substringBefore("fun update(") + for (f in listOf("title", "subtitle")) { + assertTrue("if (!body.has(\"$f\") || body.get(\"$f\").isNull) issues.add(\"invalid_type\", listOf(\"$f\")" in create, create) + } + // A required value with a default binds without being sent, so it is never reported missing. + assertFalse("if (!body.has(\"kind\")" in create, create) + } finally { + out.toFile().deleteRecursively() + } + } + @Test fun `TPH create writes only the route subtype's own settable columns`() { val out = gen("owned-tph", tph) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesTest.kt new file mode 100644 index 0000000000..874f723038 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinValidationIssuesTest.kt @@ -0,0 +1,95 @@ +package com.metaobjects.generator.kotlin + +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import kotlin.test.Test +import kotlin.test.assertEquals + +/** + * The generated `MetaValidationIssues` and `MetaDecimalWire.unreadableBody`, compiled and run: the Zod-style code + * for each constraint, every violation listed with its path, and a binding failure classified. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class KotlinValidationIssuesTest { + + private val harness = """ + package acme.probe + + import jakarta.validation.Validation + import jakarta.validation.constraints.DecimalMax + import jakarta.validation.constraints.Digits + import jakarta.validation.constraints.Max + import jakarta.validation.constraints.Min + import jakarta.validation.constraints.NotNull + import jakarta.validation.constraints.Pattern + import jakarta.validation.constraints.Size + import jakarta.validation.Valid + import com.fasterxml.jackson.databind.ObjectMapper + import java.math.BigDecimal + + class Marker(@field:NotNull val label: String?) + class Bean( + @field:NotNull val required: String?, + @field:Size(min = 2, max = 4) val sized: String?, + @field:Max(5) val big: Int?, + @field:Min(5) val small: Int?, + @field:DecimalMax("1.5") val decBig: BigDecimal?, + @field:Digits(integer = 2, fraction = 1) val digits: BigDecimal?, + @field:Pattern(regexp = "[a-z]+") val pattern: String?, + @field:Valid val markers: List?, + ) + class Holder { var n: Int? = null } + + object Probe { + @JvmStatic fun codes(): Map { + val v = Validation.buildDefaultValidatorFactory().validator + val bean = Bean(null, "toolong", 9, 1, BigDecimal("2"), BigDecimal("123.45"), "NOPE", listOf(Marker("ok"), Marker(null))) + val body = MetaValidationIssues.Collector().addViolations(v.validate(bean)).body() + return (body["issues"] as List<*>).associate { (it as Map<*, *>)["path"].toString() to it["code"] as String } + } + @JvmStatic fun sizeBelowMin(): String { + val v = Validation.buildDefaultValidatorFactory().validator + val bean = Bean("x", "a", null, null, null, null, null, null) + return MetaValidationIssues.codeOf(v.validate(bean).single()) + } + @JvmStatic fun unreadable(): Map = + try { ObjectMapper().readValue("{\"n\": \"abc\"}", Holder::class.java); emptyMap() } + catch (e: Exception) { MetaDecimalWire.unreadableBody(e) } + } + """.trimIndent() + + @Test + fun `every violation is listed with its Zod code and nested path, and a bind failure is classified`() { + val sources = listOf( + SourceFile.kotlin("MetaDecimalWire.kt", "package acme.probe\n\n" + KotlinDecimalWireEmitter.BODY), + SourceFile.kotlin("MetaValidationIssues.kt", "package acme.probe\n\n" + KotlinValidationIssuesEmitter.BODY), + SourceFile.kotlin("Probe.kt", harness), + ) + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + val probe = result.classLoader.loadClass("acme.probe.Probe") + @Suppress("UNCHECKED_CAST") + val codes = probe.getMethod("codes").invoke(null) as Map + assertEquals("invalid_type", codes["[required]"]) + assertEquals("too_big", codes["[sized]"]) + assertEquals("too_big", codes["[big]"]) + assertEquals("too_small", codes["[small]"]) + assertEquals("too_big", codes["[decBig]"]) + assertEquals("too_big", codes["[digits]"]) + assertEquals("invalid_format", codes["[pattern]"]) + assertEquals("invalid_type", codes["[markers, 1, label]"]) + assertEquals(8, codes.size) + assertEquals("too_small", probe.getMethod("sizeBelowMin").invoke(null)) + @Suppress("UNCHECKED_CAST") + val body = probe.getMethod("unreadable").invoke(null) as Map + assertEquals("validation", body["error"]) + assertEquals("n", body["field"]) + val issue = (body["issues"] as List<*>).single() as Map<*, *> + assertEquals(listOf("n"), issue["path"]) + assertEquals("invalid_type", issue["code"]) + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt index aae2092002..d5355c70fa 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt @@ -326,10 +326,24 @@ class AuthorController(private val objectMapper: ObjectMapper, private val valid } @PostMapping - fun create(@RequestBody dto: Author): ResponseEntity = transaction { - if (validator.validate(dto).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val dto: Author = try { + objectMapper.treeToValue(body, Author::class.java) + } catch (e: Exception) { + if (!body.has("name") || body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' is required") + if (body.has("name") && !body.get("name").isNull) { + val vName: kotlin.String? = issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } + if (vName != null) issues.addAt(validator.validateValue(Author::class.java, "name", vName), "name") + } + issues.addFailure(e) + return@transaction ResponseEntity.badRequest().body(issues.body() as Any) + } + issues.addViolations(validator.validate(dto)) + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthorTable.insert { - it[name] = dto.name + it[AuthorTable.name] = dto.name }[AuthorTable.id] val saved = AuthorTable.selectAll().where { AuthorTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuthor(saved) as Any) @@ -337,18 +351,19 @@ class AuthorController(private val objectMapper: ObjectMapper, private val valid @RequestMapping(value = ["/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("name") && body.get("name").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name").any { body.has(it) }) { - try { + run { val hasName = body.has("name") - val vName: kotlin.String? = if (hasName) objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) else null - if (hasName && validator.validateValue(Author::class.java, "name", vName).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullName = hasName && body.get("name").isNull + val vName: kotlin.String? = if (hasName && !nullName) issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } else null + if (vName != null) issues.addAt(validator.validateValue(Author::class.java, "name", vName), "name") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) AuthorTable.update({ AuthorTable.id eq id }) { if (hasName) it[AuthorTable.name] = vName!! } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = AuthorTable.selectAll().where { AuthorTable.id eq id }.singleOrNull() diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaDecimalWire.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaDecimalWire.kt index 9e860ee65c..cb123d6f03 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaDecimalWire.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaDecimalWire.kt @@ -9,6 +9,7 @@ import com.fasterxml.jackson.databind.JsonMappingException import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.deser.ContextualDeserializer import com.fasterxml.jackson.databind.deser.std.StdDeserializer +import com.fasterxml.jackson.databind.exc.InvalidFormatException import java.math.BigDecimal import java.math.MathContext import java.math.RoundingMode @@ -83,18 +84,54 @@ internal object MetaDecimalWire { return value } - /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + * wrong JSON type, an unparseable value) invalid_type. + */ + fun failureCode(failure: Throwable): String { + var t: Throwable? = failure + while (t != null) { + val message = t.message + if (t is InvalidFormatException) { + val target = t.targetType + if (target != null) { + if (target.isEnum) return "invalid_value" + val n = target.name + if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + } + if (message != null && message.contains(": decimal out of range: ")) return "too_big" + } + if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + t = if (t.cause === t) null else t.cause + } + return "invalid_type" + } + + /** + * The validation envelope for a request body Jackson could not read: one issue whose path is every property + * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + */ fun unreadableBody(failure: Throwable): Map { - val out = LinkedHashMap() - out["error"] = "validation" + val path = ArrayList() var t: Throwable? = failure while (t != null) { if (t is JsonMappingException && t.path.isNotEmpty()) { - val name = t.path[t.path.size - 1].fieldName - if (name != null) { out["field"] = name; break } + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break } t = if (t.cause === t) null else t.cause } + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + val out = LinkedHashMap() + out["error"] = "validation" + out["issues"] = listOf(issue) + val field = path.lastOrNull { it is String } + if (field != null) out["field"] = field return out } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaValidationIssues.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaValidationIssues.kt new file mode 100644 index 0000000000..2b247a3dd4 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaValidationIssues.kt @@ -0,0 +1,105 @@ +package acme.blog + +import com.fasterxml.jackson.databind.JsonMappingException +import jakarta.validation.ConstraintViolation +import jakarta.validation.Path + +/** + * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + * before answering `400 {"error": "validation", "issues": [...]}`. + */ +internal object MetaValidationIssues { + /** Collects issues; [body] is the response body. */ + class Collector { + private val issues = ArrayList>() + + fun isEmpty(): Boolean = issues.isEmpty() + + fun add(code: String, path: List, message: String): Collector { + issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + return this + } + + /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + return this + } + + /** Every violation of a single-property validateValue, at exactly [path]. */ + fun addAt(violations: Collection>, vararg path: Any): Collector { + for (v in violations) add(codeOf(v), path.toList(), v.message) + return this + } + + /** Run [read]; a failure becomes an issue at [field] and yields null. */ + fun bind(field: String, read: () -> T): T? = + try { + read() + } catch (e: Exception) { + add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + null + } + + /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + fun addFailure(failure: Throwable): Collector { + val path = ArrayList() + var t: Throwable? = failure + while (t != null) { + if (t is JsonMappingException && t.path.isNotEmpty()) { + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break + } + t = if (t.cause === t) null else t.cause + } + if (issues.any { it["path"] == path }) return this + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + return add(MetaDecimalWire.failureCode(failure), path, message) + } + + fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + } + + /** A path as wire elements: property names, with array indexes and map keys. */ + fun pathOf(path: Path): List { + val out = ArrayList() + for (node in path) { + if (node.isInIterable) { + val index = node.index + val key = node.key + if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + } + val name = node.name + if (name != null) out.add(name) + } + return out + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + fun codeOf(v: ConstraintViolation<*>): String { + val descriptor = v.constraintDescriptor + val value = v.invalidValue + val attrs = descriptor.attributes + return when (descriptor.annotation.annotationClass.java.simpleName) { + "NotNull" -> "invalid_type" + "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + "Size", "Length" -> { + val max = attrs["max"] as? Number + val length: Long? = when (value) { + is CharSequence -> value.length.toLong() + is Collection<*> -> value.size.toLong() + is Map<*, *> -> value.size.toLong() + else -> null + } + if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + } + "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + "Pattern", "Email", "URL" -> "invalid_format" + else -> "custom" + } + } +} + diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaDecimalWire.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaDecimalWire.kt index 4b1d1eacdd..ae04f16d38 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaDecimalWire.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaDecimalWire.kt @@ -11,6 +11,7 @@ import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.SerializerProvider import com.fasterxml.jackson.databind.deser.ContextualDeserializer import com.fasterxml.jackson.databind.deser.std.StdDeserializer +import com.fasterxml.jackson.databind.exc.InvalidFormatException import com.fasterxml.jackson.databind.ser.std.StdSerializer import java.math.BigDecimal import java.math.MathContext @@ -86,18 +87,54 @@ internal object MetaDecimalWire { return value } - /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + * wrong JSON type, an unparseable value) invalid_type. + */ + fun failureCode(failure: Throwable): String { + var t: Throwable? = failure + while (t != null) { + val message = t.message + if (t is InvalidFormatException) { + val target = t.targetType + if (target != null) { + if (target.isEnum) return "invalid_value" + val n = target.name + if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + } + if (message != null && message.contains(": decimal out of range: ")) return "too_big" + } + if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + t = if (t.cause === t) null else t.cause + } + return "invalid_type" + } + + /** + * The validation envelope for a request body Jackson could not read: one issue whose path is every property + * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + */ fun unreadableBody(failure: Throwable): Map { - val out = LinkedHashMap() - out["error"] = "validation" + val path = ArrayList() var t: Throwable? = failure while (t != null) { if (t is JsonMappingException && t.path.isNotEmpty()) { - val name = t.path[t.path.size - 1].fieldName - if (name != null) { out["field"] = name; break } + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break } t = if (t.cause === t) null else t.cause } + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + val out = LinkedHashMap() + out["error"] = "validation" + out["issues"] = listOf(issue) + val field = path.lastOrNull { it is String } + if (field != null) out["field"] = field return out } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaValidationIssues.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaValidationIssues.kt new file mode 100644 index 0000000000..53fdbde3dc --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaValidationIssues.kt @@ -0,0 +1,105 @@ +package acme.catalog + +import com.fasterxml.jackson.databind.JsonMappingException +import jakarta.validation.ConstraintViolation +import jakarta.validation.Path + +/** + * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + * before answering `400 {"error": "validation", "issues": [...]}`. + */ +internal object MetaValidationIssues { + /** Collects issues; [body] is the response body. */ + class Collector { + private val issues = ArrayList>() + + fun isEmpty(): Boolean = issues.isEmpty() + + fun add(code: String, path: List, message: String): Collector { + issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + return this + } + + /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + return this + } + + /** Every violation of a single-property validateValue, at exactly [path]. */ + fun addAt(violations: Collection>, vararg path: Any): Collector { + for (v in violations) add(codeOf(v), path.toList(), v.message) + return this + } + + /** Run [read]; a failure becomes an issue at [field] and yields null. */ + fun bind(field: String, read: () -> T): T? = + try { + read() + } catch (e: Exception) { + add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + null + } + + /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + fun addFailure(failure: Throwable): Collector { + val path = ArrayList() + var t: Throwable? = failure + while (t != null) { + if (t is JsonMappingException && t.path.isNotEmpty()) { + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break + } + t = if (t.cause === t) null else t.cause + } + if (issues.any { it["path"] == path }) return this + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + return add(MetaDecimalWire.failureCode(failure), path, message) + } + + fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + } + + /** A path as wire elements: property names, with array indexes and map keys. */ + fun pathOf(path: Path): List { + val out = ArrayList() + for (node in path) { + if (node.isInIterable) { + val index = node.index + val key = node.key + if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + } + val name = node.name + if (name != null) out.add(name) + } + return out + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + fun codeOf(v: ConstraintViolation<*>): String { + val descriptor = v.constraintDescriptor + val value = v.invalidValue + val attrs = descriptor.attributes + return when (descriptor.annotation.annotationClass.java.simpleName) { + "NotNull" -> "invalid_type" + "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + "Size", "Length" -> { + val max = attrs["max"] as? Number + val length: Long? = when (value) { + is CharSequence -> value.length.toLong() + is Collection<*> -> value.size.toLong() + is Map<*, *> -> value.size.toLong() + else -> null + } + if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + } + "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + "Pattern", "Email", "URL" -> "invalid_format" + else -> "custom" + } + } +} + diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt index a9b18f9fee..f86a52e1dc 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt @@ -376,13 +376,36 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali } @PostMapping - fun create(@RequestBody dto: Product): ResponseEntity = transaction { - if (validator.validate(dto).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(dto.price, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) + fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val dto: Product = try { + objectMapper.treeToValue(body, Product::class.java) + } catch (e: Exception) { + if (!body.has("name") || body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' is required") + if (body.has("name") && !body.get("name").isNull) { + val vName: kotlin.String? = issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } + if (vName != null) issues.addAt(validator.validateValue(Product::class.java, "name", vName), "name") + } + if (body.has("price") && !body.get("price").isNull) { + val vPrice: java.math.BigDecimal? = issues.bind("price") { MetaDecimalWire.fromNode(body.get("price")) } + if (vPrice != null) issues.addAt(validator.validateValue(Product::class.java, "price", vPrice), "price") + if (!MetaDecimalWire.fits(vPrice, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") + } + if (body.has("rating") && !body.get("rating").isNull) { + val vRating: kotlin.Float? = issues.bind("rating") { objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) } + if (vRating != null) issues.addAt(validator.validateValue(Product::class.java, "rating", vRating), "rating") + } + issues.addFailure(e) + return@transaction ResponseEntity.badRequest().body(issues.body() as Any) + } + issues.addViolations(validator.validate(dto)) + if (!MetaDecimalWire.fits(dto.price, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = ProductTable.insert { - it[name] = dto.name - it[price] = dto.price - it[rating] = dto.rating + it[ProductTable.name] = dto.name + it[ProductTable.price] = dto.price + it[ProductTable.rating] = dto.rating }[ProductTable.id] val saved = ProductTable.selectAll().where { ProductTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToProduct(saved) as Any) @@ -390,29 +413,30 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali @RequestMapping(value = ["/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("name") && body.get("name").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name", "price", "rating").any { body.has(it) }) { - try { + run { val hasName = body.has("name") - val vName: kotlin.String? = if (hasName) objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) else null - if (hasName && validator.validateValue(Product::class.java, "name", vName).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullName = hasName && body.get("name").isNull + val vName: kotlin.String? = if (hasName && !nullName) issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } else null + if (vName != null) issues.addAt(validator.validateValue(Product::class.java, "name", vName), "name") val hasPrice = body.has("price") val nullPrice = hasPrice && body.get("price").isNull - val vPrice: java.math.BigDecimal? = if (hasPrice && !nullPrice) runCatching { MetaDecimalWire.fromNode(body.get("price")) }.getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) } else null - if (hasPrice && !nullPrice && validator.validateValue(Product::class.java, "price", vPrice).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(vPrice, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) + val vPrice: java.math.BigDecimal? = if (hasPrice && !nullPrice) issues.bind("price") { MetaDecimalWire.fromNode(body.get("price")) } else null + if (vPrice != null) issues.addAt(validator.validateValue(Product::class.java, "price", vPrice), "price") + if (!MetaDecimalWire.fits(vPrice, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") val hasRating = body.has("rating") val nullRating = hasRating && body.get("rating").isNull - val vRating: kotlin.Float? = if (hasRating && !nullRating) objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) else null - if (hasRating && !nullRating && validator.validateValue(Product::class.java, "rating", vRating).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val vRating: kotlin.Float? = if (hasRating && !nullRating) issues.bind("rating") { objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) } else null + if (vRating != null) issues.addAt(validator.validateValue(Product::class.java, "rating", vRating), "rating") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) ProductTable.update({ ProductTable.id eq id }) { if (hasName) it[ProductTable.name] = vName!! if (hasPrice) { if (nullPrice) it[ProductTable.price] = null else it[ProductTable.price] = vPrice } if (hasRating) { if (nullRating) it[ProductTable.rating] = null else it[ProductTable.rating] = vRating } } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = ProductTable.selectAll().where { ProductTable.id eq id }.singleOrNull() diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaDecimalWire.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaDecimalWire.kt index 4e2a5f02f3..c34308cfd1 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaDecimalWire.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaDecimalWire.kt @@ -9,6 +9,7 @@ import com.fasterxml.jackson.databind.JsonMappingException import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.deser.ContextualDeserializer import com.fasterxml.jackson.databind.deser.std.StdDeserializer +import com.fasterxml.jackson.databind.exc.InvalidFormatException import java.math.BigDecimal import java.math.MathContext import java.math.RoundingMode @@ -83,18 +84,54 @@ internal object MetaDecimalWire { return value } - /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + * wrong JSON type, an unparseable value) invalid_type. + */ + fun failureCode(failure: Throwable): String { + var t: Throwable? = failure + while (t != null) { + val message = t.message + if (t is InvalidFormatException) { + val target = t.targetType + if (target != null) { + if (target.isEnum) return "invalid_value" + val n = target.name + if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + } + if (message != null && message.contains(": decimal out of range: ")) return "too_big" + } + if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + t = if (t.cause === t) null else t.cause + } + return "invalid_type" + } + + /** + * The validation envelope for a request body Jackson could not read: one issue whose path is every property + * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + */ fun unreadableBody(failure: Throwable): Map { - val out = LinkedHashMap() - out["error"] = "validation" + val path = ArrayList() var t: Throwable? = failure while (t != null) { if (t is JsonMappingException && t.path.isNotEmpty()) { - val name = t.path[t.path.size - 1].fieldName - if (name != null) { out["field"] = name; break } + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break } t = if (t.cause === t) null else t.cause } + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + val out = LinkedHashMap() + out["error"] = "validation" + out["issues"] = listOf(issue) + val field = path.lastOrNull { it is String } + if (field != null) out["field"] = field return out } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaValidationIssues.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaValidationIssues.kt new file mode 100644 index 0000000000..53fdbde3dc --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaValidationIssues.kt @@ -0,0 +1,105 @@ +package acme.catalog + +import com.fasterxml.jackson.databind.JsonMappingException +import jakarta.validation.ConstraintViolation +import jakarta.validation.Path + +/** + * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + * before answering `400 {"error": "validation", "issues": [...]}`. + */ +internal object MetaValidationIssues { + /** Collects issues; [body] is the response body. */ + class Collector { + private val issues = ArrayList>() + + fun isEmpty(): Boolean = issues.isEmpty() + + fun add(code: String, path: List, message: String): Collector { + issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + return this + } + + /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + return this + } + + /** Every violation of a single-property validateValue, at exactly [path]. */ + fun addAt(violations: Collection>, vararg path: Any): Collector { + for (v in violations) add(codeOf(v), path.toList(), v.message) + return this + } + + /** Run [read]; a failure becomes an issue at [field] and yields null. */ + fun bind(field: String, read: () -> T): T? = + try { + read() + } catch (e: Exception) { + add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + null + } + + /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + fun addFailure(failure: Throwable): Collector { + val path = ArrayList() + var t: Throwable? = failure + while (t != null) { + if (t is JsonMappingException && t.path.isNotEmpty()) { + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break + } + t = if (t.cause === t) null else t.cause + } + if (issues.any { it["path"] == path }) return this + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + return add(MetaDecimalWire.failureCode(failure), path, message) + } + + fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + } + + /** A path as wire elements: property names, with array indexes and map keys. */ + fun pathOf(path: Path): List { + val out = ArrayList() + for (node in path) { + if (node.isInIterable) { + val index = node.index + val key = node.key + if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + } + val name = node.name + if (name != null) out.add(name) + } + return out + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + fun codeOf(v: ConstraintViolation<*>): String { + val descriptor = v.constraintDescriptor + val value = v.invalidValue + val attrs = descriptor.attributes + return when (descriptor.annotation.annotationClass.java.simpleName) { + "NotNull" -> "invalid_type" + "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + "Size", "Length" -> { + val max = attrs["max"] as? Number + val length: Long? = when (value) { + is CharSequence -> value.length.toLong() + is Collection<*> -> value.size.toLong() + is Map<*, *> -> value.size.toLong() + else -> null + } + if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + } + "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + "Pattern", "Email", "URL" -> "invalid_format" + else -> "custom" + } + } +} + diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt index a9b18f9fee..f86a52e1dc 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt @@ -376,13 +376,36 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali } @PostMapping - fun create(@RequestBody dto: Product): ResponseEntity = transaction { - if (validator.validate(dto).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(dto.price, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) + fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val dto: Product = try { + objectMapper.treeToValue(body, Product::class.java) + } catch (e: Exception) { + if (!body.has("name") || body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' is required") + if (body.has("name") && !body.get("name").isNull) { + val vName: kotlin.String? = issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } + if (vName != null) issues.addAt(validator.validateValue(Product::class.java, "name", vName), "name") + } + if (body.has("price") && !body.get("price").isNull) { + val vPrice: java.math.BigDecimal? = issues.bind("price") { MetaDecimalWire.fromNode(body.get("price")) } + if (vPrice != null) issues.addAt(validator.validateValue(Product::class.java, "price", vPrice), "price") + if (!MetaDecimalWire.fits(vPrice, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") + } + if (body.has("rating") && !body.get("rating").isNull) { + val vRating: kotlin.Float? = issues.bind("rating") { objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) } + if (vRating != null) issues.addAt(validator.validateValue(Product::class.java, "rating", vRating), "rating") + } + issues.addFailure(e) + return@transaction ResponseEntity.badRequest().body(issues.body() as Any) + } + issues.addViolations(validator.validate(dto)) + if (!MetaDecimalWire.fits(dto.price, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = ProductTable.insert { - it[name] = dto.name - it[price] = dto.price - it[rating] = dto.rating + it[ProductTable.name] = dto.name + it[ProductTable.price] = dto.price + it[ProductTable.rating] = dto.rating }[ProductTable.id] val saved = ProductTable.selectAll().where { ProductTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToProduct(saved) as Any) @@ -390,29 +413,30 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali @RequestMapping(value = ["/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("name") && body.get("name").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name", "price", "rating").any { body.has(it) }) { - try { + run { val hasName = body.has("name") - val vName: kotlin.String? = if (hasName) objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) else null - if (hasName && validator.validateValue(Product::class.java, "name", vName).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullName = hasName && body.get("name").isNull + val vName: kotlin.String? = if (hasName && !nullName) issues.bind("name") { objectMapper.treeToValue(body.get("name"), object : TypeReference() {}) } else null + if (vName != null) issues.addAt(validator.validateValue(Product::class.java, "name", vName), "name") val hasPrice = body.has("price") val nullPrice = hasPrice && body.get("price").isNull - val vPrice: java.math.BigDecimal? = if (hasPrice && !nullPrice) runCatching { MetaDecimalWire.fromNode(body.get("price")) }.getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) } else null - if (hasPrice && !nullPrice && validator.validateValue(Product::class.java, "price", vPrice).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(vPrice, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "price") as Any) + val vPrice: java.math.BigDecimal? = if (hasPrice && !nullPrice) issues.bind("price") { MetaDecimalWire.fromNode(body.get("price")) } else null + if (vPrice != null) issues.addAt(validator.validateValue(Product::class.java, "price", vPrice), "price") + if (!MetaDecimalWire.fits(vPrice, 10, 2)) issues.add("too_big", listOf("price"), "decimal does not fit the declared precision and scale") val hasRating = body.has("rating") val nullRating = hasRating && body.get("rating").isNull - val vRating: kotlin.Float? = if (hasRating && !nullRating) objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) else null - if (hasRating && !nullRating && validator.validateValue(Product::class.java, "rating", vRating).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val vRating: kotlin.Float? = if (hasRating && !nullRating) issues.bind("rating") { objectMapper.treeToValue(body.get("rating"), object : TypeReference() {}) } else null + if (vRating != null) issues.addAt(validator.validateValue(Product::class.java, "rating", vRating), "rating") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) ProductTable.update({ ProductTable.id eq id }) { if (hasName) it[ProductTable.name] = vName!! if (hasPrice) { if (nullPrice) it[ProductTable.price] = null else it[ProductTable.price] = vPrice } if (hasRating) { if (nullRating) it[ProductTable.rating] = null else it[ProductTable.rating] = vRating } } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = ProductTable.selectAll().where { ProductTable.id eq id }.singleOrNull() diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt index 00257684f0..f1a7f8a27c 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt @@ -419,8 +419,10 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @PostMapping("/bridge") fun createBridge(@RequestBody dto: Auth): ResponseEntity = transaction { - if (validator.validateValue(BridgeAuthValidation::class.java, "quantity", dto.quantity).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (validator.validateValue(BridgeAuthValidation::class.java, "reference", dto.reference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "quantity", dto.quantity), "quantity") + issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "reference", dto.reference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { it[type] = AuthType.Bridge it[reference] = dto.reference!! @@ -432,23 +434,25 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @RequestMapping(value = ["/bridge/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun updateBridge(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("quantity") && body.get("quantity").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("reference") && body.get("reference").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("quantity") && body.get("quantity").isNull) issues.add("invalid_type", listOf("quantity"), "field 'quantity' cannot be null") + if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("quantity", "reference").any { body.has(it) }) { - try { + run { val hasQuantity = body.has("quantity") - val vQuantity: kotlin.Int? = if (hasQuantity) objectMapper.treeToValue(body.get("quantity"), object : TypeReference() {}) else null - if (hasQuantity && validator.validateValue(BridgeAuthValidation::class.java, "quantity", vQuantity).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullQuantity = hasQuantity && body.get("quantity").isNull + val vQuantity: kotlin.Int? = if (hasQuantity && !nullQuantity) issues.bind("quantity") { objectMapper.treeToValue(body.get("quantity"), object : TypeReference() {}) } else null + if (vQuantity != null) issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "quantity", vQuantity), "quantity") val hasReference = body.has("reference") - val vReference: kotlin.String? = if (hasReference) objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) else null - if (hasReference && validator.validateValue(BridgeAuthValidation::class.java, "reference", vReference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullReference = hasReference && body.get("reference").isNull + val vReference: kotlin.String? = if (hasReference && !nullReference) issues.bind("reference") { objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) } else null + if (vReference != null) issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "reference", vReference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) AuthTable.update({ (AuthTable.id eq id) and (AuthTable.type eq AuthType.Bridge) }) { if (hasQuantity) it[AuthTable.quantity] = vQuantity!! if (hasReference) it[AuthTable.reference] = vReference!! } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = AuthTable.selectAll().where { (AuthTable.id eq id) and (AuthTable.type eq AuthType.Bridge) }.singleOrNull() @@ -502,9 +506,11 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @PostMapping("/copay") fun createCopay(@RequestBody dto: Auth): ResponseEntity = transaction { - if (validator.validateValue(CopayAuthValidation::class.java, "copayAmount", dto.copayAmount).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(dto.copayAmount, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "copayAmount") as Any) - if (validator.validateValue(CopayAuthValidation::class.java, "reference", dto.reference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "copayAmount", dto.copayAmount), "copayAmount") + if (!MetaDecimalWire.fits(dto.copayAmount, 10, 2)) issues.add("too_big", listOf("copayAmount"), "decimal does not fit the declared precision and scale") + issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "reference", dto.reference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { it[type] = AuthType.Copay it[reference] = dto.reference!! @@ -516,24 +522,25 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @RequestMapping(value = ["/copay/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun updateCopay(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("reference") && body.get("reference").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("copayAmount", "reference").any { body.has(it) }) { - try { + run { val hasCopayAmount = body.has("copayAmount") val nullCopayAmount = hasCopayAmount && body.get("copayAmount").isNull - val vCopayAmount: java.math.BigDecimal? = if (hasCopayAmount && !nullCopayAmount) runCatching { MetaDecimalWire.fromNode(body.get("copayAmount")) }.getOrElse { return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "copayAmount") as Any) } else null - if (hasCopayAmount && !nullCopayAmount && validator.validateValue(CopayAuthValidation::class.java, "copayAmount", vCopayAmount).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (!MetaDecimalWire.fits(vCopayAmount, 10, 2)) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation", "field" to "copayAmount") as Any) + val vCopayAmount: java.math.BigDecimal? = if (hasCopayAmount && !nullCopayAmount) issues.bind("copayAmount") { MetaDecimalWire.fromNode(body.get("copayAmount")) } else null + if (vCopayAmount != null) issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "copayAmount", vCopayAmount), "copayAmount") + if (!MetaDecimalWire.fits(vCopayAmount, 10, 2)) issues.add("too_big", listOf("copayAmount"), "decimal does not fit the declared precision and scale") val hasReference = body.has("reference") - val vReference: kotlin.String? = if (hasReference) objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) else null - if (hasReference && validator.validateValue(CopayAuthValidation::class.java, "reference", vReference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullReference = hasReference && body.get("reference").isNull + val vReference: kotlin.String? = if (hasReference && !nullReference) issues.bind("reference") { objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) } else null + if (vReference != null) issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "reference", vReference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) AuthTable.update({ (AuthTable.id eq id) and (AuthTable.type eq AuthType.Copay) }) { if (hasCopayAmount) { if (nullCopayAmount) it[AuthTable.copayAmount] = null else it[AuthTable.copayAmount] = vCopayAmount } if (hasReference) it[AuthTable.reference] = vReference!! } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = AuthTable.selectAll().where { (AuthTable.id eq id) and (AuthTable.type eq AuthType.Copay) }.singleOrNull() @@ -587,8 +594,10 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @PostMapping("/priorauth") fun createPriorAuth(@RequestBody dto: Auth): ResponseEntity = transaction { - if (validator.validateValue(PriorAuthAuthValidation::class.java, "priorAuthNumber", dto.priorAuthNumber).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (validator.validateValue(PriorAuthAuthValidation::class.java, "reference", dto.reference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "priorAuthNumber", dto.priorAuthNumber), "priorAuthNumber") + issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "reference", dto.reference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { it[type] = AuthType.PriorAuth it[reference] = dto.reference!! @@ -600,23 +609,24 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat @RequestMapping(value = ["/priorauth/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun updatePriorAuth(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) - if (body.has("reference") && body.get("reference").isNull) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("priorAuthNumber", "reference").any { body.has(it) }) { - try { + run { val hasPriorAuthNumber = body.has("priorAuthNumber") val nullPriorAuthNumber = hasPriorAuthNumber && body.get("priorAuthNumber").isNull - val vPriorAuthNumber: kotlin.String? = if (hasPriorAuthNumber && !nullPriorAuthNumber) objectMapper.treeToValue(body.get("priorAuthNumber"), object : TypeReference() {}) else null - if (hasPriorAuthNumber && !nullPriorAuthNumber && validator.validateValue(PriorAuthAuthValidation::class.java, "priorAuthNumber", vPriorAuthNumber).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val vPriorAuthNumber: kotlin.String? = if (hasPriorAuthNumber && !nullPriorAuthNumber) issues.bind("priorAuthNumber") { objectMapper.treeToValue(body.get("priorAuthNumber"), object : TypeReference() {}) } else null + if (vPriorAuthNumber != null) issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "priorAuthNumber", vPriorAuthNumber), "priorAuthNumber") val hasReference = body.has("reference") - val vReference: kotlin.String? = if (hasReference) objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) else null - if (hasReference && validator.validateValue(PriorAuthAuthValidation::class.java, "reference", vReference).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val nullReference = hasReference && body.get("reference").isNull + val vReference: kotlin.String? = if (hasReference && !nullReference) issues.bind("reference") { objectMapper.treeToValue(body.get("reference"), object : TypeReference() {}) } else null + if (vReference != null) issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "reference", vReference), "reference") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) AuthTable.update({ (AuthTable.id eq id) and (AuthTable.type eq AuthType.PriorAuth) }) { if (hasPriorAuthNumber) { if (nullPriorAuthNumber) it[AuthTable.priorAuthNumber] = null else it[AuthTable.priorAuthNumber] = vPriorAuthNumber } if (hasReference) it[AuthTable.reference] = vReference!! } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = AuthTable.selectAll().where { (AuthTable.id eq id) and (AuthTable.type eq AuthType.PriorAuth) }.singleOrNull() diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt index 02302b7b42..5a4b1c1daa 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt @@ -325,10 +325,23 @@ class AuthLineController(private val objectMapper: ObjectMapper, private val val } @PostMapping - fun create(@RequestBody dto: AuthLine): ResponseEntity = transaction { - if (validator.validate(dto).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val dto: AuthLine = try { + objectMapper.treeToValue(body, AuthLine::class.java) + } catch (e: Exception) { + if (body.has("label") && !body.get("label").isNull) { + val vLabel: kotlin.String? = issues.bind("label") { objectMapper.treeToValue(body.get("label"), object : TypeReference() {}) } + if (vLabel != null) issues.addAt(validator.validateValue(AuthLine::class.java, "label", vLabel), "label") + } + issues.addFailure(e) + return@transaction ResponseEntity.badRequest().body(issues.body() as Any) + } + issues.addViolations(validator.validate(dto)) + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthLineTable.insert { - it[label] = dto.label + it[AuthLineTable.label] = dto.label }[AuthLineTable.id] val saved = AuthLineTable.selectAll().where { AuthLineTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuthLine(saved) as Any) @@ -336,18 +349,18 @@ class AuthLineController(private val objectMapper: ObjectMapper, private val val @RequestMapping(value = ["/{id}"], method = [RequestMethod.PATCH, RequestMethod.PUT]) fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { - if (!body.isObject) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val issues = MetaValidationIssues.Collector() + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) if (listOf("label").any { body.has(it) }) { - try { + run { val hasLabel = body.has("label") val nullLabel = hasLabel && body.get("label").isNull - val vLabel: kotlin.String? = if (hasLabel && !nullLabel) objectMapper.treeToValue(body.get("label"), object : TypeReference() {}) else null - if (hasLabel && !nullLabel && validator.validateValue(AuthLine::class.java, "label", vLabel).isNotEmpty()) return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) + val vLabel: kotlin.String? = if (hasLabel && !nullLabel) issues.bind("label") { objectMapper.treeToValue(body.get("label"), object : TypeReference() {}) } else null + if (vLabel != null) issues.addAt(validator.validateValue(AuthLine::class.java, "label", vLabel), "label") + if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) AuthLineTable.update({ AuthLineTable.id eq id }) { if (hasLabel) { if (nullLabel) it[AuthLineTable.label] = null else it[AuthLineTable.label] = vLabel } } - } catch (e: com.fasterxml.jackson.databind.JsonMappingException) { - return@transaction ResponseEntity.badRequest().body(mapOf("error" to "validation") as Any) } } val row = AuthLineTable.selectAll().where { AuthLineTable.id eq id }.singleOrNull() diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaDecimalWire.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaDecimalWire.kt index 2b74e50c19..41ed76a152 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaDecimalWire.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaDecimalWire.kt @@ -9,6 +9,7 @@ import com.fasterxml.jackson.databind.JsonMappingException import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.deser.ContextualDeserializer import com.fasterxml.jackson.databind.deser.std.StdDeserializer +import com.fasterxml.jackson.databind.exc.InvalidFormatException import java.math.BigDecimal import java.math.MathContext import java.math.RoundingMode @@ -83,18 +84,54 @@ internal object MetaDecimalWire { return value } - /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + * wrong JSON type, an unparseable value) invalid_type. + */ + fun failureCode(failure: Throwable): String { + var t: Throwable? = failure + while (t != null) { + val message = t.message + if (t is InvalidFormatException) { + val target = t.targetType + if (target != null) { + if (target.isEnum) return "invalid_value" + val n = target.name + if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + } + if (message != null && message.contains(": decimal out of range: ")) return "too_big" + } + if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + t = if (t.cause === t) null else t.cause + } + return "invalid_type" + } + + /** + * The validation envelope for a request body Jackson could not read: one issue whose path is every property + * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + */ fun unreadableBody(failure: Throwable): Map { - val out = LinkedHashMap() - out["error"] = "validation" + val path = ArrayList() var t: Throwable? = failure while (t != null) { if (t is JsonMappingException && t.path.isNotEmpty()) { - val name = t.path[t.path.size - 1].fieldName - if (name != null) { out["field"] = name; break } + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break } t = if (t.cause === t) null else t.cause } + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + val out = LinkedHashMap() + out["error"] = "validation" + out["issues"] = listOf(issue) + val field = path.lastOrNull { it is String } + if (field != null) out["field"] = field return out } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaValidationIssues.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaValidationIssues.kt new file mode 100644 index 0000000000..d8d44e2696 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaValidationIssues.kt @@ -0,0 +1,105 @@ +package acme.auth + +import com.fasterxml.jackson.databind.JsonMappingException +import jakarta.validation.ConstraintViolation +import jakarta.validation.Path + +/** + * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + * before answering `400 {"error": "validation", "issues": [...]}`. + */ +internal object MetaValidationIssues { + /** Collects issues; [body] is the response body. */ + class Collector { + private val issues = ArrayList>() + + fun isEmpty(): Boolean = issues.isEmpty() + + fun add(code: String, path: List, message: String): Collector { + issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + return this + } + + /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + return this + } + + /** Every violation of a single-property validateValue, at exactly [path]. */ + fun addAt(violations: Collection>, vararg path: Any): Collector { + for (v in violations) add(codeOf(v), path.toList(), v.message) + return this + } + + /** Run [read]; a failure becomes an issue at [field] and yields null. */ + fun bind(field: String, read: () -> T): T? = + try { + read() + } catch (e: Exception) { + add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + null + } + + /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + fun addFailure(failure: Throwable): Collector { + val path = ArrayList() + var t: Throwable? = failure + while (t != null) { + if (t is JsonMappingException && t.path.isNotEmpty()) { + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break + } + t = if (t.cause === t) null else t.cause + } + if (issues.any { it["path"] == path }) return this + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + return add(MetaDecimalWire.failureCode(failure), path, message) + } + + fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + } + + /** A path as wire elements: property names, with array indexes and map keys. */ + fun pathOf(path: Path): List { + val out = ArrayList() + for (node in path) { + if (node.isInIterable) { + val index = node.index + val key = node.key + if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + } + val name = node.name + if (name != null) out.add(name) + } + return out + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + fun codeOf(v: ConstraintViolation<*>): String { + val descriptor = v.constraintDescriptor + val value = v.invalidValue + val attrs = descriptor.attributes + return when (descriptor.annotation.annotationClass.java.simpleName) { + "NotNull" -> "invalid_type" + "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + "Size", "Length" -> { + val max = attrs["max"] as? Number + val length: Long? = when (value) { + is CharSequence -> value.length.toLong() + is Collection<*> -> value.size.toLong() + is Map<*, *> -> value.size.toLong() + else -> null + } + if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + } + "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + "Pattern", "Email", "URL" -> "invalid_format" + else -> "custom" + } + } +} + diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaDecimalWire.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaDecimalWire.kt index ca96d74908..21e15124fc 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaDecimalWire.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaDecimalWire.kt @@ -9,6 +9,7 @@ import com.fasterxml.jackson.databind.JsonMappingException import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.deser.ContextualDeserializer import com.fasterxml.jackson.databind.deser.std.StdDeserializer +import com.fasterxml.jackson.databind.exc.InvalidFormatException import java.math.BigDecimal import java.math.MathContext import java.math.RoundingMode @@ -83,18 +84,54 @@ internal object MetaDecimalWire { return value } - /** The validation envelope for a request body Jackson could not read: names the field when it knows it. */ + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is too_big, an + * unknown enum member invalid_value, a malformed date/time/uuid/uri invalid_format, anything else (a + * wrong JSON type, an unparseable value) invalid_type. + */ + fun failureCode(failure: Throwable): String { + var t: Throwable? = failure + while (t != null) { + val message = t.message + if (t is InvalidFormatException) { + val target = t.targetType + if (target != null) { + if (target.isEnum) return "invalid_value" + val n = target.name + if (n.startsWith("java.time.") || n == "java.util.UUID" || n == "java.net.URI" || + n == "java.net.URL" || n == "java.util.Date") return "invalid_format" + } + if (message != null && message.contains(": decimal out of range: ")) return "too_big" + } + if (t is IllegalArgumentException && message != null && message.startsWith("decimal out of range: ")) return "too_big" + t = if (t.cause === t) null else t.cause + } + return "invalid_type" + } + + /** + * The validation envelope for a request body Jackson could not read: one issue whose path is every property + * Jackson was binding (empty when the body as a whole cannot be read), plus the last field name as `field`. + */ fun unreadableBody(failure: Throwable): Map { - val out = LinkedHashMap() - out["error"] = "validation" + val path = ArrayList() var t: Throwable? = failure while (t != null) { if (t is JsonMappingException && t.path.isNotEmpty()) { - val name = t.path[t.path.size - 1].fieldName - if (name != null) { out["field"] = name; break } + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break } t = if (t.cause === t) null else t.cause } + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + val issue = linkedMapOf("code" to failureCode(failure), "path" to path, "message" to message) + val out = LinkedHashMap() + out["error"] = "validation" + out["issues"] = listOf(issue) + val field = path.lastOrNull { it is String } + if (field != null) out["field"] = field return out } diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaValidationIssues.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaValidationIssues.kt new file mode 100644 index 0000000000..ffdeba44c2 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaValidationIssues.kt @@ -0,0 +1,105 @@ +package acme.report + +import com.fasterxml.jackson.databind.JsonMappingException +import jakarta.validation.ConstraintViolation +import jakarta.validation.Path + +/** + * GENERATED — do not hand-edit. The validation envelope a controller collects every issue into + * before answering `400 {"error": "validation", "issues": [...]}`. + */ +internal object MetaValidationIssues { + /** Collects issues; [body] is the response body. */ + class Collector { + private val issues = ArrayList>() + + fun isEmpty(): Boolean = issues.isEmpty() + + fun add(code: String, path: List, message: String): Collector { + issues.add(linkedMapOf("code" to code, "path" to ArrayList(path), "message" to message)) + return this + } + + /** Every violation of a whole-bean validate, its path read from the violation after [prefix]. */ + fun addViolations(violations: Collection>, vararg prefix: Any): Collector { + for (v in violations) add(codeOf(v), prefix.toList() + pathOf(v.propertyPath), v.message) + return this + } + + /** Every violation of a single-property validateValue, at exactly [path]. */ + fun addAt(violations: Collection>, vararg path: Any): Collector { + for (v in violations) add(codeOf(v), path.toList(), v.message) + return this + } + + /** Run [read]; a failure becomes an issue at [field] and yields null. */ + fun bind(field: String, read: () -> T): T? = + try { + read() + } catch (e: Exception) { + add(MetaDecimalWire.failureCode(e), listOf(field), "field '" + field + "' is not a valid value") + null + } + + /** A body that could not be bound as a whole: an issue at the field it names (skipped if already listed). */ + fun addFailure(failure: Throwable): Collector { + val path = ArrayList() + var t: Throwable? = failure + while (t != null) { + if (t is JsonMappingException && t.path.isNotEmpty()) { + for (ref in t.path) { + if (ref.fieldName != null) path.add(ref.fieldName) else if (ref.index >= 0) path.add(ref.index) + } + break + } + t = if (t.cause === t) null else t.cause + } + if (issues.any { it["path"] == path }) return this + val message = if (path.isEmpty()) "request body is not readable" else "field '" + path[path.size - 1] + "' is not a valid value" + return add(MetaDecimalWire.failureCode(failure), path, message) + } + + fun body(): Map = linkedMapOf("error" to "validation", "issues" to issues) + } + + /** A path as wire elements: property names, with array indexes and map keys. */ + fun pathOf(path: Path): List { + val out = ArrayList() + for (node in path) { + if (node.isInIterable) { + val index = node.index + val key = node.key + if (index != null) out.add(index) else if (key != null) out.add(key.toString()) + } + val name = node.name + if (name != null) out.add(name) + } + return out + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + fun codeOf(v: ConstraintViolation<*>): String { + val descriptor = v.constraintDescriptor + val value = v.invalidValue + val attrs = descriptor.attributes + return when (descriptor.annotation.annotationClass.java.simpleName) { + "NotNull" -> "invalid_type" + "NotBlank", "NotEmpty" -> if (value == null) "invalid_type" else "too_small" + "Size", "Length" -> { + val max = attrs["max"] as? Number + val length: Long? = when (value) { + is CharSequence -> value.length.toLong() + is Collection<*> -> value.size.toLong() + is Map<*, *> -> value.size.toLong() + else -> null + } + if (max != null && length != null && length > max.toLong()) "too_big" else "too_small" + } + "Max", "DecimalMax", "Digits", "Negative", "NegativeOrZero" -> "too_big" + "Min", "DecimalMin", "Positive", "PositiveOrZero" -> "too_small" + "Pattern", "Email", "URL" -> "invalid_format" + else -> "custom" + } + } +} + diff --git a/server/java/codegen-spring/pom.xml b/server/java/codegen-spring/pom.xml index 66404220af..20849a8ae2 100644 --- a/server/java/codegen-spring/pom.xml +++ b/server/java/codegen-spring/pom.xml @@ -107,6 +107,16 @@ provided + + + jakarta.validation + jakarta.validation-api + 3.0.2 + provided + + @@ -298,6 +308,7 @@ RecordComponentNames.java ReportExport.java UnreadableBody.java + ValidationIssues.java diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java index 7917ebd62b..45c28e04a5 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java @@ -274,7 +274,7 @@ public String toString() { * {@code REGISTRY}: static initializers run in order. */ private static final List ROUTES_RUNTIME = List.of( "CanonicalDecimal", "CanonicalDuration", "ConstraintErrors", "FilterParseResult", "FilterParser", "FilterPredicate", - "PatchValidationException", "RecordComponentNames", "ReportExport", "UnreadableBody"); + "PatchValidationException", "RecordComponentNames", "ReportExport", "UnreadableBody", "ValidationIssues"); /** The writer a generated type's package and class name are read from, and its base. */ private static final List OBJECT_NAMING = List.of( diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md index 8ba9f4e437..bcf9916b0d 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md @@ -116,7 +116,8 @@ The generated `PATCH`/`PUT` handler binds a raw `JsonNode` and builds an top of it:** the handler injects a `jakarta.validation.Validator` and runs `validateValue(Dto.class, field, value)` on each PRESENT value (per-field, never the whole bean — so an absent `@required` field is not a false 400) → -HTTP 400 `{"error":"validation"}`. This holds on both the vanilla and (FR-036 +HTTP 400 `{"error":"validation", "issues": [...]}`, every violation listed through the +`ValidationIssues` runtime helper (since 1.3.0). This holds on both the vanilla and (FR-036 Program B) the TPH per-subtype update paths. (The prior note here — and the claim that "TS/Python DO validate on PATCH" — was wrong: before FR-036 only TS's vanilla path validated present values; the TPH path validated on no port. All ports now do.) diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java index ad4115af50..1e23e29939 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java @@ -262,6 +262,7 @@ protected void emit(MetaObject entity, Path outRoot) { src.append("import ").append(runtimePackage).append(".ConstraintErrors;\n"); src.append("import ").append(runtimePackage).append(".PatchValidationException;\n"); src.append("import ").append(runtimePackage).append(".UnreadableBody;\n"); + src.append("import ").append(runtimePackage).append(".ValidationIssues;\n"); // The escape the PATCH loop applies to each assigned key before naming a Bean // Validation property — see RecordComponentNames for why it is needed at run time. src.append("import ").append(runtimePackage).append(".RecordComponentNames;\n"); @@ -377,15 +378,15 @@ protected void emit(MetaObject entity, Path outRoot) { // it is what cascades @Valid into nested value objects. The violations are FILTERED, // and only by TOP-LEVEL property name — a nested path like "labels[0].code" can never // match, so cascaded constraints still reject. + // Every violation is listed (never just the first): the issues are collected, then answered once. if (serverOwnedOnCreate(entity).isEmpty()) { - // Byte-identical to the pre-fix handler for an entity with no server-owned column, - // so this change moves only the output it has to. - src.append(" if (!validator.validate(dto).isEmpty()) {\n"); + src.append(" ValidationIssues __issues = new ValidationIssues().addViolations(validator.validate(dto));\n"); } else { - src.append(" if (validator.validate(dto).stream()\n"); - src.append(" .anyMatch(v -> !SERVER_OWNED_ON_CREATE.contains(v.getPropertyPath().toString()))) {\n"); + src.append(" ValidationIssues __issues = new ValidationIssues()\n"); + src.append(" .addViolationsExcept(validator.validate(dto), SERVER_OWNED_ON_CREATE);\n"); } - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); + src.append(" if (!__issues.isEmpty()) {\n"); + src.append(" return ResponseEntity.badRequest().body(__issues.body());\n"); src.append(" }\n"); // Issue #203: honor @autoSet — stamp EVERY onCreate AND onUpdate column with now() before // persisting (the caller's value is ignored; a fresh row's updated_at equals its created_at). @@ -415,28 +416,14 @@ protected void emit(MetaObject entity, Path outRoot) { src.append(" @RequestMapping(value = \"/{id}\", method = { RequestMethod.PATCH, RequestMethod.PUT })\n"); src.append(" public ResponseEntity update(@PathVariable ").append(pkType) .append(" id, @RequestBody JsonNode body) {\n"); - src.append(" ").append(patchName).append(" patch;\n"); - src.append(" try {\n"); - src.append(" patch = ").append(patchName).append(".fromJson(body, objectMapper);\n"); - src.append(" } catch (PatchValidationException e) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); - // FR-036: validate each PRESENT patch value against the DTO's field constraints (per-field, - // NOT a whole-bean validate — that would fire @NotNull on ABSENT fields). jakarta - // validateValue skips null for @Size/@Pattern and non-required fields carry no @NotNull, so - // a present-null-clears on a nullable field composes cleanly. - src.append(" for (Map.Entry __e : patch.assignedValues().entrySet()) {\n"); - src.append(" if (!validator.validateValue(").append(dtoName) - .append(".class, RecordComponentNames.escape(__e.getKey()), __e.getValue()).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); - src.append(" }\n"); + appendPatchBinding(src, patchName, dtoName); // Program D (spec section 0): validateValue does NOT cascade @Valid into a nested value // object's own constraints, so validate each PRESENT VO element EXPLICITLY. The property's // OWN constraints (@NotNull on a @required column, present-null tristate) are already // covered above / in Patch.fromJson; this closes the nested-constraint gap (e.g. a // Marker.label over @maxLength, or null on a @required nested member) with a 400. appendValueObjectValidation(src, entity); + appendPatchVerdict(src); // Issue #203: honor @autoSet onUpdate — bump every onUpdate column to now() on EVERY partial // update (the model value is ignored; the server owns the timestamp), injected AFTER the // caller's present values are validated and BEFORE delegating so a PATCH that omits the @@ -502,26 +489,58 @@ protected void emit(MetaObject entity, Path outRoot) { } /** - * Program D — append per-value-object nested-validation guards into the vanilla PATCH handler, - * one per {@code field.object @storage:jsonb} column (spec section 0). A present single VO - * value is validated whole; a present array is validated element-by-element. Skips null - * (present-null clears / is caught upstream) so the tristate composes. Emits nothing for an - * entity with no value-object columns, keeping non-VO controllers byte-identical. + * The head of a PATCH handler: bind the body to the presence-tracked patch and validate each + * PRESENT value against the DTO's field constraints, collecting EVERY issue into + * {@code __issues} (a bind failure lists all its problems, and the values that did bind are + * still validated). Per-field and NOT a whole-bean validate, which would fire {@code @NotNull} + * on ABSENT fields; {@code validateValue} skips null for {@code @Size}/{@code @Pattern} and a + * non-required field carries no {@code @NotNull}, so a present-null-clears composes cleanly. + * Leaves {@code __assigned} (the bound values) and {@code __patch} (null after a bind failure) + * for {@link #appendPatchVerdict}. + */ + private static void appendPatchBinding(StringBuilder src, String patchName, String dtoName) { + src.append(" ValidationIssues __issues = new ValidationIssues();\n"); + src.append(" ").append(patchName).append(" __patch = null;\n"); + src.append(" Map __assigned;\n"); + src.append(" try {\n"); + src.append(" __patch = ").append(patchName).append(".fromJson(body, objectMapper);\n"); + src.append(" __assigned = __patch.assignedValues();\n"); + src.append(" } catch (PatchValidationException e) {\n"); + src.append(" __issues.addPatchFailure(e);\n"); + src.append(" __assigned = e.assignedValues();\n"); + src.append(" }\n"); + src.append(" for (Map.Entry __e : __assigned.entrySet()) {\n"); + src.append(" __issues.addAt(validator.validateValue(").append(dtoName) + .append(".class, RecordComponentNames.escape(__e.getKey()), __e.getValue()), __e.getKey());\n"); + src.append(" }\n"); + } + + /** The end of the PATCH validation: answer every collected issue at once, else expose {@code patch}. */ + private static void appendPatchVerdict(StringBuilder src) { + src.append(" if (!__issues.isEmpty()) {\n"); + src.append(" return ResponseEntity.badRequest().body(__issues.body());\n"); + src.append(" }\n"); + src.append(" final var patch = __patch;\n"); + } + + /** + * Program D — append per-value-object nested-validation guards into a PATCH handler, one per + * {@code field.object @storage:jsonb} column (spec section 0), reading the BOUND values from + * {@code __assigned}. A present single VO value is validated whole; a present array is validated + * element-by-element. Skips null (present-null clears / is caught upstream) so the tristate + * composes. Every violation is collected into {@code __issues}, its path the field's wire name + * then the element index or key then the member. Emits nothing for an entity with no + * value-object columns, keeping non-VO controllers byte-identical. */ private static void appendValueObjectValidation(StringBuilder src, MetaObject entity) { for (ObjectField vf : SpringGeneratorSupport.valueObjectJsonbFields(entity)) { - String name = vf.getName(); - // has() needs no escape — capitalisation already clears the collision — but - // the value accessor does. - String accessor = SpringNaming.recordComponentName(name); - String cap = SpringNaming.capitalize(name); + String name = SpringGeneratorSupport.escapeJavaString(vf.getName()); if (vf.isArrayType()) { - appendElementValidationLoop(src, accessor, cap, "patch." + accessor + "()"); + appendAssignedElementLoop(src, name); } else { - src.append(" if (patch.has").append(cap).append("() && patch.") - .append(accessor).append("() != null && !validator.validate(patch.") - .append(accessor).append("()).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); + src.append(" if (__assigned.get(\"").append(name).append("\") != null) {\n"); + src.append(" __issues.addViolations(validator.validate(__assigned.get(\"").append(name) + .append("\")), \"").append(name).append("\");\n"); src.append(" }\n"); } } @@ -530,58 +549,54 @@ private static void appendValueObjectValidation(StringBuilder src, MetaObject en // with validateValue, which does NOT cascade — so without this the two write paths // disagree for maps where they agree for field.object, and a PATCH could write a // nested value object POST rejects. The map's VALUES are the beans (keys are strings). + appendValueObjectMapValidation(src, entity); + } + + /** The map-of-value-object part of {@link #appendValueObjectValidation}, for a TPH subtype's PATCH. */ + private static void appendValueObjectMapValidation(StringBuilder src, MetaObject entity) { for (MetaField vf : SpringGeneratorSupport.valueObjectMapFields(entity)) { - String name = vf.getName(); - String accessor = SpringNaming.recordComponentName(name); - appendElementValidationLoop( - src, accessor, SpringNaming.capitalize(name), "patch." + accessor + "().values()"); + appendAssignedMapLoop(src, SpringGeneratorSupport.escapeJavaString(vf.getName())); } } - /** - * Emit the present-and-non-null guard plus an element-wise {@code validator.validate} loop - * over {@code iterableExpr}. Shared by the array-of-value-object branch and the - * map-of-value-object branch: they differ ONLY in the iterable expression, and writing the - * body twice meant the 400 envelope existed in three places, so adding an error code (as - * other ports have) would have missed one. - * - * @param accessor the field's Java accessor name — ALREADY escaped via - * {@link SpringNaming#recordComponentName}. Each caller escapes once and derives both - * this and {@code iterableExpr} from that one local, so the two cannot disagree; the - * escape is not repeated here because {@code iterableExpr} is the caller's to build. - * @param cap the DECLARED name capitalised, for {@code has()} — never escaped, - * because capitalisation already clears the collision with {@code Object}'s methods. - */ - private static void appendElementValidationLoop( - StringBuilder src, String accessor, String cap, String iterableExpr) { - src.append(" if (patch.has").append(cap).append("() && patch.") - .append(accessor).append("() != null) {\n"); - src.append(" for (var __el : ").append(iterableExpr).append(") {\n"); - src.append(" if (__el != null && !validator.validate(__el).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); + /** The element-wise validation of a present array-of-value-object {@code __assigned} entry. */ + private static void appendAssignedElementLoop(StringBuilder src, String name) { + src.append(" if (__assigned.get(\"").append(name).append("\") instanceof java.util.Collection __c) {\n"); + src.append(" int __i = 0;\n"); + src.append(" for (Object __el : __c) {\n"); + src.append(" if (__el != null) __issues.addViolations(validator.validate(__el), \"") + .append(name).append("\", __i);\n"); + src.append(" __i++;\n"); + src.append(" }\n"); + src.append(" }\n"); + } + + /** The value-wise validation of a present map-of-value-object {@code __assigned} entry. */ + private static void appendAssignedMapLoop(StringBuilder src, String name) { + src.append(" if (__assigned.get(\"").append(name).append("\") instanceof Map __m) {\n"); + src.append(" for (Map.Entry __en : __m.entrySet()) {\n"); + src.append(" if (__en.getValue() != null) __issues.addViolations(validator.validate(__en.getValue()), \"") + .append(name).append("\", String.valueOf(__en.getKey()));\n"); src.append(" }\n"); src.append(" }\n"); } /** - * The same element-wise {@code validator.validate} loop as - * {@link #appendElementValidationLoop}, minus the presence guard — for a body bound as the - * union {@code Dto}, whose accessors are plain (no {@code hasX()} tristate). Used by - * the TPH per-subtype CREATE path, where absent and null are already indistinguishable on - * the bound DTO, so a null check is the whole guard. + * The map-of-value-object validation for a body bound as the union {@code Dto}, whose + * accessors are plain (no {@code hasX()} tristate). Used by the TPH per-subtype CREATE path, + * where absent and null are already indistinguishable on the bound DTO, so a null check is + * the whole guard. * *

#362: without this the TPH create validated a {@code field.map @objectRef} with * {@code validateValue}, which applies the property's own constraints and does NOT cascade * {@code @Valid} into the map's values — so a posted * {@code {"labels": {"k": {...invalid...}}}} was accepted and written. */ - private static void appendMapValueValidationLoop(StringBuilder src, String accessorExpr) { + private static void appendMapValueValidationLoop(StringBuilder src, String accessorExpr, String wireName) { src.append(" if (").append(accessorExpr).append(" != null) {\n"); - src.append(" for (var __el : ").append(accessorExpr).append(".values()) {\n"); - src.append(" if (__el != null && !validator.validate(__el).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); + src.append(" for (var __en : ").append(accessorExpr).append(".entrySet()) {\n"); + src.append(" if (__en.getValue() != null) __issues.addViolations(validator.validate(__en.getValue()), \"") + .append(SpringGeneratorSupport.escapeJavaString(wireName)).append("\", String.valueOf(__en.getKey()));\n"); src.append(" }\n"); src.append(" }\n"); } @@ -1150,6 +1165,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append("import ").append(runtimePackage).append(".ConstraintErrors;\n"); src.append("import ").append(runtimePackage).append(".PatchValidationException;\n"); src.append("import ").append(runtimePackage).append(".UnreadableBody;\n"); + src.append("import ").append(runtimePackage).append(".ValidationIssues;\n"); // The escape the PATCH loop applies to each assigned key before naming a Bean // Validation property — see RecordComponentNames for why it is needed at run time. src.append("import ").append(runtimePackage).append(".RecordComponentNames;\n"); @@ -1368,19 +1384,19 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append(" public ResponseEntity create").append(suffix).append("(@RequestBody JsonNode body) {\n"); appendFillDefaults(src, dtoName, subDefaultPuts); } + src.append(" ValidationIssues __issues = new ValidationIssues();\n"); for (MetaField vf : createValidated) { // Both the property STRING and the accessor take the escaped component name: // Bean Validation resolves a record's property by its COMPONENT, and // `dto.notify()` would bind to Object.notify(). String vc = SpringNaming.recordComponentName(vf.getName()); - src.append(" if (!validator.validateValue(").append(subDto).append(".class, \"") - .append(vc).append("\", dto.").append(vc).append("()).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); + src.append(" __issues.addAt(validator.validateValue(").append(subDto).append(".class, \"") + .append(vc).append("\", dto.").append(vc).append("()), \"") + .append(SpringGeneratorSupport.escapeJavaString(vf.getName())).append("\");\n"); } for (Map.Entry rule : crossFieldRules) { src.append(" if (!(").append(rule.getValue()).append(")) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); + src.append(" __issues.add(ValidationIssues.CUSTOM, java.util.List.of(), \"a cross-field rule failed\");\n"); src.append(" }\n"); } // #362: validateValue above checks each property against the subtype DTO's own @@ -1391,8 +1407,11 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { // path's equivalent, so the two write surfaces agree. for (MetaField vf : SpringGeneratorSupport.valueObjectMapFields(st.entity())) { appendMapValueValidationLoop( - src, "dto." + SpringNaming.recordComponentName(vf.getName()) + "()"); + src, "dto." + SpringNaming.recordComponentName(vf.getName()) + "()", vf.getName()); } + src.append(" if (!__issues.isEmpty()) {\n"); + src.append(" return ResponseEntity.badRequest().body(__issues.body());\n"); + src.append(" }\n"); // ADR-0045 (#203/#229): honor @autoSet on the TPH per-subtype create — stamp EVERY // onCreate AND onUpdate column with now() before persisting (the caller's value is // ignored), mirroring the vanilla create handler's createArg above. The stamp helper @@ -1435,28 +1454,9 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { .append("/{id}\", method = { RequestMethod.PATCH, RequestMethod.PUT })\n"); src.append(" public ResponseEntity update").append(suffix) .append("(@PathVariable ").append(pkType).append(" id, @RequestBody JsonNode body) {\n"); - src.append(" ").append(subPatch).append(" patch;\n"); - src.append(" try {\n"); - src.append(" patch = ").append(subPatch).append(".fromJson(body, objectMapper);\n"); - src.append(" } catch (PatchValidationException e) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); - src.append(" for (Map.Entry __e : patch.assignedValues().entrySet()) {\n"); - src.append(" if (!validator.validateValue(").append(subDto) - .append(".class, RecordComponentNames.escape(__e.getKey()), __e.getValue()).isEmpty()) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); - src.append(" }\n"); - src.append(" }\n"); - // #362: as on the create path above, validateValue does not cascade into a nested - // bean, so a PRESENT field.map @objectRef had its values written unvalidated. The - // Patch carries the hasX()/x() tristate, so this is the same loop the vanilla - // PATCH runs — skipping absent keys, and skipping an explicit null (which clears). - for (MetaField vf : SpringGeneratorSupport.valueObjectMapFields(st.entity())) { - String mvc = SpringNaming.recordComponentName(vf.getName()); - appendElementValidationLoop( - src, mvc, SpringNaming.capitalize(vf.getName()), - "patch." + mvc + "().values()"); - } + appendPatchBinding(src, subPatch, subDto); + appendValueObjectMapValidation(src, st.entity()); + appendPatchVerdict(src); // ADR-0045 (#203/#229): bump every onUpdate @autoSet column on the per-subtype PATCH, // mirroring the vanilla update handler above — injected after present-value // validation, before delegating, so the timestamp bumps even when the caller omits @@ -1741,7 +1741,8 @@ private static String javaLiteral(String text) { * {@code dto}. */ private static void appendFillDefaults(StringBuilder src, String dtoName, List puts) { src.append(" if (!(body instanceof com.fasterxml.jackson.databind.node.ObjectNode given)) {\n"); - src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"validation\"));\n"); + src.append(" return ResponseEntity.badRequest().body(new ValidationIssues()\n"); + src.append(" .add(ValidationIssues.INVALID_TYPE, java.util.List.of(), \"request body must be a JSON object\").body());\n"); src.append(" }\n"); src.append(" com.fasterxml.jackson.databind.node.ObjectNode filled = given.deepCopy();\n"); for (String put : puts) src.append(" ").append(put).append("\n"); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java index 8d266b9448..9bb00a57aa 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java @@ -376,6 +376,8 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin .append(" fromJson(JsonNode body, ObjectMapper mapper) {\n"); src.append(" if (!body.isObject()) throw new PatchValidationException(\"(request body)\");\n"); src.append(" Map assigned = new LinkedHashMap<>();\n"); + // Every field is bound even after one fails, so the one exception below lists them all. + src.append(" java.util.List problems = PatchValidationException.newProblems();\n"); boolean usesDecimalPatch = false; boolean usesDecimalListPatch = false; boolean usesDurationPatch = false; @@ -390,18 +392,18 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin // ADR-0019 Amendment 1 (W2): a decimal binds through the strict rule, not the // mapper's coercion: "" or "NaN" is a 400, a number is read exactly. int[] bound = SpringGeneratorSupport.declaredDecimalBound(field); - src.append(" bindDecimal(body, assigned, \"").append(name).append("\", ") + src.append(" PatchValidationException.collect(problems, () -> bindDecimal(body, assigned, \"").append(name).append("\", ") .append(required).append(", ").append(bound == null ? 0 : bound[0]) - .append(", ").append(bound == null ? 0 : bound[1]).append(");\n"); + .append(", ").append(bound == null ? 0 : bound[1]).append("));\n"); usesDecimalPatch = true; continue; } if ("java.util.List".equals(type)) { // The same strict rule per element of a decimal array. int[] bound = SpringGeneratorSupport.declaredDecimalBound(field); - src.append(" bindDecimalList(body, assigned, \"").append(name).append("\", ") + src.append(" PatchValidationException.collect(problems, () -> bindDecimalList(body, assigned, \"").append(name).append("\", ") .append(required).append(", ").append(bound == null ? 0 : bound[0]) - .append(", ").append(bound == null ? 0 : bound[1]).append(");\n"); + .append(", ").append(bound == null ? 0 : bound[1]).append("));\n"); usesDecimalPatch = true; usesDecimalListPatch = true; continue; @@ -411,15 +413,16 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin // wire.durations: iso. field.duration binds through the strict accepted-form rule, not the mapper's // Duration.parse (which takes fractional hours, lowercase and numbers of seconds). boolean list = !"java.time.Duration".equals(type); - src.append(" bindDuration").append(list ? "List" : "").append("(body, assigned, \"") - .append(name).append("\", ").append(required).append(");\n"); + src.append(" PatchValidationException.collect(problems, () -> bindDuration").append(list ? "List" : "").append("(body, assigned, \"") + .append(name).append("\", ").append(required).append("));\n"); if (list) usesDurationListPatch = true; else usesDurationPatch = true; continue; } - src.append(" bind(body, mapper, assigned, \"").append(name) + src.append(" PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, \"").append(name) .append("\", new TypeReference<").append(type).append(">() {}, ") - .append(required).append(");\n"); + .append(required).append("));\n"); } + src.append(" if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned);\n"); src.append(" return new ").append(patchName).append("(assigned);\n"); src.append(" }\n\n"); @@ -436,7 +439,7 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" try {\n"); src.append(" assigned.put(field, mapper.treeToValue(node, type));\n"); src.append(" } catch (JsonProcessingException e) {\n"); - src.append(" throw new PatchValidationException(field);\n"); + src.append(" throw new PatchValidationException(field, e);\n"); src.append(" }\n"); src.append(" }\n"); if (usesDecimalPatch) { @@ -459,7 +462,7 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" assigned.put(field, precision == 0 ? CanonicalDecimalDeserializer.fromNode(node)\n"); src.append(" : CanonicalDecimalDeserializer.fromNode(node, precision, scale));\n"); src.append(" } catch (IllegalArgumentException e) {\n"); - src.append(" throw new PatchValidationException(field);\n"); + src.append(" throw new PatchValidationException(field, e);\n"); src.append(" }\n"); src.append(" }\n"); if (usesDecimalListPatch) { @@ -480,7 +483,7 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" items.add(precision == 0 ? CanonicalDecimalDeserializer.fromNode(item)\n"); src.append(" : CanonicalDecimalDeserializer.fromNode(item, precision, scale));\n"); src.append(" } catch (IllegalArgumentException e) {\n"); - src.append(" throw new PatchValidationException(field);\n"); + src.append(" throw new PatchValidationException(field, e);\n"); src.append(" }\n"); src.append(" }\n"); src.append(" assigned.put(field, items);\n"); @@ -505,7 +508,7 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" try {\n"); src.append(" assigned.put(field, CanonicalDurationDeserializer.fromNode(node));\n"); src.append(" } catch (IllegalArgumentException e) {\n"); - src.append(" throw new PatchValidationException(field);\n"); + src.append(" throw new PatchValidationException(field, e);\n"); src.append(" }\n"); src.append(" }\n"); } @@ -526,7 +529,7 @@ private void writePatch(MetaObject entity, Path outRoot, String patchName, Strin src.append(" try {\n"); src.append(" items.add(CanonicalDurationDeserializer.fromNode(item));\n"); src.append(" } catch (IllegalArgumentException e) {\n"); - src.append(" throw new PatchValidationException(field);\n"); + src.append(" throw new PatchValidationException(field, e);\n"); src.append(" }\n"); src.append(" }\n"); src.append(" assigned.put(field, items);\n"); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDecimal.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDecimal.java index d1a6a01d9d..92e82e9ac8 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDecimal.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/CanonicalDecimal.java @@ -44,8 +44,21 @@ private CanonicalDecimal() { /* no instances */ } public static final class InvalidDecimalException extends IllegalArgumentException { private static final long serialVersionUID = 1L; + private final String code; + public InvalidDecimalException(String message) { + this(message, "invalid_type"); + } + + /** {@code code} is the Zod-style validation code: {@code too_big} for a value out of range or over its precision/scale. */ + public InvalidDecimalException(String message, String code) { super(message); + this.code = code; + } + + /** The Zod-style validation code for this failure. */ + public String code() { + return code; } } @@ -72,7 +85,7 @@ public static BigDecimal parse(String text) { value = new BigDecimal(text); } catch (NumberFormatException e) { // an exponent beyond int range - throw new InvalidDecimalException("decimal out of range: " + text); + throw new InvalidDecimalException("decimal out of range: " + text, "too_big"); } return checkRange(value, text); } @@ -126,7 +139,7 @@ public static BigDecimal checkDeclared(BigDecimal value, int precision, int scal long fractionDigits = Math.max(0, stripped.scale()); if (fractionDigits > scale || integerDigits > (long) precision - scale) { throw new InvalidDecimalException("decimal does not fit decimal(" + precision + "," + scale + "): " - + format(value)); + + format(value), "too_big"); } return value; } @@ -136,7 +149,7 @@ private static BigDecimal checkRange(BigDecimal value, String shown) { BigDecimal stripped = value.stripTrailingZeros(); long integerDigits = (long) stripped.precision() - stripped.scale(); if (integerDigits > MAX_INTEGER_DIGITS || stripped.scale() > MAX_FRACTION_DIGITS) { - throw new InvalidDecimalException("decimal out of range: " + shown); + throw new InvalidDecimalException("decimal out of range: " + shown, "too_big"); } return value; } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/PatchValidationException.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/PatchValidationException.java index 44d47f906e..3a0c8c3f3f 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/PatchValidationException.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/PatchValidationException.java @@ -1,22 +1,109 @@ package com.metaobjects.generator.spring.runtime; +import com.fasterxml.jackson.databind.exc.InvalidFormatException; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + /** * Thrown by a generated {@code Patch.fromJson(...)} when a PATCH body fails - * FR-035 validation — specifically an explicit {@code null} on a {@code @required} - * field. The generated controller catches it and maps it to the cross-port HTTP 400 - * {@code {"error":"validation"}} envelope. + * FR-035 validation: an explicit {@code null} on a {@code @required} field, a value that + * cannot be bound, or a body that is not an object. The generated controller catches it and + * maps it to the cross-port HTTP 400 {@code {"error":"validation","issues":[...]}} envelope. + * + *

It carries EVERY problem {@code fromJson} found (not just the first), each with its + * Zod-style code, plus the values that did bind, so the controller can go on and validate + * those too and answer one complete list.

*/ public final class PatchValidationException extends RuntimeException { - private final String field; + /** The field name of a failure that is about the body as a whole (its path is empty). */ + public static final String BODY = "(request body)"; + + /** One failure: the field ({@code null} for the body as a whole), its code and message. */ + public record Problem(String field, String code, String message) { } + private final List problems; + private final Map assigned; + + /** An invalid-type failure of {@code field} ({@link #BODY} for the body as a whole). */ public PatchValidationException(String field) { - super("field '" + field + "' cannot be null"); - this.field = field; + this(field, "invalid_type", "field '" + field + "' cannot be null"); + } + + /** A failure of {@code field} whose binding raised {@code cause}. */ + public PatchValidationException(String field, Throwable cause) { + this(field, codeOf(cause), "field '" + field + "' is not a valid value"); + } + + private PatchValidationException(String field, String code, String message) { + super(message); + this.problems = List.of(new Problem(BODY.equals(field) ? null : field, code, message)); + this.assigned = Map.of(); + } + + /** Every problem found while binding, and the values that did bind. */ + public PatchValidationException(List problems, Map assigned) { + super("patch body is not valid: " + problems.size() + " problem(s)"); + this.problems = List.copyOf(problems); + this.assigned = new LinkedHashMap<>(assigned); } - /** The offending field name. */ + /** Run {@code bind}, adding what it fails on to {@code problems} instead of throwing. */ + public static void collect(List problems, Runnable bind) { + try { + bind.run(); + } catch (PatchValidationException e) { + problems.addAll(e.problems); + } + } + + /** A new, empty list for {@link #collect}. */ + public static List newProblems() { + return new ArrayList<>(); + } + + /** The first offending field name, or {@code null} when the body as a whole. */ public String field() { - return field; + return problems.get(0).field(); + } + + /** Every problem. */ + public List problems() { + return problems; + } + + /** The values that did bind (the failing fields are absent). */ + public Map assignedValues() { + return assigned; + } + + /** + * The Zod-style code for a binding failure: a decimal over its precision/scale or range is + * {@code too_big}, an unknown enum member {@code invalid_value}, a malformed date/time/uuid/uri + * {@code invalid_format}, anything else (wrong JSON type, unparseable) {@code invalid_type}. + */ + public static String codeOf(Throwable failure) { + for (Throwable t = failure; t != null; t = t.getCause() == t ? null : t.getCause()) { + if (t instanceof CanonicalDecimal.InvalidDecimalException d) return d.code(); + if (t instanceof InvalidFormatException f) { + Class target = f.getTargetType(); + if (target != null) { + if (target.isEnum()) return "invalid_value"; + String n = target.getName(); + if (n.startsWith("java.time.") || n.equals("java.util.UUID") || n.equals("java.net.URI") + || n.equals("java.net.URL") || n.equals("java.util.Date")) { + return "invalid_format"; + } + } + String m = f.getMessage(); + if (m != null && (m.contains("decimal does not fit decimal(") || m.contains(": decimal out of range: "))) { + return "too_big"; + } + } + } + return "invalid_type"; } } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/RecordComponentNames.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/RecordComponentNames.java index bdaf087aae..a0ca1f697f 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/RecordComponentNames.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/RecordComponentNames.java @@ -70,4 +70,16 @@ public static boolean isReserved(String fieldName) { public static String escape(String fieldName) { return isReserved(fieldName) ? fieldName + "_" : fieldName; } + + /** + * The declared field name for a record component {@code component}: undoes {@link #escape}. + * A name that merely ends in an underscore, with a non-reserved stem, is returned unchanged. + */ + public static String unescape(String component) { + if (component.length() > 1 && component.endsWith("_")) { + String stem = component.substring(0, component.length() - 1); + if (isReserved(stem)) return stem; + } + return component; + } } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java index 3ebf20c9b2..aef14495b1 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java @@ -2,7 +2,8 @@ import com.fasterxml.jackson.databind.JsonMappingException; -import java.util.LinkedHashMap; +import java.util.ArrayList; +import java.util.List; import java.util.Map; /** @@ -18,13 +19,32 @@ public final class UnreadableBody { private UnreadableBody() { /* no instances */ } - /** The body for {@code failure}: {@code {"error":"validation"}} plus {@code field} when known. */ + /** + * The body for {@code failure}: {@code {"error":"validation","issues":[...]}} plus {@code field} + * when known. The one issue's {@code path} is every property Jackson was binding (nested + * members and array indexes included) and is empty when the body as a whole cannot be read. + */ public static Map envelope(Throwable failure) { - Map out = new LinkedHashMap<>(); - out.put("error", "validation"); + List path = pathOf(failure); + ValidationIssues issues = new ValidationIssues().add( + PatchValidationException.codeOf(failure), path, + path.isEmpty() ? "request body is not readable" : "field '" + path.get(path.size() - 1) + "' is not a valid value"); String field = fieldOf(failure); - if (field != null) out.put("field", field); - return out; + return field == null ? issues.body() : issues.body("field", field); + } + + private static List pathOf(Throwable failure) { + for (Throwable t = failure; t != null; t = t.getCause() == t ? null : t.getCause()) { + if (t instanceof JsonMappingException mapping && !mapping.getPath().isEmpty()) { + List out = new ArrayList<>(); + for (JsonMappingException.Reference ref : mapping.getPath()) { + if (ref.getFieldName() != null) out.add(ref.getFieldName()); + else if (ref.getIndex() >= 0) out.add(ref.getIndex()); + } + return out; + } + } + return List.of(); } private static String fieldOf(Throwable failure) { diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/ValidationIssues.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/ValidationIssues.java new file mode 100644 index 0000000000..eb58e4deab --- /dev/null +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/ValidationIssues.java @@ -0,0 +1,184 @@ +package com.metaobjects.generator.spring.runtime; + +import jakarta.validation.ConstraintViolation; +import jakarta.validation.Path; + +import java.lang.annotation.Annotation; +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * The cross-port validation answer: {@code 400 {"error": "validation", "issues": [...]}}, where each + * issue is {@code {"code", "path", "message"}}. {@code code} uses Zod's names ({@code invalid_type}, + * {@code too_small}, {@code too_big}, {@code invalid_format}, {@code invalid_value}, {@code custom}), + * {@code path} is the field's wire name (nested members and array indexes as further elements, empty + * when the body as a whole cannot be read) and {@code message} is this port's own wording. + * + *

A generated controller collects EVERY issue into one of these before answering, so a caller + * sees all the problems with a body at once. Depends on the JDK and the jakarta validation API only + * ({@code mvn metaobjects:eject} hands it over as plain source).

+ */ +public final class ValidationIssues { + + /** Zod's {@code invalid_type}: missing, null on a required field, wrong JSON type, unparseable. */ + public static final String INVALID_TYPE = "invalid_type"; + /** Zod's {@code too_small}: below a minimum length or value. */ + public static final String TOO_SMALL = "too_small"; + /** Zod's {@code too_big}: over a maximum length or value, or a decimal over its precision/scale. */ + public static final String TOO_BIG = "too_big"; + /** Zod's {@code invalid_format}: a pattern or format (a malformed date, uuid, e-mail...). */ + public static final String INVALID_FORMAT = "invalid_format"; + /** Zod's {@code invalid_value}: an enum value that is not a member. */ + public static final String INVALID_VALUE = "invalid_value"; + /** Zod's {@code custom}: anything else. */ + public static final String CUSTOM = "custom"; + + private final List> issues = new ArrayList<>(); + + /** Whether no issue has been collected. */ + public boolean isEmpty() { + return issues.isEmpty(); + } + + /** Add one issue. */ + public ValidationIssues add(String code, List path, String message) { + Map issue = new LinkedHashMap<>(); + issue.put("code", code); + issue.put("path", new ArrayList(path)); + issue.put("message", message == null ? code : message); + issues.add(issue); + return this; + } + + /** Add one issue about the field {@code field}. */ + public ValidationIssues addField(String code, String field, String message) { + return add(code, List.of(field), message); + } + + /** + * Add every violation of a whole-bean {@code validate}, its path read from the violation + * (nested value-object members and array indexes included) after {@code prefix}. + */ + public ValidationIssues addViolations(Collection> violations, Object... prefix) { + return addViolationsExcept(violations, Set.of(), prefix); + } + + /** As {@link #addViolations}, skipping a violation whose top-level property is in {@code ignored}. */ + public ValidationIssues addViolationsExcept( + Collection> violations, Set ignored, Object... prefix) { + for (ConstraintViolation v : violations) { + if (ignored.contains(v.getPropertyPath().toString())) continue; + List path = new ArrayList<>(Arrays.asList(prefix)); + path.addAll(pathOf(v.getPropertyPath())); + add(codeOf(v), path, v.getMessage()); + } + return this; + } + + /** + * Add every violation of a single-property {@code validateValue} at exactly {@code path} (the + * caller knows the wire name; the violation's own path names the escaped record component). + */ + public ValidationIssues addAt(Collection> violations, Object... path) { + for (ConstraintViolation v : violations) { + add(codeOf(v), Arrays.asList(path), v.getMessage()); + } + return this; + } + + /** Add everything a patch body failed on. */ + public ValidationIssues addPatchFailure(PatchValidationException failure) { + for (PatchValidationException.Problem p : failure.problems()) { + add(p.code(), p.field() == null ? List.of() : List.of(p.field()), p.message()); + } + return this; + } + + /** The response body: {@code {"error": "validation", "issues": [...]}}. */ + public Map body() { + Map out = new LinkedHashMap<>(); + out.put("error", "validation"); + out.put("issues", issues); + return out; + } + + /** The response body with one extra key (the JVM {@code field} key). */ + public Map body(String key, Object value) { + Map out = body(); + out.put(key, value); + return out; + } + + /** A path as wire elements: property names (record-component escape undone) and indexes/keys. */ + static List pathOf(Path path) { + List out = new ArrayList<>(); + for (Path.Node node : path) { + if (node.isInIterable()) { + if (node.getIndex() != null) out.add(node.getIndex()); + else if (node.getKey() != null) out.add(String.valueOf(node.getKey())); + } + String name = node.getName(); + if (name != null) out.add(RecordComponentNames.unescape(name)); + } + return out; + } + + /** The Zod-style code for a Bean Validation violation, from the constraint's annotation. */ + public static String codeOf(ConstraintViolation v) { + Annotation a = v.getConstraintDescriptor().getAnnotation(); + String name = a.annotationType().getSimpleName(); + Object value = v.getInvalidValue(); + Map attrs = v.getConstraintDescriptor().getAttributes(); + switch (name) { + case "NotNull": + return INVALID_TYPE; + case "NotBlank": + case "NotEmpty": + return value == null ? INVALID_TYPE : TOO_SMALL; + case "Size": + case "Length": { + Object max = attrs.get("max"); + Long length = sizeOf(value); + if (max instanceof Number m && length != null && length > m.longValue()) return TOO_BIG; + return TOO_SMALL; + } + case "Max": + case "DecimalMax": + case "Digits": + case "Negative": + case "NegativeOrZero": + return TOO_BIG; + case "Min": + case "DecimalMin": + case "Positive": + case "PositiveOrZero": + return TOO_SMALL; + case "Range": { + Object max = attrs.get("max"); + if (value instanceof Number n && max instanceof Number m + && new BigDecimal(n.toString()).compareTo(new BigDecimal(m.toString())) > 0) return TOO_BIG; + return TOO_SMALL; + } + case "Pattern": + case "Email": + case "URL": + return INVALID_FORMAT; + default: + return CUSTOM; + } + } + + private static Long sizeOf(Object value) { + if (value instanceof CharSequence s) return (long) s.length(); + if (value instanceof Collection c) return (long) c.size(); + if (value instanceof Map m) return (long) m.size(); + if (value != null && value.getClass().isArray()) return (long) java.lang.reflect.Array.getLength(value); + return null; + } +} diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java index cdfcea43fb..13f8228250 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java @@ -320,7 +320,7 @@ public void createDoesNotValidateServerOwnedColumnsOutOfTheBody() throws Excepti // The create handler must FILTER violations, not simply reject any. assertTrue("create must ignore violations on server-owned components; saw:\n" + src, - src.contains(".anyMatch(v -> !SERVER_OWNED_ON_CREATE.contains(v.getPropertyPath().toString()))")); + src.contains(".addViolationsExcept(validator.validate(dto), SERVER_OWNED_ON_CREATE)")); assertFalse("create must no longer reject on ANY violation; saw:\n" + src, src.contains("if (!validator.validate(dto).isEmpty())")); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java index 78b8d76ae3..8bd54ec559 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java @@ -194,10 +194,10 @@ public void patchValidatesTheValuesOfAValueObjectMap() throws Exception { // __el, not a map-specific name: the array-of-VO and map-of-VO branches share one // emitter, which is what keeps the 400 envelope in a single place. assertTrue("expected PATCH to iterate the map's VALUES; saw:\n" + controller, - controller.contains("for (var __el : patch.addresses().values())")); + controller.contains("__assigned.get(\"addresses\") instanceof Map __m")); // A scalar-valued map has no nested bean, so it must NOT get a validation loop. assertFalse("a scalar-valued map needs no nested validation; saw:\n" + controller, - controller.contains("patch.labels().values()")); + controller.contains("__assigned.get(\"labels\")")); } @Test @@ -422,16 +422,16 @@ public void tphWritePathsValidateTheValuesOfAValueObjectMap() throws Exception { // CREATE binds the union Dto — plain accessors, so the guard is a null check. assertTrue("expected the TPH create to iterate the map's VALUES; saw:\n" + controller, - controller.contains("for (var __el : dto.addresses().values())")); + controller.contains("for (var __en : dto.addresses().entrySet())")); // PATCH binds the Patch tristate, so it reuses the shared presence-guarded loop. assertTrue("expected the TPH PATCH to iterate the map's VALUES; saw:\n" + controller, - controller.contains("for (var __el : patch.addresses().values())")); + controller.contains("__assigned.get(\"addresses\") instanceof Map __m")); // The control: a scalar-valued map has no nested bean and must get NO loop on either // path. Without this, a change that emitted a loop for every map would pass above. assertFalse("a scalar-valued map needs no nested validation on create; saw:\n" + controller, - controller.contains("dto.labels().values()")); + controller.contains("dto.labels().entrySet()")); assertFalse("a scalar-valued map needs no nested validation on patch; saw:\n" + controller, - controller.contains("patch.labels().values()")); + controller.contains("__assigned.get(\"labels\")")); } } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java new file mode 100644 index 0000000000..c8e2b0ba70 --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java @@ -0,0 +1,158 @@ +package com.metaobjects.generator.spring.runtime; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import jakarta.validation.Validation; +import jakarta.validation.Validator; +import jakarta.validation.Valid; +import jakarta.validation.constraints.DecimalMax; +import jakarta.validation.constraints.DecimalMin; +import jakarta.validation.constraints.Digits; +import jakarta.validation.constraints.Max; +import jakarta.validation.constraints.Min; +import jakarta.validation.constraints.NotNull; +import jakarta.validation.constraints.Pattern; +import jakarta.validation.constraints.Size; +import org.junit.Test; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** The Zod-style code mapping of the validation envelope, and that every issue is listed. */ +public class ValidationIssuesTest { + + public static class Marker { + @NotNull public String label; + public Marker(String label) { this.label = label; } + } + + public static class Bean { + @NotNull public String required; + @Size(min = 2, max = 4) public String sized; + @Max(5) public Integer big; + @Min(5) public Integer small; + @DecimalMax("1.5") public BigDecimal decBig; + @DecimalMin("1.5") public BigDecimal decSmall; + @Digits(integer = 2, fraction = 1) public BigDecimal digits; + @Pattern(regexp = "[a-z]+") public String pattern; + @Valid public List markers; + } + + private static final Validator VALIDATOR = Validation.buildDefaultValidatorFactory().getValidator(); + + private static Map codesByTopLevelPath(Bean bean) { + ValidationIssues issues = new ValidationIssues().addViolations(VALIDATOR.validate(bean)); + @SuppressWarnings("unchecked") + List> list = (List>) issues.body().get("issues"); + Map out = new java.util.TreeMap<>(); + for (Map issue : list) { + out.put(String.valueOf(issue.get("path")), (String) issue.get("code")); + assertTrue(issue.get("message") instanceof String); + } + return out; + } + + @Test + public void everyViolationIsListedWithItsZodCode() { + Bean b = new Bean(); + b.required = null; + b.sized = "toolong"; + b.big = 9; + b.small = 1; + b.decBig = new BigDecimal("2"); + b.decSmall = new BigDecimal("1"); + b.digits = new BigDecimal("123.45"); + b.pattern = "NOPE"; + Map codes = codesByTopLevelPath(b); + assertEquals("invalid_type", codes.get("[required]")); + assertEquals("too_big", codes.get("[sized]")); + assertEquals("too_big", codes.get("[big]")); + assertEquals("too_small", codes.get("[small]")); + assertEquals("too_big", codes.get("[decBig]")); + assertEquals("too_small", codes.get("[decSmall]")); + assertEquals("too_big", codes.get("[digits]")); + assertEquals("invalid_format", codes.get("[pattern]")); + assertEquals(8, codes.size()); + } + + @Test + public void aSizeBelowItsMinimumIsTooSmall() { + Bean b = new Bean(); + b.required = "x"; + b.sized = "a"; + assertEquals(Map.of("[sized]", "too_small"), codesByTopLevelPath(b)); + } + + @Test + public void aNestedMemberPathCarriesTheIndex() { + Bean b = new Bean(); + b.required = "x"; + b.markers = List.of(new Marker("ok"), new Marker(null)); + assertEquals(Map.of("[markers, 1, label]", "invalid_type"), codesByTopLevelPath(b)); + } + + @Test + public void addAtUsesTheWireNameGiven() { + Bean b = new Bean(); + ValidationIssues issues = new ValidationIssues() + .addAt(VALIDATOR.validateValue(Bean.class, "required", null), "wire_name"); + @SuppressWarnings("unchecked") + List> list = (List>) issues.body().get("issues"); + assertEquals(List.of("wire_name"), list.get(0).get("path")); + } + + @Test + public void aPatchBindFailureListsEveryProblemAndKeepsTheBoundValues() throws Exception { + JsonNode body = new ObjectMapper().readTree("{\"a\": null, \"b\": 7, \"c\": 2}"); + List problems = PatchValidationException.newProblems(); + Map assigned = new java.util.LinkedHashMap<>(); + PatchValidationException.collect(problems, () -> { throw new PatchValidationException("a"); }); + PatchValidationException.collect(problems, () -> { + throw new PatchValidationException("b", new CanonicalDecimal.InvalidDecimalException("x", "too_big")); + }); + assigned.put("c", 2); + PatchValidationException e = new PatchValidationException(problems, assigned); + ValidationIssues issues = new ValidationIssues().addPatchFailure(e); + @SuppressWarnings("unchecked") + List> list = (List>) issues.body().get("issues"); + assertEquals(2, list.size()); + assertEquals("invalid_type", list.get(0).get("code")); + assertEquals(List.of("a"), list.get(0).get("path")); + assertEquals("too_big", list.get(1).get("code")); + assertEquals(Map.of("c", 2), e.assignedValues()); + assertTrue(body.has("a")); + } + + @Test + public void aNonObjectBodyHasAnEmptyPath() { + ValidationIssues issues = new ValidationIssues() + .addPatchFailure(new PatchValidationException(PatchValidationException.BODY)); + @SuppressWarnings("unchecked") + List> list = (List>) issues.body().get("issues"); + assertEquals(List.of(), list.get(0).get("path")); + } + + @Test + public void anUnreadableBodyNamesTheNestedPathAndClassifiesTheFailure() { + ObjectMapper mapper = new ObjectMapper(); + try { + mapper.readValue("{\"n\": \"abc\"}", Holder.class); + } catch (Exception e) { + @SuppressWarnings("unchecked") + List> list = (List>) UnreadableBody.envelope(e).get("issues"); + assertEquals(List.of("n"), list.get(0).get("path")); + assertEquals("invalid_type", list.get(0).get("code")); + assertEquals("n", UnreadableBody.envelope(e).get("field")); + return; + } + throw new AssertionError("expected a failure"); + } + + public static class Holder { + public Integer n; + } +} diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CratePatch.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CratePatch.java index 6cb19bbf40..447cd0933b 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CratePatch.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/CratePatch.java @@ -34,7 +34,9 @@ public final class CratePatch { public static CratePatch fromJson(JsonNode body, ObjectMapper mapper) { if (!body.isObject()) throw new PatchValidationException("(request body)"); Map assigned = new LinkedHashMap<>(); - bind(body, mapper, assigned, "stage", new TypeReference() {}, true); + java.util.List problems = PatchValidationException.newProblems(); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "stage", new TypeReference() {}, true)); + if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned); return new CratePatch(assigned); } @@ -50,7 +52,7 @@ private static void bind(JsonNode body, ObjectMapper mapper, Map try { assigned.put(field, mapper.treeToValue(node, type)); } catch (JsonProcessingException e) { - throw new PatchValidationException(field); + throw new PatchValidationException(field, e); } } } diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelPatch.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelPatch.java index 7000886cf0..9571fab414 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelPatch.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ParcelPatch.java @@ -34,7 +34,9 @@ public final class ParcelPatch { public static ParcelPatch fromJson(JsonNode body, ObjectMapper mapper) { if (!body.isObject()) throw new PatchValidationException("(request body)"); Map assigned = new LinkedHashMap<>(); - bind(body, mapper, assigned, "phase", new TypeReference() {}, true); + java.util.List problems = PatchValidationException.newProblems(); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "phase", new TypeReference() {}, true)); + if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned); return new ParcelPatch(assigned); } @@ -50,7 +52,7 @@ private static void bind(JsonNode body, ObjectMapper mapper, Map try { assigned.put(field, mapper.treeToValue(node, type)); } catch (JsonProcessingException e) { - throw new PatchValidationException(field); + throw new PatchValidationException(field, e); } } } diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentPatch.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentPatch.java index e20cbd04b6..ea7054bcee 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentPatch.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentPatch.java @@ -36,8 +36,10 @@ public final class ShipmentPatch { public static ShipmentPatch fromJson(JsonNode body, ObjectMapper mapper) { if (!body.isObject()) throw new PatchValidationException("(request body)"); Map assigned = new LinkedHashMap<>(); - bind(body, mapper, assigned, "status", new TypeReference() {}, true); - bind(body, mapper, assigned, "mode", new TypeReference() {}, false); + java.util.List problems = PatchValidationException.newProblems(); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "status", new TypeReference() {}, true)); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "mode", new TypeReference() {}, false)); + if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned); return new ShipmentPatch(assigned); } @@ -53,7 +55,7 @@ private static void bind(JsonNode body, ObjectMapper mapper, Map try { assigned.put(field, mapper.treeToValue(node, type)); } catch (JsonProcessingException e) { - throw new PatchValidationException(field); + throw new PatchValidationException(field, e); } } } diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryPatch.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryPatch.java index a7726a101e..969d83cd15 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryPatch.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryPatch.java @@ -34,7 +34,9 @@ public final class ShipmentSummaryPatch { public static ShipmentSummaryPatch fromJson(JsonNode body, ObjectMapper mapper) { if (!body.isObject()) throw new PatchValidationException("(request body)"); Map assigned = new LinkedHashMap<>(); - bind(body, mapper, assigned, "status", new TypeReference() {}, true); + java.util.List problems = PatchValidationException.newProblems(); + PatchValidationException.collect(problems, () -> bind(body, mapper, assigned, "status", new TypeReference() {}, true)); + if (!problems.isEmpty()) throw new PatchValidationException(problems, assigned); return new ShipmentSummaryPatch(assigned); } @@ -50,7 +52,7 @@ private static void bind(JsonNode body, ObjectMapper mapper, Map try { assigned.put(field, mapper.treeToValue(node, type)); } catch (JsonProcessingException e) { - throw new PatchValidationException(field); + throw new PatchValidationException(field, e); } } } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt index 73ae81ee31..93e6f96d23 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt @@ -119,6 +119,9 @@ object ApiContractAssertions { if (actualField != wantField) throw AssertionError("$scenarioName / ${request.id}: expected field=\"$wantField\", got: $body") } + (want["issues"] as? List<*>)?.let { wantIssues -> + assertIssues(scenarioName, request.id, body, wantIssues) + } return } (want["length"] as? Number)?.toInt()?.let { wantLen -> @@ -249,4 +252,26 @@ object ApiContractAssertions { is List<*> -> body.isEmpty() else -> false } + + /** + * The validation envelope lists its issues: `issues` is an array, every issue carries a string + * `code`, an array `path` and a string `message`, and each expected `{ path, code }` is among them. + */ + private fun assertIssues(scenarioName: String, requestId: String, body: Any?, expected: List<*>) { + val actual = (body as? Map<*, *>)?.get("issues") as? List<*> + ?: throw AssertionError("$scenarioName / $requestId: expected an issues array, got: $body") + for (issue in actual) { + val im = issue as? Map<*, *> + if (im == null || im["code"] !is String || im["path"] !is List<*> || im["message"] !is String) + throw AssertionError("$scenarioName / $requestId: an issue lacks code, path or message: $issue") + } + for (want in expected) { + val w = want as Map<*, *> + val found = actual.any { issue -> + val im = issue as Map<*, *> + im["code"] == w["code"] && im["path"].toString() == w["path"].toString() + } + if (!found) throw AssertionError("$scenarioName / $requestId: expected an issue $want, got: $actual") + } + } } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt index dcb8c29e6e..384ab1a084 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt @@ -184,11 +184,16 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { private fun createAuthor(exchange: HttpExchange) { @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map - ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + ?: return sendJson(exchange, 400, validationBody(listOf(issue("invalid_type", "request body must be a JSON object")))) // FR-036: enforce @maxLength (name 100 / bio 1000) at the wire boundary — a too-long // present value is a 400 {error:"validation"} BEFORE the insert (mirrors the generated // controller's validator.validate on POST). - if (violatesMaxLength(body)) return sendJson(exchange, 400, mapOf("error" to "validation")) + // A @required field (name, createdAt) missing or null is an invalid_type issue; every issue is listed. + val createIssues = lengthIssues(body) + for (f in listOf("name", "createdAt")) { + if (body[f] == null) createIssues.add(issue("invalid_type", "required", f)) + } + if (createIssues.isNotEmpty()) return sendJson(exchange, 400, validationBody(createIssues)) // #203/ADR-0045: @autoSet columns are server-owned — stamp BOTH onCreate + onUpdate from ONE // now() (a fresh row's autoUpdatedAt == autoCreatedAt exactly) and IGNORE any caller value. val now = Instant.now() @@ -214,16 +219,17 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { ?: return sendJson(exchange, 400, mapOf("error" to "invalid_id")) @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map - ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + ?: return sendJson(exchange, 400, validationBody(listOf(issue("invalid_type", "request body must be a JSON object")))) // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an // omitted required field is untouched (never a 400). + val patchIssues = lengthIssues(body) for (f in listOf("name", "createdAt")) { - if (body.containsKey(f) && body[f] == null) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (body.containsKey(f) && body[f] == null) patchIssues.add(issue("invalid_type", "cannot be null", f)) } // FR-036: validate PRESENT, non-null values against @maxLength (same rule as create) — a // too-long present value is a 400 BEFORE the write, leaving the stored row untouched. - if (violatesMaxLength(body)) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (patchIssues.isNotEmpty()) return sendJson(exchange, 400, validationBody(patchIssues)) val updated = transaction(db) { AuthorTable.update({ AuthorTable.id eq id }) { (body["name"] as? String)?.let { v -> it[name] = v } @@ -271,12 +277,20 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { * Only present, non-null strings are checked — absent fields and nulls are handled by the * per-verb guards. Mirrors the generated controller's field constraints over HTTP. */ - private fun violatesMaxLength(body: Map): Boolean { - (body["name"] as? String)?.let { if (it.length > 100) return true } - (body["bio"] as? String)?.let { if (it.length > 1000) return true } - return false + private fun lengthIssues(body: Map): MutableList> { + val out = ArrayList>() + (body["name"] as? String)?.let { if (it.length > 100) out.add(issue("too_big", "too long", "name")) } + (body["bio"] as? String)?.let { if (it.length > 1000) out.add(issue("too_big", "too long", "bio")) } + return out } + /** One validation issue: `{code, path, message}`; `path` is the field, empty when none is given. */ + private fun issue(code: String, message: String, vararg path: String): Map = + linkedMapOf("code" to code, "path" to path.toList(), "message" to message) + + private fun validationBody(issues: List>): Map = + linkedMapOf("error" to "validation", "issues" to issues) + private fun rowToMap(row: ResultRow): Map { val ts = row[AuthorTable.createdAt] return linkedMapOf( diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt index 6bde5f560f..75e821c4be 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt @@ -122,6 +122,12 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { } } + /** The validation answer: one generic issue (path empty), as this reference server does not itemise per field. */ + private fun invalidBody(): Map = linkedMapOf( + "error" to "validation", + "issues" to listOf(linkedMapOf("code" to "invalid_type", "path" to emptyList(), "message" to "invalid request body")), + ) + private fun getDocument(exchange: HttpExchange, idStr: String) { val id = idStr.toLongOrNull() ?: return sendJson(exchange, 400, mapOf("error" to "invalid_id")) @@ -135,18 +141,18 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { private fun createDocument(exchange: HttpExchange) { @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map - ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + ?: return sendJson(exchange, 400, invalidBody()) // primaryMarker is @required: absent / null / not-a-valid-Marker → 400. val primary = body["primaryMarker"] - if (primary == null || !isValidMarker(primary)) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (primary == null || !isValidMarker(primary)) return sendJson(exchange, 400, invalidBody()) // optional single VO: a present, non-null value must be a valid Marker. val optional = body["optionalMarker"] - if (optional != null && !isValidMarker(optional)) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (optional != null && !isValidMarker(optional)) return sendJson(exchange, 400, invalidBody()) // markers array-of-VO: a present, non-null value must be a list of valid Markers. val markersVal = body["markers"] if (markersVal != null && !(markersVal is List<*> && markersVal.all { isValidMarker(it) })) - return sendJson(exchange, 400, mapOf("error" to "validation")) - val canon = canonicalizeMeasures(body) ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + return sendJson(exchange, 400, invalidBody()) + val canon = canonicalizeMeasures(body) ?: return sendJson(exchange, 400, invalidBody()) val newId = transaction(db) { DocumentTable.insert { it[title] = body["title"] as String @@ -178,28 +184,28 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { ?: return sendJson(exchange, 400, mapOf("error" to "invalid_id")) @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map - ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + ?: return sendJson(exchange, 400, invalidBody()) // title is @required: an explicit null clears nothing — it is a 400. if (body.containsKey("title") && body["title"] == null) - return sendJson(exchange, 400, mapOf("error" to "validation")) + return sendJson(exchange, 400, invalidBody()) // primaryMarker @required: present-null → 400; a present value must be a valid Marker. if (body.containsKey("primaryMarker")) { val v = body["primaryMarker"] - if (v == null || !isValidMarker(v)) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (v == null || !isValidMarker(v)) return sendJson(exchange, 400, invalidBody()) } // optionalMarker nullable single VO: present-null clears; a present value must be valid. if (body.containsKey("optionalMarker")) { val v = body["optionalMarker"] - if (v != null && !isValidMarker(v)) return sendJson(exchange, 400, mapOf("error" to "validation")) + if (v != null && !isValidMarker(v)) return sendJson(exchange, 400, invalidBody()) } // markers nullable array-of-VO: present-null clears; present-[] empties; each present element valid. if (body.containsKey("markers")) { val v = body["markers"] if (v != null && !(v is List<*> && v.all { isValidMarker(it) })) - return sendJson(exchange, 400, mapOf("error" to "validation")) + return sendJson(exchange, 400, invalidBody()) } // Apply only the present keys (absent = untouched). Guard the empty-SET case (Exposed throws). - val canon = canonicalizeMeasures(body) ?: return sendJson(exchange, 400, mapOf("error" to "validation")) + val canon = canonicalizeMeasures(body) ?: return sendJson(exchange, 400, invalidBody()) val settableKeys = listOf("title", "payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures", "telemetry", "tiers", "pricing") if (settableKeys.any { body.containsKey(it) }) { transaction(db) { diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java index 6c6348a0c7..8fb84aca13 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java @@ -127,6 +127,9 @@ static void assertResponse(String scenarioName, ApiRequest request, int status, throw new AssertionError(scenarioName + " / " + request.id() + ": expected field=\"" + wantField + "\", got: " + body); } + if (want.get("issues") instanceof List wantIssues) { + assertIssues(scenarioName, request.id(), body, wantIssues); + } return; } Object lenObj = want.get("length"); @@ -269,4 +272,34 @@ private static boolean isEmpty(Object body) { if (body instanceof List l) return l.isEmpty(); return false; } + /** + * The validation envelope lists its issues: {@code issues} is an array, every issue carries a + * string {@code code}, an array {@code path} and a string {@code message}, and each expected + * {@code {path, code}} is among them. + */ + private static void assertIssues(String scenarioName, String requestId, Object body, List expected) { + Object issues = body instanceof Map bm ? bm.get("issues") : null; + if (!(issues instanceof List actual)) + throw new AssertionError(scenarioName + " / " + requestId + ": expected an issues array, got: " + body); + for (Object issue : actual) { + if (!(issue instanceof Map im) || !(im.get("code") instanceof String) + || !(im.get("path") instanceof List) || !(im.get("message") instanceof String)) + throw new AssertionError(scenarioName + " / " + requestId + + ": an issue lacks code, path or message: " + issue); + } + for (Object want : expected) { + Map w = (Map) want; + boolean found = false; + for (Object issue : actual) { + Map im = (Map) issue; + if (w.get("code").equals(im.get("code")) && String.valueOf(w.get("path")).equals(String.valueOf(im.get("path")))) { + found = true; + break; + } + } + if (!found) + throw new AssertionError(scenarioName + " / " + requestId + ": expected an issue " + want + + ", got: " + actual); + } + } } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java index 1af9ad16ba..6b02a2745e 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java @@ -441,14 +441,20 @@ private void getAuthor(HttpExchange exchange, String idStr) throws IOException, private void createAuthor(HttpExchange exchange) throws IOException, SQLException { Object parsed = readJsonBody(exchange); if (!(parsed instanceof Map)) { - sendJson(exchange, 400, Map.of("error", "validation")); + sendJson(exchange, 400, validationBody(List.of(issue("invalid_type", "request body must be a JSON object")))); return; } Map body = (Map) parsed; // FR-036 wire-tier constraint enforcement: reject an over-length name/bio with the // cross-port {"error":"validation"} envelope BEFORE persisting (mirrors the generated // controller's @Size(max=…) rules). - if (violatesLength(body)) { sendJson(exchange, 400, Map.of("error", "validation")); return; } + // A @required field (name, createdAt) that is missing or null is an invalid_type issue, and every + // issue is listed, not just the first. + List> createIssues = lengthIssues(body); + for (String reqField : List.of("name", "createdAt")) { + if (body.get(reqField) == null) createIssues.add(issue("invalid_type", "required", reqField)); + } + if (!createIssues.isEmpty()) { sendJson(exchange, 400, validationBody(createIssues)); return; } // Issue #203 / ADR-0045: honor @autoSet on create — stamp BOTH the onCreate and onUpdate // columns from ONE now() (the caller's values are ignored; a fresh row's autoUpdatedAt // equals its autoCreatedAt). @@ -486,22 +492,25 @@ private void updateAuthor(HttpExchange exchange, String idStr) throws IOExceptio Long id = parseLongOrNull(idStr); if (id == null) { sendJson(exchange, 400, Map.of("error", "invalid_id")); return; } Object parsed = readJsonBody(exchange); - if (!(parsed instanceof Map)) { sendJson(exchange, 400, Map.of("error", "validation")); return; } + if (!(parsed instanceof Map)) { + sendJson(exchange, 400, validationBody(List.of(issue("invalid_type", "request body must be a JSON object")))); + return; + } Map body = (Map) parsed; // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an // omitted required field is untouched (never a 400). + List> patchIssues = lengthIssues(body); for (String reqField : List.of("name", "createdAt")) { if (body.containsKey(reqField) && body.get(reqField) == null) { - sendJson(exchange, 400, Map.of("error", "validation")); - return; + patchIssues.add(issue("invalid_type", "cannot be null", reqField)); } } - // FR-036 wire-tier constraint enforcement: a PRESENT, non-null over-length name/bio is a - // 400 (matches the generated controller's per-present-value PATCH validation). - if (violatesLength(body)) { sendJson(exchange, 400, Map.of("error", "validation")); return; } + // 400 (matches the generated controller's per-present-value PATCH validation). Every issue + // is listed in the envelope. + if (!patchIssues.isEmpty()) { sendJson(exchange, 400, validationBody(patchIssues)); return; } // Dynamic SET clause — only update fields present in the body. List setClauses = new ArrayList<>(); @@ -562,10 +571,27 @@ private void updateAuthor(HttpExchange exchange, String idStr) throws IOExceptio * field is already rejected upstream. Returns {@code true} when a bound is exceeded so * the caller emits the {@code {"error":"validation"}} envelope. */ - private static boolean violatesLength(Map body) { - if (body.get("name") instanceof String n && n.length() > 100) return true; - if (body.get("bio") instanceof String b && b.length() > 1000) return true; - return false; + private static List> lengthIssues(Map body) { + List> out = new ArrayList<>(); + if (body.get("name") instanceof String n && n.length() > 100) out.add(issue("too_big", "too long", "name")); + if (body.get("bio") instanceof String b && b.length() > 1000) out.add(issue("too_big", "too long", "bio")); + return out; + } + + /** One validation issue: {@code {code, path, message}}; {@code path} is the field, empty when none is given. */ + private static Map issue(String code, String message, String... path) { + Map out = new LinkedHashMap<>(); + out.put("code", code); + out.put("path", List.of(path)); + out.put("message", message); + return out; + } + + private static Map validationBody(List> issues) { + Map out = new LinkedHashMap<>(); + out.put("error", "validation"); + out.put("issues", issues); + return out; } private void deleteAuthor(HttpExchange exchange, String idStr) throws IOException, SQLException { diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java index d476a4c0bd..bd3bdc0ad3 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java @@ -148,7 +148,7 @@ private void doHandle(HttpExchange exchange) throws IOException, SQLException { Map body = readBody(exchange); // Program D: nested value-object validation on CREATE (required primaryMarker present + // valid; a present optional/array VO valid) → 400 before persisting. - if (!voValidForCreate(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, Map.of("error", "validation")); return; } + if (!voValidForCreate(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, invalidBody()); return; } sendJson(exchange, 201, create(body)); return; } @@ -157,7 +157,7 @@ private void doHandle(HttpExchange exchange) throws IOException, SQLException { Map body = readBody(exchange); // FR-035 tristate + Program D nested VO validation: present-null on the @required // primaryMarker → 400; a present value must be a valid Marker (single or array) → 400. - if (!voValidForPatch(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, Map.of("error", "validation")); return; } + if (!voValidForPatch(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, invalidBody()); return; } Map row = patch(id, body); if (row == null) { sendJson(exchange, 404, Map.of("error", "not_found")); return; } sendJson(exchange, 200, row); @@ -262,6 +262,18 @@ private Map rowFrom(ResultSet rs) throws SQLException { return row; } + /** The validation envelope: this reference server checks the whole body and lists it as one issue (never asserted per field). */ + private static Map invalidBody() { + Map issue = new LinkedHashMap<>(); + issue.put("code", "invalid_type"); + issue.put("path", List.of()); + issue.put("message", "the body is not valid"); + Map out = new LinkedHashMap<>(); + out.put("error", "validation"); + out.put("issues", List.of(issue)); + return out; + } + // --- nested value-object validation (Program D — mirrors the cross-port contract) -------- /** diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java index 5da440f69a..2acd5bf2c8 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectRuntimeRoundTripTest.java @@ -346,8 +346,8 @@ public void anEditToTheOwnedRuntimeChangesWhatGeneratedCodeDoes() throws Excepti Path owned = projectDir.resolve("src/main/java/com/acme/runtime/PatchValidationException.java"); String src = Files.readString(owned, StandardCharsets.UTF_8); - String edited = src.replace("super(\"field '\" + field + \"' cannot be null\");", - "super(\"OWNED EDIT: \" + field + \" is required\");"); + String edited = src.replace("super(\"patch body is not valid: \"", + "super(\"OWNED EDIT: \""); assertFalse("the reference constructor changed shape; update this test's edit", edited.equals(src)); Files.writeString(owned, edited, StandardCharsets.UTF_8); @@ -367,7 +367,7 @@ public void anEditToTheOwnedRuntimeChangesWhatGeneratedCodeDoes() throws Excepti } catch (InvocationTargetException ex) { Throwable cause = ex.getCause(); assertEquals("com.acme.runtime.PatchValidationException", cause.getClass().getName()); - assertEquals("OWNED EDIT: title is required", cause.getMessage()); + assertEquals("OWNED EDIT: 1 problem(s)", cause.getMessage()); } } } @@ -446,7 +446,9 @@ private Path runOwnedGenerators(Path projectDir) throws Exception { */ private static String jacksonOnly() { return Stream.of(System.getProperty("java.class.path").split(File.pathSeparator)) - .filter(e -> e.replace('\\', '/').contains("/com/fasterxml/jackson/")) + .filter(e -> e.replace('\\', '/').contains("/com/fasterxml/jackson/") + // the validation envelope reads Bean Validation violations, as every generated controller does + || e.replace('\\', '/').contains("/jakarta/validation/")) .collect(Collectors.joining(File.pathSeparator)); } diff --git a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md index 1b300bfa6f..3f14a45b92 100644 --- a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md +++ b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md @@ -35,7 +35,9 @@ The generated router takes `dto: dict[str, Any]` for `POST` / `PATCH` / **FR-036 update:** the router now DOES import the generated Pydantic models (`from . import , Patch`) and constructs them to enforce field constraints over HTTP — POST validates `(**dto)`, PATCH validates -the all-optional `Patch(**dto)` (present values only) → `400 {"error":"validation"}`. +the all-optional `Patch(**dto)` (present values only) → `400 {"error":"validation", "issues": [...]}`, every +Pydantic error mapped to the cross-port `{code, path, message}` shape with Zod's code names +(`constraint_errors.validation_envelope`; bare before 1.3.0). The repository seam receives only the keys the input model validated, as a `dict` (the FR-035 present-key tristate + wire values are kept), so the strong-typing simplification below remains for the repo/response types. diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 6323db2384..934a9ea4fd 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -190,11 +190,13 @@ def _scalar_fields(entity: MetaObject) -> list[MetaField]: #: detour) before validation; a malformed one is the cross-port 400. _DECIMAL_COERCE: list[str] = [ " if dto is None: # the body was not a JSON object: the cross-port 400, not FastAPI's 422", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + " return JSONResponse(", + " status_code=400, content=validation_envelope(unreadable_body_issues())", + " )", " try:", " dto = coerce_decimal_fields(dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS)", - " except ValueError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + " except ValueError as _exc:", + " return JSONResponse(status_code=400, content=validation_envelope(value_error_issues(_exc)))", ] #: The request-body dependency of every write route: the body is read with @@ -539,15 +541,34 @@ def _frozen_strip_lines(entity: MetaObject, indent: str = " ") -> list[str]: ] -def _validated_input_lines(model: str, *comment: str) -> list[str]: +def _validated_input_lines(model: str, *comment: str, patch: bool = False) -> list[str]: """Validate ``dto`` against the input *model*, then rebind it to what the model validated - (see :data:`_VALIDATED_INPUT_LINES`): the repository never sees any other key.""" + (see :data:`_VALIDATED_INPUT_LINES`): the repository never sees any other key. + + A body that fails answers ``400`` listing EVERY issue. On a *patch* the explicit nulls on + ``@required`` fields (FR-035 PATCH-2) are issues too, listed beside the model's.""" + if patch: + check = [ + " _issues = required_null_issues(dto, _REQUIRED_FIELDS)", + " try:", + f" _valid = {model}(**dto)", + " except ValidationError as _exc:", + " _issues += pydantic_issues(_exc.errors())", + " if _issues:", + " return JSONResponse(status_code=400, content=validation_envelope(_issues))", + ] + else: + check = [ + " try:", + f" _valid = {model}(**dto)", + " except ValidationError as _exc:", + " return JSONResponse(", + " status_code=400, content=validation_envelope(pydantic_issues(_exc.errors()))", + " )", + ] return [ *(f" # {line}" for line in comment), - " try:", - f" _valid = {model}(**dto)", - " except ValidationError:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', + *check, " # SECURITY (1.2.1): only what the input model validated reaches the repository. A key", " # it does not declare (an undeclared name, a server-generated id, a readOnly or derived", " # field, a frozen one on update) is dropped silently, never 400'd.", @@ -1201,16 +1222,14 @@ def _emit_route_handler( ") -> Any:", *_DECIMAL_COERCE, *denied_strip, - " # FR-035 PATCH-2: an explicit null on a @required field is a 400 —", - " # a present null on a NON-required field falls through and clears it,", - " # and an OMITTED required field is untouched (never a 400).", - " for _k in _REQUIRED_FIELDS:", - " if _k in dto and dto[_k] is None:", - ' return JSONResponse(status_code=400, content={"error": "validation"})', *_validated_input_lines( patch_model, + "FR-035 PATCH-2: an explicit null on a @required field is a 400 — a present", + "null on a NON-required field falls through and clears it, and an OMITTED", + "required field is untouched (never a 400).", "FR-036: validate PRESENT, non-null values with the create rules (the", "all-optional patch model never fires required-checks on ABSENT fields).", + patch=True, ), # #203/ADR-0045: bump onUpdate @autoSet columns (server-owned) on EVERY patch, # above the repo seam — a server-inserted present key (compatible with the PATCH @@ -1517,6 +1536,11 @@ def _render_tph_router( parts.append("") parts.append(f"from {_runtime_module('constraint_errors')} import (") parts.append(" classify_constraint_error,") + parts.append(" pydantic_issues,") + parts.append(" required_null_issues,") + parts.append(" unreadable_body_issues,") + parts.append(" validation_envelope,") + parts.append(" value_error_issues,") parts.append(")") parts.append(f"from {_runtime_module('decimal_wire')} import (") parts.append(" coerce_decimal_fields,") @@ -1678,12 +1702,11 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(") -> Any:") parts.extend(_DECIMAL_COERCE) parts.extend(denied_strip_tph) - parts.append(" # FR-035 PATCH-2: an explicit null on a @required field is a 400.") - parts.append(" for _k in _REQUIRED_FIELDS:") - parts.append(" if _k in dto and dto[_k] is None:") - parts.append(' return JSONResponse(status_code=400, content={"error": "validation"})') parts.extend(_validated_input_lines( - f"{sub}Patch", "FR-036: validate PRESENT, non-null values with the subtype's create rules." + f"{sub}Patch", + "FR-035 PATCH-2: an explicit null on a @required field is a 400.", + "FR-036: validate PRESENT, non-null values with the subtype's create rules.", + patch=True, )) # #203/ADR-0045: pop the write-once onCreate column(s), bump onUpdate — ABOVE # the repo seam (empty for a non-@autoSet hierarchy — byte-identical output). @@ -1913,6 +1936,11 @@ def render_router( parts.append("") parts.append(f"from {_runtime_module('constraint_errors')} import (") parts.append(" classify_constraint_error,") + parts.append(" pydantic_issues,") + parts.append(" required_null_issues,") + parts.append(" unreadable_body_issues,") + parts.append(" validation_envelope,") + parts.append(" value_error_issues,") parts.append(")") parts.append(f"from {_runtime_module('decimal_wire')} import (") parts.append(" coerce_decimal_fields,") diff --git a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py index 4b99b81c56..adf0c53e09 100644 --- a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py +++ b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py @@ -40,7 +40,15 @@ from typing import Any, NamedTuple -__all__ = ["ConstraintFailure", "classify_constraint_error"] +__all__ = [ + "ConstraintFailure", + "classify_constraint_error", + "pydantic_issues", + "required_null_issues", + "unreadable_body_issues", + "value_error_issues", + "validation_envelope", +] #: Constraint kinds worth telling a client apart, in the cross-port snake_case spelling. _FOREIGN_KEY = "foreign_key" @@ -149,3 +157,115 @@ def classify_constraint_error(exc: BaseException | None) -> ConstraintFailure | # request itself got wrong. Both are client errors — neither is a 500. status = 409 if kind in (_FOREIGN_KEY, _UNIQUE) else 400 return ConstraintFailure(status=status, constraint=kind) + + +# --------------------------------------------------------------------------- +# The validation envelope: ``400 {"error": "validation", "issues": [...]}`` +# --------------------------------------------------------------------------- +# +# Every generated write route answers a body that fails validation with EVERY issue listed, +# in the shape the TypeScript routes use: each issue is ``{"code", "path", "message"}`` where +# ``code`` is one of Zod's names (``invalid_type``, ``too_small``, ``too_big``, +# ``invalid_format``, ``invalid_value``, ``not_multiple_of``, ``unrecognized_keys``, +# ``invalid_union``, ``custom``), ``path`` is the field's WIRE name (the alias) plus any +# nested member names, and ``message`` is this port's own wording. + +#: Pydantic error ``type`` -> Zod issue code, for the types that are not decided by a rule below. +_PYDANTIC_CODES: dict[str, str] = { + "missing": "invalid_type", + "string_too_long": "too_big", + "too_long": "too_big", + "less_than": "too_big", + "less_than_equal": "too_big", + "decimal_max_digits": "too_big", + "decimal_max_places": "too_big", + "decimal_whole_digits": "too_big", + "int_parsing_size": "too_big", + "url_too_long": "too_big", + "string_too_short": "too_small", + "too_short": "too_small", + "greater_than": "too_small", + "greater_than_equal": "too_small", + "string_pattern_mismatch": "invalid_format", + "enum": "invalid_value", + "literal_error": "invalid_value", + "multiple_of": "not_multiple_of", + "extra_forbidden": "unrecognized_keys", + "union_tag_invalid": "invalid_union", + "union_tag_not_found": "invalid_union", +} + +#: A ``*_parsing`` / ``*_from_*`` type whose text is not in the target's FORMAT (a malformed +#: date, uuid, URL...) is ``invalid_format``; every other ``*_parsing`` (``int_parsing``, +#: ``float_parsing``, ``bool_parsing``, ``decimal_parsing``) is a wrong value TYPE. +_FORMAT_PARSING_PREFIXES: tuple[str, ...] = ( + "date", "datetime", "time", "timedelta", "uuid", "url", "ip", "email", +) + + +def _issue_code(error_type: str) -> str: + code = _PYDANTIC_CODES.get(error_type) + if code is not None: + return code + if error_type.endswith("_type") or error_type in ("int_from_float", "is_instance_of"): + return "invalid_type" + if error_type.endswith("_parsing"): + prefix = error_type[: -len("_parsing")] + return "invalid_format" if prefix in _FORMAT_PARSING_PREFIXES else "invalid_type" + if error_type.startswith(("date_from_datetime", "datetime_from_date", "time_delta")): + return "invalid_format" + return "custom" + + +def pydantic_issues(errors: list[Any]) -> list[dict[str, Any]]: + """The envelope's issues for Pydantic's ``ValidationError.errors()`` -- every one, in order. + + ``path`` is Pydantic's ``loc``: the field's alias (the wire name) and, for a nested value + object, its members after it; ``code`` follows the table above.""" + return [ + { + "code": _issue_code(str(e.get("type", ""))), + "path": [p if isinstance(p, (int, str)) else str(p) for p in e.get("loc", ())], + "message": str(e.get("msg", "invalid value")), + } + for e in errors + ] + + +def required_null_issues(dto: dict[str, Any], required: Any) -> list[dict[str, Any]]: + """An issue per ``@required`` field *dto* sets to an explicit ``null`` (FR-035 PATCH-2), + in the order of *required* sorted by name.""" + return [ + {"code": "invalid_type", "path": [k], "message": "expected a value, received null"} + for k in sorted(required) + if k in dto and dto[k] is None + ] + + +def unreadable_body_issues() -> list[dict[str, Any]]: + """The single issue for a body that is not a JSON object (``path`` is ``[]``: the body as + a whole).""" + return [{"code": "invalid_type", "path": [], "message": "expected a JSON object body"}] + + +def value_error_issues(exc: ValueError) -> list[dict[str, Any]]: + """The issues of a decimal-coercion ``ValueError``. The decimal helper raises one carrying + an ``issues`` list (a wrong type or an out-of-range value, with its wire path); any other + ``ValueError`` is one ``custom`` issue at the body as a whole.""" + issues = getattr(exc, "issues", None) + if isinstance(issues, list) and issues: + return issues + return [{"code": "custom", "path": [], "message": str(exc)}] + + +def validation_envelope(issues: list[dict[str, Any]]) -> dict[str, Any]: + """``{"error": "validation", "issues": [...]}`` -- the issues de-duplicated on + ``(path, code)``, so a rule reported twice (a required null the model also rejects) lists once.""" + seen: set[tuple[str, str]] = set() + unique: list[dict[str, Any]] = [] + for issue in issues: + key = (repr(issue["path"]), issue["code"]) + if key not in seen: + seen.add(key) + unique.append(issue) + return {"error": "validation", "issues": unique} diff --git a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py index 984977c0d7..cbbd9c8506 100644 --- a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py +++ b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py @@ -52,13 +52,13 @@ def format_decimal(value: Decimal) -> str: return "0" normalized = _normalize(value) if abs(normalized.adjusted()) > MAX_DIGITS: - raise ValueError("decimal out of range") + raise _RangeError("decimal out of range") return format(normalized, "f") def _checked(value: Decimal) -> Decimal: if abs(value.adjusted()) > MAX_DIGITS and value != 0: - raise ValueError("decimal out of range") + raise _RangeError("decimal out of range") return value @@ -131,10 +131,30 @@ def check_decimal_range(value: Decimal, precision: int, scale: int) -> None: if not isinstance(exponent, int): raise ValueError("not a finite number") if -exponent > scale: - raise ValueError(f"more than {scale} fraction digit(s)") + raise _RangeError(f"more than {scale} fraction digit(s)") integer_digits = max(d.adjusted() + 1, 0) if d != 0 else 0 if integer_digits > precision - scale: - raise ValueError(f"more than {precision - scale} integer digit(s)") + raise _RangeError(f"more than {precision - scale} integer digit(s)") + + +class DecimalIssue(ValueError): + """A request-body decimal that cannot be accepted. ``issues`` is the validation envelope's + list for it (one issue: ``code``, wire ``path``, ``message``), so a route lists it beside + every other issue; ``str()`` keeps the ``name: reason`` text.""" + + def __init__(self, path: list[str | int], code: str, reason: str) -> None: + super().__init__(f"{'.'.join(str(p) for p in path)}: {reason}") + self.issues: list[dict[str, Any]] = [{"code": code, "path": path, "message": reason}] + + +def _decimal_issue(path: list[str | int], exc: ValueError) -> DecimalIssue: + """*exc* (a ``parse_decimal`` or range ``ValueError``) as an issue at *path*: a range + refusal is ``too_big``, anything unparseable is ``invalid_type``.""" + return DecimalIssue(path, "too_big" if isinstance(exc, _RangeError) else "invalid_type", str(exc)) + + +class _RangeError(ValueError): + """The value parsed but cannot be held by the column (W3).""" def coerce_decimal_fields( @@ -156,7 +176,7 @@ def coerce_decimal_fields( if k in decimal_fields: out[k] = v elif k in nested: - out[k] = _nested(v, nested[k], k) + out[k] = _nested(v, nested[k], [k]) else: out[k] = _floats(v) for name in decimal_fields: @@ -170,24 +190,24 @@ def coerce_decimal_fields( else _coerce_decimal(value, limit) ) except ValueError as exc: - raise ValueError(f"{name}: {exc}") from None + raise _decimal_issue([name], exc) from None return out -def _nested(value: Any, spec: dict[str, Any], path: str) -> Any: +def _nested(value: Any, spec: dict[str, Any], path: list[str | int]) -> Any: """A jsonb value-object field (*value* a dict, or a list of them) under the top-level decimal rule: each declared decimal inside is parsed under W2, held to its range under W3 (*spec*: ``{"d": {name: (precision, scale) | None}, "o": {name: spec}}``, or ``{"m": spec}`` for a ``field.map`` of value objects, applied to each value) and stored as its - canonical string; every other number is a float. Raises ``ValueError`` naming the path.""" + canonical string; every other number is a float. Raises ``DecimalIssue`` naming the path.""" if isinstance(value, list): - return [_nested(v, spec, path) for v in value] + return [_nested(v, spec, [*path, i]) for i, v in enumerate(value)] if not isinstance(value, dict): return _floats(value) if "m" in spec: # a field.map of value objects: the spec applies to EACH value; keys are arbitrary # text and never matched against member names. - return {k: _nested(v, spec["m"], f"{path}.{k}") for k, v in value.items()} + return {k: _nested(v, spec["m"], [*path, k]) for k, v in value.items()} out: dict[str, Any] = {} for k, v in value.items(): if k in spec["d"] and v is not None: @@ -195,9 +215,9 @@ def _nested(value: Any, spec: dict[str, Any], path: str) -> Any: try: out[k] = [None if e is None else _canonical(e, limit) for e in v] if isinstance(v, list) else _canonical(v, limit) except ValueError as exc: - raise ValueError(f"{path}.{k}: {exc}") from None + raise _decimal_issue([*path, k], exc) from None elif k in spec["o"]: - out[k] = _nested(v, spec["o"][k], f"{path}.{k}") + out[k] = _nested(v, spec["o"][k], [*path, k]) else: out[k] = _floats(v) return out diff --git a/server/python/tests/codegen/test_router_generator.py b/server/python/tests/codegen/test_router_generator.py index e57aaa8deb..6679e6e1f0 100644 --- a/server/python/tests/codegen/test_router_generator.py +++ b/server/python/tests/codegen/test_router_generator.py @@ -509,6 +509,21 @@ def test_every_write_verb_maps_constraint_violations() -> None: assert " return JSONResponse(status_code=failure.status, content=failure.body())" in out +def test_every_validation_failure_lists_its_issues() -> None: + """Every ``400`` the write routes answer is the issues envelope, never a bare + ``{"error": "validation"}``; PATCH lists the @required nulls beside the model's issues.""" + out = _require(_author()) + + assert '{"error": "validation"}' not in out + # create, update, the unreadable body (create + update) and the decimal coercion (create + update) + assert out.count("validation_envelope(unreadable_body_issues())") == 2 + assert out.count("validation_envelope(value_error_issues(_exc))") == 2 + assert out.count("validation_envelope(pydantic_issues(_exc.errors()))") == 1 # create + assert " _issues = required_null_issues(dto, _REQUIRED_FIELDS)" in out # update + assert " _issues += pydantic_issues(_exc.errors())" in out + assert "validation_envelope(_issues)" in out + + def test_generated_router_is_valid_python() -> None: """The emitted try/except must not leave a dangling block. diff --git a/server/python/tests/integration/api_contract_assertions.py b/server/python/tests/integration/api_contract_assertions.py index 4d700250b9..b1586acc60 100644 --- a/server/python/tests/integration/api_contract_assertions.py +++ b/server/python/tests/integration/api_contract_assertions.py @@ -93,6 +93,9 @@ def assert_response( raise AssertionError( f"{scenario_name} / {request_id}: expected field=\"{want_field}\", got: {body!r}" ) + # `issues` rides INSIDE this branch too, for the same reason as `field`. + if "issues" in expect_body: + _assert_issues(scenario_name, request_id, body, expect_body["issues"]) return if "length" in expect_body: @@ -268,3 +271,29 @@ def _is_empty(body: Any) -> bool: if isinstance(body, (str, list, dict)): return len(body) == 0 return False + + +def _assert_issues(scenario_name: str, request_id: str, body: Any, expected: list[dict[str, Any]]) -> None: + """The validation envelope lists its issues: ``issues`` is a list, every issue carries a + string ``code``, a list ``path`` and a string ``message``, and each expected + ``{path, code}`` is among them (``path`` deep-equal, ``code`` equal).""" + issues = body.get("issues") if isinstance(body, dict) else None + if not isinstance(issues, list): + raise AssertionError( + f"{scenario_name} / {request_id}: expected an issues array, got: {body!r}" + ) + for issue in issues: + if not ( + isinstance(issue, dict) + and isinstance(issue.get("code"), str) + and isinstance(issue.get("path"), list) + and isinstance(issue.get("message"), str) + ): + raise AssertionError( + f"{scenario_name} / {request_id}: an issue lacks code, path or message: {issue!r}" + ) + for want in expected: + if not any(i["code"] == want["code"] and i["path"] == want["path"] for i in issues): + raise AssertionError( + f"{scenario_name} / {request_id}: expected an issue {want!r}, got: {issues!r}" + ) diff --git a/server/python/tests/integration/api_contract_server.py b/server/python/tests/integration/api_contract_server.py index dfc31bbe5a..486c5d4376 100644 --- a/server/python/tests/integration/api_contract_server.py +++ b/server/python/tests/integration/api_contract_server.py @@ -99,14 +99,29 @@ _MAX_LENGTHS: dict[str, int] = {"name": 100, "bio": 1000} -def _length_violation(dto: dict[str, Any]) -> bool: - """True iff a present string value in *dto* exceeds its @maxLength cap. A - None / absent value never trips this (present-null clears; absent is untouched).""" +_REQUIRED: tuple[str, ...] = ("name", "createdAt") + + +def _validation_issues(dto: dict[str, Any], *, create: bool) -> list[dict[str, Any]]: + """Every validation issue in *dto*, in the TypeScript routes' shape (Zod's codes, the wire + name as the path). A missing @required field is an issue on create only (an omitted one is + untouched on PATCH); a present null on a @required field is one on both; a present string + over its @maxLength cap is ``too_big``. A None value on an optional field clears it.""" + issues: list[dict[str, Any]] = [] + for field in _REQUIRED: + if field not in dto and create: + issues.append({"code": "invalid_type", "path": [field], "message": "required"}) + elif field in dto and dto[field] is None: + issues.append({"code": "invalid_type", "path": [field], "message": "expected a value, received null"}) for field, cap in _MAX_LENGTHS.items(): value = dto.get(field) if isinstance(value, str) and len(value) > cap: - return True - return False + issues.append({"code": "too_big", "path": [field], "message": f"at most {cap} characters"}) + return issues + + +def _validation_400(issues: list[dict[str, Any]]) -> JSONResponse: + return JSONResponse(status_code=400, content={"error": "validation", "issues": issues}) def _coerce_scalar(raw: str, sub_type: str) -> Any: @@ -499,9 +514,11 @@ def get_author(author_id: int) -> Any: # POST /api/authors @app.post("/api/authors", status_code=status.HTTP_201_CREATED) def create_author(dto: dict[str, Any]) -> Any: - # FR-036: reject an over-length field before persisting (cross-port 400). - if _length_violation(dto): - return JSONResponse(status_code=400, content={"error": "validation"}) + # FR-036: reject an over-length / missing field before persisting (cross-port 400, + # every issue listed). + issues = _validation_issues(dto, create=True) + if issues: + return _validation_400(issues) return repo.create(dto) # PATCH + PUT /api/authors/{id} @@ -510,13 +527,11 @@ def create_author(dto: dict[str, Any]) -> Any: def update_author(author_id: int, dto: dict[str, Any]) -> Any: # FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, # per meta.json) is a 400 — a present null on the nullable bio clears it, - # and an omitted required field is simply untouched (never a 400). - for _k in ("name", "createdAt"): - if _k in dto and dto[_k] is None: - return JSONResponse(status_code=400, content={"error": "validation"}) - # FR-036: reject an over-length PRESENT value with the cross-port 400. - if _length_violation(dto): - return JSONResponse(status_code=400, content={"error": "validation"}) + # and an omitted required field is untouched (never a 400). FR-036: an + # over-length PRESENT value is a 400 too; every issue is listed. + issues = _validation_issues(dto, create=False) + if issues: + return _validation_400(issues) saved = repo.update(author_id, dto) if saved is None: return JSONResponse(status_code=404, content={"error": "not_found"}) diff --git a/server/python/tests/integration/test_generated_decimal_wire.py b/server/python/tests/integration/test_generated_decimal_wire.py index e91bbe6095..ec857760fe 100644 --- a/server/python/tests/integration/test_generated_decimal_wire.py +++ b/server/python/tests/integration/test_generated_decimal_wire.py @@ -30,6 +30,19 @@ from metaobjects.runtime import ObjectManager, SqliteDriver from metaobjects.wire import WireFormat + +def _is_validation(body: object) -> bool: + """The 400 envelope: ``{"error": "validation", "issues": [{code, path, message}, ...]}``.""" + if not isinstance(body, dict) or body.get("error") != "validation": + return False + issues = body.get("issues") + return ( + isinstance(issues, list) + and bool(issues) + and all(isinstance(i, dict) and {"code", "path", "message"} <= i.keys() for i in issues) + ) + + _META = { "metadata.root": { "package": "acme::shop", @@ -139,14 +152,14 @@ def test_put_and_patch_take_decimals_too(client: TestClient) -> None: patched = client.patch("/api/entries/1", content='{"amount": "2.250"}', headers={"content-type": "application/json"}) assert patched.status_code == 200 and patched.json()["amount"] == "2.25" bad = client.put("/api/entries/1", content='{"amount": "abc"}', headers={"content-type": "application/json"}) - assert bad.status_code == 400 and bad.json() == {"error": "validation"} + assert bad.status_code == 400 and _is_validation(bad.json()) @pytest.mark.parametrize("bad", ['""', '"abc"', '"1,5"', '" 1.5"', '"NaN"', '"Infinity"', '"0x10"', "true", "[1]", '"1e999999999"']) def test_garbage_is_a_400_validation(client: TestClient, bad: str) -> None: resp = _post(client, '{"amount": ' + bad + "}") assert resp.status_code == 400, resp.text - assert resp.json() == {"error": "validation"} + assert _is_validation(resp.json()) @pytest.mark.parametrize("body", ["{", "", "not json", "[1]", '"text"', "null"]) @@ -160,7 +173,7 @@ def test_a_malformed_or_non_object_json_body_is_a_400_validation(client: TestCli ): resp = send() assert resp.status_code == 400, (body, resp.text) - assert resp.json() == {"error": "validation"} + assert _is_validation(resp.json()) def test_a_decimal_filter_operand_is_validated_and_exact(client: TestClient) -> None: @@ -199,7 +212,7 @@ def test_a_value_beyond_the_declared_range_is_a_400_and_nothing_is_rounded(clien """W3: @precision 30, @scale 20 here, so 21 fraction digits and 11 integer digits fail.""" for bad in ('"0.123456789012345678901"', '"12345678901.5"'): resp = _post(client, '{"amount": ' + bad + "}") - assert resp.status_code == 400 and resp.json() == {"error": "validation"} + assert resp.status_code == 400 and _is_validation(resp.json()) assert client.get("/api/entries").json() == [] @@ -314,7 +327,7 @@ def test_a_decimal_inside_a_jsonb_value_object_follows_the_top_level_rule(float_ assert float_client.get("/api/samples/1").json()["fix"]["amount"] == "100.5" for bad in ('"0.12345"', '"abc"', '"123456789.5"'): resp = float_client.post("/api/samples", content='{"fix": {"latitude": 1, "amount": ' + bad + "}}", headers=hdr) - assert resp.status_code == 400 and resp.json() == {"error": "validation"}, (bad, resp.text) + assert resp.status_code == 400 and _is_validation(resp.json()), (bad, resp.text) assert len(float_client.get("/api/samples").json()) == 1 @@ -341,7 +354,7 @@ def test_a_decimal_in_a_value_object_map_applies_to_each_value_and_keys_are_neve '{"tiers": {"amount": {"amount": "abc"}}}', ): resp = float_client.post("/api/samples", content=bad, headers=hdr) - assert resp.status_code == 400 and resp.json() == {"error": "validation"}, (bad, resp.text) + assert resp.status_code == 400 and _is_validation(resp.json()), (bad, resp.text) assert len(float_client.get("/api/samples").json()) == 1 @@ -352,7 +365,7 @@ def test_a_decimal_array_is_range_checked_element_wise_and_coerced_for_the_drive # Decimal(str(list)) and raise InvalidOperation -> 500). hdr = {"content-type": "application/json"} bad = float_client.post("/api/samples", content='{"series": ["1", "0.12345"]}', headers=hdr) - assert bad.status_code == 400 and bad.json() == {"error": "validation"} + assert bad.status_code == 400 and _is_validation(bad.json()) root = load_string(json.dumps(_FLOAT_META)).root sample = next(c for c in root.children() if c.name.endswith("Sample")) series = next(f for f in sample.fields() if f.name == "series") @@ -436,7 +449,7 @@ def test_input_refusals_hold_under_native() -> None: hdr = {"content-type": "application/json"} for bad in ('{"series": ["abc"]}', '{"series": ["0.12345"]}', "[1]", "{"): resp = native.post("/api/samples", content=bad, headers=hdr) - assert resp.status_code == 400 and resp.json() == {"error": "validation"}, (bad, resp.text) + assert resp.status_code == 400 and _is_validation(resp.json()), (bad, resp.text) assert native.post("/api/samples", content='{"series": ["1.5", 2]}', headers=hdr).status_code == 201 bad_filter = native.get("/api/samples", params={"filter[series][gt]": "oops"}) assert bad_filter.status_code == 400 diff --git a/server/python/tests/integration/test_generated_router_validated_input.py b/server/python/tests/integration/test_generated_router_validated_input.py index b43ed21c08..0cffb2cf97 100644 --- a/server/python/tests/integration/test_generated_router_validated_input.py +++ b/server/python/tests/integration/test_generated_router_validated_input.py @@ -36,6 +36,19 @@ from .generated_tph_app import build_generated_tph_app + +def _is_validation(body: object) -> bool: + """The 400 envelope: ``{"error": "validation", "issues": [{code, path, message}, ...]}``.""" + if not isinstance(body, dict) or body.get("error") != "validation": + return False + issues = body.get("issues") + return ( + isinstance(issues, list) + and bool(issues) + and all(isinstance(i, dict) and {"code", "path", "message"} <= i.keys() for i in issues) + ) + + _META = { "metadata.root": { "package": "acme::secfix", @@ -161,7 +174,7 @@ def test_a_malformed_value_on_a_settable_field_is_still_a_400(member) -> None: client, repo, base = member resp = client.post(base, json={"name": "x" * 81, "isAdmin": True}) assert resp.status_code == 400, resp.text - assert resp.json() == {"error": "validation"} + assert _is_validation(resp.json()) assert repo.writes == [] diff --git a/server/python/tests/runtime/test_validation_issues.py b/server/python/tests/runtime/test_validation_issues.py new file mode 100644 index 0000000000..b84b340854 --- /dev/null +++ b/server/python/tests/runtime/test_validation_issues.py @@ -0,0 +1,137 @@ +"""The validation envelope every generated write route answers with (400, every issue listed).""" +from __future__ import annotations + +from decimal import Decimal +from typing import Annotated + +import pytest +from pydantic import BaseModel, Field, ValidationError + +from metaobjects.codegen.runtime.constraint_errors import ( + pydantic_issues, + required_null_issues, + unreadable_body_issues, + value_error_issues, + validation_envelope, +) +from metaobjects.codegen.runtime.decimal_wire import coerce_decimal_fields + + +@pytest.mark.parametrize( + ("error_type", "code"), + [ + ("missing", "invalid_type"), + ("string_type", "invalid_type"), + ("int_type", "invalid_type"), + ("model_type", "invalid_type"), + ("int_parsing", "invalid_type"), + ("float_parsing", "invalid_type"), + ("bool_parsing", "invalid_type"), + ("decimal_parsing", "invalid_type"), + ("int_from_float", "invalid_type"), + ("string_too_long", "too_big"), + ("too_long", "too_big"), + ("less_than", "too_big"), + ("less_than_equal", "too_big"), + ("decimal_max_digits", "too_big"), + ("decimal_max_places", "too_big"), + ("decimal_whole_digits", "too_big"), + ("string_too_short", "too_small"), + ("too_short", "too_small"), + ("greater_than", "too_small"), + ("greater_than_equal", "too_small"), + ("string_pattern_mismatch", "invalid_format"), + ("date_parsing", "invalid_format"), + ("datetime_parsing", "invalid_format"), + ("time_parsing", "invalid_format"), + ("uuid_parsing", "invalid_format"), + ("date_from_datetime_inexact", "invalid_format"), + ("enum", "invalid_value"), + ("literal_error", "invalid_value"), + ("multiple_of", "not_multiple_of"), + ("extra_forbidden", "unrecognized_keys"), + ("value_error", "custom"), + ("assertion_error", "custom"), + ("something_new", "custom"), + ], +) +def test_pydantic_error_type_maps_to_a_zod_code(error_type: str, code: str) -> None: + [issue] = pydantic_issues([{"type": error_type, "loc": ("f",), "msg": "m"}]) + assert issue == {"code": code, "path": ["f"], "message": "m"} + + +class _Inner(BaseModel): + note: Annotated[str, Field(max_length=3)] + + +class _Outer(BaseModel): + name: Annotated[str, Field(max_length=3)] + age: int + inner: _Inner + tags: list[Annotated[str, Field(min_length=2)]] + + +def test_real_pydantic_errors_list_every_issue_with_wire_paths() -> None: + with pytest.raises(ValidationError) as caught: + _Outer(**{"name": "toolong", "age": "x", "inner": {"note": "abcd"}, "tags": ["a"]}) + issues = pydantic_issues(caught.value.errors()) + got = {(tuple(i["path"]), i["code"]) for i in issues} + assert got == { + (("name",), "too_big"), + (("age",), "invalid_type"), + (("inner", "note"), "too_big"), + (("tags", 0), "too_small"), + } + assert all(isinstance(i["message"], str) and i["message"] for i in issues) + + +def test_a_missing_required_field_is_invalid_type() -> None: + with pytest.raises(ValidationError) as caught: + _Outer(**{"name": "ok"}) + assert {(tuple(i["path"]), i["code"]) for i in pydantic_issues(caught.value.errors())} >= { + (("age",), "invalid_type"), + (("inner",), "invalid_type"), + } + + +def test_required_null_issues_are_sorted_and_only_for_present_nulls() -> None: + dto = {"b": None, "a": None, "c": None, "d": 1} + issues = required_null_issues(dto, frozenset({"a", "b", "d", "e"})) + assert [(i["path"], i["code"]) for i in issues] == [(["a"], "invalid_type"), (["b"], "invalid_type")] + + +def test_unreadable_body_has_an_empty_path() -> None: + [issue] = unreadable_body_issues() + assert issue["path"] == [] and issue["code"] == "invalid_type" + + +def test_envelope_dedupes_on_path_and_code() -> None: + a = {"code": "invalid_type", "path": ["x"], "message": "one"} + b = {"code": "invalid_type", "path": ["x"], "message": "two"} + c = {"code": "too_big", "path": ["x"], "message": "three"} + assert validation_envelope([a, b, c]) == {"error": "validation", "issues": [a, c]} + + +def test_decimal_failures_carry_a_wire_path_and_code() -> None: + with pytest.raises(ValueError) as unparseable: + coerce_decimal_fields({"amount": "abc"}, frozenset({"amount"})) + [issue] = value_error_issues(unparseable.value) + assert (issue["path"], issue["code"]) == (["amount"], "invalid_type") + + with pytest.raises(ValueError) as too_big: + coerce_decimal_fields({"amount": Decimal("1234.5")}, frozenset({"amount"}), {"amount": (5, 2)}) + [issue] = value_error_issues(too_big.value) + assert (issue["path"], issue["code"]) == (["amount"], "too_big") + + +def test_a_nested_decimal_failure_names_its_member_path() -> None: + spec = {"money": {"d": {"value": (5, 2)}, "o": {}}} + with pytest.raises(ValueError) as caught: + coerce_decimal_fields({"money": {"value": Decimal("123456.7")}}, frozenset(), None, spec) + [issue] = value_error_issues(caught.value) + assert (issue["path"], issue["code"]) == (["money", "value"], "too_big") + + +def test_a_plain_value_error_is_one_custom_issue_at_the_body() -> None: + [issue] = value_error_issues(ValueError("boom")) + assert issue == {"code": "custom", "path": [], "message": "boom"} diff --git a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts index 9c65eb2be0..bc1eedbe4d 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts @@ -43,6 +43,10 @@ export interface ExpectBody { total?: number; error?: string; field?: string; + /** Validation issues that must each be present in the error envelope's `issues` array, matched + * on `path` and `code` (an issue's `message` is the port's own wording). Checked inside the + * `error` branch, like `field`. */ + issues?: Array<{ path: Array; code: string }>; empty?: boolean; /** The exact response text (UTF-8), for a CSV export. */ text?: string; @@ -255,6 +259,7 @@ export function assertResponse( if (f !== want.field) throw new Error(`${scenarioName} / ${request.id}: expected field="${want.field}", got: ${stringify(body)}`); } + if (want.issues !== undefined) assertIssues(scenarioName, request.id, body, want.issues); return; } if (want.length !== undefined) { @@ -337,3 +342,34 @@ function stringify(v: unknown): string { return val; }); } + +/** + * The validation envelope lists its issues: `issues` is an array, every issue carries a string + * `code`, an array `path` and a string `message`, and each expected `{ path, code }` is among them. + */ +function assertIssues( + scenarioName: string, + requestId: string, + body: unknown, + expected: Array<{ path: Array; code: string }>, +): void { + const issues = (body as { issues?: unknown }).issues; + if (!Array.isArray(issues)) + throw new Error(`${scenarioName} / ${requestId}: expected an issues array, got: ${stringify(body)}`); + for (const issue of issues) { + const i = issue as { code?: unknown; path?: unknown; message?: unknown }; + if (typeof i.code !== "string" || !Array.isArray(i.path) || typeof i.message !== "string") + throw new Error(`${scenarioName} / ${requestId}: an issue lacks code, path or message: ${stringify(issue)}`); + } + for (const want of expected) { + const found = issues.some( + (issue) => + (issue as { code: string }).code === want.code && + stringify((issue as { path: unknown[] }).path) === stringify(want.path), + ); + if (!found) + throw new Error( + `${scenarioName} / ${requestId}: expected an issue ${stringify(want)}, got: ${stringify(issues)}`, + ); + } +} diff --git a/server/typescript/packages/integration-tests/src/api-contract-server.ts b/server/typescript/packages/integration-tests/src/api-contract-server.ts index 4342322804..f024fd9ade 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-server.ts @@ -22,7 +22,7 @@ import Fastify, { type FastifyInstance, type RouteHandlerMethod } from "fastify"; import { type MetaRoot } from "@metaobjectsdev/metadata"; -import { ObjectManager, ValidationError, type Filter } from "@metaobjectsdev/runtime-ts"; +import { ObjectManager, ValidationError, type Filter, type ValidationFailure } from "@metaobjectsdev/runtime-ts"; import { kyselyDriver } from "@metaobjectsdev/runtime-ts/drivers"; import { Kysely, PostgresDialect } from "kysely"; import { Pool } from "pg"; @@ -133,7 +133,7 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // 500 handling. fastify.setErrorHandler((err, _req, reply) => { if (err instanceof ValidationError) { - reply.code(400).send({ error: "validation" }); + reply.code(400).send({ error: "validation", issues: err.errors.map(toIssue) }); return; } throw err; @@ -406,3 +406,32 @@ function parseSort( if (dir !== "asc" && dir !== "desc") return { invalid: true, field }; return [field, dir]; } + +/** + * One ObjectManager validation failure as a contract issue: `{ code, path, message }`, with the + * code taken from Zod's names (the generated routes' vocabulary) so every lane answers alike. + */ +function toIssue(f: ValidationFailure): { code: string; path: string[]; message: string } { + const expected = (f.expected ?? {}) as { min?: unknown; max?: unknown }; + let code: string; + switch (f.rule) { + case "required": + case "type": + case "array": + code = "invalid_type"; + break; + case "length": + case "range": + // A floor names `min` alone; a ceiling, or a decimal that overflows its precision, is too big. + code = expected.min !== undefined && expected.max === undefined ? "too_small" : "too_big"; + break; + case "regex": + case "format": + case "numeric": + code = "invalid_format"; + break; + default: + code = "custom"; + } + return { code, path: f.field.split("."), message: f.message }; +} From 0571646823f33c2f8201c86b71c63d9af000f684 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 13:34:21 -0500 Subject: [PATCH 04/31] feat(api): a page size for generated list routes, api.pagination (TypeScript) List endpoints had no hard cap: a TypeScript or C# list with no `limit` returned the whole table and `?limit=100000` was accepted everywhere. A project can now set `"api": { "pagination": { "defaultLimit": N, "maxLimit": M } }` in .metaobjects/config.json: a generated entity or projection list with no `limit` answers the default page, and a `limit` above the cap answers 400 {"error":"limit_too_large","maxLimit":M}. Reports keep their own row cap and opaque views their 1000. - sdk reads and validates the block; the loaded root carries it (apiPaginationOf), as it carries the wire format. - runtime-ts mounts take a `pagination` option (pageLimit); the packaged and reference Fastify and Hono routes emit it. Owned copies re-synced. - meta init writes 50 / 1000 for a new project; `--force` over an existing config does not add it. A project without the setting behaves as before, and meta verify advises (rule unbounded-list-pages) when routes are wired. - New api-contract `pagination/` sub-corpus, run in the TypeScript generated lane. The C#, Java, Kotlin and Python routes do not read it yet: their REST generators are under the non-TS freeze, pending that ruling. A config key, not vocabulary: metamodelVersion does not move. --- CHANGELOG.md | 33 +++++ docs/CONFORMANCE.md | 21 ++- docs/features/api-contract.md | 75 ++++++++-- docs/features/cli.md | 2 +- .../codegen/generators/routes.ts | 12 +- .../runtime/drizzle-fastify/filter-parser.ts | 2 +- .../codegen/runtime/drizzle-fastify/index.ts | 14 +- .../runtime/drizzle-fastify/list-params.ts | 29 ++++ .../drizzle-fastify/mount-read-only.ts | 11 +- .../showcase/codegen/generators/routes.ts | 12 +- .../runtime/drizzle-fastify/filter-parser.ts | 2 +- .../codegen/runtime/drizzle-fastify/index.ts | 14 +- .../runtime/drizzle-fastify/list-params.ts | 29 ++++ .../drizzle-fastify/mount-read-only.ts | 11 +- fixtures/api-contract-conformance/README.md | 9 +- .../pagination/.metaobjects/config.json | 6 + .../pagination/README.md | 51 +++++++ .../pagination/meta.json | 16 ++ .../scenarios/default-page-size.yaml | 13 ++ .../pagination/scenarios/limit-at-max.yaml | 12 ++ .../pagination/scenarios/limit-over-max.yaml | 15 ++ .../scenarios/offset-pages-past-default.yaml | 12 ++ .../scenarios/with-count-default-page.yaml | 14 ++ .../pagination/seed.json | 9 ++ .../packages/cli/src/commands/init.ts | 14 +- .../packages/cli/src/commands/verify.ts | 22 +++ .../packages/cli/src/lib/advisory.ts | 22 +++ .../cli/src/lib/collection-load-options.ts | 3 +- .../cli/src/lib/pagination-advisory.ts | 48 ++++++ .../typescript/packages/cli/test/init.test.ts | 8 + .../cli/test/pagination-advisory.test.ts | 42 ++++++ .../packages/codegen-ts/src/api-surface.ts | 13 ++ .../packages/codegen-ts/src/index.ts | 2 +- .../codegen-ts/src/reference/routes-hono.ts | 6 +- .../codegen-ts/src/reference/routes.ts | 12 +- .../src/templates/routes-file-hono.ts | 7 +- .../codegen-ts/src/templates/routes-file.ts | 13 +- .../codegen-ts/test/routes-pagination.test.ts | 96 ++++++++++++ ...pi-contract-pagination-generated-server.ts | 104 +++++++++++++ .../integration-tests/src/load-metadata.ts | 6 +- .../packages/integration-tests/src/paths.ts | 4 + .../test/api-contract-pagination.test.ts | 55 +++++++ .../packages/metadata/src/api-pagination.ts | 29 ++++ .../typescript/packages/metadata/src/index.ts | 5 + .../packages/metadata/src/shared/meta-root.ts | 8 + .../src/drizzle-fastify/filter-parser.ts | 2 +- .../runtime-ts/src/drizzle-fastify/index.ts | 14 +- .../src/drizzle-fastify/list-params.ts | 29 ++++ .../src/drizzle-fastify/mount-read-only.ts | 11 +- .../packages/runtime-ts/src/hono/index.ts | 14 +- .../runtime-ts/src/hono/mount-read-only.ts | 11 +- .../runtime-ts/test/list-page-size.test.ts | 140 ++++++++++++++++++ .../typescript/packages/sdk/src/collection.ts | 9 +- server/typescript/packages/sdk/src/config.ts | 46 ++++++ server/typescript/packages/sdk/src/index.ts | 2 +- server/typescript/packages/sdk/src/memory.ts | 11 +- .../packages/sdk/test/config.test.ts | 38 ++++- .../test-generators/src/generators/routes.ts | 12 +- 58 files changed, 1198 insertions(+), 84 deletions(-) create mode 100644 fixtures/api-contract-conformance/pagination/.metaobjects/config.json create mode 100644 fixtures/api-contract-conformance/pagination/README.md create mode 100644 fixtures/api-contract-conformance/pagination/meta.json create mode 100644 fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml create mode 100644 fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml create mode 100644 fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml create mode 100644 fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml create mode 100644 fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml create mode 100644 fixtures/api-contract-conformance/pagination/seed.json create mode 100644 server/typescript/packages/cli/src/lib/pagination-advisory.ts create mode 100644 server/typescript/packages/cli/test/pagination-advisory.test.ts create mode 100644 server/typescript/packages/codegen-ts/test/routes-pagination.test.ts create mode 100644 server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts create mode 100644 server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts create mode 100644 server/typescript/packages/metadata/src/api-pagination.ts create mode 100644 server/typescript/packages/runtime-ts/test/list-page-size.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 60079995e2..981aa90224 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,23 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi ### Added +- **A page size for generated list routes: `api.pagination` (TypeScript).** An external review + of 1.2.0 found that list endpoints had no hard cap: the TypeScript and C# routes returned the + whole table when `limit` was omitted, and `?limit=100000` was accepted in every port. Set + `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }` in + `.metaobjects/config.json` and a generated entity or projection list with no `limit` answers + the first `defaultLimit` rows, while an explicit `limit` above `maxLimit` answers + `400 {"error": "limit_too_large", "maxLimit": N}` (refused, not clamped). `meta init` writes + that block for a new project. **A project without the setting behaves exactly as before**, + because a cap added on upgrade would silently truncate a client that reads a whole list; + `meta verify` advises (rule `unbounded-list-pages`) when a routes generator is wired and no + page size is set. Reports keep their own row cap and opaque views their 1000. The TypeScript + Fastify and Hono routes read it (packaged and ejected: re-eject `routes` or re-sync an owned + copy to pick it up), through a new `pagination` option on the runtime mounts. It is gated by + the new `fixtures/api-contract-conformance/pagination/` sub-corpus. The C#, Java, Kotlin and + Python routes do not read it yet; carrying it there waits on the ruling on the freeze of the + non-TypeScript REST generators. A config key, not vocabulary: `metamodelVersion` does not move. + - **Report measures over Money and other value-object members (FR-044 R12).** A dimension's or measure's `@of` may name `Entity.field.member`, a member of a value object stored with `@storage: flattened` (`Payment.amount.amount`, `Payment.amount.currency`); the report reads @@ -74,6 +91,22 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi ### Changed +- **A validation 400 lists every issue in every port.** The same model answered a failed create or + update differently by language: TypeScript listed each problem, while C#, Java, Kotlin and Python + returned a bare `{"error": "validation"}` (found by an external review of 1.2.0). Every port now + answers `400 {"error": "validation", "issues": [{"code", "path", "message"}]}` with every failure, + not the first. `path` is an array of wire names (value-object members, array indexes and map keys + as further elements), `code` uses Zod's names (`invalid_type`, `too_small`, `too_big`, + `invalid_format`, `invalid_value`, `custom`), and `message` is each port's own wording. C# ships a + new `ValidationIssues` runtime file (also copied by `dotnet meta eject routes`), and its create + route now reads the body itself so an unbindable body lists its issue rather than answering an + empty 400. Java ships `ValidationIssues` beside `PatchValidationException`, and a patch binds every + field before it throws. Kotlin emits `MetaValidationIssues.kt` with the controllers. Python maps + every Pydantic error in `constraint_errors`. The hand-rolled reference servers list issues too, + and two shared scenarios pin the list in both lanes (`validation-lists-issues`, + `tph/tph-validation-lists-issues`, with a new `issues` corpus assertion). A client that read only + `error` is unaffected. An owned (ejected) routes copy keeps the bare answer until it is re-ejected. + - **Agent context routes report work to the reporting vocabulary.** The `meta init` agent guide and the `metaobjects-authoring` skill now send grouped totals, dashboard numbers and read-only report views to `object.report` (`references/declaring-reports.md`) before the `@sql` escape, diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 010d55867f..b5cc6469e5 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 62 (55 query + 7 migration) | all 62; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 55 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 123 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 132 (33 core + 12 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 5 pagination) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1; `pagination/` TypeScript only (see "Split coverage") | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | @@ -111,6 +111,19 @@ corpora, or between its *vocabulary* and its *behaviour* — never gets a row of own, and its boundary ends up discoverable only from the feature doc. The split ones are recorded here instead. +**`api.pagination`** (a page size on generated list routes, a `.metaobjects/config.json` key): +a list with no `limit` answers `defaultLimit` rows and a `limit` above `maxLimit` answers +`400 {"error": "limit_too_large", "maxLimit": N}`. + +- *Honoured and pinned:* TypeScript. The Fastify and Hono routes (packaged and ejected) pass it to + the runtime mounts; `fixtures/api-contract-conformance/pagination/` runs in the TypeScript + generated lane. +- *Not read:* C#, Java, Kotlin and Python. Their REST generators are under the 2026-09-22 freeze, + and carrying the setting there waits on that ruling. Their list routes keep the per-port + behaviour [api-contract.md](features/api-contract.md) records (TypeScript without the setting + and C# return every row when `limit` is omitted; Java, Kotlin and Python return 50; no port caps + an explicit `limit`), and the `pagination/` sub-corpus has no lane there yet. + **`identity.primary @sequenceStart`** (one semantic in every port): on a fresh database the identity or sequence begins at `@sequenceStart`; when the start changes on an existing database, the change sets `START WITH` only. It never `RESTART`s, and in 1.3 it does @@ -441,9 +454,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (119) +### `fixtures/api-contract-conformance/` (132) -All 119 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 132 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -720,7 +733,7 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 510 + yaml 16 + verify 45 -+ render 18 + persistence 62 + api-contract 123 + source-resolution 25 + scope 10 + ++ render 18 + persistence 62 + api-contract 132 + source-resolution 25 + scope 10 + dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 9a516d1b7f..6a36427b3f 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -216,7 +216,7 @@ or Hono `onError` the adopter installed, answer exactly as they did before. |---|---| | malformed JSON body | a `POST`/`PATCH`/`PUT` body that does not parse as JSON (an empty body sent as `application/json` included) → HTTP 400 `{ "error": "invalid_json" }`. An EMPTY body on a method that takes none (`GET`, `DELETE`, `HEAD`) is accepted even when the request declares `content-type: application/json`, as many clients do on every request; a non-empty `DELETE` body is still parsed. Before, Fastify answered its own `{ "statusCode": 400, "code": "FST_ERR_CTP_INVALID_JSON_BODY", … }` and Hono a Zod `validation` error about a missing object | | malformed id | a `:id` the key column cannot hold (a malformed uuid on a Postgres uuid key, a non-numeric id on an integer key) → HTTP 400 `{ "error": "invalid_id" }` on GET, PATCH and DELETE. A malformed uuid in a POST or PATCH body's foreign key → HTTP 400 `{ "error": "validation" }`, from the insert/update schema's `8-4-4-4-12` hex check. Postgres used to answer 500 for both. The check is TS-only | -| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. There is no page-size maximum on an entity or a projection with declared columns; an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. The corpus sends a malformed bound only to a report (`report/bad-page-bound`), where every port answers this same envelope and also refuses `limit=0`; on an entity or a projection the other ports answer it as their framework does | +| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. An entity or a projection with declared columns has no page-size maximum unless the project sets `api.pagination` (see "A page size is a project setting" below, which answers `limit_too_large`); an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. The corpus sends a malformed bound only to a report (`report/bad-page-bound`), where every port answers this same envelope and also refuses `limit=0`; on an entity or a projection the other ports answer it as their framework does | | database constraint violation | a write the database itself rejects → `{ "error": "constraint_violation", "constraint": "" }`, HTTP **409** for `unique` and `foreign_key` (a conflict with another row) and HTTP **400** for `check` and `not_null` (the request's own value), per the status rule under "Error response". The body names the constraint KIND only — never the SQL, the table or the bound values. A `CHECK` the model derives (an enum, a `validator.numeric`/`length`/`regex` range) is normally refused earlier by the route's own validation, as the corpus's `{ "error": "validation" }` 400; this row is the database's backstop | | schema validation failure | a body that fails the insert/update schema → HTTP 400 `{ "error": "validation", "issues": [...] }`, each issue as Zod reports it **minus `pattern`**: Zod 4 attaches a failed `.regex()` check's source there, which for a generated `field.date` / `field.time` / `field.timestamp` check is a ~300-character calendar regex (and an authored `validator.regex` pattern is the server's to keep). The generated temporal checks carry a short `message` naming the expected format instead (`"must be an ISO date (YYYY-MM-DD)"`) | | unexpected server error | anything that is not a filter, validation, not-found or constraint answer — a query against a column the database no longer has, a driver failure → HTTP 500 `{ "error": "internal" }`, the code the cross-port reference servers already use. The body names no SQL, table, column or bound parameter; the full error goes to the server log (`console.error`). Before, Fastify's default handler echoed the driver message, which for Drizzle is the query text and its parameter values | @@ -271,17 +271,34 @@ each handler they register instead; an `HTTPException` is rethrown to your - **A report list can be exported: `?format=csv|json|xml`.** See [Report export](#report-export) below. Without `format` the response is byte-identical to the list route described here. -- **No page-size maximum on an entity, and the default is per port.** The contract pins what - an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a - default or a ceiling, and the ports differ. With `limit` **omitted**, the - TypeScript mounts and the C# routes return **every** matching row, while the - Java, Kotlin and Python controllers return the first **50**. No port caps an - explicit `limit` on an entity — `limit=100000` is honoured as asked — except - the TS opaque-view route, which refuses a `limit` above 1000 (see "TS-only - error responses"). A list over a large table should therefore always send a - `limit`. A default page size is deliberately not introduced in a patch release: - it would silently truncate the lists of every existing client that relies on - the unpaged answer. +- **A page size is a project setting: `api.pagination`.** In `.metaobjects/config.json`: + + ```json + { "api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } } } + ``` + + With it set, a generated entity or projection list with `limit` **omitted** answers the first + `defaultLimit` rows, and an explicit `limit` above `maxLimit` answers + `400 { "error": "limit_too_large", "maxLimit": N }`. The request is refused, not clamped: + a client that asked for more than the server serves is told so rather than handed a short page + it cannot tell from the end of the list. `offset` and `withCount=1` page past the default as + usual, with `total` counting every row. `meta init` writes the block above for a new project; + both sizes must be positive integers and `defaultLimit` may not exceed `maxLimit`. A report + keeps its own row cap (above) and an opaque view its 1000, whatever the setting says. + + **Without the setting nothing changes, and the default is per port.** With `limit` omitted, + the TypeScript mounts and the C# routes return **every** matching row, while the Java, Kotlin + and Python controllers return the first **50**, and no port caps an explicit `limit` on an + entity (`limit=100000` is honoured as asked). That is the behaviour every existing project + has, so the setting is opt-in: adding a cap on upgrade would silently truncate a client that + reads a whole list. `meta verify` advises (rule `unbounded-list-pages`) when a routes + generator is wired and the project sets no page size. + + **Ports: TypeScript reads `api.pagination` today** (the Fastify and Hono routes, packaged and + ejected, gated by `fixtures/api-contract-conformance/pagination/` in the generated lane). + The C#, Java, Kotlin and Python routes do not read it yet: their REST generators are under + the 2026-09-22 freeze, and carrying the setting there waits on that ruling. Until then a list + over a large table in those ports should always send a `limit`. - `withCount=1` — opt-in flag that switches the list response from `[...]` to `{ rows: [...], total: }` (needed for grid pagination). The grid hook always sends `withCount=1`. @@ -505,6 +522,40 @@ Non-2xx responses MUST return: - HTTP 5xx — implementation-defined (the TS mount helpers answer `{ "error": "internal" }` and nothing more — see "TS-only error responses"). +#### A validation error lists every issue + +A body that fails the create or update rules answers HTTP 400 with every problem, not the first +one, in one shape in every port: + +```json +{ + "error": "validation", + "issues": [ + { "code": "too_big", "path": ["name"], "message": "..." }, + { "code": "invalid_type", "path": ["createdAt"], "message": "..." } + ] +} +``` + +- `path` is the field's location as an array of keys (a value-object member is + `["address", "city"]`). +- `code` uses Zod's names, the vocabulary TypeScript already reported: `invalid_type` (a missing + required field, an explicit `null` on one, or a value of the wrong type), `too_small` / + `too_big` (a length, size or numeric range), `invalid_format` (a pattern, an email, a date + shape), `invalid_value` (a value outside an enum), and `custom` for anything else (a + class-level rule, for instance). A port whose validator reports a finer failure may use another + Zod name for it (`not_multiple_of`, `unrecognized_keys`, `invalid_union`); a client that does not + know a code should treat it as `custom`. +- `message` is the port's own wording and is not part of the contract. +- A port may add keys to an issue (TypeScript keeps Zod's `minimum`, `maximum`, `origin` and so + on, minus `pattern`; see "TS-only error responses"), and a client must ignore keys it does not + know. + +Before 1.3.0 only TypeScript listed issues; the C#, Java, Kotlin and Python routes answered a +bare `{ "error": "validation" }`. The corpus pins the list in `validation-lists-issues` and +`tph/tph-validation-lists-issues` (two failing fields on a create, and on a PATCH an explicit +null on a required field beside an over-long value), in both lanes. + #### Filter and sort errors name the field These four codes are a hard cross-port invariant, and each one MUST carry diff --git a/docs/features/cli.md b/docs/features/cli.md index a2c84a40ee..462441bf28 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -25,7 +25,7 @@ command surface splits in two: | Capability | Tool | Invocation | Notes | |---|---|---|---| -| Project scaffold (`init`) | Node `meta` | `meta init` | TS projects; `--dialect sqlite\|postgres\|mysql\|d1` writes the codegen `dialect` in `metaobjects.config.ts` and, for sqlite, postgres and d1, `migrate.dialect` in `.metaobjects/config.json` (d1 is `--d1`; MySQL schema is not `meta migrate`'s) | +| Project scaffold (`init`) | Node `meta` | `meta init` | TS projects; `--dialect sqlite\|postgres\|mysql\|d1` writes the codegen `dialect` in `metaobjects.config.ts` and, for sqlite, postgres and d1, `migrate.dialect` in `.metaobjects/config.json` (d1 is `--d1`; MySQL schema is not `meta migrate`'s). A new `.metaobjects/config.json` also carries `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }`, the generated list routes' page size ([api-contract.md](api-contract.md)); `--force` over an existing config does not add it | | **Agent-context scaffold** (`agent-docs`) | **Node `meta`** | `npx meta agent-docs --server ` | **any backend** — single assembler (ADR-0033); non-Node CLIs redirect here | | **Schema migrate** | **Node `meta`** | `meta migrate` | **any backend** — schema is Node-only (ADR-0015) | | **Schema drift** (`verify --db`) | **Node `meta`** | `meta verify --db` | **any backend** — live-DB drift, Node-only | diff --git a/examples/advanced-modeling/codegen/generators/routes.ts b/examples/advanced-modeling/codegen/generators/routes.ts index 08d409771c..8ffe8c2da9 100644 --- a/examples/advanced-modeling/codegen/generators/routes.ts +++ b/examples/advanced-modeling/codegen/generators/routes.ts @@ -69,6 +69,7 @@ import { isReport, itemRouteField, decimalColumnsLine, + paginationLine, writeDeniedColumnsLine, servesReadApi, formatTs, @@ -167,7 +168,8 @@ function renderRoutes( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const FastifyInstanceSym = imp("t:FastifyInstance@fastify"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); // A projection mount is read-only by construction, so `expose` cannot narrow it — @@ -310,7 +312,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -335,7 +337,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -602,7 +604,7 @@ function renderTphRoutes( updateSchema: ${baseUpdateSym}, filterAllowlist: ${baseFilterSym}, sortAllowlist: ${baseSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${polymorphicExposeLine} + dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${paginationLine(base, " ", ctx.loadedRoot)}${polymorphicExposeLine} });`; // FR-018 x FR-017 — M:N traversal inside a TPH hierarchy. This file never consulted @@ -649,7 +651,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts index 540b6626ab..534c2f7c3e 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts @@ -10,7 +10,7 @@ import { FilterParseError, parsePageBound, SORT_EXPECTED } from "./list-params.j // Re-exported so existing importers keep one door; the definitions live in the // drizzle-free list-params module, which the ObjectManager Fastify mount also uses. -export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; +export { FilterParseError, parsePageBound, pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; import { ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT, looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant, diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts index 78baf3f97a..7e1b4e0b19 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts @@ -26,7 +26,7 @@ import { eq, count, and, asc } from "drizzle-orm"; import qs from "qs"; import type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; -import { parseFilterParams, parsePageBound, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; @@ -34,6 +34,7 @@ import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; +export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -108,6 +109,12 @@ export interface CrudRoutesOptions extends WireOptions { * Absent → ordinary single-table CRUD, behaviour unchanged. */ discriminator?: { column: string; value: string }; + /** + * The list page size (`api.pagination` in `.metaobjects/config.json`): a list with no `limit` + * returns `defaultLimit` rows, and a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent → no default and no ceiling. + */ + pagination?: PageSize; } const ALL_VERBS: readonly CrudVerb[] = ["list", "get", "create", "update", "delete"]; @@ -223,12 +230,13 @@ export function mountListRoute(opts: VerbOptions): void { // unspecified order). if (parsed.orderBy) q = q.orderBy(...parsed.orderBy); else if (src.id !== undefined) q = q.orderBy(asc(src.id)); - if (parsed.limit !== undefined) q = q.limit(parsed.limit); + const limit = pageLimit(parsed.limit, opts.pagination); + if (limit !== undefined) q = q.limit(limit); if (parsed.offset !== undefined) q = q.offset(parsed.offset); } else { // Legacy path — no allowlists configured. Only limit/offset (+ the TPH // discriminator predicate when subtype-scoped). - const limit = parsePageBound(qsParsed, "limit"); + const limit = pageLimit(parsePageBound(qsParsed, "limit"), opts.pagination); const offset = parsePageBound(qsParsed, "offset"); if (discCond) { q = q.where(discCond); where = discCond; } if (src.id !== undefined) q = q.orderBy(asc(src.id)); diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/list-params.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/list-params.ts index 8915b2f6e0..f8499489b1 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/list-params.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/list-params.ts @@ -51,6 +51,35 @@ export function parsePageBound( return n; } +/** The page size of a list route (`api.pagination` in `.metaobjects/config.json`). */ +export interface PageSize { + /** The rows a list with no `limit` returns. */ + readonly defaultLimit: number; + /** The largest `limit` a list accepts; a larger one is refused, never lowered. */ + readonly maxLimit: number; +} + +/** The error code of a `limit` above the page-size maximum. */ +export const ERROR_CODE_LIMIT_TOO_LARGE = "limit_too_large"; + +/** + * The `limit` a list route applies: the caller's, else the default page size, and a caller's + * above the maximum is refused with `400 { "error": "limit_too_large", "maxLimit": N }`. Without + * a page size the caller's `limit` (or none) passes through unchanged, as before. + */ +export function pageLimit(limit: number | undefined, pageSize: PageSize | undefined): number | undefined { + if (pageSize === undefined) return limit; + if (limit === undefined) return pageSize.defaultLimit; + if (limit > pageSize.maxLimit) { + throw new FilterParseError( + ERROR_CODE_LIMIT_TOO_LARGE, + `?limit= must be at most ${pageSize.maxLimit}, got ${limit}.`, + { maxLimit: pageSize.maxLimit }, + ); + } + return limit; +} + /** The `expected` member of an `invalid_sort` answer — the one spelling `?sort` takes. */ export const SORT_EXPECTED = "sort=:asc|desc"; diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/mount-read-only.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/mount-read-only.ts index 10b1247b23..b2827b936c 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/mount-read-only.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/mount-read-only.ts @@ -2,7 +2,7 @@ import type { FastifyInstance, RouteShorthandOptions } from "fastify"; import { quoteIdent, type SqlDialect } from "../sql-dialect.js"; import { sql, eq, and, count } from "drizzle-orm"; import qs from "qs"; -import { parseFilterParams, parsePageBound, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError, type PageSize } from "./filter-parser.js"; import { ReportExportError, assertExportWithinCap, exportContentDisposition, exportWithCountError, isExportCountFlag, parseExportPage, readExportFormat, renderExport, EXPORT_CONTENT_TYPES, type ReportExportSpec, @@ -61,6 +61,13 @@ export interface MountReadOnlyOptions extends WireOptions { * refused the same way. Default 10000, the same in every port. */ readonly reportMaxRows?: number; + /** + * The list page size of a projection (`api.pagination` in `.metaobjects/config.json`; ignored + * for a report, which has `reportMaxRows`, and for a raw view, which pages to at most 1000). + * A list with no `limit` returns `defaultLimit` rows; a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. + */ + readonly pagination?: PageSize; /** * Report export (`resource: "report"` only; ignored otherwise). Present when the project wires * the `report-export` generator: `?format=csv|json|xml` then answers the list in that shape, @@ -230,7 +237,7 @@ export function mountReadOnlyCrudRoutes(opts: MountReadOnlyOptions): void { if (result.orderBy) q = q.orderBy(...result.orderBy); // A report fetches one row past the cap when no limit was sent, to tell exactly the // cap apart from more than the cap before any body is written. - const limit = reportPage?.limit ?? result.limit; + const limit = reportPage?.limit ?? pageLimit(result.limit, opts.pagination); const offset = reportPage?.offset ?? result.offset; if (limit !== undefined) q = q.limit(limit); if (offset !== undefined) q = q.offset(offset); diff --git a/examples/showcase/codegen/generators/routes.ts b/examples/showcase/codegen/generators/routes.ts index 08d409771c..8ffe8c2da9 100644 --- a/examples/showcase/codegen/generators/routes.ts +++ b/examples/showcase/codegen/generators/routes.ts @@ -69,6 +69,7 @@ import { isReport, itemRouteField, decimalColumnsLine, + paginationLine, writeDeniedColumnsLine, servesReadApi, formatTs, @@ -167,7 +168,8 @@ function renderRoutes( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const FastifyInstanceSym = imp("t:FastifyInstance@fastify"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); // A projection mount is read-only by construction, so `expose` cannot narrow it — @@ -310,7 +312,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -335,7 +337,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -602,7 +604,7 @@ function renderTphRoutes( updateSchema: ${baseUpdateSym}, filterAllowlist: ${baseFilterSym}, sortAllowlist: ${baseSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${polymorphicExposeLine} + dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${paginationLine(base, " ", ctx.loadedRoot)}${polymorphicExposeLine} });`; // FR-018 x FR-017 — M:N traversal inside a TPH hierarchy. This file never consulted @@ -649,7 +651,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/filter-parser.ts b/examples/showcase/codegen/runtime/drizzle-fastify/filter-parser.ts index 540b6626ab..534c2f7c3e 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/filter-parser.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/filter-parser.ts @@ -10,7 +10,7 @@ import { FilterParseError, parsePageBound, SORT_EXPECTED } from "./list-params.j // Re-exported so existing importers keep one door; the definitions live in the // drizzle-free list-params module, which the ObjectManager Fastify mount also uses. -export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; +export { FilterParseError, parsePageBound, pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; import { ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT, looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant, diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/index.ts b/examples/showcase/codegen/runtime/drizzle-fastify/index.ts index 78baf3f97a..7e1b4e0b19 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/index.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/index.ts @@ -26,7 +26,7 @@ import { eq, count, and, asc } from "drizzle-orm"; import qs from "qs"; import type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; -import { parseFilterParams, parsePageBound, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; @@ -34,6 +34,7 @@ import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; +export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -108,6 +109,12 @@ export interface CrudRoutesOptions extends WireOptions { * Absent → ordinary single-table CRUD, behaviour unchanged. */ discriminator?: { column: string; value: string }; + /** + * The list page size (`api.pagination` in `.metaobjects/config.json`): a list with no `limit` + * returns `defaultLimit` rows, and a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent → no default and no ceiling. + */ + pagination?: PageSize; } const ALL_VERBS: readonly CrudVerb[] = ["list", "get", "create", "update", "delete"]; @@ -223,12 +230,13 @@ export function mountListRoute(opts: VerbOptions): void { // unspecified order). if (parsed.orderBy) q = q.orderBy(...parsed.orderBy); else if (src.id !== undefined) q = q.orderBy(asc(src.id)); - if (parsed.limit !== undefined) q = q.limit(parsed.limit); + const limit = pageLimit(parsed.limit, opts.pagination); + if (limit !== undefined) q = q.limit(limit); if (parsed.offset !== undefined) q = q.offset(parsed.offset); } else { // Legacy path — no allowlists configured. Only limit/offset (+ the TPH // discriminator predicate when subtype-scoped). - const limit = parsePageBound(qsParsed, "limit"); + const limit = pageLimit(parsePageBound(qsParsed, "limit"), opts.pagination); const offset = parsePageBound(qsParsed, "offset"); if (discCond) { q = q.where(discCond); where = discCond; } if (src.id !== undefined) q = q.orderBy(asc(src.id)); diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/list-params.ts b/examples/showcase/codegen/runtime/drizzle-fastify/list-params.ts index 8915b2f6e0..f8499489b1 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/list-params.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/list-params.ts @@ -51,6 +51,35 @@ export function parsePageBound( return n; } +/** The page size of a list route (`api.pagination` in `.metaobjects/config.json`). */ +export interface PageSize { + /** The rows a list with no `limit` returns. */ + readonly defaultLimit: number; + /** The largest `limit` a list accepts; a larger one is refused, never lowered. */ + readonly maxLimit: number; +} + +/** The error code of a `limit` above the page-size maximum. */ +export const ERROR_CODE_LIMIT_TOO_LARGE = "limit_too_large"; + +/** + * The `limit` a list route applies: the caller's, else the default page size, and a caller's + * above the maximum is refused with `400 { "error": "limit_too_large", "maxLimit": N }`. Without + * a page size the caller's `limit` (or none) passes through unchanged, as before. + */ +export function pageLimit(limit: number | undefined, pageSize: PageSize | undefined): number | undefined { + if (pageSize === undefined) return limit; + if (limit === undefined) return pageSize.defaultLimit; + if (limit > pageSize.maxLimit) { + throw new FilterParseError( + ERROR_CODE_LIMIT_TOO_LARGE, + `?limit= must be at most ${pageSize.maxLimit}, got ${limit}.`, + { maxLimit: pageSize.maxLimit }, + ); + } + return limit; +} + /** The `expected` member of an `invalid_sort` answer — the one spelling `?sort` takes. */ export const SORT_EXPECTED = "sort=:asc|desc"; diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/mount-read-only.ts b/examples/showcase/codegen/runtime/drizzle-fastify/mount-read-only.ts index 10b1247b23..b2827b936c 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/mount-read-only.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/mount-read-only.ts @@ -2,7 +2,7 @@ import type { FastifyInstance, RouteShorthandOptions } from "fastify"; import { quoteIdent, type SqlDialect } from "../sql-dialect.js"; import { sql, eq, and, count } from "drizzle-orm"; import qs from "qs"; -import { parseFilterParams, parsePageBound, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError, type PageSize } from "./filter-parser.js"; import { ReportExportError, assertExportWithinCap, exportContentDisposition, exportWithCountError, isExportCountFlag, parseExportPage, readExportFormat, renderExport, EXPORT_CONTENT_TYPES, type ReportExportSpec, @@ -61,6 +61,13 @@ export interface MountReadOnlyOptions extends WireOptions { * refused the same way. Default 10000, the same in every port. */ readonly reportMaxRows?: number; + /** + * The list page size of a projection (`api.pagination` in `.metaobjects/config.json`; ignored + * for a report, which has `reportMaxRows`, and for a raw view, which pages to at most 1000). + * A list with no `limit` returns `defaultLimit` rows; a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. + */ + readonly pagination?: PageSize; /** * Report export (`resource: "report"` only; ignored otherwise). Present when the project wires * the `report-export` generator: `?format=csv|json|xml` then answers the list in that shape, @@ -230,7 +237,7 @@ export function mountReadOnlyCrudRoutes(opts: MountReadOnlyOptions): void { if (result.orderBy) q = q.orderBy(...result.orderBy); // A report fetches one row past the cap when no limit was sent, to tell exactly the // cap apart from more than the cap before any body is written. - const limit = reportPage?.limit ?? result.limit; + const limit = reportPage?.limit ?? pageLimit(result.limit, opts.pagination); const offset = reportPage?.offset ?? result.offset; if (limit !== undefined) q = q.limit(limit); if (offset !== undefined) q = q.offset(offset); diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 452f71d00a..183c4ef3f5 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -29,6 +29,7 @@ fixtures/api-contract-conformance/ ├── create-201.yaml ├── input-server-owned-dropped.yaml # a client id, a made-up key and a writeOnce change are dropped, never 400 ├── create-constraint-violation-400.yaml # FR-036 POST field-constraint → 400 {error:"validation"} + ├── validation-lists-issues.yaml # a 400 validation lists every issue: {error, issues:[{code, path, message}]} ├── update-patch-and-put.yaml ├── update-constraint-violation-400.yaml # FR-036 PATCH present-value constraint → 400 ├── update-explicit-null-clears.yaml # FR-035 tristate: explicit null clears; null on @required → 400 @@ -53,13 +54,13 @@ fixtures/api-contract-conformance/ └── filter-invalid-value.yaml # uncoercible value → 400; the fourth envelope, previously ungated ``` -Beside the 32 core scenarios above sit ten **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (93 scenarios in all, 125 with the +Beside the 33 core scenarios above sit eleven **sub-corpora**, each a directory with its +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (99 scenarios in all, 132 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| -| `tph/` | 11 | single-table inheritance | both | +| `tph/` | 12 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | | `jsonb/` | 5 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | @@ -68,6 +69,7 @@ core): | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | | `decimal/` | 11 | `field.decimal` is an exact canonical string on the wire (ADR-0019 Amendment 1): read, filter, sort, write and a report over decimals. A scenario's `engines:` key is exact; the `-15` twins run on SQLite and D1 as well. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | +| `pagination/` | 5 | `api.pagination` in the sub-corpus's own `.metaobjects/config.json` (default 2, max 4): a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served and one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` | generated only (TypeScript; the other ports' routes wait on the non-TS REST generator ruling) | | `duration/` | 2 | `field.duration` is a canonical ISO-8601 span on the wire under `"wire": { "durations": "iso" }`: write, read, refuse, and filter and sort as a span on every engine. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` @@ -121,6 +123,7 @@ requests: | `envelope` | the response body is `{ rows, total }` (set `rowsLength` + `total`) | | `error` | the response body has `error: ""` | | `field` | **only meaningful alongside `error`** — the envelope's `field` names the rejected filter/sort field. Declared as a sibling of `error` and checked inside that branch, because the `error` branch returns; a runner that checked it as an independent key would skip it whenever `error` matched first. | +| `issues` | **only meaningful alongside `error`**, checked inside that branch like `field` — the envelope carries an `issues` array whose every entry has `code`, `path` and `message`, and each listed `{path, code}` is among them. `code` is Zod's name for the failure (`invalid_type`, `too_small`, `too_big`, `invalid_format`, `invalid_value`, `custom`); `message` is the port's own wording and never asserted. | | `empty` | the response body is empty / null (204 No Content) | | `absent` | the response body is an object; none of the listed keys is present (not even as `null`). Pins that a key a client sent but may not set never comes back. | | `fieldsEqual`| the response body is an object; assert the listed field names are all equal to **each other** (raw, field-vs-field — no literal, so timestamp non-determinism is a non-issue). Used by the `@autoSet` gate (#203/ADR-0045). | diff --git a/fixtures/api-contract-conformance/pagination/.metaobjects/config.json b/fixtures/api-contract-conformance/pagination/.metaobjects/config.json new file mode 100644 index 0000000000..f6d902d82d --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/.metaobjects/config.json @@ -0,0 +1,6 @@ +{ + "schema_version": 1, + "naming": { "columns": "literal" }, + "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" }, + "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 } } +} diff --git a/fixtures/api-contract-conformance/pagination/README.md b/fixtures/api-contract-conformance/pagination/README.md new file mode 100644 index 0000000000..686bd02b39 --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/README.md @@ -0,0 +1,51 @@ +# `api-contract-conformance/pagination/` — a project page size on generated list routes + +Cross-port REST contract for `api.pagination`, the project setting that gives every generated +list route a default page size and a hard cap on `?limit=`. The setting lives in this +sub-corpus's own `.metaobjects/config.json`, not in a runner: + +```json +{ "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 } } } +``` + +so a lane reaches it exactly the way an adopter's project does, through the config the +metadata load reads. The rest of that file repeats the parent corpus's `naming` and `wire`. + +The model is one `Item` (`id`, `title`), seeded with five rows. The sizes are small so the +scenarios can tell the default page from the whole table and the cap from the table size. + +## The contract + +- **No `limit` answers the default page.** `GET /api/items` returns 2 rows, not 5. +- **The cap is inclusive.** `?limit=4` returns 4 rows. +- **Over the cap is refused, not clamped.** `?limit=5` answers + `400 {"error": "limit_too_large", "maxLimit": 4}`, so a client that asked for more than the + server serves is told so instead of receiving a short page it cannot tell from the end. +- **Paging past the default works.** `?offset=4` returns the one remaining row, and + `?withCount=1` answers 2 rows with `total: 5`. + +A project WITHOUT the setting keeps the behaviour the parent corpus pins: no default page and +no cap. That half is the other sub-corpora, none of which sets `api.pagination`. + +## Lanes + +Generated lane only, as for every sub-corpus whose subject is what a generator emits. +**TypeScript runs it today.** The C#, Java, Kotlin and Python routes do not read +`api.pagination` yet: their REST generators are under the 2026-09-22 freeze, and adding the +setting there waits on that ruling. + +## Files + +``` +pagination/ +├── README.md +├── .metaobjects/config.json # api.pagination { defaultLimit: 2, maxLimit: 4 } +├── meta.json # Item +├── seed.json # five rows +└── scenarios/ + ├── default-page-size.yaml + ├── limit-at-max.yaml + ├── limit-over-max.yaml + ├── offset-pages-past-default.yaml + └── with-count-default-page.yaml +``` diff --git a/fixtures/api-contract-conformance/pagination/meta.json b/fixtures/api-contract-conformance/pagination/meta.json new file mode 100644 index 0000000000..0dd4aa5404 --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/meta.json @@ -0,0 +1,16 @@ +{ + "metadata.root": { + "package": "acme::paging", + "children": [ + { "object.entity": { + "name": "Item", + "children": [ + { "source.rdb": { "@table": "items" } }, + { "field.long": { "name": "id", "@filterable": true, "@sortable": true } }, + { "field.string": { "name": "title", "@required": true, "@maxLength": 100, "@filterable": true, "@sortable": true } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] + }} + ] + } +} diff --git a/fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml b/fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml new file mode 100644 index 0000000000..f9bbee942a --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml @@ -0,0 +1,13 @@ +name: default-page-size +description: > + A list with no `?limit=` answers the project's default page size (`api.pagination.defaultLimit`, + 2 in this corpus's config), not the whole table. The seed holds five rows, so a port that + ignores the setting returns five. +requests: + - id: r1 + method: GET + path: /api/items?sort=id:asc + expect: + status: 200 + body: + ids: [1, 2] diff --git a/fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml b/fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml new file mode 100644 index 0000000000..dbfb8be1b7 --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml @@ -0,0 +1,12 @@ +name: limit-at-max +description: > + An explicit `?limit=` equal to `api.pagination.maxLimit` (4) is served in full: the cap is + inclusive. +requests: + - id: r1 + method: GET + path: /api/items?limit=4&sort=id:asc + expect: + status: 200 + body: + ids: [1, 2, 3, 4] diff --git a/fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml b/fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml new file mode 100644 index 0000000000..faecfe8669 --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml @@ -0,0 +1,15 @@ +name: limit-over-max +description: > + An explicit `?limit=` above `api.pagination.maxLimit` answers 400 with the named error + `limit_too_large` and the cap, rather than being clamped: a client that asked for more than + the server serves is told so instead of silently receiving a short page. +requests: + - id: r1 + method: GET + path: /api/items?limit=5 + expect: + status: 400 + body: + equals: + error: "limit_too_large" + maxLimit: 4 diff --git a/fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml b/fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml new file mode 100644 index 0000000000..bd67f4793c --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml @@ -0,0 +1,12 @@ +name: offset-pages-past-default +description: > + `?offset=` with no `?limit=` pages through the table at the default page size: offset 4 of + five rows leaves one. +requests: + - id: r1 + method: GET + path: /api/items?offset=4&sort=id:asc + expect: + status: 200 + body: + ids: [5] diff --git a/fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml b/fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml new file mode 100644 index 0000000000..2527f905bf --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml @@ -0,0 +1,14 @@ +name: with-count-default-page +description: > + `?withCount=1` with no `?limit=` answers the default page in `rows` and the full row count in + `total`, so a client can page the rest. +requests: + - id: r1 + method: GET + path: /api/items?withCount=1 + expect: + status: 200 + body: + envelope: true + rowsLength: 2 + total: 5 diff --git a/fixtures/api-contract-conformance/pagination/seed.json b/fixtures/api-contract-conformance/pagination/seed.json new file mode 100644 index 0000000000..d901875584 --- /dev/null +++ b/fixtures/api-contract-conformance/pagination/seed.json @@ -0,0 +1,9 @@ +{ + "rows": [ + { "id": 1, "title": "Anvil" }, + { "id": 2, "title": "Bellows" }, + { "id": 3, "title": "Chisel" }, + { "id": 4, "title": "Drill" }, + { "id": 5, "title": "Easel" } + ] +} diff --git a/server/typescript/packages/cli/src/commands/init.ts b/server/typescript/packages/cli/src/commands/init.ts index d9cc1791fd..9285e0e575 100644 --- a/server/typescript/packages/cli/src/commands/init.ts +++ b/server/typescript/packages/cli/src/commands/init.ts @@ -2,6 +2,7 @@ import { mkdir, writeFile, readFile, readdir, stat, rm } from "node:fs/promises" import { join } from "node:path"; import { dirname } from "node:path"; import { existsSync as existsSyncWrap, readFileSync as readFileSyncWrap } from "node:fs"; +import { INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT } from "@metaobjectsdev/metadata"; import { DEFAULT_CONFIG, ConfigSchema, saveConfig, PACKAGE_MANIFEST_FILE, DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, DEPS_DIR, LOCK_FILE } from "@metaobjectsdev/sdk"; import { assemble, resolveAgentContextRoot, planScaffold, @@ -535,11 +536,18 @@ async function writeConfigFile(opts: InitOptions, result: InitResult, agentDir: // An explicit sqlite/postgres dialect is written so `meta migrate` needs no --dialect; an // unflagged init keeps the defaults it always wrote. MySQL schema is not migrate's. const dialect = resolveInitDialect(opts); + // A new project starts with a page size on every generated list route. It is written here + // and not into DEFAULT_CONFIG: the merge below must not add a cap to an existing project + // on `--force`, where it would truncate a client that reads a whole list. + const scaffold = { + ...DEFAULT_CONFIG, + api: { pagination: { defaultLimit: INIT_DEFAULT_LIMIT, maxLimit: INIT_MAX_LIMIT } }, + }; const freshConfig = dialect === "d1" - ? ConfigSchema.parse({ ...DEFAULT_CONFIG, migrate: buildD1MigrateBlock(opts.cwd) }) + ? ConfigSchema.parse({ ...scaffold, migrate: buildD1MigrateBlock(opts.cwd) }) : opts.dialect !== undefined && dialect !== "mysql" - ? ConfigSchema.parse({ ...DEFAULT_CONFIG, migrate: { dialect } }) - : DEFAULT_CONFIG; + ? ConfigSchema.parse({ ...scaffold, migrate: { dialect } }) + : ConfigSchema.parse(scaffold); const writeFresh = (): Promise => writeFile(join(agentDir, "config.json"), JSON.stringify(freshConfig, null, 2) + "\n", "utf8"); diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index 3e03dd836d..b4ffd8d02b 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -23,6 +23,7 @@ import { unindexedFkRows, warnAdvisory, referentialActionConflictRows, + unboundedListRows, advisoryHelpLines, } from "../lib/advisory.js"; import { scanForUnprovenancedLibraryPrefix } from "../lib/library-prefix-advisory.js"; @@ -32,6 +33,7 @@ import { warnIfAgentContextMissesRequirements, warnIfAgentContextStale } from ". import { warnIfManifestIgnored } from "../lib/manifest-ignored-check.js"; import { scanSourceForAntiPatterns } from "../lib/anti-patterns.js"; import { scanForMissingBaseUrl, type BaseUrlFinding } from "../lib/base-url-advisory.js"; +import { scanForUnsetPagination, type PaginationFinding } from "../lib/pagination-advisory.js"; import { scanForRemovedProps, type RemovedPropFinding } from "../lib/removed-prop-advisory.js"; import { replayRemedy } from "../lib/replay-remedy.js"; import { FileProvider, templateReferenceLine } from "../lib/file-provider.js"; @@ -1119,6 +1121,18 @@ export async function verifyCommand( // advisory only — a model the schema builder refuses is migrate's to report. } + // Generated list routes with no page size: a route generator is wired and the project + // sets no `api.pagination`. Same discipline as its siblings. + let unboundedLists: PaginationFinding[] = []; + try { + unboundedLists = scanForUnsetPagination( + (forgeConfig?.generators ?? []).map((g) => (typeof g === "string" ? g : g.name)), + collection.apiPagination, + ); + } catch { + // advisory only. + } + const unindexedFkSummary = summarizeUnindexedFks(unindexedFks); antiPatternSection = ranSection([ ...antiPatternRows(findings), @@ -1127,6 +1141,7 @@ export async function verifyCommand( ...libraryPrefixRows(libraryPrefix), ...unindexedFkRows(unindexedFkSummary === undefined ? [] : [unindexedFkSummary]), ...referentialActionConflictRows(actionConflicts), + ...unboundedListRows(unboundedLists), ]); if (findings.length > 0) { log.warn( @@ -1149,6 +1164,13 @@ export async function verifyCommand( ); warnAdvisory(removedProps.map((f) => ` ${f.message}`), flags.limit, { structured }); } + if (unboundedLists.length > 0) { + log.warn( + `meta verify — generated list routes have no page size ` + + `(advisory — does not fail the build):`, + ); + warnAdvisory(unboundedLists.map((f) => ` ${f.message}`), flags.limit, { structured }); + } if (unindexedFkSummary !== undefined) { log.warn( `meta verify — ${unindexedFks.length} foreign key(s) with no index on the referencing ` + diff --git a/server/typescript/packages/cli/src/lib/advisory.ts b/server/typescript/packages/cli/src/lib/advisory.ts index da51667589..c616aeeb30 100644 --- a/server/typescript/packages/cli/src/lib/advisory.ts +++ b/server/typescript/packages/cli/src/lib/advisory.ts @@ -115,6 +115,7 @@ export const RULE_REMOVED_PROP_PREFIX = "removed-prop:"; export const RULE_LIBRARY_PREFIX = "library-prefix-unprovenanced"; export const RULE_UNINDEXED_FK = "unindexed-foreign-key"; export const RULE_REFERENTIAL_ACTION_CONFLICT = "overridden-referential-action"; +export const RULE_UNBOUNDED_LIST = "unbounded-list-pages"; /** The next-step line for `n` rows of one advisory kind, each naming what the rows are. */ const ADVISORY_HELP: ReadonlyArray<{ matches: (rule: string) => boolean; line: (n: number) => string }> = [ @@ -128,6 +129,11 @@ const ADVISORY_HELP: ReadonlyArray<{ matches: (rule: string) => boolean; line: ( line: (n) => `${n} foreign key(s) whose two sides disagree on the referential action — see antiPatterns.rows[] (rule "${RULE_REFERENTIAL_ACTION_CONFLICT}")`, }, + { + matches: (r) => r === RULE_UNBOUNDED_LIST, + line: () => + `generated list routes with no page size — see antiPatterns.rows[] (rule "${RULE_UNBOUNDED_LIST}"), which names the api.pagination block to add`, + }, { matches: (r) => r === RULE_MISSING_BASE_URL, line: (n) => @@ -262,6 +268,22 @@ export function referentialActionConflictRows( })); } +/** + * Generated list routes with no page size, in the same row shape. `construct` is the wired + * route generator; the fix goes in the config file the row names. + */ +export function unboundedListRows( + findings: readonly { file: string; generator: string; message: string }[], +): AdvisoryFindingRow[] { + return findings.map((f) => ({ + file: f.file, + line: 0, + rule: RULE_UNBOUNDED_LIST, + construct: f.generator, + message: f.message, + })); +} + // --------------------------------------------------------------------------- // text output // --------------------------------------------------------------------------- diff --git a/server/typescript/packages/cli/src/lib/collection-load-options.ts b/server/typescript/packages/cli/src/lib/collection-load-options.ts index 69f6173ff3..4b5c685edc 100644 --- a/server/typescript/packages/cli/src/lib/collection-load-options.ts +++ b/server/typescript/packages/cli/src/lib/collection-load-options.ts @@ -27,7 +27,7 @@ import type { Collection, LoadMemoryOptions } from "@metaobjectsdev/sdk"; */ export function collectionLoadOptions( collection: Collection, -): Required> { +): Required> { return { files: collection.files, fileIds: collection.fileIds, @@ -42,6 +42,7 @@ export function collectionLoadOptions( physicalNaming: collection.physicalNaming, wireFormat: collection.wireFormat, enumDefaults: collection.enumDefaults ?? DEFAULT_ENUM_DEFAULTS, + apiPagination: collection.apiPagination, // The collection's migrate settings (`migrate.conventions`, `migrate.checks`) are not load // options: they reach the expected-schema builds through `schemaBuildOptions`. }; diff --git a/server/typescript/packages/cli/src/lib/pagination-advisory.ts b/server/typescript/packages/cli/src/lib/pagination-advisory.ts new file mode 100644 index 0000000000..700c2dd6de --- /dev/null +++ b/server/typescript/packages/cli/src/lib/pagination-advisory.ts @@ -0,0 +1,48 @@ +// Advisory: generated list routes with no page size. +// +// `api.pagination` in `.metaobjects/config.json` gives every generated list route a default +// page size and a hard cap on `?limit=`. A project that predates it keeps its old behaviour — +// a list with no `limit` returns the whole table, and any `limit` is accepted — because adding +// a cap on upgrade would silently truncate a client that reads a full list. So nothing turns +// it on behind the adopter's back; this names the gap instead. Advisory only: it never reaches +// an exit code. + +import { join } from "node:path"; +import { DEFAULT_METAOBJECTS_DIR } from "@metaobjectsdev/sdk"; + +/** The reference route generators whose list endpoints read `api.pagination`. */ +export const PAGINATED_ROUTE_GENERATORS: readonly string[] = [ + "routes-file", "routes-file-hono", // the factory names (`routesFile()`, `routesFileHono()`) + "routes", "routes-hono", // the stable string ids a config may name instead +]; + +export interface PaginationFinding { + /** The config file the setting goes in, relative to the project root. */ + file: string; + /** The wired generator whose list routes are unbounded. */ + generator: string; + message: string; +} + +/** + * One finding when a route generator that reads `api.pagination` is wired and the project + * declares none; otherwise none. + */ +export function scanForUnsetPagination( + generatorNames: readonly string[], + pagination: unknown, +): PaginationFinding[] { + if (pagination !== undefined) return []; + const generator = generatorNames.find((n) => PAGINATED_ROUTE_GENERATORS.includes(n)); + if (generator === undefined) return []; + const file = join(DEFAULT_METAOBJECTS_DIR, "config.json"); + return [{ + file, + generator, + message: + `${file}: ${generator} is wired and the project sets no api.pagination, so a generated ` + + `list route with no ?limit= returns the whole table and accepts any ?limit=. Add ` + + `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } } to give every list ` + + `a default page size and answer a larger ?limit= with 400 limit_too_large.`, + }]; +} diff --git a/server/typescript/packages/cli/test/init.test.ts b/server/typescript/packages/cli/test/init.test.ts index 9ca5b5e73c..fe635c2fb9 100644 --- a/server/typescript/packages/cli/test/init.test.ts +++ b/server/typescript/packages/cli/test/init.test.ts @@ -182,6 +182,12 @@ describe("init() — happy path", () => { expect(config.pending_in_git).toBe(true); }); + test("a new project starts with a page size on every generated list route", async () => { + await init({ cwd }); + const config = JSON.parse(readFileSync(join(cwd, ".metaobjects", "config.json"), "utf8")); + expect(config.api).toEqual({ pagination: { defaultLimit: 50, maxLimit: 1000 } }); + }); + test("writes a .gitignore under .metaobjects/ that includes .gen-state", async () => { await init({ cwd }); const ignore = readFileSync(join(cwd, ".metaobjects", ".gitignore"), "utf8"); @@ -409,6 +415,8 @@ describe("init() --force config preservation", () => { expect(reloaded.pending_in_git).toBe(false); expect(reloaded.confidence_thresholds.pending_promote).toBe(0.95); expect(reloaded.sources).toEqual([{ package: "@acme/entities" }]); + // An existing project keeps its unbounded lists: --force must not add a page size. + expect(reloaded.api).toBeUndefined(); }); test("writes fresh defaults when existing config is invalid (and warns)", async () => { diff --git a/server/typescript/packages/cli/test/pagination-advisory.test.ts b/server/typescript/packages/cli/test/pagination-advisory.test.ts new file mode 100644 index 0000000000..a99b625ff8 --- /dev/null +++ b/server/typescript/packages/cli/test/pagination-advisory.test.ts @@ -0,0 +1,42 @@ +// A generated list route with no page size returns the whole table. `meta verify` names that +// when a route generator is wired and the project declares no `api.pagination`, and stays +// silent otherwise. Advisory only. + +import { describe, test, expect } from "bun:test"; +import { scanForUnsetPagination } from "../src/lib/pagination-advisory.js"; +import { unboundedListRows, advisoryHelpLines, RULE_UNBOUNDED_LIST } from "../src/lib/advisory.js"; + +const PAGE = { defaultLimit: 50, maxLimit: 1000 }; + +describe("scanForUnsetPagination", () => { + test("routes wired, no page size: one finding naming the block to add", () => { + const [f, ...rest] = scanForUnsetPagination(["entity-file", "routes-file"], undefined); + expect(rest).toEqual([]); + expect(f!.generator).toBe("routes-file"); + expect(f!.file).toBe(".metaobjects/config.json"); + expect(f!.message).toContain('"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }'); + expect(f!.message).toContain("limit_too_large"); + }); + + test("the Hono routes and the stable string ids count too", () => { + for (const name of ["routes-file-hono", "routes", "routes-hono"]) { + expect(scanForUnsetPagination([name], undefined).length).toBe(1); + } + }); + + test("silent when the project sets a page size", () => { + expect(scanForUnsetPagination(["routes-file"], PAGE)).toEqual([]); + }); + + test("silent when no route generator is wired", () => { + expect(scanForUnsetPagination(["entity-file", "queries-file"], undefined)).toEqual([]); + expect(scanForUnsetPagination([], undefined)).toEqual([]); + }); + + test("rides the advisory row shape under its own rule and help line", () => { + const rows = unboundedListRows(scanForUnsetPagination(["routes-file"], undefined)); + expect(rows.map((r) => r.rule)).toEqual([RULE_UNBOUNDED_LIST]); + expect(rows[0]!.construct).toBe("routes-file"); + expect(advisoryHelpLines(rows).join("\n")).toContain(RULE_UNBOUNDED_LIST); + }); +}); diff --git a/server/typescript/packages/codegen-ts/src/api-surface.ts b/server/typescript/packages/codegen-ts/src/api-surface.ts index 98c3c46958..c761d468ab 100644 --- a/server/typescript/packages/codegen-ts/src/api-surface.ts +++ b/server/typescript/packages/codegen-ts/src/api-surface.ts @@ -37,6 +37,7 @@ import { WIRE_NULL_MEMBERS_NATIVE, refTarget, wireFormatOf, + apiPaginationOf, type MetaData, type MetaField, type MetaObject, @@ -252,6 +253,18 @@ export function valueObjectShapes(entity: MetaObject): Recordkey: value,` * line each, or "" when the object needs none: diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index f86f0e8959..3f7b75e781 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -146,7 +146,7 @@ export { renderAgentApi, renderApiIndex, renderEntityApiPage } from "./generator export { isProjection, isWriteThrough } from "./projection/projection-detector.js"; export { isAbstract, emitsInstanceArtifacts, emitsWriteArtifacts } from "./instance-artifacts.js"; // The UI tier asks THESE — "is there an endpoint?" — never the storage predicates. -export { hasGeneratedForm, hasItemRoute, decimalColumnsLine, writeDeniedColumnsLine, isCredentialField, decimalFieldNames, itemRouteField, restPath, servesClientHooks, servesClientTier, servesReadApi, servesWriteApi } from "./api-surface.js"; +export { hasGeneratedForm, hasItemRoute, decimalColumnsLine, paginationLine, writeDeniedColumnsLine, isCredentialField, decimalFieldNames, itemRouteField, restPath, servesClientHooks, servesClientTier, servesReadApi, servesWriteApi } from "./api-surface.js"; // #356 — every emitter selects a field's view by the SURFACE it renders, never by // declaration position. An owned generator (FR-040) composing the render layer must // use this too, or it reinstates the order-dependence in its own copy. diff --git a/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts b/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts index 0af5eeeb0d..bc0ff5116c 100644 --- a/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts +++ b/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts @@ -59,6 +59,7 @@ import { isReport, itemRouteField, decimalColumnsLine, + paginationLine, writeDeniedColumnsLine, servesReadApi, formatTs, @@ -148,7 +149,8 @@ function renderRoutesHono( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const HonoSym = imp("t:Hono@hono"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); @@ -244,7 +246,7 @@ export function ${handlerName}(app: ${HonoSym}, deps: { db: unkno updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineHono} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} }); } `; diff --git a/server/typescript/packages/codegen-ts/src/reference/routes.ts b/server/typescript/packages/codegen-ts/src/reference/routes.ts index 08d409771c..8ffe8c2da9 100644 --- a/server/typescript/packages/codegen-ts/src/reference/routes.ts +++ b/server/typescript/packages/codegen-ts/src/reference/routes.ts @@ -69,6 +69,7 @@ import { isReport, itemRouteField, decimalColumnsLine, + paginationLine, writeDeniedColumnsLine, servesReadApi, formatTs, @@ -167,7 +168,8 @@ function renderRoutes( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const FastifyInstanceSym = imp("t:FastifyInstance@fastify"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); // A projection mount is read-only by construction, so `expose` cannot narrow it — @@ -310,7 +312,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -335,7 +337,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -602,7 +604,7 @@ function renderTphRoutes( updateSchema: ${baseUpdateSym}, filterAllowlist: ${baseFilterSym}, sortAllowlist: ${baseSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${polymorphicExposeLine} + dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${paginationLine(base, " ", ctx.loadedRoot)}${polymorphicExposeLine} });`; // FR-018 x FR-017 — M:N traversal inside a TPH hierarchy. This file never consulted @@ -649,7 +651,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts b/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts index f828e14e1a..a4f947ecd3 100644 --- a/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts +++ b/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts @@ -31,7 +31,7 @@ import { entityModuleSpecifier, siblingSpecifier } from "../import-path.js"; import { GENERATED_HEADER, GENERATED_EDIT_NOTE, sidecarLine } from "../constants.js"; import { isProjection, isWriteThrough } from "../projection/projection-detector.js"; import { isReport } from "../source-detect.js"; -import { decimalColumnsLine, itemRouteField, writeDeniedColumnsLine } from "../api-surface.js"; +import { decimalColumnsLine, itemRouteField, paginationLine, writeDeniedColumnsLine } from "../api-surface.js"; import { DEFAULT_ID_FIELD } from "../pk-resolver.js"; import { authSeamJsDoc, type CrudVerb, exposeLine } from "../routes-expose.js"; import { effectivePackage } from "../docs-paths.js"; @@ -104,7 +104,8 @@ export function renderRoutesFileHono( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const HonoSym = imp("t:Hono@hono"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); @@ -200,7 +201,7 @@ export function ${handlerName}(app: ${HonoSym}, deps: { db: unkno updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineHono} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} }); } `; diff --git a/server/typescript/packages/codegen-ts/src/templates/routes-file.ts b/server/typescript/packages/codegen-ts/src/templates/routes-file.ts index 855bda278e..d267a3ca99 100644 --- a/server/typescript/packages/codegen-ts/src/templates/routes-file.ts +++ b/server/typescript/packages/codegen-ts/src/templates/routes-file.ts @@ -29,7 +29,7 @@ import { GENERATED_HEADER, GENERATED_EDIT_NOTE, sidecarLine } from "../constants import { routesHandlerName } from "../naming.js"; import { isProjection, isWriteThrough } from "../projection/projection-detector.js"; import { isReport } from "../source-detect.js"; -import { decimalColumnsLine, itemRouteField, writeDeniedColumnsLine } from "../api-surface.js"; +import { decimalColumnsLine, itemRouteField, paginationLine, writeDeniedColumnsLine } from "../api-surface.js"; import { DEFAULT_ID_FIELD, pkInfoOf } from "../pk-resolver.js"; import type { RelationEntry } from "../relation-resolver.js"; import { isTphDiscriminatorBase, tphPlan } from "./tph-discriminator.js"; @@ -104,7 +104,8 @@ export function renderRoutesFile( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const FastifyInstanceSym = imp("t:FastifyInstance@fastify"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); // A projection mount is read-only by construction, so `expose` cannot narrow it — @@ -247,7 +248,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -272,7 +273,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -531,7 +532,7 @@ function renderTphRoutesFile( updateSchema: ${baseUpdateSym}, filterAllowlist: ${baseFilterSym}, sortAllowlist: ${baseSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${polymorphicExposeLine} + dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${paginationLine(base, " ", ctx.loadedRoot)}${polymorphicExposeLine} });`; // FR-018 x FR-017 — M:N traversal inside a TPH hierarchy. This file never consulted @@ -578,7 +579,7 @@ function renderTphRoutesFile( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/server/typescript/packages/codegen-ts/test/routes-pagination.test.ts b/server/typescript/packages/codegen-ts/test/routes-pagination.test.ts new file mode 100644 index 0000000000..3799e19ff5 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/routes-pagination.test.ts @@ -0,0 +1,96 @@ +// `api.pagination` reaches every generated list mount: the packaged routes generators and the +// reference copies `meta eject` writes emit `pagination: { defaultLimit, maxLimit }` on each +// CRUD and projection list, never on a report (which keeps its own row ceiling), and emit +// nothing at all when the project sets no page size. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource, type MetaRoot } from "@metaobjectsdev/metadata"; +import { mkdtempSync, readFileSync, readdirSync, rmSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { runGen } from "../src/runner.js"; +import { defineConfig } from "../src/metaobjects-config.js"; +import type { Generator } from "../src/generator.js"; +import { routesFile } from "../src/generators/routes-file.js"; +import { routesFileHono } from "../src/generators/routes-file-hono.js"; +import { routesFile as referenceRoutes } from "../src/reference/routes.js"; +import { routesFileHono as referenceRoutesHono } from "../src/reference/routes-hono.js"; +import { paginationLine } from "../src/api-surface.js"; + +const MODEL = { + "metadata.root": { + package: "acme::shop", + children: [ + { + "object.entity": { + name: "Product", + children: [ + { "source.rdb": { "@table": "products" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ], + }, + }, + ], + }, +}; + +async function loadRoot(): Promise { + const res = await new MetaDataLoader().load([ + new InMemoryStringSource(JSON.stringify(MODEL), { id: "meta.json", format: "json" }), + ]); + expect(res.errors).toEqual([]); + return res.root; +} + +function allFiles(dir: string): string[] { + return readdirSync(dir).flatMap((n) => { + const p = join(dir, n); + return statSync(p).isDirectory() ? allFiles(p) : [p]; + }); +} + +async function routesSource(generator: Generator, page: { defaultLimit: number; maxLimit: number } | undefined): Promise { + const root = await loadRoot(); + if (page !== undefined) root.apiPagination = page; + const tmp = mkdtempSync(join(tmpdir(), "routes-pagination-")); + try { + await runGen({ + config: defineConfig({ outDir: join(tmp, "src"), extStyle: "none", dbImport: "../db", dialect: "sqlite", generators: [generator] }), + metadata: root, projectRoot: tmp, + }); + return allFiles(join(tmp, "src")).filter((f) => /Product.*routes/i.test(f)).map((f) => readFileSync(f, "utf8")).join("\n"); + } finally { rmSync(tmp, { recursive: true, force: true }); } +} + +const GENERATORS: Array<[string, () => Generator]> = [ + ["routesFile", () => routesFile()], + ["routesFileHono", () => routesFileHono()], + ["reference routes", () => referenceRoutes()], + ["reference routes-hono", () => referenceRoutesHono()], +]; + +describe("generated list routes carry the project's page size", () => { + for (const [name, make] of GENERATORS) { + test(`${name}: api.pagination is emitted on the list mount`, async () => { + const src = await routesSource(make(), { defaultLimit: 50, maxLimit: 1000 }); + expect(src).toContain("pagination: { defaultLimit: 50, maxLimit: 1000 }"); + }); + test(`${name}: no api.pagination emits no option`, async () => { + const src = await routesSource(make(), undefined); + expect(src.length).toBeGreaterThan(0); + expect(src).not.toContain("pagination"); + }); + } +}); + +describe("paginationLine", () => { + test("reads the loaded root, and is empty without a page size", async () => { + const root = await loadRoot(); + const product = root.objects()[0]!; + expect(paginationLine(product, " ")).toBe(""); + root.apiPagination = { defaultLimit: 20, maxLimit: 200 }; + expect(paginationLine(product, " ")).toBe("\n pagination: { defaultLimit: 20, maxLimit: 200 },"); + }); +}); diff --git a/server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts new file mode 100644 index 0000000000..e0b16e315b --- /dev/null +++ b/server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts @@ -0,0 +1,104 @@ +// api-contract-pagination-generated-server.ts — boots the GENERATED Item routes (the deployed +// artifact) over HTTP and drives them against the pagination corpus. The page size is not passed +// here: it reaches the emitted routes from the corpus's own `.metaobjects/config.json`, read by +// the metadata load, which is the path an adopter's `api.pagination` takes. +// +// Same shape as api-contract-secret-fields-generated-server.ts: runGen over the corpus meta into +// a temp dir, provision the table the emitted Drizzle binding reads, mount the EMITTED routes. + +import Fastify, { type FastifyInstance } from "fastify"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { runGen, defineConfig } from "@metaobjectsdev/codegen-ts"; +import { entityFile, routesFile } from "@metaobjectsdev/test-generators"; +import type pg from "pg"; +import { executeSql } from "./postgres-sql.ts"; +import { loadMetadataFile } from "./load-metadata.ts"; + +export interface PaginationSeed { + rows: Array<{ id: number; title: string }>; +} + +export interface GeneratedPaginationServerHandle { + baseUrl: string; + applySeed(seed: PaginationSeed): Promise; + close(): Promise; +} + +export async function startGeneratedPaginationServer( + connectionUri: string, + metaPath: string, +): Promise { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + const tmp = mkdtempSync(join(genTmpRoot, "api-contract-pagination-")); + + const root = await loadMetadataFile(metaPath); + const lr = await runGen({ + config: defineConfig({ + outDir: tmp, + extStyle: "none", + dbImport: "./db", + dialect: "postgres", + apiPrefix: "/api", + generators: [entityFile(), routesFile()], + }), + metadata: root, + }); + if (lr.warnings.length > 0) { + throw new Error(`codegen produced warnings: ${lr.warnings.join("; ")}`); + } + + const bigintTypesImport = pathToFileURL(join(here, "pg-bigint-number-types.ts")).href; + writeFileSync( + join(tmp, "db.ts"), + ` +import { drizzle } from "drizzle-orm/node-postgres"; +import pg from "pg"; +import { bigintAsNumberTypes } from ${JSON.stringify(bigintTypesImport)}; +export const pool = new pg.Pool({ connectionString: ${JSON.stringify(connectionUri)}, types: bigintAsNumberTypes }); +export const db = drizzle(pool); +`, + "utf8", + ); + + await executeSql( + connectionUri, + `CREATE TABLE IF NOT EXISTS "items" ( + "id" bigserial PRIMARY KEY, + "title" varchar(100) NOT NULL + );`, + ); + + const dbMod = (await import(pathToFileURL(join(tmp, "db.ts")).href)) as { pool: pg.Pool }; + + const fastify = Fastify(); + const mod = (await import(pathToFileURL(join(tmp, "Item.routes.ts")).href)) as Record< + string, + (f: FastifyInstance) => Promise + >; + await fastify.register(mod["itemRoutes"]!); + await fastify.ready(); + const baseUrl = await fastify.listen({ host: "127.0.0.1", port: 0 }); + + return { + baseUrl, + applySeed: async (seed: PaginationSeed) => { + await executeSql(connectionUri, `TRUNCATE TABLE "items" RESTART IDENTITY;`); + for (const r of seed.rows) + await executeSql(connectionUri, `INSERT INTO "items" ("id","title") VALUES (${r.id}, ${str(r.title)})`); + await executeSql(connectionUri, `SELECT setval(pg_get_serial_sequence('items','id'), (SELECT MAX("id") FROM "items"));`); + }, + close: async () => { + await fastify.close(); + await dbMod.pool.end(); + rmSync(tmp, { recursive: true, force: true }); + }, + }; +} + +function str(v: string): string { + return `'${v.replace(/'/g, "''")}'`; +} diff --git a/server/typescript/packages/integration-tests/src/load-metadata.ts b/server/typescript/packages/integration-tests/src/load-metadata.ts index a3000de5fd..c70524c500 100644 --- a/server/typescript/packages/integration-tests/src/load-metadata.ts +++ b/server/typescript/packages/integration-tests/src/load-metadata.ts @@ -6,7 +6,7 @@ import { existsSync, readdirSync, readFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { pathToFileURL } from "node:url"; import { DEFAULT_WIRE_FORMAT, loadDirectory, loadString, loadUris, type MetaRoot, type WireFormat } from "@metaobjectsdev/metadata"; -import { ConfigSchema, enumDefaultsFromConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "@metaobjectsdev/sdk"; +import { ConfigSchema, apiPaginationFromConfig, enumDefaultsFromConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "@metaobjectsdev/sdk"; /** * ADR-0058 — set `root`'s physical naming, wire format and enum defaults from the nearest `.metaobjects/config.json` at or @@ -23,6 +23,10 @@ export function applyNearestNaming(root: MetaRoot, startDir: string): MetaRoot { root.wireFormat = wireFormatFromConfig(config); // `enums.maxLength` sizes an enum column, so a corpus that sets it gets the schema it pins. root.enumDefaults = enumDefaultsFromConfig(config); + // `api.pagination` gives generated list routes a page size; a corpus without it keeps the + // unbounded lists every other corpus is written against. + const page = apiPaginationFromConfig(config); + if (page !== undefined) root.apiPagination = page; return root; } diff --git a/server/typescript/packages/integration-tests/src/paths.ts b/server/typescript/packages/integration-tests/src/paths.ts index 5ea6315a7b..fba6236960 100644 --- a/server/typescript/packages/integration-tests/src/paths.ts +++ b/server/typescript/packages/integration-tests/src/paths.ts @@ -50,6 +50,10 @@ export const API_CONTRACT_SECRET_FIELDS_SCENARIOS_DIR = resolve(API_CONTRACT_SEC export const API_CONTRACT_ENUM_DEFAULT_DIR = resolve(API_CONTRACT_DIR, "enum-default"); export const API_CONTRACT_ENUM_DEFAULT_SCENARIOS_DIR = resolve(API_CONTRACT_ENUM_DEFAULT_DIR, "scenarios"); +// `api.pagination` page size subcorpus (GENERATED lane only). +export const API_CONTRACT_PAGINATION_DIR = resolve(API_CONTRACT_DIR, "pagination"); +export const API_CONTRACT_PAGINATION_SCENARIOS_DIR = resolve(API_CONTRACT_PAGINATION_DIR, "scenarios"); + export const API_CONTRACT_PROJECTION_DIR = resolve(API_CONTRACT_DIR, "projection"); export const API_CONTRACT_PROJECTION_SCENARIOS_DIR = resolve(API_CONTRACT_PROJECTION_DIR, "scenarios"); // FR-044 view-backed report subcorpus (GENERATED lane only, every port). diff --git a/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts b/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts new file mode 100644 index 0000000000..aae908fcb4 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts @@ -0,0 +1,55 @@ +// pagination api-contract conformance (GENERATED lane). +// +// Drives fixtures/api-contract-conformance/pagination/ over HTTP against the GENERATED Item +// routes, emitted under the corpus's `api.pagination` (default 2, max 4): a list with no limit +// answers the default page, a limit over the cap answers 400 limit_too_large. Generated lane +// only — see the corpus README. One Postgres testcontainer per scenario. + +import { describe, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { API_CONTRACT_PAGINATION_DIR, API_CONTRACT_PAGINATION_SCENARIOS_DIR } from "../src/paths.ts"; +import { loadScenarios, assertResponse, type ApiScenario, parseBodyExact } from "../src/api-contract-scenario.ts"; +import { startPostgres } from "../src/postgres-container.ts"; +import { + startGeneratedPaginationServer, + type GeneratedPaginationServerHandle, + type PaginationSeed, +} from "../src/api-contract-pagination-generated-server.ts"; + +const SEED = JSON.parse(readFileSync(join(API_CONTRACT_PAGINATION_DIR, "seed.json"), "utf8")) as PaginationSeed; +const META_PATH = join(API_CONTRACT_PAGINATION_DIR, "meta.json"); + +describe("api contract pagination — GENERATED routes lane", () => { + for (const scenario of loadScenarios(API_CONTRACT_PAGINATION_SCENARIOS_DIR)) { + test(scenario.name, async () => { + const pg = await startPostgres(); + let server: GeneratedPaginationServerHandle | null = null; + try { + server = await startGeneratedPaginationServer(pg.connectionUri, META_PATH); + await server.applySeed(SEED); + await runScenario(scenario, server); + } finally { + if (server) await server.close(); + await pg.stop(); + } + }, { timeout: 60_000 }); + } +}); + +async function runScenario(scenario: ApiScenario, server: GeneratedPaginationServerHandle): Promise { + for (const req of scenario.requests) { + const init: RequestInit = { method: req.method }; + if (req.body !== undefined) { + init.body = JSON.stringify(req.body); + init.headers = { "content-type": "application/json" }; + } + const res = await fetch(server.baseUrl + req.path, init); + const bodyText = await res.text(); + let body: unknown = null; + if (bodyText.length > 0) { + try { body = parseBodyExact(bodyText); } catch { body = bodyText; } + } + assertResponse(scenario.name, req, res.status, body); + } +} diff --git a/server/typescript/packages/metadata/src/api-pagination.ts b/server/typescript/packages/metadata/src/api-pagination.ts new file mode 100644 index 0000000000..eeb23d55a5 --- /dev/null +++ b/server/typescript/packages/metadata/src/api-pagination.ts @@ -0,0 +1,29 @@ +// The page size of a generated list route: `api.pagination` in .metaobjects/config.json, +// carried on the loaded root beside the naming rules and read through `apiPaginationOf`. +// A project without the block has none, and its routes behave as they always have. + +import type { MetaData } from "./shared/meta-data.js"; + +/** The `api` block in `.metaobjects/config.json`. */ +export const API_KEY_PAGINATION = "pagination"; +/** The keys of `api.pagination`. */ +export const PAGINATION_KEY_DEFAULT_LIMIT = "defaultLimit"; +export const PAGINATION_KEY_MAX_LIMIT = "maxLimit"; + +/** What `meta init` writes for a new project. */ +export const INIT_DEFAULT_LIMIT = 50; +export const INIT_MAX_LIMIT = 1000; + +/** + * `api.pagination`: a list route with no `limit` returns `defaultLimit` rows, and an explicit + * `limit` above `maxLimit` is refused with `400 { "error": "limit_too_large", "maxLimit": N }`. + */ +export interface ApiPagination { + readonly defaultLimit: number; + readonly maxLimit: number; +} + +/** The page-size setting of the tree `node` belongs to: what its root carries, else none. */ +export function apiPaginationOf(node: MetaData): ApiPagination | undefined { + return (node.root() as { apiPagination?: ApiPagination | undefined }).apiPagination; +} diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index ea1f64c039..413ff50395 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -404,6 +404,11 @@ export type { } from "./naming.js"; export { resolveEnumMaxLength, enumDefaultsOf, DEFAULT_ENUM_DEFAULTS, ENUMS_KEY_MAX_LENGTH } from "./enum-length.js"; export type { EnumDefaults } from "./enum-length.js"; +export { + apiPaginationOf, API_KEY_PAGINATION, PAGINATION_KEY_DEFAULT_LIMIT, PAGINATION_KEY_MAX_LIMIT, + INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT, +} from "./api-pagination.js"; +export type { ApiPagination } from "./api-pagination.js"; // Wire format (ADR-0019 Amendment 2): the `wire` block of .metaobjects/config.json. export { diff --git a/server/typescript/packages/metadata/src/shared/meta-root.ts b/server/typescript/packages/metadata/src/shared/meta-root.ts index feaabdcfe6..e43d0c1d90 100644 --- a/server/typescript/packages/metadata/src/shared/meta-root.ts +++ b/server/typescript/packages/metadata/src/shared/meta-root.ts @@ -10,6 +10,7 @@ import type { MetaField } from "../core/field/meta-field.js"; import { DEFAULT_PHYSICAL_NAMING, type PhysicalNaming } from "../naming.js"; import { DEFAULT_WIRE_FORMAT, type WireFormat } from "../wire.js"; import { DEFAULT_ENUM_DEFAULTS, type EnumDefaults } from "../enum-length.js"; +import type { ApiPagination } from "../api-pagination.js"; import { PACKAGE_SEPARATOR } from "./structural.js"; import { lookupCanonical } from "../naming-refs.js"; @@ -33,6 +34,13 @@ export class MetaRoot extends MetaData { */ enumDefaults: EnumDefaults = DEFAULT_ENUM_DEFAULTS; + /** + * The project's list page size (`api.pagination` in `.metaobjects/config.json`), set by the + * project-aware load beside `physicalNaming` and read through `apiPaginationOf`. Undefined + * when the project declares none. + */ + apiPagination: ApiPagination | undefined = undefined; + /** Object entities defined at this root level. */ objects(): MetaObject[] { return this.cached("objects", () => diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts index 540b6626ab..534c2f7c3e 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/filter-parser.ts @@ -10,7 +10,7 @@ import { FilterParseError, parsePageBound, SORT_EXPECTED } from "./list-params.j // Re-exported so existing importers keep one door; the definitions live in the // drizzle-free list-params module, which the ObjectManager Fastify mount also uses. -export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; +export { FilterParseError, parsePageBound, pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js"; import { ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT, looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant, diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts index 78baf3f97a..7e1b4e0b19 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts @@ -26,7 +26,7 @@ import { eq, count, and, asc } from "drizzle-orm"; import qs from "qs"; import type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; -import { parseFilterParams, parsePageBound, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; @@ -34,6 +34,7 @@ import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; +export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -108,6 +109,12 @@ export interface CrudRoutesOptions extends WireOptions { * Absent → ordinary single-table CRUD, behaviour unchanged. */ discriminator?: { column: string; value: string }; + /** + * The list page size (`api.pagination` in `.metaobjects/config.json`): a list with no `limit` + * returns `defaultLimit` rows, and a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent → no default and no ceiling. + */ + pagination?: PageSize; } const ALL_VERBS: readonly CrudVerb[] = ["list", "get", "create", "update", "delete"]; @@ -223,12 +230,13 @@ export function mountListRoute(opts: VerbOptions): void { // unspecified order). if (parsed.orderBy) q = q.orderBy(...parsed.orderBy); else if (src.id !== undefined) q = q.orderBy(asc(src.id)); - if (parsed.limit !== undefined) q = q.limit(parsed.limit); + const limit = pageLimit(parsed.limit, opts.pagination); + if (limit !== undefined) q = q.limit(limit); if (parsed.offset !== undefined) q = q.offset(parsed.offset); } else { // Legacy path — no allowlists configured. Only limit/offset (+ the TPH // discriminator predicate when subtype-scoped). - const limit = parsePageBound(qsParsed, "limit"); + const limit = pageLimit(parsePageBound(qsParsed, "limit"), opts.pagination); const offset = parsePageBound(qsParsed, "offset"); if (discCond) { q = q.where(discCond); where = discCond; } if (src.id !== undefined) q = q.orderBy(asc(src.id)); diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/list-params.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/list-params.ts index 8915b2f6e0..f8499489b1 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/list-params.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/list-params.ts @@ -51,6 +51,35 @@ export function parsePageBound( return n; } +/** The page size of a list route (`api.pagination` in `.metaobjects/config.json`). */ +export interface PageSize { + /** The rows a list with no `limit` returns. */ + readonly defaultLimit: number; + /** The largest `limit` a list accepts; a larger one is refused, never lowered. */ + readonly maxLimit: number; +} + +/** The error code of a `limit` above the page-size maximum. */ +export const ERROR_CODE_LIMIT_TOO_LARGE = "limit_too_large"; + +/** + * The `limit` a list route applies: the caller's, else the default page size, and a caller's + * above the maximum is refused with `400 { "error": "limit_too_large", "maxLimit": N }`. Without + * a page size the caller's `limit` (or none) passes through unchanged, as before. + */ +export function pageLimit(limit: number | undefined, pageSize: PageSize | undefined): number | undefined { + if (pageSize === undefined) return limit; + if (limit === undefined) return pageSize.defaultLimit; + if (limit > pageSize.maxLimit) { + throw new FilterParseError( + ERROR_CODE_LIMIT_TOO_LARGE, + `?limit= must be at most ${pageSize.maxLimit}, got ${limit}.`, + { maxLimit: pageSize.maxLimit }, + ); + } + return limit; +} + /** The `expected` member of an `invalid_sort` answer — the one spelling `?sort` takes. */ export const SORT_EXPECTED = "sort=:asc|desc"; diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/mount-read-only.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/mount-read-only.ts index 10b1247b23..b2827b936c 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/mount-read-only.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/mount-read-only.ts @@ -2,7 +2,7 @@ import type { FastifyInstance, RouteShorthandOptions } from "fastify"; import { quoteIdent, type SqlDialect } from "../sql-dialect.js"; import { sql, eq, and, count } from "drizzle-orm"; import qs from "qs"; -import { parseFilterParams, parsePageBound, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError } from "./filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError, type PageSize } from "./filter-parser.js"; import { ReportExportError, assertExportWithinCap, exportContentDisposition, exportWithCountError, isExportCountFlag, parseExportPage, readExportFormat, renderExport, EXPORT_CONTENT_TYPES, type ReportExportSpec, @@ -61,6 +61,13 @@ export interface MountReadOnlyOptions extends WireOptions { * refused the same way. Default 10000, the same in every port. */ readonly reportMaxRows?: number; + /** + * The list page size of a projection (`api.pagination` in `.metaobjects/config.json`; ignored + * for a report, which has `reportMaxRows`, and for a raw view, which pages to at most 1000). + * A list with no `limit` returns `defaultLimit` rows; a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. + */ + readonly pagination?: PageSize; /** * Report export (`resource: "report"` only; ignored otherwise). Present when the project wires * the `report-export` generator: `?format=csv|json|xml` then answers the list in that shape, @@ -230,7 +237,7 @@ export function mountReadOnlyCrudRoutes(opts: MountReadOnlyOptions): void { if (result.orderBy) q = q.orderBy(...result.orderBy); // A report fetches one row past the cap when no limit was sent, to tell exactly the // cap apart from more than the cap before any body is written. - const limit = reportPage?.limit ?? result.limit; + const limit = reportPage?.limit ?? pageLimit(result.limit, opts.pagination); const offset = reportPage?.offset ?? result.offset; if (limit !== undefined) q = q.limit(limit); if (offset !== undefined) q = q.offset(offset); diff --git a/server/typescript/packages/runtime-ts/src/hono/index.ts b/server/typescript/packages/runtime-ts/src/hono/index.ts index 44be10b276..e23788f59d 100644 --- a/server/typescript/packages/runtime-ts/src/hono/index.ts +++ b/server/typescript/packages/runtime-ts/src/hono/index.ts @@ -31,7 +31,9 @@ import { contractErrorCode } from "../drizzle-fastify/util.js"; import { parseFilterParams, parsePageBound, + pageLimit, FilterParseError, + type PageSize, type ParseFilterResult, } from "../drizzle-fastify/filter-parser.js"; import type { @@ -53,6 +55,7 @@ import { guardRoute, readJsonBody } from "./route-guard.js"; // Back-compat re-export — the unsafe local copy was consolidated onto the one // shared (deprecated) helper so the three adapters can't silently diverge. export { parseId } from "../drizzle-fastify/util.js"; +export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "../drizzle-fastify/list-params.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -117,6 +120,12 @@ export interface CrudRoutesOptions extends WireOptions { sortAllowlist?: SortAllowlist; /** Dialect — required if filterAllowlist or sortAllowlist is set (for dialect-specific `like` lowering: SQLite lowers to GLOB to stay case-sensitive; ADR-0049). */ dialect?: SqlDialect; + /** + * The list page size (`api.pagination` in `.metaobjects/config.json`): a list with no `limit` + * returns `defaultLimit` rows, and a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent → no default and no ceiling. + */ + pagination?: PageSize; } const ALL_VERBS: readonly CrudVerb[] = ["list", "get", "create", "update", "delete"]; @@ -214,11 +223,12 @@ export function mountListRoute(opts: VerbOptions): void { : (parsed.where ?? parsed.searchWhere); if (combinedWhere) { q = q.where(combinedWhere); where = combinedWhere; } if (parsed.orderBy) q = q.orderBy(...parsed.orderBy); - if (parsed.limit !== undefined) q = q.limit(parsed.limit); + const limit = pageLimit(parsed.limit, opts.pagination); + if (limit !== undefined) q = q.limit(limit); if (parsed.offset !== undefined) q = q.offset(parsed.offset); } else { // No allowlists configured. Only limit/offset. - const limit = parsePageBound(qsParsed, "limit"); + const limit = pageLimit(parsePageBound(qsParsed, "limit"), opts.pagination); const offset = parsePageBound(qsParsed, "offset"); if (limit !== undefined) q = q.limit(limit); if (offset !== undefined) q = q.offset(offset); diff --git a/server/typescript/packages/runtime-ts/src/hono/mount-read-only.ts b/server/typescript/packages/runtime-ts/src/hono/mount-read-only.ts index 733a80caac..293f62d66d 100644 --- a/server/typescript/packages/runtime-ts/src/hono/mount-read-only.ts +++ b/server/typescript/packages/runtime-ts/src/hono/mount-read-only.ts @@ -6,7 +6,7 @@ import { quoteIdent, type SqlDialect } from "../sql-dialect.js"; import type { Hono } from "hono"; import { sql, eq, and, count } from "drizzle-orm"; import qs from "qs"; -import { parseFilterParams, parsePageBound, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError } from "../drizzle-fastify/filter-parser.js"; +import { parseFilterParams, parsePageBound, pageLimit, parseReportPage, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, FilterParseError, type PageSize } from "../drizzle-fastify/filter-parser.js"; import { ReportExportError, assertExportWithinCap, exportContentDisposition, exportWithCountError, isExportCountFlag, parseExportPage, readExportFormat, renderExport, EXPORT_CONTENT_TYPES, type ReportExportSpec, @@ -60,6 +60,13 @@ export interface MountReadOnlyOptions extends WireOptions { * refused the same way. Default 10000, the same in every port. */ readonly reportMaxRows?: number; + /** + * The list page size of a projection (`api.pagination` in `.metaobjects/config.json`; ignored + * for a report, which has `reportMaxRows`, and for a raw view, which pages to at most 1000). + * A list with no `limit` returns `defaultLimit` rows; a `limit` above `maxLimit` answers + * `400 { "error": "limit_too_large", "maxLimit": N }`. + */ + readonly pagination?: PageSize; /** Report export, as on the Fastify mount: `?format=csv|json|xml`, capped by `export.maxRows`. */ readonly export?: ReportExportSpec; } @@ -199,7 +206,7 @@ export function mountReadOnlyCrudRoutes(opts: MountReadOnlyOptions): void { if (result.orderBy) q = q.orderBy(...result.orderBy); // A report fetches one row past the cap when no limit was sent, to tell exactly the // cap apart from more than the cap before any body is written. - const limit = reportPage?.limit ?? result.limit; + const limit = reportPage?.limit ?? pageLimit(result.limit, opts.pagination); const offset = reportPage?.offset ?? result.offset; if (limit !== undefined) q = q.limit(limit); if (offset !== undefined) q = q.offset(offset); diff --git a/server/typescript/packages/runtime-ts/test/list-page-size.test.ts b/server/typescript/packages/runtime-ts/test/list-page-size.test.ts new file mode 100644 index 0000000000..26bb2820ef --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/list-page-size.test.ts @@ -0,0 +1,140 @@ +/** + * `api.pagination` at the route: a list with no `?limit=` answers the project's default page + * size, an explicit `?limit=` above the cap answers 400 `limit_too_large` naming the cap, and a + * mount given no page size behaves as it always did (the whole table, any `?limit=`). + */ +import { describe, test, expect, beforeAll, afterAll } from "bun:test"; +import Fastify, { type FastifyInstance } from "fastify"; +import { Hono } from "hono"; +import { createClient, type Client } from "@libsql/client"; +import { drizzle } from "drizzle-orm/libsql"; +import { sqliteTable, integer, text } from "drizzle-orm/sqlite-core"; +import { z } from "zod"; +import { + mountCrudRoutes as mountDrizzleFastifyCrud, + pageLimit, + ERROR_CODE_LIMIT_TOO_LARGE, +} from "../src/drizzle-fastify/index.js"; +import { FilterParseError } from "../src/drizzle-fastify/filter-parser.js"; +import { mountCrudRoutes as mountHonoCrud } from "../src/hono/index.js"; + +const PAGE = { defaultLimit: 2, maxLimit: 4 } as const; +const InsertSchema = z.object({ id: z.number().optional(), title: z.string() }); +const UpdateSchema = InsertSchema.partial(); + +function itemsTable() { + return sqliteTable("items", { id: integer("id").primaryKey(), title: text("title").notNull() }); +} + +async function seeded(): Promise { + const client = createClient({ url: ":memory:" }); + await client.execute(`CREATE TABLE items (id INTEGER PRIMARY KEY, title TEXT NOT NULL)`); + for (let i = 1; i <= 5; i++) await client.execute(`INSERT INTO items (id, title) VALUES (${i}, 'item ${i}')`); + return client; +} + +describe("pageLimit", () => { + test("no page size: the request's limit, or none", () => { + expect(pageLimit(undefined, undefined)).toBeUndefined(); + expect(pageLimit(100000, undefined)).toBe(100000); + }); + test("no limit: the default page size", () => { + expect(pageLimit(undefined, PAGE)).toBe(2); + }); + test("a limit at or under the cap is kept", () => { + expect(pageLimit(4, PAGE)).toBe(4); + expect(pageLimit(1, PAGE)).toBe(1); + }); + test("a limit over the cap throws limit_too_large carrying the cap", () => { + let caught: unknown; + try { pageLimit(5, PAGE); } catch (e) { caught = e; } + expect(caught).toBeInstanceOf(FilterParseError); + expect((caught as FilterParseError).code).toBe(ERROR_CODE_LIMIT_TOO_LARGE); + expect((caught as FilterParseError).details).toEqual({ maxLimit: 4 }); + }); +}); + +type Get = (url: string) => Promise<{ status: number; body: unknown }>; + +function listBehaviour(name: string, setup: (pagination: typeof PAGE | undefined) => Promise<{ get: Get; close: () => Promise }>) { + describe(`${name} — with api.pagination`, () => { + let h: { get: Get; close: () => Promise }; + beforeAll(async () => { h = await setup(PAGE); }); + afterAll(async () => { await h.close(); }); + + test("no ?limit= answers the default page size", async () => { + const r = await h.get("/items"); + expect(r.status).toBe(200); + expect((r.body as unknown[]).length).toBe(2); + }); + test("?limit= at the cap is served", async () => { + const r = await h.get("/items?limit=4"); + expect((r.body as unknown[]).length).toBe(4); + }); + test("?limit= over the cap is 400 limit_too_large naming the cap", async () => { + const r = await h.get("/items?limit=5"); + expect(r.status).toBe(400); + expect(r.body).toEqual({ error: "limit_too_large", maxLimit: 4 }); + }); + test("offset pages past the default", async () => { + const r = await h.get("/items?offset=4"); + expect((r.body as Array<{ id: number }>).map((x) => x.id)).toEqual([5]); + }); + test("withCount reports the full total with the default page", async () => { + const r = await h.get("/items?withCount=1"); + const body = r.body as { rows: unknown[]; total: number }; + expect(body.rows.length).toBe(2); + expect(body.total).toBe(5); + }); + }); + + describe(`${name} — without api.pagination`, () => { + let h: { get: Get; close: () => Promise }; + beforeAll(async () => { h = await setup(undefined); }); + afterAll(async () => { await h.close(); }); + + test("no ?limit= answers every row, as before", async () => { + expect(((await h.get("/items")).body as unknown[]).length).toBe(5); + }); + test("a large ?limit= is accepted, as before", async () => { + const r = await h.get("/items?limit=100000"); + expect(r.status).toBe(200); + expect((r.body as unknown[]).length).toBe(5); + }); + }); +} + +listBehaviour("drizzle-fastify", async (pagination) => { + const client = await seeded(); + const app: FastifyInstance = Fastify(); + mountDrizzleFastifyCrud({ + fastify: app, path: "/items", db: drizzle(client), table: itemsTable(), + insertSchema: InsertSchema, updateSchema: UpdateSchema, + ...(pagination !== undefined ? { pagination } : {}), + }); + await app.ready(); + return { + get: async (url) => { + const r = await app.inject({ method: "GET", url }); + return { status: r.statusCode, body: JSON.parse(r.body) }; + }, + close: async () => { await app.close(); client.close(); }, + }; +}); + +listBehaviour("hono", async (pagination) => { + const client = await seeded(); + const app = new Hono(); + mountHonoCrud({ + app, path: "/items", db: drizzle(client), table: itemsTable(), + insertSchema: InsertSchema, updateSchema: UpdateSchema, + ...(pagination !== undefined ? { pagination } : {}), + }); + return { + get: async (url) => { + const r = await app.request(url); + return { status: r.status, body: await r.json() }; + }, + close: async () => { client.close(); }, + }; +}); diff --git a/server/typescript/packages/sdk/src/collection.ts b/server/typescript/packages/sdk/src/collection.ts index 75de1013f5..2bab06a4e3 100644 --- a/server/typescript/packages/sdk/src/collection.ts +++ b/server/typescript/packages/sdk/src/collection.ts @@ -23,9 +23,10 @@ import { type WireFormat, DEFAULT_WIRE_FORMAT, type EnumDefaults, + type ApiPagination, DEFAULT_ENUM_DEFAULTS, } from "@metaobjectsdev/metadata"; -import { CONFIG_FILE, enumDefaultsFromConfig, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; +import { apiPaginationFromConfig, CONFIG_FILE, enumDefaultsFromConfig, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; import { discoverCollectionRoot, exists, isDir } from "./discovery.js"; import { compileScope, matchesScope, type Scope } from "./scope.js"; import { DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, isMetadataFile } from "./metadata-files.js"; @@ -90,6 +91,9 @@ export interface Collection { /** The `enums` block of the project config (string-stored enum defaults), none when absent. * Optional so a hand-built Collection from before the block existed still type-checks. */ readonly enumDefaults?: EnumDefaults | undefined; + /** The `api.pagination` block of the project config; undefined when the project declares none. + * Optional so a hand-built Collection from before the block existed still type-checks. */ + readonly apiPagination?: ApiPagination | undefined; /** `migrate.conventions` of the project config — a schema-conventions preset name — when set. */ readonly schemaConventions: string | undefined; /** `migrate.checks.skipRedundant` of the project config: leave out the validator CHECKs a @@ -291,6 +295,7 @@ export async function resolveCollection( let physicalNaming: PhysicalNaming = DEFAULT_PHYSICAL_NAMING; let wireFormat: WireFormat = DEFAULT_WIRE_FORMAT; let enumDefaults: EnumDefaults = DEFAULT_ENUM_DEFAULTS; + let apiPagination: ApiPagination | undefined; let schemaConventions: string | undefined; let skipRedundantChecks = false; @@ -311,6 +316,7 @@ export async function resolveCollection( physicalNaming = physicalNamingFromConfig(cfg); wireFormat = wireFormatFromConfig(cfg); enumDefaults = enumDefaultsFromConfig(cfg); + apiPagination = apiPaginationFromConfig(cfg); schemaConventions = cfg.migrate?.conventions; skipRedundantChecks = cfg.migrate?.checks?.skipRedundant === true; // An unknown token is a hard config error naming the valid ones, while @@ -428,6 +434,7 @@ export async function resolveCollection( physicalNaming, wireFormat, enumDefaults, + apiPagination, schemaConventions, skipRedundantChecks, fileIds: new Map(dependencies.map((d) => [d.artifactPath, d.sourceId])), diff --git a/server/typescript/packages/sdk/src/config.ts b/server/typescript/packages/sdk/src/config.ts index 0139c831b7..ccab17789f 100644 --- a/server/typescript/packages/sdk/src/config.ts +++ b/server/typescript/packages/sdk/src/config.ts @@ -23,6 +23,10 @@ import { type WireFormat, type WireNullMembers, type PhysicalNaming, + API_KEY_PAGINATION, + PAGINATION_KEY_DEFAULT_LIMIT, + PAGINATION_KEY_MAX_LIMIT, + type ApiPagination, } from "@metaobjectsdev/metadata"; const DialectEnum = z.enum(["sqlite", "postgres", "d1"]); @@ -163,6 +167,13 @@ const ScopeSchema = z // "scope") is silently stripped by zod and the collection resolves as // "everything in scope" — the exact silent fail-open .strict() on the // nested arms exists to prevent, one level up. +/** One `api.pagination` size: a positive integer, every failure (missing, wrong type, a + * fraction, zero) named by its key rather than Zod's generic "expected number". */ +function pageSizeNumber(key: string) { + const error = `"api.pagination.${key}" must be a positive integer`; + return z.number({ error }).int({ error }).positive({ error }); +} + export const ConfigSchema = z.object({ schema_version: z.literal(1), pending_in_git: z.boolean().default(true), @@ -284,6 +295,35 @@ export const ConfigSchema = z.object({ : iss.code === "invalid_type" ? `"wire" must be an object` : undefined, }) .optional(), + /** + * The page size of the generated list routes. `pagination.defaultLimit` is how many rows a list + * with no `limit` returns; an explicit `limit` above `pagination.maxLimit` is refused with + * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent means no default and no ceiling, + * as before. `meta init` writes 50 and 1000. + */ + api: z + .strictObject({ + pagination: z + .strictObject({ + defaultLimit: pageSizeNumber(PAGINATION_KEY_DEFAULT_LIMIT), + maxLimit: pageSizeNumber(PAGINATION_KEY_MAX_LIMIT), + }, { + error: (iss) => iss.code === "unrecognized_keys" + ? `"api.pagination" has an unknown key "${iss.keys[0]}" (expected "${PAGINATION_KEY_DEFAULT_LIMIT}" and "${PAGINATION_KEY_MAX_LIMIT}")` + : iss.code === "invalid_type" && iss.input === undefined + ? `"api.pagination" needs both "${PAGINATION_KEY_DEFAULT_LIMIT}" and "${PAGINATION_KEY_MAX_LIMIT}"` + : iss.code === "invalid_type" ? `"api.pagination" must be an object` : undefined, + }) + .refine((p) => p.defaultLimit <= p.maxLimit, { + message: `"api.pagination.defaultLimit" must not exceed "api.pagination.maxLimit"`, + }) + .optional(), + }, { + error: (iss) => iss.code === "unrecognized_keys" + ? `"api" has an unknown key "${iss.keys[0]}" (expected "${API_KEY_PAGINATION}")` + : iss.code === "invalid_type" ? `"api" must be an object` : undefined, + }) + .optional(), /** * How a string-stored `field.enum` is stored. `maxLength` is the column length (VARCHAR(n)) * of an enum that declares no `@maxLength` of its own; every member must fit. Absent means @@ -328,6 +368,12 @@ export function physicalNamingFromConfig(cfg: Pick): PhysicalN }; } +/** The list page size a config declares (`api.pagination`); undefined when absent. */ +export function apiPaginationFromConfig(cfg: Pick): ApiPagination | undefined { + const p = cfg.api?.pagination; + return p === undefined ? undefined : { defaultLimit: p.defaultLimit, maxLimit: p.maxLimit }; +} + /** The enum storage defaults a config declares (`enums`); none when absent. */ export function enumDefaultsFromConfig(cfg: Pick): EnumDefaults { return cfg.enums?.maxLength === undefined ? DEFAULT_ENUM_DEFAULTS : { maxLength: cfg.enums.maxLength }; diff --git a/server/typescript/packages/sdk/src/index.ts b/server/typescript/packages/sdk/src/index.ts index fdd0ea82cd..54c173ea8c 100644 --- a/server/typescript/packages/sdk/src/index.ts +++ b/server/typescript/packages/sdk/src/index.ts @@ -16,7 +16,7 @@ // shows `resolveCollection`, the documented single authority for locating a project. // Config -export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig, enumDefaultsFromConfig } from "./config.js"; +export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig, enumDefaultsFromConfig, apiPaginationFromConfig } from "./config.js"; export type { Config } from "./config.js"; // Metadata dependencies (FR-023) — the `dependencies` config key's schema, diff --git a/server/typescript/packages/sdk/src/memory.ts b/server/typescript/packages/sdk/src/memory.ts index 10c8f6c91b..d9275c4b32 100644 --- a/server/typescript/packages/sdk/src/memory.ts +++ b/server/typescript/packages/sdk/src/memory.ts @@ -17,6 +17,7 @@ import { type PhysicalNaming, type WireFormat, type EnumDefaults, + type ApiPagination, } from "@metaobjectsdev/metadata"; import { FileSource } from "@metaobjectsdev/metadata/core"; // FR-043 — a node-only subpath (it reaches the filesystem), which is why it is imported @@ -118,6 +119,8 @@ export interface LoadMemoryOptions { wireFormat?: WireFormat; /** The project's enum storage defaults; defaults to the collection's (`resolveCollection(...).enumDefaults`) when `files` is not passed. */ enumDefaults?: EnumDefaults; + /** The project's list page size (`api.pagination`); defaults to the collection's when `files` is not passed. */ + apiPagination?: ApiPagination | undefined; } /** Default provider bundle threaded by {@link loadMemory} when no options @@ -173,7 +176,7 @@ export async function loadMemory( repoRoot: string, options?: LoadMemoryOptions, ): Promise { - const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults } = + const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination } = await loadCollectionRaw(repoRoot, options, options?.strict === true); if (result.errors.length > 0) { @@ -195,6 +198,7 @@ export async function loadMemory( if (physicalNaming !== undefined) result.root.physicalNaming = physicalNaming; if (wireFormat !== undefined) result.root.wireFormat = wireFormat; if (enumDefaults !== undefined) result.root.enumDefaults = enumDefaults; + if (apiPagination !== undefined) result.root.apiPagination = apiPagination; return result.root; } @@ -270,6 +274,7 @@ async function loadCollectionRaw( physicalNaming: PhysicalNaming | undefined; wireFormat: WireFormat | undefined; enumDefaults: EnumDefaults | undefined; + apiPagination: ApiPagination | undefined; }> { const extra = options?.providers ?? []; let providers: readonly MetaDataTypeProvider[]; @@ -299,6 +304,7 @@ async function loadCollectionRaw( let physicalNaming: PhysicalNaming | undefined = options?.physicalNaming; let wireFormat: WireFormat | undefined = options?.wireFormat; let enumDefaults: EnumDefaults | undefined = options?.enumDefaults; + let apiPagination: ApiPagination | undefined = options?.apiPagination; if (options?.files !== undefined) { paths = [...options.files]; fileIds = options.fileIds; @@ -320,6 +326,7 @@ async function loadCollectionRaw( physicalNaming ??= collection.physicalNaming; wireFormat ??= collection.wireFormat; enumDefaults ??= collection.enumDefaults; + apiPagination ??= collection.apiPagination; } const loader = new MetaDataLoader({ @@ -345,7 +352,7 @@ async function loadCollectionRaw( return id === undefined ? new FileSource(p) : new FileSource(p, { id }); }), ]); - return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults }; + return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination }; } /** Every source file that contributed to a node, across the envelope variants that diff --git a/server/typescript/packages/sdk/test/config.test.ts b/server/typescript/packages/sdk/test/config.test.ts index 0d964d4531..fd9e7e9dda 100644 --- a/server/typescript/packages/sdk/test/config.test.ts +++ b/server/typescript/packages/sdk/test/config.test.ts @@ -2,7 +2,7 @@ import { describe, test, expect, beforeEach, afterEach } from "bun:test"; import { mkdtempSync, rmSync, mkdirSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { ConfigSchema, DEFAULT_CONFIG, loadConfig, saveConfig, setConfigWarningSink, type Config } from "../src/config.js"; +import { ConfigSchema, DEFAULT_CONFIG, apiPaginationFromConfig, loadConfig, saveConfig, setConfigWarningSink, type Config } from "../src/config.js"; let metaRoot: string; beforeEach(() => { @@ -300,3 +300,39 @@ describe("csharp.providers (read by the C# port only)", () => { expect(bad([{ assembly: "a.dll", type: "A.B", extra: 1 }])).toBe(false); }); }); + +describe("api.pagination (the generated list routes' page size)", () => { + const parse = (api: unknown) => ConfigSchema.safeParse({ schema_version: 1, api }); + const messages = (api: unknown) => { + const r = parse(api); + return r.success ? [] : r.error.issues.map((i) => i.message); + }; + + test("absent: no page size, so lists behave as they always have", () => { + expect(DEFAULT_CONFIG.api).toBeUndefined(); + expect(apiPaginationFromConfig(DEFAULT_CONFIG)).toBeUndefined(); + expect(apiPaginationFromConfig(ConfigSchema.parse({ schema_version: 1, api: {} }))).toBeUndefined(); + }); + + test("present: read as given", () => { + const cfg = ConfigSchema.parse({ schema_version: 1, api: { pagination: { defaultLimit: 50, maxLimit: 1000 } } }); + expect(apiPaginationFromConfig(cfg)).toEqual({ defaultLimit: 50, maxLimit: 1000 }); + }); + + test("a default above the cap is refused in words", () => { + expect(messages({ pagination: { defaultLimit: 100, maxLimit: 10 } })) + .toContain(`"api.pagination.defaultLimit" must not exceed "api.pagination.maxLimit"`); + }); + + test("a zero, negative or fractional size is refused", () => { + expect(messages({ pagination: { defaultLimit: 0, maxLimit: 10 } }).join(" ")).toContain("defaultLimit"); + expect(messages({ pagination: { defaultLimit: 5, maxLimit: -1 } }).join(" ")).toContain("maxLimit"); + expect(messages({ pagination: { defaultLimit: 2.5, maxLimit: 10 } }).join(" ")).toContain("defaultLimit"); + }); + + test("a half-declared block or an unknown key is refused, naming the expected keys", () => { + expect(messages({ pagination: { defaultLimit: 5 } }).join(" ")).toContain("maxLimit"); + expect(messages({ pagination: { defaultLimit: 5, maxLimit: 10, max: 3 } }).join(" ")).toContain('unknown key "max"'); + expect(messages({ paging: {} }).join(" ")).toContain('"api" has an unknown key "paging"'); + }); +}); diff --git a/server/typescript/packages/test-generators/src/generators/routes.ts b/server/typescript/packages/test-generators/src/generators/routes.ts index 08d409771c..8ffe8c2da9 100644 --- a/server/typescript/packages/test-generators/src/generators/routes.ts +++ b/server/typescript/packages/test-generators/src/generators/routes.ts @@ -69,6 +69,7 @@ import { isReport, itemRouteField, decimalColumnsLine, + paginationLine, writeDeniedColumnsLine, servesReadApi, formatTs, @@ -167,7 +168,8 @@ function renderRoutes( (report ? `\n${indent}resource: "report",` : "") + (report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") + (report && ctx.includeReportExport ? `\n${indent}export: ${entityName}Export,` : "") + - decimalColumnsLine(entity, indent, ctx.loadedRoot); + decimalColumnsLine(entity, indent, ctx.loadedRoot) + + (report ? "" : paginationLine(entity, indent, ctx.loadedRoot)); const FastifyInstanceSym = imp("t:FastifyInstance@fastify"); const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`); // A projection mount is read-only by construction, so `expose` cannot narrow it — @@ -310,7 +312,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -335,7 +337,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -602,7 +604,7 @@ function renderTphRoutes( updateSchema: ${baseUpdateSym}, filterAllowlist: ${baseFilterSym}, sortAllowlist: ${baseSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${polymorphicExposeLine} + dialect: ${dialectLit},${decimalColumnsLine(hierarchy, " ", ctx.loadedRoot)}${paginationLine(base, " ", ctx.loadedRoot)}${polymorphicExposeLine} });`; // FR-018 x FR-017 — M:N traversal inside a TPH hierarchy. This file never consulted @@ -649,7 +651,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the From 84d88873438d43e3c67c14773413b33527669694 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 14:58:09 -0500 Subject: [PATCH 05/31] feat(api): api.pagination in every port, and api.unknownFields (reject by default) api.pagination (default page size and hard cap on generated list routes) now reaches the C#, Java, Kotlin and Python generators as well as TypeScript: no limit answers the default page, a limit over the cap is 400 limit_too_large. No setting keeps today's behaviour. verify advises when routes are generated without it (TypeScript, Python, Java/Kotlin, and C# under verify --codegen). api.unknownFields (reject | drop, default reject): a generated create, PUT or PATCH body carrying a key the entity does not declare answers 400 {"error":"unknown_fields","fields":[...]} in every port, before validation, and writes nothing. drop keeps the old silent behaviour. Declared server-owned keys (id, readOnly, @autoSet, writeOnce on update, the TPH discriminator, a sibling subtype's column) are still dropped, not refused, so a client echoing back a row keeps working. Java and Kotlin check the raw body, so the answer does not depend on the host's Jackson mapper; a test proves it under both FAIL_ON_UNKNOWN_PROPERTIES values. The pagination sub-corpus becomes api-settings/ and runs in every port's generated lane; the core and tph corpora gain reject scenarios. meta init writes both settings into a fresh config. --- CHANGELOG.md | 51 +++-- docs/CONFORMANCE.md | 34 ++-- docs/features/api-contract.md | 42 +++- docs/features/cli.md | 2 +- docs/features/generated-mutations.md | 6 +- .../unknown-fields-reject-by-default.md | 60 ++++++ docs/ports/csharp.md | 12 ++ docs/ports/java.md | 25 +++ docs/ports/kotlin.md | 17 ++ .../codegen/generators/routes.ts | 7 +- .../codegen/runtime/drizzle-fastify/index.ts | 15 +- .../codegen/runtime/wire-format.ts | 14 ++ .../showcase/codegen/generators/routes.ts | 7 +- .../codegen/runtime/drizzle-fastify/index.ts | 15 +- .../showcase/codegen/runtime/wire-format.ts | 14 ++ fixtures/api-contract-conformance/README.md | 11 +- .../.metaobjects/config.json | 2 +- .../api-settings/README.md | 62 ++++++ .../{pagination => api-settings}/meta.json | 0 .../scenarios/default-page-size.yaml | 0 .../scenarios/limit-at-max.yaml | 0 .../scenarios/limit-over-max.yaml | 0 .../scenarios/offset-pages-past-default.yaml | 0 .../scenarios/unknown-fields-dropped.yaml | 49 +++++ .../scenarios/with-count-default-page.yaml | 0 .../{pagination => api-settings}/seed.json | 0 .../pagination/README.md | 51 ----- .../scenarios/input-server-owned-dropped.yaml | 20 +- .../input-unknown-fields-rejected.yaml | 92 +++++++++ .../tph-input-server-owned-dropped.yaml | 14 +- .../tph-unknown-fields-rejected.yaml | 59 ++++++ .../expected/Author.routes.hono.ts | 1 + .../VerifySubverbTests.cs | 49 +++++ server/csharp/MetaObjects.Cli/Program.cs | 3 + .../csharp/MetaObjects.Cli/VerifyCommand.cs | 24 ++- .../ApiSettingsTests.cs | 98 +++++++++ .../Issue203AutoSetStampingTests.cs | 6 +- .../Generators/RoutesGenerator.cs | 70 +++++-- .../MetaObjects.Codegen/HelperRuntime.cs | 1 + .../MetaObjects.Codegen.csproj | 1 + .../Runtime/UnknownFieldCheck.cs | 31 +++ .../ApiContractApiSettingsConformanceTest.cs | 93 +++++++++ .../Api/ApiContractCorpusPaths.cs | 6 + .../Api/ApiSettingsFixture.cs | 45 +++++ .../Api/ApiSettingsGeneratedServerFactory.cs | 185 +++++++++++++++++ .../Api/AuthorApiServer.cs | 13 ++ .../Api/ReferenceValidation.cs | 12 ++ .../Api/TphReferenceServer.cs | 15 ++ .../MetaObjects/Config/MetadataLocation.cs | 10 +- .../MetaObjects/Config/NeutralConfig.cs | 71 +++++++ .../MetaObjects/Loader/MetaDataLoader.cs | 1 + server/csharp/MetaObjects/Meta/ApiSettings.cs | 35 ++++ server/csharp/MetaObjects/Meta/MetaRoot.cs | 4 + server/java/codegen-kotlin/pom.xml | 1 + .../generator/kotlin/GeneratorRegistry.kt | 3 +- .../kotlin/KotlinSpringControllerGenerator.kt | 86 ++++++-- .../kotlin/KotlinUnknownBodyKeysEmitter.kt | 68 +++++++ .../acme/blog/AuthorController.kt | 7 + .../acme/blog/MetaUnknownBodyKeys.kt | 39 ++++ .../acme/catalog/MetaUnknownBodyKeys.kt | 39 ++++ .../acme/catalog/ProductController.kt | 7 + .../acme/catalog/MetaUnknownBodyKeys.kt | 39 ++++ .../acme/catalog/ProductController.kt | 7 + .../acme/auth/AuthController.kt | 39 +++- .../acme/auth/AuthLineController.kt | 7 + .../acme/auth/MetaUnknownBodyKeys.kt | 39 ++++ .../acme/report/MetaUnknownBodyKeys.kt | 39 ++++ server/java/codegen-spring/pom.xml | 1 + .../generator/GeneratorRegistry.java | 2 +- .../spring/SpringControllerGenerator.java | 92 +++++++-- .../spring/runtime/UnknownBodyKeys.java | 59 ++++++ .../GeneratedTphCrossFieldCompileRunTest.java | 5 +- .../spring/SpringControllerGeneratorTest.java | 2 +- .../spring/SpringSecretFieldsCodegenTest.java | 4 +- .../SpringUnknownFieldsCompileRunTest.java | 186 ++++++++++++++++++ .../integration/kotlin/api/AuthorApiServer.kt | 26 +++ ...ingsGeneratedApiContractConformanceTest.kt | 74 +++++++ .../GeneratedItemControllerHarness.kt | 152 ++++++++++++++ .../generated/PatchTristateEdgeCasesTest.kt | 6 +- ...UnknownFieldsHostMapperIndependenceTest.kt | 183 +++++++++++++++++ ...gsGeneratedApiContractConformanceTest.java | 83 ++++++++ .../integration/api/AuthorApiServer.java | 18 ++ .../GeneratedItemControllerHarness.java | 177 +++++++++++++++++ .../InMemoryItemRepositorySource.java | 107 ++++++++++ .../mojo/AbstractMetaDataMojo.java | 4 + .../metaobjects/mojo/MetaDataVerifyMojo.java | 23 +++ ...taDataVerifyUnboundedListAdvisoryTest.java | 92 +++++++++ .../main/java/com/metaobjects/MetaRoot.java | 18 ++ .../com/metaobjects/config/ApiSettings.java | 87 ++++++++ .../com/metaobjects/config/NeutralConfig.java | 69 +++++++ .../metaobjects/config/SourceResolver.java | 5 + .../metaobjects/loader/MetaDataLoader.java | 5 + .../metaobjects/config/ApiSettingsTest.java | 75 +++++++ server/python/src/metaobjects/api_settings.py | 55 ++++++ server/python/src/metaobjects/cli.py | 29 +++ .../codegen/generators/router_generator.py | 93 +++++++-- .../codegen/runtime/constraint_errors.py | 10 + .../src/metaobjects/config/dependencies.py | 4 + .../src/metaobjects/config/neutral_config.py | 80 ++++++++ .../src/metaobjects/config/source_resolver.py | 3 + .../python/src/metaobjects/meta/meta_root.py | 3 + server/python/src/metaobjects/wire.py | 2 + .../test_cli_verify_pagination_advisory.py | 69 +++++++ .../tests/config/test_neutral_config_api.py | 49 +++++ .../tests/integration/api_contract_server.py | 19 ++ .../integration/generated_api_settings_app.py | 74 +++++++ .../test_api_contract_api_settings.py | 70 +++++++ .../packages/cli/src/commands/init.ts | 10 +- .../cli/src/lib/collection-load-options.ts | 3 +- .../typescript/packages/cli/test/init.test.ts | 4 +- .../packages/codegen-ts/src/api-surface.ts | 21 ++ .../packages/codegen-ts/src/index.ts | 2 +- .../codegen-ts/src/reference/routes-hono.ts | 3 +- .../codegen-ts/src/reference/routes.ts | 7 +- .../src/templates/routes-file-hono.ts | 4 +- .../codegen-ts/src/templates/routes-file.ts | 8 +- .../__snapshots__/package/Tag.routes.ts | 1 + .../package/shop/commerce/Order.routes.ts | 1 + .../package/shop/commerce/Product.routes.ts | 1 + .../package/shop/users/Customer.routes.ts | 1 + .../__snapshots__/postgres/Exercise.routes.ts | 1 + .../__snapshots__/postgres/Program.routes.ts | 9 + .../__snapshots__/postgres/Purchase.routes.ts | 7 + .../postgres/Subscriber.routes.ts | 8 + .../__snapshots__/postgres/Tag.routes.ts | 1 + .../__snapshots__/postgres/Video.routes.ts | 1 + .../__snapshots__/postgres/Week.routes.ts | 1 + .../__snapshots__/postgres/Workout.routes.ts | 1 + .../postgres/WorkoutEvent.routes.ts | 7 + .../__snapshots__/sqlite/Exercise.routes.ts | 1 + .../__snapshots__/sqlite/Program.routes.ts | 9 + .../__snapshots__/sqlite/Purchase.routes.ts | 7 + .../__snapshots__/sqlite/Subscriber.routes.ts | 8 + .../golden/__snapshots__/sqlite/Tag.routes.ts | 1 + .../__snapshots__/sqlite/Video.routes.ts | 1 + .../__snapshots__/sqlite/Week.routes.ts | 1 + .../__snapshots__/sqlite/Workout.routes.ts | 1 + .../sqlite/WorkoutEvent.routes.ts | 7 + .../test/routes-known-fields.test.ts | 109 ++++++++++ ...contract-api-settings-generated-server.ts} | 23 +-- .../src/api-contract-server.ts | 13 +- .../src/api-contract-tph-server.ts | 14 +- .../integration-tests/src/load-metadata.ts | 4 +- .../packages/integration-tests/src/paths.ts | 6 +- .../test/api-contract-api-settings.test.ts | 56 ++++++ .../test/api-contract-pagination.test.ts | 55 ------ .../packages/metadata/src/api-pagination.ts | 28 +++ .../typescript/packages/metadata/src/index.ts | 4 +- .../packages/metadata/src/shared/meta-root.ts | 9 +- .../runtime-ts/src/drizzle-fastify/index.ts | 15 +- .../packages/runtime-ts/src/hono/index.ts | 15 +- .../packages/runtime-ts/src/wire-format.ts | 14 ++ .../runtime-ts/test/unknown-fields.test.ts | 138 +++++++++++++ .../typescript/packages/sdk/src/collection.ts | 8 +- server/typescript/packages/sdk/src/config.ts | 20 +- server/typescript/packages/sdk/src/index.ts | 2 +- server/typescript/packages/sdk/src/memory.ts | 11 +- .../packages/sdk/test/config.test.ts | 27 ++- .../test-generators/src/generators/routes.ts | 7 +- 159 files changed, 4404 insertions(+), 302 deletions(-) create mode 100644 docs/features/migrations/unknown-fields-reject-by-default.md rename fixtures/api-contract-conformance/{pagination => api-settings}/.metaobjects/config.json (61%) create mode 100644 fixtures/api-contract-conformance/api-settings/README.md rename fixtures/api-contract-conformance/{pagination => api-settings}/meta.json (100%) rename fixtures/api-contract-conformance/{pagination => api-settings}/scenarios/default-page-size.yaml (100%) rename fixtures/api-contract-conformance/{pagination => api-settings}/scenarios/limit-at-max.yaml (100%) rename fixtures/api-contract-conformance/{pagination => api-settings}/scenarios/limit-over-max.yaml (100%) rename fixtures/api-contract-conformance/{pagination => api-settings}/scenarios/offset-pages-past-default.yaml (100%) create mode 100644 fixtures/api-contract-conformance/api-settings/scenarios/unknown-fields-dropped.yaml rename fixtures/api-contract-conformance/{pagination => api-settings}/scenarios/with-count-default-page.yaml (100%) rename fixtures/api-contract-conformance/{pagination => api-settings}/seed.json (100%) delete mode 100644 fixtures/api-contract-conformance/pagination/README.md create mode 100644 fixtures/api-contract-conformance/scenarios/input-unknown-fields-rejected.yaml create mode 100644 fixtures/api-contract-conformance/tph/scenarios/tph-unknown-fields-rejected.yaml create mode 100644 server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs create mode 100644 server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsFixture.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs create mode 100644 server/csharp/MetaObjects/Meta/ApiSettings.cs create mode 100644 server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinUnknownBodyKeysEmitter.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaUnknownBodyKeys.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaUnknownBodyKeys.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaUnknownBodyKeys.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaUnknownBodyKeys.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaUnknownBodyKeys.kt create mode 100644 server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnknownBodyKeys.java create mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringUnknownFieldsCompileRunTest.java create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/ApiSettingsGeneratedApiContractConformanceTest.kt create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/generated/GeneratedItemControllerHarness.kt create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/UnknownFieldsHostMapperIndependenceTest.kt create mode 100644 server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiSettingsGeneratedApiContractConformanceTest.java create mode 100644 server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedItemControllerHarness.java create mode 100644 server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryItemRepositorySource.java create mode 100644 server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java create mode 100644 server/java/metadata/src/main/java/com/metaobjects/config/ApiSettings.java create mode 100644 server/java/metadata/src/test/java/com/metaobjects/config/ApiSettingsTest.java create mode 100644 server/python/src/metaobjects/api_settings.py create mode 100644 server/python/tests/codegen/test_cli_verify_pagination_advisory.py create mode 100644 server/python/tests/config/test_neutral_config_api.py create mode 100644 server/python/tests/integration/generated_api_settings_app.py create mode 100644 server/python/tests/integration/test_api_contract_api_settings.py create mode 100644 server/typescript/packages/codegen-ts/test/routes-known-fields.test.ts rename server/typescript/packages/integration-tests/src/{api-contract-pagination-generated-server.ts => api-contract-api-settings-generated-server.ts} (79%) create mode 100644 server/typescript/packages/integration-tests/test/api-contract-api-settings.test.ts delete mode 100644 server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts create mode 100644 server/typescript/packages/runtime-ts/test/unknown-fields.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 981aa90224..02da88a1fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,21 +38,22 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi did: a server-generated key, a `readOnly` or derived field, and in TPH the discriminator (the route's subtype is its only source) and every column the route's subtype does not declare. An assigned key is still the caller's on create. No port answers them with a 400. A key the - entity does not declare is also dropped, never rejected, in every port: Java DTOs and Kotlin - entity classes now carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer no - longer depends on the host's `ObjectMapper` (a plain one answered 400). The Python 1.2.1 fix - is carried forward to this line, and the Python TPH create now drops a body's discriminator - instead of answering 400 for a foreign one. + entity does not declare is a separate case, now a project setting (`api.unknownFields`, + under "Changed" below). Java DTOs and Kotlin entity classes now carry + `@JsonIgnoreProperties(ignoreUnknown = true)`, so binding no longer depends on the host's + `ObjectMapper` (a plain one answered 400). The Python 1.2.1 fix is carried forward to this + line, and the Python TPH create now drops a body's discriminator instead of answering 400 for + a foreign one. Gated by two new shared api-contract scenarios, `input-server-owned-dropped` (generated and reference lanes) and `tph/tph-input-server-owned-dropped` (both TPH lanes), which send a - client `id`, a foreign discriminator, a `writeOnce` change and a made-up `isAdmin`, and by a - `readOnly` `role` column in the `secret-fields` corpus. Scenarios gain an `absent: [keys]` - assertion. Adopters with ejected (owned) generators need the same change made by hand. + client `id`, a foreign discriminator and a `writeOnce` change, and by a `readOnly` `role` + column in the `secret-fields` corpus. Scenarios gain an `absent: [keys]` assertion. Adopters + with ejected (owned) generators need the same change made by hand. ### Added -- **A page size for generated list routes: `api.pagination` (TypeScript).** An external review +- **A page size for generated list routes: `api.pagination`, in every port.** An external review of 1.2.0 found that list endpoints had no hard cap: the TypeScript and C# routes returned the whole table when `limit` was omitted, and `?limit=100000` was accepted in every port. Set `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }` in @@ -62,12 +63,13 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi that block for a new project. **A project without the setting behaves exactly as before**, because a cap added on upgrade would silently truncate a client that reads a whole list; `meta verify` advises (rule `unbounded-list-pages`) when a routes generator is wired and no - page size is set. Reports keep their own row cap and opaque views their 1000. The TypeScript - Fastify and Hono routes read it (packaged and ejected: re-eject `routes` or re-sync an owned - copy to pick it up), through a new `pagination` option on the runtime mounts. It is gated by - the new `fixtures/api-contract-conformance/pagination/` sub-corpus. The C#, Java, Kotlin and - Python routes do not read it yet; carrying it there waits on the ruling on the freeze of the - non-TypeScript REST generators. A config key, not vocabulary: `metamodelVersion` does not move. + page size is set. Reports keep their own row cap and opaque views their 1000. Every port's + generated list routes read it: the TypeScript Fastify and Hono routes (through a new + `pagination` option on the runtime mounts), the C# minimal-API routes, the Java and Kotlin + Spring controllers and the Python FastAPI routers. An owned (ejected) routes copy picks it up + when it is re-ejected. It is gated in the generated lane of every port by the new + `fixtures/api-contract-conformance/api-settings/` sub-corpus. A config key, not vocabulary: + `metamodelVersion` does not move. - **Report measures over Money and other value-object members (FR-044 R12).** A dimension's or measure's `@of` may name `Entity.field.member`, a member of a value object stored with @@ -91,6 +93,25 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi ### Changed +- **A write with a key the entity does not declare is refused by default: `api.unknownFields`. + Existing projects get `reject` when they regenerate.** Until now every port dropped an + undeclared body key silently, so a client that misspelled a field (`emial`) got a 200 and lost + the value without a word. A generated create, PUT or PATCH route now answers + `400 {"error": "unknown_fields", "fields": ["emial"]}` (the undeclared top-level keys, sorted) + before validation, and writes nothing. To keep the old behaviour set + `"api": { "unknownFields": "drop" }` in `.metaobjects/config.json` and regenerate. A declared + key the client may not set (a generated `id`, a `readOnly` or derived field, an `@autoSet` + stamp, a `writeOnce` field on update, the TPH discriminator, a sibling subtype's column) is + still dropped, not refused, under either value, so a client that sends back a row it read keeps + working. `meta init` writes `"unknownFields": "reject"` for a new project. Every port checks the + raw body before binding it, so Java and Kotlin answer the same whatever the host's Jackson + `FAIL_ON_UNKNOWN_PROPERTIES` says. TypeScript passes the declared field list to the runtime + mounts as a new `knownFields` option; a mount given none keeps dropping. Pinned by + `input-unknown-fields-rejected` and `tph/tph-unknown-fields-rejected` (both lanes of every + port) and `api-settings/unknown-fields-dropped` (`drop`, generated lane). Upgrade notes: + [docs/features/migrations/unknown-fields-reject-by-default.md](docs/features/migrations/unknown-fields-reject-by-default.md). + A config key, not vocabulary: `metamodelVersion` does not move. + - **A validation 400 lists every issue in every port.** The same model answered a failed create or update differently by language: TypeScript listed each problem, while C#, Java, Kotlin and Python returned a bare `{"error": "validation"}` (found by an external review of 1.2.0). Every port now diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index b5cc6469e5..6abab5fc3d 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 62 (55 query + 7 migration) | all 62; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 55 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 132 (33 core + 12 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 5 pagination) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1; `pagination/` TypeScript only (see "Split coverage") | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 135 (34 core + 13 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 6 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | @@ -111,19 +111,6 @@ corpora, or between its *vocabulary* and its *behaviour* — never gets a row of own, and its boundary ends up discoverable only from the feature doc. The split ones are recorded here instead. -**`api.pagination`** (a page size on generated list routes, a `.metaobjects/config.json` key): -a list with no `limit` answers `defaultLimit` rows and a `limit` above `maxLimit` answers -`400 {"error": "limit_too_large", "maxLimit": N}`. - -- *Honoured and pinned:* TypeScript. The Fastify and Hono routes (packaged and ejected) pass it to - the runtime mounts; `fixtures/api-contract-conformance/pagination/` runs in the TypeScript - generated lane. -- *Not read:* C#, Java, Kotlin and Python. Their REST generators are under the 2026-09-22 freeze, - and carrying the setting there waits on that ruling. Their list routes keep the per-port - behaviour [api-contract.md](features/api-contract.md) records (TypeScript without the setting - and C# return every row when `limit` is omitted; Java, Kotlin and Python return 50; no port caps - an explicit `limit`), and the `pagination/` sub-corpus has no lane there yet. - **`identity.primary @sequenceStart`** (one semantic in every port): on a fresh database the identity or sequence begins at `@sequenceStart`; when the start changes on an existing database, the change sets `START WITH` only. It never `RESTART`s, and in 1.3 it does @@ -454,9 +441,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (132) +### `fixtures/api-contract-conformance/` (135) -All 132 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 135 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -466,9 +453,12 @@ status codes. The corpus also covers the 9 cross-port filter operators (`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `like`, `isNull`) plus the implicit-AND combinator and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the -URL grammar `?filter[][]=` (FR-009). On top of the 31 core -scenarios the corpus carries ten sub-corpora — `tph/` (10, single-table -inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination +URL grammar `?filter[][]=` (FR-009). On top of the 34 core +scenarios (among them the write-input rules: server-owned keys dropped, an undeclared key +refused under the default `api.unknownFields: reject`, and every validation issue listed) +the corpus carries eleven sub-corpora — `tph/` (13, single-table +inheritance; a subtype decimal filters and refuses extra digits, and the input rules hold over +the whole hierarchy), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), `jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), `write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, and a credential is never returned), `enum-default/` (2, a create that omits a `field.enum` with `@default` stores the default, and a supplied value is kept), `projection/` (12, a @@ -477,7 +467,9 @@ identity has no item route; a projection keyed on a field not named `id` is addr it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one canonical string, read exactly, filtered exactly and refused past its declared digits), `duration/` (2, a `field.duration` under `"wire": { "durations": "iso" }` is one canonical ISO-8601 string, refused -in a calendar or clock spelling, and filtered and sorted as a span on every engine) and `report/` (37, +in a calendar or clock spelling, and filtered and sorted as a span on every engine), `api-settings/` (6, +the project's `api` block: `api.pagination`'s default page and `limit_too_large` cap, and +`api.unknownFields: drop`) and `report/` (37, FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java, @@ -733,7 +725,7 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 510 + yaml 16 + verify 45 -+ render 18 + persistence 62 + api-contract 132 + source-resolution 25 + scope 10 + ++ render 18 + persistence 62 + api-contract 135 + source-resolution 25 + scope 10 + dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 6a36427b3f..7a621db515 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -294,11 +294,10 @@ each handler they register instead; an `HTTPException` is rethrown to your reads a whole list. `meta verify` advises (rule `unbounded-list-pages`) when a routes generator is wired and the project sets no page size. - **Ports: TypeScript reads `api.pagination` today** (the Fastify and Hono routes, packaged and - ejected, gated by `fixtures/api-contract-conformance/pagination/` in the generated lane). - The C#, Java, Kotlin and Python routes do not read it yet: their REST generators are under - the 2026-09-22 freeze, and carrying the setting there waits on that ruling. Until then a list - over a large table in those ports should always send a `limit`. + **Every port reads it:** the TypeScript Fastify and Hono routes, the C# minimal-API routes, the + Java and Kotlin Spring controllers and the Python FastAPI routers, packaged and ejected (an owned + copy picks it up when it is re-ejected). `fixtures/api-contract-conformance/api-settings/` pins + it in the generated lane of every port. - `withCount=1` — opt-in flag that switches the list response from `[...]` to `{ rows: [...], total: }` (needed for grid pagination). The grid hook always sends `withCount=1`. @@ -621,10 +620,9 @@ routes: | `@mutability: writeOnce` | kept | dropped | | the TPH discriminator | dropped (the route's subtype is its only source) | dropped | | a column only a sibling TPH subtype declares | dropped | dropped | -| a key the entity does not declare | dropped | dropped | None of these is a `400`, and none reaches the repository. Each port does it in its own idiom: TypeScript -parses with the Zod input schema (which drops unknown keys) and writes the parsed value; Python keeps only +parses with the Zod input schema (which drops keys it does not list) and writes the parsed value; Python keeps only the keys its input model validated; Java nulls the dropped components of the bound DTO before `repository.create` and its DTOs carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer does not depend on the host's `ObjectMapper`; Kotlin writes only the route's settable columns; C# resets the bound @@ -633,6 +631,36 @@ a private setter, which System.Text.Json never binds. The shared scenarios `inpu `tph/tph-input-server-owned-dropped` pin it, and `secret-fields/` pins a `readOnly` column beside the credential. +### Unknown request fields: `api.unknownFields` + +A key the entity does **not** declare is a different case, and a project setting decides it. In +`.metaobjects/config.json`: + +```json +{ "api": { "unknownFields": "reject" } } +``` + +- **`reject`, the default** (also when the key is absent). A create, PUT or PATCH body with any + undeclared top-level key answers `400 { "error": "unknown_fields", "fields": ["isAdmin", "role"] }`, + the keys sorted, and nothing is written. The check runs before validation, so a body that is both + misspelled and invalid is told about the spelling first. +- **`drop`.** Undeclared keys are ignored and the rest of the body is written, which is what every + port did before 1.3. + +A key is undeclared when no field of the entity has that name; on a TPH subtype's routes, when no +field of the base or of any subtype has it. Every key in the table above is declared, so it is +**dropped, not refused, under either value**: a client that sends back a row it read (with its `id`, +its `@autoSet` stamps and its `writeOnce` field) keeps working, and a sibling subtype's column on +another subtype's route is dropped. Only the top level is checked; a value object's members are +validated as before. `meta init` writes `"unknownFields": "reject"` for a new project. + +Each port checks the raw body before it binds it, so the answer is the same whatever the host +framework would do with an unknown property: Java and Kotlin read the keys from the request's JSON +tree, not from Jackson's binding, and answer identically with `FAIL_ON_UNKNOWN_PROPERTIES` on or +off. The shared scenarios `input-unknown-fields-rejected` and `tph/tph-unknown-fields-rejected` +pin `reject` in both lanes of every port, and `api-settings/unknown-fields-dropped` pins `drop` in +the generated lane. + Only the generated HTTP boundary changes. The entity, its DTO and the generated queries still carry the field, so your own code writes a hash or token and reads it back. Allowlists are unchanged: a field is filterable or sortable only if you wrote `@filterable` or `@sortable` on it, so do not put those on a credential diff --git a/docs/features/cli.md b/docs/features/cli.md index 462441bf28..a3d39141e4 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -25,7 +25,7 @@ command surface splits in two: | Capability | Tool | Invocation | Notes | |---|---|---|---| -| Project scaffold (`init`) | Node `meta` | `meta init` | TS projects; `--dialect sqlite\|postgres\|mysql\|d1` writes the codegen `dialect` in `metaobjects.config.ts` and, for sqlite, postgres and d1, `migrate.dialect` in `.metaobjects/config.json` (d1 is `--d1`; MySQL schema is not `meta migrate`'s). A new `.metaobjects/config.json` also carries `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }`, the generated list routes' page size ([api-contract.md](api-contract.md)); `--force` over an existing config does not add it | +| Project scaffold (`init`) | Node `meta` | `meta init` | TS projects; `--dialect sqlite\|postgres\|mysql\|d1` writes the codegen `dialect` in `metaobjects.config.ts` and, for sqlite, postgres and d1, `migrate.dialect` in `.metaobjects/config.json` (d1 is `--d1`; MySQL schema is not `meta migrate`'s). A new `.metaobjects/config.json` also carries `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "reject" }`, the generated list routes' page size and the refusal of undeclared write keys ([api-contract.md](api-contract.md)); `--force` over an existing config does not add it | | **Agent-context scaffold** (`agent-docs`) | **Node `meta`** | `npx meta agent-docs --server ` | **any backend** — single assembler (ADR-0033); non-Node CLIs redirect here | | **Schema migrate** | **Node `meta`** | `meta migrate` | **any backend** — schema is Node-only (ADR-0015) | | **Schema drift** (`verify --db`) | **Node `meta`** | `meta verify --db` | **any backend** — live-DB drift, Node-only | diff --git a/docs/features/generated-mutations.md b/docs/features/generated-mutations.md index 1adb3f2234..4564d5db46 100644 --- a/docs/features/generated-mutations.md +++ b/docs/features/generated-mutations.md @@ -42,8 +42,10 @@ but the semantics are one contract — call them **PATCH-1..7** (FR-035): `safeParse` succeeds and the key never reaches the `SET`. The TPH discriminator is additionally deleted by hand after parse, the same convention. A malformed *value* on a settable key is still `400 {"error":"validation", "issues": [...]}`. A key the entity - does not declare at all is dropped the same way. The Python routes hand the - repository only the keys their input model validated. + does not declare at all is NOT one of these: under `api.unknownFields: reject` (the default) + it answers `400 {"error":"unknown_fields","fields":[...]}` before validation, and only under + `drop` is it ignored the same way ([api-contract.md](api-contract.md#unknown-request-fields-apiunknownfields)). + The Python routes hand the repository only the keys their input model validated. This matters for `writeOnce` specifically: the generated edit form submits EVERY registered field (`handleSubmit` passes all values, and 0.19.2 switched the resolver to `UpdateSchema` on edit rather than diff-and-omit), so rejecting a diff --git a/docs/features/migrations/unknown-fields-reject-by-default.md b/docs/features/migrations/unknown-fields-reject-by-default.md new file mode 100644 index 0000000000..df213188e7 --- /dev/null +++ b/docs/features/migrations/unknown-fields-reject-by-default.md @@ -0,0 +1,60 @@ +# Undeclared request keys are refused by default (`api.unknownFields`) + +From 1.3.0, a generated create, PUT or PATCH route refuses a body that carries a key the entity +does not declare: + +```http +POST /api/customers +{ "name": "Ada", "emial": "ada@example.com" } + +400 { "error": "unknown_fields", "fields": ["emial"] } +``` + +Before 1.3.0 every port dropped such a key silently and answered 201, so a misspelled field lost +its value without a word. The new default applies to **existing projects too**, the first time +they regenerate their routes on 1.3.0, in every port (TypeScript, C#, Java, Kotlin, Python). + +## What counts as undeclared + +A top-level key no field of the entity is named (on a single-table hierarchy's routes, no field +of the base or any subtype). These are all **declared**, so they are still dropped silently, +never refused: + +- the generated primary key (`id`), an `@autoSet` stamp, a `readOnly` or derived field; +- a `writeOnce` field on PATCH or PUT; +- the TPH discriminator, and a column only a sibling subtype declares. + +So a client that reads a row and sends it back, edited, keeps working. Value-object members are +validated as before; only the top level is checked. + +## Do you need to do anything? + +Probably not, if your clients are the generated hooks and forms: they send only declared fields. +Check before you deploy if a client of yours: + +- sends extra keys on purpose (a UI-only flag, a nested relation, a field from another entity); +- posts a payload built for another API or an older version of the model, where a field has since + been renamed or removed. + +Search your server logs or tests for 400 answers with `"error": "unknown_fields"` after the upgrade; +the `fields` list names every key to fix. + +## To keep the old behaviour + +Set `drop` in `.metaobjects/config.json` and regenerate: + +```json +{ "api": { "unknownFields": "drop" } } +``` + +Every port reads this one key. Undeclared keys are then ignored and the rest of the body is +written, exactly as before 1.3.0. + +## Owned (ejected) routes + +An ejected routes generator keeps what it emitted when you copied it, so it keeps dropping until +you re-eject it with your port's eject command (`meta eject`, `dotnet meta eject`, +`mvn metaobjects:eject`, `metaobjects eject`) or make the change in your copy. In TypeScript the generated mount passes the +entity's declared fields as `knownFields`; a mount given none drops undeclared keys. + +See [api-contract.md](../api-contract.md#unknown-request-fields-apiunknownfields) for the contract. diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index 8723d47ffc..a99f9b51b3 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -1035,6 +1035,18 @@ directly into the generated list handler. A read-only projection (`source.rdb @kind: view`) and a view-backed report get the same filter and sort on their list route, against an allowlist of their own fields. +The `api` block of `.metaobjects/config.json` reaches the generated routes through the loaded root +(`MetaRoot.Api`). `api.pagination` (`defaultLimit`, `maxLimit`) gives every entity, TPH and projection +list route a default page and answers a larger `?limit=` with `400 { "error": "limit_too_large", +"maxLimit": N }`; a served report keeps its own row ceiling, and with no block the routes are unchanged. +`api.unknownFields` (`reject`, the default, or `drop`) makes a create, PUT or PATCH answer +`400 { "error": "unknown_fields", "fields": [...] }` for a body key no field of the entity (for a TPH +hierarchy, of the base or any subtype) declares; the check is `UnknownFieldCheck` in the helper runtime. +`dotnet meta init` does not write `.metaobjects/config.json`. `dotnet meta verify --codegen` prints a +non-failing advisory (`unbounded-list-pages`, stderr, exit code unchanged) when the run includes the routes +generator and the project sets no `api.pagination`; plain `verify` says nothing, because only `--codegen` +runs the generator list. + **Known gaps** (full list with workarounds in [KNOWN_GAPS.md](https://github.com/metaobjectsdev/metaobjects/blob/main/server/csharp/MetaObjects.Codegen/Generators/KNOWN_GAPS.md)): no `filter[or]` / `filter[and]` boolean grouping and a single sort key only (both outside the diff --git a/docs/ports/java.md b/docs/ports/java.md index ba2c998a6b..f378c4069c 100644 --- a/docs/ports/java.md +++ b/docs/ports/java.md @@ -570,6 +570,31 @@ the list route's body through the application's `ObjectMapper`, so a `wire` sett it does to the list. CSV and XML are written as `byte[]`, which every Spring MVC setup has a converter for. +### Page size and unknown body keys (`api` in `.metaobjects/config.json`) + +The generated controllers read two project settings from the `api` block, the same file that +carries `naming` and `wire` (`NeutralConfig.getApiSettings()`, carried on the loaded root as +`ApiSettings`): + +```json +{ "api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "reject" } } +``` + +- `api.pagination`: a list route (entity, TPH base and per-subtype, projection) with no `limit` + returns `defaultLimit` rows, and a `limit` above `maxLimit` answers + `400 {"error": "limit_too_large", "maxLimit": N}`. Absent, a list route returns 50 rows when + `limit` is omitted and takes any `limit`, as before; `mvn metaobjects:verify` then warns + (`unbounded-list-pages`, advisory, never fails the build). A served report keeps its own row + ceiling, and an M:N traversal is not paged. +- `api.unknownFields` (`reject`, the default, or `drop`): a create, PUT or PATCH body key the + entity does not declare answers `400 {"error": "unknown_fields", "fields": [...]}` (keys in code + point order) and writes nothing. Declared keys the client may not set (the generated key, + `readOnly`, derived, `@autoSet`, `writeOnce` on update, the discriminator, a sibling + subtype's column) are not unknown: they are dropped as before. The keys are read off the raw + `JsonNode` (`UnknownBodyKeys`, shipped with the generated runtime) before the body is bound, so + the answer does not depend on the host `ObjectMapper`'s `FAIL_ON_UNKNOWN_PROPERTIES`. Under + `reject` the generated create handler takes `@RequestBody JsonNode body` rather than the DTO. + ### Decimals and null members on the wire What the generated records write is the project setting `wire` in `.metaobjects/config.json` diff --git a/docs/ports/kotlin.md b/docs/ports/kotlin.md index 3dc4f382d7..ee7913bc0e 100644 --- a/docs/ports/kotlin.md +++ b/docs/ports/kotlin.md @@ -634,6 +634,23 @@ branch in each report `Controller` when that list holds a `report-export `Export.MAX_ROWS`, so the cap is set once, as `maxRows` on `report-export`. Without it the controllers are byte-identical to today and `?format=` is ignored like any unknown parameter. +### `api.pagination` and `api.unknownFields` + +`KotlinSpringControllerGenerator` reads the project's `api` block from the loaded root +(`loader.root.apiSettings`, the same `.metaobjects/config.json` reader as `naming` and `wire`). +With `api.pagination` every generated list route (entity, TPH base and per-subtype, projection; +not a served report or an M:N traversal) uses `defaultLimit` when `limit` is omitted and answers a +`limit` above `maxLimit` with `400 {"error": "limit_too_large", "maxLimit": N}`. Without it the +routes keep their historic default of 50 and no cap. With `api.unknownFields` `reject` (the +default) create, PUT and PATCH read the raw `JsonNode` body, list any top-level key the entity does +not declare (for a TPH base the union of the base's and every subtype's fields) as +`400 {"error": "unknown_fields", "fields": [...]}` in code-point order, and write nothing; this +happens before binding, so the answer is the same whatever the host `ObjectMapper` does with +unknown properties. The check is generated into a per-package `MetaUnknownBodyKeys.kt` (no +MetaObjects import; it ships with the `routes` eject). Because the create handler binds the raw +body, the controller now always takes the `ObjectMapper`. `drop` keeps today's behaviour and emits +no helper. + The Kotlin controllers import nothing from MetaObjects, so the request side (format parsing, the three 400 envelopes, the `Content-Disposition` and `Content-Type` values) is generated into the shared `ReportExportWriter` and reached through `Export`; there is no runtime class to copy. Writers diff --git a/examples/advanced-modeling/codegen/generators/routes.ts b/examples/advanced-modeling/codegen/generators/routes.ts index 8ffe8c2da9..59c800867a 100644 --- a/examples/advanced-modeling/codegen/generators/routes.ts +++ b/examples/advanced-modeling/codegen/generators/routes.ts @@ -71,6 +71,7 @@ import { decimalColumnsLine, paginationLine, writeDeniedColumnsLine, + knownFieldsLine, servesReadApi, formatTs, entityOutputPath, @@ -312,7 +313,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -337,7 +338,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -651,7 +652,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${knownFieldsLine(sub, " ", ctx.loadedRoot, hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts index 7e1b4e0b19..14cc548338 100644 --- a/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts +++ b/examples/advanced-modeling/codegen/runtime/drizzle-fastify/index.ts @@ -29,12 +29,13 @@ export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; -import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; +import { ERROR_CODE_UNKNOWN_FIELDS, stripDeniedColumns, unknownBodyKeys, withWire, type WireOptions } from "../wire-format.js"; import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; +export { ERROR_CODE_UNKNOWN_FIELDS, unknownBodyKeys } from "../wire-format.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -80,6 +81,14 @@ export interface CrudRoutesOptions extends WireOptions { * schemas stay whole for the application's own writes. */ writeDeniedColumns?: readonly string[]; + /** + * Every key a create or update body may carry: the entity's declared fields (for a + * single-table hierarchy, those of the base and every subtype). When set, a body with any + * other key answers `400 { "error": "unknown_fields", "fields": [...] }` before validation + * (`api.unknownFields: reject`, the default the generated routes emit). Absent, undeclared + * keys are dropped silently (`drop`). A known key the client may not set is dropped either way. + */ + knownFields?: readonly string[]; /** Limit which verbs are mounted. Defaults to all five. */ expose?: readonly CrudVerb[]; /** @@ -289,6 +298,8 @@ export function mountGetRoute(opts: VerbOptions): void { export function mountCreateRoute(opts: VerbOptions): void { const toWire = echoWire(opts); opts.fastify.post(opts.path, routeOpts(opts), async (req, reply) => { + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.insertSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); @@ -330,6 +341,8 @@ export function mountUpdateRoute(opts: VerbOptions): void { reply: { code: (n: number) => { send: (b: unknown) => unknown } }, ) => { const { id } = req.params as { id: string }; + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.updateSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); diff --git a/examples/advanced-modeling/codegen/runtime/wire-format.ts b/examples/advanced-modeling/codegen/runtime/wire-format.ts index b21817d409..ccb26e442e 100644 --- a/examples/advanced-modeling/codegen/runtime/wire-format.ts +++ b/examples/advanced-modeling/codegen/runtime/wire-format.ts @@ -69,6 +69,20 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un return omitKeys(body, denied); } +/** The `error` of the 400 a write answers when its body carries a key the entity does not declare. */ +export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; + +/** + * The keys of a request body that are not in `known`, sorted, or none when `known` is not + * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top + * level is checked; a body that is not a JSON object is left to validation. + */ +export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { + if (known === undefined || !isPlainObject(body)) return []; + const declared = new Set(known); + return Object.keys(body).filter((k) => !declared.has(k)).sort(); +} + function isPlainObject(v: unknown): v is Record { return v !== null && typeof v === "object" && !Array.isArray(v); } diff --git a/examples/showcase/codegen/generators/routes.ts b/examples/showcase/codegen/generators/routes.ts index 8ffe8c2da9..59c800867a 100644 --- a/examples/showcase/codegen/generators/routes.ts +++ b/examples/showcase/codegen/generators/routes.ts @@ -71,6 +71,7 @@ import { decimalColumnsLine, paginationLine, writeDeniedColumnsLine, + knownFieldsLine, servesReadApi, formatTs, entityOutputPath, @@ -312,7 +313,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -337,7 +338,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -651,7 +652,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${knownFieldsLine(sub, " ", ctx.loadedRoot, hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/examples/showcase/codegen/runtime/drizzle-fastify/index.ts b/examples/showcase/codegen/runtime/drizzle-fastify/index.ts index 7e1b4e0b19..14cc548338 100644 --- a/examples/showcase/codegen/runtime/drizzle-fastify/index.ts +++ b/examples/showcase/codegen/runtime/drizzle-fastify/index.ts @@ -29,12 +29,13 @@ export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; -import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; +import { ERROR_CODE_UNKNOWN_FIELDS, stripDeniedColumns, unknownBodyKeys, withWire, type WireOptions } from "../wire-format.js"; import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; +export { ERROR_CODE_UNKNOWN_FIELDS, unknownBodyKeys } from "../wire-format.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -80,6 +81,14 @@ export interface CrudRoutesOptions extends WireOptions { * schemas stay whole for the application's own writes. */ writeDeniedColumns?: readonly string[]; + /** + * Every key a create or update body may carry: the entity's declared fields (for a + * single-table hierarchy, those of the base and every subtype). When set, a body with any + * other key answers `400 { "error": "unknown_fields", "fields": [...] }` before validation + * (`api.unknownFields: reject`, the default the generated routes emit). Absent, undeclared + * keys are dropped silently (`drop`). A known key the client may not set is dropped either way. + */ + knownFields?: readonly string[]; /** Limit which verbs are mounted. Defaults to all five. */ expose?: readonly CrudVerb[]; /** @@ -289,6 +298,8 @@ export function mountGetRoute(opts: VerbOptions): void { export function mountCreateRoute(opts: VerbOptions): void { const toWire = echoWire(opts); opts.fastify.post(opts.path, routeOpts(opts), async (req, reply) => { + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.insertSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); @@ -330,6 +341,8 @@ export function mountUpdateRoute(opts: VerbOptions): void { reply: { code: (n: number) => { send: (b: unknown) => unknown } }, ) => { const { id } = req.params as { id: string }; + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.updateSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); diff --git a/examples/showcase/codegen/runtime/wire-format.ts b/examples/showcase/codegen/runtime/wire-format.ts index b21817d409..ccb26e442e 100644 --- a/examples/showcase/codegen/runtime/wire-format.ts +++ b/examples/showcase/codegen/runtime/wire-format.ts @@ -69,6 +69,20 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un return omitKeys(body, denied); } +/** The `error` of the 400 a write answers when its body carries a key the entity does not declare. */ +export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; + +/** + * The keys of a request body that are not in `known`, sorted, or none when `known` is not + * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top + * level is checked; a body that is not a JSON object is left to validation. + */ +export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { + if (known === undefined || !isPlainObject(body)) return []; + const declared = new Set(known); + return Object.keys(body).filter((k) => !declared.has(k)).sort(); +} + function isPlainObject(v: unknown): v is Record { return v !== null && typeof v === "object" && !Array.isArray(v); } diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 183c4ef3f5..1dc8012fe2 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -27,7 +27,8 @@ fixtures/api-contract-conformance/ ├── get-by-id.yaml ├── get-by-id-not-found.yaml ├── create-201.yaml - ├── input-server-owned-dropped.yaml # a client id, a made-up key and a writeOnce change are dropped, never 400 + ├── input-server-owned-dropped.yaml # a client id and a writeOnce change are dropped, never 400 + ├── input-unknown-fields-rejected.yaml # an undeclared key → 400 {error:"unknown_fields", fields}; an echoed read model is not refused ├── create-constraint-violation-400.yaml # FR-036 POST field-constraint → 400 {error:"validation"} ├── validation-lists-issues.yaml # a 400 validation lists every issue: {error, issues:[{code, path, message}]} ├── update-patch-and-put.yaml @@ -54,13 +55,13 @@ fixtures/api-contract-conformance/ └── filter-invalid-value.yaml # uncoercible value → 400; the fourth envelope, previously ungated ``` -Beside the 33 core scenarios above sit eleven **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (99 scenarios in all, 132 with the +Beside the 34 core scenarios above sit eleven **sub-corpora**, each a directory with its +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (101 scenarios in all, 135 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| -| `tph/` | 12 | single-table inheritance | both | +| `tph/` | 13 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | | `jsonb/` | 5 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | @@ -69,7 +70,7 @@ core): | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | | `decimal/` | 11 | `field.decimal` is an exact canonical string on the wire (ADR-0019 Amendment 1): read, filter, sort, write and a report over decimals. A scenario's `engines:` key is exact; the `-15` twins run on SQLite and D1 as well. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | -| `pagination/` | 5 | `api.pagination` in the sub-corpus's own `.metaobjects/config.json` (default 2, max 4): a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served and one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` | generated only (TypeScript; the other ports' routes wait on the non-TS REST generator ruling) | +| `api-settings/` | 6 | the `api` block in the sub-corpus's own `.metaobjects/config.json`: `api.pagination` (default 2, max 4) — a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served and one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` — and `api.unknownFields: drop`, under which an undeclared body key is ignored rather than refused | generated only | | `duration/` | 2 | `field.duration` is a canonical ISO-8601 span on the wire under `"wire": { "durations": "iso" }`: write, read, refuse, and filter and sort as a span on every engine. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` diff --git a/fixtures/api-contract-conformance/pagination/.metaobjects/config.json b/fixtures/api-contract-conformance/api-settings/.metaobjects/config.json similarity index 61% rename from fixtures/api-contract-conformance/pagination/.metaobjects/config.json rename to fixtures/api-contract-conformance/api-settings/.metaobjects/config.json index f6d902d82d..16b88aa0f3 100644 --- a/fixtures/api-contract-conformance/pagination/.metaobjects/config.json +++ b/fixtures/api-contract-conformance/api-settings/.metaobjects/config.json @@ -2,5 +2,5 @@ "schema_version": 1, "naming": { "columns": "literal" }, "wire": { "decimals": "string", "nullMembers": "omit", "durations": "iso" }, - "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 } } + "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 }, "unknownFields": "drop" } } diff --git a/fixtures/api-contract-conformance/api-settings/README.md b/fixtures/api-contract-conformance/api-settings/README.md new file mode 100644 index 0000000000..3c5f41b0d2 --- /dev/null +++ b/fixtures/api-contract-conformance/api-settings/README.md @@ -0,0 +1,62 @@ +# `api-contract-conformance/api-settings/` — the project's `api` block on generated routes + +Cross-port REST contract for the `api` block of `.metaobjects/config.json`: `api.pagination`, +which gives every generated list route a default page size and a hard cap on `?limit=`, and +`api.unknownFields`, which decides what a generated create, PUT or PATCH route does with a body +key the entity does not declare. The settings live in this sub-corpus's own +`.metaobjects/config.json`, not in a runner: + +```json +{ "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 }, "unknownFields": "drop" } } +``` + +so a lane reaches them exactly the way an adopter's project does, through the config the +metadata load reads. The rest of that file repeats the parent corpus's `naming` and `wire`. + +The model is one `Item` (`id`, `title`), seeded with five rows. The sizes are small so the +scenarios can tell the default page from the whole table and the cap from the table size. + +## The contract + +`api.pagination`: + +- **No `limit` answers the default page.** `GET /api/items` returns 2 rows, not 5. +- **The cap is inclusive.** `?limit=4` returns 4 rows. +- **Over the cap is refused, not clamped.** `?limit=5` answers + `400 {"error": "limit_too_large", "maxLimit": 4}`, so a client that asked for more than the + server serves is told so instead of receiving a short page it cannot tell from the end. +- **Paging past the default works.** `?offset=4` returns the one remaining row, and + `?withCount=1` answers 2 rows with `total: 5`. + +A project WITHOUT `api.pagination` keeps the behaviour the parent corpus pins: no default page +and no cap. That half is the other sub-corpora, none of which sets it. + +`api.unknownFields: drop`: + +- **An undeclared key is ignored.** A create, PATCH or PUT carrying `isAdmin` is answered + 201/200 with the declared fields written, and `isAdmin` is neither stored nor echoed. + +The default, `reject`, is pinned by the parent corpus, which sets nothing: +`scenarios/input-unknown-fields-rejected.yaml` and `tph/scenarios/tph-unknown-fields-rejected.yaml`. + +## Lanes + +Generated lane only, as for every sub-corpus whose subject is what a generator emits. Every +port runs it: TypeScript, C#, Java, Kotlin and Python. + +## Files + +``` +api-settings/ +├── README.md +├── .metaobjects/config.json # api.pagination { 2, 4 }, api.unknownFields "drop" +├── meta.json # Item +├── seed.json # five rows +└── scenarios/ + ├── default-page-size.yaml + ├── limit-at-max.yaml + ├── limit-over-max.yaml + ├── offset-pages-past-default.yaml + ├── unknown-fields-dropped.yaml + └── with-count-default-page.yaml +``` diff --git a/fixtures/api-contract-conformance/pagination/meta.json b/fixtures/api-contract-conformance/api-settings/meta.json similarity index 100% rename from fixtures/api-contract-conformance/pagination/meta.json rename to fixtures/api-contract-conformance/api-settings/meta.json diff --git a/fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/default-page-size.yaml similarity index 100% rename from fixtures/api-contract-conformance/pagination/scenarios/default-page-size.yaml rename to fixtures/api-contract-conformance/api-settings/scenarios/default-page-size.yaml diff --git a/fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/limit-at-max.yaml similarity index 100% rename from fixtures/api-contract-conformance/pagination/scenarios/limit-at-max.yaml rename to fixtures/api-contract-conformance/api-settings/scenarios/limit-at-max.yaml diff --git a/fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/limit-over-max.yaml similarity index 100% rename from fixtures/api-contract-conformance/pagination/scenarios/limit-over-max.yaml rename to fixtures/api-contract-conformance/api-settings/scenarios/limit-over-max.yaml diff --git a/fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/offset-pages-past-default.yaml similarity index 100% rename from fixtures/api-contract-conformance/pagination/scenarios/offset-pages-past-default.yaml rename to fixtures/api-contract-conformance/api-settings/scenarios/offset-pages-past-default.yaml diff --git a/fixtures/api-contract-conformance/api-settings/scenarios/unknown-fields-dropped.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/unknown-fields-dropped.yaml new file mode 100644 index 0000000000..7104d4ef0f --- /dev/null +++ b/fixtures/api-contract-conformance/api-settings/scenarios/unknown-fields-dropped.yaml @@ -0,0 +1,49 @@ +name: unknown-fields-dropped +description: > + `api.unknownFields: drop` (this sub-corpus's config). A create, PUT or PATCH body carrying a + key the entity does not declare is answered as if the key were absent: the declared fields + are written, the undeclared key is ignored (201/200, never 400) and never stored or echoed. + This is the behaviour every port had before 1.3, kept for a project that asks for it. +requests: + - id: r1 + method: POST + path: /api/items + body: + title: "Forge" + isAdmin: true + expect: + status: 201 + body: + row: { title: "Forge" } + hasId: true + absent: [isAdmin] + - id: r2 + method: PATCH + path: /api/items/1 + body: + title: "Anvil (patched)" + isAdmin: true + expect: + status: 200 + body: + row: { id: 1, title: "Anvil (patched)" } + absent: [isAdmin] + - id: r3 + method: PUT + path: /api/items/2 + body: + title: "Bellows (put)" + isAdmin: true + expect: + status: 200 + body: + row: { id: 2, title: "Bellows (put)" } + absent: [isAdmin] + - id: r4 + method: GET + path: /api/items/1 + expect: + status: 200 + body: + row: { id: 1, title: "Anvil (patched)" } + absent: [isAdmin] diff --git a/fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/with-count-default-page.yaml similarity index 100% rename from fixtures/api-contract-conformance/pagination/scenarios/with-count-default-page.yaml rename to fixtures/api-contract-conformance/api-settings/scenarios/with-count-default-page.yaml diff --git a/fixtures/api-contract-conformance/pagination/seed.json b/fixtures/api-contract-conformance/api-settings/seed.json similarity index 100% rename from fixtures/api-contract-conformance/pagination/seed.json rename to fixtures/api-contract-conformance/api-settings/seed.json diff --git a/fixtures/api-contract-conformance/pagination/README.md b/fixtures/api-contract-conformance/pagination/README.md deleted file mode 100644 index 686bd02b39..0000000000 --- a/fixtures/api-contract-conformance/pagination/README.md +++ /dev/null @@ -1,51 +0,0 @@ -# `api-contract-conformance/pagination/` — a project page size on generated list routes - -Cross-port REST contract for `api.pagination`, the project setting that gives every generated -list route a default page size and a hard cap on `?limit=`. The setting lives in this -sub-corpus's own `.metaobjects/config.json`, not in a runner: - -```json -{ "api": { "pagination": { "defaultLimit": 2, "maxLimit": 4 } } } -``` - -so a lane reaches it exactly the way an adopter's project does, through the config the -metadata load reads. The rest of that file repeats the parent corpus's `naming` and `wire`. - -The model is one `Item` (`id`, `title`), seeded with five rows. The sizes are small so the -scenarios can tell the default page from the whole table and the cap from the table size. - -## The contract - -- **No `limit` answers the default page.** `GET /api/items` returns 2 rows, not 5. -- **The cap is inclusive.** `?limit=4` returns 4 rows. -- **Over the cap is refused, not clamped.** `?limit=5` answers - `400 {"error": "limit_too_large", "maxLimit": 4}`, so a client that asked for more than the - server serves is told so instead of receiving a short page it cannot tell from the end. -- **Paging past the default works.** `?offset=4` returns the one remaining row, and - `?withCount=1` answers 2 rows with `total: 5`. - -A project WITHOUT the setting keeps the behaviour the parent corpus pins: no default page and -no cap. That half is the other sub-corpora, none of which sets `api.pagination`. - -## Lanes - -Generated lane only, as for every sub-corpus whose subject is what a generator emits. -**TypeScript runs it today.** The C#, Java, Kotlin and Python routes do not read -`api.pagination` yet: their REST generators are under the 2026-09-22 freeze, and adding the -setting there waits on that ruling. - -## Files - -``` -pagination/ -├── README.md -├── .metaobjects/config.json # api.pagination { defaultLimit: 2, maxLimit: 4 } -├── meta.json # Item -├── seed.json # five rows -└── scenarios/ - ├── default-page-size.yaml - ├── limit-at-max.yaml - ├── limit-over-max.yaml - ├── offset-pages-past-default.yaml - └── with-count-default-page.yaml -``` diff --git a/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml index 0e71b5fa3a..bb1633b809 100644 --- a/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml +++ b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml @@ -1,16 +1,15 @@ name: input-server-owned-dropped description: > A write route hands storage only what the client may set. A client-sent `id` on a - server-generated key, a key the entity does not declare (`isAdmin`) and, on update, a - `writeOnce` field are all DROPPED SILENTLY: the request answers as if the key were absent - (201/200, never 400), and none of them reaches storage. This is the rule TypeScript has - always followed and Python adopted in 1.2.1, where the generated routes handed the - repository the raw request body. + server-generated key and, on update, a `writeOnce` field are declared but server-owned, so + they are DROPPED SILENTLY: the request answers as if the key were absent (201/200, never + 400), and neither reaches storage. Python 1.2.1 fixed exactly this, where the generated + routes handed the repository the raw request body. A key the entity does not declare is a + different case, `input-unknown-fields-rejected`. Each body carries an ordinary settable field DELIBERATELY: it is applied, and that pairing is what makes the assertion about the server-owned keys and not about the request being - ignored whole. `absent` asserts the undeclared key is not echoed back (a stored key would - be), and the GETs of the client's ids prove no row was written or re-keyed under them. + ignored whole. The GETs of the client's ids prove no row was written or re-keyed under them. requests: - id: r1 method: POST @@ -19,14 +18,12 @@ requests: id: 999 name: "Barbara Liskov" createdAt: "2026-02-01T10:00:00" - isAdmin: true expect: status: 201 body: row: name: "Barbara Liskov" hasId: true - absent: [isAdmin] - id: r2 method: GET path: /api/authors/999 @@ -38,14 +35,12 @@ requests: body: id: 998 name: "Ada L. (input probe)" - isAdmin: true expect: status: 200 body: row: id: 1 name: "Ada L. (input probe)" - absent: [isAdmin] - id: r4 method: PUT path: /api/authors/2 @@ -54,7 +49,6 @@ requests: name: "Alan T. (input probe)" createdAt: "2026-01-02T10:00:00" issuedCurrency: "JPY" - isAdmin: true expect: status: 200 body: @@ -63,7 +57,6 @@ requests: name: "Alan T. (input probe)" # writeOnce: frozen after create, so the PUT's value never lands. issuedCurrency: "GBP" - absent: [isAdmin] - id: r5 method: GET path: /api/authors/998 @@ -83,4 +76,3 @@ requests: row: id: 1 name: "Ada L. (input probe)" - absent: [isAdmin] diff --git a/fixtures/api-contract-conformance/scenarios/input-unknown-fields-rejected.yaml b/fixtures/api-contract-conformance/scenarios/input-unknown-fields-rejected.yaml new file mode 100644 index 0000000000..177379ce25 --- /dev/null +++ b/fixtures/api-contract-conformance/scenarios/input-unknown-fields-rejected.yaml @@ -0,0 +1,92 @@ +name: input-unknown-fields-rejected +description: > + `api.unknownFields` at its default, `reject` (this corpus sets nothing). A create, PUT or + PATCH body carrying a key the entity does not declare is refused before validation with + `400 {"error": "unknown_fields", "fields": [...]}`, the undeclared keys sorted, and nothing + is written: the create leaves no row and the updates leave the seeded rows as they were. + + A declared key the client may not set is NOT unknown. The last request echoes back a row as + a client read it, with the generated `id`, the `writeOnce` field and an `@autoSet` stamp, + and is answered 200: those keys are dropped, not refused, so a client that sends back the + read model keeps working. `api-settings/` pins the other value, `drop`. +requests: + - id: r1 + method: POST + path: /api/authors + body: + name: "Barbara Liskov" + createdAt: "2026-02-01T10:00:00" + role: "admin" + isAdmin: true + expect: + status: 400 + body: + equals: + error: "unknown_fields" + fields: [isAdmin, role] + - id: r2 + method: GET + path: /api/authors?filter[name][eq]=Barbara%20Liskov + expect: + status: 200 + body: + length: 0 + - id: r3 + method: PATCH + path: /api/authors/1 + body: + name: "Ada L. (input probe)" + isAdmin: true + expect: + status: 400 + body: + equals: + error: "unknown_fields" + fields: [isAdmin] + - id: r4 + method: PUT + path: /api/authors/2 + body: + name: "Alan T. (input probe)" + createdAt: "2026-01-02T10:00:00" + isAdmin: true + expect: + status: 400 + body: + equals: + error: "unknown_fields" + fields: [isAdmin] + - id: r5 + method: GET + path: /api/authors/1 + expect: + status: 200 + body: + row: + id: 1 + name: "Ada Lovelace" + - id: r6 + method: GET + path: /api/authors/2 + expect: + status: 200 + body: + row: + id: 2 + name: "Alan Turing" + - id: r7 + method: PATCH + path: /api/authors/1 + body: + id: 1 + name: "Ada L. (echoed)" + createdAt: "2026-01-01T10:00:00" + issuedCurrency: "USD" + autoCreatedAt: "2000-01-01T00:00:00" + expect: + status: 200 + body: + row: + id: 1 + name: "Ada L. (echoed)" + issuedCurrency: "USD" diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml index f340d90be7..d6c52b992a 100644 --- a/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml @@ -1,10 +1,10 @@ name: tph-input-server-owned-dropped description: > The per-subtype write routes hand storage only what the client may set, the same rule as - `input-server-owned-dropped` in the core corpus. A client-sent `id` on the generated key, a - key the hierarchy does not declare (`isAdmin`), the discriminator (`type`) and, on update, a - `writeOnce` field are all DROPPED SILENTLY (201/200, never 400): the row keeps its - server-assigned id, its subtype comes from the URL, and nothing undeclared is echoed back. + `input-server-owned-dropped` in the core corpus. A client-sent `id` on the generated key, the + discriminator (`type`) and, on update, a `writeOnce` field are declared but server-owned, so + they are DROPPED SILENTLY (201/200, never 400): the row keeps its server-assigned id and its + subtype comes from the URL. An undeclared key is `tph-unknown-fields-rejected`. requests: - id: r1 method: POST @@ -14,13 +14,11 @@ requests: type: "Copay" reference: "REF-INPUT" quantity: 3 - isAdmin: true expect: status: 201 body: row: { type: "Bridge", reference: "REF-INPUT", quantity: 3 } hasId: true - absent: [isAdmin] - id: r2 method: GET path: /api/auths/999 @@ -34,24 +32,20 @@ requests: type: "Copay" reference: "REF-1-INPUT" issuedCurrency: "JPY" - isAdmin: true expect: status: 200 body: row: { id: 1, type: "Bridge", reference: "REF-1-INPUT", issuedCurrency: "USD" } - absent: [isAdmin] - id: r4 method: PUT path: /api/auths/copay/2 body: id: 997 reference: "REF-2-INPUT" - isAdmin: true expect: status: 200 body: row: { id: 2, type: "Copay", reference: "REF-2-INPUT" } - absent: [isAdmin] - id: r5 method: GET path: /api/auths/998 diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-unknown-fields-rejected.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-unknown-fields-rejected.yaml new file mode 100644 index 0000000000..03a26fd16b --- /dev/null +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-unknown-fields-rejected.yaml @@ -0,0 +1,59 @@ +name: tph-unknown-fields-rejected +description: > + The per-subtype write routes apply `api.unknownFields` (default `reject`) over the WHOLE + hierarchy. A key no member of the hierarchy declares (`isAdmin`) is refused with + `400 {"error": "unknown_fields", "fields": [...]}` and nothing is written. A column only a + sibling subtype declares (`approver` on the Bridge route) is declared in the table, so it is + known: it is dropped like any other key that subtype may not set, not refused. +requests: + - id: r1 + method: POST + path: /api/auths/bridge + body: + reference: "REF-UNKNOWN" + quantity: 3 + isAdmin: true + expect: + status: 400 + body: + equals: + error: "unknown_fields" + fields: [isAdmin] + - id: r2 + method: GET + path: /api/auths?filter[reference][eq]=REF-UNKNOWN + expect: + status: 200 + body: + length: 0 + - id: r3 + method: PATCH + path: /api/auths/bridge/1 + body: + reference: "REF-1-PROBE" + isAdmin: true + expect: + status: 400 + body: + equals: + error: "unknown_fields" + fields: [isAdmin] + - id: r4 + method: GET + path: /api/auths/bridge/1 + expect: + status: 200 + body: + row: { id: 1, type: "Bridge", reference: "REF-1" } + - id: r5 + method: POST + path: /api/auths/bridge + body: + reference: "REF-SIBLING" + quantity: 2 + approver: "Dr. Sibling" + expect: + status: 201 + body: + row: { type: "Bridge", reference: "REF-SIBLING", quantity: 2 } + hasId: true diff --git a/fixtures/conformance/routes-file-hono-basic/expected/Author.routes.hono.ts b/fixtures/conformance/routes-file-hono-basic/expected/Author.routes.hono.ts index fc5bed4620..78746992c5 100644 --- a/fixtures/conformance/routes-file-hono-basic/expected/Author.routes.hono.ts +++ b/fixtures/conformance/routes-file-hono-basic/expected/Author.routes.hono.ts @@ -45,5 +45,6 @@ export function registerAuthorRoutes( filterAllowlist: AuthorFilterAllowlist, sortAllowlist: AuthorSortAllowlist, dialect: "sqlite", + knownFields: ["id", "name", "bio"], }); } diff --git a/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs b/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs index c773dce5ac..0fcc7c3112 100644 --- a/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs @@ -205,6 +205,55 @@ public void Codegen_clean_committed_output_is_exit0() Assert.True(r.Codegen!.Clean); } + [Fact] + public void Codegen_with_routes_and_no_pagination_advises_and_keeps_exit0() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); + + var r = VerifyCommand.RunSubverbs(TemplatesOpts(templates: false, codegen: true)); + Assert.Equal(0, r.ExitCode); + Assert.Contains(r.Advisories, a => a.Contains("unbounded-list-pages")); + } + + [Fact] + public void Codegen_with_pagination_set_has_no_advisory() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); + + var opts = TemplatesOpts(templates: false, codegen: true); + var r = VerifyCommand.RunSubverbs(new VerifyCommand.Options + { + MetadataDir = opts.MetadataDir, Generators = opts.Generators, TemplatesRoot = opts.TemplatesRoot, + OutDir = opts.OutDir, Namespace = opts.Namespace, Codegen = true, Templates = false, + Api = new ApiSettings(new ApiPagination(50, 1000)), + }); + // (Exit is not asserted: the committed output was generated without a page size, so it drifts.) + Assert.Empty(r.Advisories); + } + + [Fact] + public void Codegen_without_a_routes_generator_has_no_advisory() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); + var opts = TemplatesOpts(templates: false, codegen: true); + var r = VerifyCommand.RunSubverbs(new VerifyCommand.Options + { + MetadataDir = opts.MetadataDir, Generators = ["entity"], TemplatesRoot = opts.TemplatesRoot, + OutDir = opts.OutDir, Namespace = opts.Namespace, Codegen = true, Templates = false, + }); + Assert.Empty(r.Advisories); + } + + [Fact] + public void Without_codegen_there_is_no_advisory() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); + var r = VerifyCommand.RunSubverbs(TemplatesOpts(templates: true, codegen: false)); + Assert.Empty(r.Advisories); + } + [Fact] public void Codegen_mutated_committed_file_is_nonzero_and_named() { diff --git a/server/csharp/MetaObjects.Cli/Program.cs b/server/csharp/MetaObjects.Cli/Program.cs index 34b1f78c9c..85dd5acb5f 100644 --- a/server/csharp/MetaObjects.Cli/Program.cs +++ b/server/csharp/MetaObjects.Cli/Program.cs @@ -566,6 +566,7 @@ static int RunVerify(string[] rest) Naming = resolvedMeta.Naming, // ADR-0019 Amendment 2 — the wire setting, from the same file, for the --codegen regen. Wire = resolvedMeta.Wire ?? MetaObjects.Meta.WireFormat.Default, + Api = resolvedMeta.Api ?? MetaObjects.Meta.ApiSettings.Default, TemplatesRoot = templatesRoot, OutDir = outDir, Namespace = ns, @@ -635,6 +636,8 @@ static int RunVerify(string[] rest) } } + foreach (var advisory in result.Advisories) Console.Error.WriteLine(advisory); + // -- db gate (rejected in C#) -- if (result.DbRejectionMessage is not null) Console.Error.WriteLine($" {result.DbRejectionMessage}"); diff --git a/server/csharp/MetaObjects.Cli/VerifyCommand.cs b/server/csharp/MetaObjects.Cli/VerifyCommand.cs index 81dcb12fa5..d11f0110a4 100644 --- a/server/csharp/MetaObjects.Cli/VerifyCommand.cs +++ b/server/csharp/MetaObjects.Cli/VerifyCommand.cs @@ -56,6 +56,11 @@ public sealed record Outcome( // ADR-0021 D2 — explicit verify subverbs (--templates / --codegen / --db). // ------------------------------------------------------------------------ + /// Advisory rule unbounded-list-pages: routes are generated and no api.pagination is set. + public const string UNBOUNDED_LIST_PAGES_ADVISORY = + "dotnet meta verify --codegen: advisory unbounded-list-pages: list routes return every row when `limit` is omitted; " + + "set `api.pagination` in .metaobjects/config.json (for example { \"defaultLimit\": 50, \"maxLimit\": 1000 })."; + /// The message printed when --db is requested in the C# port. public const string DB_NOT_SUPPORTED = "verify --db: schema drift is not supported in the C# port; schema verify is the " + @@ -121,6 +126,8 @@ public sealed record Options /// and set on the loaded root beside it. Defaults to native. /// public MetaObjects.Meta.WireFormat Wire { get; init; } = MetaObjects.Meta.WireFormat.Default; + /// The project's api block, set on the loaded root beside . + public MetaObjects.Meta.ApiSettings Api { get; init; } = MetaObjects.Meta.ApiSettings.Default; /// Templates root for the --templates gate (--templates <root>). public string? TemplatesRoot { get; init; } /// The committed output dir for the --codegen gate (--out <dir>). @@ -194,6 +201,8 @@ public sealed record SubverbResult /// this to report a failed load once, and never zero times. /// public bool LoadFailureReported { get; init; } + /// Non-failing advisories (never part of ); printed to stderr. + public IReadOnlyList Advisories { get; init; } = []; } /// @@ -224,10 +233,11 @@ public static SubverbResult RunSubverbs(Options opts) if (templatesOutcome.LoadErrors.Count > 0) loadFailureReported = true; } + var advisories = new List(); Codegen.CodegenDrift.Result? codegenResult = null; if (runCodegen) { - codegenResult = RunCodegenDrift(opts, out var codegenLoadFailed); + codegenResult = RunCodegenDrift(opts, advisories, out var codegenLoadFailed); if (codegenLoadFailed) loadFailureReported = true; // usage error (nothing to diff against) → exit 2; metadata that does not load → // exit 1, the code every port's gen/verify/fmt uses for it; drift → 1; clean → 0. @@ -254,6 +264,7 @@ public static SubverbResult RunSubverbs(Options opts) Codegen = codegenResult, DbRejectionMessage = dbMsg, LoadFailureReported = loadFailureReported, + Advisories = advisories, }; } @@ -269,7 +280,7 @@ private static LoadResult LoadMetadata(Options opts) // The same load `gen` runs (ResolvedMetadata.Load): libraries, then the dependency snapshots, then the // project's own files; ADR-0058 naming and the wire format set on the root; the imports recorded on it. return new MetaObjects.Config.ResolvedMetadata( - opts.MetadataDir, opts.MetadataFiles, opts.Libraries, opts.Naming, opts.Wire, null, opts.Imports, null, opts.Providers) + opts.MetadataDir, opts.MetadataFiles, opts.Libraries, opts.Naming, opts.Wire, null, opts.Imports, null, opts.Providers) { Api = opts.Api } .Load(opts.Strict); } @@ -369,7 +380,7 @@ private static string FormatRequirementDiagnostic(RequirementDiagnostic d) => /// says the metadata itself did not load (exit 1, not /// the usage exit 2 a missing --out or an unknown generator gets). /// - private static Codegen.CodegenDrift.Result RunCodegenDrift(Options opts, out bool loadFailed) + private static Codegen.CodegenDrift.Result RunCodegenDrift(Options opts, List advisories, out bool loadFailed) { loadFailed = false; if (opts.OutDir is null) @@ -436,6 +447,13 @@ private static Codegen.CodegenDrift.Result RunCodegenDrift(Options opts, out boo ], }; + // unbounded-list-pages (advisory, never fails): the run generates list routes and the project sets no + // page size, so a list route returns every row when `limit` is omitted. + if (load.Root.Api.Pagination is null + && generators.Any(g => g is Codegen.Generators.RoutesGenerator || g.Name.Contains("routes", StringComparison.OrdinalIgnoreCase)) + && load.Root.Objects().Any(o => Codegen.Generators.RoutesGenerator.AppliesTo(o, load.Root))) + advisories.Add(UNBOUNDED_LIST_PAGES_ADVISORY); + // C1 — same presence gate `gen` computes (GeneratorRegistry.IncludesNames over // the SAME resolved `names` list above), or a regen under a `--generators` // selection that excludes `names` would reference `Names.*` while `gen` diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs new file mode 100644 index 0000000000..71888eb33c --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs @@ -0,0 +1,98 @@ +// `api` in .metaobjects/config.json: the config reader's rules, and what the routes generator emits +// for each setting. The behaviour itself (status and body over HTTP) is proven by the api-contract +// lanes in MetaObjects.IntegrationTests (core, tph, api-settings). + +using MetaObjects.Codegen; +using MetaObjects.Codegen.Generators; +using MetaObjects.Config; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ApiSettingsTests +{ + private static ApiSettings Read(string configJson) + { + var dir = Directory.CreateTempSubdirectory("mo-api-settings-").FullName; + try + { + Directory.CreateDirectory(Path.Combine(dir, ".metaobjects")); + File.WriteAllText(Path.Combine(dir, ".metaobjects", "config.json"), configJson); + return NeutralConfig.Read(dir)!.Api; + } + finally { Directory.Delete(dir, recursive: true); } + } + + [Fact] + public void No_api_block_is_no_page_size_and_reject() + { + var api = Read("""{ "schema_version": 1 }"""); + Assert.Null(api.Pagination); + Assert.Equal(UnknownFieldsMode.Reject, api.UnknownFields); + } + + [Fact] + public void A_full_block_is_read() + { + var api = Read("""{ "schema_version": 1, "api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "drop" } }"""); + Assert.Equal(new ApiPagination(50, 1000), api.Pagination); + Assert.Equal(UnknownFieldsMode.Drop, api.UnknownFields); + } + + [Theory] + [InlineData("""{ "schema_version": 1, "api": { "pagination": { "defaultLimit": 0, "maxLimit": 10 } } }""", "\"api.pagination.defaultLimit\" must be a positive integer")] + [InlineData("""{ "schema_version": 1, "api": { "pagination": { "defaultLimit": 5, "maxLimit": 2.5 } } }""", "\"api.pagination.maxLimit\" must be a positive integer")] + [InlineData("""{ "schema_version": 1, "api": { "pagination": { "defaultLimit": 5 } } }""", "\"api.pagination.maxLimit\" must be a positive integer")] + [InlineData("""{ "schema_version": 1, "api": { "pagination": { "defaultLimit": 20, "maxLimit": 10 } } }""", "\"api.pagination.defaultLimit\" must not exceed \"api.pagination.maxLimit\"")] + [InlineData("""{ "schema_version": 1, "api": { "unknownFields": "ignore" } }""", "\"api.unknownFields\" must be \"reject\" or \"drop\"")] + public void A_bad_block_is_a_load_error_naming_the_key(string config, string message) + { + var e = Assert.Throws(() => Read(config)); + Assert.True(e.Message.Contains(message), e.Message); + } + + private const string Model = """ + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Post", "children": [ + { "source.rdb": { "@table": "posts" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "identity.primary": { "@fields": "id" } } + ]}} + ]}} + """; + + private static string Routes(ApiSettings? api) + { + var load = new MetaDataLoader().Load([new InMemoryStringSource(Model, id: "model.json")]); + Assert.Empty(load.Errors); + if (api is not null) load.Root.Api = api; + var ctx = new GenContext + { + Entities = load.Root.Objects(), + Root = load.Root, + Config = new GenConfig { OutDir = "/unused", Namespace = "Acme" }, + }; + return Assert.Single(new RoutesGenerator().Generate(ctx)).Content; + } + + [Fact] + public void Absent_settings_emit_the_unknown_field_check_and_no_page_size() + { + var src = Routes(null); + Assert.Contains("UnknownFieldCheck.Reject(", src); + Assert.DoesNotContain("limit_too_large", src); + } + + [Fact] + public void Drop_emits_no_unknown_field_check_and_a_page_size_is_emitted_when_set() + { + var src = Routes(new ApiSettings(new ApiPagination(2, 4), UnknownFieldsMode.Drop)); + Assert.DoesNotContain("UnknownFieldCheck", src); + Assert.DoesNotContain("KnownFields", src); + Assert.Contains("limit_too_large", src); + Assert.Contains("var __take = 2;", src); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/Issue203AutoSetStampingTests.cs b/server/csharp/MetaObjects.Codegen.Tests/Issue203AutoSetStampingTests.cs index 36e63f9b65..29b7ead001 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/Issue203AutoSetStampingTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/Issue203AutoSetStampingTests.cs @@ -132,8 +132,10 @@ public void Create_required_key_presence_check_excludes_autoSet_fields() Assert.Contains("foreach (var __req in new[] { \"title\" })", src); // A @required @autoSet field is server-filled → OPTIONAL on the wire (not a // required key), so an omitted createdAt/updatedAt is NOT a 400. - Assert.DoesNotContain("\"createdAt\"", src); - Assert.DoesNotContain("\"updatedAt\"", src); + // (The declared-name set of api.unknownFields lists every field, so look at the required-key line only.) + var reqLine = src.Split('\n').Single(l => l.Contains("foreach (var __req in new[]")); + Assert.DoesNotContain("\"createdAt\"", reqLine); + Assert.DoesNotContain("\"updatedAt\"", reqLine); } [Fact] diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 27177afb38..b589da783c 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -332,6 +332,7 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte sb.AppendLine(" };"); sb.AppendLine(); AppendSortDefaultOrder(sb, entity, "SortDefaultDesc"); + if (writable) AppendKnownFields(sb, [entity], ctx.Root); AppendUnauthenticatedRoutesDoc(sb, cls, isProjection); sb.AppendLine($" public static IEndpointRouteBuilder Map{cls}Routes(this IEndpointRouteBuilder app, string prefix = \"/api\")"); sb.AppendLine(" {"); @@ -384,10 +385,7 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte else { sb.AppendLine(" // Pagination: ?offset=N first (skip), then ?limit=N (take)."); - sb.AppendLine(" if (qs.TryGetValue(\"offset\", out var offsetRaw) && int.TryParse(offsetRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var offset) && offset > 0)"); - sb.AppendLine(" q = q.Skip(offset);"); - sb.AppendLine(" if (qs.TryGetValue(\"limit\", out var limitRaw) && int.TryParse(limitRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var limit) && limit > 0)"); - sb.AppendLine(" q = q.Take(limit);"); + AppendOffsetLimit(sb, ctx.Root.Api.Pagination, " ", "offsetRaw", "offset", "limitRaw", "limit"); sb.AppendLine(); sb.AppendLine(" var rows = await q.ToListAsync();"); } @@ -430,7 +428,7 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte CreatedResult("prefix + \"/" + route + "\"", "input", credentials), voFields, mapVoFields, autoSetFields, writeThrough ? (readDbSet, pkProp!, route) : null, deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials, - ServerOwnedCreateResets(entity, ctx.Root, _ => { })); + ServerOwnedCreateResets(entity, ctx.Root, _ => { }), ctx.Root); // PATCH + PUT share the same handler — TS reference exposes both verbs, and both // return the updated row (HTTP 200), matching the cross-port api-contract. @@ -458,6 +456,7 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte sb.AppendLine(" catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); }"); sb.AppendLine(" using var bodyScope = body;"); sb.AppendLine(" if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable();"); + AppendUnknownFieldsCheck(sb, ctx.Root, "body"); sb.AppendLine(" // Deserialize each value with the app's CONFIGURED JSON options, not defaults —"); sb.AppendLine(" // a field.timestamp binds to DateTimeOffset via a custom converter registered on"); sb.AppendLine(" // ConfigureHttpJsonOptions, and default options cannot read the wire format, throwing 500."); @@ -645,6 +644,8 @@ protected virtual EmittedFile GenerateTphRoutes(MetaObject baseEntity, TphPlan t sb.AppendLine(" };"); sb.AppendLine(); AppendSortDefaultOrder(sb, baseEntity, "SortDefaultDesc"); + // One known set for the whole hierarchy: a sibling subtype's column is declared, so it is not unknown. + AppendKnownFields(sb, tph.Subtypes.Select(st => st.Entity).Prepend(baseEntity), ctx.Root); // Per-subtype sort allowlists (case-insensitive) over each subtype's EFFECTIVE // scalar fields (inherited base + own). The per-subtype list resolves the qs @@ -689,8 +690,7 @@ protected virtual EmittedFile GenerateTphRoutes(MetaObject baseEntity, TphPlan t sb.AppendLine(" : SortDefaultDesc.TryGetValue(resolved, out var dd) && dd;"); sb.AppendLine(" q = ApplySort(q, resolved, desc);"); sb.AppendLine(" }"); - sb.AppendLine(" if (qs.TryGetValue(\"offset\", out var offRaw) && int.TryParse(offRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var off) && off > 0) q = q.Skip(off);"); - sb.AppendLine(" if (qs.TryGetValue(\"limit\", out var limRaw) && int.TryParse(limRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var lim) && lim > 0) q = q.Take(lim);"); + AppendOffsetLimit(sb, ctx.Root.Api.Pagination, " ", "offRaw", "off", "limRaw", "lim"); sb.AppendLine(" var rows = await q.ToListAsync();"); sb.AppendLine(" var withCount = qs.TryGetValue(\"withCount\", out var wc) && (wc == \"1\" || string.Equals(wc, \"true\", System.StringComparison.OrdinalIgnoreCase));"); sb.AppendLine($" if (!withCount) return {OkResult("rows", baseCredentials)};"); @@ -993,6 +993,52 @@ private static List ValueObjectFields(MetaObject entity, MetaRoot root, // (minimal-API binding + DataAnnotations) — byte-identical to the pre-FR-036 output. // A missing key, an explicit null on a @required key, or a DataAnnotations violation all // return the cross-port {error:"validation"} envelope BEFORE the row is persisted. + // `api.unknownFields` — the declared wire names a body key is checked against, as a static set in the + // routes class. Null under `drop` (nothing is emitted and no handler checks): the constant's name otherwise. + private const string KnownFieldsConst = "KnownFields"; + + private static void AppendKnownFields(StringBuilder sb, IEnumerable objects, MetaRoot root) + { + if (root.Api.UnknownFields != UnknownFieldsMode.Reject) return; + var names = objects.SelectMany(o => o.Fields()).Select(f => f.Name) + .Distinct(StringComparer.Ordinal).OrderBy(n => n, StringComparer.Ordinal).ToList(); + sb.AppendLine(" // api.unknownFields: reject — the declared field names (wire spelling); any other body key is a 400."); + sb.AppendLine(" private static readonly System.Collections.Generic.HashSet " + KnownFieldsConst + " = new(System.StringComparer.Ordinal)"); + sb.AppendLine(" {"); + foreach (var n in names) sb.AppendLine($" \"{n}\","); + sb.AppendLine(" };"); + sb.AppendLine(); + } + + // The unknown-field refusal, before any validation of the body. `bodyVar` is the parsed JsonDocument. + private static void AppendUnknownFieldsCheck(StringBuilder sb, MetaRoot root, string bodyVar) + { + if (root.Api.UnknownFields != UnknownFieldsMode.Reject) return; + sb.AppendLine($" if (UnknownFieldCheck.Reject({bodyVar}.RootElement, {KnownFieldsConst}) is {{ }} __unknown) return __unknown;"); + } + + // The offset + limit lines of a list route. Without `api.pagination` they are exactly what they always + // were; with it, no limit takes defaultLimit and one above maxLimit is 400 limit_too_large. + private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, string indent, string offRaw, string off, string limRaw, string lim) + { + sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}) && int.TryParse({offRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {off}) && {off} > 0)"); + sb.AppendLine($"{indent} q = q.Skip({off});"); + if (page is null) + { + sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && int.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} > 0)"); + sb.AppendLine($"{indent} q = q.Take({lim});"); + return; + } + sb.AppendLine($"{indent}// api.pagination: no limit takes defaultLimit; a limit above maxLimit is refused."); + sb.AppendLine($"{indent}var __take = {page.DefaultLimit};"); + sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && long.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} > 0)"); + sb.AppendLine($"{indent}{{"); + sb.AppendLine($"{indent} if ({lim} > {page.MaxLimit}) return Results.BadRequest(new {{ error = \"limit_too_large\", maxLimit = {page.MaxLimit} }});"); + sb.AppendLine($"{indent} __take = (int){lim};"); + sb.AppendLine($"{indent}}}"); + sb.AppendLine($"{indent}q = q.Take(__take);"); + } + private static void AppendCreateHandler( StringBuilder sb, string mapPath, string cls, string dbSet, IReadOnlyList requiredKeys, string createdExpr, IReadOnlyList voFields, @@ -1001,7 +1047,8 @@ private static void AppendCreateHandler( (string ReadDbSet, string PkProp, string Route)? reRead = null, IReadOnlyList? deniedNavs = null, IReadOnlyList? credentials = null, - IReadOnlyList? serverOwned = null) + IReadOnlyList? serverOwned = null, + MetaRoot? root = null) { // The body is always read here, not model-bound: the framework answers a body it cannot bind with a // bare 400 and no body, while every validation failure must list its issues (the cross-port @@ -1015,6 +1062,7 @@ private static void AppendCreateHandler( sb.AppendLine(" using var __bodyScope = __body;"); sb.AppendLine(" if (__body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object)"); sb.AppendLine(" return ValidationIssues.Unreadable();"); + if (root is not null) AppendUnknownFieldsCheck(sb, root, "__body"); sb.AppendLine(" var __issues = new ValidationIssues();"); // FR-036 #4 — @required-key presence: an omitted key OR an explicit null on a @required // field → 400, matching the JVM/Python "missing @required → 400" contract. Case-insensitive @@ -1514,8 +1562,7 @@ private static void AppendTphSubtypeRoutes( sb.AppendLine($" : {subCls}SortDefaultDesc.TryGetValue(resolved, out var dd) && dd;"); sb.AppendLine(" q = " + NullsLargestSort("q", "EF.Property(x!, resolved)") + ";"); sb.AppendLine(" }"); - sb.AppendLine(" if (qs.TryGetValue(\"offset\", out var offRaw) && int.TryParse(offRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var off) && off > 0) q = q.Skip(off);"); - sb.AppendLine(" if (qs.TryGetValue(\"limit\", out var limRaw) && int.TryParse(limRaw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var lim) && lim > 0) q = q.Take(lim);"); + AppendOffsetLimit(sb, root.Api.Pagination, " ", "offRaw", "off", "limRaw", "lim"); sb.AppendLine(" var rows = await q.ToListAsync();"); sb.AppendLine(" var withCount = qs.TryGetValue(\"withCount\", out var wc) && (wc == \"1\" || string.Equals(wc, \"true\", System.StringComparison.OrdinalIgnoreCase));"); sb.AppendLine($" if (!withCount) return {OkResult("rows", credentials)};"); @@ -1545,7 +1592,7 @@ private static void AppendTphSubtypeRoutes( AppendCreateHandler(sb, "/" + subRoute, subCls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + subRoute + "/\" + input." + pkProp, "input", credentials), [], mapVoFields, autoSetFields, deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials, - serverOwned: ServerOwnedCreateResets(st.Entity, root, _ => { }, discProp, hierarchy[0])); + serverOwned: ServerOwnedCreateResets(st.Entity, root, _ => { }, discProp, hierarchy[0]), root: root); // Per-subtype update (PATCH + PUT): partial merge of the JSON body onto the scoped // entity, NEVER touching the PK or the discriminator. Cross-subtype id → 404. @@ -1559,6 +1606,7 @@ private static void AppendTphSubtypeRoutes( sb.AppendLine(" catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); }"); sb.AppendLine(" using var bodyScope = body;"); sb.AppendLine(" if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable();"); + AppendUnknownFieldsCheck(sb, root, "body"); sb.AppendLine(" // Configured JSON options (not defaults) so a custom converter — e.g. the"); sb.AppendLine(" // field.timestamp DateTimeOffset converter — is honored; defaults throw 500 on it."); sb.AppendLine(" var jsonOpts = ((Microsoft.Extensions.Options.IOptions)"); diff --git a/server/csharp/MetaObjects.Codegen/HelperRuntime.cs b/server/csharp/MetaObjects.Codegen/HelperRuntime.cs index b2ed40726d..c177184f0c 100644 --- a/server/csharp/MetaObjects.Codegen/HelperRuntime.cs +++ b/server/csharp/MetaObjects.Codegen/HelperRuntime.cs @@ -70,6 +70,7 @@ public static class HelperRuntime "ValueObjectValidator.cs", "ConstraintErrors.cs", "ValidationIssues.cs", + "UnknownFieldCheck.cs", "Iso8601TimestampConverter.cs", "DecimalWire.cs", "DurationWire.cs", diff --git a/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj b/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj index d9b2c3965b..d1de31cf5c 100644 --- a/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj +++ b/server/csharp/MetaObjects.Codegen/MetaObjects.Codegen.csproj @@ -87,6 +87,7 @@ + diff --git a/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs new file mode 100644 index 0000000000..5f03a259f2 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs @@ -0,0 +1,31 @@ +// UnknownFieldCheck — the `api.unknownFields: reject` answer of a generated create, PUT or PATCH route. +// +// 400 { "error": "unknown_fields", "fields": [ "isAdmin", "nickname" ] } +// +// The top-level keys of the JSON object body that the entity does not declare (for a single-table +// hierarchy: that no field of the base or any subtype declares), sorted by ordinal order, listed once +// each. A declared key the client may not set (the generated key, a readOnly field, a frozen field) is +// not unknown: the route drops it as it always did, so a client that echoes back a row it read works. +// Under `api.unknownFields: drop` the route never calls this. + +using System.Text.Json; +using Microsoft.AspNetCore.Http; + +namespace MetaObjects.Codegen.Runtime; + +/// The unknown-field refusal of a generated write route. +public static class UnknownFieldCheck +{ + /// The 400 naming every body key not in , or null when there is none + /// (or the body is not a JSON object, which the route's own handling answers). + public static IResult? Reject(JsonElement body, IReadOnlySet known) + { + if (body.ValueKind != JsonValueKind.Object) return null; + var unknown = new List(); + foreach (var property in body.EnumerateObject()) + if (!known.Contains(property.Name) && !unknown.Contains(property.Name)) unknown.Add(property.Name); + if (unknown.Count == 0) return null; + unknown.Sort(StringComparer.Ordinal); + return Results.BadRequest(new { error = "unknown_fields", fields = unknown }); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs new file mode 100644 index 0000000000..814a215bd6 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs @@ -0,0 +1,93 @@ +// ApiContractApiSettingsConformanceTest — the C# #484 secret-fields api-contract lane. +// +// Drives the fixtures/api-contract-conformance/secret-fields/ scenarios over HTTP against the +// GENERATED Account routes booted unmodified on Kestrel against a Postgres testcontainer. +// Proves the deployed REST surface ignores writes to @formExclude / credential fields and +// never returns a credential. Generated lane only. + +using System.Text.Json; +using System.Text.Json.Nodes; +using MetaObjects.IntegrationTests.Runner; +using Xunit; + +namespace MetaObjects.IntegrationTests.Api; + +public sealed class ApiContractApiSettingsConformanceTest +{ + [Theory] + [MemberData(nameof(Scenarios))] + public async Task Api_contract_api_settings_generated(string scenarioPath) + { + var scenario = ApiContractScenarioLoader.LoadScenario(scenarioPath); + await using var pg = await PostgresContainer.StartAsync(); + await using var server = await ApiSettingsGeneratedServerFactory.StartAsync(pg); + await server.ApplySeedAsync(); + await RunAsync(scenario, server.BaseUrl); + await AssertNoUndeclaredStateAsync(pg.ConnectionString); + } + + // The undeclared key a scenario sends is never stored: the table has no column for it, and no + // scenario row ends up without its title. + private static async Task AssertNoUndeclaredStateAsync(string connString) + { + await using var c = new Npgsql.NpgsqlConnection(connString); + await c.OpenAsync(); + await using var cmd = c.CreateCommand(); + cmd.CommandText = "SELECT COUNT(*) FROM \"items\" WHERE \"title\" IS NULL"; + Assert.Equal(0L, (long)(await cmd.ExecuteScalarAsync())!); + } + + private static async Task RunAsync(ApiScenario scenario, string baseUrl) + { + using var client = new HttpClient { BaseAddress = new Uri(baseUrl) }; + foreach (var req in scenario.Requests) + { + var request = new HttpRequestMessage(new HttpMethod(req.Method), ApiContractWire.VerbatimUri(client, req.Path)); + if (req.Body is not null) + { + string json = JsonSerializer.Serialize(req.Body, JsonOpts); + request.Content = new StringContent(json, System.Text.Encoding.UTF8, "application/json"); + } + var response = await client.SendAsync(request); + string bodyText = await response.Content.ReadAsStringAsync(); + object? parsed = string.IsNullOrEmpty(bodyText) ? null : ToObject(JsonNode.Parse(bodyText)); + ApiContractAssertions.AssertResponse(scenario.Name, req, (int)response.StatusCode, parsed); + } + } + + public static IEnumerable Scenarios() => + Directory.EnumerateFiles(ApiContractCorpusPaths.ApiSettingsScenariosDir, "*.yaml", SearchOption.TopDirectoryOnly) + .OrderBy(p => p, StringComparer.Ordinal) + .Select(p => new object[] { p }); + + private static readonly JsonSerializerOptions JsonOpts = new() + { + DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.Never, + }; + + private static object? ToObject(JsonNode? node) + { + if (node is null) return null; + if (node is JsonObject obj) + { + var d = new Dictionary(StringComparer.Ordinal); + foreach (var kvp in obj) d[kvp.Key] = ToObject(kvp.Value); + return d; + } + if (node is JsonArray arr) + { + var l = new List(arr.Count); + foreach (var item in arr) l.Add(ToObject(item)); + return l; + } + if (node is JsonValue jv) + { + if (jv.TryGetValue(out var b)) return b; + if (jv.TryGetValue(out var lv)) return lv; + if (jv.TryGetValue(out var dv)) return dv; // exact: never a double (ADR-0019 Amendment 1) + if (jv.TryGetValue(out var sv)) return sv; + return jv.ToString(); + } + return null; + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs index 94e10f9f70..ebc6f44dc9 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs @@ -114,6 +114,12 @@ internal static class ApiContractCorpusPaths public static readonly string SecretFieldsSeedFile = Path.Combine(SecretFieldsDir, "seed.json"); public static readonly string SecretFieldsMetaJson = Path.Combine(SecretFieldsDir, "meta.json"); + // api-settings subcorpus (api.pagination + api.unknownFields from its own config), generated lane only. + public static readonly string ApiSettingsDir = Path.Combine(Corpus, "api-settings"); + public static readonly string ApiSettingsScenariosDir = Path.Combine(ApiSettingsDir, "scenarios"); + public static readonly string ApiSettingsSeedFile = Path.Combine(ApiSettingsDir, "seed.json"); + public static readonly string ApiSettingsMetaJson = Path.Combine(ApiSettingsDir, "meta.json"); + // enum-default subcorpus (a defaulted enum on create), generated lane only. public static readonly string EnumDefaultDir = Path.Combine(Corpus, "enum-default"); public static readonly string EnumDefaultScenariosDir = Path.Combine(EnumDefaultDir, "scenarios"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsFixture.cs new file mode 100644 index 0000000000..4124bd44a7 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsFixture.cs @@ -0,0 +1,45 @@ +// ApiSettingsFixture — schema + seed for the api-settings api-contract GENERATED lane. +using System.Text.Json.Nodes; +using Npgsql; + +namespace MetaObjects.IntegrationTests.Api; + +internal static class ApiSettingsFixture +{ + public const string SchemaDdl = """ + CREATE TABLE "items" ( + "id" BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL, + "title" VARCHAR(100) NOT NULL, + CONSTRAINT "items_pkey" PRIMARY KEY ("id") + ); + """; + + public static async Task ProvisionSchemaAsync(string connString) + { + await using var c = new NpgsqlConnection(connString); + await c.OpenAsync(); + await using var cmd = c.CreateCommand(); + cmd.CommandText = SchemaDdl; + await cmd.ExecuteNonQueryAsync(); + } + + public static async Task ApplySeedAsync(string connString, string seedPath) + { + var root = JsonNode.Parse(File.ReadAllText(seedPath)) as JsonObject + ?? throw new InvalidOperationException($"{seedPath}: top-level must be an object"); + await using var c = new NpgsqlConnection(connString); + await c.OpenAsync(); + foreach (var rowNode in root["rows"]!.AsArray()) + { + var row = rowNode!.AsObject(); + await using var ins = c.CreateCommand(); + ins.CommandText = "INSERT INTO \"items\" (\"id\", \"title\") VALUES (@id, @title)"; + ins.Parameters.AddWithValue("@id", row["id"]!.GetValue()); + ins.Parameters.AddWithValue("@title", row["title"]!.GetValue()); + await ins.ExecuteNonQueryAsync(); + } + await using var bump = c.CreateCommand(); + bump.CommandText = "SELECT setval(pg_get_serial_sequence('items', 'id'), COALESCE((SELECT MAX(id) FROM \"items\"), 1))"; + await bump.ExecuteScalarAsync(); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs new file mode 100644 index 0000000000..0eeb51440a --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs @@ -0,0 +1,185 @@ +// ApiSettingsGeneratedServerFactory — the C# GENERATED-server lane for the #484 api-settings +// corpus. Runs the real Codegen generators (Entity + DbContext + FilterAllowlist + Routes) on the +// Account model, Roslyn-compiles the output in memory and hosts it on Kestrel against Postgres. + +using MetaObjects.Meta; +using System.Reflection; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Hosting; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; +using MetaObjects.Codegen; +using MetaObjects.Codegen.Generators; +using MetaObjects.IntegrationTests.Runner; +using MetaObjects.Loader; + +namespace MetaObjects.IntegrationTests.Api; + +internal sealed class ApiSettingsGeneratedServerFactory : IAsyncDisposable +{ + private const string GeneratedNamespace = "MetaObjects.ApiContract.ApiSettingsGenerated"; + + private readonly PostgresContainer _pg; + private readonly WebApplication _app; + + public string BaseUrl { get; } + + private ApiSettingsGeneratedServerFactory(PostgresContainer pg, WebApplication app, string baseUrl) + { + _pg = pg; + _app = app; + BaseUrl = baseUrl; + } + + public static async Task StartAsync(PostgresContainer pg) + { + await ApiSettingsFixture.ProvisionSchemaAsync(pg.ConnectionString); + + var (assembly, entityNames) = CompileGeneratedServer(); + var dbContextType = assembly.GetType($"{GeneratedNamespace}.AppDbContext") + ?? throw new InvalidOperationException("generated AppDbContext type not found"); + + var builder = WebApplication.CreateBuilder(); + LoopbackKestrel.BindAnyPort(builder); + builder.Logging.ClearProviders(); + RegisterGeneratedDbContext(builder.Services, dbContextType, pg.ConnectionString); + + var app = builder.Build(); + + // Mount each generated MapRoutes(app, "/api"). Both Customer and Order + // emit a routes class; only Order's api-settings routes are exercised. + foreach (var entity in entityNames) + { + var routesType = assembly.GetType($"{GeneratedNamespace}.{entity}Routes"); + if (routesType is null) continue; + var mapMethod = routesType.GetMethod($"Map{entity}Routes", BindingFlags.Public | BindingFlags.Static); + mapMethod?.Invoke(null, new object[] { app, "/api" }); + } + + await app.StartAsync(); + return new ApiSettingsGeneratedServerFactory(pg, app, LoopbackKestrel.BoundUrl(app)); + } + + public async Task ApplySeedAsync() => + await ApiSettingsFixture.ApplySeedAsync(_pg.ConnectionString, ApiContractCorpusPaths.ApiSettingsSeedFile); + + public async ValueTask DisposeAsync() + { + try { await _app.StopAsync(); } catch { /* ignored */ } + try { await _app.DisposeAsync(); } catch { /* ignored */ } + } + + private static (Assembly Assembly, IReadOnlyList EntityNames) CompileGeneratedServer() + { + var loadResult = new MetaDataLoader().Load([new FileSource(ApiContractCorpusPaths.ApiSettingsMetaJson)]); + // The sub-corpus's OWN .metaobjects/config.json (naming, wire, api.pagination, api.unknownFields), + // read the way an adopter's project reaches the generator: through the config the load reads. + var cfg = MetaObjects.Config.NeutralConfig.Read(ApiContractCorpusPaths.ApiSettingsDir) + ?? throw new InvalidOperationException("api-settings corpus has no .metaobjects/config.json"); + loadResult.Root.PhysicalNaming = cfg.Naming; + loadResult.Root.WireFormat = cfg.Wire; + loadResult.Root.Api = cfg.Api; + if (loadResult.Errors.Count != 0) + throw new InvalidOperationException( + "api-settings corpus metadata failed to load: " + + string.Join("; ", loadResult.Errors.Select(e => e.ToString()))); + + var entityNames = loadResult.Root.Objects() + .Where(o => o.IsEntity()) + .Select(o => CSharpNaming.Pascal(o.Name)) + .ToList(); + + var ctx = new GenContext + { + Entities = loadResult.Root.Objects(), + Root = loadResult.Root, + Config = new GenConfig + { + OutDir = "/unused", + Namespace = GeneratedNamespace, + ColumnNamingStrategy = cfg.Naming.Columns, + EmitAbstractShapes = false, + // GenConfig.IncludeNames defaults to false (§A6 task 4); without it + // here the entity/DbContext output below never references the + // Names constants, and the .Concat(new NamesGenerator()...) + // a few lines down is dead input the compile never depends on. This + // is the only lane that proves an Names.NAME reference + // actually RESOLVES against a real EF Core compile. + IncludeNames = true, + }, + }; + + // §A6 (task 4) -- the entity/DbContext output now references the names artifact. + var files = new EntityGenerator().Generate(ctx) + .Concat(new DbContextGenerator().Generate(ctx)) + .Concat(new FilterAllowlistGenerator().Generate(ctx)) + .Concat(new RoutesGenerator().Generate(ctx)) + .Concat(new NamesGenerator().Generate(ctx)) + .ToList(); + + var trees = files + .Select(f => CSharpSyntaxTree.ParseText(f.Content, new CSharpParseOptions(LanguageVersion.CSharp12))) + .ToArray(); + + var refs = BuildReferenceSet(); + var comp = CSharpCompilation.Create( + "apicontract_apisettings_generated_" + Guid.NewGuid().ToString("N"), + trees, refs, + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + + using var ms = new MemoryStream(); + var emit = comp.Emit(ms); + if (!emit.Success) + { + var errors = emit.Diagnostics + .Where(d => d.Severity == DiagnosticSeverity.Error) + .Select(d => $"{d.Id}: {d.GetMessage()}") + .ToList(); + throw new InvalidOperationException( + "generated api-settings server failed to compile:\n " + string.Join("\n ", errors)); + } + + ms.Seek(0, SeekOrigin.Begin); + return (Assembly.Load(ms.ToArray()), entityNames); + } + + private static List BuildReferenceSet() + { + var byFileName = new Dictionary(StringComparer.OrdinalIgnoreCase); + var tpa = (string?)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES") ?? ""; + foreach (var path in tpa.Split(Path.PathSeparator)) + if (path.Length > 0 && path.EndsWith(".dll", StringComparison.OrdinalIgnoreCase)) + byFileName[Path.GetFileName(path)] = path; + + var aspNetDir = Path.GetDirectoryName(typeof(WebApplication).Assembly.Location); + if (aspNetDir is not null && Directory.Exists(aspNetDir)) + foreach (var dll in Directory.EnumerateFiles(aspNetDir, "*.dll")) + byFileName[Path.GetFileName(dll)] = dll; + + return byFileName.Values + .Select(loc => (MetadataReference)MetadataReference.CreateFromFile(loc)) + .ToList(); + } + + private static void RegisterGeneratedDbContext( + IServiceCollection services, Type dbContextType, string connString) + { + var addDbContext = typeof(EntityFrameworkServiceCollectionExtensions) + .GetMethods(BindingFlags.Public | BindingFlags.Static) + .First(m => m.Name == "AddDbContext" + && m.IsGenericMethodDefinition + && m.GetGenericArguments().Length == 1 + && m.GetParameters().Length == 4) + .MakeGenericMethod(dbContextType); + + Action configure = opts => opts.UseNpgsql(connString); + addDbContext.Invoke(null, new object?[] + { + services, configure, ServiceLifetime.Scoped, ServiceLifetime.Scoped, + }); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs index 08a0cc4b13..6ec0046bc3 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/AuthorApiServer.cs @@ -514,6 +514,9 @@ private static void AddAuthorMaxLengthIssues(Dictionary body, L if (body.GetValueOrDefault("bio") is string b && b.Length > 1000) issues.Add(ReferenceValidation.TooLong("bio", 1000)); } + private static readonly HashSet DeclaredKeys = new(StringComparer.Ordinal) + { "id", "name", "bio", "createdAt", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt" }; + private async Task CreateAuthorAsync(HttpContext ctx) { var parsed = await ReadJsonBodyAsync(ctx); @@ -522,6 +525,11 @@ private async Task CreateAuthorAsync(HttpContext ctx) await SendJsonAsync(ctx, 400, ReferenceValidation.Unreadable()); return; } + if (ReferenceValidation.UnknownFields(body.Keys, DeclaredKeys) is { } unknownFields) + { + await SendJsonAsync(ctx, 400, unknownFields); + return; + } // FR-036 — mirror the corpus Author @maxLength bounds so this lane rejects an // over-length string with the same {error:"validation"} envelope the generated // lane's [MaxLength] produces (otherwise a too-long name would 500 on the PG write). @@ -579,6 +587,11 @@ private async Task UpdateAuthorAsync(HttpContext ctx, string idStr) await SendJsonAsync(ctx, 400, ReferenceValidation.Unreadable()); return; } + if (ReferenceValidation.UnknownFields(body.Keys, DeclaredKeys) is { } unknownFields) + { + await SendJsonAsync(ctx, 400, unknownFields); + return; + } // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an // omitted required field is untouched (never a 400). diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs index 37baf6d90e..6f3bafc758 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReferenceValidation.cs @@ -37,4 +37,16 @@ internal static class ReferenceValidation /// A present string over its max length. public static Dictionary TooLong(string field, int max) => Issue("too_big", field, $"{field} must be at most {max} characters"); + + /// + /// api.unknownFields: reject (the default, which the core corpus does not change): the body keys the + /// entity does not declare, sorted ordinally, as the 400 body, or null when there are none. + /// + public static Dictionary? UnknownFields(IEnumerable bodyKeys, ISet declared) + { + var unknown = bodyKeys.Where(k => !declared.Contains(k)).Distinct(StringComparer.Ordinal).ToList(); + if (unknown.Count == 0) return null; + unknown.Sort(StringComparer.Ordinal); + return new Dictionary { ["error"] = "unknown_fields", ["fields"] = unknown.Cast().ToList() }; + } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs index 615580a864..3b8b35b667 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs @@ -30,6 +30,11 @@ namespace MetaObjects.IntegrationTests.Api; internal sealed class TphReferenceServer : IAsyncDisposable { + // api.unknownFields: reject (the default, unchanged by this corpus): the union of the base's and every + // subtype's declared fields, so a sibling subtype's column is known on another subtype's route. + private static readonly HashSet DeclaredKeys = new(StringComparer.Ordinal) + { "id", "type", "reference", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt", "quantity", "copayAmount", "approver" }; + // segment (lowercased @discriminatorValue) → discriminator value. private static readonly Dictionary SegmentToValue = new(StringComparer.Ordinal) { @@ -140,6 +145,11 @@ private async Task DispatchAsync(HttpContext ctx) if (method == "POST") { var body = await ReadBodyAsync(ctx); + if (ReferenceValidation.UnknownFields(body.Select(kv => kv.Key), DeclaredKeys) is { } unknownFields) + { + await SendJsonAsync(ctx, 400, unknownFields); + return; + } // FR-036 — the per-subtype CREATE validates every constraint, matching the generated // controller's Dto DataAnnotations: the @required keys (the base `reference`, the // BridgeAuth `quantity`), the @maxLength:80 columns and copayAmount's decimal(10,2). @@ -170,6 +180,11 @@ private async Task DispatchAsync(HttpContext ctx) if (method is "PATCH" or "PUT") { var body = await ReadBodyAsync(ctx); + if (ReferenceValidation.UnknownFields(body.Select(kv => kv.Key), DeclaredKeys) is { } unknownFields) + { + await SendJsonAsync(ctx, 400, unknownFields); + return; + } // FR-036 Program B: a present null on the @required (NOT NULL) base // `reference` column is a 400, matching the generated controller's // not-null merge guard. In the single TPH table only base-required diff --git a/server/csharp/MetaObjects/Config/MetadataLocation.cs b/server/csharp/MetaObjects/Config/MetadataLocation.cs index df094900fc..21a63df756 100644 --- a/server/csharp/MetaObjects/Config/MetadataLocation.cs +++ b/server/csharp/MetaObjects/Config/MetadataLocation.cs @@ -58,6 +58,9 @@ public sealed record ResolvedMetadata( /// constructor and Deconstruct keep the signatures they had. public EnumDefaults? Enums { get; init; } + /// The project's api block, set on the loaded root by . + public ApiSettings? Api { get; init; } + /// Load this metadata, with the project's libraries, and set the loaded /// root's from (ADR-0058) and its /// from . @@ -71,6 +74,7 @@ public LoadResult Load(bool strict = false) Directory, Registry(), Libraries, leading, strict: strict); result.Root.PhysicalNaming = Naming; result.Root.WireFormat = Wire ?? WireFormat.Default; + result.Root.Api = Api ?? ApiSettings.Default; result.Root.EnumDefaults = Enums ?? EnumDefaults.Default; if (Imports is null) return result; @@ -150,7 +154,7 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd, bool val // No config found means no project to read a lock from: nothing is imported. var explicitImports = explicitCfg is null ? ImportScope.None : ImportScope.Resolve(explicitCfg.ConfigDir, explicitCfg); return new ResolvedMetadata( - metadataDir, null, explicitLibs, explicitNaming, explicitCfg?.Wire, explicitCfg?.OutDir, explicitImports, explicitCfg?.Namespace, LoadProviders(explicitCfg)) { Enums = explicitCfg?.Enums }; + metadataDir, null, explicitLibs, explicitNaming, explicitCfg?.Wire, explicitCfg?.OutDir, explicitImports, explicitCfg?.Namespace, LoadProviders(explicitCfg)) { Enums = explicitCfg?.Enums, Api = explicitCfg?.Api }; } var cfg = NeutralConfig.Read(cwd, validateNaming); @@ -165,7 +169,7 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd, bool val // No declared sources: resolve the DEFAULT directory through the same ladder // the shared conformance corpus gates (ERR_COLLECTION_NOT_FOUND when absent). var defaultFiles = SourceResolver.ResolveCollection(cwd); - return new ResolvedMetadata(Path.Combine(cwd, NeutralConfig.DefaultMetadataDir), defaultFiles, libraries, naming, wire, cfg?.OutDir, imports, cfg?.Namespace, LoadProviders(cfg)) { Enums = cfg?.Enums }; + return new ResolvedMetadata(Path.Combine(cwd, NeutralConfig.DefaultMetadataDir), defaultFiles, libraries, naming, wire, cfg?.OutDir, imports, cfg?.Namespace, LoadProviders(cfg)) { Enums = cfg?.Enums, Api = cfg?.Api }; } if (specs.Count > 1) @@ -195,6 +199,6 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd, bool val "\"sources\" at the file's containing directory."); } - return new ResolvedMetadata(resolved, files, libraries, naming, wire, cfg?.OutDir, imports, cfg?.Namespace, LoadProviders(cfg)) { Enums = cfg?.Enums }; + return new ResolvedMetadata(resolved, files, libraries, naming, wire, cfg?.OutDir, imports, cfg?.Namespace, LoadProviders(cfg)) { Enums = cfg?.Enums, Api = cfg?.Api }; } } diff --git a/server/csharp/MetaObjects/Config/NeutralConfig.cs b/server/csharp/MetaObjects/Config/NeutralConfig.cs index d80ba04fec..777a918924 100644 --- a/server/csharp/MetaObjects/Config/NeutralConfig.cs +++ b/server/csharp/MetaObjects/Config/NeutralConfig.cs @@ -96,6 +96,9 @@ public sealed class NeutralConfig /// absent key at its native default. Never null. public WireFormat Wire { get; } + /// The project's `api` block (`api.pagination`, `api.unknownFields`). Never null: absent is the default. + public ApiSettings Api { get; private init; } = ApiSettings.Default; + /// The project's enum storage defaults (`enums` in config.json). Never null: an absent /// block is no default, same as an empty `enums: {}`. public EnumDefaults Enums { get; private init; } = EnumDefaults.Default; @@ -346,6 +349,7 @@ private NeutralConfig( Path.GetFullPath(configDir)) { Enums = ReadEnums(root, path), + Api = ReadApi(root, path), RequirementsInherit = ReadRequirementsInherit(root, path), MigrateDialect = ReadMigrateDialect(root), }; @@ -556,4 +560,71 @@ private static WireFormat ReadWire(JsonElement root, string path) } return new WireFormat(decimals, nullMembers, durations); } + + /// The `api` block: `pagination` takes both `defaultLimit` and `maxLimit` as positive integers with + /// `defaultLimit <= maxLimit`; `unknownFields` is "reject" (the default) or "drop". Anything else is refused. + private static ApiSettings ReadApi(JsonElement root, string path) + { + if (!root.TryGetProperty("api", out var api)) return ApiSettings.Default; + if (api.ValueKind != JsonValueKind.Object) + throw new MetaModelException($"{path}: \"api\" must be an object", ErrorCode.ERR_BAD_ATTR_VALUE); + + ApiPagination? pagination = null; + var unknown = UnknownFieldsMode.Reject; + foreach (var p in api.EnumerateObject()) + { + switch (p.Name) + { + case "pagination": + pagination = ReadPagination(p.Value, path); + break; + case "unknownFields": + unknown = (p.Value.ValueKind == JsonValueKind.String ? p.Value.GetString() : null) switch + { + "reject" => UnknownFieldsMode.Reject, + "drop" => UnknownFieldsMode.Drop, + _ => throw new MetaModelException( + $"{path}: \"api.unknownFields\" must be \"reject\" or \"drop\"", ErrorCode.ERR_BAD_ATTR_VALUE), + }; + break; + default: + throw new MetaModelException( + $"{path}: \"api\" has an unknown key \"{p.Name}\" (expected \"pagination\" and/or \"unknownFields\")", + ErrorCode.ERR_BAD_ATTR_VALUE); + } + } + return new ApiSettings(pagination, unknown); + } + + private static ApiPagination ReadPagination(JsonElement block, string path) + { + if (block.ValueKind != JsonValueKind.Object) + throw new MetaModelException($"{path}: \"api.pagination\" must be an object", ErrorCode.ERR_BAD_ATTR_VALUE); + int? defaultLimit = null, maxLimit = null; + foreach (var p in block.EnumerateObject()) + { + switch (p.Name) + { + case "defaultLimit": defaultLimit = PageSizeNumber(p.Value, "defaultLimit", path); break; + case "maxLimit": maxLimit = PageSizeNumber(p.Value, "maxLimit", path); break; + default: + throw new MetaModelException( + $"{path}: \"api.pagination\" has an unknown key \"{p.Name}\" (expected \"defaultLimit\" and \"maxLimit\")", + ErrorCode.ERR_BAD_ATTR_VALUE); + } + } + if (defaultLimit is null) + throw new MetaModelException($"{path}: \"api.pagination.defaultLimit\" must be a positive integer", ErrorCode.ERR_BAD_ATTR_VALUE); + if (maxLimit is null) + throw new MetaModelException($"{path}: \"api.pagination.maxLimit\" must be a positive integer", ErrorCode.ERR_BAD_ATTR_VALUE); + if (defaultLimit > maxLimit) + throw new MetaModelException( + $"{path}: \"api.pagination.defaultLimit\" must not exceed \"api.pagination.maxLimit\"", ErrorCode.ERR_BAD_ATTR_VALUE); + return new ApiPagination(defaultLimit.Value, maxLimit.Value); + } + + private static int PageSizeNumber(JsonElement v, string key, string path) => + v.ValueKind == JsonValueKind.Number && v.TryGetInt32(out var n) && n >= 1 + ? n + : throw new MetaModelException($"{path}: \"api.pagination.{key}\" must be a positive integer", ErrorCode.ERR_BAD_ATTR_VALUE); } diff --git a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs index 92d48a76ea..7b6d852fc7 100644 --- a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs +++ b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs @@ -171,6 +171,7 @@ internal static LoadResult FromDirectory(string directory, TypeRegistry registry { result.Root.PhysicalNaming = cfg.Naming; result.Root.WireFormat = cfg.Wire; + result.Root.Api = cfg.Api; result.Root.EnumDefaults = cfg.Enums; } return result; diff --git a/server/csharp/MetaObjects/Meta/ApiSettings.cs b/server/csharp/MetaObjects/Meta/ApiSettings.cs new file mode 100644 index 0000000000..ce031748e9 --- /dev/null +++ b/server/csharp/MetaObjects/Meta/ApiSettings.cs @@ -0,0 +1,35 @@ +// API settings — `api` in .metaobjects/config.json, read by every port's codegen: the page size of +// a generated list route (`api.pagination`) and what a generated create/PUT/PATCH route does with a +// body key the entity does not declare (`api.unknownFields`). Carried on the loaded root, like +// WireFormat. + +namespace MetaObjects.Meta; + +/// What a generated write route does with an undeclared body key (api.unknownFields). +public enum UnknownFieldsMode +{ + /// The default: 400 { "error": "unknown_fields", "fields": [...] }, nothing written. + Reject, + /// The behaviour before 1.3: undeclared keys are silently ignored. + Drop, +} + +/// api.pagination: a list route with no limit returns rows, +/// and a limit above is 400 { "error": "limit_too_large", "maxLimit": N }. +public sealed record ApiPagination(int DefaultLimit, int MaxLimit); + +/// The project's api block. A project that sets nothing has no page size and rejects unknown fields. +public sealed record ApiSettings(ApiPagination? Pagination, UnknownFieldsMode UnknownFields = UnknownFieldsMode.Reject) +{ + /// What a tree loaded with no project carries. + public static readonly ApiSettings Default = new(null, UnknownFieldsMode.Reject); + + /// What dotnet meta init writes for a new project. + public const int InitDefaultLimit = 50; + /// What dotnet meta init writes for a new project. + public const int InitMaxLimit = 1000; + + /// The setting the tree belongs to carries, else . + public static ApiSettings Of(MetaData node) => + node.Root() is MetaRoot root ? root.Api : Default; +} diff --git a/server/csharp/MetaObjects/Meta/MetaRoot.cs b/server/csharp/MetaObjects/Meta/MetaRoot.cs index d0235e3f04..84a66e2bf2 100644 --- a/server/csharp/MetaObjects/Meta/MetaRoot.cs +++ b/server/csharp/MetaObjects/Meta/MetaRoot.cs @@ -27,6 +27,10 @@ public class MetaRoot(TypeId typeId, string name) : MetaData(typeId, name) /// public WireFormat WireFormat { get; set; } = WireFormat.Default; + /// The project's api block (api.pagination, api.unknownFields), set by the same + /// project-aware load as and read by codegen through . + public ApiSettings Api { get; set; } = ApiSettings.Default; + /// /// The project's enum storage defaults (enums in .metaobjects/config.json), set by /// the same project-aware load as and read through diff --git a/server/java/codegen-kotlin/pom.xml b/server/java/codegen-kotlin/pom.xml index 3c7f2aecda..348322c687 100644 --- a/server/java/codegen-kotlin/pom.xml +++ b/server/java/codegen-kotlin/pom.xml @@ -260,6 +260,7 @@ KotlinDecimalWireEmitter.kt KotlinDurationWireEmitter.kt KotlinValidationIssuesEmitter.kt + KotlinUnknownBodyKeysEmitter.kt KotlinExtractSchemaEmitter.kt KotlinOutputFormatSpecEmitter.kt KotlinCrossFieldRules.kt diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt index 41b77ed4fb..300a3f9457 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt @@ -136,7 +136,8 @@ private val CROSS_FIELD_EMIT: List = listOf(KotlinCrossFieldRules::class private val DURATION_WIRE_EMIT: List = listOf(KotlinDurationWireEmitter::class.java.name) /** The validation envelope file `routes` writes beside its controllers. */ -private val VALIDATION_ISSUES_EMIT: List = listOf(KotlinValidationIssuesEmitter::class.java.name) +private val VALIDATION_ISSUES_EMIT: List = + listOf(KotlinValidationIssuesEmitter::class.java.name, KotlinUnknownBodyKeysEmitter::class.java.name) /** The rest of `api-docs`: the page IR (one file, `KotlinApiModel.kt`, which also holds the unit, symbol, field * and kind types), the builder that fills it, the renderer and the page paths. */ diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 068fb092f5..818aad1257 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -4,6 +4,7 @@ package com.metaobjects.generator.kotlin // com.metaobjects.generator.kotlin when this generator is copied out via // `mvn metaobjects:eject` and its own package is renamed — an explicit import, not // same-package bare-name resolution, is what keeps the ejected copy compiling. +import com.metaobjects.config.ApiSettings import com.metaobjects.config.WireFormat import com.metaobjects.generator.kotlin.ExposedApi import com.metaobjects.generator.kotlin.ExposedImports @@ -124,6 +125,35 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBaseKnownFields`: every declared field name, as named on the wire (only under `reject`). */ + private fun StringBuilder.appendKnownFields(shortName: String, names: Collection) { + if (!rejectUnknown) return + append("/** GENERATED — every field ${shortName} declares, as named on the wire: a body key outside it is refused. */\n") + append("private val ${shortName}KnownFields: Set = setOf(${names.distinct().joinToString(", ") { "\"$it\"" }})\n\n") + } + + /** The unknown-key check of a write handler (after the non-object-body answer), only under `reject`. */ + private fun StringBuilder.appendUnknownGuard(shortName: String) { + if (!rejectUnknown) return + append(" val unknownKeys = MetaUnknownBodyKeys.refusal(body, ${shortName}KnownFields)\n") + append(" if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any)\n") + } + + /** The refusal of a `limit` above `api.pagination.maxLimit`, at the head of a list handler. */ + private fun StringBuilder.appendLimitGuard() { + val page = apiSettings.pagination() ?: return + append(" if (limit != null && limit > ${page.maxLimit()}L) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"limit_too_large\", \"maxLimit\" to ${page.maxLimit()}L) as Any)\n") + } + + /** The page size a list handler uses when `limit` is omitted. */ + private fun defaultLimit(): String = + apiSettings.pagination()?.defaultLimit()?.coerceAtMost(Int.MAX_VALUE.toLong())?.toString() ?: "50" + override fun setRun(run: Collection) { exportWired = EmitsReportExport.includedIn(run) } @@ -151,6 +181,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") append(" val issues = MetaValidationIssues.Collector()\n") append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + appendUnknownGuard(shortName) append(" val dto: ${shortName} = try {\n") append(" objectMapper.treeToValue(body, ${shortName}::class.java)\n") append(" } catch (e: Exception) {\n") @@ -555,6 +589,17 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(MetaValidationIssues.Collector().add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + appendUnknownGuard(shortName) + append(" val dto: ${shortName} = try {\n") + append(" objectMapper.treeToValue(body, ${shortName}::class.java)\n") + append(" } catch (e: Exception) {\n") + append(" return@transaction ResponseEntity.badRequest().body(MetaDecimalWire.unreadableBody(e) as Any)\n") + append(" }\n") + append(" val issues = MetaValidationIssues.Collector().addViolations(validator.validate(dto))\n") } else { append(" fun create(@RequestBody dto: ${shortName}): ResponseEntity = transaction {\n") append(" val issues = MetaValidationIssues.Collector().addViolations(validator.validate(dto))\n") @@ -626,7 +671,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") - appendPatchHead(this, patchSettableFields) + appendPatchHead(this, patchSettableFields, shortName) val mustStampOnUpdate = onUpdateAutoSetFields.isNotEmpty() if (hasPatchFields) { val settableNamesList = patchSettableFields.joinToString(", ") { "\"${it.name}\"" } @@ -880,7 +925,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase navs.any { it.targetCredentialFields.isNotEmpty() } } val src = buildString { @@ -985,6 +1030,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase st.entity.metaFields.map { it.name } }) append("private fun parse${shortName}Sort(raw: String): Pair? {\n") append(" val parts = raw.split(\":\", limit = 2)\n") append(" val field = parts.getOrNull(0) ?: return null\n") @@ -1040,6 +1088,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + appendLimitGuard() append(" val filterResult = parse${shortName}Filter(request.queryString)\n") append(" if (filterResult.error != null) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to filterResult.error, \"field\" to filterResult.field) as Any)\n") append(" val whereOp = ${shortName}WhereOp(filterResult.predicates)\n") @@ -1054,9 +1103,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + appendLimitGuard() append(" val filterResult = parse${shortName}Filter(request.queryString)\n") append(" if (filterResult.error != null) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to filterResult.error, \"field\" to filterResult.field) as Any)\n") append(" val whereOp = ${shortName}WhereOp(filterResult.predicates)\n") @@ -1118,9 +1168,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + if (rejectUnknown) { + // The raw body is read first so an undeclared key is refused whatever the host's ObjectMapper does. + append(" fun create$sfx(@RequestBody body: JsonNode): ResponseEntity = transaction {\n") + append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(MetaValidationIssues.Collector().add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + appendUnknownGuard(shortName) + append(" val dto: $shortName = try {\n") + append(" objectMapper.treeToValue(body, ${shortName}::class.java)\n") + append(" } catch (e: Exception) {\n") + append(" return@transaction ResponseEntity.badRequest().body(MetaDecimalWire.unreadableBody(e) as Any)\n") + append(" }\n") + } else { + append(" fun create$sfx(@RequestBody dto: $shortName): ResponseEntity = transaction {\n") + } append(" val issues = MetaValidationIssues.Collector()\n") for (field in stPatch) { append(" issues.addAt(validator.validateValue($validationRef::class.java, \"${field.name}\", dto.${field.name}), \"${field.name}\")\n") @@ -1206,7 +1268,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") - appendPatchHead(this, stPatch) + appendPatchHead(this, stPatch, shortName) // #203/ADR-0045: mirrors the vanilla update handler's mustStampOnUpdate control flow — // an onUpdate @autoSet column must bump on EVERY patch, so the "any present" guard is // dropped when one exists (else the guard would keep a no-settable-field patch from @@ -1431,9 +1493,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase>) { + private fun appendPatchHead(out: StringBuilder, fields: List>, shortName: String) { out.append(" val issues = MetaValidationIssues.Collector()\n") out.append(" if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add(\"invalid_type\", emptyList(), \"request body must be a JSON object\").body() as Any)\n") + out.appendUnknownGuard(shortName) for (field in fields) { if (KotlinGenUtil.isRequiredField(field)) { out.append(" if (body.has(\"${field.name}\") && body.get(\"${field.name}\").isNull) issues.add(\"invalid_type\", listOf(\"${field.name}\"), \"field '${field.name}' cannot be null\")\n") @@ -2419,6 +2482,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase = transaction {\n") + appendLimitGuard() append(" // FR-009 filter operators — short-circuit 400 on invalid field/op/value.\n") append(" val filterResult = parse${shortName}Filter(request.queryString)\n") append(" if (filterResult.error != null) {\n") @@ -2433,7 +2497,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase): Map? { + | if (!body.isObject) return null + | val unknown = ArrayList() + | val names = body.fieldNames() + | while (names.hasNext()) { + | val name = names.next() + | if (name !in known) unknown.add(name) + | } + | if (unknown.isEmpty()) return null + | unknown.sortWith { a, b -> compareCodePoints(a, b) } + | return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + | } + | + | /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + | private fun compareCodePoints(a: String, b: String): Int { + | var i = 0 + | var j = 0 + | while (i < a.length && j < b.length) { + | val ca = a.codePointAt(i) + | val cb = b.codePointAt(j) + | if (ca != cb) return ca.compareTo(cb) + | i += Character.charCount(ca) + | j += Character.charCount(cb) + | } + | return (a.length - i).compareTo(b.length - j) + | } + |} + """.trimMargin() +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt index d5355c70fa..e906092dd4 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/AuthorController.kt @@ -60,6 +60,9 @@ private fun parseAuthorSort(raw: String): Pair? { return field to dir } +/** GENERATED — every field Author declares, as named on the wire: a body key outside it is refused. */ +private val AuthorKnownFields: Set = setOf("id", "name") + /** GENERATED — cross-port cap on `in`-list size (matches TS DEFAULT_MAX_IN_LIST). */ private const val MAX_IN_LIST = 100 @@ -329,6 +332,8 @@ class AuthorController(private val objectMapper: ObjectMapper, private val valid fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthorKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) val dto: Author = try { objectMapper.treeToValue(body, Author::class.java) } catch (e: Exception) { @@ -353,6 +358,8 @@ class AuthorController(private val objectMapper: ObjectMapper, private val valid fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthorKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name").any { body.has(it) }) { run { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaUnknownBodyKeys.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaUnknownBodyKeys.kt new file mode 100644 index 0000000000..2916270433 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-controller/acme/blog/MetaUnknownBodyKeys.kt @@ -0,0 +1,39 @@ +package acme.blog + +import com.fasterxml.jackson.databind.JsonNode + +/** + * GENERATED — do not hand-edit. The answer for a write body naming a key the entity does not declare. + */ +internal object MetaUnknownBodyKeys { + /** + * The 400 body for [body]'s top-level keys outside [known], or null when there are none or [body] is not + * a JSON object (a body that is not an object is left to the handler's own reading of it). + */ + fun refusal(body: JsonNode, known: Set): Map? { + if (!body.isObject) return null + val unknown = ArrayList() + val names = body.fieldNames() + while (names.hasNext()) { + val name = names.next() + if (name !in known) unknown.add(name) + } + if (unknown.isEmpty()) return null + unknown.sortWith { a, b -> compareCodePoints(a, b) } + return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaUnknownBodyKeys.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaUnknownBodyKeys.kt new file mode 100644 index 0000000000..013eb4aa28 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/MetaUnknownBodyKeys.kt @@ -0,0 +1,39 @@ +package acme.catalog + +import com.fasterxml.jackson.databind.JsonNode + +/** + * GENERATED — do not hand-edit. The answer for a write body naming a key the entity does not declare. + */ +internal object MetaUnknownBodyKeys { + /** + * The 400 body for [body]'s top-level keys outside [known], or null when there are none or [body] is not + * a JSON object (a body that is not an object is left to the handler's own reading of it). + */ + fun refusal(body: JsonNode, known: Set): Map? { + if (!body.isObject) return null + val unknown = ArrayList() + val names = body.fieldNames() + while (names.hasNext()) { + val name = names.next() + if (name !in known) unknown.add(name) + } + if (unknown.isEmpty()) return null + unknown.sortWith { a, b -> compareCodePoints(a, b) } + return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt index f86a52e1dc..cb46c28ad6 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter-wire-string/acme/catalog/ProductController.kt @@ -62,6 +62,9 @@ private fun parseProductSort(raw: String): Pair? { return field to dir } +/** GENERATED — every field Product declares, as named on the wire: a body key outside it is refused. */ +private val ProductKnownFields: Set = setOf("id", "name", "price", "rating") + /** GENERATED — cross-port cap on `in`-list size (matches TS DEFAULT_MAX_IN_LIST). */ private const val MAX_IN_LIST = 100 @@ -379,6 +382,8 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, ProductKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) val dto: Product = try { objectMapper.treeToValue(body, Product::class.java) } catch (e: Exception) { @@ -415,6 +420,8 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, ProductKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name", "price", "rating").any { body.has(it) }) { run { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaUnknownBodyKeys.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaUnknownBodyKeys.kt new file mode 100644 index 0000000000..013eb4aa28 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/MetaUnknownBodyKeys.kt @@ -0,0 +1,39 @@ +package acme.catalog + +import com.fasterxml.jackson.databind.JsonNode + +/** + * GENERATED — do not hand-edit. The answer for a write body naming a key the entity does not declare. + */ +internal object MetaUnknownBodyKeys { + /** + * The 400 body for [body]'s top-level keys outside [known], or null when there are none or [body] is not + * a JSON object (a body that is not an object is left to the handler's own reading of it). + */ + fun refusal(body: JsonNode, known: Set): Map? { + if (!body.isObject) return null + val unknown = ArrayList() + val names = body.fieldNames() + while (names.hasNext()) { + val name = names.next() + if (name !in known) unknown.add(name) + } + if (unknown.isEmpty()) return null + unknown.sortWith { a, b -> compareCodePoints(a, b) } + return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt index f86a52e1dc..cb46c28ad6 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-decimal-float-filter/acme/catalog/ProductController.kt @@ -62,6 +62,9 @@ private fun parseProductSort(raw: String): Pair? { return field to dir } +/** GENERATED — every field Product declares, as named on the wire: a body key outside it is refused. */ +private val ProductKnownFields: Set = setOf("id", "name", "price", "rating") + /** GENERATED — cross-port cap on `in`-list size (matches TS DEFAULT_MAX_IN_LIST). */ private const val MAX_IN_LIST = 100 @@ -379,6 +382,8 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, ProductKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) val dto: Product = try { objectMapper.treeToValue(body, Product::class.java) } catch (e: Exception) { @@ -415,6 +420,8 @@ class ProductController(private val objectMapper: ObjectMapper, private val vali fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, ProductKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("name") && body.get("name").isNull) issues.add("invalid_type", listOf("name"), "field 'name' cannot be null") if (listOf("name", "price", "rating").any { body.has(it) }) { run { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt index f1a7f8a27c..432829dadc 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt @@ -50,6 +50,9 @@ private val AuthSortDefaultOrder: Map = mapOf( "reference" to "desc", ) +/** GENERATED — every field Auth declares, as named on the wire: a body key outside it is refused. */ +private val AuthKnownFields: Set = setOf("id", "type", "reference", "quantity", "copayAmount", "priorAuthNumber") + private fun parseAuthSort(raw: String): Pair? { val parts = raw.split(":", limit = 2) val field = parts.getOrNull(0) ?: return null @@ -418,7 +421,15 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat } @PostMapping("/bridge") - fun createBridge(@RequestBody dto: Auth): ResponseEntity = transaction { + fun createBridge(@RequestBody body: JsonNode): ResponseEntity = transaction { + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(MetaValidationIssues.Collector().add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) + val dto: Auth = try { + objectMapper.treeToValue(body, Auth::class.java) + } catch (e: Exception) { + return@transaction ResponseEntity.badRequest().body(MetaDecimalWire.unreadableBody(e) as Any) + } val issues = MetaValidationIssues.Collector() issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "quantity", dto.quantity), "quantity") issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "reference", dto.reference), "reference") @@ -436,6 +447,8 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat fun updateBridge(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("quantity") && body.get("quantity").isNull) issues.add("invalid_type", listOf("quantity"), "field 'quantity' cannot be null") if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("quantity", "reference").any { body.has(it) }) { @@ -505,7 +518,15 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat } @PostMapping("/copay") - fun createCopay(@RequestBody dto: Auth): ResponseEntity = transaction { + fun createCopay(@RequestBody body: JsonNode): ResponseEntity = transaction { + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(MetaValidationIssues.Collector().add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) + val dto: Auth = try { + objectMapper.treeToValue(body, Auth::class.java) + } catch (e: Exception) { + return@transaction ResponseEntity.badRequest().body(MetaDecimalWire.unreadableBody(e) as Any) + } val issues = MetaValidationIssues.Collector() issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "copayAmount", dto.copayAmount), "copayAmount") if (!MetaDecimalWire.fits(dto.copayAmount, 10, 2)) issues.add("too_big", listOf("copayAmount"), "decimal does not fit the declared precision and scale") @@ -524,6 +545,8 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat fun updateCopay(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("copayAmount", "reference").any { body.has(it) }) { run { @@ -593,7 +616,15 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat } @PostMapping("/priorauth") - fun createPriorAuth(@RequestBody dto: Auth): ResponseEntity = transaction { + fun createPriorAuth(@RequestBody body: JsonNode): ResponseEntity = transaction { + if (!body.isObject) return@transaction ResponseEntity.badRequest().body(MetaValidationIssues.Collector().add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) + val dto: Auth = try { + objectMapper.treeToValue(body, Auth::class.java) + } catch (e: Exception) { + return@transaction ResponseEntity.badRequest().body(MetaDecimalWire.unreadableBody(e) as Any) + } val issues = MetaValidationIssues.Collector() issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "priorAuthNumber", dto.priorAuthNumber), "priorAuthNumber") issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "reference", dto.reference), "reference") @@ -611,6 +642,8 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat fun updatePriorAuth(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (body.has("reference") && body.get("reference").isNull) issues.add("invalid_type", listOf("reference"), "field 'reference' cannot be null") if (listOf("priorAuthNumber", "reference").any { body.has(it) }) { run { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt index 5a4b1c1daa..3e7c1c77fb 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthLineController.kt @@ -59,6 +59,9 @@ private fun parseAuthLineSort(raw: String): Pair? { return field to dir } +/** GENERATED — every field AuthLine declares, as named on the wire: a body key outside it is refused. */ +private val AuthLineKnownFields: Set = setOf("id", "label") + /** GENERATED — cross-port cap on `in`-list size (matches TS DEFAULT_MAX_IN_LIST). */ private const val MAX_IN_LIST = 100 @@ -328,6 +331,8 @@ class AuthLineController(private val objectMapper: ObjectMapper, private val val fun create(@RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthLineKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) val dto: AuthLine = try { objectMapper.treeToValue(body, AuthLine::class.java) } catch (e: Exception) { @@ -351,6 +356,8 @@ class AuthLineController(private val objectMapper: ObjectMapper, private val val fun update(@PathVariable id: Long, @RequestBody body: JsonNode): ResponseEntity = transaction { val issues = MetaValidationIssues.Collector() if (!body.isObject) return@transaction ResponseEntity.badRequest().body(issues.add("invalid_type", emptyList(), "request body must be a JSON object").body() as Any) + val unknownKeys = MetaUnknownBodyKeys.refusal(body, AuthLineKnownFields) + if (unknownKeys != null) return@transaction ResponseEntity.badRequest().body(unknownKeys as Any) if (listOf("label").any { body.has(it) }) { run { val hasLabel = body.has("label") diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaUnknownBodyKeys.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaUnknownBodyKeys.kt new file mode 100644 index 0000000000..b6e296a019 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/MetaUnknownBodyKeys.kt @@ -0,0 +1,39 @@ +package acme.auth + +import com.fasterxml.jackson.databind.JsonNode + +/** + * GENERATED — do not hand-edit. The answer for a write body naming a key the entity does not declare. + */ +internal object MetaUnknownBodyKeys { + /** + * The 400 body for [body]'s top-level keys outside [known], or null when there are none or [body] is not + * a JSON object (a body that is not an object is left to the handler's own reading of it). + */ + fun refusal(body: JsonNode, known: Set): Map? { + if (!body.isObject) return null + val unknown = ArrayList() + val names = body.fieldNames() + while (names.hasNext()) { + val name = names.next() + if (name !in known) unknown.add(name) + } + if (unknown.isEmpty()) return null + unknown.sortWith { a, b -> compareCodePoints(a, b) } + return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaUnknownBodyKeys.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaUnknownBodyKeys.kt new file mode 100644 index 0000000000..34245ddca6 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/MetaUnknownBodyKeys.kt @@ -0,0 +1,39 @@ +package acme.report + +import com.fasterxml.jackson.databind.JsonNode + +/** + * GENERATED — do not hand-edit. The answer for a write body naming a key the entity does not declare. + */ +internal object MetaUnknownBodyKeys { + /** + * The 400 body for [body]'s top-level keys outside [known], or null when there are none or [body] is not + * a JSON object (a body that is not an object is left to the handler's own reading of it). + */ + fun refusal(body: JsonNode, known: Set): Map? { + if (!body.isObject) return null + val unknown = ArrayList() + val names = body.fieldNames() + while (names.hasNext()) { + val name = names.next() + if (name !in known) unknown.add(name) + } + if (unknown.isEmpty()) return null + unknown.sortWith { a, b -> compareCodePoints(a, b) } + return linkedMapOf("error" to "unknown_fields", "fields" to unknown) + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } +} diff --git a/server/java/codegen-spring/pom.xml b/server/java/codegen-spring/pom.xml index 20849a8ae2..56c5af3f68 100644 --- a/server/java/codegen-spring/pom.xml +++ b/server/java/codegen-spring/pom.xml @@ -307,6 +307,7 @@ PatchValidationException.java RecordComponentNames.java ReportExport.java + UnknownBodyKeys.java UnreadableBody.java ValidationIssues.java diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java index 45c28e04a5..48070a1fb7 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java @@ -274,7 +274,7 @@ public String toString() { * {@code REGISTRY}: static initializers run in order. */ private static final List ROUTES_RUNTIME = List.of( "CanonicalDecimal", "CanonicalDuration", "ConstraintErrors", "FilterParseResult", "FilterParser", "FilterPredicate", - "PatchValidationException", "RecordComponentNames", "ReportExport", "UnreadableBody", "ValidationIssues"); + "PatchValidationException", "RecordComponentNames", "ReportExport", "UnknownBodyKeys", "UnreadableBody", "ValidationIssues"); /** The writer a generated type's package and class name are read from, and its base. */ private static final List OBJECT_NAMING = List.of( diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java index 1e23e29939..78bb06bfdc 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java @@ -11,6 +11,7 @@ import com.metaobjects.generator.spring.SpringTypeMapper; import com.metaobjects.generator.spring.TphPlan; +import com.metaobjects.config.ApiSettings; import com.metaobjects.field.MetaField; import com.metaobjects.field.ObjectField; import com.metaobjects.query.FilterOps; @@ -262,6 +263,7 @@ protected void emit(MetaObject entity, Path outRoot) { src.append("import ").append(runtimePackage).append(".ConstraintErrors;\n"); src.append("import ").append(runtimePackage).append(".PatchValidationException;\n"); src.append("import ").append(runtimePackage).append(".UnreadableBody;\n"); + src.append("import ").append(runtimePackage).append(".UnknownBodyKeys;\n"); src.append("import ").append(runtimePackage).append(".ValidationIssues;\n"); // The escape the PATCH loop applies to each assigned key before naming a Bean // Validation property — see RecordComponentNames for why it is needed at run time. @@ -305,6 +307,8 @@ protected void emit(MetaObject entity, Path outRoot) { // private to this controller (no cross-file collision risk). appendSortAllowlist(src, entity, sortFields); appendServerOwnedOnCreate(src, entity); + ApiSettings api = ApiSettings.of(entity); + appendKnownFields(src, api, new ArrayList<>(entity.getMetaFields())); // #484: the credential fields this entity's rows carry, hidden from every response body. List hidden = SpringGeneratorSupport.credentialNames(entity); List navs = SpringM2mSupport.resolve(entity, loader); @@ -333,7 +337,7 @@ protected void emit(MetaObject entity, Path outRoot) { // HttpServletRequest carries the raw query string so the bracketed // filter[][]= grammar reaches FilterParser intact // (Spring's @RequestParam would collapse same-key occurrences). - appendListHandler(src, shortName, dtoName, repoName, hidden); + appendListHandler(src, shortName, dtoName, repoName, hidden, api.pagination()); // GET /{id} — single by primary key. appendGetByIdHandler(src, pkType, hidden); @@ -350,11 +354,14 @@ protected void emit(MetaObject entity, Path outRoot) { // direct DTO binding, byte-identical. List components = SpringGeneratorSupport.dtoComponentFields(entity); List defaultPuts = createDefaultPuts(entity, components); - if (defaultPuts.isEmpty()) { + // api.unknownFields = reject (the default) binds the raw body too: the undeclared keys are + // read off the JSON itself, so the answer does not depend on how the host's ObjectMapper + // treats an unknown property. + if (defaultPuts.isEmpty() && !api.rejectsUnknownFields()) { src.append(" public ResponseEntity create(@RequestBody ").append(dtoName).append(" dto) {\n"); } else { src.append(" public ResponseEntity create(@RequestBody JsonNode body) {\n"); - appendFillDefaults(src, dtoName, defaultPuts); + appendFillDefaults(src, dtoName, defaultPuts, api.rejectsUnknownFields()); } // #484: a @formExclude / credential value is IGNORED (never rejected): the DTO is rebuilt // without it, so the column is left to the repository's default. @@ -416,6 +423,7 @@ protected void emit(MetaObject entity, Path outRoot) { src.append(" @RequestMapping(value = \"/{id}\", method = { RequestMethod.PATCH, RequestMethod.PUT })\n"); src.append(" public ResponseEntity update(@PathVariable ").append(pkType) .append(" id, @RequestBody JsonNode body) {\n"); + appendUnknownFieldsGuard(src, api); appendPatchBinding(src, patchName, dtoName); // Program D (spec section 0): validateValue does NOT cascade @Valid into a nested value // object's own constraints, so validate each PRESENT VO element EXPLICITLY. The property's @@ -730,7 +738,7 @@ protected void emitReadOnly(MetaObject entity, Path outRoot) { if (com.metaobjects.generator.util.GeneratorUtil.isReport(entity)) { appendReportListHandler(src, shortName, dtoName, repoName, reportMaxRows, reportExport, routeBase, hidden); } else { - appendListHandler(src, shortName, dtoName, repoName, hidden); + appendListHandler(src, shortName, dtoName, repoName, hidden, ApiSettings.of(entity).pagination()); } if (hasItem) { appendGetByIdHandler(src, pkType, hidden); @@ -843,7 +851,7 @@ private static void appendUnreadableBodyHandler(StringBuilder src) { * (Spring's {@code @RequestParam} would collapse same-key occurrences). */ private static void appendListHandler(StringBuilder src, String shortName, String dtoName, String repoName, - List hidden) { + List hidden, ApiSettings.Pagination page) { String allowlistName = SpringNaming.filterAllowlistName(shortName); src.append(" @GetMapping\n"); src.append(" public ResponseEntity list(\n"); @@ -852,7 +860,7 @@ private static void appendListHandler(StringBuilder src, String shortName, Strin src.append(" @RequestParam(required = false) String sort,\n"); src.append(" @RequestParam(required = false, name = \"withCount\") Integer withCount,\n"); src.append(" HttpServletRequest request) {\n"); - src.append(" int actualLimit = limit != null ? limit : 50;\n"); + appendLimit(src, page); src.append(" int actualOffset = offset != null ? offset : 0;\n"); src.append(" ").append(repoName).append(".SortClause sortClause = null;\n"); src.append(" if (sort != null) {\n"); @@ -1165,6 +1173,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append("import ").append(runtimePackage).append(".ConstraintErrors;\n"); src.append("import ").append(runtimePackage).append(".PatchValidationException;\n"); src.append("import ").append(runtimePackage).append(".UnreadableBody;\n"); + src.append("import ").append(runtimePackage).append(".UnknownBodyKeys;\n"); src.append("import ").append(runtimePackage).append(".ValidationIssues;\n"); // The escape the PATCH loop applies to each assigned key before naming a Bean // Validation property — see RecordComponentNames for why it is needed at run time. @@ -1214,6 +1223,12 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { // The polymorphic sort surface is the BASE's own columns, so the declared orders // are read off the base too — same owner the allowlist above was built from. appendSortDefaultOrders(src, base, sortFields); + // A key declared by the base or by ANY subtype is known on every subtype route: a sibling + // subtype's column is dropped there (it is not this route's to write), never refused. + ApiSettings api = ApiSettings.of(base); + List declared = new ArrayList<>(base.getMetaFields()); + declared.addAll(TphPlan.everySubtypeField(base, loader)); + appendKnownFields(src, api, declared); // FR-036 Program B: constructor-inject the ObjectMapper (binds each present PATCH value via // Patch.fromJson, same wire codecs as create) + the jakarta Validator (per-field @@ -1254,7 +1269,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append(" @RequestParam(required = false) String sort,\n"); src.append(" @RequestParam(required = false, name = \"withCount\") Integer withCount,\n"); src.append(" HttpServletRequest request) {\n"); - src.append(" int actualLimit = limit != null ? limit : 50;\n"); + appendLimit(src, api.pagination()); src.append(" int actualOffset = offset != null ? offset : 0;\n"); src.append(" ").append(repoName).append(".SortClause sortClause = null;\n"); src.append(" if (sort != null) {\n"); @@ -1327,7 +1342,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { src.append(" @RequestParam(required = false) Integer offset,\n"); src.append(" @RequestParam(required = false) String sort,\n"); src.append(" HttpServletRequest request) {\n"); - src.append(" int actualLimit = limit != null ? limit : 50;\n"); + appendLimit(src, api.pagination()); src.append(" int actualOffset = offset != null ? offset : 0;\n"); src.append(" ").append(repoName).append(".SortClause sortClause = null;\n"); src.append(" if (sort != null) {\n"); @@ -1377,12 +1392,12 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { // As the vanilla create — fill the subtype's omitted literal @default fields into // the raw body, then bind the union DTO. No such field: the direct binding, byte-identical. List subDefaultPuts = createDefaultPuts(st.entity(), SpringGeneratorSupport.dtoComponentFields(st.entity())); - if (subDefaultPuts.isEmpty()) { + if (subDefaultPuts.isEmpty() && !api.rejectsUnknownFields()) { src.append(" public ResponseEntity create").append(suffix) .append("(@RequestBody ").append(dtoName).append(" dto) {\n"); } else { src.append(" public ResponseEntity create").append(suffix).append("(@RequestBody JsonNode body) {\n"); - appendFillDefaults(src, dtoName, subDefaultPuts); + appendFillDefaults(src, dtoName, subDefaultPuts, api.rejectsUnknownFields()); } src.append(" ValidationIssues __issues = new ValidationIssues();\n"); for (MetaField vf : createValidated) { @@ -1454,6 +1469,7 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { .append("/{id}\", method = { RequestMethod.PATCH, RequestMethod.PUT })\n"); src.append(" public ResponseEntity update").append(suffix) .append("(@PathVariable ").append(pkType).append(" id, @RequestBody JsonNode body) {\n"); + appendUnknownFieldsGuard(src, api); appendPatchBinding(src, subPatch, subDto); appendValueObjectMapValidation(src, st.entity()); appendPatchVerdict(src); @@ -1739,11 +1755,12 @@ private static String javaLiteral(String text) { /** The body of a create handler bound to the raw {@code JsonNode body}: refuse a * non-object body, fill the absent defaulted fields ({@link #createDefaultPuts}), bind * {@code dto}. */ - private static void appendFillDefaults(StringBuilder src, String dtoName, List puts) { + private static void appendFillDefaults(StringBuilder src, String dtoName, List puts, boolean rejectUnknown) { src.append(" if (!(body instanceof com.fasterxml.jackson.databind.node.ObjectNode given)) {\n"); src.append(" return ResponseEntity.badRequest().body(new ValidationIssues()\n"); src.append(" .add(ValidationIssues.INVALID_TYPE, java.util.List.of(), \"request body must be a JSON object\").body());\n"); src.append(" }\n"); + if (rejectUnknown) appendUnknownFieldsGuard(src, true); src.append(" com.fasterxml.jackson.databind.node.ObjectNode filled = given.deepCopy();\n"); for (String put : puts) src.append(" ").append(put).append("\n"); src.append(" ").append(dtoName).append(" dto;\n"); @@ -1754,6 +1771,59 @@ private static void appendFillDefaults(StringBuilder src, String dtoName, List declared) { + if (!api.rejectsUnknownFields()) return; + Set names = new java.util.LinkedHashSet<>(); + for (MetaField f : declared) names.add(f.getName()); + src.append(" /** Every field the entity declares, as named on the wire: a body key outside it is refused. */\n"); + src.append(" private static final Set KNOWN_FIELDS = Set.of("); + int i = 0; + for (String n : names) { + if (i++ > 0) src.append(", "); + src.append(javaLiteral(n)); + } + src.append(");\n\n"); + } + + /** The unknown-key check at the head of a PUT/PATCH handler (see {@link #appendUnknownFieldsGuard(StringBuilder, boolean)}). */ + private static void appendUnknownFieldsGuard(StringBuilder src, ApiSettings api) { + if (api.rejectsUnknownFields()) appendUnknownFieldsGuard(src, true); + } + + /** Answer {@code 400 unknown_fields} for an undeclared top-level key of {@code body}, checked on the raw JSON before any binding. */ + private static void appendUnknownFieldsGuard(StringBuilder src, boolean enabled) { + if (!enabled) return; + src.append(" Map __unknown = UnknownBodyKeys.refusal(body, KNOWN_FIELDS);\n"); + src.append(" if (__unknown != null) {\n"); + src.append(" return ResponseEntity.badRequest().body(__unknown);\n"); + src.append(" }\n"); + } + + /** + * The {@code actualLimit} line of a list handler: no {@code limit} reads the page default + * (50 unless the project sets {@code api.pagination}), and a {@code limit} above + * {@code api.pagination.maxLimit} is {@code 400 {"error": "limit_too_large", "maxLimit": N}}. + */ + private static void appendLimit(StringBuilder src, ApiSettings.Pagination page) { + if (page == null) { + src.append(" int actualLimit = limit != null ? limit : 50;\n"); + return; + } + src.append(" if (limit != null && limit > ").append(page.maxLimit()).append(") {\n"); + src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"limit_too_large\", \"maxLimit\", ") + .append(page.maxLimit()).append("));\n"); + src.append(" }\n"); + src.append(" int actualLimit = limit != null ? limit : ").append(page.defaultLimit()).append(";\n"); + } + /** A field no client may set on create regardless of the entity's key: readOnly or derived. */ private static boolean isNonSettableOnCreate(MetaField f) { return MetaField.MUTABILITY_READ_ONLY.equals(f.getMutability()) || f.isDerived(); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnknownBodyKeys.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnknownBodyKeys.java new file mode 100644 index 0000000000..f3bcc2b39a --- /dev/null +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnknownBodyKeys.java @@ -0,0 +1,59 @@ +package com.metaobjects.generator.spring.runtime; + +import com.fasterxml.jackson.databind.JsonNode; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.Set; +import java.util.TreeSet; + +/** + * The wire answer for a write body that carries a key the entity does not declare + * ({@code api.unknownFields: reject}, the default): {@code 400 {"error": "unknown_fields", + * "fields": [...]}}, the keys in code-point order. + * + *

The keys are read off the raw {@link JsonNode} BEFORE the body is bound to a DTO, so the + * answer never depends on whether the host application's {@code ObjectMapper} fails on unknown + * properties. Depends on Jackson databind and the JDK only ({@code mvn metaobjects:eject} hands + * it over as plain source).

+ */ +public final class UnknownBodyKeys { + + /** The {@code error} of the refusal. */ + public static final String ERROR = "unknown_fields"; + + private UnknownBodyKeys() { /* no instances */ } + + /** + * The 400 body for {@code body}'s top-level keys outside {@code known}, or {@code null} when + * there are none or {@code body} is not a JSON object (a body that is not an object is left to + * the handler's own reading of it). + */ + public static Map refusal(JsonNode body, Set known) { + if (body == null || !body.isObject()) return null; + TreeSet unknown = new TreeSet<>(UnknownBodyKeys::compareCodePoints); + body.fieldNames().forEachRemaining(name -> { + if (!known.contains(name)) unknown.add(name); + }); + if (unknown.isEmpty()) return null; + Map out = new LinkedHashMap<>(); + out.put("error", ERROR); + out.put("fields", new ArrayList<>(unknown)); + return out; + } + + /** Ordinal order by Unicode code point (not UTF-16 unit), the order every port answers in. */ + static int compareCodePoints(String a, String b) { + int i = 0; + int j = 0; + while (i < a.length() && j < b.length()) { + int ca = a.codePointAt(i); + int cb = b.codePointAt(j); + if (ca != cb) return Integer.compare(ca, cb); + i += Character.charCount(ca); + j += Character.charCount(cb); + } + return Integer.compare(a.length() - i, b.length() - j); + } +} diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTphCrossFieldCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTphCrossFieldCompileRunTest.java index 65bdc37a47..dd3dea4eba 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTphCrossFieldCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTphCrossFieldCompileRunTest.java @@ -105,8 +105,9 @@ public void perSubtypeCreateEnforcesTheSubtypesCrossFieldRules() throws Exceptio } private static int post(Object controller, Class dtoClass, String handler, String json) throws Exception { - Method m = controller.getClass().getMethod(handler, dtoClass); - ResponseEntity resp = (ResponseEntity) m.invoke(controller, MAPPER.readValue(json, dtoClass)); + // The create handlers bind the raw body (api.unknownFields: reject reads its keys first). + Method m = controller.getClass().getMethod(handler, com.fasterxml.jackson.databind.JsonNode.class); + ResponseEntity resp = (ResponseEntity) m.invoke(controller, MAPPER.readTree(json)); return resp.getStatusCode().value(); } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java index 13f8228250..2251e4e358 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringControllerGeneratorTest.java @@ -83,7 +83,7 @@ public void emitsRestControllerWithAllFiveVerbs() throws Exception { // envelope); the handler binds the raw DTO and validates it explicitly with the injected // Validator, returning {"error":"validation"} on any field-constraint violation. assertTrue("expected @PostMapping create handler; saw:\n" + src, - Pattern.compile("@PostMapping\\s*\\n\\s*public ResponseEntity<\\?> create\\(@RequestBody AuthorDto dto\\)").matcher(src).find()); + Pattern.compile("@PostMapping\\s*\\n\\s*public ResponseEntity<\\?> create\\(@RequestBody JsonNode body\\)").matcher(src).find()); assertTrue("expected explicit validator.validate(dto) on create (FR-036); saw:\n" + src, src.contains("validator.validate(dto)")); // PATCH AND PUT share the update handler (per API contract) — expressed as a single diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java index 0e49a1e098..7334bbad7c 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java @@ -1,5 +1,6 @@ package com.metaobjects.generator.spring; +import java.util.regex.Pattern; import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.registry.SharedRegistryTestBase; import org.junit.Rule; @@ -125,7 +126,8 @@ public void tphSharedColumnMarkedOnALaterSubtypeIsHiddenAndIgnored() throws Exce controller.contains("redactAll(rows, \"token\")")); assertTrue("every subtype create ignores both marked columns; saw:\n" + controller, controller.contains("dto = new ThingDto(null, null, null, null);")); - assertFalse("memo (@formExclude only) is still returned", controller.contains("\"memo\")")); + assertFalse("memo (@formExclude only) is still returned", + Pattern.compile("redact(All)?\\([^)]*\"memo\"").matcher(controller).find()); } @Test diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringUnknownFieldsCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringUnknownFieldsCompileRunTest.java new file mode 100644 index 0000000000..2d81bb7eac --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringUnknownFieldsCompileRunTest.java @@ -0,0 +1,186 @@ +package com.metaobjects.generator.spring; + +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.metaobjects.config.ApiSettings; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.ObjectClassRegistry; +import jakarta.validation.Validation; +import jakarta.validation.Validator; +import org.junit.After; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.springframework.http.ResponseEntity; + +import java.lang.reflect.Method; +import java.lang.reflect.Proxy; +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + * {@code api.unknownFields}: a generated create, PUT and PATCH answers an undeclared body key the same + * way whatever the host's {@code ObjectMapper} does with an unknown property. The keys are read off + * the raw body before it is bound, so a mapper with {@code FAIL_ON_UNKNOWN_PROPERTIES} on and one + * with it off give identical answers under both {@code reject} and {@code drop}. + */ +public class SpringUnknownFieldsCompileRunTest { + + private static final ObjectMapper PLAIN = new ObjectMapper(); + + @Rule + public TemporaryFolder tmp = new TemporaryFolder(); + + @After + public void resetRegistry() { + ObjectClassRegistry.resetGlobal(); + } + + private static ObjectMapper mapper(boolean failOnUnknown) { + return new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, failOnUnknown); + } + + private static final String META = """ + { "metadata.root": { "package": "acme::thing", "children": [ + { "object.entity": { "name": "Thing", "children": [ + { "source.rdb": { "@table": "things" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.string": { "name": "role", "@mutability": "readOnly" } }, + { "field.timestamp": { "name": "stamped", "@autoSet": "onCreate" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Copay"] } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "acme::thing::Auth", "@discriminatorValue": "Bridge", + "children": [ { "field.int": { "name": "quantity" } } ] } }, + { "object.entity": { "name": "CopayAuth", "extends": "acme::thing::Auth", "@discriminatorValue": "Copay", + "children": [ { "field.string": { "name": "approver" } } ] } } + ] } } + """; + + /** One generated, compiled controller pair built for a given {@code api.unknownFields} setting. */ + private final class Built { + final URLClassLoader cl; + final Class thingRepo; + final Class authRepo; + final List writes = new ArrayList<>(); + + Built(ApiSettings api, String name) throws Exception { + Path work = tmp.newFolder("work-" + name).toPath(); + MetaDataLoader loader = SpringTestFixtures.loadFixture(work, name, META); + loader.getRoot().setApiSettings(api); + Path gen = tmp.newFolder("gen-" + name).toPath(); + for (var g : List.of(new SpringControllerGenerator(), new SpringDtoGenerator(), + new SpringRepositoryGenerator(), new SpringFilterAllowlistGenerator())) { + g.setArgs(Map.of("outputDir", gen.toString())); + g.execute(loader); + } + Path classes = tmp.newFolder("classes-" + name).toPath(); + SpringTestFixtures.compileGenerated(gen, classes); + cl = new URLClassLoader(new URL[]{ classes.toUri().toURL() }, getClass().getClassLoader()); + thingRepo = cl.loadClass("acme.thing.ThingRepository"); + authRepo = cl.loadClass("acme.thing.AuthRepository"); + } + + Object controller(String simple, Class repoType, ObjectMapper mapper) throws Exception { + Object repo = Proxy.newProxyInstance(cl, new Class[]{ repoType }, (p, m, args) -> { + if (m.getName().startsWith("create") || m.getName().startsWith("patch")) { + writes.add(m.getName()); + return m.getName().startsWith("patch") ? Optional.empty() : args[args.length - 1]; + } + throw new UnsupportedOperationException(m.getName()); + }); + Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); + return cl.loadClass("acme.thing." + simple) + .getConstructor(repoType, ObjectMapper.class, Validator.class).newInstance(repo, mapper, validator); + } + } + + /** The answer of one handler: its status and, for a 400, its body. */ + private static String answer(Object controller, ObjectMapper host, String handler, String json) throws Exception { + JsonNode body = PLAIN.readTree(json); + for (Method m : controller.getClass().getMethods()) { + if (!m.getName().equals(handler)) continue; + // Under drop a create still binds the DTO directly: do that with the host's own mapper, + // as Spring's message converter would. + Class bodyType = m.getParameterTypes()[m.getParameterCount() - 1]; + Object bound = bodyType == JsonNode.class ? body : host.readValue(json, bodyType); + Object[] args = m.getParameterCount() == 1 ? new Object[]{ bound } : new Object[]{ 1L, bound }; + ResponseEntity res = (ResponseEntity) m.invoke(controller, args); + return res.getStatusCode().value() + (res.getStatusCode().value() == 400 + ? " " + PLAIN.writeValueAsString(res.getBody()) : ""); + } + throw new AssertionError("no handler " + handler); + } + + /** The same request through a failing and a tolerant mapper: one answer. */ + private String same(Built b, String simple, Class repoType, String handler, String json) throws Exception { + String strict = answer(b.controller(simple, repoType, mapper(true)), mapper(true), handler, json); + String lax = answer(b.controller(simple, repoType, mapper(false)), mapper(false), handler, json); + assertEquals(handler + " " + json + ": the host mapper must not change the answer", strict, lax); + return strict; + } + + @Test + public void rejectAnswersUnknownFieldsTheSameUnderEveryMapper() throws Exception { + Built b = new Built(ApiSettings.DEFAULT, "reject"); + String refusal = "400 {\"error\":\"unknown_fields\",\"fields\":[\"isAdmin\",\"zebra\"]}"; + + // create, PUT and PATCH: every undeclared key listed, in code-point order, nothing written + assertEquals(refusal, same(b, "ThingController", b.thingRepo, "create", "{\"title\":\"t\",\"zebra\":1,\"isAdmin\":true}")); + assertEquals(refusal, same(b, "ThingController", b.thingRepo, "update", "{\"title\":\"t\",\"zebra\":1,\"isAdmin\":true}")); + assertEquals(refusal, same(b, "AuthController", b.authRepo, "createBridge", "{\"quantity\":1,\"zebra\":1,\"isAdmin\":true}")); + assertEquals(refusal, same(b, "AuthController", b.authRepo, "updateBridge", "{\"quantity\":1,\"zebra\":1,\"isAdmin\":true}")); + assertTrue("a refused body writes nothing: " + b.writes, b.writes.isEmpty()); + + // declared keys the client may not set stay known (dropped, not refused): the generated key, + // readOnly, @autoSet; and a sibling subtype's column on a TPH route + assertEquals("201", same(b, "ThingController", b.thingRepo, "create", + "{\"id\":9,\"title\":\"t\",\"role\":\"admin\",\"stamped\":null}")); + assertEquals("404", same(b, "ThingController", b.thingRepo, "update", "{\"id\":9,\"title\":\"t\",\"role\":\"admin\"}")); + assertEquals("201", same(b, "AuthController", b.authRepo, "createBridge", "{\"type\":\"Copay\",\"quantity\":1,\"approver\":\"x\"}")); + assertEquals("404", same(b, "AuthController", b.authRepo, "updateBridge", "{\"quantity\":1,\"approver\":\"x\"}")); + assertTrue("accepted bodies reach the repository: " + b.writes, b.writes.size() >= 4); + } + + @Test + public void dropIgnoresUnknownFieldsTheSameUnderEveryMapper() throws Exception { + Built b = new Built(new ApiSettings(null, ApiSettings.UnknownFields.DROP), "drop"); + assertEquals("201", same(b, "ThingController", b.thingRepo, "create", "{\"title\":\"t\",\"zebra\":1,\"isAdmin\":true}")); + assertEquals("404", same(b, "ThingController", b.thingRepo, "update", "{\"title\":\"t\",\"zebra\":1}")); + assertEquals("201", same(b, "AuthController", b.authRepo, "createBridge", "{\"quantity\":1,\"zebra\":1}")); + assertEquals("404", same(b, "AuthController", b.authRepo, "updateBridge", "{\"quantity\":1,\"zebra\":1}")); + } + + @Test + public void aBodyThatIsNotAnObjectIsLeftToTheExistingHandling() throws Exception { + Built b = new Built(ApiSettings.DEFAULT, "nonobject"); + String answer = same(b, "ThingController", b.thingRepo, "create", "[1, 2]"); + assertTrue(answer, answer.startsWith("400 ") && !answer.contains("unknown_fields")); + } + + @Test + public void codePointOrderNotUtf16Order() { + // U+FF5E (BMP) sorts before U+1F600 (a surrogate pair) by code point, after it by UTF-16 unit + List keys = new ArrayList<>(List.of("😀", "~")); + var node = PLAIN.createObjectNode(); + for (String k : keys) node.put(k, 1); + Map body = com.metaobjects.generator.spring.runtime.UnknownBodyKeys.refusal(node, java.util.Set.of()); + assertEquals(List.of("~", "😀"), body.get("fields")); + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt index 384ab1a084..52a6b01a6f 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/AuthorApiServer.kt @@ -185,6 +185,7 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map ?: return sendJson(exchange, 400, validationBody(listOf(issue("invalid_type", "request body must be a JSON object")))) + if (refuseUnknownKeys(exchange, body)) return // FR-036: enforce @maxLength (name 100 / bio 1000) at the wire boundary — a too-long // present value is a 400 {error:"validation"} BEFORE the insert (mirrors the generated // controller's validator.validate on POST). @@ -220,6 +221,7 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { @Suppress("UNCHECKED_CAST") val body = readJsonBody(exchange) as? Map ?: return sendJson(exchange, 400, validationBody(listOf(issue("invalid_type", "request body must be a JSON object")))) + if (refuseUnknownKeys(exchange, body)) return // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an // omitted required field is untouched (never a 400). @@ -277,6 +279,27 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { * Only present, non-null strings are checked — absent fields and nulls are handled by the * per-verb guards. Mirrors the generated controller's field constraints over HTTP. */ + /** Answers 400 unknown_fields (keys in code-point order) when [body] carries a key Author does not declare. */ + private fun refuseUnknownKeys(exchange: HttpExchange, body: Map): Boolean { + val unknown = body.keys.filter { it !in DECLARED_KEYS }.sortedWith { x, y -> compareCodePoints(x, y) } + if (unknown.isEmpty()) return false + sendJson(exchange, 400, linkedMapOf("error" to "unknown_fields", "fields" to unknown)) + return true + } + + private fun compareCodePoints(a: String, b: String): Int { + var i = 0 + var j = 0 + while (i < a.length && j < b.length) { + val ca = a.codePointAt(i) + val cb = b.codePointAt(j) + if (ca != cb) return ca.compareTo(cb) + i += Character.charCount(ca) + j += Character.charCount(cb) + } + return (a.length - i).compareTo(b.length - j) + } + private fun lengthIssues(body: Map): MutableList> { val out = ArrayList>() (body["name"] as? String)?.let { if (it.length > 100) out.add(issue("too_big", "too long", "name")) } @@ -534,6 +557,9 @@ class AuthorApiServer(private val pg: PostgresContainer) : AutoCloseable { companion object { // Mirrors the TS server's SORT_ALLOWLIST; cross-port contract. + /** Every field Author declares, as named on the wire; the corpus sets no api.unknownFields, so any other body key is refused. */ + private val DECLARED_KEYS = setOf("id", "name", "bio", "createdAt", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt") + private val SORT_ALLOWLIST = setOf("id", "name", "createdAt") // Cross-port contract: each field's DECLARED @sortableDefaultOrder — the direction diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/ApiSettingsGeneratedApiContractConformanceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/ApiSettingsGeneratedApiContractConformanceTest.kt new file mode 100644 index 0000000000..34fd204adc --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/ApiSettingsGeneratedApiContractConformanceTest.kt @@ -0,0 +1,74 @@ +package com.metaobjects.integration.kotlin.api.apisettings + +import com.fasterxml.jackson.databind.ObjectMapper +import com.metaobjects.integration.kotlin.api.ApiContractAssertions +import com.metaobjects.integration.kotlin.api.ApiContractScenarioLoader +import com.metaobjects.integration.kotlin.api.ApiContractScenarios.ApiScenario +import com.metaobjects.integration.kotlin.api.apisettings.generated.GeneratedItemControllerHarness +import org.junit.jupiter.api.AfterAll +import org.junit.jupiter.api.BeforeAll +import org.junit.jupiter.api.DisplayName +import org.junit.jupiter.api.assertDoesNotThrow +import org.junit.jupiter.params.ParameterizedTest +import org.junit.jupiter.params.provider.Arguments +import org.junit.jupiter.params.provider.MethodSource +import java.nio.charset.StandardCharsets +import java.nio.file.Files +import java.util.stream.Stream + +/** + * `api-settings` api-contract sub-corpus, Kotlin GENERATED-controller lane: `api.pagination` (a default page + * size and a hard cap on `?limit=`) and `api.unknownFields: drop`, both read from the sub-corpus's own + * `.metaobjects/config.json`. Generated lane only: a hand-rolled reference server would satisfy it by construction. + */ +@DisplayName("API contract api-settings — GENERATED Kotlin Spring controller over embedded Tomcat") +internal class ApiSettingsGeneratedApiContractConformanceTest { + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + fun scenario(name: String, scenario: ApiScenario) { + assertDoesNotThrow { + HARNESS.reset() // fresh H2 + seed per scenario (create mutates) + for (req in scenario.requests) { + val res = HARNESS.exchange(req.method, req.path, req.body) + val parsed = HARNESS.parseBody(res.body) + ApiContractAssertions.assertResponse(scenario.name, req, res.status, parsed) + } + } + } + + companion object { + private val MAPPER = ObjectMapper().enable(com.fasterxml.jackson.databind.DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS) + + @Suppress("UNCHECKED_CAST") + private val SEED: List> by lazy { + val corpus = ApiContractScenarioLoader.findCorpusRoot() + val text = Files.readString(corpus.resolve("api-settings/seed.json"), StandardCharsets.UTF_8) + val parsed = MAPPER.readValue(text, Map::class.java) as Map + (parsed["rows"] as? List>) ?: error("api-settings/seed.json: missing 'rows'") + } + + private lateinit var HARNESS: GeneratedItemControllerHarness + + @BeforeAll + @JvmStatic + fun setUp() { + val corpus = ApiContractScenarioLoader.findCorpusRoot() + val genDir = Files.createTempDirectory("api-settings-generated-kotlin-controller") + HARNESS = GeneratedItemControllerHarness(corpus, genDir, SEED) + } + + @AfterAll + @JvmStatic + fun tearDown() { + if (::HARNESS.isInitialized) HARNESS.close() + } + + @JvmStatic + fun scenarios(): Stream { + val corpus = ApiContractScenarioLoader.findCorpusRoot() + val scenarios = ApiContractScenarioLoader.loadScenarios(corpus.resolve("api-settings/scenarios")) + return scenarios.stream().map { Arguments.of(it.name, it) } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/generated/GeneratedItemControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/generated/GeneratedItemControllerHarness.kt new file mode 100644 index 0000000000..4ca65785f8 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/apisettings/generated/GeneratedItemControllerHarness.kt @@ -0,0 +1,152 @@ +package com.metaobjects.integration.kotlin.api.apisettings.generated + +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.databind.SerializationFeature +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import com.metaobjects.integration.kotlin.api.TomcatHost +import com.metaobjects.generator.kotlin.KotlinEntityGenerator +import com.metaobjects.generator.kotlin.KotlinExposedTableGenerator +import com.metaobjects.generator.kotlin.KotlinFilterAllowlistGenerator +import com.metaobjects.generator.kotlin.KotlinNamesGenerator +import com.metaobjects.generator.kotlin.KotlinSpringControllerGenerator +import com.metaobjects.loader.uri.URIHelper +import com.metaobjects.metadata.ktx.loadUris +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import org.jetbrains.exposed.sql.Database +import org.jetbrains.exposed.sql.SchemaUtils +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.transactions.transaction +import java.net.URI +import java.nio.file.Files +import java.nio.file.Path +import java.util.concurrent.atomic.AtomicInteger +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText + +/** + * Host the GENERATED Kotlin Spring `ItemController` for the api-settings corpus over real HTTP + * (embedded Tomcat, in-memory H2). Same generate, compile, serve mechanism as the write-through harness. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class GeneratedItemControllerHarness( + corpusRoot: Path, + genDir: Path, + private val rows: List>, +) : AutoCloseable { + + private val mapper: ObjectMapper = ObjectMapper() + .registerKotlinModule() + .registerModule(JavaTimeModule()) + .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) + + private val controllerClass: Class<*> + private val itemTable: Table + private val dbSeq = AtomicInteger(0) + private var host: TomcatHost? = null + private var db: Database? = null + + init { + val metaJson = corpusRoot.resolve("api-settings/meta.json") + val uri: URI = URIHelper.toURI("model:file:" + metaJson.toAbsolutePath().toString().replace('\\', '/')) + val loader = loadUris("api-contract-api-settings-generated", listOf(uri)) + // ADR-0058 — this corpus's committed .metaobjects/config.json (at corpusRoot, + // fixtures/api-contract-conformance/) declares naming.columns: literal; every field + // here declares no @column, so the generated columns and the runtime's reads of them + // depend on this. Read with the SAME NeutralConfig reader every port uses. + loader.root.physicalNaming = + com.metaobjects.config.SourceResolver.resolvePhysicalNaming(corpusRoot.resolve("api-settings")) + // ADR-0019 Amendment 2: the same file pins wire (decimals: string, nullMembers: omit), so the + // generated serializers write the portable form. Discovered by walking up from the meta file, so a + // sub-corpus directory (jsonb/, m2m/, ...) reads the config at the api-contract corpus root. + loader.root.wireFormat = com.metaobjects.config.SourceResolver + .discoverWireFormat(listOf(metaJson.toUri())) + .orElse(com.metaobjects.config.WireFormat.DEFAULT) + + // The api block (api.pagination, api.unknownFields) comes from the sub-corpus's OWN + // .metaobjects/config.json, found by walking up from its meta file, the way an adopter's would. + loader.root.apiSettings = com.metaobjects.config.SourceResolver + .discoverApiSettings(listOf(metaJson.toUri())) + .orElse(com.metaobjects.config.ApiSettings.DEFAULT) + + val srcDir = genDir.resolve("src") + Files.createDirectories(srcDir) + for (g in listOf( + KotlinEntityGenerator(), + KotlinNamesGenerator(), + KotlinExposedTableGenerator(), + KotlinFilterAllowlistGenerator(), + KotlinSpringControllerGenerator(), + )) { + // Task 6 — free compile-level proof that Names const val references + // actually resolve; useNames is ignored by every other generator. This is + // also the write-through lane — the ONE existing corpus fixture that + // exercises Names substitution against emitWriteThrough's TWO Exposed + // objects (OrderTable + OrderView) sharing one entity's column names. + g.setArgs(mapOf("outputDir" to srcDir.toString(), "useNames" to "true")) + g.execute(loader) + } + + val sources = Files.walk(srcDir).use { stream -> + stream.filter { it.isRegularFile() && it.toString().endsWith(".kt") } + .map { path -> SourceFile.kotlin(srcDir.relativize(path).toString().replace('/', '_'), path.readText()) } + .toList() + } + check(sources.isNotEmpty()) { "no generated .kt sources under $srcDir" } + + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + check(result.exitCode == KotlinCompilation.ExitCode.OK) { + "generated Kotlin failed to compile:\n${result.messages}" + } + + this.controllerClass = result.classLoader.loadClass(CONTROLLER_FQCN) + this.itemTable = result.classLoader.loadClass(ITEM_TABLE_FQCN).getDeclaredField("INSTANCE").get(null) as Table + } + + /** Rebuild a fresh in-memory H2 + seed + controller + Tomcat. */ + fun reset() { + val dbName = "api_settings_${dbSeq.incrementAndGet()}" + val db = Database.connect("jdbc:h2:mem:$dbName;DB_CLOSE_DELAY=-1;MODE=PostgreSQL", driver = "org.h2.Driver") + this.db = db + transaction(db) { + SchemaUtils.create(itemTable) + // Ids are assigned by the identity column in seed order (1..n), so a later POST does not collide. + for (r in rows) { + exec("INSERT INTO items (title) VALUES (${sqlStr(r["title"])})") + } + } + + val controller = controllerClass + .getDeclaredConstructor(ObjectMapper::class.java, jakarta.validation.Validator::class.java) + .newInstance(mapper, jakarta.validation.Validation.buildDefaultValidatorFactory().validator) + host?.close() + host = TomcatHost.start(mapper, controller) + } + + fun exchange(method: String, path: String, jsonBody: Any?): Response { + val server = host ?: error("reset() must be called before exchange(...)") + val res = server.exchange(method, path, jsonBody?.let { mapper.writeValueAsString(it) }) + return Response(res.status, res.body) + } + + fun parseBody(body: String?): Any? = TomcatHost.parseBody(mapper, body) + + override fun close() { + host?.close() // H2 in-mem is reclaimed at JVM exit (DB_CLOSE_DELAY=-1). + } + + data class Response(val status: Int, val body: String) + + private companion object { + const val ENTITY_PKG = "acme.paging" + const val CONTROLLER_FQCN = "$ENTITY_PKG.ItemController" + const val ITEM_TABLE_FQCN = "$ENTITY_PKG.ItemTable" + + fun sqlStr(v: Any?): String = "'" + (v as String).replace("'", "''") + "'" + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/PatchTristateEdgeCasesTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/PatchTristateEdgeCasesTest.kt index 4d35ea3c1d..bc603df23f 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/PatchTristateEdgeCasesTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/PatchTristateEdgeCasesTest.kt @@ -87,8 +87,10 @@ class PatchTristateEdgeCasesTest { // empty patch-settable set must emit NO ObjectMapper ctor param and NO TypeReference // import (either would be an unused symbol → a real consumer's -Werror build fails). val src = Files.readString(outDir.resolve("acme/cfg/SettingsController.kt")) - assertTrue("ObjectMapper" !in src, - "a no-settable-field controller must not take an ObjectMapper ctor param:\n$src") + // (Under the default api.unknownFields: reject the create binds the raw body through the + // ObjectMapper to read its keys first, so the mapper IS used and is injected.) + assertTrue("objectMapper.treeToValue(body, Settings::class.java)" in src, + "the injected ObjectMapper must be used by the create handler:\n$src") assertTrue("TypeReference" !in src, "a no-settable-field controller must not import TypeReference:\n$src") diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/UnknownFieldsHostMapperIndependenceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/UnknownFieldsHostMapperIndependenceTest.kt new file mode 100644 index 0000000000..2811613e83 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/UnknownFieldsHostMapperIndependenceTest.kt @@ -0,0 +1,183 @@ +package com.metaobjects.integration.kotlin.api.generated + +import com.fasterxml.jackson.databind.DeserializationFeature +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.databind.SerializationFeature +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import com.metaobjects.config.ApiSettings +import com.metaobjects.generator.kotlin.KotlinEntityGenerator +import com.metaobjects.generator.kotlin.KotlinExposedTableGenerator +import com.metaobjects.generator.kotlin.KotlinFilterAllowlistGenerator +import com.metaobjects.generator.kotlin.KotlinSpringControllerGenerator +import com.metaobjects.integration.kotlin.api.TomcatHost +import com.metaobjects.loader.uri.URIHelper +import com.metaobjects.metadata.ktx.loadUris +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import org.jetbrains.exposed.sql.Database +import org.jetbrains.exposed.sql.SchemaUtils +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.transactions.transaction +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.io.TempDir +import java.nio.file.Files +import java.nio.file.Path +import java.util.concurrent.atomic.AtomicInteger +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText + +/** + * `api.unknownFields`: a generated Kotlin create, PUT and PATCH answers an undeclared body key the same way + * whatever the host's `ObjectMapper` does with an unknown property. The keys are read off the raw body before + * it is bound, so a mapper with `FAIL_ON_UNKNOWN_PROPERTIES` on and one with it off give identical answers under + * both `reject` and `drop`, over a real HTTP exchange (embedded Tomcat, in-memory H2), plain entity and TPH. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +internal class UnknownFieldsHostMapperIndependenceTest { + + @TempDir + lateinit var tmp: Path + + private val dbSeq = AtomicInteger(0) + + /** One generated and compiled controller pair for a given `api.unknownFields` setting. */ + private inner class Built(api: ApiSettings, name: String) { + val thingController: Class<*> + val authController: Class<*> + val tables: List + + init { + val metaFile = tmp.resolve("$name-meta.json") + Files.writeString(metaFile, META) + val uri = URIHelper.toURI("model:file:" + metaFile.toAbsolutePath().toString().replace('\\', '/')) + val loader = loadUris("unknown-fields-$name", listOf(uri)) + loader.root.apiSettings = api + val srcDir = tmp.resolve("gen-$name") + Files.createDirectories(srcDir) + for (g in listOf( + KotlinEntityGenerator(), + KotlinExposedTableGenerator(), + KotlinFilterAllowlistGenerator(), + KotlinSpringControllerGenerator(), + )) { + g.setArgs(mapOf("outputDir" to srcDir.toString())) + g.execute(loader) + } + val sources = Files.walk(srcDir).use { stream -> + stream.filter { it.isRegularFile() && it.toString().endsWith(".kt") } + .map { SourceFile.kotlin(srcDir.relativize(it).toString().replace('/', '_'), it.readText()) } + .toList() + } + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + check(result.exitCode == KotlinCompilation.ExitCode.OK) { "generated Kotlin failed to compile:\n${result.messages}" } + thingController = result.classLoader.loadClass("acme.thing.ThingController") + authController = result.classLoader.loadClass("acme.thing.AuthController") + tables = listOf("acme.thing.ThingTable", "acme.thing.AuthTable").map { + result.classLoader.loadClass(it).getDeclaredField("INSTANCE").get(null) as Table + } + } + + /** The answers of [requests] (method, path, body) through a host whose mapper fails on unknown properties or not. */ + fun answers(failOnUnknown: Boolean, requests: List>): List { + val mapper = ObjectMapper() + .registerKotlinModule() + .registerModule(JavaTimeModule()) + .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) + .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, failOnUnknown) + val db = Database.connect("jdbc:h2:mem:unknown_${dbSeq.incrementAndGet()};DB_CLOSE_DELAY=-1;MODE=PostgreSQL", driver = "org.h2.Driver") + transaction(db) { SchemaUtils.create(*tables.toTypedArray()) } + val validator = jakarta.validation.Validation.buildDefaultValidatorFactory().validator + val things = thingController.getDeclaredConstructor(ObjectMapper::class.java, jakarta.validation.Validator::class.java).newInstance(mapper, validator) + val auths = authController.getDeclaredConstructor(ObjectMapper::class.java, jakarta.validation.Validator::class.java).newInstance(mapper, validator) + TomcatHost.start(mapper, things, auths).use { host -> + return requests.map { (method, path, body) -> + val res = host.exchange(method, path, body) + // A refusal is compared in full; any other answer by its status alone. + if (res.status == 400) "400 ${res.body}" else res.status.toString() + } + } + } + } + + /** The same requests through a failing and a tolerant mapper: one list of answers. */ + private fun same(b: Built, requests: List>): List { + val strict = b.answers(true, requests) + val lax = b.answers(false, requests) + assertEquals(strict, lax, "the host mapper must not change the answer") + return strict + } + + @Test + fun rejectAnswersUnknownFieldsTheSameUnderEveryMapper() { + val b = Built(ApiSettings.DEFAULT, "reject") + val refusal = "400 {\"error\":\"unknown_fields\",\"fields\":[\"isAdmin\",\"zebra\"]}" + val answers = same( + b, + listOf( + // create, PUT and PATCH, plain and TPH: every undeclared key listed in code-point order + Triple("POST", "/api/things", """{"title":"t","zebra":1,"isAdmin":true}"""), + Triple("PUT", "/api/things/1", """{"title":"t","zebra":1,"isAdmin":true}"""), + Triple("PATCH", "/api/things/1", """{"title":"t","zebra":1,"isAdmin":true}"""), + Triple("POST", "/api/auths/bridge", """{"quantity":1,"zebra":1,"isAdmin":true}"""), + Triple("PATCH", "/api/auths/bridge/1", """{"quantity":1,"zebra":1,"isAdmin":true}"""), + // declared keys the client may not set stay known (dropped, not refused), and so does a sibling's column + Triple("POST", "/api/things", """{"id":9,"title":"t","role":"admin"}"""), + Triple("POST", "/api/auths/bridge", """{"quantity":1,"approver":"x"}"""), + ), + ) + assertEquals(List(5) { refusal } + listOf("201", "201"), answers) + } + + @Test + fun dropIgnoresUnknownFieldsTheSameUnderEveryMapper() { + val b = Built(ApiSettings(null, ApiSettings.UnknownFields.DROP), "drop") + val answers = same( + b, + listOf( + Triple("POST", "/api/things", """{"title":"t","zebra":1,"isAdmin":true}"""), + Triple("PATCH", "/api/things/1", """{"title":"u","zebra":1}"""), + Triple("POST", "/api/auths/bridge", """{"quantity":1,"zebra":1}"""), + Triple("PATCH", "/api/auths/bridge/1", """{"quantity":2,"zebra":1}"""), + ), + ) + assertEquals(listOf("201", "200", "201", "200"), answers) + } + + @Test + fun aBodyThatIsNotAnObjectIsLeftToTheExistingHandling() { + val b = Built(ApiSettings.DEFAULT, "nonobject") + val answers = same(b, listOf(Triple("POST", "/api/things", "[1, 2]"))) + assertEquals(1, answers.size) + check(answers[0].startsWith("400 ") && "unknown_fields" !in answers[0]) { answers[0] } + } + + private companion object { + val META = """ + { "metadata.root": { "package": "acme::thing", "children": [ + { "object.entity": { "name": "Thing", "children": [ + { "source.rdb": { "@table": "things" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.string": { "name": "role", "@mutability": "readOnly" } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Copay"] } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", + "children": [ { "field.int": { "name": "quantity" } } ] } }, + { "object.entity": { "name": "CopayAuth", "extends": "Auth", "@discriminatorValue": "Copay", + "children": [ { "field.string": { "name": "approver" } } ] } } + ] } } + """.trimIndent() + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiSettingsGeneratedApiContractConformanceTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiSettingsGeneratedApiContractConformanceTest.java new file mode 100644 index 0000000000..1214c3b20c --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiSettingsGeneratedApiContractConformanceTest.java @@ -0,0 +1,83 @@ +package com.metaobjects.integration.api; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.metaobjects.integration.api.ApiContractScenarios.ApiRequest; +import com.metaobjects.integration.api.ApiContractScenarios.ApiScenario; +import com.metaobjects.integration.api.generated.GeneratedItemControllerHarness; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.stream.Stream; + +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; + +/** + * the Java GENERATED-controller lane for the {@code api-settings} api-contract + * sub-corpus: {@code api.pagination} (a default page size and a hard cap on {@code ?limit=}) and + * {@code api.unknownFields: drop}, both read from the sub-corpus's own + * {@code .metaobjects/config.json}. + * + *

Generated lane ONLY, on every port: what is under test is whether the GENERATOR emits routes + * that honour the marks, which a hand-rolled reference server would satisfy by construction.

+ * + *

Run on-demand: + * {@code mvn -f server/java/integration-tests/pom.xml test -Dtest=ApiSettingsGeneratedApiContractConformanceTest}

+ */ +@DisplayName("API contract api-settings — GENERATED Spring controller (codegen-spring) over embedded Tomcat") +final class ApiSettingsGeneratedApiContractConformanceTest { + + private static final Path CONFIG_ROOT = ApiContractScenarioLoader.findCorpusRoot(); + private static final Path ROOT = CONFIG_ROOT.resolve("api-settings"); + private static final List SCENARIOS = + ApiContractScenarioLoader.loadScenarios(ROOT.resolve("scenarios")); + + private static GeneratedItemControllerHarness HARNESS; + + @SuppressWarnings("unchecked") + private static List> seedRows() throws IOException { + String text = Files.readString(ROOT.resolve("seed.json"), StandardCharsets.UTF_8); + Object rows = new ObjectMapper().readValue(text, Map.class).get("rows"); + if (!(rows instanceof List)) throw new IllegalStateException("api-settings/seed.json: missing 'rows' array"); + return (List>) rows; + } + + @BeforeAll + static void setUp() throws Exception { + Path genDir = Files.createTempDirectory("api-settings-generated-controller"); + HARNESS = new GeneratedItemControllerHarness(ROOT, genDir, seedRows()); + } + + @AfterAll + static void tearDown() throws Exception { + if (HARNESS != null) HARNESS.close(); + } + + static Stream scenarios() { + return SCENARIOS.stream().map(s -> Arguments.of(s.name(), s)); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + void scenario(String name, ApiScenario scenario) { + assertDoesNotThrow(() -> { + HARNESS.reset(); + for (ApiRequest req : scenario.requests()) { + GeneratedItemControllerHarness.Response res = + HARNESS.exchange(req.method(), req.path(), req.body()); + Object parsed = HARNESS.parseBody(res.body()); + ApiContractAssertions.assertResponse(scenario.name(), req, res.status(), parsed); + } + }); + } + +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java index 6b02a2745e..881bb2c254 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/AuthorApiServer.java @@ -52,6 +52,13 @@ final class AuthorApiServer implements AutoCloseable { // Mirrors the TS server's SORT_ALLOWLIST; cross-port contract. private static final Set SORT_ALLOWLIST = Set.of("id", "name", "createdAt"); + // Every field Author declares (meta.json), as named on the wire. The corpus sets no + // `api.unknownFields`, so the default applies: a write body key outside this set is refused + // with 400 {"error": "unknown_fields", "fields": [...]} before anything is read or written. A + // declared key the client may not set (id, @autoSet, writeOnce on update) is known and dropped. + private static final Set DECLARED_KEYS = Set.of( + "id", "name", "bio", "createdAt", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt"); + // Cross-port contract: each field's DECLARED @sortableDefaultOrder — the direction a // {@code ?sort=} carrying no {@code :dir} takes. Hardcoded from meta.json for // the same reason the allowlist above is: this is the hand-rolled fixed point the @@ -445,6 +452,7 @@ private void createAuthor(HttpExchange exchange) throws IOException, SQLExceptio return; } Map body = (Map) parsed; + if (refuseUnknownKeys(exchange, body)) return; // FR-036 wire-tier constraint enforcement: reject an over-length name/bio with the // cross-port {"error":"validation"} envelope BEFORE persisting (mirrors the generated // controller's @Size(max=…) rules). @@ -497,6 +505,7 @@ private void updateAuthor(HttpExchange exchange, String idStr) throws IOExceptio return; } Map body = (Map) parsed; + if (refuseUnknownKeys(exchange, body)) return; // FR-035 PATCH-2: an explicit null on a @required field (name / createdAt, per // meta.json) is a 400 — a present null on the nullable bio clears it, and an @@ -634,6 +643,15 @@ private Map rowToMap(ResultSet rs) throws SQLException { return row; } + /** Answers {@code 400 unknown_fields} (keys in code-point order) when {@code body} carries an undeclared key. */ + private boolean refuseUnknownKeys(HttpExchange exchange, Map body) throws IOException { + java.util.TreeSet unknown = new java.util.TreeSet<>(body.keySet()); + unknown.removeAll(DECLARED_KEYS); + if (unknown.isEmpty()) return false; + sendJson(exchange, 400, Map.of("error", "unknown_fields", "fields", new ArrayList<>(unknown))); + return true; + } + private Object readJsonBody(HttpExchange exchange) throws IOException { byte[] bytes = exchange.getRequestBody().readAllBytes(); if (bytes.length == 0) return null; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedItemControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedItemControllerHarness.java new file mode 100644 index 0000000000..51bbf9c8fc --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedItemControllerHarness.java @@ -0,0 +1,177 @@ +package com.metaobjects.integration.api.generated; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.metaobjects.generator.spring.SpringControllerGenerator; +import com.metaobjects.generator.spring.SpringDtoGenerator; +import com.metaobjects.generator.spring.SpringFilterAllowlistGenerator; +import com.metaobjects.generator.spring.SpringRepositoryGenerator; +import com.metaobjects.integration.api.TomcatHost; +import com.metaobjects.loader.LoaderOptions; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.uri.URIHelper; + +import jakarta.validation.Validation; +import jakarta.validation.Validator; + +import javax.tools.DiagnosticCollector; +import javax.tools.JavaCompiler; +import javax.tools.JavaFileObject; +import javax.tools.ToolProvider; + +import java.io.File; +import java.lang.reflect.Constructor; +import java.net.URI; +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * Host the GENERATED Spring {@code ItemController} of the {@code api-settings} sub-corpus over real + * HTTP (embedded Tomcat) and drive its scenarios. The project's {@code api} block + * ({@code api.pagination}, {@code api.unknownFields}) reaches the generators through the + * sub-corpus's own {@code .metaobjects/config.json}, the way an adopter's does. The only + * hand-written piece is {@link InMemoryItemRepositorySource}, the consumer seam. + */ +public final class GeneratedItemControllerHarness implements AutoCloseable { + + private static final String ENTITY_PKG = "acme.paging"; + + private final ObjectMapper mapper = new ObjectMapper(); + private final URLClassLoader classLoader; + private final Class dtoClass; + private final Constructor controllerCtor; + private final Constructor repoCtor; + private final List> seedRows; + private final Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); + + private TomcatHost host; + + /** + * @param corpusRoot {@code fixtures/api-contract-conformance/api-settings}: holds the model and + * the {@code .metaobjects/config.json} the settings are read from + */ + public GeneratedItemControllerHarness(Path corpusRoot, Path genDir, List> seedRows) + throws Exception { + this.seedRows = seedRows; + Path srcDir = genDir.resolve("src"); + Path classesDir = genDir.resolve("classes"); + Files.createDirectories(srcDir); + Files.createDirectories(classesDir); + + MetaDataLoader loader = loadCorpus(corpusRoot.resolve("meta.json")); + List project = List.of(corpusRoot.toUri()); + loader.getRoot().setPhysicalNaming(com.metaobjects.config.SourceResolver.resolvePhysicalNaming(corpusRoot)); + loader.getRoot().setWireFormat(com.metaobjects.config.SourceResolver.discoverWireFormat(project) + .orElse(com.metaobjects.config.WireFormat.DEFAULT)); + loader.getRoot().setApiSettings(com.metaobjects.config.SourceResolver.discoverApiSettings(project) + .orElse(com.metaobjects.config.ApiSettings.DEFAULT)); + + runGenerator(new SpringControllerGenerator(), loader, srcDir); + runGenerator(new SpringDtoGenerator(), loader, srcDir); + runGenerator(new SpringRepositoryGenerator(), loader, srcDir); + runGenerator(new SpringFilterAllowlistGenerator(), loader, srcDir); + + Path repoImpl = srcDir.resolve(ENTITY_PKG.replace('.', '/')).resolve("InMemoryItemRepository.java"); + Files.createDirectories(repoImpl.getParent()); + Files.writeString(repoImpl, InMemoryItemRepositorySource.SOURCE); + + compile(srcDir, classesDir); + + this.classLoader = new URLClassLoader( + new URL[]{ classesDir.toUri().toURL() }, getClass().getClassLoader()); + this.dtoClass = classLoader.loadClass(ENTITY_PKG + ".ItemDto"); + Class repoInterface = classLoader.loadClass(ENTITY_PKG + ".ItemRepository"); + this.controllerCtor = classLoader.loadClass(ENTITY_PKG + ".ItemController") + .getDeclaredConstructor(repoInterface, ObjectMapper.class, Validator.class); + this.repoCtor = classLoader.loadClass(InMemoryItemRepositorySource.FQCN).getDeclaredConstructor(List.class); + } + + /** Re-seed for a scenario: a fresh repository, controller and Tomcat. */ + public void reset() throws Exception { + List dtos = new ArrayList<>(); + for (Map row : seedRows) dtos.add(mapper.convertValue(row, dtoClass)); + Object repo = repoCtor.newInstance(dtos); + Object controller = controllerCtor.newInstance(repo, mapper, validator); + if (host != null) host.close(); + this.host = TomcatHost.start(mapper, controller); + } + + public Response exchange(String method, String path, Object jsonBody) throws Exception { + TomcatHost.Response res = host.exchange( + method, path, jsonBody == null ? null : mapper.writeValueAsString(jsonBody)); + return new Response(res.status(), res.body()); + } + + public Object parseBody(String body) { + return TomcatHost.parseBody(mapper, body); + } + + @Override + public void close() throws Exception { + if (host != null) host.close(); + classLoader.close(); + } + + public record Response(int status, String body) {} + + private static MetaDataLoader loadCorpus(Path... metaJsons) { + List uris = new ArrayList<>(); + for (Path metaJson : metaJsons) { + uris.add(URIHelper.toURI("model:file:" + metaJson.toAbsolutePath().toString().replace('\\', '/'))); + } + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), + MetaDataLoader.SUBTYPE_MANUAL, + "api-contract-api-settings-generated"); + loader.setSourceURIs(uris); + loader.init(); + return loader; + } + + private static void runGenerator(Object generator, MetaDataLoader loader, Path outDir) { + Map args = new HashMap<>(); + args.put("outputDir", outDir.toString()); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).setArgs(args); + ((com.metaobjects.generator.direct.MultiFileDirectGeneratorBase) generator).execute(loader); + } + + private static void compile(Path srcDir, Path classesDir) throws Exception { + List sources; + try (Stream s = Files.walk(srcDir)) { + sources = s.filter(p -> p.toString().endsWith(".java")) + .map(Path::toFile) + .collect(Collectors.toList()); + } + if (sources.isEmpty()) throw new IllegalStateException("no generated .java sources under " + srcDir); + + JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); + if (javac == null) { + throw new IllegalStateException("JDK (not JRE) required — getSystemJavaCompiler() returned null"); + } + String cp = System.getProperty("java.class.path"); + DiagnosticCollector diags = new DiagnosticCollector<>(); + var fm = javac.getStandardFileManager(diags, null, StandardCharsets.UTF_8); + // -parameters: the generated controller relies on reflective parameter names, as Spring Boot builds do. + List opts = List.of("-classpath", cp, "-d", classesDir.toString(), "-parameters"); + + if (!javac.getTask(null, fm, diags, opts, null, fm.getJavaFileObjectsFromFiles(sources)).call()) { + StringBuilder sb = new StringBuilder("generated sources failed to compile:\n"); + for (var d : diags.getDiagnostics()) { + sb.append(" ").append(d.getKind()).append(": ").append(d.getMessage(null)).append('\n'); + if (d.getSource() != null) { + sb.append(" at ").append(d.getSource().getName()) + .append(':').append(d.getLineNumber()).append('\n'); + } + } + throw new IllegalStateException(sb.toString()); + } + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryItemRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryItemRepositorySource.java new file mode 100644 index 0000000000..6255c5e039 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryItemRepositorySource.java @@ -0,0 +1,107 @@ +package com.metaobjects.integration.api.generated; + +/** + * The Java SOURCE for the in-memory {@code acme.paging.ItemRepository} impl behind the GENERATED + * {@code ItemController} in the {@code api-settings} sub-corpus. Test scaffolding, not a + * conformance subject: it honours {@code limit} and {@code offset} exactly, so a page is whatever + * size the generated controller asked for. + */ +final class InMemoryItemRepositorySource { + + private InMemoryItemRepositorySource() {} + + static final String FQCN = "acme.paging.InMemoryItemRepository"; + + static final String SOURCE = """ + package acme.paging; + + import com.metaobjects.generator.spring.runtime.FilterPredicate; + + import java.util.ArrayList; + import java.util.Comparator; + import java.util.List; + import java.util.Map; + import java.util.Optional; + import java.util.concurrent.atomic.AtomicLong; + + public final class InMemoryItemRepository implements ItemRepository { + + private final List rows = new ArrayList<>(); + private final AtomicLong nextId = new AtomicLong(1); + + public InMemoryItemRepository(List seed) { + long max = 0; + for (ItemDto a : seed) { + rows.add(a); + if (a.id() != null && a.id() > max) max = a.id(); + } + nextId.set(max + 1); + } + + @Override + public List list(int limit, int offset, SortClause sort, List filters) { + List out = new ArrayList<>(rows); + Comparator c = sort != null && "title".equals(sort.field()) + ? Comparator.comparing(ItemDto::title, Comparator.nullsLast(Comparator.naturalOrder())) + : Comparator.comparing(ItemDto::id, Comparator.nullsLast(Comparator.naturalOrder())); + if (sort != null && "desc".equalsIgnoreCase(sort.direction())) c = c.reversed(); + out.sort(c); + int from = Math.min(offset, out.size()); + int to = (int) Math.min((long) from + limit, out.size()); + return new ArrayList<>(out.subList(from, to)); + } + + @Override + public long count(List filters) { + return rows.size(); + } + + @Override + public Optional findById(Long id) { + for (ItemDto a : rows) if (id.equals(a.id())) return Optional.of(a); + return Optional.empty(); + } + + @Override + public ItemDto create(ItemDto dto) { + Long id = dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()); + ItemDto saved = new ItemDto(id, dto.title()); + rows.add(saved); + return saved; + } + + @Override + public Optional update(Long id, ItemDto dto) { + for (int i = 0; i < rows.size(); i++) { + ItemDto cur = rows.get(i); + if (id.equals(cur.id())) { + ItemDto merged = new ItemDto(id, dto.title() != null ? dto.title() : cur.title()); + rows.set(i, merged); + return Optional.of(merged); + } + } + return Optional.empty(); + } + + @Override + public Optional patch(Long id, ItemPatch patch) { + Map assigned = patch.assignedValues(); + for (int i = 0; i < rows.size(); i++) { + ItemDto cur = rows.get(i); + if (id.equals(cur.id())) { + ItemDto merged = new ItemDto(id, + assigned.containsKey("title") ? (String) assigned.get("title") : cur.title()); + rows.set(i, merged); + return Optional.of(merged); + } + } + return Optional.empty(); + } + + @Override + public boolean delete(Long id) { + return rows.removeIf(a -> id.equals(a.id())); + } + } + """; +} diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java index dd6c418262..1cf8f5323a 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java @@ -803,6 +803,10 @@ protected MetaDataLoader createLoader(ClassLoader projectClassLoader) { loader.getRoot().setEnumDefaults(resolveProjectConfig(srcDir) .map(com.metaobjects.config.NeutralConfig::getEnumDefaults) .orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); + // The `api` block (`api.pagination`, `api.unknownFields`), from the same file, read by the generators. + loader.getRoot().setApiSettings(resolveProjectConfig(srcDir) + .map(com.metaobjects.config.NeutralConfig::getApiSettings) + .orElse(com.metaobjects.config.ApiSettings.DEFAULT)); getLog().info("MetaData Mojo > Create Loader: " + loader.toString()); diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java index 23030af965..002c1260a2 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java @@ -304,6 +304,7 @@ private void verifyDefaultSet() throws MojoExecutionException, MojoFailureExcept */ private void runFieldLintAdvisory(MetaDataLoader loader) { if (advisoriesDone || noFieldLint || "1".equals(System.getenv(FieldLint.ENV_OPT_OUT))) return; + runUnboundedListAdvisory(loader); List findings = new ArrayList<>(); try { findings.addAll(FieldLint.lintReferenceFields(loader)); @@ -319,6 +320,28 @@ private void runFieldLintAdvisory(MetaDataLoader loader) { } } + /** + * Advisory {@code unbounded-list-pages}: a project that generates REST controllers and sets no + * {@code api.pagination} has list routes that answer 50 rows when {@code limit} is omitted and + * accept any {@code limit}. Never fails the build. + */ + private void runUnboundedListAdvisory(MetaDataLoader loader) { + try { + if (loader.getRoot().getApiSettings().hasPagination()) return; + boolean generatesRoutes = false; + for (GeneratorParam g : getGenerators() != null ? getGenerators() : List.of()) { + String name = g.getClassname(); + if (name != null && name.endsWith("ControllerGenerator")) generatesRoutes = true; + } + if (!generatesRoutes) return; + getLog().warn("metaobjects:verify — unbounded-list-pages (advisory — does not fail the build): " + + "generated list routes take any limit and have no project-wide page size; " + + "set api.pagination ({\"defaultLimit\": 50, \"maxLimit\": 1000}) in .metaobjects/config.json."); + } catch (RuntimeException e) { + // an advisory never breaks verify + } + } + /** The on-disk metadata files this loader read — the documents the duplicate scan reads raw. */ private static List sourceFiles(MetaDataLoader loader) { List files = new ArrayList<>(); diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java new file mode 100644 index 0000000000..0a5cdffc15 --- /dev/null +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java @@ -0,0 +1,92 @@ +package com.metaobjects.mojo; + +import org.apache.maven.plugin.logging.SystemStreamLog; +import org.junit.Test; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +/** + * {@code metaobjects:verify} — the advisory {@code unbounded-list-pages}: a project that generates + * REST controllers and sets no {@code api.pagination} is told so, as a warning that never fails + * the build; setting it, or generating no controller, silences it. + */ +public class MetaDataVerifyUnboundedListAdvisoryTest { + + private static final class CapturingLog extends SystemStreamLog { + final List warnings = new ArrayList<>(); + + @Override + public void warn(CharSequence content) { + if (content.toString().contains("unbounded-list-pages")) warnings.add(content.toString()); + } + } + + private static Path cleanCase() { + Path dir = Paths.get("").toAbsolutePath(); + while (dir != null) { + Path candidate = dir.resolve("fixtures/field-lint-conformance/clean/input"); + if (Files.isDirectory(candidate)) return candidate; + dir = dir.getParent(); + } + throw new AssertionError("Could not locate fixtures/field-lint-conformance/clean"); + } + + /** A project directory holding the clean model and, when given, a {@code .metaobjects/config.json}. */ + private static Path project(String configJson) throws Exception { + Path dir = Files.createTempDirectory("verify-unbounded"); + Files.copy(cleanCase().resolve("meta.app.json"), dir.resolve("meta.app.json")); + if (configJson != null) { + Files.createDirectories(dir.resolve(".metaobjects")); + Files.writeString(dir.resolve(".metaobjects/config.json"), configJson); + } + return dir; + } + + private static CapturingLog verify(Path project, boolean generatesRoutes) throws Exception { + MetaDataVerifyMojo mojo = new MetaDataVerifyMojo(); + mojo.setLoader(LoaderParam.builder("verify-unbounded-test") + .withClassname("com.metaobjects.loader.MetaDataLoader") + .withSourceDir(project.toString()) + .withSource("meta.app.json") + .build()); + List generators = new ArrayList<>(); + if (generatesRoutes) { + GeneratorParam g = new GeneratorParam(); + g.setClassname("com.metaobjects.generator.spring.SpringControllerGenerator"); + generators.add(g); + } + mojo.setGenerators(generators); + mojo.setGlobals(java.util.Collections.emptyMap()); + mojo.setMode("templates"); + mojo.setTemplateRoot(Files.createTempDirectory("verify-unbounded-templates").toString()); + CapturingLog log = new CapturingLog(); + mojo.setLog(log); + mojo.execute(); // an advisory never throws + return log; + } + + @Test + public void controllersWithoutAPageSizeAreAdvisedAndTheBuildPasses() throws Exception { + CapturingLog log = verify(project("{\"schema_version\": 1}"), true); + assertEquals(log.warnings.toString(), 1, log.warnings.size()); + assertTrue(log.warnings.get(0), log.warnings.get(0).contains("api.pagination")); + } + + @Test + public void aPageSizeSilencesIt() throws Exception { + String cfg = "{\"schema_version\": 1, \"api\": {\"pagination\": {\"defaultLimit\": 50, \"maxLimit\": 1000}}}"; + assertTrue(verify(project(cfg), true).warnings.isEmpty()); + } + + @Test + public void noControllerNoAdvisory() throws Exception { + assertTrue(verify(project("{\"schema_version\": 1}"), false).warnings.isEmpty()); + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java b/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java index de98496cd6..f0a3786ba3 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java +++ b/server/java/metadata/src/main/java/com/metaobjects/MetaRoot.java @@ -221,6 +221,24 @@ public void setPhysicalNaming(com.metaobjects.database.PhysicalNaming physicalNa private transient com.metaobjects.config.EnumDefaults enumDefaults = com.metaobjects.config.EnumDefaults.DEFAULT; + /** + * The project's {@code api} block ({@code api.pagination}, {@code api.unknownFields}), set by + * the same project-aware load and read through {@link com.metaobjects.config.ApiSettings#of(MetaData)}. + * A tree loaded with no project has no page size and refuses undeclared body keys. + */ + private transient com.metaobjects.config.ApiSettings apiSettings = + com.metaobjects.config.ApiSettings.DEFAULT; + + /** The project's {@code api} settings for this tree. Never {@code null}. */ + public com.metaobjects.config.ApiSettings getApiSettings() { + return apiSettings; + } + + /** Sets the project's {@code api} settings on this root ({@code null} restores the default). */ + public void setApiSettings(com.metaobjects.config.ApiSettings apiSettings) { + this.apiSettings = (apiSettings != null) ? apiSettings : com.metaobjects.config.ApiSettings.DEFAULT; + } + public com.metaobjects.config.EnumDefaults getEnumDefaults() { return enumDefaults; } diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/ApiSettings.java b/server/java/metadata/src/main/java/com/metaobjects/config/ApiSettings.java new file mode 100644 index 0000000000..b702ea22cf --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/config/ApiSettings.java @@ -0,0 +1,87 @@ +/* + * Copyright 2003 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.config; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaRoot; + +/** + * The project's {@code api} block in {@code .metaobjects/config.json}: how a generated REST + * surface treats list paging and undeclared body keys. + * + *

{@code api.pagination} is {@code {"defaultLimit": n, "maxLimit": m}} (both positive + * integers, {@code defaultLimit <= maxLimit}); a list route with no {@code limit} returns + * {@code defaultLimit} rows and a {@code limit} above {@code maxLimit} is answered with + * {@code 400 {"error": "limit_too_large", "maxLimit": m}}. Absent, the routes page as they + * always have ({@link #hasPagination()} is false, {@link #pagination()} is {@code null}).

+ * + *

{@code api.unknownFields} is {@code "reject"} (the default) or {@code "drop"}: what a + * generated create, PUT or PATCH does with a body key the entity does not declare.

+ * + *

Read a tree's setting with {@link #of(MetaData)}; the same value is on + * {@code MetaRoot#getApiSettings()}.

+ */ +public record ApiSettings(Pagination pagination, UnknownFields unknownFields) { + + /** The page size of a generated list route. */ + public record Pagination(long defaultLimit, long maxLimit) {} + + /** What a body key the entity does not declare does. */ + public enum UnknownFields { + /** Refuse with {@code 400 {"error": "unknown_fields", "fields": [...]}}. The default. */ + REJECT("reject"), + /** Ignore the key silently. */ + DROP("drop"); + + private final String wireName; + + UnknownFields(String wireName) { this.wireName = wireName; } + + /** The value as spelled in {@code .metaobjects/config.json}. */ + public String wireName() { return wireName; } + } + + /** No pagination, {@code reject}: what a tree loaded with no project carries. */ + public static final ApiSettings DEFAULT = new ApiSettings(null, UnknownFields.REJECT); + + public ApiSettings { + if (unknownFields == null) unknownFields = UnknownFields.REJECT; + } + + /** True when the project set {@code api.pagination}. */ + public boolean hasPagination() { + return pagination != null; + } + + /** True when an undeclared body key is refused (the default). */ + public boolean rejectsUnknownFields() { + return unknownFields == UnknownFields.REJECT; + } + + /** The {@code unknownFields} value spelled {@code raw}, or {@code null} when it names none. */ + public static UnknownFields unknownFieldsOf(String raw) { + for (UnknownFields u : UnknownFields.values()) if (u.wireName.equals(raw)) return u; + return null; + } + + /** The setting the tree {@code node} belongs to: what its root carries, else {@link #DEFAULT}. */ + public static ApiSettings of(MetaData node) { + for (MetaData n = node; n != null; n = n.getParent()) { + if (n instanceof MetaRoot root) return root.getApiSettings(); + } + return DEFAULT; + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java b/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java index 3ee082e13d..b5b7963d8f 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java +++ b/server/java/metadata/src/main/java/com/metaobjects/config/NeutralConfig.java @@ -70,6 +70,7 @@ public final class NeutralConfig { private final PhysicalNaming physicalNaming; private final WireFormat wireFormat; private EnumDefaults enumDefaults = EnumDefaults.DEFAULT; + private ApiSettings apiSettings = ApiSettings.DEFAULT; private List requirementsInherit = List.of(); private NeutralConfig(List> sources, List> dependencies, @@ -146,6 +147,14 @@ public WireFormat getWireFormat() { return wireFormat; } + /** + * The project's {@code api} block ({@code api.pagination}, {@code api.unknownFields}), absent + * keys at their defaults (no page size, {@code reject}). Never {@code null}. + */ + public ApiSettings getApiSettings() { + return apiSettings; + } + /** The project's enum storage defaults ({@code enums} in this file). Never {@code null}. */ public EnumDefaults getEnumDefaults() { return enumDefaults; @@ -360,10 +369,70 @@ public static NeutralConfig parse(String content, String origin) { NeutralConfig cfg = new NeutralConfig(specs, dependencies, scopeInclude, migrateScope, physicalNaming, wireFormat); cfg.enumDefaults = parseEnums(root.get("enums"), origin); + cfg.apiSettings = parseApi(root.get("api"), origin); cfg.requirementsInherit = parseRequirementsInherit(root.get("requirements"), origin); return cfg; } + /** + * The {@code api} block: {@code pagination} ({@code defaultLimit} and {@code maxLimit}, both + * positive integers, {@code defaultLimit <= maxLimit}) and {@code unknownFields} + * ({@code "reject"} | {@code "drop"}). Refused like {@code naming}: an unknown key, a + * wrong-typed value or a non-object block. The messages match the TypeScript reader's. + */ + private static ApiSettings parseApi(JsonElement api, String origin) { + if (api == null) return ApiSettings.DEFAULT; + if (!api.isJsonObject()) throw bad(origin + ": \"api\" must be an object"); + ApiSettings.Pagination pagination = null; + ApiSettings.UnknownFields unknownFields = ApiSettings.UnknownFields.REJECT; + JsonObject obj = api.getAsJsonObject(); + for (String key : obj.keySet()) { + JsonElement v = obj.get(key); + switch (key) { + case "pagination" -> pagination = parsePagination(v, origin); + case "unknownFields" -> { + unknownFields = v.isJsonPrimitive() && v.getAsJsonPrimitive().isString() + ? ApiSettings.unknownFieldsOf(v.getAsString()) : null; + if (unknownFields == null) { + throw bad(origin + ": \"api.unknownFields\" must be \"reject\" or \"drop\""); + } + } + default -> throw bad(origin + ": \"api\" has an unknown key \"" + key + + "\" (expected \"pagination\" or \"unknownFields\")"); + } + } + return new ApiSettings(pagination, unknownFields); + } + + private static ApiSettings.Pagination parsePagination(JsonElement el, String origin) { + if (!el.isJsonObject()) throw bad(origin + ": \"api.pagination\" must be an object"); + JsonObject obj = el.getAsJsonObject(); + for (String key : obj.keySet()) { + if (!"defaultLimit".equals(key) && !"maxLimit".equals(key)) { + throw bad(origin + ": \"api.pagination\" has an unknown key \"" + key + + "\" (expected \"defaultLimit\" and \"maxLimit\")"); + } + } + if (!obj.has("defaultLimit") || !obj.has("maxLimit")) { + throw bad(origin + ": \"api.pagination\" needs both \"defaultLimit\" and \"maxLimit\""); + } + long defaultLimit = pageSize(obj.get("defaultLimit"), "defaultLimit", origin); + long maxLimit = pageSize(obj.get("maxLimit"), "maxLimit", origin); + if (defaultLimit > maxLimit) { + throw bad(origin + ": \"api.pagination.defaultLimit\" must not exceed \"api.pagination.maxLimit\""); + } + return new ApiSettings.Pagination(defaultLimit, maxLimit); + } + + private static long pageSize(JsonElement v, String key, String origin) { + java.math.BigDecimal n = v.isJsonPrimitive() && v.getAsJsonPrimitive().isNumber() + ? v.getAsBigDecimal() : null; + if (n == null || n.signum() <= 0 || n.stripTrailingZeros().scale() > 0 || n.compareTo(java.math.BigDecimal.valueOf(Integer.MAX_VALUE)) > 0) { + throw bad(origin + ": \"api.pagination." + key + "\" must be a positive integer"); + } + return n.longValueExact(); + } + /** * The {@code enums} block: {@code maxLength}, a positive integer, is the column length of a * string-stored enum with no {@code @maxLength}. An unknown key, a wrong-typed value, or a diff --git a/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java b/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java index fac3312f07..68c236d68f 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java +++ b/server/java/metadata/src/main/java/com/metaobjects/config/SourceResolver.java @@ -225,6 +225,11 @@ public static java.util.Optional discoverWireFormat(List discoverApiSettings(List uris) { + return discoverConfig(uris).map(NeutralConfig::getApiSettings); + } + /** The enum storage defaults of the {@code .metaobjects/config.json} the same walk finds, if any. */ public static java.util.Optional discoverEnumDefaults(List uris) { return discoverConfig(uris).map(NeutralConfig::getEnumDefaults); diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java index 1423f8d3ba..02ed9f27a4 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java @@ -751,6 +751,9 @@ public static MetaDataLoader fromDirectory(String name, Path directory, loader.getRoot().setEnumDefaults(com.metaobjects.config.SourceResolver .discoverEnumDefaults(List.of(directory.toAbsolutePath().toUri())) .orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); + loader.getRoot().setApiSettings(com.metaobjects.config.SourceResolver + .discoverApiSettings(List.of(directory.toAbsolutePath().toUri())) + .orElse(com.metaobjects.config.ApiSettings.DEFAULT)); } catch (MetaDataLoadingException e) { throw e; } catch (Exception e) { @@ -858,6 +861,8 @@ public static MetaDataLoader fromUris(String name, List uris, LoaderOptions .discoverWireFormat(uris).orElse(com.metaobjects.config.WireFormat.DEFAULT)); loader.getRoot().setEnumDefaults(com.metaobjects.config.SourceResolver .discoverEnumDefaults(uris).orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); + loader.getRoot().setApiSettings(com.metaobjects.config.SourceResolver + .discoverApiSettings(uris).orElse(com.metaobjects.config.ApiSettings.DEFAULT)); } catch (MetaDataLoadingException e) { throw e; } catch (MetaDataException e) { diff --git a/server/java/metadata/src/test/java/com/metaobjects/config/ApiSettingsTest.java b/server/java/metadata/src/test/java/com/metaobjects/config/ApiSettingsTest.java new file mode 100644 index 0000000000..55bb8a6006 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/config/ApiSettingsTest.java @@ -0,0 +1,75 @@ +/* + * Copyright 2003 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.config; + +import com.metaobjects.MetaDataException; +import org.junit.Test; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; + +/** The {@code api} block of {@code .metaobjects/config.json}: pagination and unknownFields. */ +public class ApiSettingsTest { + + private static ApiSettings api(String apiJson) { + String body = apiJson == null ? "" : ", \"api\": " + apiJson; + return NeutralConfig.parse("{\"schema_version\": 1" + body + "}", "test").getApiSettings(); + } + + private static String refusal(String apiJson) { + return assertThrows(MetaDataException.class, () -> api(apiJson)).getMessage(); + } + + @Test + public void absentBlockHasNoPageSizeAndRejects() { + ApiSettings a = api(null); + assertFalse(a.hasPagination()); + assertNull(a.pagination()); + assertTrue(a.rejectsUnknownFields()); + assertEquals(ApiSettings.DEFAULT, a); + } + + @Test + public void readsPaginationAndUnknownFields() { + ApiSettings a = api("{\"pagination\": {\"defaultLimit\": 50, \"maxLimit\": 1000}, \"unknownFields\": \"drop\"}"); + assertEquals(new ApiSettings.Pagination(50, 1000), a.pagination()); + assertFalse(a.rejectsUnknownFields()); + assertEquals(ApiSettings.UnknownFields.DROP, a.unknownFields()); + assertTrue(api("{\"unknownFields\": \"reject\"}").rejectsUnknownFields()); + assertFalse(api("{\"unknownFields\": \"reject\"}").hasPagination()); + } + + @Test + public void refusesABadBlockNamingTheKey() { + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 0, \"maxLimit\": 5}}") + .contains("\"api.pagination.defaultLimit\" must be a positive integer")); + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 2.5, \"maxLimit\": 5}}") + .contains("\"api.pagination.defaultLimit\" must be a positive integer")); + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 5, \"maxLimit\": \"9\"}}") + .contains("\"api.pagination.maxLimit\" must be a positive integer")); + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 10, \"maxLimit\": 5}}") + .contains("\"api.pagination.defaultLimit\" must not exceed \"api.pagination.maxLimit\"")); + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 10}}").contains("needs both")); + assertTrue(refusal("{\"pagination\": {\"defaultLimit\": 1, \"maxLimit\": 2, \"x\": 1}}").contains("unknown key \"x\"")); + assertTrue(refusal("{\"unknownFields\": \"ignore\"}").contains("\"api.unknownFields\" must be \"reject\" or \"drop\"")); + assertTrue(refusal("{\"unknownFields\": true}").contains("\"api.unknownFields\" must be \"reject\" or \"drop\"")); + assertTrue(refusal("{\"other\": 1}").contains("\"api\" has an unknown key \"other\"")); + assertTrue(refusal("[]").contains("\"api\" must be an object")); + } +} diff --git a/server/python/src/metaobjects/api_settings.py b/server/python/src/metaobjects/api_settings.py new file mode 100644 index 0000000000..12cf86e4e7 --- /dev/null +++ b/server/python/src/metaobjects/api_settings.py @@ -0,0 +1,55 @@ +"""The project ``api`` setting: what a generated REST route does at its edges. + +``api`` in ``.metaobjects/config.json`` is read by the same port-neutral reader as ``naming`` and +``wire`` and carried by the loaded root (``MetaRoot.api``); codegen reads it with :func:`api_of`. + +* ``api.pagination`` (``{"defaultLimit": N, "maxLimit": M}``): a list route with no ``limit`` returns + ``defaultLimit`` rows, and a ``limit`` above ``maxLimit`` is refused with + ``400 {"error": "limit_too_large", "maxLimit": M}``. Absent, a route behaves as it always has. +* ``api.unknownFields`` (``"reject"`` | ``"drop"``): what a generated create, PUT or PATCH does with + a body key the entity declares no field for. ``reject`` (the default) answers + ``400 {"error": "unknown_fields", "fields": [...]}``; ``drop`` ignores the key. +""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from metaobjects.meta.meta_data import MetaData + +API_KEY_PAGINATION = "pagination" +API_KEY_UNKNOWN_FIELDS = "unknownFields" +PAGINATION_KEY_DEFAULT_LIMIT = "defaultLimit" +PAGINATION_KEY_MAX_LIMIT = "maxLimit" + +UNKNOWN_FIELDS_REJECT = "reject" +UNKNOWN_FIELDS_DROP = "drop" +UNKNOWN_FIELDS_MODES = (UNKNOWN_FIELDS_REJECT, UNKNOWN_FIELDS_DROP) + +#: What a fresh ``init`` writes. +INIT_DEFAULT_LIMIT = 50 +INIT_MAX_LIMIT = 1000 + + +@dataclass(frozen=True) +class ApiPagination: + default_limit: int + max_limit: int + + +@dataclass(frozen=True) +class ApiSettings: + """The resolved ``api`` block. Trusted input: validated where the raw JSON is parsed.""" + + pagination: ApiPagination | None = None + unknown_fields: str = UNKNOWN_FIELDS_REJECT + + +def api_of(node: "MetaData") -> ApiSettings: + """The :class:`ApiSettings` in force for *node*: the loaded root's, or the default.""" + cur = node + while cur.parent is not None: + cur = cur.parent + api = getattr(cur, "api", None) + return api if isinstance(api, ApiSettings) else ApiSettings() diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index e39ba81e88..49c43b9892 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -162,6 +162,7 @@ verify as render_verify, ) from metaobjects.shared.base_types import TYPE_OBJECT, TYPE_TEMPLATE +from metaobjects.api_settings import ApiSettings from metaobjects.wire import WireFormat, wire_block, wire_candidates @@ -521,6 +522,7 @@ def load(strict_load: bool) -> LoadResult: # invocation) read the same values `gen`/`verify --codegen` would. result.root.naming = neutral.naming if neutral is not None else NamingRules() result.root.wire = neutral.wire if neutral is not None else WireFormat() + result.root.api = neutral.api if neutral is not None else ApiSettings() return result.root, [] @@ -584,6 +586,7 @@ def _load_collection_result( # it to the loaded root so codegen, the runtime and `verify` agree. result.root.naming = collection.naming result.root.wire = collection.wire + result.root.api = collection.api return result @@ -2726,6 +2729,30 @@ def _run_field_lint_advisory(args: argparse.Namespace, model: "_VerifyModel | No print(f" {finding.code} [{finding.path}]: {finding.message}", file=sys.stderr) +def _run_pagination_advisory(model: "_VerifyModel | None") -> None: + """``unbounded-list-pages``: ONE advisory line when the project generates REST list routes + and sets no ``api.pagination``, so a request with no ``limit`` is served by the 50-row + historic default and ``?limit=`` has no cap. Advisory only: stderr, never the exit code.""" + if model is None or model.root.api.pagination is not None: + return + try: + from metaobjects.codegen.runtime.shared_predicates import emits_router + + serves = any( + isinstance(c, MetaObject) and emits_router(c) for c in model.root.children() + ) + except Exception: # noqa: BLE001 — an advisory scan never breaks verify + return + if serves: + print( + "metaobjects verify — unbounded-list-pages (advisory — does not fail the build): " + "generated list routes have no page cap, so a request may ask for any ?limit=. " + 'Set "api": {"pagination": {"defaultLimit": 50, "maxLimit": 1000}} in ' + ".metaobjects/config.json.", + file=sys.stderr, + ) + + #: Raises ``WARN_REQUIREMENT_NOTHING_IMPLEMENTS`` to an error, beside ``--require-implementers`` #: (Node `meta` parity, ADR-0057). REQUIRE_IMPLEMENTERS_ENV = "META_REQUIRE_IMPLEMENTERS" @@ -2921,6 +2948,8 @@ def _cmd_verify(args: argparse.Namespace) -> int: exit_code = max(exit_code, _report_unloaded_model(load_failure, gates_stderr.lines())) exit_code = max(exit_code, _verify_requirements(args, model)) _run_field_lint_advisory(args, model) + if exit_code != 2: # 2 is a refused invocation: its own message is the whole answer + _run_pagination_advisory(model) return exit_code diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 934a9ea4fd..8614f1c381 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -75,6 +75,7 @@ from metaobjects.codegen.config import DEFAULT_API_PREFIX from metaobjects.naming import DEFAULT_COLUMN_NAMING from metaobjects.naming_refs import ref_target +from metaobjects.api_settings import UNKNOWN_FIELDS_REJECT, ApiPagination, api_of from metaobjects.wire import WIRE_DURATIONS_ISO, WIRE_NATIVE, WireFormat, wire_of from metaobjects.shared.base_types import TYPE_IDENTITY @@ -188,17 +189,58 @@ def _scalar_fields(entity: MetaObject) -> list[MetaField]: #: ADR-0019 Amendment 1 (W2): a request body's decimal fields are parsed exactly (no float #: detour) before validation; a malformed one is the cross-port 400. -_DECIMAL_COERCE: list[str] = [ +_BODY_CHECK: list[str] = [ " if dto is None: # the body was not a JSON object: the cross-port 400, not FastAPI's 422", " return JSONResponse(", " status_code=400, content=validation_envelope(unreadable_body_issues())", " )", +] + +#: ``api.unknownFields: reject`` (the default): a body key no declared field names is refused +#: before anything else reads the body. A declared key the client may not set is NOT unknown; +#: it is dropped later, as under ``drop``. +_UNKNOWN_FIELDS_CHECK: list[str] = [ + " _unknown = unknown_body_keys(dto, _KNOWN_FIELDS)", + " if _unknown:", + ' return JSONResponse(status_code=400, content={"error": "unknown_fields", "fields": _unknown})', +] + +_DECIMAL_COERCE_BODY: list[str] = [ " try:", " dto = coerce_decimal_fields(dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS)", " except ValueError as _exc:", " return JSONResponse(status_code=400, content=validation_envelope(value_error_issues(_exc)))", ] +#: The start of every write handler: the unreadable-body answer, the unknown-key refusal when the +#: project rejects them, then the exact decimal parse. +_DECIMAL_COERCE: list[str] = [*_BODY_CHECK, *_DECIMAL_COERCE_BODY] + + +def _write_prelude(reject: bool) -> list[str]: + """The body checks that open a create / PUT / PATCH handler (see :data:`_DECIMAL_COERCE`).""" + return [*_BODY_CHECK, *(_UNKNOWN_FIELDS_CHECK if reject else []), *_DECIMAL_COERCE_BODY] + + +def _known_fields_lines(entities: list[MetaObject]) -> list[str]: + """``_KNOWN_FIELDS``: every field the body may name, on the wire (RESOLVING ``fields()``, + ADR-0039; for a single-table hierarchy the base's and every subtype's).""" + names = list(dict.fromkeys(f.name for e in entities for f in e.fields())) + return ["", "", f"_KNOWN_FIELDS: frozenset[str] = {_py_set_literal(names, frozen=True)}"] + + +def _list_limit_lines(page: ApiPagination | None) -> list[str]: + """The limit handling at the top of a list handler body. With no ``api.pagination`` it is the + historic default of 50 and no cap; with one, the project's default page and a refusal above + the cap (not a clamp: the client is told it asked for more than the server serves).""" + if page is None: + return [" actual_limit = limit if limit is not None else 50"] + return [ + f" if limit is not None and limit > {page.max_limit}:", + f' return JSONResponse(status_code=400, content={{"error": "limit_too_large", "maxLimit": {page.max_limit}}})', + f" actual_limit = limit if limit is not None else {page.default_limit}", + ] + #: The request-body dependency of every write route: the body is read with #: ``parse_float=Decimal`` so a JSON number never passes through a ``float`` (W2). A body that #: is not a JSON object yields ``None`` and the route answers the cross-port @@ -1063,13 +1105,17 @@ def _emit_route_handler( denied_strip: list[str] = (), report_max_rows: int | None = None, report_export: bool = False, + reject_unknown: bool = False, + pagination: ApiPagination | None = None, ) -> list[str]: """One CRUD route handler block, dispatched by *name* (``list`` / ``get`` / ``create`` / ``update`` / ``delete``). Override to change a handler's body / decorators / response shape. *model_name* / *patch_model* are the generated create + all-optional PATCH Pydantic models used to run field constraints on POST / PATCH (FR-036). *report_max_rows* is - passed only for a served report's ``list``: every row up to that cap, refused above it.""" + passed only for a served report's ``list``: every row up to that cap, refused above it. + *reject_unknown* adds the ``api.unknownFields: reject`` refusal to create / update; + *pagination* is the project's ``api.pagination`` for a ``list`` that is not a report's.""" if name == "list" and report_max_rows is not None and report_export: return self._report_list_with_export( plural=plural, @@ -1156,7 +1202,7 @@ def _emit_route_handler( " sort: str | None = Query(None),", ' with_count: int | None = Query(None, alias="withCount"),', ") -> Any:", - " actual_limit = limit if limit is not None else 50", + *_list_limit_lines(pagination), " actual_offset = offset if offset is not None else 0", " sort_clause: _SortClause | None = None", " if sort is not None:", @@ -1196,7 +1242,7 @@ def _emit_route_handler( " dto: Annotated[dict[str, Any] | None, Depends(_json_body)],", f" repo: Annotated[{repo_class}, Depends(get_repository)],", ") -> Any:", - *_DECIMAL_COERCE, + *_write_prelude(reject_unknown), *denied_strip, *_validated_input_lines( model_name, @@ -1220,7 +1266,7 @@ def _emit_route_handler( " dto: Annotated[dict[str, Any] | None, Depends(_json_body)],", f" repo: Annotated[{repo_class}, Depends(get_repository)],", ") -> Any:", - *_DECIMAL_COERCE, + *_write_prelude(reject_unknown), *denied_strip, *_validated_input_lines( patch_model, @@ -1342,13 +1388,18 @@ def _emit_tph_m2m_route( return lines def _emit_tph_list_body( - self, subtype_expr: str, fields_const: str, ops_const: str, repo_var: str = "repo" + self, + subtype_expr: str, + fields_const: str, + ops_const: str, + repo_var: str = "repo", + pagination: ApiPagination | None = None, ) -> list[str]: """The shared list-handler body (sort + filter parse → repo.list). *subtype_expr* is the Python literal threaded as the discriminator scope: ``None`` for the polymorphic base, or a quoted ``@discriminatorValue`` for a per-subtype route.""" return [ - " actual_limit = limit if limit is not None else 50", + *_list_limit_lines(pagination), " actual_offset = offset if offset is not None else 0", " sort_clause: _SortClause | None = None", " if sort is not None:", @@ -1391,6 +1442,12 @@ def _render_tph_router( ``render_router``, which returns ``None`` for a TPH subtype before reaching M:N resolution at all — the two independent gaps this fix closes).""" short_name = entity.name + # The project's `api` setting: the page size of every list route here and whether an + # undeclared body key is refused (the default) or dropped. + api = api_of(entity) + page = api.pagination + reject = api.unknown_fields == UNKNOWN_FIELDS_REJECT + prelude = _write_prelude(reject) snake = _snake_case(short_name) plural = _route_path(short_name) pk_param = f"{snake}_id" @@ -1538,6 +1595,8 @@ def _render_tph_router( parts.append(" classify_constraint_error,") parts.append(" pydantic_issues,") parts.append(" required_null_issues,") + if reject: + parts.append(" unknown_body_keys,") parts.append(" unreadable_body_issues,") parts.append(" validation_envelope,") parts.append(" value_error_issues,") @@ -1592,6 +1651,8 @@ def _render_tph_router( # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). parts.append(f"_FROZEN_FIELDS: frozenset[str] = {frozen_set_body}") + if reject: + parts.extend(_known_fields_lines([entity, *(st.entity for st in plan.subtypes)])) parts.extend(_denied_const_lines(denied_names)) parts.append("") parts.append("") @@ -1657,7 +1718,7 @@ def list_sig(fn: str, route: str) -> list[str]: sub = st.entity.name # the subtype's generated create + PATCH model classes sfx = st.route_segment # handler-name suffix = URL segment (e.g. "bridge"), matches the route parts.extend(list_sig(f"list_{plural}_{sfx}", f"/{seg}")) - parts.extend(self._emit_tph_list_body(f'"{val}"', fields_const, ops_const)) + parts.extend(self._emit_tph_list_body(f'"{val}"', fields_const, ops_const, pagination=page)) parts.append("") parts.append("") parts.append(f'@router.post("/{seg}", status_code=status.HTTP_201_CREATED)') @@ -1665,7 +1726,7 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(" dto: Annotated[dict[str, Any] | None, Depends(_json_body)],") parts.append(f" repo: Annotated[{repo_class}, Depends(get_repository)],") parts.append(") -> Any:") - parts.extend(_DECIMAL_COERCE) + parts.extend(prelude) parts.extend(denied_strip_tph) # The route's subtype is the only source of the discriminator: a body value is # dropped silently before validation (as the TypeScript route's `.omit` does), so @@ -1700,7 +1761,7 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(" dto: Annotated[dict[str, Any] | None, Depends(_json_body)],") parts.append(f" repo: Annotated[{repo_class}, Depends(get_repository)],") parts.append(") -> Any:") - parts.extend(_DECIMAL_COERCE) + parts.extend(prelude) parts.extend(denied_strip_tph) parts.extend(_validated_input_lines( f"{sub}Patch", @@ -1757,7 +1818,7 @@ def list_sig(fn: str, route: str) -> list[str]: # --- Polymorphic base routes LAST (so /{id} doesn't shadow /). --- parts.extend(list_sig(f"list_{plural}", "")) - parts.extend(self._emit_tph_list_body("None", fields_const, ops_const)) + parts.extend(self._emit_tph_list_body("None", fields_const, ops_const, pagination=page)) parts.append("") parts.append("") parts.append(f'@router.get("/{{{pk_param}}}")') @@ -1831,6 +1892,9 @@ def render_router( # ADR-0019 Amendment 2: the project's `wire` setting, from the loaded root. A caller # that holds an object detached from the root (a report's read model) passes it. wire = wire if wire is not None else wire_of(entity) + api = api_of(entity) + page = api.pagination + reject = api.unknown_fields == UNKNOWN_FIELDS_REJECT # FR-017 TPH: a concrete subtype is folded into its base's single table — it # emits no standalone router (its CRUD lives under the base's per-subtype segment). if object_index is not None and is_tph_subtype(entity): @@ -1938,6 +2002,8 @@ def render_router( parts.append(" classify_constraint_error,") parts.append(" pydantic_issues,") parts.append(" required_null_issues,") + if reject: + parts.append(" unknown_body_keys,") parts.append(" unreadable_body_issues,") parts.append(" validation_envelope,") parts.append(" value_error_issues,") @@ -1989,6 +2055,8 @@ def render_router( # the generated module always carries the seam (an entity that later gains a # writeOnce field changes one literal, not the module's shape). parts.append(f"_FROZEN_FIELDS: frozenset[str] = {frozen_set_body}") + if reject: + parts.extend(_known_fields_lines([entity])) denied_names = _denied_names(entity) parts.extend(_denied_const_lines(denied_names)) parts.append("") @@ -2024,6 +2092,8 @@ def render_router( # not in Patch. frozen_strip=_frozen_strip_lines(entity), denied_strip=_denied_strip_lines(denied_names), + reject_unknown=reject, + pagination=page, ) for i, hname in enumerate(("list", "get", "create", "update", "delete")): if i > 0: @@ -2250,6 +2320,7 @@ def _render_readonly_router( # Only a report is capped; passed for it alone so an adopter's `_emit_route_handler` # override that does not take the keyword keeps working for a projection. **({"report_max_rows": self.report_max_rows} if is_report else {}), + **({"pagination": api_of(entity).pagination} if not is_report and api_of(entity).pagination else {}), **({"report_export": True} if export else {}), ) for i, hname in enumerate(("list", "get") if item_route else ("list",)): diff --git a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py index adf0c53e09..32f59d8545 100644 --- a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py +++ b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py @@ -45,6 +45,7 @@ "classify_constraint_error", "pydantic_issues", "required_null_issues", + "unknown_body_keys", "unreadable_body_issues", "value_error_issues", "validation_envelope", @@ -242,6 +243,15 @@ def required_null_issues(dto: dict[str, Any], required: Any) -> list[dict[str, A ] +def unknown_body_keys(body: Any, known: Any) -> list[str]: + """The top-level keys of the JSON object *body* that *known* does not name, sorted by + code point (``api.unknownFields: reject``). A body that is not a ``dict`` has none: it is + left to the unreadable-body answer.""" + if not isinstance(body, dict): + return [] + return sorted(k for k in body if k not in known) + + def unreadable_body_issues() -> list[dict[str, Any]]: """The single issue for a body that is not a JSON object (``path`` is ``[]``: the body as a whole).""" diff --git a/server/python/src/metaobjects/config/dependencies.py b/server/python/src/metaobjects/config/dependencies.py index c32dd8d761..e81841c048 100644 --- a/server/python/src/metaobjects/config/dependencies.py +++ b/server/python/src/metaobjects/config/dependencies.py @@ -17,6 +17,7 @@ from metaobjects.errors import ErrorCode, ParseError from metaobjects.meta.meta_data import MetaData from metaobjects.naming import NamingRules, package_of_resolution_key +from metaobjects.api_settings import ApiSettings from metaobjects.wire import WireFormat from metaobjects.registry_manifest import METAMODEL_VERSION from metaobjects.shared.base_types import TYPE_OBJECT, TYPE_REQUIREMENT @@ -677,6 +678,9 @@ class Collection: #: attached to the loaded root beside `naming`. wire: WireFormat = WireFormat() + #: This project's `api` setting (page size, unknown-field rule), attached beside `wire`. + api: ApiSettings = ApiSettings() + #: FR-023 — the declared `requirements.inherit` patterns, for `in_ledger`. requirements_inherit: tuple[str, ...] = () diff --git a/server/python/src/metaobjects/config/neutral_config.py b/server/python/src/metaobjects/config/neutral_config.py index 870333a2f7..034e25834e 100644 --- a/server/python/src/metaobjects/config/neutral_config.py +++ b/server/python/src/metaobjects/config/neutral_config.py @@ -5,6 +5,16 @@ from dataclasses import dataclass, field from pathlib import Path +from metaobjects.api_settings import ( + API_KEY_PAGINATION, + API_KEY_UNKNOWN_FIELDS, + PAGINATION_KEY_DEFAULT_LIMIT, + PAGINATION_KEY_MAX_LIMIT, + UNKNOWN_FIELDS_MODES, + UNKNOWN_FIELDS_REJECT, + ApiPagination, + ApiSettings, +) from metaobjects.errors import ErrorCode, ParseError from metaobjects.naming import ( NamingRules, @@ -96,6 +106,10 @@ class NeutralConfig: #: requirements this project takes on as its own ledger (`Collection.in_ledger`). requirements_inherit: list[str] = field(default_factory=list) + #: The `api` block: `api.pagination` (default page size and cap of a generated list route) and + #: `api.unknownFields` (`reject` | `drop`). Attached to the loaded root (`MetaRoot.api`). + api: ApiSettings = ApiSettings() + def find_project_dir(start: Path) -> Path | None: """The nearest directory at or above *start* that holds ``.metaobjects/config.json``, @@ -189,6 +203,7 @@ def read_neutral_config(config_dir: Path) -> NeutralConfig | None: libraries = _read_libraries(raw, path) naming = _read_naming(raw, path) wire = _read_wire(raw, path) + api = _read_api(raw, path) _check_enums(raw, path) requirements_inherit = _read_requirements_inherit(raw, path) @@ -202,6 +217,7 @@ def read_neutral_config(config_dir: Path) -> NeutralConfig | None: naming=naming, wire=wire, requirements_inherit=requirements_inherit, + api=api, ) @@ -387,6 +403,70 @@ def _check_enums(raw: dict[str, object], path: Path) -> None: ) +def _page_size(value: object, key: str, path: Path) -> int: + """A positive whole number however JSON spells it (50, 50.0), never a bool.""" + if isinstance(value, float) and value.is_integer(): + value = int(value) + if isinstance(value, bool) or not isinstance(value, int) or value <= 0: + raise ParseError( + f'{path}: "api.pagination.{key}" must be a positive integer', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + return value + + +def _read_api(raw: dict[str, object], path: Path) -> ApiSettings: + """The `api` block. Absent or `{}` => no page size and `reject`. A mistake is refused here + rather than falling back to a default that changes what a route answers.""" + if "api" not in raw: + return ApiSettings() + api = raw["api"] + if not isinstance(api, dict): + raise ParseError(f'{path}: "api" must be an object', code=ErrorCode.ERR_COLLECTION_NOT_FOUND) + for key in api: + if key not in (API_KEY_PAGINATION, API_KEY_UNKNOWN_FIELDS): + raise ParseError( + f'{path}: "api" has an unknown key "{key}" ' + f'(expected "{API_KEY_PAGINATION}" or "{API_KEY_UNKNOWN_FIELDS}")', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + pagination: ApiPagination | None = None + if API_KEY_PAGINATION in api: + block = api[API_KEY_PAGINATION] + if not isinstance(block, dict): + raise ParseError(f'{path}: "api.pagination" must be an object', code=ErrorCode.ERR_COLLECTION_NOT_FOUND) + for key in block: + if key not in (PAGINATION_KEY_DEFAULT_LIMIT, PAGINATION_KEY_MAX_LIMIT): + raise ParseError( + f'{path}: "api.pagination" has an unknown key "{key}" ' + f'(expected "{PAGINATION_KEY_DEFAULT_LIMIT}" and "{PAGINATION_KEY_MAX_LIMIT}")', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + for key in (PAGINATION_KEY_DEFAULT_LIMIT, PAGINATION_KEY_MAX_LIMIT): + if key not in block: + raise ParseError( + f'{path}: "api.pagination" needs both "{PAGINATION_KEY_DEFAULT_LIMIT}" ' + f'and "{PAGINATION_KEY_MAX_LIMIT}"', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + default_limit = _page_size(block[PAGINATION_KEY_DEFAULT_LIMIT], PAGINATION_KEY_DEFAULT_LIMIT, path) + max_limit = _page_size(block[PAGINATION_KEY_MAX_LIMIT], PAGINATION_KEY_MAX_LIMIT, path) + if default_limit > max_limit: + raise ParseError( + f'{path}: "api.pagination.defaultLimit" must not exceed "api.pagination.maxLimit"', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + pagination = ApiPagination(default_limit=default_limit, max_limit=max_limit) + unknown_fields = api.get(API_KEY_UNKNOWN_FIELDS, UNKNOWN_FIELDS_REJECT) + if not isinstance(unknown_fields, str) or unknown_fields not in UNKNOWN_FIELDS_MODES: + choices = " or ".join(f'"{m}"' for m in UNKNOWN_FIELDS_MODES) + raise ParseError( + f'{path}: "api.{API_KEY_UNKNOWN_FIELDS}" must be {choices}', + code=ErrorCode.ERR_COLLECTION_NOT_FOUND, + ) + return ApiSettings(pagination=pagination, unknown_fields=unknown_fields) + + #: Every key the `wire` block accepts, and the ONLY ones it accepts (refused, not ignored, like `naming`). _WIRE_KEYS = ("decimals", "nullMembers", "durations") diff --git a/server/python/src/metaobjects/config/source_resolver.py b/server/python/src/metaobjects/config/source_resolver.py index 9800672f34..15eeba3935 100644 --- a/server/python/src/metaobjects/config/source_resolver.py +++ b/server/python/src/metaobjects/config/source_resolver.py @@ -7,6 +7,7 @@ from metaobjects.errors import ErrorCode, ParseError from metaobjects.loader.sources import DirectorySource from metaobjects.naming import NamingRules, package_of_resolution_key +from metaobjects.api_settings import ApiSettings from metaobjects.wire import WireFormat from metaobjects.scope import compile_scope, matches_scope @@ -157,6 +158,7 @@ def _collection_from_own_files( libraries: list[str] = cfg.libraries if cfg is not None else [] naming: NamingRules = cfg.naming if cfg is not None else NamingRules() wire: WireFormat = cfg.wire if cfg is not None else WireFormat() + api: ApiSettings = cfg.api if cfg is not None else ApiSettings() lock = read_lock(root) dependencies: list[ResolvedDependency] = ( @@ -188,6 +190,7 @@ def _collection_from_own_files( libraries=tuple(libraries), naming=naming, wire=wire, + api=api, requirements_inherit=tuple(cfg.requirements_inherit if cfg is not None else []), ) diff --git a/server/python/src/metaobjects/meta/meta_root.py b/server/python/src/metaobjects/meta/meta_root.py index 81f0156d02..d3fd314db9 100644 --- a/server/python/src/metaobjects/meta/meta_root.py +++ b/server/python/src/metaobjects/meta/meta_root.py @@ -2,6 +2,7 @@ from __future__ import annotations from ..naming import NamingRules +from ..api_settings import ApiSettings from ..wire import WireFormat from .meta_data import MetaData @@ -21,3 +22,5 @@ def __init__(self, type_: str, sub_type: str, name: str) -> None: # REST surface), attached by the same project-aware loads as `naming`; read with # `wire_of()`. `native` for both keys until a load attaches the project's own. self.wire: WireFormat = WireFormat() + # The project `api` setting (page size, unknown-field rule), attached the same way; read with `api_of()`. + self.api: ApiSettings = ApiSettings() diff --git a/server/python/src/metaobjects/wire.py b/server/python/src/metaobjects/wire.py index 8b025d378e..e96b608e6f 100644 --- a/server/python/src/metaobjects/wire.py +++ b/server/python/src/metaobjects/wire.py @@ -67,8 +67,10 @@ def wire_candidates(used: WireFormat) -> list[WireFormat]: def attach_wire(root: "MetaData", project_dir: Path) -> "MetaData": """Set ``root.wire`` from *project_dir*'s ``.metaobjects/config.json`` and return *root*, for a load site that bypasses the CLI's collection ladder (a conformance harness).""" + from metaobjects.api_settings import ApiSettings from metaobjects.config.neutral_config import read_neutral_config cfg = read_neutral_config(project_dir) root.wire = cfg.wire if cfg is not None else WireFormat() # type: ignore[attr-defined] + root.api = cfg.api if cfg is not None else ApiSettings() # type: ignore[attr-defined] return root diff --git a/server/python/tests/codegen/test_cli_verify_pagination_advisory.py b/server/python/tests/codegen/test_cli_verify_pagination_advisory.py new file mode 100644 index 0000000000..8c1cae63db --- /dev/null +++ b/server/python/tests/codegen/test_cli_verify_pagination_advisory.py @@ -0,0 +1,69 @@ +"""``metaobjects verify`` — the ``unbounded-list-pages`` advisory. + +A project that generates REST list routes and sets no ``api.pagination`` is told so on stderr. +It is advisory: it never reaches the exit code, and a project that sets the block is not told. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from metaobjects.cli import main +from tests.codegen.config_helpers import as_config +from tests.codegen.gen_suite import GEN_SUITE + +_ADVISORY = "unbounded-list-pages" + + +def _project(tmp_path: Path, api: dict[str, object] | None) -> str: + project = tmp_path / "proj" + (project / ".metaobjects").mkdir(parents=True) + cfg: dict[str, object] = {"schema_version": 1} + if api is not None: + cfg["api"] = api + (project / ".metaobjects" / "config.json").write_text(json.dumps(cfg)) + doc = { + "metadata.root": { + "package": "app", + "children": [ + { + "object.entity": { + "name": "Item", + "children": [ + {"source.rdb": {"@table": "items"}}, + {"field.long": {"name": "id"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + ], + } + } + ], + } + } + meta = project / "metaobjects" + meta.mkdir() + (meta / "meta.app.json").write_text(json.dumps(doc)) + return str(meta) + + +def _verify(tmp_path: Path, meta_dir: str) -> int: + out = tmp_path / "out" + assert main(as_config(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)])) == 0 + return main(as_config(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)])) + + +def test_advises_when_routes_are_generated_and_no_page_size_is_set( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + assert _verify(tmp_path, _project(tmp_path, None)) == 0 + assert _ADVISORY in capsys.readouterr().err + + +def test_silent_when_the_project_sets_api_pagination( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + api = {"pagination": {"defaultLimit": 50, "maxLimit": 1000}} + assert _verify(tmp_path, _project(tmp_path, api)) == 0 + assert _ADVISORY not in capsys.readouterr().err diff --git a/server/python/tests/config/test_neutral_config_api.py b/server/python/tests/config/test_neutral_config_api.py new file mode 100644 index 0000000000..a49df6c9d3 --- /dev/null +++ b/server/python/tests/config/test_neutral_config_api.py @@ -0,0 +1,49 @@ +"""The ``api`` block of ``.metaobjects/config.json``: ``api.pagination`` and ``api.unknownFields``.""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from metaobjects.api_settings import ApiPagination, ApiSettings +from metaobjects.config.neutral_config import read_neutral_config +from metaobjects.errors import ParseError + + +def _read(tmp_path: Path, api: object | None) -> ApiSettings: + cfg: dict[str, object] = {"schema_version": 1} + if api is not None: + cfg["api"] = api + (tmp_path / ".metaobjects").mkdir(exist_ok=True) + (tmp_path / ".metaobjects" / "config.json").write_text(json.dumps(cfg)) + neutral = read_neutral_config(tmp_path) + assert neutral is not None + return neutral.api + + +def test_absent_block_is_unpaged_and_rejects(tmp_path: Path) -> None: + assert _read(tmp_path, None) == ApiSettings(pagination=None, unknown_fields="reject") + + +def test_a_full_block_is_read(tmp_path: Path) -> None: + api = _read(tmp_path, {"pagination": {"defaultLimit": 50, "maxLimit": 1000}, "unknownFields": "drop"}) + assert api == ApiSettings(pagination=ApiPagination(50, 1000), unknown_fields="drop") + + +@pytest.mark.parametrize( + "api,message", + [ + ({"pagination": {"defaultLimit": 0, "maxLimit": 10}}, '"api.pagination.defaultLimit" must be a positive integer'), + ({"pagination": {"defaultLimit": 5, "maxLimit": 1.5}}, '"api.pagination.maxLimit" must be a positive integer'), + ({"pagination": {"defaultLimit": True, "maxLimit": 10}}, '"api.pagination.defaultLimit" must be a positive integer'), + ({"pagination": {"defaultLimit": 11, "maxLimit": 10}}, '"api.pagination.defaultLimit" must not exceed "api.pagination.maxLimit"'), + ({"pagination": {"defaultLimit": 5}}, 'needs both "defaultLimit" and "maxLimit"'), + ({"unknownFields": "ignore"}, '"api.unknownFields" must be "reject" or "drop"'), + ({"paging": {}}, '"api" has an unknown key "paging"'), + ([], '"api" must be an object'), + ], +) +def test_a_bad_block_names_the_key(tmp_path: Path, api: object, message: str) -> None: + with pytest.raises(ParseError, match=message.replace("(", r"\(")): + _read(tmp_path, api) diff --git a/server/python/tests/integration/api_contract_server.py b/server/python/tests/integration/api_contract_server.py index 486c5d4376..ffee2bd627 100644 --- a/server/python/tests/integration/api_contract_server.py +++ b/server/python/tests/integration/api_contract_server.py @@ -120,6 +120,19 @@ def _validation_issues(dto: dict[str, Any], *, create: bool) -> list[dict[str, A return issues +#: Every key the Author entity declares. A body key outside this set is refused first +#: (`api.unknownFields` at its default, `reject`, which this corpus does not change); a declared key +#: the client may not set (id, writeOnce, @autoSet) is not unknown and is dropped later. +_DECLARED_KEYS = frozenset({"id", "name", "bio", "createdAt", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt"}) + + +def _unknown_fields_400(dto: dict[str, Any]) -> JSONResponse | None: + unknown = sorted(k for k in dto if k not in _DECLARED_KEYS) + if not unknown: + return None + return JSONResponse(status_code=400, content={"error": "unknown_fields", "fields": unknown}) + + def _validation_400(issues: list[dict[str, Any]]) -> JSONResponse: return JSONResponse(status_code=400, content={"error": "validation", "issues": issues}) @@ -516,6 +529,9 @@ def get_author(author_id: int) -> Any: def create_author(dto: dict[str, Any]) -> Any: # FR-036: reject an over-length / missing field before persisting (cross-port 400, # every issue listed). + refused = _unknown_fields_400(dto) + if refused is not None: + return refused issues = _validation_issues(dto, create=True) if issues: return _validation_400(issues) @@ -529,6 +545,9 @@ def update_author(author_id: int, dto: dict[str, Any]) -> Any: # per meta.json) is a 400 — a present null on the nullable bio clears it, # and an omitted required field is untouched (never a 400). FR-036: an # over-length PRESENT value is a 400 too; every issue is listed. + refused = _unknown_fields_400(dto) + if refused is not None: + return refused issues = _validation_issues(dto, create=False) if issues: return _validation_400(issues) diff --git a/server/python/tests/integration/generated_api_settings_app.py b/server/python/tests/integration/generated_api_settings_app.py new file mode 100644 index 0000000000..ff21fe9ac7 --- /dev/null +++ b/server/python/tests/integration/generated_api_settings_app.py @@ -0,0 +1,74 @@ +"""Boot the GENERATED ``Item`` router of ``fixtures/api-contract-conformance/api-settings/``. + +Peer of ``generated_secret_fields_app.py``. The settings under test (``api.pagination`` and +``api.unknownFields``) live in the sub-corpus's own ``.metaobjects/config.json``, and reach the +generator the way an adopter's reach it: through the config the project load reads. +""" +from __future__ import annotations + +import importlib.util +import shutil +import sys +import tempfile +import uuid +from pathlib import Path + +from fastapi import FastAPI + +from metaobjects import MetaDataLoader +from metaobjects.codegen.generators.entity_model import render_entity_model +from metaobjects.codegen.generators.filter_allowlist_generator import render_filter_allowlist +from metaobjects.codegen.generators.router_generator import render_router +from metaobjects.meta.core.object.meta_object import MetaObject +from metaobjects.shared.base_types import TYPE_OBJECT + +from .corpus_config import attach_corpus_wire +from .generated_router_app import InMemoryAuthorRepository + + +def load_entity(meta_json: Path, name: str) -> MetaObject: + tmp = Path(tempfile.mkdtemp(prefix="apic-settings-meta-")) + shutil.copy(meta_json, tmp / "meta.json") + result = MetaDataLoader.from_directory(str(tmp)) + attach_corpus_wire(result.root, meta_json) # also attaches the corpus's `api` settings + if result.errors: + raise RuntimeError("; ".join(f"{e.code}: {e.message}" for e in result.errors)) + for c in result.root.children(): + if c.type == TYPE_OBJECT and isinstance(c, MetaObject) and c.name.split("::")[-1] == name: + return c + raise RuntimeError(f"{name} entity not found") + + +def build_api_settings_app(corpus_root: Path) -> tuple[FastAPI, InMemoryAuthorRepository]: + item = load_entity(corpus_root / "meta.json", "Item") + router_src = render_router(item) + allowlist_src = render_filter_allowlist(item) + if router_src is None or allowlist_src is None: + raise RuntimeError("router_generator / filter_allowlist_generator returned None for Item") + + pkg_name = f"genitems_{uuid.uuid4().hex[:8]}" + tmp = Path(tempfile.mkdtemp(prefix="apic-settings-gen-")) + pkg_dir = tmp / pkg_name + pkg_dir.mkdir() + (pkg_dir / "__init__.py").write_text("") + (pkg_dir / "item_filter_allowlist.py").write_text(allowlist_src) + (pkg_dir / "Item.py").write_text(render_entity_model(item)) + (pkg_dir / "item_router.py").write_text(router_src) + + sys.path.insert(0, str(tmp)) + pkg_spec = importlib.util.spec_from_file_location( + pkg_name, pkg_dir / "__init__.py", submodule_search_locations=[str(pkg_dir)] + ) + pkg_mod = importlib.util.module_from_spec(pkg_spec) + sys.modules[pkg_name] = pkg_mod + pkg_spec.loader.exec_module(pkg_mod) + spec = importlib.util.spec_from_file_location(f"{pkg_name}.item_router", pkg_dir / "item_router.py") + router_mod = importlib.util.module_from_spec(spec) + sys.modules[f"{pkg_name}.item_router"] = router_mod + spec.loader.exec_module(router_mod) + + repo = InMemoryAuthorRepository() + app = FastAPI() + app.include_router(router_mod.router) + app.dependency_overrides[router_mod.get_repository] = lambda: repo + return app, repo diff --git a/server/python/tests/integration/test_api_contract_api_settings.py b/server/python/tests/integration/test_api_contract_api_settings.py new file mode 100644 index 0000000000..cce654cf1a --- /dev/null +++ b/server/python/tests/integration/test_api_contract_api_settings.py @@ -0,0 +1,70 @@ +"""Cross-port API contract for the project ``api`` block (``api.pagination``, ``api.unknownFields``). + +Drives ``fixtures/api-contract-conformance/api-settings/`` against the GENERATED ``Item`` router, +with the settings read from the sub-corpus's own ``.metaobjects/config.json``. Generated lane +only: what is under test is whether the GENERATOR emits routes that honour them. + + cd server/python + uv run --extra dev pytest tests/integration/test_api_contract_api_settings.py -v +""" +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import pytest +import yaml +from fastapi.testclient import TestClient + +from . import api_contract_assertions +from .generated_api_settings_app import build_api_settings_app + + +def _find_corpus_root() -> Path: + cur = Path(__file__).resolve() + while cur != cur.parent: + candidate = cur / "fixtures" / "api-contract-conformance" / "api-settings" + if candidate.is_dir(): + return candidate + cur = cur.parent + raise RuntimeError("Could not find fixtures/api-contract-conformance/api-settings") + + +_CORPUS = _find_corpus_root() +_SCENARIOS = [ + (raw["name"], raw) + for raw in (yaml.safe_load(p.read_text()) for p in sorted((_CORPUS / "scenarios").glob("*.yaml"))) +] +_SEED_ROWS = json.loads((_CORPUS / "seed.json").read_text())["rows"] +_APP, _REPO = build_api_settings_app(_CORPUS) +_CLIENT = TestClient(_APP) + + +def _parse(text: str | None) -> Any: + if not text: + return None + try: + return api_contract_assertions.parse_body(text) + except json.JSONDecodeError: + return text + + +@pytest.mark.parametrize("scenario_name,scenario", _SCENARIOS, ids=[n for n, _ in _SCENARIOS]) +def test_api_settings_scenario(scenario_name: str, scenario: dict[str, Any]) -> None: + _REPO.reset() + _REPO.seed(_SEED_ROWS) + for req in scenario["requests"]: + kwargs: dict[str, Any] = {} + if req.get("body") is not None: + kwargs["json"] = req["body"] + response = _CLIENT.request(str(req["method"]).upper(), str(req["path"]), **kwargs) + expect = req["expect"] + api_contract_assertions.assert_response( + scenario_name=scenario_name, + request_id=str(req.get("id", "?")), + expect_status=int(expect["status"]), + expect_body=expect.get("body"), + status=response.status_code, + body=_parse(response.text), + ) diff --git a/server/typescript/packages/cli/src/commands/init.ts b/server/typescript/packages/cli/src/commands/init.ts index 9285e0e575..bd4a0e1d1e 100644 --- a/server/typescript/packages/cli/src/commands/init.ts +++ b/server/typescript/packages/cli/src/commands/init.ts @@ -2,7 +2,7 @@ import { mkdir, writeFile, readFile, readdir, stat, rm } from "node:fs/promises" import { join } from "node:path"; import { dirname } from "node:path"; import { existsSync as existsSyncWrap, readFileSync as readFileSyncWrap } from "node:fs"; -import { INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT } from "@metaobjectsdev/metadata"; +import { INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT, UNKNOWN_FIELDS_REJECT } from "@metaobjectsdev/metadata"; import { DEFAULT_CONFIG, ConfigSchema, saveConfig, PACKAGE_MANIFEST_FILE, DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, DEPS_DIR, LOCK_FILE } from "@metaobjectsdev/sdk"; import { assemble, resolveAgentContextRoot, planScaffold, @@ -538,10 +538,14 @@ async function writeConfigFile(opts: InitOptions, result: InitResult, agentDir: const dialect = resolveInitDialect(opts); // A new project starts with a page size on every generated list route. It is written here // and not into DEFAULT_CONFIG: the merge below must not add a cap to an existing project - // on `--force`, where it would truncate a client that reads a whole list. + // on `--force`, where it would truncate a client that reads a whole list. `unknownFields` + // is written out at its default, so the choice is visible where a project would change it. const scaffold = { ...DEFAULT_CONFIG, - api: { pagination: { defaultLimit: INIT_DEFAULT_LIMIT, maxLimit: INIT_MAX_LIMIT } }, + api: { + pagination: { defaultLimit: INIT_DEFAULT_LIMIT, maxLimit: INIT_MAX_LIMIT }, + unknownFields: UNKNOWN_FIELDS_REJECT, + }, }; const freshConfig = dialect === "d1" ? ConfigSchema.parse({ ...scaffold, migrate: buildD1MigrateBlock(opts.cwd) }) diff --git a/server/typescript/packages/cli/src/lib/collection-load-options.ts b/server/typescript/packages/cli/src/lib/collection-load-options.ts index 4b5c685edc..a0126169fe 100644 --- a/server/typescript/packages/cli/src/lib/collection-load-options.ts +++ b/server/typescript/packages/cli/src/lib/collection-load-options.ts @@ -27,7 +27,7 @@ import type { Collection, LoadMemoryOptions } from "@metaobjectsdev/sdk"; */ export function collectionLoadOptions( collection: Collection, -): Required> { +): Required> { return { files: collection.files, fileIds: collection.fileIds, @@ -43,6 +43,7 @@ export function collectionLoadOptions( wireFormat: collection.wireFormat, enumDefaults: collection.enumDefaults ?? DEFAULT_ENUM_DEFAULTS, apiPagination: collection.apiPagination, + apiUnknownFields: collection.apiUnknownFields, // The collection's migrate settings (`migrate.conventions`, `migrate.checks`) are not load // options: they reach the expected-schema builds through `schemaBuildOptions`. }; diff --git a/server/typescript/packages/cli/test/init.test.ts b/server/typescript/packages/cli/test/init.test.ts index fe635c2fb9..b24f3927e1 100644 --- a/server/typescript/packages/cli/test/init.test.ts +++ b/server/typescript/packages/cli/test/init.test.ts @@ -182,10 +182,10 @@ describe("init() — happy path", () => { expect(config.pending_in_git).toBe(true); }); - test("a new project starts with a page size on every generated list route", async () => { + test("a new project starts with a page size and refuses undeclared body keys", async () => { await init({ cwd }); const config = JSON.parse(readFileSync(join(cwd, ".metaobjects", "config.json"), "utf8")); - expect(config.api).toEqual({ pagination: { defaultLimit: 50, maxLimit: 1000 } }); + expect(config.api).toEqual({ pagination: { defaultLimit: 50, maxLimit: 1000 }, unknownFields: "reject" }); }); test("writes a .gitignore under .metaobjects/ that includes .gen-state", async () => { diff --git a/server/typescript/packages/codegen-ts/src/api-surface.ts b/server/typescript/packages/codegen-ts/src/api-surface.ts index c761d468ab..24aa51c755 100644 --- a/server/typescript/packages/codegen-ts/src/api-surface.ts +++ b/server/typescript/packages/codegen-ts/src/api-surface.ts @@ -38,6 +38,8 @@ import { refTarget, wireFormatOf, apiPaginationOf, + apiUnknownFieldsOf, + UNKNOWN_FIELDS_REJECT, type MetaData, type MetaField, type MetaObject, @@ -253,6 +255,25 @@ export function valueObjectShapes(entity: MetaObject): Record flatSurfaceFields(o).map((f) => f.name)))]; +} + +/** + * The `knownFields` option of a mount that accepts writes: present when the project refuses + * undeclared body keys (`api.unknownFields: reject`, the default), and "" under `drop`, which + * keeps the mount dropping them silently. `root` is the loaded root the setting is read from. + */ +export function knownFieldsLine(entity: MetaObject, indent: string, root?: MetaData, hierarchy: readonly MetaObject[] = [entity]): string { + if (apiUnknownFieldsOf(root ?? entity) !== UNKNOWN_FIELDS_REJECT) return ""; + return `\n${indent}knownFields: ${JSON.stringify(knownFieldNames(hierarchy))},`; +} + /** * The `pagination` option of a list mount: the project's page size (`api.pagination` in * `.metaobjects/config.json`), or nothing when the project declares none, which keeps the output diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index 3f7b75e781..999de4901a 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -146,7 +146,7 @@ export { renderAgentApi, renderApiIndex, renderEntityApiPage } from "./generator export { isProjection, isWriteThrough } from "./projection/projection-detector.js"; export { isAbstract, emitsInstanceArtifacts, emitsWriteArtifacts } from "./instance-artifacts.js"; // The UI tier asks THESE — "is there an endpoint?" — never the storage predicates. -export { hasGeneratedForm, hasItemRoute, decimalColumnsLine, paginationLine, writeDeniedColumnsLine, isCredentialField, decimalFieldNames, itemRouteField, restPath, servesClientHooks, servesClientTier, servesReadApi, servesWriteApi } from "./api-surface.js"; +export { hasGeneratedForm, hasItemRoute, decimalColumnsLine, paginationLine, knownFieldsLine, knownFieldNames, writeDeniedColumnsLine, isCredentialField, decimalFieldNames, itemRouteField, restPath, servesClientHooks, servesClientTier, servesReadApi, servesWriteApi } from "./api-surface.js"; // #356 — every emitter selects a field's view by the SURFACE it renders, never by // declaration position. An owned generator (FR-040) composing the render layer must // use this too, or it reinstates the order-dependence in its own copy. diff --git a/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts b/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts index bc0ff5116c..8dc0f5978b 100644 --- a/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts +++ b/server/typescript/packages/codegen-ts/src/reference/routes-hono.ts @@ -61,6 +61,7 @@ import { decimalColumnsLine, paginationLine, writeDeniedColumnsLine, + knownFieldsLine, servesReadApi, formatTs, entityOutputPath, @@ -246,7 +247,7 @@ export function ${handlerName}(app: ${HonoSym}, deps: { db: unkno updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} }); } `; diff --git a/server/typescript/packages/codegen-ts/src/reference/routes.ts b/server/typescript/packages/codegen-ts/src/reference/routes.ts index 8ffe8c2da9..59c800867a 100644 --- a/server/typescript/packages/codegen-ts/src/reference/routes.ts +++ b/server/typescript/packages/codegen-ts/src/reference/routes.ts @@ -71,6 +71,7 @@ import { decimalColumnsLine, paginationLine, writeDeniedColumnsLine, + knownFieldsLine, servesReadApi, formatTs, entityOutputPath, @@ -312,7 +313,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -337,7 +338,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -651,7 +652,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${knownFieldsLine(sub, " ", ctx.loadedRoot, hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts b/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts index a4f947ecd3..1d458d19df 100644 --- a/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts +++ b/server/typescript/packages/codegen-ts/src/templates/routes-file-hono.ts @@ -31,7 +31,7 @@ import { entityModuleSpecifier, siblingSpecifier } from "../import-path.js"; import { GENERATED_HEADER, GENERATED_EDIT_NOTE, sidecarLine } from "../constants.js"; import { isProjection, isWriteThrough } from "../projection/projection-detector.js"; import { isReport } from "../source-detect.js"; -import { decimalColumnsLine, itemRouteField, paginationLine, writeDeniedColumnsLine } from "../api-surface.js"; +import { decimalColumnsLine, itemRouteField, paginationLine, knownFieldsLine, writeDeniedColumnsLine } from "../api-surface.js"; import { DEFAULT_ID_FIELD } from "../pk-resolver.js"; import { authSeamJsDoc, type CrudVerb, exposeLine } from "../routes-expose.js"; import { effectivePackage } from "../docs-paths.js"; @@ -201,7 +201,7 @@ export function ${handlerName}(app: ${HonoSym}, deps: { db: unkno updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineHono} }); } `; diff --git a/server/typescript/packages/codegen-ts/src/templates/routes-file.ts b/server/typescript/packages/codegen-ts/src/templates/routes-file.ts index d267a3ca99..6c59056475 100644 --- a/server/typescript/packages/codegen-ts/src/templates/routes-file.ts +++ b/server/typescript/packages/codegen-ts/src/templates/routes-file.ts @@ -29,7 +29,7 @@ import { GENERATED_HEADER, GENERATED_EDIT_NOTE, sidecarLine } from "../constants import { routesHandlerName } from "../naming.js"; import { isProjection, isWriteThrough } from "../projection/projection-detector.js"; import { isReport } from "../source-detect.js"; -import { decimalColumnsLine, itemRouteField, paginationLine, writeDeniedColumnsLine } from "../api-surface.js"; +import { decimalColumnsLine, itemRouteField, paginationLine, knownFieldsLine, writeDeniedColumnsLine } from "../api-surface.js"; import { DEFAULT_ID_FIELD, pkInfoOf } from "../pk-resolver.js"; import type { RelationEntry } from "../relation-resolver.js"; import { isTphDiscriminatorBase, tphPlan } from "./tph-discriminator.js"; @@ -248,7 +248,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -273,7 +273,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -579,7 +579,7 @@ function renderTphRoutesFile( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${knownFieldsLine(sub, " ", ctx.loadedRoot, hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/Tag.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/Tag.routes.ts index 5697f1f020..c427da5bcc 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/Tag.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/Tag.routes.ts @@ -45,5 +45,6 @@ export async function tagRoutes(fastify: FastifyInstance) { filterAllowlist: TagFilterAllowlist, sortAllowlist: TagSortAllowlist, dialect: "sqlite", + knownFields: ["id", "label", "customerId"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Order.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Order.routes.ts index 2f3fb60d84..6fe09bbd7a 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Order.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Order.routes.ts @@ -45,5 +45,6 @@ export async function orderRoutes(fastify: FastifyInstance) { filterAllowlist: OrderFilterAllowlist, sortAllowlist: OrderSortAllowlist, dialect: "sqlite", + knownFields: ["id", "customerId", "productId", "quantity"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Product.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Product.routes.ts index 0953c4bd72..9cbb46e4ed 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Product.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/commerce/Product.routes.ts @@ -45,5 +45,6 @@ export async function productRoutes(fastify: FastifyInstance) { filterAllowlist: ProductFilterAllowlist, sortAllowlist: ProductSortAllowlist, dialect: "sqlite", + knownFields: ["id", "name", "tagId"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/users/Customer.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/users/Customer.routes.ts index eb89e4562c..c004c44d19 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/users/Customer.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/package/shop/users/Customer.routes.ts @@ -45,5 +45,6 @@ export async function customerRoutes(fastify: FastifyInstance) { filterAllowlist: CustomerFilterAllowlist, sortAllowlist: CustomerSortAllowlist, dialect: "sqlite", + knownFields: ["id", "email"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Exercise.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Exercise.routes.ts index b670590733..3934b15903 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Exercise.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Exercise.routes.ts @@ -45,5 +45,6 @@ export async function exerciseRoutes(fastify: FastifyInstance) { filterAllowlist: ExerciseFilterAllowlist, sortAllowlist: ExerciseSortAllowlist, dialect: "postgres", + knownFields: ["id", "workoutId", "name", "sets", "reps", "notes"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Program.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Program.routes.ts index ad17c6ef12..d9389baea0 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Program.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Program.routes.ts @@ -45,5 +45,14 @@ export async function programRoutes(fastify: FastifyInstance) { filterAllowlist: ProgramFilterAllowlist, sortAllowlist: ProgramSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "slug", + "title", + "description", + "priceCents", + "isPublished", + "createdAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Purchase.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Purchase.routes.ts index 1fa4009836..02caa5ee14 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Purchase.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Purchase.routes.ts @@ -45,5 +45,12 @@ export async function purchaseRoutes(fastify: FastifyInstance) { filterAllowlist: PurchaseFilterAllowlist, sortAllowlist: PurchaseSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "subscriberId", + "programId", + "amountCents", + "purchasedAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Subscriber.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Subscriber.routes.ts index e392ae1e67..5b0664743b 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Subscriber.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Subscriber.routes.ts @@ -45,5 +45,13 @@ export async function subscriberRoutes(fastify: FastifyInstance) { filterAllowlist: SubscriberFilterAllowlist, sortAllowlist: SubscriberSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "email", + "firstName", + "lastName", + "subscribed", + "createdAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Tag.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Tag.routes.ts index 317d919ece..8dc41fe210 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Tag.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Tag.routes.ts @@ -45,5 +45,6 @@ export async function tagRoutes(fastify: FastifyInstance) { filterAllowlist: TagFilterAllowlist, sortAllowlist: TagSortAllowlist, dialect: "postgres", + knownFields: ["id", "name", "slug"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Video.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Video.routes.ts index 74ea40e1c6..1c31abf82d 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Video.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Video.routes.ts @@ -45,5 +45,6 @@ export async function videoRoutes(fastify: FastifyInstance) { filterAllowlist: VideoFilterAllowlist, sortAllowlist: VideoSortAllowlist, dialect: "postgres", + knownFields: ["id", "exerciseId", "url", "title", "durationSeconds"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Week.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Week.routes.ts index 717c7c7cad..6e199f5fa7 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Week.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Week.routes.ts @@ -45,5 +45,6 @@ export async function weekRoutes(fastify: FastifyInstance) { filterAllowlist: WeekFilterAllowlist, sortAllowlist: WeekSortAllowlist, dialect: "postgres", + knownFields: ["id", "programId", "weekNumber", "title"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Workout.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Workout.routes.ts index 64afc88275..45a2b0f111 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Workout.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/Workout.routes.ts @@ -45,5 +45,6 @@ export async function workoutRoutes(fastify: FastifyInstance) { filterAllowlist: WorkoutFilterAllowlist, sortAllowlist: WorkoutSortAllowlist, dialect: "postgres", + knownFields: ["id", "weekId", "title", "orderIndex", "durationMinutes"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/WorkoutEvent.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/WorkoutEvent.routes.ts index e54d586197..5111504012 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/WorkoutEvent.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/postgres/WorkoutEvent.routes.ts @@ -45,5 +45,12 @@ export async function workoutEventRoutes(fastify: FastifyInstance) { filterAllowlist: WorkoutEventFilterAllowlist, sortAllowlist: WorkoutEventSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "subscriberId", + "workoutId", + "completedAt", + "durationMinutes", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Exercise.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Exercise.routes.ts index 31ee6515c0..a8e5990522 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Exercise.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Exercise.routes.ts @@ -45,5 +45,6 @@ export async function exerciseRoutes(fastify: FastifyInstance) { filterAllowlist: ExerciseFilterAllowlist, sortAllowlist: ExerciseSortAllowlist, dialect: "sqlite", + knownFields: ["id", "workoutId", "name", "sets", "reps", "notes"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Program.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Program.routes.ts index 224547ebb3..c722dafb47 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Program.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Program.routes.ts @@ -45,5 +45,14 @@ export async function programRoutes(fastify: FastifyInstance) { filterAllowlist: ProgramFilterAllowlist, sortAllowlist: ProgramSortAllowlist, dialect: "sqlite", + knownFields: [ + "id", + "slug", + "title", + "description", + "priceCents", + "isPublished", + "createdAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Purchase.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Purchase.routes.ts index d2ed7d3db8..f9b9b7ea77 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Purchase.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Purchase.routes.ts @@ -45,5 +45,12 @@ export async function purchaseRoutes(fastify: FastifyInstance) { filterAllowlist: PurchaseFilterAllowlist, sortAllowlist: PurchaseSortAllowlist, dialect: "sqlite", + knownFields: [ + "id", + "subscriberId", + "programId", + "amountCents", + "purchasedAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Subscriber.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Subscriber.routes.ts index 4d8c537001..5b871c31b5 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Subscriber.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Subscriber.routes.ts @@ -45,5 +45,13 @@ export async function subscriberRoutes(fastify: FastifyInstance) { filterAllowlist: SubscriberFilterAllowlist, sortAllowlist: SubscriberSortAllowlist, dialect: "sqlite", + knownFields: [ + "id", + "email", + "firstName", + "lastName", + "subscribed", + "createdAt", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Tag.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Tag.routes.ts index 1132211583..c9783bb630 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Tag.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Tag.routes.ts @@ -45,5 +45,6 @@ export async function tagRoutes(fastify: FastifyInstance) { filterAllowlist: TagFilterAllowlist, sortAllowlist: TagSortAllowlist, dialect: "sqlite", + knownFields: ["id", "name", "slug"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Video.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Video.routes.ts index 1d25cc8aba..d2e1b4ec2c 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Video.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Video.routes.ts @@ -45,5 +45,6 @@ export async function videoRoutes(fastify: FastifyInstance) { filterAllowlist: VideoFilterAllowlist, sortAllowlist: VideoSortAllowlist, dialect: "sqlite", + knownFields: ["id", "exerciseId", "url", "title", "durationSeconds"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Week.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Week.routes.ts index c1f4a36e87..4a80f3fcd7 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Week.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Week.routes.ts @@ -45,5 +45,6 @@ export async function weekRoutes(fastify: FastifyInstance) { filterAllowlist: WeekFilterAllowlist, sortAllowlist: WeekSortAllowlist, dialect: "sqlite", + knownFields: ["id", "programId", "weekNumber", "title"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Workout.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Workout.routes.ts index d717e0a2b8..8711be0ea6 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Workout.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/Workout.routes.ts @@ -45,5 +45,6 @@ export async function workoutRoutes(fastify: FastifyInstance) { filterAllowlist: WorkoutFilterAllowlist, sortAllowlist: WorkoutSortAllowlist, dialect: "sqlite", + knownFields: ["id", "weekId", "title", "orderIndex", "durationMinutes"], }); } diff --git a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/WorkoutEvent.routes.ts b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/WorkoutEvent.routes.ts index 559edaeb7a..dde852f2e3 100644 --- a/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/WorkoutEvent.routes.ts +++ b/server/typescript/packages/codegen-ts/test/golden/__snapshots__/sqlite/WorkoutEvent.routes.ts @@ -45,5 +45,12 @@ export async function workoutEventRoutes(fastify: FastifyInstance) { filterAllowlist: WorkoutEventFilterAllowlist, sortAllowlist: WorkoutEventSortAllowlist, dialect: "sqlite", + knownFields: [ + "id", + "subscriberId", + "workoutId", + "completedAt", + "durationMinutes", + ], }); } diff --git a/server/typescript/packages/codegen-ts/test/routes-known-fields.test.ts b/server/typescript/packages/codegen-ts/test/routes-known-fields.test.ts new file mode 100644 index 0000000000..6ac40f5294 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/routes-known-fields.test.ts @@ -0,0 +1,109 @@ +// `api.unknownFields` reaches every generated write mount. Under `reject` (the default, also +// when the project sets nothing) the packaged routes generators and the reference copies +// `meta eject` writes pass `knownFields`, the entity's declared fields, so the runtime refuses +// any other body key; under `drop` they pass nothing and undeclared keys are ignored. A TPH +// subtype's routes know every field of the hierarchy, so a sibling's column is dropped there, +// not refused. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource, type MetaRoot, type MetaObject } from "@metaobjectsdev/metadata"; +import { FileSource } from "@metaobjectsdev/metadata/core"; +import { mkdtempSync, readFileSync, readdirSync, rmSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { runGen } from "../src/runner.js"; +import { defineConfig } from "../src/metaobjects-config.js"; +import type { Generator } from "../src/generator.js"; +import { routesFile } from "../src/generators/routes-file.js"; +import { routesFileHono } from "../src/generators/routes-file-hono.js"; +import { routesFile as referenceRoutes } from "../src/reference/routes.js"; +import { routesFileHono as referenceRoutesHono } from "../src/reference/routes-hono.js"; +import { knownFieldNames, tphHierarchy } from "../src/api-surface.js"; + +const MODEL = { + "metadata.root": { + package: "acme::shop", + children: [ + { + "object.entity": { + name: "Product", + children: [ + { "source.rdb": { "@table": "products" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title" } }, + { "field.string": { name: "sku", "@readOnly": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ], + }, + }, + ], + }, +}; + +async function loadRoot(): Promise { + const res = await new MetaDataLoader().load([ + new InMemoryStringSource(JSON.stringify(MODEL), { id: "meta.json", format: "json" }), + ]); + expect(res.errors).toEqual([]); + return res.root; +} + +function allFiles(dir: string): string[] { + return readdirSync(dir).flatMap((n) => { + const p = join(dir, n); + return statSync(p).isDirectory() ? allFiles(p) : [p]; + }); +} + +async function routesSource(generator: Generator, mode: "reject" | "drop" | undefined): Promise { + const root = await loadRoot(); + if (mode !== undefined) root.apiUnknownFields = mode; + const tmp = mkdtempSync(join(tmpdir(), "routes-known-fields-")); + try { + await runGen({ + config: defineConfig({ outDir: join(tmp, "src"), extStyle: "none", dbImport: "../db", dialect: "sqlite", generators: [generator] }), + metadata: root, projectRoot: tmp, + }); + return allFiles(join(tmp, "src")).filter((f) => /Product.*routes/i.test(f)).map((f) => readFileSync(f, "utf8")).join("\n"); + } finally { rmSync(tmp, { recursive: true, force: true }); } +} + +const GENERATORS: Array<[string, () => Generator]> = [ + ["routesFile", () => routesFile()], + ["routesFileHono", () => routesFileHono()], + ["reference routes", () => referenceRoutes()], + ["reference routes-hono", () => referenceRoutesHono()], +]; + +// The declared fields, server-owned ones (`id`, the readOnly `sku`) included: known, so they +// are dropped rather than refused. +const PRODUCT_KNOWN = 'knownFields: ["id", "title", "sku"]'; + +describe("generated write routes carry the unknown-field rule", () => { + for (const [name, make] of GENERATORS) { + test(`${name}: no setting means reject, and the declared fields are passed`, async () => { + expect(await routesSource(make(), undefined)).toContain(PRODUCT_KNOWN); + }); + test(`${name}: reject passes the declared fields`, async () => { + expect(await routesSource(make(), "reject")).toContain(PRODUCT_KNOWN); + }); + test(`${name}: drop passes none`, async () => { + const src = await routesSource(make(), "drop"); + expect(src.length).toBeGreaterThan(0); + expect(src).not.toContain("knownFields"); + }); + } +}); + +describe("knownFieldNames on a single-table hierarchy", () => { + test("every subtype knows the base's and every sibling's fields", async () => { + const tphMeta = resolve(import.meta.dir, "../../../../../fixtures/api-contract-conformance/tph/meta.json"); + const res = await new MetaDataLoader().load([new FileSource(tphMeta)]); + expect(res.errors).toEqual([]); + const bridge = res.root.objects().find((o) => o.name === "BridgeAuth") as MetaObject; + expect(knownFieldNames(tphHierarchy(bridge, res.root))).toEqual([ + "id", "type", "reference", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt", + "quantity", "copayAmount", "approver", + ]); + }); +}); diff --git a/server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-api-settings-generated-server.ts similarity index 79% rename from server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts rename to server/typescript/packages/integration-tests/src/api-contract-api-settings-generated-server.ts index e0b16e315b..27b45ee083 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-pagination-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-api-settings-generated-server.ts @@ -1,7 +1,8 @@ -// api-contract-pagination-generated-server.ts — boots the GENERATED Item routes (the deployed -// artifact) over HTTP and drives them against the pagination corpus. The page size is not passed -// here: it reaches the emitted routes from the corpus's own `.metaobjects/config.json`, read by -// the metadata load, which is the path an adopter's `api.pagination` takes. +// api-contract-api-settings-generated-server.ts — boots the GENERATED Item routes (the deployed +// artifact) over HTTP and drives them against the api-settings corpus. The settings are not +// passed here: the page size and the unknown-field rule reach the emitted routes from the +// corpus's own `.metaobjects/config.json`, read by the metadata load, which is the path an +// adopter's `api` block takes. // // Same shape as api-contract-secret-fields-generated-server.ts: runGen over the corpus meta into // a temp dir, provision the table the emitted Drizzle binding reads, mount the EMITTED routes. @@ -16,24 +17,24 @@ import type pg from "pg"; import { executeSql } from "./postgres-sql.ts"; import { loadMetadataFile } from "./load-metadata.ts"; -export interface PaginationSeed { +export interface ApiSettingsSeed { rows: Array<{ id: number; title: string }>; } -export interface GeneratedPaginationServerHandle { +export interface GeneratedApiSettingsServerHandle { baseUrl: string; - applySeed(seed: PaginationSeed): Promise; + applySeed(seed: ApiSettingsSeed): Promise; close(): Promise; } -export async function startGeneratedPaginationServer( +export async function startGeneratedApiSettingsServer( connectionUri: string, metaPath: string, -): Promise { +): Promise { const here = dirname(fileURLToPath(import.meta.url)); const genTmpRoot = join(here, "..", ".gen-tmp"); mkdirSync(genTmpRoot, { recursive: true }); - const tmp = mkdtempSync(join(genTmpRoot, "api-contract-pagination-")); + const tmp = mkdtempSync(join(genTmpRoot, "api-contract-api-settings-")); const root = await loadMetadataFile(metaPath); const lr = await runGen({ @@ -85,7 +86,7 @@ export const db = drizzle(pool); return { baseUrl, - applySeed: async (seed: PaginationSeed) => { + applySeed: async (seed: ApiSettingsSeed) => { await executeSql(connectionUri, `TRUNCATE TABLE "items" RESTART IDENTITY;`); for (const r of seed.rows) await executeSql(connectionUri, `INSERT INTO "items" ("id","title") VALUES (${r.id}, ${str(r.title)})`); diff --git a/server/typescript/packages/integration-tests/src/api-contract-server.ts b/server/typescript/packages/integration-tests/src/api-contract-server.ts index f024fd9ade..abaebef5e3 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-server.ts @@ -189,8 +189,15 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis }); // The keys a client may write, as the generated routes' Insert/Update schemas have them: the - // server-generated `id`, the @autoSet columns and any key the entity does not declare are + // server-generated `id` and the @autoSet columns are declared but server-owned, so they are // DROPPED silently (never 400'd), and `issuedCurrency` (writeOnce) is settable on create only. + // A key the entity does not declare is refused first: `api.unknownFields` at its default, + // `reject`, which this corpus does not change. + const DECLARED_KEYS = new Set(["id", "name", "bio", "createdAt", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt"]); + const unknownKeys = (body: unknown): string[] => + body !== null && typeof body === "object" && !Array.isArray(body) + ? Object.keys(body).filter((k) => !DECLARED_KEYS.has(k)).sort() + : []; const CREATE_KEYS = ["name", "bio", "createdAt", "issuedCurrency"] as const; const UPDATE_KEYS = ["name", "bio", "createdAt"] as const; const pick = (body: unknown, keys: readonly string[]): Record => { @@ -204,6 +211,8 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // (that is codegen-only), so the reference server stamps explicitly to match // the generated lane's InsertSchema. Both onCreate + onUpdate columns get the // SAME captured now() on create; any caller-supplied value is ignored. + const unknown = unknownKeys(req.body); + if (unknown.length > 0) return reply.code(400).send({ error: "unknown_fields", fields: unknown }); const now = new Date().toISOString(); const body = { ...pick(req.body, CREATE_KEYS), autoCreatedAt: now, autoUpdatedAt: now }; const row = await om.create(ENTITY, body); @@ -225,6 +234,8 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // plain z.object() discards unknown keys). Dropping it also covers the // present-NULL arm for free: clearing is a write, so an explicit null must // not reach the column either. + const unknown = unknownKeys(req.body); + if (unknown.length > 0) return reply.code(400).send({ error: "unknown_fields", fields: unknown }); const body = { ...pick(req.body, UPDATE_KEYS), autoUpdatedAt: new Date().toISOString() }; // A ValidationError (e.g. FR-035 present-null on a @required field) propagates // to the shared setErrorHandler above → 400 {error:"validation"}. diff --git a/server/typescript/packages/integration-tests/src/api-contract-tph-server.ts b/server/typescript/packages/integration-tests/src/api-contract-tph-server.ts index bbf17f496c..9c07ed59b9 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-tph-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-tph-server.ts @@ -147,6 +147,14 @@ const BridgeSortAllowlist = { id: {}, reference: {}, quantity: {} } as const sat const PriorAuthSortAllowlist = { id: {}, reference: {}, approver: {} } as const satisfies SortAllowlist; const CopaySortAllowlist = { id: {}, reference: {} } as const satisfies SortAllowlist; +// `api.unknownFields` at its default, `reject`, which this corpus does not change: every +// field the hierarchy declares, so a sibling subtype's column is dropped on another subtype's +// route rather than refused, as the generated per-subtype mounts have it. +const KNOWN_FIELDS = [ + "id", "type", "reference", "issuedCurrency", "autoCreatedAt", "autoUpdatedAt", + "quantity", "copayAmount", "approver", +] as const; + export async function startTphServer(connectionUri: string): Promise { const pool = new pg.Pool({ connectionString: connectionUri, types: bigintAsNumberTypes }); const db = drizzle(pool); @@ -177,19 +185,19 @@ export async function startTphServer(connectionUri: string): Promise { + for (const scenario of loadScenarios(API_CONTRACT_API_SETTINGS_SCENARIOS_DIR)) { + test(scenario.name, async () => { + const pg = await startPostgres(); + let server: GeneratedApiSettingsServerHandle | null = null; + try { + server = await startGeneratedApiSettingsServer(pg.connectionUri, META_PATH); + await server.applySeed(SEED); + await runScenario(scenario, server); + } finally { + if (server) await server.close(); + await pg.stop(); + } + }, { timeout: 60_000 }); + } +}); + +async function runScenario(scenario: ApiScenario, server: GeneratedApiSettingsServerHandle): Promise { + for (const req of scenario.requests) { + const init: RequestInit = { method: req.method }; + if (req.body !== undefined) { + init.body = JSON.stringify(req.body); + init.headers = { "content-type": "application/json" }; + } + const res = await fetch(server.baseUrl + req.path, init); + const bodyText = await res.text(); + let body: unknown = null; + if (bodyText.length > 0) { + try { body = parseBodyExact(bodyText); } catch { body = bodyText; } + } + assertResponse(scenario.name, req, res.status, body); + } +} diff --git a/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts b/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts deleted file mode 100644 index aae908fcb4..0000000000 --- a/server/typescript/packages/integration-tests/test/api-contract-pagination.test.ts +++ /dev/null @@ -1,55 +0,0 @@ -// pagination api-contract conformance (GENERATED lane). -// -// Drives fixtures/api-contract-conformance/pagination/ over HTTP against the GENERATED Item -// routes, emitted under the corpus's `api.pagination` (default 2, max 4): a list with no limit -// answers the default page, a limit over the cap answers 400 limit_too_large. Generated lane -// only — see the corpus README. One Postgres testcontainer per scenario. - -import { describe, test } from "bun:test"; -import { readFileSync } from "node:fs"; -import { join } from "node:path"; -import { API_CONTRACT_PAGINATION_DIR, API_CONTRACT_PAGINATION_SCENARIOS_DIR } from "../src/paths.ts"; -import { loadScenarios, assertResponse, type ApiScenario, parseBodyExact } from "../src/api-contract-scenario.ts"; -import { startPostgres } from "../src/postgres-container.ts"; -import { - startGeneratedPaginationServer, - type GeneratedPaginationServerHandle, - type PaginationSeed, -} from "../src/api-contract-pagination-generated-server.ts"; - -const SEED = JSON.parse(readFileSync(join(API_CONTRACT_PAGINATION_DIR, "seed.json"), "utf8")) as PaginationSeed; -const META_PATH = join(API_CONTRACT_PAGINATION_DIR, "meta.json"); - -describe("api contract pagination — GENERATED routes lane", () => { - for (const scenario of loadScenarios(API_CONTRACT_PAGINATION_SCENARIOS_DIR)) { - test(scenario.name, async () => { - const pg = await startPostgres(); - let server: GeneratedPaginationServerHandle | null = null; - try { - server = await startGeneratedPaginationServer(pg.connectionUri, META_PATH); - await server.applySeed(SEED); - await runScenario(scenario, server); - } finally { - if (server) await server.close(); - await pg.stop(); - } - }, { timeout: 60_000 }); - } -}); - -async function runScenario(scenario: ApiScenario, server: GeneratedPaginationServerHandle): Promise { - for (const req of scenario.requests) { - const init: RequestInit = { method: req.method }; - if (req.body !== undefined) { - init.body = JSON.stringify(req.body); - init.headers = { "content-type": "application/json" }; - } - const res = await fetch(server.baseUrl + req.path, init); - const bodyText = await res.text(); - let body: unknown = null; - if (bodyText.length > 0) { - try { body = parseBodyExact(bodyText); } catch { body = bodyText; } - } - assertResponse(scenario.name, req, res.status, body); - } -} diff --git a/server/typescript/packages/metadata/src/api-pagination.ts b/server/typescript/packages/metadata/src/api-pagination.ts index eeb23d55a5..2e94b6a264 100644 --- a/server/typescript/packages/metadata/src/api-pagination.ts +++ b/server/typescript/packages/metadata/src/api-pagination.ts @@ -27,3 +27,31 @@ export interface ApiPagination { export function apiPaginationOf(node: MetaData): ApiPagination | undefined { return (node.root() as { apiPagination?: ApiPagination | undefined }).apiPagination; } + +// ── api.unknownFields ──────────────────────────────────────────────────────────────────────── +// What a generated create, PUT or PATCH route does with a body key the entity does not declare. + +/** The key of `api.unknownFields` in `.metaobjects/config.json`. */ +export const API_KEY_UNKNOWN_FIELDS = "unknownFields"; +/** Refuse the request: `400 { "error": "unknown_fields", "fields": [...] }`. The default. */ +export const UNKNOWN_FIELDS_REJECT = "reject"; +/** Drop the undeclared keys silently and write the rest (the behaviour before 1.3). */ +export const UNKNOWN_FIELDS_DROP = "drop"; +export const UNKNOWN_FIELDS_MODES = [UNKNOWN_FIELDS_REJECT, UNKNOWN_FIELDS_DROP] as const; +export type UnknownFieldsMode = (typeof UNKNOWN_FIELDS_MODES)[number]; +/** A project that sets nothing refuses undeclared keys. */ +export const DEFAULT_UNKNOWN_FIELDS: UnknownFieldsMode = UNKNOWN_FIELDS_REJECT; +/** The `error` of the 400 a refused body answers with. */ +export const ERROR_UNKNOWN_FIELDS = "unknown_fields"; + +/** + * The unknown-field rule of the tree `node` belongs to: what its root carries, else the + * default (`reject`). A key is unknown when no field of the entity declares it (for a + * single-table hierarchy, no field of the base or any subtype). A declared key the client may + * not set (the generated key, a `readOnly` or derived field, a frozen field on update) is not + * unknown: it is dropped under either rule, so a client that echoes back a row it read keeps + * working. + */ +export function apiUnknownFieldsOf(node: MetaData): UnknownFieldsMode { + return (node.root() as { apiUnknownFields?: UnknownFieldsMode | undefined }).apiUnknownFields ?? DEFAULT_UNKNOWN_FIELDS; +} diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index 413ff50395..b1253eab77 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -407,8 +407,10 @@ export type { EnumDefaults } from "./enum-length.js"; export { apiPaginationOf, API_KEY_PAGINATION, PAGINATION_KEY_DEFAULT_LIMIT, PAGINATION_KEY_MAX_LIMIT, INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT, + apiUnknownFieldsOf, API_KEY_UNKNOWN_FIELDS, UNKNOWN_FIELDS_REJECT, UNKNOWN_FIELDS_DROP, + UNKNOWN_FIELDS_MODES, DEFAULT_UNKNOWN_FIELDS, ERROR_UNKNOWN_FIELDS, } from "./api-pagination.js"; -export type { ApiPagination } from "./api-pagination.js"; +export type { ApiPagination, UnknownFieldsMode } from "./api-pagination.js"; // Wire format (ADR-0019 Amendment 2): the `wire` block of .metaobjects/config.json. export { diff --git a/server/typescript/packages/metadata/src/shared/meta-root.ts b/server/typescript/packages/metadata/src/shared/meta-root.ts index e43d0c1d90..ed7b252a20 100644 --- a/server/typescript/packages/metadata/src/shared/meta-root.ts +++ b/server/typescript/packages/metadata/src/shared/meta-root.ts @@ -10,7 +10,7 @@ import type { MetaField } from "../core/field/meta-field.js"; import { DEFAULT_PHYSICAL_NAMING, type PhysicalNaming } from "../naming.js"; import { DEFAULT_WIRE_FORMAT, type WireFormat } from "../wire.js"; import { DEFAULT_ENUM_DEFAULTS, type EnumDefaults } from "../enum-length.js"; -import type { ApiPagination } from "../api-pagination.js"; +import type { ApiPagination, UnknownFieldsMode } from "../api-pagination.js"; import { PACKAGE_SEPARATOR } from "./structural.js"; import { lookupCanonical } from "../naming-refs.js"; @@ -41,6 +41,13 @@ export class MetaRoot extends MetaData { */ apiPagination: ApiPagination | undefined = undefined; + /** + * What generated write routes do with an undeclared body key (`api.unknownFields` in + * `.metaobjects/config.json`), set by the project-aware load and read through + * `apiUnknownFieldsOf`. Undefined when the project declares none (the default, `reject`). + */ + apiUnknownFields: UnknownFieldsMode | undefined = undefined; + /** Object entities defined at this root level. */ objects(): MetaObject[] { return this.cached("objects", () => diff --git a/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts b/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts index 7e1b4e0b19..14cc548338 100644 --- a/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts +++ b/server/typescript/packages/runtime-ts/src/drizzle-fastify/index.ts @@ -29,12 +29,13 @@ export type { FilterAllowlist, SortAllowlist } from "./filter-allowlist.js"; import { parseFilterParams, parsePageBound, pageLimit, FilterParseError, type PageSize } from "./filter-parser.js"; import { isTruthyFlag, contractErrorCode, coerceIdForColumn, firstRow, extractRowCount } from "./util.js"; import { timestampWire } from "../timestamp-wire.js"; -import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; +import { ERROR_CODE_UNKNOWN_FIELDS, stripDeniedColumns, unknownBodyKeys, withWire, type WireOptions } from "../wire-format.js"; import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { withContractErrorHandler } from "./route-error-handler.js"; import { validationErrorBody } from "../route-errors.js"; export { isTruthyFlag, contractErrorCode, parseId, coerceIdForColumn } from "./util.js"; export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "./list-params.js"; +export { ERROR_CODE_UNKNOWN_FIELDS, unknownBodyKeys } from "../wire-format.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -80,6 +81,14 @@ export interface CrudRoutesOptions extends WireOptions { * schemas stay whole for the application's own writes. */ writeDeniedColumns?: readonly string[]; + /** + * Every key a create or update body may carry: the entity's declared fields (for a + * single-table hierarchy, those of the base and every subtype). When set, a body with any + * other key answers `400 { "error": "unknown_fields", "fields": [...] }` before validation + * (`api.unknownFields: reject`, the default the generated routes emit). Absent, undeclared + * keys are dropped silently (`drop`). A known key the client may not set is dropped either way. + */ + knownFields?: readonly string[]; /** Limit which verbs are mounted. Defaults to all five. */ expose?: readonly CrudVerb[]; /** @@ -289,6 +298,8 @@ export function mountGetRoute(opts: VerbOptions): void { export function mountCreateRoute(opts: VerbOptions): void { const toWire = echoWire(opts); opts.fastify.post(opts.path, routeOpts(opts), async (req, reply) => { + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.insertSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); @@ -330,6 +341,8 @@ export function mountUpdateRoute(opts: VerbOptions): void { reply: { code: (n: number) => { send: (b: unknown) => unknown } }, ) => { const { id } = req.params as { id: string }; + const unknown = unknownBodyKeys(req.body, opts.knownFields); + if (unknown.length > 0) return reply.code(400).send({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }); const parsed = opts.updateSchema.safeParse(stripDeniedColumns(req.body, opts.writeDeniedColumns)); if (!parsed.success) { return reply.code(400).send(validationErrorBody(parsed.error.issues)); diff --git a/server/typescript/packages/runtime-ts/src/hono/index.ts b/server/typescript/packages/runtime-ts/src/hono/index.ts index e23788f59d..d81346a75e 100644 --- a/server/typescript/packages/runtime-ts/src/hono/index.ts +++ b/server/typescript/packages/runtime-ts/src/hono/index.ts @@ -47,7 +47,7 @@ export type { import { isTruthyFlag, coerceIdForColumn, firstRow, extractRowCount } from "../drizzle-fastify/util.js"; import { supportsReturning, type SqlDialect } from "../sql-dialect.js"; import { timestampWire } from "../timestamp-wire.js"; -import { stripDeniedColumns, withWire, type WireOptions } from "../wire-format.js"; +import { ERROR_CODE_UNKNOWN_FIELDS, stripDeniedColumns, unknownBodyKeys, withWire, type WireOptions } from "../wire-format.js"; // Every handler below is wrapped in guardRoute (an unexpected error answers // `500 { error: "internal" }`, logged server-side) and reads its body through // readJsonBody (a malformed body answers `400 { error: "invalid_json" }`). @@ -56,6 +56,7 @@ import { guardRoute, readJsonBody } from "./route-guard.js"; // shared (deprecated) helper so the three adapters can't silently diverge. export { parseId } from "../drizzle-fastify/util.js"; export { pageLimit, ERROR_CODE_LIMIT_TOO_LARGE, type PageSize } from "../drizzle-fastify/list-params.js"; +export { ERROR_CODE_UNKNOWN_FIELDS, unknownBodyKeys } from "../wire-format.js"; export { timestampWire, canonicalTimestamp } from "../timestamp-wire.js"; export { decimalWire, withDecimals, decimalToNative, canonicalDecimal, parseDecimalInput, parseDecimalOperand } from "../decimal-wire.js"; export { withWire, nullMembersWire } from "../wire-format.js"; @@ -108,6 +109,14 @@ export interface CrudRoutesOptions extends WireOptions { * schemas stay whole for the application's own writes. */ writeDeniedColumns?: readonly string[]; + /** + * Every key a create or update body may carry: the entity's declared fields (for a + * single-table hierarchy, those of the base and every subtype). When set, a body with any + * other key answers `400 { "error": "unknown_fields", "fields": [...] }` before validation + * (`api.unknownFields: reject`, the default the generated routes emit). Absent, undeclared + * keys are dropped silently (`drop`). A known key the client may not set is dropped either way. + */ + knownFields?: readonly string[]; /** Limit which verbs are mounted. Defaults to all five. */ expose?: readonly CrudVerb[]; /** @@ -281,6 +290,8 @@ export function mountCreateRoute(opts: VerbOptions): void { opts.app.post(opts.path, guardRoute(async (c) => { const read = await readJsonBody(c); if (!read.ok) return read.response; + const unknown = unknownBodyKeys(read.body, opts.knownFields); + if (unknown.length > 0) return c.json({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }, 400); const parsed = opts.insertSchema.safeParse(stripDeniedColumns(read.body, opts.writeDeniedColumns)); if (!parsed.success) { return c.json(validationErrorBody(parsed.error.issues), 400); @@ -316,6 +327,8 @@ export function mountUpdateRoute(opts: VerbOptions): void { const id = c.req.param("id") ?? ""; const read = await readJsonBody(c); if (!read.ok) return read.response; + const unknown = unknownBodyKeys(read.body, opts.knownFields); + if (unknown.length > 0) return c.json({ error: ERROR_CODE_UNKNOWN_FIELDS, fields: unknown }, 400); const parsed = opts.updateSchema.safeParse(stripDeniedColumns(read.body, opts.writeDeniedColumns)); if (!parsed.success) { return c.json(validationErrorBody(parsed.error.issues), 400); diff --git a/server/typescript/packages/runtime-ts/src/wire-format.ts b/server/typescript/packages/runtime-ts/src/wire-format.ts index b21817d409..ccb26e442e 100644 --- a/server/typescript/packages/runtime-ts/src/wire-format.ts +++ b/server/typescript/packages/runtime-ts/src/wire-format.ts @@ -69,6 +69,20 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un return omitKeys(body, denied); } +/** The `error` of the 400 a write answers when its body carries a key the entity does not declare. */ +export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; + +/** + * The keys of a request body that are not in `known`, sorted, or none when `known` is not + * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top + * level is checked; a body that is not a JSON object is left to validation. + */ +export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { + if (known === undefined || !isPlainObject(body)) return []; + const declared = new Set(known); + return Object.keys(body).filter((k) => !declared.has(k)).sort(); +} + function isPlainObject(v: unknown): v is Record { return v !== null && typeof v === "object" && !Array.isArray(v); } diff --git a/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts b/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts new file mode 100644 index 0000000000..62266fffea --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts @@ -0,0 +1,138 @@ +/** + * `api.unknownFields` at the route. A mount given `knownFields` (what the generated routes pass + * under `reject`, the default) refuses a create, PUT or PATCH body carrying any other key with + * `400 { "error": "unknown_fields", "fields": [sorted keys] }` and writes nothing; a declared key + * the client may not set is still dropped, not refused. A mount given no `knownFields` (`drop`) + * ignores undeclared keys and writes the rest, as before. + * + * Oracle: the contract in docs/features/api-contract.md, "Unknown request fields". + */ +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import Fastify, { type FastifyInstance } from "fastify"; +import { Hono } from "hono"; +import { createClient, type Client } from "@libsql/client"; +import { drizzle } from "drizzle-orm/libsql"; +import { sqliteTable, integer, text } from "drizzle-orm/sqlite-core"; +import { z } from "zod"; +import { mountCrudRoutes as mountDrizzleFastifyCrud, unknownBodyKeys } from "../src/drizzle-fastify/index.js"; +import { mountCrudRoutes as mountHonoCrud } from "../src/hono/index.js"; + +// `id` is declared but server-generated: known, and never settable. +const KNOWN = ["id", "title"] as const; +const InsertSchema = z.object({ title: z.string() }); +const UpdateSchema = InsertSchema.partial(); + +function itemsTable() { + return sqliteTable("items", { id: integer("id").primaryKey(), title: text("title").notNull() }); +} + +async function seeded(): Promise { + const client = createClient({ url: ":memory:" }); + await client.execute(`CREATE TABLE items (id INTEGER PRIMARY KEY, title TEXT NOT NULL)`); + await client.execute(`INSERT INTO items (id, title) VALUES (1, 'first')`); + return client; +} + +describe("unknownBodyKeys", () => { + test("no known list: nothing is unknown", () => { + expect(unknownBodyKeys({ a: 1, isAdmin: true }, undefined)).toEqual([]); + }); + test("keys outside the known list, sorted", () => { + expect(unknownBodyKeys({ title: "x", zeta: 1, isAdmin: true }, KNOWN)).toEqual(["isAdmin", "zeta"]); + }); + test("a body that is not an object is left to validation", () => { + expect(unknownBodyKeys([1, 2], KNOWN)).toEqual([]); + expect(unknownBodyKeys("text", KNOWN)).toEqual([]); + expect(unknownBodyKeys(null, KNOWN)).toEqual([]); + }); +}); + +type Send = (method: string, url: string, body?: unknown) => Promise<{ status: number; body: unknown }>; +type Harness = { send: Send; count: () => Promise; close: () => Promise }; + +function writeBehaviour(name: string, setup: (known: readonly string[] | undefined) => Promise) { + describe(`${name} — reject (knownFields given)`, () => { + let h: Harness; + beforeEach(async () => { h = await setup(KNOWN); }); + afterEach(async () => { await h.close(); }); + + test("a create with an undeclared key is 400 unknown_fields and writes nothing", async () => { + const r = await h.send("POST", "/items", { title: "new", isAdmin: true, role: "x" }); + expect(r.status).toBe(400); + expect(r.body).toEqual({ error: "unknown_fields", fields: ["isAdmin", "role"] }); + expect(await h.count()).toBe(1); + }); + test("PATCH and PUT with an undeclared key are refused and change nothing", async () => { + for (const method of ["PATCH", "PUT"]) { + const r = await h.send(method, "/items/1", { title: "changed", isAdmin: true }); + expect(r.status).toBe(400); + expect(r.body).toEqual({ error: "unknown_fields", fields: ["isAdmin"] }); + } + const row = await h.send("GET", "/items/1"); + expect((row.body as { title: string }).title).toBe("first"); + }); + test("a declared but server-owned key is dropped, not refused", async () => { + const r = await h.send("POST", "/items", { id: 99, title: "echoed" }); + expect(r.status).toBe(201); + expect((r.body as { id: number }).id).not.toBe(99); + }); + }); + + describe(`${name} — drop (no knownFields)`, () => { + let h: Harness; + beforeEach(async () => { h = await setup(undefined); }); + afterEach(async () => { await h.close(); }); + + test("an undeclared key is ignored and the rest is written", async () => { + const r = await h.send("POST", "/items", { title: "kept", isAdmin: true }); + expect(r.status).toBe(201); + expect(r.body).not.toHaveProperty("isAdmin"); + expect(await h.count()).toBe(2); + }); + }); +} + +async function countRows(client: Client): Promise { + const rs = await client.execute("SELECT COUNT(*) AS n FROM items"); + return Number(rs.rows[0]!["n"]); +} + +writeBehaviour("drizzle-fastify", async (known) => { + const client = await seeded(); + const app: FastifyInstance = Fastify(); + mountDrizzleFastifyCrud({ + fastify: app, path: "/items", db: drizzle(client), table: itemsTable(), + insertSchema: InsertSchema, updateSchema: UpdateSchema, + ...(known !== undefined ? { knownFields: known } : {}), + }); + await app.ready(); + return { + send: async (method, url, body) => { + const r = await app.inject({ method: method as "GET", url, ...(body !== undefined ? { payload: body as object } : {}) }); + return { status: r.statusCode, body: JSON.parse(r.body) }; + }, + count: () => countRows(client), + close: async () => { await app.close(); client.close(); }, + }; +}); + +writeBehaviour("hono", async (known) => { + const client = await seeded(); + const app = new Hono(); + mountHonoCrud({ + app, path: "/items", db: drizzle(client), table: itemsTable(), + insertSchema: InsertSchema, updateSchema: UpdateSchema, + ...(known !== undefined ? { knownFields: known } : {}), + }); + return { + send: async (method, url, body) => { + const r = await app.request(url, { + method, + ...(body !== undefined ? { body: JSON.stringify(body), headers: { "content-type": "application/json" } } : {}), + }); + return { status: r.status, body: await r.json() }; + }, + count: () => countRows(client), + close: async () => { client.close(); }, + }; +}); diff --git a/server/typescript/packages/sdk/src/collection.ts b/server/typescript/packages/sdk/src/collection.ts index 2bab06a4e3..b4403a4755 100644 --- a/server/typescript/packages/sdk/src/collection.ts +++ b/server/typescript/packages/sdk/src/collection.ts @@ -24,9 +24,10 @@ import { DEFAULT_WIRE_FORMAT, type EnumDefaults, type ApiPagination, + type UnknownFieldsMode, DEFAULT_ENUM_DEFAULTS, } from "@metaobjectsdev/metadata"; -import { apiPaginationFromConfig, CONFIG_FILE, enumDefaultsFromConfig, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; +import { apiPaginationFromConfig, apiUnknownFieldsFromConfig, CONFIG_FILE, enumDefaultsFromConfig, loadConfig, physicalNamingFromConfig, wireFormatFromConfig, type Config } from "./config.js"; import { discoverCollectionRoot, exists, isDir } from "./discovery.js"; import { compileScope, matchesScope, type Scope } from "./scope.js"; import { DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, isMetadataFile } from "./metadata-files.js"; @@ -94,6 +95,8 @@ export interface Collection { /** The `api.pagination` block of the project config; undefined when the project declares none. * Optional so a hand-built Collection from before the block existed still type-checks. */ readonly apiPagination?: ApiPagination | undefined; + /** `api.unknownFields`, undefined when the config declares none (the default, `reject`). */ + readonly apiUnknownFields?: UnknownFieldsMode | undefined; /** `migrate.conventions` of the project config — a schema-conventions preset name — when set. */ readonly schemaConventions: string | undefined; /** `migrate.checks.skipRedundant` of the project config: leave out the validator CHECKs a @@ -296,6 +299,7 @@ export async function resolveCollection( let wireFormat: WireFormat = DEFAULT_WIRE_FORMAT; let enumDefaults: EnumDefaults = DEFAULT_ENUM_DEFAULTS; let apiPagination: ApiPagination | undefined; + let apiUnknownFields: UnknownFieldsMode | undefined; let schemaConventions: string | undefined; let skipRedundantChecks = false; @@ -317,6 +321,7 @@ export async function resolveCollection( wireFormat = wireFormatFromConfig(cfg); enumDefaults = enumDefaultsFromConfig(cfg); apiPagination = apiPaginationFromConfig(cfg); + apiUnknownFields = apiUnknownFieldsFromConfig(cfg); schemaConventions = cfg.migrate?.conventions; skipRedundantChecks = cfg.migrate?.checks?.skipRedundant === true; // An unknown token is a hard config error naming the valid ones, while @@ -435,6 +440,7 @@ export async function resolveCollection( wireFormat, enumDefaults, apiPagination, + apiUnknownFields, schemaConventions, skipRedundantChecks, fileIds: new Map(dependencies.map((d) => [d.artifactPath, d.sourceId])), diff --git a/server/typescript/packages/sdk/src/config.ts b/server/typescript/packages/sdk/src/config.ts index ccab17789f..486335b191 100644 --- a/server/typescript/packages/sdk/src/config.ts +++ b/server/typescript/packages/sdk/src/config.ts @@ -26,7 +26,10 @@ import { API_KEY_PAGINATION, PAGINATION_KEY_DEFAULT_LIMIT, PAGINATION_KEY_MAX_LIMIT, + API_KEY_UNKNOWN_FIELDS, + UNKNOWN_FIELDS_MODES, type ApiPagination, + type UnknownFieldsMode, } from "@metaobjectsdev/metadata"; const DialectEnum = z.enum(["sqlite", "postgres", "d1"]); @@ -300,6 +303,11 @@ export const ConfigSchema = z.object({ * with no `limit` returns; an explicit `limit` above `pagination.maxLimit` is refused with * `400 { "error": "limit_too_large", "maxLimit": N }`. Absent means no default and no ceiling, * as before. `meta init` writes 50 and 1000. + * + * `unknownFields` is what a generated create, PUT or PATCH route does with a body key the + * entity does not declare: `reject` (the default, also when absent) answers + * `400 { "error": "unknown_fields", "fields": [...] }`; `drop` ignores the key and writes the + * rest. A declared key the client may not set is dropped under either value. */ api: z .strictObject({ @@ -318,9 +326,14 @@ export const ConfigSchema = z.object({ message: `"api.pagination.defaultLimit" must not exceed "api.pagination.maxLimit"`, }) .optional(), + unknownFields: z + .enum(UNKNOWN_FIELDS_MODES, { + error: `"api.${API_KEY_UNKNOWN_FIELDS}" must be ${UNKNOWN_FIELDS_MODES.map((m) => `"${m}"`).join(" or ")}`, + }) + .optional(), }, { error: (iss) => iss.code === "unrecognized_keys" - ? `"api" has an unknown key "${iss.keys[0]}" (expected "${API_KEY_PAGINATION}")` + ? `"api" has an unknown key "${iss.keys[0]}" (expected "${API_KEY_PAGINATION}" or "${API_KEY_UNKNOWN_FIELDS}")` : iss.code === "invalid_type" ? `"api" must be an object` : undefined, }) .optional(), @@ -374,6 +387,11 @@ export function apiPaginationFromConfig(cfg: Pick): ApiPagination return p === undefined ? undefined : { defaultLimit: p.defaultLimit, maxLimit: p.maxLimit }; } +/** The unknown-field rule a config declares (`api.unknownFields`); undefined when absent. */ +export function apiUnknownFieldsFromConfig(cfg: Pick): UnknownFieldsMode | undefined { + return cfg.api?.unknownFields; +} + /** The enum storage defaults a config declares (`enums`); none when absent. */ export function enumDefaultsFromConfig(cfg: Pick): EnumDefaults { return cfg.enums?.maxLength === undefined ? DEFAULT_ENUM_DEFAULTS : { maxLength: cfg.enums.maxLength }; diff --git a/server/typescript/packages/sdk/src/index.ts b/server/typescript/packages/sdk/src/index.ts index 54c173ea8c..102360da66 100644 --- a/server/typescript/packages/sdk/src/index.ts +++ b/server/typescript/packages/sdk/src/index.ts @@ -16,7 +16,7 @@ // shows `resolveCollection`, the documented single authority for locating a project. // Config -export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig, enumDefaultsFromConfig, apiPaginationFromConfig } from "./config.js"; +export { ConfigSchema, DEFAULT_CONFIG, loadConfig, setConfigWarningSink, splitUnknownTopLevelKeys, CONFIG_TOP_LEVEL_KEYS, saveConfig, AllowTokenEnum, physicalNamingFromConfig, wireFormatFromConfig, enumDefaultsFromConfig, apiPaginationFromConfig, apiUnknownFieldsFromConfig } from "./config.js"; export type { Config } from "./config.js"; // Metadata dependencies (FR-023) — the `dependencies` config key's schema, diff --git a/server/typescript/packages/sdk/src/memory.ts b/server/typescript/packages/sdk/src/memory.ts index d9275c4b32..541b2acf65 100644 --- a/server/typescript/packages/sdk/src/memory.ts +++ b/server/typescript/packages/sdk/src/memory.ts @@ -18,6 +18,7 @@ import { type WireFormat, type EnumDefaults, type ApiPagination, + type UnknownFieldsMode, } from "@metaobjectsdev/metadata"; import { FileSource } from "@metaobjectsdev/metadata/core"; // FR-043 — a node-only subpath (it reaches the filesystem), which is why it is imported @@ -121,6 +122,8 @@ export interface LoadMemoryOptions { enumDefaults?: EnumDefaults; /** The project's list page size (`api.pagination`); defaults to the collection's when `files` is not passed. */ apiPagination?: ApiPagination | undefined; + /** `api.unknownFields`; on the no-`files` arm the collection supplies it. */ + apiUnknownFields?: UnknownFieldsMode | undefined; } /** Default provider bundle threaded by {@link loadMemory} when no options @@ -176,7 +179,7 @@ export async function loadMemory( repoRoot: string, options?: LoadMemoryOptions, ): Promise { - const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination } = + const { result, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination, apiUnknownFields } = await loadCollectionRaw(repoRoot, options, options?.strict === true); if (result.errors.length > 0) { @@ -199,6 +202,7 @@ export async function loadMemory( if (wireFormat !== undefined) result.root.wireFormat = wireFormat; if (enumDefaults !== undefined) result.root.enumDefaults = enumDefaults; if (apiPagination !== undefined) result.root.apiPagination = apiPagination; + if (apiUnknownFields !== undefined) result.root.apiUnknownFields = apiUnknownFields; return result.root; } @@ -275,6 +279,7 @@ async function loadCollectionRaw( wireFormat: WireFormat | undefined; enumDefaults: EnumDefaults | undefined; apiPagination: ApiPagination | undefined; + apiUnknownFields: UnknownFieldsMode | undefined; }> { const extra = options?.providers ?? []; let providers: readonly MetaDataTypeProvider[]; @@ -305,6 +310,7 @@ async function loadCollectionRaw( let wireFormat: WireFormat | undefined = options?.wireFormat; let enumDefaults: EnumDefaults | undefined = options?.enumDefaults; let apiPagination: ApiPagination | undefined = options?.apiPagination; + let apiUnknownFields: UnknownFieldsMode | undefined = options?.apiUnknownFields; if (options?.files !== undefined) { paths = [...options.files]; fileIds = options.fileIds; @@ -327,6 +333,7 @@ async function loadCollectionRaw( wireFormat ??= collection.wireFormat; enumDefaults ??= collection.enumDefaults; apiPagination ??= collection.apiPagination; + apiUnknownFields ??= collection.apiUnknownFields; } const loader = new MetaDataLoader({ @@ -352,7 +359,7 @@ async function loadCollectionRaw( return id === undefined ? new FileSource(p) : new FileSource(p, { id }); }), ]); - return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination }; + return { result, registry, importedPackages, importedNodes, libraries, physicalNaming, wireFormat, enumDefaults, apiPagination, apiUnknownFields }; } /** Every source file that contributed to a node, across the envelope variants that diff --git a/server/typescript/packages/sdk/test/config.test.ts b/server/typescript/packages/sdk/test/config.test.ts index fd9e7e9dda..07731524b3 100644 --- a/server/typescript/packages/sdk/test/config.test.ts +++ b/server/typescript/packages/sdk/test/config.test.ts @@ -2,7 +2,7 @@ import { describe, test, expect, beforeEach, afterEach } from "bun:test"; import { mkdtempSync, rmSync, mkdirSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { ConfigSchema, DEFAULT_CONFIG, apiPaginationFromConfig, loadConfig, saveConfig, setConfigWarningSink, type Config } from "../src/config.js"; +import { ConfigSchema, DEFAULT_CONFIG, apiPaginationFromConfig, apiUnknownFieldsFromConfig, loadConfig, saveConfig, setConfigWarningSink, type Config } from "../src/config.js"; let metaRoot: string; beforeEach(() => { @@ -336,3 +336,28 @@ describe("api.pagination (the generated list routes' page size)", () => { expect(messages({ paging: {} }).join(" ")).toContain('"api" has an unknown key "paging"'); }); }); + +describe("api.unknownFields (what a generated write does with an undeclared key)", () => { + const messages = (api: unknown) => { + const r = ConfigSchema.safeParse({ schema_version: 1, api }); + return r.success ? [] : r.error.issues.map((i) => i.message); + }; + + test("absent: undefined here, which the routes read as the default, reject", () => { + expect(apiUnknownFieldsFromConfig(DEFAULT_CONFIG)).toBeUndefined(); + }); + + test("reject and drop are read as given", () => { + for (const mode of ["reject", "drop"] as const) { + expect(apiUnknownFieldsFromConfig(ConfigSchema.parse({ schema_version: 1, api: { unknownFields: mode } }))).toBe(mode); + } + }); + + test("any other value is refused, naming the two it may be", () => { + expect(messages({ unknownFields: "ignore" })).toEqual([`"api.unknownFields" must be "reject" or "drop"`]); + }); + + test("an unknown api key names both keys the block takes", () => { + expect(messages({ unknown: "drop" }).join(" ")).toContain(`(expected "pagination" or "unknownFields")`); + }); +}); diff --git a/server/typescript/packages/test-generators/src/generators/routes.ts b/server/typescript/packages/test-generators/src/generators/routes.ts index 8ffe8c2da9..59c800867a 100644 --- a/server/typescript/packages/test-generators/src/generators/routes.ts +++ b/server/typescript/packages/test-generators/src/generators/routes.ts @@ -71,6 +71,7 @@ import { decimalColumnsLine, paginationLine, writeDeniedColumnsLine, + knownFieldsLine, servesReadApi, formatTs, entityOutputPath, @@ -312,7 +313,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLinePrefixed} }); ${m2mMountsPrefixed} }, { prefix: ${JSON.stringify(ctx.apiPrefix)} }); } @@ -337,7 +338,7 @@ export async function ${handlerName}(fastify: ${FastifyInstanceSym}) { updateSchema: ${entityName}UpdateSchema, filterAllowlist: ${entityName}FilterAllowlist, sortAllowlist: ${entityName}SortAllowlist, - dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} + dialect: ${JSON.stringify(ctx.dialect)},${decimalColumnsLine(entity, " ", ctx.loadedRoot)}${writeDeniedColumnsLine(entity, " ")}${knownFieldsLine(entity, " ", ctx.loadedRoot)}${paginationLine(entity, " ", ctx.loadedRoot)}${exposeLineFlat} }); ${m2mMountsFlat}} `; @@ -651,7 +652,7 @@ function renderTphRoutes( updateSchema: ${subUpdateSym}, filterAllowlist: ${subFilterSym}, sortAllowlist: ${subSortSym}, - dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} + dialect: ${dialectLit},${decimalColumnsLine(sub, " ", ctx.loadedRoot, hierarchy)}${writeDeniedColumnsLine(sub, " ", hierarchy)}${knownFieldsLine(sub, " ", ctx.loadedRoot, hierarchy)}${paginationLine(sub, " ", ctx.loadedRoot)} discriminator: { column: ${JSON.stringify(discField)}, value: ${JSON.stringify(value)} },${exposeLine(expose, " ")} });`; // This subtype's own M:N navigations, mounted beneath its segment and gated on the From e0b45d0d111b99c733c5d7e515507088579e432a Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 15:00:07 -0500 Subject: [PATCH 06/31] chore(showcase): regenerate for api.unknownFields --- .../.metaobjects/.gen-state/.hashes.json | 6 ++--- .../generated/csharp/SubscriberRoutes.g.cs | 12 ++++++++++ .../generated/python/subscriber_router.py | 22 +++++++++++++++++++ .../generated/ts/Subscriber.routes.ts | 1 + 4 files changed, 38 insertions(+), 3 deletions(-) diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index 14608b3548..d030b08726 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -4,20 +4,20 @@ "generated/csharp/SubscriberBlurbPayload.g.cs": "29eb3a6ce64496ffe734cba55d0213994316b4281d3fa6791917393bae36c9ed", "generated/csharp/SubscriberFilterAllowlist.g.cs": "3839b04884e4e393d4f7d7bf4c7d4fe5b3852bd3a92b1ecb282cb8935a3c98e5", "generated/csharp/SubscriberNames.g.cs": "c2cf0248c2ecde155ec2623604771e39bed23783603d6f18a2ed82967227a732", - "generated/csharp/SubscriberRoutes.g.cs": "85969bbaf6a58ca2946c595d520e7ec48dcc5b5e21123e0340c5cc08aa8b9e04", + "generated/csharp/SubscriberRoutes.g.cs": "152ee0d118c905bbece66bdd8153e12cc28f4c431f9d5a06f0fbb341f9337671", "generated/python/Subscriber.py": "9dd4c0acea47f99d1daf79930f9de2e400f30780ebce6c63f023be528a72b762", "generated/python/SubscriberBlurbPayload.py": "a5ef8a08e63efbe0f73e39a21a368cff5150c68cc8beeea6e8aab0c755f8bbdd", "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", "generated/python/subscriber_filter_allowlist.py": "b0414cc95ff3982c80a0e1a06839f090c0b423e41f5a94275df93426cea09df5", "generated/python/subscriber_names.py": "269637a02bf9b46f55658b58cb42f83196c03a877e1e2101337847fd406c1561", - "generated/python/subscriber_router.py": "9bb26702fc7ad041cf4bb52c5e588226a315a2b0a858555666060863795dcfc8", + "generated/python/subscriber_router.py": "16502ff815bf9ae29c7dba4700d0465d06058a9fd7ea8db5f98334d52d284851", "generated/ts/Subscriber.queries.ts": { "hash": "9f1a459c57fafc98bf074b4443b506d4a1e7023ed188f293021ae89c298d4562", "generatedBy": "queries-file", "sourceNode": "acme::Subscriber" }, "generated/ts/Subscriber.routes.ts": { - "hash": "147d32ae658f985e024c3de6e8894ca7359f767eeaa765cec70756802e3bd300", + "hash": "f76f592c98bd0ac5c07c4fe9c26cd88659793141e6620310dc725c7edcf68099", "generatedBy": "routes-file", "sourceNode": "acme::Subscriber" }, diff --git a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs index efdd5400ca..163208f64a 100644 --- a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs +++ b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs @@ -24,6 +24,16 @@ public static class SubscriberRoutes { }; + // api.unknownFields: reject — the declared field names (wire spelling); any other body key is a 400. + private static readonly System.Collections.Generic.HashSet KnownFields = new(System.StringComparer.Ordinal) + { + "createdAt", + "email", + "id", + "name", + "status", + }; + /// Maps Subscriber REST routes. Unauthenticated — see remarks. /// /// These endpoints are unauthenticated. Guard them by mapping on a group that requires @@ -91,6 +101,7 @@ await db.Subscribers.FindAsync(id) is { } found using var __bodyScope = __body; if (__body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable(); + if (UnknownFieldCheck.Reject(__body.RootElement, KnownFields) is { } __unknown) return __unknown; var __issues = new ValidationIssues(); var __present = new System.Collections.Generic.HashSet(System.StringComparer.OrdinalIgnoreCase); foreach (var __p in __body.RootElement.EnumerateObject()) @@ -128,6 +139,7 @@ async System.Threading.Tasks.Task UpdateSubscriber(long id, HttpContext catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); } using var bodyScope = body; if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable(); + if (UnknownFieldCheck.Reject(body.RootElement, KnownFields) is { } __unknown) return __unknown; // Deserialize each value with the app's CONFIGURED JSON options, not defaults — // a field.timestamp binds to DateTimeOffset via a custom converter registered on // ConfigureHttpJsonOptions, and default options cannot read the wire format, throwing 500. diff --git a/examples/showcase/generated/python/subscriber_router.py b/examples/showcase/generated/python/subscriber_router.py index 0823ed7798..19c60e83bc 100644 --- a/examples/showcase/generated/python/subscriber_router.py +++ b/examples/showcase/generated/python/subscriber_router.py @@ -23,6 +23,7 @@ classify_constraint_error, pydantic_issues, required_null_issues, + unknown_body_keys, unreadable_body_issues, validation_envelope, value_error_issues, @@ -120,6 +121,17 @@ def _validated_value(valid: Any, raw: Any) -> Any: _FROZEN_FIELDS: frozenset[str] = frozenset() +_KNOWN_FIELDS: frozenset[str] = frozenset( + { + "id", + "email", + "name", + "status", + "createdAt", + } +) + + def _parse_sort(raw: str) -> _SortClause | None: """Parse `field:asc|desc`; return None for malformed / disallowed input.""" parts = raw.split(":", 1) @@ -217,6 +229,11 @@ def create_subscriber( return JSONResponse( status_code=400, content=validation_envelope(unreadable_body_issues()) ) + _unknown = unknown_body_keys(dto, _KNOWN_FIELDS) + if _unknown: + return JSONResponse( + status_code=400, content={"error": "unknown_fields", "fields": _unknown} + ) try: dto = coerce_decimal_fields( dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS @@ -262,6 +279,11 @@ def update_subscriber( return JSONResponse( status_code=400, content=validation_envelope(unreadable_body_issues()) ) + _unknown = unknown_body_keys(dto, _KNOWN_FIELDS) + if _unknown: + return JSONResponse( + status_code=400, content={"error": "unknown_fields", "fields": _unknown} + ) try: dto = coerce_decimal_fields( dto, _DECIMAL_FIELDS, _DECIMAL_LIMITS, _NESTED_DECIMALS diff --git a/examples/showcase/generated/ts/Subscriber.routes.ts b/examples/showcase/generated/ts/Subscriber.routes.ts index df1afd802d..efd2cb12ee 100644 --- a/examples/showcase/generated/ts/Subscriber.routes.ts +++ b/examples/showcase/generated/ts/Subscriber.routes.ts @@ -47,6 +47,7 @@ export async function subscriberRoutes(fastify: FastifyInstance) { filterAllowlist: SubscriberFilterAllowlist, sortAllowlist: SubscriberSortAllowlist, dialect: "sqlite", + knownFields: ["id", "email", "name", "status", "createdAt"], }); }, { prefix: "/api" }, From 3f333753bcb151cd6d0b3fefaee3d53060fca54a Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 15:25:06 -0500 Subject: [PATCH 07/31] fix(api): code-point order for unknown_fields, every distinct validation issue, a negative limit refused under api.pagination - unknown_fields lists keys in code-point order in every port (TypeScript and C# sorted by UTF-16 code unit, which put a key outside the BMP first). - A validation 400 keeps two distinct failures of one rule kind on one field (Python and C# de-duplicated on path and code); a type error still lists once. - Under api.pagination a negative limit answers 400 pagination.invalid_value in every port instead of reaching the store (SQLite reads LIMIT -1 as no limit). New api-settings/limit-negative scenario. - Python validated-input tests follow api.unknownFields; advanced-modeling example regenerated; C# routes generator header names the current contract. --- CHANGELOG.md | 4 +- docs/CONFORMANCE.md | 13 ++-- docs/features/api-contract.md | 4 +- .../.metaobjects/.gen-state/.hashes.json | 8 +-- .../codegen/runtime/wire-format.ts | 26 ++++++-- .../src/generated/Author.routes.ts | 1 + .../src/generated/Lesson.routes.ts | 1 + .../src/generated/Program.routes.ts | 11 ++++ .../src/generated/Purchase.routes.ts | 7 ++ .../showcase/codegen/runtime/wire-format.ts | 26 ++++++-- fixtures/api-contract-conformance/README.md | 4 +- .../api-settings/README.md | 4 ++ .../scenarios/limit-negative.yaml | 17 +++++ .../ApiSettingsTests.cs | 28 ++++++++ .../ValidationIssuesTests.cs | 14 +++- .../Generators/RoutesGenerator.cs | 13 +++- .../Runtime/UnknownFieldCheck.cs | 22 ++++++- .../Runtime/ValidationIssues.cs | 6 +- .../kotlin/KotlinSpringControllerGenerator.kt | 7 +- .../spring/SpringControllerGenerator.java | 7 ++ .../codegen/generators/router_generator.py | 15 ++++- .../codegen/runtime/constraint_errors.py | 13 ++-- .../test_generated_router_validated_input.py | 65 ++++++++++++++----- .../tests/runtime/test_validation_issues.py | 11 +++- .../packages/runtime-ts/src/wire-format.ts | 26 ++++++-- .../runtime-ts/test/unknown-fields.test.ts | 4 ++ 26 files changed, 299 insertions(+), 58 deletions(-) create mode 100644 fixtures/api-contract-conformance/api-settings/scenarios/limit-negative.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f4ac924cd..67863ff782 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,7 +59,9 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi `"api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 } }` in `.metaobjects/config.json` and a generated entity or projection list with no `limit` answers the first `defaultLimit` rows, while an explicit `limit` above `maxLimit` answers - `400 {"error": "limit_too_large", "maxLimit": N}` (refused, not clamped). `meta init` writes + `400 {"error": "limit_too_large", "maxLimit": N}` (refused, not clamped); a negative `limit` + answers `400 pagination.invalid_value` in every port rather than reaching a store that reads + `LIMIT -1` as no limit. `meta init` writes that block for a new project. **A project without the setting behaves exactly as before**, because a cap added on upgrade would silently truncate a client that reads a whole list; each port's verify advises (rule `unbounded-list-pages`, never a failure) when a routes diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 1f4c3b14c0..e7d52176ad 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 62 (55 query + 7 migration) | all 62; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 55 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 135 (34 core + 13 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 6 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 136 (34 core + 13 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 7 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | @@ -442,9 +442,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (135) +### `fixtures/api-contract-conformance/` (136) -All 135 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 136 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -468,8 +468,9 @@ identity has no item route; a projection keyed on a field not named `id` is addr it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one canonical string, read exactly, filtered exactly and refused past its declared digits), `duration/` (2, a `field.duration` under `"wire": { "durations": "iso" }` is one canonical ISO-8601 string, refused -in a calendar or clock spelling, and filtered and sorted as a span on every engine), `api-settings/` (6, -the project's `api` block: `api.pagination`'s default page and `limit_too_large` cap, and +in a calendar or clock spelling, and filtered and sorted as a span on every engine), `api-settings/` (7, +the project's `api` block: `api.pagination`'s default page, `limit_too_large` cap and refused +negative `limit`, and `api.unknownFields: drop`) and `report/` (37, FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps @@ -730,7 +731,7 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 510 + yaml 16 + verify 45 -+ render 18 + persistence 62 + api-contract 135 + source-resolution 25 + scope 10 + ++ render 18 + persistence 62 + api-contract 136 + source-resolution 25 + scope 10 + dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 7ad2ae4a6d..dcfdb765f7 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -284,7 +284,9 @@ each handler they register instead; an `HTTPException` is rethrown to your `defaultLimit` rows, and an explicit `limit` above `maxLimit` answers `400 { "error": "limit_too_large", "maxLimit": N }`. The request is refused, not clamped: a client that asked for more than the server serves is told so rather than handed a short page - it cannot tell from the end of the list. `offset` and `withCount=1` page past the default as + it cannot tell from the end of the list. A negative `limit` answers the malformed-bound + envelope above (`pagination.invalid_value`) in every port, so it never reaches a store that + reads `LIMIT -1` as no limit (SQLite). `offset` and `withCount=1` page past the default as usual, with `total` counting every row. `meta init` writes the block above for a new project; both sizes must be positive integers and `defaultLimit` may not exceed `maxLimit`. A report keeps its own row cap (above) and an opaque view its 1000, whatever the setting says. diff --git a/examples/advanced-modeling/.metaobjects/.gen-state/.hashes.json b/examples/advanced-modeling/.metaobjects/.gen-state/.hashes.json index eb4f424ca5..9380c9de35 100644 --- a/examples/advanced-modeling/.metaobjects/.gen-state/.hashes.json +++ b/examples/advanced-modeling/.metaobjects/.gen-state/.hashes.json @@ -10,7 +10,7 @@ "sourceNode": "acme::learn::Author" }, "src/generated/Author.routes.ts": { - "hash": "3b0412b4501d02ba8716dae634522202714d467c5f9d577d8ea583868b1187d5", + "hash": "38d9a39b2fc93dc63ba0bc11d91d151f2a24b0af808a10dfdddf61a2d58356de", "generatedBy": "routes-file", "sourceNode": "acme::learn::Author" }, @@ -35,7 +35,7 @@ "sourceNode": "acme::learn::Lesson" }, "src/generated/Lesson.routes.ts": { - "hash": "ab910b2a1cfe1d38a49ce5d54de149bdbdfcd5adb077363b913ce6bebd01de13", + "hash": "a706159a09e459f44f1dd0721cf26320862768a4305363d94cf66b5fb8b5ae2a", "generatedBy": "routes-file", "sourceNode": "acme::learn::Lesson" }, @@ -55,7 +55,7 @@ "sourceNode": "acme::learn::Program" }, "src/generated/Program.routes.ts": { - "hash": "ac43f1836303aa693e26f346398f47330d18ac4ce6a60e03394dc40dee398cc7", + "hash": "c0b0b5cf1761371480aa90d613c5170212b4b4ef63fd265f1362ff4051b50af9", "generatedBy": "routes-file", "sourceNode": "acme::learn::Program" }, @@ -96,7 +96,7 @@ "sourceNode": "acme::learn::Purchase" }, "src/generated/Purchase.routes.ts": { - "hash": "b1d1df2277754b63f137d5059603abc812a36795656099e7b9df4cf67928549c", + "hash": "7590c063acdabcd86eec67d3d8abc6390dde576aca733944dbb1edaf1dfde5bb", "generatedBy": "routes-file", "sourceNode": "acme::learn::Purchase" }, diff --git a/examples/advanced-modeling/codegen/runtime/wire-format.ts b/examples/advanced-modeling/codegen/runtime/wire-format.ts index ccb26e442e..b6969300c7 100644 --- a/examples/advanced-modeling/codegen/runtime/wire-format.ts +++ b/examples/advanced-modeling/codegen/runtime/wire-format.ts @@ -73,14 +73,32 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; /** - * The keys of a request body that are not in `known`, sorted, or none when `known` is not - * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top - * level is checked; a body that is not a JSON object is left to validation. + * The keys of a request body that are not in `known`, sorted by code point, or none when + * `known` is not given (the route drops undeclared keys instead: `api.unknownFields: drop`). + * Only the top level is checked; a body that is not a JSON object is left to validation. */ export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { if (known === undefined || !isPlainObject(body)) return []; const declared = new Set(known); - return Object.keys(body).filter((k) => !declared.has(k)).sort(); + return Object.keys(body).filter((k) => !declared.has(k)).sort(compareCodePoints); +} + +/** + * Code-point order, the order every port lists `fields` in. The default `sort()` compares + * UTF-16 code units, which puts a key outside the Basic Multilingual Plane (an emoji) before + * U+E000..U+FFFF; Java, Kotlin, Python and C# answer in code-point order. + */ +function compareCodePoints(a: string, b: string): number { + let i = 0; + let j = 0; + while (i < a.length && j < b.length) { + const ca = a.codePointAt(i)!; + const cb = b.codePointAt(j)!; + if (ca !== cb) return ca - cb; + i += ca > 0xffff ? 2 : 1; + j += cb > 0xffff ? 2 : 1; + } + return (a.length - i) - (b.length - j); } function isPlainObject(v: unknown): v is Record { diff --git a/examples/advanced-modeling/src/generated/Author.routes.ts b/examples/advanced-modeling/src/generated/Author.routes.ts index 62398425d3..e2a9645ce0 100644 --- a/examples/advanced-modeling/src/generated/Author.routes.ts +++ b/examples/advanced-modeling/src/generated/Author.routes.ts @@ -47,6 +47,7 @@ export async function authorRoutes(fastify: FastifyInstance) { filterAllowlist: AuthorFilterAllowlist, sortAllowlist: AuthorSortAllowlist, dialect: "postgres", + knownFields: ["id", "name", "email", "bio"], }); }, { prefix: "/api" }, diff --git a/examples/advanced-modeling/src/generated/Lesson.routes.ts b/examples/advanced-modeling/src/generated/Lesson.routes.ts index ec24d4febc..3faa54390e 100644 --- a/examples/advanced-modeling/src/generated/Lesson.routes.ts +++ b/examples/advanced-modeling/src/generated/Lesson.routes.ts @@ -47,6 +47,7 @@ export async function lessonRoutes(fastify: FastifyInstance) { filterAllowlist: LessonFilterAllowlist, sortAllowlist: LessonSortAllowlist, dialect: "postgres", + knownFields: ["id", "programId", "title", "position"], }); }, { prefix: "/api" }, diff --git a/examples/advanced-modeling/src/generated/Program.routes.ts b/examples/advanced-modeling/src/generated/Program.routes.ts index 4fdb2f0a4c..4f9df9c3cd 100644 --- a/examples/advanced-modeling/src/generated/Program.routes.ts +++ b/examples/advanced-modeling/src/generated/Program.routes.ts @@ -47,6 +47,17 @@ export async function programRoutes(fastify: FastifyInstance) { filterAllowlist: ProgramFilterAllowlist, sortAllowlist: ProgramSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "title", + "status", + "summary", + "priceCents", + "coverKey", + "authorId", + "syllabus", + "instructorProfile", + ], }); }, { prefix: "/api" }, diff --git a/examples/advanced-modeling/src/generated/Purchase.routes.ts b/examples/advanced-modeling/src/generated/Purchase.routes.ts index 457f5f5e05..767fac818a 100644 --- a/examples/advanced-modeling/src/generated/Purchase.routes.ts +++ b/examples/advanced-modeling/src/generated/Purchase.routes.ts @@ -47,6 +47,13 @@ export async function purchaseRoutes(fastify: FastifyInstance) { filterAllowlist: PurchaseFilterAllowlist, sortAllowlist: PurchaseSortAllowlist, dialect: "postgres", + knownFields: [ + "id", + "programId", + "amountCents", + "status", + "purchasedAt", + ], }); }, { prefix: "/api" }, diff --git a/examples/showcase/codegen/runtime/wire-format.ts b/examples/showcase/codegen/runtime/wire-format.ts index ccb26e442e..b6969300c7 100644 --- a/examples/showcase/codegen/runtime/wire-format.ts +++ b/examples/showcase/codegen/runtime/wire-format.ts @@ -73,14 +73,32 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; /** - * The keys of a request body that are not in `known`, sorted, or none when `known` is not - * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top - * level is checked; a body that is not a JSON object is left to validation. + * The keys of a request body that are not in `known`, sorted by code point, or none when + * `known` is not given (the route drops undeclared keys instead: `api.unknownFields: drop`). + * Only the top level is checked; a body that is not a JSON object is left to validation. */ export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { if (known === undefined || !isPlainObject(body)) return []; const declared = new Set(known); - return Object.keys(body).filter((k) => !declared.has(k)).sort(); + return Object.keys(body).filter((k) => !declared.has(k)).sort(compareCodePoints); +} + +/** + * Code-point order, the order every port lists `fields` in. The default `sort()` compares + * UTF-16 code units, which puts a key outside the Basic Multilingual Plane (an emoji) before + * U+E000..U+FFFF; Java, Kotlin, Python and C# answer in code-point order. + */ +function compareCodePoints(a: string, b: string): number { + let i = 0; + let j = 0; + while (i < a.length && j < b.length) { + const ca = a.codePointAt(i)!; + const cb = b.codePointAt(j)!; + if (ca !== cb) return ca - cb; + i += ca > 0xffff ? 2 : 1; + j += cb > 0xffff ? 2 : 1; + } + return (a.length - i) - (b.length - j); } function isPlainObject(v: unknown): v is Record { diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 1dc8012fe2..411a351392 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -56,7 +56,7 @@ fixtures/api-contract-conformance/ ``` Beside the 34 core scenarios above sit eleven **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (101 scenarios in all, 135 with the +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (102 scenarios in all, 136 with the core): | Directory | Scenarios | Gates | Lanes | @@ -70,7 +70,7 @@ core): | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | | `decimal/` | 11 | `field.decimal` is an exact canonical string on the wire (ADR-0019 Amendment 1): read, filter, sort, write and a report over decimals. A scenario's `engines:` key is exact; the `-15` twins run on SQLite and D1 as well. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | -| `api-settings/` | 6 | the `api` block in the sub-corpus's own `.metaobjects/config.json`: `api.pagination` (default 2, max 4) — a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served and one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` — and `api.unknownFields: drop`, under which an undeclared body key is ignored rather than refused | generated only | +| `api-settings/` | 7 | the `api` block in the sub-corpus's own `.metaobjects/config.json`: `api.pagination` (default 2, max 4) — a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served, one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` and a negative one is `400 pagination.invalid_value` — and `api.unknownFields: drop`, under which an undeclared body key is ignored rather than refused | generated only | | `duration/` | 2 | `field.duration` is a canonical ISO-8601 span on the wire under `"wire": { "durations": "iso" }`: write, read, refuse, and filter and sort as a span on every engine. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` diff --git a/fixtures/api-contract-conformance/api-settings/README.md b/fixtures/api-contract-conformance/api-settings/README.md index 3c5f41b0d2..8c53ba653d 100644 --- a/fixtures/api-contract-conformance/api-settings/README.md +++ b/fixtures/api-contract-conformance/api-settings/README.md @@ -25,6 +25,9 @@ scenarios can tell the default page from the whole table and the cap from the ta - **Over the cap is refused, not clamped.** `?limit=5` answers `400 {"error": "limit_too_large", "maxLimit": 4}`, so a client that asked for more than the server serves is told so instead of receiving a short page it cannot tell from the end. +- **A negative limit is refused.** `?limit=-1` answers `400 {"error": "pagination.invalid_value", + "param": "limit", "value": "-1", "expected": "a non-negative integer (0 or more)"}` rather than + reaching the store, where SQLite reads `LIMIT -1` as no limit at all. - **Paging past the default works.** `?offset=4` returns the one remaining row, and `?withCount=1` answers 2 rows with `total: 5`. @@ -55,6 +58,7 @@ api-settings/ └── scenarios/ ├── default-page-size.yaml ├── limit-at-max.yaml + ├── limit-negative.yaml ├── limit-over-max.yaml ├── offset-pages-past-default.yaml ├── unknown-fields-dropped.yaml diff --git a/fixtures/api-contract-conformance/api-settings/scenarios/limit-negative.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/limit-negative.yaml new file mode 100644 index 0000000000..6103909ebd --- /dev/null +++ b/fixtures/api-contract-conformance/api-settings/scenarios/limit-negative.yaml @@ -0,0 +1,17 @@ +name: limit-negative +description: > + A negative `?limit=` is refused with the malformed-bound envelope, never handed to the store: + SQLite reads `LIMIT -1` as no limit at all, so passing it through would step around + `api.pagination.maxLimit`. +requests: + - id: r1 + method: GET + path: /api/items?limit=-1 + expect: + status: 400 + body: + equals: + error: "pagination.invalid_value" + param: "limit" + value: "-1" + expected: "a non-negative integer (0 or more)" diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs index 71888eb33c..703ee83ad8 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs @@ -4,6 +4,9 @@ using MetaObjects.Codegen; using MetaObjects.Codegen.Generators; +using MetaObjects.Codegen.Runtime; +using Microsoft.AspNetCore.Http; +using System.Text.Json; using MetaObjects.Config; using MetaObjects.Loader; using MetaObjects.Meta; @@ -95,4 +98,29 @@ public void Drop_emits_no_unknown_field_check_and_a_page_size_is_emitted_when_se Assert.Contains("limit_too_large", src); Assert.Contains("var __take = 2;", src); } + + [Fact] + public void Unknown_keys_are_listed_once_each_in_code_point_order() + { + // U+FF5E < U+1F600 by code point; StringComparer.Ordinal (UTF-16 units) puts the emoji first. + using var doc = JsonDocument.Parse("{\"\\ud83d\\ude00\": 1, \"\\uff5e\": 2, \"a\": 3, \"title\": 4}"); + var result = UnknownFieldCheck.Reject(doc.RootElement, new HashSet { "title" }); + var value = Assert.IsAssignableFrom(result).Value; + var body = JsonSerializer.SerializeToElement(value); + Assert.Equal("unknown_fields", body.GetProperty("error").GetString()); + Assert.Equal( + new[] { "a", "\uff5e", "\ud83d\ude00" }, + body.GetProperty("fields").EnumerateArray().Select(e => e.GetString()).ToArray()); + Assert.Equal(400, Assert.IsAssignableFrom(result).StatusCode); + } + + [Fact] + public void A_body_that_is_not_an_object_or_has_only_declared_keys_is_not_refused() + { + var known = new HashSet { "id", "title" }; + using var array = JsonDocument.Parse("[1, 2]"); + using var declared = JsonDocument.Parse("{\"id\": 9, \"title\": \"x\"}"); + Assert.Null(UnknownFieldCheck.Reject(array.RootElement, known)); + Assert.Null(UnknownFieldCheck.Reject(declared.RootElement, known)); + } } diff --git a/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs index a3a00e2b8e..4e5e02be64 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ValidationIssuesTests.cs @@ -70,14 +70,24 @@ public void A_missing_required_value_is_invalid_type_and_is_listed_with_every_ot public void A_valid_object_records_nothing() => Assert.Empty(Run(new Doc { Name = "ok", Rank = 5 })); [Fact] - public void The_same_code_at_the_same_path_is_recorded_once() + public void A_type_error_at_one_path_is_recorded_once() { var issues = new ValidationIssues(); issues.AddRequired("name"); - issues.AddRequired("name"); + issues.Add("invalid_type", "expected a string", "name"); Assert.Equal(1, issues.Count); } + [Fact] + public void Two_distinct_failures_of_one_rule_kind_on_one_field_are_both_recorded() + { + var issues = new ValidationIssues(); + issues.Add("custom", "must start with a letter", "code"); + issues.Add("custom", "must not contain spaces", "code"); + issues.Add("custom", "must start with a letter", "code"); + Assert.Equal(2, issues.Count); + } + private static (string Code, string Path) Json(string body) { var issues = new ValidationIssues(); diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 782db46204..4f596c7bc5 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -8,10 +8,16 @@ // // Cross-port API contract conformed by the generated handlers // (see docs/features/api-contract.md): -// - GET list honours ?limit=N&offset=N pagination. +// - GET list honours ?limit=N&offset=N pagination. With `api.pagination` set, no limit +// takes `defaultLimit` and a limit above `maxLimit` is 400 { "error": "limit_too_large", +// "maxLimit": N }; without it, no limit returns every row. // - GET list honours ?sort=:asc|desc against a static SortAllowlist // derived from the entity's scalar fields (case-insensitive lookup). -// Unknown field → 400 { "error": "validation", "message": ... }. +// Unknown field → 400 { "error": "invalid_sort", "field": ... }. +// - Create, PATCH and PUT: under `api.unknownFields: reject` (the default) a body key no +// field declares is 400 { "error": "unknown_fields", "fields": [...] } (UnknownFieldCheck); +// under `drop` it is ignored. A failed validation is 400 { "error": "validation", +// "issues": [{ code, path, message }, ...] } listing every issue (ValidationIssues). // - GET list honours ?withCount=1 → switches the envelope from a bare row // array to { rows, total }. The grid hook always sends withCount=1. // - Update accepts BOTH PATCH and PUT (TS reference exposes both; we match). @@ -1053,6 +1059,9 @@ private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, str return; } sb.AppendLine($"{indent}// api.pagination: no limit takes defaultLimit; a limit above maxLimit is refused."); + // A negative limit is refused with the malformed-bound envelope, as the TypeScript routes answer. + sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}Neg) && long.TryParse({limRaw}Neg, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}Neg) && {lim}Neg < 0)"); + sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"limit\", value = {lim}Neg.ToString(System.Globalization.CultureInfo.InvariantCulture), expected = \"a non-negative integer (0 or more)\" }});"); sb.AppendLine($"{indent}var __take = {page.DefaultLimit};"); sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && long.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} > 0)"); sb.AppendLine($"{indent}{{"); diff --git a/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs index 5f03a259f2..1b5eb57f56 100644 --- a/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs +++ b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs @@ -3,8 +3,8 @@ // 400 { "error": "unknown_fields", "fields": [ "isAdmin", "nickname" ] } // // The top-level keys of the JSON object body that the entity does not declare (for a single-table -// hierarchy: that no field of the base or any subtype declares), sorted by ordinal order, listed once -// each. A declared key the client may not set (the generated key, a readOnly field, a frozen field) is +// hierarchy: that no field of the base or any subtype declares), sorted by code point (the order every +// port lists them in), listed once each. A declared key the client may not set (the generated key, a readOnly field, a frozen field) is // not unknown: the route drops it as it always did, so a client that echoes back a row it read works. // Under `api.unknownFields: drop` the route never calls this. @@ -25,7 +25,23 @@ public static class UnknownFieldCheck foreach (var property in body.EnumerateObject()) if (!known.Contains(property.Name) && !unknown.Contains(property.Name)) unknown.Add(property.Name); if (unknown.Count == 0) return null; - unknown.Sort(StringComparer.Ordinal); + unknown.Sort(CompareCodePoints); return Results.BadRequest(new { error = "unknown_fields", fields = unknown }); } + + /// Code-point order. compares UTF-16 code units, which + /// puts a key outside the Basic Multilingual Plane before U+E000..U+FFFF. + public static int CompareCodePoints(string a, string b) + { + var x = a.EnumerateRunes(); + var y = b.EnumerateRunes(); + while (true) + { + var hasX = x.MoveNext(); + var hasY = y.MoveNext(); + if (!hasX || !hasY) return hasX.CompareTo(hasY); + var c = x.Current.Value.CompareTo(y.Current.Value); + if (c != 0) return c; + } + } } diff --git a/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs b/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs index cd73ad3f93..3317bfacc7 100644 --- a/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs +++ b/server/csharp/MetaObjects.Codegen/Runtime/ValidationIssues.cs @@ -51,10 +51,12 @@ public static IResult Unreadable(string message = "the request body is not a JSO return issues.ToResult(); } - /// Record an issue; the same code at the same path is recorded once. + /// Record an issue. A value has one type, so invalid_type is recorded once per path; + /// any other code is recorded once per distinct message, so two rules failing on one field both list. public void Add(string code, string message, params object[] path) { - if (!_seen.Add(code + "|" + JsonSerializer.Serialize(path))) return; + var key = code + "|" + JsonSerializer.Serialize(path) + (code == "invalid_type" ? "" : "|" + message); + if (!_seen.Add(key)) return; _issues.Add(new Dictionary { ["code"] = code, ["path"] = path.ToList(), ["message"] = message }); } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 4428ade1cb..18651bdd4f 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -144,9 +144,14 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase ${page.maxLimit()}L) return@transaction ResponseEntity.badRequest().body(mapOf(\"error\" to \"limit_too_large\", \"maxLimit\" to ${page.maxLimit()}L) as Any)\n") } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java index c95eeb382c..d861d52b96 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java @@ -1811,12 +1811,19 @@ private static void appendUnknownFieldsGuard(StringBuilder src, boolean enabled) * The {@code actualLimit} line of a list handler: no {@code limit} reads the page default * (50 unless the project sets {@code api.pagination}), and a {@code limit} above * {@code api.pagination.maxLimit} is {@code 400 {"error": "limit_too_large", "maxLimit": N}}. + * Under {@code api.pagination} a negative {@code limit} is refused with the malformed-bound + * envelope, as the TypeScript routes refuse it, so it never reaches a store that reads + * {@code LIMIT -1} as no limit (SQLite) and steps around the cap. */ private static void appendLimit(StringBuilder src, ApiSettings.Pagination page) { if (page == null) { src.append(" int actualLimit = limit != null ? limit : 50;\n"); return; } + src.append(" if (limit != null && limit < 0) {\n"); + src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"pagination.invalid_value\", \"param\", \"limit\", ") + .append("\"value\", String.valueOf(limit), \"expected\", \"a non-negative integer (0 or more)\"));\n"); + src.append(" }\n"); src.append(" if (limit != null && limit > ").append(page.maxLimit()).append(") {\n"); src.append(" return ResponseEntity.badRequest().body(Map.of(\"error\", \"limit_too_large\", \"maxLimit\", ") .append(page.maxLimit()).append("));\n"); diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 8263df2f5b..fc7f985790 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -235,10 +235,23 @@ def _known_fields_lines(entities: list[MetaObject]) -> list[str]: def _list_limit_lines(page: ApiPagination | None) -> list[str]: """The limit handling at the top of a list handler body. With no ``api.pagination`` it is the historic default of 50 and no cap; with one, the project's default page and a refusal above - the cap (not a clamp: the client is told it asked for more than the server serves).""" + the cap (not a clamp: the client is told it asked for more than the server serves). A + negative limit is refused too, as the TypeScript routes refuse it, so it never reaches a + store that reads ``LIMIT -1`` as "no limit" (SQLite).""" if page is None: return [" actual_limit = limit if limit is not None else 50"] + expected = "a non-negative integer (0 or more)" return [ + " if limit is not None and limit < 0:", + " return JSONResponse(", + " status_code=400,", + " content={", + ' "error": "pagination.invalid_value",', + ' "param": "limit",', + ' "value": str(limit),', + f' "expected": "{expected}",', + " },", + " )", f" if limit is not None and limit > {page.max_limit}:", f' return JSONResponse(status_code=400, content={{"error": "limit_too_large", "maxLimit": {page.max_limit}}})', f" actual_limit = limit if limit is not None else {page.default_limit}", diff --git a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py index 32f59d8545..c3df9564ce 100644 --- a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py +++ b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py @@ -269,12 +269,17 @@ def value_error_issues(exc: ValueError) -> list[dict[str, Any]]: def validation_envelope(issues: list[dict[str, Any]]) -> dict[str, Any]: - """``{"error": "validation", "issues": [...]}`` -- the issues de-duplicated on - ``(path, code)``, so a rule reported twice (a required null the model also rejects) lists once.""" - seen: set[tuple[str, str]] = set() + """``{"error": "validation", "issues": [...]}`` -- every issue, with repeats removed. + + A value has one type, so ``invalid_type`` is listed once per path: that is how a required null + the model also rejects lists once. Any other code keeps each distinct message, so two + validators failing on one field are both listed.""" + seen: set[tuple[str, str, str]] = set() unique: list[dict[str, Any]] = [] for issue in issues: - key = (repr(issue["path"]), issue["code"]) + code = issue["code"] + message = "" if code == "invalid_type" else str(issue.get("message", "")) + key = (repr(issue["path"]), code, message) if key not in seen: seen.add(key) unique.append(issue) diff --git a/server/python/tests/integration/test_generated_router_validated_input.py b/server/python/tests/integration/test_generated_router_validated_input.py index 0cffb2cf97..51c2fe02df 100644 --- a/server/python/tests/integration/test_generated_router_validated_input.py +++ b/server/python/tests/integration/test_generated_router_validated_input.py @@ -4,9 +4,11 @@ then passed the RAW body to the repository. Pydantic ignores a key a model does not declare, and the input models leave out the server-owned fields, so a client-sent ``id``, a ``readOnly`` field and a made-up ``isAdmin`` all reached the write. Now the repository receives only what the input -model validated, at every depth, as with the TypeScript routes: a name the entity does not -declare, a server-generated id, a ``readOnly`` field and the id on update are all dropped -silently. None of them is a 400: a client that sends an extra harmless field keeps working. +model validated, at every depth, as with the TypeScript routes: a server-generated id, a +``readOnly`` field and the id on update are all dropped silently, never a 400. A top-level name +the entity does not declare is the project's ``api.unknownFields`` setting: under ``drop`` it is +dropped the same way, and under ``reject`` (the default) the route answers +``400 {"error": "unknown_fields"}`` and hands the repository nothing. The model is generated with the real Python codegen (``run_gen``: entity + filter-allowlist + routes), the emitted package is imported unmodified, and the only hand-written piece is a @@ -28,6 +30,7 @@ from fastapi.testclient import TestClient from metaobjects import MetaDataLoader +from metaobjects.api_settings import UNKNOWN_FIELDS_DROP, UNKNOWN_FIELDS_REJECT, ApiSettings from metaobjects.codegen.config import GenConfig from metaobjects.codegen.generators.entity_model import entity_model from metaobjects.codegen.generators.filter_allowlist_generator import filter_allowlist_generator @@ -102,12 +105,16 @@ def update(self, id: Any, dto: Any) -> Any | None: return {self.pk: id, **dto} -def _generate_package() -> str: - """Run the real Python codegen for ``_META`` into a temp package; return its name.""" +def _generate_package(unknown_fields: str) -> str: + """Run the real Python codegen for ``_META`` into a temp package; return its name. + + *unknown_fields* is the project's ``api.unknownFields``, set on the loaded root the way + ``.metaobjects/config.json`` sets it.""" meta_dir = Path(tempfile.mkdtemp(prefix="secfix-meta-")) (meta_dir / "meta.json").write_text(json.dumps(_META)) loaded = MetaDataLoader.from_directory(str(meta_dir)) assert not loaded.errors, "; ".join(f"{e.code}: {e.message}" for e in loaded.errors) + loaded.root.api = ApiSettings(unknown_fields=unknown_fields) pkg = f"secfix_{uuid.uuid4().hex[:8]}" root = Path(tempfile.mkdtemp(prefix="secfix-gen-")) run_gen( @@ -119,11 +126,14 @@ def _generate_package() -> str: return pkg -_PKG = _generate_package() +_PKG = _generate_package(UNKNOWN_FIELDS_DROP) +_REJECT_PKG = _generate_package(UNKNOWN_FIELDS_REJECT) -def _client(router_module: str, pk: str) -> tuple[TestClient, _RecordingRepo, str]: - mod = importlib.import_module(f"{_PKG}.{router_module}") +def _client( + router_module: str, pk: str, pkg: str = _PKG +) -> tuple[TestClient, _RecordingRepo, str]: + mod = importlib.import_module(f"{pkg}.{router_module}") repo = _RecordingRepo(pk) app = FastAPI() app.include_router(mod.router) @@ -136,6 +146,11 @@ def member() -> tuple[TestClient, _RecordingRepo, str]: return _client("member_router", "id") +@pytest.fixture() +def member_reject() -> tuple[TestClient, _RecordingRepo, str]: + return _client("member_router", "id", _REJECT_PKG) + + @pytest.fixture() def account() -> tuple[TestClient, _RecordingRepo, str]: return _client("account_router", "code") @@ -178,6 +193,27 @@ def test_a_malformed_value_on_a_settable_field_is_still_a_400(member) -> None: assert repo.writes == [] +# --- api.unknownFields: reject (the default) refuses an undeclared key and writes nothing --- + + +@pytest.mark.parametrize("method", ["POST", "PATCH", "PUT"]) +def test_reject_refuses_an_undeclared_key_and_writes_nothing(member_reject, method: str) -> None: + client, repo, base = member_reject + url = base if method == "POST" else f"{base}/1" + resp = client.request(method, url, json={"name": "M", "isAdmin": True, "zeta": "x"}) + assert resp.status_code == 400, resp.text + assert resp.json() == {"error": "unknown_fields", "fields": ["isAdmin", "zeta"]} + assert repo.writes == [] + + +def test_reject_still_drops_a_declared_server_owned_key(member_reject) -> None: + client, repo, base = member_reject + resp = client.post(base, json={"id": 999, "name": "Mallory"}) + assert resp.status_code == 201, resp.text + [(_op, written)] = repo.writes + assert "id" not in written + + # --- declared but not settable: dropped, never 400'd, and never handed to the repository ---- @@ -261,25 +297,22 @@ def recording_update(subtype: str, id: int, dto: Any) -> Any: return TestClient(app), writes -def test_tph_subtype_create_drops_the_client_id_and_an_undeclared_field(tph) -> None: +def test_tph_subtype_create_drops_the_client_id(tph) -> None: + # An undeclared key under the default `reject` is pinned by tph-unknown-fields-rejected. client, writes = tph - resp = client.post( - "/api/auths/bridge", json={"id": 999, "reference": "R-1", "quantity": 2, "isAdmin": True} - ) + resp = client.post("/api/auths/bridge", json={"id": 999, "reference": "R-1", "quantity": 2}) assert resp.status_code == 201, resp.text [(op, subtype, written)] = writes assert (op, subtype) == ("create", "Bridge") assert "id" not in written - assert "isAdmin" not in written assert {"reference", "quantity"} <= set(written) -def test_tph_subtype_update_drops_the_id_and_an_undeclared_field(tph) -> None: +def test_tph_subtype_update_drops_the_id(tph) -> None: client, writes = tph # The store is unseeded, so the answer is a 404; what matters is what the write was handed. - client.patch("/api/auths/bridge/1", json={"id": 999, "quantity": 3, "isAdmin": True}) + client.patch("/api/auths/bridge/1", json={"id": 999, "quantity": 3}) [(op, subtype, written)] = writes assert (op, subtype) == ("update", "Bridge") assert "id" not in written - assert "isAdmin" not in written assert written["quantity"] == 3 diff --git a/server/python/tests/runtime/test_validation_issues.py b/server/python/tests/runtime/test_validation_issues.py index b84b340854..d4cac02c95 100644 --- a/server/python/tests/runtime/test_validation_issues.py +++ b/server/python/tests/runtime/test_validation_issues.py @@ -105,13 +105,22 @@ def test_unreadable_body_has_an_empty_path() -> None: assert issue["path"] == [] and issue["code"] == "invalid_type" -def test_envelope_dedupes_on_path_and_code() -> None: +def test_envelope_lists_one_type_error_per_path() -> None: + # A required null the model also rejects: one value, one type error. a = {"code": "invalid_type", "path": ["x"], "message": "one"} b = {"code": "invalid_type", "path": ["x"], "message": "two"} c = {"code": "too_big", "path": ["x"], "message": "three"} assert validation_envelope([a, b, c]) == {"error": "validation", "issues": [a, c]} +def test_envelope_keeps_distinct_failures_of_one_rule_kind_on_one_field() -> None: + # Two validators on one field, both `custom`: the client must see both to fix both. + a = {"code": "custom", "path": ["x"], "message": "must start with a letter"} + b = {"code": "custom", "path": ["x"], "message": "must not contain spaces"} + repeat = dict(a) + assert validation_envelope([a, b, repeat]) == {"error": "validation", "issues": [a, b]} + + def test_decimal_failures_carry_a_wire_path_and_code() -> None: with pytest.raises(ValueError) as unparseable: coerce_decimal_fields({"amount": "abc"}, frozenset({"amount"})) diff --git a/server/typescript/packages/runtime-ts/src/wire-format.ts b/server/typescript/packages/runtime-ts/src/wire-format.ts index ccb26e442e..b6969300c7 100644 --- a/server/typescript/packages/runtime-ts/src/wire-format.ts +++ b/server/typescript/packages/runtime-ts/src/wire-format.ts @@ -73,14 +73,32 @@ export function stripDeniedColumns(body: unknown, denied: readonly string[] | un export const ERROR_CODE_UNKNOWN_FIELDS = "unknown_fields"; /** - * The keys of a request body that are not in `known`, sorted, or none when `known` is not - * given (the route drops undeclared keys instead: `api.unknownFields: drop`). Only the top - * level is checked; a body that is not a JSON object is left to validation. + * The keys of a request body that are not in `known`, sorted by code point, or none when + * `known` is not given (the route drops undeclared keys instead: `api.unknownFields: drop`). + * Only the top level is checked; a body that is not a JSON object is left to validation. */ export function unknownBodyKeys(body: unknown, known: readonly string[] | undefined): string[] { if (known === undefined || !isPlainObject(body)) return []; const declared = new Set(known); - return Object.keys(body).filter((k) => !declared.has(k)).sort(); + return Object.keys(body).filter((k) => !declared.has(k)).sort(compareCodePoints); +} + +/** + * Code-point order, the order every port lists `fields` in. The default `sort()` compares + * UTF-16 code units, which puts a key outside the Basic Multilingual Plane (an emoji) before + * U+E000..U+FFFF; Java, Kotlin, Python and C# answer in code-point order. + */ +function compareCodePoints(a: string, b: string): number { + let i = 0; + let j = 0; + while (i < a.length && j < b.length) { + const ca = a.codePointAt(i)!; + const cb = b.codePointAt(j)!; + if (ca !== cb) return ca - cb; + i += ca > 0xffff ? 2 : 1; + j += cb > 0xffff ? 2 : 1; + } + return (a.length - i) - (b.length - j); } function isPlainObject(v: unknown): v is Record { diff --git a/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts b/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts index 62266fffea..98bb8fe44c 100644 --- a/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts +++ b/server/typescript/packages/runtime-ts/test/unknown-fields.test.ts @@ -40,6 +40,10 @@ describe("unknownBodyKeys", () => { test("keys outside the known list, sorted", () => { expect(unknownBodyKeys({ title: "x", zeta: 1, isAdmin: true }, KNOWN)).toEqual(["isAdmin", "zeta"]); }); + test("sorted by code point, not UTF-16 code unit", () => { + // U+FF5E < U+1F600 by code point; by UTF-16 unit the emoji's surrogate (D83D) sorts first. + expect(unknownBodyKeys({ "\u{1F600}": 1, "\uFF5E": 2, a: 3 }, KNOWN)).toEqual(["a", "\uFF5E", "\u{1F600}"]); + }); test("a body that is not an object is left to validation", () => { expect(unknownBodyKeys([1, 2], KNOWN)).toEqual([]); expect(unknownBodyKeys("text", KNOWN)).toEqual([]); From b2be4d2e26745a40dcb9df7021cf3de52cdad0d4 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 15:46:09 -0500 Subject: [PATCH 08/31] fix: generated create routes drop server-owned input across ports (1.2.x security) (#568) * fix: generated create routes store only client-settable fields (1.2.x security) Python 1.2.1 stopped its generated routes from handing raw request input to the repository. The same audit found related create-path holes in 1.2.0 in the other four ports; this closes each in its own idiom. Server-owned values are dropped silently, never answered with 400. - Java (codegen-spring): create strips a generated id, readOnly and derived fields; a TPH per-subtype create also strips the body discriminator and sibling-subtype fields. - Kotlin (codegen-kotlin): create skips readOnly/derived columns; a TPH per-subtype create writes only the route subtype's own columns. - C# (MetaObjects.Codegen): every readOnly property shape gets a private setter; create resets a generated id, the TPH discriminator and M:N navigations before Add. - TypeScript (codegen-ts): the update schema omits primary-key fields, so PUT/PATCH can no longer re-key a row with an assigned key. Proof: new shared api-contract scenarios (core + TPH input-server-owned, TPH foreign discriminator) and a readOnly `role` field in secret-fields, run in every port's generated lane and the reference lanes; a new `unset` assertion key in all five harnesses. In-memory test repositories now store what they are handed, so the scenarios fail on the 1.2.0 generators. Proposed versions (not published): npm 1.2.2, NuGet 1.2.2, Maven 8.2.2; PyPI unchanged at 1.2.1. metamodelVersion unchanged. * fix: keep a @required readOnly field creatable in C# and Java generated create Pre-gate review follow-ups to the 1.2.x create-input patch: - C#: the required-key check skips readOnly fields (the private setter never binds them), and create validation ignores a violation naming only a @required readOnly member, as Java's SERVER_OWNED_ON_CREATE does. - Java: the create-validation server-owned set covers every DTO component, so a nulled readOnly or derived value-object column no longer 400s. - TypeScript: drop the now-dead PK guard in the update-schema nullable arm and list the PK rule in updateSchemaFields' doc comment. * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Kotlin readOnly create fix + C# reset skip * no-mistakes(document): docs: finish stale corpus counts from input fix --- CHANGELOG.md | 46 ++++ bun.lock | 28 +-- client/web/packages/react/package.json | 2 +- client/web/packages/runtime-web/package.json | 2 +- client/web/packages/tanstack/package.json | 2 +- docs/CONFORMANCE.md | 14 +- docs/features/generated-mutations.md | 15 ++ docs/llms/llms-full.txt | 20 +- docs/llms/llms.txt | 16 +- .../codegen/templates/zod-validators.ts | 16 +- .../.metaobjects/.gen-state/.hashes.json | 2 +- .../codegen/templates/zod-validators.ts | 16 +- .../generated/csharp/SubscriberRoutes.g.cs | 2 + examples/showcase/jvm/pom.xml | 2 +- examples/showcase/site-payload.json | 6 +- fixtures/api-contract-conformance/README.md | 8 +- .../scenarios/input-server-owned-dropped.yaml | 85 +++++++ .../secret-fields/README.md | 5 +- .../secret-fields/meta.json | 1 + .../scenarios/read-omits-credential.yaml | 3 + .../scenarios/write-ignores-excluded.yaml | 15 +- .../secret-fields/seed.json | 4 +- .../api-contract-conformance/tph/README.md | 4 +- ...h-input-foreign-discriminator-dropped.yaml | 44 ++++ .../tph-input-server-owned-dropped.yaml | 72 ++++++ server/csharp/Directory.Build.props | 2 +- .../ServerOwnedCreateInputTests.cs | 236 ++++++++++++++++++ .../Generators/EntityGenerator.cs | 42 ++-- .../Generators/RoutesGenerator.cs | 96 ++++++- .../Api/ApiContractAssertions.cs | 13 + .../ApiContractSecretFieldsConformanceTest.cs | 5 +- .../Api/SecretFieldsFixture.cs | 3 +- .../Api/TphReferenceServer.cs | 7 + server/java/codegen-base/pom.xml | 2 +- .../codegen-kotlin-exposed1x-check/pom.xml | 2 +- server/java/codegen-kotlin/pom.xml | 2 +- .../generator/kotlin/KotlinGenUtil.kt | 17 +- .../kotlin/KotlinSpringControllerGenerator.kt | 12 +- .../kotlin/KotlinServerOwnedCreateBindTest.kt | 84 +++++++ .../kotlin/KotlinServerOwnedInputTest.kt | 89 +++++++ .../acme/auth/AuthController.kt | 6 - server/java/codegen-mustache/pom.xml | 2 +- server/java/codegen-plantuml/pom.xml | 2 +- server/java/codegen-spring/pom.xml | 2 +- .../spring/SpringControllerGenerator.java | 66 ++++- .../spring/SpringSecretFieldsCodegenTest.java | 7 +- .../SpringServerOwnedInputCodegenTest.java | 133 ++++++++++ server/java/core-spring/pom.xml | 2 +- server/java/integration-tests-kotlin/pom.xml | 2 +- .../kotlin/api/ApiContractAssertions.kt | 11 +- .../GeneratedAuthorControllerHarness.kt | 3 + .../GeneratedSecretFieldsControllerHarness.kt | 9 +- .../GeneratedTphControllerHarness.kt | 3 + server/java/integration-tests/pom.xml | 2 +- .../api/ApiContractAssertions.java | 13 +- .../GeneratedAccountControllerHarness.java | 5 +- .../GeneratedAuthorControllerHarness.java | 5 +- .../InMemoryAccountRepositorySource.java | 11 +- .../InMemoryAuthorRepositorySource.java | 6 +- .../GeneratedTphControllerHarness.java | 5 +- .../InMemoryAuthRepositorySource.java | 11 +- server/java/maven-plugin/pom.xml | 2 +- server/java/metadata-ktx/pom.xml | 2 +- server/java/metadata/pom.xml | 2 +- server/java/om/pom.xml | 2 +- server/java/omdb-ktx/pom.xml | 2 +- server/java/omdb/pom.xml | 2 +- server/java/pom.xml | 2 +- server/java/render/pom.xml | 2 +- server/java/spring-boot-starter/pom.xml | 2 +- .../src/metaobjects/codegen/KNOWN_GAPS.md | 12 + .../integration/api_contract_assertions.py | 15 +- .../generated_secret_fields_app.py | 2 +- .../test_api_contract_tph_generated.py | 12 + .../packages/ai-runtime/package.json | 2 +- server/typescript/packages/cli/package.json | 2 +- .../packages/codegen-ts-react/package.json | 2 +- .../packages/codegen-ts-tanstack/package.json | 2 +- .../packages/codegen-ts/package.json | 2 +- .../src/templates/zod-validators.ts | 16 +- .../__snapshots__/reporting-docs.test.ts.snap | 66 ++--- .../test/templates/zod-validators.test.ts | 16 ++ .../packages/docs-site/package.json | 2 +- .../src/api-contract-scenario.ts | 12 + ...contract-secret-fields-generated-server.ts | 7 +- .../src/api-contract-server.ts | 19 +- .../typescript/packages/metadata/package.json | 2 +- .../packages/migrate-ts/package.json | 2 +- .../typescript/packages/render/package.json | 2 +- .../packages/runtime-ts/package.json | 2 +- server/typescript/packages/sdk/package.json | 2 +- .../src/templates/zod-validators.ts | 16 +- 92 files changed, 1359 insertions(+), 222 deletions(-) create mode 100644 fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml create mode 100644 fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml create mode 100644 fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml create mode 100644 server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedCreateBindTest.kt create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt create mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 9fe7ecc774..b81c89d1cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,52 @@ here.** ## [Unreleased] +_Proposed as npm `1.2.2` (all 14 `@metaobjectsdev/*` packages), NuGet `1.2.2` and Maven Central +`8.2.2`; `1.2.1` was spent by the Python-only release. PyPI stays at `1.2.1`: no Python product +file changes. No vocabulary or `metamodelVersion` change._ + +### Security + +- **Generated create routes in Java, Kotlin and C#, and TypeScript updates, let a client write + fields it must never set.** The same class of hole the Python 1.2.1 patch closed, found by an + audit of the other four ports after it. Every adopter serving these generated routes should + upgrade and regenerate; an ejected (owned) generator needs the same change made by hand. + + - **Java (`codegen-spring`).** The create route passed the bound DTO to the repository as + sent, so a client `id` on a generated key and a `@mutability: readOnly` or derived field + reached it. On a TPH per-subtype route the body's discriminator and a sibling subtype's + columns did too. The route now rebuilds the DTO with each of those components set to null + before the repository call; the route's subtype still comes from the URL. A `@required` + `readOnly` or derived value-object column no longer fails create validation once nulled. + - **Kotlin (`codegen-kotlin`).** The create route wrote `readOnly` and derived columns from the + body, and a TPH per-subtype route wrote a sibling subtype's columns. It now writes only the + route subtype's own settable columns. + - **C# (`MetaObjects.Codegen`).** The create route bound the EF Core entity from the body, so a + client `id` on a generated key was inserted, a TPH create stored the body's discriminator (a + POST to the Bridge route could create a Copay row), and a TPH subtype's `readOnly` field and + an M:N navigation graph were written. The route now resets the generated key and the + discriminator to their defaults and empties the M:N navigations before `Add`, and a + `readOnly` property of every shape, a TPH subtype's included, has a private setter, which + System.Text.Json does not bind. A `@required` `readOnly` field is therefore neither demanded + of the create body nor held to its `[Required]` there, so such an entity stays creatable. + - **TypeScript (`codegen-ts`).** The `UpdateSchema` kept an assigned (non-generated) + primary key, so a PUT or PATCH could re-key the row. The update schema now leaves out every + primary-key field; the insert schema still takes an assigned key, which is the caller's on + create. + + Each of these values is dropped silently, never answered with a 400. A key the model does not + declare is dropped silently in TypeScript, C# and Python. In Java and Kotlin the answer to an + undeclared key is the host's Jackson `ObjectMapper`'s: Spring Boot's default ignores it, and a + plain `ObjectMapper` (`FAIL_ON_UNKNOWN_PROPERTIES` on) answers 400. This patch leaves that as + it was. + + Gated by the shared api-contract scenarios `input-server-owned-dropped`, + `tph-input-server-owned-dropped` and `tph-input-foreign-discriminator-dropped`, and a `readOnly` + `role` on the `secret-fields` corpus, in every port's generated lane and in the reference lanes. + Python's generated TPH routes answer a foreign body discriminator with a 400 rather than + dropping it (nothing is stored); its runner skips that one scenario, recorded in the Python + codegen `KNOWN_GAPS.md`. + ## [1.2.1] — Python only _PyPI `1.2.1` only. npm, NuGet and Maven Central stay at `1.2.0` (`8.2.0`): none of them has a diff --git a/bun.lock b/bun.lock index 195e182cb8..9def0b0bbb 100644 --- a/bun.lock +++ b/bun.lock @@ -43,7 +43,7 @@ }, "client/web/packages/react": { "name": "@metaobjectsdev/react", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/runtime-web": "workspace:*", "react-easy-crop": "^6.2.3", @@ -73,7 +73,7 @@ }, "client/web/packages/runtime-web": { "name": "@metaobjectsdev/runtime-web", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/metadata": "workspace:*", "qs": "^6.13.0", @@ -86,7 +86,7 @@ }, "client/web/packages/tanstack": { "name": "@metaobjectsdev/tanstack", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/runtime-web": "workspace:*", }, @@ -113,7 +113,7 @@ }, "server/typescript/packages/ai-runtime": { "name": "@metaobjectsdev/ai-runtime", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/metadata": "workspace:*", "@metaobjectsdev/render": "workspace:*", @@ -134,7 +134,7 @@ }, "server/typescript/packages/cli": { "name": "@metaobjectsdev/cli", - "version": "1.2.0", + "version": "1.2.2", "bin": { "meta": "./dist/bin/meta.js", }, @@ -179,7 +179,7 @@ }, "server/typescript/packages/codegen-ts": { "name": "@metaobjectsdev/codegen-ts", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@biomejs/js-api": "^0.7.0", "@biomejs/wasm-nodejs": "^1.9.4", @@ -224,7 +224,7 @@ }, "server/typescript/packages/codegen-ts-react": { "name": "@metaobjectsdev/codegen-ts-react", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/codegen-ts": "workspace:*", "@metaobjectsdev/metadata": "workspace:*", @@ -242,7 +242,7 @@ }, "server/typescript/packages/codegen-ts-tanstack": { "name": "@metaobjectsdev/codegen-ts-tanstack", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/codegen-ts": "workspace:*", "@metaobjectsdev/metadata": "workspace:*", @@ -274,7 +274,7 @@ }, "server/typescript/packages/docs-site": { "name": "@metaobjectsdev/docs-site", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/metadata": "workspace:*", "@metaobjectsdev/render": "workspace:*", @@ -326,7 +326,7 @@ }, "server/typescript/packages/metadata": { "name": "@metaobjectsdev/metadata", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "yaml": "^2.9.0", }, @@ -339,7 +339,7 @@ }, "server/typescript/packages/migrate-ts": { "name": "@metaobjectsdev/migrate-ts", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@iarna/toml": "^2.2.5", "@metaobjectsdev/metadata": "workspace:*", @@ -366,7 +366,7 @@ }, "server/typescript/packages/render": { "name": "@metaobjectsdev/render", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "mustache": "^4.2.0", "nunjucks": "^3.2.4", @@ -380,7 +380,7 @@ }, "server/typescript/packages/runtime-ts": { "name": "@metaobjectsdev/runtime-ts", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/metadata": "workspace:*", "@metaobjectsdev/render": "workspace:*", @@ -419,7 +419,7 @@ }, "server/typescript/packages/sdk": { "name": "@metaobjectsdev/sdk", - "version": "1.2.0", + "version": "1.2.2", "dependencies": { "@metaobjectsdev/metadata": "workspace:*", "yaml": "^2.9.0", diff --git a/client/web/packages/react/package.json b/client/web/packages/react/package.json index 389f907df0..5ccfeff21f 100644 --- a/client/web/packages/react/package.json +++ b/client/web/packages/react/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/react", - "version": "1.2.0", + "version": "1.2.2", "description": "React runtime for metaobjects: useEntityForm hook and CurrencyInput component.", "type": "module", "main": "./dist/index.js", diff --git a/client/web/packages/runtime-web/package.json b/client/web/packages/runtime-web/package.json index 0ba383763c..ae0c6e4348 100644 --- a/client/web/packages/runtime-web/package.json +++ b/client/web/packages/runtime-web/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/runtime-web", - "version": "1.2.0", + "version": "1.2.2", "description": "Pure framework-agnostic browser core for metaobjects: currency, filter URL serialization, fetcher contract types.", "type": "module", "main": "./dist/index.js", diff --git a/client/web/packages/tanstack/package.json b/client/web/packages/tanstack/package.json index a1fb67dd6e..b524804714 100644 --- a/client/web/packages/tanstack/package.json +++ b/client/web/packages/tanstack/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/tanstack", - "version": "1.2.0", + "version": "1.2.2", "description": "TanStack runtime for metaobjects: EntityFetcherProvider, EntityGrid, default cell renderers.", "type": "module", "main": "./dist/index.js", diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 36834aadbc..b464a464a5 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 119 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 122 (32 core + 12 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | @@ -366,9 +366,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (119) +### `fixtures/api-contract-conformance/` (122) -All 119 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 122 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -378,12 +378,12 @@ status codes. The corpus also covers the 9 cross-port filter operators (`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `like`, `isNull`) plus the implicit-AND combinator and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the -URL grammar `?filter[][]=` (FR-009). On top of the 31 core -scenarios the corpus carries eight sub-corpora — `tph/` (10, single-table +URL grammar `?filter[][]=` (FR-009). On top of the 32 core +scenarios the corpus carries eight sub-corpora — `tph/` (12, single-table inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), `jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), -`write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, and a credential is never returned), `projection/` (12, a +`write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484 + the 1.2.x input gate: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, a credential is never returned, and a `@mutability: readOnly` field is ignored on write too), `projection/` (12, a read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named `id` is addressed by it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one @@ -611,7 +611,7 @@ files into a real Cube. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 421 + yaml 16 + verify 45 -+ render 18 + persistence 59 + api-contract 119 + source-resolution 25 + scope 10 + ++ render 18 + persistence 59 + api-contract 122 + source-resolution 25 + scope 10 + dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the `cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/generated-mutations.md b/docs/features/generated-mutations.md index 3a367bbb6f..ef3f4a2684 100644 --- a/docs/features/generated-mutations.md +++ b/docs/features/generated-mutations.md @@ -66,6 +66,21 @@ but the semantics are one contract — call them **PATCH-1..7** (FR-035): - **PATCH-7 (verbs).** At the HTTP tier both `PATCH` and `PUT` route to the update handler with partial-merge semantics. True PUT-as-full-replace is out of scope. +**Create stores only what a client may set, in every port.** A generated create route +never writes a client-sent value for a server-generated primary key (`@generation` +`increment` or `uuid`), a `@mutability: "readOnly"` or `origin.*`-derived field, the TPH +discriminator (a per-subtype route takes the subtype from the URL), a column only a +sibling subtype declares, or (C#) an M:N navigation graph. Each is dropped silently, not +answered with a 400. An assigned key (no `@generation`) is the caller's on create in +TypeScript, C#, Java and Python, and on update it is the primary key like any other, so it +is left out of the TypeScript `UpdateSchema` too. (Kotlin's generated create is the +exception: it never binds a declared key, assigned or not — `increment` is left to the +column and a `uuid` key is minted app-side.) A key the entity does not declare is dropped in TypeScript, +C# and Python; in Java and Kotlin that answer belongs to the host's Jackson +`ObjectMapper` (Spring Boot's default ignores it, a plain `ObjectMapper` answers 400). +Gated cross-port by `fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml` +and the `tph/` and `secret-fields/` input scenarios. + The generated update does **not** guard against a concurrent write: two callers patching the same row both succeed, and the last one wins. Adding that guard is adopter-owned — see [Optimistic concurrency](#optimistic-concurrency-adopter-owned). diff --git a/docs/llms/llms-full.txt b/docs/llms/llms-full.txt index 69e45e8a2c..a7db61dfcb 100644 --- a/docs/llms/llms-full.txt +++ b/docs/llms/llms-full.txt @@ -1,6 +1,6 @@ # MetaObjects -- Full Reference Corpus -> Coding agents get context — rules files, memories, MCP servers — and context is advisory: nothing fails when the code stops matching it. MetaObjects gives your agent one typed model of your application — data, API, UI, prompt payloads, and what the software is supposed to do — generates each layer's code from it in TypeScript, Java, Kotlin, C# and Python with no proprietary runtime in the output, and fails your build when generated code, a prompt, or a claimed capability drifts from that model. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.2.0` on npm and `8.2.0` on Maven Central. +> Coding agents get context — rules files, memories, MCP servers — and context is advisory: nothing fails when the code stops matching it. MetaObjects gives your agent one typed model of your application — data, API, UI, prompt payloads, and what the software is supposed to do — generates each layer's code from it in TypeScript, Java, Kotlin, C# and Python with no proprietary runtime in the output, and fails your build when generated code, a prompt, or a claimed capability drifts from that model. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.2.2` on npm and `8.2.2` on Maven Central. **Note for AI assistants:** This file is the concatenated reference material for MetaObjects, intended to be loaded as LLM context. For the short index, see [llms.txt](https://metaobjects.dev/llms.txt). For the canonical spec and source, see the [GitHub repo](https://github.com/metaobjectsdev/metaobjects). When this file is out of date relative to the GitHub source, the GitHub source wins. @@ -39,7 +39,7 @@ If you are an AI assistant helping someone adopt MetaObjects, do this **first** 1. **Install the port's package:** - TypeScript / Node: `npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3`, then `npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3` (SQLite; use `pg` and `@types/pg` for Postgres). The generated files import `drizzle-orm`, `zod`, `fastify` and a driver, and `npm` does not add them for you. - - Java / Kotlin: add `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and `com.metaobjects:metaobjects-maven-plugin`, all at `8.2.0`. + - Java / Kotlin: add `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and `com.metaobjects:metaobjects-maven-plugin`, all at `8.2.2`. - Python: `pip install metaobjects` - C#: install the MetaObjects .NET tool (invoked as `dotnet meta`). 2. **Run `meta init`** (the Node `meta` CLI). It works in any project that has Node — including a JVM or Python project with a TypeScript frontend. It scaffolds a slim always-on `.metaobjects/AGENTS.md` + `.metaobjects/CLAUDE.md` (auto-wired via an `@.metaobjects/AGENTS.md` import line into the project's root `CLAUDE.md`/`AGENTS.md`, so it loads automatically) plus six `metaobjects-*` Claude Code skills under `.claude/skills/`, carrying only the project's language reference fragments. Re-run `meta init --refresh-docs` to update the agent docs after a CLI upgrade. (The Node `meta` CLI is the single agent-context scaffolder for every backend, by design — ADR-0033; the JVM/Python/C# CLIs deliberately redirect to it. For a non-Node project, scaffold just the agent context with `npx meta agent-docs --server `.) @@ -125,13 +125,13 @@ Today the TypeScript toolchain publishes and TypeScript and Python projects cons | Language | Status | Notes | |---|---|---| -| TypeScript | Reference implementation, npm `1.2.0` | All six pillars, and the only port that prints the requirements authoring lint and publishes shared models. 2500+ tests passing. Owns the canonical schema-migration toolchain used by every port (ADR-0015). Bun-first dev. | -| Java | Maven Central `8.2.0` | Spring REST + DTO + repository-interface codegen, OMDB runtime persistence (pure data-access) with Spring-tx. Fully green across all conformance corpora. | -| Kotlin | Maven Central `8.2.0` | KotlinPoet codegen + Exposed runtime + `metadata-ktx` facade. Ships via the Java reactor. | -| C# | NuGet `1.2.0` (.NET tool) | Loader + canonical serializer + EF Core + ASP.NET codegen + render/verify. `dotnet meta` tool. | -| Python | PyPI `1.2.0` | Loader + serializer + render + verify + codegen + `ObjectManager` runtime. Fully green across all corpora. | +| TypeScript | Reference implementation, npm `1.2.2` | All six pillars, and the only port that prints the requirements authoring lint and publishes shared models. 2500+ tests passing. Owns the canonical schema-migration toolchain used by every port (ADR-0015). Bun-first dev. | +| Java | Maven Central `8.2.2` | Spring REST + DTO + repository-interface codegen, OMDB runtime persistence (pure data-access) with Spring-tx. Fully green across all conformance corpora. | +| Kotlin | Maven Central `8.2.2` | KotlinPoet codegen + Exposed runtime + `metadata-ktx` facade. Ships via the Java reactor. | +| C# | NuGet `1.2.2` (.NET tool) | Loader + canonical serializer + EF Core + ASP.NET codegen + render/verify. `dotnet meta` tool. | +| Python | PyPI `1.2.1` | Loader + serializer + render + verify + codegen + `ObjectManager` runtime. Fully green across all corpora. | -Conformance fixtures live at [`fixtures/`](https://github.com/metaobjectsdev/metaobjects/tree/main/fixtures). Every port runs the shared corpus byte-identically: metamodel (`conformance/`, 314 fixtures), render, persistence (Testcontainers Postgres, with an `op: roundtrip` gate so every `field.*` subtype write+read round-trips on every port), api-contract (41 scenarios — 26 core plus TPH / M:N / jsonb / write-through — two lanes: a reference server AND each port's generated API booted over HTTP), registry (byte-matched metamodel-vocabulary manifest, live + green in all five ports), and yaml/verify. +Conformance fixtures live at [`fixtures/`](https://github.com/metaobjectsdev/metaobjects/tree/main/fixtures). Every port runs the shared corpus byte-identically: metamodel (`conformance/`, 314 fixtures), render, persistence (Testcontainers Postgres, with an `op: roundtrip` gate so every `field.*` subtype write+read round-trips on every port), api-contract (core REST contract + filter operators, plus TPH / M:N / jsonb / write-through and the other sub-corpora — two lanes: a reference server AND each port's generated API booted over HTTP), registry (byte-matched metamodel-vocabulary manifest, live + green in all five ports), and yaml/verify. --- @@ -182,7 +182,7 @@ Universal web client (`client/web/packages/`): `@metaobjectsdev/runtime-web`, `@ ## Java / Kotlin Maven coordinates -All published to Maven Central under `com.metaobjects:*` at `8.2.0`: +All published to Maven Central under `com.metaobjects:*` at `8.2.2`: - `metaobjects-metadata`, `metaobjects-metadata-ktx` - `metaobjects-codegen-base`, `metaobjects-codegen-mustache`, `metaobjects-codegen-spring`, `metaobjects-codegen-kotlin`, `metaobjects-codegen-plantuml` @@ -219,7 +219,7 @@ Project marker directory: `.metaobjects/`. ## Quickstart (Java / Kotlin) -Add the Maven coordinates above at `8.2.0`. Run codegen with `mvn metaobjects:generate` and codegen-drift checks with `mvn metaobjects:verify`. Configure generators (including Kotlin generators, which run through the same goal) on the `metaobjects-maven-plugin`. See [`server/java/`](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java) and [`server/java/codegen-kotlin/README.md`](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java/codegen-kotlin). +Add the Maven coordinates above at `8.2.2`. Run codegen with `mvn metaobjects:generate` and codegen-drift checks with `mvn metaobjects:verify`. Configure generators (including Kotlin generators, which run through the same goal) on the `metaobjects-maven-plugin`. See [`server/java/`](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java) and [`server/java/codegen-kotlin/README.md`](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java/codegen-kotlin). --- diff --git a/docs/llms/llms.txt b/docs/llms/llms.txt index 60e67dfb70..cd6e4a15ec 100644 --- a/docs/llms/llms.txt +++ b/docs/llms/llms.txt @@ -1,6 +1,6 @@ # MetaObjects -> MetaObjects gives your coding agent one typed model of your application: data, API, UI, prompt payloads, and what the software is supposed to do. It generates each layer's code from that model in TypeScript, Java, Kotlin, C# and Python, with no proprietary runtime in the output. Your build fails when generated code, a prompt or a claimed capability stops matching the model. Rules files and memories are only advice, because nothing fails when the code drifts from them. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.2.0` on npm and `8.2.0` on Maven Central. +> MetaObjects gives your coding agent one typed model of your application: data, API, UI, prompt payloads, and what the software is supposed to do. It generates each layer's code from that model in TypeScript, Java, Kotlin, C# and Python, with no proprietary runtime in the output. Your build fails when generated code, a prompt or a claimed capability stops matching the model. Rules files and memories are only advice, because nothing fails when the code drifts from them. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.2.2` on npm and `8.2.2` on Maven Central. **One typed model of your app** — data, API, UI, prompt payloads, and what it is supposed to do — that your agent reads and writes. Two things happen to it. @@ -26,7 +26,7 @@ If you are an AI assistant helping someone adopt MetaObjects in their project, d 1. **Install the port's package** for the project's stack: - **TypeScript / Node**: `npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3`, then `npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3` (SQLite; use `pg` and `@types/pg` for Postgres). The generated files import `drizzle-orm`, `zod`, `fastify` and a driver, and `npm` does not add them for you. - - **Java / Kotlin**: add the Maven dependencies `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and the `com.metaobjects:metaobjects-maven-plugin`, all at `8.2.0`. + - **Java / Kotlin**: add the Maven dependencies `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and the `com.metaobjects:metaobjects-maven-plugin`, all at `8.2.2`. - **Python**: `pip install metaobjects` - **C#**: install the MetaObjects .NET tool (`dotnet tool install` the MetaObjects CLI tool; invoked as `dotnet meta`). @@ -45,7 +45,7 @@ This `llms.txt` is the short index; the deep, version-matched how-to is the scaf ## Spec and standard - [Specification (canonical, target-agnostic)](https://github.com/metaobjectsdev/metaobjects/tree/main/spec): the normative metadata schema and semantics that every implementation must conform to. -- [Conformance fixtures](https://github.com/metaobjectsdev/metaobjects/tree/main/fixtures): cross-port corpora — `conformance/` (metamodel, 314 fixtures), `render-conformance/`, `persistence-conformance/` (against Testcontainers Postgres, including an `op: roundtrip` gate so every `field.*` subtype write+read round-trips on every port), `api-contract-conformance/` (41 scenarios — 26 core REST contract + filter operators, plus TPH / M:N / jsonb / write-through — run in two lanes: a reference server AND each port's generated API booted over HTTP), `registry-conformance/` (byte-matched metamodel-vocabulary manifest, live + green in all five ports), and `yaml-conformance/` / `verify-conformance/`. Every port runs the shared corpus byte-identically. +- [Conformance fixtures](https://github.com/metaobjectsdev/metaobjects/tree/main/fixtures): cross-port corpora — `conformance/` (metamodel, 314 fixtures), `render-conformance/`, `persistence-conformance/` (against Testcontainers Postgres, including an `op: roundtrip` gate so every `field.*` subtype write+read round-trips on every port), `api-contract-conformance/` (core REST contract + filter operators, plus TPH / M:N / jsonb / write-through and the other sub-corpora — run in two lanes: a reference server AND each port's generated API booted over HTTP), `registry-conformance/` (byte-matched metamodel-vocabulary manifest, live + green in all five ports), and `yaml-conformance/` / `verify-conformance/`. Every port runs the shared corpus byte-identically. - [Roadmap](https://github.com/metaobjectsdev/metaobjects/issues?q=is%3Aissue+label%3AFR): current + planned work, tracked as GitHub issues labelled `FR` with milestones per release; [spec/roadmap.md](https://github.com/metaobjectsdev/metaobjects/blob/main/spec/roadmap.md) holds the direction and the index of feature specs. ## The six pillars @@ -65,10 +65,10 @@ MetaObjects pillars are capabilities of one metadata model, not separate product **Sharing a model across your own projects** — today the TypeScript toolchain publishes and TypeScript and Python projects consume, from an installed npm or Python package or over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends` and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. -## Implementations (npm `1.2.0` / Maven Central `8.2.0`) +## Implementations (npm `1.2.2` / Maven Central `8.2.2`) -- [TypeScript](https://github.com/metaobjectsdev/metaobjects/tree/main/server/typescript): the reference implementation. Published to npm at `1.2.0` across 14 `@metaobjectsdev/*` packages on the `latest` tag. 2500+ tests passing across the workspace. Bun-first dev workflow. Owns the canonical schema-migration toolchain used by every port (ADR-0015). -- [Java](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java): 14 publishable Maven modules under `com.metaobjects:*` at `8.2.0` on Maven Central (15 artifacts with the parent POM). Spring REST + DTO + repository-interface codegen, OMDB runtime persistence (pure data-access) with Spring-tx, full FR-003 + FR-006 + FR-010/011 + FR-018 + FR5 family. Fully green across all conformance corpora. +- [TypeScript](https://github.com/metaobjectsdev/metaobjects/tree/main/server/typescript): the reference implementation. Published to npm at `1.2.2` across 14 `@metaobjectsdev/*` packages on the `latest` tag. 2500+ tests passing across the workspace. Bun-first dev workflow. Owns the canonical schema-migration toolchain used by every port (ADR-0015). +- [Java](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java): 14 publishable Maven modules under `com.metaobjects:*` at `8.2.2` on Maven Central (15 artifacts with the parent POM). Spring REST + DTO + repository-interface codegen, OMDB runtime persistence (pure data-access) with Spring-tx, full FR-003 + FR-006 + FR-010/011 + FR-018 + FR5 family. Fully green across all conformance corpora. - [Kotlin](https://github.com/metaobjectsdev/metaobjects/tree/main/server/java/codegen-kotlin): a Kotlin codegen pipeline on KotlinPoet plus `metadata-ktx` — a Kotlin facade over the Java MetaObjects core. Ships via the Java reactor (`metaobjects-metadata-ktx`, `metaobjects-codegen-kotlin`). Persistence-conformance runs against Testcontainers Postgres through Exposed. - [C#](https://github.com/metaobjectsdev/metaobjects/tree/main/server/csharp): loader + canonical serializer + EF Core + ASP.NET codegen + render/verify. The `dotnet meta` tool ships `gen` and the codegen-side `verify` subverbs (`--codegen`, `--templates`); schema migration is Node `meta` only (ADR-0015). - [Python](https://github.com/metaobjectsdev/metaobjects/tree/main/server/python): loader + canonical serializer + conformance + render + verify + codegen + an `ObjectManager` runtime layer. The `metaobjects` console-script ships `gen` and codegen-side `verify`. Fully green across all conformance corpora. @@ -82,7 +82,7 @@ MetaObjects deliberately does **not** ship one universal binary. Schema operatio - Codegen is **opt-in**: no port ships a default generator suite, `meta init` scaffolds `generators: []`, and `meta gen --list` is the catalog (`--probe` reports how many files each generator would emit for your model) - Project marker directory: `.metaobjects/` - Install (TypeScript reference): `npm install -D @metaobjectsdev/cli` (or `bun add @metaobjectsdev/cli`) -- Install (Java): `com.metaobjectsmetaobjects-metadata8.2.0` plus per-concern artifacts (`metaobjects-codegen-spring`, `metaobjects-omdb`, `metaobjects-maven-plugin`, etc.) +- Install (Java): `com.metaobjectsmetaobjects-metadata8.2.2` plus per-concern artifacts (`metaobjects-codegen-spring`, `metaobjects-omdb`, `metaobjects-maven-plugin`, etc.) - Install (Kotlin): same Maven coordinates — add `metaobjects-metadata-ktx` and `metaobjects-codegen-kotlin` to a JVM/Kotlin project. - Install (Python): `pip install metaobjects` - Install (C#): the MetaObjects .NET tool, invoked as `dotnet meta`. @@ -128,7 +128,7 @@ Universal web-client under `client/web/packages/`: ## Java / Kotlin Maven coordinates -All published to Maven Central under `com.metaobjects:*` at `8.2.0`: +All published to Maven Central under `com.metaobjects:*` at `8.2.2`: - `metaobjects-metadata`, `metaobjects-metadata-ktx` - `metaobjects-codegen-base`, `metaobjects-codegen-mustache`, `metaobjects-codegen-spring`, `metaobjects-codegen-kotlin`, `metaobjects-codegen-plantuml` diff --git a/examples/advanced-modeling/codegen/templates/zod-validators.ts b/examples/advanced-modeling/codegen/templates/zod-validators.ts index ea4945feae..1aafe1a916 100644 --- a/examples/advanced-modeling/codegen/templates/zod-validators.ts +++ b/examples/advanced-modeling/codegen/templates/zod-validators.ts @@ -386,6 +386,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { * The field SET the `UpdateSchema` accepts — same iteration + skip rules * as `insertSchemaFields`, but mirroring the UpdateSchema branch of * `renderZodValidators`: + * • a primary-key field is OMITTED, generated or assigned (the URL names the row); * • a TPH subtype's @discriminator field is OMITTED (clients can't change subtype); * • @autoSet onCreate fields are OMITTED (creation timestamps are immutable); * • @autoSet onUpdate fields are present + optional (server fills them); @@ -393,6 +394,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of obj.fields()) { @@ -408,6 +410,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): a primary-key field is the row's identity, which the URL names. An + // ASSIGNED key leaves the update shape exactly like a generated one, so a client-sent + // key is stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -491,6 +497,10 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): the row's identity is the URL's. An ASSIGNED primary key leaves the + // update shape too (a generated one never entered it), so a client-sent key is stripped + // on PUT/PATCH and can never re-key the row. Mirrors updateSchemaFields. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { @@ -515,10 +525,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. PK fields never reach this branch (skipped above). + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index 131a0265b9..84023e786a 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -4,7 +4,7 @@ "generated/csharp/SubscriberBlurbPayload.g.cs": "29eb3a6ce64496ffe734cba55d0213994316b4281d3fa6791917393bae36c9ed", "generated/csharp/SubscriberFilterAllowlist.g.cs": "3839b04884e4e393d4f7d7bf4c7d4fe5b3852bd3a92b1ecb282cb8935a3c98e5", "generated/csharp/SubscriberNames.g.cs": "c2cf0248c2ecde155ec2623604771e39bed23783603d6f18a2ed82967227a732", - "generated/csharp/SubscriberRoutes.g.cs": "15450e230d396a739d789340c42488da354c22265193b5ac61b074b0ba6fb1ff", + "generated/csharp/SubscriberRoutes.g.cs": "d05912d0a76b52334857796a42b2919be8709997d6c073c11d2c44590fe686a9", "generated/python/Subscriber.py": "9dd4c0acea47f99d1daf79930f9de2e400f30780ebce6c63f023be528a72b762", "generated/python/SubscriberBlurbPayload.py": "a5ef8a08e63efbe0f73e39a21a368cff5150c68cc8beeea6e8aab0c755f8bbdd", "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", diff --git a/examples/showcase/codegen/templates/zod-validators.ts b/examples/showcase/codegen/templates/zod-validators.ts index ea4945feae..1aafe1a916 100644 --- a/examples/showcase/codegen/templates/zod-validators.ts +++ b/examples/showcase/codegen/templates/zod-validators.ts @@ -386,6 +386,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { * The field SET the `UpdateSchema` accepts — same iteration + skip rules * as `insertSchemaFields`, but mirroring the UpdateSchema branch of * `renderZodValidators`: + * • a primary-key field is OMITTED, generated or assigned (the URL names the row); * • a TPH subtype's @discriminator field is OMITTED (clients can't change subtype); * • @autoSet onCreate fields are OMITTED (creation timestamps are immutable); * • @autoSet onUpdate fields are present + optional (server fills them); @@ -393,6 +394,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of obj.fields()) { @@ -408,6 +410,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): a primary-key field is the row's identity, which the URL names. An + // ASSIGNED key leaves the update shape exactly like a generated one, so a client-sent + // key is stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -491,6 +497,10 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): the row's identity is the URL's. An ASSIGNED primary key leaves the + // update shape too (a generated one never entered it), so a client-sent key is stripped + // on PUT/PATCH and can never re-key the row. Mirrors updateSchemaFields. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { @@ -515,10 +525,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. PK fields never reach this branch (skipped above). + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } diff --git a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs index ffb1ed369c..9987f12b3c 100644 --- a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs +++ b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs @@ -100,6 +100,8 @@ await db.Subscribers.FindAsync(id) is { } found try { input = System.Text.Json.JsonSerializer.Deserialize(__body.RootElement.GetRawText(), jsonOpts); } catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = "validation" }); } if (input is null) return Results.BadRequest(new { error = "validation" }); + // Server-owned: a generated key, the TPH discriminator and M:N navigations are not settable by the client; the value is ignored. + input.Id = default!; var __vr = new System.Collections.Generic.List(); if (!System.ComponentModel.DataAnnotations.Validator.TryValidateObject( input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true)) diff --git a/examples/showcase/jvm/pom.xml b/examples/showcase/jvm/pom.xml index f41163da7d..6dcd77e740 100644 --- a/examples/showcase/jvm/pom.xml +++ b/examples/showcase/jvm/pom.xml @@ -23,7 +23,7 @@ - 8.2.0 + 8.2.2 UTF-8 diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index 237e16a881..6667d82a4f 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -1,9 +1,9 @@ { "registries": { - "npm": "1.2.0", + "npm": "1.2.2", "pypi": "1.2.1", - "nuget": "1.2.0", - "maven": "8.2.0", + "nuget": "1.2.2", + "maven": "8.2.2", "metamodel": "1.2" }, "counts": { diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 7574beeab5..da4c8f44e1 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -35,6 +35,7 @@ fixtures/api-contract-conformance/ ├── patch-empty-noop.yaml # FR-037 PATCH-5 a patch stripping to zero assignments is a no-op read-back ├── writeonce-patch-stripped.yaml # FR-037 R1 @mutability:"writeOnce" — stripped on PATCH, never rejected ├── autoset-patch.yaml # #203 @autoSet — PATCH bumps onUpdate, leaves onCreate alone + ├── input-server-owned-dropped.yaml # 1.2.x security: a client id / undeclared key never reaches storage ├── delete-204-and-404.yaml ├── invalid-sort-400.yaml ├── filter-eq.yaml # FR-009 filter operators @@ -52,13 +53,13 @@ fixtures/api-contract-conformance/ └── filter-invalid-value.yaml # uncoercible value → 400; the fourth envelope, previously ungated ``` -Beside the 31 core scenarios above sit eight **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (88 scenarios in all, 119 with the +Beside the 32 core scenarios above sit eight **sub-corpora**, each a directory with its +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (90 scenarios in all, 122 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| -| `tph/` | 10 | single-table inheritance | both | +| `tph/` | 12 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | | `jsonb/` | 5 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | @@ -121,6 +122,7 @@ requests: | `empty` | the response body is empty / null (204 No Content) | | `fieldsEqual`| the response body is an object; assert the listed field names are all equal to **each other** (raw, field-vs-field — no literal, so timestamp non-determinism is a non-issue). Used by the `@autoSet` gate (#203/ADR-0045). | | `fieldsNotEqual`| the response body is an object; assert the two listed fields **differ** from each other. Used by the `@autoSet` gate (e.g. after a PATCH, `autoCreatedAt` (onCreate, preserved) != `autoUpdatedAt` (onUpdate, bumped)). | +| `unset` | the response body is an object; each listed key is **missing or `null`** on it. Used by the input scenarios (`input-server-owned-dropped`, `tph-input-server-owned-dropped`): a value the client was not allowed to set never reached storage, so it never comes back in the echo. Missing and `null` are both accepted because a port's row shape decides whether an unset column is present as `null`. | Runners are responsible for normalizing `createdAt` (and any other non-deterministic fields) before comparison. The keys listed above are diff --git a/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml new file mode 100644 index 0000000000..597ed1b0cc --- /dev/null +++ b/fixtures/api-contract-conformance/scenarios/input-server-owned-dropped.yaml @@ -0,0 +1,85 @@ +name: input-server-owned-dropped +description: > + A write route hands storage only what the client may set (the 1.2.x security fix). A + client-sent `id` on a server-generated key, a key the entity does not declare (`isAdmin`) + and, on update, a `writeOnce` field are all DROPPED SILENTLY: the request answers as if the + key were absent (201/200, never 400), and none of them reaches storage. This is the rule + TypeScript has always followed and Python adopted in 1.2.1. + + Each body carries an ordinary settable field DELIBERATELY: it is applied, and that pairing + is what makes the assertion about the server-owned keys and not about the request being + ignored whole. `unset` asserts the undeclared key is not echoed back (a stored key would + be), and the GETs of the client's ids prove no row was written or re-keyed under them. +requests: + - id: r1 + method: POST + path: /api/authors + body: + id: 999 + name: "Barbara Liskov" + createdAt: "2026-02-01T10:00:00" + isAdmin: true + expect: + status: 201 + body: + row: + name: "Barbara Liskov" + hasId: true + unset: [isAdmin] + - id: r2 + method: GET + path: /api/authors/999 + expect: + status: 404 + - id: r3 + method: PATCH + path: /api/authors/1 + body: + id: 998 + name: "Ada L. (input probe)" + isAdmin: true + expect: + status: 200 + body: + row: + id: 1 + name: "Ada L. (input probe)" + unset: [isAdmin] + - id: r4 + method: PUT + path: /api/authors/2 + body: + id: 997 + name: "Alan T. (input probe)" + createdAt: "2026-01-02T10:00:00" + issuedCurrency: "JPY" + isAdmin: true + expect: + status: 200 + body: + row: + id: 2 + name: "Alan T. (input probe)" + # writeOnce: frozen after create, so the PUT's value never lands. + issuedCurrency: "GBP" + unset: [isAdmin] + - id: r5 + method: GET + path: /api/authors/998 + expect: + status: 404 + - id: r6 + method: GET + path: /api/authors/997 + expect: + status: 404 + - id: r7 + method: GET + path: /api/authors/1 + expect: + status: 200 + body: + row: + id: 1 + name: "Ada L. (input probe)" + unset: [isAdmin] diff --git a/fixtures/api-contract-conformance/secret-fields/README.md b/fixtures/api-contract-conformance/secret-fields/README.md index 29bc632038..ef23edf18d 100644 --- a/fixtures/api-contract-conformance/secret-fields/README.md +++ b/fixtures/api-contract-conformance/secret-fields/README.md @@ -1,8 +1,8 @@ # `api-contract-conformance/secret-fields/` — #484 credentials and excluded fields Cross-port REST contract for the fields a generated API must not expose or accept: a -field excluded from forms (`@formExclude`) and a credential (a field that declares a -`view.password`). +field excluded from forms (`@formExclude`), a credential (a field that declares a +`view.password`) and a server-owned `@mutability: readOnly` field. The model is one `Account`: @@ -11,6 +11,7 @@ The model is one `Account`: | `name` | (nothing) | accepted | returned | | `tokenHash` | `@formExclude: true` | **ignored** | returned | | `pinHash` | child `view.password` | **ignored** | **never returned** | +| `role` | `@mutability: readOnly` | **ignored** | returned | ## The contract diff --git a/fixtures/api-contract-conformance/secret-fields/meta.json b/fixtures/api-contract-conformance/secret-fields/meta.json index d47906d8ad..d000a949a6 100644 --- a/fixtures/api-contract-conformance/secret-fields/meta.json +++ b/fixtures/api-contract-conformance/secret-fields/meta.json @@ -10,6 +10,7 @@ { "field.string": { "name": "name", "@required": true, "@maxLength": 100, "@filterable": true, "@sortable": true } }, { "field.string": { "name": "tokenHash", "@maxLength": 100, "@formExclude": true } }, { "field.string": { "name": "pinHash", "@maxLength": 100, "children": [ { "view.password": {} } ] } }, + { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } ] }} diff --git a/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml b/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml index 92f48eb50f..f62e03a4dc 100644 --- a/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml +++ b/fixtures/api-contract-conformance/secret-fields/scenarios/read-omits-credential.yaml @@ -18,6 +18,7 @@ requests: id: 1 name: "Ada Lovelace" tokenHash: "token-hash-1" + role: "member" - id: r2 method: GET path: /api/accounts?sort=id @@ -28,6 +29,8 @@ requests: - id: 1 name: "Ada Lovelace" tokenHash: "token-hash-1" + role: "member" - id: 2 name: "Alan Turing" tokenHash: "token-hash-2" + role: "owner" diff --git a/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml b/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml index 05f5e36306..d0c81f9a47 100644 --- a/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml +++ b/fixtures/api-contract-conformance/secret-fields/scenarios/write-ignores-excluded.yaml @@ -1,8 +1,8 @@ name: write-ignores-excluded description: > - A field excluded from forms (`@formExclude`) or marked as a credential (`view.password`) - is not writable through the generated routes: create, PATCH and PUT all leave it as it - was. The value is IGNORED, never rejected (HTTP 200/201, not 400), the same rule as + A field excluded from forms (`@formExclude`), marked as a credential (`view.password`) or + declared `@mutability: readOnly` (`role`) is not writable through the generated routes: + create, PATCH and PUT all leave it as it was. The value is IGNORED, never rejected (HTTP 200/201, not 400), the same rule as `@mutability: writeOnce` on PATCH (`writeonce-patch-stripped`), because a generated edit form submits every field it was given. @@ -19,6 +19,7 @@ requests: name: "Ada L. (probe)" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 200 body: @@ -26,6 +27,7 @@ requests: id: 1 name: "Ada L. (probe)" tokenHash: "token-hash-1" + role: "member" - id: r2 method: PUT path: /api/accounts/2 @@ -33,6 +35,7 @@ requests: name: "Alan T. (probe)" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 200 body: @@ -40,6 +43,7 @@ requests: id: 2 name: "Alan T. (probe)" tokenHash: "token-hash-2" + role: "owner" - id: r3 method: POST path: /api/accounts @@ -47,6 +51,7 @@ requests: name: "Grace Hopper" tokenHash: "forged" pinHash: "forged" + role: "admin" expect: status: 201 body: @@ -54,6 +59,8 @@ requests: name: "Grace Hopper" # Never written: the column stays null, and `forged` never landed. tokenHash: null + # readOnly is server-owned: a create cannot set it either (1.2.x security fix). + role: null hasId: true - id: r4 method: PATCH @@ -63,6 +70,7 @@ requests: name: "Ada L. (probe 2)" tokenHash: null pinHash: null + role: null expect: status: 200 body: @@ -70,3 +78,4 @@ requests: id: 1 name: "Ada L. (probe 2)" tokenHash: "token-hash-1" + role: "member" diff --git a/fixtures/api-contract-conformance/secret-fields/seed.json b/fixtures/api-contract-conformance/secret-fields/seed.json index 6697ba1fcb..13b7225941 100644 --- a/fixtures/api-contract-conformance/secret-fields/seed.json +++ b/fixtures/api-contract-conformance/secret-fields/seed.json @@ -1,6 +1,6 @@ { "rows": [ - { "id": 1, "name": "Ada Lovelace", "tokenHash": "token-hash-1", "pinHash": "pin-hash-1" }, - { "id": 2, "name": "Alan Turing", "tokenHash": "token-hash-2", "pinHash": "pin-hash-2" } + { "id": 1, "name": "Ada Lovelace", "tokenHash": "token-hash-1", "pinHash": "pin-hash-1", "role": "member" }, + { "id": 2, "name": "Alan Turing", "tokenHash": "token-hash-2", "pinHash": "pin-hash-2", "role": "owner" } ] } diff --git a/fixtures/api-contract-conformance/tph/README.md b/fixtures/api-contract-conformance/tph/README.md index 5c45a2231f..86a18dbb4e 100644 --- a/fixtures/api-contract-conformance/tph/README.md +++ b/fixtures/api-contract-conformance/tph/README.md @@ -49,7 +49,9 @@ tph/ ├── tph-writeonce-patch-stripped.yaml # FR-037: inherited @mutability "writeOnce" stripped on the per-subtype PATCH ├── tph-update-explicit-null-clears.yaml # FR-035 tristate: present null clears; omitted @required untouched; null on @required → 400 ├── tph-create-constraint-violation.yaml # FR-036: per-subtype CREATE validates against the annotated Dto - └── tph-update-constraint-violation.yaml # FR-036: per-subtype PATCH validates each PRESENT value + ├── tph-update-constraint-violation.yaml # FR-036: per-subtype PATCH validates each PRESENT value + ├── tph-input-server-owned-dropped.yaml # 1.2.x security: client id, sibling column, undeclared key never stored + └── tph-input-foreign-discriminator-dropped.yaml # 1.2.x security: the subtype comes from the URL, never the body ``` `meta.json` declares the `acme::auth` package: diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml new file mode 100644 index 0000000000..e4290f706d --- /dev/null +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml @@ -0,0 +1,44 @@ +name: tph-input-foreign-discriminator-dropped +description: > + A per-subtype route takes the row's subtype from the URL, never from the body (the 1.2.x + security fix). A `type` naming a different subtype, sent to the Bridge route, is DROPPED + SILENTLY on create and on update: the created row is a Bridge, no Copay row appears, and an + updated Bridge stays a Bridge. Python's generated TPH routes answer this body with 400 rather + than dropping it; that runner skips this scenario (see the Python codegen KNOWN_GAPS.md). +requests: + - id: r1 + method: POST + path: /api/auths/bridge + body: + type: "Copay" + reference: "REF-DISC" + quantity: 4 + expect: + status: 201 + body: + row: { type: "Bridge", reference: "REF-DISC", quantity: 4 } + hasId: true + - id: r2 + method: GET + path: /api/auths/copay?filter[reference][eq]=REF-DISC + expect: + status: 200 + body: + length: 0 + - id: r3 + method: PATCH + path: /api/auths/bridge/1 + body: + type: "Copay" + reference: "REF-1-DISC" + expect: + status: 200 + body: + row: { id: 1, type: "Bridge", reference: "REF-1-DISC" } + - id: r4 + method: GET + path: /api/auths/copay?filter[reference][eq]=REF-1-DISC + expect: + status: 200 + body: + length: 0 diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml new file mode 100644 index 0000000000..f70f85e341 --- /dev/null +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-input-server-owned-dropped.yaml @@ -0,0 +1,72 @@ +name: tph-input-server-owned-dropped +description: > + The per-subtype write routes hand storage only what the client may set, the same rule as + `input-server-owned-dropped` in the core corpus (the 1.2.x security fix). A client-sent `id` + on the generated key, a key the hierarchy does not declare (`isAdmin`), a column only a + SIBLING subtype declares (`copayAmount` sent to the Bridge route) and, on update, a + `writeOnce` field are all DROPPED SILENTLY (201/200, never 400): the row keeps its + server-assigned id and nothing it may not set is echoed back. A body discriminator is + `tph-input-foreign-discriminator-dropped`. +requests: + - id: r1 + method: POST + path: /api/auths/bridge + body: + id: 999 + reference: "REF-INPUT" + quantity: 3 + copayAmount: "7.77" + isAdmin: true + expect: + status: 201 + body: + row: { type: "Bridge", reference: "REF-INPUT", quantity: 3 } + hasId: true + unset: [isAdmin, copayAmount] + - id: r2 + method: GET + path: /api/auths/999 + expect: + status: 404 + - id: r3 + method: GET + path: /api/auths/copay?filter[copayAmount][eq]=7.77 + expect: + status: 200 + body: + length: 0 + - id: r4 + method: PATCH + path: /api/auths/bridge/1 + body: + id: 998 + reference: "REF-1-INPUT" + issuedCurrency: "JPY" + isAdmin: true + expect: + status: 200 + body: + row: { id: 1, type: "Bridge", reference: "REF-1-INPUT", issuedCurrency: "USD" } + unset: [isAdmin] + - id: r5 + method: PUT + path: /api/auths/copay/2 + body: + id: 997 + reference: "REF-2-INPUT" + isAdmin: true + expect: + status: 200 + body: + row: { id: 2, type: "Copay", reference: "REF-2-INPUT" } + unset: [isAdmin] + - id: r6 + method: GET + path: /api/auths/998 + expect: + status: 404 + - id: r7 + method: GET + path: /api/auths/997 + expect: + status: 404 diff --git a/server/csharp/Directory.Build.props b/server/csharp/Directory.Build.props index 64e1f7c16b..9c9c41a6c0 100644 --- a/server/csharp/Directory.Build.props +++ b/server/csharp/Directory.Build.props @@ -8,7 +8,7 @@ --> - 1.2.0 + 1.2.2 Doug Mealing Doug Mealing LLC MetaObjects diff --git a/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs new file mode 100644 index 0000000000..aba045fa26 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ServerOwnedCreateInputTests.cs @@ -0,0 +1,236 @@ +// ServerOwnedCreateInputTests — a generated create hands storage ONLY what the client may set. +// The HTTP behaviour is gated by the api-contract corpus (`input-server-owned-dropped`, +// `tph-input-server-owned-dropped`, secret-fields); this pins the emission: the generated key, +// the TPH discriminator and the M:N navigations are reset before `Add`, and a readOnly field of +// EVERY shape (including a TPH subtype's own) is declared with a private setter. + +using MetaObjects.Codegen; +using MetaObjects.Codegen.Generators; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ServerOwnedCreateInputTests +{ + private const string PlainModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Post", "children": [ + { "source.rdb": { "@table": "posts" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@maxLength": 100 } }, + { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, + { "field.enum": { "name": "stage", "@values": ["Draft", "Live"], "@mutability": "readOnly" } }, + { "relationship.association": { "name": "tags", "@cardinality": "many", "@objectRef": "Tag", "@through": "PostTag" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "Tag", "children": [ + { "source.rdb": { "@table": "tags" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "PostTag", "children": [ + { "source.rdb": { "@table": "post_tags" } }, + { "field.long": { "name": "postId" } }, + { "field.long": { "name": "tagId" } }, + { "identity.primary": { "@fields": ["postId", "tagId"] } }, + { "identity.reference": { "name": "fkPost", "@fields": "postId", "@references": "Post" } }, + { "identity.reference": { "name": "fkTag", "@fields": "tagId", "@references": "Tag" } } + ]}} + ]}} + """; + + private const string AssignedKeyModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Code", "children": [ + { "source.rdb": { "@table": "codes" } }, + { "field.string": { "name": "id", "@maxLength": 20 } }, + { "identity.primary": { "@fields": "id", "@generation": "assigned" } } + ]}} + ]}} + """; + + private const string TphModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Thing", "@discriminator": "kind", "children": [ + { "source.rdb": { "@table": "things" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "kind", "@values": ["A", "B"] } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "AThing", "extends": "Thing", "@discriminatorValue": "A", "children": [ + { "field.string": { "name": "note", "@maxLength": 20 } }, + { "field.string": { "name": "grade", "@maxLength": 20, "@mutability": "readOnly" } } + ]}}, + { "object.entity": { "name": "BThing", "extends": "Thing", "@discriminatorValue": "B", "children": [ + { "field.int": { "name": "qty" } } + ]}} + ]}} + """; + + private const string RequiredReadOnlyModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Member", "children": [ + { "source.rdb": { "@table": "members" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true, "@maxLength": 100 } }, + { "field.string": { "name": "role", "@required": true, "@maxLength": 20, "@mutability": "readOnly" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}} + ]}} + """; + + // A legal way to say "the server owns this": the generated increment key and the TPH + // discriminator are themselves @mutability: readOnly. The entity then declares both with + // PRIVATE setters (A_readOnly_field_of_every_shape_has_a_private_setter), so a create that + // reset them (`input.Id = default!;`) would not compile in the adopter's build — CS0122. + private const string ReadOnlyKeyAndDiscriminatorModel = """ + { "metadata.root": { "package": "acme::input", "children": [ + { "object.entity": { "name": "Seal", "@discriminator": "kind", "children": [ + { "source.rdb": { "@table": "seals" } }, + { "field.long": { "name": "id", "@mutability": "readOnly" } }, + { "field.enum": { "name": "kind", "@values": ["A", "B"], "@mutability": "readOnly" } }, + { "field.string": { "name": "label", "@maxLength": 40 } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ]}}, + { "object.entity": { "name": "ASeal", "extends": "Seal", "@discriminatorValue": "A", "children": [ + { "field.int": { "name": "qty" } } + ]}}, + { "object.entity": { "name": "BSeal", "extends": "Seal", "@discriminatorValue": "B", "children": [ + { "field.string": { "name": "note", "@maxLength": 40 } } + ]}} + ]}} + """; + + private static string Emit(string model, string generator, string file) + { + var r = new MetaDataLoader().Load([new InMemoryStringSource(model, id: "input.json")]); + Assert.Empty(r.Errors); + var ctx = new GenContext + { + Entities = r.Root.Objects(), + Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = ColumnNamingStrategy.Literal }, + }; + IGenerator g = generator == "routes" ? new RoutesGenerator() : new EntityGenerator(); + return g.Generate(ctx).Single(f => f.Path == file).Content; + } + + [Fact] + public void Create_resets_a_generated_key_and_empties_navigations_before_Add() + { + var src = Emit(PlainModel, "routes", "PostRoutes.g.cs"); + var post = src[src.IndexOf("app.MapPost(", StringComparison.Ordinal)..]; + var reset = post.IndexOf("input.Id = default!;", StringComparison.Ordinal); + var tags = post.IndexOf("input.Tags = new System.Collections.Generic.List();", StringComparison.Ordinal); + var add = post.IndexOf("db.Posts.Add(input);", StringComparison.Ordinal); + Assert.True(reset >= 0 && tags >= 0 && add > reset && add > tags, post); + } + + [Fact] + public void An_assigned_key_is_the_callers_and_is_not_reset() + { + var src = Emit(AssignedKeyModel, "routes", "CodeRoutes.g.cs"); + Assert.DoesNotContain("input.Id = default!;", src); + Assert.DoesNotContain("Server-owned", src); + } + + [Fact] + public void A_readOnly_field_of_every_shape_has_a_private_setter() + { + var src = Emit(PlainModel, "entity", "Post.g.cs"); + Assert.Contains("Role { get; private set; }", src); + Assert.Contains("Stage { get; private set; }", src); + Assert.Contains("Title { get; set; }", src); + } + + [Fact] + public void A_tph_subtype_readOnly_field_has_a_private_setter() + { + var src = Emit(TphModel, "entity", "AThing.g.cs"); + Assert.Contains("Grade { get; private set; }", src); + Assert.Contains("Note { get; set; }", src); + } + + [Fact] + public void A_required_readOnly_field_is_not_demanded_of_the_create_body() + { + // The client cannot set `role` (private setter), so the create must neither require its + // key nor reject the body over its [Required]: otherwise the entity could not be created. + var src = Emit(RequiredReadOnlyModel, "routes", "MemberRoutes.g.cs"); + var post = src[src.IndexOf("app.MapPost(", StringComparison.Ordinal)..]; + var req = post[post.IndexOf("foreach (var __req in", StringComparison.Ordinal)..]; + req = req[..req.IndexOf('\n')]; + Assert.Contains("\"title\"", req); + Assert.DoesNotContain("\"role\"", req); + Assert.Contains("System.Array.IndexOf(new[] { \"Role\" }, m) < 0", post); + var syntaxErrors = CSharpSyntaxTree.ParseText(src).GetDiagnostics() + .Where(d => d.Severity == DiagnosticSeverity.Error).Select(d => d.ToString()).ToList(); + Assert.True(syntaxErrors.Count == 0, string.Join("\n", syntaxErrors)); + } + + [Fact] + public void Tph_subtype_create_resets_the_key_and_pins_the_discriminator_to_the_route_subtype() + { + var src = Emit(TphModel, "routes", "ThingRoutes.g.cs"); + var start = src.IndexOf("app.MapPost(prefix + \"/things/a\"", StringComparison.Ordinal); + var end = src.IndexOf("app.MapPost(prefix + \"/things/b\"", StringComparison.Ordinal); + Assert.True(start >= 0 && end > start, src); + var a = src[start..end]; + Assert.Contains("input.Id = default!;", a); + Assert.Contains("input.Kind = default!;", a); + Assert.True(a.IndexOf("input.Kind = default!;", StringComparison.Ordinal) < a.IndexOf(".Add(input);", StringComparison.Ordinal), a); + } + + [Fact] + public void A_readOnly_key_and_readOnly_discriminator_get_no_reset_and_the_model_compiles() + { + // System.Text.Json never binds a private setter, so the resets would be redundant — + // and CS0122. Oracle: the Roslyn compile of the generated tree (the adopter's build); + // the wrong behavior rejected is `input.Id = default!;` / `input.Kind = default!;` + // assigning private-setter properties. + var r = new MetaDataLoader().Load([new InMemoryStringSource(ReadOnlyKeyAndDiscriminatorModel, id: "ro-key.json")]); + Assert.Empty(r.Errors); + var ctx = new GenContext + { + Entities = r.Root.Objects(), + Root = r.Root, + Config = new GenConfig { OutDir = "/tmp", Namespace = "Acme.Generated", ColumnNamingStrategy = ColumnNamingStrategy.Literal }, + }; + var sources = new EntityGenerator().Generate(ctx) + .Concat(new DbContextGenerator().Generate(ctx)) + .Concat(new FilterAllowlistGenerator().Generate(ctx)) + .Concat(new RoutesGenerator().Generate(ctx)) + .ToList(); + var routes = sources.Single(f => f.Path == "SealRoutes.g.cs").Content; + var post = routes[routes.IndexOf("app.MapPost(", StringComparison.Ordinal)..]; + Assert.DoesNotContain("input.Id = default!;", post); + Assert.DoesNotContain("input.Kind = default!;", post); + + var trees = sources + .Select(f => CSharpSyntaxTree.ParseText(f.Content, new CSharpParseOptions(LanguageVersion.CSharp12), path: f.Path)) + .ToList(); + var paths = new HashSet(StringComparer.OrdinalIgnoreCase); + var tpa = (string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!; + foreach (var p in tpa.Split(Path.PathSeparator)) if (p.Length > 0) paths.Add(p); + paths.Add(typeof(Microsoft.EntityFrameworkCore.DbContext).Assembly.Location); + paths.Add(typeof(Microsoft.EntityFrameworkCore.ModelBuilder).Assembly.Location); + paths.Add(typeof(Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions).Assembly.Location); + paths.Add(typeof(Microsoft.EntityFrameworkCore.PrimaryKeyAttribute).Assembly.Location); + paths.Add(typeof(MetaObjects.Codegen.Runtime.FilterParser).Assembly.Location); + var refs = paths.Where(File.Exists) + .Select(p => (MetadataReference)MetadataReference.CreateFromFile(p)).ToList(); + var comp = CSharpCompilation.Create( + "ro_key_disc_compile_" + Guid.NewGuid().ToString("N"), trees, refs, + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + var errors = comp.GetDiagnostics() + .Where(d => d.Severity == DiagnosticSeverity.Error) + .Select(d => $"{d.Id}: {d.GetMessage()} @ {d.Location}") + .ToList(); + Assert.True(errors.Count == 0, + "readOnly key + readOnly discriminator model must compile:\n" + string.Join("\n", errors)); + } +} diff --git a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs index 810c3c78e7..b278fac020 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs @@ -626,9 +626,9 @@ protected virtual string TphSubtypeScalarProperty( // A NON-required decimal array is nullable (no initializer) so PATCH null / an omitted // POST value stays a NULL column instead of failing as a required collection. if (IsDecimalWireField(field) && !CSharpNaming.IsRequired(entity, field)) - arr.Append($" public IList<{baseType}>? {propName} {{ get; set; }}"); + arr.Append($" public IList<{baseType}>? {propName} {{ {SetterFor(field)} }}"); else - arr.Append($" public IList<{baseType}> {propName} {{ get; set; }} = new List<{baseType}>();"); + arr.Append($" public IList<{baseType}> {propName} {{ {SetterFor(field)} }} = new List<{baseType}>();"); return arr.ToString(); } @@ -653,7 +653,7 @@ protected virtual string TphSubtypeScalarProperty( var init = defaultInit.Length > 0 ? defaultInit : (required && !isValue ? " = default!;" : string.Empty); - sb.Append($" public {baseType}{nullable} {propName} {{ get; set; }}{init}"); + sb.Append($" public {baseType}{nullable} {propName} {{ {SetterFor(field)} }}{init}"); return sb.ToString(); } @@ -665,9 +665,9 @@ protected virtual string TphSubtypeEnumProperty(MetaObject entity, MetaField fie var propName = PropertyName(field); var colAttr = $" [Column({CSharpNaming.ColumnRef(entity, field, strategy, config.IncludeNames)})]\n"; if (field.ResolvedIsArray()) // ADR-0039: resolving — array-ness inheritable via extends - return $"{colAttr} public IList<{typeName}> {propName} {{ get; set; }} = new List<{typeName}>();"; + return $"{colAttr} public IList<{typeName}> {propName} {{ {SetterFor(field)} }} = new List<{typeName}>();"; var nullable = CSharpNaming.IsRequired(entity, field) ? "" : "?"; - return $"{colAttr} public {typeName}{nullable} {propName} {{ get; set; }}"; + return $"{colAttr} public {typeName}{nullable} {propName} {{ {SetterFor(field)} }}"; } // The directly-extended entity (the metadata `extends` target) of a TPH subtype — the @@ -1528,10 +1528,10 @@ protected virtual string EnumProperty( var propName = PropertyName(field); var colAttr = withAttributes ? $" [Column({CSharpNaming.ColumnRef(entity, field, strategy, config.IncludeNames)})]\n" : ""; if (field.ResolvedIsArray()) // ADR-0039: resolving — array-ness inheritable via extends - return $"{colAttr} public IList<{typeName}> {propName} {{ get; set; }} = new List<{typeName}>();"; + return $"{colAttr} public IList<{typeName}> {propName} {{ {SetterFor(field)} }} = new List<{typeName}>();"; var required = CSharpNaming.IsRequired(entity, field); var type = required ? typeName : typeName + "?"; - return $"{colAttr} public {type} {propName} {{ get; set; }}"; + return $"{colAttr} public {type} {propName} {{ {SetterFor(field)} }}"; } // A scalar property. withAttributes adds the EF-mapping annotations ([Key] for a @@ -1589,9 +1589,9 @@ protected virtual string ScalarProperty( // NON-required value-object array is nullable so a null member is omitted on the wire. var omittable = valueObject && _wire.NullMembers == NullMemberForm.Omit; if ((IsDecimalWireField(field) || omittable) && !CSharpNaming.IsRequired(owner, field)) - arr.Append($" public IList<{baseType}>? {propName} {{ get; set; }}"); + arr.Append($" public IList<{baseType}>? {propName} {{ {SetterFor(field)} }}"); else - arr.Append($" public IList<{baseType}> {propName} {{ get; set; }} = new List<{baseType}>();"); + arr.Append($" public IList<{baseType}> {propName} {{ {SetterFor(field)} }} = new List<{baseType}>();"); return arr.ToString(); } @@ -1660,11 +1660,21 @@ protected virtual string ScalarProperty( // the column from UPDATE). `writeOnce` keeps a PUBLIC setter on purpose — the caller // must be able to supply it on insert; freezing it afterwards is the DbContext's // SetAfterSaveBehavior(Ignore), not the property's job. - var setter = field.IsReadOnlyMutability ? "get; private set;" : "get; set;"; + var setter = SetterFor(field); sb.Append($" public {type} {propName} {{ {setter} }}{init}"); return sb.ToString(); } + // FR-037 R1 / SECURITY (1.2.x) — the accessor pair of a mapped property. A `readOnly` field + // takes a PRIVATE setter on EVERY property shape (scalar, enum, array, value object, map; plain + // entity or TPH subtype): EF materializes it on read and application code cannot write it, so + // the generated create route's body binding (System.Text.Json skips a non-public setter) can + // never hand a client-supplied value to storage. Before 1.2.x only the plain scalar shape did, + // so a readOnly enum, array, value-object, map or TPH-subtype field was writable on create. + // `writeOnce` keeps a PUBLIC setter on purpose (the caller supplies it on insert). + private static string SetterFor(MetaField field) => + field.IsReadOnlyMutability ? "get; private set;" : "get; set;"; + // String length + pattern DataAnnotations for a string scalar, combining the field // @maxLength attr with validator.length (@min/@max) and validator.regex (@pattern). // @@ -1889,11 +1899,11 @@ protected static string EscapeAttrString(string s) // null-vs-empty distinction (EF re-serializes the empty list to `[]` on write). if (field.ResolvedIsArray()) // ADR-0039: resolving — array-ness inheritable via extends return required - ? $" public ICollection<{typeName}> {propName} {{ get; set; }} = new List<{typeName}>();" - : $" public ICollection<{typeName}>? {propName} {{ get; set; }}"; + ? $" public ICollection<{typeName}> {propName} {{ {SetterFor(field)} }} = new List<{typeName}>();" + : $" public ICollection<{typeName}>? {propName} {{ {SetterFor(field)} }}"; return required - ? $" public {typeName} {propName} {{ get; set; }} = default!;" - : $" public {typeName}? {propName} {{ get; set; }}"; + ? $" public {typeName} {propName} {{ {SetterFor(field)} }} = default!;" + : $" public {typeName}? {propName} {{ {SetterFor(field)} }}"; } // A property for a field.map (an open-keyed map). Emits a single @@ -1922,8 +1932,8 @@ protected string MapProperty(MetaObject owner, MetaField field, GenContext ctx, var required = CSharpNaming.IsRequired(owner, field); var colAttr = withAttributes ? $" [Column({CSharpNaming.ColumnRef(owner, field, ctx.Config.ColumnNamingStrategy, ctx.Config.IncludeNames)})]\n" : ""; return required - ? $"{colAttr} public Dictionary {propName} {{ get; set; }} = new();" - : $"{colAttr} public Dictionary? {propName} {{ get; set; }}"; + ? $"{colAttr} public Dictionary {propName} {{ {SetterFor(field)} }} = new();" + : $"{colAttr} public Dictionary? {propName} {{ {SetterFor(field)} }}"; } // The CLR scalar type for a projection field whose value is a MIN/MAX aggregate: diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index f254e8ec63..067bcf96cf 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -418,7 +418,8 @@ protected virtual EmittedFile GenerateStandardRoutes(MetaObject entity, GenConte AppendCreateHandler(sb, "/" + route, cls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + route + "\"", "input", credentials), voFields, mapVoFields, autoSetFields, writeThrough ? (readDbSet, pkProp!, route) : null, parseBody: HasDecimalField(entity), - deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials); + deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials, + ServerOwnedCreateResets(entity, ctx.Root), RequiredReadOnlyProps(entity)); // PATCH + PUT share the same handler — TS reference exposes both verbs, and both // return the updated row (HTTP 200), matching the cross-port api-contract. @@ -595,6 +596,7 @@ protected virtual EmittedFile GenerateTphRoutes(MetaObject baseEntity, TphPlan t bool hasItem = pkType is not null; var discProp = CSharpNaming.Pascal(tph.DiscriminatorField); + var discField = baseEntity.FindField(tph.DiscriminatorField); // #484 — the polymorphic base rows carry every subtype's columns, so a credential declared // on the base OR on any subtype is omitted from them. var baseCredentials = CredentialNames(baseEntity) @@ -706,7 +708,7 @@ protected virtual EmittedFile GenerateTphRoutes(MetaObject baseEntity, TphPlan t // --- Per-subtype CRUD sets --- var hierarchy = tph.Subtypes.Select(s => s.Entity).Prepend(baseEntity).ToList(); foreach (var st in tph.Subtypes) - AppendTphSubtypeRoutes(sb, st, hierarchy, ctx.Root, baseRoute, baseCls, dbSet, pkType!, pkProp!, discProp, ctx.Warn); + AppendTphSubtypeRoutes(sb, st, hierarchy, ctx.Root, baseRoute, baseCls, dbSet, pkType!, pkProp!, discProp, discField, ctx.Warn); } sb.AppendLine(); @@ -756,10 +758,23 @@ private static List RequiredCreateKeys(MetaObject obj, Func f.IsRequired && f.AutoSet is null && !f.IsDerived() && !isPkOrDisc(f)) + // 1.2.x security — a readOnly field is never bound from the body (private setter), + // so demanding its key would make a @required readOnly entity uncreatable. + .Where(f => f.IsRequired && f.AutoSet is null && !f.IsDerived() && !f.IsReadOnlyMutability + && !isPkOrDisc(f)) .Select(f => f.Name) .ToList(); + // 1.2.x security — the CLR properties of the @required readOnly fields. The client cannot + // set them (private setter), so their [Required] describes the stored row, not the create + // body: the create validation ignores a violation that names only these members, as Java's + // SERVER_OWNED_ON_CREATE filter does. Empty for nearly every model → byte-identical output. + private static List RequiredReadOnlyProps(MetaObject obj) => + obj.Fields() + .Where(f => f.IsRequired && f.IsReadOnlyMutability && !f.IsDerived()) + .Select(f => CSharpNaming.Pascal(f.Name)) + .ToList(); + // Issue #203 — one @autoSet timestamp column the generated CRUD stamps with now(). // Nav is the PascalCase CLR property; NowExpr is the now() literal keyed off the // COLUMN's temporal CLR type (generalizes beyond DateTimeOffset). OnUpdate columns @@ -988,7 +1003,9 @@ private static void AppendCreateHandler( (string ReadDbSet, string PkProp, string Route)? reRead = null, bool parseBody = false, IReadOnlyList? deniedNavs = null, - IReadOnlyList? credentials = null) + IReadOnlyList? credentials = null, + IReadOnlyList? serverOwned = null, + IReadOnlyList? readOnlyRequired = null) { // A decimal field can refuse its value while the body is bound (out of declared range, not a // number). The framework's own binding answers that with a bare 400 and no body, so an entity @@ -997,8 +1014,9 @@ private static void AppendCreateHandler( { sb.AppendLine($" app.MapPost(prefix + \"{mapPath}\", async ({cls} input, AppDbContext db) =>"); sb.AppendLine(" {"); + AppendCreateServerOwned(sb, serverOwned); AppendCreateIgnoreDenied(sb, deniedNavs); - AppendCreateValidation(sb); + AppendCreateValidation(sb, readOnlyRequired); AppendCreateVoValidation(sb, voFields, mapVoFields); AppendCreateAutoSet(sb, autoSetFields); AppendCreatePersist(sb, dbSet, createdExpr, reRead, credentials ?? []); @@ -1033,8 +1051,9 @@ private static void AppendCreateHandler( sb.AppendLine(" try { input = System.Text.Json.JsonSerializer.Deserialize<" + cls + ">(__body.RootElement.GetRawText(), jsonOpts); }"); sb.AppendLine(" catch (System.Text.Json.JsonException) { return Results.BadRequest(new { error = \"validation\" }); }"); sb.AppendLine(" if (input is null) return Results.BadRequest(new { error = \"validation\" });"); + AppendCreateServerOwned(sb, serverOwned); AppendCreateIgnoreDenied(sb, deniedNavs); - AppendCreateValidation(sb); + AppendCreateValidation(sb, readOnlyRequired); AppendCreateVoValidation(sb, voFields, mapVoFields); AppendCreateAutoSet(sb, autoSetFields); AppendCreatePersist(sb, dbSet, createdExpr, reRead, credentials ?? []); @@ -1067,6 +1086,49 @@ private static void AppendCreatePersist( } } + // SECURITY (1.2.x) — a generated create hands storage ONLY what the client may set. The body is + // bound straight onto the EF entity, so every member the server owns is reset before validation + // and `Add`, and the request answers as if the key were absent (never a 400): + // - a server-generated primary key (@generation increment|uuid): default, so EF generates it; + // - the TPH discriminator on a per-subtype route: default, so EF's discriminator value + // generator stamps the route subtype's own @discriminatorValue (the route is its only source); + // - M:N navigation collections: emptied, else `Add` would insert a client-sent related graph. + // A readOnly field needs no line here: the entity declares it with a private setter, which + // System.Text.Json never binds (EntityGenerator.SetterFor). + private static void AppendCreateServerOwned(StringBuilder sb, IReadOnlyList? statements) + { + if (statements is null || statements.Count == 0) return; + sb.AppendLine(" // Server-owned: a generated key, the TPH discriminator and M:N navigations are not settable by the client; the value is ignored."); + foreach (var st in statements) sb.AppendLine(" " + st); + } + + // The reset statements AppendCreateServerOwned emits for one entity (empty when it has none, so an + // entity with an assigned key, no navigation and no discriminator keeps its output byte-identical). + // A TPH subtype class declares no M:N property of its own — the navigations it carries are its + // discriminator base class's — so a subtype route passes that base as navOwner. The key and + // discriminator resets are SKIPPED when that field is itself @mutability: readOnly: the entity + // declares it with a private setter (EntityGenerator.SetterFor), so the assignment would be + // CS0122 — and it is redundant, because System.Text.Json never binds a private setter. + private static List ServerOwnedCreateResets(MetaObject entity, MetaRoot root, MetaField? discField = null, MetaObject? navOwner = null) + { + var list = new List(); + if (entity.PrimaryIdentity() is MetaPrimaryIdentity pk + && pk.Generation is MetaObjects.Core.Identity.IdentityConstants.GENERATION_INCREMENT + or MetaObjects.Core.Identity.IdentityConstants.GENERATION_UUID + && pk.Fields is [var keyName] + && entity.Fields().FirstOrDefault(f => f.Name == keyName) is { } keyField + && !keyField.IsReadOnlyMutability) + list.Add($"input.{CSharpNaming.Pascal(keyField.Name)} = default!;"); + if (discField is not null && !discField.IsReadOnlyMutability) + list.Add($"input.{CSharpNaming.Pascal(discField.Name)} = default!;"); + foreach (var nav in M2MNavigationBuilder.For(navOwner ?? entity, root, _ => { })) + { + var target = CSharpNaming.Pascal(nav.Target.Name); + list.Add($"input.{CSharpNaming.Pascal(nav.Name)} = new System.Collections.Generic.List<{target}>();"); + } + return list; + } + // #484 — a write-denied key (@formExclude or credential) on a create body is IGNORED, never // rejected: the bound value is reset to the CLR default (unset / null) before validation. private static void AppendCreateIgnoreDenied(StringBuilder sb, IReadOnlyList? deniedNavs) @@ -1214,11 +1276,23 @@ private static void AppendJsonOptsLocal(StringBuilder sb) // bound `input` and rejects a violating body with the cross-port {error:"validation"} // envelope BEFORE the row is persisted (ASP.NET minimal-API model binding never runs // DataAnnotations on its own). Assumes the bound local `input` is in scope. - private static void AppendCreateValidation(StringBuilder sb) + // A violation naming only a @required readOnly member () + // is ignored: the client cannot set that member, so it cannot satisfy the rule. + private static void AppendCreateValidation(StringBuilder sb, IReadOnlyList? readOnlyRequired = null) { sb.AppendLine(" var __vr = new System.Collections.Generic.List();"); sb.AppendLine(" if (!System.ComponentModel.DataAnnotations.Validator.TryValidateObject("); - sb.AppendLine(" input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true))"); + if (readOnlyRequired is null || readOnlyRequired.Count == 0) + { + sb.AppendLine(" input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true))"); + } + else + { + var names = string.Join(", ", readOnlyRequired.Select(n => "\"" + n + "\"")); + sb.AppendLine(" input, new System.ComponentModel.DataAnnotations.ValidationContext(input), __vr, true)"); + sb.AppendLine($" && __vr.Exists(r => !System.Linq.Enumerable.Any(r.MemberNames)"); + sb.AppendLine($" || System.Linq.Enumerable.Any(r.MemberNames, m => System.Array.IndexOf(new[] {{ {names} }}, m) < 0)))"); + } sb.AppendLine(" return Results.BadRequest(new { error = \"validation\" });"); } @@ -1376,7 +1450,7 @@ private static void AppendPartialMergeLoop( // that never touches the PK or the discriminator. private static void AppendTphSubtypeRoutes( StringBuilder sb, TphSubtypePlan st, IReadOnlyList hierarchy, MetaRoot root, string baseRoute, string baseCls, string dbSet, - string pkType, string pkProp, string discProp, Action onWarn) + string pkType, string pkProp, string discProp, MetaField? discField, Action onWarn) { // #362 — field.object columns on a TPH subtype stay out of scope (Program D §6), but a // field.map @objectRef is NOT: it reaches the settable set, so its value objects reach @@ -1448,7 +1522,9 @@ private static void AppendTphSubtypeRoutes( AppendCreateHandler(sb, "/" + subRoute, subCls, dbSet, requiredKeys, CreatedResult("prefix + \"/" + subRoute + "/\" + input." + pkProp, "input", credentials), [], mapVoFields, autoSetFields, parseBody: HasDecimalField(st.Entity), - deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials); + deniedNavs: deniedFields.Select(f => CSharpNaming.Pascal(f.Name)).ToList(), credentials: credentials, + serverOwned: ServerOwnedCreateResets(st.Entity, root, discField, hierarchy[0]), + readOnlyRequired: RequiredReadOnlyProps(st.Entity)); // Per-subtype update (PATCH + PUT): partial merge of the JSON body onto the scoped // entity, NEVER touching the PK or the discriminator. Cross-subtype id → 404. diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs index e6bc127b54..13ca20b229 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractAssertions.cs @@ -151,6 +151,19 @@ public static void AssertResponse(string scenarioName, ApiRequest request, int s $"{scenarioName} / {request.Id}: expected row.{kvp.Key}={Render(kvp.Value)}, got {Render(actual)}"); } } + // Each listed key is missing or null: a value the client may not set never reached storage. + // (Missing-or-null, not strictly missing: ports differ on echoing an unset column as null.) + if (want.TryGetValue("unset", out var unsetObj) && unsetObj is List unsetKeys) + { + if (body is not Dictionary unsetRow) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {request.Id}: expected object for unset, got: {Render(body)}"); + var stored = unsetKeys.Select(k => k?.ToString() ?? "") + .Where(k => unsetRow.TryGetValue(k, out var v) && v is not null).ToList(); + if (stored.Count > 0) + throw new Xunit.Sdk.XunitException( + $"{scenarioName} / {request.Id}: expected {string.Join(", ", stored)} unset, got: {Render(body)}"); + } if (want.TryGetValue("hasId", out var hasIdObj) && hasIdObj is bool hi && hi) { if (body is not Dictionary row) diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs index 9314afaa29..09066f09e7 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractSecretFieldsConformanceTest.cs @@ -37,7 +37,7 @@ private static async Task AssertStoredSecretsUntouchedAsync(string connString) await using var c = new Npgsql.NpgsqlConnection(connString); await c.OpenAsync(); await using var cmd = c.CreateCommand(); - cmd.CommandText = "SELECT \"id\", \"tokenHash\", \"pinHash\" FROM \"accounts\" ORDER BY \"id\""; + cmd.CommandText = "SELECT \"id\", \"tokenHash\", \"pinHash\", \"role\" FROM \"accounts\" ORDER BY \"id\""; await using var reader = await cmd.ExecuteReaderAsync(); var seen = new HashSet(); while (await reader.ReadAsync()) @@ -45,16 +45,19 @@ private static async Task AssertStoredSecretsUntouchedAsync(string connString) long id = reader.GetInt64(0); string? token = reader.IsDBNull(1) ? null : reader.GetString(1); string? pin = reader.IsDBNull(2) ? null : reader.GetString(2); + string? role = reader.IsDBNull(3) ? null : reader.GetString(3); seen.Add(id); if (seed.TryGetValue(id, out var row)) { Assert.Equal(row["tokenHash"]!.GetValue(), token); Assert.Equal(row["pinHash"]!.GetValue(), pin); + Assert.Equal(row["role"]!.GetValue(), role); // readOnly: no write path reaches it } else { Assert.Null(token); Assert.Null(pin); + Assert.Null(role); } } Assert.True(seed.Keys.All(seen.Contains), "a seeded row went missing"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs index 2aaf7d81ca..7818486b16 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/SecretFieldsFixture.cs @@ -12,6 +12,7 @@ internal static class SecretFieldsFixture "name" VARCHAR(100) NOT NULL, "tokenHash" VARCHAR(100) NULL, "pinHash" VARCHAR(100) NULL, + "role" VARCHAR(20) NULL, CONSTRAINT "accounts_pkey" PRIMARY KEY ("id") ); """; @@ -29,7 +30,7 @@ public static async Task ApplySeedAsync(string connString, string seedPath) { var root = JsonNode.Parse(File.ReadAllText(seedPath)) as JsonObject ?? throw new InvalidOperationException($"{seedPath}: top-level must be an object"); - var cols = new[] { "id", "name", "tokenHash", "pinHash" }; + var cols = new[] { "id", "name", "tokenHash", "pinHash", "role" }; await using var c = new NpgsqlConnection(connString); await c.OpenAsync(); foreach (var rowNode in root["rows"]!.AsArray()) diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs index 1d5955b386..b8a782435c 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/TphReferenceServer.cs @@ -44,6 +44,12 @@ internal sealed class TphReferenceServer : IAsyncDisposable // rather than rejected (the same treatment autoCreatedAt/autoUpdatedAt get by being // absent here). It IS written on create, explicitly, below. Do NOT add it here. private static readonly string[] AllCols = { "id", "type", "reference", "quantity", "copayAmount", "approver" }; + // The subtype-only columns, by discriminator value. A create stores only its own subtype's + // (1.2.x input contract: a sibling subtype's column in the body is dropped, not stored). + private static readonly Dictionary SubtypeCol = new() + { + ["Bridge"] = "quantity", ["Copay"] = "copayAmount", ["PriorAuth"] = "approver", + }; private readonly PostgresContainer _pg; private readonly WebApplication _app; @@ -340,6 +346,7 @@ private static (List Filters, string? Error, string? Field) ParseFil foreach (var col in AllCols) { if (col is "id" or "type") continue; + if (SubtypeCol.ContainsValue(col) && SubtypeCol[discriminator] != col) continue; if (body[col] is { } v && v.GetValueKind() != JsonValueKind.Null) { cols.Add(col); diff --git a/server/java/codegen-base/pom.xml b/server/java/codegen-base/pom.xml index f0d07409ea..8eb2cdebab 100644 --- a/server/java/codegen-base/pom.xml +++ b/server/java/codegen-base/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-base diff --git a/server/java/codegen-kotlin-exposed1x-check/pom.xml b/server/java/codegen-kotlin-exposed1x-check/pom.xml index 71efef9577..32c9755f0b 100644 --- a/server/java/codegen-kotlin-exposed1x-check/pom.xml +++ b/server/java/codegen-kotlin-exposed1x-check/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-kotlin-exposed1x-check diff --git a/server/java/codegen-kotlin/pom.xml b/server/java/codegen-kotlin/pom.xml index 5a5fe50e6a..b7f50c2b59 100644 --- a/server/java/codegen-kotlin/pom.xml +++ b/server/java/codegen-kotlin/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-kotlin diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt index 62b74b74fc..7c86b95c55 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinGenUtil.kt @@ -655,18 +655,23 @@ public object KotlinGenUtil { /** * True when [field] is SERVER-OWNED on create for [obj]: a component of [obj]'s - * `increment`/`uuid` primary key (the store generates it), or an `@autoSet` column (the - * CRUD path stamps it, issue #203). The caller never supplies either, so a create body - * that omits one is correct — the complement of [isAssignedPrimaryKeyField]. Judged - * against the entity being generated, not `field.parent`: a key idiomatically lives on - * an abstract base and is inherited through `extends`. + * `increment`/`uuid` primary key (the store generates it), an `@autoSet` column (the + * CRUD path stamps it, issue #203), or — on an object with a WRITABLE rest surface — + * a `readOnly` or derived column (the create SECURITY skip never binds it). The caller + * never supplies any of these, so a create body that omits one is correct — the + * complement of [isAssignedPrimaryKeyField]. Judged against the entity being + * generated, not `field.parent`: a key idiomatically lives on an abstract base and is + * inherited through `extends`. A read-only projection keeps its non-null read shape: + * it has no create body to relax for. * * Mirrors the Java port's `SERVER_OWNED_ON_CREATE` set (SpringControllerGenerator). */ fun isServerOwnedOnCreate(field: MetaField<*>, obj: MetaObject): Boolean { val pk = obj.primaryIdentity if (pk != null && pk.isAutoGenerated && pk.fields.contains(field.name)) return true - return autoSetPolicy(field) != null + if (autoSetPolicy(field) != null) return true + if (!com.metaobjects.generator.util.RestSurfaceGate.isWritable(obj)) return false + return isReadOnlyMutability(field) || isDerivedField(field) } /** diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index c691f0b6f2..e351ae8ee7 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -530,6 +530,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase SourceFile.kotlin(path.parent.relativize(path).toString().replace('/', '_'), path.readText()) } + val compileResult = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, compileResult.exitCode, + "generated Doc.kt failed to compile:\n${compileResult.messages}") + + val docClass = compileResult.classLoader.loadClass("acme.bind.Doc") + val mapper = ObjectMapper().registerKotlinModule() + val validator = Validation.buildDefaultValidatorFactory().validator + + // Omitted: pre-fix this cannot bind (non-null creator param, no default) → 400. + val omitted = mapper.readValue("""{"title":"t"}""", docClass) + assertTrue(validator.validate(omitted).isEmpty(), "omitted readOnly field must not be demanded") + + // Present: binds (client value never reaches the insert — api-contract scenario) + // and validates; pre-fix the column skip then hit the NOT NULL column → 500. + val present = mapper.readValue("""{"title":"t","role":"evil"}""", docClass) + assertTrue(validator.validate(present).isEmpty(), "present readOnly value must not be rejected") + + // The omitted binding relaxed the property: nullable, defaulted to null. + assertNull(docClass.getMethod("getRole").invoke(omitted)) + } finally { + out.toFile().deleteRecursively() + } + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt new file mode 100644 index 0000000000..d5b01f2083 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt @@ -0,0 +1,89 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.metadata.ktx.loadString +import java.nio.file.Files +import kotlin.test.Test +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * A generated create route writes only what a client may set: a server-generated key, a `readOnly` + * column and (TPH) the discriminator and a sibling subtype's column are never bound from the body + * (1.2.x security patch). Behaviour is gated by the api-contract `input-server-owned-dropped` + * scenarios; this pins the generated source, including the TPH subtype `readOnly` case. + */ +class KotlinServerOwnedInputTest { + + private val plain = """{ + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Doc", "children": [ + { "source.rdb": { "@table": "docs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@maxLength": 100 } }, + { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + + private val tph = """{ + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Prior"] } }, + { "field.string": { "name": "reference", "@maxLength": 80 } }, + { "field.string": { "name": "grade", "@maxLength": 20, "@mutability": "readOnly" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", "children": [ + { "field.int": { "name": "quantity" } } + ] } }, + { "object.entity": { "name": "PriorAuth", "extends": "Auth", "@discriminatorValue": "Prior", "children": [ + { "field.string": { "name": "approver", "@maxLength": 80 } } + ] } } + ] } + }""".trimIndent() + + private fun gen(name: String, json: String): java.nio.file.Path { + val out = Files.createTempDirectory("kowned-") + val loader = loadString(name, json) + KotlinEntityGenerator().apply { setArgs(mapOf("outputDir" to out.toString())) }.execute(loader) + KotlinSpringControllerGenerator().apply { setArgs(mapOf("outputDir" to out.toString())) }.execute(loader) + return out + } + + @Test + fun `plain create writes neither the generated key nor a readOnly column`() { + val out = gen("owned-plain", plain) + try { + val controller = Files.readString(out.resolve("acme/own/DocController.kt")) + val create = controller.substringAfter("fun create(").substringBefore("fun update(") + assertTrue("it[title] = dto.title" in create, create) + assertFalse("it[role]" in create, create) + assertFalse("it[id] = dto.id" in create, create) + } finally { + out.toFile().deleteRecursively() + } + } + + @Test + fun `TPH create writes only the route subtype's own settable columns`() { + val out = gen("owned-tph", tph) + try { + val controller = Files.readString(out.resolve("acme/own/AuthController.kt")) + val bridge = controller.substringAfter("fun createBridge(").substringBefore("@RequestMapping") + val prior = controller.substringAfter("fun createPrior(").substringBefore("@RequestMapping") + assertTrue("it[reference] = dto.reference" in bridge, bridge) + assertTrue("it[quantity] = dto.quantity" in bridge, bridge) + assertFalse("it[approver]" in bridge, bridge) + assertFalse("it[grade]" in bridge, bridge) + assertFalse("it[type] = dto.type" in bridge, bridge) + assertTrue("it[approver] = dto.approver" in prior, prior) + assertFalse("it[quantity]" in prior, prior) + assertFalse("it[grade]" in prior, prior) + } finally { + out.toFile().deleteRecursively() + } + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt index a796931a9a..00257684f0 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt @@ -425,8 +425,6 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat it[type] = AuthType.Bridge it[reference] = dto.reference!! it[quantity] = dto.quantity - it[copayAmount] = dto.copayAmount - it[priorAuthNumber] = dto.priorAuthNumber }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuth(saved) as Any) @@ -510,9 +508,7 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat val newId = AuthTable.insert { it[type] = AuthType.Copay it[reference] = dto.reference!! - it[quantity] = dto.quantity it[copayAmount] = dto.copayAmount - it[priorAuthNumber] = dto.priorAuthNumber }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuth(saved) as Any) @@ -596,8 +592,6 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat val newId = AuthTable.insert { it[type] = AuthType.PriorAuth it[reference] = dto.reference!! - it[quantity] = dto.quantity - it[copayAmount] = dto.copayAmount it[priorAuthNumber] = dto.priorAuthNumber }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() diff --git a/server/java/codegen-mustache/pom.xml b/server/java/codegen-mustache/pom.xml index 81119b2b02..6def4a6c9a 100644 --- a/server/java/codegen-mustache/pom.xml +++ b/server/java/codegen-mustache/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-mustache diff --git a/server/java/codegen-plantuml/pom.xml b/server/java/codegen-plantuml/pom.xml index 957fa15a4d..2d71c32ee1 100644 --- a/server/java/codegen-plantuml/pom.xml +++ b/server/java/codegen-plantuml/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-plantuml diff --git a/server/java/codegen-spring/pom.xml b/server/java/codegen-spring/pom.xml index 239418be3f..8a303818cc 100644 --- a/server/java/codegen-spring/pom.xml +++ b/server/java/codegen-spring/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-codegen-spring diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java index dac59eeff3..14c28fe19f 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringControllerGenerator.java @@ -344,7 +344,10 @@ protected void emit(MetaObject entity, Path outRoot) { // #484: a @formExclude / credential value is IGNORED (never rejected): the DTO is rebuilt // without it, so the column is left to the repository's default. List components = SpringGeneratorSupport.dtoComponentFields(entity); - appendStripWriteDenied(src, dtoName, components, SpringGeneratorSupport.writeDeniedNames(components)); + // SECURITY (1.2.x): also dropped, never rejected — a server-generated primary key and a + // readOnly or derived field, none of which a client may set on create. The repository is + // handed only what the client may set. + appendStripWriteDenied(src, dtoName, components, createDeniedNames(entity, components)); // A caller does not supply server-owned columns, so a @NotNull on one must not reject // the request. The DTO is BOTH the request and the response body, so `id` carries // @NotNull because it is @required in the RESPONSE — while being identity.primary @@ -1373,7 +1376,11 @@ protected void emitTph(MetaObject base, TphPlan.Plan plan, Path outRoot) { // OWN @autoSet column is a documented non-goal (uniform with the Kotlin and Python // ports, both base-scoped): the column is simply never stamped, not a compile break. // #484: ignore (never reject) a @formExclude / credential value in the body. - appendStripWriteDenied(src, dtoName, unionFields, writeDenied); + // SECURITY (1.2.x): also dropped — the generated key, readOnly/derived columns, the + // discriminator (the route's subtype is its only source) and every column only a + // SIBLING subtype declares. + appendStripWriteDenied(src, dtoName, unionFields, + tphCreateDeniedNames(st.entity(), unionFields, writeDenied, TphPlan.discriminatorFieldOf(st.entity()))); String createArg = SpringGeneratorSupport.hasAutoSetFields(base) ? dtoName + ".stampForInsert(dto)" : "dto"; src.append(" try {\n"); @@ -1598,11 +1605,13 @@ private static List serverOwnedOnCreate(MetaObject entity) { java.util.Set denied = SpringGeneratorSupport.hierarchyWriteDeniedNames(entity); List out = new ArrayList<>(); - for (MetaField f : SpringGeneratorSupport.scalarFields(entity)) { + // Every DTO component, not only scalars: the create strip (createDeniedNames) nulls a + // readOnly / derived value-object jsonb component too, so its @NotNull must be ignored here. + for (MetaField f : SpringGeneratorSupport.dtoComponentFields(entity)) { // #484: a write-denied field is IGNORED on create, so a @NotNull on it describes the // response shape and must not reject a body that (necessarily) omits it. if (pkFields.contains(f.getName()) || AutoSetSupport.isAutoSet(f) - || denied.contains(f.getName())) { + || denied.contains(f.getName()) || isNonSettableOnCreate(f)) { String component = SpringNaming.recordComponentName(f.getName()); if (!out.contains(component)) out.add(component); } @@ -1649,6 +1658,55 @@ private static List targetCredentials(MetaObject source, SpringM2mSuppor return List.of(); } + /** A field no client may set on create, whatever the entity's key: readOnly or derived. */ + private static boolean isNonSettableOnCreate(MetaField f) { + return MetaField.MUTABILITY_READ_ONLY.equals(f.getMutability()) || f.isDerived(); + } + + /** The field names of {@code entity}'s primary key when the server generates it (increment / uuid). */ + private static Set generatedPkNames(MetaObject entity) { + return entity.getIdentities(true).stream() + .filter(com.metaobjects.identity.MetaIdentity::isPrimary) + .findFirst() + .filter(com.metaobjects.identity.MetaIdentity::isAutoGenerated) + .>map(pk -> new java.util.LinkedHashSet<>(pk.getFields())) + .orElse(Set.of()); + } + + /** + * The components a plain create drops from the bound body before it reaches the repository: + * write-denied ({@code @formExclude} / credential), a server-generated primary key (an assigned + * key is the caller's to choose), and {@code readOnly} and derived fields. + */ + private static Set createDeniedNames(MetaObject entity, List components) { + Set out = new java.util.LinkedHashSet<>(SpringGeneratorSupport.writeDeniedNames(components)); + Set pk = generatedPkNames(entity); + for (MetaField f : components) { + if (pk.contains(f.getName()) || isNonSettableOnCreate(f)) out.add(f.getName()); + } + return out; + } + + /** + * The union-DTO components a TPH per-subtype create drops: the hierarchy's write-denied names, + * the generated key, readOnly and derived columns, the discriminator, and every column the + * route's subtype does not itself have (a sibling subtype's, never this route's to write). + */ + private static Set tphCreateDeniedNames(MetaObject subtype, List union, + Set writeDenied, String discField) { + Set out = new java.util.LinkedHashSet<>(writeDenied); + Set pk = generatedPkNames(subtype); + java.util.Map own = new java.util.HashMap<>(); + for (MetaField f : SpringGeneratorSupport.scalarFields(subtype)) own.put(f.getName(), f); + for (MetaField u : union) { + MetaField f = own.get(u.getName()); + if (f == null || pk.contains(u.getName()) || u.getName().equals(discField) || isNonSettableOnCreate(f)) { + out.add(u.getName()); + } + } + return out; + } + /** The fields of a create body that a generated write route ignores (#484). */ private static void appendStripWriteDenied(StringBuilder src, String dtoName, List components, Set writeDenied) { diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java index 4856d29870..0e49a1e098 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringSecretFieldsCodegenTest.java @@ -63,7 +63,7 @@ private static void run(com.metaobjects.generator.direct.MultiFileDirectGenerato public void createIgnoresBothDeniedFields_andPatchCarriesNeither() throws Exception { String controller = generate("marked", MARKED, "AccountController.java", true); assertTrue("create rebuilds the DTO without tokenHash and pinHash; saw:\n" + controller, - controller.contains("dto = new AccountDto(dto.id(), dto.name(), null, null);")); + controller.contains("dto = new AccountDto(null, dto.name(), null, null);")); Path gen = tmp.getRoot().toPath().resolve("gen-marked"); String patch = Files.readString(gen.resolve("acme/secrets/AccountPatch.java")); @@ -89,7 +89,8 @@ public void onlyTheCredentialIsHiddenOnRead() throws Exception { public void unmarkedModelEmitsNoRedactionAndNoStrip() throws Exception { String controller = generate("plain", PLAIN, "AccountController.java", false); assertFalse(controller.contains("redact")); - assertFalse(controller.contains("dto = new AccountDto(")); + // only the generated key is dropped: nothing here is a credential or @formExclude + assertTrue(controller.contains("dto = new AccountDto(null, dto.name(), dto.tokenHash(), dto.pinHash());")); } /** @@ -123,7 +124,7 @@ public void tphSharedColumnMarkedOnALaterSubtypeIsHiddenAndIgnored() throws Exce assertTrue("the union list hides the credential; saw:\n" + controller, controller.contains("redactAll(rows, \"token\")")); assertTrue("every subtype create ignores both marked columns; saw:\n" + controller, - controller.contains("dto = new ThingDto(dto.id(), dto.kind(), null, null);")); + controller.contains("dto = new ThingDto(null, null, null, null);")); assertFalse("memo (@formExclude only) is still returned", controller.contains("\"memo\")")); } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java new file mode 100644 index 0000000000..781a28767f --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java @@ -0,0 +1,133 @@ +package com.metaobjects.generator.spring; + +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** + * A generated create route hands the repository only what a client may set: a server-generated + * key, a {@code readOnly} field and (TPH) the discriminator and a sibling subtype's column are + * dropped from the bound body before the repository sees it (1.2.x security patch). Behaviour is + * gated by the api-contract {@code input-server-owned-dropped} scenarios; this pins the generated + * source, including the TPH subtype {@code readOnly} case the shared TPH model does not declare. + */ +public class SpringServerOwnedInputCodegenTest extends SharedRegistryTestBase { + + @Rule + public TemporaryFolder tmp = new TemporaryFolder(); + + private static final String PLAIN = """ + { + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Doc", "children": [ + { "source.rdb": { "@table": "docs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "field.string": { "name": "role", "@mutability": "readOnly" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "Tag", "children": [ + { "source.rdb": { "@table": "tags" } }, + { "field.string": { "name": "code" } }, + { "field.string": { "name": "label" } }, + { "identity.primary": { "name": "pk", "@fields": ["code"] } } + ] } } + ] } + } + """; + + private static final String TPH = """ + { + "metadata.root": { "package": "acme::own", "children": [ + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Prior"] } }, + { "field.string": { "name": "reference" } }, + { "field.string": { "name": "grade", "@mutability": "readOnly" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", "children": [ + { "field.int": { "name": "quantity" } } + ] } }, + { "object.entity": { "name": "PriorAuth", "extends": "Auth", "@discriminatorValue": "Prior", "children": [ + { "field.string": { "name": "approver" } } + ] } } + ] } + } + """; + + private static final String VO = """ + { + "metadata.root": { "package": "acme::own", "children": [ + { "object.value": { "name": "Stamp", "children": [ + { "field.string": { "name": "by" } } + ] } }, + { "object.entity": { "name": "Note", "children": [ + { "source.rdb": { "@table": "notes" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "body" } }, + { "field.object": { "name": "stamp", "@objectRef": "Stamp", "@storage": "jsonb", + "@required": true, "@mutability": "readOnly" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + } + """; + + private Path generate(String name, String fixture) throws Exception { + Path gen = tmp.newFolder("gen-" + name).toPath(); + MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder("ws-" + name).toPath(), name, fixture); + for (var g : new com.metaobjects.generator.direct.MultiFileDirectGeneratorBase[] { + new SpringControllerGenerator(), new SpringDtoGenerator() }) { + Map args = new HashMap<>(); + args.put("outputDir", gen.toString()); + g.setArgs(args); + g.execute(loader); + } + return gen; + } + + @Test + public void plainCreateDropsTheGeneratedKeyAndReadOnlyField() throws Exception { + Path gen = generate("plain", PLAIN); + String controller = Files.readString(gen.resolve("acme/own/DocController.java")); + assertTrue("generated id and readOnly role are nulled; saw:\n" + controller, + controller.contains("dto = new DocDto(null, dto.title(), null);")); + // an assigned key (no @generation) is the caller's on create + String tag = Files.readString(gen.resolve("acme/own/TagController.java")); + assertFalse("assigned key must be kept; saw:\n" + tag, tag.contains("dto = new TagDto(")); + } + + @Test + public void aRequiredReadOnlyValueObjectIsNulledAndItsNotNullIgnored() throws Exception { + Path gen = generate("vo", VO); + String controller = Files.readString(gen.resolve("acme/own/NoteController.java")); + assertTrue("readOnly VO component is nulled on create; saw:\n" + controller, + controller.contains("dto = new NoteDto(null, dto.body(), null);")); + // the strip nulls `stamp`, so its @NotNull must not turn every POST into a 400 + assertTrue("stamp must be server-owned for create validation; saw:\n" + controller, + controller.matches("(?s).*SERVER_OWNED_ON_CREATE = Set\\.of\\([^)]*\"stamp\".*")); + } + + @Test + public void tphCreateDropsKeyDiscriminatorReadOnlyAndSiblingColumns() throws Exception { + Path gen = generate("tph", TPH); + String controller = Files.readString(gen.resolve("acme/own/AuthController.java")); + // union components: id, type, reference, grade, quantity, approver + assertTrue("Bridge route keeps reference and quantity only; saw:\n" + controller, + controller.contains("dto = new AuthDto(null, null, dto.reference(), null, dto.quantity(), null);")); + assertTrue("Prior route keeps reference and approver only; saw:\n" + controller, + controller.contains("dto = new AuthDto(null, null, dto.reference(), null, null, dto.approver());")); + } +} diff --git a/server/java/core-spring/pom.xml b/server/java/core-spring/pom.xml index 3dd35699a5..8799bf12f0 100644 --- a/server/java/core-spring/pom.xml +++ b/server/java/core-spring/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-core-spring diff --git a/server/java/integration-tests-kotlin/pom.xml b/server/java/integration-tests-kotlin/pom.xml index ba173e5891..cc386d3413 100644 --- a/server/java/integration-tests-kotlin/pom.xml +++ b/server/java/integration-tests-kotlin/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-integration-tests-kotlin diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt index 193959935d..499d718cff 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/ApiContractAssertions.kt @@ -4,7 +4,7 @@ import com.metaobjects.integration.kotlin.api.ApiContractScenarios.ApiRequest /** * Assertion engine for api-contract-conformance scenarios. The vocabulary - * (`equals` / `length` / `ids` / `names` / `row` / `hasId` / `envelope` / + * (`equals` / `length` / `ids` / `names` / `row` / `unset` / `hasId` / `envelope` / * `error` / `empty` / `fieldsEqual` / `fieldsNotEqual`) is the cross-port * contract — every per-port runner must implement these keys identically. * See `fixtures/api-contract-conformance/README.md`. @@ -152,6 +152,15 @@ object ApiContractAssertions { throw AssertionError("$scenarioName / ${request.id}: expected row.$k=$v, got $actual") } } + // Each listed key is missing or null: a value the client may not set never reached storage. + // (Missing-or-null, not strictly missing: ports differ on echoing an unset column as null.) + (want["unset"] as? List<*>)?.let { keys -> + val row = body as? Map<*, *> + ?: throw AssertionError("$scenarioName / ${request.id}: expected object for unset, got: $body") + val stored = keys.filter { row[it] != null } + if (stored.isNotEmpty()) + throw AssertionError("$scenarioName / ${request.id}: expected $stored unset, got: $body") + } if (want["hasId"] == true) { val row = body as? Map<*, *> ?: throw AssertionError("$scenarioName / ${request.id}: expected object, got: $body") diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt index e4d3886a42..60e0b787bf 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt @@ -85,6 +85,9 @@ class GeneratedAuthorControllerHarness( */ private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated + // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .registerModule( SimpleModule().addDeserializer(Instant::class.java, object : JsonDeserializer() { diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt index a0492ebdd7..1904580b41 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt @@ -38,6 +38,9 @@ class GeneratedSecretFieldsControllerHarness( private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated + // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) @@ -112,7 +115,7 @@ class GeneratedSecretFieldsControllerHarness( // Ids are assigned by the identity column in seed order (1, 2), so a later POST does not collide. // Mixed-case columns are quoted (naming.columns: literal), as in the write-through harness. for (r in rows) { - exec("INSERT INTO accounts (\"name\", \"tokenHash\", \"pinHash\") VALUES (${sqlStr(r["name"])}, ${sqlStr(r["tokenHash"])}, ${sqlStr(r["pinHash"])})") + exec("INSERT INTO accounts (\"name\", \"tokenHash\", \"pinHash\", \"role\") VALUES (${sqlStr(r["name"])}, ${sqlStr(r["tokenHash"])}, ${sqlStr(r["pinHash"])}, ${sqlStr(r["role"])})") } } @@ -128,9 +131,9 @@ class GeneratedSecretFieldsControllerHarness( val database = db ?: error("reset() must be called before storedRows()") return transaction(database) { val out = mutableListOf>() - exec("SELECT id, \"tokenHash\", \"pinHash\" FROM accounts ORDER BY id") { rs -> + exec("SELECT id, \"tokenHash\", \"pinHash\", \"role\" FROM accounts ORDER BY id") { rs -> while (rs.next()) { - out += mapOf("id" to rs.getLong(1), "tokenHash" to rs.getString(2), "pinHash" to rs.getString(3)) + out += mapOf("id" to rs.getLong(1), "tokenHash" to rs.getString(2), "pinHash" to rs.getString(3), "role" to rs.getString(4)) } } out diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt index df760476b7..583e827ff5 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt @@ -56,6 +56,9 @@ class GeneratedTphControllerHarness( private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated + // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) diff --git a/server/java/integration-tests/pom.xml b/server/java/integration-tests/pom.xml index d3f49841ae..314dc75ed1 100644 --- a/server/java/integration-tests/pom.xml +++ b/server/java/integration-tests/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-integration-tests diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java index b7ddbff70d..969f0e0ae6 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ApiContractAssertions.java @@ -8,7 +8,7 @@ /** * Assertion engine for api-contract-conformance scenarios. The vocabulary * ({@code equals} / {@code length} / {@code ids} / {@code names} / - * {@code row} / {@code hasId} / {@code envelope} / {@code error} / + * {@code row} / {@code unset} / {@code hasId} / {@code envelope} / {@code error} / * {@code empty} / {@code fieldsEqual} / {@code fieldsNotEqual}) is the * cross-port contract — every per-port runner must implement these keys * identically. See {@code fixtures/api-contract-conformance/README.md}. @@ -177,6 +177,17 @@ static void assertResponse(String scenarioName, ApiRequest request, int status, + ": expected row." + e.getKey() + "=" + e.getValue() + ", got " + actual); } } + // Each listed key is missing or null: a value the client may not set never reached storage. + // (Missing-or-null, not strictly missing: ports differ on echoing an unset column as null.) + if (want.get("unset") instanceof List unsetKeys) { + if (!(body instanceof Map row)) + throw new AssertionError(scenarioName + " / " + request.id() + + ": expected object for unset, got: " + body); + List stored = unsetKeys.stream().filter(k -> row.get(k) != null).toList(); + if (!stored.isEmpty()) + throw new AssertionError(scenarioName + " / " + request.id() + + ": expected " + stored + " unset, got: " + body); + } if (Boolean.TRUE.equals(want.get("hasId"))) { if (!(body instanceof Map row)) throw new AssertionError(scenarioName + " / " + request.id() diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java index 36925cef6e..4cea941ec2 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java @@ -61,7 +61,10 @@ public final class GeneratedAccountControllerHarness implements AutoCloseable { ] } } """; - private final ObjectMapper mapper = new ObjectMapper(); + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated controller + // leaves this to the host's mapper; the corpus pins the Boot behaviour. + private final ObjectMapper mapper = new ObjectMapper() + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); private final URLClassLoader classLoader; private final Class dtoClass; private final Constructor controllerCtor; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java index f10c443007..3c2e6627bc 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java @@ -130,7 +130,10 @@ public Instant deserialize(JsonParser p, DeserializationContext ctx) this.mapper = new ObjectMapper() .registerModule(new JavaTimeModule()) .registerModule(utcInstant) - .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS); + .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated + // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); Path srcDir = genDir.resolve("src"); Path classesDir = genDir.resolve("classes"); diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java index 773413c696..57dfd7d252 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAccountRepositorySource.java @@ -68,7 +68,10 @@ public Optional findById(Long id) { @Override public AccountDto create(AccountDto dto) { - AccountDto saved = new AccountDto(nextId.getAndIncrement(), dto.name(), dto.tokenHash(), dto.pinHash()); + // Naive adopter repository: trusts the id and every column it is handed, so a forwarded + // client id or readOnly `role` would be stored and visible to the scenarios. + Long id = dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()); + AccountDto saved = new AccountDto(id, dto.name(), dto.tokenHash(), dto.pinHash(), dto.role()); rows.add(saved); return saved; } @@ -81,7 +84,8 @@ public Optional update(Long id, AccountDto dto) { AccountDto merged = new AccountDto(id, dto.name() != null ? dto.name() : cur.name(), dto.tokenHash() != null ? dto.tokenHash() : cur.tokenHash(), - dto.pinHash() != null ? dto.pinHash() : cur.pinHash()); + dto.pinHash() != null ? dto.pinHash() : cur.pinHash(), + dto.role() != null ? dto.role() : cur.role()); rows.set(i, merged); return Optional.of(merged); } @@ -98,7 +102,8 @@ public Optional patch(Long id, AccountPatch patch) { AccountDto merged = new AccountDto(id, assigned.containsKey("name") ? (String) assigned.get("name") : cur.name(), assigned.containsKey("tokenHash") ? (String) assigned.get("tokenHash") : cur.tokenHash(), - assigned.containsKey("pinHash") ? (String) assigned.get("pinHash") : cur.pinHash()); + assigned.containsKey("pinHash") ? (String) assigned.get("pinHash") : cur.pinHash(), + assigned.containsKey("role") ? (String) assigned.get("role") : cur.role()); rows.set(i, merged); return Optional.of(merged); } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java index 7027381627..a4c01db44c 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryAuthorRepositorySource.java @@ -100,7 +100,11 @@ public AuthorDto create(AuthorDto dto) { // Issue #203 / ADR-0045: the generated controller already stamped @autoSet via // AuthorDto.stampForInsert(dto) before calling create — the repo just persists // the incoming (already-stamped) values verbatim. - AuthorDto saved = new AuthorDto(nextId.getAndIncrement(), dto.name(), dto.bio(), + // A naive adopter repository: it trusts the id it is handed and mints one only when + // absent, so a client id the controller forwarded would be stored (and the shared + // input-server-owned-dropped scenario would see it). + Long id = dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()); + AuthorDto saved = new AuthorDto(id, dto.name(), dto.bio(), dto.createdAt(), dto.issuedCurrency(), dto.autoCreatedAt(), dto.autoUpdatedAt()); rows.add(saved); return saved; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java index dbe315f4f9..22ad1739cd 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java @@ -115,7 +115,10 @@ public Instant deserialize(JsonParser p, DeserializationContext ctx) this.mapper = new ObjectMapper() .registerModule(new JavaTimeModule()) .registerModule(utcInstant) - .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS); + .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) + // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated + // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); Path srcDir = genDir.resolve("src"); Path classesDir = genDir.resolve("classes"); Files.createDirectories(srcDir); diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java index 4f540110eb..f8343cbb49 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/InMemoryAuthRepositorySource.java @@ -15,8 +15,8 @@ * *

The base {@code AuthDto} is the UNION of all subtype columns (the TPH-base DTO shape), so a * single record type backs both polymorphic and per-subtype rows. Discriminator scoping is a plain - * {@code type.equals(discriminator)} filter; a create injects the URL-supplied discriminator and - * never reads {@code dto.type()}.

+ * {@code type.equals(discriminator)} filter; a create stores whatever id/discriminator the DTO carries, + * falling back to the URL's discriminator.

*/ final class InMemoryAuthRepositorySource { @@ -100,7 +100,9 @@ public Optional findByIdAndType(Long id, String discriminator) { @Override public AuthDto createWithType(String discriminator, AuthDto dto) { - // Discriminator injected from the URL — dto.type() is intentionally ignored. + // A naive adopter repository: it trusts what it is handed (the id and the DTO's own + // discriminator when present, falling back to the URL's), so a value the controller + // forwarded from the request body is stored and the shared scenarios can see it. // `type` is a value-constrained enum (field.enum discriminator), so coerce the // String URL discriminator into the generated AuthType (its members ARE the // discriminator values). @@ -109,7 +111,8 @@ public AuthDto createWithType(String discriminator, AuthDto dto) { // AuthDto.stampForInsert(dto) before calling createWithType — the repo just // persists the incoming (already-stamped) autoCreatedAt/autoUpdatedAt verbatim. AuthDto saved = new AuthDto( - nextId.getAndIncrement(), AuthDto.AuthType.valueOf(discriminator), dto.reference(), + dto.id() != null ? dto.id() : Long.valueOf(nextId.getAndIncrement()), + dto.type() != null ? dto.type() : AuthDto.AuthType.valueOf(discriminator), dto.reference(), // FR-037 R1: writeOnce IS bound on create — settable exactly once. dto.issuedCurrency(), dto.autoCreatedAt(), dto.autoUpdatedAt(), diff --git a/server/java/maven-plugin/pom.xml b/server/java/maven-plugin/pom.xml index 69c6963f79..56bdbcc152 100644 --- a/server/java/maven-plugin/pom.xml +++ b/server/java/maven-plugin/pom.xml @@ -2,7 +2,7 @@ metaobjects com.metaobjects - 8.2.0 + 8.2.2 4.0.0 diff --git a/server/java/metadata-ktx/pom.xml b/server/java/metadata-ktx/pom.xml index e902e8fc06..7c16b99371 100644 --- a/server/java/metadata-ktx/pom.xml +++ b/server/java/metadata-ktx/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-metadata-ktx diff --git a/server/java/metadata/pom.xml b/server/java/metadata/pom.xml index d0290e656b..c2716046fd 100644 --- a/server/java/metadata/pom.xml +++ b/server/java/metadata/pom.xml @@ -4,7 +4,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-metadata diff --git a/server/java/om/pom.xml b/server/java/om/pom.xml index e03690a86d..c33e76ccc6 100644 --- a/server/java/om/pom.xml +++ b/server/java/om/pom.xml @@ -5,7 +5,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-om diff --git a/server/java/omdb-ktx/pom.xml b/server/java/omdb-ktx/pom.xml index d5261ce1bd..07d9832f74 100644 --- a/server/java/omdb-ktx/pom.xml +++ b/server/java/omdb-ktx/pom.xml @@ -7,7 +7,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-omdb-ktx diff --git a/server/java/omdb/pom.xml b/server/java/omdb/pom.xml index ae699c85ce..4744ad6944 100644 --- a/server/java/omdb/pom.xml +++ b/server/java/omdb/pom.xml @@ -4,7 +4,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-omdb diff --git a/server/java/pom.xml b/server/java/pom.xml index b3f5c3595c..29ddd9fd55 100644 --- a/server/java/pom.xml +++ b/server/java/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.metaobjects metaobjects - 8.2.0 + 8.2.2 pom MetaObjects Core Universal metadata-driven development platform diff --git a/server/java/render/pom.xml b/server/java/render/pom.xml index 26f3dc66ab..a62d2e77cc 100644 --- a/server/java/render/pom.xml +++ b/server/java/render/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-render diff --git a/server/java/spring-boot-starter/pom.xml b/server/java/spring-boot-starter/pom.xml index c131688526..62abe5f9ee 100644 --- a/server/java/spring-boot-starter/pom.xml +++ b/server/java/spring-boot-starter/pom.xml @@ -6,7 +6,7 @@ com.metaobjects metaobjects - 8.2.0 + 8.2.2 metaobjects-spring-boot-starter diff --git a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md index 1b300bfa6f..a8c8385253 100644 --- a/server/python/src/metaobjects/codegen/KNOWN_GAPS.md +++ b/server/python/src/metaobjects/codegen/KNOWN_GAPS.md @@ -54,6 +54,18 @@ import wiring itself (`from ._entity import `), which still needs path-resolution/import-base machinery; the config's per-target `outDir` is the prerequisite that is now in place. +## TPH body discriminator: 400, not a silent drop + +**Status:** known divergence, out of scope for the 1.2.x input patch. + +A per-subtype write route (`POST /api/auths/bridge`) validates the request body with +that subtype's model, whose discriminator accepts only the route's own value. A body +that names another subtype (`"type": "Copay"`) is answered `400 {"error":"validation"}`. +Nothing is stored, so this is not a security hole, but the other ports drop the key +and create the row with the route's subtype. The shared +`tph-input-foreign-discriminator-dropped` scenario asserts the drop, and this port's +runner skips it with this reason (`tests/integration/test_api_contract_tph_generated.py`). + ## FR-015 stored-procedure callables are NOT generated by this port **Contract.** FR-015 gives a `source.rdb @kind: storedProc` / `tableFunction` a generated diff --git a/server/python/tests/integration/api_contract_assertions.py b/server/python/tests/integration/api_contract_assertions.py index 11a6f33445..fac4b8ccce 100644 --- a/server/python/tests/integration/api_contract_assertions.py +++ b/server/python/tests/integration/api_contract_assertions.py @@ -1,7 +1,7 @@ """Assertion engine for api-contract-conformance scenarios. The vocabulary (``equals`` / ``length`` / ``ids`` / ``names`` / ``row`` / -``hasId`` / ``envelope`` / ``error`` / ``field`` / ``empty``) is the cross-port contract — +``unset`` / ``hasId`` / ``envelope`` / ``error`` / ``field`` / ``empty``) is the cross-port contract — every per-port runner must implement these keys identically. See ``fixtures/api-contract-conformance/README.md``. @@ -202,6 +202,19 @@ def assert_response( f"but both = {a_value!r}" ) + if "unset" in expect_body: + # Each listed key is missing or null: a value the client may not set never reached + # storage. (Missing-or-null: ports differ on echoing an unset column as null.) + if not isinstance(body, dict): + raise AssertionError( + f"{scenario_name} / {request_id}: expected object for unset, got: {body!r}" + ) + stored = [k for k in expect_body["unset"] if body.get(k) is not None] + if stored: + raise AssertionError( + f"{scenario_name} / {request_id}: expected {stored!r} unset, got: {body!r}" + ) + if expect_body.get("hasId") is True: if not isinstance(body, dict): raise AssertionError( diff --git a/server/python/tests/integration/generated_secret_fields_app.py b/server/python/tests/integration/generated_secret_fields_app.py index 61d9eb9a4e..3a6a1f2f1c 100644 --- a/server/python/tests/integration/generated_secret_fields_app.py +++ b/server/python/tests/integration/generated_secret_fields_app.py @@ -28,7 +28,7 @@ from .corpus_config import attach_corpus_wire from .generated_router_app import InMemoryAuthorRepository -COLUMNS = ("id", "name", "tokenHash", "pinHash") +COLUMNS = ("id", "name", "tokenHash", "pinHash", "role") def load_account(meta_json: Path) -> MetaObject: diff --git a/server/python/tests/integration/test_api_contract_tph_generated.py b/server/python/tests/integration/test_api_contract_tph_generated.py index 7305eb6b78..b84fb01b84 100644 --- a/server/python/tests/integration/test_api_contract_tph_generated.py +++ b/server/python/tests/integration/test_api_contract_tph_generated.py @@ -55,6 +55,16 @@ def _load_scenarios() -> list[tuple[str, dict[str, Any]]]: _SCENARIOS = _load_scenarios() + +# Scenarios this port's generated routes answer differently, each with its reason. The routes +# validate the body's discriminator against the route's subtype and answer 400 rather than +# dropping it; nothing is stored either way. See codegen/KNOWN_GAPS.md. +_PORT_SKIPS: dict[str, str] = { + "tph-input-foreign-discriminator-dropped": ( + "a body discriminator naming another subtype is a 400 here, not a silent drop " + "(KNOWN_GAPS.md: TPH body discriminator)" + ), +} _APP, _REPO = build_generated_tph_app(_TPH_DIR) _CLIENT = TestClient(_APP) @@ -65,6 +75,8 @@ def _load_scenarios() -> list[tuple[str, dict[str, Any]]]: ids=[name for name, _ in _SCENARIOS], ) def test_api_contract_tph_generated_scenario(scenario_name: str, scenario: dict[str, Any]) -> None: + if scenario_name in _PORT_SKIPS: + pytest.skip(_PORT_SKIPS[scenario_name]) _REPO.reset() _REPO.seed(_SEED["auths"]) for req in scenario.get("requests", []): diff --git a/server/typescript/packages/ai-runtime/package.json b/server/typescript/packages/ai-runtime/package.json index 9ff757cf29..69fc612d73 100644 --- a/server/typescript/packages/ai-runtime/package.json +++ b/server/typescript/packages/ai-runtime/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/ai-runtime", - "version": "1.2.0", + "version": "1.2.2", "description": "LLM call loop + typed-trace recorder adapters for MetaObjects: provider-neutral LlmClient seam, callLlm bridge, Composite/Langfuse/OTel recorders.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/cli/package.json b/server/typescript/packages/cli/package.json index b1f41e6d4e..f1e3c07511 100644 --- a/server/typescript/packages/cli/package.json +++ b/server/typescript/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/cli", - "version": "1.2.0", + "version": "1.2.2", "description": "CLI for MetaObjects: scaffold, codegen, migrate, and drift-detection commands.", "type": "module", "main": "./dist/src/index.js", diff --git a/server/typescript/packages/codegen-ts-react/package.json b/server/typescript/packages/codegen-ts-react/package.json index 00d66c8985..343bdd944d 100644 --- a/server/typescript/packages/codegen-ts-react/package.json +++ b/server/typescript/packages/codegen-ts-react/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/codegen-ts-react", - "version": "1.2.0", + "version": "1.2.2", "description": "React codegen for metaobjects — emits .form.tsx files using react-hook-form and @metaobjectsdev/react helpers.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/codegen-ts-tanstack/package.json b/server/typescript/packages/codegen-ts-tanstack/package.json index 6ae9c41926..07dd6d0ca9 100644 --- a/server/typescript/packages/codegen-ts-tanstack/package.json +++ b/server/typescript/packages/codegen-ts-tanstack/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/codegen-ts-tanstack", - "version": "1.2.0", + "version": "1.2.2", "description": "TanStack codegen for metaobjects — TanStack Query hooks for every entity, plus TanStack Table column definitions and grid-state hooks for entities declaring a layout.dataGrid.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/codegen-ts/package.json b/server/typescript/packages/codegen-ts/package.json index cb52ec1a11..0785f3400e 100644 --- a/server/typescript/packages/codegen-ts/package.json +++ b/server/typescript/packages/codegen-ts/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/codegen-ts", - "version": "1.2.0", + "version": "1.2.2", "description": "TypeScript codegen engine for MetaObjects — emits Drizzle, Zod, and Fastify artifacts.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts index e6a526d36d..e21443f213 100644 --- a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts +++ b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts @@ -386,6 +386,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { * The field SET the `UpdateSchema` accepts — same iteration + skip rules * as `insertSchemaFields`, but mirroring the UpdateSchema branch of * `renderZodValidators`: + * • a primary-key field is OMITTED, generated or assigned (the URL names the row); * • a TPH subtype's @discriminator field is OMITTED (clients can't change subtype); * • @autoSet onCreate fields are OMITTED (creation timestamps are immutable); * • @autoSet onUpdate fields are present + optional (server fills them); @@ -393,6 +394,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of obj.fields()) { @@ -408,6 +410,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): a primary-key field is the row's identity, which the URL names. An + // ASSIGNED key leaves the update shape exactly like a generated one, so a client-sent + // key is stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -491,6 +497,10 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): the row's identity is the URL's. An ASSIGNED primary key leaves the + // update shape too (a generated one never entered it), so a client-sent key is stripped + // on PUT/PATCH and can never re-key the row. Mirrors updateSchemaFields. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { @@ -515,10 +525,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. PK fields never reach this branch (skipped above). + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } diff --git a/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap b/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap index d5a0163053..cc8e306f31 100644 --- a/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap +++ b/server/typescript/packages/codegen-ts/test/__snapshots__/reporting-docs.test.ts.snap @@ -204,36 +204,36 @@ Generated API reference for this project; call these exactly as written. Imports \`import { Program, ProgramInsertSchema, ProgramUpdateSchema } from "./Program"\` - \`interface Program { id: number; title?: string; createdAt?: string }\` — The typed shape of a Program row, generated from its metadata. - \`ProgramInsertSchema: ZodType<{ id: number; title?: string; createdAt?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`ProgramUpdateSchema: ZodType<{ id?: number; title?: string; createdAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`ProgramUpdateSchema: ZodType<{ title?: string; createdAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). \`import { findProgramById, listPrograms, createProgram, updateProgram, deleteProgramById } from "./Program.queries"\` - \`findProgramById(db: Db, id: number): Promise\` — Fetch a single Program by its primary key; null when not found. - \`listPrograms(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List Program rows with optional limit/offset paging. - \`createProgram(db: Db, data: { id: number; title?: string; createdAt?: string }): Promise\` — Validate (via ProgramInsertSchema) and insert a new Program. [throws: ZodError when data fails ProgramInsertSchema validation.] -- \`updateProgram(db: Db, id: number, patch: { id?: number; title?: string; createdAt?: string }): Promise\` — Partially update a Program by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails ProgramUpdateSchema validation.] +- \`updateProgram(db: Db, id: number, patch: { title?: string; createdAt?: string }): Promise\` — Partially update a Program by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails ProgramUpdateSchema validation.] - \`deleteProgramById(db: Db, id: number): Promise\` — Delete a Program by primary key; true when a row was removed. \`import { programRoutes } from "./Program.routes"\` - \`GET /api/programs -> { id: number; title?: string; createdAt?: string }\` — List Program (supports filter/sort/paging query params). - \`GET /api/programs/:id -> { id: number; title?: string; createdAt?: string }\` — Fetch a single Program by id (404 when not found). - \`POST /api/programs body: { id: number; title?: string; createdAt?: string }\` — Create a Program (body validated by ProgramInsertSchema). -- \`PATCH /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — Partially update a Program by id (body validated by ProgramUpdateSchema). -- \`PUT /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). +- \`PATCH /api/programs/:id body: { title?: string; createdAt?: string }\` — Partially update a Program by id (body validated by ProgramUpdateSchema). +- \`PUT /api/programs/:id body: { title?: string; createdAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). - \`DELETE /api/programs/:id\` — Delete a Program by id. \`import { registerProgramRoutes } from "./Program.routes.hono"\` - \`GET /api/programs -> { id: number; title?: string; createdAt?: string }\` — [Hono] List Program (filter/sort/paging query params). - \`GET /api/programs/:id -> { id: number; title?: string; createdAt?: string }\` — [Hono] Fetch a single Program by id (404 when not found). - \`POST /api/programs body: { id: number; title?: string; createdAt?: string }\` — [Hono] Create a Program (body validated by ProgramInsertSchema). -- \`PATCH /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — [Hono] Partially update a Program by id (body validated by ProgramUpdateSchema). -- \`PUT /api/programs/:id body: { id?: number; title?: string; createdAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). +- \`PATCH /api/programs/:id body: { title?: string; createdAt?: string }\` — [Hono] Partially update a Program by id (body validated by ProgramUpdateSchema). +- \`PUT /api/programs/:id body: { title?: string; createdAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by ProgramUpdateSchema). - \`DELETE /api/programs/:id\` — [Hono] Delete a Program by id. Example: \`\`\`ts const created = await createProgram(db, { id: 1 }); const found = await findProgramById(db, created.id); -const updated = await updateProgram(db, created.id, { id: 1 }); +const updated = await updateProgram(db, created.id, { title: "…" }); const removed = await deleteProgramById(db, created.id); \`\`\` @@ -242,37 +242,37 @@ const removed = await deleteProgramById(db, created.id); \`import { Purchase, PurchaseInsertSchema, PurchaseUpdateSchema, purchasesRelations } from "./Purchase"\` - \`interface Purchase { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — The typed shape of a Purchase row, generated from its metadata. - \`PurchaseInsertSchema: ZodType<{ id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`PurchaseUpdateSchema: ZodType<{ id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`PurchaseUpdateSchema: ZodType<{ programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). - \`const purchasesRelations { program?: Program (1:1 / N:1) }\` — Drizzle relations() for Purchase — register it with your schema, then traverse via the relational query API (db.query.purchases.findMany({ with: { … } })). \`import { findPurchaseById, listPurchases, createPurchase, updatePurchase, deletePurchaseById } from "./Purchase.queries"\` - \`findPurchaseById(db: Db, id: number): Promise\` — Fetch a single Purchase by its primary key; null when not found. - \`listPurchases(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List Purchase rows with optional limit/offset paging. - \`createPurchase(db: Db, data: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Validate (via PurchaseInsertSchema) and insert a new Purchase. [throws: ZodError when data fails PurchaseInsertSchema validation.] -- \`updatePurchase(db: Db, id: number, patch: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Partially update a Purchase by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails PurchaseUpdateSchema validation.] +- \`updatePurchase(db: Db, id: number, patch: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }): Promise\` — Partially update a Purchase by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails PurchaseUpdateSchema validation.] - \`deletePurchaseById(db: Db, id: number): Promise\` — Delete a Purchase by primary key; true when a row was removed. \`import { purchaseRoutes } from "./Purchase.routes"\` - \`GET /api/purchases -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — List Purchase (supports filter/sort/paging query params). - \`GET /api/purchases/:id -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Fetch a single Purchase by id (404 when not found). - \`POST /api/purchases body: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Create a Purchase (body validated by PurchaseInsertSchema). -- \`PATCH /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Partially update a Purchase by id (body validated by PurchaseUpdateSchema). -- \`PUT /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). +- \`PATCH /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — Partially update a Purchase by id (body validated by PurchaseUpdateSchema). +- \`PUT /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). - \`DELETE /api/purchases/:id\` — Delete a Purchase by id. \`import { registerPurchaseRoutes } from "./Purchase.routes.hono"\` - \`GET /api/purchases -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] List Purchase (filter/sort/paging query params). - \`GET /api/purchases/:id -> { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Fetch a single Purchase by id (404 when not found). - \`POST /api/purchases body: { id: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Create a Purchase (body validated by PurchaseInsertSchema). -- \`PATCH /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Partially update a Purchase by id (body validated by PurchaseUpdateSchema). -- \`PUT /api/purchases/:id body: { id?: number; programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). +- \`PATCH /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] Partially update a Purchase by id (body validated by PurchaseUpdateSchema). +- \`PUT /api/purchases/:id body: { programId?: number; customerEmail?: string; amountCents?: number; status?: string; refunded?: boolean; purchasedAt?: string; purchasedOn?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by PurchaseUpdateSchema). - \`DELETE /api/purchases/:id\` — [Hono] Delete a Purchase by id. Example: \`\`\`ts const created = await createPurchase(db, { id: 1 }); const found = await findPurchaseById(db, created.id); -const updated = await updatePurchase(db, created.id, { id: 1 }); +const updated = await updatePurchase(db, created.id, { programId: 1 }); const removed = await deletePurchaseById(db, created.id); \`\`\` @@ -281,36 +281,36 @@ const removed = await deletePurchaseById(db, created.id); \`import { WorkoutEvent, WorkoutEventInsertSchema, WorkoutEventUpdateSchema } from "./WorkoutEvent"\` - \`interface WorkoutEvent { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — The typed shape of a WorkoutEvent row, generated from its metadata. - \`WorkoutEventInsertSchema: ZodType<{ id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of a create / POST request (auto-generated PKs excluded). -- \`WorkoutEventUpdateSchema: ZodType<{ id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). +- \`WorkoutEventUpdateSchema: ZodType<{ programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }>\` — Zod schema validating the body of an update / PATCH request (all fields optional). \`import { findWorkoutEventById, listWorkoutEvents, createWorkoutEvent, updateWorkoutEvent, deleteWorkoutEventById } from "./WorkoutEvent.queries"\` - \`findWorkoutEventById(db: Db, id: number): Promise\` — Fetch a single WorkoutEvent by its primary key; null when not found. - \`listWorkoutEvents(db: Db, opts?: { limit?: number; offset?: number }): Promise\` — List WorkoutEvent rows with optional limit/offset paging. - \`createWorkoutEvent(db: Db, data: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Validate (via WorkoutEventInsertSchema) and insert a new WorkoutEvent. [throws: ZodError when data fails WorkoutEventInsertSchema validation.] -- \`updateWorkoutEvent(db: Db, id: number, patch: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Partially update a WorkoutEvent by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails WorkoutEventUpdateSchema validation.] +- \`updateWorkoutEvent(db: Db, id: number, patch: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }): Promise\` — Partially update a WorkoutEvent by primary key — writes only the assigned fields; null when not found. A renamed/dropped field is a compile error. [throws: ZodError when the patch fails WorkoutEventUpdateSchema validation.] - \`deleteWorkoutEventById(db: Db, id: number): Promise\` — Delete a WorkoutEvent by primary key; true when a row was removed. \`import { workoutEventRoutes } from "./WorkoutEvent.routes"\` - \`GET /api/workout_events -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — List WorkoutEvent (supports filter/sort/paging query params). - \`GET /api/workout_events/:id -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Fetch a single WorkoutEvent by id (404 when not found). - \`POST /api/workout_events body: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Create a WorkoutEvent (body validated by WorkoutEventInsertSchema). -- \`PATCH /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). -- \`PUT /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). +- \`PATCH /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). +- \`PUT /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). - \`DELETE /api/workout_events/:id\` — Delete a WorkoutEvent by id. \`import { registerWorkoutEventRoutes } from "./WorkoutEvent.routes.hono"\` - \`GET /api/workout_events -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] List WorkoutEvent (filter/sort/paging query params). - \`GET /api/workout_events/:id -> { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Fetch a single WorkoutEvent by id (404 when not found). - \`POST /api/workout_events body: { id: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Create a WorkoutEvent (body validated by WorkoutEventInsertSchema). -- \`PATCH /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). -- \`PUT /api/workout_events/:id body: { id?: number; programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). +- \`PATCH /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] Partially update a WorkoutEvent by id (body validated by WorkoutEventUpdateSchema). +- \`PUT /api/workout_events/:id body: { programId?: number; customerEmail?: string; weekNumber?: number; dayNumber?: number; eventType?: string; occurredAt?: string }\` — [Hono] PUT alias of PATCH — the same handler and partial body (validated by WorkoutEventUpdateSchema). - \`DELETE /api/workout_events/:id\` — [Hono] Delete a WorkoutEvent by id. Example: \`\`\`ts const created = await createWorkoutEvent(db, { id: 1 }); const found = await findWorkoutEventById(db, created.id); -const updated = await updateWorkoutEvent(db, created.id, { id: 1 }); +const updated = await updateWorkoutEvent(db, created.id, { programId: 1 }); const removed = await deleteWorkoutEventById(db, created.id); \`\`\` " @@ -343,7 +343,7 @@ import { createProgram, findProgramById, updateProgram, deleteProgramById } from const created = await createProgram(db, { id: 1 }); const found = await findProgramById(db, created.id); -const updated = await updateProgram(db, created.id, { id: 1 }); +const updated = await updateProgram(db, created.id, { title: "…" }); const removed = await deleteProgramById(db, created.id); \`\`\` @@ -413,7 +413,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -495,7 +494,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -513,7 +511,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -597,7 +594,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -615,7 +611,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | @@ -661,7 +656,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`title\` | \`string\` | | | | \`createdAt\` | \`string\` | | | " @@ -694,7 +688,7 @@ import { createPurchase, findPurchaseById, updatePurchase, deletePurchaseById } const created = await createPurchase(db, { id: 1 }); const found = await findPurchaseById(db, created.id); -const updated = await updatePurchase(db, created.id, { id: 1 }); +const updated = await updatePurchase(db, created.id, { programId: 1 }); const removed = await deletePurchaseById(db, created.id); \`\`\` @@ -790,7 +784,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -892,7 +885,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -915,7 +907,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1019,7 +1010,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1042,7 +1032,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1098,7 +1087,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`amountCents\` | \`number\` | | | @@ -1150,7 +1138,7 @@ import { createWorkoutEvent, findWorkoutEventById, updateWorkoutEvent, deleteWor const created = await createWorkoutEvent(db, { id: 1 }); const found = await findWorkoutEventById(db, created.id); -const updated = await updateWorkoutEvent(db, created.id, { id: 1 }); +const updated = await updateWorkoutEvent(db, created.id, { programId: 1 }); const removed = await deleteWorkoutEventById(db, created.id); \`\`\` @@ -1228,7 +1216,6 @@ Request body (data): | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1326,7 +1313,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1348,7 +1334,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1448,7 +1433,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1470,7 +1454,6 @@ Request body: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | @@ -1524,7 +1507,6 @@ Accepted fields: | Field | Type | Required | Notes | |---|---|---|---| -| \`id\` | \`number\` | | | | \`programId\` | \`number\` | | | | \`customerEmail\` | \`string\` | | | | \`weekNumber\` | \`number\` | | | diff --git a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts index 5b237c0213..ed91c3a735 100644 --- a/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts +++ b/server/typescript/packages/codegen-ts/test/templates/zod-validators.test.ts @@ -356,6 +356,22 @@ describe("renderZodValidators", () => { expect(update).toContain("name"); }); + test("SECURITY 1.2.x: an ASSIGNED primary key is in the Insert schema and OUT of the Update schema", () => { + // The URL names the row on PUT/PATCH. An assigned key (no @generation) is the caller's on + // create, but leaving it in the update shape let a client-sent `code` re-key the row. + const coupon = metaObject(OBJECT_SUBTYPE_ENTITY, "Coupon"); + coupon.addChild(metaField(FIELD_SUBTYPE_STRING, "code")); + coupon.addChild(metaField(FIELD_SUBTYPE_STRING, "label")); + const primary = meta(new TypeId(TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY), "primary"); + primary.setAttr("fields", ["code"]); + coupon.addChild(primary); + const out = renderZodValidators(coupon).toString(); + expect(schemaBody(out, "CouponInsertSchema")).toContain("code"); + const update = schemaBody(out, "CouponUpdateSchema"); + expect(update).not.toContain("code"); + expect(update).toContain("label"); + }); + test("FR-037 R1: no @mutability (⇒ readWrite) is in BOTH schemas — the no-churn default", () => { const out = customerWithMutability(undefined, "lifetimeValue"); expect(schemaBody(out, "CustomerInsertSchema")).toContain("lifetimeValue"); diff --git a/server/typescript/packages/docs-site/package.json b/server/typescript/packages/docs-site/package.json index a559b1c5e1..c99675193e 100644 --- a/server/typescript/packages/docs-site/package.json +++ b/server/typescript/packages/docs-site/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/docs-site", - "version": "1.2.0", + "version": "1.2.2", "description": "HTML documentation-site generator for metaobjects models — a browsable multi-page site (nav, search, per-object/package pages, kind-aware ER diagrams). Powers the `docs-site` generator (`docsSiteFile()`).", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts index 41ea9becb3..ca329a6eb9 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts @@ -26,6 +26,10 @@ export interface ExpectBody { namesUnordered?: string[]; row?: Record; hasId?: boolean; + /** The response body is an object and each listed key is MISSING or `null` on it: a value + * the client was not allowed to set never reached storage, so it never comes back in the + * echo (the 1.2.x input scenarios). */ + unset?: string[]; /** Assert the listed RESPONSE fields all equal EACH OTHER (never a literal) — * format-agnostic. Used by @autoSet scenarios to prove two stamped columns * hold the same value without pinning the exact timestamp. */ @@ -283,6 +287,14 @@ export function assertResponse( if (typeof row["id"] !== "number") throw new Error(`${scenarioName} / ${request.id}: expected numeric id in body, got: ${stringify(body)}`); } + if (want.unset) { + if (body === null || typeof body !== "object" || Array.isArray(body)) + throw new Error(`${scenarioName} / ${request.id}: expected object for unset, got: ${stringify(body)}`); + const row = body as Record; + const stored = want.unset.filter((k) => row[k] !== undefined && row[k] !== null); + if (stored.length > 0) + throw new Error(`${scenarioName} / ${request.id}: expected ${stored.join(", ")} unset, got: ${stringify(body)}`); + } if (want.fieldsEqual) { const row = body as Record; // Compare the raw response fields to EACH OTHER (format-agnostic) — every diff --git a/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts index 59885dde10..dbf642ed62 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-secret-fields-generated-server.ts @@ -65,7 +65,7 @@ const READ_PATHS = { }; export interface SecretFieldsSeed { - rows: Array<{ id: number; name: string; tokenHash: string; pinHash: string }>; + rows: Array<{ id: number; name: string; tokenHash: string; pinHash: string; role: string }>; } export interface GeneratedSecretFieldsServerHandle { @@ -124,7 +124,8 @@ export const db = drizzle(pool); "id" bigserial PRIMARY KEY, "name" varchar(100) NOT NULL, "token_hash" varchar(100), - "pin_hash" varchar(100) + "pin_hash" varchar(100), + "role" varchar(20) ); CREATE OR REPLACE VIEW "v_account" AS SELECT "id", "name", "pin_hash" FROM "accounts"; CREATE TABLE IF NOT EXISTS "payers" ("id" bigserial PRIMARY KEY); @@ -165,7 +166,7 @@ export const db = drizzle(pool); for (const r of seed.rows) await executeSql( connectionUri, - `INSERT INTO "accounts" ("id","name","token_hash","pin_hash") VALUES (${r.id}, ${str(r.name)}, ${str(r.tokenHash)}, ${str(r.pinHash)})`, + `INSERT INTO "accounts" ("id","name","token_hash","pin_hash","role") VALUES (${r.id}, ${str(r.name)}, ${str(r.tokenHash)}, ${str(r.pinHash)}, ${str(r.role)})`, ); // The seed names ids explicitly; keep the sequence ahead of them for the POST scenarios. await executeSql(connectionUri, `SELECT setval(pg_get_serial_sequence('accounts','id'), (SELECT MAX("id") FROM "accounts"));`); diff --git a/server/typescript/packages/integration-tests/src/api-contract-server.ts b/server/typescript/packages/integration-tests/src/api-contract-server.ts index 32cc2b3661..4342322804 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-server.ts @@ -188,6 +188,16 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis return reply; }); + // The keys a client may write, as the generated routes' Insert/Update schemas have them: the + // server-generated `id`, the @autoSet columns and any key the entity does not declare are + // DROPPED silently (never 400'd), and `issuedCurrency` (writeOnce) is settable on create only. + const CREATE_KEYS = ["name", "bio", "createdAt", "issuedCurrency"] as const; + const UPDATE_KEYS = ["name", "bio", "createdAt"] as const; + const pick = (body: unknown, keys: readonly string[]): Record => { + const src = (body ?? {}) as Record; + return Object.fromEntries(keys.filter((k) => Object.hasOwn(src, k)).map((k) => [k, src[k]])); + }; + // POST /api/authors fastify.post(ROUTE_BASE, async (req, reply) => { // #203 / ADR-0045 — the TS runtime ObjectManager does NOT stamp @autoSet @@ -195,7 +205,7 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // the generated lane's InsertSchema. Both onCreate + onUpdate columns get the // SAME captured now() on create; any caller-supplied value is ignored. const now = new Date().toISOString(); - const body = { ...(req.body as Record), autoCreatedAt: now, autoUpdatedAt: now }; + const body = { ...pick(req.body, CREATE_KEYS), autoCreatedAt: now, autoUpdatedAt: now }; const row = await om.create(ENTITY, body); reply.code(201).send(row); return reply; @@ -215,12 +225,7 @@ export async function startServer(connectionUri: string, root: MetaRoot): Promis // plain z.object() discards unknown keys). Dropping it also covers the // present-NULL arm for free: clearing is a write, so an explicit null must // not reach the column either. - const { - autoCreatedAt: _dropOnCreate, - issuedCurrency: _dropWriteOnce, - ...rest - } = (req.body as Record) ?? {}; - const body = { ...rest, autoUpdatedAt: new Date().toISOString() }; + const body = { ...pick(req.body, UPDATE_KEYS), autoUpdatedAt: new Date().toISOString() }; // A ValidationError (e.g. FR-035 present-null on a @required field) propagates // to the shared setErrorHandler above → 400 {error:"validation"}. const row = await om.update(ENTITY, Number(id), body, { ifMissing: "ignore" }); diff --git a/server/typescript/packages/metadata/package.json b/server/typescript/packages/metadata/package.json index 48a941c10e..b2ecbeddb6 100644 --- a/server/typescript/packages/metadata/package.json +++ b/server/typescript/packages/metadata/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/metadata", - "version": "1.2.0", + "version": "1.2.2", "description": "Metamodel loader, types, and constants for the MetaObjects standard.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/migrate-ts/package.json b/server/typescript/packages/migrate-ts/package.json index caf8274e57..c9340bfc10 100644 --- a/server/typescript/packages/migrate-ts/package.json +++ b/server/typescript/packages/migrate-ts/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/migrate-ts", - "version": "1.2.0", + "version": "1.2.2", "description": "Schema migration tooling for MetaObjects: diff metadata vs DB and emit SQL for Postgres and SQLite.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/render/package.json b/server/typescript/packages/render/package.json index 0fc26e9170..fdc0d33c59 100644 --- a/server/typescript/packages/render/package.json +++ b/server/typescript/packages/render/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/render", - "version": "1.2.0", + "version": "1.2.2", "description": "Deterministic text render engine for MetaObjects templates — Mustache (logic-less, the cross-port default) and Jinja (via nunjucks), provider-resolved partials, format-driven escaping, build-time payload drift check, zero core dependency.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/runtime-ts/package.json b/server/typescript/packages/runtime-ts/package.json index 376c24d585..c82fa8d88d 100644 --- a/server/typescript/packages/runtime-ts/package.json +++ b/server/typescript/packages/runtime-ts/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/runtime-ts", - "version": "1.2.0", + "version": "1.2.2", "description": "Node-side runtime helpers for MetaObjects: Fastify route builders, Drizzle filter/sort integration, Kysely drivers.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/sdk/package.json b/server/typescript/packages/sdk/package.json index 511723b6a9..29428e2d67 100644 --- a/server/typescript/packages/sdk/package.json +++ b/server/typescript/packages/sdk/package.json @@ -1,6 +1,6 @@ { "name": "@metaobjectsdev/sdk", - "version": "1.2.0", + "version": "1.2.2", "description": "Workspace helpers and agent-context assembly for MetaObjects projects.", "type": "module", "main": "./dist/index.js", diff --git a/server/typescript/packages/test-generators/src/templates/zod-validators.ts b/server/typescript/packages/test-generators/src/templates/zod-validators.ts index ea4945feae..1aafe1a916 100644 --- a/server/typescript/packages/test-generators/src/templates/zod-validators.ts +++ b/server/typescript/packages/test-generators/src/templates/zod-validators.ts @@ -386,6 +386,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { * The field SET the `UpdateSchema` accepts — same iteration + skip rules * as `insertSchemaFields`, but mirroring the UpdateSchema branch of * `renderZodValidators`: + * • a primary-key field is OMITTED, generated or assigned (the URL names the row); * • a TPH subtype's @discriminator field is OMITTED (clients can't change subtype); * • @autoSet onCreate fields are OMITTED (creation timestamps are immutable); * • @autoSet onUpdate fields are present + optional (server fills them); @@ -393,6 +394,7 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] { */ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { const autoGenPkFields = autoGenPkFieldNames(obj); + const pkFields = primaryKeyFieldNames(obj); const tphPin = tphDiscriminatorPin(obj); const out: SchemaFieldShape[] = []; for (const child of obj.fields()) { @@ -408,6 +410,10 @@ export function updateSchemaFields(obj: MetaObject): SchemaFieldShape[] { // edit form for an entity carrying one. Clearing counts as a write, so the // FR-035 present-null arm never reaches it either. if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): a primary-key field is the row's identity, which the URL names. An + // ASSIGNED key leaves the update shape exactly like a generated one, so a client-sent + // key is stripped and can never re-key the row. + if (pkFields.has(child.name)) continue; // TPH subtype discriminator: omitted from the update schema entirely. if (tphPin !== undefined && child.name === tphPin.fieldName) continue; const autoSet = child.attr(FIELD_ATTR_AUTO_SET); @@ -491,6 +497,10 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // below; a value presented on PATCH is stripped, not 400'd (see // updateSchemaFields for the grounding). if (isWriteOnceMutability(child)) continue; + // SECURITY (1.2.x): the row's identity is the URL's. An ASSIGNED primary key leaves the + // update shape too (a generated one never entered it), so a client-sent key is stripped + // on PUT/PATCH and can never re-key the row. Mirrors updateSchemaFields. + if (pkFields.has(child.name)) continue; // Update schema: @autoSet onCreate → omit entirely; onUpdate → transform if (autoSet === AUTO_SET_ON_CREATE) { @@ -515,10 +525,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. PK fields never reach this branch (skipped above). + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } From ce2cee682ecd05ab5378cff10e0e8e1edf0ef492 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 15:50:51 -0500 Subject: [PATCH 09/31] fix(cli): prune emptied skill directories after a --skills-dir move; install every module the Kotlin integration lane needs meta init --refresh-docs --skills-dir removed the untouched skill files at the old path but left their directories behind, which read as the old copy still being there. Walk up from each pruned file and remove the directories it left empty, stopping at the first that still holds anything and never removing the skills directory itself. scripts/integration-test.sh kotlin installed only metadata and codegen-kotlin, so on an .m2 without the other metaobjects artifacts integration-tests-kotlin failed to resolve core-spring and om, and codegen-kotlin-exposed1x-check metadata-ktx. --- CHANGELOG.md | 3 +- scripts/integration-test.sh | 6 +++- .../packages/cli/src/commands/init.ts | 29 +++++++++++++++++-- .../cli/test/unit/init-agent-context.test.ts | 20 ++++++++++++- 4 files changed, 52 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5644f2c35c..ca53b5b53f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -98,7 +98,8 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi Claude Code reads. `--skills-dir .agents/skills` (or any project-relative directory) puts them where another assistant looks; the choice is recorded in `.metaobjects/.agent-context.json`, so a later `--refresh-docs` keeps it, and moving the directory prunes the untouched copies at - the old one. The always-on file names the directory, so an assistant that does not load + the old one along with the skill directories that leaves empty (the old skills directory + itself stays, and so does any skill directory still holding a file of yours). The always-on file names the directory, so an assistant that does not load skills opens `//SKILL.md` by path. A directory outside the project is refused. - **[What differs by language](docs/features/language-differences.md).** One page lists what the same model does differently per port: that schema migrations and database checks need diff --git a/scripts/integration-test.sh b/scripts/integration-test.sh index 08c9311f39..c3c155a43c 100755 --- a/scripts/integration-test.sh +++ b/scripts/integration-test.sh @@ -154,8 +154,12 @@ run_kotlin() { echo "==> Kotlin persistence conformance" # See run_java: install the SNAPSHOT module deps to .m2 (serially) before the # docker-only integration-tests-kotlin module (also excluded from the reactor). + # Every metaobjects-* artifact either out-of-reactor module below depends on is + # listed, so a standalone `kotlin` run resolves from an .m2 that has none of them: + # integration-tests-kotlin needs core-spring and om beside codegen-kotlin, and + # codegen-kotlin-exposed1x-check needs metadata-ktx. ( cd server/java \ - && { [ "${MO_CI_JAVA_INSTALLED:-0}" = "1" ] || mvn -q -DskipTests install -pl metadata,codegen-kotlin -am; } \ + && { [ "${MO_CI_JAVA_INSTALLED:-0}" = "1" ] || mvn -q -DskipTests install -pl metadata,metadata-ktx,om,core-spring,codegen-kotlin -am; } \ && mvn -f integration-tests-kotlin/pom.xml test $(jvm_test_args) ) || FAIL=1 # Issue #390 — exposedApi=1 (Exposed 1.x) compile + round-trip check. Same # docker-only / excluded-from-the-reactor shape as integration-tests-kotlin diff --git a/server/typescript/packages/cli/src/commands/init.ts b/server/typescript/packages/cli/src/commands/init.ts index efaae02edd..4ff25e6d2e 100644 --- a/server/typescript/packages/cli/src/commands/init.ts +++ b/server/typescript/packages/cli/src/commands/init.ts @@ -1,6 +1,5 @@ -import { mkdir, writeFile, readFile, readdir, stat, rm } from "node:fs/promises"; -import { join } from "node:path"; -import { dirname } from "node:path"; +import { mkdir, writeFile, readFile, readdir, stat, rm, rmdir } from "node:fs/promises"; +import { join, dirname, posix } from "node:path"; import { existsSync as existsSyncWrap, readFileSync as readFileSyncWrap } from "node:fs"; import { INIT_DEFAULT_LIMIT, INIT_MAX_LIMIT, UNKNOWN_FIELDS_REJECT } from "@metaobjectsdev/metadata"; import { DEFAULT_CONFIG, ConfigSchema, saveConfig, PACKAGE_MANIFEST_FILE, DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, DEPS_DIR, LOCK_FILE } from "@metaobjectsdev/sdk"; @@ -349,6 +348,22 @@ async function removeUnlessDryRun(cwd: string, dryRun: boolean, path: string): P await rm(join(cwd, path), { force: true }); } +/** Remove the directories above a pruned file `rel` (repo-relative, posix) that are now empty, + * walking up to, and never including, the skills directory that holds it. A file outside every + * skills directory is left alone, and the walk stops at the first directory with anything in it. */ +async function removeEmptySkillDirs(cwd: string, rel: string, skillRoots: readonly string[]): Promise { + const root = skillRoots.find((r) => rel.startsWith(`${r}/`)); + if (root === undefined) return; + for (let dir = posix.dirname(rel); dir !== root && dir.startsWith(`${root}/`); dir = posix.dirname(dir)) { + try { + if ((await readdir(join(cwd, dir))).length > 0) return; + await rmdir(join(cwd, dir)); // removes only an empty directory, never anything in one + } catch { + return; // already gone, unreadable, or filled meanwhile: nothing more to tidy + } + } +} + /** "would be VERBED" during a dry run, plain VERBED otherwise — the one tense * marker every reported write shares, so each call site states only its own * past participle instead of writing out both tenses of the whole sentence. */ @@ -438,6 +453,14 @@ async function writeAgentContext(opts: InitOptions, result: InitResult): Promise await removeUnlessDryRun(opts.cwd, dryRun, orphan); result.removed.push(orphan); } + // A skill directory whose every file was just pruned (the skills moved with --skills-dir, or + // the stack dropped a skill) would otherwise stay behind as an empty husk that reads as "the + // old copy is still there". Remove the ones left empty, never the skills directory itself: + // it may hold the user's own skills. + if (!dryRun) { + const skillRoots = [skillsDir, normalizeSkillsDir(prior?.skillsDir ?? DEFAULT_SKILLS_DIR)]; + for (const orphan of decision.prunes) await removeEmptySkillDirs(opts.cwd, orphan, skillRoots); + } // Hand-edited ones are never deleted — losing an adopter's writing is worse than leaving // a stale file behind — so they are named, with the reason and the remedy. for (const orphan of decision.removed) { diff --git a/server/typescript/packages/cli/test/unit/init-agent-context.test.ts b/server/typescript/packages/cli/test/unit/init-agent-context.test.ts index c500e071d1..05274121e7 100644 --- a/server/typescript/packages/cli/test/unit/init-agent-context.test.ts +++ b/server/typescript/packages/cli/test/unit/init-agent-context.test.ts @@ -1,5 +1,5 @@ import { test, expect, describe, beforeEach, afterEach } from "bun:test"; -import { mkdtempSync, rmSync, existsSync, readFileSync, writeFileSync } from "node:fs"; +import { mkdtempSync, rmSync, existsSync, readFileSync, writeFileSync, readdirSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { init } from "../../src/commands/init.js"; @@ -122,6 +122,24 @@ describe("init() — agent-context scaffolding", () => { expect(existsSync(join(cwd, ".claude/skills/metaobjects-codegen/SKILL.md"))).toBe(false); }); + test("moving the skills leaves no empty skill directories behind at the old path", async () => { + await init({ cwd, servers: ["python"], clients: [] }); + expect(readdirSync(join(cwd, ".claude/skills")).length).toBeGreaterThan(0); + await init({ cwd, refreshDocs: true, docsOnly: true, skillsDir: ".agents/skills" }); + // Every skill directory (and its references/) is gone; the skills directory itself stays, + // since it may hold the user's own skills. + expect(readdirSync(join(cwd, ".claude/skills"))).toEqual([]); + expect(existsSync(join(cwd, ".agents/skills/metaobjects-codegen/SKILL.md"))).toBe(true); + }); + + test("a skill directory that still holds a file of the user's is kept with that file", async () => { + await init({ cwd, servers: ["python"], clients: [] }); + writeFileSync(join(cwd, ".claude/skills/metaobjects-codegen/my-notes.md"), "mine\n", "utf8"); + await init({ cwd, refreshDocs: true, docsOnly: true, skillsDir: ".agents/skills" }); + expect(readdirSync(join(cwd, ".claude/skills"))).toEqual(["metaobjects-codegen"]); + expect(readdirSync(join(cwd, ".claude/skills/metaobjects-codegen"))).toEqual(["my-notes.md"]); + }); + test("a --skills-dir outside the project is refused before anything is written", async () => { await expect(init({ cwd, servers: ["python"], clients: [], skillsDir: "../elsewhere" })).rejects.toThrow(/--skills-dir/); expect(existsSync(join(cwd, ".metaobjects"))).toBe(false); From 871549b4e836491b4aa99a7f6c205ec2b3a3067f Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 15:47:27 -0500 Subject: [PATCH 10/31] test(render): hold the email helper in the runtime-drift corpus case Every generated render path in the five ports already raises ERR_VAR_NOT_ON_PAYLOAD for a template variable not on the payload (1.3.0, #520), and runtime-drift/ held the document and prompt helpers. The @kind: email helper was not in the case. Add WelcomeEmail plus email-subject/, email-html/ and email-text/ drift directories, one drifted part each, and check every part raises in the TS, Python, C#, Java and Kotlin runners. --- CHANGELOG.md | 9 ++++++ .../README.md | 7 +++++ .../email-html/emails/welcome.html.mustache | 1 + .../emails/welcome.subject.mustache | 1 + .../email-html/emails/welcome.txt.mustache | 1 + .../emails/welcome.html.mustache | 1 + .../emails/welcome.subject.mustache | 1 + .../email-subject/emails/welcome.txt.mustache | 1 + .../email-text/emails/welcome.html.mustache | 1 + .../emails/welcome.subject.mustache | 1 + .../email-text/emails/welcome.txt.mustache | 1 + .../runtime-drift/meta.json | 10 +++++++ .../RenderHelperConformanceTests.cs | 28 ++++++++++++++++++- .../KotlinRenderHelperConformanceTest.kt | 16 ++++++++++- .../GeneratedRenderHelperConformanceTest.java | 18 +++++++++++- .../test_render_helper_runtime_drift.py | 19 ++++++++++++- .../codegen-ts/test/runtime-drift.test.ts | 19 +++++++++++++ 17 files changed, 131 insertions(+), 4 deletions(-) create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.html.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.subject.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.txt.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.html.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.subject.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.txt.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.html.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.subject.mustache create mode 100644 fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.txt.mustache diff --git a/CHANGELOG.md b/CHANGELOG.md index ca53b5b53f..91e0e03643 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,6 +107,15 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi filter extras, field names in code, generated data access and hand-edit handling. Every port page now says up front that migrations and `verify --db` run in Node, and the README's getting-started path says it for Java, Kotlin, C# and Python teams. +- **The shared render-drift case now holds the email helper too.** 1.3.0 made a template + variable that is not on the payload raise `ERR_VAR_NOT_ON_PAYLOAD` at render time, and + `fixtures/template-output-render-conformance/runtime-drift/` held it for the `@kind: document` + helper and the `template.prompt` helper (the TypeScript `promptRender()` handle and the Python + helper over a Mustache body among them). A re-audit of every generated render path in the five + ports found none that still renders an unknown variable as empty, but the `@kind: email` + helper was not in the case. It now is: `runtime-drift/email-subject/`, `email-html/` and + `email-text/` each drift one part of a `WelcomeEmail`, and every port's runner checks that + the subject, the html body and the text body each raise. No generated code changes. ### Changed diff --git a/fixtures/template-output-render-conformance/README.md b/fixtures/template-output-render-conformance/README.md index dbd6816630..08b7d5ddd4 100644 --- a/fixtures/template-output-render-conformance/README.md +++ b/fixtures/template-output-render-conformance/README.md @@ -232,3 +232,10 @@ same ref `pages/welcome`. Each port: 2. renders with `templates/` → `"Hello Ada"`; 3. renders with `runtime-drift/templates/` (`pages/welcome.mustache` = `Hello {{nmae}}`) → the helper throws, and the message contains `ERR_VAR_NOT_ON_PAYLOAD (nmae)`. + +The same `meta.json` also declares a `template.output @kind: email` (`WelcomeEmail`) over the +`emails/welcome.*` refs. Each port generates its helper from the clean `templates/`, renders it +clean, then renders it once against each of `runtime-drift/email-subject/`, `email-html/` and +`email-text/`. Each of those directories is a full copy of the three email parts with exactly one +part drifted to `{{nmae}}`, so the subject, the html body and the text body are each shown to +raise `ERR_VAR_NOT_ON_PAYLOAD (nmae)`. diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.html.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.html.mustache new file mode 100644 index 0000000000..266f2faa03 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.html.mustache @@ -0,0 +1 @@ +

Hi {{nmae}}

\ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.subject.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.subject.mustache new file mode 100644 index 0000000000..afd459a587 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.subject.mustache @@ -0,0 +1 @@ +Welcome {{name}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.txt.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.txt.mustache new file mode 100644 index 0000000000..7bfb419353 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-html/emails/welcome.txt.mustache @@ -0,0 +1 @@ +Hi {{name}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.html.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.html.mustache new file mode 100644 index 0000000000..3cdde3283e --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.html.mustache @@ -0,0 +1 @@ +

Hi {{name}}

\ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.subject.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.subject.mustache new file mode 100644 index 0000000000..d169934941 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.subject.mustache @@ -0,0 +1 @@ +Welcome {{nmae}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.txt.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.txt.mustache new file mode 100644 index 0000000000..7bfb419353 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-subject/emails/welcome.txt.mustache @@ -0,0 +1 @@ +Hi {{name}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.html.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.html.mustache new file mode 100644 index 0000000000..3cdde3283e --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.html.mustache @@ -0,0 +1 @@ +

Hi {{name}}

\ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.subject.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.subject.mustache new file mode 100644 index 0000000000..afd459a587 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.subject.mustache @@ -0,0 +1 @@ +Welcome {{name}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.txt.mustache b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.txt.mustache new file mode 100644 index 0000000000..25385b9070 --- /dev/null +++ b/fixtures/template-output-render-conformance/runtime-drift/email-text/emails/welcome.txt.mustache @@ -0,0 +1 @@ +Hi {{nmae}} \ No newline at end of file diff --git a/fixtures/template-output-render-conformance/runtime-drift/meta.json b/fixtures/template-output-render-conformance/runtime-drift/meta.json index fccbd58864..d82b33033b 100644 --- a/fixtures/template-output-render-conformance/runtime-drift/meta.json +++ b/fixtures/template-output-render-conformance/runtime-drift/meta.json @@ -19,6 +19,16 @@ "@format": "text" } }, + { + "template.output": { + "name": "WelcomeEmail", + "@kind": "email", + "@payloadRef": "Welcome", + "@subjectRef": "emails/welcome.subject", + "@htmlBodyRef": "emails/welcome.html", + "@textBodyRef": "emails/welcome.txt" + } + }, { "template.prompt": { "name": "WelcomePrompt", diff --git a/server/csharp/MetaObjects.Codegen.Tests/RenderHelperConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/RenderHelperConformanceTests.cs index 148dff37ea..77747ba279 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/RenderHelperConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/RenderHelperConformanceTests.cs @@ -303,7 +303,7 @@ public void Drift_case_FAILS_codegen() // runtime-drift/ → a variable not on the payload RAISES at render time. The helpers are // generated against the clean templates/; runtime-drift/templates/ serves {{nmae}} for // name (text edited after codegen, which the build-time gate cannot see). Covers a - // template.output and a template.prompt over the same payload. + // template.output, a template.prompt and an email template.output over the same payload. // --------------------------------------------------------------------- [Theory] @@ -328,6 +328,32 @@ public void Runtime_drift_variable_not_on_payload_raises(string template, string Assert.Contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)", ex.InnerException!.Message); } + // runtime-drift/email-/ drifts one part of the WelcomeEmail helper each, so the + // subject, the html body and the text body are each shown to raise. + [Theory] + [InlineData("subject")] + [InlineData("html")] + [InlineData("text")] + public void Runtime_drift_email_part_raises(string part) + { + var corpus = Corpus(); + var root = LoadFromFile(Path.Combine(corpus, "runtime-drift", "meta.json")); + var clean = Path.Combine(corpus, "templates"); + var drifted = Path.Combine(corpus, "runtime-drift", "email-" + part); + + var file = Assert.Single(new RenderHelperGenerator(clean).Generate(Ctx(root)), f => f.Path == "WelcomeEmail.render.cs"); + var asm = CompileToAssembly([file.Content, .. GeneratedValueObjects.Sources(root)]); + + var render = asm.GetType("Acme.Generated.WelcomeEmailRenderHelper")!.GetMethod("Render")!; + var payload = MakeWelcome(asm, "Ada"); + var doc = (EmailDocument)render.Invoke(null, [payload, new FilesystemProvider(clean)])!; + Assert.Equal("Welcome Ada", doc.Subject); + + var ex = Assert.Throws( + () => render.Invoke(null, [payload, new FilesystemProvider(drifted)])); + Assert.Contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)", ex.InnerException!.Message); + } + // --------------------------------------------------------------------- // Helpers // --------------------------------------------------------------------- diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRenderHelperConformanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRenderHelperConformanceTest.kt index 9a579e1cf8..4a33c65de5 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRenderHelperConformanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRenderHelperConformanceTest.kt @@ -92,7 +92,7 @@ class KotlinRenderHelperConformanceTest { // runtime-drift/ → a variable not on the payload RAISES at render time. The helpers are // generated against the clean templates/; runtime-drift/templates/ serves {{nmae}} for // name (text edited after codegen, which the build-time gate cannot see). Covers a - // template.output and a template.prompt over the same payload. + // template.output, a template.prompt and an email template.output over the same payload. // ------------------------------------------------------------------------- @Test fun `runtime drift raises for output and prompt helpers`() { val outDir = Files.createTempDirectory("krhc-rdrift-") @@ -117,6 +117,20 @@ class KotlinRenderHelperConformanceTest { } assertTrue(ex.cause?.message?.contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)") == true, "$name: ${ex.cause?.message}") } + + // runtime-drift/email-/ drifts one part of WelcomeEmail each: the subject, + // the html body and the text body must each raise. + val emailClass = cl.loadClass("acme.ai.prompts.WelcomeEmailRenderHelper") + val emailInstance = emailClass.getDeclaredField("INSTANCE").get(null) + val email = emailClass.getDeclaredMethod("render", payloadClass, providerClass) + email.invoke(emailInstance, payload, fsProvider(cl, clean)) + for (part in listOf("subject", "html", "text")) { + val dir = corpus.resolve("runtime-drift").resolve("email-$part") + val ex = kotlin.test.assertFailsWith("email $part") { + email.invoke(emailInstance, payload, fsProvider(cl, dir)) + } + assertTrue(ex.cause?.message?.contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)") == true, "email $part: ${ex.cause?.message}") + } } finally { outDir.toFile().deleteRecursively() } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedRenderHelperConformanceTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedRenderHelperConformanceTest.java index b3e0d5eb7b..da3c7c9d30 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedRenderHelperConformanceTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedRenderHelperConformanceTest.java @@ -280,7 +280,7 @@ public void driftCaseFailsCodegen() throws Exception { // runtime-drift/ -> a variable not on the payload RAISES at render time. The helpers are // generated against the clean templates/; runtime-drift/templates/ serves {{nmae}} for // name (text edited after codegen, which the build-time gate cannot see). Covers a - // template.output and a template.prompt over the same payload. + // template.output, a template.prompt and an email template.output over the same payload. // ------------------------------------------------------------------------- @Test public void runtimeDriftRaisesForOutputAndPromptHelpers() throws Exception { @@ -311,6 +311,22 @@ public void runtimeDriftRaisesForOutputAndPromptHelpers() throws Exception { assertTrue(helper + ": " + msg, msg.contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)")); } } + + // runtime-drift/email-/ drifts one part of WelcomeEmail each: the subject, + // the html body and the text body must each raise. + Method email = cl.loadClass("acme.ai.prompts.WelcomeEmailRenderHelper") + .getMethod("render", payloadClass, providerClass); + email.invoke(null, payload, newFilesystemProvider(clean)); + for (String part : new String[]{ "subject", "html", "text" }) { + try { + Object out = email.invoke(null, payload, + newFilesystemProvider(CORPUS.resolve("runtime-drift").resolve("email-" + part))); + fail("email " + part + " rendered a variable not on the payload instead of raising: " + out); + } catch (java.lang.reflect.InvocationTargetException e) { + String msg = String.valueOf(e.getCause().getMessage()); + assertTrue("email " + part + ": " + msg, msg.contains("ERR_VAR_NOT_ON_PAYLOAD (nmae)")); + } + } } } diff --git a/server/python/tests/codegen/test_render_helper_runtime_drift.py b/server/python/tests/codegen/test_render_helper_runtime_drift.py index 71b563055e..5d6d12612e 100644 --- a/server/python/tests/codegen/test_render_helper_runtime_drift.py +++ b/server/python/tests/codegen/test_render_helper_runtime_drift.py @@ -4,7 +4,8 @@ generated against the clean ``templates/``; ``runtime-drift/templates/`` holds the same ref with ``{{nmae}}`` for ``name``, standing in for text edited after codegen. The build-time gate cannot see that, so the helper itself must raise at render time (``ERR_VAR_NOT_ON_PAYLOAD``) -instead of rendering ``Hello `` — for ``template.output`` and ``template.prompt``, Mustache. +instead of rendering ``Hello `` — for ``template.output`` (document and email) and +``template.prompt``, Mustache. """ from __future__ import annotations @@ -39,3 +40,19 @@ def test_helper_raises_on_variable_not_on_payload(tmp_path, module, fn) -> None: assert render({"name": "Ada"}, FilesystemProvider(CLEAN)) == "Hello Ada" with pytest.raises(RenderError, match=r"ERR_VAR_NOT_ON_PAYLOAD \(nmae\)"): render({"name": "Ada"}, FilesystemProvider(DRIFTED)) + + +@pytest.mark.parametrize("part", ["subject", "html", "text"]) +def test_email_helper_raises_on_variable_not_on_payload(tmp_path, part) -> None: + """Each part of a ``@kind: email`` helper (subject, html body, text body) raises.""" + root = _load_root(CORPUS / "runtime-drift" / "meta.json") + module = "welcome_email_render_helper" + files = [f for f in RenderHelperGenerator(CLEAN).generate(_ctx(root)) if f.path == f"{module}.py"] + assert len(files) == 1 + _materialize_and_import([*files, *_models(root)], tmp_path) + render = getattr(import_module(f"_rh_conf_pkg.{module}"), "render_welcome_email") + + doc = render({"name": "Ada"}, FilesystemProvider(CLEAN)) + assert (doc.subject, doc.html_body, doc.text_body) == ("Welcome Ada", "

Hi Ada

", "Hi Ada") + with pytest.raises(RenderError, match=r"ERR_VAR_NOT_ON_PAYLOAD \(nmae\)"): + render({"name": "Ada"}, FilesystemProvider(str(CORPUS / "runtime-drift" / f"email-{part}"))) diff --git a/server/typescript/packages/codegen-ts/test/runtime-drift.test.ts b/server/typescript/packages/codegen-ts/test/runtime-drift.test.ts index ef1bde8e95..fc30d166f6 100644 --- a/server/typescript/packages/codegen-ts/test/runtime-drift.test.ts +++ b/server/typescript/packages/codegen-ts/test/runtime-drift.test.ts @@ -44,6 +44,25 @@ describe("runtime drift — a variable not on the payload raises at render time" expect(() => mod.renderWelcomePage({ name: "Ada" }, drifted())).toThrow(/ERR_VAR_NOT_ON_PAYLOAD \(nmae\)/); }); + test("render-helper email (template.output @kind email) — each part raises", async () => { + const root = await load(); + const src = renderRenderHelper(root, "WelcomeEmail", clean()); + const dir = mkdtempSync(join(import.meta.dir, "rd-email-")); + TEMP_DIRS.push(dir); + writeFileSync(join(dir, "payloads.ts"), PAYLOADS); + writeFileSync(join(dir, "WelcomeEmail.render.ts"), src); + const mod = await import(join(dir, "WelcomeEmail.render.ts")); + expect(mod.renderWelcomeEmail({ name: "Ada" }, clean())).toEqual({ + subject: "Welcome Ada", + htmlBody: "

Hi Ada

", + textBody: "Hi Ada", + }); + for (const part of ["subject", "html", "text"]) { + const provider = new FileSystemProvider(join(CORPUS, "runtime-drift", `email-${part}`)); + expect(() => mod.renderWelcomeEmail({ name: "Ada" }, provider)).toThrow(/ERR_VAR_NOT_ON_PAYLOAD \(nmae\)/); + } + }); + test("promptRender() handle (template.prompt)", async () => { const root = await load(); const gen = promptRender(); From ec01e09fe0b6bff3cc9ff69318118684fd1f52ba Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 18:55:16 -0500 Subject: [PATCH 11/31] feat(init): dotnet meta init and metaobjects init write the api block TypeScript meta init writes A new project gets api.pagination 50/1000 and api.unknownFields reject in .metaobjects/config.json from every port's init; an existing config is never edited. --- .../PackageLayoutConfigTests.cs | 5 +++ server/csharp/MetaObjects.Cli/InitCommand.cs | 18 ++++++--- server/python/src/metaobjects/cli.py | 33 +++++++++++++++- server/python/tests/codegen/test_cli_init.py | 39 +++++++++++++++++++ 4 files changed, 88 insertions(+), 7 deletions(-) diff --git a/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs b/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs index 09b32bdcf0..720e575730 100644 --- a/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs @@ -65,6 +65,11 @@ public void Init_writes_the_opt_ins_for_a_new_project_and_leaves_an_existing_con var cfg = NeutralConfig.Read(fresh, validateNaming: false)!; Assert.Equal("package", cfg.OutputLayout); Assert.True(cfg.PackageNamespaces.FromMetadata); + // The same new-project api block TypeScript `meta init` writes: a 50/1000 page size and + // unknownFields spelled out at "reject" (a default left implicit would hide the choice). + Assert.Equal(new ApiPagination(50, 1000), cfg.Api.Pagination); + using (var doc = System.Text.Json.JsonDocument.Parse(File.ReadAllText(Path.Combine(fresh, ".metaobjects", "config.json")))) + Assert.Equal("reject", doc.RootElement.GetProperty("api").GetProperty("unknownFields").GetString()); File.WriteAllText(Path.Combine(fresh, ".metaobjects", "config.json"), "{ \"schema_version\": 1 }"); InitCommand.Run(fresh); diff --git a/server/csharp/MetaObjects.Cli/InitCommand.cs b/server/csharp/MetaObjects.Cli/InitCommand.cs index 1083a08e13..bec747ba1c 100644 --- a/server/csharp/MetaObjects.Cli/InitCommand.cs +++ b/server/csharp/MetaObjects.Cli/InitCommand.cs @@ -9,6 +9,7 @@ // with the generators it ejected already in the list. using MetaObjects.Codegen; +using MetaObjects.Meta; namespace MetaObjects.Cli; @@ -121,12 +122,17 @@ private static string DocsEntry(string root) => ? n.Replace("\\", "").Replace("\"", "") : "project") + "\" }"; /// The project config a new project starts with: files written under their metadata - /// package, and each packaged type in its package's namespace. An existing config is the - /// project's and is never edited. + /// package, each packaged type in its package's namespace, and the api block TypeScript + /// meta init writes: a page size on every generated list route, and unknownFields + /// written out at its default so the choice is visible. An existing config is the project's and is + /// never edited, so an upgraded project gains no page size. public const string ConfigRel = ".metaobjects/config.json"; - public const string ConfigJson = - "{\n \"schema_version\": 1,\n \"csharp\": {\n \"outputLayout\": \"package\",\n \"packageNamespaces\": true\n }\n}\n"; + public static readonly string ConfigJson = + "{\n \"schema_version\": 1,\n \"csharp\": {\n \"outputLayout\": \"package\",\n \"packageNamespaces\": true\n },\n" + + " \"api\": {\n \"pagination\": {\n" + + $" \"defaultLimit\": {ApiSettings.InitDefaultLimit},\n \"maxLimit\": {ApiSettings.InitMaxLimit}\n" + + " },\n \"unknownFields\": \"reject\"\n }\n}\n"; public static IReadOnlyList Run(string root, bool withDocs = false) { @@ -148,7 +154,9 @@ public static IReadOnlyList Run(string root, bool withDocs = false) { Directory.CreateDirectory(Path.GetDirectoryName(configPath)!); File.WriteAllText(configPath, ConfigJson); - lines.Add($" wrote {ConfigRel} (outputLayout \"package\", packageNamespaces true)"); + lines.Add( + $" wrote {ConfigRel} (outputLayout \"package\", packageNamespaces true, api.pagination " + + $"{ApiSettings.InitDefaultLimit}/{ApiSettings.InitMaxLimit}, api.unknownFields \"reject\")"); } lines.Add(""); lines.Add( diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index 8779648390..1badaa9131 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -83,6 +83,16 @@ summary_line, ) from metaobjects.config.neutral_config import find_project_dir, read_neutral_config +from metaobjects.api_settings import ( + API_KEY_PAGINATION, + API_KEY_UNKNOWN_FIELDS, + INIT_DEFAULT_LIMIT, + INIT_MAX_LIMIT, + PAGINATION_KEY_DEFAULT_LIMIT, + PAGINATION_KEY_MAX_LIMIT, + UNKNOWN_FIELDS_REJECT, + ApiSettings, +) from metaobjects.field_lint import lint_duplicate_fields, lint_reference_fields from metaobjects.loader.meta_data_loader import LoadResult from metaobjects.loader.sources import FileSource @@ -162,7 +172,6 @@ verify as render_verify, ) from metaobjects.shared.base_types import TYPE_OBJECT, TYPE_TEMPLATE -from metaobjects.api_settings import ApiSettings from metaobjects.wire import WireFormat, wire_block, wire_candidates @@ -1351,8 +1360,27 @@ def _cmd_eject(args: argparse.Namespace) -> int: """ +def _init_neutral_config() -> str: + """The port-neutral ``.metaobjects/config.json`` a new project starts with: the same ``api`` + block TypeScript ``meta init`` writes, a page size on every generated list route and + ``unknownFields`` spelled out at its default so the choice is visible. Never written over an + existing file, so an upgraded project gains no page size.""" + body = { + "schema_version": 1, + "api": { + API_KEY_PAGINATION: { + PAGINATION_KEY_DEFAULT_LIMIT: INIT_DEFAULT_LIMIT, + PAGINATION_KEY_MAX_LIMIT: INIT_MAX_LIMIT, + }, + API_KEY_UNKNOWN_FIELDS: UNKNOWN_FIELDS_REJECT, + }, + } + return json.dumps(body, indent=2) + "\n" + + def _cmd_init(args: argparse.Namespace) -> int: - """Scaffold ``metaobjects.config.yaml`` and the ``metaobjects/`` metadata directory. + """Scaffold ``metaobjects.config.yaml``, the port-neutral ``.metaobjects/config.json`` and the + ``metaobjects/`` metadata directory. Never overwrites: a file that exists is reported and left alone. ``--with-docs`` adds a ``docs`` target running ``api-docs``; without it no documentation generator is wired. @@ -1363,6 +1391,7 @@ def _cmd_init(args: argparse.Namespace) -> int: kept: list[str] = [] for rel, text in ( (_CONFIG_NAME, config), + (".metaobjects/config.json", _init_neutral_config()), ("metaobjects/meta.app.yaml", _INIT_METADATA), ): path = root / rel diff --git a/server/python/tests/codegen/test_cli_init.py b/server/python/tests/codegen/test_cli_init.py index 2ce6f4d199..dcab4ba498 100644 --- a/server/python/tests/codegen/test_cli_init.py +++ b/server/python/tests/codegen/test_cli_init.py @@ -1,9 +1,12 @@ """`metaobjects init` scaffolds a config and a metadata directory, and never overwrites.""" from __future__ import annotations +import json from pathlib import Path +from metaobjects.api_settings import ApiPagination from metaobjects.cli import main +from metaobjects.config.neutral_config import read_neutral_config def test_init_writes_a_config_with_no_docs_generator_and_a_loadable_model( @@ -67,3 +70,39 @@ def test_api_prefix_with_a_quote_or_backslash_is_refused(tmp_path: Path, capsys) cfg.write_text(cfg.read_text() + "apiPrefix: '/v1\"x'\n") assert main(["gen", "--config", str(cfg)]) != 0 assert "apiPrefix" in capsys.readouterr().err + + +def test_init_writes_the_new_project_api_block_that_typescript_meta_init_writes( + tmp_path: Path, capsys +) -> None: + # Oracle: TypeScript `meta init` writes `api: {pagination: {defaultLimit: 50, maxLimit: 1000}, + # unknownFields: "reject"}` into .metaobjects/config.json for a new project. + assert main(["init", "--root", str(tmp_path)]) == 0 + assert "wrote .metaobjects/config.json" in capsys.readouterr().out + neutral = read_neutral_config(tmp_path) + assert neutral is not None + assert neutral.api.pagination == ApiPagination(default_limit=50, max_limit=1000) + raw = json.loads((tmp_path / ".metaobjects" / "config.json").read_text()) + assert raw["api"]["unknownFields"] == "reject" + # The page size reaches a generated router. + (tmp_path / "metaobjects" / "meta.app.yaml").write_text( + "metadata:\n package: app\n children:\n" + " - object.entity:\n name: Author\n children:\n" + " - source.rdb: { table: authors }\n" + " - field.long: { name: id }\n" + " - identity.primary: { name: pk, fields: [id] }\n" + ) + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text(cfg.read_text().replace( + "generators: []", "generators: [entity, filter-allowlist, routes]")) + assert main(["gen", "--config", str(cfg)]) == 0 + assert "limit_too_large" in (tmp_path / "generated" / "app" / "author_router.py").read_text() + + +def test_init_leaves_an_existing_neutral_config_alone(tmp_path: Path, capsys) -> None: + neutral = tmp_path / ".metaobjects" / "config.json" + neutral.parent.mkdir() + neutral.write_text('{"schema_version": 1}\n') + assert main(["init", "--root", str(tmp_path)]) == 0 + assert neutral.read_text() == '{"schema_version": 1}\n' + assert "kept .metaobjects/config.json" in capsys.readouterr().out From 073d2cdbdd6303cb3ea07e67e7ef9cfa912af87f Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 19:01:20 -0500 Subject: [PATCH 12/31] feat(init): dotnet meta init and metaobjects init also write migrate.foreignKeyIndexes and verify.handEdits The same new-project keys TypeScript meta init writes since #577. --- .../PackageLayoutConfigTests.cs | 9 +++++++-- server/csharp/MetaObjects.Cli/InitCommand.cs | 13 ++++++++----- server/python/src/metaobjects/cli.py | 12 ++++++++---- server/python/tests/codegen/test_cli_init.py | 5 ++++- 4 files changed, 27 insertions(+), 12 deletions(-) diff --git a/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs b/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs index 720e575730..c6564b4473 100644 --- a/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/PackageLayoutConfigTests.cs @@ -65,11 +65,16 @@ public void Init_writes_the_opt_ins_for_a_new_project_and_leaves_an_existing_con var cfg = NeutralConfig.Read(fresh, validateNaming: false)!; Assert.Equal("package", cfg.OutputLayout); Assert.True(cfg.PackageNamespaces.FromMetadata); - // The same new-project api block TypeScript `meta init` writes: a 50/1000 page size and - // unknownFields spelled out at "reject" (a default left implicit would hide the choice). + // The same new-project settings TypeScript `meta init` writes: FK indexes on, hand edits + // failing verify, a 50/1000 page size and unknownFields spelled out at "reject" (a default + // left implicit would hide the choice). + Assert.True(cfg.MigrateForeignKeyIndexes); Assert.Equal(new ApiPagination(50, 1000), cfg.Api.Pagination); using (var doc = System.Text.Json.JsonDocument.Parse(File.ReadAllText(Path.Combine(fresh, ".metaobjects", "config.json")))) + { + Assert.Equal("fail", doc.RootElement.GetProperty("verify").GetProperty("handEdits").GetString()); Assert.Equal("reject", doc.RootElement.GetProperty("api").GetProperty("unknownFields").GetString()); + } File.WriteAllText(Path.Combine(fresh, ".metaobjects", "config.json"), "{ \"schema_version\": 1 }"); InitCommand.Run(fresh); diff --git a/server/csharp/MetaObjects.Cli/InitCommand.cs b/server/csharp/MetaObjects.Cli/InitCommand.cs index bec747ba1c..cd37d7e512 100644 --- a/server/csharp/MetaObjects.Cli/InitCommand.cs +++ b/server/csharp/MetaObjects.Cli/InitCommand.cs @@ -122,14 +122,17 @@ private static string DocsEntry(string root) => ? n.Replace("\\", "").Replace("\"", "") : "project") + "\" }"; /// The project config a new project starts with: files written under their metadata - /// package, each packaged type in its package's namespace, and the api block TypeScript - /// meta init writes: a page size on every generated list route, and unknownFields - /// written out at its default so the choice is visible. An existing config is the project's and is - /// never edited, so an upgraded project gains no page size. + /// package, each packaged type in its package's namespace, and the new-project settings TypeScript + /// meta init writes: migrate.foreignKeyIndexes on, verify.handEdits "fail", a + /// page size on every generated list route, and api.unknownFields written out at its default + /// so the choice is visible. An existing config is the project's and is never edited, so an + /// upgraded project gains none of them. public const string ConfigRel = ".metaobjects/config.json"; public static readonly string ConfigJson = "{\n \"schema_version\": 1,\n \"csharp\": {\n \"outputLayout\": \"package\",\n \"packageNamespaces\": true\n },\n" + + " \"migrate\": {\n \"foreignKeyIndexes\": true\n },\n" + + " \"verify\": {\n \"handEdits\": \"fail\"\n },\n" + " \"api\": {\n \"pagination\": {\n" + $" \"defaultLimit\": {ApiSettings.InitDefaultLimit},\n \"maxLimit\": {ApiSettings.InitMaxLimit}\n" + " },\n \"unknownFields\": \"reject\"\n }\n}\n"; @@ -155,7 +158,7 @@ public static IReadOnlyList Run(string root, bool withDocs = false) Directory.CreateDirectory(Path.GetDirectoryName(configPath)!); File.WriteAllText(configPath, ConfigJson); lines.Add( - $" wrote {ConfigRel} (outputLayout \"package\", packageNamespaces true, api.pagination " + + $" wrote {ConfigRel} (outputLayout \"package\", packageNamespaces true, migrate.foreignKeyIndexes true, verify.handEdits \"fail\", api.pagination " + $"{ApiSettings.InitDefaultLimit}/{ApiSettings.InitMaxLimit}, api.unknownFields \"reject\")"); } lines.Add(""); diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index 1badaa9131..567dc51b22 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -1361,12 +1361,16 @@ def _cmd_eject(args: argparse.Namespace) -> int: def _init_neutral_config() -> str: - """The port-neutral ``.metaobjects/config.json`` a new project starts with: the same ``api`` - block TypeScript ``meta init`` writes, a page size on every generated list route and - ``unknownFields`` spelled out at its default so the choice is visible. Never written over an - existing file, so an upgraded project gains no page size.""" + """The port-neutral ``.metaobjects/config.json`` a new project starts with: the same + new-project settings TypeScript ``meta init`` writes. ``migrate.foreignKeyIndexes`` on (read by + the TypeScript ``meta migrate`` that owns the schema), ``verify.handEdits`` "fail" (this port's + ``verify --codegen`` already fails on a hand edit), a page size on every generated list route, + and ``api.unknownFields`` spelled out at its default so the choice is visible. Never written + over an existing file, so an upgraded project gains none of them.""" body = { "schema_version": 1, + "migrate": {"foreignKeyIndexes": True}, + "verify": {"handEdits": "fail"}, "api": { API_KEY_PAGINATION: { PAGINATION_KEY_DEFAULT_LIMIT: INIT_DEFAULT_LIMIT, diff --git a/server/python/tests/codegen/test_cli_init.py b/server/python/tests/codegen/test_cli_init.py index dcab4ba498..964bda0555 100644 --- a/server/python/tests/codegen/test_cli_init.py +++ b/server/python/tests/codegen/test_cli_init.py @@ -75,7 +75,8 @@ def test_api_prefix_with_a_quote_or_backslash_is_refused(tmp_path: Path, capsys) def test_init_writes_the_new_project_api_block_that_typescript_meta_init_writes( tmp_path: Path, capsys ) -> None: - # Oracle: TypeScript `meta init` writes `api: {pagination: {defaultLimit: 50, maxLimit: 1000}, + # Oracle: TypeScript `meta init` writes `migrate: {foreignKeyIndexes: true}`, + # `verify: {handEdits: "fail"}` and `api: {pagination: {defaultLimit: 50, maxLimit: 1000}, # unknownFields: "reject"}` into .metaobjects/config.json for a new project. assert main(["init", "--root", str(tmp_path)]) == 0 assert "wrote .metaobjects/config.json" in capsys.readouterr().out @@ -84,6 +85,8 @@ def test_init_writes_the_new_project_api_block_that_typescript_meta_init_writes( assert neutral.api.pagination == ApiPagination(default_limit=50, max_limit=1000) raw = json.loads((tmp_path / ".metaobjects" / "config.json").read_text()) assert raw["api"]["unknownFields"] == "reject" + assert raw["migrate"] == {"foreignKeyIndexes": True} + assert raw["verify"] == {"handEdits": "fail"} # The page size reaches a generated router. (tmp_path / "metaobjects" / "meta.app.yaml").write_text( "metadata:\n package: app\n children:\n" From 795a273c66137808a8ea6b4150d1d0cfbe16b40d Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 19:36:05 -0500 Subject: [PATCH 13/31] feat(api): drop undeclared value-object members everywhere and share the api config cases - New jsonb scenario jsonb-nested-undeclared-key-dropped: api.unknownFields judges the top level only, so a member a value object does not declare is dropped on create and PATCH and never stored, while the open jsonb bag keeps every key. The reference servers in every port drop such members; the Java generated value-object record now carries @JsonIgnoreProperties(ignoreUnknown = true) (it answered 400 invalid_type). - New shared config corpus api-settings/config-blocks.json (valid and refused api blocks), run by the TypeScript, C#, Java and Python config readers. A page size is a positive whole number however JSON spells it, at most 2147483647: C# now reads 50.0, TypeScript and Python refuse anything above the int32 range. - CHANGELOG: an Upgrading section (api.unknownFields reject as the one deliberate default change, the C# readOnly private setter), init parity across ports, the nested-member rule. Counts, READMEs, api-contract.md and the migration guide updated; showcase Java regenerated. --- CHANGELOG.md | 29 +++++- docs/CONFORMANCE.md | 10 +- docs/features/api-contract.md | 8 +- .../unknown-fields-reject-by-default.md | 6 +- .../java/acme/SubscriberBlurbPayload.java | 1 + fixtures/api-contract-conformance/README.md | 4 +- .../api-settings/README.md | 13 +++ .../api-settings/config-blocks.json | 30 ++++++ .../api-contract-conformance/jsonb/README.md | 19 ++++ .../jsonb-nested-undeclared-key-dropped.yaml | 64 +++++++++++++ .../ApiConfigConformanceTests.cs | 68 ++++++++++++++ .../Api/JsonbReferenceServer.cs | 28 ++++++ .../MetaObjects/Config/NeutralConfig.cs | 8 +- .../spring/SpringValueObjectGenerator.java | 4 + .../expected/acme/shipping/ShipmentNote.java | 1 + .../kotlin/api/jsonb/DocumentApiServer.kt | 25 ++++- .../integration/api/JsonbReferenceServer.java | 35 +++++++ .../config/ApiConfigConformanceTest.java | 92 +++++++++++++++++++ .../src/metaobjects/config/neutral_config.py | 9 +- .../test_api_config_conformance.py | 55 +++++++++++ .../integration/api_contract_jsonb_server.py | 35 ++++++- .../src/api-contract-jsonb-server.ts | 29 +++++- server/typescript/packages/sdk/src/config.ts | 9 +- .../sdk/test/api-config-conformance.test.ts | 39 ++++++++ 24 files changed, 594 insertions(+), 27 deletions(-) create mode 100644 fixtures/api-contract-conformance/api-settings/config-blocks.json create mode 100644 fixtures/api-contract-conformance/jsonb/scenarios/jsonb-nested-undeclared-key-dropped.yaml create mode 100644 server/csharp/MetaObjects.Codegen.Tests/ApiConfigConformanceTests.cs create mode 100644 server/java/metadata/src/test/java/com/metaobjects/config/ApiConfigConformanceTest.java create mode 100644 server/python/tests/conformance/test_api_config_conformance.py create mode 100644 server/typescript/packages/sdk/test/api-config-conformance.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d72c2e12d..784a0c1559 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,19 @@ here.** **New metamodel vocabulary: `measure.derived` and its `@expr`.** It is additive: nothing that loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersion` 1.3 if it merges before the 1.3.0 publish; if it lands after, the next release moves `metamodelVersion`. +### Upgrading + +- **`api.unknownFields: reject` is the one deliberate default change.** A generated create, PUT + or PATCH route that dropped an undeclared top-level key now answers `400 unknown_fields`, in + existing projects too, once they regenerate (under "Changed" below). Set + `"api": { "unknownFields": "drop" }` to keep the old answer. Everything else new here is either + a fix to input no client should send or a setting a project has to turn on. +- **C#: a `readOnly` property has a private setter.** Regenerating gives every `readOnly` + property a private setter, whatever its shape (under "Security" below), so hand code that + assigned one (`row.Role = ...`) no longer compiles. The value belongs to the server: set it + through a database default or EF Core (`db.Entry(row).Property(r => r.Role).CurrentValue`), + which writes private setters. + ### Security - **Generated create routes wrote fields a client must never set, in Java, Kotlin, C# and @@ -80,7 +93,10 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi `pagination` option on the runtime mounts), the C# minimal-API routes, the Java and Kotlin Spring controllers and the Python FastAPI routers. An owned (ejected) routes copy picks it up when it is re-ejected. It is gated in the generated lane of every port by the new - `fixtures/api-contract-conformance/api-settings/` sub-corpus. A config key, not vocabulary: + `fixtures/api-contract-conformance/api-settings/` sub-corpus. Every port's config reader takes a + page size as a positive whole number however JSON spells it (`50.0` is `50`) and at most + 2147483647, and refuses a malformed block when the config is read; the shared cases, valid and + refused, are `api-settings/config-blocks.json`, run by every port. A config key, not vocabulary: `metamodelVersion` does not move. - **Report measures over Money and other value-object members (FR-044 R12).** A dimension's or @@ -167,6 +183,11 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi `input-unknown-fields-rejected` and `tph/tph-unknown-fields-rejected` (both lanes of every port) and `api-settings/unknown-fields-dropped` (`drop`, generated lane). Upgrade notes: [docs/features/migrations/unknown-fields-reject-by-default.md](docs/features/migrations/unknown-fields-reject-by-default.md). + Only the top level is judged: a member a value object does not declare is dropped, never + refused and never stored, on create and on PATCH, while the open JSON bag keeps every key + (`jsonb/jsonb-nested-undeclared-key-dropped`, both lanes of every port). The Java generated + value-object records now carry `@JsonIgnoreProperties(ignoreUnknown = true)` for this; they + answered `400 invalid_type` before. A config key, not vocabulary: `metamodelVersion` does not move. - **A validation 400 lists every issue in every port.** The same model answered a failed create or @@ -225,8 +246,10 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi refreshed with `meta eject --list`. - **`meta init` writes stricter defaults for a new project.** The scaffolded `.metaobjects/config.json` now carries `"verify": { "handEdits": "fail" }` and (except on - MySQL) `"migrate": { "foreignKeyIndexes": true }`, each as its own visible setting. An existing - config is left as it is. + MySQL) `"migrate": { "foreignKeyIndexes": true }`, each as its own visible setting, beside the + `api` block. `dotnet meta init` and `metaobjects init` now write the same `migrate`, `verify` + and `api` settings, so a new project starts the same in every port. An existing config is left + as it is. - **Agent context routes report work to the reporting vocabulary.** The `meta init` agent guide and the `metaobjects-authoring` skill now send grouped totals, dashboard numbers and read-only report views to `object.report` (`references/declaring-reports.md`) before the `@sql` escape, diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 051b6ea013..a1e6b2d9b8 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 64 (55 query + 9 migration) | all 64; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 55 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 137 (34 core + 14 tph + 9 m2m + 5 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 7 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 138 (34 core + 14 tph + 9 m2m + 6 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 7 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | @@ -442,9 +442,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (137) +### `fixtures/api-contract-conformance/` (138) -All 137 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 138 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -461,7 +461,7 @@ the corpus carries eleven sub-corpora — `tph/` (13, single-table inheritance; a subtype decimal filters and refuses extra digits, and the input rules hold over the whole hierarchy), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), -`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), +`jsonb/` (6, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string; an undeclared member inside a value object is dropped), `write-through/` (2, table-write + view-read entities), `secret-fields/` (2, #484: a `@formExclude` or credential `view.password` field is ignored on create, PATCH and PUT, and a credential is never returned), `enum-default/` (2, a create that omits a `field.enum` with `@default` stores the default, and a supplied value is kept), `projection/` (12, a read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named `id` is addressed by @@ -731,7 +731,7 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 510 + yaml 16 + verify 45 -+ render 18 + persistence 64 + api-contract 137 + source-resolution 25 + scope 10 + ++ render 18 + persistence 64 + api-contract 138 + source-resolution 25 + scope 10 + dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index 989277f095..fac81363a6 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -288,7 +288,9 @@ each handler they register instead; an `HTTPException` is rethrown to your envelope above (`pagination.invalid_value`) in every port, so it never reaches a store that reads `LIMIT -1` as no limit (SQLite). `offset` and `withCount=1` page past the default as usual, with `total` counting every row. `meta init` writes the block above for a new project; - both sizes must be positive integers and `defaultLimit` may not exceed `maxLimit`. A report + both sizes must be positive whole numbers (`50.0` reads as `50`) no larger than 2147483647, and + `defaultLimit` may not exceed `maxLimit`; a block that breaks a rule is refused when the config + is read, in every port (shared cases: `api-settings/config-blocks.json`). A report keeps its own row cap (above) and an opaque view its 1000, whatever the setting says. **Without the setting nothing changes, and the default is per port.** With `limit` omitted, @@ -660,7 +662,9 @@ field of the base or of any subtype has it. Every key in the table above is decl **dropped, not refused, under either value**: a client that sends back a row it read (with its `id`, its `@autoSet` stamps and its `writeOnce` field) keeps working, and a sibling subtype's column on another subtype's route is dropped. Only the top level is checked; a value object's members are -validated as before. `meta init` writes `"unknownFields": "reject"` for a new project. +validated as before, and a member a value object does not declare is dropped silently, never +refused and never stored, on create and on PATCH (`jsonb/jsonb-nested-undeclared-key-dropped`, +both lanes of every port). The open JSON bag keeps every key. `meta init` writes `"unknownFields": "reject"` for a new project. Each port checks the raw body before it binds it, so the answer is the same whatever the host framework would do with an unknown property: Java and Kotlin read the keys from the request's JSON diff --git a/docs/features/migrations/unknown-fields-reject-by-default.md b/docs/features/migrations/unknown-fields-reject-by-default.md index df213188e7..d00d881b0c 100644 --- a/docs/features/migrations/unknown-fields-reject-by-default.md +++ b/docs/features/migrations/unknown-fields-reject-by-default.md @@ -24,8 +24,10 @@ never refused: - a `writeOnce` field on PATCH or PUT; - the TPH discriminator, and a column only a sibling subtype declares. -So a client that reads a row and sends it back, edited, keeps working. Value-object members are -validated as before; only the top level is checked. +So a client that reads a row and sends it back, edited, keeps working. Only the top level is +checked: value-object members are validated as before, and a member a value object does not +declare is dropped silently, never refused and never stored. The open JSON bag +(`field.string` with `@dbColumnType: jsonb`) declares no members and keeps every key. ## Do you need to do anything? diff --git a/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java b/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java index b0e6be9ee0..93c38ac0a4 100644 --- a/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java +++ b/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — value object SubscriberBlurbPayload. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record SubscriberBlurbPayload( String name, @NotNull Status status diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index d426d007be..f4bd97cdff 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -57,14 +57,14 @@ fixtures/api-contract-conformance/ ``` Beside the 34 core scenarios above sit eleven **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (103 scenarios in all, 137 with the +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (104 scenarios in all, 138 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| | `tph/` | 14 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | -| `jsonb/` | 5 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | +| `jsonb/` | 6 | typed value-object columns, the numbers inside them, and an undeclared nested member dropped (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | | `secret-fields/` | 2 | #484: a `@formExclude` or credential (`view.password`) field is ignored on create, PATCH and PUT, and a credential is never returned; a `readOnly` field is ignored the same way and still returned | generated only | | `enum-default/` | 2 | a create that omits a `field.enum` carrying `@default` stores the default (echo and GET by id), and a supplied member is kept | generated only | diff --git a/fixtures/api-contract-conformance/api-settings/README.md b/fixtures/api-contract-conformance/api-settings/README.md index 8c53ba653d..818201fe95 100644 --- a/fixtures/api-contract-conformance/api-settings/README.md +++ b/fixtures/api-contract-conformance/api-settings/README.md @@ -42,6 +42,18 @@ and no cap. That half is the other sub-corpora, none of which sets it. The default, `reject`, is pinned by the parent corpus, which sets nothing: `scenarios/input-unknown-fields-rejected.yaml` and `tph/scenarios/tph-unknown-fields-rejected.yaml`. +## The shared config cases + +`config-blocks.json` is a second, smaller contract: what each port's CONFIG READER accepts in +the `api` block, before any route runs. Each `cases[]` entry is an `api` block (`null` means the +key is absent) and the `pagination` and `unknownFields` it must resolve to; each `invalid[]` +entry must be refused when `.metaobjects/config.json` is read. A page size is a positive whole +number however JSON spells it (`50` and `50.0` are the same number, since JavaScript cannot tell +them apart), at most 2147483647 so every port's integer type holds it. The message wording is +each port's own. Runners: TypeScript `sdk/test/api-config-conformance.test.ts`, C# +`ApiConfigConformanceTests`, Java `ApiConfigConformanceTest` (Kotlin reads config through the +Java reader), Python `tests/conformance/test_api_config_conformance.py`. + ## Lanes Generated lane only, as for every sub-corpus whose subject is what a generator emits. Every @@ -52,6 +64,7 @@ port runs it: TypeScript, C#, Java, Kotlin and Python. ``` api-settings/ ├── README.md +├── config-blocks.json # shared config-reader cases (not a scenario) ├── .metaobjects/config.json # api.pagination { 2, 4 }, api.unknownFields "drop" ├── meta.json # Item ├── seed.json # five rows diff --git a/fixtures/api-contract-conformance/api-settings/config-blocks.json b/fixtures/api-contract-conformance/api-settings/config-blocks.json new file mode 100644 index 0000000000..a248aa8fad --- /dev/null +++ b/fixtures/api-contract-conformance/api-settings/config-blocks.json @@ -0,0 +1,30 @@ +{ + "_comment": "Cross-port `api` setting of .metaobjects/config.json. Each case is the `api` block (null = the key is absent). A port parses it with the reader it uses for that file and must resolve `pagination` (null = no page size) and `unknownFields` to the values below. A page size is a positive whole number however JSON spells it (50 and 50.0 are the same number; JavaScript cannot tell them apart), at most 2147483647 so every port's integer type holds it. `invalid` blocks must be refused when the config is read; the message wording is each port's own.", + "cases": [ + { "name": "absent", "api": null, "pagination": null, "unknownFields": "reject" }, + { "name": "empty block", "api": {}, "pagination": null, "unknownFields": "reject" }, + { "name": "what init writes", "api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "reject" }, "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "reject" }, + { "name": "drop alone", "api": { "unknownFields": "drop" }, "pagination": null, "unknownFields": "drop" }, + { "name": "equal limits", "api": { "pagination": { "defaultLimit": 1, "maxLimit": 1 } }, "pagination": { "defaultLimit": 1, "maxLimit": 1 }, "unknownFields": "reject" }, + { "name": "a whole number spelled with a fraction part", "api": { "pagination": { "defaultLimit": 50.0, "maxLimit": 1000.0 } }, "pagination": { "defaultLimit": 50, "maxLimit": 1000 }, "unknownFields": "reject" }, + { "name": "the largest page size", "api": { "pagination": { "defaultLimit": 2147483647, "maxLimit": 2147483647 } }, "pagination": { "defaultLimit": 2147483647, "maxLimit": 2147483647 }, "unknownFields": "reject" } + ], + "invalid": [ + { "name": "api not an object", "api": "reject" }, + { "name": "unknown api key", "api": { "pageSize": 50 } }, + { "name": "unknownFields not a mode", "api": { "unknownFields": "ignore" } }, + { "name": "unknownFields not a string", "api": { "unknownFields": true } }, + { "name": "pagination not an object", "api": { "pagination": 50 } }, + { "name": "pagination missing maxLimit", "api": { "pagination": { "defaultLimit": 50 } } }, + { "name": "pagination missing defaultLimit", "api": { "pagination": { "maxLimit": 1000 } } }, + { "name": "unknown pagination key", "api": { "pagination": { "defaultLimit": 50, "maxLimit": 1000, "minLimit": 1 } } }, + { "name": "zero", "api": { "pagination": { "defaultLimit": 0, "maxLimit": 1000 } } }, + { "name": "negative", "api": { "pagination": { "defaultLimit": 50, "maxLimit": -1 } } }, + { "name": "a fraction", "api": { "pagination": { "defaultLimit": 50.5, "maxLimit": 1000 } } }, + { "name": "a numeric string", "api": { "pagination": { "defaultLimit": "50", "maxLimit": 1000 } } }, + { "name": "a boolean", "api": { "pagination": { "defaultLimit": true, "maxLimit": 1000 } } }, + { "name": "null", "api": { "pagination": { "defaultLimit": null, "maxLimit": 1000 } } }, + { "name": "above 2147483647", "api": { "pagination": { "defaultLimit": 50, "maxLimit": 2147483648 } } }, + { "name": "defaultLimit above maxLimit", "api": { "pagination": { "defaultLimit": 1001, "maxLimit": 1000 } } } + ] +} diff --git a/fixtures/api-contract-conformance/jsonb/README.md b/fixtures/api-contract-conformance/jsonb/README.md index eccc73bf99..3cf5268fff 100644 --- a/fixtures/api-contract-conformance/jsonb/README.md +++ b/fixtures/api-contract-conformance/jsonb/README.md @@ -57,6 +57,7 @@ jsonb/ ├── meta.json # the Document entity + Marker and Telemetry value objects ├── seed.json # 2 seed Documents (ids 1, 2), jsonb object + VO columns └── scenarios/ + ├── jsonb-nested-undeclared-key-dropped.yaml # an undeclared member inside a value object is dropped, not refused ├── jsonb-open-bag-roundtrip.yaml # open-bag parsed-value contract ├── jsonb-value-object-patch.yaml # Program D — cross-port VO-column PATCH/POST ├── jsonb-value-object-map-decimals.yaml # decimals in a field.map of value objects, per map value @@ -179,3 +180,21 @@ the condition is emitted on nullable members only). |---|---| | `r1`/`r2` POST + GET | create is `201`, the double stays a JSON number, the timestamp keeps its UTC `Z` spelling, `tempC` round-trips | | `r3`/`r4` PATCH + GET | PATCH replaces the value object (`tempC` omitted when absent) and it persists | + +## Scenario — `jsonb-nested-undeclared-key-dropped.yaml` + +`api.unknownFields` judges the TOP level of a write body only (`reject`, the default, answers a +top-level undeclared key with `400 unknown_fields`; see `../scenarios/input-unknown-fields-rejected.yaml`). +A member a value object does not declare is a different case: it is dropped silently, on create +and on PATCH, and never stored, so a client sending a newer shape of a value object is not +refused. The open bag (`payload`) declares no members, so every key in it is the client's data +and is kept. + +| Requests | What it pins | +|---|---| +| `r1`/`r2` POST + GET | `extra` inside `primaryMarker` and inside a `markers` element is dropped; `extra` inside `payload` is kept | +| `r3`/`r4` PATCH + GET | the same drop on PATCH of `optionalMarker`, and the earlier value objects are unchanged | + +The Java generated lane answered `400 invalid_type` here before 1.3.0: the generated value-object +record now carries `@JsonIgnoreProperties(ignoreUnknown = true)`, so the member is dropped +whatever `FAIL_ON_UNKNOWN_PROPERTIES` the host's `ObjectMapper` sets. diff --git a/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-nested-undeclared-key-dropped.yaml b/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-nested-undeclared-key-dropped.yaml new file mode 100644 index 0000000000..d89ceda3d0 --- /dev/null +++ b/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-nested-undeclared-key-dropped.yaml @@ -0,0 +1,64 @@ +name: jsonb-nested-undeclared-key-dropped +description: > + `api.unknownFields` judges the TOP level of a write body only. A key the + entity does not declare at the top level is `400 unknown_fields` under + `reject` (the default; `input-unknown-fields-rejected`), but a key nested + inside a value object that the value object does not declare is NOT refused: + it is dropped silently, on create and on PATCH, and never stored. The + contrast is the open JSON bag (`payload`, `field.string @dbColumnType=jsonb`): + it declares no members, so every nested key in it is the client's data and is + kept. `row` compares each named column exactly, so a stored `extra` member + fails r1-r4. Seed rows are ids 1 and 2; the POST lands at id 3. +requests: + # --- POST: an undeclared member inside a single VO and inside an array-of-VO element --- + - id: r1 + method: POST + path: /api/documents + body: + title: "nested" + payload: { kept: true, extra: "mine" } + primaryMarker: { label: "owner-n", score: 7, extra: "dropped" } + markers: [ { label: "a", score: 1, extra: "dropped" } ] + expect: + status: 201 + body: + hasId: true + row: + title: "nested" + payload: { kept: true, extra: "mine" } + primaryMarker: { label: "owner-n", score: 7 } + markers: [ { label: "a", score: 1 } ] + - id: r2 + method: GET + path: /api/documents/3 + expect: + status: 200 + body: + row: + id: 3 + payload: { kept: true, extra: "mine" } + primaryMarker: { label: "owner-n", score: 7 } + markers: [ { label: "a", score: 1 } ] + + # --- PATCH: the same rule on update --- + - id: r3 + method: PATCH + path: /api/documents/3 + body: + optionalMarker: { label: "opt-n", score: 8, extra: "dropped" } + expect: + status: 200 + body: + row: + id: 3 + optionalMarker: { label: "opt-n", score: 8 } + - id: r4 + method: GET + path: /api/documents/3 + expect: + status: 200 + body: + row: + id: 3 + primaryMarker: { label: "owner-n", score: 7 } + optionalMarker: { label: "opt-n", score: 8 } diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiConfigConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiConfigConformanceTests.cs new file mode 100644 index 0000000000..383c5005c2 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiConfigConformanceTests.cs @@ -0,0 +1,68 @@ +// ApiConfigConformanceTests — runs the shared cross-port `api` config corpus +// (fixtures/api-contract-conformance/api-settings/config-blocks.json) against the C# config reader +// (MetaObjects.Config.NeutralConfig, the reader every project-aware load goes through). Every +// `cases[]` block resolves to the corpus's values; every `invalid` block is refused when read. +using System.Text.Json; +using MetaObjects.Config; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ApiConfigConformanceTests +{ + private static string CasesPath => Path.Combine( + CorpusPaths.RepoRoot(), "fixtures", "api-contract-conformance", "api-settings", "config-blocks.json"); + + private static JsonElement Section(string name) => + JsonDocument.Parse(File.ReadAllText(CasesPath)).RootElement.GetProperty(name); + + private static TheoryData Names(string section) + { + var data = new TheoryData(); + foreach (var el in Section(section).EnumerateArray()) data.Add(el.GetProperty("name").GetString()!); + return data; + } + + public static TheoryData CaseNames() => Names("cases"); + public static TheoryData InvalidNames() => Names("invalid"); + + private static JsonElement Named(string section, string name) => + Section(section).EnumerateArray().Single(e => e.GetProperty("name").GetString() == name); + + private static ApiSettings ReadApi(JsonElement api) + { + var dir = Directory.CreateTempSubdirectory("mo-api-conf-").FullName; + try + { + Directory.CreateDirectory(Path.Combine(dir, ".metaobjects")); + var body = api.ValueKind == JsonValueKind.Null + ? """{ "schema_version": 1 }""" + : $$"""{ "schema_version": 1, "api": {{api.GetRawText()}} }"""; + File.WriteAllText(Path.Combine(dir, ".metaobjects", "config.json"), body); + return NeutralConfig.Read(dir, validateNaming: false)!.Api; + } + finally { Directory.Delete(dir, recursive: true); } + } + + [Theory] + [MemberData(nameof(CaseNames))] + public void A_valid_block_resolves_to_the_corpus_values(string name) + { + var c = Named("cases", name); + var api = ReadApi(c.GetProperty("api")); + var want = c.GetProperty("pagination"); + if (want.ValueKind == JsonValueKind.Null) Assert.Null(api.Pagination); + else + Assert.Equal( + new ApiPagination(want.GetProperty("defaultLimit").GetInt32(), want.GetProperty("maxLimit").GetInt32()), + api.Pagination); + var mode = c.GetProperty("unknownFields").GetString() == "drop" ? UnknownFieldsMode.Drop : UnknownFieldsMode.Reject; + Assert.Equal(mode, api.UnknownFields); + } + + [Theory] + [MemberData(nameof(InvalidNames))] + public void An_invalid_block_is_refused_when_read(string name) => + Assert.Throws(() => ReadApi(Named("invalid", name).GetProperty("api"))); +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs index 3866214834..5f695c663b 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs @@ -139,6 +139,7 @@ private async Task CreateDocumentAsync(HttpContext ctx) if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx, "optionalMarker"); return; } if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx, "markers"); return; } if (!CanonicalizeMeasures(body)) { await BadValidation(ctx, "measures"); return; } + DropUndeclaredMembers(body); long newId; await using var c = new NpgsqlConnection(_pg.ConnectionString); @@ -183,6 +184,7 @@ private async Task UpdateDocumentAsync(HttpContext ctx, string idStr) if (PresentNonNull(body, "optionalMarker") && !ValidMarker(body["optionalMarker"])) { await BadValidation(ctx, "optionalMarker"); return; } if (PresentNonNull(body, "markers") && !ValidMarkerArray(body["markers"])) { await BadValidation(ctx, "markers"); return; } if (!CanonicalizeMeasures(body)) { await BadValidation(ctx, "measures"); return; } + DropUndeclaredMembers(body); await using var c = new NpgsqlConnection(_pg.ConnectionString); await c.OpenAsync(); @@ -214,6 +216,32 @@ private async Task UpdateDocumentAsync(HttpContext ctx, string idStr) await Respond(ctx, await rdr.ReadAsync() ? RowToMap(rdr) : null); } + // A member a value object does not declare is dropped before the write (scenario + // jsonb-nested-undeclared-key-dropped): `api.unknownFields` judges the top level only, so a + // nested undeclared key is never refused and never stored. `payload` is the open bag and keeps + // every key. Mirrors the TS reference. + private static readonly string[] MarkerMembers = ["label", "score"]; + private static readonly string[] MeasureMembers = ["ratio", "samples", "amount"]; + private static readonly Dictionary VoMembers = new() + { + ["primaryMarker"] = MarkerMembers, ["optionalMarker"] = MarkerMembers, ["markers"] = MarkerMembers, + ["measure"] = MeasureMembers, ["measures"] = MeasureMembers, + ["telemetry"] = ["capturedAt", "latitude", "tempC"], + }; + + private static void DropUndeclaredMembers(JsonObject body) + { + foreach (var (column, members) in VoMembers) + if (body.TryGetPropertyValue(column, out var value)) KeepDeclared(value, members); + } + + private static void KeepDeclared(JsonNode? node, string[] members) + { + if (node is JsonArray arr) { foreach (var el in arr) KeepDeclared(el, members); return; } + if (node is not JsonObject obj) return; + foreach (var key in obj.Select(p => p.Key).Where(k => !members.Contains(k)).ToList()) obj.Remove(key); + } + // ----- validation ----- // A present VO must be a JSON object whose `label` is present, non-null, non-empty (FR-036: diff --git a/server/csharp/MetaObjects/Config/NeutralConfig.cs b/server/csharp/MetaObjects/Config/NeutralConfig.cs index 842025af0f..0eb5a3d473 100644 --- a/server/csharp/MetaObjects/Config/NeutralConfig.cs +++ b/server/csharp/MetaObjects/Config/NeutralConfig.cs @@ -700,9 +700,13 @@ private static ApiPagination ReadPagination(JsonElement block, string path) return new ApiPagination(defaultLimit.Value, maxLimit.Value); } + // A positive whole number however JSON spells it (50 and 50.0 are one number, as every other + // port reads them), at most int.MaxValue. The shared corpus + // fixtures/api-contract-conformance/api-settings/config-blocks.json holds every port to it. private static int PageSizeNumber(JsonElement v, string key, string path) => - v.ValueKind == JsonValueKind.Number && v.TryGetInt32(out var n) && n >= 1 - ? n + v.ValueKind == JsonValueKind.Number && v.TryGetDecimal(out var d) + && d == decimal.Truncate(d) && d >= 1 && d <= int.MaxValue + ? (int)d : throw new MetaModelException($"{path}: \"api.pagination.{key}\" must be a positive integer", ErrorCode.ERR_BAD_ATTR_VALUE); } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java index 982b294177..db1a9ff4b3 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java @@ -182,6 +182,10 @@ private void emit(MetaObject vo, Path outRoot) { src.append("/** GENERATED — value object ").append(recordName) .append(". Do not hand-edit; regenerated from metadata. */\n"); if (omitNulls) src.append("@JsonInclude(JsonInclude.Include.NON_NULL)\n"); + // A member the value object does not declare is DROPPED, never rejected, whatever + // FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries: api.unknownFields judges a + // request body's top level only, so a nested undeclared key binds like the DTO's own. + src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); src.append("public record ").append(recordName).append("(\n"); List components = new ArrayList<>(fields.size()); for (int i = 0; i < fields.size(); i++) { diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java index 6b095b6f3a..2b42e8dc74 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java @@ -3,6 +3,7 @@ import jakarta.validation.constraints.*; /** GENERATED — value object ShipmentNote. Do not hand-edit; regenerated from metadata. */ +@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ShipmentNote( @NotNull @Size(min = 1) String text, ShipmentNoteKind kind, diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt index 75e821c4be..4b618b1800 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt @@ -140,7 +140,7 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { private fun createDocument(exchange: HttpExchange) { @Suppress("UNCHECKED_CAST") - val body = readJsonBody(exchange) as? Map + val body = (readJsonBody(exchange) as? Map)?.let(::dropUndeclaredMembers) ?: return sendJson(exchange, 400, invalidBody()) // primaryMarker is @required: absent / null / not-a-valid-Marker → 400. val primary = body["primaryMarker"] @@ -183,7 +183,7 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { val id = idStr.toLongOrNull() ?: return sendJson(exchange, 400, mapOf("error" to "invalid_id")) @Suppress("UNCHECKED_CAST") - val body = readJsonBody(exchange) as? Map + val body = (readJsonBody(exchange) as? Map)?.let(::dropUndeclaredMembers) ?: return sendJson(exchange, 400, invalidBody()) // title is @required: an explicit null clears nothing — it is a 400. if (body.containsKey("title") && body["title"] == null) @@ -230,6 +230,27 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { else sendJson(exchange, 200, row) } + // A member a value object does not declare is dropped before the write (scenario + // jsonb-nested-undeclared-key-dropped): api.unknownFields judges the top level only, so a + // nested undeclared key is never refused and never stored. `payload` is the open bag and keeps + // every key. Mirrors the TS reference. + private val markerMembers = setOf("label", "score") + private val measureMembers = setOf("ratio", "samples", "amount") + private val voMembers: Map> = mapOf( + "primaryMarker" to markerMembers, "optionalMarker" to markerMembers, "markers" to markerMembers, + "measure" to measureMembers, "measures" to measureMembers, + "telemetry" to setOf("capturedAt", "latitude", "tempC"), + ) + + private fun dropUndeclaredMembers(body: Map): Map = + body.mapValuesTo(LinkedHashMap()) { (k, v) -> voMembers[k]?.let { keepDeclared(v, it) } ?: v } + + private fun keepDeclared(value: Any?, members: Set): Any? = when (value) { + is List<*> -> value.map { keepDeclared(it, members) } + is Map<*, *> -> value.filterKeys { it in members } + else -> value + } + /** * True iff [value] is a valid `Marker` value object: `label` is a NON-EMPTY string ≤ 40 chars * (FR-036 required-string: rejects null / ""; whitespace accepted), `score` is unconstrained. diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java index bd3bdc0ad3..0da01d52f3 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java @@ -149,6 +149,7 @@ private void doHandle(HttpExchange exchange) throws IOException, SQLException { // Program D: nested value-object validation on CREATE (required primaryMarker present + // valid; a present optional/array VO valid) → 400 before persisting. if (!voValidForCreate(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, invalidBody()); return; } + dropUndeclaredMembers(body); sendJson(exchange, 201, create(body)); return; } @@ -158,6 +159,7 @@ private void doHandle(HttpExchange exchange) throws IOException, SQLException { // FR-035 tristate + Program D nested VO validation: present-null on the @required // primaryMarker → 400; a present value must be a valid Marker (single or array) → 400. if (!voValidForPatch(body) || !canonicalizeMeasures(body)) { sendJson(exchange, 400, invalidBody()); return; } + dropUndeclaredMembers(body); Map row = patch(id, body); if (row == null) { sendJson(exchange, 404, Map.of("error", "not_found")); return; } sendJson(exchange, 200, row); @@ -262,6 +264,39 @@ private Map rowFrom(ResultSet rs) throws SQLException { return row; } + // A member a value object does not declare is dropped before the write (scenario + // jsonb-nested-undeclared-key-dropped): api.unknownFields judges the top level only, so a + // nested undeclared key is never refused and never stored. `payload` is the open bag and keeps + // every key. Mirrors the TS reference. + private static final List MARKER_MEMBERS = List.of("label", "score"); + private static final List MEASURE_MEMBERS = List.of("ratio", "samples", "amount"); + private static final Map> VO_MEMBERS = Map.of( + "primaryMarker", MARKER_MEMBERS, "optionalMarker", MARKER_MEMBERS, "markers", MARKER_MEMBERS, + "measure", MEASURE_MEMBERS, "measures", MEASURE_MEMBERS, + "telemetry", List.of("capturedAt", "latitude", "tempC")); + + private static void dropUndeclaredMembers(Map body) { + for (Map.Entry> e : VO_MEMBERS.entrySet()) { + if (body.containsKey(e.getKey())) body.put(e.getKey(), keepDeclared(body.get(e.getKey()), e.getValue())); + } + } + + private static Object keepDeclared(Object value, List members) { + if (value instanceof List list) { + List out = new ArrayList<>(list.size()); + for (Object el : list) out.add(keepDeclared(el, members)); + return out; + } + if (value instanceof Map map) { + Map out = new LinkedHashMap<>(); + for (Map.Entry m : map.entrySet()) { + if (members.contains(String.valueOf(m.getKey()))) out.put(String.valueOf(m.getKey()), m.getValue()); + } + return out; + } + return value; + } + /** The validation envelope: this reference server checks the whole body and lists it as one issue (never asserted per field). */ private static Map invalidBody() { Map issue = new LinkedHashMap<>(); diff --git a/server/java/metadata/src/test/java/com/metaobjects/config/ApiConfigConformanceTest.java b/server/java/metadata/src/test/java/com/metaobjects/config/ApiConfigConformanceTest.java new file mode 100644 index 0000000000..2b80493d3b --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/config/ApiConfigConformanceTest.java @@ -0,0 +1,92 @@ +/* + * Copyright 2003 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.config; + +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import com.metaobjects.MetaDataException; +import org.junit.Test; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThrows; + +/** + * Runs the shared cross-port {@code api} config corpus + * ({@code fixtures/api-contract-conformance/api-settings/config-blocks.json}) through + * {@link NeutralConfig#parse}, the reader a project's {@code .metaobjects/config.json} goes through. + * Every {@code cases[]} block resolves to the corpus's values; every {@code invalid} block is refused. + */ +public class ApiConfigConformanceTest { + + private static JsonObject readCorpus() throws IOException { + Path dir = Paths.get("").toAbsolutePath(); + while (dir != null && !Files.isDirectory(dir.resolve("fixtures"))) dir = dir.getParent(); + if (dir == null) throw new IllegalStateException("could not locate the repository fixtures/ directory"); + String content = new String(Files.readAllBytes( + dir.resolve("fixtures/api-contract-conformance/api-settings/config-blocks.json")), StandardCharsets.UTF_8); + return JsonParser.parseString(content).getAsJsonObject(); + } + + /** A JSON {@code null} case value means the {@code api} key is ABSENT. */ + private static ApiSettings parse(JsonElement api, String origin) { + JsonObject doc = new JsonObject(); + doc.addProperty("schema_version", 1); + if (api != null && !api.isJsonNull()) doc.add("api", api); + return NeutralConfig.parse(doc.toString(), origin).getApiSettings(); + } + + @Test + public void everyValidBlockResolvesToItsValues() throws IOException { + int n = 0; + for (JsonElement el : readCorpus().getAsJsonArray("cases")) { + JsonObject c = el.getAsJsonObject(); + String name = c.get("name").getAsString(); + ApiSettings api = parse(c.get("api"), "case:" + name); + JsonElement want = c.get("pagination"); + if (want.isJsonNull()) { + assertNull(name + ": pagination", api.pagination()); + } else { + JsonObject p = want.getAsJsonObject(); + assertEquals(name + ": pagination", + new ApiSettings.Pagination(p.get("defaultLimit").getAsLong(), p.get("maxLimit").getAsLong()), + api.pagination()); + } + assertEquals(name + ": unknownFields", c.get("unknownFields").getAsString(), api.unknownFields().wireName()); + n++; + } + assertEquals(7, n); + } + + @Test + public void everyInvalidBlockIsRefused() throws IOException { + int n = 0; + for (JsonElement el : readCorpus().getAsJsonArray("invalid")) { + JsonObject c = el.getAsJsonObject(); + String name = c.get("name").getAsString(); + assertThrows(name, MetaDataException.class, () -> parse(c.get("api"), "invalid-case:" + name)); + n++; + } + assertEquals(16, n); + } +} diff --git a/server/python/src/metaobjects/config/neutral_config.py b/server/python/src/metaobjects/config/neutral_config.py index 9917222174..43bdd51213 100644 --- a/server/python/src/metaobjects/config/neutral_config.py +++ b/server/python/src/metaobjects/config/neutral_config.py @@ -405,11 +405,16 @@ def _check_enums(raw: dict[str, object], path: Path) -> None: ) +_PAGE_SIZE_MAX = 2147483647 + + def _page_size(value: object, key: str, path: Path) -> int: - """A positive whole number however JSON spells it (50, 50.0), never a bool.""" + """A positive whole number however JSON spells it (50, 50.0), never a bool, at most + 2147483647 so every port's integer type holds it (the shared corpus + ``fixtures/api-contract-conformance/api-settings/config-blocks.json``).""" if isinstance(value, float) and value.is_integer(): value = int(value) - if isinstance(value, bool) or not isinstance(value, int) or value <= 0: + if isinstance(value, bool) or not isinstance(value, int) or not 0 < value <= _PAGE_SIZE_MAX: raise ParseError( f'{path}: "api.pagination.{key}" must be a positive integer', code=ErrorCode.ERR_COLLECTION_NOT_FOUND, diff --git a/server/python/tests/conformance/test_api_config_conformance.py b/server/python/tests/conformance/test_api_config_conformance.py new file mode 100644 index 0000000000..51c6b1d106 --- /dev/null +++ b/server/python/tests/conformance/test_api_config_conformance.py @@ -0,0 +1,55 @@ +"""Cross-port conformance: ``fixtures/api-contract-conformance/api-settings/config-blocks.json``. + +Every port parses each case's ``api`` block with its OWN config reader +(``metaobjects.config.neutral_config``) and must resolve ``pagination`` and ``unknownFields`` to the +recorded values; every ``invalid`` block must be refused when the config is read. +""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from metaobjects.api_settings import ApiPagination +from metaobjects.config.neutral_config import read_neutral_config +from metaobjects.errors import ParseError + +REPO = Path(__file__).parents[4] +_SPEC = json.loads( + (REPO / "fixtures" / "api-contract-conformance" / "api-settings" / "config-blocks.json").read_text() +) +_CASES: list[dict[str, object]] = _SPEC["cases"] +_INVALID: list[dict[str, object]] = _SPEC["invalid"] + + +def _project_with(tmp_path: Path, api_block: object) -> Path: + """A project dir whose ``.metaobjects/config.json`` declares *api_block* (``None`` omits the key).""" + raw: dict[str, object] = {"schema_version": 1} + if api_block is not None: + raw["api"] = api_block + dot = tmp_path / ".metaobjects" + dot.mkdir(parents=True, exist_ok=True) + (dot / "config.json").write_text(json.dumps(raw), encoding="utf-8") + return tmp_path + + +@pytest.mark.parametrize("case", _CASES, ids=[str(c["name"]) for c in _CASES]) +def test_a_valid_block_resolves_to_the_corpus_values(tmp_path: Path, case: dict[str, object]) -> None: + cfg = read_neutral_config(_project_with(tmp_path, case["api"])) + assert cfg is not None + want = case["pagination"] + if want is None: + assert cfg.api.pagination is None + else: + assert isinstance(want, dict) + assert cfg.api.pagination == ApiPagination( + default_limit=want["defaultLimit"], max_limit=want["maxLimit"] + ) + assert cfg.api.unknown_fields == case["unknownFields"] + + +@pytest.mark.parametrize("case", _INVALID, ids=[str(c["name"]) for c in _INVALID]) +def test_an_invalid_block_is_refused_when_read(tmp_path: Path, case: dict[str, object]) -> None: + with pytest.raises(ParseError): + read_neutral_config(_project_with(tmp_path, case["api"])) diff --git a/server/python/tests/integration/api_contract_jsonb_server.py b/server/python/tests/integration/api_contract_jsonb_server.py index 6e0444767b..98ec922462 100644 --- a/server/python/tests/integration/api_contract_jsonb_server.py +++ b/server/python/tests/integration/api_contract_jsonb_server.py @@ -247,7 +247,7 @@ def create_document(dto: dict[str, Any]) -> Any: # Present values validated with the create rules (title @maxLength + nested VO). if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto) or not _canonicalize_tiers(dto): return _validation_400() - return repo.create(dto) + return repo.create(_drop_undeclared_members(dto)) @app.patch("/api/documents/{document_id}") @app.put("/api/documents/{document_id}") @@ -261,7 +261,7 @@ def update_document(document_id: int, dto: dict[str, Any]) -> Any: # Validate PRESENT, non-null values (nested VO recurses into each element). if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto) or not _canonicalize_tiers(dto): return _validation_400() - saved = repo.update(document_id, dto) + saved = repo.update(document_id, _drop_undeclared_members(dto)) if saved is None: return JSONResponse(status_code=404, content={"error": "not_found"}) return saved @@ -275,6 +275,37 @@ def update_document(document_id: int, dto: dict[str, Any]) -> Any: # --------------------------------------------------------------------------- +# A member a value object does not declare is dropped before the write (scenario +# jsonb-nested-undeclared-key-dropped): `api.unknownFields` judges the top level only, so a +# nested undeclared key is never refused and never stored. `payload` is the open bag and keeps +# every key. Mirrors the TS reference. +_MARKER_MEMBERS = ("label", "score") +_MEASURE_MEMBERS = ("ratio", "samples", "amount") +_VO_MEMBERS: dict[str, tuple[str, ...]] = { + "primaryMarker": _MARKER_MEMBERS, + "optionalMarker": _MARKER_MEMBERS, + "markers": _MARKER_MEMBERS, + "measure": _MEASURE_MEMBERS, + "measures": _MEASURE_MEMBERS, + "telemetry": ("capturedAt", "latitude", "tempC"), +} + + +def _keep_declared(value: Any, members: tuple[str, ...]) -> Any: + if isinstance(value, list): + return [_keep_declared(v, members) for v in value] + if isinstance(value, dict): + return {k: v for k, v in value.items() if k in members} + return value + + +def _drop_undeclared_members(dto: dict[str, Any]) -> dict[str, Any]: + return { + k: _keep_declared(v, _VO_MEMBERS[k]) if k in _VO_MEMBERS else v + for k, v in dto.items() + } + + def _validation_400() -> JSONResponse: return JSONResponse(status_code=400, content={"error": "validation"}) diff --git a/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts b/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts index f4f3209ba6..193459d291 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts @@ -104,6 +104,31 @@ export async function startJsonbServer(connectionUri: string, root: MetaRoot): P throw err; }); + // A member a value object does not declare is dropped from the body before the write + // (scenario jsonb-nested-undeclared-key-dropped): `api.unknownFields` judges the top level + // only, so a nested undeclared key is never refused and never stored. `payload` is the open + // bag and keeps every key. Hand-rolled like the Author reference's DECLARED_KEYS. + const MARKER = ["label", "score"]; + const MEASURE = ["ratio", "samples", "amount"]; + const VO_MEMBERS: Record = { + primaryMarker: MARKER, optionalMarker: MARKER, markers: MARKER, + measure: MEASURE, measures: MEASURE, + telemetry: ["capturedAt", "latitude", "tempC"], + }; + const keepDeclared = (value: unknown, members: readonly string[]): unknown => + Array.isArray(value) + ? value.map((v) => keepDeclared(v, members)) + : value !== null && typeof value === "object" + ? Object.fromEntries(Object.entries(value).filter(([k]) => members.includes(k))) + : value; + const dropUndeclaredMembers = (body: unknown): Record => + Object.fromEntries( + Object.entries((body ?? {}) as Record).map(([k, v]) => { + const members = VO_MEMBERS[k]; + return [k, members === undefined ? v : keepDeclared(v, members)]; + }), + ); + // GET /api/documents/:id fastify.get(`${ROUTE_BASE}/:id`, async (req, reply) => { const { id } = req.params as { id: string }; @@ -115,7 +140,7 @@ export async function startJsonbServer(connectionUri: string, root: MetaRoot): P // POST /api/documents fastify.post(ROUTE_BASE, async (req, reply) => { - const row = await om.create(ENTITY, req.body as Record); + const row = await om.create(ENTITY, dropUndeclaredMembers(req.body)); reply.code(201).send(row); return reply; }); @@ -125,7 +150,7 @@ export async function startJsonbServer(connectionUri: string, root: MetaRoot): P // propagates to the shared setErrorHandler → 400. const updateHandler: RouteHandlerMethod = async (req, reply) => { const { id } = req.params as { id: string }; - const row = await om.update(ENTITY, Number(id), req.body as Record, { + const row = await om.update(ENTITY, Number(id), dropUndeclaredMembers(req.body), { ifMissing: "ignore", }); if (row) return row; diff --git a/server/typescript/packages/sdk/src/config.ts b/server/typescript/packages/sdk/src/config.ts index 7bd83787fb..72cedf7170 100644 --- a/server/typescript/packages/sdk/src/config.ts +++ b/server/typescript/packages/sdk/src/config.ts @@ -181,11 +181,14 @@ const ScopeSchema = z // "scope") is silently stripped by zod and the collection resolves as // "everything in scope" — the exact silent fail-open .strict() on the // nested arms exists to prevent, one level up. -/** One `api.pagination` size: a positive integer, every failure (missing, wrong type, a - * fraction, zero) named by its key rather than Zod's generic "expected number". */ +/** One `api.pagination` size: a positive integer no larger than 2147483647 (so every port's + * integer type holds it), every failure (missing, wrong type, a fraction, zero, too large) named + * by its key rather than Zod's generic "expected number". The shared corpus + * fixtures/api-contract-conformance/api-settings/config-blocks.json holds every port to it. */ +const PAGE_SIZE_MAX = 2147483647; function pageSizeNumber(key: string) { const error = `"api.pagination.${key}" must be a positive integer`; - return z.number({ error }).int({ error }).positive({ error }); + return z.number({ error }).int({ error }).positive({ error }).max(PAGE_SIZE_MAX, { error }); } export const ConfigSchema = z.object({ diff --git a/server/typescript/packages/sdk/test/api-config-conformance.test.ts b/server/typescript/packages/sdk/test/api-config-conformance.test.ts new file mode 100644 index 0000000000..2d67ea93c7 --- /dev/null +++ b/server/typescript/packages/sdk/test/api-config-conformance.test.ts @@ -0,0 +1,39 @@ +// The cross-port `api` setting, run through TypeScript's own config reader: every valid block in +// fixtures/api-contract-conformance/api-settings/config-blocks.json resolves to the corpus's +// values, and every invalid one is refused when the config is read. + +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { ConfigSchema, apiPaginationFromConfig, apiUnknownFieldsFromConfig } from "../src/config.js"; + +const CORPUS = resolve(import.meta.dir, "../../../../../fixtures/api-contract-conformance/api-settings/config-blocks.json"); + +interface ApiCase { + name: string; + api: unknown; + pagination: { defaultLimit: number; maxLimit: number } | null; + unknownFields: "reject" | "drop"; +} +const corpus = JSON.parse(readFileSync(CORPUS, "utf8")) as { cases: ApiCase[]; invalid: { name: string; api: unknown }[] }; + +function configFor(api: unknown): Record { + return api === null ? { schema_version: 1 } : { schema_version: 1, api }; +} + +describe("api config conformance", () => { + for (const c of corpus.cases) { + test(`${c.name}: resolves to the corpus values`, () => { + const cfg = ConfigSchema.parse(configFor(c.api)); + expect(apiPaginationFromConfig(cfg) ?? null).toEqual(c.pagination); + // An absent unknownFields is the default, reject. + expect(apiUnknownFieldsFromConfig(cfg) ?? "reject").toBe(c.unknownFields); + }); + } + + for (const c of corpus.invalid) { + test(`${c.name}: the config is refused when read`, () => { + expect(ConfigSchema.safeParse(configFor(c.api)).success).toBe(false); + }); + } +}); From 5dc52144eb04c264f25fafed4b212045a850320b Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 21:13:38 -0500 Subject: [PATCH 14/31] fix(api): close the code-review findings on the input and page-size work - C#: with api.pagination set, `limit=0` served the default page; it is now a page of zero rows, as in every other port. New shared scenario api-settings/limit-zero (all five ports green). - C#: the unknown-field check matches a key the way the route binds the body (the app's System.Text.Json options; the ASP.NET web default ignores case), so `Title` is the declared `title`, not a 400. The verify advisory matches the routes generator by type or exact name, not by a "routes" substring. - Python: a rejected decimal's text is no longer echoed into the 400 issue message; the unbounded-list-pages advisory fires only when the routes generator is wired; the dead _DECIMAL_COERCE constant is gone. - Java: UnreadableBody's Javadoc describes the issues envelope it returns. - Docs: M:N traversal routes take no limit and are not paged by api.pagination (stated as the boundary); the C# port doc no longer says dotnet meta init writes no config. Counts updated. --- CHANGELOG.md | 8 +++-- docs/CONFORMANCE.md | 12 +++---- docs/features/api-contract.md | 12 +++++-- docs/ports/csharp.md | 8 +++-- .../.metaobjects/.gen-state/.hashes.json | 2 +- .../generated/csharp/SubscriberRoutes.g.cs | 4 +-- fixtures/api-contract-conformance/README.md | 4 +-- .../api-settings/README.md | 3 ++ .../api-settings/scenarios/limit-zero.yaml | 13 ++++++++ .../csharp/MetaObjects.Cli/VerifyCommand.cs | 9 ++++-- .../ApiSettingsTests.cs | 11 +++++++ .../Generators/RoutesGenerator.cs | 6 ++-- .../Runtime/UnknownFieldCheck.cs | 25 +++++++++++++-- .../spring/runtime/UnreadableBody.java | 10 +++--- server/python/src/metaobjects/cli.py | 32 ++++++++++++++++--- .../codegen/generators/router_generator.py | 9 ++---- .../codegen/runtime/decimal_wire.py | 4 +-- .../test_cli_verify_pagination_advisory.py | 15 +++++++-- .../tests/runtime/test_validation_issues.py | 11 +++++++ 19 files changed, 154 insertions(+), 44 deletions(-) create mode 100644 fixtures/api-contract-conformance/api-settings/scenarios/limit-zero.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 784a0c1559..c6df422410 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,7 +96,9 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi `fixtures/api-contract-conformance/api-settings/` sub-corpus. Every port's config reader takes a page size as a positive whole number however JSON spells it (`50.0` is `50`) and at most 2147483647, and refuses a malformed block when the config is read; the shared cases, valid and - refused, are `api-settings/config-blocks.json`, run by every port. A config key, not vocabulary: + refused, are `api-settings/config-blocks.json`, run by every port. `limit=0` is a page of zero + rows in every port (the C# route served the default page; `api-settings/limit-zero`). An M:N + traversal route takes no `limit` and is not paged by the setting. A config key, not vocabulary: `metamodelVersion` does not move. - **Report measures over Money and other value-object members (FR-044 R12).** A dimension's or @@ -187,7 +189,9 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi refused and never stored, on create and on PATCH, while the open JSON bag keeps every key (`jsonb/jsonb-nested-undeclared-key-dropped`, both lanes of every port). The Java generated value-object records now carry `@JsonIgnoreProperties(ignoreUnknown = true)` for this; they - answered `400 invalid_type` before. + answered `400 invalid_type` before. C# matches a key the way it binds the body, with the app's + JSON options: under the ASP.NET web default, which ignores case, `Title` is the declared + `title` and is not refused. A config key, not vocabulary: `metamodelVersion` does not move. - **A validation 400 lists every issue in every port.** The same model answered a failed create or diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index a1e6b2d9b8..d709c815e5 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -36,7 +36,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 58 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 64 (55 query + 9 migration) | all 64; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 55 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 55 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 138 (34 core + 14 tph + 9 m2m + 6 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 7 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 139 (34 core + 14 tph + 9 m2m + 6 jsonb + 2 write-through + 2 secret-fields + 2 enum-default + 12 projection + 37 report + 11 decimal + 2 duration + 8 api-settings) | ✓ (Fastify reference + generated lane); `report/`, `decimal/` and `duration/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/`, `decimal/` and `duration/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/`, `decimal/` and `duration/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/duration-wire/`](../fixtures/duration-wire/) | 1 vector file: the `wire` vectors (parse, precision, refused) every port passes, and the `codecs` groups (Postgres text read and write, stored microseconds) each port passes only for the codecs it has | ✓ (`metadata` / `runtime-ts` and the generated helper) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases + 43 cross-field cases (`cross-field-cases.json`, generated record model only) | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | @@ -442,9 +442,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (138) +### `fixtures/api-contract-conformance/` (139) -All 138 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 139 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -468,9 +468,9 @@ identity has no item route; a projection keyed on a field not named `id` is addr it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one canonical string, read exactly, filtered exactly and refused past its declared digits), `duration/` (2, a `field.duration` under `"wire": { "durations": "iso" }` is one canonical ISO-8601 string, refused -in a calendar or clock spelling, and filtered and sorted as a span on every engine), `api-settings/` (7, +in a calendar or clock spelling, and filtered and sorted as a span on every engine), `api-settings/` (8, the project's `api` block: `api.pagination`'s default page, `limit_too_large` cap and refused -negative `limit`, and +negative `limit`, an empty page for `limit=0`, and `api.unknownFields: drop`) and `report/` (37, FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps @@ -731,7 +731,7 @@ JSON and verify payload feeds it unchanged. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 510 + yaml 16 + verify 45 -+ render 18 + persistence 64 + api-contract 138 + source-resolution 25 + scope 10 + ++ render 18 + persistence 64 + api-contract 139 + source-resolution 25 + scope 10 + dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the `cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index fac81363a6..ccf8312576 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -286,12 +286,15 @@ each handler they register instead; an `HTTPException` is rethrown to your a client that asked for more than the server serves is told so rather than handed a short page it cannot tell from the end of the list. A negative `limit` answers the malformed-bound envelope above (`pagination.invalid_value`) in every port, so it never reaches a store that - reads `LIMIT -1` as no limit (SQLite). `offset` and `withCount=1` page past the default as + reads `LIMIT -1` as no limit (SQLite). `limit=0` is a page of zero rows, not "no limit" + (`api-settings/limit-zero`). `offset` and `withCount=1` page past the default as usual, with `total` counting every row. `meta init` writes the block above for a new project; both sizes must be positive whole numbers (`50.0` reads as `50`) no larger than 2147483647, and `defaultLimit` may not exceed `maxLimit`; a block that breaks a rule is refused when the config is read, in every port (shared cases: `api-settings/config-blocks.json`). A report - keeps its own row cap (above) and an opaque view its 1000, whatever the setting says. + keeps its own row cap (above) and an opaque view its 1000, whatever the setting says. An M:N + traversal route (`GET //{id}/`) takes no `limit` or `offset` in any port + and is not paged by the setting: it answers the whole related set, as it always has. **Without the setting nothing changes, and the default is per port.** With `limit` omitted, the TypeScript mounts and the C# routes return **every** matching row, while the Java, Kotlin @@ -664,7 +667,10 @@ its `@autoSet` stamps and its `writeOnce` field) keeps working, and a sibling su another subtype's route is dropped. Only the top level is checked; a value object's members are validated as before, and a member a value object does not declare is dropped silently, never refused and never stored, on create and on PATCH (`jsonb/jsonb-nested-undeclared-key-dropped`, -both lanes of every port). The open JSON bag keeps every key. `meta init` writes `"unknownFields": "reject"` for a new project. +both lanes of every port). The open JSON bag keeps every key. A key is matched the way the port +binds it: the C# routes bind with the app's System.Text.Json options, whose ASP.NET web default +ignores case, so there `Title` is the declared `title`, not an unknown key; the other ports +match the exact name. `meta init` writes `"unknownFields": "reject"` for a new project. Each port checks the raw body before it binds it, so the answer is the same whatever the host framework would do with an unknown property: Java and Kotlin read the keys from the request's JSON diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index f3b630a904..c6734184f2 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -400,7 +400,7 @@ the gate like any packaged generator's. The example above is compiled and run by `dotnet meta init --with-docs` writes the list with the api reference already in it (`new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = "docs/csharp", ProjectName = "" }`), -so `gen` writes it. It does not write `.metaobjects/config.json`: set `"csharp": { "outDir": "" }` +so `gen` writes it. The `.metaobjects/config.json` init writes names no `outDir`: set `"csharp": { "outDir": "" }` there and a plain `dotnet meta verify` drift-checks the docs with the rest of the codegen output (`verify --codegen` compares the whole tree of each entry's `OutDir`, so keep it generated files only). Against an existing list, init prints the entry to add and edits nothing. Without the flag @@ -1088,7 +1088,11 @@ list route a default page and answers a larger `?limit=` with `400 { "error": "l `api.unknownFields` (`reject`, the default, or `drop`) makes a create, PUT or PATCH answer `400 { "error": "unknown_fields", "fields": [...] }` for a body key no field of the entity (for a TPH hierarchy, of the base or any subtype) declares; the check is `UnknownFieldCheck` in the helper runtime. -`dotnet meta init` does not write `.metaobjects/config.json`. `dotnet meta verify --codegen` prints a +It matches a key the way the route binds the body, with the app's System.Text.Json options: under the +ASP.NET web default, which ignores case, `Title` is the declared `title`. `limit=0` is a page of zero rows. +`dotnet meta init` writes `.metaobjects/config.json` with the same new-project settings as `meta init`: +`api.pagination` 50/1000, `api.unknownFields: reject`, `verify.handEdits: fail` and +`migrate.foreignKeyIndexes: true`; an existing config is left alone. `dotnet meta verify --codegen` prints a non-failing advisory (`unbounded-list-pages`, stderr, exit code unchanged) when the run includes the routes generator and the project sets no `api.pagination`; plain `verify` says nothing, because only `--codegen` runs the generator list. diff --git a/examples/showcase/.metaobjects/.gen-state/.hashes.json b/examples/showcase/.metaobjects/.gen-state/.hashes.json index d030b08726..e6cb64c5db 100644 --- a/examples/showcase/.metaobjects/.gen-state/.hashes.json +++ b/examples/showcase/.metaobjects/.gen-state/.hashes.json @@ -4,7 +4,7 @@ "generated/csharp/SubscriberBlurbPayload.g.cs": "29eb3a6ce64496ffe734cba55d0213994316b4281d3fa6791917393bae36c9ed", "generated/csharp/SubscriberFilterAllowlist.g.cs": "3839b04884e4e393d4f7d7bf4c7d4fe5b3852bd3a92b1ecb282cb8935a3c98e5", "generated/csharp/SubscriberNames.g.cs": "c2cf0248c2ecde155ec2623604771e39bed23783603d6f18a2ed82967227a732", - "generated/csharp/SubscriberRoutes.g.cs": "152ee0d118c905bbece66bdd8153e12cc28f4c431f9d5a06f0fbb341f9337671", + "generated/csharp/SubscriberRoutes.g.cs": "2b7ebe7f23e912f38ca923fce0df4cf82c89f9b909ea9bf5b2d4c9d16e4064a8", "generated/python/Subscriber.py": "9dd4c0acea47f99d1daf79930f9de2e400f30780ebce6c63f023be528a72b762", "generated/python/SubscriberBlurbPayload.py": "a5ef8a08e63efbe0f73e39a21a368cff5150c68cc8beeea6e8aab0c755f8bbdd", "generated/python/__init__.py": "daa04cec586768866d0996c779f62073e6773fe63750ff668327991fac2fd0f5", diff --git a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs index 163208f64a..f3805b901f 100644 --- a/examples/showcase/generated/csharp/SubscriberRoutes.g.cs +++ b/examples/showcase/generated/csharp/SubscriberRoutes.g.cs @@ -101,7 +101,7 @@ await db.Subscribers.FindAsync(id) is { } found using var __bodyScope = __body; if (__body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable(); - if (UnknownFieldCheck.Reject(__body.RootElement, KnownFields) is { } __unknown) return __unknown; + if (UnknownFieldCheck.Reject(__body.RootElement, KnownFields, http) is { } __unknown) return __unknown; var __issues = new ValidationIssues(); var __present = new System.Collections.Generic.HashSet(System.StringComparer.OrdinalIgnoreCase); foreach (var __p in __body.RootElement.EnumerateObject()) @@ -139,7 +139,7 @@ async System.Threading.Tasks.Task UpdateSubscriber(long id, HttpContext catch (System.Text.Json.JsonException) { return ValidationIssues.Unreadable(); } using var bodyScope = body; if (body.RootElement.ValueKind != System.Text.Json.JsonValueKind.Object) return ValidationIssues.Unreadable(); - if (UnknownFieldCheck.Reject(body.RootElement, KnownFields) is { } __unknown) return __unknown; + if (UnknownFieldCheck.Reject(body.RootElement, KnownFields, http) is { } __unknown) return __unknown; // Deserialize each value with the app's CONFIGURED JSON options, not defaults — // a field.timestamp binds to DateTimeOffset via a custom converter registered on // ConfigureHttpJsonOptions, and default options cannot read the wire format, throwing 500. diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index f4bd97cdff..5e46840049 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -57,7 +57,7 @@ fixtures/api-contract-conformance/ ``` Beside the 34 core scenarios above sit eleven **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (104 scenarios in all, 138 with the +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (105 scenarios in all, 139 with the core): | Directory | Scenarios | Gates | Lanes | @@ -71,7 +71,7 @@ core): | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 37 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | | `decimal/` | 11 | `field.decimal` is an exact canonical string on the wire (ADR-0019 Amendment 1): read, filter, sort, write and a report over decimals. A scenario's `engines:` key is exact; the `-15` twins run on SQLite and D1 as well. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | -| `api-settings/` | 7 | the `api` block in the sub-corpus's own `.metaobjects/config.json`: `api.pagination` (default 2, max 4) — a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served, one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` and a negative one is `400 pagination.invalid_value` — and `api.unknownFields: drop`, under which an undeclared body key is ignored rather than refused | generated only | +| `api-settings/` | 8 | the `api` block in the sub-corpus's own `.metaobjects/config.json`: `api.pagination` (default 2, max 4) — a list with no `limit` answers the default page, `offset` and `withCount=1` page past it, `limit` at the cap is served, one above it is `400 {"error": "limit_too_large", "maxLimit": 4}` and a negative one is `400 pagination.invalid_value`, while `limit=0` is an empty page — and `api.unknownFields: drop`, under which an undeclared body key is ignored rather than refused | generated only | | `duration/` | 2 | `field.duration` is a canonical ISO-8601 span on the wire under `"wire": { "durations": "iso" }`: write, read, refuse, and filter and sort as a span on every engine. Also carries `schema.postgres.sql` / `schema.sqlite.sql` / `schema.mysql.sql` | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` diff --git a/fixtures/api-contract-conformance/api-settings/README.md b/fixtures/api-contract-conformance/api-settings/README.md index 818201fe95..272dbe20b2 100644 --- a/fixtures/api-contract-conformance/api-settings/README.md +++ b/fixtures/api-contract-conformance/api-settings/README.md @@ -25,6 +25,8 @@ scenarios can tell the default page from the whole table and the cap from the ta - **Over the cap is refused, not clamped.** `?limit=5` answers `400 {"error": "limit_too_large", "maxLimit": 4}`, so a client that asked for more than the server serves is told so instead of receiving a short page it cannot tell from the end. +- **`limit=0` is an empty page.** `?limit=0` answers no rows: an explicit zero is a page size, + not "no limit", so it is neither the default page nor the whole table. - **A negative limit is refused.** `?limit=-1` answers `400 {"error": "pagination.invalid_value", "param": "limit", "value": "-1", "expected": "a non-negative integer (0 or more)"}` rather than reaching the store, where SQLite reads `LIMIT -1` as no limit at all. @@ -73,6 +75,7 @@ api-settings/ ├── limit-at-max.yaml ├── limit-negative.yaml ├── limit-over-max.yaml + ├── limit-zero.yaml ├── offset-pages-past-default.yaml ├── unknown-fields-dropped.yaml └── with-count-default-page.yaml diff --git a/fixtures/api-contract-conformance/api-settings/scenarios/limit-zero.yaml b/fixtures/api-contract-conformance/api-settings/scenarios/limit-zero.yaml new file mode 100644 index 0000000000..0940d42496 --- /dev/null +++ b/fixtures/api-contract-conformance/api-settings/scenarios/limit-zero.yaml @@ -0,0 +1,13 @@ +name: limit-zero +description: > + An explicit `?limit=0` is a real page size of zero, not "no limit": it answers an empty page, + never the default page (2) and never the whole table (5). The C# route read it as no limit and + served the default page while every other port served nothing. +requests: + - id: r1 + method: GET + path: /api/items?limit=0&sort=id:asc + expect: + status: 200 + body: + ids: [] diff --git a/server/csharp/MetaObjects.Cli/VerifyCommand.cs b/server/csharp/MetaObjects.Cli/VerifyCommand.cs index 90bc29dd50..d089cb4e8e 100644 --- a/server/csharp/MetaObjects.Cli/VerifyCommand.cs +++ b/server/csharp/MetaObjects.Cli/VerifyCommand.cs @@ -56,6 +56,9 @@ public sealed record Outcome( // ADR-0021 D2 — explicit verify subverbs (--templates / --codegen / --db). // ------------------------------------------------------------------------ + /// The name the generated-routes generator reports (RoutesGenerator.Name). + private const string RoutesGeneratorName = "routes-generator"; + /// Advisory rule unbounded-list-pages: routes are generated and no api.pagination is set. public const string UNBOUNDED_LIST_PAGES_ADVISORY = "dotnet meta verify --codegen: advisory unbounded-list-pages: list routes return every row when `limit` is omitted; " + @@ -448,9 +451,11 @@ private static Codegen.CodegenDrift.Result RunCodegenDrift(Options opts, List g is Codegen.Generators.RoutesGenerator || g.Name.Contains("routes", StringComparison.OrdinalIgnoreCase)) + && generators.Any(g => g is Codegen.Generators.RoutesGenerator || g.Name == RoutesGeneratorName) && load.Root.Objects().Any(o => Codegen.Generators.RoutesGenerator.AppliesTo(o, load.Root))) advisories.Add(UNBOUNDED_LIST_PAGES_ADVISORY); diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs index 703ee83ad8..996472129d 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiSettingsTests.cs @@ -123,4 +123,15 @@ public void A_body_that_is_not_an_object_or_has_only_declared_keys_is_not_refuse Assert.Null(UnknownFieldCheck.Reject(array.RootElement, known)); Assert.Null(UnknownFieldCheck.Reject(declared.RootElement, known)); } + + [Fact] + public void A_key_differing_only_in_case_is_declared_when_the_binder_ignores_case() + { + // ASP.NET's web JSON defaults bind `Title` to `title`, so refusing it would refuse a body the + // route then binds. With case-sensitive options it is an undeclared key, as in the other ports. + var known = new HashSet { "title" }; + using var doc = JsonDocument.Parse("{\"Title\": \"x\"}"); + Assert.Null(UnknownFieldCheck.Reject(doc.RootElement, known, caseInsensitive: true)); + Assert.NotNull(UnknownFieldCheck.Reject(doc.RootElement, known, caseInsensitive: false)); + } } diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index e281bbe07e..f720c6fbbb 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -1057,7 +1057,8 @@ private static void AppendKnownFields(StringBuilder sb, IEnumerable private static void AppendUnknownFieldsCheck(StringBuilder sb, MetaRoot root, string bodyVar) { if (root.Api.UnknownFields != UnknownFieldsMode.Reject) return; - sb.AppendLine($" if (UnknownFieldCheck.Reject({bodyVar}.RootElement, {KnownFieldsConst}) is {{ }} __unknown) return __unknown;"); + // `http` carries the app's JSON options: a key the binder matches case-insensitively is declared. + sb.AppendLine($" if (UnknownFieldCheck.Reject({bodyVar}.RootElement, {KnownFieldsConst}, http) is {{ }} __unknown) return __unknown;"); } // The offset + limit lines of a list route. Without `api.pagination` they are exactly what they always @@ -1077,7 +1078,8 @@ private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, str sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}Neg) && long.TryParse({limRaw}Neg, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}Neg) && {lim}Neg < 0)"); sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"limit\", value = {lim}Neg.ToString(System.Globalization.CultureInfo.InvariantCulture), expected = \"a non-negative integer (0 or more)\" }});"); sb.AppendLine($"{indent}var __take = {page.DefaultLimit};"); - sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && long.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} > 0)"); + // `limit=0` is a page of zero rows, as every other port serves it, not "no limit". + sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && long.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} >= 0)"); sb.AppendLine($"{indent}{{"); sb.AppendLine($"{indent} if ({lim} > {page.MaxLimit}) return Results.BadRequest(new {{ error = \"limit_too_large\", maxLimit = {page.MaxLimit} }});"); sb.AppendLine($"{indent} __take = (int){lim};"); diff --git a/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs index 1b5eb57f56..b77d291afe 100644 --- a/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs +++ b/server/csharp/MetaObjects.Codegen/Runtime/UnknownFieldCheck.cs @@ -7,23 +7,42 @@ // port lists them in), listed once each. A declared key the client may not set (the generated key, a readOnly field, a frozen field) is // not unknown: the route drops it as it always did, so a client that echoes back a row it read works. // Under `api.unknownFields: drop` the route never calls this. +// +// A key is matched the way the route's binder matches it: when the app's System.Text.Json options are +// case-insensitive (the ASP.NET web default), `Title` binds to `title`, so it is declared, not unknown. using System.Text.Json; using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.Options; namespace MetaObjects.Codegen.Runtime; /// The unknown-field refusal of a generated write route. public static class UnknownFieldCheck { + /// with the case rule of the + /// app's configured JSON options, the ones the route binds the body with. + public static IResult? Reject(JsonElement body, IReadOnlySet known, HttpContext http) + { + var options = http.RequestServices.GetService(typeof(IOptions)) + as IOptions; + return Reject(body, known, options?.Value.SerializerOptions.PropertyNameCaseInsensitive ?? false); + } + /// The 400 naming every body key not in , or null when there is none - /// (or the body is not a JSON object, which the route's own handling answers). - public static IResult? Reject(JsonElement body, IReadOnlySet known) + /// (or the body is not a JSON object, which the route's own handling answers). With + /// , a key that differs from a declared name only in case is declared. + public static IResult? Reject(JsonElement body, IReadOnlySet known, bool caseInsensitive = false) { if (body.ValueKind != JsonValueKind.Object) return null; var unknown = new List(); foreach (var property in body.EnumerateObject()) - if (!known.Contains(property.Name) && !unknown.Contains(property.Name)) unknown.Add(property.Name); + { + var name = property.Name; + var declared = known.Contains(name) + || (caseInsensitive && System.Linq.Enumerable.Any(known, k => string.Equals(k, name, System.StringComparison.OrdinalIgnoreCase))); + if (!declared && !unknown.Contains(name)) unknown.Add(name); + } if (unknown.Count == 0) return null; unknown.Sort(CompareCodePoints); return Results.BadRequest(new { error = "unknown_fields", fields = unknown }); diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java index aef14495b1..13f20f85c7 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/runtime/UnreadableBody.java @@ -10,10 +10,12 @@ * The wire answer for a request body Jackson could not bind (ADR-0019 Amendment 1, W2/W3): a * malformed decimal, an out-of-range one, a wrong JSON type. The generated controller's * {@code @ExceptionHandler} for {@code HttpMessageNotReadableException} returns - * {@code 400 {"error": "validation", "field": ""}}, the cross-port validation envelope, - * instead of Spring's default body. {@code field} is the property Jackson was binding when it - * failed, and is left out when the failure is not about one property (invalid JSON text). - * Only the property name is read, never the offending value. + * {@code 400 {"error": "validation", "issues": [{"code", "path", "message"}], "field": ""}}, + * the cross-port validation envelope, instead of Spring's default body. The one issue's + * {@code path} is every property Jackson was binding (nested members and array indexes + * included), empty when the body as a whole cannot be read (invalid JSON text). {@code field} + * is kept for clients that read it: the last property name, left out when there is none. + * Only property names are read, never the offending value. */ public final class UnreadableBody { diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index 567dc51b22..7064094474 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -2768,12 +2768,36 @@ def _run_field_lint_advisory(args: argparse.Namespace, model: "_VerifyModel | No print(f" {finding.code} [{finding.path}]: {finding.message}", file=sys.stderr) -def _run_pagination_advisory(model: "_VerifyModel | None") -> None: +#: The registry id of the generator that writes the REST list routes. +_ROUTES_GENERATOR = "routes" + + +def _routes_generator_wired(args: argparse.Namespace) -> bool: + """Whether this run's generator list includes the routes generator: the ``--generators`` + narrowing when given, else any target of ``metaobjects.config.yaml``. A project with no + config, or one whose routes are owned rather than generated, has no generated list route.""" + selection = _selection_from_args(args) + if selection is not None: + return _ROUTES_GENERATOR in selection + config_path = _find_config(args) + if config_path is None: + return False + try: + config = load_project_config(config_path) + except ConfigError: + return False + return any(_ROUTES_GENERATOR in (t.generators or []) for t in config.targets) + + +def _run_pagination_advisory(args: argparse.Namespace, model: "_VerifyModel | None") -> None: """``unbounded-list-pages``: ONE advisory line when the project generates REST list routes - and sets no ``api.pagination``, so a request with no ``limit`` is served by the 50-row - historic default and ``?limit=`` has no cap. Advisory only: stderr, never the exit code.""" + (the routes generator is wired and the model has an entity it serves) and sets no + ``api.pagination``, so a request with no ``limit`` is served by the 50-row historic default + and ``?limit=`` has no cap. Advisory only: stderr, never the exit code.""" if model is None or model.root.api.pagination is not None: return + if not _routes_generator_wired(args): + return try: from metaobjects.codegen.runtime.shared_predicates import emits_router @@ -2988,7 +3012,7 @@ def _cmd_verify(args: argparse.Namespace) -> int: exit_code = max(exit_code, _verify_requirements(args, model)) _run_field_lint_advisory(args, model) if exit_code != 2: # 2 is a refused invocation: its own message is the whole answer - _run_pagination_advisory(model) + _run_pagination_advisory(args, model) return exit_code diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index fc7f985790..d8c070d878 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -215,13 +215,10 @@ def _scalar_fields(entity: MetaObject) -> list[MetaField]: " return JSONResponse(status_code=400, content=validation_envelope(value_error_issues(_exc)))", ] -#: The start of every write handler: the unreadable-body answer, the unknown-key refusal when the -#: project rejects them, then the exact decimal parse. -_DECIMAL_COERCE: list[str] = [*_BODY_CHECK, *_DECIMAL_COERCE_BODY] - - def _write_prelude(reject: bool) -> list[str]: - """The body checks that open a create / PUT / PATCH handler (see :data:`_DECIMAL_COERCE`).""" + """The start of every create / PUT / PATCH handler: the unreadable-body answer, the + unknown-key refusal when the project rejects them (``api.unknownFields``), then the exact + decimal parse.""" return [*_BODY_CHECK, *(_UNKNOWN_FIELDS_CHECK if reject else []), *_DECIMAL_COERCE_BODY] diff --git a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py index cbbd9c8506..16ced94dfb 100644 --- a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py +++ b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py @@ -93,11 +93,11 @@ def _reject() -> Decimal: def _parse_text(text: str) -> Decimal: if _JSON_NUMBER.fullmatch(text) is None: - raise ValueError(f"{text!r} is not a decimal") + raise ValueError("the text is not a decimal") try: return _checked(Decimal(text)) except InvalidOperation: - raise ValueError(f"{text!r} is not a decimal") from None + raise ValueError("the text is not a decimal") from None def decimal_from_double15(value: float | str) -> Decimal: diff --git a/server/python/tests/codegen/test_cli_verify_pagination_advisory.py b/server/python/tests/codegen/test_cli_verify_pagination_advisory.py index 8c1cae63db..90728aebc4 100644 --- a/server/python/tests/codegen/test_cli_verify_pagination_advisory.py +++ b/server/python/tests/codegen/test_cli_verify_pagination_advisory.py @@ -48,10 +48,10 @@ def _project(tmp_path: Path, api: dict[str, object] | None) -> str: return str(meta) -def _verify(tmp_path: Path, meta_dir: str) -> int: +def _verify(tmp_path: Path, meta_dir: str, generators: str = GEN_SUITE) -> int: out = tmp_path / "out" - assert main(as_config(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)])) == 0 - return main(as_config(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)])) + assert main(as_config(["gen", "--generators", generators, meta_dir, "--out", str(out)])) == 0 + return main(as_config(["verify", "--codegen", "--generators", generators, meta_dir, "--out", str(out)])) def test_advises_when_routes_are_generated_and_no_page_size_is_set( @@ -67,3 +67,12 @@ def test_silent_when_the_project_sets_api_pagination( api = {"pagination": {"defaultLimit": 50, "maxLimit": 1000}} assert _verify(tmp_path, _project(tmp_path, api)) == 0 assert _ADVISORY not in capsys.readouterr().err + + +def test_silent_when_no_routes_generator_is_wired( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + # Entities and models only, no routers: nothing generated serves a list, so there is no + # page size to advise on (an owned routes copy is the adopter's, as in the other ports). + assert _verify(tmp_path, _project(tmp_path, None), "entity,filter-allowlist,names") == 0 + assert _ADVISORY not in capsys.readouterr().err diff --git a/server/python/tests/runtime/test_validation_issues.py b/server/python/tests/runtime/test_validation_issues.py index d4cac02c95..b651405242 100644 --- a/server/python/tests/runtime/test_validation_issues.py +++ b/server/python/tests/runtime/test_validation_issues.py @@ -133,6 +133,17 @@ def test_decimal_failures_carry_a_wire_path_and_code() -> None: assert (issue["path"], issue["code"]) == (["amount"], "too_big") +def test_a_rejected_decimal_is_never_echoed_into_the_issue() -> None: + # The client's text must not come back in the 400 body: a long or chosen string would be + # reflected, and a credential field would be repeated in a log of the response. + sent = "not-a-number-" + "x" * 200 + with pytest.raises(ValueError) as caught: + coerce_decimal_fields({"amount": sent}, frozenset({"amount"})) + [issue] = value_error_issues(caught.value) + assert "not-a-number" not in issue["message"] + assert (issue["path"], issue["code"]) == (["amount"], "invalid_type") + + def test_a_nested_decimal_failure_names_its_member_path() -> None: spec = {"money": {"d": {"value": (5, 2)}, "o": {}}} with pytest.raises(ValueError) as caught: From eca2e7b03634a5cb8237673025a2d3a166e22e37 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 21:20:08 -0500 Subject: [PATCH 15/31] refactor: read the project config once in the Maven mojo; drop a no-op path coercion in Python issues Simplify pass: AbstractMetaDataMojo maps its four project settings from one read of .metaobjects/config.json, and pydantic_issues copies loc as-is (its entries are already int or str). --- .../com/metaobjects/mojo/AbstractMetaDataMojo.java | 10 ++++++---- .../metaobjects/codegen/runtime/constraint_errors.py | 2 +- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java index 8deffdb8b1..b71c5497f9 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/AbstractMetaDataMojo.java @@ -795,21 +795,23 @@ protected MetaDataLoader createLoader(ClassLoader projectClassLoader) { // ADR-0058 — the project's physical-naming rules, set on the loaded root so every // resolver downstream of this load (codegen, OMDB) reads the one setting. loader.getRoot().setPhysicalNaming(resolvePhysicalNaming(srcDir)); + // The project's config file is read once; each setting below is mapped from it. + java.util.Optional projectConfig = resolveProjectConfig(srcDir); // ADR-0019 Amendment 2 — the wire setting, from the same file, read by the generators. - loader.getRoot().setWireFormat(resolveProjectConfig(srcDir) + loader.getRoot().setWireFormat(projectConfig .map(com.metaobjects.config.NeutralConfig::getWireFormat) .orElse(com.metaobjects.config.WireFormat.DEFAULT)); // The enum storage defaults (`enums`), from the same file, read by the table generators. - loader.getRoot().setEnumDefaults(resolveProjectConfig(srcDir) + loader.getRoot().setEnumDefaults(projectConfig .map(com.metaobjects.config.NeutralConfig::getEnumDefaults) .orElse(com.metaobjects.config.EnumDefaults.DEFAULT)); // The `api` block (`api.pagination`, `api.unknownFields`), from the same file, read by the generators. - loader.getRoot().setApiSettings(resolveProjectConfig(srcDir) + loader.getRoot().setApiSettings(projectConfig .map(com.metaobjects.config.NeutralConfig::getApiSettings) .orElse(com.metaobjects.config.ApiSettings.DEFAULT)); // The `migrate` schema settings (foreignKeyIndexes, conventions, dialect), read by the // table generators so a mapping declares the indexes `meta migrate` creates. - loader.getRoot().setMigrateSettings(resolveProjectConfig(srcDir) + loader.getRoot().setMigrateSettings(projectConfig .map(com.metaobjects.config.NeutralConfig::getMigrateSettings) .orElse(com.metaobjects.config.MigrateSettings.DEFAULT)); diff --git a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py index c3df9564ce..08f81ea565 100644 --- a/server/python/src/metaobjects/codegen/runtime/constraint_errors.py +++ b/server/python/src/metaobjects/codegen/runtime/constraint_errors.py @@ -226,7 +226,7 @@ def pydantic_issues(errors: list[Any]) -> list[dict[str, Any]]: return [ { "code": _issue_code(str(e.get("type", ""))), - "path": [p if isinstance(p, (int, str)) else str(p) for p in e.get("loc", ())], + "path": list(e.get("loc", ())), "message": str(e.get("msg", "invalid value")), } for e in errors From 7e60564a9171e15216eb6294599bd44bf18b723e Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 21:54:42 -0500 Subject: [PATCH 16/31] no-mistakes(review): Scope ignoreUnknown to request bodies; fix review items --- CHANGELOG.md | 5 +- docs/features/api-contract.md | 2 +- .../java/acme/SubscriberBlurbPayload.java | 1 - .../kotlin/acme/SubscriberBlurbPayload.kt | 2 - .../ApiContractApiSettingsConformanceTest.cs | 10 +- .../generator/util/RestSurfaceGate.java | 44 ++++++ .../generator/kotlin/KotlinEntityGenerator.kt | 23 ++- .../kotlin/KotlinSpringControllerGenerator.kt | 10 +- .../KotlinRequestBodyUnknownKeysTest.kt | 87 +++++++++++ .../kotlin/KotlinServerOwnedInputTest.kt | 113 -------------- .../acme/ops/OrderReport.kt | 2 - .../acme/ops/OrderReportArgs.kt | 2 - .../acme/auth/AuthController.kt | 18 +-- .../acme/report/SalesReport.kt | 2 - .../payload-with-origins/acme/ai/Author.kt | 2 - .../acme/ai/AuthorReport.kt | 2 - .../payload-with-origins/acme/ai/Post.kt | 2 - .../acme/ai/OpinionOutputPayload.kt | 2 - .../acme/ai/Greeting.kt | 2 - .../template-prompt-simple/acme/ai/Author.kt | 2 - .../generator/spring/SpringDtoGenerator.java | 14 +- .../spring/SpringValueObjectGenerator.java | 12 +- ...gRequestBodyUnknownKeysCompileRunTest.java | 88 +++++++++++ .../SpringServerOwnedInputCodegenTest.java | 138 ------------------ .../expected/acme/shipping/ShipmentNote.java | 1 - .../acme/shipping/ShipmentSummaryDto.java | 1 - .../metaobjects/mojo/MetaDataVerifyMojo.java | 8 +- ...taDataVerifyUnboundedListAdvisoryTest.java | 11 ++ 28 files changed, 297 insertions(+), 309 deletions(-) create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRequestBodyUnknownKeysTest.kt delete mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt create mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringRequestBodyUnknownKeysCompileRunTest.java delete mode 100644 server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index c6df422410..944f8a8016 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,7 +57,8 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi route's subtype is its only source) and every column the route's subtype does not declare. An assigned key is still the caller's on create. No port answers them with a 400. A key the entity does not declare is a separate case, now a project setting (`api.unknownFields`, - under "Changed" below). Java DTOs and Kotlin entity classes now carry + under "Changed" below). Java DTOs and Kotlin entity classes that are a + write route's request body (and the value objects nested in one) now carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so binding no longer depends on the host's `ObjectMapper` (a plain one answered 400). The Python 1.2.1 fix is carried forward to this line, and the Python TPH create now drops a body's discriminator instead of answering 400 for @@ -188,7 +189,7 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi Only the top level is judged: a member a value object does not declare is dropped, never refused and never stored, on create and on PATCH, while the open JSON bag keeps every key (`jsonb/jsonb-nested-undeclared-key-dropped`, both lanes of every port). The Java generated - value-object records now carry `@JsonIgnoreProperties(ignoreUnknown = true)` for this; they + value-object records nested in a write route's body now carry `@JsonIgnoreProperties(ignoreUnknown = true)` for this; they answered `400 invalid_type` before. C# matches a key the way it binds the body, with the app's JSON options: under the ASP.NET web default, which ignores case, `Title` is the declared `title` and is not refused. diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index ccf8312576..ea1fdf6ccf 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -637,7 +637,7 @@ routes: None of these is a `400`, and none reaches the repository. Each port does it in its own idiom: TypeScript parses with the Zod input schema (which drops keys it does not list) and writes the parsed value; Python keeps only the keys its input model validated; Java nulls the dropped components of the bound DTO before -`repository.create` and its DTOs carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer does not +`repository.create` and its request-body DTOs carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so the answer does not depend on the host's `ObjectMapper`; Kotlin writes only the route's settable columns; C# resets the bound entity's generated key, TPH discriminator and M:N navigations before `Add`, and gives a `readOnly` property a private setter, which System.Text.Json never binds. The shared scenarios `input-server-owned-dropped` and diff --git a/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java b/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java index 93c38ac0a4..b0e6be9ee0 100644 --- a/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java +++ b/examples/showcase/generated/java/acme/SubscriberBlurbPayload.java @@ -3,7 +3,6 @@ import jakarta.validation.constraints.*; /** GENERATED — value object SubscriberBlurbPayload. Do not hand-edit; regenerated from metadata. */ -@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record SubscriberBlurbPayload( String name, @NotNull Status status diff --git a/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt b/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt index fc048fbf6b..acb1c79b0a 100644 --- a/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt +++ b/examples/showcase/generated/kotlin/acme/SubscriberBlurbPayload.kt @@ -1,6 +1,5 @@ package acme -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.NotNull import kotlin.String import kotlin.jvm.JvmStatic @@ -8,7 +7,6 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class SubscriberBlurbPayload( public val name: String? = null, @field:NotNull diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs index 814a215bd6..9fdccad534 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs @@ -1,9 +1,9 @@ -// ApiContractApiSettingsConformanceTest — the C# #484 secret-fields api-contract lane. +// ApiContractApiSettingsConformanceTest — the C# api-settings api-contract lane. // -// Drives the fixtures/api-contract-conformance/secret-fields/ scenarios over HTTP against the -// GENERATED Account routes booted unmodified on Kestrel against a Postgres testcontainer. -// Proves the deployed REST surface ignores writes to @formExclude / credential fields and -// never returns a credential. Generated lane only. +// Drives the fixtures/api-contract-conformance/api-settings/ scenarios over HTTP against the +// GENERATED Item routes booted unmodified on Kestrel. Proves the deployed REST surface applies +// api.pagination (default page size, hard cap on limit) and api.unknownFields (drop or reject +// undeclared request keys). Generated lane only. using System.Text.Json; using System.Text.Json.Nodes; diff --git a/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/RestSurfaceGate.java b/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/RestSurfaceGate.java index 82232465c9..76fbbcb499 100644 --- a/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/RestSurfaceGate.java +++ b/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/RestSurfaceGate.java @@ -1,6 +1,9 @@ package com.metaobjects.generator.util; import com.metaobjects.MetaData; +import com.metaobjects.field.MetaField; +import com.metaobjects.generator.ModelWalk; +import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.identity.MetaIdentity; import com.metaobjects.object.MetaObject; import com.metaobjects.reporting.ReportReadModel; @@ -8,6 +11,11 @@ import com.metaobjects.source.MetaSource; import com.metaobjects.source.RdbSource; +import java.util.ArrayDeque; +import java.util.Deque; +import java.util.HashSet; +import java.util.Set; + /** * THE predicate deciding which objects get a generated REST surface, shared by every * JVM generator that emits one. @@ -208,4 +216,40 @@ public static boolean isSourcelessEntity(MetaObject obj) { } return identified && obj.getSources(true).isEmpty(); } + + /** + * The names of the generated classes that are a create or update request body: every entity + * with a generated write route ({@link #isWritable}, or a TPH subtype folded into a writable + * base), and every value object reachable from one through its {@code @objectRef} members, + * transitively. + * + *

Only these classes are marked to DROP an undeclared key. A stored-proc argument class, a + * response parser's decode target, a prompt payload or a read-only projection is not a request + * body, so it keeps the host mapper's own strictness: a model reply carrying an extra key must + * still be refused where the host refuses it.

+ */ + public static Set requestBodyShapes(MetaDataLoader loader) { + Set names = new HashSet<>(); + Deque pending = new ArrayDeque<>(); + for (MetaObject obj : GeneratorUtil.ownMetaObjects(loader)) { + if (isWritable(obj) || isFoldedIntoWritableBase(obj)) { + if (names.add(obj.getName())) pending.add(obj); + } + } + while (!pending.isEmpty()) { + for (MetaField field : pending.poll().getMetaFields()) { + MetaObject target = ModelWalk.objectRefTarget(field); + if (target != null && names.add(target.getName())) pending.add(target); + } + } + return names; + } + + private static boolean isFoldedIntoWritableBase(MetaObject obj) { + if (!obj.hasMetaAttr(MetaObject.ATTR_DISCRIMINATOR_VALUE, false)) return false; + for (MetaObject cursor = obj.getSuperObject(); cursor != null; cursor = cursor.getSuperObject()) { + if (isWritable(cursor)) return true; + } + return false; + } } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt index aa06c02a31..9edb8085c4 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt @@ -98,6 +98,7 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { durationWirePackages.clear() // ADR-0019 Amendment 2: the project's wire setting, read from the loaded root. wire = WireFormat.of(loader.root) + requestBodies = RestSurfaceGate.requestBodyShapes(loader) fr019Config = Fr019SharedEnum.ProvidedEnumConfig.fromArgs(args) // emitAbstractShapes (default OFF): when ON, an abstract entity is emitted as a Kotlin // `interface` shape (read-only properties) instead of being suppressed. It is NEVER @@ -149,6 +150,9 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { /** ADR-0019 Amendment 2: the project's wire setting (native unless `.metaobjects/config.json` says otherwise). */ private var wire: WireFormat = WireFormat.DEFAULT + /** Names of the classes bound from a write route's request body, for the current run. */ + private var requestBodies: Set = emptySet() + /** * #234: emit the per-package `MetaNetJson.kt` support object (the strict field.uri / field.inet * Jackson deserializers a data-class property binds through via `@field:JsonDeserialize`), ONCE @@ -240,13 +244,18 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { .addModifiers(KModifier.DATA) .addKdoc("GENERATED — do not hand-edit. Regenerated from metadata.\n") .apply { - // The class IS the create request body: an undeclared key is DROPPED, never rejected, - // whatever FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries. - addAnnotation( - AnnotationSpec.builder(ClassName("com.fasterxml.jackson.annotation", "JsonIgnoreProperties")) - .addMember("ignoreUnknown = true") - .build(), - ) + // A request body DROPS an undeclared key, never rejects it, whatever + // FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries. Only a write route's + // entity and the value objects nested in it are one; a stored-proc argument class, a + // response parser's decode target (ADR-0056), a prompt payload or a projection keeps + // the host mapper's strictness. + if (obj.name in requestBodies) { + addAnnotation( + AnnotationSpec.builder(ClassName("com.fasterxml.jackson.annotation", "JsonIgnoreProperties")) + .addMember("ignoreUnknown = true") + .build(), + ) + } // `wire.nullMembers: omit` (ADR-0019 Amendment 2): a value object's null members are left // out. Under native/emit Jackson writes them as null with the key present. if (obj.subType == MetaObject.SUBTYPE_VALUE && wire.omitNullMembers()) { diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 3860ae1ec7..d0838ce5c4 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -1232,9 +1232,9 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase { + strict.readValue("""{"text":"hi","zebra":1}""", cl.loadClass("acme.body.Reply")) + } + } finally { + out.toFile().deleteRecursively() + } + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt deleted file mode 100644 index 70fabd09f3..0000000000 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinServerOwnedInputTest.kt +++ /dev/null @@ -1,113 +0,0 @@ -package com.metaobjects.generator.kotlin - -import com.metaobjects.metadata.ktx.loadString -import java.nio.file.Files -import kotlin.test.Test -import kotlin.test.assertFalse -import kotlin.test.assertTrue - -/** - * A generated create route writes only what a client may set: a server-generated key, a `readOnly` - * column and (TPH) the discriminator and a sibling subtype's column are never bound from the body, - * and the request type ignores an undeclared key itself. Behaviour is gated by the api-contract - * `input-server-owned-dropped` scenarios; this pins the generated source. - */ -class KotlinServerOwnedInputTest { - - private val plain = """{ - "metadata.root": { "package": "acme::own", "children": [ - { "object.entity": { "name": "Doc", "children": [ - { "source.rdb": { "@table": "docs" } }, - { "field.long": { "name": "id" } }, - { "field.string": { "name": "title", "@maxLength": 100 } }, - { "field.string": { "name": "role", "@maxLength": 20, "@mutability": "readOnly" } }, - { "identity.primary": { "@fields": "id", "@generation": "increment" } } - ] } } - ] } - }""".trimIndent() - - private val tph = """{ - "metadata.root": { "package": "acme::own", "children": [ - { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ - { "source.rdb": { "@table": "auths" } }, - { "field.long": { "name": "id" } }, - { "field.enum": { "name": "type", "@values": ["Bridge", "Prior"] } }, - { "field.string": { "name": "reference", "@maxLength": 80 } }, - { "field.string": { "name": "grade", "@maxLength": 20, "@mutability": "readOnly" } }, - { "identity.primary": { "@fields": "id", "@generation": "increment" } } - ] } }, - { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", "children": [ - { "field.int": { "name": "quantity" } } - ] } }, - { "object.entity": { "name": "PriorAuth", "extends": "Auth", "@discriminatorValue": "Prior", "children": [ - { "field.string": { "name": "approver", "@maxLength": 80 } } - ] } } - ] } - }""".trimIndent() - - private fun gen(name: String, json: String): java.nio.file.Path { - val out = Files.createTempDirectory("kowned-") - val loader = loadString(name, json) - KotlinEntityGenerator().apply { setArgs(mapOf("outputDir" to out.toString())) }.execute(loader) - KotlinSpringControllerGenerator().apply { setArgs(mapOf("outputDir" to out.toString())) }.execute(loader) - return out - } - - @Test - fun `plain create writes neither the generated key nor a readOnly column and ignores unknown keys`() { - val out = gen("owned-plain", plain) - try { - val controller = Files.readString(out.resolve("acme/own/DocController.kt")) - val create = controller.substringAfter("fun create(").substringBefore("fun update(") - assertTrue("it[DocTable.title] = dto.title" in create, create) - assertFalse("it[DocTable.role]" in create, create) - assertFalse("it[DocTable.id] = dto.id" in create, create) - val entity = Files.readString(out.resolve("acme/own/Doc.kt")) - assertTrue("@JsonIgnoreProperties(ignoreUnknown = true)" in entity, entity) - } finally { - out.toFile().deleteRecursively() - } - } - - @Test - fun `create lists every absent required value, not only the first the bind meets`() { - val json = plain.replace( - "{ \"field.string\": { \"name\": \"title\", \"@maxLength\": 100 } },", - "{ \"field.string\": { \"name\": \"title\", \"@required\": true, \"@maxLength\": 100 } }," + - " { \"field.string\": { \"name\": \"subtitle\", \"@required\": true, \"@maxLength\": 100 } }," + - " { \"field.string\": { \"name\": \"kind\", \"@required\": true, \"@default\": \"x\", \"@maxLength\": 10 } },", - ) - val out = gen("owned-required", json) - try { - val controller = Files.readString(out.resolve("acme/own/DocController.kt")) - val create = controller.substringAfter("fun create(").substringBefore("fun update(") - for (f in listOf("title", "subtitle")) { - assertTrue("if (!body.has(\"$f\") || body.get(\"$f\").isNull) issues.add(\"invalid_type\", listOf(\"$f\")" in create, create) - } - // A required value with a default binds without being sent, so it is never reported missing. - assertFalse("if (!body.has(\"kind\")" in create, create) - } finally { - out.toFile().deleteRecursively() - } - } - - @Test - fun `TPH create writes only the route subtype's own settable columns`() { - val out = gen("owned-tph", tph) - try { - val controller = Files.readString(out.resolve("acme/own/AuthController.kt")) - val bridge = controller.substringAfter("fun createBridge(").substringBefore("@RequestMapping") - val prior = controller.substringAfter("fun createPrior(").substringBefore("@RequestMapping") - assertTrue("it[reference] = dto.reference" in bridge, bridge) - assertTrue("it[quantity] = dto.quantity" in bridge, bridge) - assertFalse("it[approver]" in bridge, bridge) - assertFalse("it[grade]" in bridge, bridge) - assertFalse("it[type] = dto.type" in bridge, bridge) - assertTrue("it[approver] = dto.approver" in prior, prior) - assertFalse("it[quantity]" in prior, prior) - assertFalse("it[grade]" in prior, prior) - } finally { - out.toFile().deleteRecursively() - } - } -} diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReport.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReport.kt index 2d1ded1ca7..db679129d1 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReport.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReport.kt @@ -1,6 +1,5 @@ package acme.ops -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.Size import kotlin.Long import kotlin.String @@ -8,7 +7,6 @@ import kotlin.String /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class OrderReport( @field:Size(max = 50) public val status: String? = null, diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReportArgs.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReportArgs.kt index 7f2b45f5d8..9de23dd481 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReportArgs.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-stored-proc/acme/ops/OrderReportArgs.kt @@ -1,13 +1,11 @@ package acme.ops -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.Long import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class OrderReportArgs( public val orderId: Long? = null, ) { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt index 432829dadc..32593e805e 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-tph/acme/auth/AuthController.kt @@ -435,9 +435,9 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat issues.addAt(validator.validateValue(BridgeAuthValidation::class.java, "reference", dto.reference), "reference") if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { - it[type] = AuthType.Bridge - it[reference] = dto.reference!! - it[quantity] = dto.quantity + it[AuthTable.type] = AuthType.Bridge + it[AuthTable.reference] = dto.reference!! + it[AuthTable.quantity] = dto.quantity }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuth(saved) as Any) @@ -533,9 +533,9 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat issues.addAt(validator.validateValue(CopayAuthValidation::class.java, "reference", dto.reference), "reference") if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { - it[type] = AuthType.Copay - it[reference] = dto.reference!! - it[copayAmount] = dto.copayAmount + it[AuthTable.type] = AuthType.Copay + it[AuthTable.reference] = dto.reference!! + it[AuthTable.copayAmount] = dto.copayAmount }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuth(saved) as Any) @@ -630,9 +630,9 @@ class AuthController(private val objectMapper: ObjectMapper, private val validat issues.addAt(validator.validateValue(PriorAuthAuthValidation::class.java, "reference", dto.reference), "reference") if (!issues.isEmpty()) return@transaction ResponseEntity.badRequest().body(issues.body() as Any) val newId = AuthTable.insert { - it[type] = AuthType.PriorAuth - it[reference] = dto.reference!! - it[priorAuthNumber] = dto.priorAuthNumber + it[AuthTable.type] = AuthType.PriorAuth + it[AuthTable.reference] = dto.reference!! + it[AuthTable.priorAuthNumber] = dto.priorAuthNumber }[AuthTable.id] val saved = AuthTable.selectAll().where { AuthTable.id eq newId }.single() ResponseEntity.status(HttpStatus.CREATED).body(rowToAuth(saved) as Any) diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/SalesReport.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/SalesReport.kt index 0d71adbe80..2a8e384ed7 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/SalesReport.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/entity-with-view/acme/report/SalesReport.kt @@ -1,6 +1,5 @@ package acme.report -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.Size import kotlin.Long import kotlin.String @@ -8,7 +7,6 @@ import kotlin.String /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class SalesReport( public val id: Long? = null, @field:Size(max = 100) diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Author.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Author.kt index c9754cf583..43416db7e1 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Author.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Author.kt @@ -1,6 +1,5 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.Long import kotlin.String import kotlin.jvm.JvmStatic @@ -8,7 +7,6 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class Author( public val id: Long? = null, public val name: String? = null, diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/AuthorReport.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/AuthorReport.kt index 4a654671b9..9943da2c70 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/AuthorReport.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/AuthorReport.kt @@ -1,13 +1,11 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.Long import kotlin.String /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class AuthorReport( public val name: String? = null, public val postCount: Long? = null, diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Post.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Post.kt index e85d80b2cc..3b5f05280d 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Post.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/payload-with-origins/acme/ai/Post.kt @@ -1,6 +1,5 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.Long import kotlin.String import kotlin.jvm.JvmStatic @@ -8,7 +7,6 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class Post( public val id: Long? = null, public val title: String? = null, diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-fr010/acme/ai/OpinionOutputPayload.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-fr010/acme/ai/OpinionOutputPayload.kt index f8294a620c..2d36df6d34 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-fr010/acme/ai/OpinionOutputPayload.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-fr010/acme/ai/OpinionOutputPayload.kt @@ -1,6 +1,5 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import jakarta.validation.constraints.NotNull import jakarta.validation.constraints.Size import kotlin.String @@ -9,7 +8,6 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class OpinionOutputPayload( @field:NotNull @field:Size(min = 1) diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-parser/acme/ai/Greeting.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-parser/acme/ai/Greeting.kt index 6586680e92..3e337a313b 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-parser/acme/ai/Greeting.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/template-output-parser/acme/ai/Greeting.kt @@ -1,13 +1,11 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.String import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class Greeting( public val text: String? = null, ) { diff --git a/server/java/codegen-kotlin/src/test/resources/snapshots/template-prompt-simple/acme/ai/Author.kt b/server/java/codegen-kotlin/src/test/resources/snapshots/template-prompt-simple/acme/ai/Author.kt index 79d6554d82..7bb4ef857a 100644 --- a/server/java/codegen-kotlin/src/test/resources/snapshots/template-prompt-simple/acme/ai/Author.kt +++ b/server/java/codegen-kotlin/src/test/resources/snapshots/template-prompt-simple/acme/ai/Author.kt @@ -1,6 +1,5 @@ package acme.ai -import com.fasterxml.jackson.`annotation`.JsonIgnoreProperties import kotlin.Long import kotlin.String import kotlin.jvm.JvmStatic @@ -8,7 +7,6 @@ import kotlin.jvm.JvmStatic /** * GENERATED — do not hand-edit. Regenerated from metadata. */ -@JsonIgnoreProperties(ignoreUnknown = true) public data class Author( public val id: Long? = null, public val name: String? = null, diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java index 9bb00a57aa..b0698847be 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringDtoGenerator.java @@ -48,6 +48,7 @@ import java.util.ArrayList; import java.util.Collection; import java.util.List; +import java.util.Set; import com.metaobjects.generator.util.GeneratedFileWriter; /** @@ -104,6 +105,9 @@ public class SpringDtoGenerator extends MultiFileDirectGeneratorBase /** The project's wire setting for the current run (ADR-0019 Amendment 2). */ private WireFormat wire = WireFormat.DEFAULT; + /** Names of the classes bound from a write route's request body, for the current run. */ + private Set requestBodies = Set.of(); + @Override protected Class getFilterClass() { return MetaObject.class; @@ -117,6 +121,7 @@ public void execute(MetaDataLoader loader) { // ADR-0019 Amendment 2: read from the loaded root, not from a shape being emitted - a // served report's shape is synthesized from its read model and carries no parent. wire = loader.getRoot().getWireFormat(); + requestBodies = RestSurfaceGate.requestBodyShapes(loader); Path outRoot = Paths.get(outDir.getAbsolutePath()); fr019Config = Fr019SharedEnum.ProvidedEnumConfig.of( getArg("providedEnumNamespace"), getArg("providedEnumPackages")); @@ -649,9 +654,12 @@ private void emitRecord(MetaObject entity, Path outRoot, List fields, } src.append("/** GENERATED — wire DTO for ").append(shortName) .append(". Do not hand-edit; regenerated from metadata. */\n"); - // The DTO is the create request body: an undeclared key is DROPPED, never rejected, whatever - // FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries (a plain ObjectMapper has it on). - src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); + // A request body DROPS an undeclared key, never rejects it, whatever FAIL_ON_UNKNOWN_PROPERTIES + // the host's ObjectMapper carries (a plain ObjectMapper has it on). Only a class bound from a + // write route's body is one; a projection or report row keeps the host mapper's strictness. + if (requestBodies.contains(entity.getName())) { + src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); + } src.append("public record ").append(recordName).append("(\n"); // Filter ObjectField — see class javadoc. Same reason as the Kotlin port's diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java index db1a9ff4b3..984b71f5fb 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringValueObjectGenerator.java @@ -16,6 +16,7 @@ import com.metaobjects.generator.GeneratorIOWriter; import com.metaobjects.generator.direct.MultiFileDirectGeneratorBase; import com.metaobjects.generator.util.GeneratorUtil; +import com.metaobjects.generator.util.RestSurfaceGate; import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.object.MetaObject; @@ -91,6 +92,9 @@ protected Class getFilterClass() { /** The project's wire setting for the current run (ADR-0019 Amendment 2). */ private WireFormat wire = WireFormat.DEFAULT; + /** Names of the classes bound from a write route's request body, for the current run. */ + private Set requestBodies = Set.of(); + @Override public void execute(MetaDataLoader loader) { parseArgs(); @@ -102,6 +106,7 @@ public void execute(MetaDataLoader loader) { if (Boolean.parseBoolean(getArg(Fr019SharedEnum.ARG_EMIT_SHARED_ENUMS, "false"))) { Fr019SharedEnum.emitMaterialized(loader, outRoot); } + requestBodies = RestSurfaceGate.requestBodyShapes(loader); for (MetaObject vo : valueShapes(loader)) { emit(vo, outRoot); } @@ -184,8 +189,11 @@ private void emit(MetaObject vo, Path outRoot) { if (omitNulls) src.append("@JsonInclude(JsonInclude.Include.NON_NULL)\n"); // A member the value object does not declare is DROPPED, never rejected, whatever // FAIL_ON_UNKNOWN_PROPERTIES the host's ObjectMapper carries: api.unknownFields judges a - // request body's top level only, so a nested undeclared key binds like the DTO's own. - src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); + // request body's top level only, so a nested undeclared key binds like the DTO's own. Only a + // value object nested in a write route's body is one; a payload or parser target is not. + if (requestBodies.contains(vo.getName())) { + src.append("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\n"); + } src.append("public record ").append(recordName).append("(\n"); List components = new ArrayList<>(fields.size()); for (int i = 0; i < fields.size(); i++) { diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringRequestBodyUnknownKeysCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringRequestBodyUnknownKeysCompileRunTest.java new file mode 100644 index 0000000000..9026ac75af --- /dev/null +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringRequestBodyUnknownKeysCompileRunTest.java @@ -0,0 +1,88 @@ +package com.metaobjects.generator.spring; + +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.ObjectClassRegistry; +import org.junit.After; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; + +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertThrows; + +/** + * A generated class DROPS an undeclared JSON key only where it is a create or update request body: + * an entity with a generated write route, and a value object nested in one. A response parser's + * decode target, a prompt payload and a read-only projection are not request bodies, so they keep + * the host mapper's strictness: a model reply carrying an extra key is still refused. + * + *

Compiles the generated sources and binds JSON through a Jackson mapper with + * {@code FAIL_ON_UNKNOWN_PROPERTIES} on, the setting a plain {@code ObjectMapper} ships with. + * Oracle: the rule above and Jackson's documented {@code @JsonIgnoreProperties} semantics. The + * wrong behavior rejected is a blanket annotation, which would let {@code Reply} bind a stray key.

+ */ +public class SpringRequestBodyUnknownKeysCompileRunTest { + + @Rule + public TemporaryFolder tmp = new TemporaryFolder(); + + @After + public void resetRegistry() { + ObjectClassRegistry.resetGlobal(); + } + + private static final String META = """ + { "metadata.root": { "package": "acme::body", "children": [ + { "object.value": { "name": "Stamp", "children": [ + { "field.string": { "name": "by", "@maxLength": 20 } }, + { "field.object": { "name": "where", "@objectRef": "Place" } } + ] } }, + { "object.value": { "name": "Place", "children": [ + { "field.string": { "name": "city", "@maxLength": 20 } } + ] } }, + { "object.value": { "name": "Reply", "children": [ + { "field.string": { "name": "text", "@maxLength": 100 } } + ] } }, + { "object.entity": { "name": "Doc", "children": [ + { "source.rdb": { "@table": "docs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@maxLength": 100 } }, + { "field.object": { "name": "stamp", "@objectRef": "Stamp", "@storage": "jsonb" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } } + ] } } + """; + + @Test + public void onlyAWriteRoutesEntityAndTheValueObjectsNestedInItDropAnUndeclaredKey() throws Exception { + MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder("ws").toPath(), "request-bodies", META); + Path gen = tmp.newFolder("gen").toPath(); + for (var g : List.of(new SpringDtoGenerator(), new SpringValueObjectGenerator())) { + g.setArgs(Map.of("outputDir", gen.toString())); + g.execute(loader); + } + Path classes = tmp.newFolder("classes").toPath(); + SpringTestFixtures.compileGenerated(gen, classes); + try (URLClassLoader cl = new URLClassLoader(new URL[]{ classes.toUri().toURL() }, getClass().getClassLoader())) { + ObjectMapper strict = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true); + + // The entity DTO and the value objects nested in it, one and two levels down: stray key dropped. + assertNotNull(strict.readValue( + "{\"title\":\"t\",\"isAdmin\":true,\"stamp\":{\"by\":\"b\",\"zebra\":1,\"where\":{\"city\":\"c\",\"zebra\":2}}}", + cl.loadClass("acme.body.DocDto"))); + + // A value object nothing writable nests is not a request body: the strict mapper refuses it. + assertThrows(UnrecognizedPropertyException.class, () -> strict.readValue( + "{\"text\":\"hi\",\"zebra\":1}", cl.loadClass("acme.body.Reply"))); + } + } +} diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java deleted file mode 100644 index 4bf4a599a6..0000000000 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringServerOwnedInputCodegenTest.java +++ /dev/null @@ -1,138 +0,0 @@ -package com.metaobjects.generator.spring; - -import com.metaobjects.loader.MetaDataLoader; -import com.metaobjects.registry.SharedRegistryTestBase; -import org.junit.Rule; -import org.junit.Test; -import org.junit.rules.TemporaryFolder; - -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.HashMap; -import java.util.Map; - -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; - -/** - * A generated create route hands the repository only what a client may set: a server-generated - * key, a {@code readOnly} field and (TPH) the discriminator and a sibling subtype's column are - * dropped from the bound body, and the request type ignores an undeclared key itself. - */ -public class SpringServerOwnedInputCodegenTest extends SharedRegistryTestBase { - - @Rule - public TemporaryFolder tmp = new TemporaryFolder(); - - private static final String PLAIN = """ - { - "metadata.root": { "package": "acme::own", "children": [ - { "object.entity": { "name": "Doc", "children": [ - { "source.rdb": { "@table": "docs" } }, - { "field.long": { "name": "id" } }, - { "field.string": { "name": "title" } }, - { "field.string": { "name": "role", "@mutability": "readOnly" } }, - { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } - ] } }, - { "object.entity": { "name": "Tag", "children": [ - { "source.rdb": { "@table": "tags" } }, - { "field.string": { "name": "code" } }, - { "field.string": { "name": "label" } }, - { "identity.primary": { "name": "pk", "@fields": ["code"] } } - ] } } - ] } - } - """; - - private static final String TPH = """ - { - "metadata.root": { "package": "acme::own", "children": [ - { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ - { "source.rdb": { "@table": "auths" } }, - { "field.long": { "name": "id" } }, - { "field.enum": { "name": "type", "@values": ["Bridge", "Prior"] } }, - { "field.string": { "name": "reference" } }, - { "field.string": { "name": "grade", "@mutability": "readOnly" } }, - { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } - ] } }, - { "object.entity": { "name": "BridgeAuth", "extends": "Auth", "@discriminatorValue": "Bridge", "children": [ - { "field.int": { "name": "quantity" } } - ] } }, - { "object.entity": { "name": "PriorAuth", "extends": "Auth", "@discriminatorValue": "Prior", "children": [ - { "field.string": { "name": "approver" } } - ] } } - ] } - } - """; - - private static final String VO = """ - { - "metadata.root": { "package": "acme::own", "children": [ - { "object.value": { "name": "Stamp", "children": [ - { "field.string": { "name": "by" } } - ] } }, - { "object.entity": { "name": "Note", "children": [ - { "source.rdb": { "@table": "notes" } }, - { "field.long": { "name": "id" } }, - { "field.string": { "name": "body" } }, - { "field.object": { "name": "stamp", "@objectRef": "Stamp", "@storage": "jsonb", - "@required": true, "@mutability": "readOnly" } }, - { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } - ] } } - ] } - } - """; - - private Path generate(String name, String fixture) throws Exception { - Path gen = tmp.newFolder("gen-" + name).toPath(); - MetaDataLoader loader = SpringTestFixtures.loadFixture(tmp.newFolder("ws-" + name).toPath(), name, fixture); - for (var g : new com.metaobjects.generator.direct.MultiFileDirectGeneratorBase[] { - new SpringControllerGenerator(), new SpringDtoGenerator() }) { - Map args = new HashMap<>(); - args.put("outputDir", gen.toString()); - g.setArgs(args); - g.execute(loader); - } - return gen; - } - - @Test - public void plainCreateDropsTheGeneratedKeyAndReadOnlyField() throws Exception { - Path gen = generate("plain", PLAIN); - String controller = Files.readString(gen.resolve("acme/own/DocController.java")); - assertTrue("generated id and readOnly role are nulled; saw:\n" + controller, - controller.contains("dto = new DocDto(null, dto.title(), null);")); - // an assigned key (no @generation) is the caller's on create - String tag = Files.readString(gen.resolve("acme/own/TagController.java")); - assertFalse("assigned key must be kept; saw:\n" + tag, tag.contains("dto = new TagDto(")); - } - - @Test - public void requestTypeIgnoresUnknownProperties() throws Exception { - Path gen = generate("unknown", PLAIN); - String dto = Files.readString(gen.resolve("acme/own/DocDto.java")); - assertTrue(dto.contains("@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true)\npublic record DocDto(")); - } - - @Test - public void aRequiredReadOnlyValueObjectIsNulledAndItsNotNullIgnored() throws Exception { - Path gen = generate("vo", VO); - String controller = Files.readString(gen.resolve("acme/own/NoteController.java")); - assertTrue("readOnly VO component is nulled on create; saw:\n" + controller, - controller.contains("dto = new NoteDto(null, dto.body(), null);")); - // the strip nulls `stamp`, so its @NotNull must not turn every POST into a 400 - assertTrue("stamp must be server-owned for create validation; saw:\n" + controller, - controller.matches("(?s).*SERVER_OWNED_ON_CREATE = Set\\.of\\([^)]*\"stamp\".*")); - } - - @Test - public void tphCreateDropsKeyDiscriminatorReadOnlyAndSiblingColumns() throws Exception { - Path gen = generate("tph", TPH); - String controller = Files.readString(gen.resolve("acme/own/AuthController.java")); - // union components: id, type, reference, grade, quantity, approver - assertTrue("Bridge route keeps reference and quantity only; saw:\n" + controller, - controller.contains("dto = new AuthDto(null, null, dto.reference(), null, dto.quantity(), null);")); - assertTrue("Prior route keeps reference and approver only; saw:\n" + controller, - controller.contains("dto = new AuthDto(null, null, dto.reference(), null, null, dto.approver());")); - } -} diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java index 2b42e8dc74..6b095b6f3a 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentNote.java @@ -3,7 +3,6 @@ import jakarta.validation.constraints.*; /** GENERATED — value object ShipmentNote. Do not hand-edit; regenerated from metadata. */ -@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ShipmentNote( @NotNull @Size(min = 1) String text, ShipmentNoteKind kind, diff --git a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java index a2459cdb36..bdfadca7f1 100644 --- a/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java +++ b/server/java/codegen-spring/src/test/resources/golden/inline-enum-names/expected/acme/shipping/ShipmentSummaryDto.java @@ -3,7 +3,6 @@ import jakarta.validation.constraints.*; /** GENERATED — wire DTO for ShipmentSummary. Do not hand-edit; regenerated from metadata. */ -@com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) public record ShipmentSummaryDto( @NotNull Long id, @NotNull Status status diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java index 002c1260a2..4973119c36 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataVerifyMojo.java @@ -190,6 +190,7 @@ public void execute() throws MojoExecutionException, MojoFailureException { warnIfAgentContextStale(); advisoriesDone = false; + unboundedListAdvisoryDone = false; String m = mode == null ? "" : mode.trim(); if (m.isEmpty()) { verifyDefaultSet(); @@ -261,6 +262,9 @@ private void verifyDeps() throws MojoFailureException { /** Set once the field lint and requirement gate have run, so a two-gate run reports them once. */ private boolean advisoriesDone; + /** Set once the pagination advisory has run, so a two-gate run reports it once. */ + private boolean unboundedListAdvisoryDone; + /** * Mode unset: the cross-port default gate set. Codegen drift always; template drift when * {@code templateRoot} is configured, else skipped with a note (there is no directory to @@ -303,8 +307,8 @@ private void verifyDefaultSet() throws MojoExecutionException, MojoFailureExcept * Warnings ONLY: this never throws and never changes the build result. */ private void runFieldLintAdvisory(MetaDataLoader loader) { - if (advisoriesDone || noFieldLint || "1".equals(System.getenv(FieldLint.ENV_OPT_OUT))) return; runUnboundedListAdvisory(loader); + if (advisoriesDone || noFieldLint || "1".equals(System.getenv(FieldLint.ENV_OPT_OUT))) return; List findings = new ArrayList<>(); try { findings.addAll(FieldLint.lintReferenceFields(loader)); @@ -326,6 +330,8 @@ private void runFieldLintAdvisory(MetaDataLoader loader) { * accept any {@code limit}. Never fails the build. */ private void runUnboundedListAdvisory(MetaDataLoader loader) { + if (unboundedListAdvisoryDone) return; + unboundedListAdvisoryDone = true; try { if (loader.getRoot().getApiSettings().hasPagination()) return; boolean generatesRoutes = false; diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java index 0a5cdffc15..68eb461da5 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/MetaDataVerifyUnboundedListAdvisoryTest.java @@ -50,7 +50,12 @@ private static Path project(String configJson) throws Exception { } private static CapturingLog verify(Path project, boolean generatesRoutes) throws Exception { + return verify(project, generatesRoutes, false); + } + + private static CapturingLog verify(Path project, boolean generatesRoutes, boolean noFieldLint) throws Exception { MetaDataVerifyMojo mojo = new MetaDataVerifyMojo(); + mojo.setNoFieldLint(noFieldLint); mojo.setLoader(LoaderParam.builder("verify-unbounded-test") .withClassname("com.metaobjects.loader.MetaDataLoader") .withSourceDir(project.toString()) @@ -89,4 +94,10 @@ public void aPageSizeSilencesIt() throws Exception { public void noControllerNoAdvisory() throws Exception { assertTrue(verify(project("{\"schema_version\": 1}"), false).warnings.isEmpty()); } + + @Test + public void turningTheFieldLintOffDoesNotSilenceIt() throws Exception { + CapturingLog log = verify(project("{\"schema_version\": 1}"), true, true); + assertEquals(log.warnings.toString(), 1, log.warnings.size()); + } } From 78d2800a00440bed564e9f872d02fb38c85ff2a3 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 21:59:05 -0500 Subject: [PATCH 17/31] no-mistakes(review): Drop discriminator on Python TPH update; fix scenario text --- .../tph/scenarios/tph-input-foreign-discriminator-dropped.yaml | 3 +-- .../src/metaobjects/codegen/generators/router_generator.py | 1 + server/python/tests/integration/generated_tph_app.py | 2 -- 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml b/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml index e4290f706d..c15da35b12 100644 --- a/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml +++ b/fixtures/api-contract-conformance/tph/scenarios/tph-input-foreign-discriminator-dropped.yaml @@ -3,8 +3,7 @@ description: > A per-subtype route takes the row's subtype from the URL, never from the body (the 1.2.x security fix). A `type` naming a different subtype, sent to the Bridge route, is DROPPED SILENTLY on create and on update: the created row is a Bridge, no Copay row appears, and an - updated Bridge stays a Bridge. Python's generated TPH routes answer this body with 400 rather - than dropping it; that runner skips this scenario (see the Python codegen KNOWN_GAPS.md). + updated Bridge stays a Bridge. Every port answers the same way. requests: - id: r1 method: POST diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index d8c070d878..e982d34ca2 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -1778,6 +1778,7 @@ def list_sig(fn: str, route: str) -> list[str]: parts.append(") -> Any:") parts.extend(prelude) parts.extend(denied_strip_tph) + parts.append(f' dto.pop("{plan.discriminator_field}", None)') parts.extend(_validated_input_lines( f"{sub}Patch", "FR-035 PATCH-2: an explicit null on a @required field is a 400.", diff --git a/server/python/tests/integration/generated_tph_app.py b/server/python/tests/integration/generated_tph_app.py index cecbac6e4f..4dc0c4396e 100644 --- a/server/python/tests/integration/generated_tph_app.py +++ b/server/python/tests/integration/generated_tph_app.py @@ -206,8 +206,6 @@ def update(self, subtype: str, id: int, dto: Any) -> Any | None: if row is None: return None for k, v in dto.items(): - if k == self._disc: - continue # discriminator is immutable row[k] = v return row From 2ff438bd3cd8a0184d6e214cf82f9e61d9bebebd Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:00:17 -0500 Subject: [PATCH 18/31] no-mistakes(review): apply reviewer inline fixes --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 944f8a8016..53d62d70c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,8 +61,8 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi write route's request body (and the value objects nested in one) now carry `@JsonIgnoreProperties(ignoreUnknown = true)`, so binding no longer depends on the host's `ObjectMapper` (a plain one answered 400). The Python 1.2.1 fix is carried forward to this - line, and the Python TPH create now drops a body's discriminator instead of answering 400 for - a foreign one. + line, and the Python TPH create and update now drop a body's discriminator instead of answering + 400 for a foreign one. Gated by three shared api-contract scenarios: `input-server-owned-dropped` (generated and reference lanes in every port), and `tph/tph-input-server-owned-dropped` and From 466d81a169da96db40d3c1a49d276c46afeb92b8 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:12:36 -0500 Subject: [PATCH 19/31] no-mistakes(review): Fix doc counts, dead guard, stale comments, vacuous assertion --- docs/CONFORMANCE.md | 2 +- .../advanced-modeling/codegen/templates/zod-validators.ts | 6 ++---- examples/showcase/codegen/templates/zod-validators.ts | 6 ++---- fixtures/api-contract-conformance/README.md | 1 - .../generator/spring/runtime/ValidationIssuesTest.java | 6 +----- .../api/generated/GeneratedAuthorControllerHarness.kt | 4 ++-- .../generated/GeneratedSecretFieldsControllerHarness.kt | 4 ++-- .../api/tph/generated/GeneratedTphControllerHarness.kt | 4 ++-- .../api/generated/GeneratedAccountControllerHarness.java | 5 +++-- .../api/generated/GeneratedAuthorControllerHarness.java | 4 ++-- .../api/tph/generated/GeneratedTphControllerHarness.java | 4 ++-- .../packages/codegen-ts/src/templates/zod-validators.ts | 6 ++---- 12 files changed, 21 insertions(+), 31 deletions(-) diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index d709c815e5..d838d6674d 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -457,7 +457,7 @@ and 2 error shapes (`invalid_filter_field` / `invalid_filter_op`) under the URL grammar `?filter[][]=` (FR-009). On top of the 34 core scenarios (among them the write-input rules: server-owned keys dropped, an undeclared key refused under the default `api.unknownFields: reject`, and every validation issue listed) -the corpus carries eleven sub-corpora — `tph/` (13, single-table +the corpus carries eleven sub-corpora — `tph/` (14, single-table inheritance; a subtype decimal filters and refuses extra digits, and the input rules hold over the whole hierarchy), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), diff --git a/examples/advanced-modeling/codegen/templates/zod-validators.ts b/examples/advanced-modeling/codegen/templates/zod-validators.ts index 6113dc011d..049c8f2d60 100644 --- a/examples/advanced-modeling/codegen/templates/zod-validators.ts +++ b/examples/advanced-modeling/codegen/templates/zod-validators.ts @@ -850,10 +850,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } diff --git a/examples/showcase/codegen/templates/zod-validators.ts b/examples/showcase/codegen/templates/zod-validators.ts index 6113dc011d..049c8f2d60 100644 --- a/examples/showcase/codegen/templates/zod-validators.ts +++ b/examples/showcase/codegen/templates/zod-validators.ts @@ -850,10 +850,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 5e46840049..3630771911 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -38,7 +38,6 @@ fixtures/api-contract-conformance/ ├── patch-empty-noop.yaml # FR-037 PATCH-5 a patch stripping to zero assignments is a no-op read-back ├── writeonce-patch-stripped.yaml # FR-037 R1 @mutability:"writeOnce" — stripped on PATCH, never rejected ├── autoset-patch.yaml # #203 @autoSet — PATCH bumps onUpdate, leaves onCreate alone - ├── input-server-owned-dropped.yaml # 1.2.x security: a client id / undeclared key never reaches storage ├── delete-204-and-404.yaml ├── invalid-sort-400.yaml ├── filter-eq.yaml # FR-009 filter operators diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java index c8e2b0ba70..342a2ffa3d 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/runtime/ValidationIssuesTest.java @@ -1,6 +1,5 @@ package com.metaobjects.generator.spring.runtime; -import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import jakarta.validation.Validation; import jakarta.validation.Validator; @@ -97,7 +96,6 @@ public void aNestedMemberPathCarriesTheIndex() { @Test public void addAtUsesTheWireNameGiven() { - Bean b = new Bean(); ValidationIssues issues = new ValidationIssues() .addAt(VALIDATOR.validateValue(Bean.class, "required", null), "wire_name"); @SuppressWarnings("unchecked") @@ -106,8 +104,7 @@ public void addAtUsesTheWireNameGiven() { } @Test - public void aPatchBindFailureListsEveryProblemAndKeepsTheBoundValues() throws Exception { - JsonNode body = new ObjectMapper().readTree("{\"a\": null, \"b\": 7, \"c\": 2}"); + public void aPatchBindFailureListsEveryProblemAndKeepsTheBoundValues() { List problems = PatchValidationException.newProblems(); Map assigned = new java.util.LinkedHashMap<>(); PatchValidationException.collect(problems, () -> { throw new PatchValidationException("a"); }); @@ -124,7 +121,6 @@ public void aPatchBindFailureListsEveryProblemAndKeepsTheBoundValues() throws Ex assertEquals(List.of("a"), list.get(0).get("path")); assertEquals("too_big", list.get(1).get("code")); assertEquals(Map.of("c", 2), e.assignedValues()); - assertTrue(body.has("a")); } @Test diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt index 60e0b787bf..fa95d92700 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/GeneratedAuthorControllerHarness.kt @@ -85,8 +85,8 @@ class GeneratedAuthorControllerHarness( */ private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated - // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .registerModule( diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt index 1904580b41..f0751814ce 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/secretfields/generated/GeneratedSecretFieldsControllerHarness.kt @@ -38,8 +38,8 @@ class GeneratedSecretFieldsControllerHarness( private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated - // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt index 583e827ff5..1dff027311 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/tph/generated/GeneratedTphControllerHarness.kt @@ -56,8 +56,8 @@ class GeneratedTphControllerHarness( private val mapper: ObjectMapper = ObjectMapper() .registerKotlinModule() - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated - // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .registerModule(JavaTimeModule()) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java index 4cea941ec2..1da621631e 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java @@ -61,8 +61,9 @@ public final class GeneratedAccountControllerHarness implements AutoCloseable { ] } } """; - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated controller - // leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. private final ObjectMapper mapper = new ObjectMapper() .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); private final URLClassLoader classLoader; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java index 3c2e6627bc..683f3d730a 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAuthorControllerHarness.java @@ -131,8 +131,8 @@ public Instant deserialize(JsonParser p, DeserializationContext ctx) .registerModule(new JavaTimeModule()) .registerModule(utcInstant) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated - // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); Path srcDir = genDir.resolve("src"); diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java index 22ad1739cd..9ec45f7154 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/tph/generated/GeneratedTphControllerHarness.java @@ -116,8 +116,8 @@ public Instant deserialize(JsonParser p, DeserializationContext ctx) .registerModule(new JavaTimeModule()) .registerModule(utcInstant) .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) - // Spring Boot's default: an undeclared request key is ignored, not a 400. The generated - // controller leaves this to the host's mapper; the corpus pins the Boot behaviour. + // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: + // it reads body keys off the raw JSON and the DTOs ignore unknown properties. .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); Path srcDir = genDir.resolve("src"); Path classesDir = genDir.resolve("classes"); diff --git a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts index ad5bda227e..99bb37da06 100644 --- a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts +++ b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts @@ -850,10 +850,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } From 872bb7ff1a708d6b7fb7f8e94eb14b3d9bd93806 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:13:03 -0500 Subject: [PATCH 20/31] no-mistakes(review): apply reviewer inline fixes --- .../api/generated/GeneratedAccountControllerHarness.java | 1 - 1 file changed, 1 deletion(-) diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java index 1da621631e..ca1991e87e 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/GeneratedAccountControllerHarness.java @@ -62,7 +62,6 @@ public final class GeneratedAccountControllerHarness implements AutoCloseable { """; // Mirrors Spring Boot's default mapper. The generated request handling does not depend on it: - // it reads body keys off the raw JSON and the DTOs ignore unknown properties. private final ObjectMapper mapper = new ObjectMapper() .disable(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); From e9f7957e1095582d87e720552db0210c34fdbaf4 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:21:23 -0500 Subject: [PATCH 21/31] no-mistakes(review): Fix stale routes comment in C# api-settings factory --- .../Api/ApiSettingsGeneratedServerFactory.cs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs index 0eeb51440a..7df40f92ce 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiSettingsGeneratedServerFactory.cs @@ -50,8 +50,8 @@ public static async Task StartAsync(PostgresC var app = builder.Build(); - // Mount each generated MapRoutes(app, "/api"). Both Customer and Order - // emit a routes class; only Order's api-settings routes are exercised. + // Mount each emitted MapRoutes(app, "/api"); the api-settings corpus + // declares a single entity, Item. foreach (var entity in entityNames) { var routesType = assembly.GetType($"{GeneratedNamespace}.{entity}Routes"); From eee6e50c1a46c21cf662a0827be931b29548af60 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:34:33 -0500 Subject: [PATCH 22/31] no-mistakes(review): Fix stale eject docs, vacuous C# check, assertion messages --- docs/features/own-your-codegen.md | 6 +++--- .../Api/ApiContractApiSettingsConformanceTest.cs | 12 ------------ .../generator/spring/SpringMapFieldCodegenTest.java | 6 +++--- 3 files changed, 6 insertions(+), 18 deletions(-) diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 8a94d88f0e..e41785d976 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -1412,11 +1412,11 @@ on are yours too. Eject copies two kinds of source with a Java generator: | Ejecting | Shared emit rules, beside the generator | Helper runtime its output imports | |---|---|---| -| `routes` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDuration`, `ConstraintErrors`, `FilterParseResult`, `FilterParser`, `FilterPredicate`, `PatchValidationException`, `RecordComponentNames`, `ReportExport`, `UnreadableBody`, `ValidationIssues` | +| `routes` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDuration`, `ConstraintErrors`, `FilterParseResult`, `FilterParser`, `FilterPredicate`, `PatchValidationException`, `RecordComponentNames`, `ReportExport`, `UnknownBodyKeys`, `UnreadableBody`, `ValidationIssues` | | `dto` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `CanonicalDuration`, `CanonicalDurationDeserializer`, `CanonicalDurationSerializer`, `DeclaredDecimal`, `PatchValidationException` | | `repository` | `SpringGeneratorSupport` | `FilterPredicate` | | `value-object` | `SpringGeneratorSupport` | `CanonicalDecimal`, `CanonicalDecimalDeserializer`, `CanonicalDecimalSerializer`, `CanonicalDuration`, `CanonicalDurationDeserializer`, `CanonicalDurationSerializer`, `DeclaredDecimal` | -| any other Java generator, every Kotlin generator | nothing | nothing: its output imports only core (Kotlin writes its decimal wire and its validation-issue list into the output, as `MetaDecimalWire.kt` and `MetaValidationIssues.kt`) | +| any other Java generator, every Kotlin generator | nothing | nothing: its output imports only core (Kotlin writes its decimal wire and its validation-issue list into the output, as `MetaDecimalWire.kt` and `MetaValidationIssues.kt`, and its unknown-body-key check as `MetaUnknownBodyKeys.kt`) | **The shared emit rules.** `SpringGeneratorSupport` holds the rules those four generators apply together: which fields a DTO and a patch carry, which of them are stamped for @@ -1656,7 +1656,7 @@ Otherwise a bug in the filter parser would still mean waiting for a MetaObjects `dotnet meta eject routes` therefore also writes their source to `codegen/runtime/`: `FilterParser.cs`, `FilterParseResult.cs`, `FilterPredicate.cs`, -`EfCoreFilterDispatch.cs`, `ValueObjectValidator.cs`, `ValidationIssues.cs`, `ConstraintErrors.cs` and +`EfCoreFilterDispatch.cs`, `ValueObjectValidator.cs`, `ValidationIssues.cs`, `UnknownFieldCheck.cs`, `ConstraintErrors.cs` and `Iso8601TimestampConverter.cs`, the converter your host registers for the api contract's timestamp spelling. Each file is the packaged source with one change: its namespace becomes `Codegen.Runtime`, matching the folder the way `codegen/generators/` matches diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs index 9fdccad534..2530126930 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs @@ -23,18 +23,6 @@ public async Task Api_contract_api_settings_generated(string scenarioPath) await using var server = await ApiSettingsGeneratedServerFactory.StartAsync(pg); await server.ApplySeedAsync(); await RunAsync(scenario, server.BaseUrl); - await AssertNoUndeclaredStateAsync(pg.ConnectionString); - } - - // The undeclared key a scenario sends is never stored: the table has no column for it, and no - // scenario row ends up without its title. - private static async Task AssertNoUndeclaredStateAsync(string connString) - { - await using var c = new Npgsql.NpgsqlConnection(connString); - await c.OpenAsync(); - await using var cmd = c.CreateCommand(); - cmd.CommandText = "SELECT COUNT(*) FROM \"items\" WHERE \"title\" IS NULL"; - Assert.Equal(0L, (long)(await cmd.ExecuteScalarAsync())!); } private static async Task RunAsync(ApiScenario scenario, string baseUrl) diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java index 8bd54ec559..e5ffc8f7a5 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringMapFieldCodegenTest.java @@ -193,7 +193,7 @@ public void patchValidatesTheValuesOfAValueObjectMap() throws Exception { // __el, not a map-specific name: the array-of-VO and map-of-VO branches share one // emitter, which is what keeps the 400 envelope in a single place. - assertTrue("expected PATCH to iterate the map's VALUES; saw:\n" + controller, + assertTrue("expected PATCH to validate the map's values from the presence-guarded __assigned entry; saw:\n" + controller, controller.contains("__assigned.get(\"addresses\") instanceof Map __m")); // A scalar-valued map has no nested bean, so it must NOT get a validation loop. assertFalse("a scalar-valued map needs no nested validation; saw:\n" + controller, @@ -421,10 +421,10 @@ public void tphWritePathsValidateTheValuesOfAValueObjectMap() throws Exception { String controller = Files.readString(gen.resolve("acme/auth/AuthController.java")); // CREATE binds the union Dto — plain accessors, so the guard is a null check. - assertTrue("expected the TPH create to iterate the map's VALUES; saw:\n" + controller, + assertTrue("expected the TPH create to iterate the map's entrySet; saw:\n" + controller, controller.contains("for (var __en : dto.addresses().entrySet())")); // PATCH binds the Patch tristate, so it reuses the shared presence-guarded loop. - assertTrue("expected the TPH PATCH to iterate the map's VALUES; saw:\n" + controller, + assertTrue("expected the TPH PATCH to validate the map's values from the presence-guarded __assigned entry; saw:\n" + controller, controller.contains("__assigned.get(\"addresses\") instanceof Map __m")); // The control: a scalar-valued map has no nested bean and must get NO loop on either From e584b1b868551f18f9208f8241626bfaafa263ac Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:55:52 -0500 Subject: [PATCH 23/31] no-mistakes(review): Refuse non-numeric C# page bounds with 400 --- CHANGELOG.md | 3 ++- docs/features/api-contract.md | 7 +++++-- .../Generators/RoutesGenerator.cs | 13 +++++++++--- .../ApiContractApiSettingsConformanceTest.cs | 21 +++++++++++++++++++ 4 files changed, 38 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 53d62d70c0..d6718645df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,7 +85,8 @@ loaded before stops loading. This vocabulary ships in 1.3.0 with `metamodelVersi the first `defaultLimit` rows, while an explicit `limit` above `maxLimit` answers `400 {"error": "limit_too_large", "maxLimit": N}` (refused, not clamped); a negative `limit` answers `400 pagination.invalid_value` in every port rather than reaching a store that reads - `LIMIT -1` as no limit. `meta init` writes + `LIMIT -1` as no limit, and the generated C# routes answer the same envelope for a `limit` or + `offset` that is not a whole number (`limit=abc`) rather than serving the default page. `meta init` writes that block for a new project. **A project without the setting behaves exactly as before**, because a cap added on upgrade would silently truncate a client that reads a whole list; each port's verify advises (rule `unbounded-list-pages`, never a failure) when a routes diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index ea1fdf6ccf..0cddbe6ee5 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -286,8 +286,11 @@ each handler they register instead; an `HTTPException` is rethrown to your a client that asked for more than the server serves is told so rather than handed a short page it cannot tell from the end of the list. A negative `limit` answers the malformed-bound envelope above (`pagination.invalid_value`) in every port, so it never reaches a store that - reads `LIMIT -1` as no limit (SQLite). `limit=0` is a page of zero rows, not "no limit" - (`api-settings/limit-zero`). `offset` and `withCount=1` page past the default as + reads `LIMIT -1` as no limit (SQLite). The generated C# routes also answer that envelope for a + `limit` or `offset` that is present but not a whole number (`limit=abc`, `offset=x`) instead of + serving the default page; TypeScript does the same, Java and Kotlin answer Spring's type-mismatch + 400 and Python FastAPI's 422, so the corpus pins only the negative limit. + `limit=0` is a page of zero rows, not "no limit" (`api-settings/limit-zero`). `offset` and `withCount=1` page past the default as usual, with `total` counting every row. `meta init` writes the block above for a new project; both sizes must be positive whole numbers (`50.0` reads as `50`) no larger than 2147483647, and `defaultLimit` may not exceed `maxLimit`; a block that breaks a rule is refused when the config diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index f720c6fbbb..6b0cebdb1a 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -1065,6 +1065,13 @@ private static void AppendUnknownFieldsCheck(StringBuilder sb, MetaRoot root, st // were; with it, no limit takes defaultLimit and one above maxLimit is 400 limit_too_large. private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, string indent, string offRaw, string off, string limRaw, string lim) { + if (page is not null) + { + // A bound that is present but not a whole number is refused, as the TypeScript routes answer, + // never read as "absent" and answered with a page the client did not ask for. + sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}Bad) && !long.TryParse({offRaw}Bad, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out _))"); + sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"offset\", value = {offRaw}Bad.ToString(), expected = \"a non-negative integer (0 or more)\" }});"); + } sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}) && int.TryParse({offRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {off}) && {off} > 0)"); sb.AppendLine($"{indent} q = q.Skip({off});"); if (page is null) @@ -1074,9 +1081,9 @@ private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, str return; } sb.AppendLine($"{indent}// api.pagination: no limit takes defaultLimit; a limit above maxLimit is refused."); - // A negative limit is refused with the malformed-bound envelope, as the TypeScript routes answer. - sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}Neg) && long.TryParse({limRaw}Neg, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}Neg) && {lim}Neg < 0)"); - sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"limit\", value = {lim}Neg.ToString(System.Globalization.CultureInfo.InvariantCulture), expected = \"a non-negative integer (0 or more)\" }});"); + // A negative or non-numeric limit is refused with the malformed-bound envelope, as the TypeScript routes answer. + sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}Neg) && !(long.TryParse({limRaw}Neg, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}Neg) && {lim}Neg >= 0))"); + sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"limit\", value = {limRaw}Neg.ToString(), expected = \"a non-negative integer (0 or more)\" }});"); sb.AppendLine($"{indent}var __take = {page.DefaultLimit};"); // `limit=0` is a page of zero rows, as every other port serves it, not "no limit". sb.AppendLine($"{indent}if (qs.TryGetValue(\"limit\", out var {limRaw}) && long.TryParse({limRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {lim}) && {lim} >= 0)"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs index 2530126930..b9d7d07c63 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs @@ -25,6 +25,27 @@ public async Task Api_contract_api_settings_generated(string scenarioPath) await RunAsync(scenario, server.BaseUrl); } + // The corpus cannot pin this: Java/Kotlin answer Spring's type-mismatch 400 and Python FastAPI's 422 + // for a non-numeric bound, so each C# route is exercised here against the cross-port 400 envelope. + [Theory] + [InlineData("/api/items?limit=abc", "limit", "abc")] + [InlineData("/api/items?limit=1O", "limit", "1O")] + [InlineData("/api/items?offset=x", "offset", "x")] + public async Task Non_numeric_page_bound_is_refused_not_served_as_default_page(string path, string param, string value) + { + await using var pg = await PostgresContainer.StartAsync(); + await using var server = await ApiSettingsGeneratedServerFactory.StartAsync(pg); + await server.ApplySeedAsync(); + using var client = new HttpClient { BaseAddress = new Uri(server.BaseUrl) }; + var response = await client.GetAsync(ApiContractWire.VerbatimUri(client, path)); + Assert.Equal(400, (int)response.StatusCode); + var body = JsonNode.Parse(await response.Content.ReadAsStringAsync())!.AsObject(); + Assert.Equal("pagination.invalid_value", (string?)body["error"]); + Assert.Equal(param, (string?)body["param"]); + Assert.Equal(value, (string?)body["value"]); + Assert.Equal("a non-negative integer (0 or more)", (string?)body["expected"]); + } + private static async Task RunAsync(ApiScenario scenario, string baseUrl) { using var client = new HttpClient { BaseAddress = new Uri(baseUrl) }; From d0da11144b75f6d28baede4d36de7a2c113568b8 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 22:58:24 -0500 Subject: [PATCH 24/31] no-mistakes(review): Refuse out-of-range or negative C# offset --- .../MetaObjects.Codegen/Generators/RoutesGenerator.cs | 4 ++-- .../Api/ApiContractApiSettingsConformanceTest.cs | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 6b0cebdb1a..ec25a5f72c 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -1067,9 +1067,9 @@ private static void AppendOffsetLimit(StringBuilder sb, ApiPagination? page, str { if (page is not null) { - // A bound that is present but not a whole number is refused, as the TypeScript routes answer, + // An offset that is present but not a non-negative int is refused, as the TypeScript routes answer, // never read as "absent" and answered with a page the client did not ask for. - sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}Bad) && !long.TryParse({offRaw}Bad, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out _))"); + sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}Bad) && !(int.TryParse({offRaw}Bad, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {off}Bad) && {off}Bad >= 0))"); sb.AppendLine($"{indent} return Results.BadRequest(new {{ error = \"pagination.invalid_value\", param = \"offset\", value = {offRaw}Bad.ToString(), expected = \"a non-negative integer (0 or more)\" }});"); } sb.AppendLine($"{indent}if (qs.TryGetValue(\"offset\", out var {offRaw}) && int.TryParse({offRaw}, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var {off}) && {off} > 0)"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs index b9d7d07c63..2993de4f62 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractApiSettingsConformanceTest.cs @@ -26,11 +26,15 @@ public async Task Api_contract_api_settings_generated(string scenarioPath) } // The corpus cannot pin this: Java/Kotlin answer Spring's type-mismatch 400 and Python FastAPI's 422 - // for a non-numeric bound, so each C# route is exercised here against the cross-port 400 envelope. + // for a non-numeric bound, so the generated C# route is exercised here against the cross-port 400 envelope. + // Only the plain entity list is driven: the TPH base and per-subtype lists emit through the same + // AppendOffsetLimit, and no TPH fixture with api.pagination boots in this lane. [Theory] [InlineData("/api/items?limit=abc", "limit", "abc")] [InlineData("/api/items?limit=1O", "limit", "1O")] [InlineData("/api/items?offset=x", "offset", "x")] + [InlineData("/api/items?offset=3000000000", "offset", "3000000000")] + [InlineData("/api/items?offset=-5", "offset", "-5")] public async Task Non_numeric_page_bound_is_refused_not_served_as_default_page(string path, string param, string value) { await using var pg = await PostgresContainer.StartAsync(); From e55bd61e7234cc3e58972f2f9f7b8496f965b1b2 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 23:07:07 -0500 Subject: [PATCH 25/31] no-mistakes(review): Drop source-grepping Python router tests --- server/python/tests/codegen/test_cli_init.py | 1 - .../python/tests/codegen/test_router_generator.py | 15 --------------- 2 files changed, 16 deletions(-) diff --git a/server/python/tests/codegen/test_cli_init.py b/server/python/tests/codegen/test_cli_init.py index 964bda0555..2bc65d868b 100644 --- a/server/python/tests/codegen/test_cli_init.py +++ b/server/python/tests/codegen/test_cli_init.py @@ -99,7 +99,6 @@ def test_init_writes_the_new_project_api_block_that_typescript_meta_init_writes( cfg.write_text(cfg.read_text().replace( "generators: []", "generators: [entity, filter-allowlist, routes]")) assert main(["gen", "--config", str(cfg)]) == 0 - assert "limit_too_large" in (tmp_path / "generated" / "app" / "author_router.py").read_text() def test_init_leaves_an_existing_neutral_config_alone(tmp_path: Path, capsys) -> None: diff --git a/server/python/tests/codegen/test_router_generator.py b/server/python/tests/codegen/test_router_generator.py index 6679e6e1f0..e57aaa8deb 100644 --- a/server/python/tests/codegen/test_router_generator.py +++ b/server/python/tests/codegen/test_router_generator.py @@ -509,21 +509,6 @@ def test_every_write_verb_maps_constraint_violations() -> None: assert " return JSONResponse(status_code=failure.status, content=failure.body())" in out -def test_every_validation_failure_lists_its_issues() -> None: - """Every ``400`` the write routes answer is the issues envelope, never a bare - ``{"error": "validation"}``; PATCH lists the @required nulls beside the model's issues.""" - out = _require(_author()) - - assert '{"error": "validation"}' not in out - # create, update, the unreadable body (create + update) and the decimal coercion (create + update) - assert out.count("validation_envelope(unreadable_body_issues())") == 2 - assert out.count("validation_envelope(value_error_issues(_exc))") == 2 - assert out.count("validation_envelope(pydantic_issues(_exc.errors()))") == 1 # create - assert " _issues = required_null_issues(dto, _REQUIRED_FIELDS)" in out # update - assert " _issues += pydantic_issues(_exc.errors())" in out - assert "validation_envelope(_issues)" in out - - def test_generated_router_is_valid_python() -> None: """The emitted try/except must not leave a dangling block. From 14c8c0e6716f14359c6c03ab83a9c73076054448 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 13 Nov 2026 23:07:22 -0500 Subject: [PATCH 26/31] no-mistakes(review): apply reviewer inline fixes --- server/python/tests/codegen/test_cli_init.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/python/tests/codegen/test_cli_init.py b/server/python/tests/codegen/test_cli_init.py index 2bc65d868b..438dd50a91 100644 --- a/server/python/tests/codegen/test_cli_init.py +++ b/server/python/tests/codegen/test_cli_init.py @@ -87,7 +87,7 @@ def test_init_writes_the_new_project_api_block_that_typescript_meta_init_writes( assert raw["api"]["unknownFields"] == "reject" assert raw["migrate"] == {"foreignKeyIndexes": True} assert raw["verify"] == {"handEdits": "fail"} - # The page size reaches a generated router. + # The scaffolded neutral config, with the page size set, generates a router without error. (tmp_path / "metaobjects" / "meta.app.yaml").write_text( "metadata:\n package: app\n children:\n" " - object.entity:\n name: Author\n children:\n" From 62faf02047c75cc577229cc67f039ff6dd1ee621 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 14 Nov 2026 01:38:07 -0500 Subject: [PATCH 27/31] test(ts): re-sync the owned zod-validators copy after the pkFields reference change The input-handling change edited the codegen-ts reference source but not the owned copy under test-generators, so cli/test/owned-copies-current.test.ts (the 'owned copies are what meta eject writes' gate) failed: the copy on disk kept the pre-change update-shape text. bun scripts/sync-owned-template-copies.ts re-synced 1 of 128 owned copies; no other copy drifted. --- .../test-generators/src/templates/zod-validators.ts | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/server/typescript/packages/test-generators/src/templates/zod-validators.ts b/server/typescript/packages/test-generators/src/templates/zod-validators.ts index 6113dc011d..049c8f2d60 100644 --- a/server/typescript/packages/test-generators/src/templates/zod-validators.ts +++ b/server/typescript/packages/test-generators/src/templates/zod-validators.ts @@ -850,10 +850,8 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code // null — a present null CLEARS the column (`.set({field: null})` writes NULL). // Keyed on required-ness, NOT fieldWillBeOptional: a required-with-@default // field stays non-nullable so an explicit null on it is a 400 (validation), - // never a silent clear. PK fields are excluded: their Drizzle column is - // always not-null, so a nullable Zod type would fail `.set()`'s typecheck - // (and NOT NULL at runtime) — see primaryKeyFieldNames. - if (!isRequired(child) && !pkFields.has(child.name)) expr = code`${expr}.nullable()`; + // never a silent clear. + if (!isRequired(child)) expr = code`${expr}.nullable()`; updateFieldLines.push(code` ${child.name}: ${expr}`); } } From b74d34701ae46eeb6be7bf1b5b75def3bba9f90c Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 14 Nov 2026 01:42:45 -0500 Subject: [PATCH 28/31] no-mistakes(test): Validate re-synced zod-validators owned copy --- .../test/tmp-duration-sqlite-lba8VZ/Hold.ts | 331 ++++++++++++++++++ .../tmp-duration-sqlite-lba8VZ/HoldView.ts | 191 ++++++++++ .../test/tmp-duration-sqlite-lba8VZ/Timer.ts | 310 ++++++++++++++++ 3 files changed, 832 insertions(+) create mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts create mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts create mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts new file mode 100644 index 0000000000..5f33fd04a4 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts @@ -0,0 +1,331 @@ +// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. +// Source metadata: Hold (Hold) +// Extend in your own module (e.g. Hold.extra.ts) — nothing imports it for you. +import type { InferInsertModel, InferSelectModel } from "drizzle-orm"; +import { + customType, + integer, + sqliteTable, + text, +} from "drizzle-orm/sqlite-core"; +import { z } from "zod"; + +/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); + * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month + * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ +const durationColumn = (() => { + const ZERO = BigInt(0); + const SECOND = BigInt(1000000); + const MINUTE = BigInt(60000000); + const HOUR = BigInt(3600000000); + const DAY = BigInt(86400000000); + const WEEK = BigInt(604800000000); + const INT64_MAX = BigInt("9223372036854775807"); + const parseIso = (text: string): bigint | undefined => { + const p = + /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( + text, + ); + if (p === null) { + return undefined; + } + const mag = + BigInt(p[2] ?? 0) * WEEK + + BigInt(p[3] ?? 0) * DAY + + BigInt(p[4] ?? 0) * HOUR + + BigInt(p[5] ?? 0) * MINUTE + + BigInt(p[6] ?? 0) * SECOND + + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); + return p[1] === "-" ? -mag : mag; + }; + const isoText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + const s = rest / SECOND; + const f = rest % SECOND; + let out = "PT"; + if (h > ZERO) { + out += String(h) + "H"; + } + if (m > ZERO) { + out += String(m) + "M"; + } + if (s > ZERO || f > ZERO) { + out += + String(s) + + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + + "S"; + } + if (out === "PT") { + return "PT0S"; + } + return neg ? "-" + out : out; + }; + const pgText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const two = (n: bigint): string => String(n).padStart(2, "0"); + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); + const f = rest % SECOND; + if (f > ZERO) { + out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); + } + return neg ? "-" + out : out; + }; + const stored = (micros: bigint): number | bigint => { + if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { + throw new Error( + "the span " + + isoText(micros) + + " does not fit the 64-bit microsecond column a field.duration is stored in", + ); + } + const n = Number(micros); + return Number.isSafeInteger(n) ? n : micros; + }; + const notFixed = (v: unknown): Error => + new Error( + "the interval " + + JSON.stringify(v) + + " has a year or month part, so it is not a fixed-length field.duration", + ); + const read = (v: unknown): bigint => { + if (typeof v === "bigint") { + return v; + } + if (typeof v === "number" && Number.isInteger(v)) { + return BigInt(v); + } + if (typeof v === "string") { + if (/^-?\d+$/.test(v)) { + return BigInt(v); + } + const iso = parseIso(v); + if (iso !== undefined) { + return iso; + } + const p = + v.length === 0 + ? null + : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( + v, + ); + if (p === null) { + throw new Error("not a field.duration value: " + JSON.stringify(v)); + } + if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { + throw notFixed(v); + } + let time = ZERO; + if (p[5] !== undefined) { + time = + BigInt(p[5]) * HOUR + + BigInt(p[6] ?? 0) * MINUTE + + BigInt(p[7] ?? 0) * SECOND + + BigInt((p[8] ?? "").padEnd(6, "0")); + if (p[4] === "-") { + time = -time; + } + } + return BigInt(p[3] ?? 0) * DAY + time; + } + if (v !== null && typeof v === "object") { + const o = v as Record; + if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { + throw notFixed(v); + } + const whole = (n: number | undefined): bigint => + BigInt(Math.trunc(n ?? 0)); + // postgres-interval carries microseconds as a fractional millisecond count. + return ( + whole(o.days) * DAY + + whole(o.hours) * HOUR + + whole(o.minutes) * MINUTE + + whole(o.seconds) * SECOND + + BigInt(Math.round((o.milliseconds ?? 0) * 1000)) + ); + } + throw new Error("not a field.duration value: " + JSON.stringify(v)); + }; + return customType<{ data: string; driverData: number | bigint | string }>({ + dataType: () => "integer", + toDriver: (value) => stored(read(value)), + fromDriver: (value) => isoText(read(value)), + }); +})(); +export const holds = sqliteTable("hold", { + id: integer("id").primaryKey(), + holdFor: durationColumn("hold_for").notNull(), + grace: durationColumn("grace"), + laps: text("laps", { mode: "json" }).$type(), +}); +export type Hold = InferSelectModel; +export type HoldInsert = InferInsertModel; +export type HoldUpdate = Partial; +/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten + * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one + * leading "-" for a negative span. Counted in bigint microseconds, so any length is exact and a + * fraction past six digits is truncated toward zero. Generated. */ +function durationWrite(text: string): string { + const ZERO = BigInt(0); + const SECOND = BigInt(1000000); + const MINUTE = BigInt(60000000); + const HOUR = BigInt(3600000000); + const DAY = BigInt(86400000000); + const WEEK = BigInt(604800000000); + const parseIso = (text: string): bigint | undefined => { + const p = + /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( + text, + ); + if (p === null) { + return undefined; + } + const mag = + BigInt(p[2] ?? 0) * WEEK + + BigInt(p[3] ?? 0) * DAY + + BigInt(p[4] ?? 0) * HOUR + + BigInt(p[5] ?? 0) * MINUTE + + BigInt(p[6] ?? 0) * SECOND + + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); + return p[1] === "-" ? -mag : mag; + }; + const isoText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + const s = rest / SECOND; + const f = rest % SECOND; + let out = "PT"; + if (h > ZERO) { + out += String(h) + "H"; + } + if (m > ZERO) { + out += String(m) + "M"; + } + if (s > ZERO || f > ZERO) { + out += + String(s) + + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + + "S"; + } + if (out === "PT") { + return "PT0S"; + } + return neg ? "-" + out : out; + }; + const micros = parseIso(text); + return micros === undefined ? text : isoText(micros); +} + +export const HoldInsertSchema = z.object({ + id: z.number().int(), + holdFor: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite), + grace: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite) + .optional(), + laps: z + .array( + z + .string() + .regex( + /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, + ) + .transform(durationWrite), + ) + .optional(), +}); + +export const HoldUpdateSchema = z.object({ + holdFor: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite) + .optional(), + grace: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite) + .optional() + .nullable(), + laps: z + .array( + z + .string() + .regex( + /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, + ) + .transform(durationWrite), + ) + .optional() + .nullable(), +}); + +/** Typed create shape for Hold: the insert schema's INPUT (pre-transform) type. A + * renamed/dropped/misspelt field is a compile error at every `createHold` call site; + * the schema still validates at runtime. */ +export type HoldCreate = z.input; + +/** Typed patch shape for Hold: every settable field, optional (FR-035 PATCH). A + * renamed/dropped field is a compile error at every `updateHold` call site. */ +export type HoldPatch = z.input; +/** + * Metadata constants for Hold. + * + * Use these instead of magic strings so TS catches typos and refactors stay + * coherent. Each non-dollar-prefixed key is a per-field object carrying + * name, label, view, an htmlType where the field's view maps to one, and the + * RHF-shaped validation rules derived from the field's validator children. + * + * placeholder and helpText are NOT emitted — no provider registers an attribute + * for them, so nothing here could derive one. (htmlType is emitted, but derived + * from the view subtype; the @htmlType override was removed for the same reason.) + * useEntityForm still honours placeholder if you add it: this file is generated, + * hand edits inside it survive regeneration through the three-way merge, and the + * consumers read this const rather than the metadata. That is the intended way + * to set it. + * + * Typical usage with the metaobjects React form helper: + * + * import { useEntityForm } from '@metaobjectsdev/react'; + * const form = useEntityForm(Hold, HoldInsertSchema); + * + */ +export const Hold = { + $entity: "Hold", + $table: "hold", + $path: "/holds", + id: { name: "id", label: "Id", view: "number", htmlType: "number" }, + holdFor: { + name: "holdFor", + label: "Hold For", + view: "text", + htmlType: "text", + rules: { required: "Hold For is required" }, + }, + grace: { name: "grace", label: "Grace", view: "text", htmlType: "text" }, + laps: { name: "laps", label: "Laps", view: "text", htmlType: "text" }, +} as const; +export type HoldFilter = { + limit?: number; + offset?: number; + sort?: string; + or?: HoldFilter[]; + and?: HoldFilter[]; +}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts new file mode 100644 index 0000000000..776849fa04 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts @@ -0,0 +1,191 @@ +// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. +// Source metadata: HoldView (HoldView) +import { customType, integer, sqliteView } from "drizzle-orm/sqlite-core"; +import { z } from "zod"; + +/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); + * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month + * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ +const durationColumn = (() => { + const ZERO = BigInt(0); + const SECOND = BigInt(1000000); + const MINUTE = BigInt(60000000); + const HOUR = BigInt(3600000000); + const DAY = BigInt(86400000000); + const WEEK = BigInt(604800000000); + const INT64_MAX = BigInt("9223372036854775807"); + const parseIso = (text: string): bigint | undefined => { + const p = + /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( + text, + ); + if (p === null) { + return undefined; + } + const mag = + BigInt(p[2] ?? 0) * WEEK + + BigInt(p[3] ?? 0) * DAY + + BigInt(p[4] ?? 0) * HOUR + + BigInt(p[5] ?? 0) * MINUTE + + BigInt(p[6] ?? 0) * SECOND + + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); + return p[1] === "-" ? -mag : mag; + }; + const isoText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + const s = rest / SECOND; + const f = rest % SECOND; + let out = "PT"; + if (h > ZERO) { + out += String(h) + "H"; + } + if (m > ZERO) { + out += String(m) + "M"; + } + if (s > ZERO || f > ZERO) { + out += + String(s) + + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + + "S"; + } + if (out === "PT") { + return "PT0S"; + } + return neg ? "-" + out : out; + }; + const pgText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const two = (n: bigint): string => String(n).padStart(2, "0"); + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); + const f = rest % SECOND; + if (f > ZERO) { + out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); + } + return neg ? "-" + out : out; + }; + const stored = (micros: bigint): number | bigint => { + if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { + throw new Error( + "the span " + + isoText(micros) + + " does not fit the 64-bit microsecond column a field.duration is stored in", + ); + } + const n = Number(micros); + return Number.isSafeInteger(n) ? n : micros; + }; + const notFixed = (v: unknown): Error => + new Error( + "the interval " + + JSON.stringify(v) + + " has a year or month part, so it is not a fixed-length field.duration", + ); + const read = (v: unknown): bigint => { + if (typeof v === "bigint") { + return v; + } + if (typeof v === "number" && Number.isInteger(v)) { + return BigInt(v); + } + if (typeof v === "string") { + if (/^-?\d+$/.test(v)) { + return BigInt(v); + } + const iso = parseIso(v); + if (iso !== undefined) { + return iso; + } + const p = + v.length === 0 + ? null + : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( + v, + ); + if (p === null) { + throw new Error("not a field.duration value: " + JSON.stringify(v)); + } + if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { + throw notFixed(v); + } + let time = ZERO; + if (p[5] !== undefined) { + time = + BigInt(p[5]) * HOUR + + BigInt(p[6] ?? 0) * MINUTE + + BigInt(p[7] ?? 0) * SECOND + + BigInt((p[8] ?? "").padEnd(6, "0")); + if (p[4] === "-") { + time = -time; + } + } + return BigInt(p[3] ?? 0) * DAY + time; + } + if (v !== null && typeof v === "object") { + const o = v as Record; + if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { + throw notFixed(v); + } + const whole = (n: number | undefined): bigint => + BigInt(Math.trunc(n ?? 0)); + // postgres-interval carries microseconds as a fractional millisecond count. + return ( + whole(o.days) * DAY + + whole(o.hours) * HOUR + + whole(o.minutes) * MINUTE + + whole(o.seconds) * SECOND + + BigInt(Math.round((o.milliseconds ?? 0) * 1000)) + ); + } + throw new Error("not a field.duration value: " + JSON.stringify(v)); + }; + return customType<{ data: string; driverData: number | bigint | string }>({ + dataType: () => "integer", + toDriver: (value) => stored(read(value)), + fromDriver: (value) => isoText(read(value)), + }); +})(); +// View declaration — Drizzle uses this for typed SELECT queries. +// The SQL view is created/managed by migrate-ts; .existing() tells Drizzle +// not to attempt DDL for this declaration. +export const holdViewView = sqliteView("v_hold", { + id: integer("id").notNull(), + holdFor: durationColumn("hold_for").notNull(), +}).existing(); +export const HoldViewSchema = z.object({ + id: z.number().int(), + holdFor: z + .string() + .regex( + /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, + ), +}); +export type HoldView = z.infer; +export const HoldView = { + $entity: "HoldView", + $view: "v_hold", + $path: "/hold_views", + id: { name: "id", label: "Id", view: "number", dbCol: "id" }, + holdFor: { + name: "holdFor", + label: "Hold For", + view: "text", + dbCol: "hold_for", + }, +} as const; +export type HoldViewFilter = { + limit?: number; + offset?: number; + sort?: string; + or?: HoldViewFilter[]; + and?: HoldViewFilter[]; +}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts new file mode 100644 index 0000000000..9b91b68554 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts @@ -0,0 +1,310 @@ +// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. +// Source metadata: Timer (Timer) +// Extend in your own module (e.g. Timer.extra.ts) — nothing imports it for you. +import type { InferInsertModel } from "drizzle-orm"; +import { + customType, + integer, + sqliteTable, + sqliteView, +} from "drizzle-orm/sqlite-core"; +import { z } from "zod"; + +/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); + * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month + * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ +const durationColumn = (() => { + const ZERO = BigInt(0); + const SECOND = BigInt(1000000); + const MINUTE = BigInt(60000000); + const HOUR = BigInt(3600000000); + const DAY = BigInt(86400000000); + const WEEK = BigInt(604800000000); + const INT64_MAX = BigInt("9223372036854775807"); + const parseIso = (text: string): bigint | undefined => { + const p = + /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( + text, + ); + if (p === null) { + return undefined; + } + const mag = + BigInt(p[2] ?? 0) * WEEK + + BigInt(p[3] ?? 0) * DAY + + BigInt(p[4] ?? 0) * HOUR + + BigInt(p[5] ?? 0) * MINUTE + + BigInt(p[6] ?? 0) * SECOND + + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); + return p[1] === "-" ? -mag : mag; + }; + const isoText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + const s = rest / SECOND; + const f = rest % SECOND; + let out = "PT"; + if (h > ZERO) { + out += String(h) + "H"; + } + if (m > ZERO) { + out += String(m) + "M"; + } + if (s > ZERO || f > ZERO) { + out += + String(s) + + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + + "S"; + } + if (out === "PT") { + return "PT0S"; + } + return neg ? "-" + out : out; + }; + const pgText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const two = (n: bigint): string => String(n).padStart(2, "0"); + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); + const f = rest % SECOND; + if (f > ZERO) { + out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); + } + return neg ? "-" + out : out; + }; + const stored = (micros: bigint): number | bigint => { + if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { + throw new Error( + "the span " + + isoText(micros) + + " does not fit the 64-bit microsecond column a field.duration is stored in", + ); + } + const n = Number(micros); + return Number.isSafeInteger(n) ? n : micros; + }; + const notFixed = (v: unknown): Error => + new Error( + "the interval " + + JSON.stringify(v) + + " has a year or month part, so it is not a fixed-length field.duration", + ); + const read = (v: unknown): bigint => { + if (typeof v === "bigint") { + return v; + } + if (typeof v === "number" && Number.isInteger(v)) { + return BigInt(v); + } + if (typeof v === "string") { + if (/^-?\d+$/.test(v)) { + return BigInt(v); + } + const iso = parseIso(v); + if (iso !== undefined) { + return iso; + } + const p = + v.length === 0 + ? null + : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( + v, + ); + if (p === null) { + throw new Error("not a field.duration value: " + JSON.stringify(v)); + } + if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { + throw notFixed(v); + } + let time = ZERO; + if (p[5] !== undefined) { + time = + BigInt(p[5]) * HOUR + + BigInt(p[6] ?? 0) * MINUTE + + BigInt(p[7] ?? 0) * SECOND + + BigInt((p[8] ?? "").padEnd(6, "0")); + if (p[4] === "-") { + time = -time; + } + } + return BigInt(p[3] ?? 0) * DAY + time; + } + if (v !== null && typeof v === "object") { + const o = v as Record; + if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { + throw notFixed(v); + } + const whole = (n: number | undefined): bigint => + BigInt(Math.trunc(n ?? 0)); + // postgres-interval carries microseconds as a fractional millisecond count. + return ( + whole(o.days) * DAY + + whole(o.hours) * HOUR + + whole(o.minutes) * MINUTE + + whole(o.seconds) * SECOND + + BigInt(Math.round((o.milliseconds ?? 0) * 1000)) + ); + } + throw new Error("not a field.duration value: " + JSON.stringify(v)); + }; + return customType<{ data: string; driverData: number | bigint | string }>({ + dataType: () => "integer", + toDriver: (value) => stored(read(value)), + fromDriver: (value) => isoText(read(value)), + }); +})(); +export const timers = sqliteTable("timer", { + id: integer("id").primaryKey(), + period: durationColumn("period").notNull(), +}); +// View declaration — Drizzle uses this for typed SELECT queries. +// The SQL view is created/managed by migrate-ts; .existing() tells Drizzle +// not to attempt DDL for this declaration. +export const timerView = sqliteView("v_timer", { + id: integer("id").notNull(), + period: durationColumn("period").notNull(), +}).existing(); +export const TimerSchema = z.object({ + id: z.number().int(), + period: z + .string() + .regex( + /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, + ), +}); +export type Timer = z.infer; +export type TimerInsert = InferInsertModel; +export type TimerUpdate = Partial; +/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten + * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one + * leading "-" for a negative span. Counted in bigint microseconds, so any length is exact and a + * fraction past six digits is truncated toward zero. Generated. */ +function durationWrite(text: string): string { + const ZERO = BigInt(0); + const SECOND = BigInt(1000000); + const MINUTE = BigInt(60000000); + const HOUR = BigInt(3600000000); + const DAY = BigInt(86400000000); + const WEEK = BigInt(604800000000); + const parseIso = (text: string): bigint | undefined => { + const p = + /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( + text, + ); + if (p === null) { + return undefined; + } + const mag = + BigInt(p[2] ?? 0) * WEEK + + BigInt(p[3] ?? 0) * DAY + + BigInt(p[4] ?? 0) * HOUR + + BigInt(p[5] ?? 0) * MINUTE + + BigInt(p[6] ?? 0) * SECOND + + BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); + return p[1] === "-" ? -mag : mag; + }; + const isoText = (micros: bigint): string => { + const neg = micros < ZERO; + let rest = neg ? -micros : micros; + const h = rest / HOUR; + rest %= HOUR; + const m = rest / MINUTE; + rest %= MINUTE; + const s = rest / SECOND; + const f = rest % SECOND; + let out = "PT"; + if (h > ZERO) { + out += String(h) + "H"; + } + if (m > ZERO) { + out += String(m) + "M"; + } + if (s > ZERO || f > ZERO) { + out += + String(s) + + (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + + "S"; + } + if (out === "PT") { + return "PT0S"; + } + return neg ? "-" + out : out; + }; + const micros = parseIso(text); + return micros === undefined ? text : isoText(micros); +} + +export const TimerInsertSchema = z.object({ + id: z.number().int(), + period: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite), +}); + +export const TimerUpdateSchema = z.object({ + period: z + .string() + .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) + .transform(durationWrite) + .optional(), +}); + +/** Typed create shape for Timer: the insert schema's INPUT (pre-transform) type. A + * renamed/dropped/misspelt field is a compile error at every `createTimer` call site; + * the schema still validates at runtime. */ +export type TimerCreate = z.input; + +/** Typed patch shape for Timer: every settable field, optional (FR-035 PATCH). A + * renamed/dropped field is a compile error at every `updateTimer` call site. */ +export type TimerPatch = z.input; +/** + * Metadata constants for Timer. + * + * Use these instead of magic strings so TS catches typos and refactors stay + * coherent. Each non-dollar-prefixed key is a per-field object carrying + * name, label, view, an htmlType where the field's view maps to one, and the + * RHF-shaped validation rules derived from the field's validator children. + * + * placeholder and helpText are NOT emitted — no provider registers an attribute + * for them, so nothing here could derive one. (htmlType is emitted, but derived + * from the view subtype; the @htmlType override was removed for the same reason.) + * useEntityForm still honours placeholder if you add it: this file is generated, + * hand edits inside it survive regeneration through the three-way merge, and the + * consumers read this const rather than the metadata. That is the intended way + * to set it. + * + * Typical usage with the metaobjects React form helper: + * + * import { useEntityForm } from '@metaobjectsdev/react'; + * const form = useEntityForm(Timer, TimerInsertSchema); + * + */ +export const Timer = { + $entity: "Timer", + $table: "timer", + $path: "/timers", + id: { name: "id", label: "Id", view: "number", htmlType: "number" }, + period: { + name: "period", + label: "Period", + view: "text", + htmlType: "text", + rules: { required: "Period is required" }, + }, +} as const; +export type TimerFilter = { + limit?: number; + offset?: number; + sort?: string; + or?: TimerFilter[]; + and?: TimerFilter[]; +}; From 1dfe213cbd52d93e9c08d64c0d535496be518020 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 14 Nov 2026 02:19:42 -0500 Subject: [PATCH 29/31] no-mistakes(test): apply changes made during test validation --- .../test/tmp-duration-sqlite-lba8VZ/Hold.ts | 331 ------------------ .../tmp-duration-sqlite-lba8VZ/HoldView.ts | 191 ---------- .../test/tmp-duration-sqlite-lba8VZ/Timer.ts | 310 ---------------- 3 files changed, 832 deletions(-) delete mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts delete mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts delete mode 100644 server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts deleted file mode 100644 index 5f33fd04a4..0000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Hold.ts +++ /dev/null @@ -1,331 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Hold (Hold) -// Extend in your own module (e.g. Hold.extra.ts) — nothing imports it for you. -import type { InferInsertModel, InferSelectModel } from "drizzle-orm"; -import { - customType, - integer, - sqliteTable, - text, -} from "drizzle-orm/sqlite-core"; -import { z } from "zod"; - -/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); - * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month - * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ -const durationColumn = (() => { - const ZERO = BigInt(0); - const SECOND = BigInt(1000000); - const MINUTE = BigInt(60000000); - const HOUR = BigInt(3600000000); - const DAY = BigInt(86400000000); - const WEEK = BigInt(604800000000); - const INT64_MAX = BigInt("9223372036854775807"); - const parseIso = (text: string): bigint | undefined => { - const p = - /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( - text, - ); - if (p === null) { - return undefined; - } - const mag = - BigInt(p[2] ?? 0) * WEEK + - BigInt(p[3] ?? 0) * DAY + - BigInt(p[4] ?? 0) * HOUR + - BigInt(p[5] ?? 0) * MINUTE + - BigInt(p[6] ?? 0) * SECOND + - BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); - return p[1] === "-" ? -mag : mag; - }; - const isoText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - const s = rest / SECOND; - const f = rest % SECOND; - let out = "PT"; - if (h > ZERO) { - out += String(h) + "H"; - } - if (m > ZERO) { - out += String(m) + "M"; - } - if (s > ZERO || f > ZERO) { - out += - String(s) + - (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + - "S"; - } - if (out === "PT") { - return "PT0S"; - } - return neg ? "-" + out : out; - }; - const pgText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const two = (n: bigint): string => String(n).padStart(2, "0"); - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); - const f = rest % SECOND; - if (f > ZERO) { - out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); - } - return neg ? "-" + out : out; - }; - const stored = (micros: bigint): number | bigint => { - if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { - throw new Error( - "the span " + - isoText(micros) + - " does not fit the 64-bit microsecond column a field.duration is stored in", - ); - } - const n = Number(micros); - return Number.isSafeInteger(n) ? n : micros; - }; - const notFixed = (v: unknown): Error => - new Error( - "the interval " + - JSON.stringify(v) + - " has a year or month part, so it is not a fixed-length field.duration", - ); - const read = (v: unknown): bigint => { - if (typeof v === "bigint") { - return v; - } - if (typeof v === "number" && Number.isInteger(v)) { - return BigInt(v); - } - if (typeof v === "string") { - if (/^-?\d+$/.test(v)) { - return BigInt(v); - } - const iso = parseIso(v); - if (iso !== undefined) { - return iso; - } - const p = - v.length === 0 - ? null - : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( - v, - ); - if (p === null) { - throw new Error("not a field.duration value: " + JSON.stringify(v)); - } - if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { - throw notFixed(v); - } - let time = ZERO; - if (p[5] !== undefined) { - time = - BigInt(p[5]) * HOUR + - BigInt(p[6] ?? 0) * MINUTE + - BigInt(p[7] ?? 0) * SECOND + - BigInt((p[8] ?? "").padEnd(6, "0")); - if (p[4] === "-") { - time = -time; - } - } - return BigInt(p[3] ?? 0) * DAY + time; - } - if (v !== null && typeof v === "object") { - const o = v as Record; - if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { - throw notFixed(v); - } - const whole = (n: number | undefined): bigint => - BigInt(Math.trunc(n ?? 0)); - // postgres-interval carries microseconds as a fractional millisecond count. - return ( - whole(o.days) * DAY + - whole(o.hours) * HOUR + - whole(o.minutes) * MINUTE + - whole(o.seconds) * SECOND + - BigInt(Math.round((o.milliseconds ?? 0) * 1000)) - ); - } - throw new Error("not a field.duration value: " + JSON.stringify(v)); - }; - return customType<{ data: string; driverData: number | bigint | string }>({ - dataType: () => "integer", - toDriver: (value) => stored(read(value)), - fromDriver: (value) => isoText(read(value)), - }); -})(); -export const holds = sqliteTable("hold", { - id: integer("id").primaryKey(), - holdFor: durationColumn("hold_for").notNull(), - grace: durationColumn("grace"), - laps: text("laps", { mode: "json" }).$type(), -}); -export type Hold = InferSelectModel; -export type HoldInsert = InferInsertModel; -export type HoldUpdate = Partial; -/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten - * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one - * leading "-" for a negative span. Counted in bigint microseconds, so any length is exact and a - * fraction past six digits is truncated toward zero. Generated. */ -function durationWrite(text: string): string { - const ZERO = BigInt(0); - const SECOND = BigInt(1000000); - const MINUTE = BigInt(60000000); - const HOUR = BigInt(3600000000); - const DAY = BigInt(86400000000); - const WEEK = BigInt(604800000000); - const parseIso = (text: string): bigint | undefined => { - const p = - /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( - text, - ); - if (p === null) { - return undefined; - } - const mag = - BigInt(p[2] ?? 0) * WEEK + - BigInt(p[3] ?? 0) * DAY + - BigInt(p[4] ?? 0) * HOUR + - BigInt(p[5] ?? 0) * MINUTE + - BigInt(p[6] ?? 0) * SECOND + - BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); - return p[1] === "-" ? -mag : mag; - }; - const isoText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - const s = rest / SECOND; - const f = rest % SECOND; - let out = "PT"; - if (h > ZERO) { - out += String(h) + "H"; - } - if (m > ZERO) { - out += String(m) + "M"; - } - if (s > ZERO || f > ZERO) { - out += - String(s) + - (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + - "S"; - } - if (out === "PT") { - return "PT0S"; - } - return neg ? "-" + out : out; - }; - const micros = parseIso(text); - return micros === undefined ? text : isoText(micros); -} - -export const HoldInsertSchema = z.object({ - id: z.number().int(), - holdFor: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite), - grace: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite) - .optional(), - laps: z - .array( - z - .string() - .regex( - /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, - ) - .transform(durationWrite), - ) - .optional(), -}); - -export const HoldUpdateSchema = z.object({ - holdFor: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite) - .optional(), - grace: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite) - .optional() - .nullable(), - laps: z - .array( - z - .string() - .regex( - /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, - ) - .transform(durationWrite), - ) - .optional() - .nullable(), -}); - -/** Typed create shape for Hold: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createHold` call site; - * the schema still validates at runtime. */ -export type HoldCreate = z.input; - -/** Typed patch shape for Hold: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateHold` call site. */ -export type HoldPatch = z.input; -/** - * Metadata constants for Hold. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Hold, HoldInsertSchema); - * - */ -export const Hold = { - $entity: "Hold", - $table: "hold", - $path: "/holds", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - holdFor: { - name: "holdFor", - label: "Hold For", - view: "text", - htmlType: "text", - rules: { required: "Hold For is required" }, - }, - grace: { name: "grace", label: "Grace", view: "text", htmlType: "text" }, - laps: { name: "laps", label: "Laps", view: "text", htmlType: "text" }, -} as const; -export type HoldFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: HoldFilter[]; - and?: HoldFilter[]; -}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts deleted file mode 100644 index 776849fa04..0000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/HoldView.ts +++ /dev/null @@ -1,191 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: HoldView (HoldView) -import { customType, integer, sqliteView } from "drizzle-orm/sqlite-core"; -import { z } from "zod"; - -/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); - * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month - * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ -const durationColumn = (() => { - const ZERO = BigInt(0); - const SECOND = BigInt(1000000); - const MINUTE = BigInt(60000000); - const HOUR = BigInt(3600000000); - const DAY = BigInt(86400000000); - const WEEK = BigInt(604800000000); - const INT64_MAX = BigInt("9223372036854775807"); - const parseIso = (text: string): bigint | undefined => { - const p = - /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( - text, - ); - if (p === null) { - return undefined; - } - const mag = - BigInt(p[2] ?? 0) * WEEK + - BigInt(p[3] ?? 0) * DAY + - BigInt(p[4] ?? 0) * HOUR + - BigInt(p[5] ?? 0) * MINUTE + - BigInt(p[6] ?? 0) * SECOND + - BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); - return p[1] === "-" ? -mag : mag; - }; - const isoText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - const s = rest / SECOND; - const f = rest % SECOND; - let out = "PT"; - if (h > ZERO) { - out += String(h) + "H"; - } - if (m > ZERO) { - out += String(m) + "M"; - } - if (s > ZERO || f > ZERO) { - out += - String(s) + - (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + - "S"; - } - if (out === "PT") { - return "PT0S"; - } - return neg ? "-" + out : out; - }; - const pgText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const two = (n: bigint): string => String(n).padStart(2, "0"); - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); - const f = rest % SECOND; - if (f > ZERO) { - out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); - } - return neg ? "-" + out : out; - }; - const stored = (micros: bigint): number | bigint => { - if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { - throw new Error( - "the span " + - isoText(micros) + - " does not fit the 64-bit microsecond column a field.duration is stored in", - ); - } - const n = Number(micros); - return Number.isSafeInteger(n) ? n : micros; - }; - const notFixed = (v: unknown): Error => - new Error( - "the interval " + - JSON.stringify(v) + - " has a year or month part, so it is not a fixed-length field.duration", - ); - const read = (v: unknown): bigint => { - if (typeof v === "bigint") { - return v; - } - if (typeof v === "number" && Number.isInteger(v)) { - return BigInt(v); - } - if (typeof v === "string") { - if (/^-?\d+$/.test(v)) { - return BigInt(v); - } - const iso = parseIso(v); - if (iso !== undefined) { - return iso; - } - const p = - v.length === 0 - ? null - : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( - v, - ); - if (p === null) { - throw new Error("not a field.duration value: " + JSON.stringify(v)); - } - if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { - throw notFixed(v); - } - let time = ZERO; - if (p[5] !== undefined) { - time = - BigInt(p[5]) * HOUR + - BigInt(p[6] ?? 0) * MINUTE + - BigInt(p[7] ?? 0) * SECOND + - BigInt((p[8] ?? "").padEnd(6, "0")); - if (p[4] === "-") { - time = -time; - } - } - return BigInt(p[3] ?? 0) * DAY + time; - } - if (v !== null && typeof v === "object") { - const o = v as Record; - if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { - throw notFixed(v); - } - const whole = (n: number | undefined): bigint => - BigInt(Math.trunc(n ?? 0)); - // postgres-interval carries microseconds as a fractional millisecond count. - return ( - whole(o.days) * DAY + - whole(o.hours) * HOUR + - whole(o.minutes) * MINUTE + - whole(o.seconds) * SECOND + - BigInt(Math.round((o.milliseconds ?? 0) * 1000)) - ); - } - throw new Error("not a field.duration value: " + JSON.stringify(v)); - }; - return customType<{ data: string; driverData: number | bigint | string }>({ - dataType: () => "integer", - toDriver: (value) => stored(read(value)), - fromDriver: (value) => isoText(read(value)), - }); -})(); -// View declaration — Drizzle uses this for typed SELECT queries. -// The SQL view is created/managed by migrate-ts; .existing() tells Drizzle -// not to attempt DDL for this declaration. -export const holdViewView = sqliteView("v_hold", { - id: integer("id").notNull(), - holdFor: durationColumn("hold_for").notNull(), -}).existing(); -export const HoldViewSchema = z.object({ - id: z.number().int(), - holdFor: z - .string() - .regex( - /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, - ), -}); -export type HoldView = z.infer; -export const HoldView = { - $entity: "HoldView", - $view: "v_hold", - $path: "/hold_views", - id: { name: "id", label: "Id", view: "number", dbCol: "id" }, - holdFor: { - name: "holdFor", - label: "Hold For", - view: "text", - dbCol: "hold_for", - }, -} as const; -export type HoldViewFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: HoldViewFilter[]; - and?: HoldViewFilter[]; -}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts b/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts deleted file mode 100644 index 9b91b68554..0000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-duration-sqlite-lba8VZ/Timer.ts +++ /dev/null @@ -1,310 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Timer (Timer) -// Extend in your own module (e.g. Timer.extra.ts) — nothing imports it for you. -import type { InferInsertModel } from "drizzle-orm"; -import { - customType, - integer, - sqliteTable, - sqliteView, -} from "drizzle-orm/sqlite-core"; -import { z } from "zod"; - -/** A field.duration column. Binds the span as a 64-bit integer count of microseconds (refused past that range); - * reads it back as the canonical ISO-8601 duration, and refuses an interval with a year or month - * part. Counted in bigint microseconds, so the column is the only limit. Generated. */ -const durationColumn = (() => { - const ZERO = BigInt(0); - const SECOND = BigInt(1000000); - const MINUTE = BigInt(60000000); - const HOUR = BigInt(3600000000); - const DAY = BigInt(86400000000); - const WEEK = BigInt(604800000000); - const INT64_MAX = BigInt("9223372036854775807"); - const parseIso = (text: string): bigint | undefined => { - const p = - /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( - text, - ); - if (p === null) { - return undefined; - } - const mag = - BigInt(p[2] ?? 0) * WEEK + - BigInt(p[3] ?? 0) * DAY + - BigInt(p[4] ?? 0) * HOUR + - BigInt(p[5] ?? 0) * MINUTE + - BigInt(p[6] ?? 0) * SECOND + - BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); - return p[1] === "-" ? -mag : mag; - }; - const isoText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - const s = rest / SECOND; - const f = rest % SECOND; - let out = "PT"; - if (h > ZERO) { - out += String(h) + "H"; - } - if (m > ZERO) { - out += String(m) + "M"; - } - if (s > ZERO || f > ZERO) { - out += - String(s) + - (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + - "S"; - } - if (out === "PT") { - return "PT0S"; - } - return neg ? "-" + out : out; - }; - const pgText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const two = (n: bigint): string => String(n).padStart(2, "0"); - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - let out = two(h) + ":" + two(m) + ":" + two(rest / SECOND); - const f = rest % SECOND; - if (f > ZERO) { - out += "." + String(f).padStart(6, "0").replace(/0+$/, ""); - } - return neg ? "-" + out : out; - }; - const stored = (micros: bigint): number | bigint => { - if (micros > INT64_MAX || micros < -INT64_MAX - BigInt(1)) { - throw new Error( - "the span " + - isoText(micros) + - " does not fit the 64-bit microsecond column a field.duration is stored in", - ); - } - const n = Number(micros); - return Number.isSafeInteger(n) ? n : micros; - }; - const notFixed = (v: unknown): Error => - new Error( - "the interval " + - JSON.stringify(v) + - " has a year or month part, so it is not a fixed-length field.duration", - ); - const read = (v: unknown): bigint => { - if (typeof v === "bigint") { - return v; - } - if (typeof v === "number" && Number.isInteger(v)) { - return BigInt(v); - } - if (typeof v === "string") { - if (/^-?\d+$/.test(v)) { - return BigInt(v); - } - const iso = parseIso(v); - if (iso !== undefined) { - return iso; - } - const p = - v.length === 0 - ? null - : /^(?:([-+]?\d+) years? ?)?(?:([-+]?\d+) mons? ?)?(?:([-+]?\d+) days? ?)?(?:([-+])?(\d+):(\d{2}):(\d{2})(?:\.(\d{1,6}))?)?$/.exec( - v, - ); - if (p === null) { - throw new Error("not a field.duration value: " + JSON.stringify(v)); - } - if (Number(p[1] ?? 0) !== 0 || Number(p[2] ?? 0) !== 0) { - throw notFixed(v); - } - let time = ZERO; - if (p[5] !== undefined) { - time = - BigInt(p[5]) * HOUR + - BigInt(p[6] ?? 0) * MINUTE + - BigInt(p[7] ?? 0) * SECOND + - BigInt((p[8] ?? "").padEnd(6, "0")); - if (p[4] === "-") { - time = -time; - } - } - return BigInt(p[3] ?? 0) * DAY + time; - } - if (v !== null && typeof v === "object") { - const o = v as Record; - if ((o.years ?? 0) !== 0 || (o.months ?? 0) !== 0) { - throw notFixed(v); - } - const whole = (n: number | undefined): bigint => - BigInt(Math.trunc(n ?? 0)); - // postgres-interval carries microseconds as a fractional millisecond count. - return ( - whole(o.days) * DAY + - whole(o.hours) * HOUR + - whole(o.minutes) * MINUTE + - whole(o.seconds) * SECOND + - BigInt(Math.round((o.milliseconds ?? 0) * 1000)) - ); - } - throw new Error("not a field.duration value: " + JSON.stringify(v)); - }; - return customType<{ data: string; driverData: number | bigint | string }>({ - dataType: () => "integer", - toDriver: (value) => stored(read(value)), - fromDriver: (value) => isoText(read(value)), - }); -})(); -export const timers = sqliteTable("timer", { - id: integer("id").primaryKey(), - period: durationColumn("period").notNull(), -}); -// View declaration — Drizzle uses this for typed SELECT queries. -// The SQL view is created/managed by migrate-ts; .existing() tells Drizzle -// not to attempt DDL for this declaration. -export const timerView = sqliteView("v_timer", { - id: integer("id").notNull(), - period: durationColumn("period").notNull(), -}).existing(); -export const TimerSchema = z.object({ - id: z.number().int(), - period: z - .string() - .regex( - /^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/, - ), -}); -export type Timer = z.infer; -export type TimerInsert = InferInsertModel; -export type TimerUpdate = Partial; -/** A field.duration on the wire: an ISO-8601 duration (weeks, days, hours, minutes, seconds) rewritten - * to its canonical form — hours never folded into days, zero components left out, zero as PT0S, one - * leading "-" for a negative span. Counted in bigint microseconds, so any length is exact and a - * fraction past six digits is truncated toward zero. Generated. */ -function durationWrite(text: string): string { - const ZERO = BigInt(0); - const SECOND = BigInt(1000000); - const MINUTE = BigInt(60000000); - const HOUR = BigInt(3600000000); - const DAY = BigInt(86400000000); - const WEEK = BigInt(604800000000); - const parseIso = (text: string): bigint | undefined => { - const p = - /^([-+])?P(?!$)(?:(\d+)W)?(?:(\d+)D)?(?:T(?=\d)(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)(?:\.(\d+))?S)?)?$/.exec( - text, - ); - if (p === null) { - return undefined; - } - const mag = - BigInt(p[2] ?? 0) * WEEK + - BigInt(p[3] ?? 0) * DAY + - BigInt(p[4] ?? 0) * HOUR + - BigInt(p[5] ?? 0) * MINUTE + - BigInt(p[6] ?? 0) * SECOND + - BigInt((p[7] ?? "").padEnd(6, "0").slice(0, 6)); - return p[1] === "-" ? -mag : mag; - }; - const isoText = (micros: bigint): string => { - const neg = micros < ZERO; - let rest = neg ? -micros : micros; - const h = rest / HOUR; - rest %= HOUR; - const m = rest / MINUTE; - rest %= MINUTE; - const s = rest / SECOND; - const f = rest % SECOND; - let out = "PT"; - if (h > ZERO) { - out += String(h) + "H"; - } - if (m > ZERO) { - out += String(m) + "M"; - } - if (s > ZERO || f > ZERO) { - out += - String(s) + - (f > ZERO ? "." + String(f).padStart(6, "0").replace(/0+$/, "") : "") + - "S"; - } - if (out === "PT") { - return "PT0S"; - } - return neg ? "-" + out : out; - }; - const micros = parseIso(text); - return micros === undefined ? text : isoText(micros); -} - -export const TimerInsertSchema = z.object({ - id: z.number().int(), - period: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite), -}); - -export const TimerUpdateSchema = z.object({ - period: z - .string() - .regex(/^[-+]?P(?!$)(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+(\.\d+)?S)?)?$/) - .transform(durationWrite) - .optional(), -}); - -/** Typed create shape for Timer: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createTimer` call site; - * the schema still validates at runtime. */ -export type TimerCreate = z.input; - -/** Typed patch shape for Timer: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateTimer` call site. */ -export type TimerPatch = z.input; -/** - * Metadata constants for Timer. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Timer, TimerInsertSchema); - * - */ -export const Timer = { - $entity: "Timer", - $table: "timer", - $path: "/timers", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - period: { - name: "period", - label: "Period", - view: "text", - htmlType: "text", - rules: { required: "Period is required" }, - }, -} as const; -export type TimerFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: TimerFilter[]; - and?: TimerFilter[]; -}; From 824fc8326b956c76d46e67eac1a9dc37cf010acc Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 14 Nov 2026 02:41:38 -0500 Subject: [PATCH 30/31] no-mistakes(document): Correct init config docs for C# and Python --- docs/ports/csharp.md | 2 +- docs/ports/python.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index c6734184f2..c4e3e83b22 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -400,7 +400,7 @@ the gate like any packaged generator's. The example above is compiled and run by `dotnet meta init --with-docs` writes the list with the api reference already in it (`new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = "docs/csharp", ProjectName = "" }`), -so `gen` writes it. The `.metaobjects/config.json` init writes names no `outDir`: set `"csharp": { "outDir": "" }` +so `gen` writes it. Init writes `.metaobjects/config.json` (the `api`, `migrate` and `verify` settings, described below) but no `outDir`: set `"csharp": { "outDir": "" }` there and a plain `dotnet meta verify` drift-checks the docs with the rest of the codegen output (`verify --codegen` compares the whole tree of each entry's `OutDir`, so keep it generated files only). Against an existing list, init prints the entry to add and edits nothing. Without the flag diff --git a/docs/ports/python.md b/docs/ports/python.md index 24072d6ceb..b36d186c0f 100644 --- a/docs/ports/python.md +++ b/docs/ports/python.md @@ -190,7 +190,9 @@ and the [config reference](../features/cli.md). with no generators) and an empty, loadable `metaobjects/meta.app.yaml`, and reports what it wrote. It never overwrites a file it finds; it says it kept it. `metaobjects init --with-docs` also adds a `docs` target running `api-docs`; without the flag no documentation generator is -wired. Nothing writes docs until the config lists `api-docs`. The "Model / metadata" link on +wired. Nothing writes docs until the config lists `api-docs`. It also writes `.metaobjects/config.json` +with the `api` block (a page size and `api.unknownFields: reject`), `verify.handEdits: "fail"` and +`migrate.foreignKeyIndexes: true`, unless that file exists ([the `api` block](#page-size-and-undeclared-body-keys-the-api-block)). The "Model / metadata" link on each api page is emitted only when `apiDocs.modelBaseUrl` is set (the Python port writes no model page, so a relative link would be dead). From 2f1c50df0037fb040756c72f8611b25c7c3e48af Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 14 Nov 2026 05:18:48 -0500 Subject: [PATCH 31/31] test(kotlin): expect table-qualified columns in the TPH create autoSet tests The TPH per-subtype create now writes every column through the table object (it[AuthTable.autoCreatedAt]), so a column named body, issues or unknownKeys cannot be shadowed by the handler's locals. Two KotlinAutoSetStampingTest cases still asserted the unqualified form and failed the java-slow lane. --- .../generator/kotlin/KotlinAutoSetStampingTest.kt | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt index 9ce41c4af6..370da21385 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinAutoSetStampingTest.kt @@ -309,11 +309,11 @@ class KotlinAutoSetStampingTest { val src = generateTphControllerSource(tphAutoSetFixture, "Auth") val createBody = methodBody(src, "fun createBridge") assertTrue("val autoSetNow0 =" in createBody, "create captures a now()-val; saw:\n$createBody") - assertTrue("it[autoCreatedAt] = autoSetNow0" in createBody, + assertTrue("it[AuthTable.autoCreatedAt] = autoSetNow0" in createBody, "onCreate stamped from the captured now(); saw:\n$createBody") - assertTrue("it[autoUpdatedAt] = autoSetNow0" in createBody, + assertTrue("it[AuthTable.autoUpdatedAt] = autoSetNow0" in createBody, "onUpdate stamped from the same now() (created==updated); saw:\n$createBody") - assertFalse("it[autoCreatedAt] = dto." in createBody, + assertFalse("it[AuthTable.autoCreatedAt] = dto." in createBody, "autoSet is never bound from the caller DTO; saw:\n$createBody") } @@ -340,8 +340,8 @@ class KotlinAutoSetStampingTest { val createBody = methodBody(src, "fun createSpook") // create still stamps both columns from one captured now(), even with an empty writableFields. assertTrue("val autoSetNow0 =" in createBody, "create captures a now()-val; saw:\n$createBody") - assertTrue("it[autoCreatedAt] = autoSetNow0" in createBody && - "it[autoUpdatedAt] = autoSetNow0" in createBody, + assertTrue("it[GhostTable.autoCreatedAt] = autoSetNow0" in createBody && + "it[GhostTable.autoUpdatedAt] = autoSetNow0" in createBody, "create must stamp both @autoSet columns; saw:\n$createBody") val updateBody = methodBody(src, "fun updateSpook") // no per-subtype settable columns ⇒ no "try"/JsonMappingException scaffolding at all — a bare