Skip to content

Commit 8474465

Browse files
authored
feat(metadata): add relative dates and case expressions to view loaders (#562)
* feat(metadata): relative dates and case/now expressions in every loader Load and validate, in TypeScript, C#, Java (Kotlin through it) and Python: - a relative date ({ now: ... }) in a projection's and an origin.first / origin.aggregate's @filter, shape-checked where it is declared; - case and now in an origin.computed expression, and a ref in it that names another of the projection's own fields (never itself, never an aggregate) before a base field. error-relative-date-origin-host and error-relative-date-wrong-host are removed: those hosts now accept the value. New fixtures: projection-filter-relative-now, origin-filter-relative-now, origin-computed-case-over-joined, error-origin-computed-case-mixed-types. * no-mistakes(review): Fix case/now expression checks in all loaders * no-mistakes(review): Tighten expression keys and relative-date types * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Reject fractional literals beside integer-backed fields * no-mistakes(document): Fixed stale expression-grammar docs: error-code entry and @condition refusal note.
1 parent 57a7c6f commit 8474465

80 files changed

Lines changed: 4651 additions & 293 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
7171
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.
7272

7373
**Cross-port conformance corpora** (every port runs the shared corpus):
74-
- Metamodel: `fixtures/conformance/` (459 fixtures; 38 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
74+
- Metamodel: `fixtures/conformance/` (471 fixtures; 38 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
7575
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
7676
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
7777
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.

‎CHANGELOG.md‎

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,9 @@ _npm `1.3.0` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates
3737

3838
**`metamodelVersion` moves to `1.3`.** This release adds vocabulary (`field.duration`,
3939
`validator.exactlyOne`, `@constraintName`, `@condition`, `@maxLength` on `field.enum`,
40-
`@sequenceStart`, `@sequenceName`, `@flattenedColumnPrefix`, the `pascal` column rule and
41-
`@collectionName`) and nothing that loaded before stops loading. The registry gains two
40+
`@sequenceStart`, `@sequenceName`, `@flattenedColumnPrefix`, the `pascal` column rule,
41+
`@collectionName`, and the `case` and `now` expression nodes) and nothing that loaded before
42+
stops loading. The registry gains two
4243
fields on a type or attr entry, carried into the registry manifest: `implementedIn` (cross-port
4344
vocabulary only the named ports implement so far, e.g. the `view.*` controls, `["typescript"]`) and
4445
`ports` (vocabulary only the named ports ever act on, e.g. `@objectAdapter`, `["java", "kotlin"]`:
@@ -137,6 +138,32 @@ Each item below is described in full under Added, Changed or Fixed; this list is
137138
(`ERR_AMBIGUOUS_PATH`). A declared relationship or reference of the same name still wins. Every
138139
port's loader implements it, gated by `origin-reverse-reference-hop`,
139140
`origin-reverse-hop-collection-name` and `error-origin-reverse-hop-ambiguous`.
141+
- **Expressions and relative dates in views.** These load and validate in every port's loader,
142+
each gated by shared conformance fixtures:
143+
- a projection's and an `origin.first` / `origin.aggregate`'s `@filter` take a relative date
144+
(`{ "now": "-P7D" }`), checked for its shape (`projection-filter-relative-now`,
145+
`origin-filter-relative-now`); the fixtures `error-relative-date-origin-host` and
146+
`error-relative-date-wrong-host`, which asserted the refusal, are removed;
147+
- an `origin.computed` expression may use `now` and `case` (searched, or simple with an
148+
`operand`), and a ref in it may name another of the projection's own fields (a passthrough,
149+
an `origin.first` or another `origin.computed`, never itself or an aggregate) before a base
150+
field (`origin-computed-case-over-joined`, `error-origin-computed-case-mixed-types`). A
151+
simple `case` compares a literal `when` with any `operand`, so an enum or long operand maps
152+
its codes to labels. A `case`'s results and a `coalesce`'s arguments unify a numeric literal
153+
with any numeric field and a `now` with a date or timestamp (`origin-computed-literal-unifies`);
154+
- the loaders refuse what a view cannot lower: a `now` whose value is not an ISO-8601 duration
155+
or whose `startOf` is not day, week or month (`error-origin-computed-now-bad-duration`); a
156+
searched `case` whose `when` is not boolean (`error-origin-computed-case-non-boolean-when`);
157+
computed fields that refer to each other in a cycle (`error-origin-computed-cycle`); and a
158+
malformed relative date in a view host's `@filter` (`error-relative-date-view-host-malformed`),
159+
or one on a field that is not a `field.date` or `field.timestamp`, where a projection's key
160+
names its own field and an origin's names a field of its `@of` entity or `@via` terminal
161+
(`error-relative-date-non-temporal-field`); and an expression node that carries a key its
162+
kind does not take, such as a misspelled `else` on a `case` or a `field` node that also
163+
carries `case` (`ERR_UNKNOWN_EXPR_NODE`, `error-origin-computed-case-unknown-key`,
164+
`error-origin-computed-node-mixed-keys`);
165+
- a validator `@condition` still refuses `case` and `now` (`ERR_BAD_ATTR_VALUE`,
166+
`error-validator-condition-now`): it becomes a database CHECK, which can hold neither.
140167

141168
- **An int-backed enum can be an array.** `@intValueMap` together with `isArray: true` (the
142169
map on the field, or inherited from a shared enum) now loads in every port instead of

‎docs/CONFORMANCE.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
2626

2727
| Corpus | Fixtures | TS | Java | Kotlin | C# | Python |
2828
|---|---|---|---|---|---|---|
29-
| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 459 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ |
29+
| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 471 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ |
3030
| [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 16 | 16 / 16 | 15 / 16 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 15 / 16 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 16 / 16 |
3131
| [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 45 | ✓ | ✓ | inherits via Java | ✓ | ✓ |
3232
| [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ |
@@ -354,7 +354,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker.
354354

355355
## Fixture-to-doc mapping
356356

357-
### `fixtures/conformance/` — metamodel loader + canonical serializer (459)
357+
### `fixtures/conformance/` — metamodel loader + canonical serializer (471)
358358

359359
| Fixture prefix | Feature doc |
360360
|---|---|
@@ -716,7 +716,7 @@ JSON and verify payload feeds it unchanged.
716716

717717
## Orphaned fixtures (tested but not yet documented)
718718

719-
The fixtures in the nine corpora mapped above (metamodel 459 + yaml 16 + verify 45
719+
The fixtures in the nine corpora mapped above (metamodel 471 + yaml 16 + verify 45
720720
+ render 18 + persistence 62 + api-contract 123 + source-resolution 25 + scope 10 +
721721
dependency 47) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance`, the
722722
`cube-model` corpus and `explorer-conformance`, whose case lists live in their own READMEs. None are orphaned today. The remaining

‎docs/features/migrations-and-drift.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -430,6 +430,7 @@ attributes shape it, all database-only (no generated code reads them):
430430
it out.
431431
- **`@condition`** on a cross-field validator (`comparison`, `requiredWhen`, `presentIff`,
432432
`atLeastOne`, `exactlyOne`) is an expression, in the grammar `origin.computed` uses, over the entity's fields.
433+
It refuses `case` and `now` (`ERR_BAD_ATTR_VALUE`): a database CHECK can hold neither.
433434
The rule holds only where the condition does: the CHECK is `(NOT condition) OR (rule)`, with
434435
the negation folded in (`kind = 'Owned'` becomes `kind <> 'Owned'`). It is checked at load
435436
(`ERR_BAD_ATTR_VALUE`): every field it names is a field of the entity, it is a boolean

‎docs/features/reporting.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -969,7 +969,7 @@ only beside `now`.
969969

970970
| Rule | Code | Refused |
971971
|---|---|---|
972-
| **F1** | `ERR_BAD_ATTR_FILTER` | A relative value anywhere except the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. In a `layout.dataGrid` preset, a projection `@filter` or an `origin.aggregate` `@filter` it fails the load, because no lowering exists for those hosts. |
972+
| **F1** | `ERR_BAD_ATTR_FILTER` | A relative value anywhere except the `@filter` of a `segment`, a `measure.aggregate`, an `object.report`, an `object.projection`, an `origin.first` or an `origin.aggregate`. In a `layout.dataGrid` preset it fails the load, because no lowering exists for that host. On a projection or an origin, F2 applies in full: a projection's key names one of its own fields, and an origin's key names a field of its `@of` entity, or of its `@via` terminal when it has no `@of`. |
973973
| **F2** | `ERR_BAD_ATTR_FILTER` | A relative value on any field except a `field.date` or `field.timestamp`; under any operator except `gt, gte, lt, lte`; as an object that carries keys besides `now` and `startOf`; with a duration that does not match the ISO-8601 pattern `^[+-]?P(?!$)(\d+Y)?(\d+M)?(\d+W)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+S)?)?$`; or with a `startOf` other than `day`, `week` or `month`. |
974974

975975
An operator-less `{ "occurredAt": { "now": "-P7D" } }` is read as `eq`, and `eq` is not one

‎docs/features/source-kinds.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,14 @@ round out the read-model vocabulary: **`origin.computed`** carries a closed `@ex
196196
expression grammar (a derived scalar computed from other fields), and **`origin.first`** picks
197197
one related row's column along a `@via` path (an argmax-style projection; nullable).
198198

199+
**Expressions and relative dates.** An `origin.computed` `@expr` may use `case`
200+
(`{ "case": [ { "when": …, "then": … } ], "else": … }`, or with an `operand` for the simple
201+
form) and `now` (the current instant), and a `{ "field": … }` ref in it names one of the
202+
projection's own fields first (a passthrough, an `origin.first` or another `origin.computed`,
203+
never itself or an aggregate), else a base field. An `origin.first` or `origin.aggregate`
204+
`@filter`, like a projection's, may hold a relative date (`{ "now": "-P7D" }`, see
205+
[reporting.md](reporting.md#relative-dates-in-filters)). Every port's loader validates these.
206+
199207
**Projection row-scope `@filter` (#207).** Distinct from the *per-aggregate* `@filter` above,
200208
an **object-level `@filter` on an `object.projection`** scopes the whole view's rows — it
201209
lowers to the view's outer `WHERE`. This is the metadata-managed way to express a

‎examples/showcase/site-payload.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
"metamodel": "1.3"
88
},
99
"counts": {
10-
"fixtures": 459,
10+
"fixtures": 471,
1111
"corpora": 38,
1212
"baseTypes": 17
1313
},

‎fixtures/conformance/ERROR-CODES.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@
5252
"ERR_TEMPLATE_NOT_STATIC": "Build-time verify (ADR-0061, Jinja bodies, Python and TypeScript): a template reference the check cannot settle statically — a computed attribute or include name, a dynamic subscript, a call, a tag outside the supported set. Reported rather than passed; path = the expression or tag.",
5353
"ERR_TEMPLATE_SYNTAX": "Build-time verify: the template body, or a template it includes or imports, does not parse. Jinja bodies (ADR-0061, Python and TypeScript): path = the parser's message, which is engine-specific. Mustache bodies (every port): an unbalanced section (a close with no open, a close naming another section, a section never closed); path = the section's name.",
5454
"ERR_TEMPLATE_ENGINE_UNSUPPORTED": "Build-time verify and render (ADR-0061): a template body is written for an engine this port does not ship — a Jinja body (.jinja / .j2 / .jinja2) in the Java, Kotlin or C# port, which render Mustache only. path = the template ref.",
55-
"ERR_BAD_ATTR_FILTER": "An attr.filter references a field that is not addressable in that context or uses a disallowed op: a dataGrid @filter over a non-filterable field or an op disallowed for the field's subtype, or a projection row-scope @filter over a non-declared (dangling) or aggregate-derived field (#207). FR-044: a segment / measure.aggregate / object.report @filter names a field its entity (or @from) lacks or an op illegal for the field (S1); a relative-date value { \"now\": \"<duration>\" } appears outside the @filter of a segment, measure.aggregate or object.report (F1); or a relative-date value sits on a field that is not field.date/field.timestamp, under an op other than gt/gte/lt/lte, carries a value that is not an ISO-8601 duration, or has a startOf other than day, week or month, or a key besides now and startOf (F2).",
55+
"ERR_BAD_ATTR_FILTER": "An attr.filter references a field that is not addressable in that context or uses a disallowed op: a dataGrid @filter over a non-filterable field or an op disallowed for the field's subtype, or a projection row-scope @filter over a non-declared (dangling) or aggregate-derived field (#207). FR-044: a segment / measure.aggregate / object.report @filter names a field its entity (or @from) lacks or an op illegal for the field (S1); a relative-date value { \"now\": \"<duration>\" } appears outside the @filter of a segment, measure.aggregate, object.report, object.projection, origin.aggregate or origin.first (F1); or a relative-date value sits on a field that is not field.date/field.timestamp, under an op other than gt/gte/lt/lte, carries a value that is not an ISO-8601 duration, or has a startOf other than day, week or month, or a key besides now and startOf (F2).",
5656
"ERR_STORAGE_FLATTENED_ARRAY": "@storage \"flattened\" cannot be combined with isArray=true.",
5757
"ERR_SEQUENCE_START_OUT_OF_RANGE": "An identity.primary @sequenceStart is below 1, or above the largest value its key field can hold (2147483647 for a field.int). Postgres would refuse the sequence when the migration is applied, so the model is refused at load.",
5858
"ERR_SEQUENCE_WITHOUT_INCREMENT": "An identity.primary sets @sequenceStart or @sequenceName, but its @generation is not \"increment\". Only an increment key draws from a sequence, so the attribute would change nothing: the identity is refused rather than silently ignored.",
@@ -96,7 +96,7 @@
9696
"ERR_INVALID_REPORT": "FR-044: an object.report is invalid: @from does not resolve to an object.entity (R1); a @dimensions item names no dimension of @from, omits the grain of a time dimension, uses a grain the dimension does not declare, gives a grain to an attribute dimension, or repeats (R2); a @measures item resolves to no measure at all (R3); the report declares a field.* or identity.* child (R4) or a writable source (R5); two items derive the same report field name (R6); or @segment names no segment of @from (R7).",
9797
"ERR_REPORT_FOREIGN_MEASURE": "FR-044: an object.report lists a measure (bare or dotted) that belongs to an entity other than @from (R3). All measures of a report come from @from; combining two fact tables is a second report.",
9898
"ERR_COMPUTED_TYPE_MISMATCH": "#195: an origin.computed @expr tree's inferred root type does not equal the carrying field's declared field.<subType>. A computed column's type is DERIVED from its expression, never asserted (no @convert escape), so a mismatch is a hard load error (sibling of ERR_PASSTHROUGH_TYPE_MISMATCH).",
99-
"ERR_UNKNOWN_EXPR_NODE": "#195: an origin.computed @expr tree contains a node whose kind/op/fn is not in the closed expression grammar (field/value refs, comparisons sharing the filter op vocabulary, isNull/isNotNull, and/or/not, coalesce). Fail-closed per ADR-0023; the detail names the offending token.",
99+
"ERR_UNKNOWN_EXPR_NODE": "#195: an origin.computed @expr tree contains a node whose kind/op/fn is not in the closed expression grammar (field/value refs, comparisons sharing the filter op vocabulary, isNull/isNotNull, and/or/not, coalesce, case, now), or a node carries a key its kind does not take (a misspelled `else`, or a `field` node that also carries `case`). Fail-closed per ADR-0023; the detail names the offending token.",
100100
"ERR_SQL_BODY_WITH_UNMANAGED": "#208: a source.rdb declares both @sql (author-supplied body) and @unmanaged (DDL owned elsewhere). The two markers are the mutually exclusive non-default states of one DDL-ownership axis — contradictory on a single source.",
101101
"ERR_SQL_BODY_ON_WRITABLE_KIND": "#208: a source.rdb declares @sql with a writable @kind (\"table\", the default). A hand-written CREATE TABLE would bypass the column-diff machinery; tables are fully modeled or @unmanaged, never opaque-bodied.",
102102
"ERR_ORIGIN_UNDER_SQL_BODY": "#208: a host object whose read source carries @sql also declares an origin.*-bearing (derived) field, or (on object.projection) an @filter (#207) — two sources of truth for the same body (the synthesized derivation/outer-WHERE vs. the author's verbatim SQL). Fail-closed.",
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"errors": [
3+
{
4+
"code": "ERR_INVALID_ORIGIN",
5+
"source": {
6+
"format": "json",
7+
"files": [
8+
"meta.views.json"
9+
],
10+
"jsonPath": "$['metadata.root'].children[2]['object.projection'].children[2]['field.string'].children[0]['origin.computed']"
11+
}
12+
}
13+
],
14+
"warnings": []
15+
}

0 commit comments

Comments
 (0)