Skip to content

Commit 18d7cd9

Browse files
dmealingclaude
andcommitted
merge: the whole-object rollup into the 0.24.1 line (#335)
Absorbed into 0.24.1 rather than held for 0.25.0, per Doug's ruling, and the cut stays a PATCH. Gates green on all four ports at d98c0b9 (--quick, ts-slow, java, csharp, python). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DhpswkF1NvwxhFWMmdAT15
2 parents 128485f + d98c0b9 commit 18d7cd9

78 files changed

Lines changed: 5803 additions & 164 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,86 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
77

88
## [Unreleased]
99

10+
### Added — `@of` becomes optional on `@agg: collect`: the whole-object rollup ([#335](https://github.com/metaobjectsdev/metaobjects/issues/335))
11+
12+
**A projection could roll related rows up into an array of one COLUMN, and had no way to
13+
roll them up into an array of OBJECTS.** `origin.aggregate @agg: collect` required `@of`, so
14+
"every supplier's `{id, name}` for this product" was inexpressible — the shape had to be a
15+
second round-trip, or a hand-written view, which is unmanaged and invisible to
16+
`meta verify --db`. When `origin.collection` retired in `0.24.0` this became the one
17+
coverage gap the retirement guide had to state rather than close.
18+
19+
**`@of` is now OPTIONAL on `collect`, and omitting it means a whole-object rollup:** the
20+
carrying `field.object @isArray @objectRef` collects each related row as its declared value
21+
object.
22+
23+
```jsonc
24+
{ "field.object": {
25+
"name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief",
26+
"children": [
27+
{ "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } }
28+
]
29+
}}
30+
```
31+
32+
**The declared value object IS the exposure.** Members bind to the `@via` terminal entity's
33+
fields BY NAME, and a field the entity has but the value object omits is simply not
34+
projected. That is deliberate — it is the [#270](https://github.com/metaobjectsdev/metaobjects/issues/270)
35+
guarantee (a curated value object must not silently become the full entity) carried down to
36+
the DDL tier. Name matching, rather than `extends`, is also deliberate: it keeps one value
37+
object collectable from two different entities, which `extends` would forbid. The convention
38+
is written into the byte-gated `origin.aggregate` registry prose so no port has to infer it.
39+
40+
**Eight load errors, in all five ports.** The carrying field must be a `field.object`
41+
declaring `@objectRef`; that `@objectRef` must name an `object.value`; `@via` is required
42+
(there is no `@of` entity to infer a single hop from); the path must be to-many; `@distinct`
43+
is refused; `@orderBy` keys must resolve against the `@via` **terminal** entity; every value
44+
object member must match a terminal field; and a matched member must agree on **both** type
45+
axes — subtype and array-ness. `@distinct` is refused by CHOICE, not engine limit: it works
46+
on both engines, but it is a guaranteed no-op whenever the value object carries the primary
47+
key, and a silent no-op is worse than a refusal.
48+
49+
**A new error code, `ERR_COLLECT_WHOLE_OBJECT`,** carries the five refusals that would
50+
otherwise have shared `ERR_INVALID_ORIGIN`. That is not taxonomy for its own sake: the shared
51+
corpus compares error **code + source** and never message text, and `ERR_INVALID_ORIGIN` is
52+
exactly what a loader that still *requires* `@of` emits for this same metadata — so five of
53+
the eight negative fixtures passed against three ports containing none of the rules. With the
54+
distinct code they fail, and Task-by-task porting has a real signal.
55+
56+
**View lowering, both dialects.** Postgres emits
57+
`COALESCE(jsonb_agg(jsonb_build_object(…) ORDER BY <pk> ASC) FILTER (WHERE <pk> IS NOT NULL), '[]'::jsonb)`.
58+
`jsonb`, not `json`: PG's `json` type has neither an equality nor an ordering operator, so
59+
the `json_agg(json_build_object(… ORDER BY …))` form does not run at all. Default element
60+
order is the related entity's **primary key** ascending — ordering rows by a serialized
61+
object is meaningless — and an explicit `@orderBy` leads with the PK appended as a tie-break.
62+
The scalar `@of` arm deliberately keeps its existing no-tie-break behaviour, since changing
63+
it would move the emitted SQL of every project already using `@orderBy`.
64+
65+
**SQLite needed a shape nobody would have guessed, and only a real engine found it.** On
66+
SQLite 3.44 (D1's pinned baseline) the in-aggregate `ORDER BY` clause **destroys the JSON
67+
subtype**: `json_group_array(json_object(…) ORDER BY …)` returns an array of quoted STRINGS
68+
rather than objects, and wrapping the argument in `json()` does not survive it either.
69+
Dropping the `ORDER BY` was not an option — element order would stop being deterministic and
70+
an author's `@orderBy` would silently do nothing. So the ordered array is built first and
71+
re-wrapped element-by-element through `json_each`, which iterates in array order. The
72+
emitted SQL text alone could never have shown this; it was caught by the emit → apply →
73+
introspect → re-diff round-trip against a real engine, which is the standing rule that
74+
golden SQL is not evidence for new DDL.
75+
76+
Also worth knowing: inside the rollup a `field.long` member arrives as a JSON **number**,
77+
while the same value as a top-level `BIGINT` column arrives as a string from
78+
node-postgres. That is inherent to JSON, not a codegen choice, and it is lossy above 2^53.
79+
80+
**Also in this change — array fields are not filterable or sortable.** A
81+
`field.<scalar> isArray: true` carrying `@filterable: true` or `@sortable: true` is now a
82+
load error in all five ports (`ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` /
83+
`ERR_SORTABLE_UNSUPPORTED_SUBTYPE`): no operator in the FR-009 scalar band applies to a
84+
collection column, and no dialect can `ORDER BY` one. `@sortable` also gains the subtype
85+
validation `@filterable` already had.
86+
87+
`metamodelVersion` moves `0.11` → `0.12`.
88+
89+
1090
## [0.24.1] — npm `0.24.1` · PyPI `0.24.1` · NuGet `0.24.1` · Maven `7.24.1`
1191

1292
### Fixed — an expression index was undeclarable, and the one spelling that loaded was half-ignored ([#342](https://github.com/metaobjectsdev/metaobjects/issues/342))

‎agent-context/skills/metaobjects-authoring/SKILL.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -838,8 +838,16 @@ amendment 2026-08-06.)
838838

839839
**Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max`
840840
(numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of`
841-
forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of`
842-
into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be
841+
forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup
842+
into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one
843+
`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or
844+
omit `@of` on a `field.object @objectRef` to collect each related row as that declared
845+
value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))`
846+
on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is
847+
a no-op whenever the value object carries the primary key), and requires every value-object
848+
member to match a field on the `@via` **terminal** entity by name, with the same subtype
849+
and array-ness. The declared value object IS the exposure: a field the entity has and the
850+
value object omits is not projected. Any aggregate may be
843851
row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a
844852
derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`,
845853
ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) —

‎docs/features/downstream-metadata-decisions.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,13 @@ Most "I need a new type" needs die here. Before you register anything:
2929
- **Is it a one-off author-supplied property?** The registered `attr.properties` bag
3030
is the sanctioned escape hatch for arbitrary key/values — it does not require a new
3131
attribute and stays inside strict provenance (ADR-0023).
32+
- **Do you need an array of OBJECTS on a projection?** That is
33+
`origin.aggregate @agg: collect` with **`@of` omitted** — the whole-object rollup
34+
(#335), on a `field.object @isArray @objectRef`. It is the case most often
35+
mistaken for missing vocabulary, because the `@of` form reads as the only form and
36+
the natural next thought is a hand-written view. Its declared value object is the
37+
exposure, and the `CREATE VIEW` is generated — a hand-written one is unmanaged and
38+
therefore invisible to `meta verify --db`.
3239

3340
## Step 1 — check whether core (or the roadmap) already models it
3441

‎docs/features/migrations/origin-collection-retirement.md‎

Lines changed: 43 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -59,10 +59,16 @@ gain `@filter`, `@orderBy` and `@distinct`, none of which `collection` could exp
5959
`collect` preserves the **element** type: the array field's own `field.<subType>` must equal
6060
the `@of` column's, and the field must be `isArray: true`.
6161

62-
### 2. A whole-object rollup → delete the child
62+
### 2. A whole-object rollup → depends on the HOST
6363

64-
If the field collected whole nested value objects (`field.object @objectRef … isArray: true`),
65-
**delete the `origin.collection` child and change nothing else**:
64+
What to do splits by where the field lives, and getting it wrong on a projection is
65+
**silent** — read both cases before editing.
66+
67+
#### 2a. On a payload / `object.value` host → delete the child
68+
69+
If the field collected whole nested value objects (`field.object @objectRef … isArray: true`)
70+
and its host is a payload or value object, **delete the `origin.collection` child and change
71+
nothing else**:
6672

6773
```jsonc
6874
{ "field.object": {
@@ -78,15 +84,40 @@ The declared shape is unchanged, and payload typing has been **declared-authorit
7884
0.20.16** (#270) — the type came from `field.object` + `isArray` + `@objectRef`, never from the
7985
origin. Generated payload records, output parsers and render helpers are byte-identical.
8086

81-
**State it plainly, because it is the one real gap:** no surviving origin expresses a
82-
whole-object rollup along a relationship — `@agg: collect` reduces a *column* via `@of`. If
83-
that field was on a `source.rdb @kind: view` **projection** and you were relying on the view
84-
DDL, note that no port ever emitted DDL for `origin.collection` either, so nothing regresses;
85-
but you also cannot now declare the provenance. That shape returns with
86-
[#335](https://github.com/metaobjectsdev/metaobjects/issues/335), which makes `@of` **optional**
87-
on `@agg: collect` (absent = whole-object rollup, typed by the declared `@objectRef` +
88-
`isArray`, never derived from the `@via` target) and ships the view lowering with it. It is
89-
**additive**, so it needs no breaking window.
87+
#### 2b. On a view-kind PROJECTION host → replace the child, never delete it
88+
89+
**Deleting the child here produces metadata that loads clean and generates a broken view.**
90+
A projection field with no `origin.*` is treated as a plain base column, so the emitted DDL
91+
becomes `SELECT base."supplierBriefs"` against a column that does not exist. The load says
92+
nothing; the failure surfaces at apply.
93+
94+
Replace the child instead:
95+
96+
```jsonc
97+
{ "field.object": {
98+
"name": "supplierBriefs", "isArray": true,
99+
"@objectRef": "acme::catalog::SupplierBrief",
100+
"children": [
101+
- { "origin.collection": { "@via": "acme::catalog::Product.suppliers" } }
102+
+ { "origin.aggregate": {
103+
+ "@agg": "collect",
104+
+ "@via": "acme::catalog::Product.suppliers" } }
105+
]
106+
}}
107+
```
108+
109+
That is [#335](https://github.com/metaobjectsdev/metaobjects/issues/335)'s whole-object
110+
rollup: `@of` is **optional** on `@agg: collect`, and omitting it means "collect the related
111+
rows as the declared `@objectRef` value object". The element type comes from the declared
112+
`@objectRef` + `isArray`, never from the `@via` target (#270), and the view lowering ships
113+
with it — `jsonb_agg(jsonb_build_object(…))` on Postgres, `json_group_array(json_object(…))`
114+
on SQLite, `[]` on an empty related set.
115+
116+
Three constraints the old `origin.collection` did not have: `@via` is **required** (there is
117+
no `@of` entity to infer a single hop from), `@distinct` is **refused** (it is a guaranteed
118+
no-op whenever the value object carries the primary key), and every value-object member must
119+
match a field on the `@via` **terminal** entity by name, with the same `field.<subType>` and
120+
array-ness. Each is a load error, so a mistake here fails loudly rather than silently.
90121

91122
### 3. `ASSEMBLY_ORIGIN_SUBTYPES` shrinks to three
92123

0 commit comments

Comments
 (0)