feat(spec): list-view grouping is server-side — compile the group header query and the per-group row page from the view

.changeset/list-view-grouping-server-side-contract.md

@@ -0,0 +1,63 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): list-view grouping is server-side — the group header query and the per-group row page compile from the view (#14556)
+
+Maintainer ruling A on objectui#7189 (2026-09-02): grouping on a list view is
+server-side. The set of groups and every number in a group header — the count
+and any per-group aggregation — are properties of the query, not of the fetched
+page; rows inside a group are paged. Grouping one fetched window (the interim
+behaviour) rendered two headers (86, 14) or five (31/31/30/7/1) for the same
+186 rows in five units depending on row order, and left the rows past the
+first window unreachable.
+
+The contract reuses the query shapes the platform already has — no new query
+shape, no new engine verb, no new envelope:
+
+1. **The group keys and every header number are ONE aggregate query**
+   (`EngineAggregateOptions`, executed by `IDataEngine.aggregate`): `groupBy`
+   is `grouping.fields[].field` in nesting order (a multi-level grouping is a
+   multi-column `groupBy`), `aggregations` is a `count` node (the group's total
+   row count, alias `count`) plus the view's declared column summaries mapped
+   onto `AggregationFunction` — the one aggregation vocabulary datasets already
+   use — and `where` is the view's composed filter.
+2. **The rows inside a group are the existing paged `find`**
+   (`EngineQueryOptions`) with the group's key predicate AND-ed into the view
+   filter, `limit` / `offset` per group.
+
+New on the `ui` entry, `view-grouping-query.ts`:
+
+- `compileListViewGroupQuery(view, { where?, depth? })` → the header query;
+  `compileListViewGroupRowsQuery(view, groupKey, { where?, limit?, offset?, orderBy?, fields? })`
+  → the row page; `listViewGroupKeyPredicate` (the empty group is spelled with
+  the `$null` predicate — the spelling the view filter dialect's `is_empty`
+  lowers to).
+- `COLUMN_SUMMARY_AGGREGATION` — the `ColumnSummary` → aggregation table,
+  exhaustive by type: `count` → a fieldless `count` (`COUNT(*)`),
+  `count_unique` → `count_distinct`, `sum` / `avg` / `min` / `max` → the same
+  name, `none` → nothing; `count_filled` / `count_empty` / `percent_filled` /
+  `percent_empty` map by derivation — one `{ function: 'count', field }` node
+  (`COUNT(field)`, the non-null count, header column `count_<field>`), from
+  which `deriveColumnSummary(row, summary, field)` computes all four on the
+  header row (`count_filled` = `count_<field>`, `count_empty` = `count −
+  count_<field>`, `percent_filled` = `count_<field> / count`, 0 when the count
+  is 0, `percent_empty` = `1 − percent_filled`). Server-side "empty" is `null`
+  on every face; the footer's client-side reading of `''` / `[]` as empty is
+  the renderer's to converge. A future member with no counterpart is refused
+  loudly at compile time (`ListViewGroupQueryError`, `NOT_IMPLEMENTED` / 501,
+  the summary's path — `UNMAPPED_COLUMN_SUMMARIES`, empty today); a value that
+  is no member at all is `INVALID_QUERY` / 400.
+- Result-column naming on a header row: each grouped field under its own name
+  (raw stored value, `null` for the empty group; group keys are scalar), `count`,
+  and each summary under `<function>_<field>` (`columnSummaryAlias`).
+
+`GroupingConfigSchema` / `GroupingFieldSchema` / `ColumnSummarySchema` now say
+this in their docs, with the shape's recorded limits (a date grouping field
+groups per distinct stored instant; header cardinality is unbounded). Nothing
+changes in what parses: no key is added, removed or re-shaped. `minor` because
+a new exported helper and a declared contract semantics ship; not breaking —
+the page-scoped behaviour was never declared. Both queries ride the existing
+`POST /data/:object/query` door (`protocol.findData` → `engine.aggregate`,
+answering `{ object, records, total, hasMore }`); the grid consuming the header
+rows is objectui#7189.

content/docs/references/api/protocol.mdx

@@ -1639,7 +1639,7 @@ The published metadata item body, opaque by ruling (1C). Shape is the item's own
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -1724,7 +1724,7 @@ The published metadata item body, opaque by ruling (1C). Shape is the item's own
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |

content/docs/references/data/object.mdx

@@ -386,7 +386,7 @@ const result = ApiMethod.parse(data);
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |

content/docs/references/ui/view.mdx

@@ -127,7 +127,7 @@ Compound-cell prefix configuration
 
 ## ColumnSummary
 
-Aggregation function for column footer summary
+Aggregation function for the column footer summary — and, on a grouped list view, the per-group header summary (server-side): count (COUNT(*), the group count), count_unique (count_distinct), sum, avg, min, max map onto the query AST's AggregationFunction; count_filled, count_empty, percent_filled, percent_empty derive from one COUNT(field) node (the non-null count) and the group count — count_filled = COUNT(field), count_empty = count − COUNT(field), percent_filled = COUNT(field) / count (0 when count is 0), percent_empty = 1 − percent_filled. Server-side "empty" is null on every face; the footer's client-side reading of empty strings and empty arrays as empty is the renderer's to converge
 
 ### Allowed Values
 
@@ -600,21 +600,21 @@ Gallery/card view configuration
 
 ## GroupingConfig
 
-Record grouping configuration
+Record grouping configuration — SERVER-SIDE: the set of groups and every number in a group header (the count and the per-column summaries) are properties of the query, not of the fetched page, answered by one aggregate query (`groupBy` = the fields in nesting order, `count` + the mapped column summaries, the view filter); rows inside a group are paged by the existing find with the group key AND-ed into the view filter. Compiled by `compileListViewGroupQuery` / `compileListViewGroupRowsQuery`
 
 ### Properties
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **fields** | `{ field: string; order: Enum<'asc' \| 'desc'>; collapsed: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) |
+| **fields** | `{ field: string; order: Enum<'asc' \| 'desc'>; collapsed: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` |
 
 ### Nested Shape: `GroupingConfig.fields[number]`
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **field** | `string` | ✅ | Field name to group by |
-| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order |
-| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default |
+| **field** | `string` | ✅ | Field name to group by — one `groupBy` column of the group header query; the header row carries its raw stored value (null for the empty group) |
+| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order — applied by the consumer over the header rows (the aggregate query carries no orderBy) |
+| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default (presentation only) |
 
 
 ---
@@ -625,9 +625,9 @@ Record grouping configuration
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **field** | `string` | ✅ | Field name to group by |
-| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order |
-| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default |
+| **field** | `string` | ✅ | Field name to group by — one `groupBy` column of the group header query; the header row carries its raw stored value (null for the empty group) |
+| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order — applied by the consumer over the header rows (the aggregate query carries no orderBy) |
+| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default (presentation only) |
 
 
 ---
@@ -783,7 +783,7 @@ Map view configuration
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -1003,7 +1003,7 @@ View filter rule
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) |
+| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` |
 
 ### Nested Shape: `ListView.rowColor`
 
@@ -1178,7 +1178,7 @@ Tab configuration for multi-tab view interface
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -1389,7 +1389,7 @@ View filter rule
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) |
+| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` |
 
 ### Nested Shape: `ObjectListView.rowColor`
 
@@ -1764,7 +1764,7 @@ Tab configuration for multi-tab view interface
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -1849,7 +1849,7 @@ Tab configuration for multi-tab view interface
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -2090,7 +2090,7 @@ This schema accepts one of the following structures:
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |
@@ -2266,7 +2266,7 @@ This schema accepts one of the following structures:
 | **description** | `string \| Record<string, string>` | optional | View description for documentation/tooltips |
 | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration |
 | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting |
-| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields |
+| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) |
 | **rowColor** | `{ field: string; colors?: Record<string, string> }` | optional | Color rows based on field value |
 | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view |
 | **fieldOrder** | `string[]` | optional | Explicit field display order for this view |