From 79f007c5ba22fca5bbf5b82d869d85d21c64afe3 Mon Sep 17 00:00:00 2001 From: os-justin Date: Fri, 4 Sep 2026 07:55:59 +0000 Subject: [PATCH 1/4] =?UTF-8?q?feat(spec):=20list-view=20grouping=20is=20s?= =?UTF-8?q?erver-side=20=E2=80=94=20compile=20the=20group=20header=20query?= =?UTF-8?q?=20and=20the=20per-group=20row=20page=20from=20the=20view=20(#1?= =?UTF-8?q?4556)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer ruling A on objectui#7189: the set of groups and every number in a group header are properties of the query, not of the fetched page; rows inside a group are paged. Seat ruling: reuse, no new query shape. - `GroupingConfigSchema` / `GroupingFieldSchema` / `ColumnSummarySchema` / `ListView.grouping` state the contract in JSDoc and `.describe()`. - New `ui/view-grouping-query.ts`: `compileListViewGroupQuery` (one `EngineAggregateOptions` — `groupBy` in nesting order, a `count` node, the column summaries mapped onto `AggregationFunction`, the view filter) and `compileListViewGroupRowsQuery` (the existing paged find with the group key AND-ed into the view filter; the empty group spelled with `$null`). `COLUMN_SUMMARY_AGGREGATION` is exhaustive by type; `count_empty` / `count_filled` / `percent_empty` / `percent_filled` have no counterpart and refuse loudly (`NOT_IMPLEMENTED` / 501 + path) until the mapping is ruled. - Pins on the 186-row / five-unit / `$top: 100` fixture: the page-scoped artefacts (86, 14) and (31/31/30/7/1) reproduce, the compiled header query answers 86/61/31/7/1 in both row orders. - Generated followers regenerated by `check:generated --fix`; `QueryInput` re-exported on the `ui` entry for entry-nameability. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01H2oQebDDxYKfWZusyd8GXk --- ...list-view-grouping-server-side-contract.md | 53 ++ content/docs/references/api/protocol.mdx | 4 +- content/docs/references/data/object.mdx | 2 +- content/docs/references/ui/view.mdx | 34 +- packages/spec/api-surface/ui.json | 15 + packages/spec/export-origins/ui.json | 15 + packages/spec/src/ui/index.ts | 11 + .../spec/src/ui/view-grouping-query.test.ts | 485 +++++++++++++++++ packages/spec/src/ui/view-grouping-query.ts | 490 ++++++++++++++++++ packages/spec/src/ui/view.zod.ts | 97 +++- .../contracts/react-blocks.contract.json | 2 +- .../objectstack-ui/references/react-blocks.md | 2 +- 12 files changed, 1180 insertions(+), 30 deletions(-) create mode 100644 .changeset/list-view-grouping-server-side-contract.md create mode 100644 packages/spec/src/ui/view-grouping-query.test.ts create mode 100644 packages/spec/src/ui/view-grouping-query.ts diff --git a/.changeset/list-view-grouping-server-side-contract.md b/.changeset/list-view-grouping-server-side-contract.md new file mode 100644 index 0000000000..cad28911f1 --- /dev/null +++ b/.changeset/list-view-grouping-server-side-contract.md @@ -0,0 +1,53 @@ +--- +"@objectstack/spec": minor +--- + +feat(spec): list-view grouping is server-side — the group header query and the per-group row page compile from the view (#14556) + +Maintainer ruling A on objectui#7189 (2026-09-02): grouping on a list view is +server-side. The set of groups and every number in a group header — the count +and any per-group aggregation — are properties of the query, not of the fetched +page; rows inside a group are paged. Grouping one fetched window (the interim +behaviour) rendered two headers (86, 14) or five (31/31/30/7/1) for the same +186 rows in five units depending on row order, and left the rows past the +first window unreachable. + +The contract reuses the query shapes the platform already has — no new query +shape, no new engine verb, no new envelope: + +1. **The group keys and every header number are ONE aggregate query** + (`EngineAggregateOptions`, executed by `IDataEngine.aggregate`): `groupBy` + is `grouping.fields[].field` in nesting order (a multi-level grouping is a + multi-column `groupBy`), `aggregations` is a `count` node (the group's total + row count, alias `count`) plus the view's declared column summaries mapped + onto `AggregationFunction` — the one aggregation vocabulary datasets already + use — and `where` is the view's composed filter. +2. **The rows inside a group are the existing paged `find`** + (`EngineQueryOptions`) with the group's key predicate AND-ed into the view + filter, `limit` / `offset` per group. + +New on the `ui` entry, `view-grouping-query.ts`: + +- `compileListViewGroupQuery(view, { where?, depth? })` → the header query; + `compileListViewGroupRowsQuery(view, groupKey, { where?, limit?, offset?, orderBy?, fields? })` + → the row page; `listViewGroupKeyPredicate` (the empty group is spelled with + the `$null` predicate, never `$eq: null`). +- `COLUMN_SUMMARY_AGGREGATION` — the `ColumnSummary` → `AggregationFunction` + table, exhaustive by type: `count` → a fieldless `count` (`COUNT(*)`), + `count_unique` → `count_distinct`, `sum` / `avg` / `min` / `max` → the same + name, `none` → nothing. `count_empty`, `count_filled`, `percent_empty` and + `percent_filled` have no counterpart yet (`UNMAPPED_COLUMN_SUMMARIES`); a + grouped view declaring one is refused loudly at compile time with + `ListViewGroupQueryError` (`NOT_IMPLEMENTED` / 501, the summary's path) — + their mapping is an open contract question on #14556, and nothing is dropped + silently in the meantime. +- Result-column naming on a header row: each grouped field under its own name + (raw stored value, `null` for the empty group), `count`, and each summary + under `_` (`columnSummaryAlias`). + +`GroupingConfigSchema` / `GroupingFieldSchema` / `ColumnSummarySchema` now say +this in their docs. Nothing changes in what parses: no key is added, removed +or re-shaped. `minor` because a new exported helper and a declared contract +semantics ship; not breaking — the page-scoped behaviour was never declared. +The route that carries the header query to the grid is the platform half of +#14556; the grid consuming it is objectui#7189. diff --git a/content/docs/references/api/protocol.mdx b/content/docs/references/api/protocol.mdx index 1a123c8985..5435ac2bce 100644 --- a/content/docs/references/api/protocol.mdx +++ b/content/docs/references/api/protocol.mdx @@ -1639,7 +1639,7 @@ The published metadata item body, opaque by ruling (1C). Shape is the item's own | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -1724,7 +1724,7 @@ The published metadata item body, opaque by ruling (1C). Shape is the item's own | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | diff --git a/content/docs/references/data/object.mdx b/content/docs/references/data/object.mdx index 5cba52d166..ea105ef57e 100644 --- a/content/docs/references/data/object.mdx +++ b/content/docs/references/data/object.mdx @@ -386,7 +386,7 @@ const result = ApiMethod.parse(data); | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | diff --git a/content/docs/references/ui/view.mdx b/content/docs/references/ui/view.mdx index 5d9e15e5f4..e88861387e 100644 --- a/content/docs/references/ui/view.mdx +++ b/content/docs/references/ui/view.mdx @@ -127,7 +127,7 @@ Compound-cell prefix configuration ## ColumnSummary -Aggregation function for column footer summary +Aggregation function for the column footer summary — and, on a grouped list view, the per-group header summary (server-side): count (COUNT(*), the group count), count_unique (count_distinct), sum, avg, min, max map onto the query AST's AggregationFunction; count_empty, count_filled, percent_empty, percent_filled have no counterpart yet and are refused loudly by the group-header compiler (an open contract question) — never dropped silently ### Allowed Values @@ -600,21 +600,21 @@ Gallery/card view configuration ## GroupingConfig -Record grouping configuration +Record grouping configuration — SERVER-SIDE: the set of groups and every number in a group header (the count and the per-column summaries) are properties of the query, not of the fetched page, answered by one aggregate query (`groupBy` = the fields in nesting order, `count` + the mapped column summaries, the view filter); rows inside a group are paged by the existing find with the group key AND-ed into the view filter. Compiled by `compileListViewGroupQuery` / `compileListViewGroupRowsQuery` ### Properties | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **fields** | `{ field: string; order: Enum<'asc' \| 'desc'>; collapsed: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) | +| **fields** | `{ field: string; order: Enum<'asc' \| 'desc'>; collapsed: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` | ### Nested Shape: `GroupingConfig.fields[number]` | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **field** | `string` | ✅ | Field name to group by | -| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order | -| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default | +| **field** | `string` | ✅ | Field name to group by — one `groupBy` column of the group header query; the header row carries its raw stored value (null for the empty group) | +| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order — applied by the consumer over the header rows (the aggregate query carries no orderBy) | +| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default (presentation only) | --- @@ -625,9 +625,9 @@ Record grouping configuration | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **field** | `string` | ✅ | Field name to group by | -| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order | -| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default | +| **field** | `string` | ✅ | Field name to group by — one `groupBy` column of the group header query; the header row carries its raw stored value (null for the empty group) | +| **order** | `Enum<'asc' \| 'desc'>` | optional (default: `"asc"`) | Group sort order — applied by the consumer over the header rows (the aggregate query carries no orderBy) | +| **collapsed** | `boolean` | optional (default: `false`) | Collapse groups by default (presentation only) | --- @@ -783,7 +783,7 @@ Map view configuration | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -1003,7 +1003,7 @@ View filter rule | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) | +| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` | ### Nested Shape: `ListView.rowColor` @@ -1178,7 +1178,7 @@ Tab configuration for multi-tab view interface | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -1389,7 +1389,7 @@ View filter rule | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field) | +| **fields** | `{ field: string; order?: Enum<'asc' \| 'desc'>; collapsed?: boolean }[]` | ✅ | Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query's `groupBy` | ### Nested Shape: `ObjectListView.rowColor` @@ -1764,7 +1764,7 @@ Tab configuration for multi-tab view interface | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -1849,7 +1849,7 @@ Tab configuration for multi-tab view interface | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -2090,7 +2090,7 @@ This schema accepts one of the following structures: | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | @@ -2266,7 +2266,7 @@ This schema accepts one of the following structures: | **description** | `string \| Record` | optional | View description for documentation/tooltips | | **sharing** | `{ type?: Enum<'personal' \| 'collaborative'>; lockedBy?: string }` | optional | View sharing and access configuration | | **rowHeight** | `Enum<'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'>` | optional | Row height / density setting | -| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields | +| **grouping** | `{ fields: object[] }` | optional | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema) | | **rowColor** | `{ field: string; colors?: Record }` | optional | Color rows based on field value | | **hiddenFields** | `string[]` | optional | Fields to hide in this specific view | | **fieldOrder** | `string[]` | optional | Explicit field display order for this view | diff --git a/packages/spec/api-surface/ui.json b/packages/spec/api-surface/ui.json index 17a8988936..85509cb65c 100644 --- a/packages/spec/api-surface/ui.json +++ b/packages/spec/api-surface/ui.json @@ -57,6 +57,7 @@ "BulkActionParam (type)", "BulkActionParamSchema (const)", "CHART_AGGREGATE_COMPARISON_SUFFIX (const)", + "COLUMN_SUMMARY_AGGREGATION (const)", "CalendarConfig (type)", "CalendarConfigSchema (const)", "ChartAggregate (type)", @@ -89,9 +90,12 @@ "ColumnPrefixParsed (type)", "ColumnPrefixSchema (const)", "ColumnSummary (type)", + "ColumnSummaryAggregation (type)", "ColumnSummaryConfig (type)", "ColumnSummaryConfigSchema (const)", "ColumnSummarySchema (const)", + "CompileListViewGroupQueryOptions (interface)", + "CompileListViewGroupRowsQueryOptions (interface)", "ComponentNavItem (type)", "ComponentNavItemParsed (type)", "ComponentNavItemSchema (const)", @@ -205,6 +209,7 @@ "KNOWN_COMPONENT_TYPE_CANDIDATES (const)", "KanbanConfig (type)", "KanbanConfigSchema (const)", + "LIST_VIEW_GROUP_COUNT_ALIAS (const)", "ListChartConfig (type)", "ListChartConfigParsed (type)", "ListChartConfigSchema (const)", @@ -214,6 +219,10 @@ "ListMapConfig (type)", "ListMapConfigSchema (const)", "ListView (type)", + "ListViewGroupHeaderRow (interface)", + "ListViewGroupQueryError (class)", + "ListViewGroupQueryRefusal (type)", + "ListViewGroupQuerySource (interface)", "ListViewParsed (type)", "ListViewSchema (const)", "NavigationArea (type)", @@ -281,6 +290,7 @@ "PaginationConfig (type)", "PaginationConfigParsed (type)", "PaginationConfigSchema (const)", + "QueryInput (type)", "REACT_BLOCKS (const)", "REACT_OVERLAY_SHADOWS (const)", "REACT_RECORD_BLOCK_ALTERNATIVES (const)", @@ -341,6 +351,7 @@ "TimelineConfigSchema (const)", "TreeConfig (type)", "TreeConfigSchema (const)", + "UNMAPPED_COLUMN_SUMMARIES (const)", "UrlNavItem (type)", "UrlNavItemParsed (type)", "UrlNavItemSchema (const)", @@ -403,6 +414,9 @@ "chartAggregateCategoryKey (function)", "chartAggregateResultKeys (function)", "chartAggregateValueKey (function)", + "columnSummaryAlias (function)", + "compileListViewGroupQuery (function)", + "compileListViewGroupRowsQuery (function)", "dashboardForm (const)", "datasetForm (const)", "defineAction (function)", @@ -424,6 +438,7 @@ "isKnownComponentType (function)", "isRecordContextBlockType (function)", "isViewContainerShaped (function)", + "listViewGroupKeyPredicate (function)", "normalizeFilterOperator (function)", "normalizeInlineAction (function)", "pageForm (const)", diff --git a/packages/spec/export-origins/ui.json b/packages/spec/export-origins/ui.json index acb6eaa3f7..c5f9e80beb 100644 --- a/packages/spec/export-origins/ui.json +++ b/packages/spec/export-origins/ui.json @@ -57,6 +57,7 @@ "BulkActionParam": "src/ui/bulk-action.zod.ts#BulkActionParam (type)", "BulkActionParamSchema": "src/ui/bulk-action.zod.ts#BulkActionParamSchema (const)", "CHART_AGGREGATE_COMPARISON_SUFFIX": "src/ui/chart-aggregate.ts#CHART_AGGREGATE_COMPARISON_SUFFIX (const)", + "COLUMN_SUMMARY_AGGREGATION": "src/ui/view-grouping-query.ts#COLUMN_SUMMARY_AGGREGATION (const)", "CalendarConfig": "src/ui/view.zod.ts#CalendarConfig (type)", "CalendarConfigSchema": "src/ui/view.zod.ts#CalendarConfigSchema (const)", "ChartAggregate": "src/ui/chart.zod.ts#ChartAggregate (type)", @@ -89,9 +90,12 @@ "ColumnPrefixParsed": "src/ui/view.zod.ts#ColumnPrefixParsed (type)", "ColumnPrefixSchema": "src/ui/view.zod.ts#ColumnPrefixSchema (const)", "ColumnSummary": "src/ui/view.zod.ts#ColumnSummary (type)", + "ColumnSummaryAggregation": "src/ui/view-grouping-query.ts#ColumnSummaryAggregation (type)", "ColumnSummaryConfig": "src/ui/view.zod.ts#ColumnSummaryConfig (type)", "ColumnSummaryConfigSchema": "src/ui/view.zod.ts#ColumnSummaryConfigSchema (const)", "ColumnSummarySchema": "src/ui/view.zod.ts#ColumnSummarySchema (const)", + "CompileListViewGroupQueryOptions": "src/ui/view-grouping-query.ts#CompileListViewGroupQueryOptions (interface)", + "CompileListViewGroupRowsQueryOptions": "src/ui/view-grouping-query.ts#CompileListViewGroupRowsQueryOptions (interface)", "ComponentNavItem": "src/ui/app.zod.ts#ComponentNavItem (type)", "ComponentNavItemParsed": "src/ui/app.zod.ts#ComponentNavItemParsed (type)", "ComponentNavItemSchema": "src/ui/app.zod.ts#ComponentNavItemSchema (const)", @@ -205,6 +209,7 @@ "KNOWN_COMPONENT_TYPE_CANDIDATES": "src/ui/component-type-vocabulary.ts#KNOWN_COMPONENT_TYPE_CANDIDATES (const)", "KanbanConfig": "src/ui/view.zod.ts#KanbanConfig (type)", "KanbanConfigSchema": "src/ui/view.zod.ts#KanbanConfigSchema (const)", + "LIST_VIEW_GROUP_COUNT_ALIAS": "src/ui/view-grouping-query.ts#LIST_VIEW_GROUP_COUNT_ALIAS (const)", "ListChartConfig": "src/ui/view.zod.ts#ListChartConfig (type)", "ListChartConfigParsed": "src/ui/view.zod.ts#ListChartConfigParsed (type)", "ListChartConfigSchema": "src/ui/view.zod.ts#ListChartConfigSchema (const)", @@ -214,6 +219,10 @@ "ListMapConfig": "src/ui/view.zod.ts#ListMapConfig (type)", "ListMapConfigSchema": "src/ui/view.zod.ts#ListMapConfigSchema (const)", "ListView": "src/ui/view.zod.ts#ListView (type)", + "ListViewGroupHeaderRow": "src/ui/view-grouping-query.ts#ListViewGroupHeaderRow (interface)", + "ListViewGroupQueryError": "src/ui/view-grouping-query.ts#ListViewGroupQueryError (class)", + "ListViewGroupQueryRefusal": "src/ui/view-grouping-query.ts#ListViewGroupQueryRefusal (type)", + "ListViewGroupQuerySource": "src/ui/view-grouping-query.ts#ListViewGroupQuerySource (interface)", "ListViewParsed": "src/ui/view.zod.ts#ListViewParsed (type)", "ListViewSchema": "src/ui/view.zod.ts#ListViewSchema (const)", "NavigationArea": "src/ui/app.zod.ts#NavigationArea (type)", @@ -281,6 +290,7 @@ "PaginationConfig": "src/ui/view.zod.ts#PaginationConfig (type)", "PaginationConfigParsed": "src/ui/view.zod.ts#PaginationConfigParsed (type)", "PaginationConfigSchema": "src/ui/view.zod.ts#PaginationConfigSchema (const)", + "QueryInput": "src/data/query.zod.ts#QueryInput (type)", "REACT_BLOCKS": "src/ui/react-blocks.ts#REACT_BLOCKS (const)", "REACT_OVERLAY_SHADOWS": "src/ui/react-blocks.ts#REACT_OVERLAY_SHADOWS (const)", "REACT_RECORD_BLOCK_ALTERNATIVES": "src/ui/react-blocks.ts#REACT_RECORD_BLOCK_ALTERNATIVES (const)", @@ -341,6 +351,7 @@ "TimelineConfigSchema": "src/ui/view.zod.ts#TimelineConfigSchema (const)", "TreeConfig": "src/ui/view.zod.ts#TreeConfig (type)", "TreeConfigSchema": "src/ui/view.zod.ts#TreeConfigSchema (const)", + "UNMAPPED_COLUMN_SUMMARIES": "src/ui/view-grouping-query.ts#UNMAPPED_COLUMN_SUMMARIES (const)", "UrlNavItem": "src/ui/app.zod.ts#UrlNavItem (type)", "UrlNavItemParsed": "src/ui/app.zod.ts#UrlNavItemParsed (type)", "UrlNavItemSchema": "src/ui/app.zod.ts#UrlNavItemSchema (const)", @@ -403,6 +414,9 @@ "chartAggregateCategoryKey": "src/ui/chart-aggregate.ts#chartAggregateCategoryKey (function)", "chartAggregateResultKeys": "src/ui/chart-aggregate.ts#chartAggregateResultKeys (function)", "chartAggregateValueKey": "src/ui/chart-aggregate.ts#chartAggregateValueKey (function)", + "columnSummaryAlias": "src/ui/view-grouping-query.ts#columnSummaryAlias (function)", + "compileListViewGroupQuery": "src/ui/view-grouping-query.ts#compileListViewGroupQuery (function)", + "compileListViewGroupRowsQuery": "src/ui/view-grouping-query.ts#compileListViewGroupRowsQuery (function)", "dashboardForm": "src/ui/dashboard.form.ts#dashboardForm (const)", "datasetForm": "src/ui/dataset.form.ts#datasetForm (const)", "defineAction": "src/ui/action.zod.ts#defineAction (function)", @@ -424,6 +438,7 @@ "isKnownComponentType": "src/ui/component-type-vocabulary.ts#isKnownComponentType (function)", "isRecordContextBlockType": "src/ui/react-blocks.ts#isRecordContextBlockType (function)", "isViewContainerShaped": "src/ui/assembled-views.zod.ts#isViewContainerShaped (function)", + "listViewGroupKeyPredicate": "src/ui/view-grouping-query.ts#listViewGroupKeyPredicate (function)", "normalizeFilterOperator": "src/ui/view.zod.ts#normalizeFilterOperator (function)", "normalizeInlineAction": "src/ui/action.zod.ts#normalizeInlineAction (function)", "pageForm": "src/ui/page.form.ts#pageForm (const)", diff --git a/packages/spec/src/ui/index.ts b/packages/spec/src/ui/index.ts index e40b4ed779..affc388684 100644 --- a/packages/spec/src/ui/index.ts +++ b/packages/spec/src/ui/index.ts @@ -21,6 +21,12 @@ export * from './responsive.zod'; export * from './app.zod'; export * from './bulk-action.zod'; export * from './view.zod'; +// [#14556] List-view grouping is SERVER-SIDE (ruling A on objectui#7189): the +// group header aggregate query and the per-group row page, compiled from the +// view's `grouping` + column summaries + composed filter in the query AST's +// own vocabulary — the checkable half of the contract `GroupingConfigSchema` +// states in prose. +export * from './view-grouping-query'; // [#5320] The assembled-manifest channel for non-container view artifacts — // the `viewItems:` vocabulary and its producer-side partition helper. export * from './assembled-views.zod'; @@ -99,3 +105,8 @@ export * from './expression-bindable-text-keys.zod'; // (maintainer ruling recorded on #11350), same repair: re-export from the // declaring module. export type { FilterCondition } from '../data/filter.zod'; +// [#14556] entry-nameability, the same shape one module over: the compiled +// group queries are `/data`'s `EngineAggregateOptions` / `EngineQueryOptions`, +// whose structural expansion mentions `QueryInput` (`expand` is a recursive +// map of it), which `/ui` did not re-export. Same invariant, same repair. +export type { QueryInput } from '../data/query.zod'; diff --git a/packages/spec/src/ui/view-grouping-query.test.ts b/packages/spec/src/ui/view-grouping-query.test.ts new file mode 100644 index 0000000000..375fce8ace --- /dev/null +++ b/packages/spec/src/ui/view-grouping-query.test.ts @@ -0,0 +1,485 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +/** + * Pins for the server-side list-view grouping contract (#14556, ruling A on + * objectui#7189). + * + * The acceptance fixture is the card's: 186 rows in five business units sized + * 86 / 61 / 31 / 7 / 1, a page of `$top: 100`. The card measured what a + * page-scoped grouping renders on it — TWO headers (86, 14) when the rows are + * contiguous, FIVE (31/31/30/7/1) when they are interleaved — and ruled that + * neither is the data. The pins below reproduce both artefacts on the page + * window (so the fixture is shown to be the one that failed) and then reduce + * the COMPILED header query over the whole set in both orders: 86/61/31/7/1, + * order-independent, every unit present. + */ +import { describe, it, expect } from 'vitest'; +import { StandardErrorCode } from '../api/errors.zod'; +import { ListViewSchema } from './view.zod'; +import type { FilterCondition } from '../data/filter.zod'; +import type { EngineAggregateOptions } from '../data/data-engine.zod'; +import { + COLUMN_SUMMARY_AGGREGATION, + LIST_VIEW_GROUP_COUNT_ALIAS, + ListViewGroupQueryError, + UNMAPPED_COLUMN_SUMMARIES, + columnSummaryAlias, + compileListViewGroupQuery, + compileListViewGroupRowsQuery, + listViewGroupKeyPredicate, +} from './view-grouping-query'; + +// ─── The acceptance fixture ────────────────────────────────────────────────── + +interface Row { + id: string; + business_unit: string | null; + status: 'open' | 'done'; + amount: number; + owner: string; +} + +/** Five units, 186 rows, sized exactly as the card measured them. */ +const UNITS: ReadonlyArray = [ + ['northgate_operations', 86], + ['northgate_quality', 61], + ['riverside_plant', 31], + ['northgate_plant', 7], + ['harbour_office', 1], +]; + +function makeRow(unit: string, ordinal: number): Row { + return { + id: `${unit}-${ordinal}`, + business_unit: unit, + // Every third row is done, so a view filter and a second grouping level + // both have something to bite on. + status: ordinal % 3 === 0 ? 'done' : 'open', + amount: ordinal, + owner: `owner_${ordinal % 4}`, + }; +} + +/** Rows of one unit, then the next — the contiguous order the card measured. */ +const CONTIGUOUS: Row[] = UNITS.flatMap(([unit, size]) => + Array.from({ length: size }, (_, i) => makeRow(unit, i + 1)), +); + +/** Round-robin over the units — the interleaved order the card measured. */ +const INTERLEAVED: Row[] = (() => { + const queues = UNITS.map(([unit, size]) => Array.from({ length: size }, (_, i) => makeRow(unit, i + 1))); + const out: Row[] = []; + while (queues.some((q) => q.length > 0)) { + for (const q of queues) { + const next = q.shift(); + if (next) out.push(next); + } + } + return out; +})(); + +const PAGE_SIZE = 100; +const EXPECTED_COUNTS = { northgate_operations: 86, northgate_quality: 61, riverside_plant: 31, northgate_plant: 7, harbour_office: 1 }; + +// ─── A minimal reduction of the compiled queries over rows ─────────────────── +// +// Test-only: enough of the filter AST (`$and`, `$eq`, `$null`, implicit +// equality) and of the aggregate vocabulary to evaluate what the helpers +// compile. The platform's own faces are pinned elsewhere +// (`aggregation-conformance`, `in-memory-aggregation`); this is the pin that +// the COMPILED shape, evaluated, answers the card's numbers. + +function matches(row: Record, where: FilterCondition | Record | undefined): boolean { + if (!where) return true; + for (const [key, cond] of Object.entries(where)) { + if (key === '$and') { if (!(cond as FilterCondition[]).every((c) => matches(row, c))) return false; continue; } + if (key === '$or') { if (!(cond as FilterCondition[]).some((c) => matches(row, c))) return false; continue; } + const value = row[key]; + if (cond !== null && typeof cond === 'object' && !Array.isArray(cond)) { + const ops = cond as Record; + if ('$eq' in ops && value !== ops.$eq) return false; + if ('$null' in ops && (value == null) !== ops.$null) return false; + if ('$in' in ops && !(ops.$in as unknown[]).includes(value)) return false; + continue; + } + if (value !== cond) return false; + } + return true; +} + +function reduceHeaderQuery(rows: Row[], query: EngineAggregateOptions): Record[] { + const groupBy = (query.groupBy ?? []).map((g) => (typeof g === 'string' ? g : g.field)); + const buckets = new Map; rows: Row[] }>(); + for (const row of rows.filter((r) => matches(r as unknown as Record, query.where))) { + const key: Record = {}; + for (const g of groupBy) key[g] = (row as unknown as Record)[g] ?? null; + const id = JSON.stringify(groupBy.map((g) => key[g])); + const bucket = buckets.get(id) ?? { key, rows: [] }; + bucket.rows.push(row); + buckets.set(id, bucket); + } + return [...buckets.values()].map(({ key, rows: bucketRows }) => { + const out: Record = { ...key }; + for (const agg of query.aggregations ?? []) { + const values = agg.field ? bucketRows.map((r) => (r as unknown as Record)[agg.field as string]) : []; + const nums = values.filter((v) => v != null).map(Number); + switch (agg.function) { + case 'count': out[agg.alias] = agg.field ? values.filter((v) => v != null).length : bucketRows.length; break; + case 'count_distinct': out[agg.alias] = new Set(values.filter((v) => v != null)).size; break; + case 'sum': out[agg.alias] = nums.reduce((a, b) => a + b, 0); break; + case 'avg': out[agg.alias] = nums.length ? nums.reduce((a, b) => a + b, 0) / nums.length : null; break; + case 'min': out[agg.alias] = nums.length ? Math.min(...nums) : null; break; + case 'max': out[agg.alias] = nums.length ? Math.max(...nums) : null; break; + } + } + return out; + }); +} + +/** Header rows → `{ unit: count }`, the shape a reader compares. */ +function countsByUnit(headers: Record[]): Record { + return Object.fromEntries(headers.map((h) => [String(h.business_unit), h[LIST_VIEW_GROUP_COUNT_ALIAS] as number])); +} + +/** What a page-scoped grouping (the interim, objectui `useGroupedData`) shows on the first window. */ +function pageScopedCounts(rows: Row[]): Record { + const counts: Record = {}; + for (const row of rows.slice(0, PAGE_SIZE)) counts[String(row.business_unit)] = (counts[String(row.business_unit)] ?? 0) + 1; + return counts; +} + +const GROUPED_VIEW = { + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [ + { field: 'id' }, + { field: 'amount', summary: 'sum' as const }, + { field: 'owner', summary: 'count_unique' as const }, + ], +}; + +// ─── The fixture reproduces the card's measurements ────────────────────────── + +describe('the acceptance fixture is the one the card measured', () => { + it('holds 186 rows in five units sized 86/61/31/7/1, in both orders', () => { + expect(CONTIGUOUS).toHaveLength(186); + expect(INTERLEAVED).toHaveLength(186); + const whole = (rows: Row[]) => countsByUnit(reduceHeaderQuery(rows, { groupBy: ['business_unit'], aggregations: [{ function: 'count', alias: 'count' }] })); + expect(whole(CONTIGUOUS)).toEqual(EXPECTED_COUNTS); + expect(whole(INTERLEAVED)).toEqual(EXPECTED_COUNTS); + }); + + it('page-scoped grouping over the first 100 rows renders TWO headers (86, 14) when contiguous — three units absent', () => { + expect(pageScopedCounts(CONTIGUOUS)).toEqual({ northgate_operations: 86, northgate_quality: 14 }); + }); + + it('page-scoped grouping over the first 100 rows renders FIVE headers reading 31/31/30/7/1 when interleaved', () => { + expect(pageScopedCounts(INTERLEAVED)).toEqual({ + northgate_operations: 31, northgate_quality: 31, riverside_plant: 30, northgate_plant: 7, harbour_office: 1, + }); + }); +}); + +// ─── The header query ──────────────────────────────────────────────────────── + +describe('compileListViewGroupQuery — the group set and every header number are ONE aggregate query', () => { + it('compiles the grouping, the count node and the mapped column summaries onto EngineAggregateOptions', () => { + const where = { status: { $eq: 'open' } }; + expect(compileListViewGroupQuery(GROUPED_VIEW, { where })).toEqual({ + where, + groupBy: ['business_unit'], + aggregations: [ + { function: 'count', alias: 'count' }, + { function: 'sum', field: 'amount', alias: 'sum_amount' }, + { function: 'count_distinct', field: 'owner', alias: 'count_distinct_owner' }, + ], + }); + }); + + it('omits `where` when the view has no filter — the whole object', () => { + expect(compileListViewGroupQuery({ grouping: { fields: [{ field: 'business_unit' }] } })).toEqual({ + groupBy: ['business_unit'], + aggregations: [{ function: 'count', alias: 'count' }], + }); + }); + + it('reduced over the whole set yields 86/61/31/7/1 regardless of row order — the acceptance criterion', () => { + const query = compileListViewGroupQuery(GROUPED_VIEW); + const contiguous = reduceHeaderQuery(CONTIGUOUS, query); + const interleaved = reduceHeaderQuery(INTERLEAVED, query); + expect(countsByUnit(contiguous)).toEqual(EXPECTED_COUNTS); + expect(countsByUnit(interleaved)).toEqual(EXPECTED_COUNTS); + expect(contiguous).toHaveLength(5); + expect(interleaved).toHaveLength(5); + }); + + it('carries the per-group summaries on the same row as the count, under `_`', () => { + const headers = reduceHeaderQuery(CONTIGUOUS, compileListViewGroupQuery(GROUPED_VIEW)); + const ops = headers.find((h) => h.business_unit === 'northgate_operations')!; + // sum 1..86, and owners cycle through four names. + expect(ops).toEqual({ business_unit: 'northgate_operations', count: 86, sum_amount: (86 * 87) / 2, count_distinct_owner: 4 }); + const single = headers.find((h) => h.business_unit === 'harbour_office')!; + expect(single).toEqual({ business_unit: 'harbour_office', count: 1, sum_amount: 1, count_distinct_owner: 1 }); + }); + + it('applies the view filter to the header numbers — the same `where` the row query carries', () => { + const query = compileListViewGroupQuery(GROUPED_VIEW, { where: { status: { $eq: 'done' } } }); + const counts = countsByUnit(reduceHeaderQuery(INTERLEAVED, query)); + // Every third ordinal is done: floor(size / 3). + expect(counts).toEqual({ northgate_operations: 28, northgate_quality: 20, riverside_plant: 10, northgate_plant: 2 }); + }); + + it('a two-level grouping compiles to a two-column groupBy, in nesting order', () => { + const query = compileListViewGroupQuery({ + grouping: { fields: [{ field: 'business_unit' }, { field: 'status' }] }, + columns: ['id', 'amount'], + }); + expect(query.groupBy).toEqual(['business_unit', 'status']); + const leaves = reduceHeaderQuery(CONTIGUOUS, query); + // Five units, two of which have no `done` rows (sizes 7 → 2 done, 1 → 0 done). + expect(leaves).toHaveLength(9); + const opsOpen = leaves.find((h) => h.business_unit === 'northgate_operations' && h.status === 'open')!; + const opsDone = leaves.find((h) => h.business_unit === 'northgate_operations' && h.status === 'done')!; + expect(opsOpen.count).toBe(58); + expect(opsDone.count).toBe(28); + // The outer level folds exactly for count. + expect((opsOpen.count as number) + (opsDone.count as number)).toBe(EXPECTED_COUNTS.northgate_operations); + }); + + it('`depth` compiles an outer level\'s own query — the first N grouping fields', () => { + const view = { grouping: { fields: [{ field: 'business_unit' }, { field: 'status' }] } }; + expect(compileListViewGroupQuery(view, { depth: 1 }).groupBy).toEqual(['business_unit']); + expect(compileListViewGroupQuery(view, { depth: 2 }).groupBy).toEqual(['business_unit', 'status']); + expect(countsByUnit(reduceHeaderQuery(INTERLEAVED, compileListViewGroupQuery(view, { depth: 1 })))).toEqual(EXPECTED_COUNTS); + }); + + it('groups the empty key as its own group, keyed null', () => { + const rows: Row[] = [...CONTIGUOUS.slice(0, 3), { ...makeRow('x', 9), business_unit: null }]; + const headers = reduceHeaderQuery(rows, compileListViewGroupQuery({ grouping: { fields: [{ field: 'business_unit' }] } })); + expect(headers).toContainEqual({ business_unit: null, count: 1 }); + }); + + it('a column summary `count` IS the group count — it rides the count column, not a second node', () => { + const query = compileListViewGroupQuery({ + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [{ field: 'id', summary: 'count' }, { field: 'amount', summary: { type: 'count' } }], + }); + expect(query.aggregations).toEqual([{ function: 'count', alias: 'count' }]); + }); + + it('the object form aggregates the named field, and identical summaries are one node', () => { + const query = compileListViewGroupQuery({ + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [ + { field: 'amount', summary: { type: 'sum', field: 'amount_in_base_currency' } }, + { field: 'amount_in_base_currency', summary: 'sum' }, + { field: 'notes', summary: 'none' }, + ], + }); + expect(query.aggregations).toEqual([ + { function: 'count', alias: 'count' }, + { function: 'sum', field: 'amount_in_base_currency', alias: 'sum_amount_in_base_currency' }, + ]); + }); + + it('bare field-name columns declare no summary', () => { + expect(compileListViewGroupQuery({ grouping: { fields: [{ field: 'status' }] }, columns: ['id', 'amount'] }).aggregations) + .toEqual([{ function: 'count', alias: 'count' }]); + }); + + it('compiles from a view the schema accepts — the source is the declared ListView, not a private shape', () => { + const view = ListViewSchema.parse({ + type: 'grid', + columns: [{ field: 'name' }, { field: 'amount', summary: 'avg' }], + grouping: { fields: [{ field: 'business_unit', order: 'desc' }] }, + }); + expect(compileListViewGroupQuery({ grouping: view.grouping!, columns: view.columns })).toEqual({ + groupBy: ['business_unit'], + aggregations: [ + { function: 'count', alias: 'count' }, + { function: 'avg', field: 'amount', alias: 'avg_amount' }, + ], + }); + }); +}); + +// ─── The mapping table (fork i) ────────────────────────────────────────────── + +describe('COLUMN_SUMMARY_AGGREGATION — one vocabulary, not two', () => { + it('maps count / count_unique / sum / avg / min / max onto AggregationFunction and names the unmapped members', () => { + expect(COLUMN_SUMMARY_AGGREGATION).toEqual({ + none: { kind: 'none' }, + count: { kind: 'aggregate', function: 'count', fieldless: true }, + count_unique: { kind: 'aggregate', function: 'count_distinct', fieldless: false }, + sum: { kind: 'aggregate', function: 'sum', fieldless: false }, + avg: { kind: 'aggregate', function: 'avg', fieldless: false }, + min: { kind: 'aggregate', function: 'min', fieldless: false }, + max: { kind: 'aggregate', function: 'max', fieldless: false }, + count_empty: { kind: 'unmapped' }, + count_filled: { kind: 'unmapped' }, + percent_empty: { kind: 'unmapped' }, + percent_filled: { kind: 'unmapped' }, + }); + expect(UNMAPPED_COLUMN_SUMMARIES).toEqual(['count_empty', 'count_filled', 'percent_empty', 'percent_filled']); + }); + + it('aliases a summary `_` and the fieldless count `count`', () => { + expect(columnSummaryAlias('sum', 'amount')).toBe('sum_amount'); + expect(columnSummaryAlias('count_distinct', 'owner')).toBe('count_distinct_owner'); + expect(columnSummaryAlias('count', undefined)).toBe(LIST_VIEW_GROUP_COUNT_ALIAS); + expect(columnSummaryAlias('count', 'owner')).toBe('count_owner'); + }); + + it.each(['count_empty', 'count_filled', 'percent_empty', 'percent_filled'] as const)( + 'refuses a `%s` summary LOUDLY at compile time — code, status and the path of the summary', + (member) => { + const view = { + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [{ field: 'id' }, { field: 'notes', summary: member }], + }; + let caught: unknown; + try { compileListViewGroupQuery(view); } catch (e) { caught = e; } + expect(caught).toBeInstanceOf(ListViewGroupQueryError); + const err = caught as ListViewGroupQueryError; + expect(err.code).toBe('NOT_IMPLEMENTED'); + expect(err.status).toBe(501); + expect(err.reason).toBe('summary_unmapped'); + expect(err.path).toEqual(['columns', 1, 'summary']); + expect(err.message).toMatch(new RegExp(`^Column summary "${member}" on columns\\[1\\] \\(field "notes"\\) has no counterpart in the aggregation vocabulary`)); + expect(err.message).toContain('is an open contract question; until it is ruled, remove the summary from this column'); + expect(err.message).toContain('Nothing is dropped silently.'); + // The refusal is printed at the author: no issue-id token (maintainer ruling 2026-08-12). + expect(err.message).not.toMatch(/#\d+/); + }, + ); + + it('the refusal codes are standard-catalog members (ADR-0112) — pinned so the literals cannot drift', () => { + const unmapped = new ListViewGroupQueryError('summary_unmapped', [], 'x'); + expect(unmapped.code).toBe(StandardErrorCode.enum.NOT_IMPLEMENTED); + expect(unmapped.status).toBe(501); + const invalid = new ListViewGroupQueryError('alias_collision', [], 'x'); + expect(invalid.code).toBe(StandardErrorCode.enum.INVALID_QUERY); + expect(invalid.status).toBe(400); + expect(unmapped.name).toBe('ListViewGroupQueryError'); + }); +}); + +// ─── Structural refusals ───────────────────────────────────────────────────── + +describe('compileListViewGroupQuery — refuses what the contract cannot mean', () => { + const refusal = (fn: () => unknown): ListViewGroupQueryError => { + try { fn(); } catch (e) { return e as ListViewGroupQueryError; } + throw new Error('expected a ListViewGroupQueryError'); + }; + + it('an alias landing on a grouped field\'s own column', () => { + const err = refusal(() => compileListViewGroupQuery({ + grouping: { fields: [{ field: 'sum_amount' }] }, + columns: [{ field: 'amount', summary: 'sum' }], + })); + expect(err).toBeInstanceOf(ListViewGroupQueryError); + expect(err.reason).toBe('alias_collision'); + expect(err.code).toBe('INVALID_QUERY'); + expect(err.status).toBe(400); + expect(err.path).toEqual(['columns', 0, 'summary']); + }); + + it('an empty grouping', () => { + const err = refusal(() => compileListViewGroupQuery({ grouping: { fields: [] } })); + expect(err.reason).toBe('grouping_empty'); + expect(err.path).toEqual(['grouping', 'fields']); + expect(err.status).toBe(400); + }); + + it('a blank grouping field', () => { + const err = refusal(() => compileListViewGroupQuery({ grouping: { fields: [{ field: 'status' }, { field: ' ' }] } })); + expect(err.reason).toBe('grouping_field_blank'); + expect(err.path).toEqual(['grouping', 'fields', 1, 'field']); + }); + + it.each([0, 3, 1.5, -1])('depth %s outside 1..2', (depth) => { + const err = refusal(() => compileListViewGroupQuery( + { grouping: { fields: [{ field: 'a' }, { field: 'b' }] } }, + { depth }, + )); + expect(err.reason).toBe('depth_out_of_range'); + expect(err.code).toBe('INVALID_QUERY'); + }); +}); + +// ─── The per-group row page ────────────────────────────────────────────────── + +describe('compileListViewGroupRowsQuery — rows inside a group are the EXISTING paged find', () => { + const VIEW = { grouping: { fields: [{ field: 'business_unit' }] } }; + const viewWhere = { status: { $eq: 'open' } }; + + it('ANDs the group predicate into the view filter and carries the page', () => { + expect(compileListViewGroupRowsQuery(VIEW, { business_unit: 'northgate_operations' }, { where: viewWhere, limit: 25, offset: 25 })).toEqual({ + where: { $and: [viewWhere, { business_unit: { $eq: 'northgate_operations' } }] }, + limit: 25, + offset: 25, + }); + }); + + it('spells the empty group with the null predicate — `$eq: null` is not a comparand', () => { + expect(compileListViewGroupRowsQuery(VIEW, { business_unit: null }, { limit: 10 })).toEqual({ + where: { $and: [{ business_unit: { $null: true } }] }, + limit: 10, + }); + expect(listViewGroupKeyPredicate(VIEW.grouping, { business_unit: undefined })).toEqual([{ business_unit: { $null: true } }]); + }); + + it('a missing or empty view filter contributes no member', () => { + expect(compileListViewGroupRowsQuery(VIEW, { business_unit: 'x' }).where).toEqual({ $and: [{ business_unit: { $eq: 'x' } }] }); + expect(compileListViewGroupRowsQuery(VIEW, { business_unit: 'x' }, { where: {} }).where).toEqual({ $and: [{ business_unit: { $eq: 'x' } }] }); + }); + + it('passes `orderBy` and `fields` through to the find', () => { + const query = compileListViewGroupRowsQuery(VIEW, { business_unit: 'x' }, { + orderBy: [{ field: 'amount', order: 'desc' }], + fields: ['id', 'amount'], + limit: 5, + }); + expect(query.orderBy).toEqual([{ field: 'amount', order: 'desc' }]); + expect(query.fields).toEqual(['id', 'amount']); + expect(query).not.toHaveProperty('offset'); + }); + + it('opening the 86-row group pages its rows — every row reachable, none twice', () => { + const pages: Row[][] = []; + for (let offset = 0; offset < 200; offset += 50) { + const query = compileListViewGroupRowsQuery(VIEW, { business_unit: 'northgate_operations' }, { limit: 50, offset }); + const page = INTERLEAVED + .filter((r) => matches(r as unknown as Record, query.where)) + .slice(query.offset, (query.offset ?? 0) + (query.limit ?? 0)); + if (page.length === 0) break; + pages.push(page); + } + expect(pages.map((p) => p.length)).toEqual([50, 36]); + const ids = pages.flat().map((r) => r.id); + expect(new Set(ids).size).toBe(86); + expect(pages.flat().every((r) => r.business_unit === 'northgate_operations')).toBe(true); + }); + + it('a two-level group key selects the leaf group; an outer-level key selects the outer group', () => { + const grouping = { fields: [{ field: 'business_unit' }, { field: 'status' }] }; + expect(listViewGroupKeyPredicate(grouping, { business_unit: 'northgate_plant', status: 'done' })).toEqual([ + { business_unit: { $eq: 'northgate_plant' } }, + { status: { $eq: 'done' } }, + ]); + expect(listViewGroupKeyPredicate(grouping, { business_unit: 'northgate_plant' })).toEqual([ + { business_unit: { $eq: 'northgate_plant' } }, + ]); + const leaf = compileListViewGroupRowsQuery({ grouping }, { business_unit: 'northgate_plant', status: 'done' }); + expect(CONTIGUOUS.filter((r) => matches(r as unknown as Record, leaf.where))).toHaveLength(2); + }); + + it('refuses a group key that is not a prefix of the nesting order', () => { + const grouping = { fields: [{ field: 'business_unit' }, { field: 'status' }] }; + for (const key of [{ status: 'done' }, { business_unit: 'x', owner: 'y' }, {}]) { + let caught: unknown; + try { listViewGroupKeyPredicate(grouping, key); } catch (e) { caught = e; } + expect(caught).toBeInstanceOf(ListViewGroupQueryError); + expect((caught as ListViewGroupQueryError).reason).toBe('group_key_not_a_prefix'); + expect((caught as ListViewGroupQueryError).code).toBe('INVALID_QUERY'); + } + }); +}); diff --git a/packages/spec/src/ui/view-grouping-query.ts b/packages/spec/src/ui/view-grouping-query.ts new file mode 100644 index 0000000000..626e7fce0f --- /dev/null +++ b/packages/spec/src/ui/view-grouping-query.ts @@ -0,0 +1,490 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * List-view grouping is SERVER-SIDE — the two queries a grouped list view is + * answered with, compiled from the view's own declaration (#14556). + * + * ## The ruling this module implements + * + * Maintainer ruling A on objectui#7189 (2026-09-02, verbatim 「7189 A 其他同意」), + * carried into the spec lane by #14556: *"The set of groups and every number in + * a group header (the count and any per-group aggregation) are properties of + * the query, not of the fetched page. Rows inside a group are paged."* The + * client-side grouping of a fetched page is the interim state, not the + * contract — on 186 rows in five units with `$top: 100` it rendered two + * headers (86, 14) when the rows were contiguous and five (31/31/30/7/1) when + * they were interleaved; neither is the data (86/61/31/7/1), and the rows past + * the first window were not on page 2, they were unreachable. + * + * The seat ruling on #14556 (third tier): **reuse, no new query shape.** + * + * 1. **The group set and every header number are ONE aggregate query**, in + * the vocabulary `data/query.zod.ts` already declares: `groupBy` is + * `grouping.fields[].field` in nesting order (multi-level grouping is a + * multi-column `groupBy`), `aggregations` is a `count` node plus the view's + * declared per-column summaries mapped onto `AggregationFunction` (the one + * vocabulary datasets use — objectui#4576: one vocabulary, not two), and + * `where` is the view's composed filter. {@link compileListViewGroupQuery}. + * 2. **Rows inside a group are the EXISTING paged `find`** with the group's + * key predicate AND-ed into the same view filter, `limit` / `offset` per + * group. {@link compileListViewGroupRowsQuery}. + * 3. **Execution is the existing contract** — + * `IDataEngine.aggregate(objectName, EngineAggregateOptions, options)` for + * (1) and `IDataEngine.find(objectName, EngineQueryOptions, options)` for + * (2) (`contracts/data-engine.ts`). No new engine verb, no new envelope. + * `engine.aggregate` pushes (1) down to a driver that implements + * `aggregate()` (driver-sql groups by every `groupBy` item and lowers + * `count` / `sum` / `avg` / `min` / `max` / `count_distinct`) and otherwise + * buckets `driver.find()` output in memory (`applyInMemoryAggregation`), + * so the same header query answers on every driver. + * + * ## What a header row carries (result-column naming) + * + * One row per distinct combination of the grouped fields, and on each row: + * + * * every grouped field under **its own name**, holding the RAW STORED value + * (a lookup group key is the referenced id, never the expanded record; + * the empty group's key is `null` — the in-memory face folds an absent + * field into `null`, the SQL face groups `NULL` as one bucket). Resolving + * a key to a display label is the consumer's, exactly as for a cell. + * * `count` ({@link LIST_VIEW_GROUP_COUNT_ALIAS}) — the group's TOTAL row + * count, a fieldless `count` node (`COUNT(*)`: every row of the group, + * null cells included — the ruled semantics in `aggregation-conformance`). + * * one column per declared column summary, under + * {@link columnSummaryAlias} — `_` (`sum_amount`, + * `count_distinct_owner`). Two columns declaring the same summary over + * the same field are one number and one column. A column summary + * `count` IS the group count (it counts every row of the group, filled or + * not — the footer's own reading), so it rides the `count` column rather + * than minting a second one. + * + * A summary alias that would land on a grouped field's own column + * (`sum_amount` while grouping by a field named `sum_amount`) is refused, not + * silently overwritten — the two would be one column with two meanings. + * + * ## The mapping table — `ColumnSummary` → `AggregationFunction` (fork i) + * + * | `ListColumn.summary` | aggregation node | note | + * |---|---|---| + * | `none` | (no node) | "no summary" is not a summary | + * | `count` | `{ function: 'count' }` | fieldless — `COUNT(*)`, the group count itself | + * | `count_unique` | `{ function: 'count_distinct', field }` | `COUNT(DISTINCT field)`, nulls excluded | + * | `sum` / `avg` / `min` / `max` | the same name, `field` | | + * | `count_empty` | **refused** | no counterpart — see below | + * | `count_filled` | **refused** | no counterpart — see below | + * | `percent_empty` | **refused** | no counterpart — see below | + * | `percent_filled` | **refused** | no counterpart — see below | + * + * The four refused members are a STOP-AND-REPORT fork on #14556: the seat + * decides whether they map (`count_filled` reads as `COUNT(field)`, which the + * platform defines as the non-null count, while the footer's client-side + * reading also treats `''` and `[]` as empty; `count_empty` is spellable only + * with a per-aggregation `filter: { [field]: { $null: true } }`, which routes + * the whole header query through the engine's in-memory tier today; the two + * `percent_*` members are ratios of those counts, not aggregation functions). + * Until that ruling lands the refusal below IS the contract: a grouped view + * declaring one of them fails to compile LOUDLY, with `code`, `status` and + * the `path` of the offending summary — nothing is dropped and no third + * vocabulary is invented. {@link COLUMN_SUMMARY_AGGREGATION} is typed + * `Record`, so adding a member to `ColumnSummarySchema` + * without deciding its row here fails to type-check. + * + * ## Multi-level grouping + * + * `groupBy` carries the grouping fields in nesting order, so the header query + * answers one row per LEAF combination. An outer level's header is derived + * from the leaf rows sharing its prefix: `count`, `sum`, `min` and `max` fold + * exactly; `avg` does not (an average of averages weights the groups, not the + * rows). When an outer level must carry an exact `avg`, compile that level's + * own query with `depth` — the same query over the first `depth` grouping + * fields — rather than folding. This is still one query shape. + * + * ## Deliberately NOT here + * + * * **The REST door.** No `aggregate` route exists on the data endpoint + * today; which route carries the header query to the grid is the platform + * half of #14556 (item 2 of the card), not the spec half. + * * **Lowering the view's `filter` rules to a `FilterCondition`.** Both + * inputs here take the view's COMPOSED filter — the same `where` the + * view's row query already carries. The rule dialect → AST lowering is + * `parseFilterAST` (`data/filter.zod.ts`) on the platform and + * objectui's `filter-converter` on the client. + * * **Ordering the groups.** `EngineAggregateOptions` carries no `orderBy`; + * `GroupingField.order` is applied by the consumer over the header rows, + * a set the size of the group count. + * + * ## No business logic here (Prime Directive #2) + * + * Pure contract derivations — a declaration in, the queries the contract + * says it means out — in the same seat as `chart-aggregate.ts` (result-column + * naming) and `i18n-label-resolver.ts` (one shared rule instead of a private + * twin per producer). The header query is what the platform half executes + * and what objectui's `plugin-grid` will ask for once it lands; keeping the + * derivation here is what stops the two ends from re-deriving it apart. + */ + +import type { FilterCondition } from '../data/filter.zod'; +import type { AggregationFunction, AggregationNode } from '../data/query.zod'; +import type { EngineAggregateOptions, EngineQueryOptions } from '../data/data-engine.zod'; +import type { ColumnSummary, GroupingConfig, ListColumn, ListView } from './view.zod'; + +/** + * The alias of the per-group TOTAL row count on every header row — a + * fieldless `count` node, `COUNT(*)`. The same literal the fieldless-count + * alias already is on the object-bound chart path (`chart-aggregate.ts`). + */ +export const LIST_VIEW_GROUP_COUNT_ALIAS = 'count'; + +/** + * How one `ColumnSummary` member reaches the group header query. + * + * * `aggregate` — a node with this `function`; `fieldless` says whether the + * node names the summarised field (`COUNT(*)` does not). + * * `none` — the member means "no summary"; no node. + * * `unmapped` — no counterpart in `AggregationFunction`; the compiler + * refuses it loudly (fork i on #14556). + */ +export type ColumnSummaryAggregation = + | { readonly kind: 'aggregate'; readonly function: AggregationFunction; readonly fieldless: boolean } + | { readonly kind: 'none' } + | { readonly kind: 'unmapped' }; + +/** + * The mapping table, exhaustive over `ColumnSummarySchema` by construction — + * a new summary member without a row here is a type error, which is how the + * fork stays visible instead of silently dropping. + */ +export const COLUMN_SUMMARY_AGGREGATION: Readonly> = Object.freeze({ + none: { kind: 'none' }, + count: { kind: 'aggregate', function: 'count', fieldless: true }, + count_unique: { kind: 'aggregate', function: 'count_distinct', fieldless: false }, + sum: { kind: 'aggregate', function: 'sum', fieldless: false }, + avg: { kind: 'aggregate', function: 'avg', fieldless: false }, + min: { kind: 'aggregate', function: 'min', fieldless: false }, + max: { kind: 'aggregate', function: 'max', fieldless: false }, + count_empty: { kind: 'unmapped' }, + count_filled: { kind: 'unmapped' }, + percent_empty: { kind: 'unmapped' }, + percent_filled: { kind: 'unmapped' }, +} as const); + +/** + * The `ColumnSummary` members with no `AggregationFunction` counterpart — the + * exact list the compiler refuses (fork i on #14556), derived from the table + * rather than restated. + */ +export const UNMAPPED_COLUMN_SUMMARIES: readonly ColumnSummary[] = Object.freeze( + (Object.keys(COLUMN_SUMMARY_AGGREGATION) as ColumnSummary[]) + .filter((member) => COLUMN_SUMMARY_AGGREGATION[member].kind === 'unmapped'), +); + +/** + * The result column a column summary lands under on a header row: + * `_`. A fieldless `count` is {@link LIST_VIEW_GROUP_COUNT_ALIAS}. + */ +export function columnSummaryAlias(fn: AggregationFunction, field: string | undefined): string { + if (fn === 'count' && !field) return LIST_VIEW_GROUP_COUNT_ALIAS; + return `${fn}_${field}`; +} + +/** The slice of a list view the group queries read. */ +export interface ListViewGroupQuerySource { + /** `ListView.grouping` — the fields to group by, in nesting order. */ + grouping: GroupingConfig; + /** + * `ListView.columns` — only `ListColumn` entries carrying a `summary` + * contribute; bare field-name columns declare no summary. + */ + columns?: ListView['columns']; +} + +export interface CompileListViewGroupQueryOptions { + /** + * The view's COMPOSED filter — the same `where` its row query carries. + * Omitted or `{}` means the whole object. + */ + where?: FilterCondition; + /** + * How many grouping levels the query groups by, from the outermost: + * `1..grouping.fields.length`, default every level (one row per leaf + * combination). See "Multi-level grouping" in the module note. + */ + depth?: number; +} + +export interface CompileListViewGroupRowsQueryOptions { + /** The view's COMPOSED filter — the same `where` the header query carried. */ + where?: FilterCondition; + /** Page size within the group (`limit`; `$top` on the wire). */ + limit?: number; + /** Rows to skip within the group (`offset`; `$skip` on the wire). */ + offset?: number; + /** Row order within the group — passed through to the find. */ + orderBy?: EngineQueryOptions['orderBy']; + /** Projection — passed through to the find. */ + fields?: EngineQueryOptions['fields']; +} + +/** + * One header row as the platform answers it: the grouped fields under their + * own names, `count`, and one column per declared summary + * ({@link columnSummaryAlias}). + */ +export interface ListViewGroupHeaderRow { + [column: string]: unknown; + count: number; +} + +/** + * Why {@link compileListViewGroupQuery} / {@link compileListViewGroupRowsQuery} + * refused — the machine-readable discriminator beside the ADR-0112 envelope. + */ +export type ListViewGroupQueryRefusal = + | 'summary_unmapped' + | 'alias_collision' + | 'grouping_empty' + | 'grouping_field_blank' + | 'depth_out_of_range' + | 'group_key_not_a_prefix'; + +/** + * A refusal to compile a grouped list view into its queries. + * + * Carries the ADR-0112 envelope (`code` + `status`) so a door that serves the + * compile answers the same way the data path answers a malformed query, plus + * the `path` of the offending declaration inside the list view (a zod-style + * path: `['columns', 2, 'summary']`) and a closed `reason`. The two codes are + * standard-catalog members (`api/errors.zod.ts`): `NOT_IMPLEMENTED` / 501 for + * a summary the vocabulary has no counterpart for yet (the interim contract of + * fork i), `INVALID_QUERY` / 400 for a declaration the contract cannot mean. + * Spelled as literals here rather than imported from `../api/errors.zod` — + * `api/` already imports `ui/`, and a value edge back would be a cycle for + * two strings; `view-grouping-query.test.ts` pins them to + * `StandardErrorCode.enum` so the two cannot drift. + */ +export class ListViewGroupQueryError extends Error { + readonly code: 'NOT_IMPLEMENTED' | 'INVALID_QUERY'; + readonly status: 501 | 400; + readonly path: ReadonlyArray; + readonly reason: ListViewGroupQueryRefusal; + + constructor( + reason: ListViewGroupQueryRefusal, + path: ReadonlyArray, + message: string, + ) { + super(message); + this.name = 'ListViewGroupQueryError'; + this.reason = reason; + this.path = path; + if (reason === 'summary_unmapped') { + this.code = 'NOT_IMPLEMENTED'; + this.status = 501; + } else { + this.code = 'INVALID_QUERY'; + this.status = 400; + } + } +} + +/** The grouping fields, validated and in nesting order. */ +function groupingFieldNames(grouping: GroupingConfig | undefined): string[] { + const fields = grouping?.fields; + if (!Array.isArray(fields) || fields.length === 0) { + throw new ListViewGroupQueryError( + 'grouping_empty', + ['grouping', 'fields'], + 'A grouped list view names at least one grouping field: `grouping.fields` is empty, so there is ' + + 'no `groupBy` to compile and no group to page. Declare the field(s) to group by, in nesting order.', + ); + } + return fields.map((entry, index) => { + const name = entry && typeof entry === 'object' ? (entry as { field?: unknown }).field : undefined; + if (typeof name !== 'string' || name.trim() === '') { + throw new ListViewGroupQueryError( + 'grouping_field_blank', + ['grouping', 'fields', index, 'field'], + `grouping.fields[${index}].field is not a field name, so it cannot be a \`groupBy\` column. ` + + 'Name the field to group by at this level, or remove the level.', + ); + } + return name; + }); +} + +/** + * The per-column summaries a view declares, as aggregation nodes — the header + * query's `aggregations` after the `count` node. Refuses an unmapped member + * (fork i) and an alias landing on a grouped field's column. + */ +function summaryAggregationNodes( + columns: ListView['columns'] | undefined, + groupByNames: readonly string[], +): AggregationNode[] { + const grouped = new Set(groupByNames); + const nodes = new Map(); + const entries: ReadonlyArray = Array.isArray(columns) ? columns : []; + + entries.forEach((column, index) => { + if (!column || typeof column !== 'object') return; // a bare field name declares no summary + const summary = column.summary; + if (summary === undefined) return; + const path = ['columns', index, 'summary']; + const member: ColumnSummary = typeof summary === 'string' ? summary : summary.type; + const field = typeof summary === 'string' ? column.field : (summary.field ?? column.field); + + const mapping = COLUMN_SUMMARY_AGGREGATION[member]; + if (mapping === undefined || mapping.kind === 'unmapped') { + // The refusal is printed AT the author, so it carries the remedy and no + // issue id (the tracking card is in the module note — fork i). + throw new ListViewGroupQueryError( + 'summary_unmapped', + path, + `Column summary "${member}" on columns[${index}] (field "${field}") has no counterpart in the ` + + 'aggregation vocabulary (AggregationFunction: count / sum / avg / min / max / count_distinct), ' + + 'so a grouped list view cannot carry it in its group headers yet. Whether ' + + `${UNMAPPED_COLUMN_SUMMARIES.join(' / ')} map onto the vocabulary is an open contract question; ` + + 'until it is ruled, remove the summary from this column or group the view without it. ' + + 'Nothing is dropped silently.', + ); + } + if (mapping.kind === 'none') return; + + const aggregatedField = mapping.fieldless ? undefined : field; + const alias = columnSummaryAlias(mapping.function, aggregatedField); + if (alias === LIST_VIEW_GROUP_COUNT_ALIAS) return; // the group count column already carries it + if (grouped.has(alias)) { + throw new ListViewGroupQueryError( + 'alias_collision', + path, + `Column summary "${member}" on columns[${index}] would land under "${alias}", which is also a ` + + 'grouped field\'s own column on the header row — one column cannot carry both the group key ' + + 'and the summary. Rename the field or drop one of the two declarations.', + ); + } + if (!nodes.has(alias)) { + nodes.set(alias, { function: mapping.function, field: aggregatedField, alias }); + } + }); + + return [...nodes.values()]; +} + +/** + * The GROUP HEADER query: the set of groups and every number in every group + * header, as ONE `EngineAggregateOptions` for `IDataEngine.aggregate`. + * + * * `where` — `options.where`, the view's composed filter, verbatim + * (omitted when not given). + * * `groupBy` — `grouping.fields[].field` in nesting order (the first + * `options.depth` of them; default all). + * * `aggregations` — the `count` node first + * ({@link LIST_VIEW_GROUP_COUNT_ALIAS}), then one node per declared + * column summary ({@link COLUMN_SUMMARY_AGGREGATION}, + * {@link columnSummaryAlias}), in column order, deduplicated. + * + * @throws {ListViewGroupQueryError} on an unmapped summary (fork i — + * `NOT_IMPLEMENTED` / 501), an alias colliding with a grouped field, an + * empty grouping, a blank grouping field, or a `depth` outside + * `1..fields.length` (all `INVALID_QUERY` / 400). + */ +export function compileListViewGroupQuery( + view: ListViewGroupQuerySource, + options: CompileListViewGroupQueryOptions = {}, +): EngineAggregateOptions { + const names = groupingFieldNames(view.grouping); + const depth = options.depth ?? names.length; + if (!Number.isInteger(depth) || depth < 1 || depth > names.length) { + throw new ListViewGroupQueryError( + 'depth_out_of_range', + ['grouping', 'fields'], + `depth ${String(depth)} is outside 1..${names.length}: a header query groups by the first \`depth\` ` + + `of the ${names.length} declared grouping level(s). Omit \`depth\` for every level.`, + ); + } + const groupBy = names.slice(0, depth); + const aggregations: AggregationNode[] = [ + { function: 'count', alias: LIST_VIEW_GROUP_COUNT_ALIAS }, + ...summaryAggregationNodes(view.columns, groupBy), + ]; + const query: EngineAggregateOptions = { groupBy, aggregations }; + if (options.where !== undefined) query.where = options.where; + return query; +} + +/** + * The predicate that selects ONE group's rows: for each grouped field, in + * nesting order, `{ field: { $eq: key } }` — or `{ field: { $null: true } }` + * for the empty group, whose header key is `null` and which no `$eq` can + * select (`null` is not a comparand; `$null` is the AST's spelling for + * absence, `data/filter.zod.ts`). + * + * `groupKey` must carry a PREFIX of the nesting order — every level from the + * outermost down to the group being opened, and no level past it — so the + * rows of an outer group (a `depth`-scoped header) are spellable too. + * + * @throws {ListViewGroupQueryError} `group_key_not_a_prefix` (`INVALID_QUERY`) + */ +export function listViewGroupKeyPredicate( + grouping: GroupingConfig, + groupKey: Readonly>, +): FilterCondition[] { + const names = groupingFieldNames(grouping); + const keyed = new Set(Object.keys(groupKey)); + const levels = names.filter((name) => keyed.has(name)).length; + const isPrefix = levels > 0 + && keyed.size === levels + && names.slice(0, levels).every((name) => keyed.has(name)); + if (!isPrefix) { + throw new ListViewGroupQueryError( + 'group_key_not_a_prefix', + ['grouping', 'fields'], + `A group key names the grouping fields from the outermost level down to the group being opened ` + + `(a prefix of [${names.join(', ')}]); received [${[...keyed].join(', ')}]. ` + + 'Carry every outer level\'s key, and no field that is not a grouping field.', + ); + } + return names.slice(0, levels).map((name) => { + const value = groupKey[name]; + return value === null || value === undefined + ? { [name]: { $null: true } } + : { [name]: { $eq: value } }; + }); +} + +/** + * The PER-GROUP ROW PAGE: the EXISTING paged `find`, as `EngineQueryOptions` + * for `IDataEngine.find`, with the group's key predicate AND-ed into the + * view's composed filter: + * + * ```ts + * { where: { $and: [viewWhere, { business_unit: { $eq: 'northgate_ops' } }] }, limit: 50, offset: 0 } + * ``` + * + * `$and` is the filter AST's own composition; a missing or empty + * `options.where` contributes no member (`{ $and: [] }` is TRUE by the ruled + * reduction, so the group predicates alone would already be the whole + * condition — the empty member is simply not spelled). `limit` / `offset` / + * `orderBy` / `fields` pass through when given. + * + * @throws {ListViewGroupQueryError} see {@link listViewGroupKeyPredicate} + */ +export function compileListViewGroupRowsQuery( + view: Pick, + groupKey: Readonly>, + options: CompileListViewGroupRowsQueryOptions = {}, +): EngineQueryOptions { + const members: FilterCondition[] = []; + const viewWhere = options.where; + if (viewWhere && typeof viewWhere === 'object' && Object.keys(viewWhere).length > 0) { + members.push(viewWhere); + } + members.push(...listViewGroupKeyPredicate(view.grouping, groupKey)); + + const query: EngineQueryOptions = { where: { $and: members } }; + if (options.limit !== undefined) query.limit = options.limit; + if (options.offset !== undefined) query.offset = options.offset; + if (options.orderBy !== undefined) query.orderBy = options.orderBy; + if (options.fields !== undefined) query.fields = options.fields; + return query; +} diff --git a/packages/spec/src/ui/view.zod.ts b/packages/spec/src/ui/view.zod.ts index 2c798ec799..4ca5b96cc5 100644 --- a/packages/spec/src/ui/view.zod.ts +++ b/packages/spec/src/ui/view.zod.ts @@ -684,6 +684,29 @@ export type ViewFilterRuleParsed = z.infer; /** * Column Summary Function Schema * Aggregation function for column footer (Airtable-style column summaries) + * + * On a GROUPED list view the same declaration is the per-group HEADER summary + * (#14556, ruling A on objectui#7189: grouping is server-side, so a group's + * numbers are properties of the query, not of the fetched page). The header + * is one aggregate query in the query AST's own vocabulary — `AggregationFunction` + * (`data/query.zod.ts`), the vocabulary datasets already use, one and not two + * (objectui#4576) — and this enum maps onto it in + * `view-grouping-query.ts` (`COLUMN_SUMMARY_AGGREGATION`): + * + * * `count` → a fieldless `count` (`COUNT(*)`, every row of the group — the + * group count itself), `count_unique` → `count_distinct`, and + * `sum` / `avg` / `min` / `max` → the same name; `none` declares nothing. + * * `count_empty` / `count_filled` / `percent_empty` / `percent_filled` have + * NO counterpart yet. Their mapping is an open contract question on #14556 + * (fork i); until it is ruled, a grouped view declaring one of them is + * refused LOUDLY by `compileListViewGroupQuery` (`NOT_IMPLEMENTED` / 501, + * with the path of the summary) — never dropped, never given a third + * vocabulary. The footer keeps computing them client-side over the rows + * it holds, as it always has. + * + * ⛔ Adding a member here without deciding its row in + * `COLUMN_SUMMARY_AGGREGATION` is a type error by construction, so the fork + * cannot widen silently. */ export const ColumnSummarySchema = lazySchema(() => z.enum([ 'none', @@ -697,7 +720,15 @@ export const ColumnSummarySchema = lazySchema(() => z.enum([ 'avg', 'min', 'max', -]).describe('Aggregation function for column footer summary')); +]).describe( + // The tracking card for the open mapping question is named in the JSDoc + // above; `.describe()` prose reaches readers who cannot resolve an issue id. + 'Aggregation function for the column footer summary — and, on a grouped list view, the per-group ' + + 'header summary (server-side): count (COUNT(*), the group count), count_unique ' + + '(count_distinct), sum, avg, min, max map onto the query AST\'s AggregationFunction; ' + + 'count_empty, count_filled, percent_empty, percent_filled have no counterpart yet and are refused ' + + 'loudly by the group-header compiler (an open contract question) — never dropped silently', +)); /** * Column Summary Configuration Schema @@ -802,26 +833,76 @@ export const RowHeightSchema = lazySchema(() => z.enum([ /** * Grouping Field Configuration * Defines a single grouping level for record grouping. + * + * `field` is one `groupBy` column of the group header query + * ({@link GroupingConfigSchema}); the header row carries its RAW STORED value + * under the field's own name — a lookup's group key is the referenced id, the + * empty group's key is `null`. `order` and `collapsed` are presentation: + * `EngineAggregateOptions` carries no `orderBy`, so the consumer sorts the + * header rows (a set the size of the group count) and folds/unfolds them. */ export const GroupingFieldSchema = lazySchema(() => strictObject({ surface: 'this grouping field', history: VIEW_HISTORY, }, { - field: z.string().describe('Field name to group by'), - order: z.enum(['asc', 'desc']).default('asc').describe('Group sort order'), - collapsed: z.boolean().default(false).describe('Collapse groups by default'), + field: z.string().describe('Field name to group by — one `groupBy` column of the group header query; the header row carries its raw stored value (null for the empty group)'), + order: z.enum(['asc', 'desc']).default('asc').describe('Group sort order — applied by the consumer over the header rows (the aggregate query carries no orderBy)'), + collapsed: z.boolean().default(false).describe('Collapse groups by default (presentation only)'), })); /** * Grouping Configuration Schema (Airtable-style) * Supports multi-level grouping for grid/gallery views. + * + * ## Grouping is SERVER-SIDE (#14556) + * + * Maintainer ruling A on objectui#7189 (2026-09-02): *the set of groups and + * every number in a group header (the count and any per-group aggregation) + * are properties of the query, not of the fetched page; rows inside a group + * are paged.* Grouping the rows of one fetched window — what a grouped grid + * did before this contract — rendered two headers (86, 14) or five + * (31/31/30/7/1) for the same 186 rows in five units depending on row order, + * and left the rows past the first window unreachable. That is the interim + * state, not the contract. + * + * What the platform returns for a grouped list view, in the vocabulary the + * query AST already declares (seat ruling: reuse, no new query shape): + * + * 1. **The group keys and every header number — ONE aggregate query** + * (`EngineAggregateOptions`, executed by `IDataEngine.aggregate`): + * `groupBy` = `fields[].field` in nesting order (multi-level grouping is + * a multi-column `groupBy`), `aggregations` = a `count` node (the group's + * TOTAL row count, alias `count`) plus the view's declared column + * summaries (`ListColumn.summary`, mapped onto `AggregationFunction` — see + * {@link ColumnSummarySchema}), `where` = the view's composed filter. + * One header row per group, keyed by the grouped fields' own names. + * 2. **The rows inside a group — the EXISTING paged `find`** + * (`EngineQueryOptions`, `IDataEngine.find`) with the group's key + * predicate AND-ed into the same view filter, `limit` / `offset` per + * group (`$top` / `$skip` on the wire). + * 3. No new engine verb, no new envelope. + * + * The checkable form of this contract is `view-grouping-query.ts` — + * `compileListViewGroupQuery` (1) and `compileListViewGroupRowsQuery` (2), + * pinned on the 186-row fixture: 86/61/31/7/1 regardless of row order. + * + * Follow-ons, in order: the platform half of #14556 (the route that carries + * the header query on the REST/ObjectQL path — no `aggregate` route exists on + * the data endpoint today), then objectui#7189 (`plugin-grid` consumes the + * header rows and stops grouping the page). */ export const GroupingConfigSchema = lazySchema(() => strictObject({ surface: 'this grouping configuration', history: VIEW_HISTORY, }, { - fields: z.array(GroupingFieldSchema).min(1).describe('Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field)'), -}).describe('Record grouping configuration')); + fields: z.array(GroupingFieldSchema).min(1).describe('Fields to group by, in nesting order — the first entry is the outermost group and each later entry nests one level deeper (at least one field); the same order as the group header query\'s `groupBy`'), +}).describe( + 'Record grouping configuration — SERVER-SIDE: the set of groups and every number in a group ' + + 'header (the count and the per-column summaries) are properties of the query, not of the fetched page, ' + + 'answered by one aggregate query (`groupBy` = the fields in nesting order, `count` + the mapped column ' + + 'summaries, the view filter); rows inside a group are paged by the existing find with the group key ' + + 'AND-ed into the view filter. Compiled by `compileListViewGroupQuery` / `compileListViewGroupRowsQuery`', +)); /** * Gallery View Configuration (Airtable-style) @@ -1783,8 +1864,8 @@ const ListViewShapeSchema = lazySchema(() => strictObject({ /** Row Height / Density (Airtable-style) */ rowHeight: RowHeightSchema.optional().describe('Row height / density setting'), - /** Record Grouping (Airtable-style) */ - grouping: GroupingConfigSchema.optional().describe('Group records by one or more fields'), + /** Record Grouping (Airtable-style) — server-side, see {@link GroupingConfigSchema} (#14556). */ + grouping: GroupingConfigSchema.optional().describe('Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a group are paged (see GroupingConfigSchema)'), /** Row Color (Airtable-style) */ rowColor: RowColorConfigSchema.optional().describe('Color rows based on field value'), diff --git a/skills/objectstack-ui/contracts/react-blocks.contract.json b/skills/objectstack-ui/contracts/react-blocks.contract.json index dc37170501..c17d2dc7e6 100644 --- a/skills/objectstack-ui/contracts/react-blocks.contract.json +++ b/skills/objectstack-ui/contracts/react-blocks.contract.json @@ -374,7 +374,7 @@ "type": "object", "kind": "data", "required": false, - "description": "Group records by one or more fields" + "description": "Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a …" }, { "name": "rowHeight", diff --git a/skills/objectstack-ui/references/react-blocks.md b/skills/objectstack-ui/references/react-blocks.md index 6e1b6923d7..98c1ff47e6 100644 --- a/skills/objectstack-ui/references/react-blocks.md +++ b/skills/objectstack-ui/references/react-blocks.md @@ -73,7 +73,7 @@ Server-connected object table with toolbar and switchable visualizations (grid/k | `searchableFields` | `string[]` | data | | Fields enabled for search | | `userFilters` | `object` | data | | End-user quick-filter bar: dropdown/toggle fields or tab presets. Omit to let the renderer derive filters from select/boolean fields | | `pagination` | `object` | data | | Pagination configuration | -| `grouping` | `object` | data | | Group records by one or more fields | +| `grouping` | `object` | data | | Group records by one or more fields — server-side: the groups and their header numbers come from an aggregate query over the whole filtered set, rows within a … | | `rowHeight` | `'compact' \| 'short' \| 'medium' \| 'tall' \| 'extra_tall'` | data | | Row height / density setting | | `selection` | `object` | data | | Row selection configuration | | `rowActions` | `string[]` | data | | Actions available for individual row items | From 880d45dd087b642f8827781aab8644402c24853c Mon Sep 17 00:00:00 2001 From: os-justin Date: Fri, 4 Sep 2026 08:10:06 +0000 Subject: [PATCH 2/4] =?UTF-8?q?docs(spec):=20the=20empty=20group's=20`$nul?= =?UTF-8?q?l`=20spelling=20is=20the=20one=20`is=5Fempty`=20lowers=20to=20?= =?UTF-8?q?=E2=80=94=20not=20a=20claim=20that=20`$eq:=20null`=20is=20refus?= =?UTF-8?q?ed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `data/filter.zod.ts` accepts `$eq: null` as the "has no value" predicate; what it refuses is null as an ordering or list comparand. The helper keeps `$null` (the spelling `parseFilterAST` gives `is_empty` / `is_null`, lowered to `IS NULL`) and its JSDoc, the pin title and the changeset now say why. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01H2oQebDDxYKfWZusyd8GXk --- .changeset/list-view-grouping-server-side-contract.md | 3 ++- packages/spec/src/ui/view-grouping-query.test.ts | 2 +- packages/spec/src/ui/view-grouping-query.ts | 8 +++++--- 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/.changeset/list-view-grouping-server-side-contract.md b/.changeset/list-view-grouping-server-side-contract.md index cad28911f1..808759448a 100644 --- a/.changeset/list-view-grouping-server-side-contract.md +++ b/.changeset/list-view-grouping-server-side-contract.md @@ -31,7 +31,8 @@ New on the `ui` entry, `view-grouping-query.ts`: - `compileListViewGroupQuery(view, { where?, depth? })` → the header query; `compileListViewGroupRowsQuery(view, groupKey, { where?, limit?, offset?, orderBy?, fields? })` → the row page; `listViewGroupKeyPredicate` (the empty group is spelled with - the `$null` predicate, never `$eq: null`). + the `$null` predicate — the spelling the view filter dialect's `is_empty` + lowers to). - `COLUMN_SUMMARY_AGGREGATION` — the `ColumnSummary` → `AggregationFunction` table, exhaustive by type: `count` → a fieldless `count` (`COUNT(*)`), `count_unique` → `count_distinct`, `sum` / `avg` / `min` / `max` → the same diff --git a/packages/spec/src/ui/view-grouping-query.test.ts b/packages/spec/src/ui/view-grouping-query.test.ts index 375fce8ace..b600d9dd52 100644 --- a/packages/spec/src/ui/view-grouping-query.test.ts +++ b/packages/spec/src/ui/view-grouping-query.test.ts @@ -419,7 +419,7 @@ describe('compileListViewGroupRowsQuery — rows inside a group are the EXISTING }); }); - it('spells the empty group with the null predicate — `$eq: null` is not a comparand', () => { + it('spells the empty group with the `$null` predicate — the spelling `is_empty` lowers to', () => { expect(compileListViewGroupRowsQuery(VIEW, { business_unit: null }, { limit: 10 })).toEqual({ where: { $and: [{ business_unit: { $null: true } }] }, limit: 10, diff --git a/packages/spec/src/ui/view-grouping-query.ts b/packages/spec/src/ui/view-grouping-query.ts index 626e7fce0f..5c895b4608 100644 --- a/packages/spec/src/ui/view-grouping-query.ts +++ b/packages/spec/src/ui/view-grouping-query.ts @@ -415,9 +415,11 @@ export function compileListViewGroupQuery( /** * The predicate that selects ONE group's rows: for each grouped field, in * nesting order, `{ field: { $eq: key } }` — or `{ field: { $null: true } }` - * for the empty group, whose header key is `null` and which no `$eq` can - * select (`null` is not a comparand; `$null` is the AST's spelling for - * absence, `data/filter.zod.ts`). + * for the empty group, whose header key is `null`: the `$null` predicate is + * the AST's own spelling for absence (`data/filter.zod.ts`, lowered to + * `IS NULL` on the SQL family) and the one the view filter dialect's + * `is_empty` / `is_null` lower to (`parseFilterAST`), so a group predicate + * and a view filter agree on what "empty" means. * * `groupKey` must carry a PREFIX of the nesting order — every level from the * outermost down to the group being opened, and no level past it — so the From da0ec3c1766332d6b213539bd49bb951b044d039 Mon Sep 17 00:00:00 2001 From: os-justin Date: Fri, 4 Sep 2026 11:29:32 +0000 Subject: [PATCH 3/4] =?UTF-8?q?feat(spec):=20list-view=20grouping=20?= =?UTF-8?q?=E2=80=94=20the=20four=20*=5Ffilled=20/=20*=5Fempty=20summaries?= =?UTF-8?q?=20derive=20from=20one=20COUNT(field)=20node;=20the=20existing?= =?UTF-8?q?=20query=20door;=20scalar=20keys;=20unknown-member=20refusal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contract review conditions (seat comment on the card, 2026-09-04): - alias_collision also refuses a grouping field named `count` (checked up front, and the per-summary collision check now precedes the count early-return); pinned. - fold sentence: `count_distinct` does not fold across leaves — listed with `avg`; an outer-level `count_unique` needs the `depth` query; pinned (8 vs 4). - the "no aggregate route" premise corrected: both queries ride the existing `POST /data/:object/query` → `protocol.findData` → `engine.aggregate` door (answering `{ object, records, total, hasMore }`), `client.data.query()` and the RPC `method: 'aggregate'`; the platform half pins that door. - an unknown ColumnSummary value → `summary_unknown` (INVALID_QUERY / 400); `summary_unmapped` (NOT_IMPLEMENTED / 501) kept for a declared member with no counterpart (none today); both pinned. - group keys are scalar-valued: `group_key_not_scalar` (INVALID_QUERY) for an array/object key; per-instant date grouping and unbounded header cardinality recorded in the GroupingConfigSchema JSDoc. - fork (i) ruling implemented: count_filled / count_empty / percent_filled / percent_empty compile to ONE `{ function: 'count', field, alias: 'count_' }` node (deduplicated, never the fieldless count) and `deriveColumnSummary(row, summary, field)` computes them on the header row (percent_filled 0 when count is 0; percent_empty = 1 − percent_filled); describes and the changeset updated; pinned on the fixture's nullable field in both row orders. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01H2oQebDDxYKfWZusyd8GXk --- ...list-view-grouping-server-side-contract.md | 39 ++- .../spec/src/ui/view-grouping-query.test.ts | 227 +++++++++++---- packages/spec/src/ui/view-grouping-query.ts | 271 ++++++++++++++---- packages/spec/src/ui/view.zod.ts | 62 +++- 4 files changed, 459 insertions(+), 140 deletions(-) diff --git a/.changeset/list-view-grouping-server-side-contract.md b/.changeset/list-view-grouping-server-side-contract.md index 808759448a..109fe84521 100644 --- a/.changeset/list-view-grouping-server-side-contract.md +++ b/.changeset/list-view-grouping-server-side-contract.md @@ -33,22 +33,31 @@ New on the `ui` entry, `view-grouping-query.ts`: → the row page; `listViewGroupKeyPredicate` (the empty group is spelled with the `$null` predicate — the spelling the view filter dialect's `is_empty` lowers to). -- `COLUMN_SUMMARY_AGGREGATION` — the `ColumnSummary` → `AggregationFunction` - table, exhaustive by type: `count` → a fieldless `count` (`COUNT(*)`), +- `COLUMN_SUMMARY_AGGREGATION` — the `ColumnSummary` → aggregation table, + exhaustive by type: `count` → a fieldless `count` (`COUNT(*)`), `count_unique` → `count_distinct`, `sum` / `avg` / `min` / `max` → the same - name, `none` → nothing. `count_empty`, `count_filled`, `percent_empty` and - `percent_filled` have no counterpart yet (`UNMAPPED_COLUMN_SUMMARIES`); a - grouped view declaring one is refused loudly at compile time with - `ListViewGroupQueryError` (`NOT_IMPLEMENTED` / 501, the summary's path) — - their mapping is an open contract question on #14556, and nothing is dropped - silently in the meantime. + name, `none` → nothing; `count_filled` / `count_empty` / `percent_filled` / + `percent_empty` map by derivation — one `{ function: 'count', field }` node + (`COUNT(field)`, the non-null count, header column `count_`), from + which `deriveColumnSummary(row, summary, field)` computes all four on the + header row (`count_filled` = `count_`, `count_empty` = `count − + count_`, `percent_filled` = `count_ / count`, 0 when the count + is 0, `percent_empty` = `1 − percent_filled`). Server-side "empty" is `null` + on every face; the footer's client-side reading of `''` / `[]` as empty is + the renderer's to converge. A future member with no counterpart is refused + loudly at compile time (`ListViewGroupQueryError`, `NOT_IMPLEMENTED` / 501, + the summary's path — `UNMAPPED_COLUMN_SUMMARIES`, empty today); a value that + is no member at all is `INVALID_QUERY` / 400. - Result-column naming on a header row: each grouped field under its own name - (raw stored value, `null` for the empty group), `count`, and each summary - under `_` (`columnSummaryAlias`). + (raw stored value, `null` for the empty group; group keys are scalar), `count`, + and each summary under `_` (`columnSummaryAlias`). `GroupingConfigSchema` / `GroupingFieldSchema` / `ColumnSummarySchema` now say -this in their docs. Nothing changes in what parses: no key is added, removed -or re-shaped. `minor` because a new exported helper and a declared contract -semantics ship; not breaking — the page-scoped behaviour was never declared. -The route that carries the header query to the grid is the platform half of -#14556; the grid consuming it is objectui#7189. +this in their docs, with the shape's recorded limits (a date grouping field +groups per distinct stored instant; header cardinality is unbounded). Nothing +changes in what parses: no key is added, removed or re-shaped. `minor` because +a new exported helper and a declared contract semantics ship; not breaking — +the page-scoped behaviour was never declared. Both queries ride the existing +`POST /data/:object/query` door (`protocol.findData` → `engine.aggregate`, +answering `{ object, records, total, hasMore }`); the grid consuming the header +rows is objectui#7189. diff --git a/packages/spec/src/ui/view-grouping-query.test.ts b/packages/spec/src/ui/view-grouping-query.test.ts index b600d9dd52..efcb03d3a3 100644 --- a/packages/spec/src/ui/view-grouping-query.test.ts +++ b/packages/spec/src/ui/view-grouping-query.test.ts @@ -15,6 +15,7 @@ import { describe, it, expect } from 'vitest'; import { StandardErrorCode } from '../api/errors.zod'; import { ListViewSchema } from './view.zod'; +import type { ColumnSummary } from './view.zod'; import type { FilterCondition } from '../data/filter.zod'; import type { EngineAggregateOptions } from '../data/data-engine.zod'; import { @@ -25,8 +26,10 @@ import { columnSummaryAlias, compileListViewGroupQuery, compileListViewGroupRowsQuery, + deriveColumnSummary, listViewGroupKeyPredicate, } from './view-grouping-query'; +import type { ListViewGroupHeaderRow } from './view-grouping-query'; // ─── The acceptance fixture ────────────────────────────────────────────────── @@ -36,6 +39,8 @@ interface Row { status: 'open' | 'done'; amount: number; owner: string; + /** Nullable on purpose — the field the derived `*_filled` / `*_empty` pins read. */ + notes: string | null; } /** Five units, 186 rows, sized exactly as the card measured them. */ @@ -56,6 +61,8 @@ function makeRow(unit: string, ordinal: number): Row { status: ordinal % 3 === 0 ? 'done' : 'open', amount: ordinal, owner: `owner_${ordinal % 4}`, + // Every fifth row has no note — the server's "empty" (null), never ''. + notes: ordinal % 5 === 0 ? null : `note ${ordinal}`, }; } @@ -79,6 +86,8 @@ const INTERLEAVED: Row[] = (() => { const PAGE_SIZE = 100; const EXPECTED_COUNTS = { northgate_operations: 86, northgate_quality: 61, riverside_plant: 31, northgate_plant: 7, harbour_office: 1 }; +/** Rows whose `notes` is null per unit: floor(size / 5). */ +const EXPECTED_EMPTY_NOTES = { northgate_operations: 17, northgate_quality: 12, riverside_plant: 6, northgate_plant: 1, harbour_office: 0 }; // ─── A minimal reduction of the compiled queries over rows ─────────────────── // @@ -106,7 +115,7 @@ function matches(row: Record, where: FilterCondition | Record[] { +function reduceHeaderQuery(rows: Row[], query: EngineAggregateOptions): ListViewGroupHeaderRow[] { const groupBy = (query.groupBy ?? []).map((g) => (typeof g === 'string' ? g : g.field)); const buckets = new Map; rows: Row[] }>(); for (const row of rows.filter((r) => matches(r as unknown as Record, query.where))) { @@ -123,6 +132,7 @@ function reduceHeaderQuery(rows: Row[], query: EngineAggregateOptions): Record (r as unknown as Record)[agg.field as string]) : []; const nums = values.filter((v) => v != null).map(Number); switch (agg.function) { + // `count(field)` is the NON-NULL count on every face — the ruled semantics. case 'count': out[agg.alias] = agg.field ? values.filter((v) => v != null).length : bucketRows.length; break; case 'count_distinct': out[agg.alias] = new Set(values.filter((v) => v != null)).size; break; case 'sum': out[agg.alias] = nums.reduce((a, b) => a + b, 0); break; @@ -131,13 +141,13 @@ function reduceHeaderQuery(rows: Row[], query: EngineAggregateOptions): Record[]): Record { - return Object.fromEntries(headers.map((h) => [String(h.business_unit), h[LIST_VIEW_GROUP_COUNT_ALIAS] as number])); +function countsByUnit(headers: ListViewGroupHeaderRow[]): Record { + return Object.fromEntries(headers.map((h) => [String(h.business_unit), h[LIST_VIEW_GROUP_COUNT_ALIAS]])); } /** What a page-scoped grouping (the interim, objectui `useGroupedData`) shows on the first window. */ @@ -147,6 +157,11 @@ function pageScopedCounts(rows: Row[]): Record { return counts; } +const refusal = (fn: () => unknown): ListViewGroupQueryError => { + try { fn(); } catch (e) { return e as ListViewGroupQueryError; } + throw new Error('expected a ListViewGroupQueryError'); +}; + const GROUPED_VIEW = { grouping: { fields: [{ field: 'business_unit' }] }, columns: [ @@ -241,7 +256,7 @@ describe('compileListViewGroupQuery — the group set and every header number ar expect(opsOpen.count).toBe(58); expect(opsDone.count).toBe(28); // The outer level folds exactly for count. - expect((opsOpen.count as number) + (opsDone.count as number)).toBe(EXPECTED_COUNTS.northgate_operations); + expect(opsOpen.count + opsDone.count).toBe(EXPECTED_COUNTS.northgate_operations); }); it('`depth` compiles an outer level\'s own query — the first N grouping fields', () => { @@ -251,6 +266,21 @@ describe('compileListViewGroupQuery — the group set and every header number ar expect(countsByUnit(reduceHeaderQuery(INTERLEAVED, compileListViewGroupQuery(view, { depth: 1 })))).toEqual(EXPECTED_COUNTS); }); + it('count_distinct does NOT fold across leaves — an outer-level count_unique needs the depth query', () => { + const view = { + grouping: { fields: [{ field: 'business_unit' }, { field: 'status' }] }, + columns: [{ field: 'owner', summary: 'count_unique' as const }], + }; + const leaves = reduceHeaderQuery(CONTIGUOUS, compileListViewGroupQuery(view)); + const opsLeaves = leaves.filter((h) => h.business_unit === 'northgate_operations'); + // Both leaves of the unit see all four owners, so summing over-counts the union. + const folded = opsLeaves.reduce((a, h) => a + (h.count_distinct_owner as number), 0); + expect(folded).toBe(8); + const outer = reduceHeaderQuery(CONTIGUOUS, compileListViewGroupQuery(view, { depth: 1 })); + expect(outer.find((h) => h.business_unit === 'northgate_operations')!.count_distinct_owner).toBe(4); + expect(folded).not.toBe(4); + }); + it('groups the empty key as its own group, keyed null', () => { const rows: Row[] = [...CONTIGUOUS.slice(0, 3), { ...makeRow('x', 9), business_unit: null }]; const headers = reduceHeaderQuery(rows, compileListViewGroupQuery({ grouping: { fields: [{ field: 'business_unit' }] } })); @@ -301,10 +331,83 @@ describe('compileListViewGroupQuery — the group set and every header number ar }); }); -// ─── The mapping table (fork i) ────────────────────────────────────────────── +// ─── The derived members (fork i, ruled) ───────────────────────────────────── + +describe('count_filled / count_empty / percent_filled / percent_empty — ONE COUNT(field) node, derived on the header row', () => { + const DERIVED_VIEW = { + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [ + { field: 'notes', summary: 'count_filled' as const }, + { field: 'id', summary: { type: 'count_empty' as const, field: 'notes' } }, + { field: 'notes', summary: 'percent_filled' as const }, + { field: 'amount', summary: 'percent_empty' as const }, + ], + }; + + it('compile to one `count_` node per summarised field, beside the fieldless count', () => { + expect(compileListViewGroupQuery(DERIVED_VIEW).aggregations).toEqual([ + { function: 'count', alias: 'count' }, + { function: 'count', field: 'notes', alias: 'count_notes' }, + { function: 'count', field: 'amount', alias: 'count_amount' }, + ]); + }); + + it.each([['contiguous', CONTIGUOUS], ['interleaved', INTERLEAVED]] as const)( + 'derive per unit from the header row, in %s order: filled / empty / the two ratios', + (_order, rows) => { + const headers = reduceHeaderQuery(rows, compileListViewGroupQuery(DERIVED_VIEW)); + for (const [unit, size] of UNITS) { + const row = headers.find((h) => h.business_unit === unit)!; + const empty = EXPECTED_EMPTY_NOTES[unit as keyof typeof EXPECTED_EMPTY_NOTES]; + expect(row.count).toBe(size); + expect(deriveColumnSummary(row, 'count_filled', 'notes')).toBe(size - empty); + expect(deriveColumnSummary(row, { type: 'count_empty', field: 'notes' }, 'id')).toBe(empty); + expect(deriveColumnSummary(row, 'percent_filled', 'notes')).toBeCloseTo((size - empty) / size, 12); + expect(deriveColumnSummary(row, 'percent_empty', 'notes')).toBeCloseTo(empty / size, 12); + } + // `amount` is never null: percent_empty is 0 everywhere. + expect(headers.every((h) => deriveColumnSummary(h, 'percent_empty', 'amount') === 0)).toBe(true); + }, + ); + + it('an all-empty group reads filled 0 / percent_filled 0 / percent_empty 1; a count of 0 reads 0 and 1 — no division', () => { + const allNull: Row[] = [5, 10, 15].map((n) => ({ ...makeRow('void_unit', n) })); + const [row] = reduceHeaderQuery(allNull, compileListViewGroupQuery(DERIVED_VIEW)); + expect(row).toEqual({ business_unit: 'void_unit', count: 3, count_notes: 0, count_amount: 3 }); + expect(deriveColumnSummary(row, 'count_filled', 'notes')).toBe(0); + expect(deriveColumnSummary(row, 'count_empty', 'notes')).toBe(3); + expect(deriveColumnSummary(row, 'percent_filled', 'notes')).toBe(0); + expect(deriveColumnSummary(row, 'percent_empty', 'notes')).toBe(1); + const emptyGroup: ListViewGroupHeaderRow = { business_unit: 'x', count: 0, count_notes: 0 }; + expect(deriveColumnSummary(emptyGroup, 'percent_filled', 'notes')).toBe(0); + expect(deriveColumnSummary(emptyGroup, 'percent_empty', 'notes')).toBe(1); + expect(deriveColumnSummary(emptyGroup, 'count_empty', 'notes')).toBe(0); + }); + + it('reads the aggregate members off their own column, `none` and a missing column as undefined', () => { + const row: ListViewGroupHeaderRow = { business_unit: 'x', count: 86, sum_amount: 3741, avg_amount: null }; + expect(deriveColumnSummary(row, 'count', 'anything')).toBe(86); + expect(deriveColumnSummary(row, 'sum', 'amount')).toBe(3741); + expect(deriveColumnSummary(row, 'avg', 'amount')).toBeNull(); + expect(deriveColumnSummary(row, 'min', 'amount')).toBeUndefined(); + expect(deriveColumnSummary(row, 'none', 'amount')).toBeUndefined(); + expect(deriveColumnSummary(row, 'count_filled', 'notes')).toBeUndefined(); + }); + + it('a summary on a field named like the derived column collides loudly', () => { + const err = refusal(() => compileListViewGroupQuery({ + grouping: { fields: [{ field: 'count_notes' }] }, + columns: [{ field: 'notes', summary: 'count_filled' }], + })); + expect(err.reason).toBe('alias_collision'); + expect(err.path).toEqual(['columns', 0, 'summary']); + }); +}); + +// ─── The mapping table ─────────────────────────────────────────────────────── describe('COLUMN_SUMMARY_AGGREGATION — one vocabulary, not two', () => { - it('maps count / count_unique / sum / avg / min / max onto AggregationFunction and names the unmapped members', () => { + it('maps every member: six onto AggregationFunction, four by derivation from count, none as nothing', () => { expect(COLUMN_SUMMARY_AGGREGATION).toEqual({ none: { kind: 'none' }, count: { kind: 'aggregate', function: 'count', fieldless: true }, @@ -313,48 +416,48 @@ describe('COLUMN_SUMMARY_AGGREGATION — one vocabulary, not two', () => { avg: { kind: 'aggregate', function: 'avg', fieldless: false }, min: { kind: 'aggregate', function: 'min', fieldless: false }, max: { kind: 'aggregate', function: 'max', fieldless: false }, - count_empty: { kind: 'unmapped' }, - count_filled: { kind: 'unmapped' }, - percent_empty: { kind: 'unmapped' }, - percent_filled: { kind: 'unmapped' }, + count_filled: { kind: 'derived', from: 'count', fieldless: false }, + count_empty: { kind: 'derived', from: 'count', fieldless: false }, + percent_filled: { kind: 'derived', from: 'count', fieldless: false }, + percent_empty: { kind: 'derived', from: 'count', fieldless: false }, }); - expect(UNMAPPED_COLUMN_SUMMARIES).toEqual(['count_empty', 'count_filled', 'percent_empty', 'percent_filled']); + expect(UNMAPPED_COLUMN_SUMMARIES).toEqual([]); + expect(Object.values(COLUMN_SUMMARY_AGGREGATION).every((m) => m.kind !== 'unmapped')).toBe(true); }); it('aliases a summary `_` and the fieldless count `count`', () => { expect(columnSummaryAlias('sum', 'amount')).toBe('sum_amount'); expect(columnSummaryAlias('count_distinct', 'owner')).toBe('count_distinct_owner'); expect(columnSummaryAlias('count', undefined)).toBe(LIST_VIEW_GROUP_COUNT_ALIAS); - expect(columnSummaryAlias('count', 'owner')).toBe('count_owner'); - }); - - it.each(['count_empty', 'count_filled', 'percent_empty', 'percent_filled'] as const)( - 'refuses a `%s` summary LOUDLY at compile time — code, status and the path of the summary', - (member) => { - const view = { - grouping: { fields: [{ field: 'business_unit' }] }, - columns: [{ field: 'id' }, { field: 'notes', summary: member }], - }; - let caught: unknown; - try { compileListViewGroupQuery(view); } catch (e) { caught = e; } - expect(caught).toBeInstanceOf(ListViewGroupQueryError); - const err = caught as ListViewGroupQueryError; - expect(err.code).toBe('NOT_IMPLEMENTED'); - expect(err.status).toBe(501); - expect(err.reason).toBe('summary_unmapped'); - expect(err.path).toEqual(['columns', 1, 'summary']); - expect(err.message).toMatch(new RegExp(`^Column summary "${member}" on columns\\[1\\] \\(field "notes"\\) has no counterpart in the aggregation vocabulary`)); - expect(err.message).toContain('is an open contract question; until it is ruled, remove the summary from this column'); - expect(err.message).toContain('Nothing is dropped silently.'); - // The refusal is printed at the author: no issue-id token (maintainer ruling 2026-08-12). - expect(err.message).not.toMatch(/#\d+/); - }, - ); + expect(columnSummaryAlias('count', 'notes')).toBe('count_notes'); + }); - it('the refusal codes are standard-catalog members (ADR-0112) — pinned so the literals cannot drift', () => { - const unmapped = new ListViewGroupQueryError('summary_unmapped', [], 'x'); + it('an UNKNOWN summary value (no member at all) is INVALID_QUERY / 400 — a typo, not a capability gap', () => { + const view = { + grouping: { fields: [{ field: 'business_unit' }] }, + columns: [{ field: 'id' }, { field: 'amount', summary: 'median' as unknown as ColumnSummary }], + }; + const err = refusal(() => compileListViewGroupQuery(view)); + expect(err).toBeInstanceOf(ListViewGroupQueryError); + expect(err.reason).toBe('summary_unknown'); + expect(err.code).toBe('INVALID_QUERY'); + expect(err.status).toBe(400); + expect(err.path).toEqual(['columns', 1, 'summary']); + expect(err.message).toMatch(/^Column summary "median" on columns\[1\] \(field "amount"\) is not a column summary function\./); + expect(err.message).not.toMatch(/#\d+/); + const read = refusal(() => deriveColumnSummary({ count: 1 }, 'median' as unknown as ColumnSummary, 'amount')); + expect(read.reason).toBe('summary_unknown'); + expect(read.status).toBe(400); + }); + + it('a DECLARED member with no counterpart stays NOT_IMPLEMENTED / 501 (summary_unmapped) — the machinery, kept for a future member', () => { + const unmapped = new ListViewGroupQueryError('summary_unmapped', ['columns', 0, 'summary'], 'x'); expect(unmapped.code).toBe(StandardErrorCode.enum.NOT_IMPLEMENTED); expect(unmapped.status).toBe(501); + expect(unmapped.path).toEqual(['columns', 0, 'summary']); + const unknown = new ListViewGroupQueryError('summary_unknown', [], 'x'); + expect(unknown.code).toBe(StandardErrorCode.enum.INVALID_QUERY); + expect(unknown.status).toBe(400); const invalid = new ListViewGroupQueryError('alias_collision', [], 'x'); expect(invalid.code).toBe(StandardErrorCode.enum.INVALID_QUERY); expect(invalid.status).toBe(400); @@ -365,11 +468,6 @@ describe('COLUMN_SUMMARY_AGGREGATION — one vocabulary, not two', () => { // ─── Structural refusals ───────────────────────────────────────────────────── describe('compileListViewGroupQuery — refuses what the contract cannot mean', () => { - const refusal = (fn: () => unknown): ListViewGroupQueryError => { - try { fn(); } catch (e) { return e as ListViewGroupQueryError; } - throw new Error('expected a ListViewGroupQueryError'); - }; - it('an alias landing on a grouped field\'s own column', () => { const err = refusal(() => compileListViewGroupQuery({ grouping: { fields: [{ field: 'sum_amount' }] }, @@ -382,6 +480,22 @@ describe('compileListViewGroupQuery — refuses what the contract cannot mean', expect(err.path).toEqual(['columns', 0, 'summary']); }); + it('a grouping field named `count` — it would share its column with the group count, summaries or not', () => { + const bare = refusal(() => compileListViewGroupQuery({ grouping: { fields: [{ field: 'status' }, { field: 'count' }] } })); + expect(bare.reason).toBe('alias_collision'); + expect(bare.code).toBe('INVALID_QUERY'); + expect(bare.path).toEqual(['grouping', 'fields', 1, 'field']); + const withSummary = refusal(() => compileListViewGroupQuery({ + grouping: { fields: [{ field: 'count' }] }, + columns: [{ field: 'id', summary: 'count' }], + })); + expect(withSummary.reason).toBe('alias_collision'); + expect(withSummary.path).toEqual(['grouping', 'fields', 0, 'field']); + // Past the compiled depth, the name is not a groupBy column and does not collide. + expect(compileListViewGroupQuery({ grouping: { fields: [{ field: 'status' }, { field: 'count' }] } }, { depth: 1 }).groupBy) + .toEqual(['status']); + }); + it('an empty grouping', () => { const err = refusal(() => compileListViewGroupQuery({ grouping: { fields: [] } })); expect(err.reason).toBe('grouping_empty'); @@ -475,11 +589,26 @@ describe('compileListViewGroupRowsQuery — rows inside a group are the EXISTING it('refuses a group key that is not a prefix of the nesting order', () => { const grouping = { fields: [{ field: 'business_unit' }, { field: 'status' }] }; for (const key of [{ status: 'done' }, { business_unit: 'x', owner: 'y' }, {}]) { - let caught: unknown; - try { listViewGroupKeyPredicate(grouping, key); } catch (e) { caught = e; } - expect(caught).toBeInstanceOf(ListViewGroupQueryError); - expect((caught as ListViewGroupQueryError).reason).toBe('group_key_not_a_prefix'); - expect((caught as ListViewGroupQueryError).code).toBe('INVALID_QUERY'); + const err = refusal(() => listViewGroupKeyPredicate(grouping, key)); + expect(err).toBeInstanceOf(ListViewGroupQueryError); + expect(err.reason).toBe('group_key_not_a_prefix'); + expect(err.code).toBe('INVALID_QUERY'); } }); + + it('group keys are scalar-valued — an array or object where a key should be is refused', () => { + const grouping = { fields: [{ field: 'business_unit' }, { field: 'status' }] }; + const array = refusal(() => listViewGroupKeyPredicate(grouping, { business_unit: ['a', 'b'] })); + expect(array.reason).toBe('group_key_not_scalar'); + expect(array.code).toBe('INVALID_QUERY'); + expect(array.status).toBe(400); + expect(array.path).toEqual(['grouping', 'fields', 0, 'field']); + const object = refusal(() => listViewGroupKeyPredicate(grouping, { business_unit: 'a', status: { id: 'done' } })); + expect(object.reason).toBe('group_key_not_scalar'); + expect(object.path).toEqual(['grouping', 'fields', 1, 'field']); + // Scalars of every stored kind, and a date instant, pass. + const when = new Date('2026-09-04T00:00:00Z'); + expect(listViewGroupKeyPredicate({ fields: [{ field: 'n' }, { field: 'b' }, { field: 'd' }] }, { n: 7, b: false, d: when })) + .toEqual([{ n: { $eq: 7 } }, { b: { $eq: false } }, { d: { $eq: when } }]); + }); }); diff --git a/packages/spec/src/ui/view-grouping-query.ts b/packages/spec/src/ui/view-grouping-query.ts index 5c895b4608..3923dc3c15 100644 --- a/packages/spec/src/ui/view-grouping-query.ts +++ b/packages/spec/src/ui/view-grouping-query.ts @@ -57,53 +57,83 @@ * `count` IS the group count (it counts every row of the group, filled or * not — the footer's own reading), so it rides the `count` column rather * than minting a second one. + * * for `count_filled` / `count_empty` / `percent_filled` / `percent_empty` + * ONE column per summarised field, `count_` — `COUNT(field)`, the + * non-null count — from which all four are DERIVED on the header row by + * {@link deriveColumnSummary} (see the mapping table). * * A summary alias that would land on a grouped field's own column * (`sum_amount` while grouping by a field named `sum_amount`) is refused, not - * silently overwritten — the two would be one column with two meanings. + * silently overwritten — the two would be one column with two meanings. So + * is a grouping field named `count`: it would share its column with the + * group count. * - * ## The mapping table — `ColumnSummary` → `AggregationFunction` (fork i) + * ## The mapping table — `ColumnSummary` → the aggregation vocabulary (fork i, ruled) * - * | `ListColumn.summary` | aggregation node | note | + * | `ListColumn.summary` | aggregation node | on the header row | * |---|---|---| - * | `none` | (no node) | "no summary" is not a summary | - * | `count` | `{ function: 'count' }` | fieldless — `COUNT(*)`, the group count itself | - * | `count_unique` | `{ function: 'count_distinct', field }` | `COUNT(DISTINCT field)`, nulls excluded | - * | `sum` / `avg` / `min` / `max` | the same name, `field` | | - * | `count_empty` | **refused** | no counterpart — see below | - * | `count_filled` | **refused** | no counterpart — see below | - * | `percent_empty` | **refused** | no counterpart — see below | - * | `percent_filled` | **refused** | no counterpart — see below | - * - * The four refused members are a STOP-AND-REPORT fork on #14556: the seat - * decides whether they map (`count_filled` reads as `COUNT(field)`, which the - * platform defines as the non-null count, while the footer's client-side - * reading also treats `''` and `[]` as empty; `count_empty` is spellable only - * with a per-aggregation `filter: { [field]: { $null: true } }`, which routes - * the whole header query through the engine's in-memory tier today; the two - * `percent_*` members are ratios of those counts, not aggregation functions). - * Until that ruling lands the refusal below IS the contract: a grouped view - * declaring one of them fails to compile LOUDLY, with `code`, `status` and - * the `path` of the offending summary — nothing is dropped and no third - * vocabulary is invented. {@link COLUMN_SUMMARY_AGGREGATION} is typed + * | `none` | (no node) | — "no summary" is not a summary | + * | `count` | `{ function: 'count' }` | `count` — fieldless, `COUNT(*)`, the group count itself | + * | `count_unique` | `{ function: 'count_distinct', field }` | `count_distinct_` — `COUNT(DISTINCT field)`, nulls excluded | + * | `sum` / `avg` / `min` / `max` | the same name, `field` | `_` | + * | `count_filled` | `{ function: 'count', field }` | `count_` — `COUNT(field)`, the non-null count | + * | `count_empty` | the same node | derived: `count − count_` | + * | `percent_filled` | the same node | derived: `count_ / count` (0 when `count` is 0) | + * | `percent_empty` | the same node | derived: `1 − percent_filled` | + * + * The seat ruling on fork (i) (contract review on the card, 2026-09-04): the + * four `*_filled` / `*_empty` members map by DERIVATION from two exact + * counts the vocabulary already has — `COUNT(*)` and `COUNT(field)` — never + * by a per-aggregation `filter` (which would route the whole header query + * through the engine's in-memory tier) and never by a new + * `AggregationFunction` member (six lowering faces and the conformance + * ledger stay untouched). Several of the four over the same field are ONE + * `count_` node. "Empty" is the SERVER's meaning on every face: the + * stored value is `null` (`aggregation-conformance.ts`, the `count(col)` + * rows); the footer's client-side reading, which also treats `''` and `[]` + * as empty, is objectui's to converge under "one vocabulary". The + * `summary_unmapped` refusal (`NOT_IMPLEMENTED` / 501) stays for any FUTURE + * member with no counterpart — {@link UNMAPPED_COLUMN_SUMMARIES} is empty + * today — and {@link COLUMN_SUMMARY_AGGREGATION} is typed * `Record`, so adding a member to `ColumnSummarySchema` - * without deciding its row here fails to type-check. + * without deciding its row here fails to type-check. A value that is not a + * member at all (a typo reaching the helper unparsed) is `INVALID_QUERY` / + * 400 (`summary_unknown`): a typo is not a capability gap. * * ## Multi-level grouping * * `groupBy` carries the grouping fields in nesting order, so the header query * answers one row per LEAF combination. An outer level's header is derived - * from the leaf rows sharing its prefix: `count`, `sum`, `min` and `max` fold - * exactly; `avg` does not (an average of averages weights the groups, not the - * rows). When an outer level must carry an exact `avg`, compile that level's - * own query with `depth` — the same query over the first `depth` grouping - * fields — rather than folding. This is still one query shape. + * from the leaf rows sharing its prefix: `count`, `sum`, `min`, `max` and + * the `count_` column (hence `count_filled` / `count_empty`, and the + * two percents recomputed from the folded counts) fold exactly; `avg` and + * `count_distinct` do NOT — an average of averages weights the groups, not + * the rows, and the same value can be distinct in several leaves at once, so + * per-leaf distinct counts over-count the union. When an outer level must + * carry an exact `avg` or `count_unique`, compile that level's own query + * with `depth` — the same query over the first `depth` grouping fields — + * rather than folding. This is still one query shape. + * + * ## The door — existing, not new + * + * Both compiled queries ride the data endpoint's EXISTING query door: + * `POST /data/:object/query` (`packages/rest/src/rest-server.ts`, the + * `${dataPath}/:object/query` route) validates the body as a + * `FindDataRequest` and hands it to `protocol.findData`, which routes a body + * carrying `groupBy` / `aggregations` to `engine.aggregate` + * (`packages/metadata-protocol/src/protocol.ts`, the `hasGroupBy || + * hasAggregations` branch) and answers `{ object, records, total, hasMore }` + * with the header rows as `records`; `client.data.query()` + * (`packages/client/src/index.ts`) posts there, and the RPC face declares + * `method: 'aggregate'` with an `EngineAggregateOptions` body + * (`data/data-engine.zod.ts`, `DataEngineAggregateRequestSchema`). The row + * page is the same door with the compiled `EngineQueryOptions`. No new + * route, no new wire shape; the platform half of the card PINS that door on + * the compiled queries (the 186-row fixture through the route, on driver-sql + * and on the in-memory tier). * * ## Deliberately NOT here * - * * **The REST door.** No `aggregate` route exists on the data endpoint - * today; which route carries the header query to the grid is the platform - * half of #14556 (item 2 of the card), not the spec half. * * **Lowering the view's `filter` rules to a `FilterCondition`.** Both * inputs here take the view's COMPOSED filter — the same `where` the * view's row query already carries. The rule dialect → AST lowering is @@ -126,7 +156,7 @@ import type { FilterCondition } from '../data/filter.zod'; import type { AggregationFunction, AggregationNode } from '../data/query.zod'; import type { EngineAggregateOptions, EngineQueryOptions } from '../data/data-engine.zod'; -import type { ColumnSummary, GroupingConfig, ListColumn, ListView } from './view.zod'; +import type { ColumnSummary, ColumnSummaryConfig, GroupingConfig, ListColumn, ListView } from './view.zod'; /** * The alias of the per-group TOTAL row count on every header row — a @@ -140,12 +170,16 @@ export const LIST_VIEW_GROUP_COUNT_ALIAS = 'count'; * * * `aggregate` — a node with this `function`; `fieldless` says whether the * node names the summarised field (`COUNT(*)` does not). + * * `derived` — a `count` node over the summarised field (`COUNT(field)`, + * the non-null count, column `count_`), from which the member is + * computed on the header row by {@link deriveColumnSummary}. * * `none` — the member means "no summary"; no node. - * * `unmapped` — no counterpart in `AggregationFunction`; the compiler - * refuses it loudly (fork i on #14556). + * * `unmapped` — no counterpart in the vocabulary; the compiler refuses it + * loudly (`NOT_IMPLEMENTED` / 501). No member is in this state today. */ export type ColumnSummaryAggregation = | { readonly kind: 'aggregate'; readonly function: AggregationFunction; readonly fieldless: boolean } + | { readonly kind: 'derived'; readonly from: 'count'; readonly fieldless: false } | { readonly kind: 'none' } | { readonly kind: 'unmapped' }; @@ -162,10 +196,10 @@ export const COLUMN_SUMMARY_AGGREGATION: Readonly, + where: string, +): Exclude { + const mapping = (COLUMN_SUMMARY_AGGREGATION as Readonly>)[member]; + if (mapping === undefined) { + throw new ListViewGroupQueryError( + 'summary_unknown', + path, + `Column summary "${String(member)}" on ${where} is not a column summary function. ` + + `The members are ${Object.keys(COLUMN_SUMMARY_AGGREGATION).join(' / ')}; ` + + 'a value outside that list is refused as malformed, not treated as a capability gap.', + ); + } + if (mapping.kind === 'unmapped') { + // The refusal is printed AT the author, so it carries the remedy and no + // issue id (the tracking card is in the module note — fork i). + throw new ListViewGroupQueryError( + 'summary_unmapped', + path, + `Column summary "${member}" on ${where} has no counterpart in the ` + + 'aggregation vocabulary (AggregationFunction: count / sum / avg / min / max / count_distinct), ' + + 'so a grouped list view cannot carry it in its group headers yet. Whether ' + + `${UNMAPPED_COLUMN_SUMMARIES.join(' / ')} map onto the vocabulary is an open contract question; ` + + 'until it is ruled, remove the summary from this column or group the view without it. ' + + 'Nothing is dropped silently.', + ); + } + return mapping; +} + +/** + * Read ONE column summary's value off a header row — the aggregate members + * from their `_` column (`count` from `count`), and the four + * DERIVED members from the row's two exact counts: + * + * * `count_filled` = `count_` (`COUNT(field)`, the non-null count) + * * `count_empty` = `count − count_` + * * `percent_filled` = `count_ / count` — a ratio in `0..1`, and `0` + * when `count` is `0` (no division) + * * `percent_empty` = `1 − percent_filled` — so an empty group reads `1` + * + * `undefined` when the row carries no column for the summary (the header + * query was compiled without it) and for `none`; `null` is what the + * aggregate itself answered (`avg` / `min` / `max` over no values). Refuses + * an unknown member (`summary_unknown`) exactly as the compiler does. + * + * `summary` is the member, or the `{ type, field }` object form; `field` is + * the column's own field, which the object form's `field` overrides — the + * same resolution {@link compileListViewGroupQuery} applies. + */ +export function deriveColumnSummary( + row: ListViewGroupHeaderRow, + summary: ColumnSummary | ColumnSummaryConfig, + field: string, +): number | null | undefined { + const member: ColumnSummary = typeof summary === 'string' ? summary : summary.type; + const aggregatedField = typeof summary === 'string' ? field : (summary.field ?? field); + const mapping = columnSummaryMapping(member, [], `field "${aggregatedField}"`); + if (mapping.kind === 'none') return undefined; + if (mapping.kind === 'aggregate') { + const value = row[columnSummaryAlias(mapping.function, mapping.fieldless ? undefined : aggregatedField)]; + return value === undefined ? undefined : (value as number | null); + } + const raw = row[columnSummaryAlias(mapping.from, aggregatedField)]; + if (raw === undefined || raw === null) return undefined; + const filled = Number(raw); + const total = Number(row[LIST_VIEW_GROUP_COUNT_ALIAS]); + const percentFilled = total === 0 ? 0 : filled / total; + switch (member) { + case 'count_filled': return filled; + case 'count_empty': return total - filled; + case 'percent_filled': return percentFilled; + case 'percent_empty': return 1 - percentFilled; + default: return undefined; + } +} + /** * The GROUP HEADER query: the set of groups and every number in every group * header, as ONE `EngineAggregateOptions` for `IDataEngine.aggregate`. @@ -403,6 +517,16 @@ export function compileListViewGroupQuery( ); } const groupBy = names.slice(0, depth); + const countLevel = groupBy.indexOf(LIST_VIEW_GROUP_COUNT_ALIAS); + if (countLevel >= 0) { + throw new ListViewGroupQueryError( + 'alias_collision', + ['grouping', 'fields', countLevel, 'field'], + `grouping.fields[${countLevel}].field is "${LIST_VIEW_GROUP_COUNT_ALIAS}", the column every header row ` + + 'carries its group count under — one column cannot carry both the group key and the count. ' + + 'Rename the field.', + ); + } const aggregations: AggregationNode[] = [ { function: 'count', alias: LIST_VIEW_GROUP_COUNT_ALIAS }, ...summaryAggregationNodes(view.columns, groupBy), @@ -425,7 +549,14 @@ export function compileListViewGroupQuery( * outermost down to the group being opened, and no level past it — so the * rows of an outer group (a `depth`-scoped header) are spellable too. * - * @throws {ListViewGroupQueryError} `group_key_not_a_prefix` (`INVALID_QUERY`) + * Group keys are SCALAR-valued: what a header row carries under a grouped + * field is one stored value (string, number, boolean, bigint, a date instant, + * or `null`). An array or object where a key should be is refused + * (`group_key_not_scalar`) rather than compiled into a predicate that would + * compare a list or a record and select the wrong rows. + * + * @throws {ListViewGroupQueryError} `group_key_not_a_prefix` / + * `group_key_not_scalar` (`INVALID_QUERY`) */ export function listViewGroupKeyPredicate( grouping: GroupingConfig, @@ -446,14 +577,30 @@ export function listViewGroupKeyPredicate( + 'Carry every outer level\'s key, and no field that is not a grouping field.', ); } - return names.slice(0, levels).map((name) => { + return names.slice(0, levels).map((name, level) => { const value = groupKey[name]; + if (!isScalarGroupKey(value)) { + throw new ListViewGroupQueryError( + 'group_key_not_scalar', + ['grouping', 'fields', level, 'field'], + `The group key for "${name}" is ${Array.isArray(value) ? 'an array' : 'an object'}; a group key is ` + + 'one stored scalar value (or null for the empty group), exactly what the header row carries ' + + 'under the grouped field. Pass that value.', + ); + } return value === null || value === undefined ? { [name]: { $null: true } } : { [name]: { $eq: value } }; }); } +/** A value a header row can carry under a grouped field — one stored scalar, or absence. */ +function isScalarGroupKey(value: unknown): boolean { + if (value === null || value === undefined || value instanceof Date) return true; + const type = typeof value; + return type === 'string' || type === 'number' || type === 'boolean' || type === 'bigint'; +} + /** * The PER-GROUP ROW PAGE: the EXISTING paged `find`, as `EngineQueryOptions` * for `IDataEngine.find`, with the group's key predicate AND-ed into the diff --git a/packages/spec/src/ui/view.zod.ts b/packages/spec/src/ui/view.zod.ts index 4ca5b96cc5..70fd82a854 100644 --- a/packages/spec/src/ui/view.zod.ts +++ b/packages/spec/src/ui/view.zod.ts @@ -696,17 +696,23 @@ export type ViewFilterRuleParsed = z.infer; * * `count` → a fieldless `count` (`COUNT(*)`, every row of the group — the * group count itself), `count_unique` → `count_distinct`, and * `sum` / `avg` / `min` / `max` → the same name; `none` declares nothing. - * * `count_empty` / `count_filled` / `percent_empty` / `percent_filled` have - * NO counterpart yet. Their mapping is an open contract question on #14556 - * (fork i); until it is ruled, a grouped view declaring one of them is - * refused LOUDLY by `compileListViewGroupQuery` (`NOT_IMPLEMENTED` / 501, - * with the path of the summary) — never dropped, never given a third - * vocabulary. The footer keeps computing them client-side over the rows - * it holds, as it always has. + * * `count_filled` / `count_empty` / `percent_filled` / `percent_empty` map + * by DERIVATION (seat ruling on fork i, contract review of #14556): one + * `{ function: 'count', field }` node — `COUNT(field)`, the non-null count + * — rides the header row as `count_`, and the four are computed + * from it and the group count by `deriveColumnSummary`: `count_filled` = + * `count_`, `count_empty` = `count − count_`, + * `percent_filled` = `count_ / count` (0 when the count is 0), + * `percent_empty` = `1 − percent_filled`. "Empty" is the SERVER's meaning + * on every face — the stored value is `null` (`aggregation-conformance`); + * the footer's client-side reading, which also treats `''` and `[]` as + * empty, is objectui's to converge under "one vocabulary". * * ⛔ Adding a member here without deciding its row in - * `COLUMN_SUMMARY_AGGREGATION` is a type error by construction, so the fork - * cannot widen silently. + * `COLUMN_SUMMARY_AGGREGATION` is a type error by construction, so the table + * cannot widen silently; a member whose row says "no counterpart" is refused + * loudly by `compileListViewGroupQuery` (`NOT_IMPLEMENTED` / 501, with the + * path of the summary) — none is in that state today. */ export const ColumnSummarySchema = lazySchema(() => z.enum([ 'none', @@ -726,8 +732,11 @@ export const ColumnSummarySchema = lazySchema(() => z.enum([ 'Aggregation function for the column footer summary — and, on a grouped list view, the per-group ' + 'header summary (server-side): count (COUNT(*), the group count), count_unique ' + '(count_distinct), sum, avg, min, max map onto the query AST\'s AggregationFunction; ' - + 'count_empty, count_filled, percent_empty, percent_filled have no counterpart yet and are refused ' - + 'loudly by the group-header compiler (an open contract question) — never dropped silently', + + 'count_filled, count_empty, percent_filled, percent_empty derive from one COUNT(field) node (the ' + + 'non-null count) and the group count — count_filled = COUNT(field), count_empty = count − COUNT(field), ' + + 'percent_filled = COUNT(field) / count (0 when count is 0), percent_empty = 1 − percent_filled. ' + + 'Server-side "empty" is null on every face; the footer\'s client-side reading of empty strings and ' + + 'empty arrays as empty is the renderer\'s to converge', )); /** @@ -886,9 +895,34 @@ export const GroupingFieldSchema = lazySchema(() => strictObject({ * `compileListViewGroupQuery` (1) and `compileListViewGroupRowsQuery` (2), * pinned on the 186-row fixture: 86/61/31/7/1 regardless of row order. * - * Follow-ons, in order: the platform half of #14556 (the route that carries - * the header query on the REST/ObjectQL path — no `aggregate` route exists on - * the data endpoint today), then objectui#7189 (`plugin-grid` consumes the + * ## Known limits of the shape, recorded + * + * * **Group keys are scalar-valued.** A header row carries, under each + * grouped field, one stored value — a lookup's referenced id, a select + * value, a number, a boolean, `null` for the empty group. A date / + * datetime grouping field groups per DISTINCT STORED INSTANT: there is + * no `dateGranularity` on a grouping field (the query AST's bucketed + * `groupBy` member form is not exposed here), so "by month" is not a + * list-view grouping today. + * * **Header cardinality is unbounded.** The header query answers one row + * per group, and `EngineAggregateOptions` carries neither `orderBy` nor + * `limit` — the existing door returns the whole grouped set and slices + * `limit` after aggregation. A high-cardinality grouping field therefore + * returns as many header rows as it has distinct values; bounding that is + * `orderBy` + `limit` on the aggregate verb, an engine-contract card of + * its own, never a change to `order`'s meaning. + * + * ## The door, and the follow-ons + * + * Both queries ride the data endpoint's EXISTING door: `POST + * /data/:object/query` (`packages/rest/src/rest-server.ts`) → `protocol.findData` + * (`packages/metadata-protocol/src/protocol.ts`), which routes a body carrying + * `groupBy` / `aggregations` to `engine.aggregate` and answers `{ object, + * records, total, hasMore }`; `client.data.query()` posts there and the RPC + * face declares `method: 'aggregate'`. No new route, no new wire shape. + * Follow-ons, in order: the platform half of #14556 pins that door on the + * compiled queries (the 186-row fixture through the route, on driver-sql and + * on the in-memory tier), then objectui#7189 (`plugin-grid` consumes the * header rows and stops grouping the page). */ export const GroupingConfigSchema = lazySchema(() => strictObject({ From 49f19218b28dce64d48c7894a27baefe3c2dfa60 Mon Sep 17 00:00:00 2001 From: os-justin Date: Fri, 4 Sep 2026 11:37:28 +0000 Subject: [PATCH 4/4] chore(spec): regenerate api-surface / export-origins / references for deriveColumnSummary and the ColumnSummary describe Regenerated by `check:generated --fix` after the merge was committed (api-surface/ui.json + export-origins/ui.json: the new export; content/docs/references/ui/view.mdx: the ColumnSummarySchema describe). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01H2oQebDDxYKfWZusyd8GXk --- content/docs/references/ui/view.mdx | 2 +- packages/spec/api-surface/ui.json | 1 + packages/spec/export-origins/ui.json | 1 + 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/content/docs/references/ui/view.mdx b/content/docs/references/ui/view.mdx index e88861387e..e1349f86e0 100644 --- a/content/docs/references/ui/view.mdx +++ b/content/docs/references/ui/view.mdx @@ -127,7 +127,7 @@ Compound-cell prefix configuration ## ColumnSummary -Aggregation function for the column footer summary — and, on a grouped list view, the per-group header summary (server-side): count (COUNT(*), the group count), count_unique (count_distinct), sum, avg, min, max map onto the query AST's AggregationFunction; count_empty, count_filled, percent_empty, percent_filled have no counterpart yet and are refused loudly by the group-header compiler (an open contract question) — never dropped silently +Aggregation function for the column footer summary — and, on a grouped list view, the per-group header summary (server-side): count (COUNT(*), the group count), count_unique (count_distinct), sum, avg, min, max map onto the query AST's AggregationFunction; count_filled, count_empty, percent_filled, percent_empty derive from one COUNT(field) node (the non-null count) and the group count — count_filled = COUNT(field), count_empty = count − COUNT(field), percent_filled = COUNT(field) / count (0 when count is 0), percent_empty = 1 − percent_filled. Server-side "empty" is null on every face; the footer's client-side reading of empty strings and empty arrays as empty is the renderer's to converge ### Allowed Values diff --git a/packages/spec/api-surface/ui.json b/packages/spec/api-surface/ui.json index 85509cb65c..4c1bcb7f30 100644 --- a/packages/spec/api-surface/ui.json +++ b/packages/spec/api-surface/ui.json @@ -427,6 +427,7 @@ "defineReport (function)", "defineView (function)", "defineViewItem (function)", + "deriveColumnSummary (function)", "diagnoseViewMetadata (function)", "expandViewContainer (function)", "expandViewContainerWithDiagnostics (function)", diff --git a/packages/spec/export-origins/ui.json b/packages/spec/export-origins/ui.json index c5f9e80beb..81015d2e31 100644 --- a/packages/spec/export-origins/ui.json +++ b/packages/spec/export-origins/ui.json @@ -427,6 +427,7 @@ "defineReport": "src/ui/report.zod.ts#defineReport (function)", "defineView": "src/ui/view.zod.ts#defineView (function)", "defineViewItem": "src/ui/view.zod.ts#defineViewItem (function)", + "deriveColumnSummary": "src/ui/view-grouping-query.ts#deriveColumnSummary (function)", "diagnoseViewMetadata": "src/ui/view.zod.ts#diagnoseViewMetadata (function)", "expandViewContainer": "src/ui/view.zod.ts#expandViewContainer (function)", "expandViewContainerWithDiagnostics": "src/ui/view.zod.ts#expandViewContainerWithDiagnostics (function)",