fix(metadata-protocol): refuse a dotted projection instead of widening to every field (#7532) (#7588)

`POST /api/v1/data/:object/query {"fields":["name","account.name"]}` answered 200
carrying every business field — strictly more than was asked for — and no resolved
`account.name`. `GET ?$select=name,account.name` did the same.

`assertProjectionFieldsExist` validated only `f.split('.')[0]`, so a dotted entry
cleared the #4226 unknown-name gate on its head segment and reached the driver as a
projection column. Measured on a real SqlDriver (better-sqlite3), a dotted projection
comes back byte-identical to no projection at all: Knex renders `"account"."name"`
against a table that was never joined, sqlite answers `no such column`, and the
#3821 recovery ladder retries `select('*')`.

A dotted entry is now 400 INVALID_FIELD, ordered `unknown` > `dotted` to match
`assertSortFieldsExist`. The message names the relationship it crossed and sends the
caller to `expand`; a dotted path on a real non-reference column gets its own wording.
Follows #5918, which ruled the same way on the analytics measures axis for the same
reason — there is no correct answer to converge on.

The shape rejection's hint no longer prescribes `select=owner.name`: it pointed at the
widening defect, the dead end #6924 removed from the sort axis' hint.

`resolveQueryFields` is unchanged (additive call only, #7534 in flight on the filter
axes). SqlDriver's recovery ladder is deliberately untouched and filed separately.


Claude-Session: https://claude.ai/code/session_01TXK7QuSmTF3DKWji91cmkn

Co-authored-by: Claude <noreply@anthropic.com>

Author: claude[bot]

.changeset/dotted-projection-refused.md

@@ -0,0 +1,49 @@
+---
+"@objectstack/metadata-protocol": patch
+---
+
+fix(metadata-protocol): refuse a dotted `fields`/`$select` entry instead of widening the response to every field (#7532)
+
+`POST /api/v1/data/:object/query` with `{"fields":["name","account.name"]}` answered
+`200` carrying **every** business field — strictly more data than was asked for — and
+no resolved `account.name`. `GET /api/v1/data/:object?$select=name,account.name` did
+the same. A parameter whose entire purpose is to return LESS had "return more" as its
+failure mode, pointing away from both FLS and data minimisation.
+
+`assertProjectionFieldsExist` validated only `f.split('.')[0]`, so a dotted entry
+cleared the #4226 unknown-name gate on its **head** segment (`account` really is a
+field) and travelled on to the driver as a projection column. Measured on a real
+`SqlDriver` (better-sqlite3):
+
+```
+no projection                  -> account amount created_at id name status updated_at
+fields ['name']                -> name                        (a plain name narrows)
+fields ['name','account.name'] -> account amount created_at id name status updated_at
+fields ['account.name']        -> account amount created_at id name status updated_at
+```
+
+The dotted rows are byte-identical to no projection at all. Knex renders
+`"account"."name"` against a table that was never joined, sqlite answers `no such
+column`, and the driver's #3821 recovery ladder retries `select('*')` because rows
+matter more than the projection.
+
+A dotted entry on this axis is now `400 INVALID_FIELD`, with the `unknown` > `dotted`
+precedence {@link assertSortFieldsExist} already applies, so the two axes report the
+same complaint first. The message names the relationship it tried to cross and sends
+the caller to `expand` — the sanctioned door for related data here — or to
+denormalising the value onto the queried object. A dotted path whose head is a real
+but non-reference column gets its own wording, since `expand` would be the wrong
+prescription for it.
+
+This also settles the second half of the report: an unknown **plain** column was a
+`400` while an unknown **dotted** one was a `200` with every field, so one mistake got
+opposite verdicts on one endpoint depending on how it was spelled. Both doors —
+`POST /query` body `fields` and `GET ?$select=` — fold into the same slot before the
+gate and are pinned separately.
+
+Nothing that worked stops working: no driver ever resolved these paths. Plain
+projections still narrow, unknown plain columns still refuse per #4226, an unknown
+head still gets the unknown-name verdict with its did-you-mean, and `expand` still
+delivers related records. The engine's internal-caller projection tolerance and
+`SqlDriver`'s recovery ladder are deliberately untouched — refusing at ingress is what
+stops a request reaching them with a projection no driver can apply.

packages/metadata-protocol/src/protocol.ts

@@ -5597,6 +5597,52 @@ export class ObjectStackProtocolImplementation implements
      * narrowing it cost no caller anything; nothing equivalent has been measured
      * for the projection axis, and this sentence is not a licence to assume it.
      *
+     * [#7532] The DOTTED leg, which this gate used to pass on its head
+     * segment. `f.split('.')[0]` is what let `fields=['name','account.name']`
+     * through: `account` IS a field, so the entry cleared the unknown-name
+     * check above and reached the driver as a projection column. Measured at
+     * that commit on a REAL `SqlDriver` (better-sqlite3), against the same
+     * object the card reports:
+     *
+     * ```
+     * no projection              -> account amount created_at id name status updated_at
+     * fields ['name']            -> name                          (a plain name narrows)
+     * fields ['name','account.name'] -> account amount created_at id name status updated_at
+     * fields ['account.name']    -> account amount created_at id name status updated_at
+     * ```
+     *
+     * The dotted rows are BYTE-IDENTICAL to no projection at all — the exact
+     * "asked for less, received more" this axis' first paragraph describes,
+     * reached by a different route. Knex renders `"account"."name"` against a
+     * table that was never joined, sqlite answers `no such column`, and
+     * `SqlDriver`'s #3821 recovery ladder retries `select('*')` because rows
+     * matter more than the projection. That ladder is a DRIVER-side tolerance
+     * for internal callers and is deliberately left alone here (filed
+     * separately as defence-in-depth); refusing at this ingress is what stops a
+     * request from reaching it carrying a projection no driver can apply.
+     *
+     * It also settles the card's second complaint: an unknown PLAIN column was
+     * a 400 while an unknown DOTTED one was a 200 with every field, so one
+     * mistake got opposite verdicts on one endpoint depending on spelling.
+     *
+     * The governing precedent is #5918 on the analytics MEASURES axis, which
+     * faced this exact shape and ruled the same way: refuse the dotted member
+     * loudly, naming the caller's original spelling, *because there is no
+     * correct answer to converge on*. That is the distinction from #5739, where
+     * refusing would have rejected queries that already compiled correctly.
+     * Here — as there — nothing resolved these paths, so both the typo
+     * (`titel.name`) and the genuine traversal intent (`account.name`) eat this
+     * 400: the two are not separable at this door, and the alternative is the
+     * over-return above.
+     *
+     * NOT a removal of a working feature — nothing resolved these paths. The
+     * spec's `fields` description, `query-syntax.mdx`, `data/query.mdx` and the
+     * `query.joins` / nested-select retirement prescriptions all still offer a
+     * dotted `fields` path as the way to read one related column; every one of
+     * them describes behaviour no driver implements. Aligning that prose with
+     * `expand` is spec/docs surface with its own blast radius and is called out
+     * on the PR rather than smuggled in here.
+     *
      * [#4196] It also owns the projection's SHAPE, which is a different
      * question from its names and is answered first — see below.
      */
@@ -5620,9 +5666,15 @@ export class ObjectStackProtocolImplementation implements
                     ? ' The nested-select object form `{ field, fields, alias }` was removed in '
                       + '@objectstack/spec 17 (#4196) — no engine or driver ever read it.'
                     : '')
-                + " Select related records with `expand` (`expand=owner` / `{ expand: { owner: "
-                + "{ object: 'user', fields: ['name'] } } }`), or name one related column with a "
-                + 'dotted path (`select=owner.name`).',
+                // [#7532] The dotted-path half of this prescription is GONE.
+                // It pointed at a spelling this same gate now refuses — and
+                // before that refusal it pointed at a spelling no driver
+                // resolves, which answered with every field. Naming it here
+                // sent the author from one refusal straight into the widening
+                // defect, the same dead end #6924 removed from the SORT axis'
+                // hint. `expand` is the one door for related data on this axis.
+                + " Select related records with `expand` (`expand=owner`, or `{ expand: { owner: "
+                + "{ object: 'user', fields: ['name'] } } }` to choose its columns).",
             );
             err.code = 'INVALID_FIELD';
             err.status = 400;
@@ -5632,24 +5684,68 @@ export class ObjectStackProtocolImplementation implements
         }
         const gate = this.resolveQueryFields(object);
         if (!gate) return;
-        const unknown = (fields as string[]).filter((f) => !gate.known.has(f.split('.')[0]));
-        if (unknown.length === 0) return;
-        const first = unknown[0];
-        const err: any = new Error(
-            `Unknown field '${first}' on object '${object}'`
-            + (unknown.length > 1 ? ` (also: ${unknown.slice(1).join(', ')})` : '')
-            + `. '${param}' chooses which fields to return; dropping an unknown one silently `
-            + 'answered a NARROWER projection with a WIDER one — a projection naming no known '
-            + 'field fell all the way back to every field.'
-            + suggestFieldName(first, gate.declared),
+        const names = fields as string[];
+        const unknown = names.filter((f) => !gate.known.has(f.split('.')[0]));
+        if (unknown.length > 0) {
+            const first = unknown[0];
+            const unknownErr: any = new Error(
+                `Unknown field '${first}' on object '${object}'`
+                + (unknown.length > 1 ? ` (also: ${unknown.slice(1).join(', ')})` : '')
+                + `. '${param}' chooses which fields to return; dropping an unknown one silently `
+                + 'answered a NARROWER projection with a WIDER one — a projection naming no known '
+                + 'field fell all the way back to every field.'
+                + suggestFieldName(first, gate.declared),
+            );
+            unknownErr.code = 'INVALID_FIELD';
+            unknownErr.status = 400;
+            unknownErr.field = first;
+            unknownErr.fields = unknown;
+            unknownErr.object = object;
+            unknownErr.param = param;
+            throw unknownErr;
+        }
+        // [#7532] The DOTTED verdict — the leg the head-segment check above
+        // does not cover, and the one that made this axis fail in the very
+        // direction its own docblock warns about.
+        //
+        // Ordered `unknown` > `dotted`, the same precedence
+        // {@link assertSortFieldsExist} applies, so the two axes agree about
+        // which complaint a caller hears first when an entry is both.
+        //
+        // It sits AFTER the `gate` early-return for the same reason the sort
+        // axis' dotted verdict does: the relation-vs-not split below reads
+        // `gate.fields`, and a registry-less host has no field map to read.
+        const dotted = names.filter((f) => f.includes('.'));
+        if (dotted.length === 0) return;
+        const first = dotted[0];
+        const head = first.split('.')[0];
+        const headDef: any = gate.fields[head];
+        const crossesRelation = headDef != null && REFERENCE_VALUE_TYPES.has(headDef.type);
+        const dottedErr: any = new Error(
+            (crossesRelation
+                ? `Field '${first}' on object '${object}' follows the relationship '${head}' into `
+                  + `another object — '${param}' reaches only columns of '${object}' itself`
+                : `Field '${first}' on object '${object}' is a dotted path — '${param}' reaches only `
+                  + `whole columns of '${object}', not values inside them`)
+            + (dotted.length > 1 ? ` (also: ${dotted.slice(1).join(', ')})` : '')
+            + '. No driver resolves it: the path reaches the driver as a column name, matches no '
+            + 'column, and the projection falls back to EVERY field — a narrower request answered '
+            + 'with a wider response, which is the same failure the unknown-name refusal above '
+            + 'exists to stop.'
+            + (crossesRelation
+                ? ` Read the related record with 'expand' (\`expand=${head}\`, or `
+                  + `\`{ expand: { ${head}: { object: '<target>', fields: ['<column>'] } } }\` to `
+                  + `choose its columns), or denormalise the value onto '${object}' (a stored `
+                  + 'field, written when the source changes) and name that.'
+                : ` Name the whole column ('${head}') and read into its value in the caller.`),
         );
-        err.code = 'INVALID_FIELD';
-        err.status = 400;
-        err.field = first;
-        err.fields = unknown;
-        err.object = object;
-        err.param = param;
-        throw err;
+        dottedErr.code = 'INVALID_FIELD';
+        dottedErr.status = 400;
+        dottedErr.field = first;
+        dottedErr.fields = dotted;
+        dottedErr.object = object;
+        dottedErr.param = param;
+        throw dottedErr;
     }
 
     /**