fix(objectql): the primary key is never a __search companion source (#10290) (#10459)

* fix(objectql): the primary key is never a `__search` companion source (#10290)

* test(objectql,pinyin-search): negative + positive controls for the PK companion source (#10290)

* chore: changeset for the primary-key companion refusal (#10290)

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: Elon Musk

.changeset/companion-source-never-primary-key.md

@@ -0,0 +1,55 @@
+---
+"@objectstack/objectql": minor
+---
+
+The `__search` companion is no longer provisioned or backfilled on objects whose only companion source is the primary key (#10290)
+
+`resolveSearchCompanionSources` resolves the companion's source through
+ADR-0079's `resolveDisplayField`. That derivation ends at "first title-eligible
+field by declaration order", and on a table whose only text column IS its
+primary key — system tables, junction tables, append-only logs — it lands on
+`id`. `id` is `type: 'text'`, not hidden and carries no `requiredPermissions`,
+so it passed the eligibility gate: `provisionSearchCompanion` declared a
+`__search` column on those objects and `plugin-pinyin-search`'s backfill walked
+them at every boot.
+
+That work is doomed by construction rather than merely unlikely. Both writers —
+the `beforeInsert`/`beforeUpdate` stamp and the boot backfill — gate on
+`containsCJK(row[source])`, and a platform-generated primary key is ASCII by
+construction, so the predicate can never be true. Measured on a real
+`bootStack` of `examples/app-showcase`: **20 of the 66 objects** the backfill
+enumerated were in this state, walking whole platform tables to compute nothing
+— `sys_secret`, `sys_oauth_access_token` and `sys_jwks` among them.
+
+`resolveSearchCompanionSources` now returns `[]` when the resolved display
+field is the record's primary key, and `isPrimaryKeyField` is exported as the
+named judgement behind it.
+
+**Keyed on the field's ROLE, not on "resolved by fallback".** The registry's
+materialization seam runs `provisionPrimary(schema, { synthesize: false })`
+before this module — a contractual order — and that pass writes `nameField:
+'id'` onto the document, so by the time provisioning asks, a derived fallback
+and an author's explicit pointer are byte-identical. The role is readable from
+the name because that is where the platform keeps it: the driver provisions
+`id` on every physical table unconditionally and there is no per-field
+`primaryKey` marker in the spec, which is why `isPreservableUnderAudit` already
+keys on `SystemFieldName.ID` for the same reason. `_id` is refused as the
+alternate spelling of the same address.
+
+**This interprets ADR-0079, it does not amend it.** The title contract is
+untouched: `resolveDisplayField` still resolves `id`, `provisionPrimary` still
+designates it, and `resolveRecordDisplayName` still renders the `Record #<id>`
+floor. Only the search normalizer declines to take its input from there — the
+same distinction #4483 drew one seam over on the READ path, where the display
+field's job in the `$search` auto-default is to ORDER the set and never to
+ADMIT a field the exclusions already rejected (`SEARCH_AUTO_EXCLUDED_FIELDS`
+names `id` and `_id`).
+
+**What does not change.** Existing permanently-NULL `__search` columns on
+already-migrated tables stay: ADR-0045 migrations are additive and dropping a
+physical column is a separate decision. Those deployments still stop walking —
+the backfill skips an object whose sources resolve empty even when its schema
+still declares the column. Objects with a real name/title field are unaffected:
+provisioning, write-time stamping and the query-time `$or` clause all behave
+exactly as before, including when the object also declares an `id` field and
+when its display field is a plain text column that is not named `name`/`title`.

packages/objectql/src/core.ts

@@ -31,6 +31,7 @@ export {
   provisionSearchCompanion,
   resolveSearchCompanionSources,
   isCompanionSourceEligible,
+  isPrimaryKeyField,
   isCompanionMatchableTerm,
   isSearchCompanionRequested,
   stripSearchCompanion,

packages/objectql/src/index.ts

@@ -49,6 +49,7 @@ export {
   provisionSearchCompanion,
   resolveSearchCompanionSources,
   isCompanionSourceEligible,
+  isPrimaryKeyField,
   isCompanionMatchableTerm,
   isSearchCompanionRequested,
   stripSearchCompanion,

packages/objectql/src/search-companion.test.ts

@@ -7,12 +7,14 @@
 
 import { describe, it, expect } from 'vitest';
 import type { ServiceObject } from '@objectstack/spec/data';
+import { resolveDisplayField } from '@objectstack/spec/data';
 import {
   SEARCH_COMPANION_FIELD,
   provisionSearchCompanion,
   resolveSearchCompanionSources,
   isCompanionSourceEligible,
   isCompanionMatchableTerm,
+  isPrimaryKeyField,
   containsCJK,
 } from './search-companion';
 import { expandSearchToFilter, resolveSearchFields } from './search-filter';
@@ -195,3 +197,141 @@ describe('containsCJK / isCompanionMatchableTerm', () => {
     expect(isCompanionMatchableTerm('123')).toBe(false);
   });
 });
+
+// ─────────────────────────────────────────────────────────────────────────────
+// [#10290] The primary key is never a companion source.
+//
+// The two blocks below are a matched pair and are meant to be read together:
+// the first is the NEGATIVE control (the defect — it fails on the pre-fix
+// source), the second is the POSITIVE control (the capability survives — it
+// fails if the refusal over-reaches).
+// ─────────────────────────────────────────────────────────────────────────────
+
+/**
+ * The measured shape: a platform table whose only title-eligible column IS its
+ * primary key. `id` is declared first, so ADR-0079's derivation reaches tier 3
+ * ("first title-eligible field by declaration order") and lands on it — which
+ * is how `sys_jwks`, `sys_secret`, `sys_oauth_access_token` and 17 more came to
+ * carry a companion column their values can never fill.
+ */
+const pkOnly = (): any => ({
+  name: 'sys_jwks',
+  fields: {
+    id: { type: 'text', label: 'ID', readonly: true },
+    created_at: { type: 'datetime' },
+    expires_at: { type: 'datetime' },
+  },
+});
+
+/**
+ * The shape the registry ACTUALLY hands this seam. `materializeBaseLayer` runs
+ * `provisionPrimary(schema, { synthesize: false })` first — contractual order —
+ * so the derived fallback has already been written down as an explicit
+ * `nameField` pointer by the time provisioning asks. A refusal keyed on "the
+ * display field resolved by fallback" could not see any difference here, which
+ * is why the refusal is keyed on the field's role instead.
+ */
+const pkOnlyDesignated = (): any => ({ ...pkOnly(), nameField: 'id' });
+
+describe('[#10290] negative control — the primary key is never a companion source', () => {
+  it('refuses a tier-3 derived display field that is the primary key', () => {
+    expect(resolveDisplayField(pkOnly())).toBe('id'); // ADR-0079 is unchanged…
+    expect(resolveSearchCompanionSources(pkOnly())).toEqual([]); // …the companion declines it
+  });
+
+  it('refuses it just the same once `provisionPrimary` has written it down as `nameField`', () => {
+    expect(resolveSearchCompanionSources(pkOnlyDesignated())).toEqual([]);
+  });
+
+  it('refuses the `_id` spelling of the same address', () => {
+    expect(resolveSearchCompanionSources({
+      name: 'sys_legacy',
+      nameField: '_id',
+      fields: { _id: { type: 'text' }, created_at: { type: 'datetime' } },
+    })).toEqual([]);
+  });
+
+  it('declares no `__search` column on such an object (returns it by reference)', () => {
+    const before = pkOnly();
+    expect(provisionSearchCompanion(before)).toBe(before);
+    expect(before.fields[SEARCH_COMPANION_FIELD]).toBeUndefined();
+
+    const designated = pkOnlyDesignated();
+    expect(provisionSearchCompanion(designated)).toBe(designated);
+    expect(designated.fields[SEARCH_COMPANION_FIELD]).toBeUndefined();
+  });
+
+  it('SchemaRegistry: registering one provisions no companion, and ADR-0079 still designates the title', () => {
+    const registry = new SchemaRegistry({ multiTenant: false, searchCompanion: true });
+    registry.registerObject(pkOnly(), 'test-pkg', 'sys');
+    const schema = registry.getObject('sys_jwks')!;
+    expect(schema.fields![SEARCH_COMPANION_FIELD]).toBeUndefined();
+    // The title contract is INTERPRETED, not amended: the pointer is still set.
+    expect((schema as any).nameField).toBe('id');
+  });
+
+  it('isPrimaryKeyField answers for the address spellings only', () => {
+    expect(isPrimaryKeyField('id')).toBe(true);
+    expect(isPrimaryKeyField('_id')).toBe(true);
+    expect(isPrimaryKeyField(undefined)).toBe(false);
+    expect(isPrimaryKeyField('')).toBe(false);
+  });
+});
+
+describe('[#10290] positive control — objects with a real title keep the companion', () => {
+  /** A contact as the platform really registers it: `id` present, `name` too. */
+  const contactWithId = (): any => ({
+    name: 'crm_contact',
+    fields: {
+      id: { type: 'text', label: 'ID', readonly: true },
+      name: { type: 'text', label: 'Name' },
+      email: { type: 'email' },
+    },
+  });
+
+  it('an object carrying an `id` field still gets its companion from the name field', () => {
+    expect(resolveSearchCompanionSources(contactWithId())).toEqual(['name']);
+    const out = provisionSearchCompanion(contactWithId());
+    expect(out.fields[SEARCH_COMPANION_FIELD]).toBeDefined();
+  });
+
+  it('a plain text display field that is not named `name`/`title` is still accepted', () => {
+    // Explicit pointer …
+    expect(resolveSearchCompanionSources({
+      name: 'crm_ticket',
+      nameField: 'subject',
+      fields: { id: { type: 'text' }, subject: { type: 'text' } },
+    })).toEqual(['subject']);
+    // … and a tier-3 DERIVED text column (the same derivation path the negative
+    // control exercises), so the refusal cannot be "all `type: 'text'` sources".
+    expect(resolveSearchCompanionSources({
+      name: 'crm_code',
+      fields: { code: { type: 'text' }, note: { type: 'textarea' } },
+    })).toEqual(['code']);
+  });
+
+  it('a field whose NAME merely contains "id" is not the address', () => {
+    for (const fname of ['identifier', 'id_card', 'bid', 'valid_name', 'external_id_label']) {
+      expect(isPrimaryKeyField(fname)).toBe(false);
+      expect(resolveSearchCompanionSources({
+        name: 'crm_thing',
+        nameField: fname,
+        fields: { [fname]: { type: 'text' } },
+      })).toEqual([fname]);
+    }
+  });
+
+  it('SchemaRegistry: a titled object is still provisioned end-to-end', () => {
+    const registry = new SchemaRegistry({ multiTenant: false, searchCompanion: true });
+    registry.registerObject(contactWithId(), 'test-pkg', 'crm');
+    const schema = registry.getObject('crm_contact')!;
+    expect(schema.fields![SEARCH_COMPANION_FIELD]).toBeDefined();
+    expect((schema.fields![SEARCH_COMPANION_FIELD] as any).hidden).toBe(true);
+  });
+
+  it('the query-time OR clause still fires for a companion-bearing object', () => {
+    const fields = provisionSearchCompanion(contactWithId()).fields as any;
+    const filter = expandSearchToFilter('zhangwei', { fields });
+    expect(filter.$or).toContainEqual({ [SEARCH_COMPANION_FIELD]: { $contains: 'zhangwei' } });
+  });
+});

packages/objectql/src/search-companion.ts

@@ -41,6 +41,7 @@
  */
 
 import { resolveDisplayField, TITLE_ELIGIBLE_TYPES } from '@objectstack/spec/data';
+import { SystemFieldName } from '@objectstack/spec/system';
 
 /**
  * Name of the hidden companion column. Double-underscore prefixed so it can
@@ -92,6 +93,34 @@ function isCompanionSourceType(type: string | undefined): boolean {
   return TITLE_ELIGIBLE_TYPES.has(type);
 }
 
+/**
+ * [#10290] Field names that carry the record's PRIMARY KEY — its address —
+ * rather than any human-authored text.
+ *
+ * Keyed on the NAME because that is where the role lives: the driver
+ * provisions `id` on every physical table unconditionally
+ * (`resolveInjectedSystemColumns` reports it even under `systemFields: false`),
+ * and there is no per-field `primaryKey` marker in the spec for a def-based
+ * test to read — the same reasoning `isPreservableUnderAudit`
+ * (`validation/rule-validator.ts`) already keys on {@link SystemFieldName.ID}.
+ * `_id` is the alternate spelling of the same address: `resolveRecordDisplayName`
+ * reads `record.id ?? record._id`, and the read path's own exclusion set
+ * (`SEARCH_AUTO_EXCLUDED_FIELDS`, `@objectstack/spec/data`) names both, for
+ * this refusal, one seam over.
+ */
+const RECORD_ADDRESS_FIELD_NAMES: ReadonlySet<string> = new Set<string>([SystemFieldName.ID, '_id']);
+
+/**
+ * [#10290] Is `fieldName` the object's primary key?
+ *
+ * Exported so the refusal in {@link resolveSearchCompanionSources} is one
+ * named judgement a caller can ask about, rather than a literal buried in a
+ * conditional.
+ */
+export function isPrimaryKeyField(fieldName: string | undefined | null): boolean {
+  return typeof fieldName === 'string' && RECORD_ADDRESS_FIELD_NAMES.has(fieldName);
+}
+
 /**
  * May `fieldMeta` feed the companion column?
  *
@@ -119,11 +148,62 @@ export function isCompanionSourceEligible(fieldMeta: CompanionFieldMeta | undefi
  * ({@link provisionSearchCompanion}) and the `plugin-pinyin-search` populate
  * hook — deriving both from the same function means there is no stored
  * mapping to drift.
+ *
+ * ## [#10290] The PRIMARY KEY is never a source
+ *
+ * ADR-0079's derivation ends at "first title-eligible field by declaration
+ * order", and on a table whose only text column IS its primary key — system
+ * tables, junction tables, append-only logs — that is `id`. `id` is
+ * `type: 'text'`, not hidden and carries no `requiredPermissions`, so it
+ * passed {@link isCompanionSourceEligible} and the companion was provisioned
+ * on it.
+ *
+ * The work that follows is doomed by construction rather than merely
+ * unlikely. The companion stores search-normalized forms of CJK text and
+ * nothing else: both writers gate on {@link containsCJK} of the source value
+ * (the `beforeInsert`/`beforeUpdate` stamp and the boot backfill). A primary
+ * key is a platform-generated identifier, ASCII by construction, so
+ * `containsCJK` can never be true — the boot backfill's keyset walk over such
+ * an object scans a whole platform table to compute nothing, forever.
+ * Measured on a real `bootStack` of `examples/app-showcase`: **20 of the 66
+ * objects** the backfill enumerated were in this state, `sys_secret`,
+ * `sys_oauth_access_token` and `sys_jwks` among them.
+ *
+ * ⛔ The refusal is keyed on the field's ROLE, NOT on "the display field
+ * resolved by fallback", and that is forced rather than chosen. The registry's
+ * materialization seam (`registry.ts` `materializeBaseLayer`) runs
+ * `provisionPrimary(schema, { synthesize: false })` BEFORE this module — a
+ * contractual order — and that pass WRITES `nameField: 'id'` onto the
+ * document. By the time provisioning asks this function, a derived fallback
+ * and an author's explicit pointer are byte-identical, so "was this a
+ * fallback?" is not a question this seam can answer. It does not need to: the
+ * values cannot qualify either way.
+ *
+ * This INTERPRETS ADR-0079, it does not amend it. The title contract is
+ * untouched — `resolveDisplayField` still resolves `id`, `provisionPrimary`
+ * still designates it, and `resolveRecordDisplayName` still renders the
+ * `Record #<id>` floor. What changes is only that the search normalizer
+ * declines to take its input from there. That is the same distinction #4483
+ * drew on the READ path one seam over: the display field's job in the
+ * `$search` auto-default is to ORDER the set, never to ADMIT a field the
+ * exclusions already rejected (`SEARCH_AUTO_EXCLUDED_FIELDS`, which names `id`
+ * and `_id`). This is that rule on the WRITE/provisioning path.
+ *
+ * Existing permanently-NULL `__search` columns on already-migrated tables are
+ * deliberately left in place: ADR-0045 migrations are additive, and dropping a
+ * physical column is a separate decision. What stops here is new provisioning
+ * and the per-boot walk — the backfill skips an object whose sources resolve
+ * empty, and {@link provisionSearchCompanion} no longer declares the column at
+ * all, so `plugin-pinyin-search`'s own `fields[SEARCH_COMPANION_FIELD]`
+ * early-out closes the walk one guard earlier still.
  */
 export function resolveSearchCompanionSources(schema: CompanionObjectMeta | undefined | null): string[] {
   if (!schema?.fields) return [];
   const display = resolveDisplayField(schema as any);
   if (!display) return [];
+  // [#10290] Before the metadata gate: the primary key's METADATA is a
+  // perfectly ordinary readable text column, so only its name can refuse it.
+  if (isPrimaryKeyField(display)) return [];
   const meta = schema.fields[display];
   return isCompanionSourceEligible(meta) ? [display] : [];
 }