fix(service-analytics): compile the $icontains ASCII fold per dialect — translate() is not a SQLite function

All three of this package's SQL compilers spelled the #6520 fold as
`translate(col, 'ABC…', 'abc…')` on EVERY dialect. `translate()` is
PostgreSQL/Oracle; SQLite has none, so on a SQLite datasource an analytics
`where` carrying `$icontains` — and an ADR-0021 D-C read scope carrying it —
compiled a statement the engine refuses to parse. Measured on sql.js 1.14.1
(SQLite 3.49.1, the engine driver-sqlite-wasm runs): `SELECT
translate('ABC','ABC','abc')` answers `no such function: translate`.

`$icontains` now goes through `text-match-sql.ts`'s per-dialect construct table
with one `fold` flag, set on that operator alone:

  - sqlite   → `lower(col) GLOB lower(?)`, ASCII-only there (`lower('CAFÉ')` is
               `cafÉ`), which is the #4706 Q1 = A boundary rather than an
               approximation of it.
  - postgres → `translate()`, byte-identical to before. Never broken.
  - unknown  → `translate()`, byte-identical to before. The residue keeps the
               shape it had; note this diverges from driver-sql, whose unknown
               arm folds with LOWER(), and neither face claims the other's.
  - mysql    → the nested-REPLACE fold over CAST(… AS BINARY), matching
               driver-sql. TEXT ONLY — no MySQL server is provisionable here.

The `sql` keyword field on `ObjectQLStrategy`'s LIKE_SQL_OPS lost its last
reader in this move and is removed: a dead field named `sql` beside a compiler
invites exactly the misreading this defect was.

#15684's `$icontains` control asserted "the fold arm still emits translate() on
every dialect". That was a PROXY for the property it protected — the two text
families must not collapse onto one path — and this change makes the fold
dialect-DEPENDENT by design, so the proxy no longer states the property. It is
re-aimed rather than deleted or loosened: `$icontains` and `$contains` must now
compile to DIFFERENT text on each dialect, and `$contains` must carry no fold in
any of its three spellings. That discriminates against the collapse in both
directions where dialect-invariance discriminated against one.

Fixes #15780

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XpTx2tbq3pZRYAdoGt6E6Y

Author: os-warren

.changeset/analytics-icontains-per-dialect-fold.md

@@ -0,0 +1,15 @@
+---
+"@objectstack/service-analytics": patch
+---
+
+Analytics `$icontains` no longer compiles a `translate()` call on SQLite and MySQL, where that function does not exist and the statement failed to parse.
+
+`$icontains` folds ASCII case on both sides of the comparison (#4706 Q1 = A). All three of this package's SQL compilers — the query's own `where` (`NativeSQLStrategy.buildFilterClause`), the ADR-0021 D-C read scope (`compileScopedFilterToSql`) and the `ObjectQLStrategy` echo of that statement — spelled that fold as `translate(col, 'ABC…', 'abc…')` on **every** dialect. `translate()` is PostgreSQL/Oracle; SQLite has none. Measured on sql.js 1.14.1 (SQLite 3.49.1, the engine `driver-sqlite-wasm` runs), `SELECT translate('ABC','ABC','abc')` answers `no such function: translate` — so this was not a filter that returned the wrong rows, it was a statement the engine refused. On a SQLite datasource, an analytics `where` carrying `$icontains` and an **RLS read scope** carrying it were both unusable.
+
+The fold is now chosen per dialect, on the same construct table the case-exact text family already used, reached through one `fold` flag:
+
+- **SQLite** — `lower(col) GLOB lower(?)`. SQLite's `lower()` is ASCII-only (measured: `lower('CAFÉ')` is `cafÉ`), so this is the ruled fold rather than an approximation of it, and it runs.
+- **PostgreSQL** and the `unknown` residue (a host that wires no dialect hook) — `translate()`, **unchanged**. These arms were never broken, so the emitted SQL and its bound parameters are byte-identical to before.
+- **MySQL** — the nested-`REPLACE` fold over `CAST(… AS BINARY)`, matching what `driver-sql` emits for the same operator. Asserted as text only; no MySQL server is provisionable in the container that wrote this, so that cell is a declared skip, not a claimed pass.
+
+`$icontains` and the case-sensitive `$contains` family remain two separate constructs on every dialect — collapsing them would give `$contains` back the case fold #4706 Q2 = A took away from it. A host that answers no dialect keeps exactly the behaviour it had.

packages/services/service-analytics/src/__tests__/icontains-dialect-sql.test.ts

@@ -50,18 +50,40 @@
  * — on the same engine, and requires the same row sets from both. A third
  * hand-copy of the table anywhere is the thing to refuse.
  *
- * ## What is deliberately NOT changed
+ * ## What #15684 deliberately did not change, and what #15780 then did
  *
- * `$icontains` (#6520) keeps its own construct on every dialect: it folds BOTH
- * sides through `asciiLowerSqlExpr`, and the assertions below pin that the
- * emitted text is untouched by the dialect. That fold is `translate()`, which
- * SQLite does not have (measured: `no such function: translate` on sql.js
- * 1.14.1) — so those statements are pinned as TEXT and are NOT executed here.
- * That is a separate defect, filed as #15780, and this suite's `$icontains`
- * assertions are the control that must stay unchanged while it is open.
- * Escaping (#5567) is likewise unchanged for every `LIKE` arm; the GLOB arm
- * brings its OWN escaped character class (`*`, `?`, `[`), which is why the
- * second fixture below exists.
+ * As written, this suite pinned `$icontains` (#6520) as a CONTROL: it kept its
+ * own construct on every dialect, folding both sides through
+ * `asciiLowerSqlExpr`, and the assertion below pinned that its emitted text was
+ * untouched by the dialect. The property that control protected is the one that
+ * still matters: **the two text families must not collapse onto one path**, or
+ * `$contains` gets back the fold #4706 Q2 = A took away from it. The pin was
+ * "the fold arm still emits `translate()` on every dialect" only because, while
+ * #15684's scope was the case-exact four, dialect-invariance was a cheap
+ * PROXY for family-separation — the two happened to coincide.
+ *
+ * They stopped coinciding. That same `translate()` is a PostgreSQL/Oracle
+ * function SQLite does not have (measured: `no such function: translate` on
+ * sql.js 1.14.1), so `$icontains` did not merely go unpinned on SQLite — it
+ * failed to PARSE there. That was #15780, and closing it moved `$icontains`
+ * onto this same per-dialect table with a `fold` flag, which makes its emitted
+ * text dialect-DEPENDENT by design. The old assertion could then only be read
+ * two ways: as a defect it must go red for, or as a statement of the property,
+ * which it no longer is.
+ *
+ * ⛔ It was therefore neither deleted nor loosened — it was re-aimed at the
+ * property itself, which is now pinned DIRECTLY and more tightly than the proxy
+ * ever did (`$icontains and the case-EXACT family stay two constructs…`
+ * below): on each dialect, `$icontains` and `$contains` must compile to
+ * DIFFERENT text, and `$contains` must carry no fold. That discriminates
+ * against the collapse in both directions, where dialect-invariance only
+ * discriminated against one. The row sets that make it more than a text
+ * comparison are executed in `icontains-dialect-sql.test.ts`, which owns the
+ * `$icontains` half of the family from here on.
+ *
+ * Escaping (#5567) is unchanged for every `LIKE` arm; the GLOB arm brings its
+ * OWN escaped character class (`*`, `?`, `[`), which is why the second fixture
+ * below exists.
  */
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
@@ -234,22 +256,37 @@ describe('[#15684] the compiled TEXT, per dialect', () => {
       .toEqual({ sql: 'CAST("t"."name" AS BINARY) LIKE CAST(? AS BINARY) ESCAPE ?', params: ['%acme%', '\\'] });
   });
 
-  it('$icontains is untouched by the dialect — the fold arm still emits translate() on both sides', async () => {
-    // The control that must stay green. #6520's construct is case-INSENSITIVE
-    // by ruling, so it never wants the case-exact table; if a future edit routes
-    // it through `text-match-sql.ts`, the `$contains` family gets back the fold
-    // #4706 Q2 = A took away from it and this line reds first.
+  it('$icontains and the case-EXACT family stay two constructs on EVERY dialect', async () => {
+    // [#15684 → #15780] The control, re-aimed. This assertion used to read "the
+    // fold arm still emits translate() on every dialect", which was a PROXY for
+    // the property below and stopped being one when #15780 gave `$icontains` a
+    // per-dialect fold of its own. See this file's header for the full reading.
+    //
+    // The property is family SEPARATION: `$icontains` folds (#4706 Q1 = A) and
+    // `$contains` must not (#4706 Q2 = A). A collapse in EITHER direction reds
+    // here — the fold leaking onto `$contains`, or `$contains`' bare construct
+    // being handed to `$icontains`.
     for (const dialect of [undefined, 'sqlite', 'postgres', 'mysql'] as const) {
-      const out = await nativeSql({ name: { $icontains: 'acme' } }, dialect);
-      expect(out.sql, String(dialect)).toContain(
-        "WHERE translate(name, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') LIKE translate($1,",
-      );
-      expect(out.sql, String(dialect)).toContain('ESCAPE $2');
-      expect(out.params, String(dialect)).toEqual(['%acme%', '\\']);
+      const icontains = await nativeSql({ name: { $icontains: 'acme' } }, dialect);
+      const contains = await nativeSql({ name: { $contains: 'acme' } }, dialect);
+      expect(icontains.sql, String(dialect)).not.toBe(contains.sql);
+      // `$contains` carries NO fold, in any of the three spellings a fold has.
+      expect(contains.sql, String(dialect)).not.toMatch(/translate\(|lower\(|REPLACE\(/);
+      // …and `$icontains` carries exactly one of them, per dialect.
+      const FOLD_PER_DIALECT: Record<string, RegExp> = {
+        undefined: /translate\(name, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'/,
+        postgres: /translate\(name, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'/,
+        sqlite: /lower\(name\) GLOB lower\(\$1\)/,
+        mysql: /REPLACE\(CAST\(name AS BINARY\), 'A', 'a'\)/,
+      };
+      expect(icontains.sql, String(dialect)).toMatch(FOLD_PER_DIALECT[String(dialect)]);
     }
-    expect(
-      compileScopedFilterToSql({ name: { $icontains: 'acme' } } as FilterCondition, 't', { dialect: 'sqlite' }).params,
-    ).toEqual(['%acme%', '\\']);
+    // The read scope, the compiler where the collapse would be an ADR-0021
+    // over-reach rather than a wrong chart, holds the same separation.
+    const filterOf = (op: '$contains' | '$icontains') =>
+      compileScopedFilterToSql({ name: { [op]: 'acme' } } as FilterCondition, 't', { dialect: 'sqlite' });
+    expect(filterOf('$icontains').sql).not.toBe(filterOf('$contains').sql);
+    expect(filterOf('$contains').sql).not.toMatch(/lower\(/);
   });
 });
 

packages/services/service-analytics/src/__tests__/text-operator-case-exactness.test.ts

@@ -108,16 +108,18 @@
  *
  *   - {@link likePattern} / {@link escapeLikePattern} build the LIKE PATTERN,
  *     and are used by the `LIKE` arms — Postgres, MySQL, and the `unknown`
- *     residue — plus `$icontains` on every dialect.
+ *     residue — for BOTH text families.
  *   - Which KEYWORD those patterns hang off, and whether a GLOB pattern with a
  *     different escaped character class is built instead, is
  *     `text-match-sql.ts`'s answer. ⛔ Do not re-derive it here.
  *
- * `$icontains` IS implemented here since #6520, and it is a separate construct
- * rather than a flag on the family above: it folds ASCII case on BOTH sides via
- * {@link asciiLowerSqlExpr}, while the `$contains` family stays case-EXACT. The
- * two must not be collapsed — a shared "case-insensitive" path would give the
- * `$contains` family the fold the ruling took away from it.
+ * [#15780] `$icontains` (#6520) goes through that same table, and its fold is a
+ * per-dialect construct too — {@link asciiLowerSqlExpr} is only the Postgres /
+ * `unknown` arm of it. What the two families share is the TABLE and the
+ * escaping; what separates them is one `fold` flag set on the `$icontains` row
+ * alone. ⛔ The two must never be collapsed into one case-insensitive path —
+ * that would give the `$contains` family back the fold #4706 Q2 = A took away
+ * from it, which is the failure `text-operator-case-exactness.test.ts` guards.
  *
  * ## `String(value)` is safe here because nothing unrenderable reaches it (#5234)
  *
@@ -177,9 +179,17 @@ export function likePattern(shape: LikeShape, value: unknown): string {
   return shape === 'starts' ? `${escaped}%` : shape === 'ends' ? `%${escaped}` : `%${escaped}%`;
 }
 
-/** `A`..`Z` and the `a`..`z` they fold onto — the #4706 Q1 = A domain, as data. */
-const ASCII_UPPER_LETTERS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
-const ASCII_LOWER_LETTERS = 'abcdefghijklmnopqrstuvwxyz';
+/**
+ * `A`..`Z` and the `a`..`z` they fold onto — the #4706 Q1 = A domain, as data.
+ *
+ * [#15780] EXPORTED, because the fold is now chosen per dialect and two of the
+ * three arms are built from this domain rather than from `translate()`:
+ * `text-match-sql.ts`'s MySQL arm nests one `REPLACE` per letter. The domain
+ * itself stays here, in one copy — a second 26-character literal anywhere is
+ * how the Postgres arm and the MySQL arm start folding different alphabets.
+ */
+export const ASCII_UPPER_LETTERS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
+export const ASCII_LOWER_LETTERS = 'abcdefghijklmnopqrstuvwxyz';
 
 /**
  * [#6520] Wrap a SQL expression in `$icontains`' ASCII-ONLY case fold.
@@ -199,24 +209,28 @@ const ASCII_LOWER_LETTERS = 'abcdefghijklmnopqrstuvwxyz';
  * alone, so it is the fold the ruling names rather than the one the database
  * happens to offer.
  *
- * ## The dialect this assumes, stated so it can go red rather than stale
- *
- * Postgres. `translate()` is Postgres/Oracle; SQLite has no such function — and
- * [#15684] MEASURED that these compilers' statements do reach SQLite, so the
- * warning this paragraph used to write in the conditional is now a live defect,
- * filed as #15780: on sql.js 1.14.1, `SELECT translate('ABC','ABC','abc')`
- * answers `no such function: translate`, so an `$icontains` in an analytics
- * `where` or in an RLS read scope over a SQLite datasource does not merely
- * over-match — it fails to parse.
- *
- * ⛔ Do NOT close that by falling back to `LOWER()`, which would silently
- * restore the Unicode fold this function exists to avoid. The remedy is one
- * more arm on `text-match-sql.ts`'s per-dialect table, whose shapes
- * `driver-sql`'s `textMatchPredicate` already carries: `lower(col) GLOB
- * lower(?)` on SQLite (measured ASCII-only there — `lower('CAFÉ')` is
- * `cafÉ`), nested `REPLACE` over `CAST(… AS BINARY)` on MySQL. #15684
- * deliberately did not build it: its scope was the case-EXACT four, and its
- * suite pins THIS expression as the control that must stay unchanged.
+ * ## The dialect this arm is FOR — no longer the dialect it assumes
+ *
+ * Postgres, and the `unknown` residue. `translate()` is Postgres/Oracle and
+ * SQLite has no such function, so while this expression was emitted on EVERY
+ * dialect it did not merely over-match on SQLite — it failed to PARSE
+ * (`no such function: translate` on sql.js 1.14.1). That was #15780, and it is
+ * closed: this function is now ONE arm of `text-match-sql.ts`'s per-dialect
+ * table ({@link textMatchPredicateSql}, `fold: true`), reached on `postgres`
+ * and on `unknown`, while SQLite gets `lower(col) GLOB lower(?)` and MySQL the
+ * nested-`REPLACE` binary fold.
+ *
+ * ⛔ Do NOT "simplify" this to `LOWER()`, on any arm. Postgres' `LOWER()` is
+ * locale-aware and would silently restore the Unicode fold #4706 Q1 = A rules
+ * out; SQLite's `lower()` is ASCII-only, which is why the SQLite arm may use it
+ * and this one may not. That asymmetry is the whole reason the fold is chosen
+ * per dialect rather than written once.
+ *
+ * ⛔ Nor is `unknown` free to adopt `driver-sql`'s residue: that face folds
+ * `unknown` with `LOWER()` because `LOWER()` is the shape it emitted before
+ * #6518. This face's pre-existing shape is `translate()`, so `translate()` is
+ * what its residue keeps. Each side keeps its own, and neither claims the
+ * other's — an `unknown` dialect is by definition one nothing here measured.
  *
  * The caller must apply it to BOTH sides of the comparison. Folding only the
  * comparand compares a folded needle against a raw column and matches just the

packages/services/service-analytics/src/like-pattern.ts

@@ -2,7 +2,7 @@
 
 import type { FilterCondition } from '@objectstack/spec/data';
 import type { RegisteredErrorCode } from '@objectstack/spec/api';
-import { likePattern, LIKE_ESCAPE_CHAR, asciiLowerSqlExpr, type LikeShape } from './like-pattern.js';
+import { type LikeShape } from './like-pattern.js';
 import { textMatchPredicateSql, normalizeSqlDialect } from './text-match-sql.js';
 import { textOperatorPolarity } from './non-text-column.js';
 import {
@@ -800,13 +800,15 @@ function textMatch(
   negate: boolean,
   params: unknown[],
   opts: ReadScopeCompileOptions,
+  fold = false,
 ): string {
   return textMatchPredicateSql({
     dialect: normalizeSqlDialect(opts.dialect),
     column: col,
     shape,
     value: val,
     negate,
+    fold,
     bind: (v) => bind(params, v),
   });
 }
@@ -1331,19 +1333,25 @@ function compileOperator(
      * the rows already lower-case — and on a read scope that is a row set the
      * policy author never wrote, in the narrowing direction here but in the
      * WIDENING direction under a `$not`.
+     *
+     * [#15780] …and WHICH fold is the DIALECT's answer, exactly as the keyword
+     * is for the case-exact arms. This line used to spell its own binds and
+     * emit `translate()` unconditionally, on the reasoning that a
+     * case-INSENSITIVE operator never wants the per-dialect case-EXACT
+     * construct. The first half of that was right and the second half hid the
+     * defect: `translate()` is Postgres/Oracle, so on a SQLite datasource this
+     * read scope compiled to a statement the engine could not PARSE — an RLS
+     * policy that cannot be evaluated at all. It goes through
+     * {@link textMatch} now with `fold` set, which keeps `translate()` on
+     * Postgres and the `unknown` residue, emits `lower(col) GLOB lower(?)` on
+     * SQLite and the nested-`REPLACE` binary fold on MySQL. The `ESCAPE`
+     * binding is still never folded — the construct table owns that, and the
+     * SQLite arm has no `ESCAPE` clause to bind at all.
      */
-    case '$icontains': {
+    case '$icontains':
       assertRenderableText(op, field, val);
-      const gated = textOverNonTextColumn(op, field, opts);
-      if (gated) return gated;
-      // The two binds are spelled out rather than taken from {@link textMatch},
-      // because only the PATTERN placeholder is folded, the `ESCAPE` one must
-      // not be, and this operator is case-INSENSITIVE by ruling so it never
-      // wants the per-dialect case-exact construct. Left-to-right, so the
-      // values land in `params` in placeholder order.
-      const patternRef = asciiLowerSqlExpr(bind(params, likePattern('contains', val)));
-      return `${asciiLowerSqlExpr(col)} LIKE ${patternRef} ESCAPE ${bind(params, LIKE_ESCAPE_CHAR)}`;
-    }
+      return textOverNonTextColumn(op, field, opts)
+        ?? textMatch(col, 'contains', val, false, params, opts, true);
     // [#5298] NULL-safe: `NOT LIKE` is UNKNOWN for a NULL column, and "does not
     // contain" is true of a value that is not there.
     case '$notContains':