fix(driver-memory): a no-value row satisfies $nin / $notContains in the reference matcher (#13166)

.changeset/memory-matcher-no-value-negated-operators.md

@@ -0,0 +1,36 @@
+---
+"@objectstack/driver-memory": patch
+---
+
+fix(driver-memory): a no-value row satisfies `$nin` / `$notContains` in the reference matcher (#13166)
+
+`memory-matcher`'s `match()` — the reference face `driver-sql` was aligned TO for
+#5146 — diverged from the platform's settled answer in 3 of 6 measured cells. The
+ruling is the INCLUDE direction: a row whose field has no value SATISFIES a
+negation-carrying operator (`$ne` / `$nin` / `$notContains`). That is #5146
+extended by #5298 option A, re-affirmed on 2026-08-10 after the reversal was
+priced and withdrawn.
+
+"No value" has two readings and the divergence had two INDEPENDENT causes, one
+reachable from each:
+
+- a MISSING key short-circuited to "no match" in `checkCondition`'s pre-switch
+  guard, whose allowlist named `$ne` but not `$nin` / `$notContains`;
+- a `null` value failed the `$notContains` arm on its `typeof value !== 'string'`
+  TYPE test rather than on the predicate — which the guard above cannot reach.
+
+Both are fixed, and both now answer one named predicate rather than two spellings
+of one ruling.
+
+**Grading — what does NOT change.** `InMemoryDriver.find()` is unaffected:
+`match()` is not part of this package's export surface, and the live mingo query
+path users actually reach already answered the include direction (measured on the
+card's fixture: `['2','3']` for all three operators, before and after). Nothing
+moved on the SQL side either — `driver-sql` (2241 tests) and `formula` (643) are
+untouched and green, because this change moves `driver-memory` TO the answer they
+already gave. The observable effect is on the reference face itself: the
+package's two filter faces now agree for these operators where they used to
+disagree.
+
+`$exists` is deliberately NOT included — it is the neighbouring cell (#13195),
+with a different backend list, and remains a pinned divergence.

packages/drivers/driver-memory/src/memory-driver-document-not.test.ts

@@ -208,28 +208,41 @@ describe('[#5324] InMemoryDriver.find compiles a document-level $not', () => {
    *
    *   `$exists`      REFERENCE is correct. `$exists` means "has a value"
    *                  (#5298 ③ / #5369, PR #5962), so mingo's key-presence
-   *                  reading is the divergent one.
-   *   `$nin`         LIVE is correct. Negative operators MATCH no-value rows —
+   *                  reading is the divergent one. STILL OPEN — #13195.
+   *   `$nin`         LIVE was correct. Negative operators MATCH no-value rows —
    *                  #5146, extended by #5298, re-affirmed 2026-08-10 — so a
    *                  missing key satisfying `$nin` is the affirmed answer, and
-   *                  the reference matcher's early-exit guard is the divergence.
-   *   `$notContains` LIVE is correct, for the same reason.
+   *                  the reference matcher's early-exit guard was the
+   *                  divergence. CLOSED by #13166: the guard now exempts the
+   *                  negation-carrying operators, and the two faces agree.
+   *   `$notContains` LIVE was correct, for the same reason. CLOSED by #13166,
+   *                  through a SECOND and independent cause — the arm's
+   *                  `typeof value !== 'string'` test, which rejected a `null`
+   *                  on its type rather than on the predicate.
    *
    * A ruling that morning (07:33Z) would have made the REFERENCE column the
    * target on all three rows. Cells 1 and 3 of it were WITHDRAWN the same day,
    * once the reversal's cross-backend cost had been measured, and the include
-   * direction was re-affirmed — which leaves the split above.
+   * direction was re-affirmed — which left the split above.
    *
-   * ⛔ Nothing is flipped in either direction. The #5499 investment freeze was
-   * the reason while it stood; it dissolved 2026-08-11 (head note of
-   * `@objectstack/spec`'s `aggregation-conformance.ts`), so that excuse has
-   * lapsed and the direction is now #13166's and #13195's to settle — ⛔ not
-   * this pin's, and ⛔ not a sweep's. What the round trip confirmed is exactly
-   * why this pin exists — this package answers with two different faces, so a
-   * statement like "driver-memory already reads has-value" is true of the
-   * reference matcher and FALSE of the live query path users actually reach.
+   * ⚠️ Two of the three cells are no longer a divergence, and this block was
+   * built for exactly that edit. It used to say "⛔ Nothing is flipped in either
+   * direction. The #5499 investment freeze was the reason while it stood; it
+   * dissolved 2026-08-11 …, so that excuse has lapsed and the direction is now
+   * #13166's and #13195's to settle." #13166 settled its two: the `$nin` and
+   * `$notContains` rows below now assert live and reference AGREEING, on the
+   * affirmed include answer. They were not re-baselined onto whatever the
+   * matcher began printing — the target was the live path's pre-existing
+   * answer, named as correct in this very note before the fix existed.
+   *
+   * ⛔ The `$exists` row is untouched and stays a pinned divergence. It is a
+   * different cell with a different backend list (`driver-mongodb` reads
+   * key-presence too), and it belongs to #13195. What that row still shows is
+   * why this pin exists — this package answers with two faces, so a statement
+   * like "driver-memory already reads has-value" is true of the reference
+   * matcher and FALSE of the live query path users actually reach.
    */
-  describe('[#5299] the settled cells, live vs reference — divergence pinned, disposition open (#13166/#13195)', () => {
+  describe('[#5299] the settled cells, live vs reference — $nin / $notContains converged (#13166), $exists still open (#13195)', () => {
     const liveVsReference = async (where: unknown) => ({
       live: await idsFrom(nulled, where),
       reference: NULLED.filter((r) => match(r, where)).map((r) => r.id),
@@ -242,16 +255,22 @@ describe('[#5324] InMemoryDriver.find compiles a document-level $not', () => {
       });
     });
 
-    it('$nin on an ABSENT field', async () => {
+    it('$nin on an ABSENT field: the two faces now AGREE (#13166)', async () => {
+      // Was `reference: ['2']` — the matcher's `value === undefined` guard
+      // short-circuited before the `$nin` arm ran. The LIVE column is unchanged,
+      // and it is the column this note already named correct.
       const live = await idsFrom(missing, { stage: { $nin: ['won'] } });
       const reference = MISSING.filter((r) => match(r, { stage: { $nin: ['won'] } })).map((r) => r.id);
-      expect({ live, reference }).toEqual({ live: ['2', '3', '4'], reference: ['2'] });
+      expect({ live, reference }).toEqual({ live: ['2', '3', '4'], reference: ['2', '3', '4'] });
     });
 
-    it('$notContains on a null field', async () => {
+    it('$notContains on a null field: the two faces now AGREE (#13166)', async () => {
+      // Was `reference: ['1', '3', '4']` — `typeof null !== 'string'` failed the
+      // TYPE test, so the negation readmitted the null rows. The LIVE column is
+      // unchanged here too.
       expect(await liveVsReference({ $not: { stage: { $notContains: 'w' } } })).toEqual({
         live: ['1'],
-        reference: ['1', '3', '4'],
+        reference: ['1'],
       });
     });
   });

packages/drivers/driver-memory/src/memory-matcher-no-value-negated-operators.test.ts

@@ -0,0 +1,136 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#13166] A row with NO VALUE satisfies a negation-carrying operator — the
+ * six cells, on this matcher, stated directly rather than through a `$not`.
+ *
+ * ## The ruling this file enforces
+ *
+ * `$ne` / `$nin` / `$notContains` MATCH a row whose field has no value. That is
+ * the INCLUDE direction: #5146 made `$not` NULL-safe, #5298 option A extended it
+ * to the non-negated negative operators, and the 2026-08-10 reversal that would
+ * have taken SQL's three-valued answer as the common denominator was WITHDRAWN
+ * the same day once its cross-backend cost had been measured. Ten other surfaces
+ * already answer this way — `formula` (`matches-filter-not-null-safe.test.ts`),
+ * the four SQL compilers via `nullSafeNegative` / `nullValueSatisfiesOperator`,
+ * and `driver-mongodb`, which passes `$nin` through and compiles `$notContains`
+ * to `{ $not: { $regex } }`, both of which match a missing or null field.
+ *
+ * ## Why the table has SIX cells and not three
+ *
+ * "No value" has two readings that this matcher reaches through DIFFERENT code,
+ * so a three-cell table cannot see the difference between them:
+ *
+ *   - `name: null` — the shape a SQL NULL round-trips into a record.
+ *   - the key absent — the shape a partial write leaves.
+ *
+ * Collapsing them is how the divergence stayed invisible: two INDEPENDENT causes
+ * produced it, each reachable from only one of the two readings, so a fixture
+ * carrying one reading measures at most one of them. Keep both columns.
+ *
+ * ⚠️ Do not collapse this into a single case, and do not add `$exists` to it.
+ * `$exists` is the neighbouring cell (#13195): this package's live mingo path
+ * and `driver-mongodb` still read it as key-presence rather than has-value, so
+ * it is a different, still-open divergence with a different backend list.
+ *
+ * The complementary POSITIVE operators are asserted beside each negative one, so
+ * a matcher that started answering "every row" to everything cannot pass this
+ * file: the ruling is that a no-value row joins the negative answer, not that
+ * predicates stop discriminating.
+ */
+
+import { describe, it, expect } from 'vitest';
+
+import { match } from './memory-matcher.js';
+
+/** `name` present but null — how a SQL NULL round-trips into a record. */
+const NULLED: Array<Record<string, unknown>> = [
+  { id: '1', name: 'alpha-one' },
+  { id: '2', name: 'beta' },
+  { id: '3', name: null },
+];
+
+/** The same rows with `name` ABSENT — the shape a partial write leaves. */
+const MISSING: Array<Record<string, unknown>> = [
+  { id: '1', name: 'alpha-one' },
+  { id: '2', name: 'beta' },
+  { id: '3' },
+];
+
+const ids = (rows: Array<Record<string, unknown>>, filter: unknown): string[] =>
+  rows.filter((r) => match(r, filter)).map((r) => String(r.id));
+
+/** The ruling's answer: row 2 has a different value, row 3 has none. */
+const NO_VALUE_INCLUDED = ['2', '3'];
+
+describe('[#13166] no-value rows and the negation-carrying operators', () => {
+  describe('the six cells — three operators x two readings of "no value"', () => {
+    const OPERATORS: Array<[name: string, filter: unknown]> = [
+      ['$ne', { name: { $ne: 'alpha-one' } }],
+      ['$nin', { name: { $nin: ['alpha-one'] } }],
+      ['$notContains', { name: { $notContains: 'one' } }],
+    ];
+
+    for (const [op, filter] of OPERATORS) {
+      it(`${op}: a null value satisfies it`, () => {
+        expect(ids(NULLED, filter)).toEqual(NO_VALUE_INCLUDED);
+      });
+
+      it(`${op}: an absent key satisfies it`, () => {
+        expect(ids(MISSING, filter)).toEqual(NO_VALUE_INCLUDED);
+      });
+
+      it(`${op}: both readings of "no value" answer alike`, () => {
+        expect(ids(MISSING, filter)).toEqual(ids(NULLED, filter));
+      });
+    }
+  });
+
+  describe('the operators still discriminate — this is not "match everything"', () => {
+    it('each negation excludes the row that DOES carry the comparand', () => {
+      expect(ids(NULLED, { name: { $ne: 'alpha-one' } })).not.toContain('1');
+      expect(ids(NULLED, { name: { $nin: ['alpha-one'] } })).not.toContain('1');
+      expect(ids(NULLED, { name: { $notContains: 'one' } })).not.toContain('1');
+    });
+
+    it('the positive twins keep answering the complement over the VALUED rows', () => {
+      // A no-value row is in NEITHER answer for the positive operators: the
+      // ruling moved the negative cells only.
+      expect(ids(NULLED, { name: { $eq: 'alpha-one' } })).toEqual(['1']);
+      expect(ids(MISSING, { name: { $in: ['alpha-one'] } })).toEqual(['1']);
+      expect(ids(NULLED, { name: { $contains: 'one' } })).toEqual(['1']);
+      expect(ids(MISSING, { name: { $contains: 'one' } })).toEqual(['1']);
+    });
+
+    it('a present non-string value still fails $notContains on the type test', () => {
+      // Out of scope for this card, and stated so a later reader does not read
+      // the fix above as "any non-string satisfies the negation". Only the
+      // no-value readings moved; a value that is there and is not a string
+      // keeps the answer it had.
+      expect(ids([{ id: '9', name: 42 }], { name: { $notContains: 'one' } })).toEqual([]);
+    });
+  });
+
+  describe('cross-operator agreement, the invariant that outlives the fixture', () => {
+    it('$nin answers exactly what $ne answers — it is the list form of it', () => {
+      // `formula`'s suite states the same identity over its own fixture. `$ne`
+      // is the operator ENROLLED in `FILTER_LOGIC_CASES`, so this is the link
+      // between the enrolled cell and the two that are not enrolled yet.
+      for (const rows of [NULLED, MISSING]) {
+        expect(ids(rows, { name: { $nin: ['alpha-one'] } }))
+          .toEqual(ids(rows, { name: { $ne: 'alpha-one' } }));
+      }
+    });
+
+    it('$contains and $notContains partition the VALUED rows and both keep the no-value row out of the positive side', () => {
+      for (const rows of [NULLED, MISSING]) {
+        const inside = ids(rows, { name: { $contains: 'one' } });
+        const outside = ids(rows, { name: { $notContains: 'one' } });
+        expect(inside).toEqual(['1']);
+        expect(outside).toEqual(NO_VALUE_INCLUDED);
+        expect(inside.filter((id) => outside.includes(id))).toEqual([]);
+        expect([...inside, ...outside].sort()).toEqual(['1', '2', '3']);
+      }
+    });
+  });
+});