Skip to content

feat(spec): boolean aggregand column in the aggregation conformance fixture — ruled numeric min/max on every face - #12947

Merged
os-trump merged 4 commits into
mainfrom
claude/issue-11152-boolean-aggregation-conformance
Aug 28, 2026
Merged

feat(spec): boolean aggregand column in the aggregation conformance fixture — ruled numeric min/max on every face#12947
os-trump merged 4 commits into
mainfrom
claude/issue-11152-boolean-aggregation-conformance

Conversation

@os-trump

Copy link
Copy Markdown
Collaborator

Fixes #11152

What this lands

The cross-driver aggregation conformance fixture (AGGREGATION_ROWS, @objectstack/spec/data) gains its boolean column — flag, 3 true / 3 false, the FLAG_BY_ID distribution already landed on main (west [T,F,F,F], east [T,T]) — with seven ruled cases: sum(flag)=3, avg(flag)=0.5, min(flag)=0, max(flag)=1, count(flag)=6, count_distinct(flag)=2, and grouped min(flag) (east=1 / west=0, the asymmetric cell). This closes the reach gap #11065 and #11151 were both found through: no conformance cell could see a boolean aggregand on any face.

Ruling of record — maintainer 2026-08-28, applied on the card in comment 5448627494, ruling verbatim: 「12745 A回,其他同意。」 Option A: booleans aggregate as numbers on every face, with no per-aggregate exceptionmin/max over a boolean answer 0/1 (numeric spelling; the earlier #11249 ruling's false/true is superseded). Every value in this PR flows from that ruling and is cited, not re-litigated.

Mandatory pre-implementation PG leg (the ruling's must-answer, measured FIRST)

Question: does PostgreSQL's MIN/MAX over boolean error on driver-sql's actual storage (a third behaviour the ruling does not cover)?

Measured on a live PostgreSQL 16.13 cluster provisioned in this container (initdb, port 54911, torn down after):

  • Bare form over a real boolean column: select min(flag) … and max(flag) both ERROR — SQLSTATE 42883, function min(boolean) does not exist. The third behaviour is real for the bare form.
  • Driver-sql's actual path: the sql-driver-11635-boolean-aggregand-answers.test.ts pg cell ran against the live cluster — 19 passed / 1 skipped (the skip is the unprovisioned MySQL cell), so the fix(driver-sql): boolean aggregands answer the ruled #11249 contract — PG lowering cast + all-dialect min/max JSON-boolean presentation #11785 lowering cast (cast("flag" as int)) covers min/max as well as sum/avg and PG does not error through the driver. Raw cast-form values measured: min=0, max=1, sum=3, avg=0.5 — exactly the ruled numbers.

Verdict: no stop condition; the ruled values are producible on PG's actual storage.

The measurement that set the final surface

Baseline (tree at aef1b7e6, all enrolled faces green) → fixture+DDL-only intermediate state → post-change, every suite exit code captured before any pipe:

face baseline fixture+DDLs only post-change
driver-sql conformance (sqlite + live pg) 37 passed / 1 skip 51 passed / 1 skip — green while the face still answered false/true 51 passed / 1 skip
driver-sql 11635 boolean suite (sqlite + live pg) 19 passed / 1 skip (pinning superseded false/true) unchanged (not yet edited) 19 passed / 1 skip (pinning ruled 0/1)
driver-turso remote 19 passed 26 passed (already answers raw 0/1) 26 passed
driver-sqlite-wasm 15 passed 22 passed 22 passed
driver-mongodb translation 35 passed RAW-EXIT=1 — 4 failed: min(flag) answered false, max(flag) answered true, grouped min both booleans 43 passed
driver-mongodb 11151 boolean suite 17 passed (pinning superseded false/true + a "min/max stay bare" stage pin) unchanged (not yet edited) 17 passed (pinning ruled 0/1 + the coercion stage pin)
driver-memory conformance 34 passed RAW-EXIT=1 — 7 failed: both doors x min/max/grouped-min answered booleans, plus the typeof-number property 48 passed
driver-memory boolean-aggregand suite 15 passed unchanged 17 passed (min/max sections added, both faces)
objectql in-memory conformance 17 passed 24 passed — green while the reduce still answered false/true (its harness coerces Number(...); Number(false) is 0) 25 passed (uncoerced spelling pin added)

Two of those rows are the load-bearing finding: the SQL-face and objectql harnesses deliberately compare through Number(...) (a wire-type necessity — node-pg hands bigint counts back as strings), which makes them structurally blind to the false-vs-0 spelling. The value cases alone therefore cannot hold the ruled JSON domain; the strict pins in the per-driver boolean suites and the new uncoerced objectql pin are what hold it.

Surface beyond the dispatched five files — declared, each edit cited to the ruling

The dispatched surface assumed the SQL and mongodb faces already answered 0/1. Measurement falsified that: #11785 (issue #11635) had implemented the superseded #11249 ruling as an all-dialect JSON-boolean presentation of min/max results inside sql-driver.ts, and PR #12819 (issue #11151) had implemented the same superseded ruling in driver-mongodb (bare $min/$max + a stage pin explicitly forbidding the coercion). Landing only the dispatched files would have produced a fixture that says 0/1 while two published faces answer false/true — invisible to the conformance suites (the Number(...) blindness above), i.e. exactly the phantom-conformance class this card exists to remove.

Each widening edit is the same defect class as the card (the ruled boolean min/max cell), mechanical, pinned by the ruling's verbatim text, running under gate families this card already owed, and held by no other in-flight claim (verified against open PR branches):

  1. packages/drivers/driver-sql/src/sql-driver.ts — ruling 5448627494: the min/max result presentation now skips the 'boolean' kind (temporal presentation and the SQLite numeric repair untouched; find() row reads untouched). Ablation below proves the strict pins bite this exact layer.
  2. packages/drivers/driver-sql/src/sql-driver-11635-boolean-aggregand-answers.test.ts — in the dispatched surface for the FLAG_BY_ID deletion (done — seeds the fixture column now); additionally its ruling-B pins (toBe(false)/toBe(true), grouped booleans) are flipped to the ruled 0/1 and the head note re-anchored to the 2026-08-28 ruling. Leaving a strict pin of superseded values in an edited file was not an option.
  3. packages/drivers/driver-mongodb/src/mongodb-aggregation.ts — ruling 5448627494; the dispatch licensed a mongodb patch given evidence the red is the same ruled cell, and the intermediate-state measurement above is that evidence (4 failures, all on the ruled min/max cells, answering the superseded booleans). $min/$max now wrap the same boolean-only $cond coercion $sum/$avg use; null/missing still pass through (empty window still answers null — pinned).
  4. packages/drivers/driver-mongodb/src/mongodb-11151-boolean-aggregand-answers.test.ts — pins flipped with the lowering: ruled 0/1 values, and the "min/max stay bare" stage pin inverted to "min/max wrap the same coercion" (the stage, not the values, is what a real mongod would execute). Its private FLAG_BY_ID copy is deleted in favour of the fixture column, same argument as the 11635 one. The evaluator testkit needed no change — it already models $cond/$type/$eq per row.
  5. packages/drivers/driver-memory/src/memory-analytics.ts — the ruled "driver-memory" alignment applied to BOTH of its faces, per the package's own recorded discipline ("one face aligned alone leaves the other free to keep its own answer", the avg over a boolean field returns null on driver-memory and a number on sqlite — same rows, same dataset, same executor #11065 shape): the analytics face's $min/$max mingo arms wrap the existing numericAggregandExpr. Pinned in memory-boolean-aggregand.test.ts beside the data-face pins.
  6. packages/objectql/src/in-memory-aggregation-conformance.test.ts — one uncoerced spelling pin (min/max over flag answer the numbers), because this harness's Number(...) coercion cannot see the ruled spelling and the ruled reduce alignment would otherwise be pinned by nothing.

Cross-lane files pinned by the ruling (packages/objectql/src/in-memory-aggregation.ts, driver-memory's memory-driver.ts computeAggregate): the reduce arms map booleans to their numbers before comparison — value pinned by the maintainer ruling of 2026-08-28, whose applied record is comment 5448627494 on the card; the driver-memory investment-freeze (2026-08-05) is overridden here by that explicit newer, narrower ruling naming its reduce.

Ablation — the presentation removal can fail, and where the blindness lives

Predicted before running: re-adding the ruling-B boolean presentation to sql-driver.ts reds the 11635 suite on exactly 3 tests per executing cell (min, max, grouped) receiving booleans, while the conformance suite stays green under the same mutation (the Number(...) blindness). Measured: mutation confirmed on disk by anchored counts (guard-anchor 1 to 0, marker 0 to 1) before any result was read; 11635 RAW-EXIT=1, 6 failed — the exact 3 tests on each of the sqlite and live-pg cells; conformance RAW-EXIT=0, 51 passed. Restore proven by observation, not exit code: git diff HEAD empty and git hash-object on the worktree file equal to the HEAD blob (89cb9e2f87…), neither empty. Resolution-path note: both suites import the subject via same-package relative imports (vitest runs src), so no dist hop sits between the mutation and the measurement; the spec fixture, which IS consumed across packages, was rebuilt and its dist grepped for the new column before any suite was read.

Verification

Final tree: 373343c33 (branch head after merging origin/main at 4d1142753; same-day churn constraint honoured — all nine conformance faces re-run at this exact head, all green, and the ratchet family re-run at it too: type-check-debt re-measure "31 ledger entries, none above its recorded number", query-options-erasure "baseline key set verified against 4d11427: no files added", type-check-coverage, nul-bytes — all RAW-EXIT 0). The gate battery below first ran at the pre-merge head 1516d122; the incoming five commits touch .claude/** docs, one spec zod file, one PM script and a changeset — pnpm --filter @objectstack/spec build && check:generated was re-run at 373343c33 (all 14 artifacts up to date) alongside the faces and ratchets.

  • Faces at 373343c33, every RAW-EXIT captured pre-pipe, all 0: driver-sql conformance 51 passed/1 skip (sqlite + live pg cells; the skip is MySQL, unprovisioned here — NOT MEASURED locally, CI's live-dialect job owns it) · driver-sql 11635 19/1 skip (same MySQL skip) · turso-remote 26 · sqlite-wasm 22 · mongodb translation 43 · mongodb 11151 17 · memory conformance 48 · memory boolean 17 · objectql conformance 25.
  • Typecheck: spec, objectql, driver-sql, driver-memory, driver-mongodb, driver-turso, driver-sqlite-wasm — all green. Coverage note, stated rather than implied: driver-mongodb and objectql exclude **/*.test.ts from their tsc programs (standing package config), so the two edited test files in those packages are exercised by vitest execution, not tsc.
  • pnpm --filter @objectstack/spec check:generated — all 14 generated artifacts up to date (authorable-surface, api-surface, docs among them).
  • Derived gate battery (node scripts/pm/dispatch-gates.mjs, no hand-fed paths): variant-docs, empty-state, authorable-surface, cross-package-test-inputs, doc-authoring, doc-formula-expressions, driver-conformance, durability-log-level, merge-driver, objectql-double-limit, objectui-changeset, page-declaration-shape, pm-half-states, published-files, spec-parsed-alias, slot-lookup, test-source-alias, type-source-resolution, changeset-gate-self-tests, adr-0087-registration, changeset-no-major, empty-changeset, ci-filter-parity, comment-mask-adoption, dev-prereqs, engine-split-ratio, plugin-teardown-shape, release-rehearsal self-test, query-options-erasure, type-check-coverage, type-check-debt (workspace closure built first, as lint.yml does), engine-double-contract, where-matcher, dispatcher-error-vocabulary, nul-bytes — all RAW-EXIT 0. One exception: scripts/pm/check-half-states.mjs exits 3 ("no reading at all") — it sweeps the GitHub board and this seat's session has no repo-scoped REST access; CI owns it.
  • Changeset: .changeset/boolean-aggregands-numeric-min-max.md, patch across the five behaviour-bearing packages, FROM/TO stated per face (the same patch-level convention fix(driver-sql): boolean aggregands answer the ruled #11249 contract — PG lowering cast + all-dialect min/max JSON-boolean presentation #11785 and fix(driver-mongodb): a boolean aggregand answers the ruled values on this face too #12819 used for the previous flips of this same cell).

Out-of-scope observations (recorded, not acted on)

Clause-② note for the review chain: this PR changes published behaviour on the min/max-over-boolean cell of driver-sql (all dialects), driver-memory (both faces), driver-mongodb and objectql's fallback — false/true to 0/1 — and pins new contract expected values in the conformance fixture. Opened as draft; the PM attaches needs:contract-review; no AI seat enqueues this.

Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4

Generated by Claude Code


Generated by Claude Code

claude added 3 commits August 28, 2026 09:02
…ixture, ruled numeric on every face

Adds flag (3 true / 3 false, the FLAG_BY_ID distribution) to AGGREGATION_ROWS
with seven ruled cases (sum=3, avg=0.5, min=0, max=1, count=6,
count_distinct=2, grouped min east=1/west=0), extends the three SQL harness
DDLs, aligns min/max over booleans to the numeric domain on every face per the
2026-08-28 maintainer ruling (option A, superseding #11249's false/true):
objectql in-memory fallback, driver-memory data + analytics faces, driver-sql
result presentation (boolean kind skipped for min/max), driver-mongodb
lowering (numericAggregandExpr on min/max). FLAG_BY_ID private maps deleted in
favour of the fixture column; ruling-B pins in the 11635/11151 suites flipped
to the ruled 0/1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 5 package(s): @objectstack/driver-memory, @objectstack/driver-mongodb, @objectstack/driver-sql, @objectstack/objectql, @objectstack/spec, touching 9 documentable anchor(s).

13 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/ai/natural-language-queries.mdx (via count_distinct (literal))
  • content/docs/api/data-api.mdx (via groupBy (symbol))
  • content/docs/api/error-catalog.mdx (via groupBy (symbol))
  • content/docs/data-modeling/index.mdx (via groupBy (symbol))
  • content/docs/data-modeling/queries.mdx (via groupBy (symbol), count_distinct (literal))
  • content/docs/deployment/validating-metadata.mdx (via groupBy (symbol))
  • content/docs/kernel/contracts/data-engine.mdx (via groupBy (symbol), count_distinct (literal))
  • content/docs/kernel/runtime-services/data-service.mdx (via groupBy (symbol))
  • content/docs/protocol/objectql/query-syntax.mdx (via groupBy (symbol), count_distinct (literal))
  • content/docs/protocol/objectui/concept.mdx (via groupBy (symbol))
  • content/docs/protocol/objectui/layout-dsl.mdx (via groupBy (symbol))
  • content/docs/ui/dashboards.mdx (via count_distinct (literal))
  • content/docs/ui/react-pages.mdx (via groupBy (symbol))

3 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/implementation-status.mdx (via groupBy (symbol))
  • content/docs/releases/v15.mdx (via groupBy (symbol), count_distinct (literal))
  • content/docs/releases/v17.mdx (via groupBy (symbol), count_distinct (literal))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 4 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 129 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json d028b37cbce487f3529d5219cbc94373ad2bd2f9packageMentionDocs.

Which tree this was computed on

This run read content/docs from 858227a1d8f9e3dbeb1ec20c7388d93bb7e042dd — the merge of head 870e84442a1f9dd65829a6c2048086e1fe0af43c into base d028b37cbce487f3529d5219cbc94373ad2bd2f9, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 858227a1d8f9e3dbeb1ec20c7388d93bb7e042dd && git checkout 858227a1d8f9e3dbeb1ec20c7388d93bb7e042dd
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin d028b37cbce487f3529d5219cbc94373ad2bd2f9 870e84442a1f9dd65829a6c2048086e1fe0af43c && git checkout -B drift-repro d028b37cbce487f3529d5219cbc94373ad2bd2f9 && git merge --no-ff 870e84442a1f9dd65829a6c2048086e1fe0af43c

node scripts/docs-audit/affected-docs.mjs --json d028b37cbce487f3529d5219cbc94373ad2bd2f9

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs d028b37cbce487f3529d5219cbc94373ad2bd2f9 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

…ed 0/1

The 11782 cross-door parity suite keeps every row-read boolean pin (find,
distinct, group keys — NOT superseded) and asserts the ruled numbers on the
two order-statistic cells; the mongodb pipeline-builder unit test pins the
boolean-only coercion wrapper on min/max stages, the same shape it already
pins for sum/avg. Pin sweep across driver-sql/mongodb/memory/objectql/rest/qa
surfaced no third file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
@os-trump
os-trump marked this pull request as ready for review August 28, 2026 14:08
@os-trump
os-trump enabled auto-merge August 28, 2026 14:08
@os-trump
os-trump added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit f6fa22c Aug 28, 2026
38 checks passed
@os-trump
os-trump deleted the claude/issue-11152-boolean-aggregation-conformance branch August 28, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] AGGREGATION_ROWS has no boolean column, so the cross-driver aggregation conformance family cannot see a boolean aggregand on any face

2 participants