Skip to content

test(driver-sql): measure what each dialect materialises for a datetime JS cannot hold (#14078) - #14409

Merged
os-musk merged 1 commit into
mainfrom
claude/issue-14078-invalid-date-materialisation-measurement
Sep 2, 2026
Merged

test(driver-sql): measure what each dialect materialises for a datetime JS cannot hold (#14078)#14409
os-musk merged 1 commit into
mainfrom
claude/issue-14078-invalid-date-materialisation-measurement

Conversation

@os-musk

@os-musk os-musk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Part of #14078 — the MEASUREMENT half. ⛔ No change to the shared canonical-ISO spelling: A vs B is the maintainer's call, and this PR exists to turn "should it change" into an evidenced question. #14078 remains open on that ruling.

Headline: the reachability the card declined to claim is real, on both live dialects. Two independent driver code paths materialise a stored value as new Date(NaN) — one of them by returning a module-level constant named INVALID_DATE. Details and file references below; the p1 re-grade condition the triage wrote is met.

1. The copies census — the denominator, re-derived

Method: a mechanical scan of every instanceof Date arm in non-test source under packages/, each classified by whether a NaN/finite guard on that value's time is in lexical scope, and separately by whether the arm reaches toISOString(). Run at three window sizes (6, 8, 14 lines) — the count is identical at all three.

Positive control (required, and it caught a bad first predicate): the two guarded sites the card names — packages/rest/src/export-format.ts:291 and packages/rest/src/import-prepare.ts:115 — must land in the guarded bucket. A first pass that required toISOString() in-window failed this control (neither site calls it; one returns the Date, the other delegates), so the predicate was widened to the whole instanceof Date population. Control now PASSES. Reported because a census whose control fails is not a census.

Population at the branch point 1dcb995f: 92 arms across 54 non-test source files.

Unguarded and reaching toISOString() — 11:

file:line in the card's shared-spelling family?
packages/metadata-protocol/src/sys-metadata-repository.ts:120 yes — canonicalIsoInstant
packages/metadata/src/loaders/database-loader.ts:63 yes — canonicalIsoInstant
packages/rest/src/rest-server.ts:570 yes — canonicalIsoStamp
packages/rest/src/rest-server.ts:626 yes — formatCsvCell, the sibling arm
packages/metadata-protocol/src/protocol.ts:8082 yes — auditMetaItem's occurredAt
packages/metadata/src/migrations/migrate-sys-notification-to-event.ts:219 no — same hazard class, different family
packages/objectql/src/record-title.ts:102 no
packages/services/service-analytics/src/strategies/filter-normalizer.ts:1678 no
packages/formula/src/template-engine.ts:160 no
packages/drivers/driver-memory/src/memory-analytics.ts:1225 no
packages/drivers/driver-turso/src/remote-transport.ts:3662 no

So the shared spelling is 5 arms in 4 files — the triage's "four, not three" correction holds, and the fifth (rest-server.ts:626) is the sibling arm the dispatch flagged. Six further unguarded arms carry the same hazard in other families; they are reported as denominator, not folded into this decision.

The open question on canonicalVersionInstant is answered: it is NOT the same shape. packages/metadata-protocol/src/protocol.ts:1523 routes its Date arm through a time value and then tests it:

if (value instanceof Date) { ms = value.getTime(); } ...
if (!Number.isFinite(ms) || Math.abs(ms) > MAX_TIME_VALUE) return null;
return new Date(ms).toISOString();

Number.isFinite(NaN) is false, so an Invalid Date returns null there rather than throwing. It is already total, and it is not part of the four that need a decision. (The classifier only sees this at window 14, because the guard sits 11 lines below the arm; hand-verified either way.)

A third guard precedent, beyond the two the card names. packages/services/service-storage/src/stranded-orphan-inventory.ts:211 is the same three-arm function shape as the four copies and already carries the guard, with a comment naming this exact hazard: "An Invalid Date IS instanceof Date, and toISOString() THROWS on it (RangeError) rather than returning something odd." That matters for option B's cost: it is reuse of an established in-repo spelling, not a new invention — and one of the three precedents deliberately chose undefined over String(value) as its terminal arm, which is a design input if B is taken.

2. The driver-source reading — how a zero / legacy datetime materialises, per dialect

The client libraries materialise datetimes in pure JavaScript, so the arithmetic that decides Date vs Invalid Date was reproduced and executed in-container rather than inferred from documentation. Versions as installed: pg@8.22.0 (via pg-types@2.2.0, postgres-date@1.0.7), mysql2@3.23.1, better-sqlite3@13.0.3, knex@3.3.0.

Driver-side configuration read off packages/drivers/driver-sql/src/sql-driver.ts: dateStrings and typeCast are never set anywhere in driver-sql/src (so mysql2's default Date materialisation applies); withUtcSession (:4992) pins mysql2 to timezone: 'Z' and the session to +00:00; withPostgresCalendarDayAsText (:5116) overrides the type parser for date / date[] only, leaving timestamp / timestamptz to pg's stock parser, and says so deliberately. On the two live dialects the read door hands the client value straight through: formatOutput (:15980) applies repairNaiveUtcAuditTimestamp to the audit columns inside if (this.isSqlite) (:15998), so nothing downstream converts it.

dialect stored value materialised type getTime() shared spelling's Date arm String(value), the spelling it replaced
MySQL 0000-00-00 00:00:00 Invalid Date NaN throws RangeError: Invalid time value "Invalid Date"
MySQL 1000-01-01 00:00:00.000 Date finite serves ISO text (n/a)
Postgres 294276-01-01 00:00:00+00 Invalid Date NaN throws RangeError: Invalid time value "Invalid Date"
Postgres infinity number (Infinity) (n/a) not reached — falls to the terminal arm "Infinity" — served
Postgres 0001-01-01 00:00:00+00 Date finite serves ISO text (n/a)
SQLite any string (n/a) not reached — the string arm returns first (n/a)

MySQL, two independent paths, both landing on new Date(NaN):

  • Binary protocol — mysql2/lib/packets/packet.js, Packet#readDateTime: after reading the components it short-circuits if (y + m + d + H + M + S + ms === 0) return INVALID_DATE;, where line 13 of that file is const INVALID_DATE = new Date(NaN);. The Invalid Date is not an arithmetic accident here — the library returns one by name, deliberately, for the zero datetime.
  • Text protocol (the path knex's .query() takes) — Packet#parseDateTime(timezone) is new Date(str + timezone). Under the timezone: 'Z' the driver pins, a zero datetime composes new Date('0000-00-00 00:00:00Z'), measured to be an Invalid Date. Same for 0000-00-00 and 0000-00-00 00:00:00.000.

A zero datetime is storable whenever NO_ZERO_DATE / strict mode is not in force — the classic legacy-import and shared-database shape.

Postgres — the range gap, which is not about zero dates at all: postgres-date@1.0.7 builds every instant as new Date(Date.UTC(year, ...)). Date.UTC answers NaN outside ±8.64e15 ms, and new Date(NaN) is an Invalid Date — no throw, no null on that path. Measured boundary: 275760-09-13 is exactly 8640000000000000; 275760-09-14 is NaN. Postgres' own documented timestamp / timestamptz ceiling is 294276-12-31 AD, so every year from 275760 to 294276 is a value the server stores and the client materialises as an Invalid Date. The low end is not a hazard: 4713 BC and year 0001 both land inside JS's range, measured.

Falsified, in the safe direction: the assumption that infinity / -infinity would be a candidate. postgres-date tests for the exact tokens infinity and -infinity first and returns Number('Infinity') — a JS number. It never reaches the copies' instanceof Date arm at all; it falls to their terminal String(value) and is served as the text Infinity.

3. The live-dialect test, and its local status

New file (the whole diff): packages/drivers/driver-sql/src/sql-driver-14078-invalid-date-materialisation.test.ts.

  • §A — the wire-form half, six cases, runs on every runner with no server. Reproduces each library's own arithmetic (each case naming the library file it mirrors), plus §A2 which reads timezone: 'Z' off the real driver config rather than assuming it, so §A3's premise is measured and not asserted.
  • §B — the dialect half, gated exactly like its siblings via DIALECT_CELLS / declareDialectCell, so each live cell is a named skip without its URL and a named RED under OS_EXPECT_LIVE_DIALECT_MATRIX=1. It creates the table through initObjects (so updated_at is the column createAuditTimestampColumn produces: timestamptz / DATETIME(3) / TEXT), writes the exotic value with raw knex — deliberately bypassing the ObjectQL write door, because the question is what an existing row does on the way out, not what the write door admits — and reads it back through findOne, the door consumers of the shared spelling actually read through. MySQL's cell chains a pool.afterCreate that clears sql_mode so a zero datetime can reach disk at all; afterCreate rather than a pooled SET SESSION because sql_mode is a session variable and knex.raw takes whichever connection the pool hands it — the same trap sql-driver.ts documents for lock_wait_timeout, here removed rather than managed. SQLite is the negative control: with no temporal type there is no client-side parse, so no Invalid Date can be constructed on the way out.
  • Every probe asserts either way: a server that refuses the value records the refusal (that is a reading too) instead of passing as an unrun cell, and §B0 fails if no probe reached disk.

Local run status — the live legs are NOT MEASURED here. OS_TEST_POSTGRES_URL and OS_TEST_MYSQL_URL are unset in the dev container and no live server is reachable from it. The reading comes from the CI job Temporal Conformance (live PG + MySQL). Locally:

Test Files  1 passed (1)
     Tests  8 passed | 2 skipped (10)
 ✓ §A1 … §A6, §B0, §B1 (sqlite)
 ↓ … matrix (live postgres) > is provisioned — set OS_TEST_POSTGRES_URL …
 ↓ … matrix (live mysql)    > is provisioned — set OS_TEST_MYSQL_URL …

Non-vacuity, by ablation. Replacing the reproduced shared-spelling arm (value.toISOString()) with the spelling it replaced (String(value)) turns §A1 red — Tests 1 failed | 7 passed | 2 skipped — so §A1 measures the spelling rather than restating a tautology. Mutation confirmed on disk by grep counts before and after (anchor 1 to 0, injected text 0 to 1); restore proven by git hash-object matching the HEAD blob and an empty git diff HEAD. A first attempt at this ablation reported exit 1 that was not a red test — No test files found from a wrong path base — and was rerun rather than reported.

4. Verification

Run at HEAD 4f6f64254 (the final commit on this branch); the gate family was re-derived from the change set with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands (no paths passed) — 27 families, one more than the dispatch's hint, the addition being node scripts/check-system-context-census.mjs.

  • 26 of 27 gate families green. The remaining one, node scripts/check-test-completeness.mjs, prints PREREQUISITE NOT MET and exit 3 by its own design when run from the family list with no saved turbo log: NOT MEASURED, and by the gate's own words not a red. pnpm check:dual-build-cjs-loads and pnpm check:type-check-debt also start at that prerequisite; both were re-run after building the workspace closure (turbo run build --filter='./packages/*' --filter='./packages/*/*', 70/70 successful) and both then reported OK — check-type-check-coverage --re-measure: OK — 27 ledger entries re-measured, 1217 raw tsc errors total, none above its recorded number.
  • pnpm --filter @objectstack/driver-sql test151 test files passed, 9 skipped; 2287 tests passed, 138 skipped.
  • pnpm --filter @objectstack/driver-sql exec tsc --noEmit --listFiles — exit 0, 0 errors, and the new file confirmed present in the tsc program (include: ["src/**/*"], no test exclusion), so the typecheck really covers it.
  • pnpm lint — the full repo scan (eslint . --no-inline-config), exit 0, clean. No narrowing claimed.
  • Exit codes were captured after redirecting to a file, never through a pipe, and each verdict above quotes the gate's own printed line.

Tests-only diff (one new *.test.ts, no package publishes anything from it) ⇒ skip-changeset, applied at PR open with a read-back of the resulting label set.

Generated by Claude Code


Generated by Claude Code

…me JS cannot hold (#14078)

Part of #14078. Measurement only — no change to the shared canonical-ISO
spelling, which is a maintainer decision about four packages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68
@github-actions github-actions Bot added the size/m label Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 7307191db461df6120f2d1c7e6e2666276ac6e72packageMentionDocs.

@github-actions github-actions Bot added the tests label Sep 2, 2026
@os-musk os-musk added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 2, 2026 — with Claude
@os-musk
os-musk marked this pull request as ready for review September 2, 2026 05:20
@os-musk
os-musk enabled auto-merge September 2, 2026 05:21

os-musk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Enqueue provenance (domain:engine execution seat, session_0112hMx9hjJ9BgB28X97DS68): ACCEPT on #14078 (comment 5504612072) → at 05:20Z every check run on head 4f6f64254 was completed with success or skipped (37 runs; Lint & Repo Gates finished 05:11Z, Temporal Conformance (live PG + MySQL) green), governed-surface test on the one changed path: NOT governed, mergeable_state not dirty → marked ready and auto-merge (squash) enabled. Part of PR: #14078 stays open on the maintainer's A/B ruling; at MERGED the seat strips pm:dispatched only.


Generated by Claude Code

@os-musk
os-musk added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 3ecb7dc Sep 2, 2026
39 checks passed
@os-musk
os-musk deleted the claude/issue-14078-invalid-date-materialisation-measurement branch September 2, 2026 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants