Sqlite support: proof of concept#80
Open
ryanrasti wants to merge 33 commits into
Open
Conversation
ryanrasti
commented
Jul 1, 2026
ryanrasti
commented
Owner
- Shared SqlValue base extracted from PG's Any. Dialect is now { name, root, bool }; DialectName is the string tag.
- SQLite inference tool (src/types/sqlite/generate.ts) — reads pragma_function_list, probes with a typed input matrix, emits functions.json + class stubs. LLM-over-docs plan in the file header for later cleanup.
- SqliteDriver — better-sqlite3, row-value normalization at the boundary.
- Sweep: instanceof Any → SqlValue across builders/util; runtime.ts utilities widened.
Phase 0.1 of SQLite generalization. Moves generated/, overrides/, generate.ts, introspect.ts, and the PG-specific index.test.ts under src/types/postgres/. runtime.ts, deserialize.ts, cast.test.ts, and match.test.ts remain at top level as dialect-agnostic. src/types/index.ts is a transitional re-export shim so external callers don't have to change in one shot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0.2 of SQLite generalization. Drop the Pg* prefix from runtime expression builders since they aren't dialect-specific (they just wrap the dialect-agnostic Func/Op AST nodes in builder/sql.ts). - runtime.PgFunc → runtime.funcCall (lowercase factory; Func is taken) - runtime.PgOp → runtime.opCall (lowercase factory; Op is taken) - runtime.PgSrf → runtime.Srf (class — no name collision) The PgFunc interface in src/types/postgres/introspect.ts is unchanged — it represents a pg_catalog function row and remains PG-specific. Codegen template and all 77 generated/ files updated via npm run codegen. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0.3 of SQLite generalization. Splits the deserialize registry so
each dialect owns its own typname → TS-type mapping.
- src/types/deserialize.ts: shared TypeDef/Dialect, registerDialect(),
getTypeDef(dialect, typname). Internal map is a Map (avoids
proto-pollution lint rule on dynamic object keys).
- src/types/postgres/deserialize.ts: PG registry (bool, int2/4, float
4/8, oid). Registers itself on import.
- PG Any base class gets `static dialect: Dialect = "postgres"`;
runtime.match() and Any.{deserialize,serialize} read it for lookup.
SQLite's registry (Phase 1) will register the same way.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0.4 of SQLite generalization. Threads a CompileContext (carrying
the dialect) through Sql.bind(ctx) and compile(root, ctx) so future
dialect-aware AST nodes can dispatch without baking dialect into the
node at construction time.
- src/builder/sql.ts:
- CompileContext { dialect } + pgCtx/sqliteCtx singletons.
- Sql.bind() → Sql.bind(ctx); subclasses without dispatch can omit.
- compile(root, "pg"|"sqlite") → compile(root, ctx). Param-placeholder
style (`$N` vs `?`) derives from ctx.dialect.
- New Cast(expr, typname) node; TypedParam now delegates to Cast.
- src/driver.ts: Driver gets `readonly dialect: Dialect`. PgDriver and
PgliteDriver default to "postgres" and build ctx from their own
dialect on execute().
- src/types/postgres/overrides/any.ts: Any.cast() emits Cast node instead
of inline `CAST(... AS ...)` templating.
- All 45 compile() call sites updated to pass pgCtx/sqliteCtx instead of
string args.
- delete/insert/update FinalizedX.bind() take ctx and forward it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0.5 of SQLite generalization. Moves the body of Bool.and/or/not into dialect-agnostic helpers at src/types/bool-mixin.ts. PG's Bool calls them with its own __typname so primitive TS booleans coerce to pg `bool` via the Cast node; SQLite's Bool (Phase 1) will reuse the same helpers with its own typname. The previous inline `CAST(... AS bool)` template is gone — Cast is the single source of truth for cast emission. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hoists the dialect-agnostic behavior (cast, coalesce, from, serialize,
column, deserialize, toSql, [meta], statics) out of PG's `Any` into a
shared `SqlValue<N>` base in src/types/any.ts. Codegen'd PG root `any.ts`
now extends `SqlValue`; every other pg_catalog type inherits transitively.
Also introduces `Dialect` as a rich descriptor `{ name, root, bool }`
(name kept as the compile-ctx tag, aliased to `DialectName` in the
deserialize registry) and a shared `Bool<N>` interface + `isBool`
predicate for future dialect-agnostic detection.
`isNull` / `isNotNull` / `.in()` stay on PG `Any` (not SqlValue): the
shared `Bool<N>.and(Bool<any> | boolean)` vs concrete
`types.Bool.and(types.Bool<any> | boolean)` are contravariantly
incompatible, and duplicating three method bodies per dialect is cheaper
than plumbing around the variance.
SqlValue is *sibling* to Sql (not subclass) — matches the pre-existing
design and avoids inheriting Sql's abstract `bind()` down the whole
concrete-type chain. Values are duck-type-spliced into sql\`...\`
templates via `.toSql()`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 foundation. Introduces:
- `SqliteValue<N>` (src/types/sqlite/base.ts) — dialect root, extends
the shared `SqlValue<N>`. Carries the SQLite `Dialect` object
(barrel-getter root/bool) plus `isNull`/`isNotNull`. `.in()` is
deferred: its `<T, Vs>(this: T, ...)` shape triggers TS2589 at
SQLite's flatter class hierarchy (see file comment).
- `src/types/sqlite/deserialize.ts` — dialect entries for the
registry. Registers on module load, same side-effect pattern PG
uses. Bool is a convention (INTEGER 0/1); blob repr TBD (better-sqlite3
→ Buffer, Cloudflare DO → ArrayBuffer — reconcile at driver layer).
- `src/types/sqlite/generate.ts` — runtime introspection tool that
reads `pragma_function_list`, probes each function through a typed
input matrix, and emits both a JSON snapshot (`functions.json`,
~20k lines) and first-pass class stubs (`generated/{integer,real,
text,blob}.ts`). Bool is hand-written (via boolAnd/boolOr/boolNot
helpers from Phase 0.5).
Known limits — runtime probing conflates null-propagation with
error-signal nulls, admits nonsensical coercions (Blob.acos exists),
misbuckets aggregates onto whichever leaf type they were probed with,
and collapses multi-arity overloads. Detailed follow-up plan in the
generate.ts header comment: LLM-over-docs with this file as a
deterministic sanity check.
better-sqlite3 added as a devDependency (codegen only; peer-dep
plumbing lands with the driver in Phase 4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`SqliteDriver` wraps better-sqlite3 to satisfy the async `Driver`
contract. Single in-process db, so `runInSingleConnection` is a
pass-through. `.execute()` dispatches SELECT-shaped statements via
`.all()` and non-reader statements via `.run()`.
Row values from better-sqlite3 come back as typed JS (number, string,
Buffer, bigint, null). Normalized to strings at the driver boundary so
downstream `deserialize()` code keeps working against the PG-style
`{[key: string]: string}` contract. Buffer → \\x-prefixed hex to match
PG's bytea repr.
Smoke tests exercise the full pipeline end-to-end: typed-value
construction → composed method calls (abs, upper, lower, length,
isNull/isNotNull) → compile with sqlite ctx (validates `?` placeholder
emission, no `$N`) → better-sqlite3 execute → normalized rows.
8 tests, all passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Widens the query/mutation builders from PG-only (`instanceof Any`) to
dialect-agnostic (`instanceof SqlValue`) across ~15 runtime dispatch
points + ~20 type refs. PG callers unchanged — Any still is-a SqlValue.
Changes by file:
- src/table.ts, src/util.ts, src/builder/{values,insert,update,query}.ts:
Any → SqlValue at instanceof checks + type positions. `isColumn` /
`getColumn` imports moved from postgres/overrides/any (re-export)
to src/types/any (source).
- src/types/runtime.ts: `NullOf`, `TsTypeOf`, `Aggregate`, `AggregateRow`
widened from `T extends Any<...>` to `T extends SqlValue<...>` so the
utilities work uniformly over PG's Any and SQLite's SqliteValue.
- src/types/bool.ts: the shared `Bool<N>` interface loses `and`/`or`/`not`
and becomes a nominal marker. Concrete Bool classes' `.and(other: X
<any> | boolean)` are contravariantly incompatible with a shared
method signature — trying to require the ops here breaks assignment
from concrete to shared. Chaining callers cast through `any` (see
`combinePredicates`); runtime validation still uses the `isBool`
identity predicate.
- src/types/meta.ts (new): extracted the `meta` symbol into its own
tiny module. `types/any.ts` used to import `meta` from `runtime.ts`,
but `runtime.ts` also does `import * as types from "./index"` which
triggers the PG barrel chain — extending SqlValue via PG's generated
any before any.ts had defined it. Splitting `meta` out breaks the
cycle.
- src/driver.ts (SqliteDriver): unwraps a single matched outer paren
pair from compiled SQL. `QueryBuilder.FinalizedQuery.bind()` wraps
its output in `(...)` for subquery-splicing; PG tolerates a
parenthesized top-level, SQLite does not.
- src/builder/{delete,update,query}.ts: `@expose` validators swapped
from `z.instanceof(Bool)` (PG-only) to `z.custom<SharedBool<any>>
(isBool)` (dialect-agnostic via the identity predicate). Note: the
broader "same-database provenance" concern is filed as ISSUES.md #16;
this changes the check from PG-nominal to dialect-nominal (catches
cross-dialect but not cross-tenant).
- src/types/sqlite/table.test.ts (new): 7 end-to-end tests through the
full Table → QueryBuilder → SqliteDriver stack — INSERT, SELECT with
method composition (.upper()), .where predicates, INSERT/UPDATE/
DELETE ... RETURNING.
Deferred (documented as comments in place, not blocking Phase 1):
- QueryBuilder.scalar() stays PG-only — emits ROW() + array_agg +
COALESCE(..., '{}'). Phase 2.1 Scalar AST node lands the SQLite
dispatch (json_object + json_group_array).
- `.where(true)` shorthand in delete/update still constructs a PG
`Bool` — sqlite-only apps that need this will hit a nominal mismatch
until the shorthand becomes dialect-aware.
Test totals: 28 files, 550 passed, 1 skipped (was 27/544). PG suite
green throughout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces a proof-of-concept SQLite dialect alongside Postgres, extracting a shared SqlValue<N> base for typed SQL expressions and making compilation/deserialization dialect-aware.
Changes:
- Added SQLite dialect scaffolding (base type, deserialize registry, generated type stubs) plus end-to-end smoke tests using
better-sqlite3. - Refactored core type system to use shared
SqlValue+ a dialect-aware type registry and compile context (pgCtx/sqliteCtx). - Reorganized Postgres type generation into
src/types/postgres/*, updating generated/override imports and runtime helpers (funcCall/opCall/Srf).
Reviewed changes
Copilot reviewed 24 out of 124 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/util.ts | Updates runtime typed-expression checks from Any to SqlValue. |
| src/types/sqlite/table.test.ts | Adds SQLite mutation + Table/QueryBuilder smoke tests. |
| src/types/sqlite/smoke.test.ts | Adds SQLite end-to-end expression/compile/execute smoke tests. |
| src/types/sqlite/index.ts | SQLite dialect barrel export for generated + hand-written entries. |
| src/types/sqlite/generated/text.ts | Generated SQLite Text typed value methods. |
| src/types/sqlite/generated/bool.ts | Temporary SQLite Bool stub (Phase 1 placeholder). |
| src/types/sqlite/deserialize.ts | Registers SQLite typname→TS-type/deserializer mapping. |
| src/types/sqlite/base.ts | Defines SqliteValue root and dialect descriptor wiring. |
| src/types/runtime.ts | Generalizes nullability/type utilities to SqlValue and adds generic runtime call helpers. |
| src/types/postgres/overrides/record.ts | Fixes import paths after Postgres codegen rehome. |
| src/types/postgres/overrides/bool.ts | Refactors Bool ops to use shared bool helpers and updated runtime import paths. |
| src/types/postgres/overrides/anycompatiblearray.ts | Updates runtime/sql imports for new Postgres module layout. |
| src/types/postgres/overrides/anyarray.ts | Updates runtime/sql/exoeval imports for new Postgres module layout. |
| src/types/postgres/overrides/any.ts | Introduces PG root Any override carrying dialect metadata + pg-specific bool-returning methods. |
| src/types/postgres/introspect.ts | Adds Postgres introspection utilities for codegen. |
| src/types/postgres/index.ts | New Postgres barrel exporting generated + overrides. |
| src/types/postgres/index.test.ts | Updates tests to new compile context API and import paths. |
| src/types/postgres/generated/xml.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/xid8.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/xid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/varchar.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/varbit.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/uuid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/unknown.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/txid_snapshot.ts | Regenerated PG type bindings to use runtime.Srf and new runtime call helpers. |
| src/types/postgres/generated/tid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/regtype.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regrole.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regprocedure.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regproc.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regoperator.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regoper.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regnamespace.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regdictionary.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regconfig.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/regcollation.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regclass.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/refcursor.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_snapshot.ts | Regenerated PG type bindings to use runtime.Srf and new runtime call helpers. |
| src/types/postgres/generated/pg_node_tree.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_ndistinct.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_mcv_list.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_dependencies.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_brin_minmax_multi_summary.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_brin_bloom_summary.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/jsonpath.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/json.ts | Regenerated PG type bindings to use runtime.funcCall/opCall/Srf and new import paths. |
| src/types/postgres/generated/gtsvector.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/cidr.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/cid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/char.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/bool.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anynonarray.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/anyenum.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anyelement.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/anycompatiblerange.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblenonarray.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblemultirange.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblearray.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anycompatible.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/any.ts | Makes generated PG Any extend shared SqlValue and updates runtime call helpers. |
| src/types/postgres/generated/aclitem.ts | Regenerated PG type bindings to use runtime.opCall and new import paths. |
| src/types/postgres/generate.ts | Updates codegen to new Postgres module layout + new runtime helper names + dialect-aware typedef lookup. |
| src/types/postgres/deserialize.ts | Adds Postgres deserialize/type registry registration. |
| src/types/meta.ts | Introduces standalone meta symbol module to avoid load-order cycles. |
| src/types/match.test.ts | Updates tests to new compile context API. |
| src/types/index.ts | Transitional re-export to new Postgres barrel (./postgres). |
| src/types/deserialize.ts | Replaces single registry with dialect-aware registry + registration API. |
| src/types/bool.ts | Adds dialect-agnostic Bool marker interface + isBool helper. |
| src/types/bool-mixin.ts | Adds shared bool operator helpers for dialect-specific Bool classes. |
| src/types/any.ts | Extracts shared SqlValue base with dialect metadata and common typed-value behaviors. |
| src/tables/generate.ts | Updates introspection import path to new Postgres location. |
| src/table.ts | Switches column detection to dialect-agnostic isColumn from types/any. |
| src/driver.ts | Adds dialect to Driver, introduces SqliteDriver adapter, and updates compile to dialect context. |
| src/builder/values.ts | Replaces Any checks with SqlValue for VALUES builder row typing/coercion. |
| src/builder/update.ts | Makes UPDATE builder dialect-aware (ctx threading) and uses shared Bool validation. |
| src/builder/sql.ts | Introduces CompileContext, dialect-aware param emission, and a Cast node. |
| src/builder/sql.test.ts | Updates compile tests to use pgCtx/sqliteCtx. |
| src/builder/query.test.ts | Updates compile usage to CompileContext and adjusts validation expectations. |
| src/builder/insert.ts | Threads compile context through bind/debug and updates getColumn import. |
| src/builder/delete.ts | Threads compile context through bind/debug and uses shared Bool validation. |
| package.json | Moves codegen script to Postgres module path and adds better-sqlite3 deps/types. |
| eslint.config.js | Updates ignore path to new Postgres generated directory. |
| docs/ISSUES.md | Documents RPC security considerations for cross-dialect/tenant typed value provenance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+196
to
+211
| const normalizeValue = (v: unknown): string => { | ||
| if (v === null || v === undefined) { | ||
| // Preserved as null via cast; downstream checks for null-ness. | ||
| return v as unknown as string; | ||
| } | ||
| if (typeof v === "string") {return v;} | ||
| if (typeof v === "number" || typeof v === "bigint" || typeof v === "boolean") {return String(v);} | ||
| if (v instanceof Uint8Array) { | ||
| // Match PG bytea repr: \x-prefixed lowercase hex. | ||
| return "\\x" + Buffer.from(v).toString("hex"); | ||
| } | ||
| return String(v); | ||
| }; | ||
|
|
||
| const normalizeRow = (row: { [key: string]: unknown }): { [key: string]: string } => | ||
| Object.fromEntries(Object.entries(row).map(([k, v]) => [k, normalizeValue(v)])); |
Comment on lines
+22
to
+26
| // Blob deserialize is a placeholder — the driver returns Buffer | ||
| // directly, so this path is only used if a raw string sneaks in. | ||
| // Real repr TBD (Buffer? Uint8Array? hex? see plan.md A1). | ||
| blob: { tsType: "object", deserialize: identity }, | ||
| bool: { tsType: "boolean", deserialize: (raw) => raw === "1" }, |
Comment on lines
+42
to
+46
| const rows = await Users.from().execute(db); | ||
| expect(rows.length).toBe(2); | ||
| // Rows come out in insertion order (SQLite rowid ordering). | ||
| expect(rows[0]!.name).toBe("alice"); | ||
| expect(rows[1]!.name).toBe("bob"); |
Comment on lines
+57
to
+60
| const rows = await Users.from() | ||
| .select((u) => ({ upperName: u.users.name.upper() })) | ||
| .execute(db); | ||
| expect(rows.map((r) => r.upperName)).toEqual(["ALICE", "BOB", "CAROL"]); |
| @@ -9,6 +11,7 @@ export type QueryResult = { rows: { [key: string]: string }[] }; | |||
| export type ExecuteFn = (query: Sql) => Promise<QueryResult>; | |||
Comment on lines
11
to
14
| import { Database } from "../database"; | ||
| import { Any, getColumn } from "../types/overrides/any"; | ||
| import { SqlValue, getColumn } from "../types/any"; | ||
| import { fn, expose } from "../exoeval/tool"; | ||
| import z from "zod"; |
Comment on lines
+82
to
+83
| @expose(z.union([z.literal(true), fn.returns(z.custom<SharedBool<any>>(isBool))])) | ||
| where(fn: ((ns: Namespace<Name, T>) => SharedBool<any>) | true): UpdateBuilder<Name, T, R> { |
…+values Ground work for the provenance/tagging refactor (ISSUES.md #16). - `CompileContext = { database: DatabaseRef }` (was `{ dialect }`). DatabaseRef is a minimal structural view (id: symbol, dialect) declared in sql.ts to sidestep a value-import cycle; the concrete `Database` class implements it. - `Driver.execute(text, values)` — drivers no longer see the Sql tree. Compilation happens in `Database.#exec`, which supplies its own DatabaseRef as the ctx. `runInSingleConnection`'s callback signature updated in lockstep. - `Ident` grows an optional `database` slot + provenance check in `bind(ctx)`. Untagged Idents (server-authored raw templates, escape hatch) still pass through; tagged Idents that mismatch throw. Table classes and their emitted Idents remain untagged in this commit; the sweep to route them through `db.scopedIdent` comes next. - `Database` implements DatabaseRef (id, dialect, name?). Txn-bound Databases inherit their parent's id so a Table built outside the txn still compiles inside it. Adds `db.scopedIdent(name)` factory. - `Bus` takes a `Database` (not `Driver`) so it can compile via db.execute. - Test-helpers, live/db-live, bus tests, postgres/index.test all updated to use `db.execute(sql)` instead of `driver.execute(sql)`. All 550 tests green. Placeholder emission now reads `ctx.database.dialect`; sentinel pgCtx/sqliteCtx still work for compile-only paths (SQL emission tests, .debug()). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- `Table(name, opts, database?)` factory takes an optional DatabaseRef;
`db.Table(name)` supplies `this`. Static `Table("users")` leaves the
new `database` static undefined (escape hatch, existing callers
unchanged).
- `TableBase.bind()` emits `new Ident(tableName, this.database)`.
`TableBase.ident(name)` convenience for column-name idents from
a Table class.
- Mutation builders (insert/update/delete) route their tableName +
column-name idents through `instance.constructor.ident(...)`, so a
db-scoped Table propagates its provenance to every emitted Ident.
- `reAlias` picks up the source Table's database when rewriting column
refs into alias-qualified `Column(alias, Ident(name, db))`.
- `QB.debug()` / mutation `.debug()` derive ctx from the first
Table's database instead of hardcoding `pgCtx`. Falls back to
sentinel for non-Table sources (db.values etc.).
550 tests green. Provenance enforcement is now active for any query
whose Idents come from `db.Table(name)` — cross-db mismatches throw at
compile with a clear message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Constructors take an optional `dialect` slot; bind(ctx) throws if it disagrees with `ctx.database.dialect`. - `runtime.funcCall` / `runtime.opCall` pass `type.dialect.name`, so every codegen'd PG method emits a properly-tagged Func/Op node automatically (no codegen output changes needed — the helper does it). - `SqlValue.cast()` / `.from()` (via TypedParam) also pass dialect so CAST expressions carry provenance. - `Srf` tracks dialect from its first column's class; its bind() now takes ctx and emits `new Ident(name, ctx.database)` for the function-name identifier. - `Fromable.bind` widened to `(ctx?)`; `FinalizedQuery.bind` threads ctx to `t.source.bind(ctx)` so Srf sources see the compile ctx. Untagged nodes (Func/Op without dialect — hand-authored `sql` templates, raw fragments) still pass through unchecked. This is the escape hatch for server-authored SQL that doesn't route through a typed class. 550 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Widen `runtime.funcCall` / `runtime.opCall` `type` arg to
`typeof SqlValue` so both PG's Any-typed classes and SQLite's
SqliteValue-typed classes call sites typecheck.
- SQLite codegen switches from raw `sql\`abs(\${this.toSql()})\``
templates to `funcCall("abs", [this], types.Integer)`. Every method
now emits a Func node carrying `dialect: "sqlite"`, checked at
compile.
- `argToSql` widened to duck-typed check (`.toSql()` presence) so it
handles both dialects uniformly.
- Regenerated all four SQLite class files (integer, real, text, blob).
550 tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…fic Bool - Update/DeleteBuilder track `matchAll: boolean` alongside `where`. The two are independent — a real predicate wins at compile if both are set. Bind emits `WHERE <pred>` when a predicate exists, no WHERE clause when only matchAll is set, and finalize throws when neither. - Drops the `Bool.from(sql\`TRUE\`)` construction in the shorthand. Removes the last PG-nominal `Bool` residue from delete/update. - Live-events update transformer emits `WHERE TRUE` explicitly for the before-CTE snapshot when the outer update used matchAll (its `FOR UPDATE` snapshot always needs a WHERE clause). Semantics for `.where(pred).where(true)` and `.where(true).where(pred)` are identical — the pred is honored. matchAll is only the "user acknowledged unrestricted mutation" flag. 550 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `.in()` back to SqliteValue with a non-generic signature that sidesteps the TS2589 that killed the PG-style `<T, Vs>(this: T, ...)` version at SQLite's shallow class hierarchy. Trade-off: users don't get "same-typed args required" narrowing at TS compile time; runtime enforcement via `serialize()` still applies. Two smoke tests validate the runtime path (member/non-member cases). 552 tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Replace open-design entry #16 with a description of the shipped solution: DatabaseRef in CompileContext, Ident carries database via Table, dialect tags on Func/Op/Cast/Srf, matchAll flag on .where(true). Follow-ups (db.Int4-style factories, Database/Connection split) noted. - Consolidate inline-import type refs (CompileContext) in query.ts. - Drop unused sql import from table.ts. - Curly braces around `if (v instanceof SqliteValue) return v;` per eslint rule. `npm run check` green: 28 files / 552 tests + examples/basic 6/6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the spec: Database owns dialect + provenance identity + Table factory +
scopedIdent; Connection has the driver + execute/hydrate/transaction/close/
live. Database.attach(driver) returns a fresh Connection (multiple allowed).
- `new Database({ dialect })` constructs a metadata handle (module-load
safe; no driver required).
- `const conn = db.attach(driver)` wires up a runtime Connection.
- Sql.execute(conn), builder .execute(conn) — explicit connection
passing throughout; supports txn nesting via the tx passed to the
callback.
- Driver.execute now takes `(text, values)` — Database compiles first
with its own ctx (`{ database: this }`) then hands to driver. Drivers
never see Sql nodes anymore.
- Connection ergonomically forwards `Table` / `from` / `values` /
`scopedIdent` to its database (no need to thread `db` through code
that already holds a `conn`).
- `typegres({ ... })` returns `{ db, conn }` for scripts / playground.
- Bus + runLiveIteration take Connection (they need `.execute`).
Test migration sweep: `db.execute(qb)` → `conn.execute(qb)`,
`.execute(db)` → `.execute(conn)`, `db.transaction` → `conn.transaction`,
etc. Across bus.test, db-live.test, database.test, hydrate.test,
events.test, postgres/index.test, rpc.test, query.test, sqlite tests,
examples/basic tests, demo.ts, and the README snippet.
`npm run check` green: 28 files / 552 tests + examples/basic 6/6.
Provenance still active — Idents carry Database id, dialects tagged on
Func/Op/Cast/Srf.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes it from the "still open" list; keeps only the instance-scoped typed-value factory (`db.Int4.from(5)`) as the remaining provenance follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up on the escape-hatch discussion.
Test sweep — no more untagged Tables in test files:
- Every `class X extends Table("name")` in the PG test suite becomes
`class X extends db.Table("name")`, with `db` imported from
test-helpers (or a locally-constructed `Database` for standalone
unit tests like extractor.test).
- `db.Table<Name, LocalC>(name)` gets an optional per-class LocalC
override so scope tests (each defining a distinct Principal shape)
can override the app-wide `C` per class.
- `expectSqlEqual` auto-detects the compile ctx by walking the sql
tree for the first tagged Ident, falls back to the suite's `db`,
falls back to pgCtx.
Provenance-enforcement suite (src/provenance.test.ts, 18 tests):
- Ident from dbA rejected when compiled against dbB.
- Ident from dbA accepted against dbA.
- Untagged `sql.ident(...)` passes any ctx (escape hatch preserved).
- Column via db.Table carries the Table's database.
- PG Func in SQLite ctx → throws (`upper()`).
- SQLite Func in PG ctx → throws (`abs()`).
- PG Op in SQLite ctx → throws (`+`).
- Untagged Func/Op/UnaryOp/Cast pass any ctx.
- Tagged UnaryOp/Cast reject cross-dialect.
- Cross-dialect Ident smuggling caught (PG-tagged Ident inside a
SQLite compile).
Also: Bus's `private db` field renamed to `private conn` (was a
Connection since the last commit; field name kept for smaller diff
previously — cleaned up now).
`npm run check` green: 29 files / 570 tests + examples/basic 6/6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
External code can no longer construct untagged Idents via
\`sql.ident("x")\`. The only public path is \`db.scopedIdent(name)\`,
which yields a provenance-checked Ident.
Internal library callers (live/events, live/extractor, live/bus,
live/events-ddl, builder/query output labels, builder/query AS clause
column names) continue to use \`new Ident(name)\` directly — the class
constructor still allows a \`database?\` optional for these
schema-agnostic local-scope uses (CTE aliases, output labels,
hardcoded event-table columns).
The library-internal path is unreachable from external code:
- \`sql\` is not exposed to exoeval callbacks (attackers over RPC can't
even reference \`sql.ident\`).
- \`Ident\` is exported from builder/sql but requires import; external
users don't touch it.
Callsite updates:
- test-helpers: \`sql.ident(schema)\` → \`db.scopedIdent(schema)\`
- postgres/index.test: \`sql.ident("result")\` → \`exec.database.scopedIdent(...)\`
- builder/sql.test: unit test shim that constructs \`new Ident(name)\`
directly — kept for SQL emission testing without a real Database.
- provenance.test: added test that \`sql.ident\` is undefined on the
public helper; kept the library-internal \`new Ident(name)\` test
as documentation.
571 tests + 6 examples green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Remove stale isBool doc comment in query.ts. - Strip narrative comments from types/meta.ts. - eslint ignore: src/types/postgres/generated → src/types/*/generated (also covers SQLite codegen output). - package.json codegen + codegen:check scripts run both PG and SQLite generators; codegen:check diffs both trees. - Delete src/types/index.ts (transitional PG re-export). All callers updated to import from src/types/postgres directly. - SQLite deserialize: blob tsType flipped from "object" to a sentinel string so the overload matcher can't wrongly accept arbitrary objects (Date, plain records) as blob primitives. - isSetRow widened from `instanceof types.Any` (PG-only) to `instanceof SqlValue` — SQLite SqlValue instances now pass through the mutation SET-row validator. - QueryResult row values typed `string | null` (SQL NULL is real; the previous `string` typing was unsound). - QueryBuilder.scalar() detects a SQLite source Table via first-source.database.dialect and throws with a clear Phase 2.1 pointer, replacing the "defer to bind" TODO. 29 files / 571 tests + examples/basic 6/6 all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…d-type declares - Each test in sqlite/table.test wrapped in a rollback transaction (same pattern as PG's withinTransaction) so tests are independent and idempotent — no id-numbering assumptions carry across. - Ordering assertions explicitly ORDER BY id — no more assumption that SQLite returns rows in insertion order. - expectTypeOf assertions on every test's row shape. - SQLite codegen now emits `declare deserialize: (raw: string) => <ts>` per storage class (integer→number, real→number, text→string, blob→Uint8Array). Without it TsTypeOf<T> collapses to unknown and SQLite row assertions can't be typed. - Regenerated the four SQLite class files. Also tightens bool-mixin: boolAnd/boolOr drop the primitive-boolean coercion path (no callers) and take a Bool value directly. PG + SQLite Bool signatures narrow from `Bool | boolean` to `Bool` accordingly. Runtime.ts: removed unused `import * as types from "./postgres"`. 29 files / 571 tests + examples 6/6 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compile(root) walks the tree for a tagged Ident (yielding its database) or a dialect-tagged Func/Op/Cast/Srf/TypedParam (yielding a synthetic DatabaseRef with the matching dialect). Falls back to the pgCtx sentinel only when the tree is fully untagged. The second arg stays optional so callers that want to force a particular ctx — e.g., sqlite/smoke.test's placeholder-emission check — still work. Most `compile(qb, pgCtx)` call sites could now drop the second arg without behavior change, but I've left them untouched to keep this diff small. 571 tests + 6 examples green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parity with PG codegen shape: - Every method with args gets `<M0 extends types.X<any> | number>` (or `| string` for text, plain type for blob) — accepts the typed value or a JS-native primitive. - Runtime dispatch through runtime.match([args], [[[matchers], retType]]) validates arg types + serializes primitives, then funcCall constructs a dialect-tagged Func node. - Blob-typed args reject primitives (registry sentinel from prior commit; no meaningful JS blob primitive yet). - Return type nullability tracked via `StrictNull<N | NullOf<M0> | ...>` from the actual method type params rather than opaque `typeof arg0`. Runtime widening: match's ArgMatcher.type + MatchCase.retType + Srf's column-type slot go from `typeof Any` (PG-only) to `typeof SqlValue` (dialect-agnostic). Both PG and SQLite codegen output typechecks. Regenerated all four SQLite class files. Two smoke tests added: Integer.atan2(1) works with a JS number primitive; passing a string throws with `No matching overload` at runtime. 573 tests + 6 examples green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cross-database provenance concern is now live in the code (Ident/Func/Op/Cast/Srf tagging, sql.ident removed from public helper, provenance-enforcement test suite). No open follow-up worth carrying in the issues doc — instance-scoped typed-value factories are a speculative nice-to-have, not an active gap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `Bool as SharedBool` rename in delete/update/query was needed back when both PG's concrete `Bool` class and the shared `Bool` interface were imported side by side. Since the `.where(true)` refactor removed the PG Bool import from the builders, the alias is redundant. 573 tests + 6 examples green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
QueryBuilder.debug() had a bespoke "first table's database, else pgCtx" probe — now it just calls compile(this), since FinalizedQuery.children() already surfaces TableRefs and walkForDatabase can pick up the tagged Ident from there. Insert/Update/DeleteBuilder.debug() can't rely on the tree walk (their Idents are created lazily inside Finalized*.bind() and aren't exposed via children()), so they pass ctx explicitly from instance.constructor.database. That field is always set for db.Table-defined tables; the only untagged tables (TypegresLiveEvents) are library-internal and never reach debug(). bind() on Insert/Update/DeleteBuilder still takes ctx and threads it to the transformer's returned Sql — Sql.bind's abstract signature requires it, and dropping it would break the ternary's inferred type.
Tables are always scoped to a Database — a table without one is nonsensical. Enforce it: - `TableBase.database` is now a required `DatabaseRef` (was optional). - `Table(name, opts, database)` factory requires `database`. - `TypegresLiveEvents` no longer extends `Table` — it doesn't use the builder API (no .from() / .insert() / etc.), only static utilities (`tableName`, `createTableSql`, `makeTransformer`). Rewriting it as a plain class lets us tighten TableBase without inventing a fake Database for the shadow-events table. Sentinel DatabaseRefs are gone: - `pgCtx` / `sqliteCtx` exports and the `walkForDatabase` auto-detect fallback in `compile()` are removed. `compile(root, ctx)` now requires ctx. - `QueryBuilder` carries `database` in its options, threaded from `Table.from` / `db.from` / `db.values` at construction (spread through every chained builder). `QueryBuilder.debug()` uses it directly. - Insert/Update/DeleteBuilder `debug()` methods drop the `!` on `constructor.database` since the field is now required. - Test files construct a real `Database` for their compile ctx instead of importing sentinels; test-helpers throws when a ctx can't be resolved rather than falling back to pgCtx. bind(ctx) on the builders is unchanged: ctx comes from the OUTER compile — deriving it from the builder's own table would defeat the provenance check for cross-database mixing.
- Extract `paramForDialect(dialect, position)` from compile()'s inline ternary. Exhaustive switch on DialectName forces a compile-time update when a new dialect is added. - Rewrite the sql.ident-absence comment as a positive statement about the current design (why schema idents must carry a DatabaseRef) instead of referencing the removed helper. - Add AGENTS.md rule: comments describe present code, not deleted code.
Bus's #poll query referenced the events table via an untagged Ident — the last remaining place a schema-referencing Ident was constructed without provenance in the query path. Store the Connection's Database on the Bus and thread it through the Ident, so the provenance check covers this path too. DDL emission in events-ddl.ts still uses untagged Idents; that's schema-setup code that runs once at boot and never mixes across databases, but can be tightened next if we want zero untagged schema idents anywhere.
The last untagged schema Idents in the codebase — the events-table
DDL and the live-mutation transformer's INSERT-into-events wrap —
now carry the emitting Database as provenance.
- `eventsTableSqlStatements(database)` and
`TypegresLiveEvents.createTable{Sql,SqlStatements}(database)` take
the target DatabaseRef; callers (`Connection.installLiveEvents`,
live test helper) already have one at the call site.
- Insert/Update/DeleteBuilder expose a `database` getter so the
live-events transformer can pull the target Database off the
builder it wraps.
- events.ts `wrapInsertOrDelete` / `wrapUpdate` / `eventsInsertCte`
thread `database` and tag every schema-referencing Ident (events
shadow table, user table, columns, ctid). CTE/RETURNING alias
Idents stay untagged — they're query-local names, not schema
references.
Add `scopedIdent(name)` to the DatabaseRef interface (Database already implements it). Sweep library-internal `new Ident(name, database)` call sites — events-ddl, bus, events transformer, Table.bind/ident, Func.bind, Srf.bind — to go through the interface method instead. Untagged Idents (CTE aliases, output column labels, reAlias for non- Table rows via db.values) still use `new Ident(name)` — there's no database to route through.
Origin/main's `typeRegistry` (later split into per-dialect registries)
was a legacy of pre-per-class codegen — a central map of typname →
{tsType, deserialize} that the base SqlValue looked up at runtime.
Now that codegen emits real classes and there's already an
`overrides/` hook next to `generated/`, the registry is unnecessary
indirection. Move deserialize onto the class where it belongs.
Changes:
- `SqlValue.deserialize`: identity default typed as `unknown` so
overrides can freely narrow the return type.
- `SqlValue.primitiveTs`: static string ("string" default) — the JS
`typeof` accepted by serialize/match. Was `getTypeDef(...).tsType`.
- Concrete overrides for the non-string types:
PG: Bool, Int2, Int4, Float4, Float8, Oid
SQLite: Bool, Integer, Real
Each sets `static primitiveTs` + `override deserialize(...)` in
one place — parser + return type + primitive check colocated.
- Codegen still emits `declare deserialize: (raw) => ${tsType}` on
concrete classes WITHOUT an override, so `TsTypeOf<Text<1>>`
narrows to `string`. Classes with overrides get no `declare` —
the override's real return type is the source of truth.
- `runtime.match` reads `m.type.primitiveTs` directly.
- `DialectName` moves from the deleted `deserialize.ts` to a
new one-line `src/types/dialect.ts`. All consumers (`driver.ts`,
`database.ts`, `builder/sql.ts`, `types/any.ts`) rebind their imports.
- SQLite codegen: `updateBarrel` re-exports from `overrides/<name>.ts`
when the file exists (matches how PG's `generateIndex` does it);
`emitClassFile` receives the overrides root so the "declare or not"
decision uses the same source of truth.
Deletions: three deserialize.ts files and every side-effect import
of them (`overrides/any.ts`, `sqlite/base.ts`).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.