feat(sqlite3): translate dot-commands (.tables, .schema, .mode, .read, ...)#249
Open
subsetpark wants to merge 2 commits into
Open
feat(sqlite3): translate dot-commands (.tables, .schema, .mode, .read, ...)#249subsetpark wants to merge 2 commits into
subsetpark wants to merge 2 commits into
Conversation
|
@subsetpark is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
…, ...)
Real sqlite3 ships a family of dot-commands that are a feature of the CLI,
not the library. sql.js doesn't implement them, so agent scripts pasted
verbatim from a real sqlite3 session hit `near ".": syntax error` on the
first dot-command (.tables, .schema, .mode csv, .read seed.sql, ...).
Add a host-side preprocessor (commands/sqlite3/dot-commands.ts) that runs
before SQL reaches the worker. The scanner is char-level with state for
SQL string literals ('...', "...", including the doubled-quote escapes)
and comments (-- ..., /* ... */); a dot-command is recognized only at a
boundary (start-of-input, after `;`, or after `\n`) and only outside
strings/comments — `'a\n.tables'` round-trips intact, and
`.headers on; .mode csv; CREATE TABLE...` is three tokens on one line.
Each recognized dot-command resolves to one of:
- SQL replacement — .tables/.schema/.indexes/.databases/.help translate
to sqlite_master/PRAGMA queries (with shell-glob → SQL-LIKE conversion).
- Formatter mutation — .headers, .mode, .separator, .nullvalue adjust
output state for downstream SQL; bad args surface real-sqlite3-shaped
errors (e.g. "Error: unknown mode: parquet"). Last write wins.
- .read FILE — recursive scan of the included file, sharing formatter
state, capped at MAX_READ_DEPTH to prevent loops.
- .quit / .exit — terminate preprocessing; subsequent input is dropped.
- Silent drop — .echo, .timer, .changes, .bail, .show, .eqp, .width,
.prompt, .print, .explain are recognized and discarded.
- Not-implemented family — .dump, .save, .backup, .import, .clone,
.restore, .open, .output, .shell, .system, .cd, .load, .iotrace,
.log, .excel translate to `SELECT 'Error: ...' AS error` so the
message rides in stdout in script-order without aborting SQL.
- Passthrough — unknown dot-commands are left verbatim so sql.js
produces its native syntax error.
Pairs with two new CLI flags:
- `-init FILENAME` — read SQL from FILENAME before main SQL.
- `-batch` — no-op (just-bash is always non-interactive).
Dot-command errors are routed in-band to stdout alongside SQL errors when
-bail is unset (matches real sqlite3's single-channel reporting); with
-bail they go to stderr with exit 1.
Tests: 247 sqlite3 tests pass against the existing worker (no worker
changes needed — preprocessor runs entirely host-side). New test files:
- sqlite3.dot-commands.test.ts: comprehensive preprocessor coverage
- sqlite3.invocation-shapes.test.ts: pinned agent invocation patterns
- sqlite3.flags.test.ts: -init / -cmd / -batch flag behavior
- sqlite3.sql-features.test.ts: generic SQL features through the pipeline
Docs: packages/just-bash/docs/sqlite3-invocation-shapes.md catalogues the
canonical CLI invocation shapes agents use.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
e779c8c to
d77d3cf
Compare
This was referenced May 19, 2026
There was a problem hiding this comment.
Pull request overview
Adds a host-side sqlite3 “dot-command” preprocessor to bring just-bash’s sqlite3 CLI closer to real sqlite3 behavior when users paste interactive sessions (e.g. .tables, .schema, .mode, .read). This keeps the worker executing pure SQL while the host rewrites/handles CLI-only meta-commands.
Changes:
- Introduces
preprocessDotCommands()to translate/handle sqlite3 dot-commands (SQL replacement, formatter mutations,.read,.quit, etc.) before SQL reaches the worker. - Adds CLI flag support for
-init <file>(pre-load script) and accepts-batchas a no-op. - Adds extensive Vitest coverage and a docs catalogue of agent invocation shapes / supported features.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/just-bash/src/commands/sqlite3/sqlite3.ts | Wires in dot-command preprocessing and adds -init / -batch handling + dot-error routing. |
| packages/just-bash/src/commands/sqlite3/dot-commands.ts | Implements char-level scanner + dot-command translation, .read recursion, and formatter mutations. |
| packages/just-bash/src/commands/sqlite3/sqlite3.dot-commands.test.ts | New test suite covering dot-command behavior and scanner edge cases. |
| packages/just-bash/src/commands/sqlite3/sqlite3.flags.test.ts | New tests for -init and -batch and their interaction with dot-commands. |
| packages/just-bash/src/commands/sqlite3/sqlite3.invocation-shapes.test.ts | New tests pinning common real-world invocation patterns. |
| packages/just-bash/src/commands/sqlite3/sqlite3.sql-features.test.ts | New tests pinning important SQL feature support through the wrapper. |
| packages/just-bash/docs/sqlite3-invocation-shapes.md | Documents supported/observed invocation shapes, dot-commands, and known limitations. |
| .changeset/sqlite3-dot-commands.md | Changeset entry describing the new dot-command support and flags. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ecode
Use `??` instead of `||` when sourcing SQL so an explicit empty positional
SQL arg ("") is "provided but empty" and does not silently fall through to
piped stdin. Decode/trim stdin once and reuse it in the no-SQL guard so both
checks agree on the same value. Addresses Copilot review feedback on vercel-labs#249.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Real
sqlite3ships a family of dot-commands that are a feature of the CLI, not the library. sql.js doesn't implement them, so agent scripts pasted verbatim from a realsqlite3session hitnear ".": syntax erroron the first dot-command (.tables,.schema,.mode csv,.read seed.sql, …).This PR adds a host-side preprocessor (
commands/sqlite3/dot-commands.ts) that runs before SQL reaches the worker. No worker changes needed — the worker keeps executing pure SQL.How the preprocessor works
Char-level scanner with state for SQL string literals (
'…',"…", including the doubled-quote escapes''/"") and comments (-- …,/* … */). A dot-command is recognized only at a boundary — start-of-input, just after;, or just after\n— and only outside strings/comments. This lets'a\n.tables'round-trip intact and lets.headers on; .mode csv; CREATE TABLE…be three tokens on one line.Each recognized dot-command resolves to one of seven outcomes:
.tables,.schema,.indexes/.indices,.databases,.help→sqlite_master/PRAGMA queries (shell-glob*/?→ SQL-LIKE%/_).headers/.header,.mode,.separator,.nullvalue— adjusts output state for downstream SQL; bad args surface real-sqlite3-shaped errors (Error: unknown mode: parquet). Last write wins..read FILEMAX_READ_DEPTHto prevent loops.quit/.exit.echo,.timer,.changes,.bail,.show,.eqp,.width,.prompt,.print,.explain— recognized and discarded.dump,.save,.backup,.import,.clone,.restore,.open,.output,.shell,.system,.cd,.load,.iotrace,.log,.excel→ translates toSELECT 'Error: …' AS errorso the message rides in stdout in script-order without aborting surrounding SQLDot-command errors are routed in-band to stdout alongside SQL errors when
-bailis unset (matches real sqlite3's single-channel reporting); with-bailthey go to stderr with exit 1.Two paired CLI flags
-init FILENAME— read SQL fromFILENAMEbefore main SQL (matches real sqlite3; pairs naturally with.read)-batch— accepted as a no-op since just-bash is always non-interactiveWhat's not in this PR
This is the host-side preprocessor only. There's a small Bun-specific worker fix in our internal fork (
coerceDbBufferhandles a Bunworker_threadsstructured-clone regression that surfacesnullas a zero-lengthArrayBuffer) that I'd be happy to send as a separate PR if useful — it's orthogonal to dot-commands and Bun-runtime-specific.Test plan
pnpm --filter just-bash exec vitest run src/commands/sqlite3/— 247/247 pass (16 test files, including 4 new files from this PR; existing worker untouched)pnpm --filter just-bash typecheck— cleanpnpm --filter just-bash build— cleanNew test files
sqlite3.dot-commands.test.ts— preprocessor coverage (boundaries, string/comment skipping, all seven outcome categories,.readrecursion, formatter precedence)sqlite3.invocation-shapes.test.ts— pins the canonical agent invocation patterns (sqlite3 :memory: "SQL",echo SQL | sqlite3,sqlite3 -cmd ... :memory: "...", etc.)sqlite3.flags.test.ts—-init/-cmd/-batchflag behaviorsqlite3.sql-features.test.ts— generic SQL features through the pipelineDocs
packages/just-bash/docs/sqlite3-invocation-shapes.mdcatalogues the CLI invocation shapes agents reach for, with the equivalent results they should see.Background
We've been carrying this in
flowglad/just-bashfor a while (fork notes) because our agent pastes sqlite3 CLI sessions verbatim. Now that #190 fixed the bundled-worker shipping bug (thank you!), the dot-command preprocessor is the remaining gap between just-bash'ssqlite3and the real CLI's surface area. We'd love to retire the carry and use upstream directly.🤖 Generated with Claude Code