Skip to content

Add PostgreSQL and SQLite SQL standard library#4068

Open
rossirpaulo wants to merge 3 commits into
canaryfrom
paulo/stdlib-sql
Open

Add PostgreSQL and SQLite SQL standard library#4068
rossirpaulo wants to merge 3 commits into
canaryfrom
paulo/stdlib-sql

Conversation

@rossirpaulo

@rossirpaulo rossirpaulo commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add the baml.sql standard-library API with tagged, parameterized statements
  • support PostgreSQL pools, native arrays, typed decoding, transactions, timeouts, and SQLSTATE
    errors
  • support SQLite files and memory databases, strict storage-class decoding, transactions,
    timeouts, and extended error codes
  • add native typed SQL errors and nested standard-library namespace support
  • run PostgreSQL integration coverage against a PostgreSQL 16 CI service

Why

Implements BEP-047, providing a safe native SQL API where interpolated values are always bound
parameters and query results decode using runtime BAML types.

Validation

  • cargo fmt --all -- --check
  • strict Clippy with -D warnings across all changed crates and targets
  • changed crate unit suites
  • 14 SQL integration tests covering SQLite locally
  • PostgreSQL bind, decode, transaction, enum, array, time, and SQLSTATE coverage configured for CI

Summary by CodeRabbit

  • New Features
    • Added a SQL standard library for typed, parameterized statements, query/scalar helpers, and JSON/time support with consistent SQLite and PostgreSQL connection and transaction options.
  • Bug Fixes
    • Improved SQL error/unsupported handling and made thrown-value processing consistent for native-created exceptions across runtimes.
  • Tests
    • Added end-to-end SQLite/PostgreSQL SQL integration coverage and expanded validation of bind decoding, transactions, cardinality, and error kinds.
    • Updated CI to run the Linux SQL test lane against a live PostgreSQL 16 service and adjusted test grouping/artifact wiring.

@rossirpaulo
rossirpaulo marked this pull request as ready for review July 16, 2026 22:47
@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

Binary size checks failed

6 violations · ✅ 1 passed

⚠️ Please fix the size gate issues or acknowledge them by updating baselines.

Artifact Platform File Gzip Gated on Baseline Delta Status
baml-cli Linux 🔒 28.3 MB 12.2 MB file 25.3 MB +3.0 MB (+12.0%) FAIL
packed-program Linux 🔒 19.3 MB 8.1 MB file 17.0 MB +2.3 MB (+13.4%) FAIL
baml-cli macOS 🔒 22.2 MB 10.7 MB file 19.6 MB +2.6 MB (+13.2%) FAIL
packed-program macOS 🔒 15.3 MB 7.2 MB file 13.2 MB +2.1 MB (+15.7%) FAIL
baml-cli Windows 🔒 23.9 MB 11.0 MB file 21.1 MB +2.8 MB (+13.2%) FAIL
packed-program Windows 🔒 16.3 MB 7.3 MB file 14.2 MB +2.1 MB (+15.1%) FAIL
bridge_wasm WASM 16.4 MB 🔒 4.4 MB gzip 4.4 MB +34.6 KB (+0.8%) OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.

Details & how to fix

Violations:

  • baml-cli (Linux) file_bytes: 28.3 MB exceeds limit of 26.0 MB (exceeded by +2.3 MB, policy: max_file_bytes)
  • baml-cli (Linux) file_delta_pct: +12.0% exceeds limit of 3.0% (exceeded by +9.0pp, policy: max_delta_pct)
  • packed-program (Linux) file_bytes: 19.3 MB exceeds limit of 17.5 MB (exceeded by +1.8 MB, policy: max_file_bytes)
  • packed-program (Linux) file_delta_pct: +13.4% exceeds limit of 3.0% (exceeded by +10.4pp, policy: max_delta_pct)
  • baml-cli (macOS) file_bytes: 22.2 MB exceeds limit of 20.2 MB (exceeded by +2.0 MB, policy: max_file_bytes)
  • baml-cli (macOS) file_delta_pct: +13.2% exceeds limit of 3.0% (exceeded by +10.2pp, policy: max_delta_pct)
  • packed-program (macOS) file_bytes: 15.3 MB exceeds limit of 13.6 MB (exceeded by +1.7 MB, policy: max_file_bytes)
  • packed-program (macOS) file_delta_pct: +15.7% exceeds limit of 3.0% (exceeded by +12.7pp, policy: max_delta_pct)
  • baml-cli (Windows) file_bytes: 23.9 MB exceeds limit of 21.7 MB (exceeded by +2.2 MB, policy: max_file_bytes)
  • baml-cli (Windows) file_delta_pct: +13.2% exceeds limit of 3.0% (exceeded by +10.2pp, policy: max_delta_pct)
  • packed-program (Windows) file_bytes: 16.3 MB exceeds limit of 14.6 MB (exceeded by +1.7 MB, policy: max_file_bytes)
  • packed-program (Windows) file_delta_pct: +15.1% exceeds limit of 3.0% (exceeded by +12.1pp, policy: max_delta_pct)

Add/update baselines:

.ci/size-gate/aarch64-apple-darwin.toml:

[artifacts.baml-cli]
file_bytes = 22172080
stripped_bytes = 22172144
gzip_bytes = 10746252
[artifacts.packed-program]
file_bytes = 15304098
gzip_bytes = 7218176

.ci/size-gate/x86_64-pc-windows-msvc.toml:

[artifacts.baml-cli]
file_bytes = 23891968
stripped_bytes = 23891968
gzip_bytes = 11008241
[artifacts.packed-program]
file_bytes = 16286681
gzip_bytes = 7336817

.ci/size-gate/x86_64-unknown-linux-gnu.toml:

[artifacts.baml-cli]
file_bytes = 28321552
stripped_bytes = 28321544
gzip_bytes = 12226163
[artifacts.packed-program]
file_bytes = 19319368
gzip_bytes = 8141914

Generated by cargo size-gate · workflow run

@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview, Comment Jul 21, 2026 11:59pm
promptfiddle Ready Ready Preview, Comment Jul 21, 2026 11:59pm
promptfiddle2 Ready Ready Preview, Comment Jul 21, 2026 11:59pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a BAML SQL standard library with PostgreSQL and SQLite support, typed statements and bindings, transactions, decoding, structured errors, native sysops, compiler wiring, integration tests, and PostgreSQL-backed CI coverage.

Changes

SQL runtime

Layer / File(s) Summary
SQL contracts and statement types
baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/..., baml_language/crates/baml_builtins2/src/sql.rs
Defines SQL APIs, options, errors, database and transaction methods, backend constructors, typed bindings, and statement rendering.
VM SQL binding and thrown values
baml_language/crates/bex_vm/src/package_baml/sql.rs, baml_language/crates/sys_types/src/lib.rs, baml_language/crates/bex_engine/src/lib.rs
Converts VM values into SQL binds, validates arrays, constructs SQL errors, and propagates BAML-thrown values through sysop execution.
Native SQL engine and platform wiring
baml_language/crates/sys_native/src/sql.rs, baml_language/crates/sys_ops/src/lib.rs
Implements PostgreSQL and SQLite connections, encoding, decoding, statement validation, execution, transactions, lifecycle handling, and unsupported-platform fallbacks.
Nested namespace and error dispatch generation
baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs, baml_language/crates/baml_compiler2_hir/src/builder.rs, baml_language/crates/baml_builtins2_codegen/src/extract.rs
Normalizes nested namespace identifiers and dispatch paths, supports dynamic error categories, and extends code-generation tests.
Integration validation and PostgreSQL CI
baml_language/crates/baml_tests/tests/sql.rs, .github/workflows/cargo-tests.reusable.yaml
Adds SQLite and conditional PostgreSQL coverage and provisions PostgreSQL 16 for the Linux test lane.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant BamlSqlApi
  participant BamlNamespaceSql
  participant NativeSysOps
  participant sqlx
  BamlSqlApi->>BamlNamespaceSql: create statement and connect database
  BamlNamespaceSql->>NativeSysOps: dispatch SQL sysop
  NativeSysOps->>sqlx: execute query or transaction operation
  sqlx-->>NativeSysOps: rows, command result, or SQL error
  NativeSysOps-->>BamlSqlApi: decoded value or SqlError
Loading

Possibly related PRs

Suggested reviewers: codeshaunted

Poem

I’m a rabbit with queries to run,
Through SQLite moonlight and Postgres sun.
Bind every carrot, decode every row,
Commit what blooms, roll back what won’t grow.
SQL hops softly from burrow to gear.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 57.73% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding PostgreSQL and SQLite SQL standard-library support.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch paulo/stdlib-sql

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
baml_language/crates/sys_types/src/lib.rs (1)

186-236: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract a shared helper for BamlThrown/HostThrown display formatting.

The new BamlThrown arm duplicates the HostThrown arm's instance/message-extraction logic verbatim, differing only in the "thrown" vs "host thrown" prefix.

♻️ Proposed refactor to de-duplicate the formatting logic
+fn format_thrown_instance(
+    f: &mut std::fmt::Formatter<'_>,
+    prefix: &str,
+    boxed: &BexExternalValue,
+) -> std::fmt::Result {
+    match boxed {
+        BexExternalValue::Instance {
+            class_name, fields, ..
+        } => {
+            let message = fields.get("message").and_then(|v| match v {
+                BexExternalValue::String(s) => Some(s.as_str()),
+                _ => None,
+            });
+            match message {
+                Some(m) => write!(f, "{prefix} {class_name}: {m}"),
+                None => write!(f, "{prefix} {class_name}"),
+            }
+        }
+        other => write!(f, "{prefix} {other:?}"),
+    }
+}
+
 fn display_summary(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
     match self {
         Self::Vm(e) => std::fmt::Display::fmt(e, f),
-        Self::BamlThrown(boxed) => match boxed.as_ref() {
-            BexExternalValue::Instance {
-                class_name, fields, ..
-            } => {
-                let message = fields.get("message").and_then(|v| match v {
-                    BexExternalValue::String(s) => Some(s.as_str()),
-                    _ => None,
-                });
-                match message {
-                    Some(m) => write!(f, "thrown {class_name}: {m}"),
-                    None => write!(f, "thrown {class_name}"),
-                }
-            }
-            other => write!(f, "thrown {other:?}"),
-        },
-        Self::HostThrown(boxed) => match boxed.as_ref() {
-            BexExternalValue::Instance {
-                class_name, fields, ..
-            } => {
-                let message = fields.get("message").and_then(|v| match v {
-                    BexExternalValue::String(s) => Some(s.as_str()),
-                    _ => None,
-                });
-                match message {
-                    Some(m) => write!(f, "host thrown {class_name}: {m}"),
-                    None => write!(f, "host thrown {class_name}"),
-                }
-            }
-            other => write!(f, "host thrown {other:?}"),
-        },
+        Self::BamlThrown(boxed) => format_thrown_instance(f, "thrown", boxed),
+        Self::HostThrown(boxed) => format_thrown_instance(f, "host thrown", boxed),
     }
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/sys_types/src/lib.rs` around lines 186 - 236, Extract
the duplicated instance/message formatting from OpErrorPayload::display_summary
into a shared helper that accepts the throw-label prefix and BexExternalValue
reference. Update both BamlThrown and HostThrown arms to call the helper with
their respective “thrown” and “host thrown” labels, preserving the existing
output for instance messages and non-instance values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs`:
- Around line 726-736: Make namespace identifier conversion injective by
escaping underscores before converting dots, then apply the same encoding
consistently in ns_module_ident, namespace_pascal, and namespace_symbol so
distinct namespaces cannot collide in generated modules or dispatch methods. Add
a regression test covering namespaces such as sql.sqlite and sql_sqlite and
verify their generated identifiers remain distinct.

In `@baml_language/crates/bex_vm/src/package_baml/sql.rs`:
- Around line 58-63: Update the unsupported array element-type error in the
Object::Array handling of snapshot_bind to use backend-neutral wording, removing
the hardcoded “PostgreSQL” reference while preserving the existing error
behavior.

In `@baml_language/crates/sys_native/src/sql.rs`:
- Around line 1973-2074: Update execute_transaction and query_transaction to use
the existing discard-on-drop transaction connection handling used by
begin/finalize. Ensure cancellation during any awaited describe, execute, or
row-fetch operation prevents the live connection from being reused; preserve
normal connection reuse after successful completion.

---

Nitpick comments:
In `@baml_language/crates/sys_types/src/lib.rs`:
- Around line 186-236: Extract the duplicated instance/message formatting from
OpErrorPayload::display_summary into a shared helper that accepts the
throw-label prefix and BexExternalValue reference. Update both BamlThrown and
HostThrown arms to call the helper with their respective “thrown” and “host
thrown” labels, preserving the existing output for instance messages and
non-instance values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 518bc6ac-db3d-4ced-8131-732478ca615e

📥 Commits

Reviewing files that changed from the base of the PR and between e318148 and 51fc2a9.

⛔ Files ignored due to path filters (2)
  • baml_language/Cargo.lock is excluded by !**/*.lock
  • baml_language/crates/baml_cli/src/snapshots/baml_cli__describe_command_tests__render_builtin_package_listing.snap is excluded by !**/*.snap
📒 Files selected for processing (24)
  • .github/workflows/cargo-tests.reusable.yaml
  • baml_language/Cargo.toml
  • baml_language/crates/baml_builtins2/Cargo.toml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_postgres/postgres.baml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_sqlite/sqlite.baml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/sql.baml
  • baml_language/crates/baml_builtins2/src/lib.rs
  • baml_language/crates/baml_builtins2/src/sql.rs
  • baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs
  • baml_language/crates/baml_builtins2_codegen/src/extract.rs
  • baml_language/crates/baml_compiler2_hir/src/builder.rs
  • baml_language/crates/baml_tests/Cargo.toml
  • baml_language/crates/baml_tests/tests/sql.rs
  • baml_language/crates/bex_engine/src/lib.rs
  • baml_language/crates/bex_external_types/src/bex_external_value.rs
  • baml_language/crates/bex_vm/src/package_baml/mod.rs
  • baml_language/crates/bex_vm/src/package_baml/sql.rs
  • baml_language/crates/sys_native/Cargo.toml
  • baml_language/crates/sys_native/src/host_impls.rs
  • baml_language/crates/sys_native/src/lib.rs
  • baml_language/crates/sys_native/src/sql.rs
  • baml_language/crates/sys_ops/src/lib.rs
  • baml_language/crates/sys_types/src/lib.rs
  • baml_language/crates/sys_wasm/src/host_value.rs

Comment thread baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs
Comment thread baml_language/crates/bex_vm/src/package_baml/sql.rs
Comment thread baml_language/crates/sys_native/src/sql.rs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
baml_language/crates/baml_compiler2_hir/src/builder.rs (1)

2180-2192: 📐 Maintainability & Code Quality | 🔵 Trivial

Run cargo test --lib from baml_language/. The repo root doesn’t contain a Cargo manifest, so the test command needs to target the Rust workspace directory.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/baml_compiler2_hir/src/builder.rs` around lines 2180 -
2192, Run the Rust library tests from the baml_language workspace directory
using cargo test --lib, rather than running the command from the repository root
where no Cargo manifest exists.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@baml_language/crates/baml_compiler2_hir/src/builder.rs`:
- Around line 2180-2192: Run the Rust library tests from the baml_language
workspace directory using cargo test --lib, rather than running the command from
the repository root where no Cargo manifest exists.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 50921059-984e-4f5d-ad29-b499023df07a

📥 Commits

Reviewing files that changed from the base of the PR and between 5cee29b and 2d20e88.

⛔ Files ignored due to path filters (2)
  • baml_language/Cargo.lock is excluded by !**/*.lock
  • baml_language/crates/baml_cli/src/snapshots/baml_cli__describe_command_tests__render_builtin_package_listing.snap is excluded by !**/*.snap
📒 Files selected for processing (24)
  • .github/workflows/cargo-tests.reusable.yaml
  • baml_language/Cargo.toml
  • baml_language/crates/baml_builtins2/Cargo.toml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_postgres/postgres.baml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_sqlite/sqlite.baml
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/sql.baml
  • baml_language/crates/baml_builtins2/src/lib.rs
  • baml_language/crates/baml_builtins2/src/sql.rs
  • baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs
  • baml_language/crates/baml_builtins2_codegen/src/extract.rs
  • baml_language/crates/baml_compiler2_hir/src/builder.rs
  • baml_language/crates/baml_tests/Cargo.toml
  • baml_language/crates/baml_tests/tests/sql.rs
  • baml_language/crates/bex_engine/src/lib.rs
  • baml_language/crates/bex_external_types/src/bex_external_value.rs
  • baml_language/crates/bex_vm/src/package_baml/mod.rs
  • baml_language/crates/bex_vm/src/package_baml/sql.rs
  • baml_language/crates/sys_native/Cargo.toml
  • baml_language/crates/sys_native/src/host_impls.rs
  • baml_language/crates/sys_native/src/lib.rs
  • baml_language/crates/sys_native/src/sql.rs
  • baml_language/crates/sys_ops/src/lib.rs
  • baml_language/crates/sys_types/src/lib.rs
  • baml_language/crates/sys_wasm/src/host_value.rs
🚧 Files skipped from review as they are similar to previous changes (22)
  • baml_language/crates/baml_builtins2/Cargo.toml
  • baml_language/crates/sys_native/src/lib.rs
  • baml_language/crates/baml_builtins2_codegen/src/extract.rs
  • baml_language/crates/baml_tests/Cargo.toml
  • baml_language/crates/bex_external_types/src/bex_external_value.rs
  • baml_language/crates/sys_native/src/host_impls.rs
  • baml_language/crates/sys_wasm/src/host_value.rs
  • baml_language/Cargo.toml
  • baml_language/crates/baml_builtins2/src/lib.rs
  • baml_language/crates/baml_builtins2/src/sql.rs
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_postgres/postgres.baml
  • baml_language/crates/bex_vm/src/package_baml/mod.rs
  • baml_language/crates/bex_engine/src/lib.rs
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/ns_sqlite/sqlite.baml
  • baml_language/crates/sys_ops/src/lib.rs
  • baml_language/crates/sys_types/src/lib.rs
  • baml_language/crates/sys_native/Cargo.toml
  • baml_language/crates/bex_vm/src/package_baml/sql.rs
  • baml_language/crates/baml_builtins2/baml_std/baml/ns_sql/sql.baml
  • .github/workflows/cargo-tests.reusable.yaml
  • baml_language/crates/baml_builtins2_codegen/src/codegen_io.rs
  • baml_language/crates/baml_tests/tests/sql.rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant