perf(sqlite): preserve native options and bound Arrow loading - #820
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #820 +/- ##
==========================================
- Coverage 81.60% 81.17% -0.43%
==========================================
Files 521 521
Lines 78073 78208 +135
Branches 11235 11263 +28
==========================================
- Hits 63714 63488 -226
+ Misses 11130 11102 -28
- Partials 3229 3618 +389
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
cofin
changed the title
cofin
force-pushed
the
feat/sqlite-adapters-optimization
branch
6 times, most recently
from
September 27, 2026 18:45
2f3d7c1 to
47fd8d2
Compare
cofin
changed the title
cofin
changed the title
cofin
force-pushed
the
feat/sqlite-adapters-optimization
branch
3 times, most recently
from
September 28, 2026 19:25
be151e7 to
15672d3
Compare
…d add aiosqlite end_transaction
cofin
force-pushed
the
feat/sqlite-adapters-optimization
branch
from
September 28, 2026 19:32
15672d3 to
ae5a930
Compare
cofin
added a commit
that referenced
this pull request
Sep 29, 2026
## Summary Prepares the v0.65.0 release: - Bumps project version to `0.65.0` in `pyproject.toml` and updates `uv.lock`. - Updates `docs/changelog.rst` with the `v0.65.0` release notes covering all 14 PRs merged since `v0.64.0`: - IBM Db2 database adapter, SQLGlot dialect, and extension support (#811, #823). - Spanner and Spangres SQLGlot dialect isolation and AST support (#813). - Spanner adapter & store optimizations and release fixes (#814). - MySQL family adapter consolidation and data-dictionary reorganization (#821). - OracleDB adapter and store cleanup (#817, #823). - Native controls and performance optimizations for BigQuery (#812), PostgreSQL (#822), SQLite (#820), and MSSQL (#819). - Native Arrow capabilities and parse-once bindings (#818). - Fixture JSON export/load, conflict keys normalization, and upsert options (#824, #825, #826, #827). - Quoted and schema-qualified migration `version_table` identifiers (#815, #828, #829). - Builder CTE and column pruning ownership fixes (#829). ## Validation - `make docs` passed without warnings or errors. - `make lint` (ruff, prek, mypy, pyright, slotscheck, zizmor) passed with 0 errors. - Targeted unit tests passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserve SQLite and aiosqlite native behavior while reducing adapter overhead. Keep cached query metadata, bounded Arrow conversion, cursor cleanup, and ADK batch writes. Retain custom window functions, configurable transaction modes, and Arrow import batch sizing with validation before transaction start.
Custom window functions require Python 3.11+ and a supporting SQLite runtime; configured functions fail clearly when the Python API is unavailable. Existing defaults remain unchanged: sqlite uses deferred transactions and aiosqlite uses BEGIN IMMEDIATE. Async window registration runs on the connection's worker thread.
Fix pooled in-memory connection recovery and roll back adapter-owned Arrow transactions on conversion failures or cancellation. Preserve caller-owned transactions, stdlib JSON behavior, quoted annotations, and per-statement async script cancellation boundaries. Tuple-stream additions were incomplete and remain excluded. Existing compiled helpers remain enabled; the expanded SQLite driver mypyc target is deferred pending exact compiled-wheel validation.
Validation: earlier focused driver/pool slice passed 86 tests with 2 Python-version skips; restoration regressions passed 4 tests. Affected adapters and tests pass Ruff; targeted adapter Pyright reports 0 errors/warnings. No services, full suite, or local compiled-wheel build were run.
Rebase validation: based on main
a1067c856, with branch source changes preserved. Reconciled changelog entries, checked release-history preservation, and passed the focused documentation audit and whitespace checks. No runtime suites were rerun for this documentation-only update.CI repair: register the consumed BigQuery
storage_write_stream_typesetting in the shared feature contract (exact regression passed). Docker pre-pulls retry up to three times with bounded backoff and preserve failure after exhaustion. YAML, shell syntax, scoped zizmor, and fake-command success/retry/failure checks passed. No local database services or broad test suites were run. Fresh hosted checks are pending on the updated head.