Skip to content

build(duckdb): bump libduckdb to 2.0.0 - #3928

Merged
datlechin merged 8 commits into
mainfrom
build/libduckdb-2.0
Oct 31, 2026
Merged

datlechin merged 8 commits into
mainfrom
build/libduckdb-2.0

Conversation

@datlechin

@datlechin datlechin commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

  • libduckdb 1.5.2 to 2.0.0 everywhere it is pinned: the macOS DuckDB driver and Parquet export (libs-6), the iOS DuckDB.xcframework (libs-6), and the Linux duckdb crate 1.10506.1 to 1.20000.1.
  • Fixes .duckdb backup and restore of a database with foreign keys. On 1.5, COPY FROM DATABASE ... (DATA) copies tables in no foreign-key order. 2.0 copies them in dependency order (duckdb/duckdb@c111e955fb).
  • 2.0 creates new files in storage v2.0.0, which DuckDB 1.x cannot open. Every open now sets storage_compatibility_version = 'v0.10.2', 1.x's default, in the macOS plugin, the iOS driver and the Linux driver. New, attached and backup files stay on storage 64, readable by TablePro 0.84, the 1.5 CLI and other 1.x tools.
  • build-duckdb.sh and build-duckdb-ios.sh follow 2.0's linking model: per-extension describe functions registered by DuckDB's generated loader (duckdb_register_static_extensions) replace the DUCKDB_EXTENSION_*_LINKED defines. macOS links the loader's autoregister object, which -force_load keeps. iOS calls the loader before the first open, as DuckDB's own Swift package does.
  • Both scripts pass DUCKDB_EXPLICIT_VERSION. Without it 2.0 builds as v2.0.0-dev1, and automatic installs of httpfs and quack look in a commit-named folder.
  • Capabilities: httplib and loadable_extensions on macOS, httplib only on iOS. local_extension_repository is left out: it points automatic installs at the build machine.
  • iOS turns automatic installs off in the open config. 2.0 removed the compile-time switches the old iOS build used.
  • iOS drops httpfs and quack from duckdb-ios-extensions.cmake. httpfs needs libcurl, which the iOS SDK ships no headers for, so that config never built. The published framework (feat(ios): DuckDB support for files and in-memory databases #1537) predates them (feat(plugin-duckdb): add remote Quack protocol connections #1720), and the iOS app uses neither.
  • Vendored duckdb.h is 2.0.0's. No C API the plugin calls changed signature. The deprecated value API still compiles: DUCKDB_API_ALLOW_DEPRECATED defaults to 1.
  • The DuckDB probes link with -force_load, as the plugin does. New apple/scripts/probes/check-duckdb-storage-version.sh.
  • Three DuckDB probes had stopped compiling on main, unrelated to this bump: check-duckdb-table-listing-parity.sh and the corpus harness left out Plugins/Shared and TableProSQLVocabulary, and check-duckdb-offline-metadata.sh never linked the PluginKit that DuckDBSchemaQueries.swift imports since feat(sqlite): attach more database files to a SQLite, libSQL or DuckDB connection #3748. They now build what the plugin links.
  • Linux: duckdb 1.20000.1 brings Arrow 60. The driver reads a fixed-size list's offset with value_offset_at, since Arrow 60 deprecates value_offset.
  • Linux: the workspace's comfy-table moves from 7 to 8, the copy Arrow 60 already builds. The text table export ports its psql border preset to 8's TableStyle, and its 7 tests pass unchanged. num-bigint 0.5 joins 0.4, which oo7 and bollard need; deny.toml only warns on duplicates.
  • Linux: 2.0 takes an expression as the ATTACH path and opens the text it casts to, so '<path>'::BLOB or '<dir>/' || ('<name>') could attach a file another window holds past the open-file check. The driver now has DuckDB evaluate the path as CAST(... AS VARCHAR), reads a call's or a type's parentheses as part of the path, and refuses an ATTACH whose path it cannot evaluate instead of running it unchecked.
  • Linux: 2.0 quotes a nested string that holds a backslash (DuckDB added it to NestedToVarcharCast::LOOKUP_TABLE), and keeps an untyped NULL as type NULL, an Arrow NullArray inside a struct or list. The nested decoder follows both.
  • Linux: ADR 0031 supersedes 0025's crate version. License notices regenerated: libpg_query is gone from 2.0.
  • SQL that 2.0 reads differently reaches users' queries: single-arrow lambdas are refused, ~ matches part of a string, integer division by zero errors, and X'..' is a BLOB. TablePro's own DuckDB SQL uses none of these.
  • Tests updated on purpose, each for a DuckDB 2.0 behavior:
    • DuckDBDriverTests.testStreamThrowsAnErrorRaisedAfterItsFirstChunks (iOS) sets streaming_buffer_size = '1MB'. 2.0's default rose from 976.5 KiB to 10 MiB, so the 250,001-row query ran to its error inside duckdb_execute_pending, before any fetch.
    • a_string_is_quoted_only_where_duckdb_quotes_it (Linux): a\b now prints as 'a\\b'.
    • every_spelling_of_an_open_file_is_refused (Linux): 40 + 2 and an E'\u..' path are no longer parse errors on 2.0, so they leave the refused-by-DuckDB list. Three expression spellings of the open file join the refused list. U&'...' is now a Binder error.
    • a_rowid_moves_when_a_checkpoint_compacts_the_table (Linux): deletes enough rows that a checkpoint compacts in both formats. On a v2.0-format file the old delete left rowid 390000 in place, while the pinned format still moved it to 152400.
    • a_new_or_attached_database_stays_readable_by_duckdb_1 (Linux, new) writes each database in its own transaction.

Test plan

FK backup and restore with the plugin's .duckdb statements: 9 tables, about 926k rows, parents, children, grandchildren, two parents per row, and a self-referencing FK whose column is NULL. Each statement goes through duckdb_prepare, as the plugin sends it. 1.5.2 refuses COPY FROM DATABASE there ("Cannot prepare multiple statements at once"), so the harness sends (SCHEMA) then (DATA). 2.0 prepares it as one statement.

libduckdb Backup Restore (a storage-64 backup)
1.5.2 (libs-5) 0 of 20 0 of 20
2.0.0 (libs-6) 20 of 20 20 of 20
  • A self-referencing FK whose rows use it still fails to copy on both versions: 0 of 20 each. DuckDB checks the key before that statement's own rows are visible.

Storage, written by 2.0.0 and read back with the 1.5.2 library and the 1.5.2 CLI:

What 2.0 wrote Default Pinned v0.10.2
New file header 999, v2.0.0+, 1.5.2 refuses it header 64, v1.0.0+, 1.5.2 reads it
ATTACH of a new file v2.0.0+, refused v1.0.0+, read
.duckdb backup v2.0.0+, refused v1.0.0+, read
WAL left without a checkpoint on a 1.5.2 file: read on a new file: read
A 1.5.2 file after a write and CHECKPOINT stays v1.0.0+, read stays v1.0.0+, read
  • build-duckdb.sh checks each slice: it links under -force_load and loads core_functions, json, parquet, icu and autocomplete offline.
  • build-duckdb-ios.sh link-checks both slices without -force_load.
  • Probes on libs-6: check-duckdb-extensions, check-duckdb-access-mode (sqlite_scanner autoinstalls from the v2.0.0 folder), check-duckdb-value-api (36 types) and check-duckdb-storage-version pass. check-sql-lexical-grammar.sh duckdb on the 2.0 CLI: 14 of 14, so the PEG parser lexes as SQLLexicalProfile says.
  • On 2.0 the plugin's measured facts hold: the lock-conflict text, search_path and schema after USE, a nested BEGIN aborting with DUCKDB_ERROR_TRANSACTION, txid_current(), and no SERIAL or identity column.
  • verify.sh ios with the DuckDB suites: 32 of 32, including the new DuckDBDriverStorageVersionTests.
  • check-column-type-corpus.sh duckdb, the real driver on 2.0.0: 352 of 352. check-duckdb-table-listing-parity.sh: PASS. check-duckdb-offline-metadata.sh: every schema query runs with no extension available.
  • verify.sh plugins: PASS, and verify-plugin-loads.sh loads all 43 plugins, DuckDBDriver and ParquetExport among them.
  • verify.sh test on the DuckDB-related suites (NativeDump*, DuckDBDumpStatementTests, DatabaseFileClassifierTests, the statement gates, QueryClassifierLexicalTests, ScopedDriverRoutingTests, PluginMetadataRegistryCuratedCapabilityTests): 161 of 161.
  • verify.sh build: PASS. verify.sh lint on the changed Swift files: 0 violations. shellcheck on the changed scripts: clean.
  • Linux: cargo fmt --check passes. tablepro-net does not compile on macOS (set_tcp_user_timeout), so the driver tests and clippy run in CI's Linux fast checks.

@datlechin
datlechin enabled auto-merge (squash) October 31, 2026 07:54
@datlechin
datlechin merged commit e2227bc into main Oct 31, 2026
38 of 42 checks passed
@datlechin
datlechin deleted the build/libduckdb-2.0 branch October 31, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant