Skip to content

Commit 50b40a1

Browse files
committed
Rebase onto pyiceberg-0.10.0
This commit brings the IMC PyIceberg release up-to-date with the PyIceberg `0.10.0` release while keeping IMC's improvements on top of PyIceberg. commit c1628c2f91cd0f6b867ccb5ba76c0dbe1ffaaec2 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Thu Dec 11 15:49:38 2025 -0600 Allow for `pydantic` versions `>2.12` minus bugged versions commit 11ca959072282cdcee72fb8d6cb595bc49e65b11 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Thu Dec 11 15:40:30 2025 -0600 `pyiceberg-0.10.0` rebase commit 26876b45e3e050a16c853415dcd96bd54cc6a901 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Fri Oct 17 14:52:42 2025 -0500 Add support for `commit.retry.total-timeout-ms` property commit 1b73b77599e795c151b0f8c37f95e64faca5e7a5 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Wed Oct 8 08:39:57 2025 -0500 Update `pydantic` pin to handle broken `2.12` version commit f5354cfb12d73526acc115862c8a7769956ecc06 Author: MoIMC <mohammad.sheikh@imc.com> Date: Mon Sep 15 15:16:35 2025 -0500 Add transaction operations to retry in UpdateTableMetadata commit 25c74bb1fd924c708f96c12649ca73a4de7e2e65 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Thu Sep 4 14:09:53 2025 -0500 Check for _DeleteFiles cached attributes before deleting commit 7ca8d0d4ed3648cdb0965f98276ed13b2ee48cd3 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Mon Sep 1 22:29:11 2025 -0500 Override hive lock waiting logic to immediately fail commit 66f8d02dfd7ebe92a3866ec99199bc4cc2a435f1 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Mon Sep 1 17:59:49 2025 -0500 Add logic to skip hive lock wait if retries are set to `0` commit 81bd44e2adb2b025f251968dab37f4735d92d0cd Author: MoIMC <mohammad.sheikh@imc.com> Date: Wed Aug 27 20:01:43 2025 -0500 Add message for commit retries on failure commit 37fa11e5fc239d2ef54e6c42aa0693263c6ac5a7 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Wed Aug 6 17:24:13 2025 -0500 DAT-4687: fix uncommitted transaction modifying local table state commit 973a3a50c445d95d12b0b541336558eece1f5fd8 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Mon Aug 4 12:30:17 2025 -0500 DAT-4673: update default hive commit configuration commit d68f5bf698f693e1ed77684385fd3fd6fc414e21 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Mon Aug 4 12:26:59 2025 -0500 DAT-4672: improve logging for commits commit 6a0852652ca8a539c79200ebb3a406243e9c4adc Author: Roman Shanin <rshanin@bhft.com> Date: Thu Jun 12 18:22:29 2025 +0300 Correct filter behavior for projected columns When columns are projected from partitions (i.e not in the parquet files) AND these columns are in the row_filter of a scan, the scan returns no result because the filter is resolved to AlwaysFalse Change the code so that projected columns are correctly filtered. commit 846b86a971c95a41c2a19e4c7b1b31bf80859c5e Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Fri Mar 7 07:08:44 2025 -0600 Change deprecated `get_table` to `get_table_req` commit 7a786c7bff980a24a8f72543a2c214e4eb09ec16 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Tue Dec 17 09:54:29 2024 -0600 Add retry logic to `UpdateTableMetadata` commits Add default commit failure cleanup logic Patch transaction commit logic for concurrent appends Calculate bound expression once per scan Code analysis Code analysis 2 Code analysis 3 Bind predicate in delete snapshot usage Linting Fix binding on deletions Simplify query binding reusage Add `bound_filter` kwarg to TableScan Regenerate updates and requirements upon recommit attempt Change `_cleanup_commit_failure` to regular method from abstract commit 0e7a19e19efffd90dc48f12fc69937b34b24f746 Author: Mohammad Sheikh <mohammad.sheikh@imc.com> Date: Thu May 8 10:14:32 2025 -0500 Update CI/CD for GitLab mirroring commit 0774698803371b124c1e5c6aaee3a62525183ea5 Author: koenvo <info@koenvossen.nl> Date: Fri Apr 25 00:24:12 2025 +0200 Fallback for upsert when arrow cannot compare source rows with target rows (#1878) <!-- Fixes https://github.com/apache/iceberg-python/issues/1711 --> Upsert operations in PyIceberg rely on Arrow joins between source and target rows. However, Arrow Acero cannot compare certain complex types — like `struct`, `list`, and `map` — unless they’re part of the join key. When such types exist in non-join columns, the upsert fails with an error like: ```ArrowInvalid: Data type struct<...> is not supported in join non-key field venue_geo``` This PR introduces a **fallback mechanism**: if Arrow fails to join due to unsupported types, we fall back to comparing only the key columns. Non-key complex fields are ignored in the join condition, but still retained in the final upserted data. --- ```python txn.upsert(df, join_cols=["match_id"]) ``` > ❌ ArrowInvalid: Data type struct<...> is not supported in join non-key field venue_geo --- ```python txn.upsert(df, join_cols=["match_id"]) ``` > ✅ Successfully inserts or updates the record, skipping complex field comparison during join --- Yes: - A test was added to reproduce the failure scenario with complex non-key fields. - The new behavior is verified by asserting that the upsert completes successfully using the fallback logic. --- > ℹ️ **Note** > This change does not affect users who do not include complex types in their schemas. For those who do, it improves resilience while preserving data correctness. --- Yes — upserts involving complex non-key columns (like `struct`, `list`, or `map`) no longer fail. They now succeed by skipping unsupported comparisons during the join phase. commit 2d0acf18e4cc7b8b499b59e0836c72b736836356 Author: Kevin Liu <kevinjqliu@users.noreply.github.com> Date: Tue Apr 22 13:12:58 2025 -0400 Fix kerberized hive client (#1941) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> Closes #1744 (second try) First try (#1747) did not fully resolve the issue. See https://github.com/apache/iceberg-python/pull/1747#pullrequestreview-2781186390 yes <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: mnzpk <84433140+mnzpk@users.noreply.github.com> commit 7acdb12f5239594e593f2822de75c7a4c669f52f Author: Fokko Driesprong <fokko@apache.org> Date: Tue Sep 2 10:22:13 2025 +0200 Set the ManifestEntryStatus (#2408) A faulty refactor: https://github.com/apache/iceberg-python/commit/59742e01e8d190aa759e0154b03300bd81a28d02#diff-e9df836e263024ba54e2706853fb25c00269fbfe3726b440ba57f4a929c995dcR927 This produces incorrect metadata, but was hidden by the writer. <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change ## Are these changes tested? ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> commit 38c569b51b909c0b14f07b972a4a4783dbf7868c Author: Kevin Liu <kevinjqliu@users.noreply.github.com> Date: Mon Sep 1 12:47:01 2025 -0700 Release: Add `poetry.lock` to `sdist` (#2405) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change This PR explicitly add `poetry.lock` to the source distribution artifact so that we can build and test RCs with pinned dependencies. We want to pin dependencies because we verify a release candidate (RC) by re-building the project from source distribution. We [unzip the artifact](https://py.iceberg.apache.org/verify-release/#verifying-license-documentation), [install the dependencies with `make install`, and then run tests](https://py.iceberg.apache.org/verify-release/#testing). Since `poetry.lock` is not automatically added to the source distribution, `make install` will install packages only based on `pyproject.toml`. The resolver will pull in the latest version by default, which can lead to compatibility issues. For example, in 0.10 RC2, we see that this pulled in the latest datafusion which caused an compatibility issue with pyiceberg-core. This is due to the breaking change in the FFI boundary, see https://github.com/apache/datafusion-python/issues/1217 Alternatively, we can restrict the upperbound of the dependencies, but that since these are optional dependencies, we want to give users the freedom to choose. ## Are these changes tested? ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit e4849edbbb79df2ebeeb8f4c6ef1cfab8213355a Author: stevie9868 <151791653+stevie9868@users.noreply.github.com> Date: Mon Sep 1 11:19:16 2025 -0700 implement stageOnly Commit (#2269) # Rationale for this change In java, snapshotProducer can create stageOnly snapshot meaning only the snapshot is created but the snapshot is not set to a ref. This is a prerequisite to support wap.id in py-iceberg # Are these changes tested? Yes, tests are added. # Are there any user-facing changes? By default, it will stay with the current existing behavior. --------- Co-authored-by: Yingjian Wu <yingjianw@netflix.com> commit 19ba343a2e16823e79f0c0359411f8bfd792ea6e Author: Tom <mccormickt12@gmail.com> Date: Fri Aug 29 16:04:56 2025 -0400 feat: allow default scheme and netloc for schemeless path (#2291) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change For hdfs it's common to get scheme and netloc from config and have paths be just the uri. This PR adds properties to configure `DEFAULT_SCHEME` and `DEFAULT_NETLOC` For example ``` from pyiceberg.catalog import load_catalog catalog = load_catalog("default", { ..., "DEFAULT_SCHEME": "hdfs", "DEFAULT_NETLOC": "ltx1-yugioh-cluster01.linkfs.prod-ltx1.atd.prod.linkedin.com:9000", } ``` or if not using catalog ``` static_table = StaticTable.from_metadata( "/warehouse/wh/nyc.db/taxis/metadata/00002-6ea51ce3-62aa-4197-9cf8-43d07c3440ca.metadata.json" properties={ 'DEFAULT_SCHEME': 'hdfs', 'DEFAULT_NETLOC': 'ltx1-yugioh-cluster01.linkfs.prod-ltx1.atd.prod.linkedin.com:9000', } ) ``` Previously, schemeless paths are assumed to be for the local filesystem only. This PR allows schemeless paths to be passed to the HDFS Filesystem # Are these changes tested? Tested in test env at linkedin and with unit tests # Are there any user-facing changes? No user facing changes by default. If you add these env variables, if file path doesn't have scheme/netloc it'll use the defaults specified. <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Tom McCormick <tmccormi@linkedin.com> commit 86b0fe8766245fcc0132da82475901aae9f542e8 Author: Stanley Law <19900516+dingo4dev@users.noreply.github.com> Date: Sat Aug 30 01:25:54 2025 +0800 Add `close()` method to RestCatalog (#2403) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> Part of #2399 # Rationale for this change This PR introducs the implementation of `close()` method to the `RestCatalog`. And add corresponding test case which following the test pattern in #2390 ## Are these changes tested? Yes ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit c3c314d439b4def99d4e92b6437df405745e1b3c Author: Manu Zhang <OwenZhang1990@gmail.com> Date: Fri Aug 29 04:27:53 2025 +0800 Fix rest test with ssl_ca_bundle due to conflicts with OS environment (#2394) # Rationale for this change Request sessions could load `REQUESTS_CA_BUNDLE` or `CURL_CA_BUNDLE` from OS environment such that cabundle passed in from the UT is not used. https://github.com/psf/requests/blob/88dce9d854797c05d0ff296b70e0430535ef8aaf/src/requests/sessions.py#L766-L771 ## Are these changes tested? Yes ## Are there any user-facing changes? No --------- Co-authored-by: Fokko Driesprong <fokko@apache.org> commit 52ff6840077758d37228057683f415d8c5a0fd87 Author: Fokko Driesprong <fokko@apache.org> Date: Thu Aug 28 19:45:42 2025 +0200 V3: Fix invalid downcasting for nanos (#2397) # Rationale for this change It looks like we downcast Arrow nanosecond types always to microseconds. cc @sungwy @kevinjqliu ## Are these changes tested? ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit 159b2f37736e2375f76572865742daf8f417fc1a Author: gmweaver <10748110+gmweaver@users.noreply.github.com> Date: Thu Aug 28 10:44:02 2025 -0700 use anon argument when configuring s3fs.S3FileSystem (#2392) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change `s3fs.S3FileSystem` requires that we pass `anon` directly, it cannot be passed through `config_kwargs`. ## Are these changes tested? yes, unit tests updated. ## Are there any user-facing changes? no <!-- In the case of user-facing changes, please add the changelog label. --> commit 7ad70567ad49c592c19e8c31d9c163c4c7c05f1f Author: Kevin Liu <kevinjqliu@users.noreply.github.com> Date: Thu Aug 28 10:26:49 2025 -0700 disable bodo test (#2401) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Relates to #2400 Disables importing bodo to allow us to make changes to PyArrowFileIO in #2291 ## Are these changes tested? ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit 3457bc2628c713d6037a2b08e32acbb899bcd346 Author: Alessandro Nori <alessandro.nori@datadoghq.com> Date: Thu Aug 28 18:35:10 2025 +0200 use PyArrowFileIO as default for abfs and wasb schemes (#2395) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Now that [pyarrow FileIO supports ADLS](https://github.com/apache/iceberg-python/pull/2111), we can update the [SCHEMA_TO_FILE_IO](https://github.com/apache/iceberg-python/blob/370e770383c9c129aff2fe24cc92a68981688159/pyiceberg/io/__init__.py#L307-L319) mapping for `abfs` and `wasb` to use ARROW_FILE_IO, similar to how it’s handled for s3 We’re keeping FsspecFileIO as the preferred default, since the PyArrowFileIO implementation is only available in pyarrow >= 20.0.0 ## Are these changes tested? ## Are there any user-facing changes? Add mapping SCHEMA_TO_FILE_IO for `wasb` and `abfs` to `FsspecFileIO` and `PyArrowFileIO` <!-- In the case of user-facing changes, please add the changelog label. --> commit 3eecdadc000047ec30749fc5d6ce1f2f072a30b2 Author: Kristofer Gaudel <68076186+kris-gaudel@users.noreply.github.com> Date: Thu Aug 28 03:59:27 2025 -0400 Add close option to Catalog (#2390) commit 3c7c279d5a61f724e1e00b960691c509f0c758c7 Author: Kevin Liu <kevinjqliu@users.noreply.github.com> Date: Thu Aug 28 00:51:49 2025 -0700 docs: improve release docs (#2374) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Minor quality of life changes to release docs * Added a copy button to script lines <img width="818" height="515" alt="Screenshot 2025-08-22 at 12 15 46 PM" src="https://github.com/user-attachments/assets/87569d80-6a48-4dcd-b307-bd7b3ff84235" /> * `Apache PyIceberg` -> `PyIceberg`. "Apache PyIceberg" sounds like a top-level, standalone project. I prefer we use use just "PyIceberg" ## Are these changes tested? ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit 370e770383c9c129aff2fe24cc92a68981688159 Author: Manu Zhang <OwenZhang1990@gmail.com> Date: Thu Aug 28 02:45:32 2025 +0800 Fix heading levels of subsections of Code standards (#2393) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Heading levels of subsections of `Code standards` are incorrect such that subsection headings are rendered on the same level as `Code standards`. ## Are these changes tested? Doc changes only. ## Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> commit 06b9467f1db2b71c51d4c1be5f3a5144afa52880 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 12:29:09 2025 -0700 Build: Bump daft from 0.5.19 to 0.5.21 (#2384) Bumps [daft](https://github.com/Eventual-Inc/Daft) from 0.5.19 to 0.5.21. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/Eventual-Inc/Daft/releases">daft's releases</a>.</em></p> <blockquote> <h2>v0.5.21</h2> <h2>What's Changed 🚀</h2> <h2>✨ Features</h2> <ul> <li>feat: Propagate morsel size top-down in swordfish <a href="https://github.com/colin-ho"><code>@​colin-ho</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4894">#4894</a>)</li> <li>feat: DataFrame.write_huggingface <a href="https://github.com/kevinzwang"><code>@​kevinzwang</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5015">#5015</a>)</li> </ul> <h2>🐛 Bug Fixes</h2> <ul> <li>fix(blc): Attempt to fix the broken link checker. <a href="https://github.com/rohitkulshreshtha"><code>@​rohitkulshreshtha</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5010">#5010</a>)</li> <li>fix: Print UDF stdout and Daft logs above the progress bar <a href="https://github.com/srilman"><code>@​srilman</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4861">#4861</a>)</li> </ul> <h2>📖 Documentation</h2> <ul> <li>docs: Add audio transcription example card <a href="https://github.com/desmondcheongzx"><code>@​desmondcheongzx</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5020">#5020</a>)</li> <li>docs: improve audio transcription example <a href="https://github.com/universalmind303"><code>@​universalmind303</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4990">#4990</a>)</li> <li>docs: Spice up the examples page <a href="https://github.com/desmondcheongzx"><code>@​desmondcheongzx</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5019">#5019</a>)</li> </ul> <h2>🔧 Maintenance</h2> <ul> <li>chore: Only test MacOS on Python 3.11 in CI <a href="https://github.com/srilman"><code>@​srilman</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5014">#5014</a>)</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/Eventual-Inc/Daft/compare/v0.5.20...v0.5.21">https://github.com/Eventual-Inc/Daft/compare/v0.5.20...v0.5.21</a></p> <h2>v0.5.20</h2> <h2>What's Changed 🚀</h2> <h2>💥 Breaking Changes</h2> <ul> <li>feat!: RowWiseUdf.eval for eager evaluation <a href="https://github.com/kevinzwang"><code>@​kevinzwang</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4998">#4998</a>)</li> </ul> <h2>✨ Features</h2> <ul> <li>feat: support count(1) in dataframe and choose the cheap column <a href="https://github.com/huleilei"><code>@​huleilei</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4977">#4977</a>)</li> <li>feat: add clickhouse data sink <a href="https://github.com/huleilei"><code>@​huleilei</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4850">#4850</a>)</li> <li>feat: implement distributed sort in flotilla engine <a href="https://github.com/ohbh"><code>@​ohbh</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4991">#4991</a>)</li> <li>feat!: RowWiseUdf.eval for eager evaluation <a href="https://github.com/kevinzwang"><code>@​kevinzwang</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4998">#4998</a>)</li> <li>feat: basic read_huggingface functionality <a href="https://github.com/kevinzwang"><code>@​kevinzwang</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4996">#4996</a>)</li> <li>feat: support using max() and min() on list of boolean values <a href="https://github.com/varun117"><code>@​varun117</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4989">#4989</a>)</li> <li>feat: Flotilla pre-shuffle merge <a href="https://github.com/colin-ho"><code>@​colin-ho</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4873">#4873</a>)</li> <li>feat: Flotilla into partitions <a href="https://github.com/colin-ho"><code>@​colin-ho</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4963">#4963</a>)</li> <li>feat(optimizer): Add Lance count() pushdown optimization <a href="https://github.com/huleilei"><code>@​huleilei</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4969">#4969</a>)</li> <li>feat: adds video frame streaming source <a href="https://github.com/rchowell"><code>@​rchowell</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4979">#4979</a>)</li> <li>feat: Add offset support to Spark Connect <a href="https://github.com/plotor"><code>@​plotor</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4962">#4962</a>)</li> <li>feat: new <code>daft.File</code> datatype <a href="https://github.com/universalmind303"><code>@​universalmind303</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4959">#4959</a>)</li> <li>feat: unify all Daft type to Python type conversions <a href="https://github.com/kevinzwang"><code>@​kevinzwang</code></a> (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4972">#4972</a>)</li> </ul> <h2>🐛 Bug Fixes</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Eventual-Inc/Daft/commit/9017c8fcb3f0a1876dd7afcf0a7135a4bcb400b3"><code>9017c8f</code></a> docs: Add audio transcription example card (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5020">#5020</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/dec381e2f3f247ea6b461a844b607d2d83855113"><code>dec381e</code></a> feat: Propagate morsel size top-down in swordfish (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4894">#4894</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/a8b1556c7ddbb3060c180aa3f3757e25d58e929c"><code>a8b1556</code></a> fix(blc): Attempt to fix the broken link checker. (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5010">#5010</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/543e5e56d26941ddaf9023e4d0ca02dd0a0de03c"><code>543e5e5</code></a> docs: improve audio transcription example (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4990">#4990</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/69afe66758e1550b09b08341a95de58577da96fc"><code>69afe66</code></a> docs: Spice up the examples page (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5019">#5019</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/daeb3b6ae220b681d23ffa6d9ca32e8b26b9947e"><code>daeb3b6</code></a> feat: DataFrame.write_huggingface (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5015">#5015</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/6be67bf6aa82ab5c0cc301e0e431a78fde4ccc53"><code>6be67bf</code></a> chore: Only test MacOS on Python 3.11 in CI (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5014">#5014</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/0fb39cf1277cc8de3061b04b5eb5f4cc2f82be35"><code>0fb39cf</code></a> fix: Print UDF stdout and Daft logs above the progress bar (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4861">#4861</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/0bcaf35abf50c2d590e38e6ec673ff5e8fdbe0bc"><code>0bcaf35</code></a> feat: support count(1) in dataframe and choose the cheap column (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/4977">#4977</a>)</li> <li><a href="https://github.com/Eventual-Inc/Daft/commit/82d1f4fb90af1b39a0c343201b03a4ec83c425c9"><code>82d1f4f</code></a> fix: Can translate sort in flotilla (<a href="https://redirect.github.com/Eventual-Inc/Daft/issues/5005">#5005</a>)</li> <li>Additional commits viewable in <a href="https://github.com/Eventual-Inc/Daft/compare/v0.5.19...v0.5.21">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=daft&package-manager=pip&previous-version=0.5.19&new-version=0.5.21)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 9ef5a476c9d1774f201be92d20ead85e2a211ee4 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:46:46 2025 -0700 Build: Bump mypy-boto3-dynamodb from 1.40.10 to 1.40.14 (#2385) Bumps [mypy-boto3-dynamodb](https://github.com/youtype/mypy_boto3_builder) from 1.40.10 to 1.40.14. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/youtype/mypy_boto3_builder/releases">mypy-boto3-dynamodb's releases</a>.</em></p> <blockquote> <h2>8.8.0 - Python 3.8 runtime is back</h2> <h3>Changed</h3> <ul> <li><code>[services]</code> <code>install_requires</code> section is calculated based on dependencies in use, so <code>typing-extensions</code> version is set properly</li> <li><code>[all]</code> Replaced <code>typing</code> imports with <code>collections.abc</code> with a fallback to <code>typing</code> for Python &lt;3.9</li> <li><code>[all]</code> Added aliases for <code>builtins.list</code>, <code>builtins.set</code>, <code>builtins.dict</code>, and <code>builtins.type</code>, so Python 3.8 runtime should work as expected again (reported by <a href="https://github.com/YHallouard"><code>@​YHallouard</code></a> in <a href="https://redirect.github.com/youtype/mypy_boto3_builder/issues/340">#340</a> and <a href="https://github.com/Omri-Ben-Yair"><code>@​Omri-Ben-Yair</code></a> in <a href="https://redirect.github.com/youtype/mypy_boto3_builder/issues/336">#336</a>)</li> <li><code>[all]</code> Unions use the same type annotations as the rest of the structures due to proper fallbacks</li> </ul> <h3>Fixed</h3> <ul> <li><code>[services]</code> Universal input/output shapes were not replaced properly in service subresources</li> <li><code>[docs]</code> Simplified doc links rendering for services</li> <li><code>[services]</code> Cleaned up unnecessary imports in <code>client.pyi</code></li> <li><code>[builder]</code> Import records with fallback are always rendered</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/youtype/mypy_boto3_builder/commits">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mypy-boto3-dynamodb&package-manager=pip&previous-version=1.40.10&new-version=1.40.14)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 76455ac75312a88fedd1696b3e779e2d232bbaf6 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:44:07 2025 -0700 Build: Bump coverage from 7.10.4 to 7.10.5 (#2389) Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.10.4 to 7.10.5. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst">coverage's changelog</a>.</em></p> <blockquote> <h2>Version 7.10.5 — 2025-08-23</h2> <ul> <li>Big speed improvements for <code>coverage combine</code>: it's now about twice as fast! Huge thanks to Alex Gaynor for pull requests <code>2032 &lt;pull 2032_&gt;</code><em>, <code>2033 &lt;pull 2033_&gt;</code></em>, and <code>2034 &lt;pull 2034_&gt;</code>_.</li> </ul> <p>.. _pull 2032: <a href="https://redirect.github.com/nedbat/coveragepy/pull/2032">nedbat/coveragepy#2032</a> .. _pull 2033: <a href="https://redirect.github.com/nedbat/coveragepy/pull/2033">nedbat/coveragepy#2033</a> .. _pull 2034: <a href="https://redirect.github.com/nedbat/coveragepy/pull/2034">nedbat/coveragepy#2034</a></p> <p>.. _changes_7-10-4:</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/nedbat/coveragepy/commit/810abeb149fff925e49eab37ceaee60cb2878f7d"><code>810abeb</code></a> chore: make upgrade</li> <li><a href="https://github.com/nedbat/coveragepy/commit/3c8f1b518cf933040059fd85b482827ed205e0d1"><code>3c8f1b5</code></a> build: use --universal to keep platform conditions in .pip files</li> <li><a href="https://github.com/nedbat/coveragepy/commit/107ae05233005f8a5d497d8492c6410428dbbc28"><code>107ae05</code></a> docs: sample HTML for 7.10.5</li> <li><a href="https://github.com/nedbat/coveragepy/commit/b5bc6d409aefc11d0277fbf8af9c930bea686ed5"><code>b5bc6d4</code></a> docs: prep for 7.10.5</li> <li><a href="https://github.com/nedbat/coveragepy/commit/a5c18cc5eadc9df04005420207d53d74a94b9adf"><code>a5c18cc</code></a> style: auto-generated changes shouldn't trigger ruff re-formatting</li> <li><a href="https://github.com/nedbat/coveragepy/commit/1f9f84079aef5aab496381502aa818ea0574982a"><code>1f9f840</code></a> build: tooling for ruff formatting</li> <li><a href="https://github.com/nedbat/coveragepy/commit/9ee5b3e9e053168caa4e7bd32389b5592ad9b503"><code>9ee5b3e</code></a> chore: make upgrade</li> <li><a href="https://github.com/nedbat/coveragepy/commit/bfeb2aec1de7c298060b0c972f86d0f1f3e4da71"><code>bfeb2ae</code></a> style: fix things so pylint is happy with ruff</li> <li><a href="https://github.com/nedbat/coveragepy/commit/82467f72306efdb207af09ace27b6b3ed4c7ad6f"><code>82467f7</code></a> chore: <code>ruff format .</code></li> <li><a href="https://github.com/nedbat/coveragepy/commit/0a7b73346bc498a089df8cc7d4168dab7a326a06"><code>0a7b733</code></a> refactor: remove unused things from lab/</li> <li>Additional commits viewable in <a href="https://github.com/nedbat/coveragepy/compare/7.10.4...7.10.5">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=coverage&package-manager=pip&previous-version=7.10.4&new-version=7.10.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit c48d3ee6a6df556c33203da88d98797394fca68a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:43:59 2025 -0700 Build: Bump pandas from 2.3.1 to 2.3.2 (#2383) Bumps [pandas](https://github.com/pandas-dev/pandas) from 2.3.1 to 2.3.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pandas-dev/pandas/releases">pandas's releases</a>.</em></p> <blockquote> <h2>Pandas 2.3.2</h2> <p>We are pleased to announce the release of pandas 2.3.2. This release includes some improvements and fixes to the future string data type (preview feature for the upcoming pandas 3.0). We recommend that all users upgrade to this version.</p> <p>See the <a href="https://pandas.pydata.org/pandas-docs/version/2.3/whatsnew/v2.3.2.html">full whatsnew</a> for a list of all the changes. Pandas 2.3.2 supports Python 3.9 and higher.</p> <p>The release will be available on the conda-forge channel:</p> <pre><code>conda install pandas --channel conda-forge </code></pre> <p>Or via PyPI:</p> <pre><code>python3 -m pip install --upgrade pandas </code></pre> <p>Please report any issues with the release on the <a href="https://github.com/pandas-dev/pandas/issues">pandas issue tracker</a>.</p> <p>Thanks to all the contributors who made this release possible.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pandas-dev/pandas/commit/4665c10899bc413b639194f6fb8665a5c70f7db5"><code>4665c10</code></a> RLS: 2.3.2</li> <li><a href="https://github.com/pandas-dev/pandas/commit/633c68b91808441eba7d73b630a82b713ba19da8"><code>633c68b</code></a> DOC: fix syntax in whatsnew file</li> <li><a href="https://github.com/pandas-dev/pandas/commit/456ad4702cb017aab36e2d09a4029b1b085198f5"><code>456ad47</code></a> Backport PR <a href="https://redirect.github.com/pandas-dev/pandas/issues/62152">#62152</a> on branch 2.3.x (DOC: prepare 2.3.2 whatsnew notes for rel...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/6cae6444444ed7943ceffbd5990cd8249dd5b47c"><code>6cae644</code></a> [backport 2.3.x] DOC: move and reword whatsnew note for replace fix (<a href="https://redirect.github.com/pandas-dev/pandas/issues/57865">GH-57865</a>...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/a91c50a58c372cddf104fd60241c7284bbf8fc80"><code>a91c50a</code></a> Backport PR <a href="https://redirect.github.com/pandas-dev/pandas/issues/62147">#62147</a> on branch 2.3.x (DOC: correct and rewrite string migration...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/f7a2cfdf2ed08016167ae6c5e577c54949a849b2"><code>f7a2cfd</code></a> [backport 2.3.x] BUG/DEPR: logical operation with bool and string (<a href="https://redirect.github.com/pandas-dev/pandas/issues/61995">#61995</a>) (#...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/7981a43db4339826a3ce5020e9b1b931e604e5bd"><code>7981a43</code></a> Backport PR <a href="https://redirect.github.com/pandas-dev/pandas/issues/62124">#62124</a> on branch 2.3.x (CI/BLD: don't use strict xfail for '%m.%Y...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/fafbcbd03d1721830f4df8255abdab341db2a583"><code>fafbcbd</code></a> [backport 2.3.x] BUG(CoW): also raise for chained assignment for .at / .iat (...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/3ac64a714886d917ca24ba637a85c4aaf563fc1b"><code>3ac64a7</code></a> [backport 2.3.x] BUG: Fix Series.str.contains with compiled regex on Arrow st...</li> <li><a href="https://github.com/pandas-dev/pandas/commit/1f2dc4fd8a2503773b6f90cb5e8cc5c3172c3c11"><code>1f2dc4f</code></a> [backport 2.3.x] BUG: fix Series.str.fullmatch() and Series.str.match() with ...</li> <li>Additional commits viewable in <a href="https://github.com/pandas-dev/pandas/compare/v2.3.1...v2.3.2">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pandas&package-manager=pip&previous-version=2.3.1&new-version=2.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b94108ac5bb53e5bb428fc434f8224d9352c22ca Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:43:45 2025 -0700 Build: Bump mkdocs-material from 9.6.17 to 9.6.18 (#2382) Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.17 to 9.6.18. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/squidfunk/mkdocs-material/releases">mkdocs-material's releases</a>.</em></p> <blockquote> <h2>mkdocs-material-9.6.18</h2> <ul> <li>Updated Azerbaijani translations</li> <li>Fixed last compat issues with <a href="https://github.com/mitsuhiko/minijinja">minijinja</a>, now 100% compatible</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG">mkdocs-material's changelog</a>.</em></p> <blockquote> <p>mkdocs-material-9.6.18 (2025-08-22)</p> <ul> <li>Updated Azerbaijani translations</li> <li>Fixed last compat issues with [minijinja], now 100% compatible</li> </ul> <p>mkdocs-material-9.6.17 (2025-08-15)</p> <ul> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8396">#8396</a>: Videos do not autoplay when inside a content tab</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8394">#8394</a>: Stroke width not effective in Mermaid.js diagrams</li> <li>Fixed disappearing version selector when hiding page title</li> </ul> <p>mkdocs-material-9.6.16 (2025-07-26)</p> <ul> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8349">#8349</a>: Info plugin doesn't correctly detect virtualenv in some cases</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8334">#8334</a>: Find-in-page detects matches in hidden search result list</li> </ul> <p>mkdocs-material-9.6.15 (2025-07-01)</p> <ul> <li>Updated Mongolian translations</li> <li>Improved semantic markup of &quot;edit this page&quot; button</li> <li>Improved info plugin virtual environment resolution</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8291">#8291</a>: Large font size setting throws of breakpoints in JavaScript</li> </ul> <p>mkdocs-material-9.6.14 (2025-05-13)</p> <ul> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8215">#8215</a>: Social plugin crashes when CairoSVG is updated to 2.8</li> </ul> <p>mkdocs-material-9.6.13 (2025-05-10)</p> <ul> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8204">#8204</a>: Annotations showing list markers in print view</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8153">#8153</a>: Improve style of cardinality symbols in Mermaid.js ER diagrams</li> </ul> <p>mkdocs-material-9.6.12 (2025-04-17)</p> <ul> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8158">#8158</a>: Flip footnote back reference icon for right-to-left languages</li> </ul> <p>mkdocs-material-9.6.11 (2025-04-01)</p> <ul> <li>Updated Docker image to latest Alpine Linux</li> <li>Bump required Jinja version to 3.1</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8133">#8133</a>: Jinja filter <code>items</code> not available (9.6.10 regression)</li> <li>Fixed <a href="https://redirect.github.com/squidfunk/mkdocs-material/issues/8128">#8128</a>: Search plugin not entirely disabled via enabled setting</li> </ul> <p>mkdocs-material-9.6.10 (2025-03-30)</p> <p>This version is a pure refactoring release, and does not contain new features or bug fixes. It strives to improve the compatibility of our templates with alternative Jinja-like template engines that we're currently exploring, including minijinja.</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/51bc7ba063d3e97f9e7bf1e7acfd107e32ac92b7"><code>51bc7ba</code></a> Prepare 9.6.18 release</li> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/fe4a4314d3c39274de8dd3bb8ab1120520d839cc"><code>fe4a431</code></a> Updated dependencies</li> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/3b3c632c4e1308ebc597290192f0f435ca6e3a6c"><code>3b3c632</code></a> Updated Azerbaijani translations</li> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/26d6e5e3828172bc11502d73f1ccfc8760ea0588"><code>26d6e5e</code></a> Improved compatibility with minijinja</li> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/c79364a7e0af950022cb4d2c0528e11beaccdbf8"><code>c79364a</code></a> Fixed last minijinja compat issues in templates</li> <li><a href="https://github.com/squidfunk/mkdocs-material/commit/c89a66bf39bb7f472e925753367ff8e464e0d683"><code>c89a66b</code></a> Updated changelog</li> <li>See full diff in <a href="https://github.com/squidfunk/mkdocs-material/compare/9.6.17...9.6.18">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mkdocs-material&package-manager=pip&previous-version=9.6.17&new-version=9.6.18)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit d0d02a8e3d9b84b6090f0d8c136fd07b459bc431 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:16:53 2025 -0700 Build: Bump mypy-boto3-glue from 1.40.11 to 1.40.15 (#2387) Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.40.11 to 1.40.15. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/youtype/mypy_boto3_builder/releases">mypy-boto3-glue's releases</a>.</em></p> <blockquote> <h2>8.8.0 - Python 3.8 runtime is back</h2> <h3>Changed</h3> <ul> <li><code>[services]</code> <code>install_requires</code> section is calculated based on dependencies in use, so <code>typing-extensions</code> version is set properly</li> <li><code>[all]</code> Replaced <code>typing</code> imports with <code>collections.abc</code> with a fallback to <code>typing</code> for Python &lt;3.9</li> <li><code>[all]</code> Added aliases for <code>builtins.list</code>, <code>builtins.set</code>, <code>builtins.dict</code>, and <code>builtins.type</code>, so Python 3.8 runtime should work as expected again (reported by <a href="https://github.com/YHallouard"><code>@​YHallouard</code></a> in <a href="https://redirect.github.com/youtype/mypy_boto3_builder/issues/340">#340</a> and <a href="https://github.com/Omri-Ben-Yair"><code>@​Omri-Ben-Yair</code></a> in <a href="https://redirect.github.com/youtype/mypy_boto3_builder/issues/336">#336</a>)</li> <li><code>[all]</code> Unions use the same type annotations as the rest of the structures due to proper fallbacks</li> </ul> <h3>Fixed</h3> <ul> <li><code>[services]</code> Universal input/output shapes were not replaced properly in service subresources</li> <li><code>[docs]</code> Simplified doc links rendering for services</li> <li><code>[services]</code> Cleaned up unnecessary imports in <code>client.pyi</code></li> <li><code>[builder]</code> Import records with fallback are always rendered</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/youtype/mypy_boto3_builder/commits">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mypy-boto3-glue&package-manager=pip&previous-version=1.40.11&new-version=1.40.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit fec38e7aa22f2eb7edd67d9ded6929c1aa988af2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 11:06:31 2025 -0700 Build: Bump typing-extensions from 4.14.1 to 4.15.0 (#2388) Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.14.1 to 4.15.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/python/typing_extensions/releases">typing-extensions's releases</a>.</em></p> <blockquote> <h2>4.15.0</h2> <p>No user-facing changes since 4.15.0rc1.</p> <p>New features since 4.14.1:</p> <ul> <li>Add the <code>@typing_extensions.disjoint_base</code> decorator, as specified in PEP 800. Patch by Jelle Zijlstra.</li> <li>Add <code>typing_extensions.type_repr</code>, a backport of <a href="https://docs.python.org/3.14/library/annotationlib.html#annotationlib.type_repr"><code>annotationlib.type_repr</code></a>, introduced in Python 3.14 (CPython PR <a href="https://redirect.github.com/python/cpython/pull/124551">#124551</a>, originally by Jelle Zijlstra). Patch by Semyon Moroz.</li> <li>Fix behavior of type params in <code>typing_extensions.evaluate_forward_ref</code>. Backport of CPython PR <a href="https://redirect.github.com/python/cpython/pull/137227">#137227</a> by Jelle Zijlstra.</li> </ul> <h2>4.15.0rc1</h2> <ul> <li>Add the <code>@typing_extensions.disjoint_base</code> decorator, as specified in PEP 800. Patch by Jelle Zijlstra.</li> <li>Add <code>typing_extensions.type_repr</code>, a backport of <a href="https://docs.python.org/3.14/library/annotationlib.html#annotationlib.type_repr"><code>annotationlib.type_repr</code></a>, introduced in Python 3.14 (CPython PR <a href="https://redirect.github.com/python/cpython/pull/124551">#124551</a>, originally by Jelle Zijlstra). Patch by Semyon Moroz.</li> <li>Fix behavior of type params in <code>typing_extensions.evaluate_forward_ref</code>. Backport of CPython PR <a href="https://redirect.github.com/python/cpython/pull/137227">#137227</a> by Jelle Zijlstra.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/python/typing_extensions/blob/main/CHANGELOG.md">typing-extensions's changelog</a>.</em></p> <blockquote> <h1>Release 4.15.0 (August 25, 2025)</h1> <p>No user-facing changes since 4.15.0rc1.</p> <h1>Release 4.15.0rc1 (August 18, 2025)</h1> <ul> <li>Add the <code>@typing_extensions.disjoint_base</code> decorator, as specified in PEP 800. Patch by Jelle Zijlstra.</li> <li>Add <code>typing_extensions.type_repr</code>, a backport of <a href="https://docs.python.org/3.14/library/annotationlib.html#annotationlib.type_repr"><code>annotationlib.type_repr</code></a>, introduced in Python 3.14 (CPython PR <a href="https://redirect.github.com/python/cpython/pull/124551">#124551</a>, originally by Jelle Zijlstra). Patch by Semyon Moroz.</li> <li>Fix behavior of type params in <code>typing_extensions.evaluate_forward_ref</code>. Backport of CPython PR <a href="https://redirect.github.com/python/cpython/pull/137227">#137227</a> by Jelle Zijlstra.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/python/typing_extensions/commit/9d1637e264b5c1a6b7acee3e907015f89b20c2c9"><code>9d1637e</code></a> Prepare release 4.15.0 (<a href="https://redirect.github.com/python/typing_extensions/issues/658">#658</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/4bd67c5be5d9443c7d33c314d02a56ee125eb88d"><code>4bd67c5</code></a> Coverage: exclude some noise (<a href="https://redirect.github.com/python/typing_extensions/issues/656">#656</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/e589a26da73b075c5276bae40b86db1af0144f84"><code>e589a26</code></a> Coverage: add detailed report to job summary (<a href="https://redirect.github.com/python/typing_extensions/issues/655">#655</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/67d37fed1298e050f74d5acc95b2621bd37837ad"><code>67d37fe</code></a> Coverage: Implement fail_under (<a href="https://redirect.github.com/python/typing_extensions/issues/654">#654</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/e9ae26f5286edee9262727755ecb9ad16e999192"><code>e9ae26f</code></a> Don't delete previous coverage comment (<a href="https://redirect.github.com/python/typing_extensions/issues/653">#653</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/ac80bb728a3006fc88ef7373b92f0c25cfcc7895"><code>ac80bb7</code></a> Add Coverage workflow (<a href="https://redirect.github.com/python/typing_extensions/issues/623">#623</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/abaaafd98c1cc7e5baf098ec287a3d22cb339670"><code>abaaafd</code></a> Prepare release 4.15.0rc1 (<a href="https://redirect.github.com/python/typing_extensions/issues/650">#650</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/98104053ea8d49bcdd247804e5fa9f73136acbd4"><code>9810405</code></a> Add <code>@disjoint_base</code> (PEP 800) (<a href="https://redirect.github.com/python/typing_extensions/issues/634">#634</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/7ee9e05fd484d06899ce56e80f5e1aa4c760fc03"><code>7ee9e05</code></a> Backport type_params fix from CPython (<a href="https://redirect.github.com/python/typing_extensions/issues/646">#646</a>)</li> <li><a href="https://github.com/python/typing_extensions/commit/1e8eb9c06ef51b3a1e1f05303a16feca13f5ed98"><code>1e8eb9c</code></a> Do not refer to PEP 705 as being experimental (<a href="https://redirect.github.com/python/typing_extensions/issues/648">#648</a>)</li> <li>Additional commits viewable in <a href="https://github.com/python/typing_extensions/compare/4.14.1...4.15.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typing-extensions&package-manager=pip&previous-version=4.14.1&new-version=…
1 parent 6f7c21a commit 50b40a1

155 files changed

Lines changed: 15105 additions & 4648 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.asf.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
2222

2323
github:
24-
description: "Apache PyIceberg"
24+
description: "PyIceberg"
2525
homepage: https://py.iceberg.apache.org/
2626
labels:
2727
- iceberg

.github/ISSUE_TEMPLATE/iceberg_bug_report.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,9 @@ body:
2828
description: What Apache Iceberg version are you using?
2929
multiple: false
3030
options:
31-
- "0.8.1 (latest release)"
31+
- "0.9.1 (latest release)"
32+
- "0.9.0"
33+
- "0.8.1"
3234
- "0.8.0"
3335
- "0.7.1"
3436
- "0.7.0"

.github/dependabot.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@ updates:
2222
- package-ecosystem: "pip"
2323
directory: "/"
2424
schedule:
25-
interval: "daily"
25+
interval: "weekly"
2626
open-pull-requests-limit: 50
2727
- package-ecosystem: "github-actions"
2828
directory: "/"
2929
schedule:
30-
interval: "daily"
30+
interval: "weekly"

.github/pull_request_template.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
<!--
2+
Thanks for opening a pull request!
3+
-->
4+
5+
<!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
6+
<!-- Closes #${GITHUB_ISSUE_ID} -->
7+
8+
# Rationale for this change
9+
10+
## Are these changes tested?
11+
12+
## Are there any user-facing changes?
13+
14+
<!-- In the case of user-facing changes, please add the changelog label. -->

.github/workflows/check-md-link.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,4 +36,4 @@ jobs:
3636
runs-on: ubuntu-latest
3737
steps:
3838
- uses: actions/checkout@master
39-
- uses: gaurav-nelson/github-action-markdown-link-check@v1
39+
- uses: tcort/github-action-markdown-link-check@v1

.github/workflows/license_check.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,5 @@ jobs:
2424
rat:
2525
runs-on: ubuntu-22.04
2626
steps:
27-
- uses: actions/checkout@v4
27+
- uses: actions/checkout@v5
2828
- run: dev/check-license

.github/workflows/nightly-pypi-build.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
outputs:
3232
VERSION: ${{ steps.set-version.outputs.VERSION }}
3333
steps:
34-
- uses: actions/checkout@v4
34+
- uses: actions/checkout@v5
3535
with:
3636
fetch-depth: 1
3737

@@ -71,7 +71,7 @@ jobs:
7171

7272
steps:
7373
- name: Download all the artifacts
74-
uses: actions/download-artifact@v4
74+
uses: actions/download-artifact@v5
7575
with:
7676
merge-multiple: true
7777
path: dist/

.github/workflows/pypi-build-artifacts.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
os: [ ubuntu-22.04, windows-2022, macos-13, macos-14 ]
3636

3737
steps:
38-
- uses: actions/checkout@v4
38+
- uses: actions/checkout@v5
3939
with:
4040
fetch-depth: 1
4141

@@ -62,7 +62,7 @@ jobs:
6262
if: startsWith(matrix.os, 'ubuntu')
6363

6464
- name: Build wheels
65-
uses: pypa/cibuildwheel@v2.22.0
65+
uses: pypa/cibuildwheel@v3.1.4
6666
with:
6767
output-dir: wheelhouse
6868
config-file: "pyproject.toml"

.github/workflows/python-ci-docs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
runs-on: ubuntu-22.04
3636

3737
steps:
38-
- uses: actions/checkout@v4
38+
- uses: actions/checkout@v5
3939
- name: Install poetry
4040
run: make install-poetry
4141
- uses: actions/setup-python@v5

.github/workflows/python-ci.yml

Lines changed: 46 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,19 +50,59 @@ jobs:
5050
python: ['3.9', '3.10', '3.11', '3.12']
5151

5252
steps:
53-
- uses: actions/checkout@v4
53+
- uses: actions/checkout@v5
5454
- name: Install poetry
5555
run: make install-poetry
5656
- uses: actions/setup-python@v5
5757
with:
5858
python-version: ${{ matrix.python }}
59-
cache: poetry
60-
cache-dependency-path: ./poetry.lock
6159
- name: Install system dependencies
6260
run: sudo apt-get update && sudo apt-get install -y libkrb5-dev # for kerberos
6361
- name: Install
6462
run: make install-dependencies
65-
- name: Linters
63+
- name: Run linters
6664
run: make lint
67-
- name: Tests
68-
run: make test-coverage
65+
- name: Run unit tests with coverage
66+
run: COVERAGE=1 make test
67+
- name: Generate coverage report (85%) # Coverage threshold should only increase over time — never decrease it!
68+
run: COVERAGE_FAIL_UNDER=85 make coverage-report
69+
70+
integration-test:
71+
runs-on: ubuntu-22.04
72+
strategy:
73+
matrix:
74+
python: ['3.9', '3.10', '3.11', '3.12']
75+
76+
steps:
77+
- uses: actions/checkout@v5
78+
- name: Install system dependencies
79+
run: sudo apt-get update && sudo apt-get install -y libkrb5-dev # for kerberos
80+
- name: Install
81+
run: make install
82+
83+
- name: Run integration tests with coverage
84+
run: COVERAGE=1 make test-integration
85+
- name: Show debug logs
86+
if: ${{ failure() }}
87+
run: docker compose -f dev/docker-compose.yml logs
88+
89+
- name: Run s3 integration tests with coverage
90+
run: COVERAGE=1 make test-s3
91+
- name: Show debug logs
92+
if: ${{ failure() }}
93+
run: docker compose -f dev/docker-compose.yml logs
94+
95+
- name: Run adls integration tests with coverage
96+
run: COVERAGE=1 make test-adls
97+
- name: Show debug logs
98+
if: ${{ failure() }}
99+
run: docker compose -f dev/docker-compose-azurite.yml logs
100+
101+
- name: Run gcs integration tests with coverage
102+
run: COVERAGE=1 make test-gcs
103+
- name: Show debug logs
104+
if: ${{ failure() }}
105+
run: docker compose -f dev/docker-compose-gcs-server.yml logs
106+
107+
- name: Generate coverage report (75%) # Coverage threshold should only increase over time — never decrease it!
108+
run: COVERAGE_FAIL_UNDER=75 make coverage-report

0 commit comments

Comments
 (0)