Fix #1116: Drop the numpy constraints and make pyarrow a single range - #1117
Open
javihern98 wants to merge 3 commits into
Open
Fix #1116: Drop the numpy constraints and make pyarrow a single range#1117javihern98 wants to merge 3 commits into
javihern98 wants to merge 3 commits into
Conversation
javihern98
marked this pull request as ready for review
September 9, 2026 11:50
javihern98
enabled auto-merge
September 9, 2026 11:51
pyarrow becomes one unmarked range (>=14.0,<26.0): 24.0.0 covers Python 3.10-3.14 with wheels and requires-python handles 3.9 (-> 21.0.0), so the three Python-bounded lines bought nothing. The cap moves from <25.0 to <26.0 so the 25.x line (25.0.1, Python >= 3.10) is reachable; the Ubuntu 24.04 job's hand-maintained pyarrow==24.0.0 stays inside the range. numpy leaves [project].dependencies altogether: nothing under src/ or tests/ imports it, so it is pandas' dependency, not ours, and pandas already pins what it needs per Python. Fresh installs resolve exactly as before (3.9 -> 2.0.2, 3.10 -> 2.2.6, 3.11 -> 2.4.6, 3.12+ -> 2.5.3); the only effects are that pip no longer forces numpy>=2.0.2 onto older stacks (the Ubuntu 24.04 job already runs the APT numpy 1.26.4 with pandas 2.1.4) and that numpy 2.5.x becomes reachable on 3.12+. The old marker split never protected the lock from Dependabot's collapse either: Dependabot rewrites poetry.lock in violation of the pyproject markers and `poetry install` trusts the lock (PR #876, superseded by #885). The dev group loses the python_version < '3.10' pandas-stubs line (the Poetry environment starts at 3.10, so it never resolved) and the redundant marker on the remaining one. poetry.lock only changes its content-hash here; the next commit regenerates it from scratch, after which numpy is partitioned the way pip resolves it -- 2.2.6 (3.10), 2.4.6 (3.11), 2.5.3 (>=3.12).
A clean `poetry lock --regenerate` against the committed constraints, so transitive pins stop drifting (Dependabot only touches the packages it bumps): anyio 4.14.2 -> 4.15.1, ast-serialize 0.8.0 -> 0.11.0, duckdb 1.4.5 -> 1.5.5, lxml 6.1.2 -> 6.1.3, pyarrow 24.0.0 -> 25.0.1, coverage 7.15.4 -> 7.16.0, ruff 0.16.5 -> 0.16.6. numpy, now transitive, is partitioned the way pip resolves it: 2.2.6 (3.10), 2.4.6 (3.11), 2.5.3 (>=3.12). The Ubuntu 24.04 workflow keeps its hand-maintained pins: they only need to stay inside the pyproject ranges, and pyarrow==24.0.0 still does.
The patched Pyodide lockfile mirrors vtlengine's declared dependencies, which no longer include numpy. Nothing changes at runtime: the distribution's pandas and pyarrow entries both depend on numpy, so it is loaded with them. pysdmx keeps its numpy entry because pysdmx does declare numpy (under its data extra).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1116.
Three commits. The first replaces six numpy/pyarrow constraint lines with three different Python boundaries (3.10, 3.11, 3.14) by a single one; the second regenerates
poetry.lockfrom scratch; the third drops the now-stale numpy line from the JupyterLite lock patch.<25.0to<26.0so the 25.x line (25.0.1, Python >= 3.10) is reachable.src/ortests/imports it: it is pandas' dependency, and pandas already pins what it needs per Python (>=1.26.0below 3.14,>=2.3.3from 3.14). Declaring it bought nothing — fresh installs resolve identically without it — and the marker split never protected the lock from the Dependabot collapse it was meant to guard against: Dependabot rewritespoetry.lockin violation of the pyproject markers andpoetry installtrusts the lock (PR Bump coverage from 7.14.3 to 7.15.0 #876, superseded by Update coverage to 7.15.0 (supersedes #876) #885). Reviewing Dependabot lock diffs remains the actual protection.python_version < '3.10'pandas-stubs line goes (the Poetry environment starts at 3.10, so it never resolved) and the remaining line loses its redundant>= '3.10'marker.Resolution per interpreter
Verified with
uv pip compile pyproject.toml --python-version <X>:What changes for users — everything else resolves identically:
<2.5).pyarrow>=22.0floor any more; pip still resolves the highest in range.Lock
The first commit only moves the lock's content-hash. The second is a clean
poetry lock --regenerateagainst the committed constraints, so transitive pins stop drifting (Dependabot only touches the packages it bumps);poetry check --lockpasses. Every locked version change versus1.9.X— 74 packages before and after, none added or removed; the numpy row follows from dropping the declared dependency (Poetry then partitions it the way pip resolves it), everything else from the regeneration:python_version == "3.10")2.4.6 (
python_version >= "3.11")python_version == "3.10")2.4.6 (
python_version == "3.11")2.5.3 (
python_version >= "3.12")python_version < "3.13")python_version <= "3.14") — marker only, anyio 4.15.1 now needs it up to 3.14As a result the Testing matrix exercises numpy 2.5.3 on 3.12–3.14, pyarrow 25.0.1 on every 3.10+ job and duckdb 1.5.5 everywhere. The Ubuntu 24.04 workflow keeps its hand-maintained pins: they only need to stay inside the pyproject ranges, and
pyarrow==24.0.0still does.Checklist
ruff format,ruff check,mypy— mypy run against numpy 2.5.3 and pyarrow 25.0.1)pytest) — full suite locally on Python 3.12 against the regenerated lock (pyarrow 25.0.1, duckdb 1.5.5, numpy 2.5.3, pandas 3.0.5): pandas backend 5138 passed / 639 skipped, duckdb backend 6085 passed / 20 skipped; CI on 880df6b green, 22/22 checksImpact / Risk
row_stack, 2-Dcross,recfrom*, overflow-raisingwhere/datetime64arithmetic, tuplemeshgrid, complexlinalg.eig) have no call sites here since vtlengine does not import numpy. pyarrow 25.0.1 is likewise new for 3.10+ and covered locally and by every 3.10+ CI job. The cp39 wheel smoke test at release resolves pyarrow 21.0.0 instead of 19.0.1.<26.0; worth a line about the pyarrow ceilings (21.0.0 on 3.9, 25.0.1 on 3.10+).Notes
.github/workflows/ubuntu_test_24_04.ymlis unaffected:pyarrow==24.0.0with--no-dependenciesstays inside<26.0, and its APT numpy 1.26.4 + pandas 2.1.4 combination was always exercised there.release.ymlcovers it.docs/jupyterlite/patch_lock.pymirrors vtlengine's declared dependencies in the patched Pyodide lockfile, so itsnumpyentry for vtlengine goes too; nothing changes at load time because the distribution's pandas and pyarrow entries both depend on numpy. pysdmx keeps its numpy entry (pysdmx declares numpy under itsdataextra).mainreceives this through the periodic 1.9.X merge.