Skip to content

Modernise the dependency stack and pin it exactly #179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FWao merged 8 commits into from
Aug 20, 2026
Merged

Modernise the dependency stack and pin it exactly #179

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9bae1d6
Add static dependency-configuration checks
FWao Aug 6, 2026
b493200
Install flash-attn, mamba-ssm and causal-conv1d from the Astral index
FWao Aug 6, 2026
cbe0694
Upgrade to Python 3.13/3.14, PyTorch 2.11 and current dependencies
FWao Aug 7, 2026
dbb24f9
Remove obsolete build workarounds, rework CI and documentation
FWao Aug 7, 2026
ed94c9b
Hold transformers at 4.57.6 and huggingface-hub below 1.0
FWao Aug 7, 2026
237e6af
Pin every dependency to an exact version
FWao Aug 19, 2026
7d90d1b
Keep xformers out of CPU installs and pin ruff
FWao Aug 19, 2026
2ea3022
Run the test suite on Linux only
FWao Aug 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
289 changes: 222 additions & 67 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,101 +6,256 @@ on:
pull_request:
branches: [ main ]

env:
# Recorded so CI and contributors resolve with the same uv. Must stay inside
# the `required-version` range in pyproject.toml.
UV_VERSION: "0.12.2"

jobs:
test_extractors:
# --------------------------------------------------------------------- #
# Cheap checks that need no installed environment.
# --------------------------------------------------------------------- #
dependency_config:
name: Dependency configuration
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: '3.14'
- name: Check dependency declarations
run: python scripts/check_dependency_config.py

format_and_lint:
runs-on: ubuntu-latest
# needs: format_and_lint
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: '3.14'
# Pinned, and kept in step with the dev group: ruff 0.16 widened its
# default rule set and began formatting Python blocks inside Markdown, so
# an unpinned install turns CI red without a code change.
- name: Install the code linting and formatting tool Ruff
run: pipx install ruff==0.15.5
- name: Lint code with Ruff
run: ruff check --output-format=github --target-version=py313
- name: Check code formatting with Ruff
run: ruff format --diff --target-version=py313

# --------------------------------------------------------------------- #
# CPU installation across the supported platforms.
# --------------------------------------------------------------------- #
test_others:
name: CPU tests (${{ matrix.os }}, py${{ matrix.python-version }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: ["3.13"]
extractor: [
"ctranspath",
"chief-ctranspath",
"conch",
"conch1_5",
"uni",
"uni2",
"dino-bloom",
"gigapath",
"h-optimus-0",
"h-optimus-1",
"mstar",
"plip",
]

include:
- os: ubuntu-latest # Linux x86_64
python-version: "3.13"
- os: ubuntu-latest # Linux x86_64
python-version: "3.14"
# macOS runs the install smoke test below, not the test suite: STAMP
# is used on Linux, and the deploy tests hit a macOS-only DataLoader
# worker segfault that also reproduces on main.
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5

- name: Install uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
version: ${{ env.UV_VERSION }}
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: "uv.lock"

# - name: Set up Python ${{ matrix.python-version }}
# uses: actions/setup-python@v5
# with:
# python-version: ${{ matrix.python-version }}


# --locked so CI can never silently rewrite the lockfile.
- name: Install the project
run: uv sync --extra cpu --dev
run: uv sync --locked --extra cpu --dev

- name: Build
run: uv build

- name: Run tests for extractor ${{ matrix.extractor }}
run: uv run --extra cpu pytest -s tests/test_feature_extractors.py -k "${{ matrix.extractor }}" --verbose

- name: Verify no CUDA packages were installed
run: uv run --no-sync python scripts/verify_installed_stack.py --expect-no-cuda

- name: Run other tests
run: uv run --no-sync pytest -s tests/ --ignore=tests/test_feature_extractors.py --verbose
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}

test_others:
# --------------------------------------------------------------------- #
# Apple Silicon must install with no CUDA build step at all.
# --------------------------------------------------------------------- #
macos_cpu_smoke:
name: Apple Silicon CPU import smoke test
runs-on: macos-latest
steps:
- uses: actions/checkout@v5

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
version: ${{ env.UV_VERSION }}
python-version: "3.14"
enable-cache: true
cache-dependency-glob: "uv.lock"

# `-v` surfaces any build backend that runs. The CUDA extensions are
# `no-build-package`, so a source build of one would fail the sync instead
# of silently probing for nvcc.
- name: Install the project
run: uv sync --locked --extra cpu --dev -v 2>&1 | tee sync.log

- name: Verify no CUDA package and no CUDA build step
run: |
uv run --no-sync python scripts/verify_installed_stack.py --expect-no-cuda
if grep -Eiq 'building (flash-attn|mamba-ssm|causal-conv1d)' sync.log; then
echo "::error::a CUDA extension was built from source on macOS"
exit 1
fi

- name: Import STAMP and the CPU-compatible extractors
run: |
uv run --no-sync python - <<'PY'
import importlib, pkgutil
import stamp.preprocessing.extractor as ex
import stamp.encoding.encoder as en

# cobra and gigapath are Linux-only CUDA extras and raise a deliberate
# ModuleNotFoundError when their optional dependencies are absent.
expected_absent = {"cobra", "gigapath"}
failures = []
for pkg in (ex, en):
for module in pkgutil.iter_modules(pkg.__path__):
try:
importlib.import_module(f"{pkg.__name__}.{module.name}")
except ModuleNotFoundError as error:
if module.name not in expected_absent:
failures.append(f"{module.name}: {error}")
assert not failures, failures
print("all CPU-compatible extractors and encoders import")
PY

# --------------------------------------------------------------------- #
# The GPU extras must install from wheels alone on a machine with no GPU,
# no driver and no CUDA toolkit.
# --------------------------------------------------------------------- #
gpu_install_without_cuda:
name: GPU extras install without CUDA (${{ matrix.extra }})
runs-on: ubuntu-latest
# needs: format_and_lint
strategy:
fail-fast: false
matrix:
python-version: ["3.13"]
extra: [gpu, gpu_all]
steps:
- uses: actions/checkout@v5

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
version: ${{ env.UV_VERSION }}
python-version: "3.14"
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: Confirm there is no CUDA toolkit
run: |
if command -v nvcc; then echo "::error::nvcc is present"; exit 1; fi
if command -v nvidia-smi; then echo "::error::nvidia-smi is present"; exit 1; fi
echo "no CUDA toolkit and no driver, as required"

- name: Install ${{ matrix.extra }}
run: uv sync --locked --extra ${{ matrix.extra }}
env:
CUDA_HOME: /nonexistent-cuda

# `gpu` deliberately omits flash-attn: only the gigapath, conch1_5 and musk
# variants bundled into `gpu_all` need it.
- name: Verify the installed CUDA stack
run: |
if [ "${{ matrix.extra }}" = "gpu_all" ]; then
required="flash-attn mamba-ssm causal-conv1d"
else
required="mamba-ssm causal-conv1d"
fi
uv run --no-sync python scripts/verify_installed_stack.py \
--expect-cuda --require $required

# --------------------------------------------------------------------- #
# Linux aarch64 must select the aarch64 wheels rather than failing over to
# a source build.
# --------------------------------------------------------------------- #
gpu_install_aarch64:
name: GPU extras install on Linux aarch64
runs-on: ubuntu-24.04-arm
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5

- name: Install uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
python-version: ${{ matrix.python-version }}
version: ${{ env.UV_VERSION }}
python-version: "3.14"
enable-cache: true
cache-dependency-glob: "uv.lock"

# - name: Set up Python ${{ matrix.python-version }}
# uses: actions/setup-python@v5
# with:
# python-version: ${{ matrix.python-version }}

- name: Install the project
run: uv sync --extra cpu --dev

- name: Build
run: uv build

- name: Run other tests
run: uv run --extra cpu pytest -s tests/ -k "not test_feature_extractors.py" --verbose

- name: Install gpu_all
run: uv sync --locked --extra gpu_all
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}

format_and_lint:
runs-on: ubuntu-latest
CUDA_HOME: /nonexistent-cuda

- name: Verify aarch64 wheels were selected
run: |
uv run --no-sync python scripts/verify_installed_stack.py \
--expect-cuda --require flash-attn mamba-ssm causal-conv1d
uv run --no-sync python - <<'PY'
import platform
assert platform.machine() == "aarch64", platform.machine()
print("running on aarch64")
PY

# --------------------------------------------------------------------- #
# Model-weight downloads are kept out of the dependency jobs above.
# --------------------------------------------------------------------- #
test_extractors:
name: Extractor ${{ matrix.extractor }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.14"]
extractor: [
"ctranspath",
"chief-ctranspath",
"conch",
"conch1_5",
"uni",
"uni2",
"dino-bloom",
"gigapath",
"h-optimus-0",
"h-optimus-1",
"mstar",
"plip",
]
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
- uses: actions/checkout@v5

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
python-version: '3.13'
- name: Install the code linting and formatting tool Ruff
run: pipx install ruff
- name: Lint code with Ruff
run: ruff check --output-format=github --target-version=py313
- name: Check code formatting with Ruff
run: ruff format --diff --target-version=py313
# continue-on-error: true

version: ${{ env.UV_VERSION }}
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: Install the project
run: uv sync --locked --extra cpu --dev

- name: Run tests for extractor ${{ matrix.extractor }}
run: uv run --no-sync pytest -s tests/test_feature_extractors.py -k "${{ matrix.extractor }}" --verbose
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
2 changes: 1 addition & 1 deletion .python-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.13
3.14
Loading