Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 86 additions & 14 deletions .github/workflows/build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,46 @@ jobs:
echo ' */'
fi

pure-python-test:
name: Pure-Python Solver & API (${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.13"]
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Python test dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pytest pytest-cov numpy "scipy>=1.7" tifffile h5py

- name: Run pure-Python test suite
env:
PYTHONPATH: ${{ github.workspace }}/python
run: |
pytest python/tests/ -v --tb=short \
--cov=openimpala \
--cov-report=xml:coverage-python-pure.xml \
--cov-report=term

- name: Upload pure-Python coverage to Codecov
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v5
with:
files: coverage-python-pure.xml
flags: python-pure
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

static-analysis:
name: Static Analysis (clang-tidy)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -261,8 +301,9 @@ jobs:
coverage:
name: Code Coverage
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
if: github.event_name == 'pull_request' || (github.event_name == 'push' && github.ref == 'refs/heads/master') || github.event_name == 'workflow_dispatch'
permissions:
contents: read
pull-requests: write

steps:
Expand All @@ -288,57 +329,76 @@ jobs:
run: |
sudo apptainer build --force dependency_image.sif containers/Singularity.deps.def

- name: Build with coverage and run tests
- name: Build with coverage and run C++ + Python tests
run: |
sudo apptainer exec \
--writable-tmpfs \
--bind $PWD:/src \
./dependency_image.sif \
bash -c '
set -euo pipefail
source /opt/rh/gcc-toolset-11/enable
export OMPI_ALLOW_RUN_AS_ROOT=1
export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1

python3.11 -m pip install pytest-cov tifffile h5py

cd /src
rm -rf cmake_build_cov && mkdir cmake_build_cov && cd cmake_build_cov

# Changed cmake3 to cmake
rm -rf cmake_build_cov /src/python/openimpala/_core*.so
mkdir cmake_build_cov && cd cmake_build_cov

PYBIND11_DIR=$(python3.11 -m pybind11 --cmakedir)
cmake .. \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=$(which mpicc) \
-DCMAKE_CXX_COMPILER=$(which mpicxx) \
-DPython_EXECUTABLE=/usr/bin/python3.11 \
-Dpybind11_DIR=${PYBIND11_DIR} \
-DCMAKE_PREFIX_PATH="/opt/amrex/25.03;/opt/hypre/v2.32.0;/opt/hdf5/1.12.3;/opt/libtiff/4.6.0" \
-DBUILD_TESTING=ON \
-DOPENIMPALA_PYTHON=ON \
-DENABLE_COVERAGE=ON

make -j$(nproc)
ctest --output-on-failure --timeout 300 || true
ctest --output-on-failure --timeout 300 -LE python

cd /src
PYTHONPATH=/src/python /usr/bin/python3.11 -m pytest python/tests/ \
-v --tb=short -p no:cacheprovider \
--cov=openimpala \
--cov-report=xml:/src/coverage-python-bindings.xml \
--cov-report=term
'

- name: Generate coverage report
- name: Generate C++ coverage report
run: |
sudo apptainer exec \
--writable-tmpfs \
--bind $PWD:/src \
./dependency_image.sif \
bash -c '
set -euo pipefail
source /opt/rh/gcc-toolset-11/enable

# Dropped the dnf install and used native python3.11 pip!
python3.11 -m pip install gcovr

cd /src
gcovr --root . \
--filter "src/" \
--filter "python/bindings/" \
--print-summary \
--gcov-ignore-parse-errors=suspicious_hits.warn_once_per_file \
--xml coverage.xml \
--txt coverage.txt \
cmake_build_cov/

rm -f /src/python/openimpala/_core*.so
'

- name: Generate summary
if: always()
run: |
if [ -f coverage.txt ]; then
echo "## Code Coverage Report" >> $GITHUB_STEP_SUMMARY
echo "## Code Coverage Report (C++ Engine & Bindings)" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
cat coverage.txt >> $GITHUB_STEP_SUMMARY
Expand All @@ -353,14 +413,26 @@ jobs:
path: |
coverage.xml
coverage.txt
coverage-python-bindings.xml
retention-days: 30
if-no-files-found: warn

- name: Upload to Codecov
- name: Upload C++ coverage to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: coverage.xml
flags: cpp
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

- name: Upload Python (_core) coverage to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: coverage-python-bindings.xml
flags: python-bindings
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Expand Down Expand Up @@ -388,7 +460,7 @@ jobs:
'```',
'',
'---',
'*Generated by CI — coverage data from [gcovr](https://gcovr.com/)*',
'*Generated by CI — C++ & bindings coverage from [gcovr](https://gcovr.com/), Python coverage uploaded to [Codecov](https://codecov.io/gh/BASE-Laboratory/OpenImpala)*',
].join('\n');

const { data: comments } = await github.rest.issues.listComments({
Expand Down
31 changes: 22 additions & 9 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,23 @@ on:
paths:
- 'src/**'
- 'python/**'
- 'tutorials/**'
- 'Doxyfile'
- 'docs/**'
- '.github/workflows/docs.yml'
pull_request:
branches: [master]
paths:
- 'src/**'
- 'python/**'
- 'tutorials/**'
- 'Doxyfile'
- 'docs/**'
- '.github/workflows/docs.yml'
workflow_dispatch: # Allow manual triggering

# Sets permissions for GitHub Pages deployment
permissions:
contents: read
pages: write
id-token: write

# Only allow one deployment at a time
concurrency:
group: "pages"
cancel-in-progress: true

jobs:
build-docs:
Expand Down Expand Up @@ -49,20 +51,31 @@ jobs:
run: doxygen Doxyfile

- name: Build Sphinx documentation
run: sphinx-build -b html docs/ docs/_build/html
env:
PYTHONPATH: ${{ github.workspace }}/python
OPENIMPALA_REQUIRE_DOXYGEN: "1"
run: sphinx-build -W --keep-going -b html docs/ docs/_build/html

- name: Copy Doxygen HTML into Sphinx output
run: cp -r docs/doxygen/html docs/_build/html/doxygen

- name: Upload Pages artifact
if: github.event_name != 'pull_request'
uses: actions/upload-pages-artifact@v3
with:
path: docs/_build/html

deploy:
name: Deploy to GitHub Pages
if: github.event_name != 'pull_request'
needs: build-docs
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
concurrency:
group: "pages"
cancel-in-progress: true
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
Expand Down
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,15 @@ Backtrace*
# Python
__pycache__/
*.pyc
.venv/
.pytest_cache/

# Docs
docs/_build/
docs/xml/
docs/doxygen/
docs/jupyter_execute/
docs/tutorials/*.ipynb

# CMake
CMakeCache.txt
Expand Down
57 changes: 57 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
cff-version: 1.2.0
title: "OpenImpala: Scalable Transport Property Computation on 3D Voxel Images of Porous Media"
message: "If you use OpenImpala in published work, please cite the software and associated publications below."
type: software
authors:
- family-names: "Le Houx"
given-names: "James"
orcid: "https://orcid.org/0000-0002-1576-0673"
affiliation: "University of Greenwich"
email: "james.le-houx@gre.ac.uk"
repository-code: "https://github.com/BASE-Laboratory/OpenImpala"
url: "https://base-laboratory.github.io/OpenImpala/"
license: BSD-3-Clause
keywords:
- porous media
- tortuosity
- effective diffusivity
- homogenisation
- tomography
- battery modelling
- AMReX
- high-performance computing
preferred-citation:
type: article
title: "OpenImpala: OPEN source IMage based PArallisable Linear Algebra solver"
authors:
- family-names: "Le Houx"
given-names: "James"
orcid: "https://orcid.org/0000-0002-1576-0673"
- family-names: "Kramer"
given-names: "Denis"
journal: "SoftwareX"
volume: 15
start: 100729
year: 2021
doi: "10.1016/j.softx.2021.100729"
references:
- type: article
title: "Statistical Effective Diffusivity Estimation in Porous Media Using an Integrated On-site Imaging Workflow for Synchrotron Users"
authors:
- family-names: "Le Houx"
given-names: "James"
- family-names: "Ruiz"
given-names: "Siul"
- family-names: "McKay Fletcher"
given-names: "Daniel"
- family-names: "Ahmed"
given-names: "Sharif"
- family-names: "Roose"
given-names: "Tiina"
journal: "Transport in Porous Media"
volume: 150
issue: 1
start: 71
end: 88
year: 2023
doi: "10.1007/s11242-023-01993-7"
18 changes: 9 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ your local builds will behave identically to the CI build.
git clone https://github.com/BASE-Laboratory/OpenImpala.git
cd OpenImpala
# Build the container (one-off, ~20 min)
sudo apptainer build dependency_image.sif containers/dependency.def
sudo apptainer build dependency_image.sif containers/Singularity.deps.def
# Configure and build
apptainer exec --bind "$(pwd):/src" dependency_image.sif \
bash -c "cd /src && mkdir -p build && cd build && cmake .. && make -j"
Expand Down Expand Up @@ -74,10 +74,8 @@ that fail either check.
* **C++**: clang-tidy with the rules in `.clang-tidy` (bugprone-*,
performance-*, modernize-use-nullptr). CI runs this against every
changed file.
* **Python**: PEP 8 via `ruff` is recommended but not enforced. Public
API functions in `python/openimpala/facade.py` should have NumPy-style
docstrings.
* **Fortran files** are not processed by clang-format or clang-tidy.
* **Python**: PEP 8 via `ruff` is recommended. Public API functions in
`python/openimpala/` should have NumPy-style docstrings.

All C++ code lives in `namespace OpenImpala`. Headers use `#ifndef`
include guards rather than `#pragma once`.
Expand All @@ -87,11 +85,12 @@ include guards rather than `#pragma once`.
The test suite has three layers:

1. **C++ unit and integration tests** (under `tests/`) — Catch2-based,
register with CTest. Run with `ctest --output-on-failure`. Each test
registered with CTest. Run with `ctest --output-on-failure`. Each test
is a small standalone solve with an analytical solution or a tightly
bounded numerical reference.
2. **Python tests** (under `python/tests/`) — pytest-based, exercise the
facade and binding layers. Run with `pytest python/tests/`.
2. **Python tests** (under `python/tests/`) — pytest-based, exercising both
the pure-Python solver and C++ bindings. Run without compiling C++ via
`PYTHONPATH=python pytest python/tests/`.
3. **Regression benchmarks** (under `tests/benchmarks/`) — Python scripts
generating synthetic datasets with known analytical tortuosity values
(uniform block, series layers, parallel layers).
Expand All @@ -107,10 +106,11 @@ and passes on your fix.
3. Run the local CI loop:
```bash
clang-format -i src/**/*.{cpp,H} python/bindings/*.cpp
PYTHONPATH=python pytest python/tests/
apptainer exec --bind "$(pwd):/src" dependency_image.sif \
bash -c "cd /src/build && make -j && ctest --output-on-failure"
```
4. Open a pull request against `main`. The PR description should explain
4. Open a pull request against `master`. The PR description should explain
*why* the change is needed, not just *what* it changes.
5. CI runs clang-format check, clang-tidy static analysis, the C++ test
suite, Python tests, code-coverage reporting (via Codecov), and wheel
Expand Down
Loading
Loading