Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 39 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@
![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](LICENSE)

**OpenLSR-X** is a reproducible Python framework and benchmark suite for **physics- and topology-guided 2D/2.5D-to-3D Super-Resolution of X-Ray Computed Tomography (XCT)** porous-media scans using shifted-window Transformers (`SwinIR`), Residual-in-Residual Dense Blocks (`RRDB` / `RaGAN`), SRGAN, and **epistemic confidence-weighted 3-axis Fourier fusion**. Originally developed during the PhD research of **Calum Green** (Imperial College London & Diamond Light Source) and maintained by **BASE Laboratory**, OpenLSR-X bridges deep-learning super-resolution with quantitative **synchrotron beamline metrology, reactive multi-phase segmentation, and uncertainty-propagated 3D diffusive transport**.
**OpenLSR-X** is a reproducible Python framework and benchmark suite for **physics- and topology-guided 2D/2.5D-to-3D Super-Resolution of X-Ray Computed Tomography (XCT)** porous-media scans using shifted-window Transformers (`SwinIR`), Residual-in-Residual Dense Blocks (`RRDB` / `RaGAN`), SRGAN, and **confidence-weighted 3-axis Fourier fusion**. Originally developed during the PhD research of **Calum Green** (Imperial College London & Diamond Light Source) and maintained by **BASE Laboratory**, OpenLSR-X bridges deep-learning super-resolution with quantitative **synchrotron beamline metrology, reactive multi-phase segmentation, and uncertainty-propagated 3D diffusive transport**.

---

## Highlights

- **2D & `2.5D` Training → Uncertainty-Aware Isotropic 3D Reconstruction:** Train memory-efficient `1`-slice or `2k + 1` multi-slice (`2.5D`) generators (`SwinIR`, `RRDB` + `RaGAN`, `SRGAN`/`SRResNet`) alongside native `SR3DResNet` baselines across integer and fractional synchrotron optical magnifications (`2x`, `2.5x`, `4x`), and fuse orthogonal sweeps (`yz + xz + xy`) via **Confidence-Weighted Directional Fourier Fusion** (`confidence_fourier`) with zero-overhead voxel-wise epistemic uncertainty $\sigma_{\text{ortho}}(\mathbf{x})$.
- **Physics- & Topology-Preserving Objectives:** Combines Beer–Lambert **X-ray attenuation conservation** (`AttenuationConservationLoss`), differentiable **3D pore-throat skeleton preservation** (`SoftCLDiceLoss`), 2D/3D **Sobel phase-boundary sharpness** (`SobelGradientLoss`), and **Focal Frequency Loss** (`FocalFrequencyLoss`).
- **Leakage-Free 3D Spatial Splitting:** Enforces strictly disjoint 3D sub-volume slabs (with configurable guard margins and per-axis boundary-safe `2.5D` reflect padding) across `train`, `val`, and `test`.
- **Synchrotron Spectral, Petrophysical & Diffusive Transport Metrology:** Evaluates **3D Fourier Shell Correlation (FSC)** & **2D Fourier Ring Correlation (FRC)** (`1/2`-bit and `0.143` effective resolution in $\mu\text{m}$), tri-phase `Zn-13X` reactive core-rim segmentation (`Macropore` / `Na-13X` / `Zn-13X`), porosity/Euler/surface-area metrics, quasi-static drainage capillary pressure $P_c(S_w)$, and **3D steady-state Laplace diffusive tortuosity** ($D_{\text{eff}} / D_0$, $\tau$) with epistemic uncertainty intervals $[D_{\text{eff}}^-, D_{\text{eff}}^+]$ and direct **OpenImpala** export.
- **Out-of-the-Box Benchmarks & Open Synchrotron Profiles:** Ships with deterministic **Factorial PoreSpy** benchmarks, **Open Rock Micro-CT** (Bentheimer Sandstone & Indiana Limestone), and automated Zenodo staging + calibrated phantoms for the open Diamond Light Source I13-2 multi-resolution `Zn-13X` dataset ([*Scientific Data*, 2025](https://doi.org/10.1038/s41597-025-04605-9)).
- **Cell-Centered 2D & `2.5D` Training → Isotropic 3D Reconstruction:** Train memory-efficient `1`-slice or native-LR-strided `2k + 1` multi-slice (`2.5D`) generators (`SwinIR`, `RRDB` + `RaGAN`, `SRGAN`/`SRResNet`) alongside native `SR3DResNet` baselines across integer and fractional synchrotron optical magnifications (`2x`, `2.5x`, `4x`, `8x`), and fuse cell-centered (`grid_mode=True`) orthogonal sweeps (`yz + xz + xy`) via **Confidence-Weighted Directional Fourier Fusion** (`confidence_fourier`) with zero-overhead voxel-wise **orthogonal view-disagreement uncertainty** $\sigma_{\text{ortho}}(\mathbf{x})$ ($n=3$ views).
- **Physics- & Topology-Guided Objectives:** Combines **linear-attenuation ($\mu$) downsampling consistency** (`AttenuationConservationLoss`, with `2.5D` through-plane $z$-PSF curvature correction), differentiable **bimodal pore-throat skeleton preservation** (`SoftCLDiceLoss`), 2D/3D **Sobel phase-boundary sharpness** (`SobelGradientLoss`), and **Focal Frequency Loss** (`FocalFrequencyLoss`).
- **Split-Isolated 3D Spatial Splitting & Normalization:** Partitions 3D volumes into disjoint sub-volume slabs (separated by PSF guard margins) *before* fitting intensity normalization bounds on `train` only and *before* per-split synthetic PSF degradation.
- **Synchrotron Spectral, Petrophysical & Diffusive Transport Metrology:** Evaluates **3D Fourier Shell Correlation (FSC)** & **2D Fourier Ring Correlation (FRC)** (`FSC = 0.5` prediction-vs-reference crossover frequency & spectral AUC), ground-truth-thresholded porosity/Euler/surface-area metrics, tri-phase `Zn-13X` reactive core-rim segmentation (`Macropore` / `Na-13X` / `Zn-13X`), quasi-static drainage capillary pressure $S_{\text{nwp}}(P_c)$, and residual-verified **3D steady-state Laplace diffusive tortuosity** ($D_{\text{eff}} / D_0$, $\tau$) with view-disagreement sensitivity bounds $[D_{\text{eff}}^-, D_{\text{eff}}^+]$ and direct **OpenImpala** export.
- **Provenance-Verified Open Benchmarks:** Ships with deterministic **Factorial PoreSpy** benchmarks, **Open Rock Micro-CT** (Bentheimer Sandstone from Zenodo `5542624` & calibrated carbonate phantom), and coaxial sub-voxel-registered Zenodo staging (`10.5281/zenodo.14245751`) + explicit provenance tracking (`source_kind`, `is_experimental_xct`) for the Diamond Light Source I13-2 multi-resolution `Zn-13X` dataset ([*Scientific Data*, 2025](https://doi.org/10.1038/s41597-025-04605-9)).

---

Expand All @@ -31,52 +31,63 @@ uv venv --python 3.11
uv pip install -e ".[dev,docs]"
```

### 2. Train, Test, and Predict via Unified CLI
### 2. Train, Reconstruct 3D Volumes, and Evaluate via Unified CLI

Train the physics- and topology-guided `2.5D SwinIR` configuration on Diamond Light Source I13-2 `Zn-13X` (`2x`) or synthetic PoreSpy media:
Stage the Diamond Light Source I13-2 `Zn-13X` (`2x` & `8x`) benchmark (pass `--offline` on air-gapped nodes for explicit synthetic phantoms) and train the multi-axis `2.5D SwinIR` configuration:

```bash
openlsrx download-zn13x --synthetic-fallback
openlsrx download-zn13x --dest data/benchmarks
openlsrx fit -c configs/i13-xct/zn13x_2x_swinir_25d_physics.yaml
```

Run standalone 3-axis orthogonal `confidence_fourier` super-resolution + view-disagreement uncertainty export on any 3D LR volume:

```bash
openlsrx predict-volume \
--ckpt lightning_logs/version_0/checkpoints/last.ckpt \
--input data/benchmarks/zn13x_2x_lr.tif \
--output-dir results/zn13x_2x_sr \
--fusion confidence_fourier \
--save-uncertainty
```

Prepare open rock micro-CT benchmark stacks and train on Bentheimer Sandstone:

```bash
openlsrx download-benchmarks
openlsrx download-benchmarks --dest data/benchmarks
openlsrx fit -c configs/labxct/bs_config.yaml
```

Evaluate image-fidelity, petrophysical, 3D FSC spectral, tri-phase `Zn-13X`, and 3D diffusive transport metrics (+ OpenImpala export):

```bash
openlsrx evaluate \
--pred predicted_stack.tiff \
--target high_res_stack.tiff \
--pred results/zn13x_2x_sr/zn13x_2x_lr_sr_confidence_fourier.tif \
--target data/benchmarks/zn13x_2x_hr.tif \
--voxel-size 0.8125 \
--include-transport \
--include-spectral \
--include-triphase \
--export-openimpala outputs/openimpala_ready \
--output metrics.json
--export-openimpala-dir results/openimpala_ready \
--output results/metrics.json
```

---

## Reproducing Manuscript Benchmarks

Run the Factorial PoreSpy, Open Rock Micro-CT, and 7-arm Diamond I13-2 `Zn-13X` studies and generate publication-ready LaTeX `booktabs` tables and 300-DPI comparison figures:
Run the Factorial PoreSpy, Open Rock Micro-CT, and 9-arm Diamond I13-2 `Zn-13X` studies and generate publication-ready LaTeX `booktabs` tables and comparison figures (production runs fail loudly if real Zenodo/HDF5 data are unavailable unless `--allow-synthetic-fallback` is explicitly passed):

```bash
# Rapid CPU smoke verification (~2 minutes)
# Rapid offline CPU smoke verification (~2 minutes; explicitly labeled synthetic phantoms)
python scripts/benchmarks/porespy_study.py --fast
python scripts/benchmarks/public_rock_xct.py --fast
python scripts/benchmarks/zn13x_synchrotron_study.py --fast
python scripts/benchmarks/aggregate_campaign.py --inputs outputs --output-dir outputs/campaign_summary
python scripts/benchmarks/aggregate_campaign.py --inputs results --output-dir results/campaign_summary

# Full multi-seed GPU campaign (see docs/npj-study-design.md)
python scripts/benchmarks/zn13x_synchrotron_study.py --volume-size 256 --epochs 300 --seeds 42 43 44
python scripts/benchmarks/aggregate_campaign.py --inputs outputs --output-dir outputs/campaign_summary
# Full multi-seed GPU campaign on real Zenodo / beamline HDF5 data (see docs/npj-study-design.md)
python scripts/benchmarks/zn13x_synchrotron_study.py --lr-size 48 --epochs 120 --seeds 42 43 44
python scripts/benchmarks/aggregate_campaign.py --inputs results --output-dir results/campaign_summary
```

---
Expand All @@ -90,17 +101,17 @@ OpenLSR-X/
├── paper/ # JOSS manuscript (paper.md) and bibliography (paper.bib)
├── scripts/
│ ├── benchmarks/ # PoreSpy, Rock XCT & I13-2 Zn-13X runners + multi-seed campaign aggregator
│ └── hpc/ # SLURM and SGE ('Young') GPU batch templates
│ └── hpc/ # Isambard-AI GH200 SLURM array, generic SLURM, and SGE ('Young') templates
├── src/openlsrx/
│ ├── baselines/ # 2D & 3D classical interpolation baselines (Nearest, Trilinear, Tricubic, Lanczos)
│ ├── data/ # PoreSpy, Rock XCT, DLS I13-2 Zn-13X loaders & 2.5D leakage-free 3D splitting
│ ├── inference/ # Single-axis, 3-axis Fourier, and epistemic confidence-weighted 3D fusion
│ ├── io/ # OpenImpala 3D TIFF + .inputs deck exporter
│ ├── losses/ # Attenuation conservation, soft-clDice topology, Sobel, FFL, RaGAN & VGGLoss
│ ├── metrics/ # 2D/3D PSNR/SSIM, 3D FSC / 2D FRC, tri-phase petrophysics & 3D Laplace transport
│ ├── baselines/ # Cell-centered 2D & 3D classical interpolation baselines
│ ├── data/ # PoreSpy, Rock XCT, DLS I13-2 Zn-13X loaders & split-isolated 2.5D datasets
│ ├── inference/ # Cell-centered single-axis, 3-axis Fourier, and confidence-weighted 3D fusion
│ ├── io/ # OpenImpala 3D TIFF + AMReX .inputs deck exporter
│ ├── losses/ # Attenuation consistency, soft-clDice topology, Sobel, FFL & VGGLoss
│ ├── metrics/ # 2D/3D PSNR/SSIM, 3D FSC / 2D FRC, HR-locked petrophysics & 3D Laplace transport
│ ├── models/ # SwinIRGenerator, RRDBGenerator, SRGAN/SRResNet, and native SR3DResNet
│ └── cli.py # Unified `openlsrx` CLI entrypoint
└── tests/ # 71 unit, analytical physics/spectral/transport, Lightning & CLI tests (>90% cov)
└── tests/ # Comprehensive unit, analytical physics/spectral/transport & red-team regression suite
```

---
Expand Down
2 changes: 1 addition & 1 deletion configs/base/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ trainer:

model:
learning_rate: 1.0e-4
adaptive_size: 16
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
num_blocks: 10
Expand Down
2 changes: 1 addition & 1 deletion configs/base/srresnet_ablation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ trainer:

model:
learning_rate: 1.0e-4
adaptive_size: 16
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
num_blocks: 10
Expand Down
4 changes: 3 additions & 1 deletion configs/i13-xct/i13_predict_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ trainer:

model:
learning_rate: 1.0e-5
adaptive_size: 40
adaptive_size: 8
super_res_factor: 2
num_blocks: 10
sample_id: Zn13X
Expand All @@ -28,6 +28,8 @@ data:
low_y_range: [1010, 1650]
low_x_range: [858, 1498]
predict_axis: xy
predict_mode: orthogonal_3axis
predict_fusion: confidence_fourier
eval_batch_size: 2
num_workers: 0
seed: 42
6 changes: 3 additions & 3 deletions configs/i13-xct/i13_xct_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ trainer:

model:
learning_rate: 1.0e-5
adaptive_size: 40
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
num_blocks: 10
Expand All @@ -32,8 +32,8 @@ data:
norm_mode: fixed
low_vmin: -5.23e-4
low_vmax: 1.11e-3
high_vmin: -2.50e-4
high_vmax: 6.79e-4
high_vmin: -5.23e-4
high_vmax: 1.11e-3
low_z_range: [845, 1485]
low_y_range: [1010, 1650]
low_x_range: [858, 1498]
Expand Down
2 changes: 1 addition & 1 deletion configs/i13-xct/zn13x_2x_swinir_25d_physics.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ data:
train_frac: 0.70
val_frac: 0.15
guard_slices_lr: 1
train_axes: ["yz"]
train_axes: ["yz", "xz", "xy"]
test_axis: perpendicular
predict_axis: xy
patch_size_lr: 32
Expand Down
2 changes: 1 addition & 1 deletion configs/labxct/bs_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ trainer:

model:
learning_rate: 5.0e-5
adaptive_size: 16
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
num_blocks: 10
Expand Down
9 changes: 8 additions & 1 deletion configs/labxct/bs_config_predict.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ trainer:

model:
learning_rate: 5.0e-5
adaptive_size: 16
adaptive_size: 8
super_res_factor: 2
num_blocks: 10
sample_id: bentsand
Expand All @@ -20,7 +20,14 @@ data:
high_path: data/benchmarks/bentheimer_sandstone_hr.tif
degrade_from_hr: true
scale_factor: 2
psf_sigma: 0.85
degrade_noise_std: 0.025
norm_mode: percentile
lower_pct: 0.5
upper_pct: 99.5
predict_axis: xz
predict_mode: orthogonal_3axis
predict_fusion: confidence_fourier
eval_batch_size: 4
num_workers: 0
seed: 42
2 changes: 1 addition & 1 deletion configs/labxct/il_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ trainer:

model:
learning_rate: 5.0e-5
adaptive_size: 16
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
num_blocks: 12
Expand Down
9 changes: 8 additions & 1 deletion configs/labxct/il_config_predict.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ trainer:

model:
learning_rate: 5.0e-5
adaptive_size: 16
adaptive_size: 8
super_res_factor: 2
num_blocks: 12
sample_id: indlime
Expand All @@ -20,7 +20,14 @@ data:
high_path: data/benchmarks/indiana_limestone_hr.tif
degrade_from_hr: true
scale_factor: 2
psf_sigma: 0.90
degrade_noise_std: 0.030
norm_mode: percentile
lower_pct: 0.5
upper_pct: 99.5
predict_axis: xz
predict_mode: orthogonal_3axis
predict_fusion: confidence_fourier
eval_batch_size: 4
num_workers: 0
seed: 42
Loading
Loading