Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,4 @@ jobs:
test -f /tmp/bench/paper_artifacts/figures/orthogonal_sr_comparison.pdf
test -f /tmp/bench/campaign_summary/campaign_table.tex
test -f /tmp/bench/zn13x/openimpala_ready/zn13x_2x_rrdb_ragan_s42_openimpala.inputs
test -f /tmp/bench/zn13x/openimpala_ready/zn13x_2x_swinir_25d_phys_s42_openimpala.inputs
65 changes: 35 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,17 @@
![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](LICENSE)

**OpenLSR-X** is a reproducible Python framework and benchmark suite for **2D-to-3D Super-Resolution of X-Ray Computed Tomography (XCT)** porous-media scans using Generative Adversarial Networks (SRGAN) and multi-axis 3D Fourier fusion. Originally developed during the PhD research of **Calum Green** (Imperial College London & Diamond Light Source) and maintained by **BASE Laboratory**, OpenLSR-X bridges deep-learning super-resolution with quantitative **petrophysical and topological validation**.
**OpenLSR-X** is a reproducible Python framework and benchmark suite for **physics- and topology-guided 2D/2.5D-to-3D Super-Resolution of X-Ray Computed Tomography (XCT)** porous-media scans using shifted-window Transformers (`SwinIR`), Residual-in-Residual Dense Blocks (`RRDB` / `RaGAN`), SRGAN, and **epistemic confidence-weighted 3-axis Fourier fusion**. Originally developed during the PhD research of **Calum Green** (Imperial College London & Diamond Light Source) and maintained by **BASE Laboratory**, OpenLSR-X bridges deep-learning super-resolution with quantitative **synchrotron beamline metrology, reactive multi-phase segmentation, and uncertainty-propagated 3D diffusive transport**.

---

## Highlights

- **2D Training → Isotropic 3D Reconstruction:** Train memory-efficient 2D slice models (with reflect padding, `PixelNorm`, generator warmup, and optional VGG-19 perceptual loss) and reconstruct artifact-free 3D volumes via **3-Axis Orthogonal Fourier Fusion** (`yz + xz + xy`).
- **Leakage-Free 3D Spatial Splitting:** Enforces strictly disjoint 3D sub-volume slabs (with configurable guard margins) across `train`, `val`, and `test` so orthogonal evaluation slices never share voxels with training slices.
- **Petrophysical & Topological Metrics:** Evaluates not only 2D/3D PSNR and tri-planar SSIM, but also segmented **porosity error** ($|\Delta\phi|$), **pore Dice/IoU**, **specific surface area** ($S_v$), **3D Euler characteristic density** ($\chi_V$), **inscribed pore-radius Wasserstein-1 distance**, **two-point correlation** $S_2(r)$, and **through-plane gradient anisotropy**.
- **Out-of-the-Box Benchmarks & Open Synchrotron Profiles:** Ships with deterministic **Factorial PoreSpy** benchmarks, an **Open Rock Micro-CT** runner (Bentheimer Sandstone & Indiana Limestone), and NeXus/HDF5 profiles for the open Diamond Light Source I13-2 multi-resolution `Zn-13X` dataset ([*Scientific Data*, 2025](https://doi.org/10.1038/s41597-025-04605-9)).
- **2D & `2.5D` Training → Uncertainty-Aware Isotropic 3D Reconstruction:** Train memory-efficient `1`-slice or `2k + 1` multi-slice (`2.5D`) generators (`SwinIR`, `RRDB` + `RaGAN`, `SRGAN`/`SRResNet`) alongside native `SR3DResNet` baselines across integer and fractional synchrotron optical magnifications (`2x`, `2.5x`, `4x`), and fuse orthogonal sweeps (`yz + xz + xy`) via **Confidence-Weighted Directional Fourier Fusion** (`confidence_fourier`) with zero-overhead voxel-wise epistemic uncertainty $\sigma_{\text{ortho}}(\mathbf{x})$.
- **Physics- & Topology-Preserving Objectives:** Combines Beer–Lambert **X-ray attenuation conservation** (`AttenuationConservationLoss`), differentiable **3D pore-throat skeleton preservation** (`SoftCLDiceLoss`), 2D/3D **Sobel phase-boundary sharpness** (`SobelGradientLoss`), and **Focal Frequency Loss** (`FocalFrequencyLoss`).
- **Leakage-Free 3D Spatial Splitting:** Enforces strictly disjoint 3D sub-volume slabs (with configurable guard margins and per-axis boundary-safe `2.5D` reflect padding) across `train`, `val`, and `test`.
- **Synchrotron Spectral, Petrophysical & Diffusive Transport Metrology:** Evaluates **3D Fourier Shell Correlation (FSC)** & **2D Fourier Ring Correlation (FRC)** (`1/2`-bit and `0.143` effective resolution in $\mu\text{m}$), tri-phase `Zn-13X` reactive core-rim segmentation (`Macropore` / `Na-13X` / `Zn-13X`), porosity/Euler/surface-area metrics, quasi-static drainage capillary pressure $P_c(S_w)$, and **3D steady-state Laplace diffusive tortuosity** ($D_{\text{eff}} / D_0$, $\tau$) with epistemic uncertainty intervals $[D_{\text{eff}}^-, D_{\text{eff}}^+]$ and direct **OpenImpala** export.
- **Out-of-the-Box Benchmarks & Open Synchrotron Profiles:** Ships with deterministic **Factorial PoreSpy** benchmarks, **Open Rock Micro-CT** (Bentheimer Sandstone & Indiana Limestone), and automated Zenodo staging + calibrated phantoms for the open Diamond Light Source I13-2 multi-resolution `Zn-13X` dataset ([*Scientific Data*, 2025](https://doi.org/10.1038/s41597-025-04605-9)).

---

Expand All @@ -32,10 +33,11 @@ uv pip install -e ".[dev,docs]"

### 2. Train, Test, and Predict via Unified CLI

Run training out of the box on a synthetic 3D PoreSpy porous-media volume:
Train the physics- and topology-guided `2.5D SwinIR` configuration on Diamond Light Source I13-2 `Zn-13X` (`2x`) or synthetic PoreSpy media:

```bash
openlsrx fit -c configs/base/config.yaml
openlsrx download-zn13x --synthetic-fallback
openlsrx fit -c configs/i13-xct/zn13x_2x_swinir_25d_physics.yaml
```

Prepare open rock micro-CT benchmark stacks and train on Bentheimer Sandstone:
Expand All @@ -45,34 +47,36 @@ openlsrx download-benchmarks
openlsrx fit -c configs/labxct/bs_config.yaml
```

Predict an orthogonal super-resolved 3D BigTIFF stack from a trained checkpoint:
Evaluate image-fidelity, petrophysical, 3D FSC spectral, tri-phase `Zn-13X`, and 3D diffusive transport metrics (+ OpenImpala export):

```bash
openlsrx predict -c configs/labxct/bs_config_predict.yaml --ckpt_path path/to/checkpoint.ckpt
```

Evaluate image-fidelity and petrophysical XCT metrics on any pair of 3D TIFF/HDF5 volumes:

```bash
openlsrx evaluate --pred predicted_stack.tiff --target high_res_stack.tiff --output metrics.json
openlsrx evaluate \
--pred predicted_stack.tiff \
--target high_res_stack.tiff \
--voxel-size 0.8125 \
--include-transport \
--include-spectral \
--include-triphase \
--export-openimpala outputs/openimpala_ready \
--output metrics.json
```

---

## Reproducing Manuscript Benchmarks

Run the Factorial PoreSpy and Open Rock Micro-CT studies and generate publication-ready LaTeX `booktabs` tables and 300-DPI comparison figures:
Run the Factorial PoreSpy, Open Rock Micro-CT, and 7-arm Diamond I13-2 `Zn-13X` studies and generate publication-ready LaTeX `booktabs` tables and 300-DPI comparison figures:

```bash
# Rapid CPU smoke verification (< 2 minutes)
# Rapid CPU smoke verification (~2 minutes)
python scripts/benchmarks/porespy_study.py --fast
python scripts/benchmarks/public_rock_xct.py --fast
python scripts/benchmarks/make_paper_outputs.py
python scripts/benchmarks/zn13x_synchrotron_study.py --fast
python scripts/benchmarks/aggregate_campaign.py --inputs outputs --output-dir outputs/campaign_summary

# Full multi-seed GPU benchmark
python scripts/benchmarks/porespy_study.py --volume-size 256 --epochs 300 --use-vgg
python scripts/benchmarks/public_rock_xct.py --volume-size 256 --epochs 300
python scripts/benchmarks/make_paper_outputs.py
# Full multi-seed GPU campaign (see docs/npj-study-design.md)
python scripts/benchmarks/zn13x_synchrotron_study.py --volume-size 256 --epochs 300 --seeds 42 43 44
python scripts/benchmarks/aggregate_campaign.py --inputs outputs --output-dir outputs/campaign_summary
```

---
Expand All @@ -82,20 +86,21 @@ python scripts/benchmarks/make_paper_outputs.py
```text
OpenLSR-X/
├── configs/ # Validated LightningCLI YAML configs (base, porespy, labxct, i13-xct)
├── docs/ # MkDocs user, dataset, benchmark, and API documentation
├── docs/ # MkDocs user, dataset, benchmark, API, and npj study design docs
├── paper/ # JOSS manuscript (paper.md) and bibliography (paper.bib)
├── scripts/
│ ├── benchmarks/ # End-to-end PoreSpy & Rock XCT study runners + LaTeX/figure builder
│ ├── benchmarks/ # PoreSpy, Rock XCT & I13-2 Zn-13X runners + multi-seed campaign aggregator
│ └── hpc/ # SLURM and SGE ('Young') GPU batch templates
├── src/openlsrx/
│ ├── baselines/ # 2D & 3D classical interpolation baselines (Nearest, Trilinear, Tricubic, Lanczos)
│ ├── data/ # Synthetic PoreSpy, TIFF/HDF5 volume loaders, and leakage-free 3D splitting
│ ├── inference/ # 2D-to-3D single-axis and 3-axis orthogonal Fourier stack fusion
│ ├── losses/ # PixelNorm, FP16-safe custom sigmoid, and checkpoint-clean VGGLoss
│ ├── metrics/ # 2D/3D PSNR/SSIM and porous-media petrophysical metrics
│ ├── models/ # ConvBlock, ResidualBlock, Generator, Discriminator, and SRGAN/SRResNet
│ ├── data/ # PoreSpy, Rock XCT, DLS I13-2 Zn-13X loaders & 2.5D leakage-free 3D splitting
│ ├── inference/ # Single-axis, 3-axis Fourier, and epistemic confidence-weighted 3D fusion
│ ├── io/ # OpenImpala 3D TIFF + .inputs deck exporter
│ ├── losses/ # Attenuation conservation, soft-clDice topology, Sobel, FFL, RaGAN & VGGLoss
│ ├── metrics/ # 2D/3D PSNR/SSIM, 3D FSC / 2D FRC, tri-phase petrophysics & 3D Laplace transport
│ ├── models/ # SwinIRGenerator, RRDBGenerator, SRGAN/SRResNet, and native SR3DResNet
│ └── cli.py # Unified `openlsrx` CLI entrypoint
└── tests/ # Unit, analytical phantom, end-to-end Lightning, and YAML schema tests
└── tests/ # 71 unit, analytical physics/spectral/transport, Lightning & CLI tests (>90% cov)
```

---
Expand Down
56 changes: 56 additions & 0 deletions configs/i13-xct/zn13x_2x_swinir_25d_physics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Diamond Light Source I13-2 Physics-Consistent SwinIR-2.5D Configuration (1.625 -> 0.8125 um)
# Combines 3-slice 2.5D context, Windowed Self-Attention (SwinIR), Attenuation Conservation,
# and Soft-clDice Pore-Throat Topology Supervision (zero RGB ImageNet VGG dependency).
seed_everything: 42

trainer:
accelerator: auto
devices: 1
precision: 16-mixed
max_epochs: 300
check_val_every_n_epoch: 10
log_every_n_steps: 20

model:
backbone: swinir
context_slices: 3
out_channels: 48
num_blocks: 4
swin_window_size: 8
swin_num_heads: 4
output_activation: clamp
pixel_loss_type: l1
learning_rate: 2.0e-4
adaptive_size: 8
accumulate_n_batches: 1
super_res_factor: 2
use_adversarial_loss: false
use_vgg_loss: false
conservation_loss_weight: 0.25
conservation_psf_sigma: 0.6
topology_loss_weight: 0.05
gradient_loss_weight: 0.10
ffl_loss_weight: 0.05
sample_id: Zn13X_2x_SwinIR_25D_Physics

data:
class_path: openlsrx.data.PairedVolumeDataModule
init_args:
low_path: data/benchmarks/zn13x_2x_lr.tif
high_path: data/benchmarks/zn13x_2x_hr.tif
scale_factor: 2
context_slices: 3
norm_mode: none
train_frac: 0.70
val_frac: 0.15
guard_slices_lr: 1
train_axes: ["yz"]
test_axis: perpendicular
predict_axis: xy
patch_size_lr: 32
augment: true
batch_size: 4
eval_batch_size: 4
num_workers: 2
pin_memory: true
seed: 42
32 changes: 28 additions & 4 deletions docs/api.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
# API Reference

## Models (2D SRGAN, RRDB-ESRGAN & Native 3D CNN)
## Models (SwinIR-2.5D, RRDB-ESRGAN, SRGAN & Native 3D CNN)

::: openlsrx.models.srgan.SRGAN

::: openlsrx.models.swinir.SwinIRGenerator

::: openlsrx.models.swinir.ResidualSwinBlock

::: openlsrx.models.rrdb.RRDBGenerator

::: openlsrx.models.rrdb.ArbitraryScaleUpsample
Expand All @@ -28,7 +32,15 @@

---

## Losses & Normalizations
## Physics, Topology & Perceptual Losses

::: openlsrx.losses.physics.AttenuationConservationLoss

::: openlsrx.losses.physics.SoftCLDiceLoss

::: openlsrx.losses.physics.SobelGradientLoss

::: openlsrx.losses.physics.FocalFrequencyLoss

::: openlsrx.losses.loss.VGGLoss

Expand All @@ -38,14 +50,16 @@

---

## 3D Orthogonal Inference
## 3D Orthogonal Inference & Epistemic Uncertainty

::: openlsrx.inference.orthogonal.super_resolve_volume_orthogonal

::: openlsrx.inference.orthogonal.super_resolve_volume_single_axis

::: openlsrx.inference.orthogonal.fuse_orthogonal_volumes

::: openlsrx.inference.orthogonal.compute_orthogonal_uncertainty

---

## Data Modules, Synchrotron Registration & Transforms
Expand All @@ -70,14 +84,24 @@

---

## Metrics, Transport Physics & OpenImpala I/O
## Spectral Metrology, Petrophysics, Transport & OpenImpala I/O

::: openlsrx.metrics.spectral.compute_fsc_3d

::: openlsrx.metrics.spectral.compute_frc_2d

::: openlsrx.metrics.spectral.evaluate_spectral_metrics

::: openlsrx.metrics.image.evaluate_image_metrics

::: openlsrx.metrics.petrophysics.evaluate_petrophysical_metrics

::: openlsrx.metrics.petrophysics.evaluate_triphase_zn13x_metrics

::: openlsrx.metrics.transport.evaluate_transport_metrics

::: openlsrx.metrics.transport.compute_tortuosity_uncertainty_bounds

::: openlsrx.metrics.transport.compute_directional_tortuosity

::: openlsrx.metrics.transport.compute_drainage_rmse
Expand Down
64 changes: 64 additions & 0 deletions docs/npj-study-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# *npj Computational Materials* Study Design & Manuscript Blueprint

This guide documents the scientific thesis, methodology equations, ablation matrix, and figure plan for the flagship BASE Laboratory research manuscript built on **OpenLSR-X**, targeting ***npj Computational Materials*** (with *Journal of Synchrotron Radiation* as secondary open-access venue).

---

## Working Title

> **Physics-Consistent 2.5D Orthogonal Super-Resolution with Epistemic Uncertainty for Multi-Magnification Synchrotron Tomography and Pore-Scale Transport**

---

## 150-Word Draft Abstract

Quantitative X-ray computed tomography (XCT) of functional porous materials at synchrotron beamlines is fundamentally constrained by the hardware coupling between spatial resolution, field of view (FOV), and X-ray dose. Existing deep super-resolution methods either suffer from cubic memory scaling in 3D convolutional networks or rely on 2D natural-image generative adversarial priors that violate linear attenuation conservation and hallucinate disconnected pore throats without uncertainty estimates. Here we introduce a physics-consistent 2.5D-to-3D orthogonal super-resolution framework (`OpenLSR-X`) integrating shifted-window self-attention (`SwinIR`), coarse-voxel X-ray attenuation conservation, and differentiable soft-skeleton (`soft-clDice`) pore-throat topology supervision. Exploiting three-axis orthogonal sweeps (`yz`, `xz`, `xy`), our confidence-weighted 3D Fourier synthesis yields zero-overhead voxel-wise epistemic uncertainty maps that localize sub-resolution bottlenecks and propagate directly into calibrated 3D Fickian diffusive tortuosity bounds. Validated on paired multi-magnification Diamond Light Source I13-2 synchrotron tomograms of reactive Zn-doped Zeolite 13X (`2×` and `8×` scale jumps), our approach recovers effective Fourier Shell Correlation resolution, tri-phase `Pore / Na-13X / Zn-13X` ion-exchange stoichiometry, and pore-scale diffusivity at gigavoxel throughput.

---

## Four Core Contributions (Addressing *npj Computational Materials* Criteria)

1. **Zero-Overhead Voxel-Wise Epistemic Uncertainty $\sigma_{\text{ortho}}(\mathbf{x}) \to \tau \pm \Delta\tau$:**
Unlike 3D CNNs or $50\times$-slower conditional diffusion samplers, three-axis orthogonal reconstruction naturally measures directional epistemic disagreement across perpendicular high-resolution planes. Propagating $\hat{V} \pm \sigma_{\text{ortho}}$ through 3D finite-volume Fickian Laplace diffusion (`solve_steady_state_diffusion_axis` / `OpenImpala`) yields **calibrated transport error bars** on unseen low-magnification full-pellet tomograms.
2. **Domain-Grounded Tomographic Physics & Topology Supervision:**
Replaces RGB ImageNet `VGG-19` perceptual priors with `AttenuationConservationLoss` (preserving quantitative 27 keV linear attenuation per coarse scintillator PSF voxel), `SoftCLDiceLoss` (preventing GAN throat pinch-off), and `FocalFrequencyLoss`.
3. **2.5D Windowed Self-Attention (`SwinIR-2.5D`) + Confidence-Fourier Synthesis:**
Provides local through-plane `[z-1, z, z+1]` context at `< 0.3%` parameter overhead over 2D models while avoiding $O(N^3)$ 3D patch stitching, paired with hybrid `confidence_fourier` fusion that suppresses single-axis out-of-plane blur at sharp phase boundaries.
4. **Beamline `3D FSC` Metrology & Tri-Phase Reactive Zeolite Validation (`Zn-13X`):**
Evaluates half-bit and $0.143$ **3D Fourier Shell Correlation (`compute_fsc_3d`)** cutoffs alongside three-phase (`Macropore` / `Na-13X Core` / `Zn-Exchanged Shell`) segmentation fidelity on the open multimodal Diamond I13-2 benchmark (Green et al., *Sci. Data* 2025, `10.1038/s41597-025-04605-9`).

---

## Six-Figure Manuscript Plan

| Figure | Content & Scientific Message | Generated By |
| :--- | :--- | :--- |
| **Fig. 1** | **FOV–Resolution–Dose Trade-off & Architecture:** Diamond I13-2 multi-magnification optics (`2.60 -> 0.325 um`), `2.5D` `SwinIR` trunk, physics/topology losses, and `Confidence-Fourier` + $\sigma_{\text{ortho}}$ pipeline. | Schematic + `quickstart_walkthrough.py` |
| **Fig. 2** | **Paired Synchrotron Visual & Spectral Fidelity (`2×` and `8×`):** Cross-sections (`xy`, `xz`, `yz`), radial **3D FSC curves** with $1/2$-bit cutoffs proving true spatial frequency recovery vs. GAN hallucination. | `compute_fsc_3d` + `zn13x_synchrotron_study.py` |
| **Fig. 3** | **Tri-Phase Reactive `Zn-13X` Ion-Exchange Fidelity:** Three-phase segmentation (`Macropore`, `Na-13X` core, `Zn-13X` shell), radial Zn exchange profile, and attenuation histogram conservation. | `evaluate_triphase_zn13x_metrics` |
| **Fig. 4** | **Orthogonal Epistemic Uncertainty Calibration:** Spatial co-localization of $\sigma_{\text{ortho}}(\mathbf{x})$ with actual HR error $|\hat{V} - V_{\text{HR}}|$ at sub-voxel pore throats (Spearman $\rho$) and outlier suppression under `confidence_fourier` fusion. | `compute_orthogonal_uncertainty` |
| **Fig. 5** | **3D Pore-Scale Diffusive Tortuosity & Capillary Drainage:** Steady-state concentration fields $c(\mathbf{x})$, $D_{\text{eff}}/D_0$ & $\tau$ parity plots with $\pm 1\sigma_{\text{ortho}}$ transport envelopes, and simulated mercury/non-wetting drainage $S_{\text{nwp}}(P_c)$ curves. | `evaluate_transport_metrics` + `compute_tortuosity_uncertainty_bounds` + `OpenImpala` |
| **Fig. 6** | **Compute/VRAM Scaling & Ablation Pareto Frontier:** GPU memory, inference wall-time extrapolated to $2160 \times 2560^2$ full-field synchrotron tomograms, and loss/backbone ablations (`SRGAN` vs `RRDB-RaGAN` vs `SR3DResNet` vs `SwinIR-2.5D + Physics`). | `aggregate_campaign.py` (`campaign_tradeoffs.png` & `campaign_table.tex`) |

---

## Running the Full Isambard-AI Experimental Campaign

1. **Stage Real Diamond I13-2 `Zn-13X` Zenodo Sub-Volumes (`14245751` or raw `169065`–`169068` HDF5):**
```bash
uv run openlsrx download-benchmarks --datasets zn13x_2x zn13x_8x --output-dir data/benchmarks
```
2. **Launch Multi-Seed Array Campaign on Isambard-AI (`GH200`):**
```bash
sbatch scripts/hpc/isambard_ai_array.slurm
```
Or train the flagship configuration directly via CLI:
```bash
uv run openlsrx fit --config configs/i13-xct/zn13x_2x_swinir_25d_physics.yaml
```
3. **Aggregate Multi-Seed Statistics, Booktabs LaTeX Tables & Pareto Figures:**
```bash
uv run python scripts/benchmarks/aggregate_campaign.py \
--inputs results \
--output-dir results/campaign_summary
```
Loading
Loading