Semantic 4D kinematics, multi-scan beam-dropout gap closing, grain lineage (1 -> 2+ fracture splits & 2+ -> 1 coalescence merges), and persistent fracture precursor tracking for operando X-ray diffraction using classical 3D crystallography and foundation vision transformers.
-
Beamline-scale NeXus/HDF5 I/O,
darfix/BLISS auto-calibration & operando sync (braggtrack.io) — Auto-discovers 3D detector cubes acrossdarfixNXdata(@default/@signal/@axes), ESRF-ID03 LIMA files (/entry_0000/ESRF-ID03/pco_nf/data), ESRF BLISSinstrument/positioners, and arbitrary HDF5 trees (auto_discover=True). Streamsz-slabs viaLazyVolume, synchronizes external potentiostat / load-frame logs (CSV/TSVand Biologic EC-Lab.mpt, regularized$\mathrm{d}Q/\mathrm{d}V$ ,$\partial\varepsilon/\partial V$ ), maps transmission geometry(2theta, eta, d, |Q|)viaDetectorGeometry, and exports directly toImageD11(.flt) and NeXusNXprocessHDF5 (.nxs). -
Stabilised 3D Bragg reflection segmentation (
braggtrack.segmentation) — Streaming Otsu foreground masking with cross-scan rolling-median smoothing (smooth_thresholds), 3D LoG peak enhancement, robusth-maxima seeding, 3D watershed, nearby-basin merging, analytical Poisson + quantization centroid uncertainties (centroid_*_err), and physical or voxel-space covariance/eigenvalue extraction. Includes an experimental batched dense patch-feature + HDBSCAN slice-stitching segmenter (--method dino). -
Batched multi-view foundation vision descriptors (
braggtrack.semantic) — Orthogonal$(\mu, \chi, d)$ maximum-intensity projections (MIPs) encoded in batched forward passes (embed_batch,extract_patch_features_batch) with ungatedfacebook/dinov2-smallby default (zero HuggingFace login required), drop-in support for DINOv3 (facebook/dinov3-vitb16-pretrain-lvd1689m), or a deterministic zero-GPUmockbackend for CI. -
4D grain kinematics, uncertainty propagation, split/merge lineage & precursors (
braggtrack.tracking) — Vectorised geometry + semantic Hungarian association with runner-upassignment_margin, Jaqaman-style multi-scan gap closing (--max-gap), local strain ($\Delta d / d_0$ ), ancestor-referencedlineage_strainacross splits, propagated$\sigma_\varepsilon$ , causal angular/lattice velocities (rate_mu,rate_chi,rate_d,strain_rate), intensity-partitioned1 -> 2+split and2+ -> 1merge DAG edges, and persistent multi-scan fracture precursor ranking. -
Online causal streaming & autonomous DFXM "Scout-and-Zoom" (
braggtrack.autonomous,braggtrack-watch) — Strictly causal trailing-window Otsu streaming tracker (CausalStreamingTracker) with$O(1)$ -volume-RAM cached ROI precursors, velocity-extrapolated diffractometer setpoint prediction during rapid phase transitions (e.g., Ni92H2 -> H3collapse),bliss_dfxm_zoom.v1JSONL trigger emission (zoom_triggers.jsonl), and offline causal replay benchmarking against naive baselines. -
Publication figures, operando
$\mathrm{d}Q/\mathrm{d}V$ dashboards & offline HTML 4D Explorer (braggtrack.viz,braggtrack-report) — Generates Okabe–Ito styled lineage DAGs, strain/misorientation dashboards, 3-panel operando electrochemistry + fracture synchronization plots, precursor heatmaps, 3-view MIP cards, and a single-file interactivereport.htmlthat opens offline on air-gapped beamline networks. -
One-command reproducible pipeline (
braggtrack-run) — Runs validation, operando stimulus sync, segmentation, semantic embedding, tracking, kinematics, fracture event analysis, HTML reporting, andImageD11/NeXus export end-to-end, emitting SHA-256 input checksums, stage timings, andrun_manifest.json.
# Core package (numpy, scipy, scikit-image, scikit-learn, networkx, h5py)
pip install -e .
# With PyTorch + HuggingFace Transformers for real DINOv2 / DINOv3 backbones
pip install -e ".[torch]"
# With Jupyter, Matplotlib, and Pandas for interactive notebooks & static figures
pip install -e ".[notebook]"
# Full developer environment (torch + notebook + pytest + ruff)
pip install -e ".[dev]"BraggTrack ships with three real ESRF ID03 operando Ni-rich cathode scans under data/sample_operando/ (100 x 250 x 250 uint16 cubes). Omit the dataset path to run on the bundled scans automatically, or pass your own directory of scanXXXX/ folders:
braggtrack-run --outdir artifacts/pipeline \
--max-gap 1 \
--report \
--export-nexus artifacts/pipeline/results.nxs \
--export-imaged11 artifacts/pipeline/peaks.fltProduces:
artifacts/pipeline/validation_summary.jsonartifacts/pipeline/segmentation/(features.csv,labels.npz,summary.json, QC notebook)artifacts/pipeline/embedding/(embeddings.npz,embedding_manifest.json)artifacts/pipeline/tracking/(tracks.csv,kinematics.csv,kinematics_summary.json,tracking_summary.json, QC notebook)artifacts/pipeline/events/(events.csv,events.json,precursors.csv,events_summary.json)artifacts/pipeline/report/(report.htmlinteractive 4D Explorer +figures/*.png)artifacts/pipeline/results.nxs&artifacts/pipeline/peaks.fltartifacts/pipeline/run_manifest.json(SHA-256 input checksums, git SHA, config, stage timings)
| CLI Entry Point | Module Equivalent | Purpose |
|---|---|---|
braggtrack-inspect |
python -m braggtrack.cli.inspect_datasets |
Discover scan* folders, primary 3D HDF5 dataset paths, and metadata |
braggtrack-validate |
python -m braggtrack.cli.validate_dataset |
Validate sequence ordering, monotonic timestamps, and axis contracts |
braggtrack-segment-synthetic |
python -m braggtrack.cli.segment_synthetic |
Run fast synthetic 3D segmentation smoke test |
braggtrack-segment-dataset |
python -m braggtrack.cli.segment_dataset |
Segment 3D Bragg spots via chunked Otsu + smoothed thresholds & optional calibration |
braggtrack-embed-dataset |
python -m braggtrack.cli.embed_dataset |
Stream lazy 3D spot ROIs & compute batched MIP embeddings (auto, mock, torch) |
braggtrack-track-dataset |
python -m braggtrack.cli.track_dataset |
Build trajectory DAG (--max-gap), export tracks.csv, kinematics.csv, .flt, .nxs |
braggtrack-events |
python -m braggtrack.cli.detect_events |
Resolve 1 -> 2+ splits, 2+ -> 1 merges, and batched lazy-ROI fracture precursors |
braggtrack-report |
python -m braggtrack.cli.report_dataset |
Render publication PNG/PDF figures (--operando-log) and self-contained report.html |
braggtrack-watch |
python -m braggtrack.cli.watch_autonomous |
Live causal streaming watcher (--mode watch) & autonomous DFXM replay (--mode replay) |
braggtrack-run |
python -m braggtrack.cli.run_pipeline |
Execute the full pipeline with optional --config, --operando-log, --report, and exports |
Cropped detector cubes store integer voxel indices [0..N). For physically meaningful lattice strain AxisCalibration JSON file when segmenting:
{
"mu_origin": 12.50,
"mu_step": 0.02,
"chi_origin": -3.20,
"chi_step": 0.015,
"d_origin": 2.042,
"d_step": -0.00012
}For transmission 3DXRD / ImageD11 indexing workflows, supply a DetectorGeometry JSON file (--detector-json) to annotate omega_deg, two_theta_deg, eta_deg, d_spacing_angstrom, and (qx, qy, qz, q_mag_inv_angstrom) and export .flt columnfiles:
{
"distance_mm": 150.0,
"pixel_size_y_mm": 0.075,
"pixel_size_x_mm": 0.075,
"beam_center_y_px": 1024.0,
"beam_center_x_px": 1024.0,
"wavelength_angstrom": 0.2898,
"omega_origin_deg": 0.0,
"omega_step_deg": 0.05
}- Default (
facebook/dinov2-small): Public and ungated on HuggingFace — works out of the box withpip install -e ".[torch]"without tokens or license gates. - DINOv3 (
facebook/dinov3-vitb16-pretrain-lvd1689m): Supported via--model/--dino-modelafter accepting Meta's license on HuggingFace and settingHF_TOKEN. - Offline / CI (
mock): Deterministic SHA-256 multi-view projection encoder requiring zero GPU or network access (--backend mockorBRAGGTRACK_DINO_BACKEND=mock).
See docs/benchmarks.md for the full 5-scenario benchmark evaluation (scripts/benchmark_suite.py) comparing Geometry-Only, Mock-Hash, and Real facebook/dinov2-small alongside the ESRF-ID03 case study.
notebooks/braggtrack_demo.ipynb— End-to-end tutorial on the bundled ESRF-ID03 scans: discovery, lazy/chunked segmentation, physics & semantic tracking, multi-scan gap closing, per-grain kinematics,braggtrack.vizlineage & precursor plots, and crystallographic geometry mapping.notebooks/dino_segmentation_comparison.ipynb— Side-by-side comparison of stabilised classical 3D watershed vs. experimental DINO patch-clustering segmentation.
# Lint & format check
ruff check braggtrack/ tests/ scripts/
ruff format --check braggtrack/ tests/ scripts/
# Unit test suite with coverage
pytest tests/ -v --cov=braggtrack
# Full CI gate runner (unit tests + all 9 acceptance gates)
python scripts/ci_report.py
# Quantitative 5-scenario benchmark suite (add --include-torch for real DINOv2)
python scripts/benchmark_suite.py --trials 5
# Pre-PR hygiene check
python scripts/pre_pr_check.pyMissingH5DependencyErroror custom HDF5 dataset paths:h5pyis installed by default withpip install -e ..load_primary_volume()andLazyVolumeautomatically inspect standard NeXus paths,/entry_0000/ESRF-ID03/pco_nf/data, and fall back to the largest 3D numeric dataset in the file. Pass--allow-synthetic-fallbackonly when testing schema plumbing on empty/metadata-only HDF5 stubs.401 Unauthorized/ gated HuggingFace model error: You passed a gated model ID (such asfacebook/dinov3-*) without authenticating. Either omit--modelto use the ungated default (facebook/dinov2-small), or accept the model license on HuggingFace and exportHF_TOKEN.- Unphysical strain percentages in
kinematics.csv: Check"calibrated_axes"inkinematics_summary.json. Whenfalse,centroid_dis measured in cropped detector pixels (yindex) rather than Angstroms — pass--calibration-json(or--detector-json) to compute physical crystallographic quantities.
braggtrack/— Installable Python library (io,segmentation,semantic,tracking,autonomous,viz,cli).data/sample_operando/— Bundled ESRF-ID03 operando test scans (scan0001–scan0003).docs/— Ecosystem Positioning & Autonomous DFXM Guide, Architecture, Benchmarks, Coding Practices, and publication figures (docs/figures/).scripts/— 9 acceptance gates, 5-scenario benchmark suite (benchmark_suite.py), semantic α/β ablation (ablation_semantic.py), and CI runner (ci_report.py).artifacts/— Generated pipeline outputs (gitignored; never commit).
See CONTRIBUTING.md for contribution guidelines and paper.md for the manuscript overview.

