Skip to content

feat(v0.3.0): multi-scan gap closing, lazy HDF5 streaming, 4D HTML explorer, ImageD11/NeXus exports, and DINOv2 benchmarks - #18

Merged
jameslehoux merged 1 commit into
mainfrom
feat/v0.3-scale-viz-and-benchmarks
Nov 17, 2026
Merged

jameslehoux merged 1 commit into
mainfrom
feat/v0.3-scale-viz-and-benchmarks

Conversation

@jameslehoux

Copy link
Copy Markdown
Contributor

Summary

Stacked on #17 (feat/v0.2-fracture-tracking-and-pipeline), BraggTrack v0.3.0 adds beamline-scale lazy HDF5 streaming, multi-scan beam-dropout gap closing, crystallographic scattering geometry + ImageD11 / NeXus NXprocess exports, a publication visualization & zero-CDN interactive 4D HTML explorer (braggtrack.viz + braggtrack-report), and a reproducible 5-scenario benchmark suite + ESRF-ID03 case study with real pretrained facebook/dinov2-small.

Phase 1 — Beamline Scale, Gap-Closing & Crystallographic Interoperability

  • Jaqaman Multi-Scan Gap Closing (braggtrack.tracking.gap_closing): GapClosingConfig, GapCloseRecord, and resolve_track_gaps with diffusive displacement scaling ($\Delta r^2 / \Delta s$), bidirectional spurious jump severing, semantic cosine gating, and chronological track ID propagation (--max-gap across track_dataset, detect_events, and run_pipeline).
  • Lazy & Chunked HDF5 Streaming (braggtrack.io.lazy): Context-managed LazyVolume (open_lazy_volume, read_roi, iter_z_chunks) and exact two-pass otsu_threshold_chunked wired into Pass-1 cross-scan Otsu smoothing and ROI spot cropping.
  • Crystallographic Scattering Geometry & Exports (braggtrack.io.crystallography, braggtrack.io.export): DetectorGeometry (spot_to_scattering, annotate_crystallographic_columns) computing $(\omega, 2\theta, \eta, d_{\text{Å}}, \mathbf{Q})$, direct ImageD11 .flt export/round-trip parsing (export_imaged11_flt, read_imaged11_flt), and NeXus NXprocess HDF5 archival (export_nexus_process).

Phase 2 — Publication Visualization & Offline Interactive HTML 4D Explorer

  • braggtrack.viz & braggtrack-report (9th CLI): plot_spot_mip_card, plot_scan_projection_overlay, plot_lineage_dag, plot_kinematics_dashboard, plot_precursor_timeline, and zero-CDN offline generate_html_report (report.html with stdlib-encoded base64 MIP strips, pan/zoom SVG Lineage DAG, filter pills, and synchronized grain inspector), plus --report on braggtrack-run.

Phase 3 — Pretrained facebook/dinov2-small Benchmarks & ESRF-ID03 Case Study

  • 5-Scenario Multi-Trial Benchmark Suite (scripts/benchmark_suite.py, docs/benchmarks.md):
    • crossing_morphology: Pretrained facebook/dinov2-small eliminates the 25.0% Geometry-Only ID-switch failure (0.0% ID-switch rate, 0.9035 cosine stability).
    • intensity_jitter & mosaic_broadening: Maintains 0.9908 and 0.9899 cosine stability vs ~0.00 random hash decorrelation.
    • beam_dropout_gap2: Recovers 100% of staggered dropouts (0.0% fragmentation, 2.0 gap closures) where MockDinoEncoder (cos = 0.0048) fails cosine gating (50% fragmentation).
    • chemomechanical_fracture: 1.00 1 -> 2 split recall, 1.00 2 -> 1 merge recall, and 1.00 top-1 precursor precision (+0.56 contrast above isotropic controls).
  • ESRF-ID03 Operando Case Study: Full 3-scan facebook/dinov2-small CPU run completes in 55.17 s, identifying 2 1 -> 2 split candidates (Track 6 -> 32;34, cos 0.7561; Track 21 -> 36;44, cos 0.6837) and 4 persistent precursor candidates led by Track 14 (persistence=3, peak_score=0.8789).
  • Bug Fix (dino_segment.py, dino.py): Fixed resized ViT 16x16 patch-grid alignment on non-multiple slice shapes (e.g. 250x250 -> 224x224) and honored BRAGGTRACK_DINO_BACKEND when backend="auto".

Phase 4 — JOSS Paper Figures, Metadata Sync & 9th CI Gate

  • Embedded publication figures (docs/figures/benchmark_comparison.png, docs/figures/fracture_lineage_and_precursors.png) in paper.md & README.md; fixed @Finegan2017 and added @Jaqaman2008 in paper.bib; synced CITATION.cff, CONTRIBUTING.md, docs/architecture.md, and CHANGELOG.md to 0.3.0.
  • Added 9th CI acceptance gate (scripts/check_report_and_export_acceptance.py) into scripts/ci_report.py.

Verification

  • ruff check braggtrack/ tests/ scripts/ & ruff format --check braggtrack/ tests/ scripts/ — clean
  • python scripts/pre_pr_check.py — passed
  • BRAGGTRACK_DINO_BACKEND=mock python scripts/ci_report.py — 229 unit tests + all 9 acceptance gates PASS (overall=PASS)
  • jupyter nbconvert --to notebook --execute — both braggtrack_demo.ipynb and dino_segmentation_comparison.ipynb executed cleanly

…plorer, ImageD11/NeXus exports, and DINOv2 benchmarks
@jameslehoux
jameslehoux changed the base branch from feat/v0.2-fracture-tracking-and-pipeline to main November 17, 2026 22:11
@jameslehoux
jameslehoux merged commit 5f2764d into main Nov 17, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant