[timbeccue]

The goal of this PR is to speed up smartstack previews by decreasing stackframe reduction times.

Opening the five master calibrations for a full-frame stackframe takes about 19 s on core1 (measured on core1.ogg with the sq30 set), and about 15 s of that is gzip decompression of the same .fz calibration frames, repeated for every member of the same stack.

To address this, we decompress the .fz calibration frames when they are cached by the download worker, and modify open_fits_file to use the decompressed versions if they exist.

Changes:

• The site download worker funpacks each cached .fz calibration once, writing name.fits beside name.fits.fz.
• open_fits_file reads the uncompressed copy when it exists (which will only be the case for cached calibrations), returning the same data unpack produces. If the copy cannot be read it logs a warning and decompresses as before.
• The regular cache pruning removes the copy along with the master.

Nothing changes outside the site cache: central BANZAI never has these files, and the database still names the compressed file.

Release

Bumps the version to 1.41.0. #498 shipped separately as 1.40.1, so the 1.41.0 changelog entry covers only this change.

[Copilot]

🟡 Changes recommended

Failed copy writes can leave temporary files consuming disk space and blocking subsequent cache refreshes.

1 open issue

• Clean up partial temporary files after failed decompression · New

What changed in this PR

Speeds up site stackframe reduction by caching uncompressed calibration files and avoiding repeated decompression.

Changes:

• Creates and prunes uncompressed copies alongside cached masters.
• Prefers uncompressed copies for non-raw frames, with fallback on read failure.
• Adds regression tests, updates documentation, and bumps the release to 1.41.0.

File	Description
uv.lock	Updates package version to 1.41.0.
pyproject.toml	Declares version 1.41.0.
docs/​smartstacking_architecture.md	Documents uncompressed cache behavior.
CHANGES.md	Records caching improvements and the #498 fix.
banzai/​utils/​fits_utils.py	Reads uncompressed siblings with fallback.
banzai/​tests/​test_fits_utils.py	Tests sibling selection, equivalence, and fallback.
banzai/​tests/​test_download_worker.py	Tests copy creation, failure handling, and deletion.
banzai/​cache/​download_worker.py	Creates and prunes uncompressed calibration copies.

_{Review effort: Balanced}

[cmccully]

This makes me a little nervous because this seems problematic for prod (we don't have enough disk space). I think it's fine as is, but it's something to watch for in the prod, especially stacking, deployment.

[timbeccue]

This makes me a little nervous because this seems problematic for prod (we don't have enough disk space). I think it's fine as is, but it's something to watch for in the prod, especially stacking, deployment.

#502 should help reduce the size. Brings the cache to under 60GB per site on average. I'll keep an eye on the overall disk usage though.