Skip to content

fix: check H5AD lock before overwrite - #2569

Closed
stanbot8 wants to merge 1 commit into
scverse:mainfrom
stanbot8:agent/issue-522-safe-h5ad-overwrite
Closed

fix: check H5AD lock before overwrite#2569
stanbot8 wants to merge 1 commit into
scverse:mainfrom
stanbot8:agent/issue-522-safe-h5ad-overwrite

Conversation

@stanbot8

@stanbot8 stanbot8 commented Jul 23, 2026

Copy link
Copy Markdown

Summary

HDF5 can truncate a live H5AD target before it reports a lock failure (h5py issue 1864, HDF5 issue HDFFV-11230). This change opens an existing HDF5 target with r+ before the truncate-mode open. An active reader blocks the overwrite before truncation. Closes #522.

Design

The check changes less code. It does not need a temporary copy.

The implementation changes src/anndata/_io/h5ad.py. The design follows the reported truncation reproducer, the requested overwrite behavior, and the HDF5 lock analysis.

Limitations

  • A reader can open after the check, so the reader-open race remains.
  • HDF5 file locking must be enabled.
  • Writes remain non-atomic.
  • Concurrent writers are not serialized.

Tests

  • The Windows baseline changed a 20,480-byte file to an empty file.
  • The Scanpy reproducer preserved the file and the reader.
  • Current Windows and Linux dependencies passed.
  • Minimum Linux dependencies passed.
  • Three readers survived ten blocked writes. A retry passed after they closed.
  • All 11 CI matrix test commands passed. The report upload failed in the fork.

The benchmark and codespell checks passed.

Commands
pytest -q tests/test_readwrite.py -k 'test_write_h5ad_open_elsewhere'

@stanbot8
stanbot8 force-pushed the agent/issue-522-safe-h5ad-overwrite branch from f37debe to 8c23e3e Compare July 23, 2026 20:02
@stanbot8
stanbot8 marked this pull request as ready for review July 23, 2026 20:04
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.87%. Comparing base (b2e6177) to head (8c23e3e).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2569      +/-   ##
==========================================
- Coverage   87.81%   85.87%   -1.95%     
==========================================
  Files          49       49              
  Lines        7799     7802       +3     
==========================================
- Hits         6849     6700     -149     
- Misses        950     1102     +152     
Files with missing lines Coverage Δ
src/anndata/_io/h5ad.py 93.39% <100.00%> (+0.09%) ⬆️

... and 8 files with indirect coverage changes

@stanbot8

Copy link
Copy Markdown
Author

Superseded by #2570. The replacement writes a complete temporary file before it replaces the target. This design removes the reader-open race that remains in this preflight check.

@stanbot8 stanbot8 closed this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Opening file with "w" while it's open in backed mode elsewhere still deletes file contents

1 participant