Skip to content

fix: heap buffer overflow on ndarray shapes whose size overflows - #281

Merged
embray merged 2 commits into
asdf-format:mainfrom
cruzzil:fix-ndarray-size-overflow
Oct 22, 2026
Merged

embray merged 2 commits into
asdf-format:mainfrom
cruzzil:fix-ndarray-size-overflow

Conversation

@cruzzil

@cruzzil cruzzil commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Description

asdf_ndarray_size and asdf_ndarray_nbytes can wrap. With shape: [2**63 + 1] int16 read as int32, the size check sees 2 bytes, allocates 4, and convert_int16_to_int32 writes 2**63+1 elements (ASan heap-buffer-overflow write at ndarray_convert.c:409 via asdf_ndarray_read_all). The check from #277 doesn't catch it because nbytes wraps too.

check_bounds also computes origin + shape, which wraps: an origin of UINT64_MAX made asdf_ndarray_read_tile_ndim return ASDF_NDARRAY_OK and read out of bounds. Same for asdf_ndarray_read_at.

Fix: both size functions saturate at UINT64_MAX (a zero dimension still gives 0), and check_bounds compares without adding.

Tests: ndarray_read_size_overflow (new fixture) and ndarray_read_tile_bounds_overflow; both fail before the fix.

AI Disclosure

An LLM detected this whilst reviewing the Rust port for issues, and provided the fix.

asdf_ndarray_size and asdf_ndarray_nbytes wrapped, so a file could pass
the data size check with a tiny byte count while the conversion loop
wrote the true element count into a buffer sized from the wrapped one.
Saturate both at UINT64_MAX instead.

check_bounds had the same problem with origin + shape, letting a tile or
asdf_ndarray_read_at index read outside the array.
@cruzzil
cruzzil force-pushed the fix-ndarray-size-overflow branch from 27fc575 to 1e40ea7 Compare October 18, 2026 14:02
@embray

embray commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

Technically such a file is not even valid ASDF, as the spec states (somewhat poorly phrased though):

Integers represented as string literals in the ASDF tree must be within the valid range for a 64-bit signed integer.

So an ASDF file with such a shape parameter should probably be rejected outright. I'm OK though with accepting it with a warning. Indeed the code should still do something sensible in that case.

@embray embray added the bug Something isn't working label Oct 22, 2026
@embray embray added this to the v0.3 milestone Oct 22, 2026
@embray

embray commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

LGTM.

@embray
embray merged commit 7a158d7 into asdf-format:main Oct 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants