Block-granular reads for Caterva2, and what they needed - #702

Merged

FrancescAlted merged 32 commits into

mainfrom

cat2-block-granularity

Aug 20, 2026

Commits on Aug 17, 2026

Add the plan for block-granular Caterva2 reads

FrancescAlted
and
claude
committed
c15bda7
Pool the HTTP client C2Array requests go through

FrancescAlted
and
claude
committed
7bb0eff
Lift the frame reading out of FsspecNDSource into ByteRangeNDSource

FrancescAlted
and
claude
committed
422ba4b
Read a C2Array's blocks over HTTP ranges

FrancescAlted
and
claude
committed
d541eb3
Ask a Caterva2 subscriber for a whole wave of ranges at once

FrancescAlted
and
claude
committed
b021104
Record what the block-granularity work landed as, and what it found

FrancescAlted
and
claude
committed
1b2dd44
Keep C2NDSource's max_ranges with the rest of its class body
FrancescAlted
committed
b868c53

Commits on Aug 18, 2026

Measure the Caterva2 block path the way the fsspec one is measured

FrancescAlted
and
claude
committed
e8d9ba1
Say when a dataset has no chunks to fetch, rather than tracebacking

FrancescAlted
and
claude
committed
f9f755c
The api/chunk gap is fixed; note what a container leaf still lacks

FrancescAlted
and
claude
committed
4fe79af
Open a frame in two requests instead of four

FrancescAlted
and
claude
committed
944ca80
Notice a remote array that was replaced under a proxy's cache

FrancescAlted
and
claude
committed
b2918c8
Record the whole of the block-granularity work in the plan

FrancescAlted
and
claude
committed
7b9aa62
Plan byte ranges for container leaves

FrancescAlted
and
claude
committed
b6b7303
Say where a leaf's frame lies inside a .b2z

FrancescAlted
and
claude
committed
e0cc7da
Pin that a container leaf reads its blocks like any other array

FrancescAlted
and
claude
committed
460b060
Record what container-leaf ranges landed as

FrancescAlted
and
claude
committed
830f370
Cross off container-leaf ranges in the block-granularity plan

FrancescAlted
and
claude
committed
acf2d24
Shim to reproduce itertools.batched for python 3.11
FrancescAlted
committed
438d2fc

Commits on Aug 19, 2026

Skip the block tests on wasm, where a socket cannot listen

FrancescAlted
and
claude
committed
3c24a11
Read a frame's chunk offsets when a chunk is first asked for

FrancescAlted
and
claude
committed
aeb5a1b
Keep where the chunks and blocks are in the cache

FrancescAlted
and
claude
committed
9937188
Count the reads an S3 block fetch makes now, and pin the kept index

FrancescAlted
and
claude
committed
95850a8
Stop the subscriber stand-in idling half a second per test

FrancescAlted
and
claude
committed
eb7fd15
Check the cache's stamp, and the bytes a range answer actually carries

FrancescAlted
and
claude
committed
31a84fc

Commits on Aug 20, 2026

Empty a cache whose remote was replaced, rather than serve it

FrancescAlted
and
claude
committed
4a1b927
Give what a proxy reads from a module of its own

FrancescAlted
and
claude
committed
0753fa7
Leave a cache a proxy fits exactly as it found it

FrancescAlted
and
claude
committed
7254129
Read a .b2z leaf only where the zip kept it whole

FrancescAlted
and
claude
committed
410f89f
Keep the layouts of a cache whose offsets do not fit, and keep them once

FrancescAlted
and
claude
committed
92b02b3
Let the source batch the layout reads it already batches

FrancescAlted
and
claude
committed
8fa4b8f
Read whole chunks when a subscriber stops answering ranges

FrancescAlted
and
claude
committed
f81016b