Stream reads over HTTP Range requests and from Blobs

[OBrezhniev]

Stream reads over HTTP Range requests and from Blobs

Stacked on #138 (feature/direct_rw_optimization) — this PR
shows only the two streaming commits.

Summary

readExisting with a URL previously buffered the entire response in memory
(the browser path fetched any string whole; Node had no URL support at all).
That defeated chunked zkey streaming for browser provers: the full zkey was
resident before the first section read.

Now a URL opens a range-backed read-only fd:

• Open probe: one GET with Range: bytes=0-0. A 206 gives range
  support, the total size (from Content-Range), and a validator (strong
  ETag, else Last-Modified) in a single round trip. Anything else falls
  back to the historical buffer-it-all behavior, reusing the probe's own
  response body (no second fetch).
• Large reads (≥ page size) become one Range request each, streaming the
  response body directly into the caller's buffer (typed array or BigBuffer)
  — no second full-size copy. With snarkjs' chunked section readers, the
  zkey's point sections are never resident as a whole.
• Small header reads (binfile magic, section table scans) coalesce into
  page-aligned cached ranges (LRU) instead of issuing one HTTP request per
  4-byte field. Pages are capped at 64 KiB regardless of the caller's
  disk-tuned pageSize hints — snarkjs passes 8 MiB pages, which would
  otherwise turn a 4-byte header read into a whole-file range request.
• Consistency: every range request carries If-Range with the validator
  captured at open. If the remote file changes mid-read, the server answers
  200 and the read throws instead of silently mixing chunks of two file
  versions.
• Blob/File backend: readExisting(blob) streams via
  blob.slice(pos, pos+len).arrayBuffer() — a browser <input type="file">
  zkey reads from disk chunk-by-chunk with the same bounded footprint as the
  Node file backend (pages capped at 1 MiB).

Both backends share one implementation (src/rangefile.js), parameterized by
a positioned readRangeInto(dst, dstOffset, pos, len) primitive. Write
operations throw; the transport is strictly read-only.

Server requirements (CORS deployments)

• Allow the Range request header; expose Content-Range, ETag,
  Accept-Ranges.
• Serve without Content-Encoding (ranges address encoded bytes; zkey
  content is high-entropy anyway). S3/GCS/CloudFront work out of the box.

Validation

npm test: 31 passing, including 10 new tests against local HTTP servers:
range streaming with request accounting (no full-file GET ever issued),
single-request page-cache reuse, sequential unpositioned reads, the
no-range-support fallback (exactly one request), mid-session file-change
rejection via If-Range, out-of-bounds/write rejection, string reads through
the page cache, the disk-hint page-size cap, and Blob positioned reads.

Downstream, binfileutils gained URL round-trip tests (header scan + full and
partial section reads in both server modes) and snarkjs an e2e test proving
groth16 with the zkey served by a local range server — proof verifies, every
request is a Range request, and no single response carries the whole zkey.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wo6AVSAwvL9mHTpvnREZPR