Skip to content

feat(compile-hlo): array-free external constants via BufferHandle.Floats, Long-safe sizing (#1247) - #1251

Merged
michalharakal merged 1 commit into
feat/1247-graph-constant-extractionfrom
feat/1247-array-free-external-constants
Sep 2, 2026
Merged

michalharakal merged 1 commit into
feat/1247-graph-constant-extractionfrom
feat/1247-array-free-external-constants

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Part of #1247. Stacked on #1250 (needs the aliased FloatArray reaching the converter) — after #1250 merges, this retargets to develop.

Root cause found while implementing: the gemma3n tied embedding is 262144 × 2048 FP32 = Int.MAX_VALUE + 1 bytes — one byte over the JVM single-array ceiling. ByteArray(count * 4) overflowed to a negative size, and the resulting NegativeArraySizeException escaped into the converter's generic catch, producing the misleading Unsupported op 'weight' … Known names: […] comment the issue quotes. No widening of the byte round-trip could ever emit this tensor; the external path must be array-free.

  • New sealed variant BufferHandle.Floats(FloatArray) (lang-core): FP32 logical values as a primitive array — up to 2 Gi elements / 8 GiB logical, read-only, typically aliasing the live module weight. DefaultBufferResolver reads it through a chunked little-endian byte view; describe labels it.
  • External FP32 constants in ConstantOperationsConverter hand the aliased array to the packager as Floats with no byte serialization — zero copies from module weight to archive. Non-FP32 / under-filled values keep the padded byte path.
  • IrpaWriter streams Floats little-endian in 64 MiB chunks — byte-identical output to the eager Owned serialization (parity-tested).
  • elementCountFromShape folds in Long; oversized single-buffer serialization throws ConstantTooLargeException with the remediation — deliberately not IllegalArgumentException, so the inline-emission fallback can't catch it and try to format a multi-GiB tensor as text.

Testing: new BigConstantMaterializationTest (Long fold incl. the exact embedding count, overflow refusal, Floats identity-aliasing with zero serialization, tied-weight → one util.global) + IrpaFloatsParityTest. lang-core / compile-hlo / iree-params suites green; apiDumps updated.

Note for consumers: exhaustive whens over BufferHandle gain a branch — in-repo consumers are updated here; SKaiNET-transformers' Gemma3nExportHarness.writeSafetensors needs a Floats branch (chunked bf16 streaming) once this ships.

…ats, Long-safe sizing (#1247)

The gemma3n tied embedding is 262144 x 2048 FP32 = Int.MAX_VALUE + 1 bytes:
one byte over the JVM single-array ceiling, so no widening of the previous
ByteArray round-trip could ever emit it. External FP32 constants now hand
the aliased FloatArray straight to the packager as BufferHandle.Floats —
zero copies from module weight to .irpa — with IrpaWriter streaming the
values little-endian in 64 MiB chunks and DefaultBufferResolver reading
through a chunked byte view.

Constant element counts fold in Long (an Int fold of the embedding's byte
count went negative, surfacing as NegativeArraySizeException inside the
serializer and mistaken for a registry miss - the misleading "Unsupported
op 'weight' ... Known names" comment). Oversized single-buffer
serialization now throws ConstantTooLargeException with the remediation;
deliberately not an IllegalArgumentException, so the converter's
inline-emission fallback cannot catch it and try to format a multi-GiB
tensor as text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant