feat(memory): Layout + TensorView — zero-copy views over Storage, decoding get(), materialize() as the only copy point (SKEEP-003 P2, S1.3)

[michalharakal]

Summary

SKEEP-003 slice S1.3 (milestone M1 #1002, PRD M1-F6): Layout + TensorView — the type a kernel actually receives (§4.2, rules 4–6). A view never owns bytes; every slice / transpose / unsqueeze is another view over the same Storage; materialize() is the only copy point.

• Layout: strides in elements — or in blocks for a packed format — plus offsetElements/offsetBytes, isRowMajor, isContiguous (unit extents ignored), indexOf/byteOffsetOf, and the metadata-only narrow / transpose / unsqueeze / squeeze. Layout.rowMajor(shape, format) and Layout.blocked(shape, blockSize, bytesPerBlock) — the latter is what lets a packed weight be sliced and transposed without touching bytes (rule 5; the packed-transpose trick becomes "a layout that says transposed over an encoding that says blocked").
• TensorView(shape, format, layout, storage, id): narrow/transpose/unsqueeze/squeeze return views over the same storage (ids derive as model.w[1..3)), get() returns the decoded logical value for packed encodings and never a raw byte (rule 4 — the Quantized matmul dispatch skips rank-1 (single-token decode) activations, falls through to broken matmulGeneric #993/Quantized matmul dispatch silently falls back to a broken generic kernel for rank>2 attention projections (ClassCastException: Byte cannot be cast to Float) #991 class becomes impossible by construction), set() is refused on packed or read-only views, toFloatArray() is the reference materialization, and materialize(format, scope) is the single copy point (rule 6), allocating in the target Scope. Narrowing the block axis of a packed view must align to whole blocks.
• BlockDecoder + PackedBlockDecoder: the bridge from today's PackedBlockStorage implementations (Q4_0…Q8_0, Q4_K/Q5_K/Q6_K, ternary) into views; M2's Encoding descriptors will implement the same interface.
• Tests: LayoutTest (strides/offsets/contiguity, narrow keeps strides, transpose is an involution, unsqueeze↔squeeze, blocked geometry) and TensorViewTest (dense read/write, views sharing storage and bytes, a Q8_0 view decoding identically to PackedBlockStorage.toFloatArray(), whole-block slicing of a Q4_K weight, materialize as a real copy incl. packed→dense, views over closed storage refused). JVM 67/67, linuxX64 58/58 sk.ainet.lang.memory.* tests.
• BCV: lang-core jvm dump regenerated (additions only). Nothing consumes views yet — the TensorData façades are [S1.4a] P2: dense TensorData implementations become façades over TensorView (bit-identical) #1023/[S1.4b] P2: packed TensorData implementations (Q4_0…Q8_0, Q4_K/Q5_K/Q6_K, ternary, TurboQuant) become façades over TensorView #1024.

Test plan

Full local gate (scripts/pr-gate.sh, JDK 25) — all legs passed; results in the first comment.

Closes #1022

🤖 Generated with Claude Code

[michalharakal]

Local gate scripts/pr-gate.sh (JDK 25) on 9a2e694:

=== pr-gate: JVM tests ===                                      BUILD SUCCESSFUL
=== pr-gate: apiCheck ===                                       BUILD SUCCESSFUL
=== pr-gate: verifyNpmPins jsTest wasmJsTest wasmWasiTest ===   BUILD SUCCESSFUL in 2m 47s
=== pr-gate: linuxX64Test ===                                   BUILD SUCCESSFUL in 1m 21s
=== pr-gate: assemble ===                                       BUILD SUCCESSFUL in 2m 9s
=== pr-gate: :skainet-test:skainet-test-java:test ===           BUILD SUCCESSFUL in 21s
pr-gate: all legs passed.

Targeted: JVM 67/67 and linuxX64 58/58 sk.ainet.lang.memory.* tests.

[github-actions]

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

• Operator documentation: docs/modules/operators/_generated_/
• JSON schema output: operators.json

Artifacts:

• Download the documentation-preview-1066 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.