Skip to content

feat(types): DType carries its KClass witness; dtype-first storage constructors and loaders (SKEEP-003 P0, S0.2a) - #1050

Merged
michalharakal merged 1 commit into
developfrom
feature/1007-dtype-witness
Aug 22, 2026
Merged

michalharakal merged 1 commit into
developfrom
feature/1007-dtype-witness

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

SKEEP-003 slice S0.2a (milestone M0 #1001, decision #13 — merge LogicalDType into one sealed DType): DType carries its own witness, and every storage descriptor / loader can be built from a DType instead of a LogicalDType. Purely additive; LogicalDType is untouched (its deprecation is the separate slice #1014).

  • sk.ainet.lang.types.DType: witness: KClass<out DType> (each of the 14 objects narrows to its own class, e.g. FP32.witness: KClass<FP32>), isSigned (false only for UInt*), sizeInBytes, companion entries (storage-layer order, index-aligned with LogicalDType.entries), fromWitness(kclass) / fromWitnessOrNull(kclass) (identity-keyed map — no name-based reflection, JS/Wasm-safe). fromWitness(tensor.dtype) is how M0's Format slice turns a Tensor.dtype into a DType.
  • Additive dtype-first constructors/overloads (primary constructors unchanged): TensorStorage(shape, dtype, encoding, buffer, …), TensorStorageFactory.fromRawBytes / fromRawBytesOwned / fileBacked(… dtype: DType …), PackedBlockStorage.toTensorStorage(dtype, …); toTensorData dispatches on storage.dtype with identical semantics.
  • Internal call sites moved to DType: TensorStorageFactory.fromFloatArray/fromIntArray, DefaultKvCacheStore, TurboQuantKvCacheStore, StreamingGGUFReader (ggmlTypeToDType; quantized → FP32, Opaque fallback kept), StreamingSafeTensorsReader (BOOL → UInt8, UNKNOWN → Int8 kept).
  • Tests: DTypeWitnessTest (entries/order, witness identity, fromWitness inverse + rejection, signedness/width vs the storage enum, enum-like when), dtype twins in TensorStorageContractTest, StorageIntegrationTest asserts storage.dtype === FP32 for F32 and Q8_0 GGUF tensors.
  • BCV: lang-core jvm dump regenerated (additions only).

Test plan

Full local gate (scripts/pr-gate.sh, JDK 25) + --golden; results in the first comment.

Closes #1007

🤖 Generated with Claude Code

…nstructors and loaders (SKEEP-003 P0)

Decision #13 merges LogicalDType into one sealed DType. Second step
(milestone M0, #1001): DType becomes self-describing and every storage
descriptor can be built from it, additively.

- DType: witness: KClass<out DType> (each of the 14 objects narrows to
  its own class), isSigned (false for UInt*), sizeInBytes; companion
  entries (storage-layer order, index-aligned with LogicalDType.entries),
  fromWitness / fromWitnessOrNull (identity-keyed, no name-based
  reflection). Companion collections are lazy: with default members on
  the interface, initializing an object also initializes the interface
  statics on the JVM, and an eager map would capture a half-initialized
  object (class-init cycle).
- Additive dtype-first API (primary constructors unchanged):
  TensorStorage(shape, dtype, encoding, buffer, ...),
  TensorStorageFactory.fromRawBytes / fromRawBytesOwned / fileBacked
  (dtype overloads), PackedBlockStorage.toTensorStorage(dtype, ...);
  toTensorData dispatches on storage.dtype with identical semantics.
- Internal call sites moved to DType: TensorStorageFactory
  fromFloatArray/fromIntArray, DefaultKvCacheStore,
  TurboQuantKvCacheStore, StreamingGGUFReader (ggmlTypeToDType; quantized
  -> FP32, Opaque fallback kept), StreamingSafeTensorsReader (BOOL ->
  UInt8, UNKNOWN -> Int8 kept).
- Tests: DTypeWitnessTest; dtype twins in TensorStorageContractTest;
  StorageIntegrationTest asserts storage.dtype === FP32 for F32 and Q8_0.
- BCV: lang-core jvm dump regenerated (additions only).

Closes #1007

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@michalharakal

Copy link
Copy Markdown
Contributor Author

Local gate scripts/pr-gate.sh (JDK 25) on ede2228:

=== pr-gate: JVM tests ===                                      BUILD SUCCESSFUL in 6m 32s
=== pr-gate: apiCheck ===                                       BUILD SUCCESSFUL in 1m 14s
=== pr-gate: verifyNpmPins jsTest wasmJsTest wasmWasiTest ===   BUILD SUCCESSFUL in 2m 39s
=== pr-gate: linuxX64Test ===                                   BUILD SUCCESSFUL in 1m 29s
=== pr-gate: assemble ===                                       BUILD SUCCESSFUL in 2m 9s
=== pr-gate: :skainet-test:skainet-test-java:test ===           BUILD SUCCESSFUL in 21s
pr-gate: all legs passed.

Targeted: lang-core sk.ainet.lang.types.* + sk.ainet.lang.tensor.storage.* 256/256, io-gguf + io-safetensors jvmTest green (497 tests total in the targeted run). Note on the class-init cycle (why the companion collections are lazy) is in the commit message and KDoc.

@michalharakal
michalharakal requested a review from aharakal August 22, 2026 19:03
@github-actions

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1050 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[S0.2a] P0: DType carries its witness (DType.witness, fromWitness, entries); descriptors/readers report DType additively

2 participants