fix: Q6_K/Q5_1/Q5_0 matmul silently falls through for MemorySegment-backed activations

[michalharakal]

Summary

chooseQuantizedMatmulHeap (the fallback dispatcher DefaultCpuOpsJvm.chooseQuantizedMatmul intentionally defers to for Q6_K/Q5_1/Q5_0 — see the "handled in DefaultCpuOpsBase via the kernel registry... not intercepted here" comment) required the activation tensor to be exactly FloatArrayTensorData, silently returning null for anything else.

Real forward-pass activations produced with DirectCpuExecutionContext(tensorDataFactory = MemorySegmentTensorDataFactory()) — the configuration every production caller uses, including KLlamaJava.loadGGUF in SKaiNET-transformers — are MemorySegmentTensorData, not FloatArrayTensorData, so they never matched this check. Dispatch fell through to the unguarded matmulGeneric fallback, which has no packed-quant handling at all and threw:

java.lang.ClassCastException: class java.lang.Byte cannot be cast to class java.lang.Float
	at sk.ainet.exec.tensor.ops.DefaultCpuOpsBase.matmulGeneric$lambda$1(DefaultCpuOps.kt:726)

reading raw packed Q6_K bytes as if they were Float.

Fix

• chooseQuantizedMatmulHeap2D now calls the universal TensorData.copyToFloatArray() instead of gating on the FloatArrayTensorData subtype. FloatArrayTensorData already overrides that method with a cheap buffer.copyOf(), so the existing fast path is unaffected — this only adds correct handling for everything else (MemorySegment-backed, etc.).
• DefaultCpuOpsJvm.chooseQuantizedMatmul now flattens leading batch/sequence dimensions before its rank-2 fast-path check. Not the trigger for this specific bug (the repro's activation tensor was already rank-2), but linearProject's documented [..., in] input shape is a real case this fast path couldn't otherwise reach, so it's included as a related robustness fix.
• Added a regression test (QuantizedMemSegMatmulTest) that reproduces the exact scenario: a Q6_K weight matmul'd against a MemorySegmentTensorDataFactory-backed FP32 activation, verifying it returns a correctly-shaped, finite result instead of throwing.

Verification

Beyond the new unit test, I reproduced and verified the fix against a real model end-to-end (Llama-3.2-1B-Instruct-Q4_K_M.gguf via KLlamaJava.loadGGUF, from a downstream KMP app integrating SKaiNET-transformers 0.40.2). Before the fix: crash on the first forward pass. After: coherent generated text, in ~5s total including model load, via the NATIVE_OPTIMIZED packed-quant path (not a slow FP32-dequant fallback).

Fixes #991.

[fiwio-developer]

Pushed an additional fix on top of this one, filed as #993 — a related but distinct dispatch gap in the same functions this PR touches (chooseQuantizedMatmulHeap / chooseQuantizedMatmul): both required a.shape.rank >= 2, so a rank-1 activation (the single-token hidden-state vector every real forward pass produces once incremental decode moves past the initial batched prefill) skipped the packed-quant kernel dispatch entirely and hit the same matmulGeneric ClassCastException this PR already fixes for the rank-2 MemorySegment case.

Verified end-to-end against a real Llama-3.2-1B GGUF via SKaiNET-transformers' KLlamaJava — this PR's original fix alone was not sufficient to get past the first post-prefill decode step; both fixes together are.