FP16 simulation execution context + FP32-accumulating LayerNorm lowering rule (variance overflow repro)

[michalharakal]

Context

An FP16 LayerNorm computes variance as a mean over 1280 squared, centred activations. For real Whisper encoder activations the sum exceeds the FP16 maximum before the division, variance becomes +Inf, rsqrt becomes 0, and the block silently emits zeros for those frames — 6 of 1500 frames in the measured case, enough to drop encoder cosine from 0.9999 to 0.968. The fix on the LiteRT side was a graph rewrite ((0.25·δ)² · 16) because the runtime offered no lever.

SKaiNET will meet the same bug the moment an FP16 GPU or NPU lowering exists. It should be reproducible on a desktop CPU before that, so the lowering rule (accumulate variance in FP32, or apply the rescale) can be written and regression-tested against a known-failing input.

Scope

• Fp16SimulationExecutionContext (or a DirectCpuExecutionContext option): after every op, round the output through Fp16Codec (round-to-nearest-even, overflow to ±Inf, gradual underflow) — storage-only simulation. A second mode also rounds accumulators inside reductions (sum/mean/variance/matmul) to model true half-precision arithmetic.
• A regression test with a synthetic [1500,1280] input scaled so that naive FP16 variance overflows, asserting +Inf under accumulate-in-half and a finite, correct value under the FP32-accumulate lowering.
• layerNorm lowering rule for FP16 targets in skainet-compile-hlo: variance reduction in F32 (or the documented rescale), with a TraceEvent naming which rule fired.
• Docs: an explanation page on narrow-float numerics pitfalls with this as the worked example.

Acceptance

• The overflow reproduces in a unit test on a JVM with no GPU.
• The StableHLO emitted for an FP16 LayerNorm shows the F32 reduction, and the test passes through the IREE parity harness (.vmfb parity acceptance run (eager vs compiled) #1148) when an FP16 target exists.

Related

• Shared narrow-float layer: complete BF16, add FP16 on the same SPI #884, FP16: native FFM matmul kernel (priority 100) to match BF16's tier #885 — narrow-float layer and FP16 kernels
• .vmfb parity acceptance run (eager vs compiled) #1148 — parity acceptance

[michalharakal]

Tracked in #1282.