diagnostics(sae): SparseAutoencoder hook + top-K feature extraction

[jamesburton]

Closes #311.

ROADMAP Phase 7 Step 50 — SAE integration. Implements the SAE inference primitive + PostLayer hook on top of the #32 hook system and the #33 logit-lens precedent.

Stack

Depends on:

• Step 18: Inference hook system — zero-cost diagnostic hook points #32 — Inference hook system (IInferenceHook, HookRegistry, 8-point TransformerModel.Forward integration).
• Step 19: Logit lens — per-layer token prediction via hook system #33 — Logit lens (clone-on-fire + deferred-projection pattern this PR mirrors).

This PR is opened as a DRAFT because it stacks on the above. Will rebase to main once #32 and #33 merge.

What lands

• ISparseAutoencoder (non-breaking extension): adds HiddenSize + XML doc comments on the existing 3-member placeholder. Original Encode / Decode / FeatureCount contract preserved.
• SparseAutoencoder — concrete implementation holding W_enc [d_in, d_sae], b_enc [d_sae], W_dec [d_sae, d_in], b_dec [d_in] as 64-byte-aligned NativeMemory.AlignedAlloc buffers per project memory rules. Implements the canonical SAE forward documented in docs/DIAGNOSTICS.md:

  pre = activation - b_dec        (when ApplyBDecToInput; SAELens convention)
  features = relu(pre @ W_enc + b_enc)
  active = top-K features by magnitude
  reconstruction = sum(active @ W_dec_rows) + b_dec
  

• SaeHook : IInferenceHook at HookPoint.PostLayer. Mirrors LogitLensHook's defer pattern: OnActivation clones the hidden state on the hot path and returns Continue; encode + top-K + L2 reconstruction error compute once in GetResults(). Read-only — never returns Replace (steering is an explicit follow-up).
• SaeConfig — layer / position selection (reuses LogitLensLayerSelector) + TopK.
• SaeResult — per-(layer, position) feature indices, magnitudes, reconstruction error, pre-truncation active count.
• SaeLoader — minimal in-DotLLM.Diagnostics safetensors reader (F32 only) + SaeCfg.ParseJson for SAELens-style cfg.json (d_in, d_sae, apply_b_dec_to_input, hook_name). Kept inside Diagnostics to avoid expanding the project's dependency surface — the public Load / LoadFromBytes API is the retarget seam when a shared safetensors reader lands.
• SaeMath (internal) — TopK + L2Distance pure functions, testable without weights.

Tests

28 new unit tests in tests/DotLLM.Tests.Unit/Diagnostics/SaeHookTests.cs, all model-free and using synthetic SAEs with hand-computable expected values:

• Pure math: TopK descending order + K-clamping; L2Distance symmetry + zero-on-equality.
• Synthetic SAE forward: hand-computed Encode/Decode against a 2×3 SAE — pre-ReLU output, ReLU clamp on negative inputs, decode formula recon = b_dec + sum(value * W_dec[index]).
• ApplyBDecToInput: SAELens pre-subtraction verified against the same hand-computed test with shifted b_dec / activation.
• Top-K truncation: deliberately sparsity-inducing weights, K=2 drops the two smallest features; reconstruction error matches the closed-form sqrt(sum-of-squared-dropped-magnitudes).
• Identity SAE round-trip: encode → decode reconstructs non-negative input bit-exact (ReconstructionError == 0).
• Hook integration: fire SaeHook through HookRegistry with a synthetic PostLayer activation, verify SaeResult populates with indices / magnitudes / active count.
• Hook filtering: layer-selector + position-selector restrict captures as expected; activation length mismatch throws; hook always returns Continue.
• Loader round-trip: build a synthetic safetensors buffer in-memory (no binary fixtures), load via SaeLoader.LoadFromBytes, verify Encode matches the span-constructed SAE. cfg.json apply_b_dec_to_input flag honored; cfg/header shape mismatch fatal; F16 dtype rejected with a clear NotSupportedException.
• Dispose: idempotent; post-Dispose Encode throws ObjectDisposedException.

Full DotLLM.Tests.Unit suite: 1163/1163 non-CUDA tests pass (CUDA tests fail in this environment for lack of a CUDA device — pre-existing and unrelated).

Sample

samples/DotLLM.Sample.Interpretability/Program.cs gets an optional --sae <path> flag. When provided, an SaeHook is registered at the middle layer for the final prompt position alongside the existing logit lens, and top-K features + reconstruction error are printed. When omitted the original logit-lens-only workflow is unchanged.

Out of scope (follow-ups)

Documented in #311 — left as separate issues to keep this PR reviewable:

• Steering / ablation via HookResult.Replace
• F16 / BF16 weight dtypes in the SAE loader
• Curated SAE registry JSON (EleutherAI / Goodfire / Llama Scope)
• Neuronpedia feature-label client
• Serve / Chat UI panels
• Real-checkpoint integration test (e.g. EleutherAI/sae-llama-3-8b)