Skip to content

[https://nvbugs/6599150][fix] Initialize dt_bias in KDA verify-parity test #17584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brnguyen2 merged 4 commits into from
Aug 13, 2026
+4 −1
Merged

[https://nvbugs/6599150][fix] Initialize dt_bias in KDA verify-parity test #17584

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5a31fb7
[https://nvbugs/6599150][fix] Initialize dt_bias in KDA verify-parity…
brnguyen2 Aug 12, 2026
37e6857
Merge remote-tracking branch 'upstream/main' into fix/kda-verify-pari…
brnguyen2 Aug 12, 2026
944c4c0
[https://nvbugs/6599150][fix] Unwaive KDA verify-parity test
brnguyen2 Aug 12, 2026
21dde19
Merge branch 'main' into fix/kda-verify-parity-dt-bias-init
brnguyen2 Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion tests/integration/test_lists/waives.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,6 @@ unittest/_torch/attention/sparse/dsa/test_req_idx_per_token.py::test_on_update_k
unittest/_torch/attention/test_attention_backends.py::test_attention_backend[deepseekv3_mla-gen-bf16-HND-p32-v1] SKIP (https://nvbugs/6507109)
unittest/_torch/executor/test_overlap_scheduler.py::test_overlap_scheduler_consistency[no_reuse-cpp_scheduler-TorchSampler] SKIP (https://nvbugs/6561559)
unittest/_torch/misc/test_autotuner.py::test_cutedsl_nvfp4_heuristic_matches_full_sweep SKIP (https://nvbugs/6490028)
unittest/_torch/modeling/test_kimi_kda_verify_parity.py::test_kda_verify_matches_sequential_decode[2-1] SKIP (https://nvbugs/6599150)
unittest/_torch/modeling/test_modeling_qwen_moe.py::TestQwenMoe::test_qwen_moe_allclose_to_hf[backend:trtllm-use_cuda_graph:False] SKIP (https://nvbugs/6566765)
unittest/_torch/modeling/test_modeling_qwen_moe.py::TestQwenMoe::test_qwen_moe_allclose_to_hf[backend:trtllm-use_cuda_graph:True] SKIP (https://nvbugs/6575012)
unittest/_torch/modeling/test_modeling_qwen_moe.py::TestQwenMoe::test_qwen_moe_allclose_to_hf[backend:vanilla-use_cuda_graph:False] SKIP (https://nvbugs/6566765)
Expand Down
4 changes: 4 additions & 0 deletions tests/unittest/_torch/modeling/test_kimi_kda_verify_parity.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ def test_kda_verify_matches_sequential_decode(batch, t_steps):
w = lin["short_conv_kernel_size"]

runtime = KimiKDARuntime(cfg, layer_idx=0).to(device)
# dt_bias is torch.empty at construction and only filled by
# load_weights(); with random weights it holds heap garbage, and a
# NaN/Inf bit pattern poisons both paths identically (nvbug 6599150).
torch.nn.init.normal_(runtime.mixer.dt_bias, std=0.1)
slots = batch + 2 # non-trivial slot mapping
cache = _LayerCache(slots, 3 * dim, w, h, lin["head_dim"], lin["head_dim"], t_steps, device)
slot_indices = torch.arange(2, 2 + batch, device=device, dtype=torch.long)
Expand Down
Loading