From 084ca9884edb51f42587fbc3dfb5b13c24b90024 Mon Sep 17 00:00:00 2001 From: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> Date: Tue, 18 Aug 2026 20:03:12 -0700 Subject: [PATCH 1/3] test: give the multimodal Kimi-K2.5 dep8 accuracy test 120 minutes in the QA list accuracy/test_llm_api_pytorch_multimodal.py::TestKimiK25::test_nvfp4[dep8] carried no TIMEOUT annotation in the QA list, so it ran under the job-level 3600s default while the pre-merge entry in test-db/l0_dgx_b200.yml already grants it 120 minutes. The case has been trending against that ceiling on B300 (3604s on the last green run, 3616s on the failing one) even though the score stayed healthy at 80.111 versus a 77.683 threshold, so the failure was a wall-clock budget overrun rather than a functional regression. Annotate the QA entry with TIMEOUT (120) so both lists agree on the budget. Signed-off-by: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> --- tests/integration/test_lists/qa/llm_function_core.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/integration/test_lists/qa/llm_function_core.txt b/tests/integration/test_lists/qa/llm_function_core.txt index 2a4262426c18..fd23de69cc1f 100644 --- a/tests/integration/test_lists/qa/llm_function_core.txt +++ b/tests/integration/test_lists/qa/llm_function_core.txt @@ -789,7 +789,7 @@ accuracy/test_llm_api_pytorch_encode.py::TestEncoderEncode::test_qwen3_text_embe accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[forced_chunked_prefill] accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[full_budget] accuracy/test_llm_api_pytorch_multimodal.py::TestGemma3_27BInstruct::test_fp8_prequantized -accuracy/test_llm_api_pytorch_multimodal.py::TestKimiK25::test_nvfp4[dep8] +accuracy/test_llm_api_pytorch_multimodal.py::TestKimiK25::test_nvfp4[dep8] TIMEOUT (120) accuracy/test_llm_api_pytorch_multimodal.py::TestMistralLarge3_675B::test_nvfp4_4gpus[latency_moe_trtllm] accuracy/test_llm_api_pytorch_multimodal.py::TestMistralSmall24B::test_auto_dtype[forced_chunked_prefill] accuracy/test_llm_api_pytorch_multimodal.py::TestNanoV3Omni::test_auto_dtype[bf16] From 5b4beccc6c6d31842899f1c7284cca59e660fb7f Mon Sep 17 00:00:00 2001 From: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> Date: Tue, 18 Aug 2026 20:21:28 -0700 Subject: [PATCH 2/3] test: unwaive the multimodal Kimi-K2.5 dep8 accuracy test The B200 and B300 waives were added while the wall-clock overrun was under investigation. With the QA test list now granting the same 120-minute budget the pre-merge list already used, the case can run again. Signed-off-by: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> --- tests/integration/test_lists/waives.txt | 2 -- 1 file changed, 2 deletions(-) diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index 13cec4c43895..5462b117b0bf 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -148,7 +148,6 @@ full:B200/accuracy/test_llm_api_pytorch.py::TestNemotronV3Ultra::test_nvfp4_8gpu full:B200/accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_dummy_load_format SKIP (https://nvbugs/6525059) full:B200/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6597570) full:B200/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[full_budget] SKIP (https://nvbugs/6597570) -full:B200/accuracy/test_llm_api_pytorch_multimodal.py::TestKimiK25::test_nvfp4[dep8] SKIP (https://nvbugs/6626655) full:B200/disaggregated/test_disaggregated.py::test_disaggregated_stress_test[input8k-output1k-conc512-glm5_nvfp4_tp4_ep4_dp_stress] SKIP (https://nvbugs/6544407) full:B200/llmapi/test_llm_api_pytorch_moe_lora.py::test_qwen_moe_routed_expert_multi_lora_varying_ranks[cudagraph] SKIP (https://nvbugs/6475623) full:B200/test_e2e.py::test_multi_nodes_eval[Qwen3/Qwen3-235B-A22B-tp16-mmlu] SKIP (https://nvbugs/6424188) @@ -169,7 +168,6 @@ full:B300/accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_dummy_load_f full:B300/accuracy/test_llm_api_pytorch.py::TestStep3_7::test_fp8_block_scales[tp_size=4-ep_size=4-mtp_nextn=3] SKIP (https://nvbugs/6539941) full:B300/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6597570) full:B300/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[full_budget] SKIP (https://nvbugs/6597570) -full:B300/accuracy/test_llm_api_pytorch_multimodal.py::TestKimiK25::test_nvfp4[dep8] SKIP (https://nvbugs/6626655) full:B300/disaggregated/test_disaggregated.py::test_disaggregated_logprobs_serving[llama-3.1-8b-instruct] SKIP (https://nvbugs/6275959) full:B300/llmapi/test_llm_api_pytorch_moe_lora.py::test_qwen_moe_routed_expert_multi_lora_varying_ranks[cudagraph] SKIP (https://nvbugs/6475623) full:B300/unittest/_torch/attention/test_attention_backends.py::test_attention_backend[qwen2_0_5b_gqa_hd64-ctx-bf16-HND-p32-v1] SKIP (https://nvbugs/6610548) From f68cb2f4b23b6a7d771acb133e4d5cd4dd164f19 Mon Sep 17 00:00:00 2001 From: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> Date: Tue, 18 Aug 2026 20:33:13 -0700 Subject: [PATCH 3/3] test: mark the multimodal Kimi-K2.5 dep8 accuracy test with a 7200s timeout The test list annotation only applies to the list that carries it, which is how the QA and pre-merge lists ended up disagreeing on the budget in the first place. Declaring the timeout on the test itself gives it the same 120 minutes wherever it is collected from. Signed-off-by: Tianyu Xiong <117647511+tianyuxbear@users.noreply.github.com> --- .../integration/defs/accuracy/test_llm_api_pytorch_multimodal.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py b/tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py index e0f7c4b17ec4..9b79a69196b9 100644 --- a/tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py +++ b/tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py @@ -545,6 +545,7 @@ class TestKimiK25(LlmapiAccuracyTestHarness): preserve_caller_max_tokens=True, ) + @pytest.mark.timeout(7200) @skip_pre_blackwell @pytest.mark.skip_less_mpi_world_size(8) @pytest.mark.skip_less_device_memory(183000)