Skip to content

[https://nvbugs/6476233][fix] Cap max_seq_len on H200 DeepSeek-V3.2 blockscale test #17659

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pranav-nvidia merged 4 commits into from
Aug 15, 2026
Merged

[https://nvbugs/6476233][fix] Cap max_seq_len on H200 DeepSeek-V3.2 blockscale test #17659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
31d88fb
[https://nvbugs/6476233][fix] Cap max_seq_len on H200 DeepSeek-V3.2 b…
pranav-nvidia Aug 13, 2026
0f5cc15
[https://nvbugs/6476233][fix] Size the max_seq_len cap by evaluation …
pranav-nvidia Aug 13, 2026
dd76b8b
[https://nvbugs/6476233][fix] Derive the max_seq_len cap from the tas…
pranav-nvidia Aug 14, 2026
665120b
Merge remote-tracking branch 'origin/main' into bug6476233-h200-max-s…
pranav-nvidia Aug 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion tests/integration/defs/accuracy/test_llm_api_pytorch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
SADecodingConfig, SamplingParams, SchedulerConfig,
SkipSoftmaxAttentionConfig, SAEnhancerConfig, TorchCompileConfig)
# isort: on
from tensorrt_llm.math_utils import pad_up
from tensorrt_llm.quantization import QuantAlgo

from ..conftest import (check_device_contain, get_device_count,
Expand Down Expand Up @@ -3390,6 +3391,7 @@ def test_fp8_blockscale(self, tp_size, pp_size, ep_size, mtp_nextn, fp8kv,
attention_dp, cuda_graph, overlap_scheduler,
max_batch_size, moe_backend, disable_skip_indexer,
enable_heuristic_topk, use_cute_dsl_topk):
extra_llm_args = {}
if get_sm_version() == 100 or get_sm_version() == 103:
moe_backend = "DEEPGEMM" if moe_backend == "_DEFAULT" else moe_backend
moe_config = MoeConfig(backend=moe_backend, max_num_tokens=16384)
Expand All @@ -3403,6 +3405,15 @@ def test_fp8_blockscale(self, tp_size, pp_size, ep_size, mtp_nextn, fp8kv,
moe_config = MoeConfig()
kv_cache_config = KvCacheConfig(free_gpu_memory_fraction=0.7)

# Cap max_seq_len based on the evaluation task (NVBug 6476233).
MAX_SEQ_LEN_GRANULARITY = 8 * 1024 # Padding boundary for max_seq_len

task_classes = (GPQADiamond, ) if fp8kv else (MMLU, GSM8K)
max_task_seq_len = max(task.MAX_INPUT_LEN + task.MAX_OUTPUT_LEN
for task in task_classes)
extra_llm_args["max_seq_len"] = pad_up(max_task_seq_len,
MAX_SEQ_LEN_GRANULARITY)
Comment thread
pranav-nvidia marked this conversation as resolved.

pytorch_config = dict(
disable_overlap_scheduler=not overlap_scheduler,
cuda_graph_config=CudaGraphConfig() if cuda_graph else None,
Expand Down Expand Up @@ -3442,7 +3453,8 @@ def test_fp8_blockscale(self, tp_size, pp_size, ep_size, mtp_nextn, fp8kv,
**pytorch_config,
enable_attention_dp=attention_dp,
speculative_config=mtp_config,
sparse_attention_config=dsa_config) as llm:
sparse_attention_config=dsa_config,
**extra_llm_args) as llm:

# GPQA Diamond takes too long to run, we enable it only for fp8kv.
if fp8kv:
Expand Down
2 changes: 0 additions & 2 deletions tests/integration/test_lists/waives.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,6 @@ full:DGX_B200/accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Pro::test_gsm8k_fu
full:DGX_B200/disaggregated/test_disaggregated.py::test_disaggregated_gpt_oss_120b_harmony[gpt_oss/gpt-oss-120b] SKIP (https://nvbugs/6594241)
full:DGX_B200/perf/test_perf_sanity.py::test_e2e[aggr_upload-gemma4_26b_a4b_nvfp4_blackwell-gemma4_26b_a4b_nvfp4_tp1_1k1k] SKIP (https://nvbugs/6571410)
full:DGX_B200/perf/test_perf_sanity.py::test_e2e[aggr_upload-host_perf_llama8b_spec_decode-llama8b_spec_bs1_128_128] SKIP (https://nvbugs/6571408)
full:DGX_H200/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_fp8_blockscale[disable_skip_indexer] SKIP (https://nvbugs/6476233)
full:DGX_H200/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_fp8_blockscale[latency_default] SKIP (https://nvbugs/6476233)
full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_auto_dtype[ctx_block_reuse_only] SKIP (https://nvbugs/6525893)
full:GB200/accuracy/test_dwdp_disaggregated_serving.py::TestDwdpDeepSeekV3Lite::test_dwdp_accuracy SKIP (https://nvbugs/6276923)
full:GB200/accuracy/test_dwdp_disaggregated_serving.py::TestDwdpDeepSeekV3Lite::test_dwdp_accuracy_contention_opt SKIP (https://nvbugs/6276923)
Expand Down
Loading