Skip to content

[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests #19178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xinhe-nv wants to merge 19 commits into
base: main
Choose a base branch
from
Open

[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests #19178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
8280b79
[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests
xinhe-nv Sep 15, 2026
90c137e
[None][test] Split qwen_model_root out of llama_model_root, drop LoRA…
xinhe-nv Sep 15, 2026
3e57935
[None][test] Finish renaming stale TinyLlama identifiers to Qwen3-0.6B
xinhe-nv Sep 15, 2026
da08d2a
[None][test] Inline Qwen3-0.6B path lookup in triton_server llmapi tests
xinhe-nv Sep 15, 2026
44cb005
[None][test] Remove qwen_model_root fixture, inline the model path lo…
xinhe-nv Sep 15, 2026
9f6cea3
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 15, 2026
2cf8c0e
[None][fix] Fix regressions from the TinyLlama->Qwen3-0.6B test migra…
xinhe-nv Sep 16, 2026
98e3040
[None][fix] Restore test_llmapi_lora and its TinyLlama LoRA fixtures
xinhe-nv Sep 16, 2026
ad82cd0
[None][test] Restore LlamaForCausalLM coverage in DECODER_MODELS
xinhe-nv Sep 16, 2026
e32dc78
[None][fix] Update telemetry architectureClassName assertions to Qwen…
xinhe-nv Sep 16, 2026
a8a37a5
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 16, 2026
da7d5d0
[None][fix] Drop stale TinyLlama param from two B200 disagg waives
xinhe-nv Sep 16, 2026
f84eab5
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 16, 2026
700b27d
[None][fix] Migrate disaggregated test_configs/*.yaml model field to …
xinhe-nv Sep 17, 2026
4365f78
[None][test] Add capture script for test_disaggregated_simple_llama Q…
xinhe-nv Sep 17, 2026
91691f7
[None][test] Drop test_disaggregated_simple_llama, obsoleted by test_…
xinhe-nv Sep 17, 2026
7abad82
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 17, 2026
5fe940e
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 18, 2026
3efbb84
Merge branch 'main' into trtllm-15117
xinhe-nv Sep 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions tests/integration/defs/accuracy/test_llm_api_pytorch_encode.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,14 +239,14 @@ def test_qwen3_text_embedding_matches_huggingface(self, model_name, model_path):
# classification scoring on a causal LM backbone.
#
# One representative per distinct TRT-LLM architecture class:

@BowenFu BowenFu Sep 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment just above DECODER_MODELS says the list holds one representative per distinct architecture class, and the TinyLlama entry was the LlamaForCausalLM one — explicitly noted as also covering Mistral, which aliases LlamaModel. With it gone, Qwen3 is the only entry left, so encode() single-prefill coverage loses the whole Llama/Mistral family rather than becoming redundant with the Qwen3 case the way the description reads.

If it's TinyLlama specifically that's being retired rather than the class, pointing the param at another small Llama-family checkpoint would keep the representative intact. Failing that, the comment above the list should probably stop advertising a class the list no longer covers.

Non-blocking as far as I'm concerned if you'd rather handle the replacement checkpoint separately — but worth not losing silently.

@xinhe-nv xinhe-nv Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed properly in ad82cd0: restored the LlamaForCausalLM representative pointed at Llama-3.2-1B — a small Llama checkpoint I confirmed is already relied on elsewhere in this suite (kv_cache/test_kv_cache_v2_scheduler.py's TestKVCacheV2Llama), so it's a verified-to-exist path rather than a guess at a new NFS checkpoint. Added back the matching l0_a100.yml/llm_function_core.txt test-list entries. The comment above DECODER_MODELS now correctly lists both LlamaForCausalLM/Mistral and Qwen3ForCausalLM as covered again.

# LlamaForCausalLM — TinyLlama (also covers Mistral, which aliases LlamaModel)
# LlamaForCausalLM — Llama-3.2-1B (also covers Mistral, which aliases LlamaModel)
# Qwen3ForCausalLM — Qwen3-0.6B (QKNorm)
DECODER_MODELS = [
# -- LlamaForCausalLM (covers Llama + Mistral family) --
pytest.param(
"TinyLlama/TinyLlama-1.1B-Chat-v1.0",
f"{llm_models_root()}/llama-models-v2/TinyLlama-1.1B-Chat-v1.0",
id="tinyllama-1.1b",
"meta-llama/Llama-3.2-1B",
f"{llm_models_root()}/llama-3.2-models/Llama-3.2-1B",
Comment thread
coderabbitai[bot] marked this conversation as resolved.
id="llama-3.2-1b",
),
# -- Qwen3ForCausalLM --
pytest.param(
Expand Down
5 changes: 1 addition & 4 deletions tests/integration/defs/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -677,10 +677,7 @@ def llama_v2_tokenizer_model_root():
def llama_model_root(request):
models_root = llm_models_root()
assert models_root, "Did you set LLM_MODELS_ROOT?"
if request.param == "TinyLlama-1.1B-Chat-v1.0":
llama_model_root = os.path.join(models_root, "llama-models-v2",
"TinyLlama-1.1B-Chat-v1.0")
elif request.param == "llama-3.1-8b":
if request.param == "llama-3.1-8b":
llama_model_root = os.path.join(models_root, "llama-3.1-model",
"Meta-Llama-3.1-8B")
elif request.param == "llama-3.1-8b-instruct-hf-fp8":
Expand Down
32 changes: 16 additions & 16 deletions tests/integration/defs/disaggregated/test_ad_disagg.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def skip_b300():
"OMPI_UNIVERSE_SIZE",
)
AUTODEPLOY_DISAGG_SEED = 1234
REDUCED_TINYLLAMA_LAYERS = 2
REDUCED_QWEN3_LAYERS = 2
REDUCED_DEEPSEEK_LAYERS = 2
LLAMA_EAGLE3_EXPECTED_TEXT = " Berlin\nWhat is the capital of France? Paris\nWhat is the capital of"
LLAMA_EAGLE3_EXPECTED_TOKEN_IDS = [
Expand All @@ -91,7 +91,7 @@ def skip_b300():
MODEL_PATHS = {
"EAGLE3-LLaMA3.1-Instruct-8B": "EAGLE3-LLaMA3.1-Instruct-8B",
"Llama-3.1-8B-Instruct": "llama-3.1-model/Llama-3.1-8B-Instruct/",
"TinyLlama-1.1B-Chat-v1.0": "llama-models-v2/TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B": "Qwen3/Qwen3-0.6B",
"DeepSeek-V3-Lite": "DeepSeek-V3-Lite/bf16",
}

Expand Down Expand Up @@ -280,9 +280,9 @@ def run_aggregate_generation(
# ---------------------------------------------------------------------------


def reduced_tinyllama_config(extra_config=None):
def reduced_qwen3_config(extra_config=None):
config = {
"model_kwargs": {"num_hidden_layers": REDUCED_TINYLLAMA_LAYERS},
"model_kwargs": {"num_hidden_layers": REDUCED_QWEN3_LAYERS},
"max_batch_size": 4,
"max_seq_len": 512,
"max_num_tokens": 256,
Expand Down Expand Up @@ -459,7 +459,7 @@ def reduced_model_config(model, extra_config=None):
if "DeepSeek-V3-Lite" in model:
config = reduced_deepseek_v3_mla_config()
else:
config = reduced_tinyllama_config()
config = reduced_qwen3_config()
if extra_config:
config.update(extra_config)
return config
Expand All @@ -468,8 +468,8 @@ def reduced_model_config(model, extra_config=None):
def reduced_model_cases():
return [
pytest.param(
"TinyLlama-1.1B-Chat-v1.0",
id="tinyllama",
"Qwen3-0.6B",
id="qwen3_0_6b",
),
pytest.param(
"DeepSeek-V3-Lite",
Expand Down Expand Up @@ -563,7 +563,7 @@ def test_disaggregated_logits(model):
# The MLA generation worker reconstructs logits from the compressed KV latent
# through a different kernel/batching path than the single aggregate pass, so
# bf16 rounding yields ~1-ULP logit differences. Use a looser tolerance for the
# MLA (DeepSeek) case; MHA (tinyllama) stays tight. The functional checks above
# MLA (DeepSeek) case; MHA (Qwen3-0.6B) stays tight. The functional checks above
# (text/token_ids equality) remain strict for both.
if "DeepSeek-V3-Lite" in model:
rtol, atol = 1e-1, 1e-1
Expand All @@ -579,7 +579,7 @@ def test_disaggregated_logits(model):

@pytest.mark.skip_less_device_memory(30000)
@pytest.mark.timeout(600)
def test_tinyllama_batch_handoff_semantic_slots():
def test_qwen3_batch_handoff_semantic_slots():
prompts = capital_completion_prompts()
expected_capitals = ["Berlin", "Paris", "Rome", "Madrid"]
sampling_params_kwargs = {
Expand All @@ -589,7 +589,7 @@ def test_tinyllama_batch_handoff_semantic_slots():
"seed": AUTODEPLOY_DISAGG_SEED,
}
outputs = run_sequential_batch_handoff(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
generation_overlap=True,
prompts=prompts,
sampling_params_kwargs=sampling_params_kwargs,
Expand Down Expand Up @@ -951,13 +951,13 @@ def run_context_then_generation_handoff(
@pytest.mark.timeout(600)
def test_async_generation_matches_aggregate():
aggregate_output = run_aggregate_generation(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
world_size=1,
prompt="What is the capital of Germany?",
sampling_params_kwargs={"max_tokens": 10, "ignore_eos": True},
)
outputs = run_context_then_generation_handoff(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
worker_world_sizes=(1, 1),
generation_overlap=True,
prompt="What is the capital of Germany?",
Expand Down Expand Up @@ -989,13 +989,13 @@ def test_async_generation_no_overlap_matches_aggregate():
"""
sampling_params_kwargs = {"max_tokens": 10, "ignore_eos": True}
aggregate_output = run_aggregate_generation(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
world_size=1,
prompt="What is the capital of Germany?",
sampling_params_kwargs=sampling_params_kwargs,
)
outputs = run_context_then_generation_handoff(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
worker_world_sizes=(1, 1),
generation_overlap=False,
prompt="What is the capital of Germany?",
Expand All @@ -1014,13 +1014,13 @@ def test_async_generation_no_overlap_matches_aggregate():
@pytest.mark.timeout(900)
def test_async_sharded_generation_handoff():
aggregate_output = run_aggregate_generation(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
world_size=2,
prompt="What is the capital of Germany?",
sampling_params_kwargs={"max_tokens": 10, "ignore_eos": True},
)
outputs = run_context_then_generation_handoff(
"TinyLlama-1.1B-Chat-v1.0",
"Qwen3-0.6B",
worker_world_sizes=(2, 2),
generation_overlap=True,
prompt="What is the capital of Germany?",
Expand Down
8 changes: 4 additions & 4 deletions tests/integration/defs/disaggregated/test_ad_disagg_trtllm_serve.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,13 @@ def skip_b300():
SERVER_READY_REQUEST_TIMEOUT_S = 5
OPENAI_REQUEST_TIMEOUT_S = 60
PROXY_PORT_MAX_RETRIES = 5
TINYLLAMA_MODEL_DIR = "llama-models-v2/TinyLlama-1.1B-Chat-v1.0"
QWEN3_MODEL_DIR = "Qwen3/Qwen3-0.6B"
AUTODEPLOY_BACKEND = "_autodeploy"
EXPECTED_COMPLETION_SUBSTRING = "Berlin"


def tinyllama_model_path():
return str(Path(llm_models_root()) / TINYLLAMA_MODEL_DIR)
def qwen3_model_path():
return str(Path(llm_models_root()) / QWEN3_MODEL_DIR)


def worker_cuda_devices(num_workers):
Expand Down Expand Up @@ -175,7 +175,7 @@ async def test_openai_completion(work_dir):
verifies the trtllm-serve deployment shape: context worker, generation
worker, disaggregated proxy, and an OpenAI-compatible completion request.
"""
model = tinyllama_model_path()
model = qwen3_model_path()
ctx_device, gen_device = worker_cuda_devices(2)

last_port_conflict = None
Expand Down
3 changes: 1 addition & 2 deletions tests/integration/defs/disaggregated/test_auto_scaling.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,7 @@ def worker_env():

@pytest.fixture
def model_name():
model_path = os.path.join(llm_models_root(),
"llama-models-v2/TinyLlama-1.1B-Chat-v1.0")
model_path = os.path.join(llm_models_root(), "Qwen3/Qwen3-0.6B")
assert os.path.exists(model_path), f"Model path {model_path} does not exist"
return model_path

Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.25
backend: pytorch
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_cache_aware_balance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_cache_reuse.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_conditional.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_conversation.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_conversation_workers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxpp2_genpp2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxpp2_gentp2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxpp4_genpp4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxpp4_gentp4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_genpp2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.25
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2pp2_gentp2pp2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_cuda_graph_padding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
context_servers:
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_diff_max_tokens.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.25
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_gen_only.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
context_servers:
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_gen_only_bs1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion ...s/integration/defs/disaggregated/test_configs/disagg_config_gen_only_insufficient_kv.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
context_servers:
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_gen_only_kv_cache_aware.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_load_balance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_load_balancing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
backend: pytorch
cuda_graph_config: null
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_metrics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.25
backend: "pytorch"
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_mixed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.25
backend: "pytorch"
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_multi_orchestrator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
num_workers: 4
free_gpu_memory_fraction: 0.25
backend: pytorch
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_ngram.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
hostname: localhost
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
free_gpu_memory_fraction: 0.1
backend: pytorch
disable_overlap_scheduler: true
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_overlap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
backend: pytorch
cuda_graph_config: null
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_overlap_gen_first.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
port: 8000
backend: "pytorch"
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/defs/disaggregated/test_configs/disagg_config_overlap_gen_first_pp4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
port: 8000
backend: "pytorch"
Expand Down
2 changes: 1 addition & 1 deletion ...ion/defs/disaggregated/test_configs/disagg_config_overlap_transceiver_runtime_python.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
port: 8000
backend: "pytorch"
Expand Down
2 changes: 1 addition & 1 deletion ...s/disaggregated/test_configs/disagg_config_overlap_transceiver_runtime_python_bounce.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# TRTLLM_KV_CACHE_BOUNCE_MIN_BLOCKS env (set by the test) so the ordinary short test prompts still
# take the coalesced-bounce WRITE path (the production default of 96 would need a ~2k-token prompt).
# GB200/GB300 only, since the bounce arena is fabric (MNNVL) VMM memory.
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
port: 8000
backend: "pytorch"
Expand Down
2 changes: 1 addition & 1 deletion ...ration/defs/disaggregated/test_configs/disagg_config_python_transceiver_host_offload.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model: Qwen3/Qwen3-0.6B
hostname: localhost
port: 8000
backend: "pytorch"
Expand Down
Loading
Loading