From f6c994828cde02aac05f2a3e2eeca3b3b697a157 Mon Sep 17 00:00:00 2001 From: ADou Date: Wed, 9 Sep 2026 09:47:49 +0000 Subject: [PATCH] [None][docs] retarget moved performance doc URLs in tech blogs Point blog04/blog08/blog09 absolute links at the live developer-guide and legacy performance doc paths (docs/source/performance/ was removed). Signed-off-by: ADou --- .../blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md | 2 +- .../blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md | 2 +- .../blogs/tech_blog/blog09_Deploying_GPT_OSS_on_TRTLLM.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md b/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md index 68937fd1acc0..f77c2d10bd83 100644 --- a/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md +++ b/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md @@ -536,7 +536,7 @@ export EXPERT_STATISTIC_PATH=./expert_statistic export EXPERT_STATISTIC_ITER_RANGE=100-200 ``` -Prepare a dataset following the [benchmarking documentation](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/performance/perf-benchmarking.md#preparing-a-dataset) and save it as `./dataset.json`. +Prepare a dataset following the [benchmarking documentation](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/developer-guide/perf-benchmarking.md#preparing-a-dataset) and save it as `./dataset.json`. Run 32-way expert parallelism inference on the prepared dataset. Please refer to the [LLM API MGMN example](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/llm-api/llm_mgmn_trtllm_bench.sh) for details on running `trtllm-bench` on Slurm. diff --git a/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md b/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md index ec4745b24abd..a444b90b6aee 100644 --- a/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md +++ b/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md @@ -249,7 +249,7 @@ To address the increased host overhead when scaling parallelism in the system, w TensorRT LLM is designed to be composed of both C++ and Python code, so that C++ can handle the most performance-sensitive parts while Python handles higher-level logic. As we try to put more logic into Python to make the program easier to read and debug, there are still frequent conversations through binding interfaces between C++ and Python. Besides, since most of the logic is implemented in Python, there are several layers of implementation that communicate with each other through inter-process communication overhead. Frequent binding calls and serialization/deserialization introduced by inter-process communication slow down the core library. -To improve program efficiency, we used environment variables introduced in the [performance analysis guidance](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/performance/perf-analysis.md) to measure and profile CPU overhead, and improved performance by reducing and reusing different binding calls as much as possible, and delaying Python object deserialization to avoid duplicated serialization and reduce message size when doing inter-process communication. This optimization was added in [PR 5224](https://github.com/NVIDIA/TensorRT-LLM/pull/5224). We have also reduced Python garbage collection (GC) impacts in [PR 5141](https://github.com/NVIDIA/TensorRT-LLM/pull/5141). +To improve program efficiency, we used environment variables introduced in the [performance analysis guidance](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/developer-guide/perf-analysis.md) to measure and profile CPU overhead, and improved performance by reducing and reusing different binding calls as much as possible, and delaying Python object deserialization to avoid duplicated serialization and reduce message size when doing inter-process communication. This optimization was added in [PR 5224](https://github.com/NVIDIA/TensorRT-LLM/pull/5224). We have also reduced Python garbage collection (GC) impacts in [PR 5141](https://github.com/NVIDIA/TensorRT-LLM/pull/5141). To enable powerful NVTX markers for easier analysis of host overheads, TensorRT LLM provides several useful environment variables: diff --git a/docs/source/blogs/tech_blog/blog09_Deploying_GPT_OSS_on_TRTLLM.md b/docs/source/blogs/tech_blog/blog09_Deploying_GPT_OSS_on_TRTLLM.md index a027a59761de..ba898230665b 100644 --- a/docs/source/blogs/tech_blog/blog09_Deploying_GPT_OSS_on_TRTLLM.md +++ b/docs/source/blogs/tech_blog/blog09_Deploying_GPT_OSS_on_TRTLLM.md @@ -357,7 +357,7 @@ moe_config: ## Troubleshooting Tips -- If you encounter CUDA out-of-memory errors, try reducing `--max_batch_size`, `--max_num_tokens`, or `--kv_cache_free_gpu_memory_fraction`. See the [doc](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/performance/performance-tuning-guide/tuning-max-batch-size-and-max-num-tokens.md) for the explanation of these parameters. +- If you encounter CUDA out-of-memory errors, try reducing `--max_batch_size`, `--max_num_tokens`, or `--kv_cache_free_gpu_memory_fraction`. See the [doc](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/legacy/performance/performance-tuning-guide/tuning-max-batch-size-and-max-num-tokens.md) for the explanation of these parameters. - Add `print_iter_log: true` to extra LLM API options YAML file to inspect the per-iteration log. - Check GPU utilization with `nvidia-smi` while the server is running to inspect GPU status and memory usage. - If the container fails to start, verify that the NVIDIA Container Toolkit is properly installed