[Docs] Add Qwen2.5-VL guide for the TRT-LLM PyTorch backend, deprecate Llava1.5 guide

[faradawn]

Related TRTLLM: NVIDIA/TensorRT-LLM#18381

Related Triton Docs Indexing: triton-inference-server/server#8950

Related issue: triton-inference-server/server#8945

Tested on B200 successfully

pip install --target=$WORK/pylibs -U openai

cat > ./model_repository/tensorrt_llm/1/model.yaml <<'EOF'
model: Qwen/Qwen2.5-VL-3B-Instruct
backend: "pytorch"
tensor_parallel_size: 1
kv_cache_config:
  free_gpu_memory_fraction: 0.5

triton_config:
  max_batch_size: 0
  decoupled: False
  multimodal: True
EOF


trtllm-llmapi-launch tritonserver \
  --model-repository=$WORK/model_repository \
  --http-port=8000 --grpc-port=8001 --metrics-port=8002



root@prenyx0211:/opt/tritonserver# curl -s http://localhost:8000/v2/models/tensorrt_llm/infer -H 'Content-Type: application/json' -d '{
  "inputs": [
    {"name":"text_input","shape":[1],"datatype":"BYTES","data":["What color is the bus and what does the sign say?"]},
    {"name":"image_url","shape":[1],"datatype":"BYTES","data":["http://images.cocodataset.org/test2017/000000155781.jpg"]},
    {"name":"sampling_param_max_tokens","shape":[1],"datatype":"INT32","data":[64]},
    {"name":"sampling_param_exclude_input_from_output","shape":[1],"datatype":"BOOL","data":[true]}
  ],
  "outputs": [{"name":"text_output"}]
}'

>>> {"model_name":"tensorrt_llm","model_version":"1","outputs":[{"name":"text_output","datatype":"BYTES","shape":[1],"data":["The bus is yellow and white, and the sign on the bus says \"Out of Service.\""]}]}root@prenyx0211:/opt/tritonserver#

[whoisj]

LGTM

[greptile-apps]

Confidence Score: 2/5

The PR is not safe to merge until arbitrary media URLs are fetched safely and the optional numeric response tensors conform to their declared Triton datatypes.

Findings

1.   Unsafe image URL fetching ▶
2.  Numeric outputs use strings ▶
3.  Deprecation names wrong architecture ▶
4.  Repository copy is state-dependent ▶

Summary

• Adds optional image-URL request handling and multimodal prompt construction.
• Documents creation and launch of a Qwen2.5-VL model repository using the 26.07 Triton container.
• Adds the matching Triton input/output contract.
• Redirects users away from the legacy Llava1.5 deployment.
• The new media-fetching path needs destination and resource protections, and optional numeric responses currently use datatypes inconsistent with the model configuration.

Diagram

sequenceDiagram
    participant Client
    participant Triton as Triton llmapi backend
    participant Loader as async_load_image
    participant Target as Supplied HTTP(S) target
    participant Model as Qwen2.5-VL

    Client->>Triton: text_input + image_url[]
    Triton->>Triton: Check URL scheme only
    loop Every image URL
        Triton->>Loader: Load URL
        Loader->>Target: Outbound HTTP(S) request
        Target-->>Loader: Image response
    end
    Loader-->>Triton: Multimodal image data
    Triton->>Model: Rendered prompt + images
    Model-->>Triton: Generated text
    Triton-->>Client: text_output

Loading

_{Reviews (1) · Last reviewed commit: "docs: trim three more lines from the Qwe..."}

[greptile-apps]

Numeric outputs use strings

Optional numeric outputs are constructed with string or object NumPy dtypes even though config.pbtxt declares them as TYPE_FP32, TYPE_INT32, or TYPE_INT64. For example, requesting return_cumulative_logprob emits an object/string tensor for a TYPE_FP32 output. The performance metrics have the same problem because they use self.output_dtype, which comes from the TYPE_STRING text_output configuration. Triton can therefore reject these responses instead of returning the requested metrics. Construct each tensor with the dtype declared for that output.

[greptile-apps]

Deprecation names wrong architecture

The warning says this guide describes tensorrtllm_backend's all_models/multimodal path, but the guide actually builds TensorRT engines itself and serves a tutorial-local repository containing custom Python vision and wrapper models. This gives readers an inaccurate explanation of which deployment architecture reached end of life. Please describe the actual manual-engine, custom-repository path being deprecated or cite the precise upstream component that applies to it.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Repository copy is state-dependent

This command only creates the documented layout when /workspace/model_repository does not already exist. On a rerun or after a partial attempt, cp -r creates model_repository/llmapi/tensorrt_llm, while the following commands expect model_repository/tensorrt_llm and fail. Make the destination state explicit or copy the source directory's contents so readers can safely repeat the tutorial.