Skip to content

runtime: compile-gated TensorRT 10 baseline - #1

Merged
bowenzhu21 merged 2 commits into
mainfrom
codex/tensorrt-baseline
Sep 1, 2026
Merged

bowenzhu21 merged 2 commits into
mainfrom
codex/tensorrt-baseline

Conversation

@bowenzhu21

Copy link
Copy Markdown
Owner

Summary

  • add a stock TensorRT 10 enqueueV3 executor
  • use persistent CUDA input/output buffers and a non-blocking stream
  • validate the initial input_ids/logits tensor contract
  • add a TensorRT discovery module and compile/link smoke target
  • compile in NVIDIA's documented TensorRT 24.08 development container

Validation

  • CPU correctness, trace replay, artifact, and sanitizer suites pass locally
  • PR CI will compile and link the actual TensorRT/CUDA path without requiring a GPU

Scope

This establishes the stock host-sampling baseline. GPU execution, numerical parity, KV bindings, and measured performance remain separate gated milestones.

@bowenzhu21
bowenzhu21 merged commit 3fb427b into main Sep 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant