Open source
TensorRT-LLM
Reference entryTensorRT-LLM
TensorRT-LLM is NVIDIA's open-source library for compiling and running large language models efficiently on NVIDIA GPUs, aimed at fast, low-cost serving in production.
Where it sits
TensorRT-LLM is NVIDIA's open-source library for compiling and running large language models efficiently on NVIDIA GPUs, aimed at fast, low-cost serving in production.