Text Generation Inference (TGI) is Hugging Face's open-source server for running language models in production; it is now in maintenance mode.
In plain words
TGI is a ready-made server, written in Rust and Python, that loads an open model and answers generation requests. It streams tokens as they are produced, batches incoming requests together to raise throughput, can split a model over several GPUs, and offers an API compatible with OpenAI's chat format. Hugging Face has put it in maintenance mode and now recommends engines such as vLLM, SGLang, llama.cpp or MLX.
A quick example
Imagine a team that already runs TGI in a container to serve a Llama model to an internal chat tool. It still works, but for a new project they would follow Hugging Face's advice and start with vLLM or SGLang.
Where you'll meet it
- Existing Hugging Face-based serving setups.
- Hugging Face's own products, historically.
Related
Sources
- Docstext-generation-inference README, Hugging Face (GitHub) · read 29 Sept 2026
- DocsText Generation Inference, Hugging Face docs · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.