TRL is Hugging Face's open-source library for post-training language models, with ready-made trainers for methods such as SFT, DPO and GRPO.
In plain words
TRL packages common post-training methods as trainers: supervised fine-tuning, direct preference optimisation and group relative policy optimisation, among others. Each trainer builds on the Hugging Face Transformers trainer and can scale across many GPUs with tools like DeepSpeed and FSDP.
A quick example
Imagine a team with a small open model and a set of answer pairs where people marked one answer as better. They load the model and pairs into TRL's DPO trainer, run training, and get a model that leans toward the preferred style.
Where you'll meet it
- Instruction tuning and preference tuning of open models.
- Post-training open models with GRPO.
Related
Sources
- DocsTRL README, Hugging Face (GitHub) · read 29 Sept 2026
- DocsTRL documentation, Hugging Face · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.