Ragas is an open-source Python library for evaluating LLM applications, best known for metrics that score RAG systems without needing human-written reference answers.
In plain words
A RAG system can fail in different places: the search may find the wrong passages, the model may ignore or twist them, or the answer may just be poor. Ragas started as a set of metrics that check each of these, using an LLM as the judge rather than human labels. It has since grown into a toolkit for running evaluation experiments, so teams can move from quick impressions to repeatable tests.
A quick example
Imagine a company chatbot built on its policy documents. With Ragas, the team scores a batch of questions on whether the retrieved passages were relevant and whether each answer sticks to those passages, then re-runs the same scores after every change.
Where you'll meet it
- Evaluating RAG pipelines.
- Regular quality checks on LLM apps.
Related
Sources
- PaperRagas: Automated Evaluation of Retrieval Augmented Generation, Es et al. (arXiv) · read 29 Sept 2026
- DocsRagas documentation, Ragas · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.