LangSmith is LangChain's platform for tracing, testing and monitoring LLM apps and agents, so teams can see what happened in each run and measure quality.
In plain words
When an agent gives a bad answer, you need to see every step it took. LangSmith records traces of each run, from single calls to performance across all of production. It also runs evaluations: it scores outputs against datasets and criteria, so you can compare versions and catch regressions before shipping, and keep scoring after launch. It works with many frameworks, not only LangChain.
A quick example
Imagine a support agent that sometimes quotes the wrong refund policy. The team opens the trace of a bad run, sees which document was retrieved, and adds that run to a dataset. Every new version of the agent is then scored on it, so the same mistake is caught before it ships again.
Where you'll meet it
- Debugging and monitoring LLM agents in production.
- Offline evaluation before releasing a new version.
Related
Sources
- DocsLangSmith Observability, LangChain documentation · read 29 Sept 2026
- DocsLangSmith Evaluation, LangChain documentation · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.