AI safety is the field working to stop AI systems from causing accidents or harm, from everyday mistakes today to risks from much more powerful future systems.
In plain words
Machine learning systems can behave in unintended ways, for example by chasing the wrong goal or failing in situations unlike their training. AI safety studies these failures and how to prevent them, through better training, testing for dangerous failures, interpretability and oversight. Some labs, such as Anthropic, were founded mainly around this concern.
A quick example
Imagine a lab about to release a new model. It first runs tests to see whether the model can be tricked into giving dangerous instructions and whether it stays reliable on unusual inputs. Fixing what those tests find is AI safety work.
Where you'll meet it
- Testing models before release.
- Research on alignment, interpretability and oversight.
Related
Sources
- PaperConcrete Problems in AI Safety, Amodei et al. (arXiv) · read 29 Sept 2026
- DocsCore views on AI safety: When, why, what, and how, Anthropic · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.