CLIP is a 2021 OpenAI model that learns to match images with text, so it can label pictures it was never specifically trained to sort.
In plain words
Most image classifiers learn from a fixed list of labelled categories. CLIP instead learned from 400 million image and caption pairs from the internet, by guessing which caption belongs with which picture. Afterwards you can describe any categories in words, and CLIP picks the text that best fits an image. This is called zero-shot classification. It matched a classic ImageNet model without using any ImageNet training examples.
A quick example
Give CLIP a photo and three captions: 'a diagram', 'a dog', 'a cat'. It scores how well each caption fits and gives almost all the probability to the right one.
Where you'll meet it
- Labelling images with categories written in plain words.
Related
Sources
- PaperLearning Transferable Visual Models From Natural Language Supervision, Radford et al. (arXiv) · read 29 Sept 2026
- DocsCLIP, OpenAI (GitHub) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.