BERT is a 2018 Google language model that reads text in both directions at once, so it understands each word using the words before and after it.
In plain words
Many earlier models read text left to right. BERT, short for Bidirectional Encoder Representations from Transformers, looks at the whole sentence together. It is pre-trained by hiding some words and learning to guess them from the rest. Once pre-trained, it can be adapted to a specific task by adding just one extra output layer and fine-tuning.
A quick example
In pre-training, BERT might see 'the man [MASK] up, put his [MASK] on ...' and learn to fill the gaps with 'jumped' and 'basket' using clues from both sides of each blank.
Where you'll meet it
- Question answering and language inference.
- Building task-specific models by fine-tuning.
Related
Sources
- PaperBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al. (arXiv) · read 29 Sept 2026
- DocsBERT, Google Research (GitHub) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.