bfloat16 is a 16-bit number format built for machine learning that keeps the same range as 32-bit floats while using half the memory.
In plain words
Computers store decimal numbers in a fixed number of bits. Some bits set how big or small a number can be, others set how precise it is. bfloat16 keeps all the range bits of the common 32-bit format and drops most of the precision bits. Neural networks cope well with less precision, so training runs faster and fits more in memory.
A quick example
Imagine a team training on Google Cloud TPUs that stores its model's numbers as bfloat16. Each number takes 2 bytes instead of 4, so the chip can hold a bigger model or a larger batch, and results can match full 32-bit training.
Where you'll meet it
- Training and running models on Google Cloud TPUs.
- Mixed precision: multiplying in bfloat16 and adding up results in 32-bit.
Related
Sources
- DocsImprove your model's performance with bfloat16, Google Cloud TPU documentation · read 29 Sept 2026
- DocsBFloat16: The secret to high performance on Cloud TPUs, Google Cloud Blog · read 29 Sept 2026
- PaperA Study of BFLOAT16 for Deep Learning Training, Kalamkar et al. (arXiv) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.