FP8 is an 8-bit floating-point number format that lets AI chips train and run models faster, using a quarter of the memory of 32-bit numbers.
In plain words
Fewer bits per number means less memory and faster maths, but also less range and precision. FP8 comes in two flavours: E4M3 spends more bits on precision, E5M2 spends more on range. Software switches between FP8 and 16-bit maths layer by layer and rescales values so they stay inside FP8's narrow range.
A quick example
Imagine a lab training a language model on NVIDIA H100 chips, whose Tensor Cores support FP8. It switches on FP8, and NVIDIA's transformer engine picks FP8 or 16-bit maths for each layer and handles the rescaling. In the FP8 paper's tests, results like this matched the quality of 16-bit training.
Where you'll meet it
- Training and serving large language models on recent NVIDIA GPUs.
- Shrinking models after training (quantization).
Related
Sources
- PaperFP8 Formats for Deep Learning, Micikevicius et al. (arXiv) · read 29 Sept 2026
- DocsNVIDIA Hopper Architecture In-Depth, NVIDIA Technical Blog · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.