HBM (high bandwidth memory) is memory made by stacking DRAM chips vertically and placing them right beside a processor, giving AI chips very fast access to data.
In plain words
AI chips need to read huge amounts of data from memory, and slow memory leaves them waiting. HBM stacks several memory chips on top of each other and puts them in the same package as the GPU or CPU, so data travels a shorter distance and far more of it moves each second. That keeps the processor busy, which is why fast, large memory matters so much for running big AI models.
A quick example
NVIDIA's H200 was the first GPU with HBM3E memory. NVIDIA says its larger, faster memory speeds up generative AI and large language models, with inference up to twice as fast as the older H100 on models like Llama 2.
Where you'll meet it
- Data-centre GPUs and AI accelerators.
- High-performance computing and scientific simulation.
Related
Sources
- DocsHigh-bandwidth memory (HBM), Micron · read 29 Sept 2026
- DocsNVIDIA H200 Tensor Core GPU, NVIDIA · read 29 Sept 2026
- DocsNVIDIA H100 Tensor Core GPU, NVIDIA · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.