Mixtral is a Mistral AI open-weight model from December 2023 that uses a mixture of experts, so only part of the network works on each token.
In plain words
Mixtral 8x7B has eight groups of feed-forward layers, called experts, in each layer. For every token, a small router picks two of them to do the work. So although the model holds 46.7 billion parameters, it uses only 12.9 billion per token, so it runs at about the speed and cost of a 12.9-billion-parameter model. At release it matched or beat Llama 2 70B and GPT-3.5 on the benchmarks tested. Mistral has since retired Mixtral from its API.
A quick example
When Mixtral reads the word 'function' in some code, the router might send it to two experts, while the next word may go to a different pair. Only those experts' calculations are run.
Where you'll meet it
- Self-hosted chat and coding models.
- Research on mixture-of-experts designs.
Related
Sources
- DocsMixtral of experts, Mistral AI · read 29 Sept 2026
- PaperMixtral of Experts, Jiang et al. (arXiv) · read 29 Sept 2026
- DocsModels overview, Mistral AI docs · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.