Mixtral 8×7B: MoE takes off in open source
A mixture-of-experts (MoE) model: many parameters in total, only some of them active each time.
46.7B / 12.9Btotal / active parameters per token
Why it matters
Stronger results at lower inference cost, helping make MoE one of the mainstream architectures.
Led to
Source: mistral.ai ↗
Official page not opened for verification; URL taken from citations in multiple reports.