DeepSeek-V3
A 671B-parameter MoE model close to GPT-4o, with fully open weights.
~$5.57MGPU cost of the final training run
Why it matters
Its report put the final training run at about $5.6M of compute, upending assumptions about training costs.
Costs and controversy
That figure excludes earlier research and experiments; the true total cost is still debated.
Built on
Led to
Source: arxiv.org ↗