LLMilestones · A milestone timeline of AI and large language models

This page is an interactive 3D timeline. Here are all the milestones as text: All milestones · 全部里程碑

Prologue · Before the Transformer 1943 — 2016

From a mathematical model of one neuron to beating the world Go champion.

1943-12 The artificial neuron: the McCulloch-Pitts model — Described a neuron in math and logic: sum the weighted inputs, and fire if the total passes a threshold.
1958-11 The perceptron: a machine that learns — Proposed the perceptron, which adjusts its own weights from examples, and built it in hardware as the Mark I.
1959-10 Receptive fields in the cat's visual cortex — Found that different neurons in the visual cortex respond to edges at specific orientations, combined layer by layer.
1980-04 Neocognitron — Copied the visual cortex's hierarchy, recognizing shapes by combining local receptive fields layer by layer.
1986-10-09 Training neural networks with backpropagation — Used the chain rule to pass errors back from the output layer, layer by layer, to train multi-layer networks.
1989-12 LeNet: reading handwritten digits with a CNN — Trained a convolutional neural network with backpropagation to read handwritten zip codes.
2012-09-30 AlexNet — A deep convolutional network trained on two GPUs that won the ImageNet image-recognition challenge by a huge margin.
2013-12-19 Playing Atari with deep reinforcement learning — Combined a deep neural network with reinforcement learning and learned several Atari games from screen pixels alone.
2014-06-10 Generative adversarial networks (GANs) — A generator and a discriminator compete, and the generator gradually learns to produce convincing fakes.
2016-03-15 AlphaGo beats the world Go champion — Defeated world Go champion Lee Sedol 4 to 1.

Act I · Foundations 2017 — 2019

One paper dropped recurrence, and the pre-training paradigm was born.

2017-06-12 Attention Is All You Need — Introduced the Transformer, an architecture built entirely on self-attention, dropping RNNs and convolutions.
2018-06-11 GPT-1: Generative Pre-Training — Pre-train on huge amounts of text by predicting the next word, then fine-tune for specific tasks.
2018-10-11 BERT — A bidirectional Transformer encoder that learns language through fill-in-the-blank pre-training.
2018-12 AlphaFold wins CASP13 — Ranked first in the 13th protein-structure prediction contest, CASP13.
2019-02-14 GPT-2: Too dangerous to release? — More than ten times larger, and able to write coherent long text without fine-tuning.
2019-03-27 Bengio, Hinton and LeCun win the 2018 Turing Award — ACM awarded the 2018 Turing Award to Yoshua Bengio, Geoffrey Hinton and Yann LeCun.
2019-10-23 T5: Text-to-text for everything — Recast translation, classification, summarization and every other task as text in, text out.

Act II · Scaling up 2020 — 2022.10

Bigger models and more data meant more capability, and it turned out to be predictable.

2020-01-23 Scaling Laws for Neural Language Models — Found that loss falls smoothly, as a power law, with parameters, data and compute.
2020-05-28 GPT-3: Language Models are Few-Shot Learners — Give it a few examples in the prompt and it can do a new task without retraining.
2021-06-17 LoRA: Low-Rank Adaptation — Freeze the big model and train only small low-rank matrices, cutting fine-tuning cost dramatically.
2021-06-29 GitHub Copilot preview — Built on the Codex code model, it autocompletes whole blocks of code in the editor.
2022-01-28 Chain-of-Thought prompting — Have the model write out its reasoning before answering, and accuracy on math and logic jumps.
2022-03-04 InstructGPT: alignment with RLHF — Used reinforcement learning from human feedback (RLHF) to teach the model to follow instructions and give the answers people want.
2022-03-29 Chinchilla: data matters too — For the same compute, a smaller model trained on more data beats a bigger one.
2022-04-04 PaLM — Google's very large language model, showing abilities such as reasoning and explaining jokes.
2022-05-27 FlashAttention — Reorders the attention computation to minimize GPU memory reads and writes; exact results, much faster.
2022-10-06 ReAct: reasoning and acting together — Interleaves reasoning steps with actions such as searching or calling tools, then keeps thinking based on the results.

Act III · The ChatGPT moment 2022.11 — 2023

A chat box showed the world what large models could do, and the race began.

2022-11-30 ChatGPT launches — GPT-3.5 plus RLHF, packaged as a free chat box and launched as a "research preview".
2022-12-15 Constitutional AI: alignment from a set of principles — Instead of labeling every harmful answer by hand, the model critiques and revises itself against a written set of principles.
2023-02-24 LLaMA: the spark of open-source models — A family of 7B to 65B models, released to researchers on application.
2023-03-14 GPT-4 — A big jump in capability; it demonstrated image input and reached near-top human scores on many professional exams.
2023-03-14 Claude launches — Anthropic, founded by former OpenAI staff, released the first Claude on the same day as GPT-4.
2023-03-16 ERNIE Bot: the "hundred-model war" begins — The earliest chat-model product from a major Chinese tech company.
2023-05-23 QLoRA — Combines 4-bit quantization with LoRA to fine-tune huge models on a single GPU.
2023-05-29 DPO: Direct Preference Optimization — Learns directly from pairs of human preferences, skipping the separate reward model and reinforcement learning of RLHF.
2023-07-18 Llama 2: now commercially usable — Open weights, free for most companies to use commercially.
2023-08-03 Qwen goes open source — Alibaba open-sourced Qwen-7B, then kept releasing a full range of models from small to large.
2023-12-01 Mamba: selective state-space models — A sequence model whose cost grows linearly with sequence length, matching Transformers of the same size.
2023-12-06 Gemini 1.0 — A natively multimodal model, launched after Google merged Brain and DeepMind.
2023-12-11 Mixtral 8×7B: MoE takes off in open source — A mixture-of-experts (MoE) model: many parameters in total, only some of them active each time.

Act IV · Multimodal, open source catches up 2024

Models learned to see, hear and speak; open models closed in on the frontier.

2024-02-15 Gemini 1.5: a million-token context — Reads about 700,000 words, about an hour of video, or over 30,000 lines of code at once.
2024-03-04 Claude 3 family — Three tiers, Haiku, Sonnet and Opus; Opus beat GPT-4 on many benchmarks.
2024-04-18 Llama 3 / 3.1 405B — 8B and 70B models in April; the 405B Llama 3.1 in July.
2024-05-13 GPT-4o: natively multimodal — One model handles text, images and voice, with real-time spoken conversation, interruptions and emotion.
2024-05-21 Scaling Monosemanticity — Extracted millions of interpretable features from a production model, Claude 3 Sonnet, such as one for the Golden Gate Bridge.
2024-08-06 Scaling test-time compute optimally — Showed that spending compute wisely at inference time can beat a much larger model.
2024-09-12 o1: the year of reasoning models — Trained with reinforcement learning to think at length before answering.
2024-10-08 Hopfield and Hinton win the Nobel Prize in Physics — The 2024 Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational work on machine learning with artificial neural…
2024-10-22 Computer Use: AI operates a computer — The model uses a computer by looking at screenshots, moving the mouse and typing.
2024-11-25 MCP: Model Context Protocol — An open protocol that lets models connect to tools and data sources in a uniform way.
2024-12-26 DeepSeek-V3 — A 671B-parameter MoE model close to GPT-4o, with fully open weights.

Act V · Reasoning and agents 2025

First think, then answer, then go and do the work.

2025-01-20 DeepSeek-R1: an open reasoning model — Open-sourced under MIT, with reasoning on par with o1 and its full thinking visible.
2025-02-24 Claude 3.7 Sonnet and Claude Code — A hybrid reasoning model, plus a coding agent that reads, edits and runs code right in the terminal.
2025-03-06 Manus: a general AI agent — Plans, browses and writes code on its own inside a cloud virtual machine, delivering a finished task rather than just an answer.
2025-03-25 Gemini 2.5 Pro — Gemini with built-in thinking; it topped LMArena and other leaderboards at launch.
2025-04-05 Llama 4 — Meta's first MoE Llama, released as Scout and Maverick.
2025-04-29 Qwen3 — One model can switch between thinking and non-thinking modes; open-sourced under Apache 2.0.
2025-05-22 Claude 4 (Opus 4 / Sonnet 4) — Focused on coding and on agents that work continuously for long stretches.
2025-07-11 Kimi K2 — A trillion-parameter open-weight MoE model (modified MIT license), built for agents and tool use.
2025-08-05 gpt-oss: OpenAI opens weights again — Two open-weight reasoning models, 120B and 20B, under Apache 2.0.
2025-08-07 GPT-5 — Unifies quick answers and deep reasoning in one system that decides when to think.
2025-09-17 DeepSeek-R1 published in Nature — The R1 paper was peer-reviewed and published in Nature.
2025-11-18 Gemini 3 — Google's next-generation flagship, with the top scores at the time on many reasoning and multimodal benchmarks.
2025-11-24 Claude Opus 4.5 — Further gains in coding and agentic tasks, with a big price cut.

Act VI · The frontier and its limits 2026

Models so capable they had to be locked up first: capability, safety and openness all pushed to the edge.

2026-01-30 OpenClaw: an open-source personal agent — An open-source agent that runs on your own computer or server, takes tasks through WhatsApp, Telegram, Slack and other chat apps, and…
2026-02-16 Qwen3.5 — Open-sourced a 397B-parameter MoE model that activates only 17B parameters per inference.
2026-02-19 Gemini 3.1 Pro — Google released Gemini 3.1 Pro in preview, focused on complex reasoning.
2026-04-07 Claude Mythos Preview and Project Glasswing — A model exceptionally good at finding and exploiting software vulnerabilities, offered only to a few organizations through Project…
2026-04-08 Muse Spark — The first model from Meta Superintelligence Labs: natively multimodal, with no open weights.
2026-04-23 GPT-5.5 — OpenAI released GPT-5.5 (codename Spud), improving agentic coding and research ability.
2026-04-24 DeepSeek-V4 — Released two open-weight models, V4-Pro and V4-Flash, with a 1M-token context.
2026-06-09 Claude Fable 5 and Mythos 5 — The same model as Mythos 5, released to the public with safeguards added; Anthropic's top-tier model.
2026-07-24 Claude Opus 5 — Performance close to Fable 5 at half its API price.
2026-07-27 Kimi K3 — 2.8T parameters, native vision and a 1M-token context; weights released on July 27.
2026-09-03 GPT-6 (Astra / Sol / Luna) — On September 3 OpenAI launched its strongest model, GPT-6 Astra, to a small group of organizations, opening a restricted version to paid…
2026-09-22 Claude Opus 5.5 — The first of the Claude 5.5 family, on par with Fable 5.1 on most tasks.
2026-10-13 Gemini 4 Argon — The first Gemini 4 model, launched in Antigravity and the Gemini API.
2026-10-15 Muse Spark Open — Released weights for an 816B-parameter MoE model (about 42B active) that takes text, image, video and audio input.