LLMilestonesA milestone timeline of AI and large language models

Prologue · Before the Transformer

PaperRumelhart, Hinton & Williams

Training neural networks with backpropagation

Used the chain rule to pass errors back from the output layer, layer by layer, to train multi-layer networks.

Why it matters

Solved the problem of how to train multi-layer networks. Every large model today is still trained this way.

Costs and controversy

Similar methods had been proposed before; this Nature paper is what made it widely known.

Built on

Perceptron

Led to

LeNetAlexNetTuring AwardNobel Prize

Source: doi.org ↗

See it on the 3D timeline →