Mamba: selective state-space models
A sequence model whose cost grows linearly with sequence length, matching Transformers of the same size.
Why it matters
Revived research into alternatives to attention; hybrids of linear layers and attention later went into production models.
Led to
Source: arxiv.org ↗