Concept

State-space models

Sequence models, such as Mamba, that carry a fixed-size recurrent state through a sequence instead of attending to every earlier token, so that their memory per sequence does not grow with context length.