Linear Attention

Flash Linear Attention: Efficient Attention Mechanisms for Transformers

The transformer architecture has been the dominant model for sequence processing since its introduction, but it carries a fundamental limitation: …