Attention Variants and Memory Efficiency
Overview This post is divided into eight parts; they are: • The Checkpoint Fixes the Attention Architecture • A Closer Look at Attention Memory • Multi-Head Attention • Multi-Query and Grouped-Query Attention • Sliding-Window and Mixed Attention • Multi-Head Latent Attention • What the Inference Runtime Can Change • From Cache Size to Serving Capacity The Checkpoint Fixes the Attention…
We haven't written up this one. Machine Learning Mastery has the full story — the link below goes straight to it.
