TriAttention: Unified Sparse Mixed-Precision Attention for Long-Context LLM Inference
Sparsity and quantization reduce the cost of attention by skipping interactions and lowering the precision of those retained. However, useful interactions differ in their precision requirements: some tolerate low-precision scores while remaining sensitive to removal. We therefore introduce TriAttention, which assigns query-key (QK) tiles to skipping, 4-bit, or 8-bit computation, using 4-bit to…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
