Attention Quantization for Tabular Foundation Models
With the recent rise and adoption of tabular foundation models, optimizing their inference performance becomes an emerging field for efficiency research. While the models are architecturally similar to transformer-based large language models (LLMs), the size and serving patterns differ significantly. We show that the focus should be on the attention calculation and less on weight or KV cache…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
