DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains these formats provide. While post-training weight quantization has been extensively studied, activation quantization remains considerably more challenging and typically requires calibration data. In this work, we introduce DUALQUANT, a dual-scale,…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
