Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression
Per-matrix singular value decomposition (SVD) truncation is Eckart-Young optimal in the whitened Frobenius norm, but errors from independently compressed matrices compound through the block's nonlinear forward pass. Inspired in part by hierarchical variational optimization in quantum many-body methods, we introduce a three-level chain that widens optimization scope from individual matrices to…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
