LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving
Multi-LoRA serving allows many task-specific adapters to share a common base model, but scaling to large adapter fleets introduces substantial storage, loading, and execution overheads. Post-hoc compression can reduce the memory footprint of trained LoRA adapters, yet smaller ranks do not automatically translate into faster serving. For instance, padding, factor access, and generic kernels can…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
