Capacity Planning for Resource-Constrained Clinical LLM Serving
Deploying large language models (LLMs) under a fixed accelerator budget requires jointly deciding how computational capacity is allocated across model sizes and how incoming requests are routed across that capacity. Although prior work has extensively studied LLM routing and serving efficiency, the relative importance of fleet composition and routing under resource constraints remains less…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.