Presentation: Producing the World's Cheapest Tokens: A How-to Guide
Meryem Arik discusses strategies for designing low-cost LLM inference architectures for high-volume, non-real-time workloads. She explains how software architects and engineering leaders can achieve order-of-magnitude cost reductions by making critical trade-offs across hardware, inference runtimes, speculative decoding, and smart queue reordering. By Meryem Arik
We haven't written up this one. InfoQ has the full story — the link below goes straight to it.
