Learning the Cost of Reliable Inference
Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on these platforms typically use a fixed price per token, preventing users from achieving the most competitive price for their tasks. % workloads. In this work, we design a procurement platform where token prices for each task are driven by provider…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
