Pinocchio: Fast Uncertainty Estimates for Black-Box Language Models
In high-stakes decision-making applications of large language models (LLMs), practitioners require not only accurate LLMs but also uncertainty estimates for their predictions. Existing approaches to uncertainty estimation for LLMs require access to log-probabilities output by the model or require fine-tuning access. However, many industrial LLM products use closed-source API models, and many such…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
