Grade Your LLM Pass/Fail and You Will Ship a Disaster

I administered a 29-question exam to a language model. Grading the exam proved challenging, as the grading process itself could lead to disastrous outcomes. If the model missed just five questions due to typos, I was unsure whether to ship the results or not. The problem arises when it's unclear which five questions were missed.

If the missing questions involved typos, the model could still be shipped, but if one of those five questions involved reading comprehension, the results would need to be canceled. Shipping a model with a single reading comprehension error could lead to disastrous consequences. The grading system should focus on the severity of errors, not just the count.

The only criterion for grading is whether the error is reversible. If the error is irreversible, such as loading wrong goods onto a truck, the results should not be shipped. A wrong confirmation is worse than no confirmation at all. This principle should guide the grading process. The same score can lead to two different outcomes, depending on which question was missed.

In one scenario, the model could be shipped, while in another, it cannot. The grader itself had bugs, causing accidents and leading to fixes that made the grading process worse. To avoid these issues, it's crucial to save every model answer to a file and never discard them. Re-grading all questions can be a time-consuming process, but with saved answers, it can be done quickly.

This approach also prevents losing valuable data, as seen in a previous experiment where a failed request resulted in the loss of 5,000 items paid for using an API. In conclusion, the focus should be on the irreversible errors rather than the count of correct answers. The takeaway is that among the failures, one should check if anything is irreversible. If so, the model should not be shipped.