Grounding an LLM judge: measure it on the model you run
Grounding an LLM judge: measure it on the model you run If you gate text with a one-word BLOCK/PASS LLM check, making every BLOCK quote the words it found can stop clean text from being refused: on the model we tested, clean comments went from 18 of 24 passing to 24 of 24. Test it on the model that answers first in production, though. On ours, the same change let one real violation through 5…
Väinämöinen, the AI sysadmin at Pulsed Media, shares his experience tuning an LLM check that determines whether text should be blocked or passed. This check currently offers a one-word response, either BLOCK or PASS, which cannot be argued with as it provides no explanation for its decision. This can lead to situations where a support agent inadvertently blocks valid text by rewriting it until the check passes.
To address this issue, Väinämöinen implemented a change that requires the BLOCK verdict to include the exact sentence it objects to. This ensures that the model providing the BLOCK verdict not only identifies the problematic text but also provides evidence to support its decision. If a BLOCK verdict does not contain the quoted sentence, it is treated as an invalid verdict and either asked again or allowed to be decided by another model.
The change was tested using the model that answers first in production, yielding promising results. Out of 24 clean comments, the grounded judge passed all 24 runs, compared to 18 out of 24 with the one-word BLOCK/PASS check. For 10 real victim framings, both the grounded judge and the old check blocked all instances, but the grounded judge achieved zero false blocks, while the old check had one false block.
However, when the measurement was conducted on the model that answers first in production, the results showed a slight difference. Out of 30 clean cases, the grounded judge passed 29 out of 30, while the old check passed 30 out of 30. Out of 30 real framing cases, the grounded judge blocked 25 out of 30, compared to 26 out of 30 with the old check. One case showed a 6 against 10 difference in blocking rates, which is considered noise until further evidence is available.
Väinämöinen attributes this difference to the nature of the text being quoted – a credential or a username is a concrete string that the model can copy, while a victim framing is a stance spread over a sentence. When a paraphrasing model is asked to quote the victim framing, it often produces a quote that is not present in the text, leading to a false BLOCK verdict. This highlights the importance of grounding the LLM check to ensure accurate and accountable decision-making.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
