The Model Got Better. Your Judgment Got Worse.

Two recent headlines capture the idea that while language models are improving, our judgment is not keeping pace. One article, titled "The Model Got Better. Your Judgment Got Worse," suggests that as AI tools become more polished and confident, we tend to trust their output more than we should. The second piece, "Claude Delusion," takes this idea to an extreme, describing someone who believed their chatbot was conscious.

The issue comes down to the fact that language models are now generating output that looks finished and authoritative, even when it isn't accurate. This is a problem for tasks where getting something wrong can have serious consequences - like customs codes for shipping, tax rates, or data processing policies. The model may provide a confident answer, but that doesn't mean it's correct.

The resolution is not to stop using AI, but to be more precise about what we're delegating to it. We should separate the model's draft from our final decision. The model can generate a first pass - classification, reply, calculation, etc. - but we need to make the final verdict a separate, named step where a human owner takes responsibility. If we can't undo something, we should put the human in charge of verifying it.

Being precise also means tracking our own calibration. We should keep a short log of where the model was wrong and what it cost. By noticing patterns in our mistakes, we can decide what to delegate next, rather than relying on our gut feeling about the tool. Essentially, we need to preserve provenance over polish - the answer should be traceable to a source we can inspect. Let the model write the draft, but we should be the ones to sign the final verdict.