AI doesn’t just answer questions – it legitimises bad ones
Years ago, I watched a focus group explain itself into a contradiction. We asked people what mattered when choosing car insurance. Price won. The deductible came next. Reputation did not appear. During coffee, I asked the interviewer to change the question. How much cheaper would a less prestigious insurer have to be before they switched? One participant, who 15 minutes earlier ranked price…
In a recent study, 758 consultants from the Boston Consulting Group were tested on a challenging business task beyond AI's capability. When these consultants utilized AI, they were 19% less likely to arrive at the correct answer. However, AI can also generate good answers to both poorly framed and correctly framed questions, making it more difficult to question the initial question.
This is due to AI's ability to produce persuasive material that supports its answer, providing more structure, evidence, and reassurance. This phenomenon, known as "persuasion bombing," can prevent people from re-evaluating the question that led to the good answer. Additionally, a study in a Shenzhen court revealed that judges using large language models to generate reasoning based on an initial decision were more likely to revise the reasoning to support the final judgment.
The question of whether AI has provided the correct answer is a downstream safeguard, but moving the decision point earlier to assess whether the question itself is appropriate and well-phrased is the most effective approach. This shift in responsibility from verifying the answer to deciding on the question's validity is a crucial task for executives.
They must ensure that the question being asked is worth investigating, which may require challenging existing assumptions and reframing the problem. By addressing the question early in the process, before the analysis accumulates weight, executives can avoid investing significant resources in a flawed inquiry.
Written by urgent.news from SCMP Tech's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
