What Happens When AI Is the President’s Advisor?

As the world grapples with the rapid integration of artificial intelligence into decision-making processes, including the presidency, concerns over potential biases and oversights have emerged. The June 2023 CNN report detailed a close call between the United States and China, where a military analyst utilizing an AI chatbot to assess a Chinese vessel's cargo made a critical error, mistakenly identifying it as carrying nuclear materials.

This incident highlighted the potential for AI to misinterpret data and the rapid spread of AI-assisted intelligence within the chain of command, raising questions about its impact on presidential judgment.

While AI possesses the capability to process vast amounts of intelligence data and detect patterns swiftly, there is a risk that it may amplify existing biases, presenting findings as objective truths while hidden human assumptions prevail. Historically, leaders have struggled with confirmation bias and selective attention, tendencies that AI, built on human-generated data, inadvertently perpetuates.

In situations where human judges must reconcile competing assessments, AI's role in synthesizing these judgments into a singular probability may obscure critical information and nuances that could inform a leader's decision-making process.

Consider the historical example of President Obama's efforts to locate Osama Bin Laden in 2011. The intelligence assessments ranged from 30% to 95% certainty, illustrating the inherent uncertainties in such high-stakes decisions. Introducing an AI system to assess these probabilities could provide a more precise estimate, such as 63.7%, yet this simplified figure fails to capture the underlying reasons for the disparity in opinions, the assumptions driving each assessment, or the implications of different courses of action.

The potential for AI to create synthetic consensus, where multiple agencies converge on a similar conclusion despite differing analyses, poses significant challenges. This phenomenon, dubbed "synthetic consensus," could mislead leaders into believing that a decision is well-supported and free from dispute. However, recognizing and addressing these issues is crucial.

To mitigate these risks, AI systems should be designed to act as "benevolent antagonists," challenging assumptions and highlighting areas of uncertainty rather than providing a uniform answer. Moreover, decision-makers must critically evaluate the questions posed to AI, avoiding confirmation bias by seeking disconfirming evidence and exploring multiple explanations.

By fostering a culture of inquiry and questioning the questions themselves, policymakers can ensure that AI serves as a tool for enhancing, rather than diluting, the decision-making process.