When AI sounds certain, ask why

AI technology has become highly proficient at generating answers to complex queries. However, savvy business leaders do not solely rely on these answers when making decisions. They scrutinize the origin of the information, the assumptions that shaped the conclusion, and the level of confidence they should assign to the recommendation.

These inquiries are gaining significance as AI assumes a more prominent role in corporate decision-making processes. Marketing and insights teams utilize AI to evaluate campaign ideas and synthesize extensive consumer research, while executives leverage it to pinpoint growth opportunities, evaluate competitive threats, and stress-test substantial investments.

Once AI starts shaping decisions rather than merely expediting tasks, comprehending how it arrived at a particular conclusion becomes as critical as the conclusion itself.

To illustrate, consider a consumer packaged goods (CPG) firm eyeing a venture into convenience stores while concurrently sustaining sales in supermarkets. The decision must weigh numerous variables, including the impact on supermarket sales, pricing, customer demographics, channel growth, and long-term brand implications. No solitary report contains all this data.

A leader must gather it from various sources and analyze it comprehensively before determining whether to proceed with the expansion. AI can significantly expedite this process by synthesizing years of research, uncovering patterns across hundreds of documents, and presenting insights in mere minutes – enabling teams to dedicate less time to information gathering and more time to evaluating it.

However, AI does not eliminate the need for human judgment. Leaders remain responsible for grasping the reasoning behind the recommendations they ultimately employ. The answer merely offers a partial perspective of the story.

This is where many frequently used AI tools today may fall short. Numerous AI tools generate answers that seem convincing, yet they lack any indication of how the system arrived at those conclusions. These answers synthesize proprietary research, web data, and AI-generated content, with little transparency into how each component contributed to the final recommendation.

Consequently, many current AI tools function like black boxes, offering recommendations without the necessary context. While this opaque approach could suffice for exploratory or non-critical applications, decisions involving substantial investments, new product launches, or strategic planning demand visibility and transparency.

For instance, if AI recommends entering convenience stores due to anticipated growth in consumer demand, the recommendation itself might be reasonable; however, decision-makers should also understand the sources, the weight assigned to each source, which conclusions are backed by evidence, which are inferred, and where the available information leaves room for uncertainty.

Without such visibility, it becomes challenging to ascertain whether one is acting on well-supported evidence or merely accepting a captivating narrative.

Uncertainty is an inherent aspect of decision-making. Many people mistakenly perceive AI as a weakness when it comes to uncertainty. However, AI can streamline this process into a polished answer, potentially masking the uncertainty. Yet, those unknowns often prove the most valuable output. Acknowledging weak evidence, conflicting findings, or missing information equips organizations to ask better questions, gather additional research, and avert making crucial decisions based on an incorrect sense of certainty.

The "Glass Box" AI model embodies these principles. Instead of viewing transparency as a singular feature, this approach enhances visibility into the information, reasoning, and uncertainty within enterprise AI. At its core, every AI output should provide an explanation for its reasoning that is comprehensible and retrievable. Leaders must be able to examine the evidence assessed, the filters applied, and how the evidence transformed into a conclusion.

Each claim should also include references to specific pages and passages in the source materials rather than broadly referencing an entire library of documents that necessitate manual review. A Glass Box AI approach distinctly distinguishes what the source material stated from what was inferred by the AI. Evidence that carries thin weight should be identified thus and not obscured by presentation techniques.

This model should also flag gaps in knowledge, as well as what was found. The absence of evidence regarding a key assumption should be included in the report, allowing users to consider it during the decision-making process. Crucially, it should preserve user intervention. Leaders should have the ability to question, reject, or modify an AI system’s conclusions – and record the rationale behind their decision.

If an AI suggests that convenience stores could enable a CPG firm to charge higher prices, yet a team member in product believes otherwise, that disagreement should be documented. Transparency should permeate the AI system's processing cycle, not just when the answer is finalized. While working, the system should demonstrate what it is searching for, what it is weighing, and where it is converging.

This enables users to detect issues early and adjust the weighting before the recommendation is finalized. Trusted AI is "Glass Box" AI. Organizations are progressively relying on AI to make strategic decisions concerning enterprise development, capital allocation, product innovation, marketing strategy, and more. In this evolving landscape, trust cannot be a mere assumption.