AI Pays Off, But Few Leaders Are Very Confident Explaining Its Decisions

A recent FICO survey of 1,004 senior data and AI leaders reveals that while AI is meeting or exceeding initial return expectations for many businesses, only 28.5% of customer-facing decisions, such as personalization or low-risk customer service, are currently made by AI systems. Furthermore, just 5.2% of these decisions are made with the confidence to explain them to regulators and customers.

Rachael Hadaway, FICO's AI product vice president, suggests that companies can quickly reap benefits from AI by handling manual workload tasks without venturing into high-risk areas. However, some decisions may necessitate an explanation to the customer or regulator regarding the decision-making process. FICO's report, "State of Responsible AI 2026: The Confidence Gap in AI Decisions," highlights the challenges in explaining AI decisions, particularly for customer-impacting cases.

Scott Zoldi, FICO's chief analytics officer, emphasizes the need to understand the specific factors driving a model's decision, not just its general behavior. While 82.8% of respondents believe their organizations meet or exceed anticipated regulatory requirements, only 60.5% rated responsible AI adherence as a four or five on a five-point scale.

This suggests that confidence in regulatory readiness may depend on the level of required explanation. The survey also found a significant gap between respondents' confidence in regulatory readiness and their actual incorporation of responsible AI standards into operations. Only 71.2% of organizations have partially built a shared platform for collaboration between AI and technology leaders.

Zoldi warns that lack of trust and predictability could be barriers to generating ROI from agentic AI systems, which can operate with some autonomy.