FICO Says Businesses Need to Explain Their AI Decisions

One line from Balkrishan "BK" Kalra, president and CEO of Genpact, has stuck with reporter since reporting on the company's AI pivot: "We are proud engine room people. We are not the showroom people." Kalra argues that understanding where processes break, exceptions pile up, and where judgment still matters becomes more valuable as AI agents take on more execution.

A Genpact and HFS Research study reveals that technology, data, process, and talent debt limit AI value for 85% of leaders, while only 6% have established programs, executed them, and measured results. Genpact's solution is called agentic operations, which redesigns processes so agents handle execution while people oversee the system and remain responsible for outcomes. Kalra emphasizes that "There is no Artificial Intelligence without Process Intelligence."

At FICO, president Nikhil Behl discusses how companies can keep AI decisions explainable. The decision layer connects the data feeding a model, the model's output, and the business decision that follows, preserving a record of how the decision was reached. Visibility into what goes into a model, what comes out, and what businesses do with the result is crucial.

Behl explains that an undocumented input makes for an indefensible output. To maintain visibility, companies need a way to track the inputs, outputs, and downstream systems consuming the outputs - called "lineage." Regulated industries must explain their model decisions to auditors and regulators. Strong governance can help institutions move AI into live operations while maintaining oversight.

Every automated decision should leave behind a record of data, confidence level, and outcomes. Some high-stakes decisions still need human review.