Beyond ‘Pilot Purgatory’: What does it take to build AI that works?

Artificial intelligence (AI) has transitioned from a matter of curiosity to a pressing concern for executives and boards. Companies are seeking tangible evidence of AI's value creation and grappling with the high failure rate of AI pilot projects that seldom evolve into durable, operational advantages. The issue often lies not with the model itself but with its application in production environments plagued by complexity, exceptions, and accountability requirements.

In my experience, AI initiatives fail not due to model limitations but due to poor integration. A model may excel in a sandbox but prove irrelevant to business operations if it lacks proper embedding into workflows, access to relevant data, appropriate governance, and outcome-based measurement. Thus, AI's accessibility alone is no longer an exclusive competitive advantage; it's the accompanying proprietary data, domain expertise, and disciplined continuous improvement that truly set apart the successful AI practitioners.

Our organization's Lean AI approach embodies this disciplined methodology, inspired by a Lean operating model that champions continuous improvement through iterative testing, learning, and action. Rather than pursuing technology for its own sake, we focus on embedding AI in production, where it can demonstrably enhance service, boost productivity, and generate tangible business value.

Organizations that excel in AI operationalization prioritize business value and operational friction points, deploying AI agents to automate repetitive, high-volume tasks across entire customer workflows. Every deployment, however, must be underpinned by a clear business case, owner, measurement framework, feedback loops, and a scalable plan. This disciplined approach is particularly crucial when dealing with autonomous agents that possess greater decision-making authority than conventional software.

While the conversation often revolves around the foundational layers—infrastructure and large language models—the true competitive moat emerges at the application layer, where AI seamlessly integrates into workflows, systems, exceptions, data, and human judgment that define business processes. Companies that lack control over this layer risk reducing AI to a generic capability rather than a unique differentiator.

Consider the complexities of supply chain logistics, where a single shipment's coordination involves multiple transportation modes, customs documentation across jurisdictions, facility handoffs, and dynamic market conditions. A generic AI tool cannot inherently grasp these intricacies. The crux lies in the context unique to each industry, which encompasses historical pricing patterns, warehouse operations, customer-specific policies, shipment characteristics, driver performance, market cycles, and the tacit knowledge of seasoned operations personnel.

Such context cannot be procured off the shelf; it necessitates data collection, structuring, governance, and application. AI's efficacy flourishes when it's inscribed within the technology platform, learning from genuine industry realities rather than defaulting to generic inputs.

Moreover, human input remains indispensable in refining AI agents. Through continuous feedback mechanisms, employees impart institutional knowledge to AI systems, akin to training a new employee in traditional workflows. Even seemingly mundane tasks, like scheduling freight pickups, demand a nuanced understanding of customer requirements, freight specifics, facility policies, loading dock constraints, appointment systems, and location-specific exceptions.

An AI agent can only execute such responsibilities reliably if it possesses the requisite context and oversight. Companies that transcend the pilot phase and embrace AI as an operational model—inheriting real business problems, owning the application layer, equipping agents with proprietary context, maintaining human oversight, and relentlessly measuring outcomes—will reap the true benefits of AI.

They will not be measured by the frequency of demos but by their capacity to operationalize learning at a faster pace than their competitors.