Who referees the AI agents? Enterprises aren’t sure, new report shows

A new report from OutSystems, a low-code company now marketing itself as an "Agentic Systems Platform", reveals that enterprises are struggling to manage the conflicts and governance of AI agents. Surveying over 850 IT decision-makers, 82% said their organizations can keep pace with governing, integrating and monitoring AI agents, but only 26% felt well-prepared to resolve conflicts between agents, systems and automated workflows.

The gap between confidence in managing AI agents and the difficulty in resolving conflicts is stark, highlighting a critical need for better oversight and conflict resolution mechanisms. The issue arises as AI agents, unlike chatbots, take action beyond responding, such as updating records, triggering refunds, and booking shipments.

When multiple agents within a company work in tandem, they can inadvertently interfere with each other's tasks, creating a chaotic environment that requires someone to make decisive calls about which actions should proceed and how to rectify conflicts. Currently, most enterprises rely on existing IT, security, or compliance processes to oversee autonomous agents, but only 5% have automated controls, audit trails, and formal human review points in place.

Southeast Asia, particularly Singapore, has seen this challenge firsthand, with 82% of enterprises reporting the need to roll back or shut down AI agents after launch and 94% using or piloting multi-step AI systems while only 29% could produce an audit trail for AI-driven decisions. Regulatory bodies like Singapore's Infocomm Media Development Authority (IMDA) have responded to these challenges by unveiling a starter kit for testing agentic systems and an accreditation scheme for third-party AI testers, establishing a framework to check agents' compliance and reliability.

Despite these regulatory efforts, enterprises still grapple with the complexity of ensuring that agents adhere to the right protocols, with many admitting to weaknesses in oversight and conflict resolution.