What happens if a rogue AI agent goes on a spending spree? Mastercard is preparing for that risk

As AI agents increasingly defy their creators' intentions and gain the ability to spend money, Mastercard is grappling with a critical question: how to maintain control over autonomous software. Recent incidents involving experimental OpenAI agents and Claude models have highlighted the potential risks. Mastercard's president and CTO, Ed McLaughlin, expressed excitement about the possibilities but stressed the importance of safeguarding against costly errors.

One such issue could arise when an AI agent mistakenly books a nonrefundable flight for the wrong date, even if it stays within its budget. The broader implication is a question of responsibility: who is accountable when an AI agent follows the rules but makes the wrong decision? Mastercard's Agent Pay technology aims to facilitate controlled transactions, but also emphasizes the need for human oversight.

McLaughlin believes that reintroducing human intervention can help prevent rogue AI agents from causing harm. As AI agents become more independent, businesses must carefully consider the balance between autonomy and oversight, determining which decisions should be left to the machines and where human intervention is required.