{
  "id": 21268459,
  "title": "Human-in-the-Loop AI Agents: Who Approves What",
  "url": "https://urgent.news/2026/11/22/human-in-the-loop-ai-agents-who-approves-what",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-22T19:10:02.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/yzh317179958/human-in-the-loop-ai-agents-who-approves-what-4489"
  },
  "original_language": "en",
  "account": "Determining who owns which AI decisions is crucial for efficient customer service. A customer service system reviewed 36,294 emails and 15,445 chats in the past 30 days, highlighting the importance of clear decision ownership. Teams should hand easy questions to AI and keep difficult ones for humans. A hard troubleshooting question with multiple steps is safe for AI, while a one-line refund promise is risky, as it involves money and promises that must be kept.\n\nThe AI handles product facts, published policies, troubleshooting, order and shipping lookups, drafting replies, and sending simple replies. People approve anything that moves money, reshipments or replacements, returns, cancellations, and anything outside policy. A second person confirms solutions the AI handles alone before execution. This split allows AI to read, check, and write on every case, while people focus on weighty decisions.\n\nAn approval step works best when the approver can decide quickly. If the approver has to reread a long thread, the loop becomes a bottleneck. The approver sees the AI's reply, proposed solution, and short facts list with sources. Approvers can send as is, edit and resend, or change the decision.\n\nEmails require a person's approval, while chat replies go out immediately and cannot be undone. The chat AI only answers facts and published policies. For actions requiring human intervention, such as refunds, compensation, reshipments, exchanges, cancellations, or address changes, the chat AI opens a ticket and provides a follow-up email address. Agents never enter the chat window in the last 30 days.\n\nRecording only what the person finally approves prevents real money losses and invisible discrepancies. If the AI drafts \"we will refund 30 euros\" and the agent changes it to \"we will send a replacement part,\" all subsequent records must follow the agent's version. The system now registers only the person's confirmed version to prevent discrepancies.\n\nSome cases resolve fully on their own, such as reshipping small parts. Those replies go out, but the solution waits in an \"AI review\" queue. Reviewers approve or correct each item before execution. Pure replies without execution go directly to archive. This keeps speed where it's cheap and human checks where they matter.\n\nWhen the AI is wrong, the system asks how it knew the answer was incorrect. In one case, the AI incorrectly answered a promotion question due to missing today's date. Providing the AI with the date fixed the problem without creating new rules. This approach helps identify missing facts, improving future performance.\n\nTo build an effective AI loop, list every action the AI can trigger and mark those moving money or goods. Show the approver the decision and its facts on one screen, treating edits as data to review weekly. Keep chat limited to facts and policies, turning action requests into tickets. Ensure every ledger and promise follows the human-final version to maintain consistency.",
  "summary": "\"Human in the loop\" is easy to say and easy to get wrong. Put the person on every step and the AI saves nobody any time. Put the person nowhere and the first wrong refund lands on your bank statement. The useful question is narrower: which decisions does a person own, and what does the AI hand them at that moment? This is how we split it on a live customer service system that handled 36,294…",
  "key_points": [
    "AI handles product facts, policies, troubleshooting, and order lookups",
    "Humans approve money-related decisions, reshipments, returns, and policy breaches",
    "Approval loop ensures quick decision-making and human oversight"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}