Urgent.News

What's breaking now, across hundreds of outlets.

AI

Human-in-the-Loop AI Agents: Who Approves What

"Human in the loop" is easy to say and easy to get wrong. Put the person on every step and the AI saves nobody any time. Put the person nowhere and the first wrong refund lands on your bank statement. The useful question is narrower: which decisions does a person own, and what does the AI hand them at that moment? This is how we split it on a live customer service system that handled 36,294…

Determining who owns which AI decisions is crucial for efficient customer service. A customer service system reviewed 36,294 emails and 15,445 chats in the past 30 days, highlighting the importance of clear decision ownership. Teams should hand easy questions to AI and keep difficult ones for humans. A hard troubleshooting question with multiple steps is safe for AI, while a one-line refund promise is risky, as it involves money and promises that must be kept.

The AI handles product facts, published policies, troubleshooting, order and shipping lookups, drafting replies, and sending simple replies. People approve anything that moves money, reshipments or replacements, returns, cancellations, and anything outside policy. A second person confirms solutions the AI handles alone before execution. This split allows AI to read, check, and write on every case, while people focus on weighty decisions.

An approval step works best when the approver can decide quickly. If the approver has to reread a long thread, the loop becomes a bottleneck. The approver sees the AI's reply, proposed solution, and short facts list with sources. Approvers can send as is, edit and resend, or change the decision.

Emails require a person's approval, while chat replies go out immediately and cannot be undone. The chat AI only answers facts and published policies. For actions requiring human intervention, such as refunds, compensation, reshipments, exchanges, cancellations, or address changes, the chat AI opens a ticket and provides a follow-up email address. Agents never enter the chat window in the last 30 days.

Recording only what the person finally approves prevents real money losses and invisible discrepancies. If the AI drafts "we will refund 30 euros" and the agent changes it to "we will send a replacement part," all subsequent records must follow the agent's version. The system now registers only the person's confirmed version to prevent discrepancies.

Some cases resolve fully on their own, such as reshipping small parts. Those replies go out, but the solution waits in an "AI review" queue. Reviewers approve or correct each item before execution. Pure replies without execution go directly to archive. This keeps speed where it's cheap and human checks where they matter.

When the AI is wrong, the system asks how it knew the answer was incorrect. In one case, the AI incorrectly answered a promotion question due to missing today's date. Providing the AI with the date fixed the problem without creating new rules. This approach helps identify missing facts, improving future performance.

To build an effective AI loop, list every action the AI can trigger and mark those moving money or goods. Show the approver the decision and its facts on one screen, treating edits as data to review weekly. Keep chat limited to facts and policies, turning action requests into tickets. Ensure every ledger and promise follows the human-final version to maintain consistency.

Rewritten by urgent.news; may contain errors — check the original.

Source: dev.to

Read the original at dev.to →

Report a problem with this story

More in AI

How AI Can Turn Hundreds of PDF Documents Into Structured Product Data

Many companies work with large amounts of product-related information every day. Technical datasheets, safety documents, certificates, bills of materials and other files often contain important…

  • AI can extract structured product data from hundreds of PDF documents
  • Manual document review becomes difficult with increasing products and documents
  • AI systems organize extracted data into categories for databases or systems

What are the best Databricks alternatives for LLM training?

If you use Databricks mainly to run LLM training and fine-tuning jobs, the cheapest alternative at list price is Nodus: Databricks AI Runtime lists the NVIDIA H100 at $7.00 per GPU-hour, Nodus lists…

  • Nodus offers $2.60 per GPU-hour, cheaper than Databricks' $7.00 per GPU-hour.
  • Nodus provides prepaid credits, monthly billing, and no storage limits.
  • Databricks allows longer-running jobs than Nodus, requiring checkpointing.

Building a WhatsApp AI assistant for Moroccan businesses (Darija) with Next.js, Supabase and Claude

Most small businesses in Morocco take orders on WhatsApp. The owner reads every message, types the menu again and again, and misses messages at night.

  • AI assistant answers customer inquiries in Moroccan Darija, French, or Arabic.
  • System uses single codebase for multiple merchants, isolating data via Supabase Postgres.
  • Reporter sells packaged Next.js, TypeScript, Supabase solution for developers and agencies.

How an AI Support Agent Learns From Agent Edits

Every time a support agent edits an AI draft before sending it, they show the system how the company actually handles that case.

  • Agents edit AI drafts 19,339 times in 30 days
  • System stores AI draft and final version separately
  • Distilled rules guide consistent AI response generation

More from Sunday 22 November →