{
  "id": 9119442,
  "title": "8 of my AI agent's 30 test calls failed. Every one was my fault.",
  "url": "https://urgent.news/2026/09/22/8-of-my-ai-agents-30-test-calls-failed-every-one-was-my-fault",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-22T09:38:07.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/rizkynandapr/8-of-my-ai-agents-30-test-calls-failed-every-one-was-my-fault-3kff"
  },
  "original_language": "en",
  "account": "Eight out of 30 test calls conducted by an AI intake agent for home service contractors in the US failed, all due to human error. The AI intake agent is designed to handle HVAC, plumbing, and roofing emergencies, and it answers phone calls (or SMS, or web forms), determines urgency, collects necessary information, and writes a JSON record for the contractor's system. A total of 30 scripted test calls were created, with ten per trade. The agent's responses were scored on nine criteria, four of which were checked by plain code, while the other five required a secondary model for assessment. Four cases were found to be impossible to pass, nearly a quarter of the test suite, due to the lack of mandatory information like phone numbers and addresses that were never provided in the scripted caller lines.",
  "summary": "The record said there was a gas leak. It did not say where. { \"urgency\" : \"emergency\" , \"emergency_type\" : \"gas\" , \"callback_number\" : null , \"service_address\" : { \"street\" : null , \"city\" : null , \"postal_code\" : null }, \"outcome\" : \"escalated\" } My AI intake agent wrote that after a caller said their furnace was out and the basement \"kind of smells like eggs.\" It told them to leave the house…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}