Urgent.News

What's breaking now, across thousands of outlets.

AI

Your AI found a customer 'trend'. Check it in Python before it reaches a slide

This article was created with the help of AI. I ran and tested the code before publishing (Python 3.13.5, standard library only, 30/30 tests passing) and cross-checked the interval maths against statsmodels 0.15.0. All the data in it is invented. LLMs are now routinely used to tag customer calls, chats and tickets with themes ("invoice confusion", "login failure", "wants feature X"). The output…

In the world of customer support, AI is often employed to analyze conversations, chats, and tickets, tagging them with themes such as "invoice confusion," "login failure," or "wants feature X." The output from this tagging process usually appears as a ranked list or a chart, which is then used to inform product and marketing decisions. However, a ranked list can be misleading as it hides important information needed to trust the results.

One key issue is that a ranked list does not specify the total number of conversations analyzed, nor does it explain how many conversations were actually examined. Moreover, there is a risk of "confabulation" - where LLMs produce fluent text that is not grounded in the input data. NIST's Generative AI Profile describes this as a concern, which can be identified by supporting quotes not present in the transcript.

To address these issues, the author of this article developed a small gate that verifies quotes against the transcript before allowing them to be counted. This gate ensures that the tagging is accurate and trustworthy. The input format for this gate is a JSON file, where each conversation has an ID, period, customer ID, transcript, and AI-generated tags. An optional audit list can also be included for human validation of individual tags.

The gate follows two main steps to ensure the accuracy of the tags:

1. Verifying quotes against the transcript: To avoid issues with exact substring matching, the author normalizes both the quote and the transcript using Unicode NFKC normalization, casefolding, dropping punctuation, and squashing whitespace. Then, a substring match is performed to ensure the quote is indeed present in the transcript. If more than 5% of a theme's quotes fail this verification, the theme is marked as "NOT_READY," indicating a potential issue with the tagging prompt or pipeline.

2. Calculating a share with an honest range: The author uses the Wilson score interval instead of the traditional p ± z·√(p(1−p)/n) method. This alternative method, as described in NIST/SEMATECH e-Handbook (7.2.4.1), provides a more reliable estimate of the confidence interval for binomial proportions, ensuring that the lower limit never goes below zero. This provides a more honest representation of the data and helps avoid misleading conclusions.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

European Commission Confirms AI-Assisted ARTEX Hack That Took HR Systems Offline for 10 Days

For more than a week, European Commission staff could not check a payslip, file for leave, submit a timesheet or claim a medical reimbursement through the internal platforms they normally rely on.

  • European Commission's HR systems offline for 10 days due to AI-assisted hack.
  • Attack exploited open-source AI-powered penetration-testing tool ARTEX.
  • Commission acknowledged incident on November 19, 2026, after services restored.

Node.js Storage Pattern for AI-Generation Apps: Private Originals, Thumbnails, and Expiry

Short answer: use private object storage with a tenant-owned key namespace, keep originals and derived images as separate records, issue short-lived signed links, and make deletion a state transition…

  • Use private object storage with tenant-owned key namespace for AI image-generation apps.
  • Organize images in predictable hierarchy with tenant, asset, and variant IDs.
  • Implement disciplined deletion with row status, enqueueing, and idempotent object removal.

What broke when I put an AI voice agent on real phone lines

I build AI phone agents for home service companies. A voice agent (Retell) answers the call, and n8n workflows do the work behind it: checking a calendar, booking, writing to the CRM.

  • AI voice agents struggled to determine business open/closed status accurately.
  • Agent incorrectly claimed booked appointments, causing caller inconvenience.
  • Dead air during tool calls lasted up to 27 seconds, frustrating callers.

More from Friday 20 November →