{
  "id": 11219653,
  "title": "Questions for a chatbot",
  "url": "https://urgent.news/2026/10/01/questions-for-a-chatbot",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-01T15:25:26.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/lisandro_reinoso_d12ac7b9/questions-for-a-chatbot-2ic0"
  },
  "original_language": "en",
  "account": "I have an empty table with today's numbers from two weeks ago: out of 68 answers, just one passed. The hypotheses I wrote to prevent cheating when measuring again sit at the bottom. The audit stage, which costs about 28 dollars, was halted due to an expired API credit on September 15. I know what changes I made, but I don't know if the chatbot's performance has improved. When you evaluate something you've created, does it give you a score or a list of what needs fixing? And do you write down what you expect to change before conducting the evaluation?",
  "summary": "Today I have a file open with an empty table. At the top are the numbers from two weeks ago: out of 68 answers, one passed. At the bottom, the hypotheses I wrote so I wouldn't cheat myself when measuring again. The table in the middle, the one that would say whether the chatbot got better, has been empty for 16 days. What I ask it The chatbot answers questions about pasture growth rates with a…",
  "key_points": [
    "Chatbot evaluation lacks scoring system",
    "User unsure if performance improved after audit",
    "Importance of setting expectations before testing"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}