{
  "id": 12504233,
  "title": "Changing what my phone agent was shown beat changing the model",
  "url": "https://urgent.news/2026/10/07/changing-what-my-phone-agent-was-shown-beat-changing-the-model",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-07T00:19:31.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mycmdhub/changing-what-my-phone-agent-was-shown-beat-changing-the-model-94a"
  },
  "original_language": "en",
  "account": "In a study conducted on 18 September 2026, researchers found that changing the notes provided to an AI phone receptionist made a significant difference in its performance. Instead of altering the underlying model, the researchers focused on modifying the information given to the model during each interaction. Across six different models on Kaggle, the number of incorrect bookings dropped from 17 of 216 runs to none when the agent's notes were updated. The researchers also discovered that the weakest model after the changes actually outperformed the strongest model before the changes, with the latter only correctly answering 4 out of 18 runs.",
  "summary": "This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked On 18 September 2026, two turns after its own lookup had returned a booking's real email, my AI phone receptionist told the caller no email was on file. Then it read one back anyway: \"Dhruv Patel at gmail dot com\". That's my name, at an address that exists nowhere. I didn't change the model. I changed its notes, the…",
  "key_points": [
    "Changing notes to AI phone agent improved performance significantly",
    "Incorrect bookings dropped from 17 of 216 runs to none",
    "Weakest model after changes outperformed strongest model before changes"
  ],
  "editors_take": "Tweaking the input to an AI phone agent can dramatically improve its performance, sometimes even surpassing stronger models, highlighting the importance of contextual data in AI interactions.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}