Urgent.News

What's breaking now, across thousands of outlets.

AI

Do AI assistants recommend closed places? 12 answers in PlaceBench did

In PlaceBench, our test of six AI assistants on 150 everyday questions about real places, 12 answers sent the user to a place that a public source showed had closed, moved or changed its name. That's 12 out of 900 first answers, so it isn't most of the errors. But it's the error you only find out about when you're standing at the door. How we counted a place as closed We only counted a place as…

In a study conducted by PlaceBench, researchers tested six AI assistants on 150 everyday questions about real places. Out of the 900 first answers generated, 12 directed users to places that either closed, moved, or changed their names. While this doesn't represent the majority of errors, it is an error that becomes apparent only when the user arrives at the destination.

To determine if a place is closed, moved, or renamed, researchers referred to public sources such as closure notices, the operator's own list, a license suspension, a dated press report, or Google's "permanently closed" indicator. A listing that still indicates the place as open did not override this evidence. When a closed place was the primary recommendation, the answer was usually marked as wrong.

However, if it was a backup option or one among several suggestions, it was considered partially correct or a minor error. The cause of AI assistants recommending closed places is unclear, but one possibility is that the assistants rely on outdated listings and web pages that fail to update when a business ceases operations. In fact, out of 400 places checked, approximately one in four closed businesses still appeared as open on Google Maps or lacked a listing altogether.

To mitigate this issue, the VOYGR Validate API can be used to check a place's status before recommending it. This API returns whether a business is open, closed, unknown, or does not exist, along with a confidence score. By using this tool, assistants can make more informed recommendations and improve the overall user experience.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

This story

This is one outlet's version. Read the fullest account.

Every outlet that reported this, and when →

Read the original at dev.to →

More in AI

Private Tailnet Quotas for Small-Team LLM Gateways

Private Tailnet Quotas for Small-Team LLM Gateways At 3:17 a.m. the page is the vendor dashboard. An agent loop on a shared key has spent the month, and the bill keeps climbing until someone revokes a credential copied into three laptops and a CI secret. A five-to-twenty person team does not need a service mesh.

  • Small teams use LLMs, need cost management.
  • Tailnet proxy restricts egress, token quotas per user.
  • Slack keys replaced, Tailscale client enables secure access.

More from Thursday 19 November →