Urgent.News

What's breaking now, across thousands of outlets.

More in AI

The explanation was right. The policy ID was wrong.

This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked A support model told me the correct fee, named the applicable policy in its explanation, and then put a different policy…

  • Support model provided correct fee and policy.
  • Policy ID in sourceids field was incorrect.
  • GPT-5.4-mini passed 26 of 30 pairs, Gemini-3.7-flash passed all 15.

More from Thursday 1 October →