Home Office asylum AI flunks one in five accuracy checks

The Home Office's new attempt to expedite asylum decisions using artificial intelligence has faced criticism, with a report revealing that more than one in five AI-generated interview summaries failed accuracy tests. The Open Rights Group (ORG) conducted a 20-page investigation based on Freedom of Information requests and communications with ministers.

The report highlights issues with the two systems employed by Home Office caseworkers: Asylum Case Summarisation (ACS), which creates summaries of interviews, and Asylum Policy Search (APS), a chat-style tool assisting officials in searching policy information. Both systems utilize OpenAI's GPT-4o model via Microsoft's Azure AI Foundry.

The government maintains that humans retain responsibility for asylum decisions, but ORG raises concerns about the reliability of caseworkers in identifying mistakes in AI-generated information. The investigation found that 45 out of 203 AI-generated interview summaries failed initial expert checks, representing a 22 percent failure rate compared to the 9 percent rejection rate during the initial pilot.

Additionally, 19 summaries required further review, while 56 second-stage reviews failed to meet standards. The errors frequently included factual inaccuracies and misrepresentations of asylum seekers' statements during interviews. ORG also discovered that some summaries could pass the checks despite omitting crucial details, as long as the included information was accurate.

Despite these findings, ORG estimates that Home Office experts only examined 0.3 percent of summaries between April and July. The report also questions whether the tools are saving time, noting that the average monthly number of initial decisions fell by about 42 percent since April 2026, while the time taken from interview to decision increased by 29 percent.

While these figures don't definitively prove that AI caused the slowdown, they provide little evidence of the promised productivity gains. The ORG further questions the oversight of the Asylum Policy Search tool, which has reportedly been halted despite previous assurances of continued testing. The group also points out that asylum applicants are unaware when AI has been used in their cases and cannot inspect or correct the summaries generated from their interviews.

The Home Office's claims of human oversight are also doubted, as the department's data protection assessment suggests that ACS automates work previously done by technical specialists who reviewed and summarized cases before passing them to decision-makers, potentially removing an experienced layer of scrutiny. ORG calls for the suspension of both tools and independent auditing, arguing that current safeguards are insufficient for decisions that determine someone's right to remain in the UK.