Home Office asylum AI flunks one in five accuracy checks

The UK Home Office's latest attempt to streamline asylum decisions through artificial intelligence has encountered a significant issue: a new report reveals that over one in five AI-generated interview summaries fail accuracy checks. A 20-page investigation by the Open Rights Group (ORG) highlights concerns about the systems' accuracy, oversight, and the promised efficiency gains.

Based on Freedom of Information requests and correspondence with ministers, the investigation focuses on two systems employed by Home Office caseworkers: Asylum Case Summarisation (ACS) and Asylum Policy Search (APS). Both systems utilize OpenAI's GPT-4o model via Microsoft's Azure AI Foundry.

According to the ORG's findings, 45 out of 203 AI-generated interview summaries failed initial expert checks, representing a 22 percent failure rate. This rate is substantially higher than the 9 percent rejection rate observed during the initial pilot. Additionally, 19 summaries required further review, and 56 second-stage reviews failed to meet the required standard.

The errors encompass factual inaccuracies and misrepresentations of asylum seekers' statements during interviews. The ORG also discovered that summaries could pass the checks even if they omitted critical details, provided the included information was accurate.

Despite these findings, the ORG estimates that Home Office experts only reviewed 0.3 percent of AI-generated summaries between April and July, based on the department's projected monthly caseload of 17,515. The ORG also questions whether the tools are genuinely saving time, noting that the average monthly number of initial asylum decisions has dropped by about 42 percent since April 2026, while the average time from interview to decision has increased by 29 percent.

While these figures do not definitively prove that AI caused the slowdown, they offer little evidence of the promised productivity gains.

The ORG also criticizes the oversight of the Asylum Policy Search tool, stating that expert testing has ceased despite previous assurances to MPs that checks would continue. Moreover, asylum applicants are not notified when AI is used in their cases and cannot inspect or correct the AI-generated summaries. The ORG also challenges the Home Office's claims of human oversight, pointing out that the department's own data protection assessment reveals that ACS automates tasks previously performed by technical specialists, potentially reducing the experienced layer of scrutiny.

The ORG is advocating for the suspension of both tools and independent audits, emphasizing that the current safeguards are insufficient for decisions that determine whether someone can remain in the UK. This is not the first time the Home Office's technological initiatives have faced scrutiny; in June, over 60 rights groups called for an end to AI-powered age estimation of asylum-seeking children, raising concerns about the technology's reliability for determining the crucial 16-to-18-year-old age range.

A recent inspection also found that the department's immigration system, Atlas, failed to provide caseworkers with appeal feedback, forcing staff to rely on workarounds. The Home Office did not respond to The Register's inquiries.