Commentary: What if the HR department's AI prefers resumes written by AI?

The Financial Times’ Pilita Clark warns that AI hiring tools may unintentionally prefer applications written by AI, adding a new layer of risk to an already competitive job market. Earlier this year, Addleshaw Goddard, a law firm, faced a hiring dilemma when an AI agent used for assessing IT job applications favored candidates who used AI to craft their resumes.

The firm's HR department found that the AI agent's list of top contenders did not align with the human-selected candidates. The discrepancy led the firm to investigate the phenomenon, which could be attributed to AI's ability to enable candidates to tailor their CVs more effectively and, potentially, the AI agent's interpretation of signs of AI usage in a resume as evidence of AI capability.

An eye-opening study conducted by researchers simulated hiring processes for various jobs, revealing that candidates using the same AI large language model as recruiters were 23% to 60% more likely to be shortlisted. The effect was most pronounced for business roles, such as sales and accountancy, and least for fields like farming and automobiles. The paper, still under revision, suggests that AI models may exhibit "self-preferencing behavior," confirming the findings of this study.

While AI bias can be mitigated through strategies like telling models to ignore the origin of resumes and using multiple assessment models, the potential for AI to discriminate against candidates who do not use AI or use the wrong type remains a concern in an era of AI-driven hiring. In the case of Addleshaw Goddard, human judgment prevailed, and the successful candidate was not the one the AI agent ranked highest. However, the incident highlights the need for continued caution and improvement in AI's role in recruitment.