It’s time to open the black box of AI-driven employment decisions

Discrimination in employment decisions often feels like an opaque black box to workers. When individuals are denied promotions, overlooked for opportunities, or selected for layoffs, they frequently lack insight into the underlying process. This lack of transparency makes it challenging to ascertain whether unlawful discrimination has occurred and difficult to hold employers accountable under existing federal and state antidiscrimination laws.

With the rise of artificial intelligence (AI) in employment decisions, the "black box" is likely to become even more enigmatic. After leaving Big Tech and witnessing the aftermath of mass layoffs, the author now fervently advocates for transparency in AI-driven employment decisions.

The author's experience fighting pregnancy discrimination spurred her to investigate whether pregnant workers were disproportionately affected by layoffs at major tech companies. While the author does not claim every layoff involving a pregnant worker is unlawful, the dearth of evidence leaves critical questions unanswered. More recently, litigation against Meta highlighted AI's role in employee layoffs, particularly for those on protected medical and family leave.

Meta's AI-driven layoffs allegedly relied on performance ratings, calibration scores, and productivity metrics that employees on leave could not accumulate, further exacerbating the issue.

Washington state, home to numerous tech companies and startups, has an opportunity to lead on this challenge. The state attorney general's AI task force recently published a report recommending the development of worker-centered principles for AI use in employment contexts. These principles aim to address the risks of bias, inequity, and oversurveillance that accompany AI technologies.

Though a proposed bill did not pass during the 2025 legislative session, the task force's recommendations remain crucial. Whether achieved through legislative action or a gubernatorial convening, Washington should bring together workers, employers, technologists, and civil rights experts to establish transparent, worker-focused AI principles for employment decisions.

The need for such principles became evident during the author's participation in Seattle University School of Law's Summer Initiative for Technology, Innovation, and Ethics. Here, investigative journalist Hilke Schellmann, author of "The Algorithm: How AI Decides Who Gets Hired, Monitored, Promoted, and Fired and Why We Need to Fight Back," emphasized that many AI hiring and employment tools receive minimal testing before deployment.

Additionally, proprietary protections often obscure the development, data training, and bias evaluation processes, leaving employers unaware of the AI systems' inner workings. Yet, many assume that commercially available AI products have already undergone fairness and bias evaluations, a flawed assumption given the rapid pace of technological advancement.

As emphasized by Steve Tapia, a professor at Seattle University law school, "move fast and break things" mentality has fostered innovation but also produced unvetted technologies that frequently outpace regulators, courts, and societal understanding. With AI, the consequences of "breaking things" can be far more severe, as isolated errors may rapidly become systemic harms affecting thousands of workers.

Meaningful transparency requires appropriate testing, disclosure, and human oversight. Although proprietary protections may shield trade secrets, they should not prevent the public from obtaining information to evaluate the compliance of AI-assisted employment decisions with antidiscrimination and other employment laws. Transparency could include notifying employees when AI influences an employment decision, providing employees with avenues to appeal AI-assisted decisions, and mandating audits and reports on the impact of large-scale workforce reductions driven by AI, particularly if they disproportionately affect protected groups.

Some argue that AI regulation may hinder innovation. However, the author contends that having spent over a decade designing technology products, the potential harms of unregulated AI far outweigh the risks to innovation. By fostering transparency and accountability, Washington can help ensure that AI-driven employment decisions are fair, equitable, and compliant with existing laws.