AI bias isn’t just an error in the algorithm. It’s a chain of human decisions

In the United States, legal action has been taken against HR software producer Workday due to its AI-driven job screening tools allegedly discriminating against candidates based on age, disability, and race. Despite denying the accusations, the company faces criticism as one of many instances where AI systems have allegedly caused discriminatory harm.

These systems blur the line between technical errors and systemic injustice, turning bias into a digital issue. The blame often falls on the algorithms, yet they do not decide their output, safeguards, or how companies respond to discrimination reports. Therefore, technical adjustments alone are insufficient; a broader transformation of AI systems is necessary to ensure inclusivity.

AI systems disproportionately limit who is perceived as capable, employable, or suitable for leadership positions. A 2025 study examined how AI models, OpenAI’s now-retired GPT-4 and Microsoft Copilot, represented software engineers in a simulated recruitment scenario: 300 candidate profiles for four job roles, followed by recommendations and generated images.

Both models favored male candidates, especially for senior roles. Their images primarily depicted younger, slimmer, and lighter-skinned engineers, reproducing biases from language, imagery, employment records, and assumptions about who belongs in the profession. AI-generated recommendations are increasingly used in hiring, education, and public services, affecting those already underrepresented in AI development and leadership, as well as being disproportionately exposed to its harms.

The issue extends beyond gender and race; even when AI systems operate across languages and cultures, they often perpetuate predominantly western values, assumptions, and worldviews. Wealthy nations benefit more from AI, exacerbating global inequality. In another 2025 study, almost half of the reported AI incidents involved diversity or inclusion issues, with racial, gender, and age discrimination being most prevalent.

These harms stemmed from various stages of AI development lifecycle: non-diverse training data, and insufficient diversity and inclusion considerations during design, development, and deployment. Technical fixes, such as bias identification and data rebalancing, are necessary but insufficient, as much bias originates outside the AI system.

Decisions about data collection, labeling, and categorization, as well as whose experiences are deemed important, are shaped by historical, cultural, institutional, and power dynamics. Bias can also emerge through multiple intersecting identities, like gender, race, age, disability, and class. Even when a system appears fair when each identity is evaluated separately, it may disadvantage individuals at the intersection of several identities.

A more inclusive AI ecosystem requires interdisciplinary knowledge, such as educating AI engineers about social science theories to grasp the social origins of bias. It also demands genuine involvement from affected groups and ongoing attention to the power structures these systems operate within. AI development teams should evaluate not just the accuracy of models, but also their fairness in benefits, errors, and harms across different groups.

This approach would help tech companies better understand bias once it manifests through AI and develop methods to mitigate its impact. Organizations employing AI need stronger governance to monitor the technology's behavior, including accountability for reviewing risks and responding to incidents, as well as continuous monitoring post-deployment.

Algorithms do not determine which data matter or acceptable risk levels; humans make these choices. It's time for tech companies to recognize that the focus should be on examining the human decisions that allowed risks of harm to occur and identifying who was absent when these decisions were made.