{
  "id": 12687614,
  "title": "Why most health apps are biased even before their first lines of code are written",
  "url": "https://urgent.news/2026/10/07/why-most-health-apps-are-biased-even-before-their-first-lines-of-code",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-07T18:28:23.000Z",
  "source": {
    "name": "The Conversation AU",
    "slug": "the-conversation-au",
    "url": "https://theconversation.com/why-most-health-apps-are-biased-even-before-their-first-lines-of-code-are-written-286261"
  },
  "original_language": "en",
  "account": "In the realm of health technology, numerous applications have become an integral part of our daily lives, ranging from period trackers to AI-driven diagnostic tools. These digital resources, often powered by artificial intelligence, are fundamentally influenced by the data they are trained upon and the perspectives of their creators. However, the datasets fueling these technologies frequently represent a limited slice of the population, thereby potentially causing harm to underrepresented groups. For instance, algorithms intended to diagnose skin conditions often draw from datasets predominantly featuring lighter skin tones, resulting in misdiagnoses for individuals with darker skin. This issue isn't static, but rather self-perpetuating, leading to worsening health disparities over time. Our recent collaborative research, encompassing experts from ten countries, proposes a more inclusive and equitable approach to developing digital health tools. To truly benefit all, we must initiate crucial discussions even before the first line of code is written. What's the crux of the problem? These digital health tools predominantly embody Western, Eurocentric viewpoints, which may marginalize diverse perspectives on health and well-being. They are typically constructed based on implicit assumptions regarding what constitutes good health and wellbeing, the perception of \"normal\" bodies, and the significance of various forms of knowledge. The entities shaping these digital tools, such as technology companies, funders, and research institutions, wield considerable influence over which issues are prioritized and which knowledge is deemed credible. Frequently, these entities fail to account for the wider social and historical contexts that impact individuals' health and wellbeing, potentially leading to unintended consequences. For example, a US hospital algorithm designed to identify patients with complex medical needs for additional support only flagged under 18% of the assigned patients as Black, despite the algorithm basing risk scores on annual healthcare expenditure. This discrepancy stems from the fact that Black patients often incur less healthcare expenditure due to socioeconomic disparities and mistrust in the healthcare system, often stemming from systemic racism. We highlight that marginalized communities are rarely involved in the initial stages of developing new health technologies, or only participate after critical decisions have been made. This lack of early consultation, particularly concerning whether digital solutions are the most suitable approach, often prevents these communities from actively shaping technologies directly affecting their care. Assumptions about specific communities, such as \"they lack digital literacy\" or \"they won't use the technology,\" can also hinder their meaningful involvement. Additionally, the question of data ownership complicates matters. For Indigenous and numerous other marginalized communities worldwide, their data may carry cultural and economic significance tied to identity, sovereignty, and collective rights. What's the solution? Before any coding begins, we must critically examine the Western, Eurocentric assumptions underlying health and healthcare. This involves questioning what constitutes health, which forms of knowledge are valued, and which aspects of wellbeing are prioritized. Moreover, we must determine who has a say in shaping these technologies. Engaging marginalized communities in the early stages of development—where they can contribute their oral traditions and lived experiences—can complement dominant Western knowledge. For example, Indigenous health perspectives, which often encompass the interconnectedness of body, mind, spirit, community, and environment, can provide unique insights into risk factors or resilience that may go unnoticed by traditional Western models. Culturally appropriate digital health tools can transcend conventional notions of health. For instance, mental health programs tailored for young Indigenous individuals can bolster cultural identity and honor traditional knowledge, ultimately enhancing overall wellbeing and resilience. Stakeholders must also deeply consider the values of the communities targeted by these technologies and ensure alignment between the technology's values and those of the community. A case in point is the development of assistive technologies for individuals with mobility, communication, or cognitive challenges. Research conducted across Canada, Australia, and the US revealed that people were more inclined to adopt such technologies when they fostered family and community involvement in their care and strengthened connections with healthcare providers. The ideal solutions are those that promote interdependence rather than independence. Crucially, communities should wield control over their data's collection, analysis, linkage, interpretation, sharing, and management—a principle known as data sovereignty. Why now? Establishing digital health technologies differently necessitates a profound shift beyond merely ensuring accessibility or technical accuracy. It demands a reevaluation of our understanding of health, knowledge, power, and value. This transformation is paramount to prevent the perpetuation of health inequities.",
  "summary": "Apps and wearables are often built on invisible assumptions that might not be relevant to you. A new study shows how we can change that.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}