{
  "id": 2671206,
  "title": "Why Your ML Model Should Sometimes Say 'I Don't Know': Building Uncertainty-Aware Clinical AI",
  "url": "https://urgent.news/2026/08/22/why-your-ml-model-should-sometimes-say-i-dont-know-building",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-22T22:29:30.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/hossein_ahmadi/why-your-ml-model-should-sometimes-say-i-dont-know-building-uncertainty-aware-clinical-ai-36g8"
  },
  "original_language": "en",
  "account": "In healthcare, predictive models tell us what will happen, while causal models explain what happens if we intervene. However, neither addresses what actions we should take. This gap became evident when I developed CausalSepsis-DI v2, an end-to-end Healthcare Decision Intelligence system that connects the dots between causal analysis and safe clinical decisions. The question was whether to administer early antibiotics to an ICU patient to shorten their stay. Traditional machine learning provided a predicted ICU length of stay, but that information was insufficient for clinicians.\n\nThe breakthrough came with a three-class decision policy. Instead of imposing binary treat or don’t treat choices (which could be risky when uncertainty loomed), the system generated three options: treat, standard care, or defer to the clinician. This wasn’t a limitation but a feature— a trustworthy clinical AI should acknowledge when it lacks certainty.\n\nThe technical foundation included causal inference using LinearDML with cross-fitting for valid CATEs and confidence intervals, along with propensity diagnostics for initial checks. The decision policy was cost-sensitive, utility-based, and uncertainty-aware. Evaluation relied on Inverse Propensity Weighting with bootstrap confidence intervals for counterfactual estimation.\n\nTo address the lack of sufficient real-world data, I created a synthetic cohort of 1500 patients, complete with verified causal effects, varied treatment responses, and realistic propensity distributions. The entire pipeline is modular and can be seamlessly integrated with real Electronic Health Record data sources such as MIMIC-IV full, eICU, and hospital systems.\n\nThe results were impressive: 62.1% of patients were recommended for early antibiotics, 33.8% for standard care, and only 4.1% were directed to defer to a clinician (the uncertainty zone). Policy evaluation showed a 3.70-day reduction in mean ICU length of stay compared to current practices. The pipeline is fully adaptable and can be directly applied to real EHR data. For those interested in the full interactive notebook and architecture, it’s available at https://www.kaggle.com/code/ahmadihossein/sepsisdecision-di.\n\nI invite feedback from clinicians who work with decision support systems, causal inference experts, and MLOps engineers crafting production ML models. The biggest challenge in deploying clinical ML, as I see it, is ensuring trustworthy, uncertainty-aware systems that don’t force unsafe binary decisions when the evidence is unclear.",
  "summary": "🚨 Predictive models tell us what will happen. 🧠 Causal models tell us what happens IF we intervene . ❓ But neither tells us what we should DO . In healthcare, this gap is dangerous. I just finished building CausalSepsis-DI v2 , an end-to-end Healthcare Decision Intelligence system that bridges the gap between causal estimation and safe clinical action. 🏥 The Clinical Question: Should we…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}