Why AI systems need to be robust in safety-critical environments

UL's Prof Martin Hayes, a digital technologies professor at the University of Limerick (UL), discusses his research in systems theory for machine learning and AI, with a focus on robust performance in safety-critical environments. He explains that AI systems must be meticulously designed to ensure proper handoffs from the system to human decision-makers, as seen in health-related applications where any errors could have severe consequences.

Hayes emphasizes that simply achieving average accuracy is insufficient, and that building in basic levels of robustness is crucial for responsible deployment of AI in regulated domains like digital health. He stresses the importance of interdisciplinary collaboration and translating research findings into education to bridge skills gaps in safety-critical settings.

Hayes is UL’s principal investigator for the SUSA (Sustainable Healthcare with Digital Health Data Competence) project, a €12.4m European Commission-funded initiative aimed at closing the digital skills gap in European healthcare and supporting the EU’s Digital Decade and European Health Data Space ambitions. The project includes developing education modules, benchmarked against frameworks like the WHO Digital Health Competence Framework, to address the needs of students and prepare them for responsible AI deployment in healthcare.