From lab to life: Why translating AI advances to the real world is a major challenge

Despite remarkable advancements in artificial intelligence research, translating these innovations into practical tools for real-world use remains a significant challenge. Vision-based systems, for example, show promise in improving safety, health monitoring, and rehabilitation. However, these systems often falter when deployed outside ideal lab environments. A key challenge is ensuring these systems perform reliably in messy, unpredictable, or low-visibility conditions.

Three main obstacles stand in the way of translating AI from the lab to the real world: generalisation, human behavior, and resources. AI vision systems are typically trained on well-lit, high-quality images, often curated from sources like motion capture studios or daytime recordings. However, when deployed in real-world settings, such as dimly lit homes, hospital rooms, or nighttime streets, these systems struggle.

This is not due to flawed technology, but rather because the training data cannot fully represent the innumerable variability of the real world.

For instance, in human pose estimation – determining the positions of a person's joints to understand their movements – modern AI models perform impressively in well-lit settings. However, in low or uneven lighting conditions, their performance drops sharply. This is because the models have learned from clean, consistent data and are unprepared to generalise to tougher conditions.

In theory, increasing nighttime training data could help, but it is difficult to collect. Low light can obscure visual information in ways that make labeling accurate challenging and expensive, as joints may be hard to see, shadows appear, and glare introduces noise.

Another major hurdle is the vast variety of human behaviors, particularly when interacting with objects – known as human-object interaction detection. Detecting actions like cutting tomatoes or passing basketballs requires understanding both object recognition and context. The challenge lies in scale; there are many objects and countless ways humans interact with them.

Teaching an AI system to recognise these actions is not feasible by collecting and labeling data for every possible combination. My research explores how AI can detect interactions it hasn't seen before during training, known as a generalisation-to-unseen-classes problem, a key challenge for building adaptable models.

While low-light pose estimation involves recognising familiar movements under unfamiliar conditions, human-object interaction detection presents a different challenge. Here, the action-and-object combination may be entirely new. Nonetheless, both are generalisation problems requiring different solutions. Finally, even when researchers know how to make models more robust, applying these improvements isn't always easy, especially for academic or smaller research settings.

Today's most powerful AI systems rely on massive computational resources – data centers filled with specialised computer chips. Such resources are often too expensive for university labs, limiting progress.