System helps humans predict when self-driving cars will make mistakes

Autonomous vehicles often rely on deep learning models that can make unexpected decisions, such as abruptly braking in front of an emergency vehicle. To help humans anticipate these mistakes, researchers from MIT and Motional created a method called Concept-Wrapper Network (CW-Net). This technique translates the complex reasoning process of the deep learning model into understandable concepts, like "approaching stopped vehicle" or "close to cyclist."

By providing these explanations, CW-Net helps drivers and passengers improve their situational awareness and prevents potential collisions.

In road tests on a private track, CW-Net explanations allowed safety drivers to more accurately predict vehicle behavior, and a larger simulation study with nonexpert users yielded similar results. This technique could enhance the safety and transparency of autonomous vehicles while building trust among drivers and passengers. As Julie Shah, an MIT professor, explains, "This work shows how explanations are supportive to the human's mental model and understanding of the behavior of a system, and how it could be used in engineering and development to improve the technology." The research is published in Nature.