UAE's MBZUAI leads project to help AI understand Arab culture and regional dialects

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) is spearheading groundbreaking research to enhance artificial intelligence's comprehension of Arab culture and regional dialects. MBZUAI researchers have created the first benchmark to evaluate AI models' ability to understand and engage with Arab culture across 13 national dialects, exposing a significant disparity between the models' proficiency in understanding spoken Arabic and their capacity to converse like Arabs in everyday life.

Lead scientists Fajri Koto, an assistant professor in the Department of Natural Language Processing, and Muhammad Dehan, a researcher at the same department, informed state news agency Wam that closing this gap could set the stage for the next phase in the evolution of an Arabic-language AI. Dehan explained that while Arabic boasts over 400 million speakers globally, a major hurdle in AI development lies in models being predominantly trained on Modern Standard Arabic.

Consequently, these models may appear proficient in controlled trials but falter in authentic conversations across diverse dialects.

To address this issue, researchers developed ArabCulture-Dialogue, the inaugural benchmark designed to assess Arabic cultural reasoning in multi-turn conversations across Modern Standard Arabic and 13 national dialects from various Arab nations. The research team enlisted 26 native Arabic speakers from 13 countries to engage in discussions encompassing 12 topics, ranging from weddings and cuisine to parenting, agriculture, arts, and games.

Professor Koto explained that the models were tasked with selecting the culturally appropriate response from a selection of options, translating between Modern Standard Arabic and a specific dialect, and continuing a conversation in a designated dialect when prompted. The most adept models excelled at identifying culturally fitting replies, even when transitioning from Modern Standard Arabic to a dialect.

However, when asked to generate dialect, such as translating a line into Emirati or continuing a conversation in that dialect, the models' performance declined substantially.

The study revealed that models performed better when addressing customs shared across the Arab world, while country-specific customs posed greater challenges. Emirati and North African dialect discussions emerged as the most formidable. Dehan emphasized that the paradox lies in the fact that cultural knowledge resides within the models, but they occasionally necessitate minimal guidance.

Specifying the country and region associated with a conversation yielded improved accuracy, he added. The findings hold strategic significance for the UAE, which has designated AI as a national priority through the UAE National Strategy for Artificial Intelligence 2031 and the development of indigenous AI models, including Jais.