Falcon-Emirati: When an LLM Learns the Dialect, the Culture, and the Nuance

Arabic, a language family known as one, comprises Modern Standard Arabic (MSA) and various dialects. MSA, prevalent in news and textbooks, is not the vernacular used in everyday conversations in the UAE. Here, Emirati Arabic, a Gulf dialect, serves as the medium for day-to-day interaction, humor, negotiation, and storytelling. Rich with its own vocabulary, rhythm, and cultural context, Emirati poetry, especially nabati poetry, carries nuances that escape literal translations.

Falcon-Emirati-7B aims to bridge this gap by specializing in understanding and generating Emirati Arabic. This model, built on Falcon-H1-Arabic, brings together State Space Models (Mamba) and Transformer attention, offering both linear-time efficiency and precision for long-range dependencies. Trained on diverse Arabic dialects and languages, Falcon-Emirati-7B specifically targets the Emirati dialect, enhancing vocabulary, grammar, and cultural understanding.

Despite the challenge, the 7B variant offered the optimal balance between quality and computational cost, pushing the model closer to the nuanced understanding a native speaker would possess.