MBZUAI researchers launch Atlas-Chat (2B, 9B and 27B): a family of open models adapted to instructions for Darija (Moroccan Arabic)

Natural language processing (NLP) has made incredible strides in recent years, particularly through the use of large language models (LLM). However, one of the main problems with these LLMs is that they have largely focused on data-rich languages ​​such as English, leaving behind many underrepresented languages ​​and dialects. Moroccan Arabic, also known as Darija, is one of those dialects that has received very little attention despite being the main form of daily communication for more than 40 million people. Due to the lack of extensive data sets, adequate grammatical standards, and adequate benchmarks, Darija has been classified as a low-resource language. As a result, it has often been neglected by developers of large language models. The challenge of incorporating Darija into LLMs is further compounded by its unique combination of Modern Standard Arabic (MSA), Amazigh, French, and Spanish, along with its emerging written form that still lacks standardization. This has led to an asymmetry where dialectal Arabic such as Darija is marginalized, despite its widespread use, which has affected the ability of artificial intelligence models to effectively address the needs of these speakers.

Get to know Atlas-Chat!!

MBZUAI (Mohamed bin Zayed University of artificial intelligence) has launched Atlas-Chat, a family of open, instructional-tuned models designed specifically for Darija, the colloquial Arabic of Morocco. The introduction of Atlas-Chat marks a significant step in addressing the challenges posed by low-resource languages. Atlas-Chat consists of three models with different parameter sizes (2 billion, 9 billion, and 27 billion) that offer a variety of capabilities to users depending on their needs. The models have been fine-tuned based on the instructions, allowing them to perform effectively on different tasks, such as conversational interaction, translation, summarizing, and content creation in Darija. Furthermore, its goal is to promote cultural research through a better understanding of Morocco's linguistic heritage. This initiative is particularly noteworthy because it aligns with the mission of making advanced ai accessible to communities that have been underrepresented in the ai ​​landscape, thereby helping to close the gap between resource-rich and resource-poor languages.

Technical details and benefits of Atlas-Chat

Atlas-Chat models are developed by consolidating existing Darija language resources and creating new data sets through both manual and synthetic means. In particular, the Darija-SFT-Mixture dataset consists of 458,000 instruction samples, which were collected from existing resources and through synthetic generation from platforms such as Wikipedia and YouTube. Additionally, high-quality English instructional data sets were translated into Darija with rigorous quality control. The models have been refined on this data set using different base model options, such as the Gemma 2 models. This careful construction has led Atlas-Chat to outperform other Arabic-specialized LLMs, such as Jais and AceGPT, by significant margins. For example, on the recently introduced DarijaMMLU benchmark (a comprehensive evaluation suite for Darija covering discriminative and generative tasks), Atlas-Chat achieved a 13% performance increase over a larger 13 billion model. parameters. This demonstrates their superior ability to follow instructions, generate culturally relevant responses, and perform standard NLP tasks in Darija.

Why Atlas-Chat is important

The introduction of Atlas-Chat is crucial for multiple reasons. First, it addresses a long-standing gap in ai development by focusing on an underrepresented language. Moroccan Arabic, which has a complex cultural and linguistic composition, is often neglected in favor of MSA or other dialects that are more data-rich. With Atlas-Chat, MBZUAI has provided a powerful tool to improve communication and content creation in Darija, supporting applications such as conversational agents, automated summaries, and more nuanced cultural investigations. Second, by providing models with different parameter sizes, Atlas-Chat ensures flexibility and accessibility, addressing a wide range of user needs, from lightweight applications that require fewer computational resources to more sophisticated tasks. Atlas-Chat's evaluation results highlight its effectiveness; for example, Atlas-Chat-9B scored 58.23% on the DarijaMMLU benchmark, significantly outperforming state-of-the-art models such as AceGPT-13B. These advances indicate the potential of Atlas-Chat to deliver high-quality linguistic understanding to Moroccan Arabic speakers.

Conclusion

Atlas-Chat represents a transformative advance for Moroccan Arabic and other low-income dialects. By creating a robust, open-source solution for Darija, MBZUAI is taking an important step in making advanced ai accessible to a broader audience, allowing users to interact with the technology in their own language and cultural context. This work not only addresses the asymmetries observed in ai support for low-resource languages, but also sets a precedent for future development in underrepresented linguistic domains. As ai continues to evolve, initiatives like Atlas-Chat are crucial to ensuring that the benefits of the technology are available to everyone, regardless of the language they speak. With further improvements and refinements, Atlas-Chat is poised to close the communication gap and improve the digital experience of millions of Darija speakers.

look at the Paper and Models hugging faces. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. If you like our work, you will love our information sheet.. Don't forget to join our SubReddit over 55,000ml.

(Sponsorship opportunity with us) Promote your research/product/webinar to over 1 million monthly readers and over 500,000 community members

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of artificial intelligence for social good. Their most recent endeavor is the launch of an ai media platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is technically sound and easily understandable to a wide audience. The platform has more than 2 million monthly visits, which illustrates its popularity among the public.