NEWSLETTER
Large Language Models (LLMs) have taken the world by storm with their human-like capabilities and characteristics. The latest addition to the long line of LLMs, the GPT-4 model, has exponentially increased the utility of ChatGPT due to its multimodal nature. This latest version takes input in the form of text and images and is already being used to build high-quality websites and chatbots. Recently, a new model was introduced to democratize ChatGPT, that is, to make it more accessible and available to a wider audience, regardless of language or geographic limitations.
This latest model, dubbed Phoenix, aims to achieve competitive performance not only in English and Chinese, but also in resource-constrained languages such as Latin and non-Latin languages. Phoenix, the multilingual LLM that achieves great performance between the English and Chinese open source models, was released to make ChatGPT available in places with restrictions imposed by OpenAI or local governments.
The author has described the meaning of Phoenix as follows:
- Phoenix has been presented as the first open source, multilingual and democratized ChatGPT model. This has been achieved through the use of rich multilingual data in the pre-training and instructional refinement stages.
- The team has carried out an adaptation of the multilingual instruction trace, focusing on non-Latin languages. Both the instruction and the conversational data have been used to train the model. This approach allows Phoenix to benefit from both, enabling it to generate contextually relevant and consistent responses across different language environments.
- Phoenix is a top-tier Chinese large language model that has achieved performance close to that of ChatGPT. Its Latin version Chimera is competitive in the English language.
- The authors have claimed that Phoenix is SOTA’s open source large language model for many languages beyond Chinese and English.
- Phoenix is among the first to systematically assess large LLMs, using automated and human assessments and evaluating multiple aspects of language generations.
Phoenix has shown superior performance compared to existing open source LLMs in Chinese, including models such as BELLE and Chinese-LLaMA-Alpaca. In other non-Latin languages like Arabic, Japanese, and Korean, Phoenix far outperforms existing models. Phoenix did not achieve SOTA results for Vicuna, which is an open source chatbot with 13B parameters trained by fine-tuning LLaMA on conversations shared by users.
This is because Phoenix had to pay a multilingual tax when it comes to languages other than Latin and Cyrillic. The ‘multilingual tax’ refers to the performance degradation that a multilingual model may experience when outputting text in languages other than its primary language. The democratization team has found the tax worth paying as a way to serve smaller language-speaking groups of relatively low resources. The team has proposed a Tax-free solution Phoenix: Chimera to mitigate the multilingual tax in Latin and Cyrillic languages. This involves replacing Phoenix’s spine with LLaMA. In English, Chimera impressed GPT-4 with a 96.6% ChatGPT quality.
Phoenix looks promising because of its multilingual potential and its ability to allow people of diverse language backgrounds to use the power of language models for their specific needs.
review the Paper and Github. Don’t forget to join our 19k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at asif@marktechpost.com
Check out 100 AI tools at AI Tools Club
Tanya Malhotra is a final year student at the University of Petroleum and Power Studies, Dehradun, studying BTech in Computer Engineering with a specialization in Artificial Intelligence and Machine Learning.
She is a data science enthusiast with good analytical and critical thinking, along with a keen interest in acquiring new skills, leading groups, and managing work in an organized manner.
