The Allen Institute for AI (AI2) has announced the development of an innovative open language model called AI2 OLMo (Open Language Model). OLMo will be a next-generation generative language model with a scale of 70 billion parameters, comparable to other large language models. The Project is expected to end in 2024. Its goal is to provide the research community with access to all aspects of modeling, fostering collaboration, and advancing the science of language models.
AI2 is partnering with leading technology companies, including AMD and CSC, to develop OLMo. The collaboration involves utilizing the GPU capabilities of the LUMI pre-exascale supercomputer powered by AMD, known for its power efficiency. By harnessing the power of this green supercomputer, AI2 aims to create a single, open language model that will allow researchers to work directly on language models for the first time.
A key aspect of OLMo is its openness and accessibility to the research community. AI2 plans to make all elements of the Project openly available, including data, code, training curves, evaluation benchmarks, and ethical considerations surrounding model development. By providing full transparency, AI2 intends to empower researchers to leverage and improve OLMo, enabling faster and safer progress in the field. The objective is to develop the best open language model worldwide in a collaborative way.
The AI2 team ensures that OLMo becomes a genuinely open model that provides unique value to the AI research community. All components built for OLMo, including training data, code, model weights, intermediate checkpoints, and ablations, will be openly available, well-documented, and reproducible, with few exceptions and proper licenses. The release strategy for the model and its artifacts is currently being developed. In addition, AI2 plans to create a demo and publish interaction data from consenting users.
Parallel to model development, AI2 will make decisions to maximize model usability and efficiency without compromising performance. The goal is to make OLMo accessible to a wide range of AI researchers, fostering diversity of perspectives and accelerating improvements in language model development. AI2 also intends to create and release a thoroughly researched and documented model training dataset, encompassing pre-training data, instruction data, and human interaction data.
Recognizing the importance of ethical considerations, AI2 takes a pragmatic approach to ethics and openness throughout the OLMo project. The team will document decisions, concerns, and trade-offs regarding the ethical and social impacts of creating and launching the OLMo model. AI2 promotes knowledge and understanding of AI by sharing progress, challenges and discoveries. Legal experts, both internal and external, are actively involved in the modeling process to assess privacy and intellectual property rights issues at multiple checkpoints.
AI2 has partnered with organizations like Surge AI and MosaicML to collaborate on data and training code for OLMo. An ethics review committee made up of internal and external assessors has been established to provide feedback during the Project. The OLMo model and API will serve as valuable resources for the community at large, enabling better understanding and participation in the generative AI revolution. AI2 welcomes support and partnerships from organizations aligned with its AI values for standard, reasonable, responsible, and beneficial AI technologies.
review the Reference article. Don’t forget to join our 21k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]
🚀 Check out 100 AI tools at AI Tools Club
Niharika is a technical consulting intern at Marktechpost. She is a third year student, currently pursuing her B.Tech from the Indian Institute of Technology (IIT), Kharagpur. She is a very enthusiastic individual with a strong interest in machine learning, data science, and artificial intelligence and an avid reader of the latest developments in these fields.
