Athene-Llama3-70B was launched: an open-weight LLM trained via RLHF based on Llama-3-70B-Instruct

Nexusflow has launched ai/blogs/athene” target=”_blank” rel=”noreferrer noopener”>Athena-Llama3-70B, Athene-70B is an open-source weighted chat model refined from Meta ai’s Llama-3-70B. Athene-70B has achieved an Arena-Hard-Auto score of 77.8%, rivaling proprietary models such as GPT-4o and Claude-3.5-Sonnet. This marks a significant improvement over its predecessor, Llama-3-70B-Instruct, which scored 46.6%. The improvement is due to Nexusflow’s targeted post-training sequence, designed to improve specific model behaviors. Athene-70B is currently in public testing on Chatbot Arena.

To maximize the potential of Llama-3-70B, Nexusflow developed internal benchmarks that assess LLM’s capabilities in following instructions, coding, creative writing, and multilingual tasks. Based on these assessments, high-quality preference data was selected for reinforcement learning from human feedback (RLHF). This process resulted in substantial performance improvements compared to Llama-3-70B-Instruct. The improvements span key aspects such as accurate following instructions, math and reasoning, comprehensive coding assistance, inspired creative writing, and multilingual proficiency.

Athene-70B demonstrates Nexusflow’s ability to customize models to specific business requirements through targeted post-training. Building on previous successes with Starling-7B and NexusRaven-V2, Nexusflow aims to enhance its models to meet enterprise-grade application standards. The company offers customized solutions to help businesses excel in GenAI’s agent and co-pilot technologies. Nexusflow invites organizations to explore how Athene-70B can enhance their ai initiatives by reaching out to them for more insights and collaboration opportunities.

Athene-Llama3-70B, an open-source weighted chat model developed by Nexusflow, demonstrates significant improvements over its predecessor. The model achieves competitive performance compared to proprietary models on the Arena-Hard-Auto benchmark. Nexusflow’s specific post-training process, using internal benchmarks and reinforcement learning from human feedback, has improved the model’s capabilities across multiple domains, including instruction following, mathematics and reasoning, coding, creative writing, and multilingual tasks. This advancement showcases Nexusflow’s ability to tailor models to business needs, building on its previous successes. The company positions itself as a provider of enterprise-grade custom ai solutions, inviting organizations to explore the potential of Athene-70B for their ai initiatives.

Review the Model card. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and join our Telegram Channel and LinkedIn GrAbove!. If you like our work, you will love our Newsletter..

Don't forget to join our Subreddit with over 46 billion users

Find upcoming ai webinars here

Asjad is a consultant intern at Marktechpost. He is pursuing Bachelors in Mechanical Engineering from Indian Institute of technology, Kharagpur. Asjad is a Machine Learning and Deep Learning enthusiast who is always researching the applications of Machine Learning in the healthcare domain.