Athene-Llama3-70B Released: An Open-Weight LLM Trained through RLHF based on Llama-3-70B-Instruct

Nexusflow has released Athene-Llama3-70B, an open-weight chat model fine-tuned from Meta AI’s Llama-3-70B. Athene-70B has achieved an Arena-Hard-Auto score of 77.8%, rivaling proprietary models like GPT-4o and Claude-3.5-Sonnet. This marks a significant improvement from its predecessor, Llama-3-70B-Instruct, which scored 46.6%. The enhancement stems from Nexusflow’s targeted post-training pipeline, designed to improve specific model behaviors. Athene-70B is currently undergoing public testing on Chatbot Arena.

To maximize Llama-3-70B’s potential, Nexusflow developed internal benchmarks evaluating LLM capabilities in instruction following, coding, creative writing, and multilingual tasks. Based on these evaluations, high-quality preference data was curated for targeted Reinforcement Learning from Human Feedback (RLHF). This pipeline resulted in substantial performance improvements compared to Llama-3-70B-Instruct. The enhancements span key aspects such as precise instruction following, math and reasoning, comprehensive coding assistance, inspired creative writing, and multilingual mastery.

Athene-70B demonstrates Nexusflow’s capability to customize models for specific enterprise requirements through targeted post-training. Building on previous successes with Starling-7B and NexusRaven-V2, Nexusflow aims to advance its models to meet enterprise-grade application standards. The company offers tailored solutions to help businesses excel in GenAI copilot and agent technologies. Nexusflow invites organizations to explore how Athene-70B can enhance their AI initiatives by contacting them for further information and collaboration opportunities.

Athene-Llama3-70B, an open-weights chat model developed by Nexusflow, demonstrates significant improvements over its predecessor. The model achieves competitive performance compared to proprietary models in the Arena-Hard-Auto benchmark. Nexusflow’s targeted post-training pipeline, utilizing internal benchmarks and Reinforcement Learning from Human Feedback, has enhanced the model’s capabilities across various domains, including instruction following, math and reasoning, coding, creative writing, and multilingual tasks. This advancement showcases Nexusflow’s ability to tailor models for enterprise needs, building on their previous successes. The company positions itself as a provider of customized enterprise-grade AI solutions, inviting organizations to explore the potential of Athene-70B for their AI initiatives.

Check out the Model Card. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 46k+ ML SubReddit

Find Upcoming AI Webinars here

Asjad is an intern consultant at Marktechpost. He is persuing B.Tech in mechanical engineering at the Indian Institute of Technology, Kharagpur. Asjad is a Machine learning and deep learning enthusiast who is always researching the applications of machine learning in healthcare.