In a major advancement for ai, Together ai has introduced an innovative Mix of Agents (MoA) approach, Together MoA. This new model leverages the collective strengths of multiple large language models (LLMs) to improve next-generation quality and performance, setting new benchmarks in ai.
MoA employs a layered architecture, with each layer comprising multiple LLM agents. These agents use results from the previous layer as auxiliary information to generate refined responses. This method allows MoA to integrate various capabilities and knowledge from various models, resulting in a more robust and versatile combined model. The implementation has proven to be successful, achieving a notable score of 65.1% on the AlpacaEval 2.0 benchmark, surpassing the previous leader, GPT-4o, which scored 57.5%.
A fundamental idea driving the development of MoA is the concept of “collaborativity” among LLMs. This phenomenon suggests that an LLM tends to generate better responses when presented with results from other models, even if those models are less capable. By leveraging this information, the MoA architecture classifies models into “proposers” and “aggregators.” Proposers generate initial baseline responses, offering diverse and nuanced perspectives, while aggregators synthesize these responses into high-quality results. This iterative process continues through several layers until a comprehensive and refined answer is achieved.
The Together MoA framework has been rigorously tested on multiple benchmarks including AlpacaEval 2.0, MT-Bench and FLASK. The results are impressive: Together MoA achieved top positions in the AlpacaEval 2.0 and MT-Bench leaderboards. In particular, in AlpacaEval 2.0, Together MoA achieved an absolute improvement margin of 7.6%, from 57.5% (GPT-4o) to 65.1% using only open source models. This demonstrates the superior performance of the model compared to closed source alternatives.
In addition to its technical success, Together MoA is designed with cost-effectiveness in mind. When analyzing the trade-offs between cost and performance, research indicates that the Together MoA configuration provides the best balance, delivering high-quality results at a reasonable cost. This is particularly evident in the Together MoA-Lite configuration, which, despite having fewer layers, matches the cost of the GPT-4o and achieves superior quality.
MoA's success is attributed to the collaborative efforts of several organizations in the open source ai community, including Meta ai, Mistral ai, Microsoft, Alibaba Cloud, and DataBricks. His contributions to the development of models such as Meta Llama 3, Mixtral, WizardLM, Qwen and DBRX have been fundamental to this achievement. Additionally, benchmarks such as AlpacaEval, MT-Bench and FLASK, developed by Tatsu Labs, LMSYS and KAIST ai, played a crucial role in evaluating MoA's performance.
Looking ahead, Together ai plans to further optimize the MoA architecture by exploring various model, indication and configuration options. A key area of focus will be reducing time-to-first-token latency, which is an exciting future direction for this research. It aims to enhance MoA's capabilities in reasoning-focused tasks, further solidifying its position as a leader in ai innovation.
In conclusion, Together MoA represents a significant advance in harnessing the collective intelligence of open source models. Its layered approach and collaborative spirit exemplify the potential to improve ai systems, making them more capable, robust, and aligned with human reasoning. The ai community eagerly anticipates the continued evolution and application of this innovative technology.
Review the Paper, GitHuband ai/blog/together-moa” target=”_blank” rel=”noreferrer noopener”>Blog. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter.
Join our Telegram channel and LinkedIn Grabove.
If you like our work, you will love our Newsletter..
Don't forget to join our 44k+ ML SubReddit
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of artificial intelligence for social good. His most recent endeavor is the launch of an ai media platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is technically sound and easily understandable to a wide audience. The platform has more than 2 million monthly visits, which illustrates its popularity among the public.
<script async src="//platform.twitter.com/widgets.js” charset=”utf-8″>
