HUSKY: A unified, open source language agent for complex multi-step reasoning across domains

Recent advances in LLMs have paved the way for the development of linguistic agents capable of handling complex multi-step tasks using external tools for precise execution. While proprietary models or task-specific designs dominate existing language agents, these solutions often incur high costs and latency issues due to API dependency. Open source LLMs focus strictly on answering multi-hop questions or involve complex training and inference processes. Despite the computational and factual limitations of LLMs, linguistic agents offer a promising approach by methodically leveraging external tools to address complicated challenges.

Researchers from the University of Washington, Meta ai, and the Allen Institute for ai introduced HUSKY, a versatile, open-source linguistic agent designed to address diverse and complex tasks, including numerical, tabular, and knowledge-based reasoning. HUSKY operates through two key stages: generating the next action to take and executing it using expert models. The agent uses a unified action space and integrates tools such as code, mathematics, search, and common sense reasoning. Despite using smaller 7B models, extensive testing shows that HUSKY outperforms larger, cutting-edge models in several benchmarks. It demonstrates a robust and scalable approach to efficiently solving multi-step reasoning tasks.

Language agents have become crucial in solving complex tasks by leveraging language models to create high-level plans or assign tools to specific steps. They are usually based on closed source or open source models. Previous agents used proprietary models for planning and execution that, while effective, were costly and inefficient due to API dependency. Recent advances focus on open source models, drawn from larger teaching models, that offer more control and efficiency, but often specialize in narrow domains. Unlike these, HUSKY employs a broad, unified approach with a simple data curation process, using coding, mathematics, search, and common-sense reasoning tools to tackle various tasks efficiently.

HUSKY is a linguistic agent designed to solve complex multi-step reasoning tasks through a two-stage process: predicting and executing actions. It uses an action generator to determine the next step and the associated tool, followed by expert models to execute these actions. Expert models handle tasks such as generating code, performing mathematical reasoning, and constructing search queries. HUSKY repeats this process until reaching a final solution. Trained with synthetic data, HUSKY combines flexibility and efficiency in various domains. It is evaluated on data sets requiring various tools, including HUSKYQA, a new data set designed to test numerical reasoning and information retrieval capabilities.

HUSKY is assessed on a variety of tasks involving numerical, tabular, and knowledge-based reasoning, as well as mixed-tool tasks. Using datasets such as GSM-8K, MATH, and FinQA for training, HUSKY shows strong zero performance on unseen tasks, consistently outperforming other agents such as REACT, CHAMELEON, and proprietary models such as GPT-4. The model integrates tools and modules designed for specific reasoning tasks, leveraging fine-tuned models such as LLAMA and DeepSeekMath. This enables precise step-by-step problem solving across all domains, highlighting HUSKY's advanced capabilities in multi-tool usage and iterative task decomposition.

In conclusion, HUSKY is an open source linguistic agent designed to address complex multi-step reasoning tasks in various domains, including numerical, tabular, and knowledge-based reasoning. It uses a unified approach with an action generator that predicts steps and selects appropriate tools, fine-tuned from solid base models. Experiments show that HUSKY performs robustly on all tasks and benefits from domain-specific and cross-domain training. Variants with different models specialized for code and mathematical reasoning highlight the impact of model choice on performance. HUSKY's flexible and scalable architecture is poised to handle increasingly diverse reasoning challenges, providing a blueprint for developing advanced linguistic agents.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter. Join our Telegram channel, Discord channeland LinkedIn Grabove.

If you like our work, you will love our Newsletter..

Don't forget to join our 44k+ ML SubReddit