Microsoft Launches Orca 2: Pioneering Advanced Reasoning in Smaller Language Models with Customized Training Strategies

LLMs (Large Language Models) are trained with large volumes of textual data to understand and produce human-like language. GPT-3, GPT-4 and PaLM-2 are some examples. These models perform complex linguistic tasks, including text generation, conversational interaction, and question answering. They have been used in various domains, improving user experience in chatbots, coding, web search, customer service, and content production.

However, as the ai ​​community digs deeper into the vast landscape of smaller models, Microsoft has introduced the next version of Orca called Orca 2, designed to amplify the capabilities of compact ai models. Orca 1, through the integration of detailed explanations and traces, outperforms traditional instruction-tuned models in performance on challenging benchmarks such as BigBench Hard and AGIEval. Orca 2 delves into the potential of enhanced training signals to boost the reasoning capabilities of smaller language models

Imitation learning has been a predominant approach for refining small language models. These younger models often need to catch up on reasoning and comprehension skills, although they can produce content in a manner similar to that of their teachers. Although imitation learning has some benefits, it has drawbacks that can limit the ability of smaller models to reach their full potential and prevent them from using the best possible solutions given the particular problem and the model’s capabilities. They often need help matching the reasoning and understanding skills of their older counterparts, which hinders their full potential.

Instead of simply imitating, Orca instructs the model in various reasoning techniques. These include step-by-step processing, remember-then-generate, remember-reason-generate, and direct responses. The objective is to guide the model in acquiring the ability to discern the most effective solution strategy adapted to the nuances of each specific task.

Orca 2’s zero-shot reasoning capability highlights the possibility of improving smaller neural networks. Microsoft continues to believe that specialized training methods, such as that used for Orca 2, can reveal useful new applications. This method seeks to improve the effectiveness of these neural network deployments.

Most importantly, Orca 2 is protected from the initial cues that triggered particular behaviors during the training phase. Orca 2 transforms into Cautious Reasoner using the innovative Prompt Erasure technique. Unlike blind imitation, this method uses larger models as a source of behaviors from which the best ones for the given task are chosen.

Researchers tested Orca 2 on extensive benchmarks. They showed that it outperforms other equivalent models related to language comprehension, common sense reasoning, multi-step math problems, reading comprehension, summarization, and more. For example, on zero-shot reasoning tasks, the Orca 2-13B achieves 25% higher accuracy than comparable 13B models and is on par with a 70B model.

Orca 2 marks a significant step in the evolution of small language models. Its move away from conventional imitation learning, along with a focus on teaching various reasoning techniques, shows a new approach to unlocking the potential of compact ai models.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join. our 33k+ ML SubReddit, 41k+ Facebook community, Discord channel, and Electronic newsletterwhere we share the latest news on ai research, interesting ai projects and more.

If you like our work, you’ll love our newsletter.

<!– ai CONTENT END 2 –>