NEWSLETTER
Residual transformations improve the depth of representation and expressive power of large language models (LLM). However, the application of static residual transformations in all tokens in the automatic generation of the generation leads to suboptimal compensation between the efficiency of inference and the fidelity of the generation. The existing methods, which include early exit, jump decoding and depth mixture, address this by modulating residual transformation based on complexity at the token level. However, these approaches predominantly consider the distance crossed by the tokens through the layers of the model, neglecting the underlying speed of residual evolution. We introduce to the multiple waste mixture (M2R2), a frame that dynamically modulates the residual speed to improve early alignment, improving inference efficiency. Evaluations on reasoning-oriented tasks such as Koala, Autoinstruct, Wizardlm and MT-Bench show that M2R2 exceeds the state-based strategies, balancing the quality and acceleration of the generation. In the Self -Copeculative Decoding Configuration, M2R2 reaches up to 2.8x accelerations in the MT Bank, which exceed methods such as speculative decoding of 2 models, Medusa, the decoding of lookohead and writing. In the architectures of the expert mixture (MOE), the integration of the early residual alignment with the load of experts in advance in advance in the memory of high bandwidth (HBM) accelerates the decoding, reduces the collars of the bottle of expert change and achieves an acceleration of 2.9x, which makes it highly effective in the resource environments.
