NEWSLETTER
LLMs have demonstrated exceptional capabilities, but their substantial computational demands pose significant challenges for large -scale implementation. While above studies indicate that intermediate layers in deep neural networks can be reorder or eliminated without severely affecting performance, these ideas have not systematically used to reduce inference costs. Given the rapid expansion of the LLM, which often contain hundreds of billions of parameters, inference optimization is essential to improve efficiency, reduce latency and reduce operating expenses. High traffic applications depending on the cloud -based inference can incur monthly costs in millions, which makes efficiency solutions essential. In addition, the ability to implement these models on limited devices for resources requires strategies that maintain performance while minimizing computational overload. Despite the architectural similarities between modern transformers and deep residual networks, where the depth of the layer can sometimes be redundant, research has not yet explored these redundancies to completely optimize inference efficiency.
There are several approaches to improve the computer efficiency of the LLM, including pruning, quantization and parallel. Pruning eliminates redundant parameters to introduce scarcity, improving the use of memory and processing speed. On the other hand, quantization reduces accuracy by converting floating point calculations into whole formats of smaller bits such as INT8 or INT4, improving hardware efficiency and energy saving. In addition, parallelization techniques, such as the tensioner and parallelism of the pipeline, distribute workloads in multiple processing units to accelerate inference when addressing communication overload. Recent innovations have also explored architectural modifications at the layer level, including laying fusion and recurring dynamic execution, to expedite computational graphics. However, research has not yet focused on merging consecutive layers through tensioning parallelism, presenting an open path to further optimize inference.
Researchers at the University of Geneva, EPFL and Meta Fair propose a method to reduce the depth of the LLM prior to training while preserving performance. The modification of the computational graph allows the parallel execution of grouped cape pairs, improving the inference rate in approximately 1.20 × without requiring resentment. Its approach maintains an accuracy of 95% -99% between perplexity and the learning reference points in context (ICL). In addition, fine adjustment helps recover minor performance losses. This method significantly improves efficiency for large -scale LLM implementation, demonstrating that structural transformations, such as layer fusion and reorganization, can optimize the computational workload while maintaining the effectiveness of the model.
The study examines the effective depth of the LLM through the application of transformations such as decks, mergers and pruning pruning. The results indicate weak dependencies between the intermediate layers, which allows certain layers to be rearranged or parallel with a loss of minimal perplexity. Executing contiguous layers in parallel reduces depth while preserving performance, highlighting the independence of the layer. In addition, layer parallelism distributes calculations between GPUs, optimizing efficiency through tensioning parallelism. Modifications of attention and advance networks guarantee an effective parallel execution. Settings to layer standardization help maintain stability. These findings suggest that transformer models can take advantage of parallel to improve computational efficiency without requiring substantial architectural modifications.
The study evaluates the parallelism of the layer with respect to the speed of inference, the precision of the ICL and the fine adjustment for the recovery of performance. The experiments use Llama2 7b and call3.2 3b in Dual GPU A100. The parallelism of the layer applies to the fused layers, with tensioning parallelism in other places. The results show that beyond 14 layers for flame2 7b and 10 for flame3.2 3b, the precision of ICL decreases. The speed improves proportionally, reaching an impulse of 1.38x in aggressive parallelism. The parallel layers of fine adjustment in the REDPAJAMA data significantly restore the precision, improving the MMLU from 83.6% to 94.4% while maintaining the speed gains, which demonstrates the viability of the parallelism of the layer with specific adjustments.
In conclusion, the study introduces the parallelism of the layer (LP), which restructures the calculation of the transformer by executing pairs of parallel, improving the inference speed without training. Applied to flame2 7b and call3.2 3b, LP reduced the depth of the model by 21% and 18%, which produces acceleration of 1.29xy 1.22x, respectively. The fine adjustment recovered 10.8% of the lost precision, demonstrating its effectiveness. These findings challenge the notion that the transformer layers must process sequentially, suggesting that selective parallelization is viable. LP improves the efficiency of LLM in production, with a future work that explores the optimal coat group, interactions with quantization and deepest theoretical ideas about the independence of layers and computational efficiency.
Verify he Paper. All credit for this investigation goes to the researchers of this project. In addition, feel free to follow us <a target="_blank" href="https://x.com/intent/follow?screen_name=marktechpost” target=”_blank” rel=”noreferrer noopener”>twitter And don't forget to join our 75K+ ml of submen.
Recommended open source ai platform: 'Intellagent is a framework of multiple open source agents to evaluate the conversational the complex system' (Promoted)
Sana Hassan, a consulting intern in Marktechpost and double grade student in Iit Madras, passionate to apply technology and ai to address real world challenges. With great interest in solving practical problems, it provides a new perspective to the intersection of ai and real -life solutions.
