NEWSLETTER
Achieve ~2x speed-up in LLM inference with Medusa-1 on Amazon SageMaker AI
This blog post is co-written with Moran Beladev, Manos Stergiadis, and Ilya Gusev from Booking.com. Large language models (LLMs) have ...
Tag: speedup
EAGLE-2: An efficient, lossless speculative sampling method that achieves speedup rates of 3.05x – 4.26x, which is 20% to 40% faster than EAGLE-1
Large language models (LLMs) have significantly advanced the field of natural language processing (NLP). These models, recognized for their ability ...
Recommended.
