KIVI – A Plug-and-Play 2-bit KV Cache Quantization Algorithm without the need for any tuning
Large language models (LLMs) are incredibly useful for tasks like generating text or answering questions. However, they face a big ...
Tag: 2bit
Recommended.
