Meet QLORA: An efficient tuning approach that reduces memory usage enough to fine tune a 65B parameter model on a single 48GB GPU while retaining full 16-bit tuning task performance

Extended Language Models (LLMs) can be improved by fine tuning, which also allows you to add or remove desired behaviors. However, fitting large models is cost prohibitive; For example, a parameter model LLaMA 65B consumes more than 780 GB of GPU RAM when set to 16-bit standard mode. Although most current quantization approaches can reduce the memory footprint of LLMs, these methods only work for inference and fail during training. Researchers at the University of Washington developed QLORA, which quantizes a pretrained model using a state-of-the-art high-precision algorithm at 4-bit resolution before adding a sparse set of learnable low-range adapter weights modified by gradients of backpropagation through quantified consequences. . They show for the first time that a 4-bit quantized model can be tuned without affecting performance.

Compared to a fully tuned 16-bit baseline, QLORA reduces average memory needs to tune a 65B parameter model from >780 GB GPU RAM to 48 GB without sacrificing runtime or predictive performance. The largest publicly accessible models to date can now be tuned on a single GPU, which is a big change in the accessibility of LLM fine tuning. They train the Guanaco family of models using QLORA, and their largest model achieves 99.3% using a single professional GPU over 24 hours, effectively closing the gap with ChatGPT in the Vicuna benchmark. The second best model achieves 97.8% of the performance level of ChatGPT in the Vicuna benchmark and can be trained in less than 12 hours on a single consumer GPU.

The following QLORA technologies are intended to reduce memory usage without compromising performance: (1) 4-bit NormalFloat, a quantization data type for normally distributed data that is information theoretically optimal and produces empirical results superior to integers 4-bit and 4-bit. little floats. (2) Double quantization, which saves, on average, 0.37 bits per parameter (or about 3 GB for a 65B model), quantizes the quantization constants. (3) Paged optimizers use NVIDIA Unified Memory to avoid memory spikes caused by gradient checkpoints when processing a mini-batch with a long stream. When in use, its smallest Guanaco model (parameters 7B) uses less than 5GB of memory and outperforms a 26GB Alpaca model in the Vicuna test by more than 20 percentage points.

They incorporate these contributions into a more refined LoRA strategy that includes adapters at each network level and thus nearly eliminates the accuracy tradeoffs identified in previous work. Due to the efficiency of QLORA, we can analyze instruction fine tuning and chatbot performance on model sizes in greater detail than we could have done with conventional fine tuning due to memory cost. As a result, they train over a thousand models using a variety of instruction fit data sets, model topologies, and parameter values ​​ranging from 80M to 65B. They demonstrate that QLORA restores 16-bit performance, trains Guanaco, an advanced chatbot, and examines patterns in learned models.

First, while both are intended to provide instruction after generalization, they find that data quality is considerably more essential than data set size, with a 9k sample data set (OASST1) outperforming a 450k sample dataset (FLAN v2, subsampled) on chatbot performance. Second, they demonstrate that good Massively Multitasking Language Understanding (MMLU) benchmark performance only sometimes translates into great Vicuna chatbot benchmark performance, and vice versa. In other words, the suitability of the data set is more important than the scale for a given task. They also offer a comprehensive evaluation of the chatbot’s performance using human evaluators and GPT-4.

The models compete against each other in matches using tournament-style benchmarking to determine the best response to a given stimulus. GPT-4 or human scorers decide which player wins a game. Elo scores, which are created by combining tournament results, are used to rank the chatbot’s performance. Looking at the range of performance of the models in tournaments, they find that GPT-4 and human judgments mostly agree, but there are also some areas of marked divergence. As a result, they draw attention to the fact that model-based evaluation has uncertainties and is a less expensive option than human annotation.

They add a qualitative analysis of Guanaco models to their chatbot benchmark findings. Their study identifies instances of success and failure that the quantitative standards did not take into account. They publish all model generations with GPT-4 and human comments to aid future research. They incorporate their techniques into the Hugging Face transformer stack, open up their software and CUDA cores, and make them widely available. For 32 different open source enhanced models, they provide a collection of adapters for models of sizes 7/13/33/65B trained on 8 different instruction tracking data sets. The code repository is made public, along with a demo that can be hosted on Colab.

review the Paper, Code, and Alabama. Don’t forget to join our 22k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

🚀 Check out 100 AI tools at AI Tools Club