A new paradigm for editing machine learning models based on arithmetic operations on task vectors

It is increasingly common to use large-scale pretraining to develop models used as the basis for more specialized machine learning systems. From a practical point of view, it is often necessary to change and update such models after having previously trained them. The targets for post-processing are numerous. For example, it is critical to improve the performance of the pretrained model on specific tasks, address biases or unwanted behavior, align the model with human preferences, or incorporate new information.

The latest work by a team of researchers from the University of Washington, Microsoft Research, and the Allen Institute for AI develops a clever method to alter the behavior of pretrained models based on task vectors, which are obtained by subtracting pretrained weights. of a fitted model on a task. More precisely, the task vectors are defined as the element difference between the weights of the pre-trained and fitted models. To this end, the task vectors can be applied to any parameter in the model using the sum of elements and an optional scaling term. In the paper, the scaling terms are determined using retained validation sets.

The authors demonstrate that users can perform simple arithmetic operations on these task vectors to change models, such as negating the vector to eliminate undesirable behaviors or unlearning tasks, or adding task vectors to improve multitasking models or single-task performance. They also show that when tasks form an analogy relationship, task vectors can be combined to improve performance on tasks where data is sparse.

The authors show that the conceived approach is reliable in forgetting unwanted behavior in both the vision and text domains. They experiment with original and fitted CLIP models for the vision domain on various data sets (eg, Cars, EuroSAT, MNIST, etc.). As seen in Table 1 of the article, negation of task vectors is a reliable method of lowering performance on the target task (up to 45.8 percentage points for ViT-L) and leaving near original precision for the control task. . For the language domain (Table 2), they show that negative task vectors reduce by six times the number of toxic generations of a GPT-2 Large model while resulting in a model with similar perplexity on a control task (WikiText -103).

Adding task vectors can also improve pretrained models. In the case of image classification, adding task vectors from two tasks improves the accuracy of both, resulting in a single model that is competitive with the use of two specialized fitted models (Figure 2). In the language domain (GLUE benchmark), the authors show that adding task vectors to pretrained T5-based models is better than fitting, even if the improvements are more modest in this case.

Finally, performing task analogies with task vectors allows improving performance in domain and subpopulation generalization tasks with few data. For example, to get better performance on specific rare images (eg, lions indoors), a task vector can be constructed by adding to the lion-outdoor task vector the difference between the dog-indoor and dog-indoor task vectors. exteriors. As seen in Figure 4, such modeling allows clear improvements for domains where few images are available.

In summary, this work introduced a new approach to model editing by performing arithmetic operations on task vectors. The method is efficient and users can easily experiment with various editions of models by recycling and transferring knowledge from extensive collections of publicly available tuned models.

review the Paper Y Github. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 13k+ ML SubReddit, discord channel, Y electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.