Deep learning has recently fueled tremendous progress in a wide range of applications, ranging from generation of realistic images and impressive recovery systems for language models that can carry on human-like conversations. While this progress is very exciting, the widespread use of deep neural network models calls for caution: As guided by Google’s AI BeginningWe seek to develop AI technologies responsibly by understanding and mitigating potential risks, such as the spread and amplification of unfair bias, and by protecting user privacy.
Completely clearing the influence of the data that is requested to be deleted is challenging because, in addition to simply removing it from the databases where it is stored, it also requires clearing the influence of that data on other artifacts, such as trained machine learning models. . Additionally, recent research [1, 2] has shown that, in some cases, it is possible to infer with great accuracy whether an example was used to train a machine learning model using membership inference attacks (MINE). This can raise privacy concerns, as it means that even if a person’s data is removed from a database, it is still possible to infer whether that person’s data was used to train a model.
Given the above, machine unlearning is an emerging subfield of machine learning that aims to remove the influence of a specific subset of training examples, the “forget set”, from a trained model. Also, an ideal unlearn algorithm would remove the influence of certain examples. while keeping other beneficial properties, such as accuracy across the rest of the train set and generalization to retained examples. A simple way to produce this unlearned model is to retrain the model on a fitted training set that excludes the samples from the forgotten set. However, this is not always a viable option, as retraining deep models can be computationally expensive. Instead, an ideal unlearn algorithm would use the already trained model as a starting point and efficiently make adjustments to remove the influence of the requested data.
Today we are pleased to announce that we have partnered with a broad group of academic and industry researchers to organize the first automatic unlearning challenge. The contest considers a realistic scenario in which after the training, a certain subset of the training images must be forgotten to protect the privacy or rights of the people involved. The competition will take place in Kaggle, and submissions will be automatically scored in terms of forgetfulness quality and model usefulness. We hope that this competition will help advance the state of the art in machine unlearning and encourage the development of efficient, effective and ethical unlearning algorithms.
Automatic unlearning applications
Automatic unlearning has applications beyond the protection of user privacy. For example, unlearning can be used to delete inaccurate or out-of-date information from trained models (for example, due to labeling errors or environmental changes) or to remove harmful, manipulated, or outlier data.
The field of machine unlearning is related to other areas of machine learning, such as differential privacy, lifelong learningand justice. Differential privacy is intended to ensure that no particular training example has too great an influence on the trained model; a stronger goal compared to unlearn, which only requires erasing the influence of the designated forget set. Research on lifelong learning aims to design models that can continuously learn while maintaining previously acquired skills. As unlearning work progresses, it may also open up new ways to drive fairness in models, by correcting for unfair biases or unequal treatment of members belonging to different groups (e.g. demographics, age groups, etc.). .).
 |
| Anatomy of unlearning. An unlearn algorithm takes as input a previously trained model and one or more samples from the training set to unlearn (the “forget set”). From the model forget set and retain set, the unlearn algorithm produces an updated model. An ideal unlearning algorithm produces a model that is indistinguishable from the model trained without the forgetting set. |
Challenges of automatic unlearning
The problem of unlearning is complex and multifaceted, as it involves several conflicting goals: forgetting requested data, maintaining model utility (eg, accuracy in retained and retained data), and efficiency. Because of this, existing unlearn algorithms make different compensations. For example, full retraining achieves successful forgetting without harming the utility of the model, but with little efficiency, while adding noise to pesos achieves oblivion at the expense of utility.
Furthermore, the evaluation of forgetting algorithms in the literature so far has been very inconsistent. while some plays report the accuracy of classification in samples to unlearn, others report the distance to the fully retrained model, and others use the error rate of membership inference attacks as a metric for forgetting quality [4, 5, 6].
We believe that the inconsistency of evaluation metrics and the lack of a standardized protocol is a serious impediment to progress in the field; we cannot make direct comparisons between different methods of unlearning in the literature. This leaves us with a myopic view of the relative advantages and disadvantages of different approaches, as well as open challenges and opportunities to develop improved algorithms. To address the problem of inconsistent evaluation and advance the state of the art in the field of machine unlearning, we have partnered with a broad group of academic and industry researchers to organize the first ever unlearning challenge.
Announcement of the first automatic unlearning challenge
We are pleased to announce the first automatic unlearning challengewhich will take place as part of the NeurIPS 2023 competition track. The objective of the contest is twofold. First, by unifying and standardizing evaluation metrics for unlearning, we hope to identify the strengths and weaknesses of different algorithms through apples-to-apples comparisons. Second, by opening this competition to everyone, we hope to encourage novel solutions and shed light on open challenges and opportunities.
The competition will take place in Kaggle and will run between mid-July 2023 and mid-September 2023. As part of the competition, today we are announcing the availability of the starter kit. This starter kit provides a foundation for participants to build and test their unlearning models on a toy data set.
The competition considers a realistic scenario in which an age predictor has been trained on images of faces, and after training, a certain subset of the training images must be forgotten to protect the privacy or rights of the people involved. To do this, we will make a synthetic face dataset (samples shown below) available as part of the starter kit, and we will also use several real face datasets to evaluate submissions. Participants are asked to submit code that takes as input the trained predictor, forget and retain sets, and generates the weights of a predictor that has unlearned the designated forget set. We will evaluate submissions based on the strength of the forgetting algorithm and the utility of the model. We will also apply hard cutoff that rejects unlearn algorithms that run slower than a fraction of the time it takes to retrain. A valuable outcome of this competition will be to characterize the advantages and disadvantages of different unlearning algorithms.
 |
| Extract of images of the facial synthetics data set along with age annotations. The competition considers the scenario where an age predictor has been trained on face images as above, and after training, a certain subset of the training images has to be forgotten. |
To assess forgetfulness we will use tools inspired by MIA, such as Lira. MIAs were first developed in the privacy and security literature and are intended to infer which examples were part of the training set. Intuitively, if the unlearn is successful, the unlearned model contains no traces of the forgotten instances, which causes MIAs to fail: the attacker would be unable infer that the forgotten set was, in fact, part of the original training set. In addition, we will also use statistical tests to quantify how different the distribution of unlearned models (produced by a particular submitted unlearn algorithm) is compared to the distribution of models retrained from scratch. For an ideal unlearn algorithm, these two will be indistinguishable.
Conclusion
Machine unlearning is a powerful tool that has the potential to address several open problems in machine learning. As research in this area continues, we expect to see new methods that are more efficient, effective, and responsible. We are delighted to have the opportunity through this contest to spark interest in this field, and we look forward to sharing our insights and findings with the community.
Thanks
The authors of this post are now part of Google DeepMind. We are writing this blog post on behalf of Eleni Triantafillou*, Fabian Pedregosa* (*equal contribution), Meghdad Kurmanji, Kairan Zhao, Guintare Carolina Dziugaite, Peter Triantafillou, Ioannis Mitliagkas, Vincent Dumoulin, Lisheng Sun Hosoya, Peter Kairouz, Julius CS James Junior, Jun Wan, Sergio Scalera and Isabelle Guyon.
