UT Austin and UC Berkeley researchers present Ambient Diffusion: an AI framework for training/tuning diffusion models given only corrupted data as input

For learning high-dimensional distributions and solving inverse problems, generative diffusion models are emerging as flexible and powerful frameworks. Conditional text based models such as Dalle-2, Latent Diffusion, and Image have achieved remarkable performance in generic image domains due to several recent advances. Diffusion models have recently demonstrated their ability to memorize samples from their training set. Additionally, an adversary with simple query access to the model can sample data sets, raising privacy, security, and copyright concerns.

The researchers present the first diffusion-based framework that can learn an unknown distribution from heavily contaminated samples. This problem arises in scientific contexts where obtaining clean samples is difficult or expensive. Because generative models are never exposed to clean training data, they are less likely to memorize particular training samples. The core concept is to further corrupt the original distorted image during diffusion by introducing additional measurement distortion and then challenging the model to predict the original corrupted image from the other corrupted image. Scientific research verifies that the approach generates models capable of acquiring the conditional expectation of the full picture without corrupting in light of this additional measurement corruption. Repaint and feel compressed are two methods of corruption that fall under this generalization. By training them on industry-standard benchmarks, the scientists show that their models can learn the distribution even when all training samples are missing 90% of their pixels. They also demonstrate that basic models can be fit on small corrupted data sets, and the clean distribution can be learned without memorizing the training set.

notable features

• The core concept of this research is to further distort the image and force the model to predict the distorted image from the image.
• Their approach trains diffusion models using corrupted training data on popular benchmarks (CelebA, CIFAR-10, and AFHQ).
• Researchers give a rough sample of the desired distribution p0(x0) based on learned conditional expectations.
• As research shows, quite a bit can be learned about the distribution of original photos, even if up to 90% of the pixels are missing. They have better results than the previous best AmbientGAN and natural baselines.
• Not seeing a clean picture during training shows that the models perform similar to or better than state-of-the-art diffusion models in handling certain inverse problems. While baselines require many diffusion steps, models only need a single prediction step to do their job.
• The approach is used to further refine standard pretrained diffusion models in the research community. It is possible to learn distributions from a small number of contaminated samples, and the fine-tuning process only takes a few hours on a single GPU.
• Some damaged samples in a different domain can also be used to fit basic models like the Deepfloyd IF.
• To quantify the learning effect, researchers compare models trained with and without corruption by displaying the distribution of top-1 similarities to training samples.
• Models trained on sufficiently distorted data are shown to retain no knowledge of the original training data. They assess the tradeoff between corruption (which determines the level of memorization), the training data, and the quality of the learned generator.

limitations

• The level of corruption is inversely proportional to the quality of the generator. The generator is less likely to learn by heart when the level of corruption is increased but at the expense of quality. The precise definition of this commitment remains an unresolved research topic. And to estimate E[x0|xt] With the trained models, the researchers tested basic approximation algorithms in this work.
• In addition, it is necessary to make assumptions about data distribution to ensure strict privacy with respect to the protection of any training sample. Supplementary material shows that the restoration oracle can restore E precisely [x0|xt]although the researchers do not provide a technique.
• This method will not work if the measurements also contain noise. The use of SURE regularization can help future research to get around this restriction.

review the Paper and GitHub link. Don’t forget to join our 22k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

🚀 Check out 100 AI tools at AI Tools Club