Diffusion models became a key part of the AI domain in 2022. We’ve seen photorealistic images generated by them, and they just got better and better. The success of diffusion models can be largely attributed to stable diffusion, which laid the foundation for later techniques. It wasn’t long before diffusion models became the gold standard for imaging.
Diffusion models, also known as denoising diffusion models, belong to a class of generative neural networks. They start by selecting noise from the training distribution and gradually refining it until the result is pleasing to the eye. This gradual denoising process allows them to be easier to scale and control. In addition, they tend to produce higher quality samples compared to earlier approaches such as Generative Adversarial Networks (GANs).
The imaging capability of diffusion models is believed to be unlike previous approaches. Unlike previous large-scale imaging models, which were susceptible to overfitting and could generate images that closely resembled the training samples, diffusion models are believed to produce images that differ significantly from those of the set of samples. training. This feature has made diffusion models a promising tool for privacy-conscious researchers who need to protect the identity of individuals or sensitive information in training images. By generating novel images that deviate from the original data set, diffusion models offer a way to preserve privacy without sacrificing the quality of the generated output.
But it is true? Do diffusion models really not memorize the training images? Isn’t it possible to use them to access samples in your training set? Can we really trust them to protect the privacy of training samples? The researchers asked these questions and put together a study to show us that diffusion models really do memorize their training data.
It is possible to regenerate samples on the training data of state-of-the-art diffusion models, although it is not easy. First, certain training samples are easier to extract, especially duplicate ones. The authors use this property to extract training samples from Stable Diffusion. They first identify near-duplicate images in the training data set. Of course, doing this manually is not feasible as there are about 160 million images in the Stable Diffusion training dataset. Instead, they embed images using CLIP and then compare images in this low-dimensional space. If the CLIP embeds have high cosine similarity, these subtitles are used as input cues for the pull attack.
Once they have potential text messages for the attack, the next step is to generate many samples, 500 in this case, using the same message to find if there is any memory. These 500 images are generated using the same indicator, but they all look different due to the random seed. Then, they connect each image to one another by measuring their similarity distance and constructing a graph using these connections. If you see an accumulation at a certain location on this graph, say more than 10 images connected to a single one, that central image is assumed to be a cache. When they applied this approach to stable diffusion, they were able to generate nearly identical samples to the training data set.
They have carried out experimental attacks on state-of-the-art diffusion models and found interesting observations. State-of-the-art diffusion models store more information than comparable GANs, and stronger diffusion models store more information than weaker diffusion models. This suggests that the vulnerability of generative image models may increase over time.
review the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 14k+ ML SubReddit, discord channel, and electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.
Ekrem Çetinkaya received his B.Sc. in 2018 and M.Sc. in 2019 from Ozyegin University, Istanbul, Türkiye. She wrote her M.Sc. thesis on denoising images using deep convolutional networks. She is currently pursuing a PhD. She graduated from the University of Klagenfurt, Austria, and working as a researcher in the ATHENA project. Her research interests include deep learning, computer vision, and multimedia networks.
