What is the reverse diffusion process?

Introduction

Stable diffusion is a powerful tool (generative model) for creating high-quality images from noise. Stable diffusion consists of two steps: a forward diffusion process and a backward diffusion process. In the forward diffusion process, noise is progressively added to an image, effectively degrading its quality. This step is crucial for training the model as it helps it learn how images can transition from clarity to noise. We have covered the details of the forward diffusion process in our previous article.

In back scattering, noise is progressively removed to generate a high-quality image. This article will focus on this process and explore its mechanisms and mathematical foundations.

General description

1. Stable diffusion uses forward and inverse processes to generate high-quality images from noise.
2. The forward diffusion process progressively adds noise to a training image.
3. The back-diffusion process iteratively removes noise to reconstruct the original image.
4. This article explores the process of reverse diffusion and its mathematical foundations.
5. Training involves predicting noise at each step to improve image quality.
6. Neural network architecture and loss function are key to effective training.

What is the reverse diffusion process?

The backdiffusion process aims to convert pure noise into a clean image by iteratively removing noise. Training a diffusion model involves learning the backdiffusion process so that it can reconstruct an image from pure noise. If you are familiar with GANs, we are trying to train our generator network, but the only difference is that the diffusion network does an easier job because it does not have to do all the work in one step. Instead, it uses multiple steps to remove noise at once, which is more efficient and easier to train, as the authors of this paper discovered. paper.

Mathematical basis of reverse diffusion

What does a diffusion model do?

Many people think that a neural network (called a diffusion model to further confuse) either removes noise from an input image or predicts the noise that will be removed from an input. Both are incorrect. What the diffusion model does is predict all the noise that will be removed in a given time step. This means that if we have a time step t=600, then our diffusion model tries to predict all the noise whose removal we should get to at t=0, not at=599.

Reverse diffusion algorithm

• Initialization: The backscattering process starts with a noisy image, as you might have guessed. This image acts as a sample for the noise distribution.
• Iterative denoising: The model iteratively removes noise at each time step to recover the original data. This is done by following a sequence of denoising steps, where the model predicts the noise present in the current noisy image. Typically, the denoising steps are:
  • Estimate the noise in the current image (current time step to time step 0).
  • Subtract some of this estimated noise.
• Adding noise: A small amount of noise is introduced at each time step to prevent the process from becoming deterministic and to preserve generalization in the generated samples. This encourages exploration of the solution space and prevents the model from getting stuck in local minima. The added noise is typically reduced as the process progresses to ensure that the final image is less noisy and more in line with the desired result.
• Final score: The result after all the iterations is the generated image.

Mathematical formulation

This is the equation we took from the article. Probabilistic diffusion models for noise removal.

It basically says that 𝑝𝜃(𝑥0:𝑇) is a chain of Gaussian transitions starting at 𝑝(𝑥𝑇) and iterating T times using the equation for one step of the diffusion process 𝑝𝜃(𝑥𝑡−1∣𝑥𝑡).

Now it's time to explain how one-step works and how to get something implemented.

𝑁(𝑥𝑡−1,𝜇𝜃(𝑥𝑡,𝑡),∑𝜃(𝑥𝑡,𝑡)) has 2 parts:

• 𝜇𝜃(𝑥𝑡,𝑡) (mean)
• ∑𝜃(𝑥𝑡,𝑡) which is equal to 𝜎𝑡2𝐼 (variance)

To learn more about the mathematical foundations of the reverse diffusion process, see this article.

Model training using the reverse diffusion process

Image generation using the back-diffusion process relies heavily on the model's ability to predict the noise included in the forward diffusion process. This noise prediction ability is developed through a rigorous training process.

The main goal of model training using backdiffusion is to predict noise at each step of the diffusion process. By minimizing the error between predicted and actual noise, the model learns to effectively remove noise from the image.

Training data

The training data consists of noisy image pairs and the corresponding noise added at each step during the forward diffusion process. This data is generated by applying the forward diffusion process to a set of clean images, progressively adding noise over several steps.

Loss function

A key component of the training process is the loss function, which quantifies the difference between predicted and actual noise. A commonly used loss function is the mean square error (MSE). The model is trained to minimize this MSE loss, thereby improving its ability to accurately predict noise.

Neural network architecture

Convolutional neural networks (CNNs) are the most common type of neural network used in the back-diffusion process for noise prediction. CNNs can record spatial hierarchies in images, making them ideal for image processing applications. Multiple convolutional layers, pooling layers, and activation functions can be used in the architecture to extract and learn complex features from noisy images. There are two common backbone architecture choices for back-diffusion models: U-Net and Transformer.

Training procedure

• Initialization: Set random weights at the beginning of the neural network.
• Forward pass: To get the predicted noise, feed the noisy image through the neural network for each training sample.
• Loss calculation: Determine the loss by comparing the expected and actual noise using the selected loss function (e.g., MSE).
• Back pass: Perform backpropagation to calculate the gradients of the loss with respect to the network weights.
• Weight Update: To minimize the loss, update the network weights using an optimization technique such as Adam or Stochastic Gradient Descent (SGD).
• Iteration: Until the model converges to an ideal set of weights, repeat the forward pass, loss calculation, backward pass, and weight update for several epochs.

Assessment

Model performance is evaluated after training using a different validation dataset that was not used for training. On this validation set, the model's accuracy in predicting noise is an indication of its generalization ability. Metrics such as mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and R-squared (coefficient of determination) are often used.

Conclusion

Stable diffusion models are based on the processes of forward and reverse diffusion. These processes work together to gradually reduce noise in an image, ultimately producing high-quality results. This iterative refinement mechanism is based on solid mathematical foundations, making stable diffusion an effective tool in the field of generative models. As research in this area progresses, we can anticipate even more advanced applications and developments in this fascinating field.