Diffusion models have become a powerful tool for generating high-quality images from textual descriptions. Despite their successes, these models often exhibit limited diversity in the sampled images, particularly when sampling with a high guide weight without a classifier. To address this issue, we present Kaleido, a novel approach that improves sample diversity by incorporating autoregressive latent priors. Kaleido integrates an autoregressive language model that encodes the original title and generates latent variables, serving as abstract and intermediary representations to guide and facilitate the image generation process. In this article, we explore a variety of discrete latent representations, including textual descriptions, detection bounding boxes, object blobs, and visual tokens. These representations diversify and enrich the entry conditions to diffusion models, allowing for more diverse results. Our experimental results demonstrate that Kaleido effectively expands the diversity of image samples generated from a given textual description while maintaining high image quality. Furthermore, we show that Kaleido closely adheres to the guidance provided by the generated latent variables, demonstrating its ability to effectively control and direct the image generation process.
