Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Denoising diffusion models (DDMs) have attracted attention due to their exceptional sample quality and diversity. This success is largely attributed to the use of class- or text-conditional diffusion guidance methods. In this paper, we propose a more comprehensive approach that expands beyond traditional guidance methods. By adopting this generalized perspective, we introduce two novel condition-free strategies to enhance the quality of generated images: blur guidance and advanced Self-Attention Guidance (SAG). Employing benign properties of Gaussian blur, blur guidance enhances the suitability of intermediate samples for fine-scale information and generates higher quality samples with a moderate guidance scale. Improving upon this, SAG utilizes intermediate self-attention maps to enhance the stability and efficacy. Specifically, SAG leverages intermediate attention maps of diffusion models at each iteration to capture essential information for the generative process and guide it accordingly. Our experimental results demonstrate that our zero-shot method enhances the performance of various diffusion models, including ADM, IDDPM, and Stable Diffusion. Furthermore, combining SAG with conventional guidance methods, such as classifier-free guidance, results in further improvement.