Attention-based arbitrary style transfer studies have shown promising performance in synthesizing vivid local style details. They typically use the all-to-all attention mechanism -- each position of content features is fully matched to all positions of style features. However, all-to-all attention tends to generate distorted style patterns and has quadratic complexity, limiting the effectiveness and efficiency of arbitrary style transfer. In this paper, we propose a novel all-to-key attention mechanism -- each position of content features is matched to stable key positions of style features -- that is more in line with the characteristics of style transfer. Specifically, it integrates two newly proposed attention forms: distributed and progressive attention. Distributed attention assigns attention to key style representations that depict the style distribution of local regions; Progressive attention pays attention from coarse-grained regions to fine-grained key positions. The resultant module, dubbed StyA2K, shows extraordinary performance in preserving the semantic structure and rendering consistent style patterns. Qualitative and quantitative comparisons with state-of-the-art methods demonstrate the superior performance of our approach.
翻译:注意力机制驱动的任意风格迁移研究已经展现出良好的性能,可以产生出栩栩如生的局部风格细节。它们通常采用全对全注意力机制,即内容特征的每个位置都与所有样式特征的位置完全匹配。然而,全对全注意力会产生扭曲的样式图案,且具有二次复杂度,这限制了任意风格迁移的有效性和效率。在本文中,我们提出了一种新颖的全键注意力机制,即内容特征的每个位置与稳定的样式特征关键位置匹配,更符合风格迁移的特点。具体来说,它集成了两种新提出的注意力形式:分布式注意力和渐进注意力。分布式注意力将注意力分配给描述本地区域风格分布的关键样式表示;渐进注意力从粗糙区域关注到细粒度关键位置。由此得到的模块称为StyA2K,在保留语义结构和呈现一致的样式图案方面表现出非凡的性能。与最先进的方法进行定量和定性比较,证明了我们方法的优越性。