Gram-based and patch-based approaches are two important research lines of style transfer. Recent diversified Gram-based methods have been able to produce multiple and diverse stylized outputs for the same content and style images. However, as another widespread research interest, the diversity of patch-based methods remains challenging due to the stereotyped style swapping process based on nearest patch matching. To resolve this dilemma, in this paper, we dive into the crux of existing patch-based methods and propose a universal and efficient module, termed DivSwapper, for diversified patch-based arbitrary style transfer. The key insight is to use an essential intuition that neural patches with higher activation values could contribute more to diversity. Our DivSwapper is plug-and-play and can be easily integrated into existing patch-based and Gram-based methods to generate diverse results for arbitrary styles. We conduct theoretical analyses and extensive experiments to demonstrate the effectiveness of our method, and compared with state-of-the-art algorithms, it shows superiority in diversity, quality, and efficiency.
翻译:以格拉姆为基础的和以补丁为基础的方法是风格传输的两个重要研究线。最近,以格拉姆为基础的多种方法能够为相同的内容和风格图像产生多种和多样化的系统化产出。然而,作为另一个广泛的研究兴趣,由于基于近距离补丁的定型风格互换过程,基于补丁方法的多样性仍然具有挑战性。为了解决这一难题,我们在本文中将我们潜入现有的基于补丁的方法的柱子中,并提议一个通用和高效的模块,称为DivSwapper,用于多样化的基于补丁的任意风格的转让。关键洞察力是使用一种基本直觉,即具有更高激活值的神经质补丁能够对多样性做出更大的贡献。我们的DivSwapper是插接和功能,可以很容易地融入现有的基于格拉姆的、基于补丁方法,为任意风格产生不同的结果。我们进行理论分析和广泛的实验,以证明我们的方法的有效性,并与最先进的算法相比,它显示了多样性、质量和效率的优越性。