S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention

This paper presents a new hierarchical vision Transformer for image style transfer, called Strips Window Attention Transformer (S2WAT), which serves as an encoder of encoder-transfer-decoder architecture. With hierarchical features, S2WAT can leverage proven techniques in other fields of computer vision, such as feature pyramid networks (FPN) or U-Net, to image style transfer in future works. However, the existing window-based Transformers will cause a problem that the stylized images will be grid-like when introduced into image style transfer directly. To solve this problem, we propose S2WAT whose representation is computed with Strips Window Attention (SpW Attention). The SpW Attention can integrate both local information and long-range dependencies in horizontal and vertical directions by a novel feature fusion scheme named Attn Merge. Qualitative and quantitative experiments demonstrate that S2WAT achieves comparable performance to state-of-the-art CNN-based, Flow-based, and Transformer-based approaches. The code and models are available at https://github.com/AlienZhang1996/S2WAT.