Understanding and modeling the dynamics of human gaze behavior in 360$^\circ$ environments is a key challenge in computer vision and virtual reality. Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images. Existing methods for scanpath generation, however, do not adequately predict realistic scanpaths for 360$^\circ$ images. We present ScanGAN360, a new generative adversarial approach to address this challenging problem. Our network generator is tailored to the specifics of 360$^\circ$ images representing immersive environments. Specifically, we accomplish this by leveraging the use of a spherical adaptation of dynamic-time warping as a loss function and proposing a novel parameterization of 360$^\circ$ scanpaths. The quality of our scanpaths outperforms competing approaches by a large margin and is almost on par with the human baseline. ScanGAN360 thus allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior and novel applications in virtual scene design.
翻译：在360 $ ⁇ circ$的环境中理解和模拟人类凝视行为的动态是计算机视觉和虚拟现实中的一项关键挑战。 生成对抗性方法可以通过生成大量可能的隐形图像的扫描路径来缓解这一挑战。 但是,现有的扫描虫生成方法不能充分预测360 $ ⁇ circ$图像的现实扫描路径。 我们展示了ScanGAN360, 这是一种解决这一具有挑战性问题的新型基因化对抗方法。 我们的网络生成器是针对代表隐形环境的360 $ ⁇ circ$ 图像的具体特性设计的。 具体来说,我们通过利用动态时间扭曲的球形适应功能来实现这一目标, 并提出了360 ⁇ circ$ 扫描路径的新参数。 我们的扫描路径质量超过了与人类基线相竞争的方法, 几乎接近于人类基线。 因此, ScanGAN360 能够快速模拟大量虚拟观察者, 其行为模拟真实用户的行为,从而能够更好地了解视觉设计中的视觉行为和新应用。
相关内容
微信扫码咨询专知VIP会员