Image based Deep Feature Quality Metrics (DFQMs) have been shown to better correlate with subjective perceptual scores over traditional metrics. The fundamental focus of these DFQMs is to exploit internal representations from a large scale classification network as the metric feature space. Previously, no attention has been given to the problem of identifying which layers are most perceptually relevant. In this paper we present a new method for selecting perceptually relevant layers from such a network, based on a neuroscience interpretation of layer behaviour. The selected layers are treated as a hyperparameter to the critic network in a W-GAN. The critic uses the output from these layers in the preliminary stages to extract perceptual information. A video enhancement network is trained adversarially with this critic. Our results show that the introduction of these selected features into the critic yields up to 10% (FID) and 15% (KID) performance increase against other critic networks that do not exploit the idea of optimised feature selection.
翻译:基于图像的深特质质量计量(DFQM) 已证明与传统度量的主观感知分数有更好的关联。 DFQM 的基本重点是利用大型分类网络的内部代表作为计量特征空间。 以前,没有注意确定哪些层次在概念上最为相关。 在本文中,我们根据对层行为的神经科学解释,提出了从这种网络中选择感知相关层的新方法。 选定的层被视为W- GAN 中批评网络的超参数。 评论家在初步阶段利用这些层次的输出提取概念信息。 视频增强网络与该评论家进行了对抗性培训。 我们的结果显示,在评论中引入这些选定特征后,与其他不利用优化特征选择理念的批评网络相比,其表现会提高10%(FID)和15%(KID)。</s>