We consider the problem of task-agnostic feature upsampling in dense prediction where an upsampling operator is required to facilitate both region-sensitive tasks like semantic segmentation and detail-sensitive tasks such as image matting. Existing upsampling operators often can work well in either type of the tasks, but not both. In this work, we present FADE, a novel, plug-and-play, and task-agnostic upsampling operator. FADE benefits from three design choices: i) considering encoder and decoder features jointly in upsampling kernel generation; ii) an efficient semi-shift convolutional operator that enables granular control over how each feature point contributes to upsampling kernels; iii) a decoder-dependent gating mechanism for enhanced detail delineation. We first study the upsampling properties of FADE on toy data and then evaluate it on large-scale semantic segmentation and image matting. In particular, FADE reveals its effectiveness and task-agnostic characteristic by consistently outperforming recent dynamic upsampling operators in different tasks. It also generalizes well across convolutional and transformer architectures with little computational overhead. Our work additionally provides thoughtful insights on what makes for task-agnostic upsampling. Code is available at: http://lnkiy.in/fade_in
翻译:我们考虑的是任务- 不可知特性在密集的预测中进行取样的问题。 在密密的预测中, 需要进行抽样的操作员来协助进行对区域敏感的任务, 如语义分割和图像配对等细节敏感的任务。 现有的抽查操作员通常在两种任务类型中都能很好地工作, 但不是两者兼而有之。 在这项工作中, 我们首先介绍FADE, 一个新颖的、 插头和游戏, 以及任务- 不可知的抽查操作员。 FADE 有三个设计选择的好处 : i) 考虑在抽查内核的生成中联合编码和解码特性; ii 高效的半转移同流操作员, 能够控制每个特性点如何有助于采集内核; iii) 一个分离器, 来强化细节的划界。 我们首先研究FADE在玩具数据上的新样本采集特性, 然后评估大规模语义分解和图像配对。 特别是, FADE 显示其有效性和任务- 任务- 特征特征, 持续超越了最近动态的上上升- 模缩略的操作者 。 在不同的结构中, 我们的任务中, 也提供常规化任务中, 提供相同的解解析。