Implicit Neural Representations (INRs) encoding continuous multi-media data via multi-layer perceptrons has shown undebatable promise in various computer vision tasks. Despite many successful applications, editing and processing an INR remains intractable as signals are represented by latent parameters of a neural network. Existing works manipulate such continuous representations via processing on their discretized instance, which breaks down the compactness and continuous nature of INR. In this work, we present a pilot study on the question: how to directly modify an INR without explicit decoding? We answer this question by proposing an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR. Our key insight is that spatial gradients of neural networks can be computed analytically and are invariant to translation, while mathematically we show that any continuous convolution filter can be uniformly approximated by a linear combination of high-order differential operators. With these two knobs, INSP-Net instantiates the signal processing operator as a weighted composition of computational graphs corresponding to the high-order derivatives of INRs, where the weighting parameters can be data-driven learned. Based on our proposed INSP-Net, we further build the first Convolutional Neural Network (CNN) that implicitly runs on INRs, named INSP-ConvNet. Our experiments validate the expressiveness of INSP-Net and INSP-ConvNet in fitting low-level image and geometry processing kernels (e.g. blurring, deblurring, denoising, inpainting, and smoothening) as well as for high-level tasks on implicit fields such as image classification.
翻译:内隐隐性表示器(INRs)通过多层显示器编码连续的多媒体数据。 尽管许多应用成功, 编辑和处理内隐性表示器(INR)仍然难以理解, 因为神经网络的潜在参数代表着信号。 现有的工程通过处理其分散的事例来操纵这种连续表示器, 从而打破内核网络的紧凑性和连续性。 在这项工作中, 我们提出一个试验性研究: 如何在没有明确解码的情况下直接修改内核网络的连续多媒体数据? 我们回答这个问题的方式是提出一个隐含的神经信号处理网络, 称为INSP- Net, 通过内核网络的不同操作器操作器。 我们的主要洞察力是, 神经网络的空间梯度可以用分析来计算, 无法翻译。 我们数学显示, 任何连续的变动过滤器都可以通过高级差分解操作器的线性组合来统一。 有了这两个 knob, INSP- Net 即即时将信号处理操作器作为计算图的加权组成, IMIS- 直径分析器的高级衍生器, IMIS- 网络 的重量参数可以进一步进行数据化。