We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage
翻译:我们提出一种方法,通过单发编码图像压缩获得一个动态光场(5-D卷) (一个5-D卷) 。 我们设计了一个图像模型, 在一个曝光时间里同步应用孔径编码和像素接触编码。 这个编码方案使我们能够有效地将原始信息嵌入一个观测到的图像中。 然后, 观测到的图像被反馈到一个光场重建的动态神经网络( CNN), 这个网络与摄像头的编码模式共同培训。 我们还开发了一个硬件原型, 以捕捉一个长期移动的真正的三维场景。 我们成功地从一个被观测到的图像中获得了一个5x5视图的动态光场, 超过4个时段子框架( 总共100个视图) 。 随着时间的推移, 我们可以在摄像头框架速度的4x获得一个动态光场。 据我们了解, 我们的方法是首先在压缩光场获取时实现比相机本身更细的时间分辨率的方法。 我们的软件可以从我们的项目网页上获得。