Estimating the 3D structure of the drivable surface and surrounding environment is a crucial task for assisted and autonomous driving. It is commonly solved either by using expensive 3D sensors such as LiDAR or directly predicting the depth of points via deep learning. Instead of following existing methodologies, we propose Road Planar Parallax Attention Network (RPANet), a new deep neural network for 3D sensing from monocular image sequences based on planar parallax, which takes full advantage of the commonly seen road plane geometry in driving scenes. RPANet takes a pair of images aligned by the homography of the road plane as input and outputs a $\gamma$ map for 3D reconstruction. Beyond estimating the depth or height, the $\gamma$ map has a potential to construct a two-dimensional transformation between two consecutive frames while can be easily derived to depth or height. By warping the consecutive frames using the road plane as a reference, the 3D structure can be estimated from the planar parallax and the residual image displacements. Furthermore, to make the network better perceive the displacements caused by planar parallax, we introduce a novel cross-attention module. We sample data from the Waymo Open Dataset and construct data related to planar parallax. Comprehensive experiments are conducted on the sampled dataset to demonstrate the 3D reconstruction accuracy of our approach in challenging scenarios.
翻译:估计可流表面和周围环境的3D结构是辅助和自主驾驶的一项关键任务,通常通过使用LIDAR等昂贵的3D传感器或通过深层学习直接预测点深度来解决。我们提议不采用现有方法,而是采用Plantar Parallax 注意网(RPANet),这是一个新的深海神经网络,用于根据Plantar parlax从单视像序列进行3D感测,充分利用在驾驶场上常见的公路平面几何测量。RPANet采用一副与公路平面同影相匹配的图像,作为投入和输出的美元,用于3D重建。除了估计深度或高度之外,$\gamma美元地图还有可能在两个连续框架之间进行两维的转换,同时很容易得出深度或高度。通过利用平面平面平面图对连续框架进行扭曲,3D结构可以从平面平面平面平面平面图和残余图像迁移中进行估计。此外,为了让网络更好地了解平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面图的图,我们从模拟中进行数据模拟数据模拟,我们从样面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面图,我们,我们。我们将数据模拟,我们开始,我们将数据模拟数据模拟,我们用新的平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面图图,我们将数据模拟图图,我们将数据模型平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面平面图,我们。我们,