Depth Estimation and Object Detection Recognition play an important role in autonomous driving technology under the guidance of deep learning artificial intelligence. We propose a hybrid structure called RealNet: a co-design method combining the model-streamlined recognition algorithm, the depth estimation algorithm with information fusion, and deploying them on the Jetson-Nano for unmanned vehicles with monocular vision sensors. We use ROS for experiment. The method proposed in this paper is suitable for mobile platforms with high real-time request. Innovation of our method is using information fusion to compensate the problem of insufficient frame rate of output image, and improve the robustness of target detection and depth estimation under monocular vision.Object Detection is based on YOLO-v5. We have simplified the network structure of its DarkNet53 and realized a prediction speed up to 0.01s. Depth Estimation is based on the VNL Depth Estimation, which considers multiple geometric constraints in 3D global space. It calculates the loss function by calculating the deviation of the virtual normal vector VN and the label, which can obtain deeper depth information. We use PnP fusion algorithm to solve the problem of insufficient frame rate of depth map output. It solves the motion estimation depth from three-dimensional target to two-dimensional point based on corner feature matching, which is faster than VNL calculation. We interpolate VNL output and PnP output to achieve information fusion. Experiments show that this can effectively eliminate the jitter of depth information and improve robustness. At the control end, this method combines the results of target detection and depth estimation to calculate the target position, and uses a pure tracking control algorithm to track it.
翻译:深度估计和物体探测识别在深层学习人工智能的指导下,在自主驱动技术中发挥着重要的作用。 我们提出一个混合结构,名为 RealNet:一个共同设计方法,将模型-流线识别算法、信息聚合的深度估计算法结合起来,并在配有单眼视觉传感器的无人驾驶飞行器的Jetson-Nano上部署这些算法。 我们使用ROS 进行实验。 本文中建议的方法适合具有高实时请求的移动平台。 我们的方法创新是使用信息聚合来弥补产出图像框架深度不足的问题,提高单眼视觉下目标探测和深度估算的可靠性。 我们使用PnP 目标探测测算法, 以YOLO- v5. 为基础, 简化其 DarkNet53 的网络结构结构结构结构,并实现最高至 0.01的预测速度。 深度估计基于 VNL 深度的多重测算法, 通过计算虚拟正常矢量方法和标签的偏差, 来计算损失函数,从而获得更深的深度信息。 我们使用PnP 校准定位定位定位定位定位定位定位定位定位定位定位定位定位定位定位, 以比低的测算法, 度测测算, 深度, 度测测算出这个深度, 深度, 深度测算出这个深度测算方法比基点测测算出该方向的路径, 测算出该深度测算出该方向的精确度, 度, 测算为深度, 深度, 深度, 测算, 测算, 测算方法的精确度, 测算, 深度测算, 测算出该方向 度 度 度 度 深度, 测算为深度测算, 测算, 测算到 度 测算, 度 度 测算 度 度 测算 度 度 度 度 度 度 度 度 度 度 度 度 度 度 深度测算为 深度测算 度 度 测算 测算 测算 度 测算 测算 度 度 度 度 度 度 测算 测算 测算 测算 测算 测算 测算 测算 测算 测算 测算