This paper proposes SemCal: an automatic, targetless, extrinsic calibration algorithm for a LiDAR and camera system using semantic information. We leverage a neural information estimator to estimate the mutual information (MI) of semantic information extracted from each sensor measurement, facilitating semantic-level data association. By using a matrix exponential formulation of the $se(3)$ transformation and a kernel-based sampling method to sample from camera measurement based on LiDAR projected points, we can formulate the LiDAR-Camera calibration problem as a novel differentiable objective function that supports gradient-based optimization methods. We also introduce a semantic-based initial calibration method using 2D MI-based image registration and Perspective-n-Point (PnP) solver. To evaluate performance, we demonstrate the robustness of our method and quantitatively analyze the accuracy using a synthetic dataset. We also evaluate our algorithm qualitatively on an urban dataset (KITTI360) and an off-road dataset (RELLIS-3D) benchmark datasets using both hand-annotated ground truth labels as well as labels predicted by the state-of-the-art deep learning models, showing improvement over recent comparable calibration approaches.
翻译:本文提出SemCal : 使用语义信息为LIDAR 和相机系统提供自动、无目标、外部校准算法; 我们利用神经信息估计器来估计从每个传感器测量中提取的语义信息的相互信息(MI), 便利语义级数据协会。 通过使用以LIDAR 预测点为基础的USse(3)美元转换和内核取样方法的矩阵指数式配方, 从相机测量样本到基于LIDAR 预测点的摄像测量, 我们可以将LIDAR- Camera校准问题作为支持梯度优化方法的新型差异目标函数。 我们还采用基于语义的初始校准方法, 使用 2D MI 图像注册和透视- 点( PnP) 求求解器来进行基于语义的初步校准。 为了评估绩效, 我们用合成数据集来展示了我们的方法的稳健性和定量分析准确性。 我们还评估了城市数据集的算质量( KITTI360) 和离线数据集(RELIS-3D) 的基准数据集, 使用手记式地面校准方法, 并用最近的标签式校准方法, 以近的校准方式对现状作了预测。