Handling Object Symmetries in CNN-based Pose Estimation

In this paper we investigate the problems that Convolutional Neural Networks (CNN) based pose estimators have with symmetric objects. We find that the CNN's output representation has to form a closed loop when continuously rotating by one step of symmetry. Otherwise the CNN has to learn an uncontinuous function. On a 1-DOF toy example we show that commonly used representations do not fulfill this demand and analyze the problems caused thereby. In particular we find, that the popular min-over-symmetries approach for creating a symmetry aware loss tends not to work well with gradient based optimization, i.e. deep learning. We propose a representation called "closed symmetry loop"' (csl) from these insights, where the angle of relevant vectors is multiplied by the symmetry order and then generalize it to 6-DOF. The representation extends our previous algorithm including a method to disambiguate symmetric equivalents during the final pose estimation. The algorithm handles continuous rotational symmetry (i.e. a bottle) and discrete rotational symmetry (general boxes, boxes with a square face, uniform prims, but no cubes). It is evaluated on the T-LESS dataset.