On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors

Predicting the states of dynamic traffic actors into the fu-ture is important for autonomous systems to operate safelyand efficiently. Remarkably, the most critical scenarios aremuch less frequent and more complex than the uncriticalones. Therefore, uncritical cases dominate the prediction.In this paper, we address specifically the challenging sce-narios at the long tail of the dataset distribution. Our anal-ysis shows that the common losses tend to place challeng-ing cases sub-optimally in the embedding space. As a con-sequence, we propose to supplement the usual loss with aloss that places challenging cases closer to each other. Thistriggers sharing information among challenging cases andlearning specific predictive features. We show on four pub-lic datasets that this leads to improved performance on thechallenging scenarios while the overall performance staysstable. The approach is agnostic w.r.t. the used networkarchitecture, input modality or viewpoint, and can be inte-grated into existing solutions easily.