Harnessing Geometric Constraints from Emotion Labels to improve Face Verification

For the task of face verification, we explore the utility of harnessing auxiliary facial emotion labels to impose explicit geometric constraints on the embedding space when training deep embedding models. We introduce several novel loss functions that, in conjunction with a standard Triplet Loss [43], or ArcFace loss [10], provide geometric constraints on the embedding space; the labels for our loss functions can be provided using either manually annotated or automatically detected auxiliary emotion labels. We analyze the performance of our methods on nine different (CK+, PubFig, Tufts, VggFace2, LFW, YTF, MegaFace, IJB-B, IJB-C). Our results consistently show that the additional structure encoded about each face's emotions results in an embedding model that gives higher verification accuracy than training with just ArcFace or Triplet Loss alone. Moreover, the method is significantly more accurate than a simple multi-task learning approach. Our method is implemented purely in terms of the loss function and does not require any changes to the neural network backbone of the embedding function.