A Two-Branch Neural Network for Gait Recognition

Gait recognition, a promising long-distance biometric technology, has aroused intense interest in computer vision. Existing works on gait recognition can be divided into appearance-based methods and model-based methods, which extract features from silhouettes and skeleton data, respectively. However, since appearance-based methods are greatly affected by clothing changing and carrying condition, and model-based methods are limited by the accuracy of pose estimation approaches, gait recognition remains challenging in practical applications. In order to integrate the merits of such two approaches, a two-branch neural network (NN)-based model is proposed in this paper. The method contains two branches, namely a CNN-based branch taking silhouettes as input and a GCN-based branch taking skeletons as input. In addition, two modifications are introduced into the GCN-based branch to boost the performance. First, we present a simple fully connected graph convolution operator to integrate multi-scale graph convolutions and relieve dependence on natural connections. Second, we deploy an attention module named STC-Att after each GCN block to learn spatial, temporal and channel-wise attention simultaneously. We evaluated the proposed two-branch neural network on the CASIA-B dataset. The experimental results show that our method achieves state-of-the-art performance in various conditions.