A Higher-Order Generalized Singular Value Decomposition for Rank Deficient Matrices

The higher-order generalized singular value decomposition (HO-GSVD) is a matrix factorization technique that extends the GSVD to $N \ge 2$ data matrices, and can be used to identify shared subspaces in multiple large-scale datasets with different row dimensions. The standard HO-GSVD factors $N$ matrices $A_i\in\mathbb{R}^{m_i\times n}$ as $A_i=U_i\Sigma_i V^\text{T}$, but requires that each of the matrices $A_i$ has full column rank. We propose a reformulation of the HO-GSVD that extends its applicability to rank-deficient data matrices $A_i$. If the matrix of stacked $A_i$ has full rank, we show that the properties of the original HO-GSVD extend to our reformulation. The HO-GSVD captures shared right singular vectors of the matrices $A_i$, and we show that our method also identifies directions that are unique to the image of a single matrix. We also extend our results to the higher-order cosine-sine decomposition (HO-CSD), which is closely related to the HO-GSVD. Our extension of the standard HO-GSVD allows its application to datasets with $m_i < n$, such as are encountered in bioinformatics, neuroscience, control theory or classification problems.