Low-Rank Semidefinite Programs via Bilinear Factorization

Many machine learning problems can be reduced to learning a low-rank positive semidefinite matrix (denoted as $Z$), which encounters semidefinite program (SDP). Existing SDP solvers are often expensive for large-scale learning. To avoid directly solving SDP, some works convert SDP into a nonconvex program by factorizing $Z$ \textit{quadraticly} as $XX^\top$. However, this would bring higher-order nonlinearity, resulting in scarcity of structure in subsequent optimization. In this paper, we propose a novel surrogate for SDP learning, in which the structure of subproblem is exploited. More specifically, we surrogate unconstrained SDP by a biconvex problem, through factorizing $Z$ \textit{bilinearly} as $XY^\top$ and using a Courant penalty to penalize the difference of $X$ and $Y$, in which the resultant subproblems in terms of $X$ and $Y$ are convex respectively. Furthermore, we provide a theoretical bound for the associated penalty parameter under the assumption that the subobjective function of $X$ or $Y$ is $L$-Lipschitz-smooth and $\sigma$-strongly convex, such that the proposed surrogate will solve the original SDP when the penalty parameter is larger than this bound (that is $\gamma>\frac{1}{4}(L-\sigma)$). Experiments on two SDP-related applications demonstrate that the proposed algorithm is as accurate as the state-of-the-art, but is faster on large-scale learning.