Lagrange Multipliers and Rayleigh Quotient Iteration in Constrained Type Equations

We generalize the Rayleigh quotient iteration to a class of functions called vector Lagrangians. The convergence theorem we obtained generalizes classical and nonlinear Rayleigh quotient iterations, as well as iterations for tensor eigenpairs and constrained optimization. In the latter case, our generalized Rayleigh quotient is an estimate of the Lagrange multiplier. We discuss two methods of solving the updating equation associated with the iteration. One method leads to a generalization of Riemannian Newton method for embedded manifolds in a Euclidean space while the other leads to a generalization of the classical Rayleigh quotient formula. Applying to tensor eigenpairs, we obtain both an improvements over the state-of-the-art algorithm, and a new quadratically convergent algorithm to compute all complex eigenpairs of sizes typical in applications. We also obtain a Rayleigh-Chebyshev iteration with cubic convergence rate, and give a clear criterion for RQI to have cubic convergence rate, giving a common framework for existing algorithms.