Solving linear algebraic equations with Krylov subspace methods is still interesting!

The title of this paper is motivated by the title of the paper by Forsythe written in 1952 and published in Bull. Amer. Math. Soc., 59 (1953), pp. 299-329. Forsythe argues that solving a system of $n$ linear algebraic equations in $n$ unknowns is mathematically a lowly subject. His beautiful text graduates with what was at that time ``the newest process on the roster, the method of conjugate gradients.'' We consider it important to revisit, after 70 years, to what extent Forsythe's views, and the views presented in the related contemporary works of Hestenes, Stiefel, Lanczos, Karush, and Hayes, remain relevant today. Including, besides the conjugate gradient method (CG), also the generalized minimal residual method (GMRES), we point out building blocks that we consider central for the current mathematical and computational understanding of Krylov subspace methods. We accomplish this through a set of computed examples. We keep technical details to a minimum and provide references to the literature. This allows us to demonstrate the mathematical beauty and intricacies of the methods, and to recall some persistent misunderstandings as well as important open problems. We hope that this work can initiate further theoretical investigations of Krylov subspace methods. This paper can not cover all Krylov subspace methods. The principles discussed for CG and GMRES are, however, important for all of them. Further, practical computations always incorporate preconditioning. We will not deal with preconditioning techniques, but we will deal with the basic question of how preconditioning is motivated, and we will recall some recent analytic results.