A semi-randomized and augmented Kaczmarz method with simple random sampling for large-scale inconsistent linear systems

A greedy randomized augmented Kaczmarz (GRAK) method was proposed in [Z.-Z. Bai and W.-T. WU, SIAM J. Sci. Comput., 43 (2021), pp. A3892-A3911] for large and sparse inconsistent linear systems. However, one has to construct two new index sets via computing residual vector with respect to the augmented linear system in each iteration. Thus, the computational overhead of this method is large for extremely large-scale problems. Moreover, there is no reliable stopping criterion for this method. In this work, we are interested in solving large-scale sparse or dense inconsistent linear systems, and try to enhance the numerical performance of the GRAK method. First, we propose an accelerated greedy randomized augmented Kaczmarz method. Theoretical analysis indicates that it converges faster than the GRAK method under very weak assumptions. Second, in order to further release the overhead, we propose a semi-randomized augmented Kaczmarz method with simple random sampling. Third, to the best of our knowledge, there are no practical stopping criteria for all the randomized Kaczmarz-type methods till now. To fill-in this gap, we introduce a practical stopping criterion for Kaczmarz-type methods, and show its rationality from a theoretical point of view. Numerical experiments are performed on both real-world and synthetic data sets, which demonstrate the efficiency of the proposed methods and the effectiveness of our stopping criterion.