Kaczmarz-inspired acceleration for kernel canonical correlation analysis: Theory, algorithms, and its applications to information retrieval.

As a fundamental method in multivariate statistical analysis, Canonical Correlation Analysis (CCA) effectively captures linear relationships between paired variables. However, its linearity constraint limits applicability to complex nonlinear dependencies. Kernel CCA (KCCA) addresses this limitation by enabling nonlinear relationship modeling and high-dimensional feature selection. Existing KCCA methods face significant computational bottlenecks when processing large-scale data, primarily due to their prohibitive memory and time complexity. To overcome these constraints, we develop two accelerated KCCA solvers, termed KCCA-GRK and KCCA-MWRK, by adapting the established Greedy Randomized Kaczmarz (GRK) and Maximal Weighted Residual Kaczmarz (MWRK) row-selection principles to the SVD-reformulated KCCA system. The contribution is KCCA-specific: because the reformulated coefficient matrices have orthonormal rows, norm-proportional RK sampling becomes uniform; KCCA-GRK and KCCA-MWRK restore informative row selection through residual-driven randomized and deterministic rules, respectively. We further establish explicit convergence bounds and sufficient iteration estimates for the proposed algorithms. Experiments on one synthetic dataset and four real-world datasets demonstrate that the proposed methods preserve retrieval performance while substantially reducing iteration counts and runtime.