Accelerated Computation of Eigenvectors

Abstract

In Chapter 5, I discussed several adaptive algorithms for computing principal and minor eigenvectors of the online correlation matrix A_k∈ℜ^nXn from a sequence of vectors {x_k∈ℜⁿ}. I derived these algorithms by applying the gradient descent on an objective function. However, it is well known [Baldi and Hornik 95, Chatterjee et al. Mar 98, Haykin 94] that principal component analysis (PCA) algorithms based on gradient descents are slow to converge. Furthermore, both analytical and experimental studies show that convergence of these algorithms depends on appropriate selection of the gain sequence {η_k}. Moreover, it is proven [Chatterjee et al. Nov 97; Chatterjee et al. Mar 98; Chauvin 89] that if the gain sequence exceeds an upper bound, then the algorithms may diverge or converge to a false solution.