Sensitivity of low-rank matrix recovery

Abstract

We characterize the first-order sensitivity of approximately recovering a low-rank matrix from linear measurements, a standard problem in compressed sensing. A special case covered by our analysis is approximating an incomplete matrix by a low-rank matrix. This is one customary approach to build recommender systems. We give an algorithm for computing the associated condition number and demonstrate experimentally how the number of measurements affects it. In addition, we study the condition number of the best rank-r matrix approximation problem. It measures in the Frobenius norm by how much an infinitesimal perturbation to an arbitrary input matrix is amplified in the movement of its best rank-r approximation. We give an explicit formula for its condition number, which shows that it depends on the relative singular value gap between the rth and $(r+1)$th singular values of the input matrix.

Proof of Lemma 4

We restate Lemma 4 in terms of vector fields, as required by the proof, and then we prove it.

Lemma 4 (The second fundamental form under affine linear diffeomorphisms)

Consider Riemannian embedded submanifolds $\mathcal {U}\subset \mathbb {R}^N$ and $\mathcal {W} \subset \mathbb {R}^\ell$ both of dimension s. Let $L : \mathbb {R}^N \rightarrow \mathbb {R}^\ell , Y\mapsto M(Y)+b$ be an affine linear map that restricts to a diffeomorphism from $\mathcal {U}$ to $\mathcal {W}$. For a fixed $Y\in \mathcal {U}$ let $\mathcal {E} =(E_1,\ldots , E_s)$ be a local smooth frame of $\mathcal {U}$ in the neighborhood of Y. For each $1\le i\le s$ let $F_i$ be the vector field on $\mathcal {W}$ that is $L\vert _{\mathcal {U}}$-related to $E_i$. Then $\mathcal {F} = (F_1,\ldots ,F_s)$ is a smooth frame on $\mathcal {W}$ in the neighborhood of $X=L(Y)$, and we have

$$\begin{aligned} I\!I _{X}(F_i, F_j) = \textrm{P}_{\textrm{N}_{X} \mathcal {W}}\left( M( I\!I _Y(E_i, E_j) ) \right) . \end{aligned}$$

Herein, $I\!I _Y \in \textrm{T}_{Y} {\mathcal {U}} \otimes \textrm{T}_{Y} {\mathcal {U}} \otimes \textrm{N}_{Y} {\mathcal {U}}$ should be viewed as an element of the tensor space $(\textrm{T}_{Y} {\mathcal {U}})^* \otimes (\textrm{T}_{Y} {\mathcal {U}})^* \otimes \textrm{N}_{Y} {\mathcal {U}}$ by dualization relative to the Riemannian metric, and likewise for $I\!I _{X}$.

Proof

Our assumption of $L\vert _{\mathcal {U}}:\mathcal {U} \rightarrow \mathcal {W}$ being a diffeomorphism implies that $\mathcal {F}$ is a smooth frame. Let $1\le i,j\le s$, and $\varepsilon _i(t) \subset \mathcal {U}$ be the integral curve of $E_i$ starting at Y (see [41, Chapter 9]). Let $\phi _i(t) = L(\varepsilon _i(t))$ be the corresponding integral curve on $\mathcal {W}$ at $X=L(Y)$. Since $F_j$ is $L\vert _{\mathcal {U}}$-related to $E_j$, we have

$$\begin{aligned} F_j\vert _{\phi _i(t)} = (\textrm{d}_{\varepsilon _i(t)}L\vert _{\mathcal {U}}) (E_j\vert _{\varepsilon _i(t)}), \end{aligned}$$

where $\textrm{d}_{\varepsilon _i(t)}L\vert _{\mathcal {U}}$ is the derivative $\textrm{d}_{\varepsilon _i(t)}L: \mathbb {R}^N\rightarrow \mathbb {R}^\ell$ of L at $\varepsilon _i(t)$ restricted to the tangent space $\textrm{T}_Y\mathcal {U}\subset \mathbb {R}^N$. On the other hand, $\textrm{d}_{\varepsilon _i(t)}L\ = M$, so that

$$\begin{aligned} F_j\vert _{\phi _i(t)} = M \, E_j\vert _{\varepsilon _i(t)}. \end{aligned}$$

(26)

The fact that the derivative of L is constant is the key part in the proof. Interpreting $F_{j}\vert _{\phi _i(t)}$ as a smooth curve in $\textrm{T}_{\phi _i(t)} {\mathbb {R}^\ell } \simeq \mathbb {R}^\ell$ and $E_j\vert _{\varepsilon _i(t)}$ as a smooth curve in $\textrm{T}_{\varepsilon _i(t)} {\mathbb {R}^N} \simeq \mathbb {R}^N$, we can take the usual derivatives at $t=0$ on both sides of (26):

$$\begin{aligned} \frac{\textrm{d}}{\textrm{d} t} F_{j}\vert _{\phi _i(t)} = M\, \frac{\textrm{d}}{\textrm{d} t} E_j\vert _{\varepsilon _i(t)}. \end{aligned}$$

Recall that $Y=\varepsilon _i(0)$. We can decompose the right hand side into tangent and normal part at $Y\in \mathcal {U}$, so that

$$\begin{aligned} \frac{\textrm{d}}{\textrm{d} t} F_{j}\vert _{\phi _i(t)}= M\, \left( \textrm{P}_{\textrm{T}_{Y} {\mathcal {U}}} \frac{\textrm{d}}{\textrm{d} t} E_j\vert _{\varepsilon _i(t)} \right) + M\, \left( \textrm{P}_{\textrm{N}_{Y}\mathcal {U}} \frac{\textrm{d}}{\textrm{d} t} E_j\vert _{\varepsilon _i(t)} \right) . \end{aligned}$$

Observe that that $M(\textrm{T}_{Y} {\mathcal {U}}) = \textrm{T}_{X} {\mathcal {W}}$, where $X=L(Y)$. Projecting both sides to the normal space of $\mathcal {W}$ at X yields

$$\begin{aligned} \textrm{P}_{\textrm{N}_{X} \mathcal {W}} \left( \frac{\textrm{d}}{\textrm{d} t} F_{j}\vert _{\phi _i(t)} \right) = \textrm{P}_{\textrm{N}_{X} \mathcal {W}} \left( M\, \textrm{P}_{\textrm{N}_{Y}\mathcal {U}} \left( \frac{\textrm{d}}{\textrm{d} t} E_j\vert _{\varepsilon _i(t)} \right) \right) . \end{aligned}$$

The claim follows by applying the Gauss formula for curves [45, Corollary 8.3] on both sides. $\square$