Stable Recovery and the Coordinate Small-Ball Behaviour of Random Vectors

Abstract

Recovery procedures in Data Science are often based on stable point separation. In its simplest form, stable point separation implies that if f is “far away” from 0, and one is given a random sample $(f(Z_i))_{i=1}^m$ where a proportional number of the sample points may be corrupted by noise—even maliciously, that information is still enough to exhibit that f is far from 0.

Stable point separation is well understood in the context of iid sampling, and to explore it for general sampling methods we introduce a new notion—the coordinate small-ball of a random vector X. Roughly put, this feature captures the number of “relatively large coordinates” of $(|\left \langle TX,u_i \right \rangle |)_{i=1}^m$, where $T:\mathbb {R}^n \to \mathbb {R}^m$ is an arbitrary linear operator and $(u_i)_{i=1}^m$ is any fixed orthonormal basis of $\mathbb {R}^m$.

We show that under the bare-minimum assumptions on X, and with high probability, many of the values $|\left \langle TX,u_i \right \rangle |$ are at least of the order $\|T\|{ }_{S_2}/\sqrt {m}$. As a result, the “coordinate structure” of TX exhibits the typical Euclidean norm of TX and does so in a stable way.

One outcome of our analysis is that random sub-sampled convolutions satisfy stable point separation under minimal assumptions on the generating random vector—a fact that was known previously only in a highly restrictive setup, namely, for random vectors with iid subgaussian coordinates. As an application we address the problem of sparse signal recovery using a circulant matrix when a proportion of the given sample is corrupted by malicious noise.

Supported by NSF grant DMS-1812240 and Simons Fellows in Mathematics Award 823432.

Appendices

A Examples of Vectors That Satisfy The SBA

Here we give examples of several generic random vectors that satisfy the SBA. This is far from being an exhaustive list and should be viewed only as an indication to the fact that the SBA is a property shared by many natural random vectors.

• (1) Let $X= (\xi _{1}, \cdots , \xi _{n})$ where the $\xi _{i}$’s are independent random variables with densities bounded by ${\mathcal {L}}$. It was shown in [26] that X satisfies the SBA with constant $c{\mathcal {L}}$, where $c>0$ is an absolute constant.

  This fact was further extended in [11, 24]; most notably, it was shown in [24] that if the coordinates of $X=(\xi _i)_{i=1}^n$ are independent random variables with densities bounded by 1 and the coordinates of $Y=(\eta _i)_{i=1}^n$ are uniformly distributed in $[-\frac {1}{2} , \frac {1}{2}]$, then for every semi-norm $\| \cdot \|$ and $t>0$,

  $$\displaystyle \begin{aligned} {} \mathbb P ( \|X\|\leq t ) \leq \mathbb P ( \| Y \|\leq t ). \end{aligned} $$

  (A.1)

  In particular, among all such vectors the ‘worse’ small-ball behaviour—with respect to any semi-norm—is exhibited by the uniform measure on the cube $[-\frac {1}{2} , \frac {1}{2}]^n$.

  Observe that for the Euclidean norm, the small-ball behaviour of Y  and of the standard Gaussian vector G is the same up to absolute constants.

• (2) Perturbations: It is standard to verify that if X satisfies the SBA with a constant $\mathcal {L}$ and W is an arbitrary random vector that is independent of X, then $W+\delta X$ satisfies SBA with a constant depending on $\delta$ and $\mathcal {L}$.

• (3) The question of whether there is a constant $\mathcal {L}$ such that any isotropic log-concave random vector satisfies the SBA with constant $\mathcal {L}$ is equivalent to Bourgain’s celebrated Hyperplane Conjecture (see [2] and the discussion in [7] and [4])).

Thanks to the extensive study of log-concave measures and the connection the SBA has with the Hyperplane conjecture for such measures, there are some important examples of isotropic, log-concave random vectors that are known to satisfy the SBA with an absolute constant:

• If X is also 1-unconditional (see [20], section 8.2);

• If X is also subgaussian ([2, 3]);

• If X is also supergaussian (this follows from results of [22]).

B Proof of Remark 1.14

The proof requires some additional notation. Let X be a random vector in $\mathbb {R}^{n}$ and let $p\geq 1$. The $Z_{p}$ body of X is defined as the (centrally-symmetric) convex body whose support function is

$$\displaystyle \begin{aligned} {} h_{Z_{p} (X) } ( \theta) = \left( \mathbb{E}|\left\langle X, \theta \right\rangle |{}^{p} \right)^{\frac{1}{p}} , \ \ \ \theta \in S^{n-1} . \end{aligned} $$

(B.1)

It is straightforward to verify that if $T :\mathbb {R}^{n} \to \mathbb {R}^{m}$ is a linear operator then

$$\displaystyle \begin{aligned} {} Z_{p} (T X) = T Z_{p} ( X). \end{aligned} $$

(B.2)

Lemma B.1

There are absolute constants $c_1$ and $c_2$ for which the following holds. Let $X$ be a centred log-concave random vector in $\mathbb {R}^{n}$ that satisfies the SBA with constant ${\mathcal {L}}$ . For any $T\in GL_{n}$ and $F\in \mathcal {G}_{n,k}$ one has

$$\displaystyle \begin{aligned} {} \frac{c_{1}}{ | \mathrm{det}[(P_{F} T) ( P_{F} T)^{\ast}] |{}^{\frac{1}{2k}} } \leq f_{P_{F}TX}^{\frac{1}{k}}(0) \leq \frac{c_{2}{\mathcal{L}} }{ ( \mathrm{det}[(P_{F} T) ( P_{F} T)^{\ast}] )^{\frac{1}{2k}} }, \end{aligned} $$

(B.3)

where the left-hand side holds true under the additional assumption that X is isotropic.

The proof of Lemma B.1 is based on two facts. The first is a standard observation from linear algebra: let $T: \mathbb {R}^{n} \to \mathbb {R}^{k}$, set $E= \mathrm {ker}(T)^{\perp }= \mathrm {im} (T^{\ast } )$ and denote by $T|{ }_{E}$ the restriction of T to E. Then for any compact set $K \subset \mathbb {R}^n$,

$$\displaystyle \begin{aligned} {} \mathrm{vol}(T K) = \mathrm{det} ( T T^{\ast} )\cdot \mathrm{vol}(P_{E} K) . \end{aligned} $$

(B.4)

The second observation is Proposition 3.7 from [23]: If X is a centred, log-concave random vector then

$$\displaystyle \begin{aligned} {} f_{X}^{\frac{1}{n}} (0)\sim \mathrm{vol}^{-\frac{1}{n}}(Z_{n} ( X) ). \end{aligned} $$

(B.5)

Proof of Lemma B.1

By the Prekopá-Leindler inequality, for every linear operator S, the random vector SX is also log-concave and centred. Hence, using (B.2), (B.5) and (B.4), it is evident that

$$\displaystyle \begin{aligned} & f_{P_{F}TX}^{\frac{1}{k}}(0) \sim (\mathrm{vol} (Z_{k} ( P_{F} T X) ))^{-\frac{1}{k}} \sim (\mathrm{vol} (P_{F} T Z_{k} ( X) ))^{-\frac{1}{k}} \\ &\sim \frac{1}{ ( \mathrm{det}[(P_{F} T) ( P_{F} T)^{\ast}] )^{\frac{1}{2k}} } (\mathrm{vol} ( P_{E} Z_{k} ( X) ))^{-\frac{1}{k}}\\ & \sim \frac{1}{ ( \mathrm{det}[(P_{F} T) ( P_{F} T)^{\ast}])^{\frac{1}{2k}} } (\mathrm{vol} (Z_{k} ( P_{E} X) ))^{-\frac{1}{k}} \\ & \sim \frac{ f_{P_{E}X}^{\frac{1}{k}}(0)}{ ( \mathrm{det}[(P_{F} T) ( P_{F} T)^{\ast}] )^{\frac{1}{2k}} }. \end{aligned} $$

Clearly, $f_{P_{E}X}^{\frac {1}{k}}(0) \leq {\mathcal {L}}$, which proved the right-hand side inequality in (B.3). Moreover if X is an isotropic log-concave random vector in $\mathbb {R}^{n}$ then $f_{X}^{\frac {1}{n}} (0 ) \geq c$, where c is an absolute constant (see, e.g. [1]). And since $P_{F} X$ is also isotropic when X is, the left-hand side inequality in (B.3) follows. □

Combining (B.3) and (2.1) it is evident that:

Proposition B.2

There are absolute constants $c_1$ and $c_2$ for which the following holds. Let X be an isotropic log-concave random vector in $\mathbb {R}^{n}$ that satisfies the SBA with constant ${\mathcal {L}}$ and let $T: \mathbb {R}^{n} \to \mathbb {R}^{m}$ be a linear operator. Then

$$\displaystyle \begin{aligned} {} \left( \mathbb E \| T G/ (c_{2} {\mathcal{L}}) \|{}_{2}^{-k} \right)^{-\frac{1}{k}} \leq \left( \mathbb E \| T X \|{}_{2}^{-k} \right)^{-\frac{1}{k}} \leq \left( \mathbb E \| T G/c_{1} \|{}_{2}^{-k} \right)^{-\frac{1}{k}}. \end{aligned} $$

(B.6)