A Physically Admissible Stokes Vector Reconstruction in Linear Polarimetric Imaging

Abstract

Polarization encoded images improve on conventional intensity imaging techniques by providing access to additional parameters describing the vector nature of light. In a polarimetric image, each pixel is related to a $4 \times 1$ vector named Stokes vector ($3 \times 1$ in a linear configuration, which is the framework retained afterwards). Such images comprise a valuable set of physical information on the objects they contain, amplifying subsequently the accuracy of the analysis that can be done. A Stokes imaging polarimeter yields data named radiance images from which Stokes vectors are reconstructed, supposed to comply with a physical admissibility constraint. Classical estimation techniques such as pseudo-inverse approach exhibit defects, hampering any relevant physical interpretation of the scene: (i) first, due to their sensitivity to noise and errors that may contaminate the observed radiance images and that may then propagate to the evaluation of the Stokes vector components, thus justifying an ad hoc a posteriori treatment of Stokes vectors; (ii) second, in not taking this physical admissibility criterion explicitly into account. Motivated by this observation, the proposed contribution aims to provide a method of reconstruction addressing both issues, thus ensuring smoothness and spatial consistency of the reconstructed components, as well as compliance with the prescribed physical admissibility constraint. A by-product of the algorithm is that the resulting angle of polarization reflects more faithfully the physical properties of the materials present in the image. The mathematical formulation yields a non-smooth convex optimization problem that is then converted into a min–max problem and solved by the generic Chambolle–Pock primal-dual algorithm. Several mathematical results (such as existence/uniqueness of the minimizer of the primal problem, existence of a saddle point to the associated Lagrangian, etc.) are supplied and highlight the well-posed character of the modelling. Experiments demonstrate that our method provides significant improvements (i) over the least square-based method both in terms of quantitative criteria (physical admissibility constraint automatically met) and qualitative assessment (spatial regularization/coherency), (ii) over the physical consistency of related relevant polarimetric parameters such as the angle and degree of polarization, (iii) robustness of the method when applied on real outdoor scenes acquired in degraded conditions (poor weather conditions, etc.).

Appendices

Appendix A: Pointwise Infimum of Concave Functions

Let us denote by $f(p)=\displaystyle {\inf _{S\in {\mathcal {K}}}}\,{\mathcal {L}}(S,p)=\displaystyle {\inf _{S\in {\mathcal {K}}}}\,{\mathcal {L}}_S(p)$. In our case, for each p the infimum is reached so that we could write equivalently $f(p)=\displaystyle {\min _{S\in {\mathcal {K}}}}\,{\mathcal {L}}_{S}(p)$. Let us prove that f is concave.

We recall a preliminary result.

Theorem 3

Let X be a set and let $f : X \rightarrow \bar{{\mathbb {R}}}$. The hypograph of f is defined by

$$\begin{aligned} {\text{ hypo }}\,f=\left\{ (x,t)\in X \times {\mathbb {R}}\,\mid \,f(x)\ge t\right\} . \end{aligned}$$

A function $f: X \rightarrow \bar{{\mathbb {R}}}$ is concave if and only if its hypograph is convex.

Coming back to our problem,

$$\begin{aligned} {\text{ hypo }}\,f&=\left\{ (p,t)\in {\mathcal {B}}\times {\mathbb {R}}\,\mid \,f(p)\ge t\right\} , \end{aligned}$$

(A1)

$$\begin{aligned}&=\left\{ (p,t)\in {\mathcal {B}}\times {\mathbb {R}}\,\mid \,\displaystyle {\inf _{S\in {\mathcal {K}}}}\,{\mathcal {L}}_S(p)\ge t\right\} , \end{aligned}$$

(A2)

$$\begin{aligned}&= \bigcap \limits _{S\in {\mathcal {K}}}\, \left\{ (p,t)\in {\mathcal {B}}\times {\mathbb {R}}\,\mid \,{\mathcal {L}}_S(p)\ge t\right\} , \end{aligned}$$

(A3)

which is convex as the intersection of convex sets.

Appendix B: Pointwise Infimum of Continuous Functions

Theorem 4

Let $(X,\tau )$ be a topological space and let $f : X \rightarrow \bar{{\mathbb {R}}}$ be a function. Function f is upper semi-continuous if $\forall \alpha \in {\mathbb {R}}$, the set $\left\{ x\in X\,\mid \,f(x)<\alpha \right\}$ is open.

Let us prove that $f : p \in {\mathcal {B}} \mapsto \displaystyle {\inf _{S\in {\mathcal {K}}}}\,{\mathcal {L}}(S,p)$ is upper semi-continuous. Let $\alpha \in {\mathbb {R}}$. Let $p\in f^{-1}((-\infty ,\alpha ))$. Then, $\displaystyle {\inf _{S\in {\mathcal {K}}}}\,{\mathcal {L}}(S,p)<\alpha$ and there exists ${\bar{S}}_p\in {\mathcal {K}}$ such that ${\mathcal {L}}({\bar{S}}_p,p)={\mathcal {L}}_{{\bar{S}}_p}(p)<\alpha$. This implies that

$$\begin{aligned} f^{-1}((-\infty ,\alpha ))=\bigcup \limits _{S\in {\mathcal {K}}}\, {\mathcal {L}}_{S}^{-1}((-\infty ,\alpha )) \end{aligned}$$

which is open as the union of open sets.

Appendix C: Alternative Proof to Get the Proximal Operator of $\tau \, g$

Consider the three-dimensional optimization problem

$$\begin{aligned} u^{*}&={\text{ prox }}_{\tau \,g}(u=(u_0,u_1,u_2))\\&=\mathop {{\mathrm{arg\,min}}}\limits _{{\tilde{u}}=({\tilde{u}}_0,{\tilde{u}}_1,{\tilde{u}}_2)\in {\mathbb {R}}^3}\,\scriptstyle {\dfrac{1}{2\tau }\,\Vert {\tilde{u}}-u\Vert _{{\mathbb {R}}^3}^2+\dfrac{\mu }{2}\,\Vert A{\tilde{u}}-I\Vert _{{\mathbb {R}}^4}^2+i_{\widehat{{\mathcal {C}}}}({\tilde{u}})}. \end{aligned}$$

Using the identity

$$\begin{aligned}&\nu \,(a-b)^2+\mu \,(a-c)^2\\&\quad =(\nu +\mu )\,a^2-2a(\nu b+\mu c)+\nu \,b^2+\mu \,c^2\\&\quad =(\nu +\mu )\left( a-\dfrac{\nu b+\mu c}{\nu +\mu }\right) ^2\\&\qquad -\dfrac{(\nu b+\mu \,c)^2}{\nu +\mu }+\nu b^2+\mu c^2, \end{aligned}$$

and the fact that $A^TA=\begin{pmatrix}1&{}0&{}0\\ 0&{}\frac{1}{2}&{}0\\ 0&{}0&{}\frac{1}{2} \end{pmatrix}$, the minimization problem can be equivalently restated as:

$$\begin{aligned} u^{*}&=\mathop {{\mathrm{arg\,min}}}\limits _{{\tilde{u}}\in {\mathbb {R}}^3}\, \alpha \,\left( {\tilde{u}}_0-z_0\right) ^2+\beta \,\left( {\tilde{u}}_1-z_1\right) ^2\nonumber \\&\quad +\beta \,\left( {\tilde{u}}_2-z_2\right) ^2+i_{\widehat{{\mathcal {C}}}}({\tilde{u}})\nonumber \\&=\mathop {{\mathrm{arg\,min}}}\limits _{{\tilde{u}}\in {\mathbb {R}}^3}\, \left( \sqrt{\alpha }{\tilde{u}}_0-\sqrt{\alpha }z_0\right) ^2+\left( \sqrt{\beta }{\tilde{u}}_1-\sqrt{\beta }z_1\right) ^2\nonumber \\&\quad +\left( \sqrt{\beta }{\tilde{u}}_2-\sqrt{\beta }z_2\right) ^2+i_{\widehat{{\mathcal {C}}}}({\tilde{u}}), \end{aligned}$$

(C1)

with $\left\{ \begin{array}{cccc} \alpha &{}=&{}\frac{1}{2\tau }+\frac{\mu }{2}\\ \beta &{}=&{}\frac{1}{2\tau }+\frac{\mu }{4} \end{array}\right.$, and $z_0=\frac{\frac{1}{2\tau }u_0+\frac{\mu }{2}b_0}{\alpha }$, $z_1=\frac{\frac{1}{2\tau }u_1+\frac{\mu }{2}b_1}{\beta }$ and $z_2=\frac{\frac{1}{2\tau }u_2+\frac{\mu }{2}b_2}{\beta }$.

We then set $\left\{ \begin{array}{ccc}{\bar{u}}_0&{}=&{}\sqrt{\alpha }\,{\tilde{u}}_0\\ {\bar{u}}_1&{}=&{}\sqrt{\beta }\,{\tilde{u}}_1\\ {\bar{u}}_2&{}=&{}\sqrt{\beta }\,{\tilde{u}}_2 \end{array}\right.$, $\left\{ \begin{array}{ccc}{\bar{z}}_0&{}=&{}\sqrt{\alpha }\,z_0\\ {\bar{z}}_1&{}=&{}\sqrt{\beta }\,z_1\\ {\bar{z}}_2&{}=&{}\sqrt{\beta }\,z_2 \end{array}\right.$ and $\overline{{\mathcal {C}}}$ the closed convex set defined by

$$\begin{aligned} \overline{{\mathcal {C}}}=\left\{ t=(t_0,t_1,t_2) \in {\mathbb {R}}^3\,\mid \,t_0\ge \sqrt{\frac{\alpha }{\beta }}\sqrt{t_1^2+t_2^2}\right\} , \end{aligned}$$

and consider the auxiliary problem related to (C1)

$$\begin{aligned}&\mathop {{\mathrm{arg\,min}}}\limits _{{\bar{u}}=({\bar{u}}_0,{\bar{u}}_1,{\bar{u}}_2)\in {\mathbb {R}}^3}\,\Vert {\bar{u}}-{\bar{z}}\Vert _{{\mathbb {R}}^3}^2+i_{\overline{{\mathcal {C}}}}({\bar{u}}), \end{aligned}$$

(C2)

which thus amounts to computing the projection of ${\bar{z}}=({\bar{z}}_0,{\bar{z}}_1,{\bar{z}}_2)$ onto $\overline{{\mathcal {C}}}$. Once the (unique) solution of (C2) is obtained, it suffices to use the previous change of variable to recover the $u^{*}_i$’s.

Let us now recall the following result dedicated to orthogonal projection onto epigraphs.

Theorem 5

Orthogonal projection onto epigraphs, taken from [34, Chapter 6, Theorem 6.36] with ${\mathbb {E}}$ a Euclidean space, let

$$\begin{aligned} C={\text{ epi }}(g)=\left\{ (x,t)\in {\mathbb {E}}\times {\mathbb {R}}\,:\,g(x)\le t\right\} , \end{aligned}$$

where $g : {\mathbb {E}} \rightarrow {\mathbb {R}}$ is convex. Then,

$$\begin{aligned} P_C((x,s))=\left\{ \begin{array}{ccc} (x,s)&{}{\text{ if }}&{}g(x) \le s\\ ({\text{ prox }}_{\lambda ^{*}\,g}(x),s+\lambda ^{*})&{}{\text{ if }}&{}g(x) > s \end{array}\right. , \end{aligned}$$

where $\lambda ^{*}$ is any positive root of the function

$$\begin{aligned} \Psi (\lambda )=g({\text{ prox }}_{\lambda \,g}(x))-\lambda -s. \end{aligned}$$

In addition, $\Psi$ is nonincreasing.

We invoke the previous theorem with $g(\cdot )=\sqrt{\frac{\alpha }{\beta }}\,\Vert \cdot \Vert _{{\mathbb {R}}^2}$ to obtain the formula and follow the arguments of Beck ([34, Chapter 6]) (we respect the same order in the arguments of $P_{\overline{{\mathcal {C}}}}(\cdot )$ as in [34])

$$\begin{aligned}&P_{\overline{{\mathcal {C}}}}({\hat{z}}=(\bar{z_1},{\bar{z}}_2),{\bar{z}}_0)\\&\quad =\left\{ \begin{array}{ccc} (\bar{z_1},{\bar{z}}_2,{\bar{z}}_0)&{}{\text{ if }}&{}{\bar{z}}_0 \ge \sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}\\ ({\text{ prox }}_{\lambda ^{*}\,\sqrt{\frac{\alpha }{\beta }}\Vert \cdot \Vert _{{\mathbb {R}}^2}}({\hat{z}}),{\bar{z}}_0+\lambda ^{*})&{}{\text{ if }}&{}{\bar{z}}_0 <\sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2} \end{array}\right. , \end{aligned}$$

where $\lambda ^{*}$ is any positive root of the function $\Psi (\lambda )=\sqrt{\frac{\alpha }{\beta }}\,\Vert {\text{ prox }}_{\lambda \,\sqrt{\frac{\alpha }{\beta }}\Vert \cdot \Vert _{{\mathbb {R}}^2}}({\hat{z}})\Vert _{{\mathbb {R}}^2}-\lambda -{\bar{z}}_0$.

Let $({\hat{z}},\bar{z_0})$ be such that ${\bar{z}}_0 <\sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}$.

Recall that

$$\begin{aligned} {\text{ prox }}_{\lambda \,\sqrt{\frac{\alpha }{\beta }}\Vert \cdot \Vert _{{\mathbb {R}}^2}}({\hat{z}})=\left[ 1-\dfrac{\lambda \,\sqrt{\frac{\alpha }{\beta }}}{\displaystyle {\max }(\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2},\lambda \,\sqrt{\frac{\alpha }{\beta }})}\right] \,{\hat{z}}. \end{aligned}$$

Plugging the above into the expression of $\Psi$ leads to

$$\begin{aligned} \Psi (\lambda )&=\sqrt{\frac{\alpha }{\beta }}\,\left[ 1-\dfrac{\lambda \,\sqrt{\frac{\alpha }{\beta }}}{\displaystyle {\max }(\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2},\lambda \,\sqrt{\frac{\alpha }{\beta }})}\right] _{}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}-\lambda -{\bar{z}}_0\\&=\left\{ \begin{array}{ccc} \sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}-\lambda \,\frac{\alpha }{\beta }-\lambda -{\bar{z}}_0&{}{\text{ if }}&{}\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}\ge \lambda \,\sqrt{\frac{\alpha }{\beta }}\\ -\lambda -{{\bar{z}}}_0&{}{\text{ if }}&{}\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}< \lambda \,\sqrt{\frac{\alpha }{\beta }} \end{array}\right. . \end{aligned}$$

The unique positive root $\lambda ^{*}$ of the piecewise linear function $\Psi$ is

$$\begin{aligned} \lambda ^{*}=\left\{ \begin{array}{ccc} \dfrac{\sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}-{{\bar{z}}}_0}{1+\frac{\alpha }{\beta }} &{}{\text{ if }}&{}\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}\ge -{{\bar{z}}}_0\,\sqrt{\frac{\alpha }{\beta }}\\ -{{\bar{z}}}_0&{}{\text{ if }}&{}\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}< -{{\bar{z}}}_0\,\sqrt{\frac{\alpha }{\beta }} \end{array}\right. , \end{aligned}$$

resulting in

If we summarize the whole results, coming back to the initial variables, we see that we recover the results obtained with the necessary and sufficient KKT condition. Indeed,

• If ${\bar{z}}_0 \ge \sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}$ or equivalently, $z_0\ge \sqrt{z_1^2+z_2^2}$,

  $$\begin{aligned}&\left( \sqrt{\alpha }u^{*}_0,\sqrt{\beta }u^{*}_1,\sqrt{\beta }u^{*}_2 \right) =({\bar{z}}_0,{\bar{z}}_1,{\bar{z}}_2)\\&\quad =\left( \sqrt{\alpha }z_0,\sqrt{\beta }z_1,\sqrt{\beta }z_2\right) , \end{aligned}$$

  yielding

  $$\begin{aligned} \left( u^{*}_0,u^{*}_1,u^{*}_2 \right) =(z_0,z_1,z_2). \end{aligned}$$

• If ${\bar{z}}_0 <\sqrt{\frac{\alpha }{\beta }}\,\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}$ or equivalently, $z_0< \sqrt{z_1^2+z_2^2}$,

• (i) If ${\bar{z}}_0 \ge 0$ (or equivalently $z_0\ge 0$), then $\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}\ge -{\bar{z}}_0\sqrt{\frac{\alpha }{\beta }}$ or equivalently $\beta \,\sqrt{z_1^2+z_2^2}\ge -\alpha \,z_0$, and intermediate computations lead to

  

• (ii) If ${\bar{z}}_0<0$ (or equivalently, $z_0<0$) and $\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}<-{\bar{z}}_0\sqrt{\frac{\alpha }{\beta }}$ or equivalently $\beta \,\sqrt{z_1^2+z_2^2}< -\alpha \,z_0$, then

  $$\begin{aligned} \left( u^{*}_0,u^{*}_1,u^{*}_2 \right) =(0,0,0). \end{aligned}$$

• (iii) If ${\bar{z}}_0<0$ (or equivalently, $z_0<0$) and $\Vert {\hat{z}}\Vert _{{\mathbb {R}}^2}\ge -{\bar{z}}_0\sqrt{\frac{\alpha }{\beta }}$ or equivalently $\beta \,\sqrt{z_1^2+z_2^2}\ge -\alpha \,z_0$, then intermediate computations lead to