Rectifying Homographies for Stereo Vision: Analytical Solution for Minimal Distortion

Abstract

Stereo rectification is the determination of two image transformations (or homographies) that map corresponding points on the two images, projections of the same point in the 3D space, onto the same horizontal line in the transformed images. Rectification is used to simplify the subsequent stereo correspondence problem and speeding up the matching process. Rectifying transformations, in general, introduce perspective distortion on the obtained images, which shall be minimised to improve the accuracy of the following algorithm dealing with the stereo correspondence problem. The search for the optimal transformations is usually carried out relying on numerical optimisation. This work proposes a closed-form solution for the rectifying homographies that minimise perspective distortion. The experimental comparison confirms its capability to solve the convergence issues of the previous formulation. Its Python implementation is provided.

Appendix A

The coefficients of the $4^{th}$ order polynomial expression in $y_1$ of Eq. (38) are:

$$\begin{aligned} \begin{aligned} a&= m_2 m_4+m_6m_8\\ b&= m_1 m_4+ 3 m_2 m_3 m_4+ m_5 m_8+3 m_6 m_7 m_8\\ c&= 3 m_1 m_3 m_4+3 m_2 m_3^2 m_4+3 m_5 m_7 m_8+3 m_6 m_7^2 m_8\\ d&= 3 m_1 m_3^2 m_4+ m_2 m_3^3 m_4+3 m_5 m_7^2 m_8+ m_6 m_7^3 m_8\\ e&= m_1 m_3^3 m_4+ m_5 m_7^3 m_8 \end{aligned} \end{aligned}$$

(40)

with:

$$\begin{aligned} \begin{aligned} m_1&= \left[ \mathbf {M}_1\right] _{2 3} \left[ \mathbf {C}_1\right] _{2 3}-\left[ \mathbf {M}_1\right] _{3 3} \left[ \mathbf {C}_1\right] _{2 2}\\[2pt] m_2&= \left[ \mathbf {M}_1\right] _{2 2} \left[ \mathbf {C}_1\right] _{2 3}-\left[ \mathbf {M}_1\right] _{2 3} \left[ \mathbf {C}_1\right] _{2 2}\\[2pt] m_3&= \frac{\left[ \mathbf {C}_2\right] _{2 3}}{\left[ \mathbf {C}_2\right] _{2 2}}\\[2pt] m_4&= \frac{\left[ \mathbf {C}_2\right] _{2 2}}{\left[ \mathbf {C}_1\right] _{2 2}}\\[2pt] m_5&= \left[ \mathbf {M}_2\right] _{2 3} \left[ \mathbf {C}_2\right] _{2 3}-\left[ \mathbf {M}_2\right] _{3 3} \left[ \mathbf {C}_2\right] _{2 2}\\[2pt] m_6&= \left[ \mathbf {M}_2\right] _{2 2} \left[ \mathbf {C}_2\right] _{2 3}-\left[ \mathbf {M}_2\right] _{2 3} \left[ \mathbf {C}_2\right] _{2 2}\\[2pt] m_7&= \frac{\left[ \mathbf {C}_1\right] _{2 3}}{\left[ \mathbf {C}_1\right] _{2 2}} \\[2pt] m_8&= \frac{\left[ \mathbf {C}_1\right] _{2 2}}{\left[ \mathbf {C}_2\right] _{2 2}} \end{aligned} \end{aligned}$$

(41)

The four roots of the equation are given by:

$$\begin{aligned} y_1=\frac{-b}{4a}\pm Q\pm \frac{1}{2}\sqrt{-4 Q^2-2p+\frac{S}{Q}} \end{aligned}$$

(42)

with:

$$\begin{aligned} \begin{aligned} Q&= \frac{1}{2}\sqrt{-\frac{2}{3}p+\frac{1}{3a}\left( \varDelta _0+\frac{q}{\varDelta _0}\right) }\\ S&= \frac{8 a^2d-4 a b c +b^3}{8 a^3}\\ \varDelta _0&= \left( \frac{s+\sqrt{s^2-4q^3}}{2}\right) ^{\frac{1}{3}}\\ p&= \frac{8 a c -3 b^2}{8 a^2} \\ q&= 12 a e -3 b d +c^2\\ s&= 27 a d^2-72 a c e + 27 b^2 e -9 b c d +2 c^3 \end{aligned} \end{aligned}$$

(43)

Remark: for the case $\mathbf {A}_1 = \mathbf {A}_2$, $\mathbf {P}_1 = \mathbf {P}_2$, $\mathbf {P}_{c1} = \mathbf {P}_{c2}$ and $\mathbf {R}_1 = \mathbf {R}_2$, the solution is given by:

$$\begin{aligned} y_1=-\frac{m_1}{m_2} \end{aligned}$$

(44)