A b o u t   M e      |       P r o j e c t s     |       N o t e s       |       T h e   D a y    ︎ ︎

# 1. Affine and projective transformation

• For two arbitrary ideal points which are of the form (x,y,0)^t, these points are always on the line l_{infty}=(0,0,1)^t since (x_1,y_1,0)^t times(x_2,y_2,0)^t=l_{infty}.
• A point on l_{infty} is transformed to another point on  l_{infty} by affine transformation. But it is transformed to a point on the vanishing line by projective transformation.

# 2. How to remove the projective distortion

• Projective transformation causes the distortion such that parallel lines in affine space are not parallel any more because a vanishing line comes about.
• Let the vanishing line be l=(l_1,l_2,l_3)^t, then l is mapped to l_infty by a matrix H as follows:
\begin{equation} H= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ l_1 & l_2 & l_3 \end{pmatrix}\end{equation}
• While a point is transformed by H, a normal vector is transformed by H^{-t}. So H^{-t}l=l_infty.
\begin{equation} H^{-t}l= \begin{pmatrix}1 & 0 & -\frac{l_1}{l_3} \\0 & 1 & -\frac{l_2}{l_3} \\0 & 0 & \frac{1}{l_3} \end{pmatrix} \begin{pmatrix} l_1 \\ l_2 \\ l_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} = l_\infty \end{equation}
• Therefore, H^{-t} transforms any line l into l_infty, which means that it restores the projective distortion. But it still remains the affine distortion.

# 3. The angle between two lines

• For two lines l and m, the angle theta between them is the following:
\begin{align*} \cos\theta&=\frac{l_1m_1+l_2m_2}{\sqrt{(l_1^2+l_2^2)(m_1^2+m_2^2 )}}= \frac{l^tC_{\infty}^*m}{\sqrt{( l^tC_{\infty}^*l)(m^tC_{\infty}^*m)}},\\ \\ C_{\infty}^*&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix} \text{: dual degenerate conic} \end{align*}

# 4. How to remove the affine distortion

• Even though the projective distortion is removed, there is still the affine distortion, which a square shape turns into a parallelogram shape. In other words, the angle between lines is not preserved.
• Assume that two lines l, m and dual degenerate conic C_infty^ast in the original space are transformed to l^{prime}, m^{prime} and C_infty^{prime ast} by  affine transformation. Then lprime=H^{-t}l, mprime=H^{-t}m, and   Cprime_infty^ast =H C_infty^ast H^t.
• For the angle theta between l and m, the numerator of costheta is l^tC_infty^astm=(l ^{primet}H)(H^{-1} C_infty^{prime ast} H^{-t})(H^t m^{prime})=l ^{primet} C_infty^{prime ast} m^{prime}.
• When l and m are perpendicular,  l^{primet}C_infty^{prime ast}m^{prime}=0. It yields as follows:
\begin{align} l^{\prime t}C_{\infty}^{\prime \ast}m^{\prime} = l^{\prime t}HC_{\infty}^*H^tm^{\prime} = \begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3 ^{\prime}  \end{pmatrix} & \begin{pmatrix} A & \vec{t} \\ \vec{0} & 1 \end{pmatrix} \begin{pmatrix} I & \vec{0} \\ \vec{0} & 0 \end{pmatrix} \begin{pmatrix} A & \vec{0} \\ \vec{t} & 1 \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} =0, \\ \begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3^{\prime} \end{pmatrix} \begin{pmatrix} AA^t & \vec{0} \\ \vec{0} & 0 \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} &= \begin{pmatrix} l_1^{\prime} & l_2^{\prime} \end{pmatrix} \begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \end{pmatrix} =0 \\ \text{where}  \begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix} =S=AA^t \end{align}
• S is symmetric, s_{12}=s_{21}, and s_{22} can be set to 1 since S is defined up to scale.
• s_{11}l_1^{prime}m_1^{prime}+s_{12}(l_1^{prime}m_2 ^{prime} + l_2 ^{prime} m_1 ^{prime} )+l_2 ^{prime} m_2^{prime}=0
• If we get two pairs of l and m which are perpendicular, then S can be calculated.
• Having calculated S, H can be estimated assuming A is positive-definite. From S=A A^t, the eigenvalues of A are the positive sqaure roots of the eigenvalues of S.
• After removing the affine distortion, there still remains the similarity distortion.

# 5. Hierarchy of transformation in 3D is similar to that in 2D

• As a  point on l_{infty} is transformed to another point on l_{infty} by affine transformation, a point on the plane Pi_infty=(0,0,0,1)^t is also transformed to another point on Pi_infty.
• As a point on l_{infty} is transformed to a point on the vanishing line by projective transformation, a point on Pi_infty=(0,0,0,1)^t is transformed to a point on the specific plane.
• When a cube is transformed in projective space, each a pair of parallel lines is not parallel any more and intersects at a point which is not at infinity such as v_1, v_2, and v_3.
• These points v_1, v_2 and v_3 consist of a plane Pi which does correspond to Pi_infty in affine space.
• If we know this plane Pi, it can be transformed to Pi_infty by some matrix H, so the projective distortion can be removed.
• Let Pi=(pi_1, pi_2, pi_3, pi_4)^t=(bar{pi}^t, pi_4)^t, then the following H restores Pi to Pi_infty:
\begin{align} H= \begin{pmatrix} A &  \vec{0} \\ \bar{\pi^t} & \pi_4 \end{pmatrix}, &\quad H^{-1}= \begin{pmatrix} A^{-1} &  \vec{0} \\ -\frac{1}{\pi_4}\bar{\pi^t}A^{-1} & \frac{1}{\pi_4} \end{pmatrix}, \quad H^{-t}= \begin{pmatrix} A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\  \vec{0} & \frac{1}{\pi_4} \end{pmatrix} \\ \\ H^{-t}\Pi &= \begin{pmatrix}  A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\ \vec{0} & \frac{1}{\pi_4} \end{pmatrix} \begin{pmatrix} \bar{\pi} \\ \pi_4 \end{pmatrix} = \begin{pmatrix} \vec{0} \\ 1 \end{pmatrix} = \Pi_{\infty} \end{align}
• bar{pi} is the orientation of Pi and Pi is a homogeneous representation, so Pi~frac{1}{pi_4}Pi.

• Reference

 Hartley, R. and Zisserman, A. (2003) Multiple View Geometry in Computer Vision. 2nd Edition, Cambridge University Press, Cambridge.

emoy.net