# P r o j e c t s

N o t e s

# 1. Affine and projective transformation

• For two arbitrary ideal points which are of the form (x,y,0)^t, these points are always on the line l_{infty}=(0,0,1)^t since (x_1,y_1,0)^t times(x_2,y_2,0)^t=l_{infty}.
• A point on l_{infty} is transformed to another point on  l_{infty} by affine transformation. But it is transformed to a point on the vanishing line by projective transformation.

# 2. How to remove the projective distortion

• Projective transformation causes the distortion such that parallel lines in affine space are not parallel any more because a vanishing line comes about.
• Let the vanishing line be l=(l_1,l_2,l_3)^t, then l is mapped to l_infty by a matrix H as follows:
H=
\begin{pmatrix}
1 & 0 & 0 \\
0 & 1 & 1 \\
l_1 & l_2 & l_3
\end{pmatrix}

• While a point is transformed by H, a normal vector is transformed by H^{-t}. So H^{-t}l=l_infty.
H^{-t}l=
\begin{pmatrix}
1 & 0 & -\frac{l_1}{l_3} \\
0 & 1 & -\frac{l_2}{l_3} \\
0 & 0 & \frac{1}{l_3}
\end{pmatrix}
\begin{pmatrix} l_1 \\ l_2 \\ l_3 \end{pmatrix} =
\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}
= l_\infty

• Therefore, H^{-t} transforms any line l into l_infty, which means that it restores the projective distortion. But it still remains the affine distortion.

# 3. The angle between two lines

• For two lines l and m, the angle theta between them is the following:
\begin{align*}
\cos\theta&=\frac{l_1m_1+l_2m_2}{\sqrt{(l_1^2+l_2^2)(m_1^2+m_2^2 )}}=
\frac{l^tC_{\infty}^*m}{\sqrt{( l^tC_{\infty}^*l)(m^tC_{\infty}^*m)}},\\ \\
C_{\infty}^*&=
\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix}
\text{: dual degenerate conic}
\end{align*}

# 4. How to remove the affine distortion

• Even though the projective distortion is removed, there is still the affine distortion, which a square shape turns into a parallelogram shape. In other words, the angle between lines is not preserved.
• Assume that two lines l, m and dual degenerate conic C_infty^ast in the original space are transformed to l^{prime}, m^{prime} and C_infty^{prime ast} by  affine transformation. Then lprime=H^{-t}l, mprime=H^{-t}m, and   Cprime_infty^ast =H C_infty^ast H^t.
• For the angle theta between l and m, the numerator of costheta is l^tC_infty^astm=(l ^{primet}H)(H^{-1} C_infty^{prime ast} H^{-t})(H^t m^{prime})=l ^{primet} C_infty^{prime ast} m^{prime}.
• When l and m are perpendicular,  l^{primet}C_infty^{prime ast}m^{prime}=0. It yields as follows:
\begin{align}
l^{\prime t}C_{\infty}^{\prime \ast}m^{\prime} =
l^{\prime t}HC_{\infty}^*H^tm^{\prime} =
\begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3 ^{\prime}  \end{pmatrix} &
\begin{pmatrix} A & \vec{t} \\ \vec{0} & 1 \end{pmatrix}
\begin{pmatrix} I & \vec{0} \\ \vec{0} & 0 \end{pmatrix}
\begin{pmatrix} A & \vec{0} \\ \vec{t} & 1 \end{pmatrix}
\begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} =0, \\
\begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3^{\prime} \end{pmatrix}
\begin{pmatrix} AA^t & \vec{0} \\ \vec{0} & 0 \end{pmatrix}
\begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} &=
\begin{pmatrix} l_1^{\prime} & l_2^{\prime} \end{pmatrix}
\begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix}
\begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \end{pmatrix} =0 \\
\text{where}
\begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix} =S=AA^t
\end{align}
• S is symmetric, s_{12}=s_{21}, and s_{22} can be set to 1 since S is defined up to scale.
• s_{11}l_1^{prime}m_1^{prime}+s_{12}(l_1^{prime}m_2 ^{prime} + l_2 ^{prime} m_1 ^{prime} )+l_2 ^{prime} m_2^{prime}=0
• If we get two pairs of l and m which are perpendicular, then S can be calculated.
• Having calculated S, H can be estimated assuming A is positive-definite. From S=A A^t, the eigenvalues of A are the positive sqaure roots of the eigenvalues of S.
• After removing the affine distortion, there still remains the similarity distortion.

# 5. Hierarchy of transformation in 3D is similar to that in 2D

• As a  point on l_{infty} is transformed to another point on l_{infty} by affine transformation, a point on the plane Pi_infty=(0,0,0,1)^t is also transformed to another point on Pi_infty.
• As a point on l_{infty} is transformed to a point on the vanishing line by projective transformation, a point on Pi_infty=(0,0,0,1)^t is transformed to a point on the specific plane.
• When a cube is transformed in projective space, each a pair of parallel lines is not parallel any more and intersects at a point which is not at infinity such as v_1, v_2, and v_3.
• These points v_1, v_2 and v_3 consist of a plane Pi which does correspond to Pi_infty in affine space.
• If we know this plane Pi, it can be transformed to Pi_infty by some matrix H, so the projective distortion can be removed.
• Let Pi=(pi_1, pi_2, pi_3, pi_4)^t=(bar{pi}^t, pi_4)^t, then the following H restores Pi to Pi_infty:
\begin{align} H=
\begin{pmatrix} A &  \vec{0} \\ \bar{\pi^t} & \pi_4 \end{pmatrix}, &\quad H^{-1}=
\begin{pmatrix} A^{-1} &  \vec{0} \\ -\frac{1}{\pi_4}\bar{\pi^t}A^{-1} & \frac{1}{\pi_4} \end{pmatrix},
\begin{pmatrix} A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\  \vec{0} & \frac{1}{\pi_4} \end{pmatrix} \\ \\
H^{-t}\Pi &=
\begin{pmatrix}  A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\ \vec{0} & \frac{1}{\pi_4} \end{pmatrix}
\begin{pmatrix} \bar{\pi} \\ \pi_4 \end{pmatrix} =
\begin{pmatrix} \vec{0} \\ 1 \end{pmatrix} = \Pi_{\infty}
\end{align}
• bar{pi} is the orientation of Pi and Pi is a homogeneous representation, so Pi~frac{1}{pi_4}Pi.

• Reference

[1] Hartley, R. and Zisserman, A. (2003) Multiple View Geometry in Computer Vision. 2nd Edition, Cambridge University Press, Cambridge.

emoy.net