A b o u t   M e      |       P r o j e c t s     |       N o t e s       |       T h e   D a y    ︎ ︎

Hierarchy of Transformation

1. Affine and projective transformation

  • For two arbitrary ideal points which are of the form `(x,y,0)^t`, these points are always on the line `l_{infty}=(0,0,1)^t` since `(x_1,y_1,0)^t times(x_2,y_2,0)^t=l_{infty}`. 
  • A point on `l_{infty}` is transformed to another point on  `l_{infty}` by affine transformation. But it is transformed to a point on the vanishing line by projective transformation. 

2. How to remove the projective distortion

  • Projective transformation causes the distortion such that parallel lines in affine space are not parallel any more because a vanishing line comes about.
  • Let the vanishing line be `l=(l_1,l_2,l_3)^t`, then `l` is mapped to `l_infty` by a matrix `H` as follows:
\begin{equation} H= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ l_1 & l_2 & l_3 \end{pmatrix}\end{equation}
  • While a point is transformed by `H`, a normal vector is transformed by `H^{-t}`. So `H^{-t}l=l_infty`.
\begin{equation} H^{-t}l= \begin{pmatrix}1 & 0 & -\frac{l_1}{l_3} \\0 & 1 & -\frac{l_2}{l_3} \\0 & 0 & \frac{1}{l_3} \end{pmatrix} \begin{pmatrix} l_1 \\ l_2 \\ l_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} = l_\infty \end{equation}
  • Therefore, `H^{-t}` transforms any line `l` into `l_infty`, which means that it restores the projective distortion. But it still remains the affine distortion.

3. The angle between two lines

  • For two lines `l` and `m`, the angle `theta` between them is the following:
\begin{align*} \cos\theta&=\frac{l_1m_1+l_2m_2}{\sqrt{(l_1^2+l_2^2)(m_1^2+m_2^2 )}}= \frac{l^tC_{\infty}^*m}{\sqrt{( l^tC_{\infty}^*l)(m^tC_{\infty}^*m)}},\\ \\ C_{\infty}^*&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix} \text{: dual degenerate conic} \end{align*}

4. How to remove the affine distortion

  • Even though the projective distortion is removed, there is still the affine distortion, which a square shape turns into a parallelogram shape. In other words, the angle between lines is not preserved.
  • Assume that two lines `l`, `m` and dual degenerate conic `C_infty^ast` in the original space are transformed to `l^{prime}`, `m^{prime}` and `C_infty^{prime ast}` by  affine transformation. Then `lprime=H^{-t}l`, `mprime=H^{-t}m`, and  ` Cprime_infty^ast =H C_infty^ast H^t`.
  • For the angle `theta` between `l` and `m`, the numerator of `costheta` is `l^tC_infty^astm=``(l ^{primet}H)(H^{-1} C_infty^{prime ast} H^{-t})(H^t m^{prime})=``l ^{primet} C_infty^{prime ast} m^{prime}`.
  • When `l` and `m` are perpendicular,  `l^{primet}C_infty^{prime ast}m^{prime}=0`. It yields as follows:
\begin{align} l^{\prime t}C_{\infty}^{\prime \ast}m^{\prime} = l^{\prime t}HC_{\infty}^*H^tm^{\prime} = \begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3 ^{\prime}  \end{pmatrix} & \begin{pmatrix} A & \vec{t} \\ \vec{0} & 1 \end{pmatrix} \begin{pmatrix} I & \vec{0} \\ \vec{0} & 0 \end{pmatrix} \begin{pmatrix} A & \vec{0} \\ \vec{t} & 1 \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} =0, \\ \begin{pmatrix} l_1^{\prime} & l_2^{\prime} & l_3^{\prime} \end{pmatrix} \begin{pmatrix} AA^t & \vec{0} \\ \vec{0} & 0 \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \\ m_3^{\prime} \end{pmatrix} &= \begin{pmatrix} l_1^{\prime} & l_2^{\prime} \end{pmatrix} \begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix} \begin{pmatrix} m_1^{\prime} \\ m_2^{\prime} \end{pmatrix} =0 \\ \text{where}  \begin{pmatrix} s_{11} & s_{12} \\ s_{21} & s_{22} \end{pmatrix} =S=AA^t \end{align}
  • `S` is symmetric, `s_{12}=s_{21}`, and `s_{22}` can be set to `1` since `S` is defined up to scale.
  • `s_{11}l_1^{prime}m_1^{prime}+s_{12}(l_1^{prime}m_2 ^{prime} + l_2 ^{prime} m_1 ^{prime} )+l_2 ^{prime} m_2^{prime}=0`
  • If we get two pairs of `l` and `m` which are perpendicular, then `S` can be calculated.
  • Having calculated `S`, `H` can be estimated assuming `A` is positive-definite. From `S=A A^t`, the eigenvalues of `A` are the positive sqaure roots of the eigenvalues of `S`.  
  • After removing the affine distortion, there still remains the similarity distortion.

5. Hierarchy of transformation in 3D is similar to that in 2D

  • As a  point on `l_{infty}` is transformed to another point on `l_{infty}` by affine transformation, a point on the plane `Pi_infty=(0,0,0,1)^t` is also transformed to another point on `Pi_infty`. 
  • As a point on `l_{infty}` is transformed to a point on the vanishing line by projective transformation, a point on `Pi_infty=(0,0,0,1)^t` is transformed to a point on the specific plane.
  • When a cube is transformed in projective space, each a pair of parallel lines is not parallel any more and intersects at a point which is not at infinity such as `v_1`, `v_2`, and `v_3`.
  • These points `v_1`, `v_2` and `v_3` consist of a plane `Pi` which does correspond to `Pi_infty` in affine space. 
  • If we know this plane `Pi`, it can be transformed to `Pi_infty` by some matrix `H`, so the projective distortion can be removed.
  • Let `Pi=(pi_1, pi_2, pi_3, pi_4)^t=(bar{pi}^t, pi_4)^t`, then the following `H` restores `Pi` to `Pi_infty`:
\begin{align} H= \begin{pmatrix} A &  \vec{0} \\ \bar{\pi^t} & \pi_4 \end{pmatrix}, &\quad H^{-1}= \begin{pmatrix} A^{-1} &  \vec{0} \\ -\frac{1}{\pi_4}\bar{\pi^t}A^{-1} & \frac{1}{\pi_4} \end{pmatrix}, \quad H^{-t}= \begin{pmatrix} A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\  \vec{0} & \frac{1}{\pi_4} \end{pmatrix} \\ \\ H^{-t}\Pi &= \begin{pmatrix}  A^{-t} & -\frac{1}{\pi_4}A^{-t}\bar{\pi} \\ \vec{0} & \frac{1}{\pi_4} \end{pmatrix} \begin{pmatrix} \bar{\pi} \\ \pi_4 \end{pmatrix} = \begin{pmatrix} \vec{0} \\ 1 \end{pmatrix} = \Pi_{\infty} \end{align}
  • `bar{pi}` is the orientation of `Pi` and `Pi` is a homogeneous representation, so `Pi~frac{1}{pi_4}Pi`. 

  • Reference

[1] Hartley, R. and Zisserman, A. (2003) Multiple View Geometry in Computer Vision. 2nd Edition, Cambridge University Press, Cambridge.