Hierarchy of Transformation
1. Affine and projective transformation
- For two arbitrary ideal points which are of the form `(x,y,0)^t`, these points are always on the line `l_{infty}=(0,0,1)^t` since `(x_1,y_1,0)^t times(x_2,y_2,0)^t=l_{infty}`.
- A point on `l_{infty}` is transformed to another point on `l_{infty}` by affine transformation. But it is transformed to a point on the vanishing line by projective transformation.

2. How to remove the projective distortion
- Projective transformation causes the distortion such that parallel lines in affine space are not parallel any more because a vanishing line comes about.
- Let the vanishing line be `l=(l_1,l_2,l_3)^t`, then `l` is mapped to `l_infty` by a matrix `H` as follows:
- While a point is transformed by `H`, a normal vector is transformed by `H^{-t}`. So `H^{-t}l=l_infty`.
- Therefore, `H^{-t}` transforms any line `l` into `l_infty`, which means that it restores the projective distortion. But it still remains the affine distortion.
3. The angle between two lines
- For two lines `l` and `m`, the angle `theta` between them is the following:

4. How to remove the affine distortion
- Even though the projective distortion is removed, there is still the affine distortion, which a square shape turns into a parallelogram shape. In other words, the angle between lines is not preserved.
- Assume that two lines `l`, `m` and dual degenerate conic `C_infty^ast` in the original space are transformed to `l^{prime}`, `m^{prime}` and `C_infty^{prime ast}` by affine transformation. Then `lprime=H^{-t}l`, `mprime=H^{-t}m`, and ` Cprime_infty^ast =H C_infty^ast H^t`.
- For the angle `theta` between `l` and `m`, the numerator of `costheta` is `l^tC_infty^astm=``(l ^{primet}H)(H^{-1} C_infty^{prime ast} H^{-t})(H^t m^{prime})=``l ^{primet} C_infty^{prime ast} m^{prime}`.
- When `l` and `m` are perpendicular, `l^{primet}C_infty^{prime ast}m^{prime}=0`. It yields as follows:
- `S` is symmetric, `s_{12}=s_{21}`, and `s_{22}` can be set to `1` since `S` is defined up to scale.
- `s_{11}l_1^{prime}m_1^{prime}+s_{12}(l_1^{prime}m_2 ^{prime} + l_2 ^{prime} m_1 ^{prime} )+l_2 ^{prime} m_2^{prime}=0`
- If we get two pairs of `l` and `m` which are perpendicular, then `S` can be calculated.
- Having calculated `S`, `H` can be estimated assuming `A` is positive-definite. From `S=A A^t`, the eigenvalues of `A` are the positive sqaure roots of the eigenvalues of `S`.
- After removing the affine distortion, there still remains the similarity distortion.
5. Hierarchy of transformation in 3D is similar to that in 2D
- As a point on `l_{infty}` is transformed to another point on `l_{infty}` by affine transformation, a point on the plane `Pi_infty=(0,0,0,1)^t` is also transformed to another point on `Pi_infty`.
- As a point on `l_{infty}` is transformed to a point on the vanishing line by projective transformation, a point on `Pi_infty=(0,0,0,1)^t` is transformed to a point on the specific plane.

- When a cube is transformed in projective space, each a pair of parallel lines is not parallel any more and intersects at a point which is not at infinity such as `v_1`, `v_2`, and `v_3`.
- These points `v_1`, `v_2` and `v_3` consist of a plane `Pi` which does correspond to `Pi_infty` in affine space.
- If we know this plane `Pi`, it can be transformed to `Pi_infty` by some matrix `H`, so the projective distortion can be removed.
- Let `Pi=(pi_1, pi_2, pi_3, pi_4)^t=(bar{pi}^t, pi_4)^t`, then the following `H` restores `Pi` to `Pi_infty`:
- `bar{pi}` is the orientation of `Pi` and `Pi` is a homogeneous representation, so `Pi~frac{1}{pi_4}Pi`.

- Reference
emoy.net