A b o u t   M e      |       P r o j e c t s     |       N o t e s       |       T h e   D a y    ︎ ︎

## Finding Points in 3D Camera Coordinates from Points on Image Plane

When a camera is viewing an object on the flat ground, the bottom point P of the object is projected on the camera image plane. The camera coordinate system here is right handed whose view vector is z-axis and x-axis points outwards from the screen. Let phiin[0, frac{pi}{2}] be the tilt angle of the camera, then the view vector is parallel with the ground when phi=0 and intersects with a point on the ground when phine0. Lastly, H is the camera height in meter.
1.  The ground plane equation

If y-axis of the camera is rotated by theta around its x-axis, its y-axis will be parallel with the normal vector n of the ground. Therefore, -n is

\begin{pmatrix}1 & 0 & 0 \\ 0 & \cos\phi & -\sin\phi \\ 0 & \sin\phi & \cos\phi \end{pmatrix}
\begin{pmatrix}0 \\ 1 \\ 0 \end{pmatrix}=
\begin{pmatrix}0 \\ \cos\phi \\ \sin\phi \end{pmatrix} =-n,

so n is (0, -costheta, -sintheta)^t. It implies that the ground plane is the plane whose normal vector is n and distance from the camera origin is H. Therefore, the ground plane equation is H-ycosphi-zsinphi=0.
2. Finding P

P(x, y, z) is the object point which is on the ground plane, so this point is satisfied with the ground plane equation.  Let Pprime(u, v, 1) be the point on the image plane and K is the intrinsic matrix of the camera, then Pprime can be written by
$$KP= \begin{pmatrix} f & 0 & \frac{w}{2} \\ 0 & f & \frac{h}{2} \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix}= \begin{pmatrix} fx + \frac{wz}{2} \\ fy + \frac{yz}{2} \\ z \end{pmatrix} \cong \begin{pmatrix} f\frac{x}{z}+\frac{w}{2} \\ f\frac{y}{z}+\frac{h}{2} \\ 1 \end{pmatrix}=\begin{pmatrix} u \\ v \\ 1 \end{pmatrix}$$
where w and h are the width and height of the camera. It means P((u-frac{w}{2})frac{z}{f}, (v-frac{h}{2})frac{z}{f}, z), which z component of P is z=frac{H-ycosphi}{sinphi}(phine0) from the ground plane equation. Consequently, P is as follows.
\begin{align} y&=(v-\frac{h}{2})\frac{1}{f}\frac{H-y\cos\phi}{\sin\phi}\Rightarrow y= \bbox[border:2px solid red]{ \frac{(v-\frac{h}{2})H}{f\sin\phi+(v-\frac{h}{2})\cos\phi}}\\ \\x&=(u-\frac{w}{2})\frac{1}{f}\frac{H-y\cos\phi}{\sin\phi}\\ &=(u-\frac{w}{2})\frac{1}{f\sin\phi}\left\{H-\frac{(v-\frac{h}{2})H\cos\phi}{f\sin\phi+(v-\frac{h}{2}) \cos\phi}\right\}\\&= \bbox[border:2px solid red]{ \frac{(u-\frac{w}{2})H}{f\sin\phi+(v-\frac{h}{2})\cos\phi}}\\ \\ \\ z&=\frac{1}{\sin\phi}\left\{H-\frac{(v-\frac{h}{2})H\cos\phi}{f\sin\phi+(v-\frac{h}{2}) \cos\phi}\right\}=\bbox[border:2px solid red]{\frac{fH}{f\sin\phi+(v-\frac{h} {2})\cos\phi}} \end{align}
• What if phi=0?

When phi=0,  z=frac{H-ycosphi}{sinphi} is not defined because its denominator is zero. Instead, y=H from the ground plane equation in this case. It also implies that (v-frac{h}{2})frac{z}{f}=H and z=frac{fH}{v-frac{h}{2}}. Moreover, x=frac{(u-frac{w}{2})H}{v-frac{h}{2}} from  (u-frac{w}{2})frac{z}{f}. Note that this result is the same with P as above whose phi=0.

• What if phi=0 and v=frac{h}{2}?

When phi=0 and v=frac{h}{2}, P is not defined since its all components go to infinity. But it makes sense because if the tilt angle is zero, the view vector is parallel with the ground and there is no intersection between them, or they meet at infinity.
3. Finding a point over P

Until now, P is always on the ground plane. But objects are not always on the ground and they can be over the ground. In this case, it is required that the height of the object from the ground be known. Let r be the height of the object, then it is possible to imagine that the ground plane is going up by r. It means that this point is on the new ground plane again whose distance from the camera origin is H-r, not H. Therefore, P which is over the ground can be modified as follows:
\begin{align} x&= \frac{(u-\frac{w}{2})(H-r)}{f\sin\phi+(v-\frac{h}{2})\cos\phi} \\ y&=\frac{(v-\frac{h}{2})(H-r)}{f\sin\phi+(v-\frac{h}{2})\cos\phi} \\z&=\frac{f(H-r)}{f\sin\phi+(v-\frac{h}{2})\cos\phi}\end{align}
4. Accuracy on the image plane

The adjacent pixel points projected on the image plane from the ground may not close enough on the ground. In other words, the points that are close on the image plane can be far from each other in the camera coordinates. (u_1, v_1, 1)^t and (u_2, v_2, 1)^t are adjacent as above, and let z_1 and z_2 be z components of the 3D points on the ground which are projected on them.
\begin{align}z_1&=\frac{fH}{f\sin\phi+(v_1-\frac{h}{2})\cos\phi} \Rightarrow v_1=f(\frac{H}{z_1\cos\phi}-\tan\phi)+\frac{h}{2} \\ z_2&=\frac{fH}{f\sin\phi+(v_2-\frac{h}{2})\cos\phi} \Rightarrow v_2= f(\frac{H}{z_2\cos\phi}-\tan\phi)+\frac{h}{2} \end{align}
Now that v_1 and v_2 are adjacent, v_2-v_1=1.
\begin{align} v_2-v_1=\frac{fH}{\cos\phi}(\frac{1}{z_2}-\frac{1}{z_1})=1  \Rightarrow \frac{1}{z_2}-\frac{1}{z_1}= \frac{\cos\phi} {fH} \end{align}
This result implies that the larger phi, f, and H are, the smaller |z_1-z_2| is, the less the distance between two 3D points from (u_1, v_1, 1)^t and (u_2, v_2, 1)^t.

emoy.net