Pinhole Camera Models

Coordinate Systems

World Coordinates

The coordinate system of the world where the objects/scene live.

Camera Coordinates

The coordinate system with respect to the camera.\
The origin is the optical center or the focal point.
Principal axis is the axis orthogonal to the image plane, which aligns with the $z$ -axis.
The $y$ -axis points downwards.

Image Coordinates

The 2D coordinate system with respect to the image plane.
The $z$ -axis of the camera coordinates intersects with the image plane at the principal point.
There is usually an offset from the origin to the principal point.
The $x$ - and $y$ -axis points to the same direction as the camera coordinates.

Perspective Projection

The perspective projection transforms from camera coordinates to image coordinates. Let the image coordinates be $x_{s}$ , and the camera coordinates be $x_{c}$ , we have

(\begin{matrix} x_{s} \\ y_{s} \end{matrix}) = (\begin{matrix} f_{x} x_{c} / z_{c} \\ f_{y} y_{c} / z_{c} \end{matrix}) .

Axis Skew

Axis skew causes shear distortion of the projected image. The skew can be defined as the shift on the $x$ -axis, which is proportional to the $y$ -axis, parameterized by the factor $s$ :

x_{s} = f_{x} \frac{x_{c}}{z_{c}} + s \frac{y_{c}}{z_{c}} .

Principal Point Offset

The origin of the image coordinates may deviate from the principal point. We parameterize this with an offset on the $x$ - and $y$ -axis:

\begin{aligned} x_{s} & = f_{x} \frac{x_{c}}{z_{c}} + s \frac{y_{c}}{z_{c}} + c_{x} \\ y_{s} & = f_{y} \frac{y_{c}}{z_{c}} + c_{y} . \end{aligned}

Perspective Projection in Matrix Representation

Putting the above together, if we write $x_{s} \in R^{2}$ in homogeneous coordinates ${\tilde{x}}_{s} \in R^{3}$ , we have

{\tilde{x}}_{s} = [\begin{matrix} f_{x} & s & c_{x} & 0 \\ 0 & f_{y} & c_{y} & 0 \\ 0 & 0 & 1 & 0 \end{matrix}] {\bar{x}}_{c},

where ${\bar{x}}_{c}$ is $x_{c}$ written in homogeneous coordinates.

Chaining Transformations

The transformation from the world coordinates to the camera coordinates can be parameterized as a rotation matrix $R \in S O (3) \subset R^{3 \times 3}$ and a translation vector $t \in R^{3}$ . We call $R$ and $t$ the extrinsic parameters. The above projection matrix can be written as $[\begin{matrix} K & 0 \end{matrix}]$ , where $K \in R^{3 \times 3}$ is the intrinsic parameters. Therefore

{\tilde{x}}_{s} = [\begin{matrix} K & 0 \end{matrix}] [\begin{matrix} R & t \\ 0^{T} & 1 \end{matrix}] {\bar{x}}_{w} = P {\bar{x}}_{w} .