Understanding Projecting Modes

From Autopano

INTRODUCTION

Before we can effectively describe the various projection types, let’s go back to the notion of image assembling (i.e. image stitching).

This example illustrates the notion of image stitching in standard mode (i.e. when all the source images were taken by rotating the camera around its nodal point).
Each source image will receive a set of spherical coordinates (Yaw, Pitch and Roll).
When projected on the base sphere, it must exactly match the surrounding images.
Stitching a panorama is basically finding the location of each source image on a base sphere.

The resulting texture on the sphere’s surface is the stitched image.
The resulting image can either cover the totality of the sphere’s surface in the case of a 360° x 180° panorama, or only a fraction of it.

Note :
We are assuming here that the stitching being performed is based on a model where the camera rotates around its nodal point as illustrated in the above image.
Other stitching models are possible and will soon be supported by Autopano Pro. The most popular is the Orthogonal, also called Orthographic stitching.
All the source images are shot orthogonally to the same plane. Instead of rotating around its center, the camera is following a linear motion path, always pointing in the same direction.
This is exactly what is being done when scanning a A3 size sheet of paper with a letter size scanner. We end up with 4 or 5 files assembled using this stitching model.
It is also what happens when taking pictures of building fronts in a street by walking down the street and taking a picture facing the buildings every 10 steps.

CONCEPT

The projection modes are referring to what’s being done with the texture covering the base sphere.
If we project it on a plane, we will then have a Rectilinear or Planar projection; if we project it on a cylinder we are doing a Cylindrical projection; and if we use the texture as is we are talking about a Spherical projection.

Rectilinear Mode (or Planar projection)
Each pixel of the sphere is reprojected on a plane tangential to the sphere. This automatically implies two constraints:

Only the pixels facing the plane can be projected, representing only one half of the sphere.
The pixels located on the extreme outside limits of the half-sphere will be strongly stretched (for example when using a Planar Projection with more than 150° horizontally FOV).

Cylindrical Mode

When using this model the texture is projected on a cylinder surrounding the base sphere.
This mode is less problematic except when getting close to the poles. The same downside observed with the Planar projection will occur, the pixels located close to the poles will be stretched.

Spherical Mode (or Equirectangular)

No reprojection is to be performed when using this mode. The texture is simply recycled and saved in a latitude/longitude coordinate system.
Therefore a 360° x 180° panorama will have a width/height ratio of 360/180 =2.
The height and width in pixels is proportional to the FOV’s angle.

Note:
The Mercator mode is a compromise between Spherical and Cylindrical modes.

IN PRACTISE

Planar or rectilinear projection

Advantages:

Good choice if the angle is low, recommended for architectural shots because it is the only mode that does not curve lines that are deemed to be straight lines.

Disadvantages:

Can be used only if the view angle is less than 180 degrees along the diagonal of the image, practice shows that beyond 90 degrees the stretching of the image is already visible along the edges and in the corners.

Beyond 120 degrees, the result is no longer acceptable, because the stretching of the image causes an unpleasant loss of sharpness.

Cylindrical projection

Advantages:

Can be used up to 360° (horizontally).

Disadvantages:

The lines parallel to the horizon (curbs, tops of buildings) are still more or less curved.

The vertical angle is limited, it is imperative that it is less than 160 ° but the stretching begins to slightly deform at the top and bottom of the image when it exceeds 45 degrees above or below the skyline.

Mercator projection

Advantages :

Can be used up to 360° (horizontally). The effect of stretching up and down the image is attenuated compared to the spherical projection.

Disadvantages:

The lines parallel to the horizon (curbs, tops of buildings) are still more or less curved.

The vertical angle is limited, it is imperative that it is less than 160 ° since the stretching begins to slightly deform at the top and bottom of the image when it exceeds 55 degrees above or below the skyline.

Spherical or equirectangular projection

Advantages:

Authorizes the assembly of all panoramas.

Disadvantages:

For the direct display (without a viewer) of a panorama on the screen and for feel we need to ensure that the curvature of parallel straight lines on the horizon is still acceptable (there is no rule, you need to be the judge).

When the vertical viewing angle is large, the stretching effect at the top of the image (near the zenith) and at the bottom of the image (near the nadir) varies from unusual to very unusual.

Examples

Using the same example here is a preview of the results obtained with the three projection modes:

Projection réctilinéaire :	Projection cylindrique :	Projection sphérique :

Cylindrical Horizontal

‎ The projection modes can vary depending on the panorama’s orientation. This is the case for the Spherical and Cylindrical projections, not the Planar projection which is not sensitive to the panorama’s orientation.
When looking at the drawing illustrating the Cylindrical projection, we can see that we assumed that the cylinder’s axis was vertical as this is what we are looking for.
But multiple types of cylinders and axis can exist: we can imagine a cylinder with a vertical axis.
The visual aspect of the panorama will then be very different.

From left to right:

Spherical projection
Cylindrical projection
Planar projection
90° Rotation and Cylindrical projection
90° Rotation and Planar projection

Differences between Cylindrical mode (2) and Planar (3) look very close. It’s quite normal as the horizontal FOV of the image is very small, implying that the Cylindrical projection is almost rectilinear.
The view changes dramatically between the spherical mode (1) and the other two modes (2 and 3).
Apply a rotation to the panorama corresponds to put the cylinder of projection in the other axis, this is the cylindrical horizontal (4).
The difference between the cylindrical vertical (2) and the cylindrical horizontal (4) is very visible.
The difference between the two rectilinear (3 and 5) is zero. Exactly the same thing as a rotation does not change the rectilinear projection.

Note: Be careful to correctly set the point of view when working with this kind of very tall subject (in this example at the base of the tower: where the two grey lines intersect).

Spherical Horizontal

The Spherical mode, just like the Cylindrical mode, is dependent on the orientation of the panorama.
We will generally want for that type of projection that the verticals stay vertical in order to obtain nice views.
In some cases, using another axis than the horizontal can turn into a great creative tool.
The following three examples are Spherical views of the same 360° x 180° panorama with very different resulting looks.

This one is a standard Spherical projection with the vertical axis perfectly aligned with the cathedral’s vertical axis.	This view is a Spherical projection where the vertical axis of the panorama is a bit off compared to the real vertical. This is an anomaly we can often see in 360° panoramas: the horizon makes a wave, similar to a sinusoidal curve (like a “~”).	This view is a Spherical Horizontal projection where the sphere’s axis matches a real horizontal line in the subject. The creativity potential offered by this type of projection is really interesting