Understanding Projecting Modes

From Autopano

About Stitching Pictures

Before we can effectively describe the various projection types, let’s go back to the notion of image assembling (i.e. image stitching).

This example illustrates the notion of image stitching in standard mode (i.e. when all the source images were taken by rotating the camera around its nodal point). Each source image will receive a set of spherical coordinates (Yaw, Pitch and Roll); when projected on the base sphere, it must exactly match the surrounding images. Stitching a panorama is basically finding the location of each source image on a base sphere.

The resulting texture on the sphere’s surface is the stitched image. The resulting image can either cover the totality of the sphere’s surface in the case of a 360° x 180° panorama, or only a fraction of it.

Note :
We are assuming here that the stitching being performed is based on a model where the camera rotates around its nodal point as illustrated in the above image. Other stitching models are possible and will soon be supported by Autopano Pro. The most popular is the Orthogonal, also called Orthographic stitching. All the source images are shot orthogonally to the same plane. Instead of rotating around its center, the camera is following a linear motion path, always pointing in the same direction. This is exactly what is being done when scanning a A3 size sheet of paper with a letter size scanner. We end up with 4 or 5 files assembled using this stitching model. It is also what happens when taking pictures of building fronts in a street by walking down the street and taking a picture facing the buildings every 10 steps.

The Projection

The projection modes are referring to what’s being done with the texture covering the base sphere. If we project it on a plane, we will then have a Rectilinear or Planar projection; if we project it on a cylinder we are doing a Cylindrical projection; and if we use the texture as is we are talking about a Spherical projection.

• Rectilinear Mode ( also referred to as Planar )
  

Each pixel of the sphere is reprojected on a plane tangential to the sphere. This automatically implies two constraints. Only the pixels facing the plane can be projected, representing only one half of the sphere. The second downside is that when reaching the pixels located on the extreme outside limits of the half sphere (i.e. pixels almost parallel to the projection plane), their projected image with be stretched (zoom effect). This is what we see when using a Planar Projection with large FOVs (more than 150° horizontally for example).

• Cylindrical Mode ( Cylindrical Projection ):
  

When using this model the texture is projected on a cylinder surrounding the base sphere. This mode is less problematic except when getting close to the poles. The same downside observed with the Planar projection will occur, the pixels located close to the poles will be stretched.

• Spherical Mode ( also called Equirectangular ):
  

No reprojection is to be performed when using this mode. The texture is simply recycled and saved in a latitude/longitude coordinate system. Therefore a 360° x 180° panorama will have a width/height ratio of 360/180 =2. The height and width in pixels is proportional to the FOV’s angle.

Real World

• Mercator Projection
  • Advantages: can handle horizontal FOVs up to 360°. Stretching effect at the top and bottom of the image is reduced compared to the Spherical projection.
  • Downside: all straight lines parallel to the horizon (curbs, building tops) will, to various extends, be bent. The vertical angle is limited, it MUST be smaller than 160° because the stretching effect will start to appear at the top and at the bottom of the image passed 55° from the horizon (over and down).

• Linear Projection
  • Advantages: always a good choice when working with a small FOV (field of view), recommended for architecture as it is the only projection mode that will not curve any line (the other two modes will always, to a certain extend, “bend” curbs and building tops).
  • Downside: in theory this mode can only be used if the FOV is smaller than 180°, determined by the diagonal of the image. In real world situation the limitation is actually 90° as the stretching produced on both edges of the image is already visible at 90°, and even more pronounced at the corners. Passed 120°, the results become unacceptable as the stretching will produce a significant and uncomfortable loss of sharpness.
• Cylindrical Projection
  • Advantages: can handle horizontal FOVs up to 360°.
  • Downside: all straight lines parallel to the horizon (curbs, building tops) will, to various extends, be bent. The vertical angle is limited, it MUST be smaller than 160°, but a stretching will start to appear at the top and at the bottom of the image passed 45° from the horizon (over and down).
• Spherical Projection
  • Advantages: this is the default choice as it can handle any panorama type.
  • Downside: For a direct display on a computer screen (without using a special viewer) or to print the panorama, you must first make sure that the amount of curvature of the lines parallel to the horizon stays acceptable. There is no set rule to determine what is acceptable and what’s not, you must use your best judgment. When the vertical field of view is large, the stretching of the top of the image (close to the zenith) and of the bottom of the image (close to the nadir) can vary from very natural to quite unnatural

Using the same example here is a preview of the results obtained with the three projection modes:

Cylindrical Horizontal

‎ The projection modes can vary depending on the panorama’s orientation. This is the case for the Spherical and Cylindrical projections, not the Planar projection which is not sensitive to the panorama’s orientation. When looking at the drawing illustrating the Cylindrical projection, we can see that we assumed that the cylinder’s axis was vertical as this is what we are looking for. But multiple types of cylinders and axis can exist: we can imagine a cylinder with a vertical axis. The visual aspect of the panorama will then be very different.

From left to right: Spherical projection; Cylindrical projection; Planar projection. Then 90° Rotation and Cylindrical projection; 90° Rotation and Planar projection.
We can see that the Cylindrical and Planar projections look very close. It’s quite normal as the horizontal FOV of the image is very small, implying that the Cylindrical projection is almost rectilinear. But we can see that the Spherical mode stands apart from the other two modes.

The last two examples are provided to illustrate the visual effects obtained when rotating the panorama before changing the projection mode. Rotating the panorama is like placing the reprojection cylinder on the “Y” axis, thus doing a Cylindrical Horizontal projection. We instantly see the difference between the Cylindrical Vertical and Cylindrical Horizontal. On the other hand, there is no difference between the two rectilinear projections, this is to be expected. The resulting images are identical as a rotation does not affect the rectilinear projection.

Note: Be careful to correctly set the point of view when working with this kind of very tall subject (in this example at the base of the tower: where the two grey lines intersect).

Spherical Horizontal

The Spherical mode, just like the Cylindrical mode, is dependent on the orientation of the panorama. We will generally want for that type of projection that the verticals stay vertical in order to obtain nice views. In some cases, using another axis than the horizontal can turn into a great creative tool.

	The following three examples are Spherical views of the same 360° x 180° panorama with very different resulting looks. This one is a standard Spherical projection with the vertical axis perfectly aligned with the cathedral’s vertical axis.
	This view is a Spherical projection where the vertical axis of the panorama is a bit off compared to the real vertical. This is an anomaly we can often see in 360° panoramas: the horizon makes a wave, similar to a sinusoidal curve (like a “~”).
	This view is a Spherical Horizontal projection where the sphere’s axis matches a real horizontal line in the subject. The creativity potential offered by this type of projection is really interesting.