Anti-Ghost is the smart image-cutting algorithm (first appeared in Autopano 2.5).
During the blend of stitched images, the layered pixels are not necessarily identical.
This can be caused by stitching problems (when nodal point is not respected, when lens distortion is hard to correct...) and/or from objects in the picture moving between shots.
The Anti-Ghost is designed to find a cutting path between images in order to avoid blending pixels that don’t match.
The “smart” part is choosing a cut that preserves the integrity of the photographed subjects as much as possible.
This new algorithm replaces the cutting step performed by Smartblend before Autopano 2.5 was released.
The first part of this tutorial outlines the conceptual differences between Anti-Ghost and Smartblend and explains how Anti-Ghost (usually) makes better choices.
Since the results are not always perfect (or don’t always look quite as we expected), we will show a few possibilities that can help guide cutting decisions.
The giga-advantage of the new algorithm is that it can handle Gigapixel images, which Smartblend can’t do. The last part will illustrate this using the image Paris 26 gigapixels.
Anti-Ghost VS Smartblend
To illustrate the differences between the two algorithms, here is a scene that contains a great deal of overlap and a main moving subject:
We would expect the resulting panorama either to keep or to completely eliminate the cyclist in each image.
Here is the panorama with Smartblend (Autopano 2.0):
And the panorama we get with Anti-Ghost (Autopano 2.5):
Neither version of the panorama is perfect, but the one with Anti-Ghost is better.
The techniques used are closely related, but one major difference accounts for the different results:
- Smartblend chooses the cutting path one image at a time. In other words, each time an image is added, Smartblend validates the way it cuts using the images already in place, without knowing which images will be added subsequently.
- Anti-Ghost takes all the layered images into account when making a decision.
This is nicely illustrated on the right-hand side of the panorama: Smartblend kept a cyclist when adding an image and can’t delete it when it adds the next image.
Because Anti-Ghost reasons more holistically, it can make incorrect choices at any given moment and change them as it learns the constraints.
Even with the new Anti-Ghost, the result is still not perfect. The following chapter contains a few hints for correcting problems that might come up.
Manually Adjusting Anti-Ghost
When there are sufficient areas of overlap, it is often useful to delete certain images in order to facilitate Anti-Ghost’s selections.
With fewer constraints to reconcile, the panorama now appears as expected.
Color Correction influence
When there is not as much overlap as in the previous example, it is not possible to delete an image in its entirety to guide Anti-Ghost:
In the picture below, a moving flag is situated in an area of overlap. Unlike the previous example, it is not possible to delete an image without drastically altering the overall panorama.
Here is the rendering obtained when color correction is not applied:
Anti-Ghost chose to keep the flag from the image of the zenith (image 7), whereas we would prefer to keep the one from the central image (image 1).
In addition, this cutting selection does not compensate for the fit error around the flagpole.
This time, if we apply color correction, Anti-Ghost’s choices are different and yield the visual result we expected:
It’s partly by chance that color correction makes it possible to keep the right flag. But what’s interesting here is to understand how color correction influenced Anti-Ghost’s choice:
In order to select the best cutting path, the algorithm is based on analysis of the differences between the images. When there are differences in exposure and/or white balance between two images, Anti-Ghost is forced to find the cutting path in the places where the images are alike.
In the first example, without color correction, the hue of the sky in the zenith (image 7) is quite different from the hue in the central image (image 1). Therefore it’s less of a problem to make a cutting error involving a few pixels around the flagpole than to keep compounding differences by trying to blend the sky.
In the second example, color correction makes it possible to harmonize the hues in the sky, meaning it’s less of a problem to cut in the sky. Anti-Ghost chooses to cut the sky rather than cutting around the flagpole, where the adjustment error is now more difficult.
Adding an alpha layer
Using the same example, let’s imagine that even with color correction, Anti-ghost still doesn’t keep the “right” flag.
The idea is to get rid of the data superimposed on the object we want to keep. Thus, Anti-Ghost will have no other option than to keep it and find the best cutting path around this object.
Here, the zenith (Image 7) has been modified. An alpha layer has been painted over the area that is expected to overlap the flag in the central image (Image 1) and converted to a format that supports transparency (tif or png, for instance).
Note: Be careful, because some image editors lose EXIF information, you will need to set it manually in Image Properties (in this example, fisheye information may be lost).
Here is the stitching when the original zenith is replaced by the one with the alpha painted in.
It is not always easy to spot overlap zones in the initial images (not distorted by the assembly). We plan to add a tool to be directly integrated into Autopano that will guide Anti-Ghost’s selections.
As stated in the introduction, the new Anti-Ghost can be applied to Gigapixel images. We rendered the image Paris 26 Gigapixels, originally created with Autopano 2.0, using Autopano 2.5.
This image will soon be available on a new site. Meanwhile, here are some screenshots of the new rendering.
You can try finding these screenshots in the original image if you would like to compare the results: Paris 26 Gigapixels
The first two captures illustrate Anti-Ghost’s effectiveness. Vehicles and people were retained in their entirety. Blurry zones appearing in the rendering created with Autopano 2.0 are gone.
This last screenshot shows that some errors still remain. This is generally because some problems still can’t be solved:
The front end of an object appears in a non-overlapping zone (which must be retained) but the other end doesn’t appear in the adjacent image because in a Gigapixel image, the time gap between shots of adjacent images can be very long.