V1.3 - Color and Brightness

From Autopano

Issues involved:

You just stitched a beautiful panorama with a nice geometry arrangement, but significant brightness and color differences remain between the original source images. Why?

• A camera set on automatic mode will modify the shooting parameters from one shot to the other, thus all source images will have a different set of settings. If the camera’s white balance is set on “Auto”, the same gray object will have a slightly different color cast from one shot to the other. In the very same way, the shutter speed can vary from one shot to the other, as well as the ISO setting and the aperture value. The combination of these shooting parameters explains that shots close from each other hardly ever have the same colors, even less the same exposure. Example: a plain blue sky will be darker over a white building and lighter over a dark building.
• In a large outdoor panorama, it is possible that between the first and the last picture of the panorama the lighting condition have changed (cloud passing in front of the sun inducing a global luminosity change).
• For a given shot, the corners of the picture can be darker than the center, this is an optical anomaly referred to as vignetting. This anomaly, often minor and ignored on a single image, is much more visible when assembling pictures into a panorama: the stitching seam brings out the vignetting.
• When the FOV is very large, we often find ourselves with backlit areas and others in direct light: In such a case we make the voluntary choice of modifying the exposure from one shot to the other, but this is not always sufficient, especially in the case of a 360° x 180° panorama.

Consequences: a lot of causes resulting in the same problem, some luminosity and color variations.

Fortunately, Autopano Pro can correct these variations thanks to the use of a brand new color correction algorithm, and a set of special tools.

Three correction modes available

•  : no correction
  This is the default mode used when launching the rendering without going through the Editor. It is the default mode when opening the Editor Window. In this mode, the mechanism in charge of balancing the luminosity or the colors of the source pictures is disabled. Color correction is indeed unnecessary in some case, for example when all the source pictures where shot in manual mode. Be conscious however that, even when shooting in manual mode you can find yourself with variations (i.e. auto ISO, global luminosity change due to a cloud etc.).
•  : LDR correction also referred to as Standard Correction
  The purpose of this correction mode is to balance small brightness, contrast and color variations within the panorama’s source pictures set. This is the correction mode you will be using most of the time.
•  : HDR correction
  This is the mode to use when the subject bears great differences in exposition values. The algorithm used is different than the one used for LDR correction and can balance panoramas with greater luminosity variations.

Tip: you can change the default color correction mode in the Detection tab of the Application Settings dialog. In the Editor tab, you can also choose the default type of anchor.

LDR Correction	HDR Correction

Choosing the Right Correction Mode

If the blender you chose does not level by itself the variations between the source images, two simple and quick options are available to you:

• Small variations between the source images:
  • choose the LDR correction (also referred to as STANDARD)
• Large variations between the source images:
  • choose the HDR correction
  • In the Filter menu, choose Tone Mapping Rh2 and use the top slider "(Key) to modify the image

Color Correction Setup

You just activated the color correction but the result is not what you are looking for? No problem! Our color correction engine can be tuned up to your own needs. You must first set what we call the “Color Anchors”: an anchor represents how the corresponding image will be modified by the color correction algorithm.

Use the Edit Anchor button to enter the anchor addition mode. Activating this button also activates the Layer View button in the main toolbar (each individual source picture is now visible inside the panorama). You can now edit each individual image’s anchor (if several pictures are stacked at a given location use the mouse scroll wheel to cycle through the pictures and make your selection). To modify an anchor, right click on a picture and choose the new anchor type from the pop-up menu for this particular image. An optimization is automatically launched after each modification and the result is displayed after completion.

Tip: for very large panoramas, or when making multiple anchor changes at once, you can set Autopano Pro to wait for a confirmation before launching the optimization (in the Editor tab of the Application Settings dialog). This will allow you to change several anchors and when ready: hit the keyboard Enter key to update the preview.

Anchor Types

• Hard Anchor. This portion of the panorama will remain unchanged and the corresponding source image will be set as a reference. If this image is underexposed the whole panorama will be underexposed, if overexposed the panorama will be overexposed. If the image has a red or blue cast the whole panorama will inherit of that color cast.
• Transfer Function. Only the luminosity and contrast values will be adjusted for this area of the panorama: This is the obvious choice if the white balance was locked on a set value when the pictures were shot. Locking the white balance at shooting time is preferable, except in a situation where different light sources with different color temperatures are included in the same scene. This can be the case with a blue sky in the shadow and for the sun in direct light, this can also be the case indoors with tungsten bulbs and fluorescent bulbs illuminating different areas of the same scene.
• Transfer Function + White Balance. All parameters will be adjusted for this image (color, brightness and contrast).
• White Balance. Only the white balance will be adjusted on this image.

Setting multiple references

When only one source image is not sufficient as a reference (whatever source picture you choose some areas of the panorama remain too bright or too dark) it is possible to define two or more references and set them on areas of the panorama that are identical or close to the areas you want to correct and where the corresponding zones are acceptable: the further from each other the anchors are located, the easier it is for the software to avoid huge variations when jumping from one source image to the other.

White balance variations are often more subtle and less visible on the panorama than the exposure variations corrected with a Transfer Function. In some cases, it happens that we need to manage the white balance (can arise in indoor photography when using multiple types of light sources).

Layers

You can create several layers in a pano (for example, Sky, Middle and Land). Each layer can have it's own color correction. Normally, you get a file for each layer during a render. To create a single file output, create 3 layers, do the color correction with them, use the reset button to change back all layer into one single layer ( DO NOT use anymore color correction for this panorama ), render. The layer groups is in fact used temporary to fine tune color correction.

It doesn’t work! My panorama is all dark!

Don’t panic: sometimes the color correction constraints lead to great color divergence. In almost all cases changing all the anchors to Transfer Function will be sufficient. The color correction will also work and the colors will only change by a small amount. But first, look at the histogram. You may have to change the white point.

Dynamic Range Extension

A consequence of color and exposure correction is the broadening of the dynamic range of the resulting panorama. Let’s take a look at this simple color correction case.

When analyzing the histograms we can see that by matching colors of the two pictures the resulting histogram is transformed. We can see that the histogram of the second image becomes narrower (it shrinks). But this histogram still contains it’s 8bits of resolution, assuming we are working with JPEG source images. This implies that the global quality of the panorama is greater, the resolution between two segments of the histogram becomes smaller; the dynamic range is then widened. The histogram of the final panorama is not strictly contained between 0 and 255 anymore, but extends beyond those limits.

Related Topic: Dynamic Range

The HDR Mode

This correction mode works in a different way than the LDR mode. Autopano Pro directly calculates each pixel’s luminance; it does not work with pixel color values anymore, but working with the real luminosity values of the scene. In practice, on a low dynamic range scene you will not see any difference, but as soon as the exposure variations become greater - it changes everything.

When choosing the HDR mode, the resulting image often presents a dynamic range that is too wide. We must then reduce the dynamic range to obtain something we can display on the screen. You can refer to the Dynamic Range topic for more information. In Autopano Pro, levels are provided within the HDR mode. They allow some global adjustments of the panorama to reduce the dynamic range. You can also use the Tone Mapping Filters to compress the dynamic range.

HDR Mode Filters

The filters are applied to the whole panorama. Currently only two filters are available; much more will be added in the furture.

These Tone Mapping Filters help to obtain a satisfying image from the image that was just calculated. It's as if you would have a second opportunity to adjust the brightness and contrast, and much more depending on the selected filter.

• Tone Mapping RH2
  Key slider: brightness adjustment
  White slider: sets the white clipping threshold
• Tone Mapping RH4
  Brightness slider: brightness adjustment
  Saturation slider: saturation adjustment

Only one filter can be used at once. Only one filter of the same type can be activated at the same time, if another filter is active it will automatically be disabled.

These filters were designed for the HDR Mode but they can be used to adjust independently the shadows and the highlights of a “regular” picture.

Levels

This tool, even though it uses the same mechanics and design as the Levels Tool found in Adobe Photoshop, is a much more advanced tool than the one found in Photoshop.

The Levels Tool allows us to adjust and modify the possible inputs/outputs in terms of brightness value. The big difference between the Photoshop Levels Tool and the Autopano Pro Levels Tool is its capacity to work on any dynamic range width. In Photoshop, the displayed dynamic range (i.e. the one available to work with) is either 8bits or 16bits (only 8bits are actually displayed on the monitor). The Levels Tool in Autopano Pro can handle various dynamic range widths. You can visualize the range just above the histogram’s curve, expressed in Min. and Max. values. You also have access to the white and black eyedroppers, just like in Photoshop. The eyedroppers are used the set the panorama's white and black points. The levels will then be adjusted in response to your inputs.

The levels standard features are available; you can for example adjust the levels by color channel, adjust the gamma per channel or globally. The Auto Levels adjustment is also provided. You can use the Auto button to initiate the correction. The Option button is used to select the adjustment mode and to set the white and black clipping values.

When assembling source pictures that are already in HDR format (i.e. having a very wide dynamic range), it is very useful to work with a logarithmic curve instead of a linear curve. This can be done by double-clicking on the Levels Histogram. The sliders will then adjust to a logarithmic scale.