Arnaud Darmont-High Dynamic Range Imaging Sensors and Architectures-ch6

Chapter 6 Automatic High Dynamic Range Control 6.1 Automatic Exposure of Linear Sensors 6.1.1 Principle In most linear response sensors, there are mechanisms for selecting the correct exposure and correct gain to provide the best exposure and noise tradeoff. These regulators act on exposure and gain settings to target a certain average image brightness for a complete image or for a region of an image (or a weighted average of several regions, sometimes called tiles). If the image is too dark, exposure time or gain can be increased. If the image is too bright, exposure time or gain can be decreased. The decision to work on gain or exposure depends on the situation and on the sensor. High gains should be avoided because of the resulting noise amplification and reduction of precision in exposure time settings due to enlarged steps. Exposure time variation is preferable when possible, i.e., until exposure reaches a maximum value, defined by the type of scene and frame rate requirements. Still scenes allow for long exposures, while moving objects require short exposures and a higher gain. Compared to photography, exposure time is the shutter time, and gain is the ISO speed. An example of the regulation process follows: if (brightness < target) if (current exposure time < maximum exposure time) increase exposure time, else increase gain, else if (current gain > default gain) decrease gain, else decrease exposure time. Another possible method is to predict the required exposure or gain to reach the target. In this case, a normal proportional-integral-derivative (PID) regulator is 97 Downloaded From: http://ebooks.spiedigitallibrary.org/ on 06/07/2013 Terms of Use: http://spiedl.org/terms 98 Chapter 6 used, and the theory of linear regulation applies. It is also possible to combine both techniques—for example, predicting the correct exposure when image brightness is far from the target, and using the incremental adjustment when the actual brightness is close to the target. This solution improves speed and stability. 6.1.2 Brightness calculation The easiest method for estimating image brightness is to consider the average brightness of the image. Using the average is a time-consuming calculation, but good estimates can be obtained by using only the most significant bits of the pixel values and a subsampled version of the image. An alternative is to use the image median brightness. The median can easily be estimated using a two-bin histogram, i.e., a histogram that only counts pixels that are above and below 50% (only the most significant bit of pixel data). Although this method of estimating the median is computationally more efficient and less memory demanding than estimating the average, it has the disadvantage of not being compatible with a predictive algorithm, as an unbalanced single-bin histogram will not indicate how far from the center the median actually is. 6.1.3 Filtering and stability for machine vision There are two different cases in machine vision requirements. If an application requires several consecutive images to be acquired with the same exposure, the exposure time or gain changes in a sequence must be avoided. Oppositely, applications that are based on a single image can have variations of exposure time at every frame, the changes can be fast, and stability is less important. “Less important” means that small oscillations are not a problem (of course, the exposure should be correct and stable). 6.1.4 Filtering and stability for display The human eye is very sensitive to brightness variation. Oscillations in image brightness (flickering) must be avoided. Regulators should not be too fast (so that stability can be maintained) and should be filtered for typical illumination frequencies due to the 50- or 60-Hz frequency of the supply (100- or 120-Hz flickering). A possible technique is to use the average brightness over several frames as input to the regulation algorithm, instead of the brightness of the last acquired image (low-pass filter). 6.1.5 Guard-band-based filtering To avoid oscillations around the target position, guard bands are usually added around the target. Guard bands are similar to a Schmitt buffer (i.e., an electronic digital buffer with hysteresis). For example, exposure time increases if the brightness is sufficiently lower than the target, and decreases if the brightness is sufficiently higher than the target. If Downloaded From: http://ebooks.spiedigitallibrary.org/ on 06/07/2013 Terms of Use: http://spiedl.org/terms Automatic High Dynamic Range Control 99 there is a small variation of image brightness around the target brightness, this will not trigger an increase or decrease of the exposure time. Guard bands are of high importance when the step size of the control signal is large (for example, the smallest possible change of the exposure time makes a large change in image brightness). They are of less importance if the control is nearly continuous. 6.2 Automatic Exposure of High Dynamic Range Sensors Some HDR sensors do not require control at all, for example, the direct-current sensor that uses the logarithmic response. Other sensor responses have only one parameter to control and can therefore be regulated based on target brightness only. Difficulties arise when the response curve must be controlled by several parameters (several degrees of freedom). A two-parameter curve is common with, for example, multiple-segment responses. Response curves that depend on more than three parameters are very difficult to control. The degrees of freedom can usually be reduced to two or three parameters by introducing relationships between the parameters so that regulation becomes easier. For example, multiple-segment responses have two degrees of freedom per segment: the slope and the kneepoint position. In multiple-segment response sensors (Fig. 6.1), the distance between the kneepoint is usually constant, and the slope ratio between segments is also kept constant. The degrees of freedom are then the total exposure, the number of segments, and the exposure ratio between two consecutive segments. With a single degree of freedom, the state vector of the regulator reduces to a single value: the average (or median) image brightness. With two degrees of freedom, the state vector has two elements: the average (or median) image brightness and the number of saturated pixels. The level over which the pixels are considered saturated is known from the sensor’s characterization or for practical reasons is chosen arbitrarily as a certain percentage of the ADC range. With three degrees of freedom, the state vector has three elements. The first two are the same as in the previous case, and the third one is an estimate of the total size of the dynamic range gaps. The first two elements regulate the total exposure and Figure 6.1 Reduction of the degrees of freedom in a piecewise linear regression sensor response curve with three segments. Downloaded From: http://ebooks.spiedigitallibrary.org/ on 06/07/2013 Terms of Use: http://spiedl.org/terms 100 Chapter 6 total dynamic range, and the last one regulates the number of segments of the curve. Using more segments for the same dynamic range provides a less-steep angle between two consecutive segments of the response, hence reducing the dynamic range gaps (SNR holes). A major difficulty in the regulation of systems where multiple inputs affect multiple outputs is that the control inputs and outputs are all linked. Separating the variables so that one input affects only one output helps to reduce the complexity of the regulation loop and the stability analysis. Using the median of an image as input to the total exposure time control has the advantage of the median being almost independent of the response compression (dynamic range extension) because the compression of the bright pixels affects the average of the image, but (almost) not the median. The median is affected if there are kneepoints with strong compression located in parts of the pixel response that are lower (i.e., darker) than the median. The change in the median is always less than the change in the average. Downloaded From: http://ebooks.spiedigitallibrary.org/ on 06/07/2013 Terms of Use: http://spiedl.org/terms