Ning Sun, Hassan Mansour, Rabab Ward Proceedings of 2010 IEEE 17th International Conference on Image Processing September 26-29, 2010, Hong Kong HDR Image Construction from Multi-exposed Stereo LDR Images Andy

2 Intelligent Systems Lab. Algorithm description Two LDR images with different exposures Initial disparity map Camera response function Radiance maps of LDR images Refined disparity map HDR image Main concept: 1. Multi-exposed stereo images are captured using identical cameras placed adjacent to each other on a horizontal line. 2. Stereo matching is then used to find a disparity map that matches each pixel in one image to the corresponding pixel in another image. 3. A subset of the matched pixels is used to generate the camera response function which in turn is used to generate the scene radiance map for each view with an expanded dynamic range. 4. The disparity map is refined by performing a second stereo matching stage using the radiance maps

3 Intelligent Systems Lab. Imaging models Gamma-correction model Polynomial camera response Imaging models are used to determine the scene radiance from the measured pixel data Left imageRight image Scene radiance Correction factor Exposure ration between images Left imageRight image Scene radiance

4 Intelligent Systems Lab. Computing the disparity map Best disparity map Dissimilarity termSet of feasible disparitiesSmoothing term Pixel dissimilarityDisparity smoothness Used for initial disparity estimation

5 Intelligent Systems Lab. Pixel dissimilarity - Search window centered on p- displacement- Bilateral weight Spatial smoothingIntensity smoothing I’ - intensity in log space defined as:

6 Intelligent Systems Lab. Pixel dissimilarity

7 Intelligent Systems Lab. Disparity smoothness Initial disparity and camera response 1. Minimize using graph cut algorithmgraph cut algorithm 2. Compute polynomial coefficients for camera response function

8 Intelligent Systems Lab. Error correction Minimize energy function one more time with different dissimilarity function For valid pixels Convert images to radiance space (results should be same for both images) For erroneous pixels Hamming distance between pixels p and p+fp after applying Census transformCensus transform

9 Intelligent Systems Lab. Input LDR images

10 Intelligent Systems Lab. Disparity maps Reference disparity mapInitial disparity estimationFinal map

11 Intelligent Systems Lab. HDR images

12 Intelligent Systems Lab. Experimental results Image nameExposure RatioRMSE ErrorError pixels (%) Statue Dolls Clothes Baby

13 Intelligent Systems Lab. Conclusions Disparity map computation algorithm is proposed Proposed method is able to compute disparity between differently exposed images Can deal with saturated regions in the image Can be used for capturing motion scenes with different exposures Disadvantages - High computational costs - Generated images are slightly blurred - No rotation is considered

14 Intelligent Systems Lab. Ideal image formation system Image brightness Sensor response Camera exposure Camera response function Response = Gray-level Irradiance L I Reverse camera response function From optics Image radiance Scene radiance Focal length Aperture Angle from ray to optical axis Radiometric response Shutter speed or Where

15 Intelligent Systems Lab. Response function examples Response functions of a few popular cameras provided by their manufacturers

16 Intelligent Systems Lab. Graph-cut algorithm 1. Start with an arbitrary labeling f 2. Set success := 0 3. For each label 2 L 3.1. Find f* = arg min E(f’) among f’ within one α -expansion of f 3.2. If E(f*) < E(f), set f := f* and success := 1 4. If success = 1 goto 2 5. Return f

17 Intelligent Systems Lab. Census transform If (CurrentPixelIntensity<CentrePixelIntensity) boolean bit=0 else boolean bit=1 Input image3x3 transform5x5 transform