1. Radiometric Self Calibration
Tomoo Mitsunaga Shree K. Nayar
Hashimoto Signal Processing Lab. 　　 Dept. of Computer Science
Sony Corporation Columbia University
CVPR Conference
Ft. Collins, Colorado
June 1999

2. Usual imaging systems have :
Problem Statement
How well does the image represent the real world?
Image M1
(High exposure)
Image M2
(Low exposure)
Limited dynamic range
Usual imaging systems have :
Non-linear response
June/1999
CVPR99

3. Scene Radiance and Image IrradianceL E
Radiance
Irradiance
Image irradiance :
Ideal camera response :
Aperture area
Exposure :
June/1999
CVPR99

4. Scene Radiance and Measured Brightness
Video
Image
Formation
Image
Exposure
Camera
Electronics
Digitization
CCD
Scaled
radiance I
Measured
brightness M
Scene
radiance L
linear
Photo
Image
Formation
Image
Exposure
Film
Development
Film
Scanning
f (M) : The radiometric response function
June/1999
CVPR99

5. Calibration with Reference Objects
The scene must be controlled
The reflectance of the objects must be known
The illumination must be controlled
June/1999
CVPR99

6. Calibration without Reference Objects
Differently exposed images from an arbitrary scene
Recover the response function from the images
Calibrate the images with the response function
Response function
Input Images
High dynamic range radiance image
June/1999
CVPR99

7. Previous Works Mann and Picard (95) : Debevec and Malik (97) :
Take two images with known exposure ratio R
Restrictive model for f :
Find parameters a, b, g by regression
Debevec and Malik (97) :
General model for f : only smoothness constraint
Take several (say, 10) high quality images
At precisely measured exposures (shutter speed)
June/1999
CVPR99

8. Obtaining Exposure Information
We have only rough estimates
Mechanical error
Reading error (ex. F-stop number)
June/1999
CVPR99

9. Radiometric Self-Calibration
Works with roughly estimated exposures
Inputs :
Differently exposed images
Rough estimates of exposure values
ex. F-stop reading
Outputs :
Estimated response function
Corrected exposure values
June/1999
CVPR99

10. A Flexible Parametric Model
video
posi
nega
High order polynomial model :
f (M)
Parameters to be recovered :
Coefficients cn
Order N M
f(M) of some
popular imaging products
June/1999
CVPR99

11. Response Function and Exposure Ratio
Images: q = 1,2,….Q , Pixels: p = 1, 2, …..P
Exposure ratio:
Using polynomial model :
Objective function :
Thus, we obtain ...
June/1999
CVPR99

12. An Iterative Scheme for Optimization
Rough estimates
Rq,q+1(0)
Rq,q+1(i)
Optimize
for f
Optimize
for Rq,q+1
f (i)
Optimized f and Rq,q+1
June/1999
CVPR99

13. Evaluation : Noisy Synthetic Images
f (M) M
Solid : Computed response function
Dots : Actual response function
June/1999
CVPR99

14. Evaluation : Noisy Synthetic Images (cont’d)
Percentage
Error in
Computed
Response
Function
Trial Number
Maximum Error : %
June/1999
CVPR99

15. Computing a High Dynamic Range Image
Calibrating by the response function
Normalizing by corrected exposure values
Averaging with SNR-based weighting
June/1999
CVPR99

16. Results : Low Library (video)
Captured images I
Calibration chart M
Computed response function
June/1999
CVPR99

17. Results : Low Library (video)
Captured images
Computed radiance image
June/1999
CVPR99

18. Results : Adobe Room (photograph)
Captured images I M
Computed response function
Computed radiance image
June/1999
CVPR99

19. Results : Taos Clay Oven (photograph)
Captured images I M
Computed response function
Computed radiance image
June/1999
CVPR99

20. A Practical Radiometric Self-calibration Method
Conclusions
A Practical Radiometric Self-calibration Method
Works with
Arbitrary still scene
Rough estimates of exposure
Recovers
Response function of the imaging system
High dynamic range image of the scene
Software and Demo
June/1999
CVPR99

21. Obtaining Quality Measurements
Automatic noise reducing pre-processing
For random noise within a pixel
Temporal averaging
For object movement and risky object edges*
Selecting pixels from spatially static area
For vignetting
Preferring the center part of the image
* Object edges are sensitive to noise
June/1999
CVPR99