1. Photometric Image Formation
CSE 559: Computer Vision
Guest Lecturer: Austin Abrams
Images/Demo from Steve Seitz, Wikipedia

2. How are images made? One half: geometric vision
“how the pixel projected onto the image”
Today: photometric vision (aka radiometric)
“how the pixel got its color”

3. Vision and Graphics Image Properties of a scene Computer Graphics

4. Image Formation Approach
Come up with a model for how the scene was created
Given images, find the most likely properties that fit that model

5. Diffuse Surfaces Brightness of a pixel depends on: object color
lighting direction
surface normal
But NOT view direction!

6. Lambertian Cosine Law L N
The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction
I = ρ cos θ L N θ
= ρ |L||N| cos θ
= ρ L  N

7. = 
L  N = L N

8. = x
I = ρ L N

9. Recovering Albedo and Normals
Can you decompose a single image into its albedo and normal images?

10. x = x x

11. Photometric Stereo
Given multiple images taken with varying illumination, recover albedo and normals.
take pictures in dark room with varying illumination.
estimate lighting directions L.
recover albedo and normals.

12. Side note 1: How to get the lighting direction?
Put a shiny sphere in the scene
Sphere’s geometry (normals) are known
Find specular highlight

13. Side-note 2: Why “Stereo”?
Surface normals provide constraints on depth differences

14. Photometric Stereo
If L is known, and albedo is grayscale this is a linear problem.
I = ρ(L  N)
= ρ (Lx Nx + Ly Ny + Lz Nz )
= Lx Nxρ + Ly Nyρ + Lz Nzρ
= Lx a + Ly b + Lz c

15. For each pixel: I = ρ(L  N) = Lx a + Ly b + Lz c I1 Lx1 Ly1 Lz1 I2…
Lxn Lyn Lzn I1 I2 I3 In a b c =
Then:
ρ = sqrt(a2 + b2 + c2)
N = (a,b,c) / ρ

16. Demo

17. When does this model fail?
I ≠ ρ (L  N)

18. I = ρ max(L  N, 0) Attached shadows L  N = 0 L  N > 0

19. Cast Shadows, Ambient Light
I = ρ (S L  N + a)
S = 0 or 1

20. Radiometric Camera Calibration
Pixel intensities are usually not proportional to the energy that hit the CCD
RAW image
Published image

21. Radiometric Camera Calibration
Published f
RAW

22. Radiometric Camera Calibration
Observed = f(RAW)
(Grossberg and Nayar)
f -1 (Observed) = RAW

23. Radiometric Camera Calibration
How do you model f -1?
f -1(x) = xγ
f -1(x) = c0 + c1x + c2x2 + c3x3 + …
f -1(x) = f0(x) + f1(x) c1 + f2(x)c2 + …
mean camera curve
basis camera curves

24. Radiometric Camera Calibration
I = f (ρ (S L  N + a))
Adding exposure:
I = f (e ρ (S L  N + a))

25. Heliometric Stereo I = f (e ρ (S L  N + a))
Given lots of images from a stable webcam, use lighting from the sun to recover:
I = f (e ρ (S L  N + a))

26. Heliometric Stereo

27. Heliometric Stereo

28. Heliometric Stereo

29. Radiometric Camera Calibration
If you can control the exposure…
Take two pictures with different known exposures (e.g. 0.5 second and 1 second):
Observed2 = f (e2 RAW)
Observed1 = f (e1 RAW)
f -1 (Observed1) = e1 RAW
f -1 (Observed2) = e2 RAW
f -1 (Observed1) e1
f -1 (Observed2) e2 =
Solve for the best f -1 that fits your model

30. Heliometric Stereo I = f (e ρ (S L  N + a))
The following should hold for each pixel in each image:
I = f (e ρ (S L  N + a))
f : the camera’s response curve
e: that image’s exposure value
a: that image’s ambient light
S: 0 if that pixel is in shadow at that time, 1 otherwise
N: that pixel’s surface normal
ρ: that pixel’s albedo

31. Heliometric Stereo I = f (e ρ (S L  N + a))
Step 1: pixel-level thresholding to find shadows
Step 2: initialize all variables
I = f (e ρ (S L  N + a))

32. Heliometric Stereo I = f (e ρ (S L  N + a)) I = f (e ρ (S L  N + a))
Step 3: fix f, e, and a, solve for ρ and N.
Step 4: fix ρ and N, solve for f, e, and a.
Step 5: goto 3.
I = f (e ρ (S L  N + a))
I = f (e ρ (S L  N + a))

33. The life of a photon

34. BRDF Bi-Directional Reflectance Distribution Function
given incoming and outgoing rays, what proportion of light is reflected?
just a big word that describes how light’s energy is transferred upon hitting a surface

35. BRDF Almost nobody actually tries to model a full BRDF. Why?
Build a lighting model with fewer parameters that approximate the BRDF
Diffuse lighting model is very common

36. Diffuse Surfaces