Presentation is loading. Please wait.

Presentation is loading. Please wait.

Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia.

Similar presentations


Presentation on theme: "Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia."— Presentation transcript:

1 Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia

2 How are images made? One half: geometric vision – “how the pixel projected onto the image” Today: photometric vision (aka radiometric) – “how the pixel got its color”

3 Vision and Graphics Properties of a scene Image Computer Graphics Vision

4 Image Formation Approach Come up with a model for how the scene was created Given images, find the most likely properties that fit that model

5 Diffuse Surfaces Brightness of a pixel depends on: object color lighting direction surface normal But NOT view direction!

6 Lambertian Cosine Law The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction = ρ L  N I = ρ cos θ = ρ |L||N| cos θ L N θ

7 =  L  N = L N

8 = x I = ρ L  N

9 Recovering Albedo and Normals Can you decompose a single image into its albedo and normal images?

10 = x x x

11 Photometric Stereo Given multiple images taken with varying illumination, recover albedo and normals. – take pictures in dark room with varying illumination. – estimate lighting directions L. – recover albedo and normals.

12 Side note 1: How to get the lighting direction? Put a shiny sphere in the scene Sphere’s geometry (normals) are known Find specular highlight

13 Side-note 2: Why “Stereo”? Surface normals provide constraints on depth differences

14 Photometric Stereo If L is known, and albedo is grayscale this is a linear problem. I = ρ(L  N) = ρ (L x N x + L y N y + L z N z ) = L x N x ρ + L y N y ρ + L z N z ρ = L x a + L y b + L z c

15 L x1 L y1 L z1 L x2 L y2 L z2 L x3 L y3 L z3 … L xn L yn L zn I1I2I3…InI1I2I3…In abcabc = I = ρ(L  N) = L x a + L y b + L z c Then: ρ = sqrt(a 2 + b 2 + c 2 ) N = (a,b,c) / ρ For each pixel:

16 Demo

17 When does this model fail? I ≠ ρ (L  N)

18 Attached shadows I = ρ max(L  N, 0) L  N > 0 L  N = 0 L  N < 0

19 Cast Shadows, Ambient Light I = ρ (S L  N + a) S = 0 or 1

20 Radiometric Camera Calibration Pixel intensities are usually not proportional to the energy that hit the CCD RAW imagePublished image

21 Radiometric Camera Calibration f RAW Published

22 Radiometric Camera Calibration Observed = f (RAW) (Grossberg and Nayar) f -1 (Observed) = RAW

23 Radiometric Camera Calibration How do you model f -1 ? f -1 (x) = x γ f -1 (x) = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + … f -1 (x) = f 0 (x) + f 1 (x) c 1 + f 2 (x)c 2 + … mean camera curvebasis camera curves

24 Radiometric Camera Calibration I = f (ρ (S L  N + a)) Adding exposure: I = f (e ρ (S L  N + a))

25 Heliometric Stereo Given lots of images from a stable webcam, use lighting from the sun to recover: I = f (e ρ (S L  N + a))

26 Heliometric Stereo

27

28

29 Radiometric Camera Calibration If you can control the exposure… Take two pictures with different known exposures (e.g. 0.5 second and 1 second): Observed 2 = f (e 2 RAW) Observed 1 = f (e 1 RAW) f -1 (Observed 1 ) = e 1 RAW f -1 (Observed 2 ) = e 2 RAW f -1 (Observed 1 ) e 1 f -1 (Observed 2 ) e 2 = Solve for the best f -1 that fits your model

30 Heliometric Stereo The following should hold for each pixel in each image: I = f (e ρ (S L  N + a)) f : the camera’s response curve e: that image’s exposure value a: that image’s ambient light S: 0 if that pixel is in shadow at that time, 1 otherwise N: that pixel’s surface normal ρ: that pixel’s albedo

31 Heliometric Stereo Step 1: pixel-level thresholding to find shadows Step 2: initialize all variables I = f (e ρ (S L  N + a))

32 Heliometric Stereo Step 3: fix f, e, and a, solve for ρ and N. Step 4: fix ρ and N, solve for f, e, and a. Step 5: goto 3. I = f (e ρ (S L  N + a))

33 The life of a photon

34 BRDF Bi-Directional Reflectance Distribution Function given incoming and outgoing rays, what proportion of light is reflected?

35 BRDF Almost nobody actually tries to model a full BRDF. Why? Build a lighting model with fewer parameters that approximate the BRDF Diffuse lighting model is very common

36 Diffuse Surfaces


Download ppt "Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia."

Similar presentations


Ads by Google