Presentation on theme: "Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia."— Presentation transcript:
Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia
How are images made? One half: geometric vision – “how the pixel projected onto the image” Today: photometric vision (aka radiometric) – “how the pixel got its color”
Vision and Graphics Properties of a scene Image Computer Graphics Vision
Image Formation Approach Come up with a model for how the scene was created Given images, find the most likely properties that fit that model
Diffuse Surfaces Brightness of a pixel depends on: object color lighting direction surface normal But NOT view direction!
Lambertian Cosine Law The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction = ρ L N I = ρ cos θ = ρ |L||N| cos θ L N θ
= L N = L N
= x I = ρ L N
Recovering Albedo and Normals Can you decompose a single image into its albedo and normal images?
= x x x
Photometric Stereo Given multiple images taken with varying illumination, recover albedo and normals. – take pictures in dark room with varying illumination. – estimate lighting directions L. – recover albedo and normals.
Side note 1: How to get the lighting direction? Put a shiny sphere in the scene Sphere’s geometry (normals) are known Find specular highlight
Side-note 2: Why “Stereo”? Surface normals provide constraints on depth differences
Photometric Stereo If L is known, and albedo is grayscale this is a linear problem. I = ρ(L N) = ρ (L x N x + L y N y + L z N z ) = L x N x ρ + L y N y ρ + L z N z ρ = L x a + L y b + L z c
L x1 L y1 L z1 L x2 L y2 L z2 L x3 L y3 L z3 … L xn L yn L zn I1I2I3…InI1I2I3…In abcabc = I = ρ(L N) = L x a + L y b + L z c Then: ρ = sqrt(a 2 + b 2 + c 2 ) N = (a,b,c) / ρ For each pixel:
When does this model fail? I ≠ ρ (L N)
Attached shadows I = ρ max(L N, 0) L N > 0 L N = 0 L N < 0
Cast Shadows, Ambient Light I = ρ (S L N + a) S = 0 or 1
Radiometric Camera Calibration Pixel intensities are usually not proportional to the energy that hit the CCD RAW imagePublished image
Radiometric Camera Calibration f RAW Published
Radiometric Camera Calibration Observed = f (RAW) (Grossberg and Nayar) f -1 (Observed) = RAW
Radiometric Camera Calibration How do you model f -1 ? f -1 (x) = x γ f -1 (x) = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + … f -1 (x) = f 0 (x) + f 1 (x) c 1 + f 2 (x)c 2 + … mean camera curvebasis camera curves
Radiometric Camera Calibration I = f (ρ (S L N + a)) Adding exposure: I = f (e ρ (S L N + a))
Heliometric Stereo Given lots of images from a stable webcam, use lighting from the sun to recover: I = f (e ρ (S L N + a))
Radiometric Camera Calibration If you can control the exposure… Take two pictures with different known exposures (e.g. 0.5 second and 1 second): Observed 2 = f (e 2 RAW) Observed 1 = f (e 1 RAW) f -1 (Observed 1 ) = e 1 RAW f -1 (Observed 2 ) = e 2 RAW f -1 (Observed 1 ) e 1 f -1 (Observed 2 ) e 2 = Solve for the best f -1 that fits your model
Heliometric Stereo The following should hold for each pixel in each image: I = f (e ρ (S L N + a)) f : the camera’s response curve e: that image’s exposure value a: that image’s ambient light S: 0 if that pixel is in shadow at that time, 1 otherwise N: that pixel’s surface normal ρ: that pixel’s albedo
Heliometric Stereo Step 1: pixel-level thresholding to find shadows Step 2: initialize all variables I = f (e ρ (S L N + a))
Heliometric Stereo Step 3: fix f, e, and a, solve for ρ and N. Step 4: fix ρ and N, solve for f, e, and a. Step 5: goto 3. I = f (e ρ (S L N + a))
The life of a photon
BRDF Bi-Directional Reflectance Distribution Function given incoming and outgoing rays, what proportion of light is reflected?
BRDF Almost nobody actually tries to model a full BRDF. Why? Build a lighting model with fewer parameters that approximate the BRDF Diffuse lighting model is very common