Multiple Light Source Optical Flow Multiple Light Source Optical Flow Robert J. Woodham ICCV’90

Introduction Optical Flow Definition Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other Main Idea Use the intensity color values recorded from multiple Images of moving objects acquired simultaneously under different illumination conditions to calculate optical flow

Some considerations Object Motion vs. brightness change Not purely geometric Depends on radiometric factors (illumination, reflectance) Not purely geometric Depends on radiometric factors (illumination, reflectance) The idea is based on... Photometric stereo uses multiple conditions of illumination to determine shape from shading

Theory Optical Flow Constraint Equation dE/dt=E x u + E y v + E t where E = E(x,y,t) be the image brightness at point (x,y) as a function of time t Ex =  E/  x, Ey =  E/  y, Et =  E/  t (partial derivatives of E with respect to x, y and t) u =dx/dt and v= dy/dt (instantaneous flow in the point (x,y).

Theory (2) If the brightness does not change as consequence of motion... E x u + E y v + E t = 0 Validity conditions Purely translational motion, Orthographic projection Uniform incident illumination

Theory (3) Equation properties Cannot be solved locally – 1 equation with 2 unknowns Variation in scene illumination cause dE/dt  0 Objects acts as indirect sources of illumination (inter-reflection) Locations of brightness discontinuity – undefined points. E x u + E y v + E t = 0

Using Multiple Light Sources E 1x u + E 1y v + E 1t = 0 E 2x u + E 2y v + E 2t = 0 : : For 2 light sources

3 Light Sources E 1x u + E 1y v + E 1t = 0 E 2x u + E 2y v + E 2t = 0 E 3x u + E 3y v + E 3t = 0 Ax = b x = [u,v] T b = -[E 1t,, E 2t,E 3t ] T x = (A T A) -1 A T b Standard Least Square solution Overdetermined Problem

Implementation 3 under different illumination condition at time t0 3 same illumination as time t0, with same background but different object position 6 images 3 images taken under different illumination condition in t0

Implementation (2) Multiple light source optical flow computation at one point u v 3 Flow constraint lines

Results Optical Flow vectors Estimation is good where the surface is smoothly shaded In the collar dark points degenerate the results In the discontinuities, due change of image brightness the estimates is also inaccurate Vector in the background due the shadows and inter-reflection

Practical Implementation Can be used 3 light sources (red, green and blue) continuously illuminating a workspace The capture can be made using cameras to capture different spectral channels

Conclusion The method works better in smoothly curves (not distinct surface markings and the local brightness depends on local shading) Restrictions in surface discontinuities and surface markings because local brightness change is dominated by scene features (largely independent of the illumination direction)