Optical Flow 10-24-2005

Problem Problems in motion estimation Approaches: Applications Noise, color (intensity) smoothness, lighting (shadowing effects), occlusion, abrupt movements, etc Approaches: Block matching, Generalized block matching, Optical flow (block-based, Horn-Schunck etc) Bayesian, etc. Applications Video coding and compression, Segmentation Object reconstruction (structure-from-motion) Detection and tracking, etc.

Motion description î í ì = Y y X x 2D motion: p = [x(t),y(t)]  p’= [x(t+ t0), y(t+t0)]  d(t) = [x(t+ t0)-x(t),y(t+t0)-y(t)]  3D motion: Α = [ Χ1, Υ1, Ζ1 ]Τ  Β = [ Χ2, Υ2, Ζ2 ]Τ  = R + T Basic projection models: Orthographic Perspective î í ì = Y y X x

Optical Flow Basic assumptions: Normal flow: Image is smooth locally Pixel intensity does not change over time (no lighting changes) Normal flow: Second order differential equation:

Block-based Optical Flow Estimation Optical flow estimation within a block (smoothness assumption): all pixels of the block have the same motion Error: Motion equation: and

Horn-Schunck We want an optical flow field that satisfies the Optical Flow Equation with the minimum variance between the vectors (smoothness) Gauss-Seidel

Derivative Estimation with Finite differences

Example 1

Example 2

Example 3: frame reconstruction

Application Examples