Download presentation

1
**Computer Vision Optical Flow**

Some slides from K. H. Shafique [ and T. Darrell

2
**Correspondence Which pixel went where? Time: t Time: t + dt**

Bahadir K. Gunturk EE Image Analysis II

3
**Motion Field vs. Optical Flow**

Scene flow: 3D velocities of scene points. Motion field: 2D projection of scene flow. Optical flow: Approximation of motion field derived from apparent motion of brightness patterns in image. Bahadir K. Gunturk EE Image Analysis II

4
**Motion Field vs. Optical Flow**

Consider perfectly uniform sphere rotating in front of camera. Motion field follows surface points. Optical flow is zero since motion is not visible. Now consider stationary sphere with moving light source. Motion field is zero. But optical flow results from changing shading. But, in general, optical flow is a reliable indicator of motion field. Bahadir K. Gunturk EE Image Analysis II

5
**Applications Correct for camera jitter (stabilization)**

Object tracking Video compression Structure from motion Segmentation Correct for camera jitter (stabilization) Combining overlapping images (panoramic image construction) … Bahadir K. Gunturk EE Image Analysis II

6
Optical Flow Problem How to estimate pixel motion from one image to another? Bahadir K. Gunturk EE Image Analysis II

7
**Computing Optical Flow**

Assumption 1: Brightness is constant. Assumption 2: Motion is small. (from Taylor series expansion) Bahadir K. Gunturk EE Image Analysis II

8
**Computing Optical Flow**

Combine In the limit as u and v goes to zero, the equation becomes exact (optical flow equation) Bahadir K. Gunturk EE Image Analysis II

9
**Computing Optical Flow**

At each pixel, we have one equation, two unknowns. This means that only the flow component in the gradient direction can be determined. (optical flow equation) The motion is parallel to the edge, and it cannot be determined. This is called the aperture problem. Bahadir K. Gunturk EE Image Analysis II

10
**Computing Optical Flow**

We need more constraints. The most commonly used assumption is that optical flow changes smoothly locally. Bahadir K. Gunturk EE Image Analysis II

11
**Computing Optical Flow**

One method: The (u,v) is constant within a small neighborhood of a pixel. Optical flow equation: Use a 5x5 window: Two unknowns, 25 equation ! Bahadir K. Gunturk EE Image Analysis II

12
**Computing Optical Flow**

Find minimum least squares solution Lucas – Kanade method Bahadir K. Gunturk EE Image Analysis II

13
**Computing Optical Flow**

Lucas – Kanade When is This Solvable? ATA should be invertible ATA should not be too small. Other wise noise will be amplified when inverted.) Bahadir K. Gunturk EE Image Analysis II

14
**Computing Optical Flow**

What are the potential causes of errors in this procedure? Brightness constancy is not satisfied The motion is not small A point does not move like its neighbors window size is too large Bahadir K. Gunturk EE Image Analysis II

15
**Improving accuracy Recall our small motion assumption**

This is not exact To do better, we need to add higher order terms back in: This is a polynomial root finding problem Can solve using Newton’s method Also known as Newton-Raphson method Lucas-Kanade method does one iteration of Newton’s method Better results are obtained via more iterations Bahadir K. Gunturk EE Image Analysis II

16
**Iterative Refinement Iterative Lucas-Kanade Algorithm**

Estimate velocity at each pixel by solving Lucas-Kanade equations Warp H towards I using the estimated flow field - use image warping techniques Repeat until convergence Bahadir K. Gunturk EE Image Analysis II

17
**Revisiting the small motion assumption**

What can we do when the motion is not small? Bahadir K. Gunturk EE Image Analysis II

18
Reduce the resolution! Bahadir K. Gunturk EE Image Analysis II

19
**Coarse-to-fine optical flow estimation**

Gaussian pyramid of image H Gaussian pyramid of image I image I image H u=10 pixels u=5 pixels u=2.5 pixels u=1.25 pixels image H image I Bahadir K. Gunturk EE Image Analysis II

20
**Coarse-to-fine optical flow estimation**

Gaussian pyramid of image H Gaussian pyramid of image I image I image H run iterative L-K warp & upsample run iterative L-K . Bahadir K. Gunturk EE Image Analysis II

21
**Multi-resolution Lucas-Kanade Algorithm**

Bahadir K. Gunturk EE Image Analysis II

22
**Block-Based Motion Estimation**

Bahadir K. Gunturk EE Image Analysis II

23
**Block-Based Motion Estimation**

Bahadir K. Gunturk EE Image Analysis II

24
**Block-Based Motion Estimation**

Gaussian pyramid of image H Gaussian pyramid of image I image I image H Hierarchical search image H image I Bahadir K. Gunturk EE Image Analysis II

25
**Parametric (Global) Motion**

Sometimes few parameters are enough to represent the motion of whole image translation rotation scale Bahadir K. Gunturk EE Image Analysis II

26
**Parametric (Global) Motion**

Affine Flow Bahadir K. Gunturk EE Image Analysis II

27
**Parametric (Global) Motion**

Affine Flow Bahadir K. Gunturk EE Image Analysis II

28
**Parametric (Global) Motion**

Affine Flow – Approach Bahadir K. Gunturk EE Image Analysis II

29
**Iterative Refinement Iterative Estimation**

Estimate parameters by solving the linear system. Warp H towards I using the estimated flow field - use image warping techniques Repeat until convergence or a fixed number of iterations Bahadir K. Gunturk EE Image Analysis II

30
**Coarse-to-fine global flow estimation**

Gaussian pyramid of image H Gaussian pyramid of image I image I image H Compute Flow Iteratively warp & upsample Compute Flow Iteratively . image J image I Bahadir K. Gunturk EE Image Analysis II

31
**Global Flow Find features Match features Fit parametric model**

Application: Mosaic construction Bahadir K. Gunturk EE Image Analysis II

32
Image Warping Bahadir K. Gunturk EE Image Analysis II

33
**Image Warping Interpolate to find the intensity at (x’,y’)**

Pixel values are known in this image Pixel values are to found in this image Bahadir K. Gunturk EE Image Analysis II

34
**Other Parametric Motion Models**

Perspective and Pseudo-Perspective: Approximation to perspective. (Planar scene + Perspective projection) Bahadir K. Gunturk EE Image Analysis II

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google