Presentation is loading. Please wait.

Presentation is loading. Please wait.

Motion Tracking Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition CS4243Motion Tracking1.

Similar presentations

Presentation on theme: "Motion Tracking Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition CS4243Motion Tracking1."— Presentation transcript:

1 Motion Tracking Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition CS4243Motion Tracking1

2 CS42432Motion Tracking Changes are everywhere!

3 Illumination change CS4243Motion Tracking3

4 Shape change CS4243Motion Tracking4

5 Object motion CS4243Motion Tracking5

6 Camera motion CS4243Motion Tracking6

7 Object & camera motion CS4243Motion Tracking7

8 Everything changes! CS4243Motion Tracking8

9 Motion analysis is tough in general! We focus on object / camera motion. CS4243Motion Tracking9

10 Change Detection  Detects any change in two video frames.  Straightforward method:  Compute difference between corresponding pixels:  : intensity / colour at (x, y) in frame t.  If > threshold, has large difference. CS4243Motion Tracking10

11 Any difference? CS4243Motion Tracking11 No

12 Any difference? CS4243Motion Tracking12 Yes, illumination change

13 Any difference? CS4243Motion Tracking13 Yes, position change

14 Change Detection  Can detect  Illumination change  Position change  Illumination and position change  But, cannot distinguish between them.  Need to detect and measure position change. CS4243Motion Tracking14

15 Motion Tracking  Two approaches  Feature-based  Intensity gradient-based CS4243Motion Tracking15

16 Feature-based Motion Tracking  Look for distinct features that change positions.  Eagle’s wing tips change positions.  Tree tops don’t change positions. CS4243Motion Tracking16

17 Basic Ideas 1. Look for distinct features in current frame. 2. For each feature  Search for matching feature within neighbourhood in next frame.  Difference in positions  displacement.  Velocity = displacement / time difference. CS4243Motion Tracking17

18 Basic Ideas CS4243Motion Tracking18 feature area search area displacement

19 What feature to use?  Harris corner  Tomasi’s feature  Feature descriptors: SIFT, SURF, GLOH, etc.  Others CS4243Motion Tracking19

20 Summary  Simple algorithm.  Can be slow if search area is large.  Can constrain search area with prior knowledge. CS4243Motion Tracking20

21 Gradient-based Motion Tracking  Two basic assumptions  Intensity changes smoothly with position.  Intensity of object doesn’t change over time.  Suppose an object is in motion.  Change position (dx, dy) over time dt.  Then, from 2 nd assumption:  Apply Taylor’s series expansion: CS4243Motion Tracking21

22  Omit higher order terms and divide by dt  Denote  Then,  u, v are unknown.  2 unknowns, 1 equation: can’t solve! CS4243Motion Tracking22

23 Any difference in motion? CS4243Motion Tracking23

24 Any difference in motion? CS4243Motion Tracking24

25 Any difference in motion? CS4243Motion Tracking25

26 Any difference in motion? CS4243Motion Tracking26

27 Any difference in motion? CS4243Motion Tracking27

28 Any difference in motion? CS4243Motion Tracking28

29 Aperture Problem  Homogeneous region  I x = I y = I t = 0.  No change in local region.  Cannot detect motion. CS4243Motion Tracking29

30 Aperture Problem  Edge  I x and I y are zero along edge  Cannot measure motion tangential to edge  I x and I y are non-zero normal to edge  Can measure motion normal to edge  So, cannot measure actual motion CS4243Motion Tracking30

31 Aperture Problem  Corner  I x and I y are non-zero in two perpendicular directions  2 unknowns, 2 equations  Can measure actual motion CS4243Motion Tracking31

32 Lucas-Kanade Method  Consider two consecutive image frames I and J:  Object moves from x = (x, y) T to x + d.  d = (u, v) T CS4243Motion Tracking32 I x J x + d x d

33  So,  Or  Due to noise, there’s an error at position x:  Sum error over small window W at position x: CS4243Motion Tracking33 weight Similar to template matching

34  If E is small, patterns in I and J match well.  So, find d that minimises E:  Set  E /  d = 0, compute d that minimises E.  First, expand I(x – d) by Taylor’s series expansion:  Omit higher order terms:  Write in matrix form: CS4243Motion Tracking34 Intensity gradient

35  Now, error E at position x is:  Now, differentiate E with respect to d (exercise):  Setting  E /  d = 0 gives CS4243Motion Tracking35 the only unknown Z b

36  So, we get  2 unknowns, 2 equations. Can solve for d = (u, v).  What happen to aperture problem? Did it disappear? CS4243Motion Tracking36

37 Lucas-Kanade + Tomasi  Lucas-Kanade algorithm is often used with Tomasi’s feature  Apply Tomasi’s method to detect good features.  Apply LK method to compute d for each pixel.  Accept d only for good features. CS4243Motion Tracking37

38 Example Can you spot tracking errors? CS4243Motion Tracking38

39 Constraints  Math of LK tracker assumes d is small.  In implementation, W is also small.  LK tracker is good only for small displacement. CS4243Motion Tracking39

40 How to handle large displacement?  What if we scale down images?  Displacements are smaller! CS4243Motion Tracking40

41 Image Pyramid CS4243Motion Tracking41 downsample Usually, smoothen image with Gaussian filter before scaling down. input image scaled image

42 LK Tracking with Image Pyramid CS4243Motion Tracking42

43  Construct image pyramids.  Apply LK tracker to low-resolution images.  Propagate results to higher-resolution images.  Apply LK tracker to higher-resolution images. CS4243Motion Tracking43

44 Example Tracking results are more accurate. CS4243Motion Tracking44

45 Summary  Efficient algorithm, no explicit search.  Has aperture problem; track good features only  LK tracker can’t track large displacement.  Use LK + image pyramid for large displacement. CS4243Motion Tracking45

46 Software  OpenCV supports LK and LK with pyramid.  [Bir] offers LK with Tomasi’s features & pyramid. CS4243Motion Tracking46

47 Appendix  Calculation of I x, I y, I t  Use finite difference method  Forward difference I x = I(x  1, y, t)  I(x, y, t) I y = I(x, y  1, t)  I(x, y, t) I t = I(x, y, t  1)  I(x, y, t)  Backward difference I x = I(x, y, t)  I(x  1, y, t) I y = I(x, y, t)  I(x, y  1, t) I t = I(x, y, t)  I(x, y, t  1) CS4243Motion Tracking47

48 Further Readings  Lucas-Kanade tracking with pyramid: [BK08] Chapter 10.  Optical flow: [Sze10] Section 8.4.  Hierarchical motion estimation (with image pyramid): [Sze10] Section CS4243Motion Tracking48

49 References  [Bir] S. Birchfield. KLT: An implementation of the Kanade-Lucas- Tomasi feature tracker.  [BK08] Bradski and Kaehler. Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly,  [LK81] B. D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proceedings of 7th International Joint Conference on Artificial Intelligence, pages 674– 679,  [ST94] J. Shi and C. Tomasi. Good features to track. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 593–600,  [Sze10] R. Szeliski. Computer Vision: Algorithms and Applications. Springer, CS4243Motion Tracking49

50 References  [TK91] C. Tomasi and T. Kanade. Detection and tracking of point features. Technical Report CMU-CS , School of Computer Science, Carnegie Mellon University, CS4243Motion Tracking50

Download ppt "Motion Tracking Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition CS4243Motion Tracking1."

Similar presentations

Ads by Google