1. Planar Matchmove Using Invariant Image Features Andrew Kaufman

2. I will talk about Matchmove Problem “Augmented Reality Using Un-calibrated Video Sequences” – Cornelis et al. My project

3. Matchmove Goal: Insert virtual objects into real video sequences Applications  Film  Advertising  Sports  Augmented Reality Systems

4. AR from Video Problems with inserting virtual objects  Rigid registration  Occlusion  Proper illumination

5. AR from Video Problems with inserting virtual objects  Rigid registration  Occlusion  Proper illumination

6. Rigid Registration Main Difficulty  Jittering 3D Reconstruction Scheme  Extract motion and structure

7. Motion and Structure Recovery (MSR) Based on feature tracking  Points, lines, curves, regions  Harris Corner Detector points / corners with  Significant orthogonal intensity change

8. Corner Matching Initially  Using small search region  NCC along epipolar line RANSAC algorithm  Treats small movements as outliers Levenberg-Marquardt optimization

9. Initializing MSR m k ~ P k M P k projects 3D point onto image k  To scale factor Camera for two initial images  P 1 – world frame  P 2 – based on epipolar geometry Crude 3D reconstruction of matched corners

10. Finishing MSR For all other images  Image i has P i Retrieve m i -M corner matches  Image j Compute m i -m j corner matches Obtain m j -M matches  Determine P j

11. Attach Graphics Set virtual camera using P Roughly place graphics into the crude 3D environment Render and overlay

12. Review Initialize with two images Take 1 processed and 1 new image  Find new 2D-3D matches  Calculate new P  Construct new 3D points For each remaining frame  Find new 2D-3D matches  Calculate new P Place virtual object LOOP

13. Examples Static virtual objects Moving virtual objects Moving real objects http://homes.esat.kuleuven.be/~kcorneli/ARVi deos/ http://homes.esat.kuleuven.be/~kcorneli/ARVi deos/

14. Future Work Occlusions Illumination Reduce jittering  Smooth camera path  Lock to real object

15. My Project Planar Matchmove Track object through video  Matching SIFT features Solve for affine transformation between reference image and video frames Create graphics UI to attach graphics to reference image Transform graphics using affine parameters Render final video

16. Graphic Goals Boarder around object Billboard over object Attach special FX

17. Current Progress Objects tracked Affine parameters found Boarders drawn http://cs.ubc.ca/~akaufman/vision/

18. Still to do Build User Interface Create graphics Render final videos

19. Questions ???