Presentation is loading. Please wait.

Presentation is loading. Please wait.

875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Spring 2014.

Similar presentations


Presentation on theme: "875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Spring 2014."— Presentation transcript:

1 875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Spring 2014

2 Introductions 2

3 Grade Requirements Presentation of 2 papers in class  30 min talk,  10 min questions Papers for selection must come from:  top journals: IJCV, PAMI, CVIU, IVCJ  top conferences: CVPR (2010,2011), ICCV (2011), ECCV (2010),  approval for all other venues is needed Final project  evaluation, extension of a recent method from the above 3

4 Grading 20% first presentation 20% second presentation 30% final project 30% attendance & class participation 4

5 Schedule Jan. 7 th, Introduction Jan 7 th, Uncertainty in Stereo (guest Philippos Mordohai) (substitute for Jan 13 th class) Jan 15 th, Large-scale image localization basic concepts, First paper selection (Large –scale localization) Jan 20 th, MLK holiday no class Jan 22 nd -29th, Large-scale localization basic concepts Feb. 3 rd, 1. round of presentations starts Mar. 10 th, 12 th Spring break (no class) Mar. 17 th, Modeling dynamic objects/scenes basic concepts, Second paper selection, final project definition Mar. 19 st, Modeling dynamic objects Mar. 24 th, 2. round of presentations starts Apr. 21 st, 23 rd, final project presentation 5

6 How to give a great presentation Structure of the talk:  Motivation (motivate and explain the problem)  Overview  Related work (short concise discussion)  Approach  Experiments  Conclusion and future work 6

7 How to give a great presentation Use large enough fonts  5-6 one line bullet items on a slide max Keep it simple No complex formulas in your talk Bad Powerpoint slides How to for presentations 7

8 How to give a great presentation Abstract the material of the talk  provide understanding beyond details Use pictures to illustrate  find pictures on the internet  create a graphic (in ppt, graph tool)  animate complex pictures 8

9 How to give a good presentation Avoid bad color schemes  no red on blue looks awful Avoid using laser pointer (especially if you are nervous) Add pointing elements in your presentation Practice to stay within your time! Don’t rush through the talk! 9

10 Brush up on Stereo Reconstruction 10

11 Stereo Extraction of 3D information from 2D images 11 Images3D Point Cloud Stereo

12 Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image  Humans can do it Stereograms: Invented by Sir Charles Wheatstone, 1838

13 Depth Recovery by Stereo reference imagematching image Depth d1d1 d2d2 d3d3 d4d4 d5d5 d6d6 d7d7 d8d8 d9d9 Search Space Epipolar line 13

14 Depth Recovery from Stereo reference imagematching image Depth d1d1 d2d2 d3d3 d4d4 d5d5 d6d6 d7d7 d8d8 d9d9 Search Space Epipolar line depth Pixel similarity: measured by color differences Matching Cost 14 Ground Truth Pixel Matching Depth Map

15 Matching criteria Raw pixel values (correlation) Band-pass filtered images [Jones & Malik 92] “ Corner ” like features [Zhang, …] Edges [many people…] Gradients [Seitz 89; Scharstein 94] Rank statistics [Zabih & Woodfill 94] Intervals [Birchfield and Tomasi 96] Overview of matching metrics and their performance:  H. Hirschmüller and D. Scharstein, “Evaluation of Stereo Matching Costs on Images with Radiometric Differences”, PAMI 2008 slide: R. Szeliski

16 Adaptive Weighting Boundary Preserving More Costly

17 Simplest Case: Parallel images Image planes of cameras are parallel to each other and to the baseline Camera centers are at same height Focal lengths are the same slide: S. Lazebnik

18 Simplest Case: Parallel images Image planes of cameras are parallel to each other and to the baseline Camera centers are at same height Focal lengths are the same Then, epipolar lines fall along the horizontal scan lines of the images slide: S. Lazebnik

19 Essential matrix for parallel images R = I t = (T, 0, 0) Epipolar constraint: t x x’

20 Essential matrix for parallel images Epipolar constraint: R = I t = (T, 0, 0) t x x’

21 Aggregation Structure depth Matching Cost Pixelwise Costs 21 Search Space

22 Aggregation Structure Cost Volume Cost aggregation: cutting the cost volume. 22 Search Space

23 Aggregation Structure Cost Volume Fronto-Parallel Plane 23 Treat neighbors equally Cost of the center pixel Costs of neighboring pixels Sum of Absolute Differences (SAD) Depth Map

24 Aggregation Structure Adaptive Weight Yoon and Kweon, PAMI 2006 Depth MapCost Volume 24 Color differences Spatial distances Weighted cost of the center pixel Weighted costs of neighboring pixels

25 Aggregation Structure Adaptive Weight Depth Map Oriented Plane Cost Volume Lu et al., CVPR

26 Stereo: epipolar geometry Match features along epipolar lines viewing ray epipolar plane epipolar line slide: R. Szeliski

27 Your basic stereo algorithm For each epipolar line For each pixel in the left image compare with every pixel on same epipolar line in right image pick pixel with minimum match cost Improvement: match windows This should look familar... slide: R. Szeliski

28 Depth Map Computation Local methods  Depth with the minimum cost  Complexity: Global methods  Pairwise interactions  Complexity: Scharstein and Szeliski, “A taxonomy and evaluation of dense two- frame stereo correspondence algorithms", IJCV 2002 Image Resolution : the total number of pixels 28 N pixels aN pixels bN pixels

29 Depth from disparity f xx’ Baseline B z OO’ X f Disparity is inversely proportional to depth!

30 Depth Sampling Depth sampling for integer pixel disparity Quadratic precision loss with depth!

31 Depth Sampling Depth sampling for wider baseline

32 Depth Sampling Depth sampling is in O(resolution 6 )

33 Finding correspondences apply feature matching criterion (e.g., correlation or Lucas-Kanade) at all pixels simultaneously search only over epipolar lines (many fewer candidate positions) slide: R. Szeliski

34 Matching cost disparity LeftRight scanline Correspondence search Slide a window along the right scanline and compare contents of that window with the reference window in the left image Matching cost: SSD or normalized correlation slide: S. Lazebnik

35 LeftRight scanline Correspondence search SSD slide: S. Lazebnik

36 LeftRight scanline Correspondence search Norm. corr slide: S. Lazebnik

37 Neighborhood size Smaller neighborhood: more details Larger neighborhood: fewer isolated mistakes w = 3w = 20 slide: R. Szeliski

38 Failures of correspondence search Textureless surfaces Occlusions, repetition Non-Lambertian surfaces, specularities slide: S. Lazebnik

39 How can we improve window-based matching? The similarity constraint is local (each reference window is matched independently) Need to enforce non-local correspondence constraints slide: S. Lazebnik

40 Non-local constraints Uniqueness  For any point in one image, there should be at most one matching point in the other image slide: S. Lazebnik

41 Non-local constraints Uniqueness  For any point in one image, there should be at most one matching point in the other image Ordering  Corresponding points should be in the same order in both views slide: S. Lazebnik

42 Non-local constraints Uniqueness  For any point in one image, there should be at most one matching point in the other image Ordering  Corresponding points should be in the same order in both views Ordering constraint doesn’t hold slide: S. Lazebnik

43 Non-local constraints Uniqueness  For any point in one image, there should be at most one matching point in the other image Ordering  Corresponding points should be in the same order in both views Smoothness  We expect disparity values to change slowly (for the most part) slide: S. Lazebnik

44 I1I2I10 Multiple-baseline stereo results M. Okutomi and T. Kanade, “A Multiple-Baseline Stereo System,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(4): (1993).“A Multiple-Baseline Stereo System,”

45 Plane Sweep Stereo Choose a reference view Sweep family of planes at different depths with respect to the reference camera Each plane defines a homography warping each input image into the reference view reference camera input image R. Collins. A space-sweep approach to true multi-image matching. CVPR 1996.A space-sweep approach to true multi-image matching. input image

46 Real-time 3D reconstruction from video “Real-Time Plane-sweeping Stereo with Multiple Sweeping Directions", CVPR D scene SAD as similarity (darker is higher similarity) warped images 46

47 Real-time 3D reconstruction from video 47 “Real-Time Plane-sweeping Stereo with Multiple Sweeping Directions", CVPR D scene warped images SAD as similarity (darker is higher similarity)

48 Real-time 3D reconstruction from video 48 “Real-Time Plane-sweeping Stereo with Multiple Sweeping Directions", CVPR D scene warped images SAD as similarity (darker is higher similarity)

49 Real-time 3D reconstruction from video 49 “Real-Time Plane-sweeping Stereo with Multiple Sweeping Directions", CVPR D scene warped images SAD as similarity (darker is higher similarity) Multi-way sweep

50 3D reconstruction from video view 1view N 50

51 3D reconstruction from video 51


Download ppt "875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Spring 2014."

Similar presentations


Ads by Google