1. Announcements No midterm Project 3 will be done in pairs same partners as for project 2

2. Global Alignment and Structure from Motion Today’s Readings Photo Tourism (Snavely et al., SIGGRAPH 2006) –http://phototour.cs.washington.edu/Photo_Tourism.pdfhttp://phototour.cs.washington.edu/Photo_Tourism.pdf (Adapted from slides by Noah Snavely)

3. Problem: Drift Solution add another copy of first image at the end this gives a constraint: y n = y 1 there are a bunch of ways to solve this problem –add displacement of (y 1 – y n )/(n -1) to each image after the first –compute a global warp: y’ = y + ax –run a big optimization problem, incorporating this constraint »best solution, but more complicated »known as “bundle adjustment” (x 1,y 1 ) copy of first image (x n,y n )

4. I1I1 I2I2 I3I3 I4I4 Global optimization t1t1 (0,0) t2t2 t3t3 t4t4 Input Output p1p1 p2p2 p3p3 We want to estimate t i. We know p i how do t i relate to p i ?

5. I1I1 I2I2 I3I3 I4I4 p1p1 p2p2 p3p3 Global optimization Recipe 1.Identify the variables you want to estimate in our case: t i Identify a set of objectives you want to satisfy 1. in our case: p i - p j = t i - t j and similar for q, r, s Define an objective function F over these variables, whose minimum occurs at the “answer” for these variables Find the minimum of F q2q2 q3q3 q4q4 r3r3 r4r4 s4s4 s1s1

6. I1I1 I2I2 I3I3 I4I4 p1p1 p2p2 p3p3 Objective function + similar terms for q, r, s q2q2 q3q3 q4q4 r3r3 r4r4 s4s4 s1s1

7. p1p1 p2p2 p3p3 Objective function Matrix form p i = (u i, v i ) t i = (x i, y i )

8. Objective Function Adding in q, r, s give a larger matrix equation Axb Defines a least squares problem: minimize Solution: Problem: there are multiple solutions for ! (det = 0) We can add a global offset to a solution and get the same error

9. Ambiguity in the solution Each of these solutions has the same error Called the gauge ambiguity Solution: fix the translation of one image (t 1 = (0,0)) (0,0) (-100,-100) (200,-200)

10. Structure from motion Computed 3D structure Images on the Internet

11. Structure from motion Camera 1 Camera 2 Camera 3 R 1,t 1 R 2,t 2 R 3,t 3 p1p1 p4p4 p3p3 p2p2 p5p5 p6p6 p7p7 minimize g (R, T, P)g (R, T, P) aka “bundle adjustment” (texts: Zisserman; Faugeras)ZissermanFaugeras

12. Given point x and rotation and translation R, t Minimize sum of squared reprojection errors: predicted image location observed image location SfM objective function

13. Solving structure from motion Minimizing g is difficult: –g is non-linear due to rotations, perspective division –lots of parameters: 3 for each 3D point, 6 for each camera –difficult to initialize –gauge ambiguity: error is invariant to a similarity transform (translation, rotation, uniform scale) Many techniques use non-linear least-squares optimization (bundle adjustment) –Levenberg-Marquardt is a popular algorithm –http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithmhttp://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm Good code online –Bundler: http://phototour.cs.washington.edu/bundler/http://phototour.cs.washington.edu/bundler/ –Multicore: http://grail.cs.washington.edu/projects/mcba/http://grail.cs.washington.edu/projects/mcba/

14. Photo Tourism Photo tourism video: http://www.youtube.com/watch?v=5Ji84zb2r8s http://www.youtube.com/watch?v=5Ji84zb2r8s Photosynth: http://photosynth.net/ http://photosynth.net/

21. ? ?

22. Reconstructing Rome In a day... From ~1M images Using ~1000 cores Sameer Agarwal, Noah Snavely, Rick Szeliski, Steve Seitz http://grail.cs.washington.edu/rome

23. St. Peters (inside) Trevi Fountain St. Peters (outside) Il Vittoriano Coliseum (inside) Coliseum (outside) Forum

24. Rome 150K: Colosseum

25. Rome: St. Peters

26. Venice (250K images)

27. Venice: Canal

28. Dubrovnik

29. More info Rome-in-a-day page – http://grail.cs.washington.edu/rome http://grail.cs.washington.edu/rome