Structure From Motion Sebastian Thrun, Gary Bradski, Daniel Russakoff

Slides:

Advertisements

Similar presentations

Structure from motion.

Advertisements

The fundamental matrix F

Structure from motion Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/4/20 with slides by Richard Szeliski, Steve Seitz, Zhengyou Zhang and Marc.

MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.

3D Reconstruction – Factorization Method Seong-Wook Joo KG-VISA 3/10/2004.

Two-View Geometry CS Sastry and Yang

1 pb.  camera model  calibration  separation (int/ext)  pose Don’t get lost! What are we doing? Projective geometry Numerical tools Uncalibrated cameras.

N-view factorization and bundle adjustment CMPUT 613.

Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.

Structure from motion.

Projective structure from motion

Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.

Stanford CS223B Computer Vision, Winter 2005 Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and.

1 Structure from Motion with Unknown Correspondence (aka: How to combine EM, MCMC, Nonlinear LS!) Frank Dellaert, Steve Seitz, Chuck Thorpe, and Sebastian.

Today Feature Tracking Structure from Motion on Monday (1/29)

Stanford CS223B Computer Vision, Winter 2007 Lecture 8 Structure From Motion Professors Sebastian Thrun and Jana Košecká CAs: Vaibhav Vaish and David Stavens.

Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I

Computer Vision Projective structure from motion Marc Pollefeys COMP 256 Some slides and illustrations from J. Ponce, …

Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.

Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.

Uncalibrated Geometry & Stratification Sastry and Yang

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado.

Multiple-view Reconstruction from Points and Lines

3D reconstruction of cameras and structure x i = PX i x’ i = P’X i.

Many slides and illustrations from J. Ponce

Uncalibrated Epipolar - Calibration

Lecture 11: Structure from motion CS6670: Computer Vision Noah Snavely.

Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.

Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.

Computer Vision Structure from motion Marc Pollefeys COMP 256 Some slides and illustrations from J. Ponce, A. Zisserman, R. Hartley, Luc Van Gool, …

Multiple View Reconstruction Class 23 Multiple View Geometry Comp Marc Pollefeys.

Projected image of a cube. Classical Calibration.

CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.

Affine structure from motion

Advanced Computer Vision Structure from Motion. Geometric structure-from-motion problem: using image matches to estimate: The 3D positions of the corresponding.

Stereo Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with slides by James Rehg and.

CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.

Lecture 12: Structure from motion CS6670: Computer Vision Noah Snavely.

Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2006 Lecture 4 Camera Calibration Professor Sebastian Thrun.

Computer vision: models, learning and inference

Euclidean cameras and strong (Euclidean) calibration Intrinsic and extrinsic parameters Linear least-squares methods Linear calibration Degenerate point.

Geometry and Algebra of Multiple Views

Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –

Structure from Motion Computer Vision CS 143, Brown James Hays 11/18/11 Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz,

Structure from Motion Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/06/12 Many slides adapted from Lana Lazebnik, Silvio Saverese,

Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration Sebastian Thrun,

CSCE 643 Computer Vision: Structure from Motion

Affine Structure from Motion

Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from.

Advanced Computer Vision Structure from Motion1 Chapter 7 S TRUCTURE FROM M OTION.

EECS 274 Computer Vision Affine Structure from Motion.

776 Computer Vision Jan-Michael Frahm & Enrique Dunn Spring 2013.

Structure from Motion ECE 847: Digital Image Processing

Reconstruction from Two Calibrated Views Two-View Geometry

Geometry Reconstruction March 22, Fundamental Matrix An important problem: Determine the epipolar geometry. That is, the correspondence between.

Structure from Motion Paul Heckbert, Nov , Image-Based Modeling and Rendering.

Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : –

EECS 274 Computer Vision Projective Structure from Motion.

Structure from Motion. For now, static scene and moving cameraFor now, static scene and moving camera – Equivalently, rigidly moving scene and static.

Lecture 22: Structure from motion CS6670: Computer Vision Noah Snavely.

Reconstruction of a Scene with Multiple Linearly Moving Objects Mei Han and Takeo Kanade CISC 849.

Structure from Motion ECE 847: Digital Image Processing

Structure from motion II

Epipolar geometry.

Structure from motion Input: Output: (Tomasi and Kanade)

Multiway Cut for Stereo and Motion with Slanted Surfaces

Advanced Computer Vision

Uncalibrated Geometry & Stratification

Structure from motion.

Structure from motion Input: Output: (Tomasi and Kanade)

Presentation transcript:

Structure From Motion Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision http://robots.stanford.edu/cs223b

Structure From Motion (1) [Tomasi & Kanade 92]

Structure From Motion (2) [Tomasi & Kanade 92]

Structure From Motion (3) [Tomasi & Kanade 92]

Structure From Motion Problem 1: Problem 2: Given n points pij =(xij, yij) in m images Reconstruct structure: 3-D locations Pj =(xj, yj, zj) Reconstruct camera positions (extrinsics) Mi=(Aj, bj) Problem 2: Establish correspondence: c(pij)

Orthographic Camera Model Limit of Pinhole Model: Extrinsic Parameters Rotation Orthographic Projection

Orthographic Projection Limit of Pinhole Model: Orthographic Projection

The Affine SFM Problem

Count # Constraints vs #Unknowns m camera poses n points 2mn point constraints 8m+3n unknowns Suggests: need 2mn  8m + 3n But: Can we really recover all parameters???

How Many Parameters Can’t We Recover? We can recover all but… 3 6 8 9 10 12 n m nm Place Your Bet!

The Answer is (at least): 12

Points for Solving Affine SFM Problem m camera poses n points Need to have: 2mn  8m + 3n-12

Affine SFM Fix coordinate system by making p0=origin Rank Theorem: Proof: Rank Theorem: Q has rank 3

The Rank Theorem 2m elements n elements

Tomasi/Kanade 1992 Singular Value Decomposition

Tomasi/Kanade 1992 Gives also the optimal affine reconstruction under noise

Back To Orthographic Projection Find C and d for which constraints are met

Back To Projective Geometry Orthographic (in the limit) Projective

Projective Camera: Non-Linear Optimization Problem: Bundle Adjustment!

Structure From Motion Problem 1: Problem 2: Given n points pij =(xij, yij) in m images Reconstruct structure: 3-D locations Pj =(xj, yj, zj) Reconstruct camera positions (extrinsics) Mi=(Aj, bj) Problem 2: Establish correspondence: c(pij)

The Correspondence Problem View 1 View 2 View 3

Correspondence: Solution 1 Track features (e.g., optical flow) …but fails when images taken from widely different poses

Correspondence: Solution 2 Start with random solution A, b, P Compute soft correspondence: p(c|A,b,P) Plug soft correspondence into SFM Reiterate See Dellaert/Seitz/Thorpe/Thrun 2003

Example

Results: Cube

Animation

Tomasi’s Benchmark Problem

Reconstruction with EM

3-D Structure