Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computer vision: models, learning and inference Chapter 15 Models for transformations.

Similar presentations


Presentation on theme: "Computer vision: models, learning and inference Chapter 15 Models for transformations."— Presentation transcript:

1 Computer vision: models, learning and inference Chapter 15 Models for transformations

2 Structure 22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

3 Transformation models 33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Consider viewing a planar scene There is now a 1 to 1 mapping between points on the plane and points in the image We will investigate models for this 1 to 1 mapping – Euclidean – Similarity – Affine transform – Homography

4 Motivation: augmented reality tracking 44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

5 Consider viewing a fronto-parallel plane at a known distance D. In homogeneous coordinates, the imaging equations are: Euclidean Transformation 55Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3D rotation matrix becomes 2D (in plane) Plane at known distance D Point is on plane (w=0)

6 Euclidean transformation 66Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Simplifying Rearranging the last equation

7 Euclidean transformation 77Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Simplifying Rearranging the last equation

8 Euclidean transformation 88Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Pre-multiplying by inverse of (modified) intrinsic matrix

9 Euclidean transformation 99Computer vision: models, learning and inference. ©2011 Simon J.D. Prince or for short: Homogeneous: Cartesian: For short:

10 Similarity Transformation 10 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Consider viewing fronto-parallel plane at unknown distance D By same logic as before we have Premultiplying by inverse of intrinsic matrix

11 Similarity Transformation 11 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Simplifying: Multiply each equation by :

12 Similarity Transformation 12 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Simplifying: Incorporate the constants by defining:

13 Similarity Transformation 13 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Homogeneous: Cartesian: For short:

14 Affine Transformation 14 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Homogeneous: Cartesian: For short:

15 Affine Transform 15 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Affine transform describes mapping well when the depth variation within the planar object is small and the camera is far away When variation in depth is comparable to distance to object then the affine transformation is not a good model. Here we need the homography

16 Projective transformation / collinearity / homography 16 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Start with basic projection equation: Combining these two matrices we get:

17 Homography 17 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Homogeneous: Cartesian: For short:

18 Modeling for noise 18 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince In the real world, the measured image positions are uncertain. We model this with a normal distribution e.g.

19 Structure 19 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

20 Learning and inference problems 20 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Learning – take points on plane and their projections into the image and learn transformation parameters Inference – take the projection of a point in the image and establish point on plane

21 Learning and inference problems 21 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

22 Maximum likelihood approach Becomes a least squares problem Learning transformation models 22 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

23 Learning Euclidean parameters 23 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Solve for transformation: Remaining problem:

24 This is an orthogonal Procrustes problem. To solve: Compute SVD And then set Learning Euclidean parameters 24 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Has the general form:

25 Learning similarity parameters 25 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Solve for rotation matrix as before Solve for translation and scaling factor

26 Learning affine parameters 26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Affine transform is linear Solve using least-squares solution

27 Learning homography parameters 27 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Homography is not linear – cannot be solved in closed form. Convert to other homogeneous coordinates Both sides are 3x1 vectors; should be parallel, so cross product will be zero

28 Write out these equations in full There are only 2 independent equations here – use a minimum of four points to build up a set of equations Learning homography parameters 28 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

29 These equations have the form, which we need to solve with the constraint This is a “minimum direction” problem – Compute SVD – Take last column of Then use non-linear optimization Learning homography parameters 29 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

30 Inference problems 30 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Given point x* in image, find position w* on object

31 In the absence of noise, we have the relation, or in homogeneous coordinates Inference 31 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince To solve for the points, we simply invert this relation

32 Structure 32 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

33 Problem 1: exterior orientation 33 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

34 Problem 1: exterior orientation 34 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Writing out the camera equations in full Estimate the homography from matched points Factor out the intrinsic parameters

35 Problem 1: exterior orientation 35 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince To estimate the first two columns of rotation matrix, we compute this singular value decomposition Then we set

36 Problem 1: exterior orientation 36 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Find the last column using the cross product of first two columns Make sure the determinant is 1. If it is -1, then multiply last column by -1. Find translation scaling factor between old and new values Finally, set

37 Structure 37 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

38 Problem 2: calibration 38 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

39 Structure 39 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

40 One approach (not very efficient) is to alternately Optimize extrinsic parameters for fixed intrinsic Optimize intrinsic parameters for fixed extrinsic Calibration 40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Then use non-linear optimization.

41 Structure 41 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

42 Problem 3 - reconstruction 42 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation between plane and image: Point in frame of reference of plane: Point in frame of reference of camera

43 Structure 43 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

44 Transformations between images 44 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince So far we have considered transformations between the image and a plane in the world Now consider two cameras viewing the same plane There is a homography between camera 1 and the plane and a second homography between camera 2 and the plane It follows that the relation between the two images is also a homography

45 Properties of the homography 45 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Homography is a linear transformation of a ray Equivalently, leave rays and linearly transform image plane – all images formed by all planes that cut the same ray bundle are related by homographies.

46 Camera under pure rotation 46 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Special case is camera under pure rotation. Homography can be showed to be

47 Structure 47 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

48 Robust estimation 48 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Least squares criterion is not robust to outliers For example, the two outliers here cause the fitted line to be quite wrong. One approach to fitting under these circumstances is to use RANSAC – “Random sampling by consensus”

49 RANSAC 49 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

50 RANSAC 50 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

51 Fitting a homography with RANSAC 51 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Original imagesInitial matchesInliers from RANSAC

52 Piecewise planarity 52 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Many scenes are not planar, but are nonetheless piecewise planar Can we match all of the planes to one another?

53 Approach 1 – Sequential RANSAC 53 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Problems: greedy algorithm and no need to be spatially coherent

54 Approach 2 – PEaRL (propose, estimate and re-learn) 54 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Associate label l which indicates which plane we are in Relation between points x i in image1 and y i in image 2 Prior on labels is a Markov random field that encourages nearby labels to be similar Model solved with variation of alpha expansion algorithm

55 Approach 2 – PEaRL 55 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

56 Structure 56 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Transformation models Learning and inference in transformation models Three problems – Exterior orientation – Calibration – Reconstruction Properties of the homography Robust estimation of transformations Applications

57 Augmented reality tracking 57 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

58 Fast matching of keypoints 58 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

59 Visual panoramas 59 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

60 Conclusions Mapping between plane in world and camera is one-to-one Takes various forms, but most general is the homography Revisited exterior orientation, calibration, reconstruction problems from planes Use robust methods to estimate in the presence of outliers 60 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince


Download ppt "Computer vision: models, learning and inference Chapter 15 Models for transformations."

Similar presentations


Ads by Google