2. Determining 3D Structure and Motion of Man-made Objects from Corners

3. A great number of work has been done on motion estimation from feature correspondence since 1970’s. The features used mostly: Point & Line Here, we use the combination of point and line, i.e., Corner Determining 3D Structure and Motion of Man-made Objects from Corners

4.  First, the 3D structure of a corner is recovered easily from its image by introducing a new coordinate system ;  Then, the rotation matrix R and translation vector T are computed from the recovered 3D corner correspondence ;  Finally, it is concluded that one corner and two points correspondences over two views are sufficient to uniquely determine the motion.

5. 1. Representations of 3D and image corners  3D corner:  Image corner: is the normal of the projecting plane of the edge line:

6. 2. Recovering a 3D orthogonal corner from a single view Given the image corner of a 3D orthogonal corner: to recover the 3D structure of the corner: Introduce a new coordinate system such that

7. z x y o w v u k l p0p0 P0P0 L

8. Suppose that the edge line of a 3D corner has a slope in the coordinate then the direction of the 3D edge line in the coordinate system is The axis of the coordinate has coordinate in its own coordinate system However, in the, has the same direction with. So, (1)

9. Similarly, Write in a matrix form:

10. From Eq.(1), the direction of the correspondent edge line in the For orthogonal corners, (2) Substitute (2) into the above equations, and we can get three equations about the slope value

11. After and are found, the directions of the edge lines of the 3D corner can be easily computed Thus the 3D corner is reconstructed by Where is the 3D depth of the vertex of the corner, which cannot be determined from a single view.

12. 3. Determining the rotation matrix R From the image corner correspondence, To reconstruct

13. The directions of the edge lines of the corner at time and are related by are orthogonal unit vectors Remark: since we get two sets of slope values, that means we recover two 3D corners from one image corner, so four rotation matrix can be computed. Therefore, additional information is needed to determine a unique solution.

14. 4. Determining the translation vector T Since the rotation matrix R has been computed, we can eliminate the rotation motion in the whole motion. Then there is only the motion of translation. Time t1 Time t2 Intermediate time

15. Following, we suppose there is only translation motion between time t1 and t2. Remark: It is impossible to uniquely determine a translation from a single corner correspondence over two views of images. The rank of the coefficient matrix of the equations for translation is always less than 3. Proof: A maximum of 4 equations can be derived from a single corner over two views of images: the three equations from edge lines, the other one from the vertex. Edge lines of the corner satisfy the equations : (3)

16. Equations (3) and (4) are four linear equations about the unknown T, but they are not independent. The rank of the coefficient matrix is only 2. Another image point not lying in any of the three edge lines is needed to determine the translation. (4) the other equation for the vertex of the corner is: where

17. 5. Getting a unique solution Images over two frames Four rotation matrix R A corner correspondence One translation T for each R a corner and a nonsingular point correspondence Unique solution R and T Another nonsingular point correspondence

18. Uniqueness  If a 3D motion is a pure rotation, an orthogonal corner correspondence and a nonsingular point correspondence over two frames can uniquely determine the motion.  If a 3D motion is a pure translation, an orthogonal corner correspondence and a nonsingular point correspondence over two frames can uniquely determine the motion.  If a 3D motion is a rotation followed by a translation, an orthogonal corner correspondence and two point correspondences can uniquely determine the motion.

19. 6. Motion estimation from a corner with known space angles The process is the same as that in the orthogonal corner case. The only difference is : Orthogonal corner Corner with angles: