Geometric Transformation & Projective Geometry by: Diako Mardanbegi, PhD student

Transformation and Projective Geometry Computer Vision: Relative movement of a camera Object locations wrt. Camera Graphics Describe objects and cameras in a scene and how they should move in 3D scene. User interfaces (like WPF) relies heavy placing components relative to each other (2D) and for its 3D graphics engine.

Outline 2-D Transformations Multiple frames 3-D Transformations Representing Transformations Homogeneous form Multiple frames Cartesian Coordinate System Describe a local frame Orientation Translation Mapping between multiple frames 3-D Transformations Euler angles Quaternion Projective coordinate system Homography

2-D Translation y x

2-D Translation

2-D Rotation about the origin

2-D Rotation about the origin  Rotation about the origin

2-D Rotation about the origin  Rotation about the origin

2-D Rotation about a specified point

2-D Rotation about a specified point  a b d c Rotation about a specified point

2-D Scaling relative to the origin Vertical shift proportional to vertical position Horizontal shift proportional to Horizontal position

2-D Scaling relative to the origin b Scaling relative to the origin

2-D Scaling relative to a specified point

Representing transformations , Trick of additional coordinate makes this possible to express all transformations in a common linear form Old way: Cartesian coordinates New way: With additional coordinate (Homogeneous coordinates)

Representing transformations homogeneous transformation: linear transformation of homogeneous coordinates In homogeneous form x‘ = H x In homogeneous form

from Homogeneous coordinates to Cartesian coordinates from Cartesian coordinates to Homogeneous coordinates from Homogeneous coordinates to Cartesian coordinates

Advantages Easy to compose multiple transformations after each other Combination of all transformations into a single matrix

Advantages for example :General 2-D Pivot-Point Rotation  o’ a b d c

Inverse of transformations Easy to invert

Transformations Don’t Commute b 

Transformations Don’t Commute b 

Other 2-D transformations Reflection Shear 2

Homography matrix Full-generality 3 x 3 homogeneous transformation

Translation components Homography matrix Full-generality 3 x 3 homogeneous transformation Translation components

Scale/rotation components Homography matrix Full-generality 3 x 3 homogeneous transformation Scale/rotation components

Shear/rotation components Homography matrix Full-generality 3 x 3 homogeneous transformation Shear/rotation components

Homogeneous scaling factor Homography matrix Full-generality 3 x 3 homogeneous transformation Homogeneous scaling factor

Homography matrix Full-generality 3 x 3 homogeneous transformation When these are zero (as they have been so far), H is an affine transformation

Why we are using only one fixed Cartesian system? Why we don’t define a local coordinate system? o’  o’  a b d c

We can do it, but first know about: What is exactly a cartesian coordinate system? How to describe a local frame in a fixed frame? How to express a point in each frame?

Cartesian Coordinate System y-axis A coordinate system in Rn is defined by an origin o and n orthogonal basis vectors In R3, positive direction of each axis X, Y, Z is indicated by unit vector i, j, k Let P = (x, y, z)T be a point in R3 Coordinate is length of projection of vector from origin to point onto axis basis vector . P i j k x-axis O What do these values mean? z-axis ?

Vector Projection The projection of vector a onto u is that component of a in the direction of u

X’ Y’ B Y . P X A

Describe a local frame in a fixed frame Y X’ Y’ A,B X Y X’ Y’  A,B homogeneous form A X Y X’ Y’ (x, y) B : orientation of frame B relative to A : translation of frame B relative to A

Mapping from frame to frame X Y X’ Y’ . P A B Mapping Transformation

Transformation in 3-Dimension

c = a x b = (ya zb -za yb, za xb - xa zb, xa yb- ya xb)T Going from 2D to 3D Vector cross product Definition: If a = (xa, ya, za)T and b = (xb, yb, zb)T, then: c = a x b = (ya zb -za yb, za xb - xa zb, xa yb- ya xb)T c is orthogonal to both a and b (direction given by right-hand r with magnitude k = i xj

3-D Translation P’ = T . P

3-D Scaling

3-D Rotation In 2-D, we are always rotating in the plane of the image, but in 3-D the axis of rotation itself is a variable Three canonical rotation axes are the coordinate axes X, Y, Z

3-D Rotation Similar to 2-D rotation matrices, but with coordinate corresponding to rotation axis held constant

3-D Rotation When object is to be rotated about an axis that is parallel to one of the coordinate axes mesle mesale 2d

3-D Rotation Easiest way ?! When an object is to be rotated about an axis that is not parallel to one of the coordinate axes

Orientation in 3-D A B

Hierarchy of frames +Yworld +Xworld +Zworld

3-D Camera Coordinates +X right, -X left +Y down, -Y up Right-handed system From point of view of camera looking out into scene: +X right, -X left +Y down, -Y up +Z in front of camera, -Z behind

+Yworld +Xworld +Zworld Creating a simple WPF 3D space with camera: http://www.codeproject.com/KB/WPF/Wpf3DPrimer.aspx

Orientation in 3-D in 2D we have 4 elements in orientation matrix but we can locate the local frame by having only one angle . (we know the axis) in 3D we have 9 elements but how many angles do we need for locating the local frame in space? only 3

Euler’s Theorem Euler’s Theorem : “Any two independent orthonormal coordinate frames can be related by a sequence of rotations (not more than three) about coordinate axes, where no two successive rotations may be about the same axis”

Z-Y-X Euler angles Start with the frame coincident with a known frame {A}. Rotate {B} first about ZB by an angle , then about YB by an angle  and, finally, about XB by an angle 

Euler angles representation Advantages Matrix representation The mathematics is well-known Disadvantages: Describing a general rotation as rotations about the three basis axes is not natural for an animator. Order is important Gimbal lock

Quaternions Euler: “Any rotation of a rigid body can be described by defining an axis of rotation, often called the Euler rotation axis, and a rotation angle.” William Rowan Hamilton formulated quaternions, utilizing this Euler’s theorem, as a method of representing rotations.

Quaternions Complex numbers are represented in the form where , and a,b are real numbers. Quaternion is an extended complex number

Rotation by Quaternions Any unit quaternion has the form for some angle and unit vector We will represent a point p in space by the quaternion We compute the rotation of point P about the unit vector I by an angle  by using the unit quaternion and by Rodrigues formula :

Rotation by Quaternions We can also combine two rotation by multiplication of the involved quaternions. Example: 2

Quaternions representation Advantages: Simple composition Rotations Obvious geometrical interpretation Disanvantages: Quaternions only represent rotation Quaternion mathematics appears complicated

How does the model look in the view of the camera 2D Projective Geometry How does the model look in the view of the camera +Yworld +Xworld +Zworld +Ycamera +Xcamera +Zcamera

Texture mapping on 3D surfaces

Projective Geometry Outline Description of projective geometry in a plane Descriptions of Points and lines Projective transformations Example of application

Projective Geometry Euclidean geometry describes shapes “as they are” Properties of objects that are unchanged by rigid motions Lengths Angles Parallelism Projective geometry describes objects “as they appear” Lengths, angles, parallelism become “distorted” when we look at objects

2D Projective Geometry Coordinates are called homogeneous or projective coordinates R is the coordinate vector of point M and (x,y,w) are its homogeneous coordinates the rays (x,y,w) and (λx, λy, λw) are the same and are mapped to the same point M L is the coordinate vector of line l and (a,b,c) are its homogeneous coordinates

From Projective Plane to Euclidean Plane How do we “land” back from the projective world to the 2D world of the plane? for Point: for line:

Projective Geometry in 2D Point M belongs to line l if and only if: Two lines L=(a,b,c) and L’=(a’,b’,c’) intersect in the point R=L×L’ The line through 2 points R and R’ is L=R×R’

Projective Geometry in 2D The points R=(x,y,0) do not correspond to finite points in the plane. they are points at infinity, also called ideal points The line L=(0,0,1) passes through all points at infinity, since L.R=0 Two parallel lines L=(a,b,c) and L’=(a,b,c’) intersect at the point R=L×L’=(c’-c)(b,-a,0), i.e. (b,-a,0) Any line (a,b,c) intersects the line at infinity at (b,-a,0). so the line at infinity is the set of all points at infinity

Duality Duality principle: To any theorem of 2-D projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem

Projective Transformations in a Plane Projectivity (Collineation or Homography) Mapping from points in plane to points in plane A homography is an invertible transformation from the real projective plane to the projective plane that maps straight lines to straight lines. 3 aligned points are mapped to 3 aligned points

Transformation of lines A mapping is a projectivity if and only if the mapping consists of a linear transformation of homogeneous coordinates Definition: Projective transformation or 8DOF For a point transformation Transformation for lines

Special Projectivities

A special case A plane to plane projective transformation 2D World plane Observer 2D Image plane (retina, film, canvas)

Example of Projective Transformations Central projection maps planar scene points to image plane by a projectivity The image of the same plannar scene from a second camera can be obtained from the image from the first camera by a projectivity

Applications of homographies

Mosaics

projectivity is from one plane to another plane Remember that projectivity is from one plane to another plane

Camera Model Homogeneous form Concise form K: Camera Calibration Matrix

General mapping of a pinhole camera Xc = R(Xw − Cw) Xc = RXw − RCw Translation Homogeneous form of Point on image Rotation Camera Calibration

Corrected image (front-to-parallel)

Homography The floor (enlarged) Automatically rectified floor

Analysing patterns and shapes Automatic rectification From Martin Kemp The Science of Art (manual reconstruction) 2 patterns have been discovered !

Analysing patterns and shapes What is the (complicated) shape of the floor pattern? Automatically rectified floor St. Lucy Altarpiece, D. Veneziano

Analysing patterns and shapes Automatic rectification From Martin Kemp, The Science of Art (manual reconstruction)

The Ambassadors H. Holbein

Applications of Homographies summary Mosaics (Image processing) Involves computing homographies between pairs of input images (image-image mappings) Rendering textures (Computer graphics) Require planar scene surface and image plane. (scene-image mapping) Computing planar shadows (Computer graphics) Require apply H between two surfaces inside a 3D scene having the light source as the center of projections (Scene-Scene mapping) Remove perspective distortion (Computer vision) Involves computing H between image and 3D scene surfaces (image-scene mapping)