Digital Image Processing Lecture 7: Geometric Transformation March 16, 2005 Prof. Charlene Tsai

Digital Image ProcessingLecture 7 2 Review  A geometric transform of an image consists of two basic steps  Step1: determining the pixel co-ordinate transformation  Step2: determining the brightness of the points in the digital grid.  In the previous lecture, we discussed brightness interpolation and some variations of affine transformation.  A geometric transform of an image consists of two basic steps  Step1: determining the pixel co-ordinate transformation  Step2: determining the brightness of the points in the digital grid.  In the previous lecture, we discussed brightness interpolation and some variations of affine transformation. (x,y) T(x,y)

Digital Image ProcessingLecture 7 3 Affine Transformation (con’d)  An affine transformation is equivalent to the composed effects of translation, rotation and scaling.  The general affine transformation is commonly expressed as below:  An affine transformation is equivalent to the composed effects of translation, rotation and scaling.  The general affine transformation is commonly expressed as below: 0 th order coefficients1 st order coefficients

Digital Image ProcessingLecture 7 4 General Formulation  A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):  T x (x,y) and T y (x,y) are usually polynomial equations.  This transform is linear with respect to the coefficients a rk and b rk.  A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):  T x (x,y) and T y (x,y) are usually polynomial equations.  This transform is linear with respect to the coefficients a rk and b rk.

Digital Image ProcessingLecture 7 5 Finding Coefficients  How to find a rk and b rk, which are often unknown?  Finding pairs of corresponding points (x,y), (x’,y’) in both images,  Determining a rk and b rk by solving a set of linear equations.  More points than coefficients are usually used to get robustness. Least-squares fitting is often used.  How to find a rk and b rk, which are often unknown?  Finding pairs of corresponding points (x,y), (x’,y’) in both images,  Determining a rk and b rk by solving a set of linear equations.  More points than coefficients are usually used to get robustness. Least-squares fitting is often used.

Digital Image ProcessingLecture 7 6 Jacobian  A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes  The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).  What is the Jacobian of an affine transform?  A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes  The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).  What is the Jacobian of an affine transform?

Digital Image ProcessingLecture 7 7 Variation of Affine (2D)  Translation: displacement  Euclidean (rigid): translation + rotation  Similarity: Euclidean + scaling (for isotropic scaling, s x = s y )  Translation: displacement  Euclidean (rigid): translation + rotation  Similarity: Euclidean + scaling (for isotropic scaling, s x = s y )

Digital Image ProcessingLecture 7 8 Types of transformations (2D)  Affine: Similarity + shearing  Some illustrations of shearing effects (courtesy of Luis Ibanez)  Affine: Similarity + shearing  Some illustrations of shearing effects (courtesy of Luis Ibanez)

Digital Image ProcessingLecture 7 9 Affine transform  Shearing in x-direction y x a 01 y (x,y) ( x + a 01 y, y ) y’ x y x’

Digital Image ProcessingLecture 7 10 Affine transform  Shearing in y-direction y x b 10 x (x,y) ( x, y + b 10 x ) y x y’ x’

Digital Image ProcessingLecture 7 11 Invariant Properties  Translation:  Euclidean:  Similarity:  Affine:  Translation:  Euclidean:  Similarity:  Affine:

Digital Image ProcessingLecture 7 12 Types of transformations (2D)  Bilinear: Affine + warping  Quadratic: Affine + warping  Bilinear: Affine + warping  Quadratic: Affine + warping

Digital Image ProcessingLecture 7 13 Least-Squares Estimation  Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.  Objective: To minimize the sum of the Euclidean distances of the correspondence set C={p i,q i }, where p i ={x i,y i }, and q i is the corresponding point.  Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.  Objective: To minimize the sum of the Euclidean distances of the correspondence set C={p i,q i }, where p i ={x i,y i }, and q i is the corresponding point.

Digital Image ProcessingLecture 7 14 Least-Squares Estimation - Affine  For affine model,  The Euclidean error, where  With the new notation,  To find which minimize we want  For affine model,  The Euclidean error, where  With the new notation,  To find which minimize we want

Digital Image ProcessingLecture 7 15 Least-Squares Estimation - Affine  This leads to  Therefore,  This leads to  Therefore,

Digital Image ProcessingLecture 7 16 Some remarks  So far, we only discussed global transformation (applied to entire image)  It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.  for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.  geometric transformation (distortion) is then performed separately in each sub-image.  So far, we only discussed global transformation (applied to entire image)  It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.  for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.  geometric transformation (distortion) is then performed separately in each sub-image.

Digital Image ProcessingLecture 7 17 Real Application  Registration of retinal images of different modalities  Importance?  Registration of retinal images of different modalities  Importance? Color slideFluorocein Angiogram

Digital Image ProcessingLecture 7 18 Mosaicing Mosaic

Digital Image ProcessingLecture 7 19 Fusion of medical information

Digital Image ProcessingLecture 7 20 Change Detection

Digital Image ProcessingLecture 7 21 Computer-assisted surgery

Digital Image ProcessingLecture 7 22 What is needed?  A known transformation, which is less likely OR  Computing the transformation from features.  Feature extraction  Features in correspondence  Transformation models  Objective function  A known transformation, which is less likely OR  Computing the transformation from features.  Feature extraction  Features in correspondence  Transformation models  Objective function

Digital Image ProcessingLecture 7 23 What might be a good feature? Color slideFluorocein Angiogram

Digital Image ProcessingLecture 7 24 Feature Extraction

Digital Image ProcessingLecture 7 25 Features in Correspondence (crossover & branching points)

Digital Image ProcessingLecture 7 26 Features in Correspondence (vessel centerline points)

Digital Image ProcessingLecture 7 27 Transformation Models 12-parameter quadratic transformation p q

Digital Image ProcessingLecture 7 28 Transformation Model Hierarchy Similarity Affine Reduced Quadratic Quadratic

Digital Image ProcessingLecture 7 29 Objective Function Transformation parameters Feature point in image Mapping of features from to based on Set of correspondences

Digital Image ProcessingLecture 7 30 Result