Image Segmentation and Registration Rachel Jiang Department of Computer Science Ryerson University 2006

Rachel Jiang, Rigid registration Intensity-based methods –Have been very successful Monomodal Multimodal –Assuming a global relationship between the … images to register –Deriving a suitable similarity measure Correlation coefficient Correlation ratio Mutual information Block matching …

Rachel Jiang, Methods for fusing images There are three general methods for fusing images from different (or the same) image modalities: –landmark matching include external fiducial landmarks or anatomic landmarks. –surface matching uses an algorithm that matches different images of the same patient surface. –intensity matching.

Rachel Jiang, intensity matching uses mutual intensity information to co- register different images. The matched intensities may come from the same scanner –two different MRI scans acquired on different days – from different modalities such as MRI and PET.

Rachel Jiang, Rigid Body Model Most constraint model for medical imaging asserts that the distance and internal angles within images can not be changed during the registration

Rachel Jiang, Non rigid model can detect and correct discrepancies of small spatial extent, by deforming one of the images (source) to match the other (reference). Spatial deformation model can be based on different physical properties like elasticity or viscosity, or their generalizations and simplifications. Deformation is driven by external forces, which tend to minimize image differences, measured by image similarity measures.

Rachel Jiang, Mutual information The mutual information of two images is the amount of information that one image contains about the other or vice versa. Transforming one image with respect to the other such that their mutual information is maximized. The images are assumed to be registered.

Rachel Jiang, Mutual information A function of transformation between the images –an algorithm that searches maximum value for a function that gives the alignment information between images –different transformation estimates are evaluated (those transformation estimates will result in varying degrees of overlap between the two images) –(MI as a registration criteria is not invariant to the overlap between images)

Rachel Jiang, MI

10 MI in 2D/3D space –In 3D space We attempt to find the registration by maximizing the information that one volumetric image provides about the other. –In 2D space Two curve that are to be matched as the reference curve and the current active evolving curve. We seek an estimate of the transformation that registers the reference curve and the active curve by maximizing their mutual information

Rachel Jiang, Degree of freedom Six degree of freedom charactering the rigid movements –3 describe the Rotation –3 describe Translation

Rachel Jiang, Similarity measures Common registration methods can be grouped as –Feature based techniques Rely on the presence and identification of natural landmarks or fiducial marks in the input dataset to determine the best alignment –Intensity-based measures Operate on the pixel/voxel intensities directly –Varies statistics are calculated by using the raw intensity values

Rachel Jiang, Taking geometric constraints Special land marks on human body Manually embed land marks Spacial correlations …

Rachel Jiang, Correlation ratio Given two images I and J, the basic principle of the CR method is to search spatial transformation T and an intensity mapping f such that, by displacing J and remapping its intensities, the resulting image f(JoT) be as similar as possible to I.

Rachel Jiang, Example of Image Segmentation Bone fracture detection

Rachel Jiang, Image Segmentation Techniques threshold techniques –make decisions based on local pixel information edge-based methods –Weakness: broken contour lines causes failure region-based techniques –partitioning the image into connected regions by grouping neighbouring pixels of similar intensity levels. –Adjacent regions are then merged under certain criterion. Criteria create fragmentation or overlook blurred boundaries and overmerge. Active contour models

Rachel Jiang, Deformable models –Snakes/Balloons/Deformable Templates provide a curve as a compromise between regularity of the curve and high gradient values among the curve points. –(Kass et al., 1988; Cohen, 1991; Terzopoulos, 1992) –Level set methods Level Set Methods are numerical techniques which can follow the evolution of interfaces. These interfaces can develop sharp corners, break apart, and merge together. –(Osher and Sethian, 1988; Sethian, 2001) –Geodesic Active Contour take the advantages of both Snake and Level set methods –(Caselles et al., 1995; Malladi et al, 1995; Sapiro, 2001)

Rachel Jiang, The Snake formula

Rachel Jiang, Snake: image force

Rachel Jiang, Snake: example

Rachel Jiang, Level Set Method Level Set Methods –provide formulation of propagating interfaces, a mathematical formulation and numerical algorithm for tracking the motion of curve and surfaces (Osher and Sethian, 1988; Sethian, 2001) –For segmenting several objects simultaneously or an objects with holes, it is possible to model the contour as a level set of a surface, allow it to change its topology in a nature way (Cohen, 1997).

Rachel Jiang, Level Set

Rachel Jiang, Formula of Level Set method Curve evolving formula:

Rachel Jiang, LSM: example

Rachel Jiang, LSM: More Example

Rachel Jiang, Geodesic Active Contour (GAC) –presents some nice properties the initialization step does not impose any significant constraint can deal successfully with topological changes, finding the global minimum of energy minimizing curve can be solved by mapping the boundary detection problem into a single minimum problem. The new model mathematically inherit –the way handling the topological changes from the Level Set – the minimizing deformation energy function with ‘internal’ and ‘external’ energies along its boundary from the traditional Snake. –This model inherit the advantages of LS and ‘Snakes’ by transform mathematical formulation of Snake – Lagrenge formula with PDEs –The theory behind the GAC is the use of partial differential equations and curvature-driven flows. (Caselles et al., 1995; Malladi et al, 1995; Sapiro, 2001)

Rachel Jiang, Segmentation result using ACM

Rachel Jiang, Surface reconstruction

Rachel Jiang, Mathematical Morphology –provides the foundation for measuring topological shape, size, location. The theory behind mathematical morphology is defining computing operations by primitive shapes –Georges Matheron, Jean Serra and their colleagues of Centre de Morphologie Mathematique –G.Matheron 1975, Serra, 1982, Vicent, 1990… –offer several robust theories and algorithms to implement on digital images to extract complex features –uses ‘Set Theory’ as the foundation for its functions. The simplest functions to implement are ‘Dilation’ and ‘Erosion’.

Rachel Jiang, Erosion and Dilation (1D)

Rachel Jiang, Erosion and Dilation (2D)

Rachel Jiang, Erosion & Dilation example

Rachel Jiang, Opening & Closing

Rachel Jiang, Shape Operators  Shapes are usually combined by means of : Set Intersection (occluded objects): Set Union (overlapping objects):

Rachel Jiang, Dilation B A

Rachel Jiang, Dilation

Rachel Jiang, Extensitivity B A

Rachel Jiang, Erosion B A

Rachel Jiang, Erosion

Rachel Jiang, Erosion

Rachel Jiang, Opening and Closing  Opening and closing are iteratively applied dilation and erosion Opening Closing

Rachel Jiang, Opening and Closing

Rachel Jiang, Opening and Closing  They are idempotent. Their reapplication has not further effects to the previously transformed result

Rachel Jiang,

Rachel Jiang, Watershed

Rachel Jiang, Capturing the shape prior the curve C and the transformation S, R, T is calculated such that the curve C new = SRC + T and C * are perfectly aligned. The minimization problem now can be solved by finding steady state solutions to the following system

Rachel Jiang, Minimization processing system

Rachel Jiang, Distance measure d(x,y) = d(C *,(x,y)) is the distance of the point (x,y) from C* The function d is evaluated at SRC(p) + T

Rachel Jiang, Minimizing Energy function