Download presentation

Presentation is loading. Please wait.

Published byChelsey Gladson Modified about 1 year ago

1
Medical J. Michael Fitzpatrick, Department of Electrical Engineering and Computer Science Vanderbilt University, Nashville, TN Course on Medical Image Registration, Nov 3-Nov 24, 2008 Institute für Robotic, Leibniz Universität Hannover, Germany Image Registration

2
Schedule Nov 3: Overview of Medical Image Registration Nov 10: Point-based, rigid registration Nov 17: Intensity-based registration Nov 24: Non-rigid registration

3
C omputed T omography (1972) Siemens CT Scanner (Somatom AR)

4
3D Cross-sectional Image “voxels” (“volume elements”)

5
M agnetic R esonance Imaging GE MR Scanner (Signa 1.5T)

6
P ositron E mission T omography GE PET Scanner

7
Physician has 3 or more views. CT (bone) MR (wet tissue) PET (biological activity)

8
Combining multiple images requires image registration

9
Image Registration: Definition Determination of corresponding points in two different views

10
Motion relative to the scanners can be three-dimensional.

11
Slice orientations vary widely. transversesagittalcoronal

12
Views may be very different.

13
But all orientations and all views can be combined if we have the 3D point mapping.

14
Combining Registered Images = “Image Fusion” MR + PET CT + MR CT MR PET

15
Rigid Registration: Definition Rigid Registration = Registration using a “rigid” transformation

16
Rigid Transformation RigidNon-rigid Distances between all points remain constant. 6 degrees of freedom

17
Nonrigid Transformations can be very complex! [Thompson, 1996]

18
Non-rigid example

19
Registration Dichotomy “Retrospective” methods (nothing attached to patient before imaging) Match anatomical features: e.g., surfaces Maximize similarity of intensity patterns “Prospective” methods (something attached to patient before imaging) Non-invasive: Match skin markers Invasive: Match bone-implanted markers

20
Most Common Approaches Intensity-based* (not for surgical guidance) Surface-based (requires identified surfaces) Point-based (requires identified points) Stereotactic frames (for surgical guidance) *Sometimes called “voxel-based”

21
The Most Successful Intensity-Based Method: Mutual Information

22
2D Intensity Histogram (Hill94) CT MR CT intensity MR intensity

23
Misregistration Blurs It 0 cm 2 cm 5 cm MR CT MR PET Hill, 1994

24
A measure of histogram sharpness Most popular “intensity” method Assumes a search method is available Stochastic, multiresolution search common Requires a good starting pose May not find global optimum Not useful for surgical guidance Mutual Information (Viola, Collignon, 1996)

25
Example: Mutual Information Studholme, Hill, Hawkes, 1996, “Automated 3D registration of MR and CT images of the head”, MIA, 1996 (Open movie with QuickTime)

26
The Most Successful Surface-Based Method: The Iterative Closest-Point Algorithm

27
Minimizes a positive distance function Assumes surfaces have been delineated Guaranteed to converge Requires a good starting pose May not find global optimum Can be used for surgical guidance Iterative Closest-Point Method (Besl and McKay, 1992)

28
Start with two surfaces

29
Reorient one (somehow)

32
Pick points on moving surface

34
Remove moving surface

35
Points become proxy for surface

36
Find closest points on stationary surface

37
Measure the total distance

38
Remove stationary surface

39
Points become proxy for surface

40
Register point sets (rigid)

42
Restore stationary surface

43
Find (new) closest points

45
Remove stationary surface

47
Register Points

48
Register Points, and so on…

49
Iterative Closest-Point Algorithm: Find closest points Measure total distance Register points Stop when distance change is small.

50
ICP: Image-to-Image Dawant et al.

51
ICP: Image to Patient The BrainLab VectorVision surgical guidance system uses surface-based registration.

52
ICP requires surface delineation, which is a problem in Image Segmentation Example: Level Set Segmen- tation (Dawant et al.) http://www.vuse.vanderbilt.edu/~dawant/levelset_examples/

53
The fiducial marker is used in prospective registration for image- guided surgery. The Most Common Application of The Point-based Method: The Fiducial Marker

54
Image-Guided Surgery...and the other is the patient. One view is an image.... Just another image registration problem.

55
Acustar ™ Allen, Maciunas, Fitzpatrick, and Galloway 1988-1995 (J&J Z-Kat) are implanted into the skull. Posts [Maurer, et al., TMI, 1997]

56
Acustar ™ Allen, Maciunas, Fitzpatrick, and Galloway 1988-1995 (J&J Z-Kat) [Maurer, et al., TMI, 1997] Liquid in marker shows up in image Divot cap is localizable in OR

57
Acustar ™ Allen, Maciunas, Fitzpatrick, and Galloway 1988-1995 (J&J Z-Kat) [Maurer, et al., TMI, 1997] Marker center and cap center occupy the same position relative to the post

58
Acustar ™ Allen, Maciunas, Fitzpatrick, and Galloway 1988-1995 (J&J Z-Kat) [Maurer, et al., TMI, 1997] Marker center and cap center occupy the same position relative to the post

59
Find corresponding “fiducial” points Point-based, Rigid Registration View 2 = “Space” 2 View 1 = “Space” 1 Rigid transformation Align corresponding fiducials “targets” are also aligned Find all corresponding “fiducial” points

60
Measures of Error View 1 Registered Views View 2 Fiducial Localization Error (FLE) Target Registration Error (TRE) Fiducial Registration Error (FRE)

61
The Most Successful Point-based Method (by far!): Minimization of Sum of Squares of Fiducial Registration Errors

62
Minimizes a positive distance function Most popular point method Assumes points have been localized Guaranteed to converge Does not require a good starting pose Always finds global optimum Can be used for surgical guidance Minimization of Sum of FRE 2 (Shönemann, Farrell, 1966)

63
Accuracy: State of the Art The best accuracy is probably achieved for the head…

64
Retrospective Registration of Head: Image-to-Image Median Maximum CT-MR : 0.6 mm 3.0 mm PET-MR: 2.5 mm 6.0 mm TRE

65
Prospective Registration of Head: mean TRE ≤ 1 mm (CT) [Hill, JCAT, 1998, Maurer, TMI, 1997]

66
Error Theory for Minimization of Mean-square FRE

67
End of Overview

68
How to Do Minimization of Sum of Squares of Fiducial Registration Errors

69
Sum of Squares: Step 1 Center the points: Centered

70
Step 2 (Shönemann, Farrell, 1966) Determine the Rotation: Centered Centered and Rotated

71
Step 3 (Farrell, 1966) Determine the Translation: Before rotation After rotation, but before translation

72
Error Analysis

73
Start with Assumptions about FLE Independent, normal, isotropic, zero mean Space 1 Space 2

74
“Effective” FLE Space 1 Space 2

75
FRE Statistics: Sibson 1979 Approximate Solution: Configuration doesn’t matter !

76
Principal axes Configuration does matter. d1d1 d2d2 d3d3 [Fitzpatrick, West, Maurer, TMI, ’98] TRE statistics, 1998 Approximate Solution:

77
Got to here Nov 10, 2008

78
4mm 3mm 2mm 1mm 2mm 1mm FRE = 1mm TRE for FLE of 1mm Marker Placement [West et al., Neurosurgery, April, 2001]

79
A distribution would be better TRE 2 95% level Probability density

80
And what about direction?

81
TRE statistics, 2001 Approximate Solution: TRE 1 TRE 2 TRE 3 [Fitzpatrick and West., TMI, Sep 2001]

82
Some Remaining Problems

83
Isotropic Scaling [Actually now solved: Batchelor, West, Fitzpatrick, Proc. of Med. Im. Undstnd. & Anal., Jul 2002]

84
Anisotropic Scaling (Iterative Solution Only)

85
Register M points sets simultaneously View 1 View 2 ; View 3 View M The “Generalized” Procrustes Problem (Iterative Solution Only)

86
Anisotropic FLE (Iterative Solutions Only)

87
Other Unsolved Problems What is the statistical effect on TRE of dropping or adding a fiducial? Does anisotropy in FLE always, sometimes, or never makes TRE worse? How do we configure markers on a given surface so as to minimize TRE over a given region? Is there a correlation between FRE and TRE? It’ solved: There is no correlation! Fitzpatrick, SPIE Medical Imaging Symposium, to be presented Feb 2009.

88
Extension to perspective transformations. Extension to surface matching. Other Unsolved Problems (cont.)

89
Rigid Registration of the Head State of the Art

90
CT MR-T1 MR-T2 Finding Points = “Localization”

91
Acustar v. Leibinger: Leibinger Grows Up!

92
Retrospective Registration of Head Images: The State of the Art Median Maximum (Acustar) Best CT-MR : 0.6 mm 3.0 mm (0.5 mm) Poor CT-MR: 5.4 mm 61 mm (0.5 mm) Best PET-MR: 2.5 mm 6.0 mm (1.7 mm) Poor PET-MR: 5.3 mm 15 mm (1.7 mm) And how do we know?…

93
R etrospective I mage R egstration E valuation Access: 150+ participants in 20 countries Evaluation: 57 participants in 17 countries External site Vanderbilt 1995-2007

94
End Additional slides follow

95
Categories within error prediction Number of point sets: Two or more Scaling: Isotropic or anisotropic Point-wise weighting: equal or unequal Anisotropic weighting Cost function: squared error or other Point-wise FLE: equal or unequal Spatial FLE: isotropic or anisotropic... Key: Approximate, Negligible progress

96
Anisotropic Scaling R, t = rotation, translation w i 2 = point weighting S = diag( s x, s y, s z ) Given {x i y i w i } find R, t, S to minimize mean FRE 2 Iterative Algorithm: sysy szsz sxsx Search space Problem Statement:

97
Scaling: Anisotropic II R, t = rotation, translation w i 2 = point weighting S = diag( s x, s y, s z ) Given {x i y i w i } find R, t, S to minimize mean FRE 2 Iterative Algorithm:Problem Statement:

98
Spatial Weighting R, t = rotation, translation w i 2 = point weighting S = diag( s x, s y, s z ) A = diag( a x, a y, a z ) Given {x i y i w i } find R, t, S to minimize mean FRE 2 Iterative Algorithm:Problem Statement: Partial Solution:

99
Generalized Procrustes Problem Cost function Iterative method (only)

100
Add Isotropic Scaling Approximate Solution: FRE 2 = sum of squared fiducial registration errors

101
FRE: Generalized + Scaling Approximate Solution: FRE 2 = sum of squared fiducial registration errors

102
TRE statistics with scaling Approximate Solution: TRE 2 = target registration error

103
Applications of TRE Statistics

104
Surgical Paths Radiation Isodose Contours Error Bounds

105
Probe Design Tip = “target” IREDs are fiducials FLE TRE

106
Fiducial-Specific FRE Poor fiducial alignment tends to occur where target registration is good!!

107
Four Solution Methods ( All work equally well [Eggert91]! )

108
Generalized Procrustes Problem (We’ve already done it for M=2.) Problem Statement: Illustration:

109
Generalized Procrustes Problem Iterative Algorithm: Illustration: *Subject to S (m) normalization

110
Approximation Method (due to Sibson, 1979)

111
Approximation Method (cont.)

112
FRE Statistics Problem Statement: Approximate Solution:

113
TRE statistics with scaling Problem Statement: Approximate Solution:

114
What do “solved” and “unsolved” mean? “Solved”, working definition: Reduced to solving algebraic equations Iterative algorithm that converges to solution Approximate solution accurate to “Unsolved”: Not solved

115
Point-wise weighting: Equal or Unequal (We’ve just looked at this one.) R, t = rotation, translation w i 2 = point weighting Given {x i y i w i } find R, t to minimize mean FRE 2 Problem Statement: See previous slides again! Solution:

116
1. Performing a Registration x i = point in “from” set; y i = point in “to” set. t = translation vector. R = 3x3 rotation matrix (therefore R t R = I ). Rx i + t Given {x i y i w i } find R, t to minimize mean FRE 2 xixi yiyi ( usually w i =1) a.k.a. The “Orthogonal Procrustes Problem” Problem Statement:

117
2. Predicting Registration Error View 1 Registered Views View 2 Input --- fiducial positions target position, r FLE distribution Output --- statistics for TRE Output --- statistics for FRE

118
Isotropic Scaling R, t = rotation, translation w i 2 = point weighting s = isotropic scaling Given {x i y i w i } find R, t, s to minimize mean FRE 2 Problem Statement: Solution:

119
Acknowledgements Benoit M. Dawant, PhD, EECS Robert L. Galloway, PhD, BME William C. Chapman, MD, Surgery Jeannette L. Herring, PhD, EECS Jim Stefansic, PhD, Psychology Diane M. Muratore, MS, BME David M. Cash, MS, BME Steve Hartman, MS, BME W. Andrew Bass, BME NSF NIH Matthew Wang, PhD, IBM Jay B. West, PhD, Accuray, Inc. Derek L. G. Hill, PhD Kings College Calvin R. Maurer, Jr., PhD, Stanford U.

120
What could we choose to optimize? Mean-square “Fiducial Registration Error” (FRE 2 ) Known as the “Orthogonal Procrustes Problem” in statistics since 1950s. Robust estimators (median, M-estimators) Less sensitive to “outliers” Color key: Major problems solved, Much less done

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google