1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer Advanced Training 11: Registration Sonia Pujol, Ph.D. Surgical Planning Laboratory Radiology, Brigham and Womens Hospital Harvard Medical School Randy Gollub, M.D., Ph.D. Athinoula A. Martinos Center Psychiatry, Massachusetts General Hospital Harvard Medical School

2. Pujol S., Gollub R. National Alliance for Medical Image Computing Acknowledgments National Alliance for Medical Image Computing NIH U54EB005149 Neuroimage Analysis Center NIH P41RR013218 Surgical Planning Laboratory, Brigham and Womens Hospital Thanks to Steve Pieper, Ph.D.

3. Pujol S., Gollub R. National Alliance for Medical Image Computing Disclaimer It is the responsibility of the user of 3DSlicer to comply with both the terms of the license and with the applicable laws, regulations and rules.

4. Pujol S., Gollub R. National Alliance for Medical Image Computing Motivation Registration algorithms bring multiple image data sets into spatial alignment, in order to achieve anatomical agreement. Mutual information techniques can be applied to a wide variety monomodality and multimodality images. Dataset 1 Dataset 2

5. Pujol S., Gollub R. National Alliance for Medical Image Computing Goal of the tutorial Guiding you step-by-step through the process of automatically registering two structural MR datasets using a mutual information algorithm. In this tutorial, an example of registration of a pre-operative MR dataset with an intra- operative MR dataset is used.

6. Pujol S., Gollub R. National Alliance for Medical Image Computing Materials Software: Slicer 2.7 Dataset: RegistrationSample.zip http://www.namic.org/Wiki/index.php/Slicer:Workshops :User_Training_101

7. Pujol S., Gollub R. National Alliance for Medical Image Computing Processing pipeline Automatic registration Final Transform Semi-automatic refinement of the registration no yes Manual registration Initial transform Result OK ? (Step 2) (Step 3) (Step 4) Data loading (Step 1)

8. Pujol S., Gollub R. National Alliance for Medical Image Computing Data description Dataset 1 (I 1 ): 1.5 Tesla diagnostic MR scanner Regsample1/ : reg.nhdr and reg.img (27 slices) Dataset 2 (I 2 ): 0.5 Tesla intraoperative MR scanner Regsample2/ : I.xxx (27 slices) The datasets are images of the same subject, acquired with different scan sessions each using a different MR Scanner. The datasets are located in the directories /regsample1 and /regsample2 in the archive RegistrationSample.zip.

9. Pujol S., Gollub R. National Alliance for Medical Image Computing Overview Step 1: Load data and visualize mis- alignment Step 2: Manually define the initial transformation Step 3: Complete the registration by using the mutual information algorithm Step 4: Refine the registration by using the semi-automatic mode (optional) Step 5: Apply the registration transform

10. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 1 Click on Add Volume in the Main Panel

11. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 1 Select Properties Nrrd Reader Browse to load the file reg.nhdr Click on Apply to load the volume

12. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 1 Slicer loads the volume reg.nhdr

13. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 2 Click on AddVolume to load the dataset 2

14. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 2 Browse to load the image I.001 Click on Apply to load the volume Select the Properties Basic

15. Pujol S., Gollub R. National Alliance for Medical Image Computing Loading dataset 2 Slicer loads the volume I

16. Pujol S., Gollub R. National Alliance for Medical Image Computing Initial mis-alignment Left-click on Fg to display the volume reg-nhdr in foreground.

17. Pujol S., Gollub R. National Alliance for Medical Image Computing Initial mis-alignment Click on Fade and use the slider to visualize the initial mis-alignment between the two volumes

18. Pujol S., Gollub R. National Alliance for Medical Image Computing Initial mis-alignment Observe the misalignment on the occipital lobe (axial slice 0) using the Fade function. I2 I1

19. Pujol S., Gollub R. National Alliance for Medical Image Computing Initial mis-alignment Observe the misalignment on the boundaries between the cerebrum and the cerebellum (sagittal slice 0). I2 I1

20. Pujol S., Gollub R. National Alliance for Medical Image Computing Initial mis-alignment Observe the misalignment on the lateral edge of the brain (axial slice 30). I2 I1

21. Pujol S., Gollub R. National Alliance for Medical Image Computing Overview Step 1: Load data and visualize mis- alignment Step 2: Manually define the initial transformation Step 3: Complete the registration by using the mutual information algorithm Step 4: Refine the registration by using the semi-automatic mode (optional) Step 5: Apply the registration transform

22. Pujol S., Gollub R. National Alliance for Medical Image Computing Rigid Transformation A rigid transform T is an image coordinate transformation composed of a translation vector (Tx, Ty, Tz) and a rotation matrix defined by three Euler angles (θ,Φ,Ψ).

23. Pujol S., Gollub R. National Alliance for Medical Image Computing Rigid Transformation Image Space 1 Image Space 2 I1I1 T(I 1 )

24. Pujol S., Gollub R. National Alliance for Medical Image Computing By applying the registration transform to the initial volume I 1, well generate a new volume spatially aligned with the volume I 2. This allows the extraction of complementary information from the two volumes. Rigid Transform Image Space 1 Image Space 2 I2I2 I1I1

25. Pujol S., Gollub R. National Alliance for Medical Image Computing Adding a transformation To perform an initial manual registration between the two volumes, select the volume reg-nhdr and click on Add Transform. You will manually define the parameters of the initial registration matrix by using the mouse to superimpose the two volumes.

26. Pujol S., Gollub R. National Alliance for Medical Image Computing Adding a transformation Slicer adds the transform transform0 defined by the Identity matrix manual0. Double-click on manual0 to display the translation and rotation elements.

27. Pujol S., Gollub R. National Alliance for Medical Image Computing Adding a transformation Slicer displays the three translation parameters and the three rotation angles of the matrix manual0 (identity). The six degrees of freedom are defined in the anatomical directions Left-Right (LR), Posterior-Anterior (PA) and Inferior-Superior (IS).

28. Pujol S., Gollub R. National Alliance for Medical Image Computing Processing pipeline Automatic registration Final Transform Semi-automatic refinement of the registration no yes Manual registration Initial transform Result OK ? (Step 2) (Step 3) (Step 4) Data loading (Step 1)

29. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Click on Local and set the Mouse Action to Translate

30. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Hold the left mouse button down while clicking in the in the Axial view, and translate the slice in the anterior direction by 10 mm.

31. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Slicer displays the value of the applied manual translation in the PA direction.

32. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Click on Rotate to define the rotation component of the initial transformation.

33. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Hold the left mouse button down while clicking in the coronal view. Use the mouse to rotate the slice until you see the value of 3 degrees (counterclockwise) in the coronal view.

34. Pujol S., Gollub R. National Alliance for Medical Image Computing Defining an initial transformation Slicer displays the value of the applied manual rotation.

35. Pujol S., Gollub R. National Alliance for Medical Image Computing Overview Step 1: Load data and visualize mis- alignment Step 2: Manually define the initial transformation Step 3: Complete the registration by using the mutual information algorithm Step 4: Refine the registration by using the semi-automatic mode (optional) Step 5: Apply the registration transform

36. Pujol S., Gollub R. National Alliance for Medical Image Computing Similarity Measure I2I2 T(I 1 ) The registration algorithm computes the parameters of the transformation T that optimizes a measure of similarity between the target image I 2 and the initial image that has been manually transformed T(I 1 ). T

37. Pujol S., Gollub R. National Alliance for Medical Image Computing Mutual information The mutual information MI is a measure of similarity of the images I 2 and T(I 1 ) based on the entropy H (*): MI(I 2,T(I 1 ))= H(I 2 ) + H(T(I 1 )) – H(I 2,T(I 1 )) (*) Wells S, Viola P, Kikinis R. Multi-modal volume registration by maximization of mutual information. Medical Robotics and Computer-assisted Surgery 1995, 55-62. Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G. Automated multimodality image registration based on information theory. Information Processing in Medical Imaging, 1995, 263-274. The automatic alignment of the images I 2 and T(I 1 ) is achieved by maximizing the mutual information MI(I 2,T(I 1 )).

38. Pujol S., Gollub R. National Alliance for Medical Image Computing Processing pipeline Final Transform Semi-automatic refinement of the registration no yes Manual registration Initial transform Result OK ? (Step 2) (Step 3) (Step 4) Data loading (Step 1) Automatic registration

39. Pujol S., Gollub R. National Alliance for Medical Image Computing Automatic registration Select the panel Auto in the module Alignments. Set the Volume to Move to reg-nhdr (I 1 ) and the Reference Volume to I (I 2 ). Select the Registration Mode to Intensity

40. Pujol S., Gollub R. National Alliance for Medical Image Computing Automatic registration Slicer has two modes for intensity based registration: Semi-automatic mode : Fine or Coarse Automatic mode : Good and Slow, or Very Good and Very Slow

41. Pujol S., Gollub R. National Alliance for Medical Image Computing Automatic registration Semi-automatic mode (Fine or Coarse): the registration goes on in the background, and the transformation can be interactively manipulated during the process. The user has to stop manually this mode to obtain the final value of the registration matrix.

42. Pujol S., Gollub R. National Alliance for Medical Image Computing Automatic registration Automatic mode (Good and Slow, or Very Good and Very Slow): the registration goes on in the background, and repeats until a predefined criterion is reached* set in the parameters. * The default parameters can be modified to adjust the specificity of your data.

43. Pujol S., Gollub R. National Alliance for Medical Image Computing Choose the Run Objective Good and Slow and click on the button Start. Automatic registration

44. Pujol S., Gollub R. National Alliance for Medical Image Computing Automatic registration A Rigid Registration window appears, and Slicer displays the progress of the registration process.

45. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Slicer displays the result of the automatic registration of the two volumes.

46. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Slice through the volume to visualize the result of the registration

47. Pujol S., Gollub R. National Alliance for Medical Image Computing Observe the results of the registration in the occipital bone (axial slice 0). I2 T(I1) Registration result

48. Pujol S., Gollub R. National Alliance for Medical Image Computing Observe a better alignment of the boundaries between the cerebrum and the cerebellum (sagittal slice 0). I2 T(I1) Registration result

49. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Observe the results of the registration on the lateral edge of the brain (axial slice 30). I2 T(I1)

50. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result: summary Before registration After registration

51. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Click on the Props tab to display the parameters of the resulting rigid transformation T between the two datasets.

52. Pujol S., Gollub R. National Alliance for Medical Image Computing Overview Step 1: Load data and visualize mis- alignment Step 2: Manually define the initial transformation Step 3: Complete the registration by using the mutual information algorithm Step 4: Refine the registration by using the semi-automatic mode (optional) Step 5: Apply the registration transform

53. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Note a tilt and a misalignment in the Inferior-Superior direction: observe the difference in shape of the ventricles in T(I1) and I2. I2 T(I1)

54. Pujol S., Gollub R. National Alliance for Medical Image Computing Registration result Note a misalignment in the Inferior-Superior direction: observe the difference in white matter localization on the middle line in T(I1) and I2. I2 T(I1)

55. Pujol S., Gollub R. National Alliance for Medical Image Computing Processing pipeline Automatic registration Final Transform Semi-automatic refinement of the registration no yes Manual registration Initial transform Result OK ? (Step 2) (Step 3) (Step 4) Data loading (Step 1)

56. Pujol S., Gollub R. National Alliance for Medical Image Computing Refine the registration Click on the tab Auto and select the mode Coarse to refine the result of the registration. Click on Start to launch the algorithm.

57. Pujol S., Gollub R. National Alliance for Medical Image Computing Refine the registration Left-click in the sagittal view, and slightly move the slice with the mouse to correct the tilt.

58. Pujol S., Gollub R. National Alliance for Medical Image Computing Refine the registration Left-click in the saggital view and slightly move the slice down with the mouse to correct the vertical misalignment

59. Pujol S., Gollub R. National Alliance for Medical Image Computing Refine the registration Observe Slicer iterating the registration algorithm, and updating the position of the volume in the three anatomical views. Iterate the process until you are satisfied with the alignment of the two volumes.

60. Pujol S., Gollub R. National Alliance for Medical Image Computing Refine the registration Click on Stop to terminate the semi-automatic registration process Information on details and performances of the registration algorithm are available at http://www.itk.org/HTML/MutualInfo.htm

61. Pujol S., Gollub R. National Alliance for Medical Image Computing Example of registration result Before registration After automatic registration The results might differ very slightly: these pictures show an example of a good outcome. After semi-automatic refinement

62. Pujol S., Gollub R. National Alliance for Medical Image Computing Processing pipeline Automatic registration Final Transform Semi-automatic refinement of the registration no yes Manual registration Initial transform Result OK ? (Step 2) (Step 3) (Step 4) Data loading (Step 1)

63. Pujol S., Gollub R. National Alliance for Medical Image Computing Overview Step 1: Load data and visualize mis- alignment Step 2: Manually define the initial transformation Step 3: Complete the registration by using the mutual information algorithm Step 4: Refine the registration by using the semi-automatic mode (optional) Step 5: Apply the registration transform

64. Pujol S., Gollub R. National Alliance for Medical Image Computing By applying the registration transform to the initial volume I 1, well generate a new volume spatially aligned with the volume I 2. This allows the extraction of complementary information from the two volumes. Apply the registration transform Image Space 1 Image Space 2 I2I2 I1I1

65. Pujol S., Gollub R. National Alliance for Medical Image Computing Apply the registration transform Click on Modules Examples and select the module TransformVolume. In the following section, well use the transform Volume module to resample the initial volume reg-nhdr through the transform transform0 calculated by the registration.

66. Pujol S., Gollub R. National Alliance for Medical Image Computing Select the Reference Volume reg-nhdr and the Resample Mode ReferenceVolume Choose the Interpolation Mode Cubic Click on Show Preview to visualize a preview of the transformed volume. Apply the registration transform

67. Pujol S., Gollub R. National Alliance for Medical Image Computing A pop-up window displays a preview of the resampled volume, after applying transform0. Click on DoTransform to apply the final transform calculated through the registration to the volume reg-nhdr. Apply the registration transform

68. Pujol S., Gollub R. National Alliance for Medical Image Computing Slicer generates the final volume xformed-reg-nhdr, which has the same orientation and spacing as the volume reg-nhdr. Apply the registration transform (See SlicerTraining7: Saving Data to save the volume on disk.)

69. Pujol S., Gollub R. National Alliance for Medical Image Computing Conclusion Registration of a pre-operative dataset with an intra-operative dataset Initial registration by manual alignment Automatic and semi-automatic registration by maximization of mutual information

70. Pujol S., Gollub R. National Alliance for Medical Image Computing Appendix: TransformVolume The TransformVolume module offers the possibility to resample several volumes using the same transform. All the volumes will then be aligned to the same voxel space.

71. Pujol S., Gollub R. National Alliance for Medical Image Computing Appendix: Transform Volume Exemplar Case The Expectation-Maximization (EM) algorithm* performs automatic segmentation of brain structures from MR data. Multiple channel resampling can be accomplished using the TransformVolume module. atlas T1 normalized T(T2) normalized (*) See SlicerTraining11:EMBrainAtlasClassifier. White matter Grey matter CSF