1. MultiModality Registration Using Hilbert-Schmidt Estimators By: Srinivas Peddi Computer Integrated Surgery II April 27 th, 2001 Final Presentation

2. Outline Brief refresher of my project Things accomplished in the project Things left to improve upon Future Directions Conclusions

3. The Project T1 PD T2 I want to be able to register different modalities of MR images accurately. This means coming up with a new registration algorithm and getting around the intensity difference problem.

4. Original Maximal Deliverables To register the three different modalities accurately by first using Bayesian Segmentation to circumvent the intensity-difference problem. To compare this approach with other multimodality registration algorithms such as the Maximization of Mutual Information algorithm. To examine the feasibility of using the Hilbert- Schmidt algorithm in real-life applications.

5. What Has Been Achieved? T1 PD T2 Original images PD Segmentation T1 Segmentation T2 Segmentation Original images  Bayesian Segmentation Original images  Bayesian Segmentation  Switching of Intensities PD Switched Segmentation T1 Switched Segmentation T2 Switched Segmentation

6. Once the segmentation process is done, and the intensities have been switched, we can actually do the registration. We apply the Hilbert-Schmidt algorithm which uses a minimum mean-squared (MMSE) estimator. Registration is achieved by finding the element of the special Euclidean group (SE n ) that minimizes the error. Registration

7. Registering different modalities The maximum aim of the project has been achieved. At this point, we are able to register PD with T1 (which you saw in the checkpoint presentation) but we can now also register PD with T2 and T1 with T2. These registrations are possible at different noise levels as long as the segmentation is reasonable.

8. How accurate is this method? To examine how accurate something is, we must first define an error measure. The one that I will be using is called the Hilbert-Schmidt bound. A second thing that can help in getting an intuitive feel about the accuracy of the algorithm, is having another algorithm to compare it to. In this presentation, I will be using the Maximization of Mutual Information Algorithm.

9. What is the Hilbert-Schmidt bound? The Hilbert-Schmidt norm is defined as the norm of a matrix. Example: A = [2 -1 -4-2] ||A|| = [ 2 2 + (-1) 2 + (-4) 2 + (-2) 2 ] 1/2 = 5 The Hilbert-Schmidt bound (HSB) is defined as the matrix norm of the difference between the true matrix transformation and the calculated matrix transformation.

10. Maximization of Mutual Information This algorithm was implemented by Wells et al. at the SPL in 1996. Since then, it has become the registration tool of choice when doing multimodality registration. The algorithm attempts to find the registration by maximizing the information that one volumetric image provides about the other.

11. Comparison of the two algorithms As one can see, the Hilbert-Schmidt algorithm outperforms the mutual information algorithm at high noise but at low noise, they both register the images accurately.

12. Pros and Cons Accuracy: As mentioned, it seems that the Hilbert-Schmidt algorithm outperforms the mutual information algorithm in this category, especially at high noise levels. Speed: The Mutual Information algorithm runs much faster, at least for now, especially because it does not do much preprocessing. Generality: The Mutual Information algorithm assumes no a priori relationship between the two modalities and hence all modalities can be registered using the same algorithm. The Hilbert-Schmidt algorithm is striving to do the same.

13. Pros and Cons Cont’d Ease of use: Since the Mutual Information algorithm has less steps or at least is better integrated, it is easier to use. The hope is that later, the segmentation and the registration can be coupled in one program in which case, the Hilbert-Schmidt would also become easier to use. Robustness: Since the Mutual Information algorithm is essentially a simpler algorithm with less steps, it is very robust. With time, I hope to make the Hilbert-Schmidt algorithm as robust.

14. Improvements to be made Gradient descent algorithm has been implemented but can be improved upon especially by using a ‘blurring’ algorithm and also by selecting random points more wisely. The algorithm needs to be extended so that one can register 3D volumes rather than just 2D images which is what we have now.

15. Gradient Descent Algorithm What I presently do is pick a series of random points from 0 to 360, and then march in the direction of increasing probability. It would be nice to add two layers of random points so that there would be more density of random points near the maximum. Being able to blur the image without shifting the maximum would also be helpful.

16. Future Directions From the analysis so far it seems that the Hilbert- Schmidt algorithm outperforms other algorithms at higher noise levels. Therefore, it is possible that the HS algorithm might be very useful in functional modalities such as fMRI and PET where the SNRs are considerably higher than MR or CT. The segmentation algorithm can be improved to incorporate things such as windowing so as to improve the accuracy of the algorithm.

17. Feasibility of HS algorithm in real-life circumstances The algorithm developed in this project definitely has potential for real-life applications. However, some key improvements have to be made first which integrate the various steps of the protocol better and the algorithm needs to extended to 3D volumes. As far as speed goes, computers are becoming more powerful as we speak, and it does not seem that this will be a limiting factor in the application of this algorithm.

18. What have I learned? Projects are a lot of work, and many times, things do not go according to plan. Always have contingencies. Presentation of ideas well is very important. Almost as important as the project itself. Presentation is also a lot more work than I thought. There are many tools out there that can make one’s life easier. It is much better to use these tools than trying to reconstruct the wheel.

19. What would I do differently? Have a more detailed plan at the beginning of the project. There were points when I wasn’t sure if I was ahead or behind schedule. Have a thorough understanding of the theory of the topic before starting to code. It can save a lot of time and effort! Efficiency and ease-of-use are the two very important features of an algorithm. I wish I had started working earlier on those aspects of the algorithm.

20. Conclusions I have a learned a lot from this class and there are some things that I would do differently if I have were to start from the beginning again. However, all in all, I think that the project went relatively smoothly and that the Hilbert-Schmidt algorithm developed here has potential for success in the future. The most important conclusion to draw from this presentation though is that …

21. I’m DONE!!!