1. Tracking Objects of Interest from CT/MR Data using Dynamic 3D Ultrasound http://ssim.eng.wayne.edu/ssimi/cares/classes/group7/group7.html Group 6: Michael Shetliffe Mohammad Yaqub Mohammed Alam

2. Outline Problem Statement Problem Statement Background & Significance Background & Significance Overall Aims Overall Aims Methods Methods Ultrasound Ultrasound MRI MRI Registration Registration Results Results Problems Problems Solutions Solutions Competition Competition Future Work Future Work Conclusion Conclusion

3. Problem Statement

4. Problem There is a critical need to update information based on changes occurring during surgery. Changes: Shift Shift Deformation Deformation Vascular movement Vascular movement

5. Overview of Proposal The overall objective of this proposal is to show feasibility and develop a cost-effective and efficient approach to monitor and predict deformation during surgery, allowing accurate, and real-time intra-operative information to be provided reliably to the surgeon. The central hypothesis is that deformation can be followed intra-operatively using ultrasound technology.

6. Background & Significance Approaches to update the pre-op data: Intra-operative MR Intra-operative MR Obtain updated (lower-resolution) MR data Obtain updated (lower-resolution) MR data Costly, significant setup time, OR compatibility considerations Costly, significant setup time, OR compatibility considerations Finite Element modeling of surrounding tissues Finite Element modeling of surrounding tissues Challenges of parameter estimation Challenges of parameter estimation Parameter variation with physiologic changes Parameter variation with physiologic changes Long computation times Long computation times Use 3D Ultrasound Use 3D Ultrasound Convenient, safe, cheap Convenient, safe, cheap

7. Overall Aims CT US MR Registration Evaluate the effectiveness, accuracy, and usefulness of any techniques that were used Testing the method Image Acquisition Segmentation

8. Image Acquisition – 3D Ultrasound

9. What is Ultrasound ? High-energy sound waves (ultrasound) are bounced off internal tissues or organs and make echoes. (2-13 MHz) Pros: Non invasive Non invasive Real time imaging Real time imaging Cons: Cannot image bony structures Cannot image bony structures Poor native resolution Poor native resolution Image depends on time to echo (pixel position) and echo strength (pixel intensity)

10. Ultrasound Images (Breast) Objects of Interest Spurious Artifacts Bench Surface Reflection Characteristic US Noise

11. 3D Ultrasound – Background 3D Ultrasound: 3D Ultrasound: Tracks 2D ultrasound probe to build 3D volume from ultrasound “slices” Tracks 2D ultrasound probe to build 3D volume from ultrasound “slices” Can use conventional, portable ultrasound equipment to obtain 3D volumes Can use conventional, portable ultrasound equipment to obtain 3D volumes Still relies on ultrasonic backscattering from tissue structures  intrinsically noisy Still relies on ultrasonic backscattering from tissue structures  intrinsically noisy  Image Processing becomes especially important  Image Processing becomes especially important

12. 3D Ultrasound - System 2D Ultrasound Polaris Tracking System “Stradx” Software Slices in 3D Space 2D Screen Frames Probe 3D Position Frames / sec & Probe Motion

13. 3D Ultrasound - Output Output from Stradx: Output from Stradx:.sx file: Text-based file containing 3D positions, orientations, sizes, times, etc. of individual 2D slices.sx file: Text-based file containing 3D positions, orientations, sizes, times, etc. of individual 2D slices.sxi file: Binary file of 8-bit pixel values.sxi file: Binary file of 8-bit pixel values What we need: 3D image data in “conventional” format (e.g. dicom, Analyze) that can be read into other systems for processing 3D image data in “conventional” format (e.g. dicom, Analyze) that can be read into other systems for processing (Prior to starting this project, it was thought that this was what we would have.) (Prior to starting this project, it was thought that this was what we would have.)

14. 3D Ultrasound - Output Examples: Examples: 1-Pass Probe Motion

15. 3D Ultrasound - Output Examples: 2-pass had problems with “inter-pass” alignment Examples: 2-pass had problems with “inter-pass” alignment 2-Pass Probe Motion

16. 3D Ultrasound - Post Processing

17. 2 Additional Tools (Provided as part of the Stradx distribution): 2 Additional Tools (Provided as part of the Stradx distribution): SelectSX: (adjust for data too “dense”) SelectSX: (adjust for data too “dense”) StackSX: (create evenly spaced, uniformly aligned slices) StackSX: (create evenly spaced, uniformly aligned slices) 3D Ultrasound - Conversion

18. Read new.sx file as “Raw” 8-bit data into a medical imaging tool (xmedcon, osirix, 3d-slicer) Read new.sx file as “Raw” 8-bit data into a medical imaging tool (xmedcon, osirix, 3d-slicer) Region of Interest moves within resliced image between slices.

19. 3D Ultrasound - Conversion At this point, we can treat our US data as a standard set of images, and export to other convenient formats At this point, we can treat our US data as a standard set of images, and export to other convenient formats Slice separation distance determined from.sx file position data.

20. 3D Ultrasound – Post-Processing Basic Objectives: Basic Objectives: Reduce background noise Reduce background noise Segment (or at least highlight) object(s) of interest Segment (or at least highlight) object(s) of interestApproach: Stradx has some inbuilt segmentation/registration capabilities – not used Stradx has some inbuilt segmentation/registration capabilities – not used Used MATLAB to investigate feasible techniques that could later be integrated into a stand-alone system. Used MATLAB to investigate feasible techniques that could later be integrated into a stand-alone system.

21. 3D Ultrasound – Post-Processing Noise Reduction Techniques: Noise Reduction Techniques: Original Image Median FilterGaussian Blur

22. 3D Ultrasound – Post-Processing Noise Reduction Techniques: Noise Reduction Techniques: Original Image Median FilterGaussian Blur

23. 3D Ultrasound – Post-Processing Noise Reduction Techniques: Noise Reduction Techniques: Original Image Median FilterGaussian Blur

24. 3D Ultrasound – Summary In this portion of work we have: In this portion of work we have: Acquired updated ultrasound image data that better meets our requirements for working with objects of interest. Acquired updated ultrasound image data that better meets our requirements for working with objects of interest. Converted the image data to formats that can be easily used in a wide variety of medical imaging systems. Converted the image data to formats that can be easily used in a wide variety of medical imaging systems. Processed the ultrasound image data to enable higher quality results from subsequent registration with other modalities. Processed the ultrasound image data to enable higher quality results from subsequent registration with other modalities.

25. Image Acquisition - MRI / CT

26. What is MRI? MRI stands for Magnetic Resonance Imaging. The MR images used to image internal structures of the body, particularly the soft tissues. We used GE 4 tesla MRI machine to get some MR images for the phantom we have. MR brain image

27. GE MRI machine

28. What is CT? CT stands for Computed Tomography. CT image is a specialized form of x-ray imaging. It shows bony structures. We used CT images in our project. It did not give good results. CT brain image

29. MRI/CT Images (Breast) MR T-2 Images MR T-1 Images CT Images No big difference because the phantom is MR incompatibleNo internal information

30. Image Segmentation MRI

31. MRI Segmentation No quantitative phantom information. Introduce artificial shift of objects of interest. We did the movement using a manual linear interpolation method. We did the movement using a manual linear interpolation method.Segmentation Object matching Object matching Thresholding Thresholding

32. Example Original MRI sliceManually shifted object of interest

33. Example (cont.) Original segmented object of interest inside an MR image Automatically segmented object of interest inside the shifted MR image

34. Image Registration

35. Registration Registering different modalities: Original MR to shifted MR Original MR to shifted MR Original MR to CT Original MR to CT Original MR to Ultrasound Original MR to Ultrasound Manual landmark. An automated registration - mutual information. Need more human interaction.

36. Example Original fixed modalityMoving modality (The Shifted object of interest)

37. Example (cont.) A slice that contains the registered data Showing both the original & the shifted points Original object of interest shifted object of interest

38. Example (cont.) MR image with three objectsUltrasound image with three objects Need for deformable registration method

39. Problems, Proposed Solutions and Future Work

40. ProblemsPhantom Echo-sensitive material Echo-sensitive material Water Content Water Content MRI Compatible MRI Compatible Atlas AtlasUltrasound Quality of Images Quality of Images System Calibration System CalibrationRegistration

41. Proposed Solutions - Phantoms Multi modality Phantoms - $ 2000.00

42. Proposed Solutions - Ultrasound Better Ultrasound Transducer

43. Proposed Solutions - Registration Existing Research

44. Competitors Ultrasound Probes - Philips, Siemens, GE Registration Techniques – Related research available Overall Intraoperative Tracking System using Ultrasound – GE Overall Intraoperative Tracking System using Ultrasound +preop MRI - ????

45. Future Work Research areas Better post processing of US images Better post processing of US images Newer registration techniques Newer registration techniques Faster and effective calibration methods Faster and effective calibration methods Bring all individual modules to work as a single system Testing and evaluation techniques

46. Conclusion A very feasible and highly applicable research area. Cost effective when compared to I-MRI Cost effective when compared to I-MRI Relatively “real time” data Relatively “real time” data Accuracy from fusion with pre-op MR Accuracy from fusion with pre-op MR Improved surgical outcome Improved surgical outcome Works in current clinical settings Works in current clinical settings Adaptable to other surgical procedures involving: Adaptable to other surgical procedures involving: Brain Brain Breast Breast Prostate Prostate Others … Others …

47. Thank You! http://ssim.eng.wayne.edu/ssimi/cares/classes/group7/group7.html