1. Applied Anatomical Diagnostic Framework for Visualizing the Human Body in a 3-D, Immersive, Navigable and Interactive VR Environment May 4, 2012 Steven Beaudoin, BS George D. Lecakes Jr., MS Tony Aita, BS Lawrence Weisberg, MD Michael Goldberg, MD Vijay Rajput, MD Shreekanth Mandayam, PhD

2. Contributing Authors Steven Beaudoin George Lecakes, MS Shreekanth Mandayam, PhD Michael Goldberg, MD Vijay Rajput, MD Lawrence Weisberg, MD

3. Overview Background Objectives Approach Results Future Goals Conclusions Acknowledgements

4. -US Department of Education, 2010 “ To consider integration of 21 st century competencies, expertise, such as critical thinking, complex problem solving, collaboration, multimedia communication, and technological competencies demonstrated by professional disciplines” Cooper Medical School

5. CAVE® at Medical School Basic anatomy Clinical correlation Systems based practice educational tool Simulation scenarios (simulated patients / conditions)

6. Background The Rowan University CAVE®

7. Background The Rowan University CAVE® 7.5'h x 10'w x 10'd tracked virtual environment covering 3-walls and a floor 4- active stereo DLP projectors, 3,000 ANSI lumens, 1400x1050 resolution, with short-throw zoom lens CAVE® structure fabricated with extruded & powdercoated 80/20 aluminum, 8-camera WorldViz IR position tracking system CrystalEyes® active shutter glasses 6 - PC computer cluster WorldViz Vizard, Autodesk 3D Studio Max / Maya

8. Objectives Provide intuitive correlation of clinical information Utilize a CAVE® virtual reality (VR) environment to enhance viewing of data sets Ask “what if?” questions and simulate multiple scenarios for pathological/anatomical diagnosis Utilize multiple diagnostic imaging procedures for prognostic capability

9. Approach Data preparation CT- dataset of volunteer torso from the skull to the pelvis Consisted of 909 images at 1mm slices Provided in DICOM data format with each image having a different window center and window width Data Preparation Data Preprocessing Data Visualization Real-Time Processing

10. Approach Data preprocessing Utilize image processing tool (MATLAB) to convert images into a standard window center and window width Converted the various ranges into standard grayscale range (0-255) 0 – Black 255 – White Export images to CAVE compatible file format (tga,png,jpeg) Data Preparation Data Preprocessing Data Visualization Real-Time Processing

11. Approach Data Preparation Data Preprocessing Data Visualization Real-Time Processing Raw data Processed Data

12. Approach Data Visualization Load images (textures) sequentially Textures are applied to a series of evenly spaced polygonal planes Each plane represents one cross- sectional slice of patient data Images are given transparency Images are rendered back to front Data Preparation Data Preprocessing Data Visualization Real-Time Processing

13. Approach Pixel Values Values from a CT scan represent different densities Radiolucent regions are low density – air Radiodense regions are higher density – bone Data Preparation Data Preprocessing Data Visualization Real-Time Processing

14. Approach Data Manipulation and Interface 1.Grayscale histogram – Cull pixel values from the histogram using a range slider. 2.Pseudo-color histogram – Cull pixel values and color the ranges with red, green, or blue ranges. 3.Clip Plane Mode – Clip the polygonal planes with an OpenGL clipping plane in any orientation. Xbox 360® Controller – All functionality mapped to buttons Data Preparation Data Preprocessing Data Visualization Real-Time Processing

16. Educational / Future Medical Applications Learn anatomy / clinically applied anatomy in 3D virtual reality Learn anatomy / clinically applied anatomy in 3D virtual reality Anatomic pathology and radiological integration Anatomic pathology and radiological integration Correlation of surface and organ anatomy for surgical procedure Correlation of surface and organ anatomy for surgical procedure Application of 3D virtual reality to understand system based practice – e.g. Assess Home environment for safety for geriatric patient Application of 3D virtual reality to understand system based practice – e.g. Assess Home environment for safety for geriatric patient

17. Other uses of CAVE®

18. Work In Progress Fuse multiple image data sets (MRI, CT, PET) to distinguish redundant and complementary information for clinically applied anatomical education

19. Work In Progress

20. Acknowledgments We gratefully acknowledge the assistance of: Dr. H. Warren Goldman, MD, PhD Professor and Chair of Neurosurgery Cooper Hospital, Camden, NJ Mr. Anthony Aita Neurosurgery Department Cooper Hospital, Camden, NJ

22. Approach