1. Mechatronics Lab Workshop i Visualisering Presentation Haptic and Visual Simulation of a Material Cutting Process Using Patient Specific High Resolution CT-data for Haptic- and Graphic rendering Magnus G. Eriksson Supervisor: Professor Jan Wikander Co-supervisor: Professor Hans von Holst CTV (Center for Technology and Health Care) The Mechatronics Lab/Machine Design, KTH, Stockholm Department of Neuronic Engineering KTH-STH, Huddinge

2. Mechatronics Lab Background Since 1980s Since 1990 Since 1990s

3. Mechatronics Lab Temporal Bone Surgery Simulator

4. Mechatronics Lab Education of Surgeons “See one, do one, teach one” Patients risky situation Ethically and economically unacceptable Cadavers, plastic models or animals (high cost, ethical problems, difficulties of training results) Exampel

5. Mechatronics Lab Training in VR Simulators Avoid patients and cadavers Performance feedback Not time related Pre-operation planning Older, experienced surgeons Rare pathologies Reduce expensive costs First 60 patients Simulator training (Ahlberg et al.)

6. Mechatronics Lab VR and Haptic Simulators Used Today Film example from Simulatorcentrum AccuTouch® Endoscopy Simulator LapSim® Laparoscopy Simulator Metrics and Certified by US Food and Drug Administration (FDA)

7. Mechatronics Lab Pos. Force Feedback. Pos. ”Real Force” Drilling Operation Anatomical Model ”Calc. Force” Visual Feedback Visual Feedback surgeon 3D graphic slave master The Temporal Bone Surgery System patient

8. Mechatronics Lab Research Goal and Focus Goal: To develop a haptic and VR system for training and educating surgeons who practice bone milling Focus: 1.To develop a VR system for realistic 3D representation of the human skull, including the changes resulting from the milling process 2.To develop an efficient algorithm for realistic haptic feedback to mimic the milling procedure using CT-data of the skull Real time demands, without artifacts or delays when removal of material

9. Mechatronics Lab System Design SenseGraphics H3DAPI Initialization Graphic Thread 30 Hz 3D Visualization Material Removal Haptic Thread 1000 Hz Collision Detection Calculate Force

10. Mechatronics Lab Possible VR Haptic and Milling Applications Temporal bone surgery Craniofacial surgery Dental tooth milling Vertebral operating procedures Freeform design Research Goal: To develop a haptic and VR system for training and educating surgeons who practice bone milling

11. Mechatronics Lab Conclusion A haptic and VR surgical milling simulator prototype Patient specific DICOM data Efficient graphical rendering Haptic rendering to avoid fall-through problems Real time requirements when removing material Film, tooth milling

12. Mechatronics Lab Future Work Investigate various force models (3-DOF vs 6-DOF, mill is turned on- off, material removal rate) and benchmark Other applications, bone fractures, sculpting etc… User interface and virtual environments / visualisation Validate the simulator together with Simulatorcentrum Re-Design the haptik device and API (Matlab/Simulink) Investigate the economical and ethical benefits of using simulators Film, skull bone milling

13. Mechatronics Lab Initialization Volumetric data from a DICOM-file Density and gradient 3D matrices Octree node structure containing the voxel data  Range of x, y, and z coordinates.  Max/min density values.

14. Mechatronics Lab Graphic Rendering Update rate > 30Hz and a latency of less than 300ms Graphic thread, 30 Hz Check for milling Marching cubes algorithm on updated tree nodes OpenGL to create the shape of the object Update density and gradient values X_max Y_max r d Y_min X_min Film, skull bone milling

15. Mechatronics Lab Haptic thread, 1000 Hz Haptic Rendering Collision detection A force based on a proxy-probe method Send force to the haptic device If milling, add vibration force

16. Mechatronics Lab Verification of the Haptic Algorithm A Stiff Surface A Soft Surface