1. Remote Surgical Robotics Control Systems and Human-Machine Interfacing

2. What is a remote surgical robot? Da Vinci Surgical System, Intuitive Surgical, Inc.

3. What is a remote surgical robot? The surgical console measures the movements of the surgeon.

4. What is a remote surgical robot? The robot replicates the movements of the surgeon at the operating table.

5. What is a remote surgical robot? The movements include the manipulation of surgical implements.

6. The use of remote surgical robots is growing Over 1400 systems sold as of 2010. Used in the treatment of: Bladder Cancer Colorectal Cancer Coronary Artery Disease Endometriosis Gynecologic Cancer Heavy Uterine Bleeding Kidney Disorders Kidney Cancer Mitral Valve Prolapse Obesity Prostate Cancer Throat Cancer Thyroid Cancer Uterine Fibroids Uterine Prolapse

7. Remote surgical robots has advantages Possible tele-operations Less direct human contact at operating table (less chance of breaking sterile field) Enables smaller movements  smaller openings  less chance of infection  fast healing times  more precise surgical techniques

8. Surgeons control the robot entirely by visual feedback A high definition video system provides 3D images

9. Could tactile feedback help? Haptic interface Tying sutures is a common surgical technique

10. Does tactile feedback help the surgeon? “… visual force feedback primarily benefits novice robot-assisted surgeons, with diminishing benefits among experienced surgeons.” Reiley, CE, et al. Effects of visual force feedback on robot-assisted surgical task performance. J Thorac Cardiovasc Surg 2008;135:196-202 “… force feedback is helpful in this blunt dissection task because the artery is stiffer than the surrounding tissue.” Wagner CR, et al. Force feedback in a three-dimensional ultrasound-guided surgical task. Proc.of the 14th International Symp. on Haptic Interfaces for Virtual Environ. and Teleoperator Systems (2006).

11. Does tactile feedback help the surgeon? No force-feedback in Da Vinci Surgical Systems Prototypes with force-feedback are on the way (Neuroarm.org)

12. Problem Statement A company is investigating whether to add force- feedback capabilities to their surgical robots. Specifically, they want a proof of concept demonstration that a neurosurgeon will be able to accurately identify the surface of the brain and depth at which puncturing of tissue by a probe occurs.

13. Tactile sense augments vision

14. What is a surgical robot? (really) Manipulandum Position Sensors Interface Robotic arms Surgeon’s hands Robot’s hands

15. Our prototype surgical robot Joystick Joystick Sensors Interface Motor servo Your hand position Probe position

16. How do you add force-feedback? Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface

17. Joystick and Joystick sensors Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface Logitech Force 3D Pro Joystick with USB input (force level) and output (joystick position)

18. Interface Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface Cerbot 32mx4 Microprocessor with USB ports, servo controller, and analog to digital converter

19. Motor Servo Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface GWS Servo rotates to an angular position in proportion to its input

20. Probe and Brain Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface A blunt probe: 6-32 screw (imagine how little force a needle would produce!)

21. Probe and Brain Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface Brain: gelatin poured into a mold; sometimes called a “phantom brain”

22. Force Sensor Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface Thin film sensor sandwiched into the linkage between the servo and probe.

23. Force Sensor: Force sensing resistor (FSR) The sensor has two electrical connections (leads), and there is an electrical resistance between the leads. 7.62 mm

24. Force Sensor: FSR As force is applied to the FSR layer, the conductive protrusions within the ink make contact with the active area. Resistance decreases as more force is applied and more contacts are made.

25. Force Sensor: FSR Note: there is a “threshold” of ~10 grams before a resistance change occurs.

26. How can you change the system? Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface

27. How can you change the system? Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface

28. How can you change the system? Joystick capable of generating force Joystick Sensors Interface Motor servo Your hand position Force to move probe through Brain tissue (gelatin) Force Sensor Interface

29. Overview of concepts explored (It’s alright if you don’t understand what all of these mean; You’ll learn!) 1.Sensitivity (resolution) vs range (saturation) tradeoff 2.Varying gain allows you optimize the tradeoff 3.How to measure aspects of human perception? 4.What are human factors and how they influence the design of human-machine interfaces? 5.What does a phantom brain feel like?

30. Overview of concepts explored 4.What are human factors and how they influence the design of human-machine interfaces? Design requires that you meet the desired specifications: “a neurosurgeon will be able to accurately identify the surface of the brain and depth at which puncturing of tissue by a probe occurs” Specifications? The robot must be able to provide tactile feedback within the human operator’s sensory capabilities (human factors)

31. Summary of Activities Each day is an hour long class Day #2: Optimizing the joystick position to servo position gain. Day #3: Familiarization with and calibrating force sensor capabilities. Day #4: Matching force sensor capabilities with the objectives of the problem. Day #5: Familiarization with joystick force feedback capabilities and testing the limits of human perception.

32. Summary of Activities Each day is an hour long class Day #6: Feel a brain! Probe the gelatin brain and optimize your ability to perceive the surface of the brain and puncture depth. Day #7: Finish any testing and discussion of results. Day #8: Wrap up lecture. Optional: Optimizing the identification of the brain surface with different probe end geometries.

33. Group Logistics Each group will consist of 3-4 students (assigned by instructor). Groups are encouraged to discuss with each other groups their solutions to the assignments within the classroom activities. Each group will address a set of homework problems after each class. The answers (one per group) are due at the beginning of the next class period. The final assignment is to be done on an individual basis.