1. Seminar in Computer-Assisted Surgery Medical Robotics Medical Image Processing LECTURE 1 Dr. Leo Joskowicz Institute of Computer Science The Hebrew University

2. Computer-Assisted Surgery Medical Robotics Medical Image Processing LECTURE 1 1.What‘s in a surgery 2.Technical tools in CS 3.CAS systems

3. CAS, Spring 2001 © L. Joskowicz 3 PAST: Cut, then see

4. CAS, Spring 2001 © L. Joskowicz 4 PRESENT: See, then cut

5. CAS, Spring 2001 © L. Joskowicz 5 FUTURE: Combine, see, minimally cut

6. CAS, Spring 2001 © L. Joskowicz 6 How do surgeries proceed? Diagnosis –based on physical exams, images, lab tests Preoperative planning –determine the surgical approach –elaborate intraoperative plan (path, tools, implants) Surgery –prepare patient and assess condition –acquire intraoperative images, adapt and execute plan Postoperative follow-up –exams, lab tests, images to be corroborated

7. CAS, Spring 2001 © L. Joskowicz 7 Treatment procedures Invasive –neurosurgery: tumor removal –hear surgery: clogged arteries, transplants –orthopaedic surgery: spine, hip replacement, knee, fractures –gall bladder removal, prostate, various cancers Non-invasive –radiation therapy –kidney stone pulverization

8. CAS, Spring 2001 © L. Joskowicz 8 Medical imaging modalities Preoperative –Film X-rays, Digital X-rays, Ultrasound, Angiography, Doppler, …. –Computed Tomography (CT), Magnetic Resonance (MR), Nuclear Medicine (PET, SPECT, …) Intraoperative –X-ray fluoroscopy, ultrasound –video images (laparoscopy, arthorscopy) –Open MR

9. CAS, Spring 2001 © L. Joskowicz 9 Medical imaging modalities: X-rays Film or Digital X-ray X-ray Fluoroscopy

10. CAS, Spring 2001 © L. Joskowicz 10 Medical imaging modalities: continuous X-ray angiography

11. CAS, Spring 2001 © L. Joskowicz 11 Medical imaging modalities: Ultrasound

12. CAS, Spring 2001 © L. Joskowicz 12 Medical imaging modalities: CT Single slice Series of parallel slices 2mm apart

13. CAS, Spring 2001 © L. Joskowicz 13 Medical imaging modalities: MRI Good imaging of soft tissue

14. CAS, Spring 2001 © L. Joskowicz 14 Medical imaging modalities: Nuclear medicine (PET, SPECT, NMR) Functional imaging: colors indicate electrical activity

15. CAS, Spring 2001 © L. Joskowicz 15 Medical imaging modalities: video TV quality image from small camera (laparoscope or endoscope)

16. CAS, Spring 2001 © L. Joskowicz 16 Surgical approaches Open surgery –area of interest directly exposed by cutting – direct sight and touch of anatomy by surgeon –direct access but causes additional damage Closed surgerynot always feasible –indirect access to anatomical area of interest –no direct visual sight or tactile feel –catheterization, biopsies –intraoperative imaging is often required –require more skills: lengthier, more difficult Diagnostic surgery

17. CAS, Spring 2001 © L. Joskowicz 17 Minimally invasive surgery Provides treatment through small incisions Uses imaging equipment for seeing and instruments for touching Advantages: less damage, faster recovery Disadvantages: hand/eye coordination, time Examples: –brain tumor removal, laparoscopic surgery

18. CAS, Spring 2001 © L. Joskowicz 18 Laparoscopic surgery

19. CAS, Spring 2001 © L. Joskowicz 19 Brain surgery

20. CAS, Spring 2001 © L. Joskowicz 20 Total Hip replacement -- principle

21. CAS, Spring 2001 © L. Joskowicz 21 Total hip replacement procedure

22. CAS, Spring 2001 © L. Joskowicz 22 What is required to perform surgery? Knowledge intensive task –anatomy, procedures, cases –experience, skills, customization and generalization Manual and cognitive skills –dexterity, precision, strength, tool manipulation –spatial orientation and navigation Determination –information integration –judgement, decision, execution

23. CAS, Spring 2001 © L. Joskowicz 23 Medical and surgical trends Imaging improved dramatically diagnosis –started with X-rays last century –30% of all cases use images Move towards minimally invasive procedures –introduced in the mid ‘70s, slow acceptance (laparoscopy) –the method of choice now More precise and delicate procedures Development of sophisticated surgical hardware High degree of craftsmanship and skills

24. CAS, Spring 2001 © L. Joskowicz 24 Socio-economical medical trends Increase of aging population and associated problems: tumors, osteoporosis, Alzheimers Larger population volumes Universal, first rate, highly specialized care Health care costs reduction (managed care) Higher patient requirements Legal and regulatory aspects

25. CAS, Spring 2001 © L. Joskowicz 25 Surgical Needs Support for image-guided surgery Passive and active devices for accurate spatial positioning, tracking, and execution Modeling, planning, viewing, diagnosis systems Systems integration: from diagnosis to post-op Improve current practice and enable new procedures Simulation and training systems Augment the surgeon’s capabilities with better quantitative planning, execution, and integration

26. CAS, Spring 2001 © L. Joskowicz 26 Current clinical status Imaging –vast databases of medical images –digitized atlases –mostly uncorrelated unimodal qualitative interpretation Devices –mostly passive and non-invasive (supports) –laparoscopic camera, –some real-time tracking Planning, modeling, visualization –3D reconstruction, some registration

27. Part 2: Computers and Robots Technology and algorithms available today

28. CAS, Spring 2001 © L. Joskowicz 28 How can computers help? (or are already helping…) Image processing –single image: enhancement, noise reduction, segmentation, quantitative measurements –image stacks: 3D reconstruction, segmentation – image sets: registration, comparison, data fusion Planning and simulation –integrate medical images and CAD models –planning and simulation programs Computer vision and graphics –camera modeling, image registration, rendering

29. CAS, Spring 2001 © L. Joskowicz 29 Image processing

30. CAS, Spring 2001 © L. Joskowicz 30 Planning and simulation

31. CAS, Spring 2001 © L. Joskowicz 31 Virtual man project -- digital model

32. CAS, Spring 2001 © L. Joskowicz 32 How can robots and sensors help? (or are already helping…) Robotic devices –passive, semi-active, active devices –instrument and anatomy positioning and holding –cutting and machining Real-time tracking –optical, video, electromagnetic devices –navigation tools

33. CAS, Spring 2001 © L. Joskowicz 33 Robotic devices

34. CAS, Spring 2001 © L. Joskowicz 34 Real-time tracking devices camera instrument Passive markers Instrument has infrared LEDs attached to it Active markers

35. CAS, Spring 2001 © L. Joskowicz 35 Computer-Assisted Surgery (CAS) A computer-integrated system to enhance the dexterity, visual feedback, and information integration of the surgeon Key points: The goal is NOT to replace the surgeon A new paradigm for surgical tools Address a real clinical need Prove efficacy and cost-effectiveness

36. CAS, Spring 2001 © L. Joskowicz 36 Elements of CAS systems

37. CAS, Spring 2001 © L. Joskowicz 37 Elements of CAS systems Preoperative planning –image acquisition, modeling, analysis, simulation –plan elaboration, tool and prosthesis selection –Output: preop images, 3D models, prosthesis type and position, navigation and cutting plan Intraoperative execution –passive, semi-active, active robot –real time tracking –intraoperative imaging (fluoroscopy, ultrasound)

38. CAS, Spring 2001 © L. Joskowicz 38 State of the Art (1) Main clinical procedures –neurosurgery: biopsies, tumor removal –orthopaedics: hip and knee replacement, spine, pelvis and femur fractures –maxillofacial and cranofacial –laparoscopy: laparoscope holders –new fields: dentistry, ophtalmology, prostate Mostly rigid structures: bones!!

39. CAS, Spring 2001 © L. Joskowicz 39 State of the Art (2) Commercial navigation systems –main uses: neurosurgery and spine surgery Commercial robotic systems –ROBODOC for total hip replacement –laparoscope arm holders Research –very active, very interdisciplinary –a few dozen systems tested in-vitro

40. CAS, Spring 2001 © L. Joskowicz 40 State of the Art (3) Major players –INRIA Sophia Antipolis, Grenoble, Johns Hopkins, Brigham Women’s H./MIT, Shadyside H./CMU, Imperial College, many places in Germany and Japan Interdisciplinary conferences and journals –started in 1994: MRCAS’94; Orthopaedic CAS meetings, visualization, etc, –Journals: Computer-Aided Surgery, Medical Image Analysis

41. CAS, Spring 2001 © L. Joskowicz 41 Examples of CAS systems in use Image-guided navigation systems ROBODOC: Total hip replacement surgery LARS: Laparoscopic assistant Radiosurgery Brief overview follows; will be covered in detail later

42. CAS, Spring 2001 © L. Joskowicz 42 Image-guide navigation Purpose –accurate placement of instruments with respect to imaged anatomy for several procedures Problem addressed –provide 3D vision of unseen structures replace static 2D fluoroscopy or larger openings –improve precision of biopsies, screw placements Scope –non-invasive –creates surface model from preop images –registration of images to anatomy by direct contact

43. CAS, Spring 2001 © L. Joskowicz 43 Image-guided navigation

44. CAS, Spring 2001 © L. Joskowicz 44 Image-guided navigation (2) pedicle screw insertion

45. CAS, Spring 2001 © L. Joskowicz 45 Status In clinical use Over 7,000 neurosurgeries performed with commercial systems Gaining popularity in pedicle screw insertion Decreased the misplacement rate from 10-40% to 5-18% (clinical study of 700 cases) More clinical applications under development

46. CAS, Spring 2001 © L. Joskowicz 46 ROBODOC: Total hip replacement Purpose –precise machining of cementless hip implant canal Problem addressed –complications in canal preparation and implant fixation –improve positioning accuracy and surface finish Scope –invasive, numerically controled machining –plan from preop CT, registered via pins –adapted commercial robot –custom bone fixator and bone motion detection

47. CAS, Spring 2001 © L. Joskowicz 47 Artificial hip joint

48. CAS, Spring 2001 © L. Joskowicz 48 Total hip replacement procedure

49. CAS, Spring 2001 © L. Joskowicz 49 ROBODOC: Total Hip Replacement

50. CAS, Spring 2001 © L. Joskowicz 50 ROBODOC system diagram

51. CAS, Spring 2001 © L. Joskowicz 51 ORTHODOC Planning

52. CAS, Spring 2001 © L. Joskowicz 52 ROBODOC robot diagram

53. CAS, Spring 2001 © L. Joskowicz 53 ROBODOC robot

54. CAS, Spring 2001 © L. Joskowicz 54 ROBODOC procedure

55. CAS, Spring 2001 © L. Joskowicz 55 ROBODOC cutting

56. CAS, Spring 2001 © L. Joskowicz 56 ROBODOC History Developed by IBM Research and Integrated Surgical Systems First active surgical robot –1986: feasibility study –1989: in-vitro testing of dog system –1990: 26 dog cases –1992: development of human system –1994: first human procedure in Frankfurt –1995- clinical trials in the US for FDA approval

57. CAS, Spring 2001 © L. Joskowicz 57 ROBODOC current status Sold by Integrated Surgical Systems Over 3,000 cases performed 15 systems installed in Germany, 2 in Austria Excellent short term clinical results (3 year study) –no fractures, few failures (continue manually) Long-term clinical results to be determined –key issue: does the artificial hip last longer? Problems: OR time, pin insertion

58. CAS, Spring 2001 © L. Joskowicz 58 Laparoscopic assistant: LARS Purpose –laparocopic camera holding and precise navigation Problem addressed –cumbersome, unintuitive, and unsteady camera positioning Scope –non-invasive intraoperative device –video images interpreted by surgeon Benefits –direct camera manipulation; stability, precise targeting

59. CAS, Spring 2001 © L. Joskowicz 59 Laparoscopic assistant: LARS

60. CAS, Spring 2001 © L. Joskowicz 60 LARS characteristics Designed at IBM Research, 1993. Similar commercial devices available (AESOP) Custom redundant 7 degree-of-freedom robot Holds laparoscopic camera Fulcrum motions: no motion at point of entry Mouse-like controls on surgical scissors Position memory and replay

61. CAS, Spring 2001 © L. Joskowicz 61 Stereotactic Radiosurgery Purpose –plan and deliver precise radiation doses Problem addressed –precise positioning and dosing of radiation to avoid healthy organ damage Scope –non-invasive intraoperative device –active beam postioning and planning –complex preoperative planning based on MRI images –registers preoperative plan with stereotactic frame

62. CAS, Spring 2001 © L. Joskowicz 62 Stereotactic Radiosurgery

63. CAS, Spring 2001 © L. Joskowicz 63 CYBERKNIFE system

64. CAS, Spring 2001 © L. Joskowicz 64 CYBERKNIFE system

65. CAS, Spring 2001 © L. Joskowicz 65 Stereotactic Radiosurgery: planning

66. CAS, Spring 2001 © L. Joskowicz 66 Stereotactic Radiosurgery Developed at Stanford starting in 1992 Complex 3D radiation plans Currently in clinical use Frameless procedure under development follow head with markers, video, or X-rays Company Accuray has performed several clinical trials with frameless procedure

67. CAS, Spring 2001 © L. Joskowicz 67 Developing CAS systems Similarities –understand and address real needs of surgeons –consider established procedures, context, use –work on problems that will make qualitative difference –constant feedback from user; test ideas and prototypes Differences –system performace requirements

68. CAS, Spring 2001 © L. Joskowicz 68 Developing CAS systems understand and address real needs of surgeons consider established procedures, context, use constant feedback from user; test ideas and prototypes system requirements –safety and reliability –fail-safe systems: can always stop and proceed as usual –system integration

69. CAS, Spring 2001 © L. Joskowicz 69 CAS systems design cycle Prototype development In-vitro experiments –system refinement Cadaver studies –system refinement In-vivo experiments –first animal and human trials Clinical trials –double blind studies, Hospital and FDA protocols Agency approval and commercial release

70. CAS, Spring 2001 © L. Joskowicz 70 Summary Great potential for robots and computers inside and outside the operating room Great research and commercial interest, especially in the past 3 years Just the beginning of the road: many things remain to be invented Great role for applied computer science: –image processing, geometric planning, registration, graphics, vision, real-time systems, robotics, etc.