Computer Assisted Surgery Computer Vision Group Artificial Intelligence Laboratory Massachusetts Institute of Technology Surgical Planning Lab Brigham and Women’s Hospital Harvard Medical School

Computer assisted surgery Goal: Use computer vision techniques to plan and guide surgical interventions, by giving the surgeon the ability to sense critical information that is not normally visible to her How? Capture models of patient anatomy and function Use to guide surgeon’s actions

Conventional Surgery: See the surface Provided by Nakajima, Atsumi et al.

Image Guided Surgery: See under the surface Provided by Leventon et al.

Intelligently aiding the surgeon Convert medical images into models of patient’s: Structural anatomy Functional anatomy Vascular structure

Example patient specific models Green is tumor

Central Vessels Courtesy of Lorigo common carotid artery subclavian artery brachio cephalic trunc brachio cephalic trunc pulmonary trunk pulmonary veins aorta Courtesy of Lorigo

Functional information Movie: fmri.avi. Lots of activity in the rest state, need statistical methods to identify the brain activation that corresponds to the action in the presence of the background activation. In this example, the subject was asked to squeeze a all. The red area in the movie corresponds to the motor cortex.

Visualizing the surgical site Augmented reality visualizations combine with real imagery Surgical guidance planning and navigation Simulation of surgical navigation Use image information for diagnostics

Augmented Reality

Surgical guidance Movie: locator.avi different image modalities (MRI, fMRI, angio) are used to provide as much information as possible to the surgeons. Tools are tracked during surgery. New images are acquired as the procedure progresses.

Simulation: virtual endoscopy

Simulation: virtual endoscopy Movie: colonFly.mov: virtual fly-through constructed from the 3D CT image of a colon. Can be interactive or passive. Color indicates suspicious areas of high curvature (polyps).