Presentation is loading. Please wait.

Presentation is loading. Please wait.

Danielle F. Pace, David G. Gobbi, Chris Wedlake, Jan Gumprecht, Jonathan Boisvert, Junichi Tokuda, Nobuhiko Hata & Terry M. Peters Robarts Research Institute.

Similar presentations


Presentation on theme: "Danielle F. Pace, David G. Gobbi, Chris Wedlake, Jan Gumprecht, Jonathan Boisvert, Junichi Tokuda, Nobuhiko Hata & Terry M. Peters Robarts Research Institute."— Presentation transcript:

1 Danielle F. Pace, David G. Gobbi, Chris Wedlake, Jan Gumprecht, Jonathan Boisvert, Junichi Tokuda, Nobuhiko Hata & Terry M. Peters Robarts Research Institute and The University of Western Ontario, London, ON, Canada; Atamai Inc., Calgary, AB, Canada; Surgical Planning Laboratory, Brigham and Women’s Hospital, Boston, MA, USA; Institute for Information Technology, National Research Council Canada, Ottawa, ON, Canada An open-source real-time ultrasound reconstruction system for 4D imaging of moving organs

2 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Image by Andras Jakab with the permission of Ervin Berenyi, University of Debrecen Medical School and Health Science Center Beating heart surgery Intracardiac targets CAI for moving organs images courtesy G. Guiraudon and Slicer3 Visual Blog

3 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Ex: heart, abdominal 4D imaging required at some point in the surgical workflow Time series of 3D images Preoperative and intraoperative Intraoperative ultrasound: Easily integrated within OR Sufficient soft tissue contrast and spatial resolution High temporal resolution Safe, low-cost and portable CAI for moving organs – intraoperative imaging images courtesy G. Guiraudon and Slicer3 Visual Blog

4 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Intraoperative 4D ultrasound (US) Reconstructed ultrasound Real-time 3D (‘live 3D’) image courtesy Streaming image data Standard equipment Resolution Field of view Few artifacts Real-time imaging Real-time 3D US Reconstructed US Advantages of each approach:

5 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, D US reconstruction: 3D and 4D ultrasound reconstruction Translate tracked probe Rotational (1˚ increments) T 0, T 1 … T n Transforms 2D images 3D volume 4D ECG-gated US reconstruction: time series of 3D volumes

6 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Real-time 4D ultrasound reconstruction Typical clinical practice “image first, reconstruct later” T 0, T 1 … T n Acquire Reconstruct Real-time reconstruction View volume(s) during reconstruction Acquire and reconstruct at the same time (Real-time 3D US reconstruction by D. Gobbi) Real-time 4D ultrasound reconstruction implemented: Reconstructs several US volumes of the beating heart at various phases throughout the cardiac cycle (using ECG-gating) D.F. Pace, A.D. Wiles, J. Moore, C. Wedlake, D.G. Gobbi and T.M. Peters, Validation of four-dimensional ultrasound for targeting in minimally-invasive beating-heart surgery. SPIE Medical Imaging Validated with a dynamic phantom

7 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Real-time 4D ultrasound reconstruction

8 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Dynamic phantom experiments (D.F. Pace et al., SPIE 2009): 3D reconstruction – RMS localization error approx. 1.5 mm 4D reconstruction – RMS localization error approx. 2.5 mm Validation studies Porcine imaging: aortic valve mitral valve LA LV RV

9 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, D Slicer: Open-source, multi-platform application for data visualization, image processing / analysis and image-guided therapy Registration, segmentation, volume rendering, filtering, tool tracking… OpenIGTLink support for image/transform transfer Objective: 4D US reconstruction in 3D Slicer images courtesy Slicer3 Visual Blog Collaboration /w Boston Children’s Hospital Pediatric CardiologyImage from Andy Freudling

10 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Introducing SynchroGrab4D Project objectives: Enable 4D gated ultrasound reconstruction in 3D Slicer Provide interactive visualization during US acquisition Prior work – “SynchroGrab”: Real-time 3D ultrasound reconstruction in 3D Slicer: J. Boisvert, D. Gobbi, S. Vikal, R. Rohling, G. Fichtinger and P. Abolmaesumi, An open-source solution for interactive acquisition, processing and transfer of interventional ultrasound images. Workshop on Systems and Architectures for Computer Assisted Interventions, MICCAI Further improvements: by Jan Gumprecht Implementation: Command-line application interfacing with 3D Slicer via OpenIGTLink

11 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, SynchroGrab4D in action 6X speed, 280x240x300

12 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, “4D Image” module in Slicer (by Junichi Tokuda)

13 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, SynchroGrab4D – US reconstruction classes

14 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Frame grabbing, tracking and ECG gating Frame grabbing Video-for-Windows and Matrox Linux and Sonix RP in progress Select frame for timestamp Timestamp frames and store in buffer Tracking NDI Aurora/Polaris/Certus Ascension Flock of Birds Claron Technology’s Micron Timestamp matrices and store in buffer ECG gating “Heart” - 5V at parallel port ECG via USB - threshold to identify R-wave Prospective / retrospective ECG gating

15 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Gating subsystem – prospective & retrospective vtkSignalBox* sb = vtkSignalBox::New(); sb->SetNumberOfPhases(5); sb->Initialize(); sb->Start(); float rate = sb->GetBPMRate(); double currTime = sb->GetTimestamp(); int phase = sb->GetPhase(); double retTime = sb->CalculateRetrospectiveTimestamp(3); sb->Stop(); true phases predicted phases

16 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, D ultrasound reconstruction current time retrospective time if gating subsystem detects a change from phase i to phase j … and phase j corresponds to an output volume… and heart rate is within range… tracker timestamp = video timestamp – lag Interpolate transform matrix Retrieve frame whose timestamp is closest to that calculated by the gating subsystem T  T’ Splat each input pixel into the output volume (Pixel Nearest Neighbor or Pixel Trilinear Interpolation)

17 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Running SynchroGrab4D Command-line application Calibration file specifies acquisition and visualization parameters spatial calibration temporal calibration US fan parameters origins/spacings/extents gating parameters reconstruction parameters Calibration file specifies acquisition and visualization parameters spatial calibration temporal calibration US fan parameters origins/spacings/extents gating parameters reconstruction parameters

18 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Communication with 3D Slicer - OpenIGTLink During the acquisition, image volumes are sent to any CAI system supporting the OpenIGTLink protocol User specifies rate at which image volumes are sent Or can delay image transfer until after the reconstruction is finished Can also send transforms from tracking system User specifies rate at which image volumes are sent Or can delay image transfer until after the reconstruction is finished Can also send transforms from tracking system

19 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, SynchroGrab4D – open-source 3D/4D ultrasound reconstruction: Real-time visualization facilitates acquisition Ideal for intraoperative imaging of cyclically-moving organs, ex. heart and abdominal organs Limitations of real-time reconstruction: Number / size of output volumes, and time difference between them ECG-gating classes easily extensible, ex. for respiratory gating Example applications: Registration to preoperative images Surgical guidance Discussion / Conclusions

20 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Research Collaborators: John Moore Dr. Andrew Wiles Diego Cantor Cristian Linte Dr. Qi Zhang Clinical Collaborators: Dr. Daniel Bainbridge Dr. Gerard Guiraudon Dr. Doug Jones Acknowledgements

21 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Thank you! Source code maintained at NA-MIC Sandbox: 4DUltrasound-WithGating

22 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

23 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

24 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

25 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

26 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

27 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

28 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24,

29 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Bonus Slides

30 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Motion artifacts in 4D ultrasound imaging Caused by: –Limited video capture rate (30 Hz) - in 15 ms, phantom moves…  0.4 mm at 20 rpm  1.3 mm at 60 rpm –Variation in motor speed in the dynamic phantom

31 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Two dual-core 3.2 GHz Xeon CPUs 2 GB RAM NVIDIA Quadro FX 3500 graphics card Matrox Meteor-2 capture card 32-bit Windows XP Configuration

32 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Beating heart phantom

33 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Validation using a tracked dynamic phantom RMS centroid error: 3D: 1.5 mm 4D: 2.5 mm

34 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Results: Example ultrasound volumes

35 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, The dynamic phantom Three phantoms: –Point-source: 2.45 mm hard plastic polycarbonate sphere –Distance: 4 similar hard plastic polycarbonate spheres –Spherical: mm table tennis ball The dynamic phantom: –A simple robot (LEGO ® Mindstorms NXT™) moves the phantom along a circular path, using an open-source C++ robot control library Pace et al., “An accessible, hands-on tutorial system for image-guided therapy and medical robotics using a robot and open- source software”, Workshop on Open Source and Open Data for MICCAI, MICCAI 2007

36 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, ECG-gating with the dynamic phantom –LEGO light sensor detects light reflecting off phantom beam –Automatically detect the beginning of each new cycle

37 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Experimental setup dynamic phantom tracked 2D TEE probe NXT NDI Aurora reference tracked phantom component

38 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Expected and observed phantom position 1 Characterize phantom & divots with micro-CT 5 Manually identify phantom in the images (spherical phantom = best-fit ellipsoid) Calculate expected position in the images 4 Find divots relative to sensor on phantom 2 Acquire 3D/4D US (track phantom & probe) 3

39 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Experiments –Spatial calibration: Z-bar phantom –Temporal calibration: Moving cross-string method –7% glycerol in water to approximate speed of sound in tissue –25 volumes / phantom –0.35 mm voxels (Subsequently resampled to 0.70 mm voxels) 3D ultrasound (static) –50 volumes / phantom 5 phases / reconstruction 2 reconstructions / speed 5 speeds ( rpm = cm/s) –0.70 mm voxels –1.5% allowed deviation from expected heart rate 4D ultrasound (dynamic) Gobbi et al., “Ultrasound probe tracking for real-time ultrasound/MRI overlay and visualization of brain shift”, MICCAI 1999 Gobbi, “Brain deformation correction using interactive 3D ultrasound imaging”, PhD thesis

40 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Results: Localization

41 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Results: Distance d obs d exp Mean expected distance: 21.0 mm

42 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Results: Volume Mean expected volume: ≈ mm 3 Mean expected radius: mm v exp v obs

43 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Results: Eccentricity Ideal 1 Ideal 0 Westin et al., “Image processing for diffusion tensor magnetic resonance imaging”, MICCAI 1999

44 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, D US versus 3D US versus 4D US RMS localization error: 2.4 mm 2D Ultrasound RMS localization error: mm 3D Ultrasound RMS localization error: mm 4D Ultrasound - Averaging effects - Gain/compression - Micro-CT vs optical digitization - Motion artifacts from gating  Difficulties in manual segmentation Wiles et al., “Object identification accuracy under ultrasound enhanced virtual reality for minimally invasive cardiac surgery”, SPIE 2008

45 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Surgical Access: Universal Cardiac Introducer Introductory chamber Safety attachment cuff Heart port access Heart cavity

46 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Predicting phantom position

47 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Predicted phantom position versus median

48 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Interference between motor and magnetic tracking

49 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Open-source Real-time 3D ultrasound reconstruction: –Boisvert et al., An open source solution for interactive acquisition, processing and transfer of interventional ultrasound images”, Workshop on Systems and Architectures for Computer Assisted Interventions, MICCAI 2008 Lego robotic control library: –Pace et al., An accessible, hands-on tutorial system for image- guided therapy and medical robotics using a robot and open source software”, Workshop on Open Source and Open Data for MICCAI, MICCAI 2007 –www.na-mic.org - IGT Toolkit Tutorialswww.na-mic.org

50 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, LEGO Mindstorms NXT 3 Servo Motors (with rotation sensors) 519 LEGO TECHNIC pieces 4 Sensors NXT “Intelligent Brick” (USB and Bluetooth)

51 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Depth dependency

52 Danielle F. Pace – MICCAI Systems and Architectures for CAI – Sept 24, Speed dependency


Download ppt "Danielle F. Pace, David G. Gobbi, Chris Wedlake, Jan Gumprecht, Jonathan Boisvert, Junichi Tokuda, Nobuhiko Hata & Terry M. Peters Robarts Research Institute."

Similar presentations


Ads by Google