1. Visual Annotation of Clinically Important Anatomical Landmarks for VitreoRetinal Surgery Project Update Vincent Ng

2. Retina Project Overview Registration – Preoperative images to intraoperative images – Overlay of landmarks on live microscopic feed Requirements – Image Matching/Tracking Images courtesy of Rogerio Richa

3. 3 Solution: Annotations overlay on video Surgeon annotates preoperative image Problem: Mentally track notes & locations during surgery, adding significant load to the already challenging surgical task. Slide taken from Rogerio Richa’s presentation Relevance

4. Process RANSAC/Homography – 25 – 200ms SURF Feature Extraction – 50ms Match Features using NCC – 50-500ms Overlay Result

5. Milestones & Progress Working SURF detection/matching program, preop/intraop processing Validation using test/non-surgical data Integrating program onto platform Validation using real intra-operative dataWeek of April 25th Real-time tracking through speedupWeek of April 25th Week of March 28th Week of March 14th

6. Dependencies Access to platform (machine, microscope) Fundus image for phantom Intraoperative data from surgeons

7. Results/Video

8. Next steps / Plan Updates Speedup – Image size reduction Phantom/Real data testing Algorithm Updates – NCC Takes too long – Homography (planar projection) Too many degrees of freedom – Determine optimal settings SURF Hessian threshold NCC threshold Number of SURF feature points per landmark

9. Updated Timeline Below Task21-Feb28-Feb7-Mar14-Mar21-Mar28-Mar4-Apr11-Apr18-Apr25-Apr2-May Presentation and Proposal Feature detection and Matching Validation Intra-Intra image processing Preop Image Processing Annotation registration GUI Hardware Acceleration Final Poster Documentation Task21-Feb28-Feb7-Mar14-Mar21-Mar28-Mar4-Apr11-Apr18-Apr25-Apr2-May Presentation and Proposal Feature detection and Matching Validation Intra-Intra image processing Preop Image Processing Annotation registration / GUI Updates to Algorithm Hardware Acceleration Final Poster Documentation

10. Difficulties so far/future Validation – How? – Sensitivity of tracking to change? Speed – 200ms – slow ? Real data

11. What’s Next Working SURF detection/matching program, preop/intraop processing Validation using test/non-surgical data Integrating program onto platform Validation using real intra-operative dataWeek of April 25th Real-time tracking through speedupWeek of April 25th Week of March 28th Week of March 14th

12. What’s Next: Deliverables Minimum – Utilize, understand and implement SURF – Test/Validate program using manually picked annotations as ground truth – Data: Surgical and non-surgical Expected – Simple GUI that marks annotation for surgeon – Initial Image processing for real surgery data Maximum – Deploy solution into OR. – Integrate fundus/OCT data to surgeon’s screen

13. Documentation Updates to the wiki Documented Code – Comments – Well structured, portable (CISST Filters) Image taken from CISST website https://trac.lcsr.jhu.edu/cisst/wiki/cisstSte reoVisionTutorial

14. Questions? Thank You