IllustraSound Illustrative Visualization meets Ultrasound Veronika Solteszova Åsmund Birkeland Paolo Angelelli

Bergen, Norway Department of Informatics Haukeland University Hospital CMR Computing medviz.uib.no 2/21

IllustraSound origins: “Illustrated Ultrasound for Multimodal Interpretation” Ultrasonography is difficult to interpret Spatial undestanding of arrangement within the liver 3/21

4 Goal Aim to anwer US examiner’s question: Conceptual solution: Transfer of semantics from 3D anatomic modality to ultasound image Using already available 3D modality low contrast CT MRI Minimal overhead to existing procedures express segmentation and registration justifying precision with time and simplicity Targeting at examination training phase ”Where the hell am I within the organ?!?” 4/21

Illustrated Ultrasound Pipeline 5/21

Simple 13-Points Couinaud Segmentation Segment 1 defined as sphere (on slice 1 point)) Segments 2-8 defined through planes (on slice 4x3 points – plane definitions) 6/21

Landmark Rigid Registration + Probe tracking Three landmarks in US and three in CT Tracking information enables to co-register the rest 7/21

Results: Couinaud Segments in Ultrasound 8/21

Paolo Angelelli – Personal Information March 2008: Laurea Specialistica (MSc) in Computer Science at University of Bologna Thesis: Enterprise computing and integration: a case-study Developed for Wincor-Nixdorf Italy March 2005 Laurea Triennale (BS) in Computer Science at University of Bologna. Thesis: Design, Implementation and Analisys of TimeWarp synchronization algorithm for parallel and distributed Simulation (as a part of the ARTiS project) 9/21

Paolo Angelelli - Research Started a PhD program in medical visualization in the VisGroup bergen in Aug ( Within the scope of the MedViz cooperation ( Research focus Medical visualization Special focus on visualization of human physiology But also: MultiModal Of time-dependent data Multi variate Ultrasound data 10/21

Guided Visualization of 2D Ultrasound data Problem :”Which slice shows this specific structure?” Solution: Enrich the raw US video streams with semantic To aid the physicians to: Locate the structures of interest in each slice Locate the slices containing structures of interest within the video sequences Easily outline the findings when exchanging the data with others ”On-demand” access to the data How? Idea: Define ”somehow” the region of interest in the patient’s body Annotate each image in the US video with the amount of the region of interest present in the image 11/21

Prototype System 12/21

Åsmund Birkeland Bachelor and Master degree in Informatics at the University of Bergen Focus in illustrative volumetric visualization Worked as engineer for the VisGroup at UiB Magnetic tracking, stereo projection and Ultrasound [Forskningsdagene 09] Currently a Phd. student at UiB under the IllustraSound project 13/21

Recent Work View-Dependent Peel-Aways for Volumetric Data [Master thesis] Occlusion in 3D rendering of volumes Context is important Automatic generation of peel which ensures direct view to the feature of interest Published at the Spring Conference on Computer Graphics /21

Forskningsdagene 09 Goal: To get children interested in our research field Problem: Ultrasound is difficult to read for the laymen Solution: Show the context of the ultrasound Magnetic tracking WebCam with simplification filters Result: Children having fun! 15/21

Future Ideas Sketch-based 3D modelling using 2D ultrasound Tracked ultrasound provide spatial information Simple sketching for fast feature extraction 16/21

VS - Background Bachelor and master degree Computer Graphics and Digital Image Processing Vienna University of Technology, Austria Knowledge Broad overview and basics in various disciplines in computer science Mathematics Image processing, pattern recognition Graphics programming and visualization 17/21

VS - Background Master thesis in visualization ”Visual Queries in Neuronal Data Exploration” Published at IEEE Visualization 2009, Atlantic City, NJ Collaboration with Institute of Molecular Pathology Confocal microscopy imaging Spatial resolution 1μm Neurobiology data of Drosophila (fruit fly) Brains, neurons,... 18/21

Future Ideas in IllustraSound Project Rendering of 4D ultrasound data 3D volumes in real-time Ultrasound image characteristics Lower resolution than CT, MR Speckle, Reflection Traditional volume rendering approaches fail Incorporating the temporal aspect Using different approaches for surface shading 19/21

Future Ideas in IllustraSound Project Direct volume rendering Rendering incorporating temporal coherence 20/21

Thank you for your attention Questions? 21/21