1. © Fraunhofer MEVIS Toward Automated Validation of Sketch-based 3D Segmentation Editing Tools Frank Heckel 1, Momchil I. Ivanov 2, Jan H. Moltz 1, Horst K. Hahn 1,2 1 Fraunhofer MEVIS, Bremen, Germany, 2 Jacobs University, Bremen, Germany 18th Scandinavian Conference on Image Analysis, Espoo, Finland, June 2013

2. © Fraunhofer MEVIS 1 / 15 Motivation Segmentation is one of the essential tasks in medical image analysis Many sophisticated automatic segmentation algorithms exist … … which might fail in some cases Low contrast, noise, biological variability, … What is segmentation editing and why isn’t it trivial? Solution Results Outlook Conclusion What to do?

3. © Fraunhofer MEVIS 2 / 15 Motivation Intuitive interaction in 2D – Estimate the user’s intention in 3D As few interactions as possible The segmentation problems are typically hard What is segmentation editing and why isn’t it trivial? Solution Results Outlook Conclusion Locally correct the error until it satisfies the specific needs F. Heckel et al., ”3D contour based local manual correction of tumor segmentations in CT scans”, SPIE Medical Imaging: Image Processing, 2009 F. Heckel et al., “Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation”, Eurographics Workshop on Visual Computing for Biology and Medicine, 2012

4. © Fraunhofer MEVIS 3 / 15 Motivation What is segmentation editing and why isn’t it trivial? Solution Results Outlook Conclusion F. Heckel et al., “Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation”, Eurographics Workshop on Visual Computing for Biology and Medicine, 2012

5. © Fraunhofer MEVIS 4 / 15 Motivation The segmentation editing process Solution Results Outlook Conclusion intended result that the user only has in mind visually performed by the user

6. © Fraunhofer MEVIS 5 / 15 Motivation “Static” quality measurements exist Volume overlap / dice coefficient Average / maximum surface distance Interactive segmentation process Measuring the quality of the final result only is not enough Acceptance suffers from bad intermediate results Additional quality factors like number of steps, computation time, … User is mandatory High effort – New evaluations after algorithmic changes Bad reproducibility The difficulty in validation of segmentation editing tools Solution Results Outlook Conclusion

7. © Fraunhofer MEVIS 6 / 15 Solution The automatic validation process Results Outlook Conclusion Motivation once generated by an expert use common quality measurements

8. © Fraunhofer MEVIS 7 / 15 Solution Step 1: Find the most probably corrected error Subtract intermediate from reference segmentation 3D connected components define “errors” Select an error to be corrected Largest volume + compactness Largest Hausdorff distance Editing simulation Results Outlook Conclusion Motivation reference segmentation intermediate segmentation

9. © Fraunhofer MEVIS 8 / 15 Solution Step 2: Select slice and view where the error is most probably corrected Largest area + compactness Largest Hausdorff distance Editing simulation Results Outlook Conclusion Motivation Slice 48Slice 44Slice 52

10. © Fraunhofer MEVIS 9 / 15 Solution Step 3: Generate user-input for sketching Get surface of error Remove voxels that are on the surface of the intermediate segmentation as well Step 4: Apply editing algorithm Editing simulation Results Outlook Conclusion Motivation

11. © Fraunhofer MEVIS 10 / 15 Results Volume-based strategy Outlook Conclusion Solution

12. © Fraunhofer MEVIS 11 / 15 Results Distance-based strategy Outlook Conclusion Solution

13. © Fraunhofer MEVIS 12 / 15 Results Outlook Conclusion Solution Volume-based strategyDistance-based strategy

14. © Fraunhofer MEVIS 13 / 15 Outlook Solve current limitations (e.g., correction of holes) Extend simulation Model inaccuracy in drawing sketches Model more correction strategies “Finish” an error before moving to the next Perform correction in “one of the error’s first slices” Investigate how the quality of editing tools is measured best Apply simulation-based validation to a larger database Conclusion Results

15. © Fraunhofer MEVIS 14 / 15 Conclusion Segmentation editing: Is an indispensable step in the segmentation process Efficient editing in 3D is challenging Validation of 3D editing algorithms: Needs to consider the dynamic nature of such tools User studies are time consuming and lack reproducibility Proposed solution: Simulate the user Allows objective and reproducible validation and comparison Allows better quality assessment Outlook

16. © Fraunhofer MEVIS 15 / 15 frank.heckel@mevis.fraunhofer.de Thank you!

17. © Fraunhofer MEVIS 16 / 15 Appendix Computation time Volume-based strategyDistance-based strategy