Presentation is loading. Please wait.

Presentation is loading. Please wait.

S I E M E N S C O R P O R A T E R E S E A R C H 1 1 General Purpose Image Segmentation with Random Walks Leo Grady Department of Imaging and Visualization.

Similar presentations


Presentation on theme: "S I E M E N S C O R P O R A T E R E S E A R C H 1 1 General Purpose Image Segmentation with Random Walks Leo Grady Department of Imaging and Visualization."— Presentation transcript:

1 S I E M E N S C O R P O R A T E R E S E A R C H 1 1 General Purpose Image Segmentation with Random Walks Leo Grady Department of Imaging and Visualization Siemens Corporate Research

2 S I E M E N S C O R P O R A T E R E S E A R C H 2 2 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

3 S I E M E N S C O R P O R A T E R E S E A R C H 3 3 About 200 full time research staff 75+ people working on medical imaging Basic research  clinical products 1/3 mid/long term research - 2/3 applied projects SCR Siemens Medical Solutions Clinical & University Partners Princeton, USA Overview of SCR

4 S I E M E N S C O R P O R A T E R E S E A R C H 4 4 Overview of SCR Clinical Imaging Goals of clinical application software Measures something that could not be measured practically before Makes diagnosis more accurate or treatment more effective Enables therapy that was not possible before Increases patient control Saves time Reduces cost

5 S I E M E N S C O R P O R A T E R E S E A R C H 5 5 Overview of SCR Core interests – Segmentation, registration, visualization

6 S I E M E N S C O R P O R A T E R E S E A R C H 6 6 Offline & Online: Intervention So far: diagnostic radiology: offline problem Interventional imaging: online problem Continuous imaging, constant human input Rich source of new problems Overview of SCR

7 S I E M E N S C O R P O R A T E R E S E A R C H 7 7 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

8 S I E M E N S C O R P O R A T E R E S E A R C H 8 8 General Purpose Segmentation Goal: Input an image and output the desired segmentation Why? Image editing, etc. (e.g., Magic Wand) Do not want to reinvent segmentation for each new product Problem: Two users might want different objects from same image

9 S I E M E N S C O R P O R A T E R E S E A R C H 9 9 General Purpose Segmentation Requires user interaction NCuts, watershed, mean shift Snakes, level sets, intelligent scissors Graph cuts, magic wand region growing

10 S I E M E N S C O R P O R A T E R E S E A R C H 10 General Purpose Segmentation Atomic methods: Semi-automatic Atoms must be subsets of true segmentation Boundary methods: Easier to incorporate shape prior Can be used to improve segmentation of another algorithm Iterative – local minima Difficult to initialize automatically and in higher dimension Harder to generalize to point sets, surfaces, nonuniform sampling, etc. Seeding methods: Leads naturally to steady-state and graph-based algorithms Easy to seed (even automatically) in arbitrary dimensions Generalizes easily to other data modalities

11 S I E M E N S C O R P O R A T E R E S E A R C H 11 General Purpose Segmentation Popular seeding algorithms Region growing: Grow segment from initial seed until distance/contrast/etc. requirement is met Simple Fast Leaks through weak boundaries Killed by noise

12 S I E M E N S C O R P O R A T E R E S E A R C H 12 General Purpose Segmentation Popular seeding algorithms Graph cuts: Max-flow/min-cut found between seeds Fast Probabilistic interpretation Requires lots of seeds to avoid “small cut” problem Metrication artifacts True minimum only for two objects (i.e., foreground/background)

13 S I E M E N S C O R P O R A T E R E S E A R C H 13 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

14 S I E M E N S C O R P O R A T E R E S E A R C H 14 Given labeled voxels, for each voxel ask: What is the probability that a random walker starting from this voxel first reaches each set of labels? Random Walker - Concept Do not despair – Can be computed analytically!

15 S I E M E N S C O R P O R A T E R E S E A R C H 15 GreenRedYellowBlue Partially labeled imageSegmented image Probabilities Random Walker - Concept

16 S I E M E N S C O R P O R A T E R E S E A R C H 16 Random Walker - Concept

17 S I E M E N S C O R P O R A T E R E S E A R C H 17 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

18 S I E M E N S C O R P O R A T E R E S E A R C H 18 Random Walker - Properties Naturally respects weak object boundaries Solid borderWeak border

19 S I E M E N S C O R P O R A T E R E S E A R C H 19 Naturally respects weak object boundaries Random Walker - Properties

20 S I E M E N S C O R P O R A T E R E S E A R C H 20 Provably robust to identically distributed noise No texture or filtering used – Based purely on intensity weighting Random Walker - Properties

21 S I E M E N S C O R P O R A T E R E S E A R C H 21 1.Segmented regions are connected to a seed 2.The probabilities for a blank image (e.g., all black) yield a Voronoi-like segmentation 3.The expected segmentation for an image of pure noise (identical r.v.s) is equal to the Voronoi-like segmentation obtained from a blank image Random Walker - Properties

22 S I E M E N S C O R P O R A T E R E S E A R C H 22 Random walker Graph cuts Random Walker - Properties

23 S I E M E N S C O R P O R A T E R E S E A R C H 23 Random Walker - Properties

24 S I E M E N S C O R P O R A T E R E S E A R C H 24 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

25 S I E M E N S C O R P O R A T E R E S E A R C H 25 How to compute? Solution to random walk problem equivalent to minimization of the Dirichlet integral with appropriate boundary conditions. The solution is given by a harmonic function, i.e., a function satisfying D [ u ] = 1 2 Z ­ ( g r u ( x ; y )) 2 r ¢ g r u = 0 Random Walker - Theory

26 S I E M E N S C O R P O R A T E R E S E A R C H 26 Discrete or continuous space? Discrete Finite Exact Generalizable Continuous Euclidean Approximate Convergence, etc. 4-connected8-connected Random Walker - Theory

27 S I E M E N S C O R P O R A T E R E S E A R C H 27 Mean value theorem: Maximum/Minimum principle: Attractive numerical properties of a harmonic function x i = P w ij ( x i ¡ x j ) P w ij m i n ( x B oun d ary ) · x I n t er i or · max ( x B oun d ary ) Random Walker - Theory

28 S I E M E N S C O R P O R A T E R E S E A R C H 28 Need to represent Laplacian on a graph: In the notation of algebraic topology, the Laplacian is given by 0-coboundary operator (since we operate on nodes) is the incidence matrix: With the constituitive matrix C e ij e ij =w ij playing the role of the metric tensor, the combinatorial Laplace-Beltrami operator is given as Random Walker - Theory r 2 u A e ij v k = 8 > < > : + 1 i f i = k ; ¡ 1 i f j = k ; 0 o t h erw i se L = A T CA

29 S I E M E N S C O R P O R A T E R E S E A R C H 29 D [ u ] = 1 2 Z ­ ( g r u ( x ; y )) 2 Subject to boundary conditions at seed locations x F = 1 ; x B = 0 Energy functional: r ¢ g r u = 0 L x = 0 Euler-Lagrange: D [ x ] = 1 2 ¡ x T A T ¢ C ( A x ) = 1 2 x T L x Random Walker - Theory

30 S I E M E N S C O R P O R A T E R E S E A R C H 30 Random Walker - Theory Laplacian matrix defined by graph as: Decompose Laplacian matrix into labeled (marked) and unlabeled blocks and define an indicator vector for the marked nodes: Must solve a sparse, SPD, system of linear equations for probabilities Since probabilities must sum to unity, for K labels, only K-1 systems must be solved L v i v j = 8 > < > : d v i i f i = j ; ¡ w ij i f v i an d v j area d j acen t no d es ; 0 o t h erw i se L = · L M B B T L U ¸ m s j = ( 1 i f Q ( v j ) = s ; 0 i f Q ( v j ) 6 = s : L U x s = ¡ B m s x K = 1 ¡ X i < K x i

31 S I E M E N S C O R P O R A T E R E S E A R C H 31 Input imageOverlaid graph (lattice)Edge strength (line width) encodes image gradient Random walk formulated on a lattice (graph) that represents the image Random Walker - Concept w ij = e ¡ ¯ ( I i ¡ I j ) 2

32 S I E M E N S C O R P O R A T E R E S E A R C H 32 Therefore, we can formulate a combinatorial Dirichlet integral: Represents minimum power distribution of an electrical circuit We can analytically solve the equivalent circuit problem for the random walker probabilities Random Walker - Theory D [ x ] = 1 2 ¡ x T A T ¢ C ( A x ) = 1 2 x T L x A T z = f ( K i rc hh o ®' s C urren t L aw ) ; C p = z ( O h m ' s L aw ) ; p = A x ( K i rc hh o ®' s V o l t age L aw ) ;

33 S I E M E N S C O R P O R A T E R E S E A R C H 33 Situation exactly analogous to DC circuit steady-state Labels – Unit voltage sources or grounds Weights – Branch conductances Probabilities – Steady-state potentials Label 1 prob. Initial labeling Label 2 prob.Label 3 prob. Random Walker - Theory

34 S I E M E N S C O R P O R A T E R E S E A R C H 34 Algorithm summary: 1.Generate weights based on image intensities 2.Build Laplacian matrix 3.Solve system of equations for each label 4.Assign pixel (voxel) to label for which it has the highest probability Random Walker - Theory

35 S I E M E N S C O R P O R A T E R E S E A R C H 35 Equally valid interpretations of algorithm: 1.What is the steady-state temperature distribution in the inhomogeneous domain, given fixed temperatures at the seeds? 2.What is the probability that a random walker leaving this node first reaches a label of each color? 3.What is the electrical potential at this node when the labeled nodes are fixed to unity voltage (w.r.t. ground)? 4.What is the (normalized) effective resistance between this node and the labeled nodes? Random Walker - Theory

36 S I E M E N S C O R P O R A T E R E S E A R C H 36 Equally valid interpretations of algorithm: 5. If a 2-tree (tree with a missing edge) is drawn randomly, what is the probability that this node is connected to each label? Interpretation used to prove noise robustness Random Walker - Theory

37 S I E M E N S C O R P O R A T E R E S E A R C H 37 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

38 S I E M E N S C O R P O R A T E R E S E A R C H 38 Main computational burden is solving the system of linear equations Fortunately, system is sparse, symmetric, positive definite For a lattice (or any regular graph), the sparsity structure of the matrix is circulant Random Walker - Numerics L U x s = ¡ B m s

39 S I E M E N S C O R P O R A T E R E S E A R C H 39 Structure of the Laplacian matrix allows for efficient storage and operations – Off diagonals may be packed into RGBA Progressive visualization of solution possible Z-buffer allows masking out of seeds Advantages of a GPU implementation Random Walker - Numerics

40 S I E M E N S C O R P O R A T E R E S E A R C H 40 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

41 S I E M E N S C O R P O R A T E R E S E A R C H 41 Random Walker - Results

42 S I E M E N S C O R P O R A T E R E S E A R C H 42 Random Walker - Results

43 S I E M E N S C O R P O R A T E R E S E A R C H 43 Random Walker - Results

44 S I E M E N S C O R P O R A T E R E S E A R C H 44 Random Walker - Results

45 S I E M E N S C O R P O R A T E R E S E A R C H 45 Cardiac segmentation across modalities Random Walker - Results

46 S I E M E N S C O R P O R A T E R E S E A R C H 46 Segmentation of objects with varying size, shape and texture Random Walker - Results

47 S I E M E N S C O R P O R A T E R E S E A R C H 47 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

48 S I E M E N S C O R P O R A T E R E S E A R C H 48 Original imagePriors only Random walker only Combined Possible to incorporate other terms – Intensity priors Useful for multiple, disconnected objects Random Walker - New

49 S I E M E N S C O R P O R A T E R E S E A R C H 49 Random Walker - New w ij = e ¡ ¯ ( I i ¡ I j ) 2 Gaussian weighting Systematic study of weighting function Reciprocal weighting w ij = ¯ ( I i ¡ I j ) 2 Run on 62 CT datasets with seeds and manual segmentations

50 S I E M E N S C O R P O R A T E R E S E A R C H 50 Random Walker - New Systematic study of edge topology 6-connected 10-connected 26-connected

51 S I E M E N S C O R P O R A T E R E S E A R C H 51 Rand om Walke r - New Formulate as special case of general segmentation approach - Compare with other instances of algorithm

52 S I E M E N S C O R P O R A T E R E S E A R C H 52 Random Walker - New 5 eigs20 eigs40 eigs 60 eigs 80 eigs 100 eigs Exact Precomputation Precompute eigenvectors of Laplacian Input seeds Instant result (approximation)

53 S I E M E N S C O R P O R A T E R E S E A R C H 53 Outline Overview of Siemens Corporate Research (SCR) General purpose segmentation Random walker algorithm – Concept – Properties – Theory – Numerics – Results – New Conclusion

54 S I E M E N S C O R P O R A T E R E S E A R C H 54 Conclusion 1.General-purpose 2.Robust to noise and weak boundaries 3.Has a single parameter (not adjusted for these results) 4.Stable 5.Accurate 6.Available Random walker algorithm is:

55 S I E M E N S C O R P O R A T E R E S E A R C H 55 Conclusion – More Information My webpage: Random walkers paper: Random walker demo page: CVPR Short Course: Fundamentals linking discrete and continuous approaches to computer vision - A topological view MATLAB toolbox for graph theoretic image processing at: Writings and code Random walkers MATLAB code:


Download ppt "S I E M E N S C O R P O R A T E R E S E A R C H 1 1 General Purpose Image Segmentation with Random Walks Leo Grady Department of Imaging and Visualization."

Similar presentations


Ads by Google