1. User Steered Segmentation: Live Wire and Live Lane Presented by Marina Joseph

2. Image Segmentation Partitioning of an image into several constituent components is called segmentation. Segmentation is an important part of practically any automated image recognition system, because it is at this moment that one extracts the interesting objects, for further processing such as description or recognition Two types Automatic: Avoids user intervention, but their complete success cannot be guaranteed. Interactive: This range from totally manual painting of object regions or drawing of object boundaries to the detection of object region/boundaries with minimal user assistance. More attractive method since they are guaranteed to work.

3. . Examples of image segmentation

5. Goal of Interactive Segmentation 1) To provide effective control to the user on the segmentation process while it is being executed. 2) To minimize the total user’s time required in the process. Segmentation Methods Live Wire The user first selects an initial point on the boundary. For any subsequent point indicated by the cursor, an optimal path from the initial point to the current point is found and displayed in real time. Live Lane The user selects only the initial point. Subsequent points are selected automatically as the cursor is moved within a lane surrounding the boundary

6. Live Wire Method Consider pixel array as constituting a directed graph with the vertices of the pixels representing the vertices of the graph. Oriented pixel edges representing arcs. Assign set of features whose values characterize whether the oriented edge is part of a boundary. Features describe certain properties of the object (“the interior of the boundary), the background (exterior of the boundary) and the boundary itself. Feature values are converted to cost per oriented edge. The minimum cost path between the two vertices are determined to find the optimal path. Consider a 2D Rectangular array C of Pixels also called the Scene Domain. The boundary we seek in C are made of Oriented edges of Pixels.

7. The boundary should satisfy the following properties Closedness Boundary Partitions the space into two disjoint components such that any path starting from a pixel in one component and ending in the other meets the boundary. Orientedness One of the two components can be identified as constituting the interior of the boundary and the other representing the exterior of the boundary. Correctedness Guarantees that the boundary is a single connected curve

8. 1 Feature Selection- Describe certain properties of the object a. Positive side selection It is the Intensity on inside of the boundary b. Negative side selection It is the Intensity on the outside of the boundary c. Orientation Sensitive gradient magnitude At points on the boundary that we are seeking, the gradient of g is directed either from the interior of the boundary to its exterior or vice versa. The orientation of each boundary element can be used to ascertain locally whether or not Steps for successful completion of Description of live wire method

9. Orientation Sensitive gradient magnitude continued >>>>>>>> the gradient of g evaluated at the center of b is consistent with the desired boundary, since we know to which side of b lies the interior of the boundary and to which side lies its exterior. d. Distance from the previous boundary Represents the shortest distance of the center of b from the boundary that is defined on the previous slice and that is projected onto the current slice. This feature is utilized in two ways: –modify the cost assigned to boundary elements –to delimit subsets of the scene domain that are used for optimal boundary search.

10. 2 Cost Assignment The selected feature values are converted to cost per oriented edge: Joint Cost c(b) for each boundary element b of C is a linear combination of the costs assigned to its features: C(b)= Sigma i w i f i (f i (b)) ----------------------- Sigma i w i Where w i is a positive constant indicating the emphasis given to feature f i and cf i is the cost associated with feature f i

11. 3 Training Automatic parameter selection –Here the assumption is made that the user has decided which feature to use and what the accompanying feature transforms are. The aim of training then is to determine the values to be selected for the parameters describing each feature transform.Training consists of the user painting typical segments of the boundary. The user is allowed to select the paint brushes of various sizes. For sharper boundaries, thinner brushes should be used. At the completion of painting, the values of the following parameters are computed for all features: minimum, maximum, mean and standard deviation. Optimum feature selection –If B is the set of boundary elements of a scene C which are indicated (via painting) in training. The optimum feature selection is to determine the combination of features, which results in the set of boundary elements selected to be as close as possible to B.

12. 4 Finding Optimum Path The path between any two vertices vs and ve in a given scene C is optimum in the sense that that the total joint cost of boundary elements is the smallest of the total joint cost of boundary elements in any path from vs to ve. Algorithm to determine optimal path: Algorithm LW Input The joint cost function c; an initial vertex vs; a terminal vertex ve and a threshold Tcc on cummulative cost. Ouput A set of boundary elements forming an optimal path from vs to ve. Auxillary data structures A 2D cummulative cost array cc representing the total cost of the optimal path found so far from vs to each vertex of C; a 2D direction of array “dir” indicating; for each vertex, to which of its immediate neighboring vertices the optimal path goes; a queue Q of vertices; and a list L of vertices which have already been processed.

13. Begin 1.Set cc(v) to infinity for all vertices v in C and set cc(vs) to 0 2.Put vs in Q. 3.While q is not empty do a) remove a vertex v from Q such that cc(v)=minv’EQ{cc(v’)}, and put v in L b)for each vertex v’ such that v’ is the set of the 4-adjacent neighbours of v and v’ not element of L do I) compute cctmp=cc(v)+c(b’) where b’ is the bel whose direction goes from v’ to v and c(b’) is the joint cost of b’; II) if cctmp<cc(v’) and cctmp<Tcc then a) set cc(v’) to cctmp and dir(v’) to the direction from v’ to v; b) if v’ not element of Q then insert v’ in Q. end if end for end while 4) Starting from ve, trace recursively the next vertex pointed to by the current vertex using the direction information in dir until vs is reached and output the bels so traced. End

14. Live Wire Demo http://www.cs.rochester.edu/~brown/PLivewire/home.html http://www.cs.rochester.edu/~brown/PLivewire/home.html LIVE LANE The methods of feature selection, cost assignment, training and optimal path finding for the live lane process is same as LIVE WIRE To begin the live lane process, the user selects an initial vertex vs on the boundary of interest and subsequently steers the cursor roughly in the vicinity of the boundary. Live lane is thus a more dynamic process than live wire. It differs from live wire in a)the mechanism by which the end points of the live-wire segments are specified, b)the directed graph used for finding optimal paths. c)the underlying process of human-machine interaction

15. Two strategies for Live Lane method 1)Tracking with Lane of Fixed Width As the user moves the cursor within S(t0) (a square array) at time t0, each position of the cursor v(ti) defines a vertex in S(t0) for which an optimal path from vs is computed and displayed in real time using Algorithm LW. We assume in this method of tracking that the size of S(ti) is fixed for all ti. 2)Tracking with Adaptive Lane For any ti E T, we consider v(ti) to be a 2-component vector indicating the two co-ordinates of the vertex. The speed s(ti) and acceleration a(ti) are defined as S(t0)=a(t0)=0, S(ti)=||v(ti)-v(ti-1)||, I>0, ____________ ti-ti-1 a(ti)=s(ti)-s(ti-1), I>0, _________ ti-ti-1 where ||.|| denotes vector magnitude.

16. Let lw(ti) denote the number of columns in array S(ti) which we call lane width at time ti. Adaptive live lane differs from fixed live lane in two aspects: 1) The lane width lw(ti) may vary with ti, and 2) end vertices ve are selected for optimal boundary segment generation whenever the lane width changes Evaluation In evaluating user –steered segmentation methods, three factors seem to be of prime importance: speed, repeatability (precision) and accuracy. Speed expresses the extent of user involvement required by the process. Repeatability expresses the variations in the segmentation outcome as a result of the subjective actions taken by the user in the steering process. Accuracy expresses the degree of agreement of the segmentation result with truth. These factors may have to be given different emphasis in assessing the utility of a method in a given application as a function of these factors.

17. Discussion and Conclusions Live Wire is used in applications where the proper parameter settings permit to segment boundaries in slices selecting a few points. In situations where a larger number of points is necessary to segment such boundaries, compromising the good performance of live wire, we use live lane. As the users become more familiar with the behavior of the algorithms, they are able to finish segmentation quicker. The imaging modalities/objects for which the live methods may be most usefully applied may be characterized by any combination of the following: ultrasound, PET, SPECT, and MR images