1. 1 GRAPHS - ADVANCED APPLICATIONS Minimim Spanning Trees Shortest Path Transitive Closure

2. 2 Graph applications Graphs can be used for: Finding a route to drive from one city to another Finding connecting flights from one city to another Determining least-cost highway connections Designing optimal connections on a computer chip Implementing compilers Doing garbage collection Representing family histories Doing similarity testing (e.g. for a dating service) Playing games

3. 3 Elementary Graph Operations Graph traversals provide the basis for many elementary graph operations : Spanning trees on graphs Graph cycles Connected components of a graph

4. 4 Spanning Tree  Connected sub graph that includes all vertices of the original connected graph  The subgraph is a tree  If original graph has n vertices, the spanning tree has  n vertices and n-1 edges.  No cycle allowed in this sub graph

5. 5 Minimum-cost spanning tree A minimum-cost spanning tree of a connected weighted graph is : a collection of edges connecting all vertices such that The sum of the weights of the edges is the smallest possible.

6. 6 Spanning trees  Suppose you have a connected undirected graph  Connected  Connected: every node is reachable from every other node  Undirected  Undirected: edges do not have an associated direction ...  then a spanning tree is a connected sub graph in which there are no cycles 6 A connected, undirected graph Four of the spanning trees of the graph

7. 7 Finding a spanning tree – We did this in class  To find a spanning tree of a graph, 1. pick an initial node and call it part of the spanning tree 2. do a search from the initial node: Each time you find a node that is not in the spanning tree, add to the spanning tree both the new node and the edge you followed to get to it 7 An undirected graph Result of a BFS starting from top Result of a DFS starting from top

8. 8 Spanning Tree - Minimum Edges  Minimum number of edges to keep the graph connected N-1  If have N vertices, spanning tree has N-1 edges

9. 9 Minimizing costs  Suppose you want to supply a set of houses (say, in a new subdivision) with:  electric power  Water  sewage lines  telephone lines 9

10. 10 Minimum cost spanning tree  To keep costs down:  Connect these houses with a spanning tree (of, for example, power lines) all equal distances apart  However, the houses are not all equal distances apart  To reduce costs even further, you could connect the  houses with a minimum-cost spanning tree

11. 11 Minimum Spanning Tree (MST) 6 7 1 5 10 20 6 10 1 5 Spanning tree with minimum weight

12. 12 Algorithm For Finding MST 1. All nodes are unselected, 2. Mark node v selected to get started – can be any node 3. For each node of graph, { minimum weight Find the edge with minimum weight that connects an unselected node with a selected node Mark this unselected node as selected }

13. 13 Demos For Finding MST A  Step 1: mark vertex A as selected A B C D 2 1 10 5 6

14. 14 Demos For Finding MST  Step 2: find the minimum weighted edge connected to vertex A, and mark the other vertex on this edge as selected.  We choose {A to D} A B C D 2 1 10 5 6

15. 15 Demos For Finding MST  Step 3: find the minimum weighted edge connected to vertices set { A, D },  and mark the other vertex on this edge as selected.  We choose { A, B } A B C D 2 1 10 5 6

16. 16 Demos For Finding MST  Step 4:  find the minimum weighted edge connected to vertices set  { A, D, B}, and mark the other vertex on this edge as selected.  We choose {B,C} { A, D, B, C}  vertices set { A, D, B, C}  No more nodes! A B C D 2 1 10 5 6

17. 17 Demos For Finding MST  Step 5: All vertices are marked as selected, So we find the minimum spanning tree A B C D 2 1 10 5 6

18. 18 Complexity For Finding MST  N nodes, E edges N * E = O(N*E)  Analysis: N * E = O(N*E)  Number of nodes * the number of Edges  Better implementation:  Heap O(logE) where is E is number of edges(weights) O(E log E) Initialization of heap: O(E log E) Number of edges * log of E

19. 19 Cost - Definition  Routing Protocol  Cost = congestion, bandwidth, delay … cost Umass Router cost

20. 20 Minimum-Cost Spanning Trees collection of edges connecting all vertices  A minimum-cost spanning tree of a connected weighted graph is a collection of edges connecting all vertices such that the sum of the weights of the edges is the smallest possible. Prim’s algorithmalways pick the edge with the smallest weight to any node. It is a greedy algorithm. Prim’s algorithm: always pick the edge with the smallest weight to any node. It is a greedy algorithm. h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9

21. 21 Kruskal’s algorithm T = empty spanning tree; E = set of edges; N = number of nodes in graph; while T has fewer than N - 1 edges { remove an edge (v, w) of lowest cost from E if adding (v, w) to T would create a cycle then discard (v, w) else add (v, w) to T } 21

22. 22 MST Algorithms  Finding an edge of lowest cost can be done just by sorting the edges –  so could use a heap  Efficient testing for a cycle requires a fairly complex algorithm (UNION-FIND)

23. 23 MCST --- Another Example h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9

24. 24 MCST --- Example (cont’d) h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 h 2 6 4 2 8 3 7 5 4 a f e b c d g 9 All nodes visited!

25. 25 Shortest Path Problem  Weight: cost, distance, travel time, hop … 1 3 4 10 5 7 2 5 6 u v

26. 26 Prim’s Algorithm: Heap Implementation Consider using a heap to hold the keys D[0…n-1]. We use a ( Min Heap): To find the minimum in heap array using tree formulas,  To find the minimum in heap array using tree formulas, we call Extract_Min(heap) we call Extract_Min(heap)

27. 27 ExtractMin  Extract_Min removes the minimum element from the heap  which is in the root  by swapping in the last element. O(1)  Then sift down last element(LgV)  We get O(log V) using the usual removal heap operation

28. 28 PRIMS ALGORITHM- Heap implementation  Extract_Min removes the minimum element from the heap which is in the root. Heap Implementation: O(E log V) or Number of Edges times the log of # of Vertices

29. 29 Single-Source Shortest Paths – Prim’s  To find the shortest path from one given node to all the others  Given a source x.  Find the path from x to v  such that v is in V-x and the path is minimum.  If the weights in the edges are all one we already have a solution: breadth first search

30. 30 Single Source Shortest Paths Given a weighted connected graph G=(V,E), and given a pair of vertices v s (source) and v d (destination) Î V what is the shortest path from v s to v d ? That is, what is the path that has the smallest sum of edge weights? 1 2 4 6 3 5 10 7 8 1 5 6 7 2 3 5 1 2 4 6 3 5 7 8 1 5 6 7 2 3 5 Source vertex v s is vertex 1 Bold line is shortest path from 1(v s ) to 2 (v d )

31. 31 Single Source Shortest Paths SP from A to H = SP from A to E + SP from E to H. SP from A to H = SP from A to B + SP from B to H. SP from A to H = SP from A to C + SP from C to H. In general: SP from A to H = SP from A to v i + SP from v i to H;  v i. A B E F H15 A B E G H14 A C E F H16 A C E G H15 A D E F H26 A D E G H25 A B D CE F G H 2 1 6 3 8 1 7 56 5

32. 32 Single Source Shortest Paths Use Dijkstra’s (greedy) algorithm if graph has only positive weights Use Bellman-Ford’s algorithm if graph has positive and negative weights (but not negative cycles…) Both algorithms can run on directed or undirected graphs

33. 33 Single-Source Shortest Paths  The solution when we have different weights is a variation of Prim's algorithm  Before we chose the node that was closest to the partial minimum tree.  Now we want the node which is closest to the source node

34. 34 Single-Source Shortest Paths Algorithm: 1. Start from the source node 2. make this node part of the a shortest path tree A=(V A,E A ) 3. Find an edge (x,y) such that x in the V A and y is not, so that the path from the initial node to y is minimum 4. add y and the edge (x,y) to the tree A. Algorithm: 1. Start from the source node 2. make this node part of the a shortest path tree A=(V A,E A ) 3. Find an edge (x,y) such that x in the V A and y is not, so that the path from the initial node to y is minimum 4. add y and the edge (x,y) to the tree A.

35. 35 Single-Source Shortest Paths  The "question" here is: how do we find the next edge to be added in the tree?  We again can make use of existing data structures  As the case with Prim's algorithms for the MST we can use a priority queue  The algorithms described in the following slides is normally attributed to E. Dijkstra  Let's understand the algorithm graphically first.

36. 36 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 24 2 5 1 6 1 4 5 3 2 1 2 1 3 1 1 2

37. 37 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 24 2 5 1 2 6 1 4 5 3 2 1 2 1 3 1 1 0A0A 0A0A

38. 38 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 24 2 5 1 6 1 4 5 3 2 1 2 1 3 1 1 2 1B1B 1B1B 2F2F 2F2F 6G6G 6G6G

39. 39 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 24 2 5 1 2 6 1 4 5 3 2 1 2 1 3 1 1 C C F F D D E E G G

40. 40 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 2 5 1 2 6 1 4 5 3 2 1 2 1 3 1 1 2 2F2F 2F2F 3D3D 3D3D 6E6E 6E6E 6G6G 6G6G

41. 41 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 2 1 2 6 1 4 5 3 2 1 2 1 3 1 1 3D3D 3D3D 4L4L 4L4L 4E4E 4E4E 6G6G 6G6G

42. 42 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 6 1 4 5 3 2 1 2 1 3 1 1 4L4L 4L4L 4E4E 4E4E 6G6G 6G6G

43. 43 4E4E 4E4E 5M5M 5M5M An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 6 1 3 2 1 2 1 3 1 1 6G6G 6G6G 7J7J 7J7J 1

44. 44 5M5M 5M5M An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 2 1 2 1 3 1 1 5G5G 5G5G 7J7J 7J7J 2

45. 45 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 2 1 1 3 1 1 5G5G 5G5G 7J7J 7J7J

46. 46 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 2 1 1 1 1 6J6J 6J6J 8H8H 8H8H

47. 47 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 2 1 1 1 1 7K7K 7K7K 8H8H 8H8H 1

48. 48 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 2 1 1 1 1 8I8I 8I8I 8H8H 8H8H 3

49. 49 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 1 1 1 1 8H8H 8H8H

50. 50 An Example (adapted from Sedgewick) A F B D C E G HI JK LM 1 2 2 2 1 2 1 3 1 1 1 1

51. 51 The Algorithm  The algorithm here is the almost the same as the one for the minimum spanning tree  The distance to the initial node is considered instead of the absolute value of the edge  Inserting the node in the priority queue is done by considering all the edges that leave the node plus the value (the priority) assigned to the node that is being removed.  Delete is done based on the priority queue strategy

52. 52 ANOTHER EXAMPLE

53. 53 Dijkstra’s Algorithm (Cont’d) A B D CE F G H 2 1 6 3 8 1 7 56 5 -- A 2 B 1 C 6 D 3 E ? F ? G ? H A B D CE F G H 2 1 6 3 8 1 7 56 5 A 2 B 1 C 6 D 3 E 10 F 8 G ? H S = {A,C,B,E} S = {A,C,B} Cost of SP from A to v i through S

54. 54 Dijkstra’s Algorithm (Cont’d) A B D CE F G H 2 1 6 3 8 1 7 56 5 -- A 2 B 1 C 6 D 3 E 10 F 8 G ? H A B D CE F G H 2 1 6 3 8 1 7 56 5 -- A 2 B 1 C 6 D 3 E 10 F 8 G 14 H S = {A,C,B,E,D,G} S = {A,C,B,E,D} Cost of SP from A to v i through S

55. 55 Dijkstra’s Algorithm (Cont’d) A B D CE F G H 2 1 6 3 8 1 7 56 5 -- A 2 B 1 C 6 D 3 E 10 F 8 G 14 H A B D CE F G H 2 1 6 3 8 1 7 56 5 -- A 2 B 1 C 6 D 3 E 10 F 8 G 14 H S = {A,C,B,E,D,G,F,H} S = {A,C,B,E,D,G,F} Cost of SP from A to v i through S

56. 56 Dijkstra’s Algorithm for Shortest Paths S = {0} /* Current MST */ for i = 0 to n D[i] = M[0][i] /* Shortest path length from 0 to i */ for i = 1 to n-1 find the smallest D[v] such that v  S S = S  {v} for all vertices u  S if (D[u] > D[v] + M[v][u]) then D[u] = D[v] + M[v][u]

57. 57 Dijkstra’s Algorithm --- Example 1 2 4 6 3 5 10 7 8 1 5 6 7 2 3 5 -- 1 3 2 ? 3 ? 4 ? 5 5 6 1 3 2 10 3 ? 4 ? 5 5 6 -- 1 3 2 10 3 7 4 12 5 5 6 -- 1 3 2 10 3 7 4 12 5 5 6 -- 1 3 2 10 3 7 4 11 5 5 6 -- 1 3 2 10 3 7 4 11 5 5 6 -- 3 35 375 3

58. 58 Example – Dijkstra Algorithm  Greedy Algorithm  Assume all weight of edge >0 0 1 2 3 4 5 10 23 1 2 7 4 9 6 source node from node V 0 to other nodes V1V1 10 V2V2 5 V3V3  V4V4  best

59. 59 Animation Online  Dijkstra’s Shortest Path Algorithm - at my website