1. CS 3343: Analysis of Algorithms Lecture 21: Introduction to Graphs

2. Uniform-profit Restaurant location problem 5226663107 d 0 57915 0 69 0 10 0 7 Goal: maximize number of restaurants open Subject to: distance constraint (min-separation >= 10)

3. Events scheduling problem Time e1e1 e2e2 e3e3 e4e4 e5e5 e6e6 e7e7 e8e8 e9e9 Goal: maximize number of non-conflicting events

4. Fractional knapsack problem Goal: maximize value without exceeding bag capacity Weight limit: 10LB 1.5 2 1.2 1 0.75 1 $ / LB 966 425 654 333 342 221 Value ($) Weight (LB) item

5. Example Goal: maximize value without exceeding bag capacity Weight limit: 10LB 2 + 6 + 2 = 10 LB 4 + 9 + 1.2*2 = 15.4 itemWeight (LB) Value ($) $ / LB 5242 6691.5 4561.2 1221 3331 2430.75

6. The remaining lectures Graph algorithms Very important in practice –Tons of computational problems can be defined in terms of graphs –We’ll study a few interesting ones Minimum spanning tree Shortest path Graph search Topological sort, connected components

7. Graphs A graph G = (V, E) –V = set of vertices –E = set of edges = subset of V  V –Thus |E| = O(|V| 2 ) 1 24 3 Vertices: {1, 2, 3, 4} Edges: {(1, 2), (2, 3), (1, 3), (4, 3)}

8. Graph Variations (1) Directed / undirected: –In an undirected graph: Edge (u,v)  E implies edge (v,u)  E Road networks between cities –In a directed graph: Edge (u,v): u  v does not imply v  u Street networks in downtown –Degree of vertex v : The number of edges adjacency to v For directed graph, there are in-degree and out-degree

9. 1 24 3 Directed 1 24 3 Undirected Degree = 3 In-degree = 3 Out-degree = 0

10. Graph Variations (2) Weighted / unweighted: –In a weighted graph, each edge or vertex has an associated weight (numerical value) E.g., a road map: edges might be weighted w/ distance 1 24 3 1 24 3 Unweighted Weighted 0.3 0.4 1.2 1.9

11. Graph Variations (3) Connected / disconnected: –A connected graph has a path from every vertex to every other –A directed graph is strongly connected if there is a directed path between any two vertices 1 24 3 Connected but not strongly connected

12. Graph Variations (4) Dense / sparse: –Graphs are sparse when the number of edges is linear to the number of vertices |E|  O(|V|) –Graphs are dense when the number of edges is quadratic to the number of vertices |E|  O(|V| 2 ) –Most graphs of interest are sparse –If you know you are dealing with dense or sparse graphs, different data structures may make sense

13. Representing Graphs Assume V = {1, 2, …, n} An adjacency matrix represents the graph as a n x n matrix A: –A[i, j] = 1 if edge ( i, j )  E = 0 if edge (i, j)  E For weighted graph –A[i, j] = w ij if edge ( i, j )  E = 0 if edge (i, j)  E For undirected graph –Matrix is symmetric: A[i, j] = A[j, i]

14. Graphs: Adjacency Matrix Example: 1 24 3 A1234 1 2 3 ?? 4

15. Graphs: Adjacency Matrix Example: 1 24 3 A1234 10110 20010 30000 40010 How much storage does the adjacency matrix require? A: O(V 2 )

16. Graphs: Adjacency Matrix Example: 1 24 3 4 3 2 0100 1011 0101 01101 4321A Undirected graph

17. Graphs: Adjacency Matrix Example: 1 24 3 5 6 94 4 3 2 0400 4096 0905 06501 4321A Weighted graph

18. Graphs: Adjacency Matrix Time to answer if there is an edge between vertex u and v: Θ(1) Memory required: Θ(n 2 ) regardless of |E| –Usually too much storage for large graphs –But can be very efficient for small graphs Most large interesting graphs are sparse –E.g., road networks (due to limit on junctions) –For this reason the adjacency list is often a more appropriate representation

19. Graphs: Adjacency List Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: –Adj[1] = {2,3} –Adj[2] = {3} –Adj[3] = {} –Adj[4] = {3} Variation: can also keep a list of edges coming into vertex 1 24 3

20. Graph representations Adjacency list 1 24 3 23 3 3 How much storage does the adjacency list require? A: O(V+E)

21. Graph representations Undirected graph 1 24 3 4 3 2 0100 1011 0101 01101 4321A 23 1 3 3 124

22. Graph representations Weighted graph 1 24 3 5 6 94 4 3 2 0400 4096 0905 06501 4321A 2,53,6 1,53,9 3,4 1,62,94,4

23. Graphs: Adjacency List How much storage is required? For directed graphs –|adj[v]| = out-degree(v) –Total # of items in adjacency lists is  out-degree(v) = |E| For undirected graphs –|adj[v]| = degree(v) –# items in adjacency lists is  degree(v) = 2 |E| So: Adjacency lists take  (V+E) storage Time needed to test if edge (u, v)  E is O(n)

24. Tradeoffs between the two representations Adj MatrixAdj List test (u, v)  E Θ(1)O(n) Degree(u)Θ(n)O(n) MemoryΘ(n 2 )Θ(n+m) Edge insertionΘ(1) Edge deletionΘ(1)O(n) Graph traversalΘ(n 2 )Θ(n+m) |V| = n, |E| = m Both representations are very useful and have different properties.

25. Minimum Spanning Tree Problem: given a connected, undirected, weighted graph: 14 10 3 64 5 2 9 15 8

26. Minimum Spanning Tree Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 14 10 3 64 5 2 9 15 8 A spanning tree is a tree that connects all vertices Number of edges = ? A spanning tree has no designated root.

27. How to find MST? Connect every node to the closest node? –Does not guarantee a spanning tree

28. Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees –Let T be an MST of G with an edge (u,v) in the middle –Removing (u,v) partitions T into two trees T 1 and T 2 –w(T) = w(u,v) + w(T 1 ) + w(T 2 ) Claim 1: T 1 is an MST of G 1 = (V 1, E 1 ), and T 2 is an MST of G 2 = (V 2, E 2 ) T1T1 T2T2 u v Proof by contradiction: if T 1 is not optimal, we can replace T 1 with a better spanning tree, T 1 ’ T 1 ’, T 2 and ( u, v ) form a new spanning tree T’ W(T’) < W(T). Contradiction. T1’T1’

29. Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees –Let T be an MST of G with an edge (u,v) in the middle –Removing (u,v) partitions T into two trees T 1 and T 2 –w(T) = w(u,v) + w(T 1 ) + w(T 2 ) Claim 2: (u, v) is the lightest edge connecting G 1 = (V 1, E 1 ) and G 2 = (V 2, E 2 ) T1T1 T2T2 u v Proof by contradiction: if (u, v) is not the lightest edge, we can remove it, and reconnect T 1 and T 2 with a lighter edge ( x, y ) T 1, T 2 and ( x, y ) form a new spanning tree T’ W(T’) < W(T). Contradiction. x y

30. Algorithms Generic idea: –Compute MSTs for sub-graphs –Connect two MSTs for sub-graphs with the lightest edge Two of the most well-known algorithms –Prim’s algorithm –Kruskal’s algorithm –Let’s first talk about the ideas behind the algorithms without worrying about the implementation and analysis

31. Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST 6 5 7 Fully explored nodes Discovered but not fully explored nodes Not yet discovered

32. Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST 6 5 7 2 9 4 Fully explored nodes Discovered but not fully explored nodes Not yet discovered

33. Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST 6 5 7 2 9 4

34. Prim’s algorithm in words Randomly pick a vertex as the initial tree T Gradually expand into a MST: –For each vertex that is not in T but directly connected to some nodes in T Compute its minimum distance to any vertex in T –Select the vertex that is closest to T Add it to T

35. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

36. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

37. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

38. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

39. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

40. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

41. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

42. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 Total weight = 3 + 8 + 6 + 5 + 7 + 9 + 15 = 53

43. Kruskal’s algorithm Basic idea: –Grow many small trees –Find two trees that are closest (i.e., connected with the lightest edge), join them with the lightest edge –Terminate when a single tree forms

44. Claim If edge (u, v) is the lightest among all edges, (u, v) is in a MST Proof by contradiction: –Suppose that (u, v) is not in any MST –Given a MST T, if we connect (u, v), we create a cycle –Remove an edge in the cycle, have a new tree T’ –W(T’) < W(T) u v By the same argument, the second, third, …, lightest edges, if they do not create a cycle, must be in MST

45. Kruskal’s algorithm in words Procedure: –Sort all edges into non-decreasing order –Initially each node is in its own tree –For each edge in the sorted list If the edge connects two separate trees, then –join the two trees together with that edge

46. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

47. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

48. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

49. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

50. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

51. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

52. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

53. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

54. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

55. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

56. Example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15