CS 3343: Analysis of Algorithms Lecture 21: Introduction to Graphs

Uniform-profit Restaurant location problem d Goal: maximize number of restaurants open Subject to: distance constraint (min-separation >= 10)

Events scheduling problem Time e1e1 e2e2 e3e3 e4e4 e5e5 e6e6 e7e7 e8e8 e9e9 Goal: maximize number of non-conflicting events

Fractional knapsack problem Goal: maximize value without exceeding bag capacity Weight limit: 10LB $ / LB Value ($) Weight (LB) item

Example Goal: maximize value without exceeding bag capacity Weight limit: 10LB = 10 LB *2 = 15.4 itemWeight (LB) Value ($) $ / LB

The remaining lectures Graph algorithms Very important in practice –Tons of computational problems can be defined in terms of graphs –We’ll study a few interesting ones Minimum spanning tree Shortest path Graph search Topological sort, connected components

Graphs A graph G = (V, E) –V = set of vertices –E = set of edges = subset of V  V –Thus |E| = O(|V| 2 ) Vertices: {1, 2, 3, 4} Edges: {(1, 2), (2, 3), (1, 3), (4, 3)}

Graph Variations (1) Directed / undirected: –In an undirected graph: Edge (u,v)  E implies edge (v,u)  E Road networks between cities –In a directed graph: Edge (u,v): u  v does not imply v  u Street networks in downtown –Degree of vertex v : The number of edges adjacency to v For directed graph, there are in-degree and out-degree

Directed Undirected Degree = 3 In-degree = 3 Out-degree = 0

Graph Variations (2) Weighted / unweighted: –In a weighted graph, each edge or vertex has an associated weight (numerical value) E.g., a road map: edges might be weighted w/ distance Unweighted Weighted

Graph Variations (3) Connected / disconnected: –A connected graph has a path from every vertex to every other –A directed graph is strongly connected if there is a directed path between any two vertices Connected but not strongly connected

Graph Variations (4) Dense / sparse: –Graphs are sparse when the number of edges is linear to the number of vertices |E|  O(|V|) –Graphs are dense when the number of edges is quadratic to the number of vertices |E|  O(|V| 2 ) –Most graphs of interest are sparse –If you know you are dealing with dense or sparse graphs, different data structures may make sense

Representing Graphs Assume V = {1, 2, …, n} An adjacency matrix represents the graph as a n x n matrix A: –A[i, j] = 1 if edge ( i, j )  E = 0 if edge (i, j)  E For weighted graph –A[i, j] = w ij if edge ( i, j )  E = 0 if edge (i, j)  E For undirected graph –Matrix is symmetric: A[i, j] = A[j, i]

Graphs: Adjacency Matrix Example: A ?? 4

Graphs: Adjacency Matrix Example: A How much storage does the adjacency matrix require? A: O(V 2 )

Graphs: Adjacency Matrix Example: A Undirected graph

Graphs: Adjacency Matrix Example: A Weighted graph

Graphs: Adjacency Matrix Time to answer if there is an edge between vertex u and v: Θ(1) Memory required: Θ(n 2 ) regardless of |E| –Usually too much storage for large graphs –But can be very efficient for small graphs Most large interesting graphs are sparse –E.g., road networks (due to limit on junctions) –For this reason the adjacency list is often a more appropriate representation

Graphs: Adjacency List Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: –Adj[1] = {2,3} –Adj[2] = {3} –Adj[3] = {} –Adj[4] = {3} Variation: can also keep a list of edges coming into vertex

Graph representations Adjacency list How much storage does the adjacency list require? A: O(V+E)

Graph representations Undirected graph A

Graph representations Weighted graph A 2,53,6 1,53,9 3,4 1,62,94,4

Graphs: Adjacency List How much storage is required? For directed graphs –|adj[v]| = out-degree(v) –Total # of items in adjacency lists is  out-degree(v) = |E| For undirected graphs –|adj[v]| = degree(v) –# items in adjacency lists is  degree(v) = 2 |E| So: Adjacency lists take  (V+E) storage Time needed to test if edge (u, v)  E is O(n)

Tradeoffs between the two representations Adj MatrixAdj List test (u, v)  E Θ(1)O(n) Degree(u)Θ(n)O(n) MemoryΘ(n 2 )Θ(n+m) Edge insertionΘ(1) Edge deletionΘ(1)O(n) Graph traversalΘ(n 2 )Θ(n+m) |V| = n, |E| = m Both representations are very useful and have different properties.

Minimum Spanning Tree Problem: given a connected, undirected, weighted graph:

Minimum Spanning Tree Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight A spanning tree is a tree that connects all vertices Number of edges = ? A spanning tree has no designated root.

How to find MST? Connect every node to the closest node? –Does not guarantee a spanning tree

Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees –Let T be an MST of G with an edge (u,v) in the middle –Removing (u,v) partitions T into two trees T 1 and T 2 –w(T) = w(u,v) + w(T 1 ) + w(T 2 ) Claim 1: T 1 is an MST of G 1 = (V 1, E 1 ), and T 2 is an MST of G 2 = (V 2, E 2 ) T1T1 T2T2 u v Proof by contradiction: if T 1 is not optimal, we can replace T 1 with a better spanning tree, T 1 ’ T 1 ’, T 2 and ( u, v ) form a new spanning tree T’ W(T’) < W(T). Contradiction. T1’T1’

Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees –Let T be an MST of G with an edge (u,v) in the middle –Removing (u,v) partitions T into two trees T 1 and T 2 –w(T) = w(u,v) + w(T 1 ) + w(T 2 ) Claim 2: (u, v) is the lightest edge connecting G 1 = (V 1, E 1 ) and G 2 = (V 2, E 2 ) T1T1 T2T2 u v Proof by contradiction: if (u, v) is not the lightest edge, we can remove it, and reconnect T 1 and T 2 with a lighter edge ( x, y ) T 1, T 2 and ( x, y ) form a new spanning tree T’ W(T’) < W(T). Contradiction. x y

Algorithms Generic idea: –Compute MSTs for sub-graphs –Connect two MSTs for sub-graphs with the lightest edge Two of the most well-known algorithms –Prim’s algorithm –Kruskal’s algorithm –Let’s first talk about the ideas behind the algorithms without worrying about the implementation and analysis

Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST Fully explored nodes Discovered but not fully explored nodes Not yet discovered

Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST Fully explored nodes Discovered but not fully explored nodes Not yet discovered

Prim’s algorithm Basic idea: –Start from an arbitrary single node A MST for a single node has no edge –Gradually build up a single larger and larger MST

Prim’s algorithm in words Randomly pick a vertex as the initial tree T Gradually expand into a MST: –For each vertex that is not in T but directly connected to some nodes in T Compute its minimum distance to any vertex in T –Select the vertex that is closest to T Add it to T

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h

Example a a b b f f c c e e d d g g h h Total weight = = 53

Kruskal’s algorithm Basic idea: –Grow many small trees –Find two trees that are closest (i.e., connected with the lightest edge), join them with the lightest edge –Terminate when a single tree forms

Claim If edge (u, v) is the lightest among all edges, (u, v) is in a MST Proof by contradiction: –Suppose that (u, v) is not in any MST –Given a MST T, if we connect (u, v), we create a cycle –Remove an edge in the cycle, have a new tree T’ –W(T’) < W(T) u v By the same argument, the second, third, …, lightest edges, if they do not create a cycle, must be in MST

Kruskal’s algorithm in words Procedure: –Sort all edges into non-decreasing order –Initially each node is in its own tree –For each edge in the sorted list If the edge connects two separate trees, then –join the two trees together with that edge

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

Example a a b b f f c c e e d d g g h h c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15