Liaquat Majeed Sheikh 1 1/25/2016 Graph Algorithms

Liaquat Majeed Sheikh 2 1/25/2016 Graph Algorithms ● Graphs ● Search Algorithms ■ Breadth First Search ■ Depth first search

Liaquat Majeed Sheikh 3 1/25/2016 Review: Graphs ● A graph G = (V, E) ■ V = set of vertices, E = set of edges ■ Dense graph: |E|  |V| 2 ; Sparse graph: |E|  |V| ■ Undirected graph: ○ Edge (u,v) = edge (v,u) ○ No self-loops ■ Directed graph: ○ Edge (u,v) goes from vertex u to vertex v, notated u  v ■ A weighted graph associates weights with either the edges or the vertices

Liaquat Majeed Sheikh 4 1/25/2016 Review: Representing Graphs ● Assume V = {1, 2, …, n} ● An adjacency matrix represents the graph as a n x n matrix A: ■ A[i, j] = 1 if edge (i, j)  E (or weight of edge) = 0 if edge (i, j)  E ■ Storage requirements: O(V 2 ) ○ A dense representation ■ But, can be very efficient for small graphs ○ Especially if store just one bit/edge ○ Undirected graph: only need one diagonal of matrix

Liaquat Majeed Sheikh 5 1/25/2016 Review: Graph Searching ● Given: a graph G = (V, E), directed or undirected ● Goal: methodically explore every vertex and every edge ● Ultimately: build a tree on the graph ■ Pick a vertex as the root ■ Choose certain edges to produce a tree ■ Note: might also build a forest if graph is not connected

Liaquat Majeed Sheikh 6 1/25/2016 Graph Definitions: Path ● A path is a sequence of vertices w 1, w 2, w 3,....w n such that (w i, w i+1 )  E ● Length of a path = # edges in the path ● A loop is an edge from a vertex onto itself. It is denoted by (v, v) ● A simple path is a path where no vertices are repeated along the path ● A cycle is a path with at least one edge such that the first and last vertices are the same, i.e. w 1 = w n

Liaquat Majeed Sheikh 7 1/25/2016 Graph Definitions: Connectedness ● A graph is said to be connected if there is a path from every vertex to every other vertex ● A connected graph is strongly connected if it is a connected graph as well as a directed graph ● A connected graph is weakly connected if it is ■ a directed graph that is not strongly connected, but, ■ the underlying undirected graph is connected Strong or Weak?

Liaquat Majeed Sheikh 8 1/25/2016 Application of Graphs ● Driving Map ■ Edge = Road ■ Vertex = Intersection ■ Edge weight = Time required to cover the road ● Airline Traffic ■ Vertex = Cities serviced by the airline ■ Edge = Flight exists between two cities ■ Edge weight = Flight time or flight cost or both ● Computer networks ■ Vertex = Server nodes ■ Edge = Data link ■ Edge weight = Connection speed ● CAD/VLSI

Liaquat Majeed Sheikh 9 1/25/2016 Breadth-First Search ● “Explore” a graph, turning it into a tree ■ One vertex at a time ■ Expand frontier of explored vertices across the breadth of the frontier ● Builds a tree over the graph ■ Pick a source vertex to be the root ■ Find (“discover”) its children, then their children, etc.

Liaquat Majeed Sheikh 10 1/25/2016 Breadth-First Search ● Again will associate vertex “colors” to guide the algorithm ■ White vertices have not been discovered ○ All vertices start out white ■ Grey vertices are discovered but not fully explored ○ They may be adjacent to white vertices ■ Black vertices are discovered and fully explored ○ They are adjacent only to black and gray vertices ● Explore vertices by scanning adjacency list of grey vertices

Liaquat Majeed Sheikh 11 1/25/2016 Breadth-First Search: Example         rstu vwxy

Liaquat Majeed Sheikh 12 1/25/2016 Breadth-First Search: Example   0      rstu vwxy s Q:

Liaquat Majeed Sheikh 13 1/25/2016 Breadth-First Search: Example 1  0 1     rstu vwxy w Q: r

Liaquat Majeed Sheikh 14 1/25/2016 Breadth-First Search: Example 1    rstu vwxy r Q: tx

Liaquat Majeed Sheikh 15 1/25/2016 Breadth-First Search: Example   rstu vwxy Q: txv

Liaquat Majeed Sheikh 16 1/25/2016 Breadth-First Search: Example  rstu vwxy Q: xvu

Liaquat Majeed Sheikh 17 1/25/2016 Breadth-First Search: Example rstu vwxy Q: vuy

Liaquat Majeed Sheikh 18 1/25/2016 Breadth-First Search: Example rstu vwxy Q: uy

Liaquat Majeed Sheikh 19 1/25/2016 Breadth-First Search: Example rstu vwxy Q: y

Liaquat Majeed Sheikh 20 1/25/2016 Breadth-First Search: Example rstu vwxy Q: Ø

Liaquat Majeed Sheikh 21 1/25/2016 Breadth-First Search BFS(G, s) { initialize vertices; Q = {s};// Q is a queue (duh); initialize to s while (Q not empty) { u = RemoveTop(Q); for each v  u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; } What does v->p represent? What does v->d represent?

Liaquat Majeed Sheikh 22 1/25/2016 BFS: The Code Again BFS(G, s) { initialize vertices; Q = {s}; while (Q not empty) { u = RemoveTop(Q); for each v  u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; } What will be the running time? Touch every vertex: O(V) u = every vertex, but only once (Why?) So v = every vertex that appears in some other vert’s adjacency list Total running time: O(V+E)

Liaquat Majeed Sheikh 23 1/25/2016 BFS: The Code Again BFS(G, s) { initialize vertices; Q = {s}; while (Q not empty) { u = RemoveTop(Q); for each v  u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; } What will be the storage cost in addition to storing the graph? Total space used: O(max(degree(v))) = O(E)

Liaquat Majeed Sheikh 24 1/25/2016 Properties Notation G s : connected component of s Property 1 BFS(G, s) visits all the vertices and edges of G s Property 2 The discovery edges labeled by BFS(G, s) form a spanning tree T s of G s Property 3 For each vertex v in L i ■ The path of T s from s to v has i edges ■ Every path from s to v in G s has at least i edges CB A E D L0L0 L1L1 F L2L2 CB A E D F

Liaquat Majeed Sheikh 25 1/25/2016 Depth-First Search ● Depth-first search is another strategy for exploring a graph ■ Explore “deeper” in the graph whenever possible ■ Edges are explored out of the most recently discovered vertex v that still has unexplored edges ■ When all of v’s edges have been explored, backtrack to the vertex from which v was discovered

Liaquat Majeed Sheikh 26 1/25/2016 Depth-First Search ● Vertices initially colored white ● Then colored gray when discovered ● Then black when finished

Liaquat Majeed Sheikh 27 1/25/2016 DFS Example source vertex

Liaquat Majeed Sheikh 28 1/25/2016 DFS Example 1 | | | | | | | | source vertex d f

Liaquat Majeed Sheikh 29 1/25/2016 DFS Example 1 | | | | | | 2 | | source vertex d f

Liaquat Majeed Sheikh 30 1/25/2016 DFS Example 1 | | | | |3 | 2 | | source vertex d f

Liaquat Majeed Sheikh 31 1/25/2016 DFS Example 1 | | | | |3 | 4 2 | | source vertex d f

Liaquat Majeed Sheikh 32 1/25/2016 DFS Example 1 | | | |5 |3 | 4 2 | | source vertex d f

Liaquat Majeed Sheikh 33 1/25/2016 DFS Example 1 | | | |5 | 63 | 4 2 | | source vertex d f

Liaquat Majeed Sheikh 34 1/25/2016 DFS Example 1 |8 | | |5 | 63 | 4 2 | 7 | source vertex d f

Liaquat Majeed Sheikh 35 1/25/2016 DFS Example 1 |8 | | |5 | 63 | 4 2 | 7 | source vertex d f

Liaquat Majeed Sheikh 36 1/25/2016 DFS Example 1 |8 | | |5 | 63 | 4 2 | 79 | source vertex d f What is the structure of the grey vertices? What do they represent?

Liaquat Majeed Sheikh 37 1/25/2016 DFS Example 1 |8 | | |5 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 38 1/25/2016 DFS Example 1 |8 |11 | |5 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 39 1/25/2016 DFS Example 1 |128 |11 | |5 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 40 1/25/2016 DFS Example 1 |128 |1113| |5 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 41 1/25/2016 DFS Example 1 |128 |1113| 14|5 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 42 1/25/2016 DFS Example 1 |128 |1113| 14|155 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 43 1/25/2016 DFS Example 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 source vertex d f

Liaquat Majeed Sheikh 44 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; }

Liaquat Majeed Sheikh 45 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } What does u->d represent?

Liaquat Majeed Sheikh 46 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } What does u->f represent?

Liaquat Majeed Sheikh 47 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } Will all vertices eventually be colored black?

Liaquat Majeed Sheikh 48 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } What will be the running time?

Liaquat Majeed Sheikh 49 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } Running time: O(n 2 ) because call DFS_Visit on each vertex, and the loop over Adj[] can run as many as |V| times

Liaquat Majeed Sheikh 50 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } BUT, there is actually a tighter bound. How many times will DFS_Visit() actually be called?

Liaquat Majeed Sheikh 51 1/25/2016 Depth-First Search: The Code DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; } So, running time of DFS = O(V+E)

Liaquat Majeed Sheikh 52 1/25/2016 Depth-First Sort Analysis ● This running time argument is an informal example of amortized analysis ■ “Charge” the exploration of edge to the edge: ○ Each loop in DFS_Visit can be attributed to an edge in the graph ○ Runs once/edge if directed graph, twice if undirected ○ Thus loop will run in O(E) time, algorithm O(V+E)  Considered linear for graph, b/c adj list requires O(V+E) storage ■ Important to be comfortable with this kind of reasoning and analysis

Liaquat Majeed Sheikh 53 1/25/2016 DFS: Kinds of edges ● DFS introduces an important distinction among edges in the original graph: ■ Tree edge: encounter new (white) vertex ○ The tree edges form a spanning forest ○ Can tree edges form cycles? Why or why not?

Liaquat Majeed Sheikh 54 1/25/2016 DFS Example 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 source vertex d f Tree edges

Liaquat Majeed Sheikh 55 1/25/2016 DFS: Kinds of edges ● DFS introduces an important distinction among edges in the original graph: ■ Tree edge: encounter new (white) vertex ■ Back edge: from descendent to ancestor ○ Encounter a grey vertex (grey to grey)

Liaquat Majeed Sheikh 56 1/25/2016 DFS Example 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 source vertex d f Tree edgesBack edges

Liaquat Majeed Sheikh 57 1/25/2016 DFS: Kinds of edges ● DFS introduces an important distinction among edges in the original graph: ■ Tree edge: encounter new (white) vertex ■ Back edge: from descendent to ancestor ■ Forward edge: from ancestor to descendent ○ Not a tree edge, though ○ From grey node to black node

Liaquat Majeed Sheikh 58 1/25/2016 DFS Example 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 source vertex d f Tree edgesBack edgesForward edges

Liaquat Majeed Sheikh 59 1/25/2016 DFS: Kinds of edges ● DFS introduces an important distinction among edges in the original graph: ■ Tree edge: encounter new (white) vertex ■ Back edge: from descendent to ancestor ■ Forward edge: from ancestor to descendent ■ Cross edge: between a tree or subtrees ○ From a grey node to a black node

Liaquat Majeed Sheikh 60 1/25/2016 DFS Tree – Directed Graphs d e f a b c g 1/10 a 2/5 6/9 12/13 11/14 3/4 7/8 b c f g C F B d e C C ● For directed graphs, the other edges of the graph can be classified as follows: ■ Back Edges: (u, v) where v is ancestor of u in the tree ○ Thus a self-loop is considered to be a back-edge ■ Forward Edges: (u, v) where v is a proper descendant of u in the tree ■ Cross Edges: (u, v) where u and v are not ancestors or descendants of one another in the tree (in fact, the edge may go between different trees in the forest)

Liaquat Majeed Sheikh 61 1/25/2016 DFS Example 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 source vertex d f Tree edgesBack edgesForward edgesCross edges

Liaquat Majeed Sheikh 62 1/25/2016 DFS: Kinds of edges ● DFS introduces an important distinction among edges in the original graph: ■ Tree edge: encounter new (white) vertex ■ Back edge: from descendent to ancestor ■ Forward edge: from ancestor to descendent ■ Cross edge: between a tree or subtrees ● Note: tree & back edges are important; most algorithms don’t distinguish forward & cross

Liaquat Majeed Sheikh 63 1/25/2016 DFS: Kinds Of Edges ● Theorem : If G is undirected, a DFS produces only tree and back edges ● Proof by contradiction: ■ Assume there’s a forward edge ○ But F? edge must actually be a back edge (why?) source F?

Liaquat Majeed Sheikh 64 1/25/2016 DFS: Kinds Of Edges ● Theorem : If G is undirected, a DFS produces only tree and back edges ● Proof by contradiction: ■ Assume there’s a cross edge ○ But C? edge cannot be cross: ○ must be explored from one of the vertices it connects, becoming a tree vertex, before other vertex is explored ○ So in fact the picture is wrong…both lower tree edges cannot in fact be tree edges source C?

Liaquat Majeed Sheikh 65 1/25/2016 DFS And Graph Cycles ● Thm: An undirected graph is acyclic iff a DFS yields no back edges ■ If acyclic, no back edges (because a back edge implies a cycle ■ If no back edges, acyclic ○ No back edges implies only tree edges (Why?) ○ Only tree edges implies we have a tree or a forest ○ Which by definition is acyclic ● Thus, can run DFS to find whether a graph has a cycle

Liaquat Majeed Sheikh 66 1/25/2016 DFS And Cycles ● How would you modify the code to detect cycles? DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; }

Liaquat Majeed Sheikh 67 1/25/2016 DFS And Cycles ● What will be the running time? DFS(G) { for each vertex u  G->V { u->color = WHITE; } time = 0; for each vertex u  G->V { if (u->color == WHITE) DFS_Visit(u); } DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v  u->Adj[] { if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; time = time+1; u->f = time; }

Liaquat Majeed Sheikh 68 1/25/2016 DFS And Cycles ● What will be the running time? ● A: O(V+E) ● We can actually determine if cycles exist in O(V) time: ■ In an undirected acyclic forest, |E|  |V| - 1 ■ So count the edges: if ever see |V| distinct edges, must have seen a back edge along the way

Liaquat Majeed Sheikh 69 1/25/2016 Depth-First Search - Classification of Edges Theorem In a depth-first search of an undirected graph G, every edge of G is either a tree edge or a back edge.

Liaquat Majeed Sheikh 70 1/25/2016 Depth-First Search - White Path Theorem Theorem If during depth-first search a “white” vertex u is reachable from a “grey” vertex v via path that contains only “white” vertices, then vertex u will be a descendant on v in depth-first search forest.

Liaquat Majeed Sheikh 71 1/25/2016 Depth-First Search - Timestamps Parenthesis Theorem After DepthFirstSearch algorithm terminates for any two vertices u and v exactly one from the following three conditions holds the intervals [d[u],f[u]] and [d[v],f[v]] are entirely disjoint the intervals [d[u],f[u]] is contained entirely within the interval [d[v],f[v]] and u is a descendant of v in depth- first tree the intervals [d[v],f[v]] is contained entirely within the interval [d[u],f[u]] and v is a descendant of u in depth- first tree

Liaquat Majeed Sheikh 72 1/25/2016 Depth-First Search - Timestamps 3/6 7/8 1/102/9 12/134/5 abs def BF 11/16 14/15 c g C C C C B

Liaquat Majeed Sheikh 73 1/25/2016 Depth-First Search – Timestamps (s (b (a (d d) a) (e e) b) s) (c (f f) (g g) c) s b c a e d fg

Liaquat Majeed Sheikh 74 1/25/2016 Depth-First Search - Timestamps ae s bf d B F c g C C C C B

Liaquat Majeed Sheikh 75 1/25/2016 DFS - Checking for cycles [Adapted from M.Golin]

Liaquat Majeed Sheikh 76 1/25/2016 Applications of DFS In O(|V| + |E|) time, we can Find connected components of G. Determine if G has a cycle. Determine if removing a vertex or edge will disconnect G. Determine if G is planar. …

Liaquat Majeed Sheikh 77 1/25/2016 Algorithm for Detecting Cycle (v, u) is a back edge if v is a descendant of u in the DFS tree. for each u  V do onpath(u)  false // on path from the root of the DFS tree DFS-visit(v) time  time + 1 disc(v)  time onpath(v)  true for each u  Adj(v) do if disc(u) = unseen then DFS-visit(u) else if onpath(u) then a cycle has been found; halt onpath(v)  false // backtrack: v no longer on path from root u v root

Liaquat Majeed Sheikh 78 1/25/2016 [Adapted from M.Golin]