Dijkstra’s Algorithm

Announcements Assignment #2 Due Tonight Exams Graded Assignment #3 Posted

Announcements

Last Time Breadth First Search, Depth First Search ◦ Different algorithms for searching all nodes in a graph. Topological Sort ◦ Produces an ordering of vertices in a graph, given certain constrains (A before B, B before C, etc.) Dijkstra’s Algorithm ◦ Finds the shortest paths from a source vertex to all other vertices.

Today Finish up Dijkstra’s ◦ Path reconstruction MSTs – Minimum Spanning Tree algorithms ◦ Prim’s ◦ Kruskal’s

Dijkstra’s Algorithm This algorithm finds the shortest path from a source vertex to all other vertices in a weighted directed graph without negative edge weights. ec bad

Dijkstra’s Path Reconstruction Example shown on the Board

Minimum Spanning Trees In a weighted, undirected graph, it is a tree formed by connecting all of the vertices with minimal cost. ◦ The MST is a tree because it’s acyclic. ◦ It’s spanning because it covers every vertex. ◦ And it’s minimum because it has minimum cost. v1v1 v7v7 v6v6 v4v4 v3v3 v5v5 v2v v1v1 v7v7 v6v6 v4v4 v3v3 v5v5 v2v

MSTs – proof that greedy works Let G be a graph with vertices in the set V partitioned into two sets V 1 and V 2. Then the minimum weight edge, e, that connects a vertex from V 1 to V 2 is part of a minimum spanning tree of G. Proof: Consider a MST T of G that does NOT contain the minimum weight edge e. ◦ This MUST have at least one edge in between a vertex from V 1 to V 2. (Otherwise, no vertices between those two sets would be connected.) ◦ Let G contain edge f that connects V 1 to V 2. ◦ Now, add in edge e to T. ◦ This creates a cycle. In particular, there was already one path from every vertex in V 1 to V 2 and with the addition of e, there are two. ◦ Thus, we can form a cycle involving both e and f. Now, imagine removing f from this cycle. ◦ This new graph, T' is also a spanning tree, but it's total weight is less than or equal to T because we replaced e with f, and e was the minimum weight edge.

MSTs – proof that greedy works As a spanning tree is created ◦ If the edge that is added is the one of minimum cost that avoids creation of a cycle. ◦ Then the cost of the resulting spanning tree cannot be improved  Because any replacement edge would have cost at least as much as an edge already in the spanning tree. This is why greedy works!

Kruskal’s Algorithm Let V =  For i=1 to n-1, (where there are n vertices in a graph) V = V  e, where e is the edge with the minimum edge weight not already in V, and that does NOT form a cycle when added to V. Return V Basically, you build the MST of the graph by continually adding in the smallest weighted edge into the MST that doesn't form a cycle. ◦ When you are done, you'll have an MST. ◦ You HAVE to make sure you never add an edge the forms a cycle and that you always add the minimum of ALL the edges left that don't.

Kruskal’s v1v1 v7v7 v6v6 v4v4 v3v3 v5v5 v2v v1v1 v4v4 1 v1v1 v4v4 1 v7v7 v6v6 1 v7v7 v6v6 1 v1v1 v2v2 2 v4v4 v3v3 2 v4v4 v2v2 3 v7v7 v4v4 4 v6v6 v3v3 5 v7v7 v5v5 6 v1v1 v3v3 4 v4v4 v5v5 7 v6v6 v4v4 8 v5v5 v2v2 v2v2 2 v3v3 2 3 CYCLE! 4 4 v5v5 6 All Vertices, we’re done!! Determine the MST: Edges in sorted order: Given Graph G:

Kruskal’s The reason this works: ◦ is that each added edge is connecting between two sets of vertices, ◦ and since we select the edges in order by weight, ◦ we are always selecting the minimum edge weight that connects the two sets of vertices. Cycle detection: ◦ Keep track of disjoint sets. ◦ Initially, each vertex is in its own disjoint set. ◦ When you add an edge you are unioning two sets. ◦ A union cannot happen if the two vertices are already in the same set.  This would create a cycle.

Prim’s Algorithm This is quite similar to Kruskal's with one big difference: ◦ The tree that you are "growing" ALWAYS stays connected.  Whereas in Kruskal's you could add an edge to your growing tree that wasn't connected to the rest of it, here you can NOT do it. Here is the algorithm: 1) Set S = . 1) Pick any vertex in the graph. 2) Add the minimum edge incident to that vertex to S. 3) Continue to add edges into S (n-2 more times) using the following rule: Add the minimum edge weight to S that is incident to S but that doesn't form a cycle when added to S.

Prim’s v1v1 v7v7 v6v6 v4v4 v3v3 v5v5 v2v v1v1 v4v4 1 v7v7 v6v6 1 v1v1 v2v2 2 v4v4 v3v3 2 v4v4 v2v2 3 v7v7 v4v4 4 v6v6 v3v3 5 v7v7 v5v5 6 v1v1 v3v3 4 v4v4 v5v5 7 v6v6 v4v4 8 v5v5 v2v2 Edges in sorted order: Given Graph G: Determine the MST, using Prim’s starting with vertexV1: v1v1 v4v4 1 v7v7 v6v6 1 v2v2 2 v3v3 2 3 CYCLE! 4 4 v5v5 6 All Vertices, we’re done!!

Example on the board

Huffman Encoding Compress the storage of data using variable length codes. ◦ For example, each character in a text file is stored using 8 bits. ◦ Nice and easy because we always read in 8 bits for a single character. Not the most efficient… ◦ What if ‘e’ is used 10 times more frequently than ‘q’. ◦ It would be more advantageous for us to use a 7 bit code for e and a 9 bit code for q.

Huffman Coding Finds the optimal way to take advantage of varying character frequencies in a particular file. ◦ On average, standard files can shrink them anywhere from 10% to 30% depending to the character distribution. The idea behind the coding is to give less frequent characters and groups of characters longer codes. Also, the coding is constructed in such a way that no two constructed codes are prefixes of each other. This property about the code is crucial with respect to easily deciphering the code.

Building a Huffman Tree Example on the board

References Slides adapted from Arup Guha’s Computer Science II Lecture notes: ctures/ ctures/ Additional material from the textbook: Data Structures and Algorithm Analysis in Java (Second Edition) by Mark Allen Weiss Additional images: xkcd.com