1. Greedy Algorithms

2. Expected Outcomes Students should be able to
Explain the idea Dijkstra’s algorithm
Explain the idea of Bellman-Ford algorithm
Prove the correctness of Dijkstra’s algorithm
Analyze the time complexity of Dijkstra’s and Bellman-Ford algorithm
Solve the system of difference constraints problem using the two algorithms for single source shortest paths problem

3. Shortest Paths – Dijkstra’s Algorithm
Shortest Path Problems
All pair shortest paths (Floy’s algorithm)
Single Source Shortest Paths Problem (Dijkstra’s algorithm): Given a weighted graph G, find the shortest paths from a source vertex s to each of the other vertices. a e d c b 3 4 6 2 5 7

4. Tree vertices Remaining vertices a(-,0) b(a,3) c(-,∞) d(a,7) e(-,∞)4 a b e 3 7 6 2 5 c
Tree vertices Remaining vertices
a(-,0) b(a,3) c(-,∞) d(a,7) e(-,∞) a b d 4 c e 3 7 6 2 5 4
b(a,3) c(b,3+4) d(b,3+2) e(-,∞) b c 3 6 2 5 a d e 7 4
d(b,5) c(b,7) e(d,5+4) 4 b c 3 6 2 5 a d 7 e 4 4
c(b,7) e(d,9) b c 3 6 2 5 a d e 7 4
e(d,9)

5. Prim’s and Dijkstra’s Algorithms
Generate different kinds of spanning trees
Prim’s: a minimum spanning tree.
Dijkstra’s : a spanning tree rooted at a given source s, such that the distance from s to every other vertex is the shortest.
Different greedy strategies
Prims’: Always choose the closest (to the tree) vertex in the priority queue Q to add to the expanding tree VT.
Dijkstra’s : Always choose the closest (to the source) vertex in the priority queue Q to add to the expanding tree VT.
Different labels for each vertex
Prims’: parent vertex and the distance from the tree to the vertex..
Dijkstra’s : parent vertex and the distance from the source to the vertex.

6. Shortest Paths-Dijkstra’s Algorithm
Dijkstra’s algorithm: Similar to Prim’s MST algorithm, with the following difference:
Start with tree consisting of one vertex.
“Grow” tree one vertex/edge at a time.
Construct a series of expanding subtrees T1, T2, …
Keep track of shortest path from source to each of the vertices in Ti
At each stage construct Ti+1 from Ti: add minimum weight edge connecting a vertex in tree (Ti) to one not yet in tree
choose from “fringe” edges
(this is the “greedy” step!)
Algorithm stops when all vertices are included.
Here Dijkstra’s algorithm is shown in comparison to Prim’s algorithm to emphasize
that the idea is pretty much the same, but to also highlight the differences.
edge (u*,u) with lowest du* + w(u*, u)

7. Dijkstra’s Algorithm ALGORITHM Dijkstra(G, s)
//Input: A weighted connected graph G = <V, E> and a source vertex s
//Output: The length dv of a shortest path from s to v and its penultimate vertex pv for every vertex v in V
Initialize (Q) //initialize vertex priority in the priority queue
for every vertex v in V do
dv ∞ ; Pv  null // Pv , the parent of v
insert(Q, v, dv) //initialize vertex priority in the priority queue
ds 0; Decrease(Q, s, ds) //update priority of s with ds, making ds, the minimum
VT  
for i  0 to |V| - 1 do //produce |V| - 1 edges for the tree
u*  DeleteMin(Q) //delete the minimum priority element
VT VT U {u*} //expanding the tree, choosing the locally best vertex
for every vertex u in V – VT that is adjacent to u* do
if du* + w(u*, u) < du
du  du + w(u*, u); pu  u*
Decrease(Q, u, du)
Dijkstra’s : Always choosing the closest (closest to the source) vertex in Q to add to VT greedy.
Prims’: Always choosing the closest (to the tree) vertex in Q to add to VT greedy.

8. Correctness of Dijkstra’s
Dijkstra’s algorithm, run on a weighted, directed graph G=(V, E), with non-negative weight function w and source s, terminates with d[u]= (s,u) for all vertices uV. Here (s,u) means the shortest distance from s to u.
Proof (by contradiction)
Since S = V in the end and for each vertex v, after it was put into S, the value of d[u] would never be changed. we only need to prove that for each vertex v added to S, there holds d[v]= (s, v) when v is added to S.
Suppose that u is the first vertex for which d[u]  (s, u) when it was added to S
Note
u is not s because d[s] = 0= d(s, s)
There must be a path s...u, since otherwise d[u]= (s, u) = .
Since there’s a path, there must be a shortest path (note there is no negative cycle).

9. Dijkstra’s Algorithm - Proof
Let s®x®y®u be the shortest path from s to u, where at the moment u is chosen to S,
x is in S and y is the first outside S (y may not exist)
When x was added to S, d[x] = d(s, x)
Edge x®y will be considered at that time, d[y] <= d(s, x)+w(x, y) = d(s, y) (why?)

10. Dijkstra’s Algorithm - Proof
so d[y] = d(s, y) £ d(s, u) £ d[u]
But, when we chose u, both u and y are in Q, so d[u] £ d[y] (otherwise we would have chosen y)
Thus the inequalities must be equalities
d[y] = d(s, y) = d(s, u) = d[u]
And our hypothesis (d[u] ¹ d(s, u)) is contradicted!
Note! if such y does not exist, proof is easier

11. Notes on Dijkstra’s Algorithm
Doesn’t work with negative weights
Can you give a counter example?
How about if the negative weight edges are from s?
Applicable to both undirected and directed graphs
Efficiency
Use unordered array to store the priority queue: Θ(n2)
Use min-heap to store the priority queue: O(m log n)
Use Fibonacci-heap, O(nlog n+m)

12. Bellman’s Algorithm
Negative weight edge is permitted!

14. Linear Programming Linear programming problem: Notes
Input: matrix Amn, vector bm, and vector cn.
Output: vector xn, such that
maximize cTx
subject to Ax  b.
Notes
This problem has polynomial time solution.
Many problems can be reduced to LP.

15. Feasibility Problem of LP
Feasible solution: xn, subject to Ax  b.
Feasibility problem of LP:
Given A, b, c
Output: either a feasible solution x if one exists, or a judgment that no feasible solution exists.

16. Systems of difference constraints
Each row of A contains exactly one “1” and one “-1”. All other elements are “0”.
Each row is a difference constraint:
xj-xi  bk, 1  k  m.
An example
x1-x2  0
x1-x5  -1
x2--x5  1
x3-x1  5
x4-x1  4
x4-x3  -1
x5-x3  -3
x5-x4  -3
Solution x = (-5,-3,0,-1,-4); x’=(-5+d,-3+d,0+d,-1+d,-4+d)

17. Constraints graph Given a system Ax  b of difference constraints
The constraint graph is a weighted, directed graph G = (V, E;W), where:
V={v0,v1,…,vn} //v0 is an extra vertex
E={(vi, vj): xj-xi  bk is a constraint}
{(v0, v1), (v0, v2),…, (v0, vn)}. //each vertex is reachable from v0
w(vi, vj) = bk, if xj-xi  bk is a constraint;
w(v0, vi) = 0.

18. Constraint graph of example

19. Solve Difference Constraints by SSSP in constraint graph
Given a system Ax  b of difference constraints, let G = (V, E; W) be the corresponding constraint graph.
If G contains no negative-weight cycles, then x = ((v0,v1), (v0,v2), …, (v0,vn)) is a feasible solution for the system.
If G contains a negative-weight cycle, then there is no feasible solution for the system.

20. Solution to Difference Constraints
Bellman-Ford algorithm.
If G contains no negative-weight cycle, then, it contains no negative-weight cycle reachable from v0, then Bellman-Ford returns TRUE, and x = ((v0,v1), …, (v0,vn)) gives a solution.
If G contains a negative-weight cycle, then this cycle must be reachable from v0. Then Bellman-Ford returns FALSE.
Time complexity: O(VE) = O((n+1)(n+m)) = O(n2+nm)