1. Richard Anderson Lecture 1
CSE 421 Algorithms
Richard Anderson
Lecture 1

2. Course Introduction Instructor Teaching Assistant
Richard Anderson,
Teaching Assistant
Yiannis Giotas,

3. All of Computer Science is the Study of Algorithms

4. Mechanics It’s on the web Weekly homework Midterm Final exam
Subscribe to the mailing list

5. Text book Algorithm Design Jon Kleinberg, Eva Tardos
Read Chapters 1 & 2

6. How to study algorithms
Zoology
Mine is faster than yours is
Algorithmic ideas
Where algorithms apply
What makes an algorithm work
Algorithmic thinking

7. Introductory Problem: Stable Matching
Setting:
Assign TAs to Instructors
Avoid having TAs and Instructors wanting changes
E.g., Prof A. would rather have student X than her current TA, and student X would rather work for Prof A. than his current instructor.

8. Formal notions Perfect matching Ranked preference lists Stability m1w1 m2 w2

9. Examples m1: w1 w2 m2: w2 w1 w1: m1 m2 w2: m2 m1 m1: w1 w2 m2: w1 w2

10. Examples
m1: w1 w2
m2: w2 w1
w1: m2 m1
w2: m1 m2

11. Intuitive Idea for an Algorithm
m proposes to w
If w is unmatched, w accepts
If w is matched to m2
If w prefers m to m2, w accepts
If w prefers m2 to m, w rejects
Unmatched m proposes to highest w on its preference list

12. Algorithm Initially all m in M and w in W are free
While there is a free m
w highest on m’s list that m has not proposed to
if w is free, then match (m, w)
else
suppose (m2, w) is matched
if w prefers m to m2
unmatch (m2, w)
match (m, w)

13. Does this work? Does it terminate? Is the result a stable matching?
Begin by identifying invariants and measures of progress
m’s proposals get worse
Once w is matched, w stays matched
w’s partners get better

14. Claim: The algorithms stops in at most n2 steps
Why?

15. The algorithm terminates with a perfect matching
Why?

16. The resulting matching is stable
Suppose
m1 prefers w2 to w1
w2 prefers m1 to m2
How could this happen? m1 w1 m2 w2

17. Richard Anderson Lecture 2
CSE 421 Algorithms
Richard Anderson
Lecture 2

18. Announcements Office Hours Homework Reading Richard Anderson, CSE 582
Monday, 10:00 – 11:00
Friday, 11:00 – 12:00
Yiannis Giotas, CSE 220
Monday, 2:30-3:20
Friday, 2:30-3:20
Homework
Assignment 1, Due Wednesday, October 5
Reading
Read Chapters 1 & 2

19. Stable Matching Find a perfect matching with no instabilities
Instability
(m1, w1) and (m2, w2) matched
m1 prefers w2 to w1
w2 prefers m1 to m2 m1 w1 m2 w2

20. Intuitive Idea for an Algorithm
m proposes to w
If w is unmatched, w accepts
If w is matched to m2
If w prefers m to m2, w accepts
If w prefers m2 to m, w rejects
Unmatched m proposes to highest w on its preference list

21. Algorithm Initially all m in M and w in W are free
While there is a free m
w highest on m’s list that m has not proposed to
if w is free, then match (m, w)
else
suppose (m2, w) is matched
if w prefers m to m2
unmatch (m2, w)
match (m, w)

22. Does this work? Does it terminate? Is the result a stable matching?
Begin by identifying invariants and measures of progress
m’s proposals get worse
Once w is matched, w stays matched
w’s partners get better

23. Claim: The algorithm stops in at most n2 steps
Why?
Each m asks each w at most once

24. The algorithm terminates with a perfect matching
Why?
If m is free, there is a w that has not been proposed to

25. The resulting matching is stable
Suppose
m1 prefers w2 to w1
w2 prefers m1 to m2
How could this happen? m1 w1 m2 w2
m1 proposed to w2 before w1
m2 rejected m1
m2 prefers m3 to m1
m2 prefers m2 to m3

26. Result Simple, O(n2) algorithm to compute a stable matching Corollary
A stable matching always exists

27. A closer look Stable matchings are not necessarily fair m1: w1 w2 w3
w1: m2 m3 m1
w2: m3 m1 m2
w3: m1 m2 m3 m2 w2 m3 w3

28. Algorithm under specified
Many different ways of picking m’s to propose
Surprising result
All orderings of picking free m’s give the same result
Proving this type of result
Reordering argument
Prove algorithm is computing something mores specific
Show property of the solution – so it computes a specific stable matching

29. Proposal Algorithm finds the best possible solution for M
And the worst possible for W
(m, w) is valid if (m, w) is in some stable matching
best(m): the highest ranked w for m such that (m, w) is valid
S* = {(m, best(m)}
Every execution of the proposal algorithm computes S*

30. Proof Argument by contradiction
Suppose the algorithm computes a matching S different from S*
There must be some m rejected by a valid partner.
Let m be the first man rejected by a valid partner w. w rejects m for m1.
w = best(m)

31. S+ stable matching including (m, w) Suppose m1 is paired with w1 in S+
m1 prefers w to w1
w prefers m1 to m
Hence, (m1, w) is an instability in S+ m w m1 w1
Since m1 could not have been rejected by w1 at this point, because (m, w) was the first valid pair rejected. (m1, v1) is valid because it is in S+. m w m1 w1

32. The proposal algorithm is worst case for W
In S*, each w is paired with its worst valid partner
Suppose (m, w) in S* but not m is not the worst valid partner of w
S- a stable matching containing the worst valid partner of w
Let (m1, w) be in S-, w prefers m to m1
Let (m, w1) be in S-, m prefers w to w1
(m, w) is an instability in S- m w1
w prefers m to m1 because m1 is the wvp
w prefers w to w1 because S* has all the bvp’s m1 w

33. Could you do better? Is there a fair matching
Design a configuration for problem of size n:
M proposal algorithm:
All m’s get first choice, all w’s get last choice
W proposal algorithm:
All w’s get first choice, all m’s get last choice
There is a stable matching where everyone gets their second choice

34. Key ideas Formalizing real world problem
Model: graph and preference lists
Mechanism: stability condition
Specification of algorithm with a natural operation
Proposal
Establishing termination of process through invariants and progress measure
Underspecification of algorithm
Establishing uniqueness of solution

35. Richard Anderson Lecture 3
CSE 421 Algorithms
Richard Anderson
Lecture 3

36. Classroom Presenter Project
Understand how to use Pen Computing to support classroom instruction
Writing on electronic slides
Distributed presentation
Student submissions
Classroom Presenter 2.0, started January 2002
Classroom Presenter 3.0, started June 2005

37. Key ideas for Stable Matching
Formalizing real world problem
Model: graph and preference lists
Mechanism: stability condition
Specification of algorithm with a natural operation
Proposal
Establishing termination of process through invariants and progress measure
Underspecification of algorithm
Establishing uniqueness of solution

38. Question Goodness of a stable matching:
Add up the ranks of all the matched pairs
M-rank, W-rank
Suppose that the preferences are completely random
If there are n M’s, and n W’s, what is the expected value of the M-rank and the W-rank

39. What is the run time of the Stable Matching Algorithm?
Initially all m in M and w in W are free
While there is a free m
w highest on m’s list that m has not proposed to
if w is free, then match (m, w)
else
suppose (m2, w) is matched
if w prefers m to m2
unmatch (m2, w)
match (m, w)
Executed at most n2 times

40. O(1) time per iteration Find free m Find next available w
If w is matched, determine m2
Test if w prefer m to m2
Update matching

41. What does it mean for an algorithm to be efficient?

42. Definitions of efficiency
Fast in practice
Qualitatively better worst case performance than a brute force algorithm

43. Polynomial time efficiency
An algorithm is efficient if it has a polynomial run time
Run time as a function of problem size
Run time: count number of instructions executed on an underlying model of computation
T(n): maximum run time for all problems of size at most n

44. Polynomial Time
Algorithms with polynomial run time have the property that increasing the problem size by a constant factor increases the run time by at most a constant factor (depending on the algorithm)

45. Why Polynomial Time?
Generally, polynomial time seems to capture the algorithms which are efficient in practice
The class of polynomial time algorithms has many good, mathematical properties

46. Ignoring constant factors
Express run time as O(f(n))
Emphasize algorithms with slower growth rates
Fundamental idea in the study of algorithms
Basis of Tarjan/Hopcroft Turing Award

47. Why ignore constant factors?
Constant factors are arbitrary
Depend on the implementation
Depend on the details of the model
Determining the constant factors is tedious and provides little insight

48. Why emphasize growth rates?
The algorithm with the lower growth rate will be faster for all but a finite number of cases
Performance is most important for larger problem size
As memory prices continue to fall, bigger problem sizes become feasible
Improving growth rate often requires new techniques

49. Formalizing growth rates
T(n) is O(f(n)) [T : Z+  R+]
If sufficiently large n, T(n) is bounded by a constant multiple of f(n)
Exist c, n0, such that for n > n0, T(n) < c f(n)
T(n) is O(f(n)) will be written as: T(n) = O(f(n))
Be careful with this notation

50. Prove 3n2 + 5n + 20 is O(n2)
Choose c = 6, n0 = 5

51. Lower bounds T(n) is (f(n)) Warning: definitions of  vary
T(n) is at least a constant multiple of f(n)
There exists an n0, and  > 0 such that T(n) > f(n) for all n > n0
Warning: definitions of  vary
T(n) is (f(n)) if T(n) is O(f(n)) and T(n) is (f(n))

52. Useful Theorems
If lim (f(n) / g(n)) = c for c > 0 then f(n) = (g(n))
If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) is O(h(n)))
If f(n) is O(h(n)) and g(n) is O(h(n)) then f(n) + g(n) is O(h(n))

53. Ordering growth rates For b > 1 and x > 0
logb n is O(nx)
For r > 1 and d > 0
nd is O(rn)

54. Richard Anderson Lecture 4
CSE 421 Algorithms
Richard Anderson
Lecture 4

55. Announcements Homework 2, Due October 12, 1:30 pm. Reading Chapter 3
Start on Chapter 4

56. Polynomial time efficiency
An algorithm is efficient if it has a polynomial run time
Run time as a function of problem size
Run time: count number of instructions executed on an underlying model of computation
T(n): maximum run time for all problems of size at most n

57. Polynomial Time
Algorithms with polynomial run time have the property that increasing the problem size by a constant factor increases the run time by at most a constant factor (depending on the algorithm)

58. Why Polynomial Time?
Generally, polynomial time seems to capture the algorithms which are efficient in practice
The class of polynomial time algorithms has many good, mathematical properties

59. Constant factors and growth rates
Express run time as O(f(n))
Ignore constant factors
Prefer algorithms with slower growth rates
Fundamental ideas in the study of algorithms
Basis of Tarjan/Hopcroft Turing Award

60. Why ignore constant factors?
Constant factors are arbitrary
Depend on the implementation
Depend on the details of the model
Determining the constant factors is tedious and provides little insight

61. Why emphasize growth rates?
The algorithm with the lower growth rate will be faster for all but a finite number of cases
Performance is most important for larger problem size
As memory prices continue to fall, bigger problem sizes become feasible
Improving growth rate often requires new techniques

62. Formalizing growth rates
T(n) is O(f(n)) [T : Z+  R+]
If sufficiently large n, T(n) is bounded by a constant multiple of f(n)
Exist c, n0, such that for n > n0, T(n) < c f(n)
T(n) is O(f(n)) will be written as: T(n) = O(f(n))
Be careful with this notation

63. Prove 3n2 + 5n + 20 is O(n2)
Choose c = 6, n0 = 5

64. Lower bounds T(n) is (f(n)) Warning: definitions of  vary
T(n) is at least a constant multiple of f(n)
There exists an n0, and  > 0 such that T(n) > f(n) for all n > n0
Warning: definitions of  vary
T(n) is (f(n)) if T(n) is O(f(n)) and T(n) is (f(n))

65. Useful Theorems
If lim (f(n) / g(n)) = c for c > 0 then f(n) = (g(n))
If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) is O(h(n)))
If f(n) is O(h(n)) and g(n) is O(h(n)) then f(n) + g(n) is O(h(n))

66. Ordering growth rates For b > 1 and x > 0
logb n is O(nx)
For r > 1 and d > 0
nd is O(rn)

67. Graph Theory G = (V, E) Undirected graphs Directed graphs
Explain that there will be some review from 326
Graph Theory
G = (V, E)
V – vertices
E – edges
Undirected graphs
Edges sets of two vertices {u, v}
Directed graphs
Edges ordered pairs (u, v)
Many other flavors
Edge / vertices weights
Parallel edges
Self loops
By default |V| = n and |E| = m

68. Definitions Path: v1, v2, …, vk, with (vi, vi+1) in E Distance
Simple Path
Cycle
Simple Cycle
Distance
Connectivity
Undirected
Directed (strong connectivity)
Trees
Rooted
Unrooted

69. Graph search Find a path from s to t S = {s}
While there exists (u, v) in E with u in S and v not in S
Pred[v] = u
Add v to S
if (v = t) then path found

70. Breadth first search Explore vertices in layers s in layer 1
Neighbors of s in layer 2
Neighbors of layer 2 in layer

71. Key observation
All edges go between vertices on the same layer or adjacent layers 1 2 3 4 5 6 7 8

72. Bipartite
A graph V is bipartite if V can be partitioned into V1, V2 such that all edges go between V1 and V2
A graph is bipartite if it can be two colored

73. Testing Bipartiteness
If a graph contains an odd cycle, it is not bipartite

74. Algorithm Run BFS Color odd layers red, even layers blue
If no edges between the same layer, the graph is bipartite
If edge between two vertices of the same layer, then there is an odd cycle, and the graph is not bipartite

75. Richard Anderson Lecture 5 Graph Theory
CSE 421 Algorithms
Richard Anderson
Lecture 5
Graph Theory

76. Announcements Monday’s class will be held in CSE 305 Reading Chapter 3
Start on Chapter 4

77. Graph Theory G = (V, E) Undirected graphs Directed graphs
Explain that there will be some review from 326
Graph Theory
G = (V, E)
V – vertices
E – edges
Undirected graphs
Edges sets of two vertices {u, v}
Directed graphs
Edges ordered pairs (u, v)
Many other flavors
Edge / vertices weights
Parallel edges
Self loops
By default |V| = n and |E| = m

78. Definitions Path: v1, v2, …, vk, with (vi, vi+1) in E Distance
Simple Path
Cycle
Simple Cycle
Distance
Connectivity
Undirected
Directed (strong connectivity)
Trees
Rooted
Unrooted

79. Graph search Find a path from s to t S = {s}
While there exists (u, v) in E with u in S and v not in S
Pred[v] = u
Add v to S
if (v = t) then path found

80. Breadth first search Explore vertices in layers s in layer 1
Neighbors of s in layer 2
Neighbors of layer 2 in layer

81. Key observation
All edges go between vertices on the same layer or adjacent layers 1 2 3 4 5 6 7 8

82. Bipartite
A graph V is bipartite if V can be partitioned into V1, V2 such that all edges go between V1 and V2
A graph is bipartite if it can be two colored

83. Testing Bipartiteness
If a graph contains an odd cycle, it is not bipartite

84. Algorithm Run BFS Color odd layers red, even layers blue
If no edges between the same layer, the graph is bipartite
If edge between two vertices of the same layer, then there is an odd cycle, and the graph is not bipartite

85. Corollary
A graph is bipartite if and only if it has no Odd Length Cycle

86. Depth first search
Explore vertices from most recently visited

87. Recursive DFS DFS(u) Mark u as “Explored”
Foreach v in Neighborhood of u
If v is not “Explored”, DFS(v)

88. Key observation Each edge goes between vertices on the same branch1
Each edge goes between vertices on the same branch
No cross edges 2 6 4 4 7 12 5 8 9 10 11

89. Connected Components
Undirected Graphs

90. Strongly Connected Components
There is an O(n+m) algorithm that we will not be covering
Directed Graphs

91. Richard Anderson Lecture 6 Graph Theory
CSE 421 Algorithms
Richard Anderson
Lecture 6
Graph Theory

92. Draw a picture of David Notkin
To submit your drawing, press the button

93. Describe an algorithm to determine if an undirected graph has a cycle

94. Cycle finding Does a graph have a cycle? Find a cycle
Find a cycle through a specific vertex v
Linear runtime: O(n+m)

95. Find a cycle through a vertex v
Not obvious how to do this with BFS from vertex v v

96. Depth First Search Each edge goes between vertices on the same branch1
Each edge goes between vertices on the same branch
No cross edges 2 6 3 4 7 12 5 8 9 10 11

97. A DFS from vertex v gives a simple algorithm for finding a cycle containing v
How does this algorithm work and why?

98. Connected Components
Undirected Graphs

99. Directed Graphs
A Strongly Connected Component is a subset of the vertices with paths between every pair of vertices.

100. Identify the Strongly Connected Components
There is an O(n+m) algorithm that we will not be covering

101. Topological Sort
Given a set of tasks with precedence constraints, find a linear order of the tasks
321
322
401
142
143
341
326
421
370
431
378

102. Find a topological order for the following graphJ C F K B L

103. If a graph has a cycle, there is no topological sort
Consider the first vertex on the cycle in the topological sort
It must have an incoming edge A F B E C D

104. Lemma: If a graph is acyclic, it has a vertex with in degree 0
Proof:
Pick a vertex v1, if it has in-degree 0 then done
If not, let (v2, v1) be an edge, if v2 has in- degree 0 then done
If not, let (v3, v2) be an edge . . .
If this process continues for more than n steps, we have a repeated vertex, so we have a cycle

105. Topological Sort Algorithm
While there exists a vertex v with in-degree 0
Output vertex v
Delete the vertex v and all out going edges H E I A D G J C F K B L

106. Details for O(n+m) implementation
Maintain a list of vertices of in-degree 0
Each vertex keeps track of its in-degree
Update in-degrees and list when edges are removed
m edge removals at O(1) cost each

107. Richard Anderson Lecture 7 Greedy Algorithms
CSE 421 Algorithms
Richard Anderson
Lecture 7
Greedy Algorithms

108. Greedy Algorithms Solve problems with the simplest possible algorithm
The hard part: showing that something simple actually works
Pseudo-definition
An algorithm is Greedy if it builds its solution by adding elements one at a time using a simple rule

109. Scheduling Theory Tasks Processors Precedence constraints
Processing requirements, release times, deadlines
Processors
Precedence constraints
Objective function
Jobs scheduled, lateness, total execution time

110. Interval Scheduling Tasks occur at fixed time Single processor
Maximize number of tasks completed
Tasks {1, 2, N}
Start and finish times, s(i), f(i)

111. Simple heuristics Schedule earliest available task
Instructor note counter examples
Schedule shortest available task
Schedule task with fewest conflicts

112. Schedule available task with the earliest deadline
Let A be the set of tasks computed by this algorithm, and let O be an optimal set of tasks. We want to show that |A| = |O|
Let A = {i1, . . ., ik}, O = {j1, . . ., jm}, both in increasing order of finish times

113. Correctness Proof A always stays ahead of O, f(ir) <= f(jr)
Induction argument
f(i1) <= f(j1)
If f(ir-1) <= f(jr-1) then f(ir) <= f(jr)

114. Scheduling all intervals
Minimize number of processors to schedule all intervals

115. Lower bound
In any instance of the interval partitioning problem, the number of processors is at least the depth of the set of intervals

116. Algorithm Sort by start times
Suppose maximum depth is d, create d slots
Schedule items in increasing order, assign each item to an open slot
Correctness proof: When we reach an item, we always have an open slot

117. Scheduling tasks Each task has a length ti and a deadline di
All tasks are available at the start
One task may be worked on at a time
All tasks must be completed
Goal minimize maximum lateness
Lateness = fi – di if fi >= di

118. Example Show the schedule 2, 3, 4, 5 first and compute lateness6 2 3 4 4 5 5 12

119. Greedy Algorithm Earliest deadline first Order jobs by deadline
This algorithm is optimal
This result may be surprising, since it ignores the job lengths

120. Analysis
Suppose the jobs are ordered by deadlines, d1 <= d2 <= <= dn
A schedule has an inversion if job j is scheduled before i where j > i
The schedule A computed by the greedy algorithm has no inversions.
Let O be the optimal schedule, we want to show that A has the same maximum lateness as O

121. Proof Lemma: There is an optimal schedule with no idle time.
Lemma: There is an optimal schedule with no inversions and no idle time.
Let O be an optimal schedule k inversions, we construct a new optimal schedule with k-1 inversions

122. If there is an inversion, there is an inversion of adjacent jobs
Interchange argument
Suppose there is a pair of jobs i and j, with i < j, and j scheduled immediately before i. Interchanging i and j does not increase the maximum lateness. Recall, di <= dj
di dj
di dj

123. Summary Simple algorithms for scheduling problems Correctness proofs
Method 1: Identify an invariant and establish by induction that it holds
Method 2: Show that the algorithm’s solution is as good as an optimal one by converting the optimal solution to the algorithm’s solution while preserving value

124. Greedy Algorithms: Homework Scheduling and Optimal Caching
CSE 421 Algorithms
Richard Anderson
Lecture 8
Greedy Algorithms: Homework Scheduling and Optimal Caching

125. Announcements Monday, October 17 Class will meeting in CSE 305
Tablets again!
Read sections 4.4 and 4.5 before class
Lecture will be designed with the assumption that you have read the text

126. Greedy Algorithms Solve problems with the simplest possible algorithm
The hard part: showing that something simple actually works
Today’s problem
Homework Scheduling
Optimal Caching

127. Homework Scheduling Tasks to perform Deadlines on the tasks
Freedom to schedule tasks in any order

128. Scheduling tasks Each task has a length ti and a deadline di
All tasks are available at the start
One task may be worked on at a time
All tasks must be completed
Goal minimize maximum lateness
Lateness = fi – di if fi >= di

129. Example Show the schedule 2, 3, 4, 5 first and compute lateness
Deadline 6 2 3 4 4 5 5 12

130. Greedy Algorithm Earliest deadline first Order jobs by deadline
This algorithm is optimal
This result may be surprising, since it ignores the job lengths

131. Analysis
Suppose the jobs are ordered by deadlines, d1 <= d2 <= <= dn
A schedule has an inversion if job j is scheduled before i where j > i
The schedule A computed by the greedy algorithm has no inversions.
Let O be the optimal schedule, we want to show that A has the same maximum lateness as O

132. Proof Lemma: There is an optimal schedule with no idle time.
Lemma: There is an optimal schedule with no inversions and no idle time.
Let O be an optimal schedule k inversions, we construct a new optimal schedule with k-1 inversions

133. If there is an inversion, there is an inversion of adjacent jobs
Interchange argument
Suppose there is a pair of jobs i and j, with i < j, and j scheduled immediately before i. Interchanging i and j does not increase the maximum lateness. Recall, di <= dj
di dj
di dj

134. Result
Earliest Deadline First algorithm constructs a schedule that minimizes the maximum lateness

135. Extensions
What if the objective is to minimize the sum of the lateness?
EDF does not seem to work
If the tasks have release times and deadlines, and are non-preemptable, the problem is NP-complete
What about the case with release times and deadlines where tasks are preemptable?

136. Optimal Caching Caching problem:
Maintain collection of items in local memory
Minimize number of items fetched

137. Caching example
A, B, C, D, A, E, B, A, D, A, C, B, D, A

138. Optimal Caching
If you know the sequence of requests, what is the optimal replacement pattern?
Note – it is rare to know what the requests are in advance – but we still might want to do this:
Some specific applications, the sequence is known
Competitive analysis, compare performance on an online algorithm with an optimal offline algorithm

139. Farthest in the future algorithm
Discard element used farthest in the future
A, B, C, A, C, D, C, B, C, A, D

140. Correctness Proof Sketch Start with Optimal Solution O
Convert to Farthest in the Future Solution F-F
Look at the first place where they differ
Convert O to evict F-F element
There are some technicalities here to ensure the caches have the same configuration . . .

141. Richard Anderson Lecture 9 Dijkstra’s algorithm
CSE 421 Algorithms
Richard Anderson
Lecture 9
Dijkstra’s algorithm

142. Who was Dijkstra?
What were his major contributions?

143. Edsger Wybe Dijkstra

144. Single Source Shortest Path Problem
Given a graph and a start vertex s
Determine distance of every vertex from s
Identify shortest paths to each vertex
Express concisely as a “shortest paths tree”
Each vertex has a pointer to a predecessor on shortest path 1 u u 1 2 3 5 s x s x 4 3 3 v v

145. Construct Shortest Path Tree from s2 d d 1 a a 5 4 4 4 e e -3 c c s -2 s 3 3 2 6 g g b b 3 7 f f

146. Warmup
If P is a shortest path from s to v, and if t is on the path P, the segment from s to t is a shortest path between s and t
WHY? (submit an answer) v t s

147. Careful Proof Suppose s-v is a shortest path
Suppose s-t is not a shortest path
Therefore s-v is not a shortest path
Therefore s-t is a shortest path

148. Prove if s-t not a shortest path then s-v is not a shortest path

149. Dijkstra’s Algorithm S = {}; d[s] = 0; d[v] = inf for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], d[v] + c(v, w) 4 3 y 1 1 u 1 1 4 s x 2 2 2 2 v 2 3 5 z

150. Simulate Dijkstra’s algorithm (strarting from s) on the graph
Vertex Added 1
Round s a b c d c a 1 1 2 3 4 5 3 2 1 s 4 4 6 1 b d 3

151. Dijkstra’s Algorithm as a greedy algorithm
Elements committed to the solution by order of minimum distance

152. Correctness Proof Elements in S have the correct label
Key to proof: when v is added to S, it has the correct distance label. y x s u v

153. Proof Let Pv be the path of length d[v], with an edge (u,v)
Let P be some other path to v. Suppose P first leaves S on the edge (x, y)
P = Psx + c(x,y) + Pyv
Len(Psx) + c(x,y) >= d[y]
Len(Pyv) >= 0
Len(P) >= d[y] + 0 >= d[v] y x s u v

154. Negative Cost Edges
Draw a small example a negative cost edge and show that Dijkstra’s algorithm fails on this example

155. Bottleneck Shortest Path
Define the bottleneck distance for a path to be the maximum cost edge along the path u 6 5 s x 2 5 4 3 v

156. Compute the bottleneck shortest paths6 d d 6 a a 5 4 4 4 e e -3 c c s -2 s 3 3 2 6 g g b b 4 7 f f

157. How do you adapt Dijkstra’s algorithm to handle bottleneck distances
Does the correctness proof still apply?

158. Richard Anderson Lecture 10 Minimum Spanning Trees
CSE 421 Algorithms
Richard Anderson
Lecture 10
Minimum Spanning Trees

159. Announcements Homework 3 is due now
Homework 4, due 10/26, available now
Reading
Chapter 5
(Sections 4.8, 4.9 will not be covered in class)
Guest lecturers (10/28 – 11/4)
Anna Karlin
Venkat Guruswami

160. Shortest Paths Negative Cost Edges
Dijkstra’s algorithm assumes positive cost edges
For some applications, negative cost edges make sense
Shortest path not well defined if a graph has a negative cost cycle a 6 4 -4 4 e -3 c s -2 3 3 2 6 g b f 4 7

161. Negative Cost Edge Preview
Topological Sort can be used for solving the shortest path problem in directed acyclic graphs
Bellman-Ford algorithm finds shortest paths in a graph with negative cost edges (or reports the existence of a negative cost cycle).

162. Dijkstra’s Algorithm Implementation and Runtime
S = {}; d[s] = 0; d[v] = inf for v != s
While S != V
Choose v in V-S with minimum d[v]
Add v to S
For each w in the neighborhood of v
d[w] = min(d[w], d[v] + c(v, w)) y u a
HEAP OPERATIONS
n Extract Min
m Heap Update s x v b z
Edge costs are assumed to be non-negative

163. Bottleneck Shortest Path
Define the bottleneck distance for a path to be the maximum cost edge along the path u 6 5 s x 2 5 4 3 v

164. Compute the bottleneck shortest paths6 d d 6 a a 5 4 4 4 e e -3 c c s -2 s 3 3 2 6 g g b b 7 7 f f

165. How do you adapt Dijkstra’s algorithm to handle bottleneck distances
Does the correctness proof still apply?

166. Minimum Spanning Tree 15 t a 6 9 14 3 4 e 10 13 c s 11 20 5 17 2 7 g b8 f 22 u 12 1 16 v

167. Greedy Algorithm 1 Prim’s Algorithm
Extend a tree by including the cheapest out going edge 15 t a 6 9 14 3 4 e 10 13 c s 11 20 5 17 2 7 g b 8 f 22 u 12 1 16 v

168. Greedy Algorithm 2 Kruskal’s Algorithm
Add the cheapest edge that joins disjoint components 15 t a 6 9 14 3 4 e 10 13 c s 11 20 5 17 2 7 g b 8 f 22 u 12 1 16 v

169. Greedy Algorithm 3 Reverse-Delete
Delete the most expensive edge that does not disconnect the graph 15 t a 6 9 14 3 4 e 10 13 c s 11 20 5 17 2 7 g b 8 f 22 u 12 1 16 v

170. Why do the greedy algorithms work?
For simplicity, assume all edge costs are distinct
Let S be a subset of V, and suppose e = (u, v) be the minimum cost edge of E, with u in S and v in V-S
e is in every minimum spanning tree

171. Proof Suppose T is a spanning tree that does not contain e
Add e to T, this creates a cycle
The cycle must have some edge e1 = (u1, v1) with u1 in S and v1 in V-S
T1 = T – {e1} + {e} is a spanning tree with lower cost
Hence, T is not a minimum spanning tree

172. Optimality Proofs Prim’s Algorithm computes a MST
Kruskal’s Algorithm computes a MST

173. Reverse-Delete Algorithm
Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

174. Dealing with the assumption
Force the edge weights to be distinct
Add small quantities to the weights
Give a tie breaking rule for equal weight edges

175. Richard Anderson Lecture 11 Minimum Spanning Trees
CSE 421 Algorithms
Richard Anderson
Lecture 11
Minimum Spanning Trees

176. Announcements
Monday – Class in EE1 003 (no tablets)

177. Foreign Exchange Arbitrage
USD
1.2
1.2
USD
EUR
CAD
------
0.8
1.2
1.6
0.6
-----
EUR
CAD
0.6
USD
0.8
0.8
EUR
CAD
1.6

178. Minimum Spanning Tree 15 t a 6 9 14 3 4 e 10 13 c s 11 20 5 17 2 7 g b8 f 22 u 12 1 16 v
Temporary Assumption: Edge costs distinct

179. Why do the greedy algorithms work?
For simplicity, assume all edge costs are distinct
Let S be a subset of V, and suppose e = (u, v) be the minimum cost edge of E, with u in S and v in V-S
e is in every minimum spanning tree
Or equivalently, if e is not in T, then T is not a minimum spanning tree e S
V - S

180. Proof Suppose T is a spanning tree that does not contain e
Add e to T, this creates a cycle
The cycle must have some edge e1 = (u1, v1) with u1 in S and v1 in V-S
T1 = T – {e1} + {e} is a spanning tree with lower cost
Hence, T is not a minimum spanning tree S
V - S e
Why is e lower cost than e1?

181. Optimality Proofs Prim’s Algorithm computes a MST
Kruskal’s Algorithm computes a MST

182. Reverse-Delete Algorithm
Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

183. Dealing with the distinct cost assumption
Force the edge weights to be distinct
Add small quantities to the weights
Give a tie breaking rule for equal weight edges

184. MST Fun Facts
The minimum spanning tree is determined only by the order of the edges – not by their magnitude
Finding a maximum spanning tree is just like finding a minimum spanning tree

185. Divide and Conquer Array Mergesort(Array a){ n = a.Length;
if (n <= 1)
return a;
b = Mergesort(a[0..n/2]);
c = Mergesort(a[n/ n-1]);
return Merge(b, c);

186. Algorithm Analysis
Cost of Merge
Cost of Mergesort

187. T(n) = 2T(n/2) + cn; T(1) = c;

188. Recurrence Analysis Solution methods Unrolling recurrence
Guess and verify
Plugging in to a “Master Theorem”

189. A better mergesort (?) Divide into 3 subarrays and recursively sort
Apply 3-way merge

190. T(n) = aT(n/b) + f(n)

191. T(n) = T(n/2) + cn

192. T(n) = 4T(n/2) + cn

193. T(n) = 2T(n/2) + n2

194. T(n) = 2T(n/2) + n1/2

195. Recurrences Three basic behaviors Dominated by initial case
Dominated by base case
All cases equal – we care about the depth

196. Richard Anderson Lecture 12 Recurrences
CSE 421 Algorithms
Richard Anderson
Lecture 12
Recurrences

197. Announcements
Wednesday class will meet in CSE 305.

198. Divide and Conquer Array Mergesort(Array a){ n = a.Length;
if (n <= 1)
return a;
b = Mergesort(a[0..n/2]);
c = Mergesort(a[n/ n-1]);
return Merge(b, c);

199. Algorithm Analysis
Cost of Merge
Cost of Mergesort

200. T(n) = 2T(n/2) + cn; T(1) = c;

201. Recurrence Analysis Solution methods Unrolling recurrence
Guess and verify
Plugging in to a “Master Theorem”

203. A better mergesort (?) Divide into 3 subarrays and recursively sort
Apply 3-way merge

204. T(n) = aT(n/b) + f(n)

205. T(n) = T(n/2) + cn

206. T(n) = 4T(n/2) + cn

207. T(n) = 2T(n/2) + n2

208. T(n) = 2T(n/2) + n1/2

209. Recurrences Three basic behaviors Dominated by initial case
Dominated by base case
All cases equal – we care about the depth

210. Richard Anderson Lecture 13 Divide and Conquer
CSE 421 Algorithms
Richard Anderson
Lecture 13
Divide and Conquer

211. Announcements Guest Lecturers Homework 5 and Homework 6 are available
Anna Karlin (10/31, 11/2)
Venkat Guruswami (10/28, 11/4)
Homework 5 and Homework 6 are available
I’m going to try to be clear when student submissions are expected
Instructor Example
Student Submission

212. What is the solution to:
Student Submission
What is the solution to:
What are the asymptotic bounds for x < 1 and x > 1?

213. Solve by unrolling T(n) = n + 3T(n/4)
Instructor Example

214. Solve by unrolling T(n) = n + 5T(n/2)
Student Submission
Answer: nlog(5/2)

215. A non-linear additive term
Instructor Example
T(n) = n2 + 3T(n/2)

216. What you really need to know about recurrences
Work per level changes geometrically with the level
Geometrically increasing (x > 1)
The bottom level wins
Geometrically decreasing (x < 1)
The top level wins
Balanced (x = 1)
Equal contribution

217. Classify the following recurrences (Increasing, Decreasing, Balanced)
Student Submission
T(n) = n + 5T(n/8)
T(n) = n + 9T(n/8)
T(n) = n2 + 4T(n/2)
T(n) = n3 + 7T(n/2)
T(n) = n1/2 + 3T(n/4)

218. Divide and Conquer Algorithms
Split into sub problems
Recursively solve the problem
Combine solutions
Make progress in the split and combine stages
Quicksort – progress made at the split step
Mergesort – progress made at the combine step

219. Closest Pair Problem
Given a set of points find the pair of points p, q that minimizes dist(p, q)

220. Divide and conquer
If we solve the problem on two subsets, does it help? (Separate by median x coordinate) 1 2

221. Student Submission
Packing Lemma
Suppose that the minimum distance between points is at least , what is the maximum number of points that can be packed in a ball of radius ?

222. Combining Solutions
Suppose the minimum separation from the sub problems is 
In looking for cross set closest pairs, we only need to consider points with  of the boundary
How many cross border interactions do we need to test?

223. A packing lemma bounds the number of distances to check

224. Details Preprocessing: sort points by y Merge step
Select points in boundary zone
For each point in the boundary
Find highest point on the other side that is at most  above
Find lowest point on the other side that is at most  below
Compare with the points in this interval (there are at most 6)

225. Identify the pairs of points that are compared in the merge step following the recursive calls
Student Submission

226. Algorithm run time
After preprocessing:
T(n) = cn + 2 T(n/2)

227. Counting Inversions Count inversions on lower half11 12 4 1 7 2 3 15 9 5 16 8 6 13 10 14
Count inversions on lower half
Count inversions on upper half
Count the inversions between the halves

228. Count the Inversions 11 12 4 1 7 2 3 15 9 5 16 8 6 13 10 14 9 5 16 8 6 13 10 14 11 12 4 1 7 2 3 15 11 12 4 1 7 2 3 15 9 5 16 8 6 13 10 14

229. Problem – how do we count inversions between sub problems in O(n) time?
Solution – Count inversions while merging 1 2 3 4 7 11 12 15 5 6 8 9 10 13 14 16
Standard merge algorithms – add to inversion count when an element is moved from the upper array to the solution
Instructor Example

230. Use the merge algorithm to count inversions1 4 11 12 2 3 7 15 5 8 9 16 6 10 13 14
Student Submission

231. Richard Anderson Lecture 18 Dynamic Programming
CSE 421 Algorithms
Richard Anderson
Lecture 18
Dynamic Programming

234. Announcements
Wednesday class will meet in CSE 305.

235. Dynamic Programming
The most important algorithmic technique covered in CSE 421
Key ideas
Express solution in terms of a polynomial number of sub problems
Order sub problems to avoid recomputation

236. Today - Examples Examples Optimal Billboard Placement
Text, Solved Exercise, Pg 307
Linebreaking with hyphenation
Compare with HW problem 6, Pg 317
String concatenation
Text, Solved Exercise, Page 309

237. Billboard Placement Maximize income in placing billboards Constraint:
(pi, vi), vi: value of placing billboard at position pi
Constraint:
At most one billboard every five miles
Example
{(6,5), (8,6), (12, 5), (14, 1)}

238. Opt[k]
What are the sub problems?

239. Opt[k] = fun(Opt[0],…,Opt[k-1])
How is the solution determined from sub problems?

240. Solution j = 0; // j is five miles behind the current position
// the last valid location for a billboard, if one placed at P[k]
for k := 1 to n
while (P[j] < P[k] – 5)
j := j + 1;
j := j – 1;
Opt[k] = Max(Opt[k-1], V[k] + Opt[j]);

241. Optimal line breaking and hyphen-ation
Problem: break lines and insert hyphens to make lines as balanced as possible
Typographical considerations:
Avoid excessive white space
Limit number of hyphens
Avoid widows and orphans
Etc.

242. Penalty Function
Pen(i, j) – penalty of starting a line a position i, and ending at position j
Key technical idea
Number the breaks between words/syllables
Opt-i-mal line break-ing and hyph-en-a-tion is com-put-ed with dy-nam-ic pro-gram-ming

243. Opt[k]
What are the sub problems?

244. Opt[k] = fun(Opt[0],…,Opt[k-1])
How is the solution determined from sub problems?

245. Solution for k := 1 to n Opt[k] := infinity; for j := 0 to k-1
Opt[k] := Min(Opt[k], Opt[j] + Pen(j, k));

246. But what if you want to layout the text?
And not just know the minimum penalty?

247. Solution for k := 1 to n Opt[k] := infinity; for j := 0 to k-1
temp := Opt[j] + Pen(j, k);
if (temp < Opt[k])
Opt[k] = temp;
Best[k] := j;

248. String approximation
Given a string S, and a library of strings B = {b1, …bk}, construct an approximation of the string S by using copies of strings in B.
B = {abab, bbbaaa, ccbb, ccaacc}
S = abaccbbbaabbccbbccaabab

249. Formal Model
Strings from B assigned to non- overlapping positions of s
Strings from B may be used multiple times
Cost of  for unmatched character in s
Cost of  for mismatched character in s
MisMatch(i, j) – number of mismatched characters of bj, when aligned starting with position i in s.

250. Opt[k]
What are the sub problems?

251. Opt[k] = fun(Opt[0],…,Opt[k-1])
How is the solution determined from sub problems?

252. Solution for i := 1 to n Opt[k] = Opt[k-1] + ; for j := 1 to |B|
p = i – len(bj);
Opt[k] = min(Opt[k], Opt[p-1] +  MisMatch(p, j));

253. Richard Anderson Lecture 19 Longest Common Subsequence
CSE 421 Algorithms
Richard Anderson
Lecture 19
Longest Common Subsequence

254. Longest Common Subsequence
C=c1…cg is a subsequence of A=a1…am if C can be obtained by removing elements from A (but retaining order)
LCS(A, B): A maximum length sequence that is a subsequence of both A and B
ocurranec
occurrence
attacggct
tacgacca
Instructor Example

255. Determine the LCS of the following strings
BARTHOLEMEWSIMPSON
KRUSTYTHECLOWN
Student Submission

256. String Alignment Problem
Align sequences with gaps
Charge x if character x is unmatched
Charge xy if character x is matched to character y
CAT TGA AT
CAGAT AGGA

257. LCS Optimization A = a1a2…am B = b1b2…bn
Opt[j, k] is the length of LCS(a1a2…aj, b1b2…bk)

258. Optimization recurrence
If aj = bk, Opt[j,k] = 1 + Opt[j-1, k-1]
If aj != bk, Opt[j,k] = max(Opt[j-1,k], Opt[j,k-1])

259. Give the Optimization Recurrence for the String Alignment Problem
Charge x if character x is unmatched
Charge xy if character x is matched to character y
Student Submission

260. Dynamic Programming Computation

261. Write the code to compute Opt[j,k]
Student Submission

262. Storing the path information
A[1..m], B[1..n]
for i := 1 to m Opt[i, 0] := 0;
for j := 1 to n Opt[0,j] := 0;
Opt[0,0] := 0;
for i := 1 to m
for j := 1 to n
if A[i] = B[j] { Opt[i,j] := 1 + Opt[i-1,j-1]; Best[i,j] := Diag; }
else if Opt[i-1, j] >= Opt[i, j-1]
{ Opt[i, j] := Opt[i-1, j], Best[i,j] := Left; }
else { Opt[i, j] := Opt[i, j-1], Best[i,j] := Down; }
b1…bn
a1…am

263. How good is this algorithm?
Is it feasible to compute the LCS of two strings of length 100,000 on a standard desktop PC? Why or why not.
Student Submission

264. Observations about the Algorithm
The computation can be done in O(m+n) space if we only need one column of the Opt values or Best Values
The algorithm can be run from either end of the strings

265. Divide and Conquer Algorithm
Where does the best path cross the middle column?
For a fixed i, and for each j, compute the LCS that has ai matched with bj

266. Constrained LCS LCSi,j(A,B): The LCS such that LCS4,3(abbacbb, cbbaa)
a1,…,ai paired with elements of b1,…,bj
ai+1,…am paired with elements of bj+1,…,bn
LCS4,3(abbacbb, cbbaa)

267. A = RRSSRTTRTS B=RTSRRSTST
Compute LCS5,1(A,B), LCS5,2(A,B),…,LCS5,9(A,B)
Student Submission

268. A = RRSSRTTRTS B=RTSRRSTST
Compute LCS5,1(A,B), LCS5,2(A,B),…,LCS5,9(A,B) j
left
right 3 1 2 4 5 6 7 8 9
Instructor Example

269. Computing the middle column
From the left, compute LCS(a1…am/2,b1…bj)
From the right, compute LCS(am/2+1…am,bj+1…bn)
Add values for corresponding j’s
Note – this is space efficient

270. Divide and Conquer A = a1,…,am B = b1,…,bn Find j such that Recurse
LCS(a1…am/2, b1…bj) and
LCS(am/2+1…am,bj+1…bn) yield optimal solution
Recurse

271. Algorithm Analysis
T(m,n) = T(m/2, j) + T(m/2, n-j) + cnm

272. Prove by induction that T(m,n) <= 2cmn
Instructor Example

273. Richard Anderson Lecture 20 Space Efficient LCS
CSE 421 Algorithms
Richard Anderson
Lecture 20
Space Efficient LCS

274. Longest Common Subsequence
C=c1…cg is a subsequence of A=a1…am if C can be obtained by removing elements from A (but retaining order)
LCS(A, B): A maximum length sequence that is a subsequence of both A and B
Wednesday’s Result:
O(mn) time, O(mn) space LCS Algorithm
Today’s Result:
O(mn) time, O(m+n) space LCS Algorithm

275. Digression: String Alignment Problem
Align sequences with gaps
Charge x if character x is unmatched
Charge xy if character x is matched to character y
Find alignment to minimize sum of costs
CAT TGA AT
CAGAT AGGA

276. Optimization Recurrence for the String Alignment Problem
Charge x if character x is unmatched
Charge xy if character x is matched to character y
A = a1a2…am; B = b1b2…bn
Opt[j, k] is the value of the minimum cost alignment a1a2…aj and b1b2…bk

277. Dynamic Programming Computation

278. Storing the path information
A[1..m], B[1..n]
for i := 1 to m Opt[i, 0] := 0;
for j := 1 to n Opt[0,j] := 0;
Opt[0,0] := 0;
for i := 1 to m
for j := 1 to n
if A[i] = B[j] { Opt[i,j] := 1 + Opt[i-1,j-1]; Best[i,j] := Diag; }
else if Opt[i-1, j] >= Opt[i, j-1]
{ Opt[i, j] := Opt[i-1, j], Best[i,j] := Left; }
else { Opt[i, j] := Opt[i, j-1], Best[i,j] := Down; }
b1…bn
a1…am

279. Observations about the Algorithm
The computation can be done in O(m+n) space if we only need one column of the Opt values or Best Values
The algorithm can be run from either end of the strings

280. Divide and Conquer Algorithm
Where does the best path cross the middle column?
For a fixed i, and for each j, compute the LCS that has ai matched with bj

281. Constrained LCS LCSi,j(A,B): The LCS such that LCS4,3(abbacbb, cbbaa)
a1,…,ai paired with elements of b1,…,bj
ai+1,…am paired with elements of bj+1,…,bn
LCS4,3(abbacbb, cbbaa)

282. A = RRSSRTTRTS B=RTSRRSTST
Compute LCS5,0(A,B), LCS5,1(A,B), LCS5,2(A,B),…,LCS5,9(A,B)

283. A = RRSSRTTRTS B=RTSRRSTST
Compute LCS5,0(A,B), LCS5,1(A,B), LCS5,2(A,B),…,LCS5,9(A,B) j
left
right 4 1 2 3 5 6 7 8 9

284. Computing the middle column
From the left, compute LCS(a1…am/2,b1…bj)
From the right, compute LCS(am/2+1…am,bj+1…bn)
Add values for corresponding j’s
Note – this is space efficient

285. Divide and Conquer A = a1,…,am B = b1,…,bn Find j such that Recurse
LCS(a1…am/2, b1…bj) and
LCS(am/2+1…am,bj+1…bn) yield optimal solution
Recurse

286. Algorithm Analysis
T(m,n) = T(m/2, j) + T(m/2, n-j) + cnm

287. Prove by induction that T(m,n) <= 2cmn

288. Shortest Path Problem
Dijkstra’s Single Source Shortest Paths Algorithm
O(mlog n) time, positive cost edges
General case – handling negative edges
If there exists a negative cost cycle, the shortest path is not defined
Bellman-Ford Algorithm
O(mn) time for graphs with negative cost edges

289. Lemma
If a graph has no negative cost cycles, then the shortest paths are simple paths
Shortest paths have at most n-1 edges

290. Shortest paths with a fixed number of edges
Find the shortest path from v to w with exactly k edges

291. Express as a recurrence
Optk(w) = minx [Optk-1(x) + cxw]
Opt0(w) = 0 if v=w and infinity otherwise

292. Algorithm, Version 1 foreach w M[0, w] = infinity; M[0, v] = 0;
for i = 1 to n-1
M[i, w] = minx(M[i-1,x] + cost[x,w]);

293. Algorithm, Version 2 foreach w M[0, w] = infinity; M[0, v] = 0;
for i = 1 to n-1
M[i, w] = min(M[i-1, w], minx(M[i-1,x] + cost[x,w]))

294. Algorithm, Version 3 foreach w M[w] = infinity; M[v] = 0;
for i = 1 to n-1
M[w] = min(M[w], minx(M[x] + cost[x,w]))

295. Correctness Proof for Algorithm 3
Key lemma – at the end of iteration i, for all w, M[w] <= M[i, w];
Reconstructing the path:
Set P[w] = x, whenever M[w] is updated from vertex x

296. If the pointer graph has a cycle, then the graph has a negative cost cycle
If P[w] = x then M[w] >= M[x] + cost(x,w)
Equal after update, then M[x] could be reduced
Let v1, v2,…vk be a cycle in the pointer graph with (vk,v1) the last edge added
Just before the update
M[vj] >= M[vj+1] + cost(vj+1, vj) for j < k
M[vk] > M[v1] + cost(v1, vk)
Adding everything up
0 > cost(v1,v2) + cost(v2,v3) + … + cost(vk, v1) v1 v4 v2 v3

297. Negative Cycles
If the pointer graph has a cycle, then the graph has a negative cycle
Therefore: if the graph has no negative cycles, then the pointer graph has no negative cycles

298. Finding negative cost cycles
What if you want to find negative cost cycles?

299. Richard Anderson Lecture 21 Shortest Path Network Flow Introduction
CSE 421 Algorithms
Richard Anderson
Lecture 21
Shortest Path
Network Flow Introduction

300. Announcements Friday, 11/18, Class will meet in CSE 305
Reading ,
Section 7.4 will not be covered

301. Find the shortest paths from v with exactly k edges7 v y 5 1 -2 -2 3 1 x 3 z

302. Express as a recurrence
Optk(w) = minx [Optk-1(x) + cxw]
Opt0(w) = 0 if v=w and infinity otherwise

303. Algorithm, Version 1 foreach w M[0, w] = infinity; M[0, v] = 0;
for i = 1 to n-1
M[i, w] = minx(M[i-1,x] + cost[x,w]);

304. Algorithm, Version 2 foreach w M[0, w] = infinity; M[0, v] = 0;
for i = 1 to n-1
M[i, w] = min(M[i-1, w], minx(M[i-1,x] + cost[x,w]))

305. Algorithm, Version 3 foreach w M[w] = infinity; M[v] = 0;
for i = 1 to n-1
M[w] = min(M[w], minx(M[x] + cost[x,w]))

306. Algorithm 2 vs Algorithm 3x y z 7 v y 5 1 -2 -2 3 1 x 3 z i v x y z

307. Correctness Proof for Algorithm 3
Key lemma – at the end of iteration i, for all w, M[w] <= M[i, w];
Reconstructing the path:
Set P[w] = x, whenever M[w] is updated from vertex x 7 v y 5 1 -2 -2 3 1 x 3 z

308. Negative Cost Cycle example7 i v x y z v y 3 1 -2 -2 3 3 x -2 z

309. If the pointer graph has a cycle, then the graph has a negative cost cycle
If P[w] = x then M[w] >= M[x] + cost(x,w)
Equal after update, then M[x] could be reduced
Let v1, v2,…vk be a cycle in the pointer graph with (vk,v1) the last edge added
Just before the update
M[vj] >= M[vj+1] + cost(vj+1, vj) for j < k
M[vk] > M[v1] + cost(v1, vk)
Adding everything up
0 > cost(v1,v2) + cost(v2,v3) + … + cost(vk, v1) v1 v4 v2 v3

310. Negative Cycles
If the pointer graph has a cycle, then the graph has a negative cycle
Therefore: if the graph has no negative cycles, then the pointer graph has no negative cycles

311. Finding negative cost cycles
What if you want to find negative cost cycles?

312. Network Flow

313. Network Flow Definitions
Capacity
Source, Sink
Capacity Condition
Conservation Condition
Value of a flow

314. Flow Example u 20 10 30 s t 10 20 v

315. Residual Graph u u 15/20 0/10 5 10 15 15/30 s t 15 15 s t 5 5/10 20/20v v

316. Richard Anderson Lecture 22 Network Flow
CSE 421 Algorithms
Richard Anderson
Lecture 22
Network Flow

317. Outline Network flow definitions Flow examples Augmenting Paths
Residual Graph
Ford Fulkerson Algorithm
Cuts
Maxflow-MinCut Theorem

318. Network Flow Definitions
Flowgraph: Directed graph with distinguished vertices s (source) and t (sink)
Capacities on the edges, c(e) >= 0
Problem, assign flows f(e) to the edges such that:
0 <= f(e) <= c(e)
Flow is conserved at vertices other than s and t
Flow conservation: flow going into a vertex equals the flow going out
The flow leaving the source is a large as possible

319. Flow Example a d g s b e h t c f i 20 20 5 20 5 5 10 5 5 20 5 20 20 30

320. Find a maximum flow a d g s b e h t c f i Student Submission 20 20 530 20 30 s b e h t 20 5 5 10 20 5 25 20 c f i 20 10
Student Submission

321. Find a maximum flow a d g s b e h t c f i 20 20 20 20 20 5 20 5 5 20 525 30 20 20 30 s b e h t 20 20 30 5 20 5 5 15 10 15 20 5 25 5 20 c f i 20 20 10 10
Discussion slide

322. Augmenting Path Algorithm
Vertices v1,v2,…,vk
v1 = s, vk = t
Possible to add b units of flow between vj and vj+1 for j = 1 … k-1 u
10/20
0/10
10/30 s t
5/10
15/20 v

323. Find two augmenting paths
2/5
2/2
0/1
2/4
3/4
3/4
3/3
3/3
1/3
1/3 s t
3/3
3/3
2/2
1/3
3/3
1/3
2/2
1/3
Student Submission

324. Residual Graph Flow graph showing the remaining capacity
Flow graph G, Residual Graph GR
G: edge e from u to v with capacity c and flow f
GR: edge e’ from u to v with capacity c – f
GR: edge e’’ from v to u with capacity f

325. Residual Graph u u 15/20 0/10 5 10 15 15/30 s t 15 15 s t 5 5/10 20/20v v

326. Build the residual graph
3/5 d g
2/4
2/2
1/5
1/5 s
1/1 t
3/3
2/5 e h
1/1
Student Submission

327. Augmenting Path Lemma
Let P = v1, v2, …, vk be a path from s to t with minimum capacity b in the residual graph.
b units of flow can be added along the path P in the flow graph. u
15/20
0/10
15/30 s t
5/10
20/20 v

328. Richard Anderson Lecture 23 Network Flow
CSE 421 Algorithms
Richard Anderson
Lecture 23
Network Flow

329. Review Network flow definitions Flow examples Augmenting Paths
Residual Graph
Ford Fulkerson Algorithm
Cuts
Maxflow-MinCut Theorem

330. Network Flow Definitions
Flowgraph: Directed graph with distinguished vertices s (source) and t (sink)
Capacities on the edges, c(e) >= 0
Problem, assign flows f(e) to the edges such that:
0 <= f(e) <= c(e)
Flow is conserved at vertices other than s and t
Flow conservation: flow going into a vertex equals the flow going out
The flow leaving the source is a large as possible

331. Find a maximum flow a d g s b e h t c f i 20/20 20/20 5 20/20 5 5
5/5 5
5/5 20
25/30
20/20
30/30 s b e h t
20/20 5
5/5 10
20/20 5
15/20
15/25 c f i
20/20
10/10

332. Residual Graph Flow graph showing the remaining capacity
Flow graph G, Residual Graph GR
G: edge e from u to v with capacity c and flow f
GR: edge e’ from u to v with capacity c – f
GR: edge e’’ from v to u with capacity f

333. Augmenting Path Lemma
Let P = v1, v2, …, vk be a path from s to t with minimum capacity b in the residual graph.
b units of flow can be added along the path P in the flow graph. u u 5 10
15/20
0/10 15 15 15 s t
15/30 s t 5 5 20
5/10
20/20 v v

334. Proof Add b units of flow along the path P
What do we need to verify to show we have a valid flow after we do this?
Student Submission

335. Ford-Fulkerson Algorithm (1956)
while not done
Construct residual graph GR
Find an s-t path P in GR with capacity b > 0
Add b units along in G
If the sum of the capacities of edges leaving S is at most C, then the algorithm takes at most C iterations

336. Cuts in a graph
Cut: Partition of V into disjoint sets S, T with s in S and t in T.
Cap(S,T) – sum of the capacities of edges from S to T
Flow(S,T) – net flow out of S
Sum of flows out of S minus sum of flows into S
Flow(S,T) <= Cap(S,T)

337. What is Cap(S,T) and Flow(S,T)
S={s, a, b, e, h}, T = {c, f, i, d, g, t}
20/20
20/20 a d g 5
20/20 5 5
20/20
5/5 5
5/5 20
25/30
20/20
30/30 s b e h t
20/20 5
5/5 10
20/20 5
15/20
15/25 c f i
20/20
10/10
Student Submission

338. Minimum value cut u 10 40 10 s t 10 40 v

339. Find a minimum value cut6 6 5 8 10 3 6 2 t s 7 4 5 3 8 5 4
Student Submission

340. Find a minimum value cut6 6 5 8 10 3 6 2 t s 7 4 5 3 8 5 4
Duplicate slide
For discussion

341. MaxFlow – MinCut Theorem
There exists a flow which has the same value of the minimum cut
Proof: Consider a flow where the residual graph has no s-t path with positive capacity
Let S be the set of vertices in GR reachable from s with paths of positive capacity s t

342. Let S be the set of vertices in GR reachable from s with paths of positive capacityu v t T S
What is Cap(u,v) in GR?
What can you say about Cap(u,v) and Flow(u,v) in G?
What can you say about Cap(v,u) and Flow(v,u) in G?
Student Submission

343. Max Flow - Min Cut Theorem
Ford-Fulkerson algorithm finds a flow where the residual graph is disconnected, hence FF finds a maximum flow.
If we want to find a minimum cut, we begin by looking for a maximum flow.

344. Performance
The worst case performance of the Ford- Fulkerson algorithm is horrible u
1000
1000 1 s t
1000
1000 v

345. Better methods of finding augmenting paths
Find the maximum capacity augmenting path
O(m2log(C)) time
Find the shortest augmenting path
O(m2n)
Find a blocking flow in the residual graph
O(mnlog n)

346. Richard Anderson Lecture 24 Maxflow MinCut Theorem
CSE 421 Algorithms
Richard Anderson
Lecture 24
Maxflow MinCut Theorem

347. Find a minimum value cut
5/6
5/6
5/5 8
3/10
3/3
5/6 t s
6/7
2/2
4/4 5
3/3
3/8 5
3/4

348. MaxFlow – MinCut Theorem
There exists a flow which has the same value of the minimum cut
Proof: Consider a flow where the residual graph has no s-t path with positive capacity
Let S be the set of vertices in GR reachable from s with paths of positive capacity s t

349. Let S be the set of vertices in GR reachable from s with paths of positive capacityu v t T S
What is Cap(u,v) in GR?
What can you say about Cap(u,v) and Flow(u,v) in G?
What can you say about Cap(v,u) and Flow(v,u) in G?

350. Max Flow - Min Cut Theorem
Ford-Fulkerson algorithm finds a flow where the residual graph is disconnected, hence FF finds a maximum flow.
If we want to find a minimum cut, we begin by looking for a maximum flow.

351. Performance
The worst case performance of the Ford- Fulkerson algorithm is horrible u
1000
1000 1 s t
1000
1000 v

352. Better methods of finding augmenting paths
Find the maximum capacity augmenting path
O(m2log(C)) time
Find the shortest augmenting path
O(m2n)
Find a blocking flow in the residual graph
O(mnlog n)

353. Bipartite Matching
A graph G=(V,E) is bipartite if the vertices can be partitioned into disjoints sets X,Y
A matching M is a subset of the edges that does not share any vertices
Find a matching as large as possible

354. Application A collection of teachers A collection of courses
And a graph showing which teachers can teach which course RA
303 PB
321 CC
326 DG
401 AK
421

355. Converting Matching to Network Flows t

356. Finding edge disjoint paths

357. Theorem
The maximum number of edge disjoint paths equals the minimum number of edges whose removal separates s from t

358. Richard Anderson Lecture 25 Network Flow Applications
CSE 421 Algorithms
Richard Anderson
Lecture 25
Network Flow Applications

359. Today’s topics Problem Reductions Circulations
Lowerbound constraints on flows
Survey design
Airplane scheduling

360. Problem Reduction Reduce Problem A to Problem B Practical Theoretical
Convert an instance of Problem A to an instance Problem B
Use a solution of Problem B to get a solution to Problem A
Practical
Use a program for Problem B to solve Problem A
Theoretical
Show that Problem B is at least as hard as Problem A

361. Problem Reduction Examples
Reduce the problem of finding the Maximum of a set of integers to finding the Minimum of a set of integers

362. Undirected Network Flow
Undirected graph with edge capacities
Flow may go either direction along the edges (subject to the capacity constraints) u 10 20 5 s t 20 10 v

363. Circulation Problem
Directed graph with capacities, c(e) on the edges, and demands d(v) on vertices
Find a flow function that satisfies the capacity constraints and the vertex demands
0 <= f(e) <= c(e)
fin(v) – fout(v) = d(v)
Circulation facts:
Feasibility problem
d(v) < 0: source; d(v) > 0: sink
Must have vd(v)=0 to be feasible 10 u 20 10
-25 10 s t 20 20 10 v -5

364. Find a circulation in the following graph2 3 4 b e 3 2 6 5 3 -4 -2 2 7 2 1 a c f h 4 9 3 2 8 d g 5 -5 4 10

365. Reducing the circulation problem to Network flow

366. Formal reduction Add source node s, and sink node t
For each node v, with d(v) < 0, add an edge from s to v with capacity -d(v)
For each node v, with d(v) > 0, add an edge from v to t with capacity d(v)
Find a maximum s-t flow. If this flow has size vcap(s,v) than the flow gives a circulation satisifying the demands

367. Circulations with lowerbounds on flows on edges
Each edge has a lowerbound l(e).
The flow f must satisfy l(e) <= f(e) <= c(e) -2 3
1,2 a x
0,2
1,4 b y
2,4 -4 3

368. Removing lowerbounds on edges
Lowerbounds can be shifted to the demands b y
2,5 -4 3 b y 3 -2 1

369. Formal reduction Lin(v): sum of lowerbounds on incoming edges
Lout(v): sum of lowerbounds on outgoing edges
Create new demands d’ and capacities c’ on vertices and edges
d’(v) = d(v) + lout(v) – lin(v)
c’(e) = c(e) – l(e)

370. Application Customized surveys Ask customers about products
Only ask customers about products they use
Limited number of questions you can ask each customer
Need to ask a certain number of customers about each product
Information available about which products each customer has used

371. Details Customer C1, . . ., Cn Products P1, . . ., Pm
Si is the set of products used by Ci
Customer Ci can be asked between ci and c’i questions
Questions about product Pj must be asked on between pj and p’j surveys

372. Circulation construction

373. Airplane Scheduling
Given an airline schedule, and starting locations for the planes, is it possible to use a fixed set of planes to satisfy the schedule.
Schedule
[segments] Departure, arrival pairs (cities and times)

374. Compatible segments
Segments S1 and S2 are compatible if the same plane can be used on S1 and S2
End of S1 equals start of S2, and enough time for turn around between arrival and departure times
End of S1 is different from S2, but there is enough time to fly between cities

375. Graph representation
Each segment, Si, is represented as a pair of vertices (di, ai, for departure and arrival), with an edge between them.
Add an edge between ai and dj if Si is compatible with Sj. di ai ai dj

376. Setting up a flow problem
1,1 di ai
0,1 ai dj 1 -1
P’i Pi

377. Result
The planes can satisfy the schedule iff there is a feasible circulation

378. Richard Anderson Lecture 26 Open Pit Mining
CSE 421 Algorithms
Richard Anderson
Lecture 26
Open Pit Mining

379. Today’s topics Open Pit Mining Problem Task Selection Problem
Reduction to Min cut problem

380. Open Pit Mining Each unit of earth has a profit (possibly negative)
Getting to the ore below the surface requires removing the dirt above
Test drilling gives reasonable estimates of costs
Plan an optimal mining operation

381. Determine an optimal mine-5 -3 -4 -2 2 -4 -6 -1 -7
-10 -1 -3 -3 3 -2 4 -3
-10
-10
-10 4 7 -2 7 3
-10
-10
-10
-10 6 8 6 -3
-10
-10
-10
-10
-10
-10 1 4 4
-10
-10
-10
-10
-10
-10
-10
-10
-10
-10
-10
-10
Student Submission

382. Generalization Precedence graph G=(V,E) Each v in V has a profit p(v)
A set F if feasible if when w in F, and (v,w) in E, then v in F.
Find a feasible set to maximize the profit -4 6 -2 -3 5 -1
-10 4

383. Min cut algorithm for profit maximization
Construct a flow graph where the minimum cut identifies a feasible set that maximizes profit

384. Precedence graph construction
Precedence graph G=(V,E)
Each edge in E has infinite capacity
Add vertices s, t
Each vertex in V is attached to s and t with finite capacity edges

385. Show a finite value cut with at least two vertices on each side of the cut
Student Submission

386. The sink side of the cut is a feasible set
No edges permitted from S to T
If a vertex is in T, all of its ancestors are in T

387. Setting the costs If p(v) > 0, If p(v) < 0 If p(v) = 0
cap(v,t) = p(v)
cap(s,v) = 0
If p(v) < 0
cap(s,v) = -p(v)
cap(v,t) = 0
If p(v) = 0 3 3 1 1 -3 -1 -3 3 2 1 2 3 t

388. Enumerate all finite s,t cuts and show their capacities2 2 -2 1 2 -2 1 1 2 1 t
Student Submission

389. Minimum cut gives optimal solution Why?2 2 -2 1 2 -2 1 1 2 1 t

390. Computing the Profit Cost(W) = {w in W; p(w) < 0}-p(w)
Benefit(W) = {w in W; p(w) > 0} p(w)
Profit(W) = Benefit(W) – Cost(W)
Maximum cost and benefit
C = Cost(V)
B = Benefit(V)

391. Express Cap(S,T) in terms of B, C, Cost(T), Benefit(T), and Profit(T)
Student Submission

392. Cap(S,T) = B – Profit(T)
Cap(S,T) = Cost(T) + Ben(S) = Cost(T) + Ben(S) + Ben(T) – Ben(T)
= B + Cost(T) – Ben(T) = B – Profit(T)

393. Summary Construct flow graph Finite cut gives a feasible set of tasks
Infinite capacity for precedence edges
Capacities to source/sink based on cost/benefit
Finite cut gives a feasible set of tasks
Minimizing the cut corresponds to maximizing the profit
Find minimum cut with a network flow algorithm

394. Richard Anderson Lecture 27 Network Flow Applications
CSE 421 Algorithms
Richard Anderson
Lecture 27
Network Flow Applications

395. Today’s topics More network flow reductions Airplane scheduling
Image segmentation
Baseball elimination

396. Airplane Scheduling
Given an airline schedule, and starting locations for the planes, is it possible to use a fixed set of planes to satisfy the schedule.
Schedule
[segments] Departure, arrival pairs (cities and times)
Approach
Construct a circulation problem where paths of flow give segments flown by each plane

397. Compatible segments
Segments S1 and S2 are compatible if the same plane can be used on S1 and S2
End of S1 equals start of S2, and enough time for turn around between arrival and departure times
End of S1 is different from S2, but there is enough time to fly between cities

398. Graph representation
Each segment, Si, is represented as a pair of vertices (di, ai, for departure and arrival), with an edge between them.
Add an edge between ai and dj if Si is compatible with Sj. di ai ai dj

399. Setting up a flow problem
1,1 di ai
0,1 ai dj 1 -1
P’i Pi

400. Result
The planes can satisfy the schedule iff there is a feasible circulation

401. Image Segmentation
Separate foreground from background

403. Image analysis ai: value of assigning pixel i to the foreground
bi: value of assigning pixel i to the background
pij: penalty for assigning i to the foreground, j to the background or vice versa
A: foreground, B: background
Q(A,B) = {i in A}ai + {j in B}bj - {(i,j) in E, i in A, j in B}pij

404. Pixel graph to flow graphs t

405. Mincut Construction s av
pvu u v
puv bv t

406. Baseball elimination Can the Dung Beetles win the league?
Remaining games:
AB, AC, AD, AD, AD, BC, BC, BC, BD, CD W L
Ants 4 2
Bees
Cockroaches 3
Dung Beetles 1 5

407. Baseball elimination Can the Fruit Flies win the league?
Remaining games:
AC, AD, AD, AD, AF, BC, BC, BC, BC, BC, BD, BE, BE, BE, BE, BF, CE, CE, CE, CF, CF, DE, DF, EF, EF W L
Ants 17 12
Bees 16 7
Cockroaches
Dung Beetles 14 13
Earthworms 10
Fruit Flies 15

408. Assume Fruit Flies win remaining games
Fruit Flies are tied for first place if no team wins more than 19 games
Allowable wins
Ants (2)
Bees (3)
Cockroaches (3)
Dung Beetles (5)
Earthworms (5)
18 games to play
AC, AD, AD, AD, BC, BC, BC, BC, BC, BD, BE, BE, BE, BE, CE, CE, CE, DE W L
Ants 17 13
Bees 16 8
Cockroaches 9
Dung Beetles 14
Earthworms 12
Fruit Flies 19 15

409. Remaining games
AC, AD, AD, AD, BC, BC, BC, BC, BC, BD, BE, BE, BE, BE, CE, CE, CE, DE s AC AD BC BD BE CE DE E A B C D T

410. Network flow applications summary
Bipartite Matching
Disjoint Paths
Airline Scheduling
Survey Design
Baseball Elimination
Project Selection
Image Segmentation

411. Richard Anderson Lecture 28 NP Completeness
CSE 421 Algorithms
Richard Anderson
Lecture 28
NP Completeness

412. Announcements Final Exam HW 10, due Friday, 1:30 pm This weeks topic
Monday, December 12, 2:30-4:20 pm, EE1 003
Closed book, closed notes
Practice final and answer key available
HW 10, due Friday, 1:30 pm
This weeks topic
NP-completeness
Reading: : Skim the chapter, and pay more attention to particular points emphasized in class

413. Algorithms vs. Lower bounds
Algorithmic Theory
What we can compute
I can solve problem X with resources R
Proofs are almost always to give an algorithm that meets the resource bounds

414. Lower bounds How do you show that something can’t be done?
Establish evidence that a whole bunch of smart people can’t do something!
Lower bounds
How do you show that something can’t be done?

415. Theory of NP Completeness
Most significant mathematical theory associated with computing

416. The Universe
NP-Complete NP P

417. Polynomial Time
P: Class of problems that can be solved in polynomial time
Corresponds with problems that can be solved efficiently in practice
Right class to work with “theoretically”

418. Polynomial time reductions
Y Polynomial Time Reducible to X
Solve problem Y with a polynomial number of computation steps and a polynomial number of calls to a black box that solves X
Notations: Y <P X

419. Lemma
Suppose Y <P X. If X can be solved in polynomial time, then Y can be solved in polynomial time.

420. Lemma
Suppose Y <P X. If Y cannot be solved in polynomial time, then X cannot be solved in polynomial time.

421. Sample Problems Independent Set
Graph G = (V, E), a subset S of the vertices is independent if there are no edges between vertices in S 1 2 3 5 4 6 7

422. Vertex Cover Vertex Cover
Graph G = (V, E), a subset S of the vertices is a vertex cover if every edge in E has at least one endpoint in S 1 2 3 5 4 6 7

423. Decision Problems Theory developed in terms of yes/no problems
Independent set
Given a graph G and an integer K, does G have an independent set of size at least K
Vertex cover
Given a graph G and an integer K, does the graph have a vertex cover of size at most K.

424. IS <P VC Lemma: A set S is independent iff V-S is a vertex cover
To reduce IS to VC, we show that we can determine if a graph has an independent set of size K by testing for a Vertex cover of size n - K

425. Satisfiability
Given a boolean formula, does there exist a truth assignment to the variables to make the expression true

426. Definitions Boolean variable: x1, …, xn Term: xi or its negation !xi
Clause: disjunction of terms
t1 or t2 or … tj
Problem:
Given a collection of clauses C1, . . ., Ck, does there exist a truth assignment that makes all the clauses true
(x1 or !x2), (!x1 or !x3), (x2 or !x3)

427. 3-SAT Each clause has exactly 3 terms Variables x1, . . ., xn
Clauses C1, . . ., Ck
Cj = (tj1 or tj2 or tj3)
Fact: Every instance of SAT can be converted in polynomial time to an equivalent instance of 3-SAT

428. Theorem: 3-SAT <P IS
Build a graph that represents the 3-SAT instance
Vertices yi, zi with edges (yi, zi)
Truth setting
Vertices uj1, uj2, and uj3 with edges (uj1, uj2), (uj2,uj3), (uj3, uj1)
Truth testing
Connections between truth setting and truth testing:
If tjl = xi, then put in an edge (ujl, zi)
If tjl = !xi, then put in an edge (ujl, yi)

429. Example
C1 = x1 or x2 or !x3
C2 = x1 or !x2 or x3
C3 = !x1 or x2 or x3

430. Thm: 3-SAT instance is satisfiable iff there is an IS of size n + k

431. Richard Anderson Lecture 29 NP-Completeness
CSE 421 Algorithms
Richard Anderson
Lecture 29
NP-Completeness

432. Sample Problems Independent Set
Graph G = (V, E), a subset S of the vertices is independent if there are no edges between vertices in S 1 2 3 5 4 6 7

433. Satisfiability
Given a boolean formula, does there exist a truth assignment to the variables to make the expression true

434. Definitions Boolean variable: x1, …, xn Term: xi or its negation !xi
Clause: disjunction of terms
t1 or t2 or … tj
Problem:
Given a collection of clauses C1, . . ., Ck, does there exist a truth assignment that makes all the clauses true
(x1 or !x2), (!x1 or !x3), (x2 or !x3)

435. 3-SAT Each clause has exactly 3 terms Variables x1, . . ., xn
Clauses C1, . . ., Ck
Cj = (tj1 or tj2 or tj3)
Fact: Every instance of SAT can be converted in polynomial time to an equivalent instance of 3-SAT

436. Theorem: 3-SAT <P IS
Build a graph that represents the 3-SAT instance
Vertices yi, zi with edges (yi, zi)
Truth setting
Vertices uj1, uj2, and uj3 with edges (uj1, uj2), (uj2,uj3), (uj3, uj1)
Truth testing
Connections between truth setting and truth testing:
If tjl = xi, then put in an edge (ujl, zi)
If tjl = !xi, then put in an edge (ujl, yi)

437. Example
C1 = x1 or x2 or !x3
C2 = x1 or !x2 or x3
C3 = !x1 or x2 or x3

438. Thm: 3-SAT instance is satisfiable iff there is an IS of size n + k

439. What is NP?
Problems solvable in non-deterministic polynomial time . . .
Problems where “yes” instances have polynomial time checkable certificates

440. Certificate examples Independent set of size K Satifisfiable formula
The Independent Set
Satifisfiable formula
Truth assignment to the variables
Hamiltonian Circuit Problem
A cycle including all of the vertices
K-coloring a graph
Assignment of colors to the vertices

441. NP-Completeness A problem X is NP-complete if
X is in NP
For every Y in NP, Y <P X
X is a “hardest” problem in NP
If X is NP-Complete, Z is in NP and X <P Z
Then Z is NP-Complete

442. Cook’s Theorem
The Circuit Satisfiability Problem is NP- Complete

443. Garey and Johnson

444. History Jack Edmonds Steve Cook Dick Karp Leonid Levin Identified NP
Cook’s Theorem – NP-Completeness
Dick Karp
Identified “standard” collection of NP-Complete Problems
Leonid Levin
Independent discovery of NP-Completeness in USSR

445. Populating the NP-Completeness Universe
Circuit Sat <P 3-SAT
3-SAT <P Independent Set
Independent Set <P Vertex Cover
3-SAT <P Hamiltonian Circuit
Hamiltonian Circuit <P Traveling Salesman
3-SAT <P Integer Linear Programming
3-SAT <P Graph Coloring
3-SAT <P Subset Sum
Subset Sum <P Scheduling with Release times and deadlines

446. Hamiltonian Circuit Problem
Hamiltonian Circuit – a simple cycle including all the vertices of the graph

447. Traveling Salesman Problem
Minimum cost tour highlighted
Traveling Salesman Problem
Given a complete graph with edge weights, determine the shortest tour that includes all of the vertices (visit each vertex exactly once, and get back to the starting point) 3 7 7 2 2 5 4 1 1 4

448. Thm: HC <P TSP

449. Richard Anderson Lecture 30 NP-Completeness
CSE 421 Algorithms
Richard Anderson
Lecture 30
NP-Completeness

450. NP-Completeness A problem X is NP-complete if
X is in NP
For every Y in NP, Y <P X
X is a “hardest” problem in NP
To show X is NP complete, we must show how to reduce every problem in NP to X

451. Cook’s Theorem The Circuit Satisfiability Problem is NP- Complete
Given a boolean circuit, determine if there is an assignment of boolean values to the input to make the output true

452. Circuit SAT Satisfying assignment x1 = T, x2 = F, x3 = F
x4 = T, x5 = T
AND
Circuit SAT OR OR
AND
AND
AND
AND
NOT OR
NOT OR
AND OR
AND
NOT
AND
NOT OR
NOT
AND x3 x4 x5 x1 x2

453. Proof of Cook’s Theorem
Reduce an arbitrary problem Y in NP to X
Let A be a non-deterministic polynomial time algorithm for Y
Convert A to a circuit, so that Y is a Yes instance iff and only if the circuit is satisfiable

454. Garey and Johnson

455. History Jack Edmonds Steve Cook Dick Karp Leonid Levin Identified NP
Cook’s Theorem – NP-Completeness
Dick Karp
Identified “standard” collection of NP-Complete Problems
Leonid Levin
Independent discovery of NP-Completeness in USSR

456. Populating the NP-Completeness Universe
Circuit Sat <P 3-SAT
3-SAT <P Independent Set
Independent Set <P Vertex Cover
3-SAT <P Hamiltonian Circuit
Hamiltonian Circuit <P Traveling Salesman
3-SAT <P Integer Linear Programming
3-SAT <P Graph Coloring
3-SAT <P Subset Sum
Subset Sum <P Scheduling with Release times and deadlines

457. Hamiltonian Circuit Problem
Hamiltonian Circuit – a simple cycle including all the vertices of the graph

458. Traveling Salesman Problem
Minimum cost tour highlighted
Traveling Salesman Problem
Given a complete graph with edge weights, determine the shortest tour that includes all of the vertices (visit each vertex exactly once, and get back to the starting point) 3 7 7 2 2 5 4 1 1 4

459. Thm: HC <P TSP

460. Number Problems Subset sum problem Subset sum problem is NP-Complete
Given natural numbers w1,. . ., wn and a target number W, is their a subset that adds up to exactly W
Subset sum problem is NP-Complete
Subset Sum problem can be solved in O(nW) time

461. Subset sum problem
The reduction to show Subset Sum is NP- complete involves numbers with n digits
In that case, the O(nW) algorithm is an exponential time and space algorithm

462. Course summary What did we cover in the last 30 lectures?
Stable Matching
Models of computation and efficiency
Basic graph algorithms
BFS, Bipartiteness, SCC, Cycles, Topological Sort
Greedy Algorithms
Interval Scheduling, HW Scheduling
Correctness proofs
Dijkstra’s Algorithm
Minimum Spanning Trees
Recurrences
Divide and Conquer Algorithms
Closest Pair, FFT
Dynamic Programming
Weighted interval scheduling, subset sum, knapsack, longest common subsequence, shortest paths
Network Flow
Ford Fulkerson, Maxflow/mincut, Applications
NP-Completeness