1. Karger’s Min-Cut Approximate Max-Cut Monday, July 21st 1

2. Announcements  HW 4 is due this Friday (July 25 th )  HW 5: (short hw, 2 or 3 questions)  Out on Wednesday (July 23 rd )  Due next Friday (August 1 st )  Programming Assignment  Out on Wednesday (July 23 rd )  Due 16 days later on Friday (August 8 th )  HW 6: (short hw, 2 or 3 questions)  Out on Wednesday (August 6 th )  Due next Wednesday (August 13 rd ) 2

3. Fundamental Algorithms to Tractable Problems Common Algorithm Design Paradigms Mathematical Tools to Analyze Algorithms Intractable Problems MergeSort Strassen’s MM BFS/DFS Dijkstra’s SSSP Kosaraju’s SCC Kruskal’s MST Floyd Marshall APSP Karger’s Min-Cut Quicksort Bellman-Ford … Big-oh notation Recursion Tree Master method Substitution method Exchange Arguments P vs NP Approximation Algs. Divide-and-Conquer Greedy Dynamic Programming Other Data Structures  Hash Functions/Tables Parallel Algorithms Course RoadMap

4. Outline For Today 1.Karger’s MinCut 2.Approximate Max-Cut 4

5. Min Cut Problem 5  Input: Undirected G(V, E)  not necessarily simple, i.e., parallel (u, v) edges are OK.  Output: cut (X, Y) with min # of crossing edges  Intuitively a measure of how-well connected the graph is  i.e., how robust the graph is to edge removals  Applications:  transportation networks  clustering  graph partitioning  image segmentation

6. Example Cut 6 A B C D E

7. 7 A B C D E Cut Size: 4

8. Example Min Cut 8 A B C D E

9. 9 A B C D E Min Cut Size: 2

10. Observation 1 10 Claim: |min-cut| ≤ min-degree k in the graph Proof: Let v be s.t deg(v) = k then ({v}, V – {v}) has k edges crossing it v … V-{v} |({v}, V-{v})| = k deg(v) = k

11. Corollary 11 If min-cut has size k: Q1: what’s a bound on the degrees of vertices? A: ≥ k Q2: what’s a bound on m=|E|? A: v … V-{v} |({v}, V-{v})| = k deg(v) = k

12. Karger’s Algorithm Pseudocode (1993) procedure Karger(G(V,E)): while |V| > 2: pick an edge (u, v) uniformly at random merge u, v into a single node uv remove edges between u, v return (X,Y) represented by last 2 vertices 12

13. Karger’s Algorithm Simulation 1 13 A B C D E G 1 =(m 1 =m, n)

14. Karger’s Algorithm Simulation 1 14 A B C D E G 1 =(m 1 =m, n)

15. Karger’s Algorithm Simulation 1 15 AB C D E G 2 =(m 2, n-1)

16. Karger’s Algorithm Simulation 1 16 AB C D E G 2 =(m 2, n-1)

17. Karger’s Algorithm Simulation 1 17 ABC D E G 3 =(m 3, n-2)

18. Karger’s Algorithm Simulation 1 18 ABC D E G 3 =(m 3, n-2)

19. Karger’s Algorithm Simulation 1 19 ABC DE Output Cut: ({A, B, C}, {D, E}) G 4 =(m 4, n-3) Q: What’s the size of the cut? A: m 4 =3 Observation: Edges crossing two last 2 (super)nodes are exactly the edges crossing the cut Karger outputs!

20. Karger’s Algorithm Simulation 1 20 A B C D E Output Cut: ({A, B, C}, {D, E}), size: m 4 =3

21. Karger’s Algorithm Simulation 1 21 A B C D E Output Cut: ({A, B, C}, {D, E}), size: m 4 =3 Incorrect: Not Minimum-Cut

22. Karger’s Algorithm Simulation 2 22 A B C D E G 1 =(m 1 =m, n)

23. Karger’s Algorithm Simulation 2 23 A B C D E G 1 =(m 1 =m, n)

24. Karger’s Algorithm Simulation 2 24 AC B D E G 2 =(m 2`, n-1)

25. Karger’s Algorithm Simulation 2 25 AC B D E G 2 =(m 2`, n-1)

26. Karger’s Algorithm Simulation 2 26 ACD B E G 3 =(m 3`, n-2)

27. Karger’s Algorithm Simulation 2 27 ACD B E G 3 =(m 3`, n-2)

28. Karger’s Algorithm Simulation 2 28 ACDE B Output Cut: ({A, C, D, E}, {B}), size: m 4` =2 G 4 =(m 4`, n-2)

29. Karger’s Algorithm Simulation 2 29 A B C D E Output Cut: ({A, C, D, E}, {B}), size: m 4` =2

30. Karger’s Algorithm Simulation 2 30 A B C D E Output Cut: ({A, C, D, E}, {B}), size: m 4` =2 Correct: Minimum Cut

31. Fundamental Question 31 How often does Karger return a min cut? OR: What’s the probability that Karger returns a min-cut? Weaker Question: Let (X, Y) be a min-cut? What’s the probability Karger returns (X, Y).

32. Warm-up Question 1 32 Assume (X, Y) is a min cut and|(X, Y)| = k. Assume first iteration Karger picks a (u, v) crossing (X, Y). Q: Can we conclude whether Karger succeeds or fails at the end? A: Karger fails to return (X, Y). Why? => (u, v) crossing implies u ∈ X, and v ∈ Y (or vice versa) => u and v will be merged into a supernode  u and v will be on the same side of the final cut output  the final cut will NOT be (X, Y)

33. Warm-up Question 2 33 Assume the min cut is (X, Y) and |(X, Y)| = k. Assume in any iteration Karger picks (u, v) ∈ (X, Y) crossing edges. Q: Can we conclude whether Karger succeeds or fails at the end? A: Karger fails. Why? => Same argument

34. Converse is Also True 34 Assume the min cut is (X, Y) and |(X, Y)| = k. If Karger never picks a (u, v) crossing (X, Y)  All contractions were internal to X and internal to Y  At most |X|-1 possible contractions internal to X  At most (|Y|-1) possible contractions internal to Y  Since|X|-1+|Y|-1 is exactly n-2, all of X and Y must have been contracted into supernodes  Karger outputs exactly (X, Y)

35. Question We Are After 35 What’s the probability Karger never picks in any of the k edges crossing (X, Y)?

36. Key Events S i 36 Let S i be the event that one of the k (X, Y) edges is contracted in iteration i. **We’re after: Pr(¬S 1 ∩ ¬S 2 ∩ … ∩ ¬S n-2 )** Q: Pr(S 1 )? A: Since we know that m ≥ kn/2 Pr(S 1 ) Pr(¬S 1 ) Next Question: Pr(¬S 1 ∩ ¬S 2 )

37. Probability Review: Conditional Probability 37 Let S, T be two events S T S∩TS∩T Ex: Pr(roll > 4|roll>3):?

38. Pr(¬S 1 ∩ ¬S 2 )=Pr(¬S 1 )Pr(¬S 2 |¬S 1 ) 38 Recall after contracting one edge we have G 2 =(m 2, n-1). Pr(S 2 |¬S 1 ) = Question: What do we know about the degrees of nodes/super-nodes in the G i ? Answer: ≥ k Why? Every cut in the contracted graph induces a cut in the original graph.

39. Cuts in Contracted Graph 39 A B C D E ACD B E G1G1 G3G3

40. Cuts in Contracted Graph 40 A B C D E ACD B E G1G1 G3G3

41. Number of edges in G 2 41 Degree of every vertex in contracted graph ≥ k In general:

42. Pr(¬S 1 ∩ ¬S 2 )=Pr(¬S 1 )Pr(¬S 2 |¬S 1 ) 42 Pr(S 2 |¬S 1 ) = Pr(¬S 2 |¬S 1 ) Pr(¬S 2 ∩ ¬S 1 ) = Pr(¬S 1 )Pr(¬S 2 |¬S 1 ) Next Question: Pr(¬S 1 ∩ ¬S 2 ∩ ¬S 3 )

43. Pr(¬S 1 ∩ ¬S 2 ∩ ¬S 3 )=Pr(¬S 1 ∩ ¬S 2 )Pr(¬S 3 |¬S 1 ∩ ¬S 2 ) 43 Pr(S 3 |¬S 1 ∩ ¬S 2 ) = Pr(¬S 3 |¬S 1 ∩ ¬S 2 ) Pr(¬S 1 ∩ ¬S 2 ∩ ¬S 3 ) = Pr(¬S 1 ∩ ¬S 2 )Pr(¬S 3 |¬S 1 ∩ ¬S 2 )

44. Pr(success)=Pr(¬S 1 ∩ ¬S 2 …. ∩ ¬S n-2 ) 44 **Karger outputs (X, Y) with probability at least 1/n 2 **

45. Pr(success) Looks Very Small! 45 n=100 n=10 6 But we can easily boost this probability!

46. Probability Recap: Independence 46 Recall notion of independent events: By definition: S and T are independent iff Pr(S ∩ T) = Pr(S)Pr(T)  Pr(S|T)=Pr(S)  Pr(T|S)=Pr(T)

47. Trick: Running Karger Over and Over Again 47 Karger G(V,E) O1O1 Karger G(V,E) O2O2 Karger G(V,E) O3O3 Karger G(V,E) ONON Return best O j …

48. Trick: Running Karger Over and Over Again 48 Hope: At least one of the N runs succeeds T i be the event Karger returns (X, Y) in ith iteration. By independence of T i

49. What’s the Rate of Decrease? 49 Calculus fact: ∀ x real, 1+x ≤ e x Let N=n 2 => Let N=n 2 ln(n) => quite small! quite large!

50. Let’s Boost Success Probability Further… 50 Let N=2n 2 ln(n) => very small! very large! Let N=4n 2 ln(n) =>very very small! very very large!

51. Definition: With High Probability (w.h.p) 51 We say an algorithm succeeds with high probability if Pr(success) ≥ (1-1/n) probability.

52. Summary 1.Described Karger’s Algorithm 2.Analyzed the probability of success in one iteration  Pr(success in one iteration) ≥ 1/n 2  (polynomially) small 3.Repeated the algorithm O(n 2 log(n)) (poly) times  with high probability found the min-cut

53. Run-time Comments  O(n 2 log(n)) iterations  Each iteration at least O(m) times  at least O(mn 2 log(n))  A simple extension (Karger/Stein): O(n 2 log 3 (n)), which is almost linear when m is O(n 2 )

54. Interesting Consequence of Our Analysis  We showed that the prob. of Karger outputting (X, Y) cut is ≥ 1/n 2  Q: What can we conclude about the number of distinct min-cuts?  A: ≤ n 2

55.  Given a fixed input, they may give: 1.Different (possibly wrong) outputs (Monte Carlo Algs) 2.Different runtimes (possibly exponential) (Las Vegas Algs) depending on the outcomes of the coins Final Comments (2) Randomized Algorithm Input Output flip coins 55  Randomized Algorithms

56. Outline For Today 1.Karger’s MinCut 2.Approximate Max-Cut 56

57. Max Cut Problem 57  Input: Undirected G(V, E)  not necessarily simple, i.e., parallel (u, v) edges are OK.  Output: cut (X, Y) with max # of crossing edges Fact: Unlike Min-Cut, Max-Cut is Intractable (NP-complete) => can’t hope to solve it exactly within reasonable (poly- time) amount of time

58. Simple Randomized Algorithm 58 procedure Approximate-MaxCut(G(V,E)): let L, R be 2 empty sets for each vertex v: toss a fair coin if heads: put v into L if tails: put v into R

59. Worst & Best Scenarios 59  Worst Scenario:  every vertex goes to L or R, so 0 edges cross (L, R)  Best Scenario:  let (X, Y) be a max-cut  algorithm returns (X, Y)  Let Z be the # edges crossing (L, R)  Q:What’s E[Z]?

60. E[Z] 60  Let Z be # edges crossing (L, R)  ∀ e ∈ E, Let X e be an indicator random variable, s.t.  X e = 1 if e crosses (L, R)  X e = 0 o.w. Consider e=(u,v). What’s E[X e ]?

61. E[X e ]=Pr(X e = 1) 61 v u L R v u L R v u L R u v L R E[X e ]=Pr(X e = 1)=1/2

62. Back to E[Z] 62 All cuts have size ≤ m, so this is always within a factor of 2 of optimal! => we have randomized 1/2-approximation algorithm to max-cut. Key Takeaway: Solving max-cut exactly is impossible but we have a very simple linear time approximate randomized algorithm!

63. Interesting Consequence of Our Analysis Claim: Every graph G(V, E) has a cut of size m/2! Why? Since the expected number of edges of our algorithm is m/2, there has to be at least one cut with ≥ m/2.

64. Problem With Expectations Expectation of a R.V. does not give enough information about the probability that the R.V. takes a value around its expectation.

65. Different RV With Same Expectations (1) 100150200500 50% 0.5*50 + 0.5*150 = 100 Probability Distribution of RV

66. Different RV With Same Expectations (2) 100150200500 50% 0.5* + 0.5*200 = 100 Probability Distribution of RV

67. Different RV With Same Expectations (3) 100150200500 100% 1*100 = 100 Probability Distribution of RV

68. Expected Analysis of Previous Algorithms Q2: We know **on expectation** Approximate-Max-Cut outputs a large cut. But what’s the probability that it actually outputs a large cut! Q1: We know **on expectation** Quicksort takes ≤2nln(n) time. What’s the probability that it takes ~2nln(n) time.

69. Markov’s Inequality For any random variable X, c > 1 Pr(X ≥ cE[X]) ≤ 1/c  Pr(QuickSort takes ≥ 2(2nlnn)) ≤ 1/2  Pr(QuickSort takes ≥ 8nlnn) ≤ 1/4 =>Pr(QuickSort takes ≥ 200nlnn)) ≤ 1/100 There are other techniques you can use to show that Quicksort takes time around its expectation with much higher probability

70. Analyzing Approximate Max-Cut Q2: What’s the probability that Approximate Maxcut outputs a large cut! Recall Z = size of the cut output by Approximate Maxcut E[Z] = m/2 Let Y = m – Z E[Y] = m – E[Z] = m/2 By Markov: Pr(Y ≥ 1.5(m/2))=Pr(Y > 3m/4) ≤ 1/1.5=2/3 Pr(Z ≤ m/4) ≤ 2/3 => with 33% probability Approx. Max-Cut outputs 0.25- approximation to the maximum cut

71. Boosting Success Probability With Repeats Approx-MC G(V,E) O1O1 Approx-MC G(V,E) O2O2 Approx-MC G(V,E) O3O3 Approx-MC G(V,E) ONON Return best O j …

72. Boosting Success Probability With Repeats Let Z 1, Z 2, …, Z N be the outputs of each independent trial. Pr(Z i ≤ m/4) ≤ 2/3 Pr(return a small cut) = Pr(Z 1 ≤m/4 ∩ … ∩ Z N ≤m/4) ≤ (2/3) N Pick N=log 3/2 (n)=O(log(n))  Pr(return a small cut) ≤ 1/n  Pr(retuning a large cut) Pr(max(Z i ) ≥ m/4) ≥ (1-1/n) Repeating Approx. MaxCut O(logn) times returns w.h.p a 0.25-approximation to the maximum cut!