1. CSE 2331/5331 Topic 5: Prob. Analysis Randomized Alg.

2. Expected Complexity CSE 2331/5331 Probabilistic method: Given a distribution for all possible inputs Derive expected time based on distribution Randomized algorithm: Add randomness in the algorithm Analyze the expected behavior of the algorithm

3. First Example What is worst case time complexity? What is expected / average time complexity? CSE 2331/5331

4. Expected Running Time CSE 2331/5331

5. SeqSearch Alg CSE 2331/5331

6. Remarks For probabilistic analysis An input probabilistic distribution input model has be assumed! For a fixed input, the running time is fixed. The average / expected time complexity is for if we consider running it for a range of inputs, what the average behavior is. Randomized algorithm No assumption in input distribution! Randomness is added in the algorithm For a fixed input, the running time is NOT fixed. The expected time is what we can expect when we run the algorithm on ANY SINGLE input. CSE 2331/5331

7. Randomized Algorithms CSE 2331/5331

8. Expectation CSE 2331/5331

9. Linearity of Expectation CSE 2331/5331

10. Conditional Expectation CSE 2331/5331

11. A Randomized Example CSE 2331/5331 Worst case complexity? Expected case?

12. Running Time Analysis CSE 2331/5331

14. Time Analysis CSE 2331/5331

15. A more complicated variation.

16. Analysis CSE 2331/5331

17. An Example with Recursion CSE 2331/5331

19. Another Example with Recursion CSE 2331/5331

20. Analysis CSE 2331/5331

21. Another Example CSE 2331/5331

22. Analysis CSE 2331/5331

23. Back To QuickSort CSE 2331/5331 Randomized-QuickSort ( A, p, r ) If ( p ≥ r) return; s = Randomized-Partition ( A, p, r ); A1 = Randomized-QuickSort ( A, p, s-1 ); A2 = Randomized-QuickSort ( A, s+1, r );

24. CSE 2331/5331

25. Randomized-QuickSort ( A, 1, n ) If ( 1 ≥ n) return; s = Randomized-Partition ( A, 1, n ); A1 = Randomized-QuickSort ( A, 1, s-1 ); A2 = Randomized-QuickSort ( A, s+1, n ); CSE 2331/5331

26. One can use substitution method to show that ET(n) = O(n lg n)

27. Another Way of Analysis CSE 2331/5331

29. Yet Another Way of Analysis Using the indicator random variable Requires global understanding of the algorithm The key is to identify the right indicator random variable Similar to identify the right events to compute expectation. CSE 2331/5331

30. Indicator Random Variable Goal: count the expected number of heads in n coin flips Straightforward: X : number of heads in n coin flips E[X] = ∑ k Prob[X = k] New approach: X[j] = I { the j’th flip is a head } = 1 if j’th flip is a head 0 otherwise Indicator random variable

31. CSE 2331/5331 Indicator Random Variable cont. Lemma: Let X = I { claim A }. Then, E[X] = Prob[claim A is true].

32. CSE 2331/5331 Counting Heads

33. CSE 2331/5331 Rand-QuickSort Given A, suppose z 1, z 2, …., z n are its elements in sorted order. Indicator random variable: X(i,j) = 1 if is compared to 0 otherwise z i z j Complexity = O(# comparisons Y) + O(n) Y = ∑ ∑ X(i,j) ij Why?

34. CSE 2331/5331 Pr[i,j] Expected Complexity E[Y] = E [ ∑ ∑ X(i,j) ] = ∑ ∑ E [ X(i,j) ] = ∑ ∑ Pr { z i is compared to z j } Z[i, j] : { z i, z i+1 …. z j }

35. CSE 2331/5331 Key Observation If two elements in different subarray Will never be compared from then on X[i,j] = 1 if and only if z i or z j is the first among Z[i,j] as privot Pr[i,j] = Pr[ z i or z j is the first pivot chosen from Z[i,j] ] = Pr [ z i is the first pivot chosen from Z[i,j] ] + Pr[ z j is the first pivot chosen from Z[i,j] ] = 1 / (j - i + 1) + 1 / (j - i + 1) = 2 / ( j - i + 1) Why?

36. CSE 2331/5331 Expected Complexity E[X] = ∑ ∑ 2 / ( j - i + 1 ) … = O ( n lg n )

37. Summary CSE 2331/5331