1. Topic 3: Prob. Analysis Randomized Alg.
CSE 2331/5331
Topic 3:
Prob. Analysis
Randomized Alg.
CSE 2331/5331

2. Expected Complexity Probabilistic method: Randomized algorithm:
Given a distribution for all possible inputs
Derive expected time based on distribution
Randomized algorithm:
Add randomness in the algorithm
Analyze the expected behavior of the algorithm
CSE 2331/5331

3. First Example What is worst case time complexity?
What is expected / average time complexity?
CSE 2331/5331

4. Expected Running Time Expected / average running time
𝐸𝑇 𝑛 = 𝐼 Pr 𝐼 𝑡 𝐼
Pr 𝐼 = probability of event I
𝑡 𝐼 = running time given event I
To analyze, need to assume a probabilistic distribution for all inputs
CSE 2331/5331

5. SeqSearch Alg Expected running time =
Pr K ∉A t K∉A + 𝑖=1 𝑛 Pr 𝐴 𝑖 =𝐾 𝑡 𝐴 𝑖 =𝐾
If we assume
Pr K ∉A = 0
All permutations are equally likely
implies Pr 𝐴 𝑖 =𝐾 = 1 𝑛
Then expected running time = 𝑖 1 𝑛 ∗𝑐𝑖 =Θ 𝑛
CSE 2331/5331

6. Remarks For probabilistic analysis Randomized algorithm
An input probabilistic distribution input model has be assumed!
For a fixed input, the running time is fixed.
The average / expected time complexity is for if we consider running it for a range of inputs, what the average behavior is.
Randomized algorithm
No assumption in input distribution!
Randomness is added in the algorithm
For a fixed input, the running time is NOT fixed.
The expected time is what we can expect when we run the algorithm on ANY SINGLE input.
CSE 2331/5331

7. Randomized Algorithms
CSE 2331/5331

8. Expectation X is a random variable The expectation of X is
𝐸 𝑋 = 𝐼 Pr 𝑋=𝐼 𝐼
E.g, coin flip
Linearity of expectation:
𝐸 𝑋 1 + 𝑋 2 =𝐸 𝑋 1 +𝐸 𝑋 2
Conditional expectation:
𝐸 𝑋 =𝐸 𝑋 𝑌) Pr 𝑌 +𝐸(𝑋 𝑁𝑜𝑡 𝑌 1 − Pr 𝑌
CSE 2331/5331

9. Linearity of Expectation
𝐸 𝑇 1 𝑛 =𝑛 𝐸 𝑇 2 𝑛 + cn
𝐸 𝑇 2 𝑛 = expected running time for func2
What is 𝐸 𝑇 1 𝑛 ?
CSE 2331/5331

10. Conditional Expectation
𝐸 𝑇 1 𝑛 = Pr ℎ𝑒𝑎𝑑 𝐸 𝑇 2 𝑛
+(1 − Pr ℎ𝑒𝑎𝑑 ) 𝐸 𝑇 3 𝑛
+ 𝑐
𝐸 𝑇 2 𝑛 = expected running time of func2
𝐸 𝑇 3 𝑛 = expected running time of func3
What is the expected running time of func1?
CSE 2331/5331

11. A Randomized Example Worst case complexity? Expected case?
Worst case \Theta(n^2)
Expected case: T(n) = \sum(1/n) ck^2 = \Theta(n^2) too
CSE 2331/5331

12. Running Time Analysis Worst Case: Expected running time: 𝑇 𝑛 =Θ( 𝑛 2 )
𝐸𝑇 𝑛 = 𝑖=1 𝑛 Pr 𝑘=𝑖 ⋅(𝑐 𝑖 2 ) = 𝑖=1 𝑛 1 𝑛 ⋅(𝑐 𝑖 2 )
= 𝑐 𝑛 𝑖=1 𝑛 𝑖 2 =Θ 𝑛 2
CSE 2331/5331

13. Another Example
CSE 2331/5331

14. Running time analysis Worst case: 𝑇 𝑛 = 𝑖=1 𝑛 𝑐 𝑖 3 2 =Θ 𝑛 2.5 .
Expected running time:
Lines 4—8 takes time: 𝑐 𝑘 3 2
Let 𝑋 𝑖 denote the running time for the i-th iteration of outer for-loop (i.e, from line 3 – 8)
𝐸 𝑋 𝑖 = 𝑞=1 𝑖 𝑃𝑟𝑜𝑏 𝑘=𝑞 𝑖𝑛 𝑡ℎ𝑒 𝑖𝑡ℎ 𝑖𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛 𝑇𝑖𝑚𝑒 𝑙𝑖𝑛𝑒𝑠 3−8 𝑘=𝑞
= 𝑞=1 𝑖 1 𝑖 ⋅𝑐 𝑞 = 𝑐 𝑖 𝑞=1 𝑖 𝑞 =Θ( 𝑖 1.5 )
Hence we can assume that 𝐸 𝑋 𝑖 = 𝑐 ′ 𝑖 1.5
𝐸𝑇 𝑛 =𝐸 𝑖=1 𝑛 𝑋 𝑖 = 𝑖=1 𝑛 𝐸 𝑋 𝑖 = 𝑖=1 𝑛 𝑐 ′ 𝑖 1.5 =Θ 𝑛 2.5
CSE 2331/5331

15. CSE 2331/5331

16. Time Analysis Worst case: Expected running time: 𝑇 𝑛 =Θ 𝑛 2
𝑇 𝑛 =Θ 𝑛 2
Expected running time:
𝐸𝑇 𝑛 = Pr 𝑘< 𝑛 ⋅𝐸𝑥𝑝𝑇𝑖𝑚𝑒(𝑘< 𝑛 )+
Pr 𝑘≥ 𝑛 ⋅𝐸𝑥𝑝𝑇𝑖𝑚𝑒 𝑘≥ 𝑛
= 𝑛 𝑛 ⋅ 𝑐 𝑛 − 𝑛 𝑛 ⋅( 𝑐 ′ 𝑛 lg 𝑛)
⇒𝐸𝑇(𝑛)=Θ( 𝑛 1.5 )
CSE 2331/5331

17. A more complicated
variation.
CSE 2331/5331

18. Analysis Worst case: Expected case – analysis (1) 𝑇 𝑛 =Θ(𝑛 lg 𝑛 )
𝐸𝑇 𝑛 = Pr 𝑘< 𝑛 ⋅𝐸𝑇 𝑙𝑖𝑛𝑒𝑠 4−7 𝑘< 𝑛 +
Pr 𝑘≥ 𝑛 ⋅𝐸𝑇 𝑙𝑖𝑛𝑒𝑠 11−17∣ 𝑘≥ 𝑛
= 𝑛 𝑛 𝑖=1 𝑛 Pr 𝑘=𝑖∣𝑘< 𝑛 ⋅𝐸𝑇 𝑙𝑖𝑛𝑒𝑠 4−7∣𝑘=𝑖 +
= 1 𝑛 𝑖=1 𝑛 𝑛 ⋅ 𝑐 𝑖 − 𝑛 𝑛 ⋅( 𝑐 ′ 𝑛 lg 𝑛)
=Θ( 𝑛 ) +Θ 𝑛 lg 𝑛 =Θ(𝑛 lg 𝑛)
CSE 2331/5331

19. Analysis Expected case – analysis (2)
𝐸𝑇 𝑛 = 𝑖=1 𝑛 Pr 𝑘=𝑖 ⋅𝐸𝑇 𝑙𝑖𝑛𝑒𝑠 4−7∣𝑘=𝑖 +
Pr 𝑘≥ 𝑛 ⋅𝐸𝑇 𝑙𝑖𝑛𝑒𝑠 11−17∣𝑘≥ 𝑛
= 𝑖=1 𝑛 1 𝑛 ⋅ 𝑐 𝑖 − 𝑛 𝑛 ⋅( 𝑐 ′ 𝑛 lg 𝑛)
=Θ( 𝑛 ) +Θ 𝑛 lg 𝑛 =Θ(𝑛 lg 𝑛)
CSE 2331/5331

20. An Example with Recursion
CSE 2331/5331

21. Expected running time 𝐸𝑇 𝑛
Worst case:
T n =T n−1 +cn= Θ 𝑛 2
Expected running time 𝐸𝑇 𝑛
𝐸𝑇 𝑛 =𝑐𝑛 𝐸𝑇 𝑛−1
Solving this we have
𝐸𝑇 𝑛 = 1 2 ET n−1 +cn≤ 1 2 𝐸𝑇 𝑛 +𝑐n
⇒ 1 2 𝐸𝑇 𝑛 ≤𝑐𝑛⇒𝐸𝑇 𝑛 ≤2𝑐𝑛⇒𝐸𝑇 𝑛 =𝑂 𝑛
⇒𝐸𝑇 𝑛 = Θ 𝑛
CSE 2331/5331

22. Another Example with Recursion
CSE 2331/5331

23. Analysis Worst case: Expected running time: 𝑇 𝑛 =𝑐+𝑇 𝑛−1 +𝑇 𝑛−2
𝑇 𝑛 =Ω 2 𝑛/2 !
Expected running time:
𝐸𝑇 𝑛 =𝑐 𝐸𝑇 𝑛−1 +𝐸𝑇 𝑛−2
⇒𝐸𝑇 𝑛 ≤𝑐+ 1 2 ET n ⇒ 1 2 ET n ≤𝑐
⇒𝐸𝑇 𝑛 ≤2𝑐⇒𝐸𝑇 𝑛 =O(1)
𝐸𝑇 𝑛 =Θ 1 !
CSE 2331/5331

24. Another Example
CSE 2331/5331

25. Analysis Worst case: Expected running time: 𝑇 𝑛 =2𝑇 𝑛 2 +𝑐𝑛=Θ 𝑛 lg 𝑛
𝐸𝑇 𝑛 = 1 4 𝐸𝑇 𝑛 2 +𝐸𝑇 𝑛 2 +𝑐𝑛= 1 2 𝐸𝑇 𝑛 2 +𝑐𝑛
⇒𝐸𝑇 𝑛 ≤ 1 2 𝐸𝑇 𝑛 + 𝑐𝑛⇒ 1 2 𝐸𝑇 𝑛 ≤𝑐𝑛
⇒𝐸𝑇 𝑛 =𝑂 𝑛 ⇒𝐸𝑇 𝑛 =Θ 𝑛
CSE 2331/5331

26. Example 5
CSE 2331/5331

27. 𝐸𝑇(𝑛) = expected running time of 𝐹𝑢𝑛𝑐5 on 𝑛 values
CSE 2331/5331

28. Bounding 𝐸𝑇(𝑛) 𝑃𝑟 𝑘< 𝑛 4 = 1 4 ; 𝑃𝑟 𝑛 4 ≤𝑘≤ 3𝑛 4 = 1 2 ;
𝑃𝑟 𝑘< 𝑛 4 = 1 4 ;
𝑃𝑟 𝑛 4 ≤𝑘≤ 3𝑛 4 = 1 2 ;
𝑃𝑟 𝑘> 3𝑛 4 = 1 4 ;
CSE 2331/5331

29. Upper Bound
CSE 2331/5331

30. Upper Bound cont. 𝐸𝑇 𝑛 ≤𝑐𝑛+ 1 2 𝐸𝑇 𝑛 + 1 2 (𝐸𝑇 𝑛 4 +𝐸𝑇 3𝑛 4 )
𝐸𝑇 𝑛 ≤𝑐𝑛+ 1 2 𝐸𝑇 𝑛 (𝐸𝑇 𝑛 4 +𝐸𝑇 3𝑛 4 )
CSE 2331/5331

31. Lower Bound In the best case, 𝑘= 𝑛 2 Hence
Putting upper and lower bound together, we have:
𝐸𝑇 𝑛 =Θ 𝑛 𝑙𝑔 𝑛
CSE 2331/5331

32. Example 6
CSE 2331/5331

33. Recurrence for Func6
𝐸𝑇(𝑛) = expected running time of 𝐹𝑢𝑛𝑐6 on 𝑛 values
CSE 2331/5331

34. Bounding ET(n)
CSE 2331/5331

35. Upper Bound
CSE 2331/5331

36. Lower Bound Whatever 𝑘 is, 𝐸𝑇(𝑘) is at least 𝑐𝑛. Hence 𝐸𝑇(𝑛) =Ω(𝑛)
Putting upper and lower bounds together, we have
𝐸𝑇(𝑛) =Θ(𝑛)
CSE 2331/5331