1. ICS 353: Design and Analysis of Algorithms
King Fahd University of Petroleum & Minerals
Information & Computer Science Department
ICS 353: Design and Analysis of Algorithms
Randomized Algorithms

2. Reading Assignment
M. Alsuwaiyel, Introduction to Algorithms: Design Techniques and Analysis, World Scientific Publishing Co., Inc
Chapter 14
Rajeev Motwani , Prabhakar Raghavan, Randomized algorithms, Cambridge University Press, New York, 1995

3. Introduction
Definition: A randomized algorithm is one that receives a stream of random bits that it can use in the course of its action for the purpose of making random choices, in addition to its input.
may give different results when applied to the same input in different runs.
the execution time may vary from one run to another when applied to the same input.
Does that remind you of something?

4. Main Advantages of Randomized Algorithms
Execution time or space requirement of a randomized algorithm is smaller than that of the best deterministic algorithm known for the same problem.
Randomized algorithms are extremely simple to comprehend and implement.

5. Example 1: Polynomial Equivalence
Given two polynomial expressions, f(x) and g(x) in n variables x = (x1, x2, …, xn), check whether f(x) = g(x).
Analytically:
Using a Randomized Algorithm (RA):
we generate a random n-vector r of real numbers and evaluate q (r) = f(r) - g(r).
If q(r) 0, we know that q  0.
If q(r) = 0, then either q is identically zero or we have been extremely lucky in our choice of r.
If we repeat this several times and keep on getting q = 0, then we conclude that q is identically zero. The probability that we have made an error is negligible.
Monte-Carlo: Find a witness that says: Yes the polynomials are different or halt without success and say: well, maybe they are equal!!!!!

6. Example 2: Randomized QuickSort
It has been shown that the average case time complexity of standard QuickSort is (n log n).
The worst case time complexity of standard QuickSort is O(n2)!
The time complexity of sorting a sorted array is……
Different ways to “eliminate” the worst case behavior
Introduce a preprocessing step that permutes the elements randomly before choosing the pivot.
Can be done in ( )
Select the pivot on which to split the array randomly.

7. Randomized QuickSort Algorithm
Algorithm 14.1 RandomizedQuickSort
Input: An array A[1..n] of n elements.
Output: The elements in A sorted in nondecreasing order.
1. rquicksort(1, n)
Procedure rquicksort(low, high)
1. if low < high then
2. v  random(low, high);
3. interchange A[low] and A[v];
w  split(A[low..high]);
rquicksort(low, w – 1);
6. rquicksort(w + 1, high);
7. end if

8. Complexity Analysis of Randomized Algorithms
The worst case time complexity of RandomizedQuickSort is O(n2)!!!!!!!
So, what does RA buy us?
In randomized algorithms, we look at the expected running time of the algorithm, instead of its worst case time complexity.
Worst-case expected running time.
Average-case expected running time.
What is the expected running time of RandomizedQuickSort?

9. Classes of Randomized Algorithms
Monte Carlo Algorithms
Always gives an answer, but may occasionally produce an incorrect answer
The probability of producing an incorrect answer can be made arbitrarily small by running the algorithm repeatedly with independent random choices in each run.
Note: The probability of a Monte Carlo algorithm to succeed after T(n) steps is independent of the input. It depends on the random choices made.

10. Classes of Randomized Algorithms (Cont.)
Las Vegas Algorithms
Either give a correct answer, or do not give an answer at all.

11. Occupancy Problems
Problem Statement: Given m identical balls and n identical bins, we want to place each ball in a bin independently and uniformly at random.
What is the expected number of bins with k balls?
What is the maximum number of balls in any bin?
What is the probability that one bin contains at least two balls?
How many ball are needed to be thrown before filling all the bins?

12. Number of Balls in Each Bin
Let Xi be the random variable that counts the number of balls in bin i.
Let
Clearly E(X) =
By linearity of expectation, E(Xi) =
What if m = n ?

13. Example
An instructor collects all homeworks from n students, shuffles the homeworks, and then returns them back to students. What is the expected number of students who will get their own homework back?
Exactly similar to the problem of the number of fixed points in a permutation of n numbers.

14. Number of Empty Bins Let (called indicator RV)
given bin i, a ball goes into a different bin with probability……………
Therefore, after throwing m balls,
Pr(Xi = 1) =
Question: How good is randomized linear search?

15. Balls Falling into the Same Bin
Let Ei be the event that bin i will contain more than one ball. To compute
The Birthday Paradox: What is the probability that there are two people in a group of m people who happen to have the same birthday?

16. Number of Balls to Fill All Bins
Geometric Distribution
Given a biased coin with probability p of HEADS.
Let Y = number of coin tosses before a HEAD appears for the first time.
E(Y) = 1 / p
Suppose we throw m balls at random into n bins, where throwing one ball at random is referred to as a trial.
A trial is called a success if the ball is thrown in an empty bin.
Xi = the number of trials until the ith success.
E(Xi) =
Let , what is the value of E(X)?

17. Random Sampling

18. Randomized Select