1. CS200: Algorithm Analysis

2. RANDOMIZED ALGORITHMS
General Quicksort:
Quicksort(A,p,r)
if p < r then
q = Partition(A,p,r)
Quicksort(A,p,q-1)
Quicksort(A,q+1,r)
Partition subdivides array so that the lower sub- array is <= the pivot element while the upper sub- array is > pivot.
Runtime : Worst-case = ? Occurs when array is ?

3. Best-case = ? Occurs when ?
Average-case = ? Occurs when all permutations of input are equally likely.
How can we avoid the worst-case runtime for Quicksort?

4. RANDOM METHODOLOGY
impose a random distribution on inputs so that all permutations are equally likely.
the run-time thereby becomes independent of the input order.
no specific input elicits worst-case behavior.
worst-case is determined only by the output of the random number generator.

5. Randomized Quicksort:
RandomizedPartition(A,p,r)
i = Random(p,r)
swap(A[i],A[r])
return(Partition(A,p,r)
RandomizedQuicksort(A,p,r)
if p < r then
q = RandomizedPartition(A,p,r)
RandomizedQuicksort(A,p,q-1)
RandomizedQuicksort(A,q+1,r)

6. Average case runtime recurrence for randomized quicksort
T(n) = 1/n( S n-1[T(k) +T(n-1-k)]) + Q(n)
k=0
Notice that for k = 0, 1, 2, 3,…, n-1, each term T(q) of the sum appears once as T(k) and once as T(n-1-k). These terms can be collapsed to get
= 2/n( S n-1[T(k)]) + Q(n) = O(nlogn)
k=1
Note: 2/n is dropped for k = 0. Please refer to text for solution to recurrence (on pages ). Solution requires an iterated sum. Not responsible for this solution but it does rely on next topic, which you should understand.

7. PROBABILITY THEORY- read Appendix C2 and C3
In order to do analysis of random algorithms some basic concepts of probability theory must be understood.
PROBABILITY: is defined over a sample space, S, that is a set of elementary events (elements).
i.e. S = {the set of 36 ways 2 dice can fall}, where each roll outcome is an elementary event.
What is the sample space of flipping 2 coins?
What is the sample space for choosing 1 letter at random from the word DIVIDE?
What is the sample space for choosing 1 jelly bean at random from a jar containing 5 red, 7 blue and 2 green jelly beans?
each elementary event is viewed as a possible outcome of an experiment where event A, is a subset of S.

8. i.e. A = {(1,6) (6,1) (3,4) (4,3) (2,5) (5,2)}, then A is the event of rolling a sum of 7 with 2 dice.
A probability distribution, Pr{} on S is a mapping from events of S to the real numbers ST the following probability axioms are satisfied.
1. Pr{A} >= 0 for any event A, where Pr{A} is the probability of event A occurring.
i.e. A = {(1,1)}, where A is the event of rolling a sum of 2 with 2 dice, then Pr{A}= 1/36 or |A|/36.
2. Pr{S} = 1, the probability of a CERTAIN event is taken as 1 for convenience.
3. Pr{A  B}= Pr{A} + Pr{B}; for any two mutually exclusive events, that is, A and B do not intersect.

9. i.e. A = rolling a sum of 2, B = rolling a sum of 7, then Pr{A  B}= 1/36 + 6/36 = 7/36.
Theorem : For any 2 events A and B that are not mutually exclusive:
Pr{A B} = Pr{A} + Pr{B} - Pr{A  B}
<= Pr{A} + Pr{B}
i.e. Pr{roll sum of 4 or doubles}
where Pr{roll 4} = {(2,2) (1,3) (3,1)} and
Pr{roll doubles} = {(1,1) (2,2) (3,3) (4,4) (5,5) (6,6)} thus Pr{roll sum of 4 or doubles} = 3/36 + 6/36 - {(2,2)} = 8/36

10. 15 minute activity – Probability Worksheet

11. Probability Distributions:
A probability distribution is discrete if S is a finite or countable infinite sample space. Let S be a sample space then for any event A
Pr{A} = S (Pr{s})
s e A
i.e. if A = {roll doubles} then s= {(2,2)} is an element of A = { (1,1) (2,2) (3,3) (4,4) (5,5) (6,6)}
{(2,2)} = s , Pr{s} = 1 / S => uniform probability distribution on S.
and Pr{A} =1/36 + 1/36 + 1/36 + 1/36 +1/36 + 1/36

12. DISCRETE RANDOM VARIABLES
A discrete random variable C is a function from a finite or countable infinite sample space to the real numbers.
X –> S x R, X is a mapping from a sample space S to the real numbers. X associates a real number x with each event in S.
for a random variable X and a real number x, the event X = x is defined as {s e S : X(s) = x}
i.e. Roll 2 dice then |S| = 36 possible outcomes.
Each element s of S has a uniform distribution, that is, each element has a probability of occurrence =1/|S| = 1/36.
Let X be the sum of dice, then Pr{X=5} = 4/36; {(1,4) (4,1) (2,3) (3,2)}

13. The expected value of a discrete random variable X is 
Expected Value of a Random Variable (average of the values taken on by a discrete random variable)
expectation, mean, average are synonymous.
The expected value of a discrete random variable X is 
E[X] = S (x * Pr{X=x})
x = 0

14. i.e. X = sum of dice. Thus E[X] = 252/36 = 7 the average value of X is 7. Note : E[X] = S (x * Pr{X=x})
x = 0
Sum of dice(x) Pr{X=x} x* Pr{X=x}
1 0/36 0/36
2 1/36 2/36
3 2/36 6/36
4 3/36 12/36
5 4/36 20/36
6 5/36 30/36
7 6/36 42/36
8 5/36 40/36
9 4/36 36/36
10 3/36 30/36
11 2/36 22/36
12 1/36 12/36

15. Apply Expectation to average case analysis of Linear Search
Assume searchVal is in A. Sample space is number of comparisons needed to find searchVal. What is X = x? What is the Pr{X=x}? Construct table? Expectation of comparisons?
Number of comparisons (x) Pr{X=x} x* Pr{X=x}
1 1/n 1/n
2 1/n 2/n
3 1/n 3/n
4 1/n 4/n
5 1/n 5/n
n 1/n n/n
What is E[ S (x * Pr{X=x})] ?
x = 1
1/n(Sn x ) = (n(n+1)/2n) = (n+1)/2
x=1

16. Expectations of discrete random variables are linear: for any 2 random variables X and Y and any constant a, E[aX + Y] = aE[X] + E[Y]
Let Z = number on die 1 and Y = number on die 2 and let X = sum of dice then X = Z + Y.
E[Z] = E[Y] =
(x) Prob x*Prob
1 1/6 1/6
2 1/6 2/6
3 1/6 3/6
4 1/6 4/6
5 1/6 5/6
6 1/6 6/6
21/6 = 3.5
E[X] = E[Z] + E[Y] = = 7

17. Independence If two random variables X and Y are independent then
for all x,y: Pr{X=x and Y=y} = Pr{X=x}*Pr{Y=y}
if X and Y are independent then E[XY] = E[X]*E[Y]
i.e. X = 5, Y = 6, dice values
Pr{X=5 and Y=6}= 1/6 * 1/6 = 1/36 (one of 36 possibilities).

18. Summary Know how to avoid worst case run time of QuickSort
Understand how average runtime recurrence for QuickSort is constructed.
Know basic probability theorems and their application
Be able to apply expectation, E[X], to compute average case runtime for simple algorithms