1. Lec Nov 12
Probability – definitions and simulation

2. (Discrete) Sample space
Experiment: a physical act such as tossing a coin or rolling a die.
Sample space – set of outcomes.
Coin toss Sample Space S = { head, tail}
Rolling a die Sample space S = {1, 2, 3, 4, 5, 6}
Tossing a coin twice.
Sample space S = {(h,h), (h,t), (t,h), (t,t)}

3. Events and probability
Event E is any subset of sample space S.
You flip 2 coins
Sample space S = {(h,h), (h,t), (t,h), (t,t)}
Event: both tosses produce same result
E = {(h,h), (t,t)}
Prob(E) = |E|/ |S|
In the above example, p(E) = 2/4 = 0.5
Question: what is the probability of getting at least one six in three roles of a die?

4. Bernoulli trial
Bernoulli trials are experiments with two outcomes. (success with prob = p and failure with prob = 1 – p.)
Example: rolling an unloaded die. Success is defined as getting a role of 1.
p(success) = 1/6

5. Random Variable
Random variable (RV) is a function that maps the sample space to a number.
E.g. the total number of heads X you get if you flip 100 coins
Another example:
Keep tossing a coin until you get a head. The RV n is the number of tosses.
Event = { H, TH, TTH, TTTH, … }
RV n(H) = 1, n(TH) = 2, n(TTH) = 3, … etc.

6. Common Distributions Uniform X: U[1, N] Binomial distribution
X takes values 1, 2, …, N
E.g. picking balls of different colors from a box
Binomial distribution
X takes values 0, 1, …, n
N coin tosses. What is the prob. That there are exactly k tails?

7. Conditional Probability
P(A|B) is the probability of event A given that B has occurred.
Suppose 6 coins are tossed. Given that there is at least one head, what is the probability that the number of heads is 3?
Definition:
p(A|B) =

8. Baye’s Rule If X and Y are events, then p(X|Y) = p(Y|X) p(X)/p(Y)
Useful in situation where p(X), p(Y) and p(Y|X) are easier to compute than p(X|Y).

9. Independent events
Definition: X and Y are independent if

10. Monty Hall Problem
You're given the choice of three doors: Behind one door is a car; behind the others, goats. You want to pick the car.
You pick a door, say No. 1
The host, who knows what's behind the doors, opens another door, say No. 3, which has a goat.
Do you want to pick door No. 2 instead?

11. Host reveals Goat A or Host reveals Goat B
                        
Host must reveal Goat B                         
Host must reveal Goat A                        
                             
Host reveals Goat A or Host reveals Goat B
           
             

12. Monty Hall Problem: Bayes Rule
: the car is behind door i, i = 1, 2, 3
: the host opens door j after you pick door i

13. Monty Hall Problem: Bayes Rule continued
WLOG, i=1, j=3

14. Monty Hall Problem: Bayes Rule continued

15. Monty Hall Problem: Bayes Rule continued
You should switch!

16. Continuous Random Variables
What if X is continuous?
Probability density function (pdf) instead of probability mass function (pmf)
A pdf is any function that describes the probability density in terms of the input variable x.

17. Probability Density Function
Properties of pdf
Actual probability can be obtained by taking the integral of pdf
E.g. the probability of X being between 0 and 1 is

18. Cumulative Distribution Function
Discrete RVs
Continuous RVs

19. Common Distributions
Normal
E.g. the height of the entire population

20. Moments
Mean (Expectation):
Discrete RVs:
Continuous RVs:
Variance:

21. Properties of Moments
Mean
If X and Y are independent,
Variance

22. Moments of Common Distributions
Uniform
Mean ; variance
Binomial
Mean ; variance
Normal
Mean ; variance

23. Simulating events by Matlab programs
Write a program in Matlab to distribute the 52 cards of a deck to 4 people, each getting 13 cards. All the choices must be equally likely.
One way to do this is as follows: map each card to a number 1, 2, …, 52. Generate a random permutation of the array a[1 2 … 52], then give the cards a[1:13] to first player, a[14:26] to second player etc.

24. Random permutation generation
We can use ceil(rand()*n) to generate a random number from the set {1, 2, …, n}.
Algorithm generate a random permutation:
1. Start with array a = [ 1 2 … n]
2. For j = n: -1: 1
randomly pick a number r in [1..j]. Switch a[r] and a[j]
3. Output a.