1. ASV Chapters 1 - Sample Spaces and Probabilities
2 - Conditional Probability and Independence
3 - Random Variables
4 - Approximations of the Binomial Distribution
5 - Transforms and Transformations
6 - Joint Distribution of Random Variables
7 - Sums and Symmetry
8 - Expectation and Variance in the Multivariate Setting
9 - Tail Bounds and Limit Theorems
10 - Conditional Distribution
11 - Appendix A, B, C, D, E, F

2. Discrete and Continuous
Recall that events can be described by random variables (from which their corresponding probabilities can be calculated).
Also recall that there are two types of numerical (versus categorical) random variable:
Discrete and Continuous

3. Discrete and Continuous
Recall that events can be described by random variables (from which their corresponding probabilities can be calculated).
Also recall that there are two types of numerical (versus categorical) random variable:
Discrete and Continuous

4. Random Variable X = “Value shown”
Example: Roll a fair die once.
Sample space
{1,2,3,4,5,6}
Random Variable X = “Value shown”
Discrete

5. “probability mass function” Probability Histogram
Example: Roll a fair die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
X is uniformly distributed over 1, 2, 3, 4, 5, 6.
“probability mass function”
pmf
Probability Histogram X
P(X = x) x
p(x) 1
1/6 2 3 4 5 6
Total Area = 1
“What is the probability of rolling a 4?”

6. “probability mass function” Probability Histogram
Example: Roll a fair die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
X is uniformly distributed over 1, 2, 3, 4, 5, 6.
“probability mass function”
pmf
Probability Histogram X
P(X = x) x
p(x) 1
1/6 2 3 4 5 6
Total Area = 1
“What is the probability of rolling a 4?”

7. “probability mass function” “Cumulative distribution function”
Example: Roll a fair die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
“probability mass function”
“Cumulative distribution function”
pmf
cdf
P(X = x) x
p(x) 1
1/6 2 3 4 5 6
P(X  x)
F(x)
1/6
2/6
3/6
4/6
5/6 1

8. pmf cdf Discrete Random Variable X = “Value shown” P(X = x) x p(x) 1
Example: Roll a fair die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
“probability mass function”
“Cumulative distribution function”
pmf
cdf
P(X = x) x
p(x) 1
1/6 2 3 4 5 6
P(X  x)
“jump discontinuities”
“piecewise constant”
F(x)
1/6
2/6
3/6
4/6
5/6 1
“staircase graph” from 0 to 1

9. Random Variable X = “Value shown”
Example: Roll a biased die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
0.20
0.30
0.15
0.10
0.05
pmf
P(X = x) x
p (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05

10. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
0.14
0.16
0.20
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

11. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
p (y) 2 3 4 5 6 7 8 9 10 11 12 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

12. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2 3 4 5 6 7 8 9 10 11 12 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

13. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) 3
(1,2), (2,1) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

14. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) 3
(1,2), (2,1) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

15. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

16. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

17. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

18. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1)
(0.20)(0.16) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

19. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1)
(0.20)(0.16) (0.30)(0.14) 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

20. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1) via disjoint
(0.20)(0.16) + (0.30)(0.14) = 0.074 4
(1,3), (2,2), (3,1) 5 . 6 7 8 9 10 11
(5,6), (6,5) 12
(6,6) 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

21. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
P(Y = y) y
Outcomes
p (y) 2
(1,1) via independence
(0.20)(0.14) = 0.028 3
(1,2), (2,1) via disjoint
(0.20)(0.16) + (0.30)(0.14) = 0.074 4
(1,3), (2,2), (3,1)
0.116 5 .
0.153 6
0.170 7
0.169 8
0.132 9
0.082 10
0.047 11
(5,6), (6,5)
0.022 12
(6,6)
0.007 1
pmf
pmf
P(X1 = x) x
p1 (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X2 = x)
p2 (x)
0.14
0.16
0.20 1

22. Probability Histogram
POPULATION
Pop vals x
pmf
p(x) x1
p(x1) x2
p(x2) x3
p(x3) ⋮
Total 1
Example: X = Cholesterol level (mg/dL)
random variable X
Discrete
Probability Histogram
“Density” X
p(x) = Probability that the random variable X is equal to a specific value x, i.e.,
p(x) = P(X = x)
“probability mass function” (pmf) | x

23. Probability Histogram
POPULATION
Pop vals x
pmf
p(x) x1
p(x1) x2
p(x2) x3
p(x3) ⋮
Total 1
Pop vals x
pmf
p(x)
cdf
F(x) = P(X  x) x1
p(x1)
F(x1) = p(x1) x2
p(x2)
F(x2) = p(x1) + p(x2) x3
p(x3)
F(x3) = p(x1) + p(x2) + p(x3) ⋮
Total 1
increases from 0 to 1 in a “staircase graph”
Example: X = Cholesterol level (mg/dL)
random variable X
Discrete
Probability Histogram X
F(x) = Probability that the random variable X is less than or equal to a specific value x, i.e.,
F(x) = P(X  x)
“cumulative distribution function” (cdf) | x

24. x1 p(x1) x2 p(x2) x3 p(x3) F(b) = P(X  b) F(a–) = P(X  a–)
POPULATION
Pop vals x
pmf
p(x)
cdf
F(x) = P(X  x) x1
p(x1)
F(x1) = p(x1) x2
p(x2)
F(x2) = p(x1) + p(x2) x3
p(x3)
F(x3) = p(x1) + p(x2) + p(x3) ⋮
Total 1
increases from 0 to 1 in a “staircase graph”
Example: X = Cholesterol level (mg/dL)
random variable X
Discrete
Calculating
“interval probabilities”… X
F(b) = P(X  b)
F(a–) = P(X  a–)
F(b) – F(a–) =
P(X  b) – P(X  a–)
= P(a  X  b)
p(x) | a– | a | b

25. FUNDAMENTAL THEOREM OF CALCULUS
POPULATION
Pop vals x
pmf
p(x)
cdf
F(x) = P(X  x) x1
p(x1)
F(x1) = p(x1) x2
p(x2)
F(x2) = p(x1) + p(x2) x3
p(x3)
F(x3) = p(x1) + p(x2) + p(x3) ⋮
Total 1
increases from 0 to 1 in a “staircase graph”
Example: X = Cholesterol level (mg/dL)
random variable X
Discrete
Calculating
“interval probabilities”… X
F(b) = P(X  b)
F(a–) = P(X  a–)
F(b) – F(a–) =
P(X  b) – P(X  a–)
FUNDAMENTAL THEOREM OF CALCULUS
(discrete form)
= P(a  X  b)
p(x) | a– | a | b

26. Random Variable X = “Value shown”
Example: Roll a biased die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
Method 1
pmf
P(X = x) x
p (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
0.20
0.30
0.15
0.10
0.05

27. Random Variable X = “Value shown”
Example: Roll a biased die once.
Sample space
{1,2,3,4,5,6}
Discrete
Random Variable X = “Value shown”
Method 2
pmf
cdf
P(X = x) x
p (x) 1
0.20 2
0.30 3 4
0.15 5
0.10 6
0.05
P(X  x)
F(x)
0.20
0.50
0.70
0.85
0.95 1
0.20
0.30
0.15
0.10
0.05

28. x1 p(x1) x2 p(x2) x3 p(x3) POPULATION x p(x) ⋮ Total 1 “balance point”
Pop vals x
pmf
p(x) x1
p(x1) x2
p(x2) x3
p(x3) ⋮
Total 1
Example: X = Cholesterol level (mg/dL)
random variable X
Discrete
“balance point”
Two issues…
“Measure of Spread”
How to define a “typical” distance of a random population value from this mean
“Measure of Center”
How to define a “typical” population value
Mean (of X):
Variance (of X):
= “expected value” or “expectation” of X
Moreover, we will prove that…
Standard deviation (of X):

29. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
0.14
0.16
0.20
pmf
pmf x
p1 (x)
p2 (x) 1
0.20
0.14 2
0.30
0.16 3 4
0.15 5
0.10 6
0.05

30. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
0.20
0.30
0.15
0.10
0.05
pmf
pmf x
p1 (x)
p2 (x) 1
0.20
0.14 2
0.30
0.16 3 4
0.15 5
0.10 6
0.05

31. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
0.20
0.30
0.15
0.10
0.05
pmf
pmf x
p1 (x)
p2 (x) 1
0.20
0.14 2
0.30
0.16 3 4
0.15 5
0.10 6
0.05

32. Random Variable Y = “Sum”
Example: Roll two dice, independently.
Sample space
{(1,1),…,(6,6)}
Discrete
Random Variable Y = “Sum”
0.20
0.30
0.15
0.10
0.05
pmf
pmf x
p1 (x)
p2 (x) 1
0.20
0.14 2
0.30
0.16 3 4
0.15 5
0.10 6
0.05

33. To summarize…

34. Probability Histogram
Probability Table
Probability Histogram X
Total Area = 1
POPULATION
Pop xi
Probabilities
pmf p(xi ) x1
p(x1) x2
p(x2) x3
p(x3) ⋮ 1
Discrete random variable X
Frequency Table
Density Histogram X
Total Area = 1
Data xi
Relative Frequencies
p(xi ) = fi /n x1
p(x1) x2
p(x2) x3
p(x3) ⋮ xk
p(xk) 1
SAMPLE of size n x1 x2 x3 x4 x5 x6
…etc…. xn

35. Probability Histogram
Probability Table
Probability Histogram X
Total Area = 1
POPULATION
Pop xi
Probabilities
pmf p(xi ) x1
p(x1) x2
p(x2) x3
p(x3) ⋮ 1 ?
Discrete random variable X
Continuous
Frequency Table
Density Histogram X
Total Area = 1
Data xi
Relative Frequencies
p(xi ) = fi /n x1
p(x1) x2
p(x2) x3
p(x3) ⋮ xk
p(xk) 1
SAMPLE of size n x1 x2 x3 x4 x5 x6
…etc…. xn