COMP 170 L2 L18: Random Variables: Independence and Variance Page 1

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 2

COMP 170 L2 Distribution Functions of RVs l P(X=k) viewed as a function of k: The distribution function of X. l In general Page 3

COMP 170 L2 Probability Weight and Distribution Function Page 4

COMP 170 L2 Distribution Functions of RVs Page 5

COMP 170 L2 Distribution Functions of RVs Page 6 l For coin example n D(1, 9) = P(1 <= X <=9 ) ~= 1

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 7

COMP 170 L2 Independence Between Events l E is independent of F If P(E|F) = P(E) n The information that “Event F occurred” does not change the probability of E

COMP 170 L2 Random Variable and Event l Given a rand variable, we can define many difference events Page 9

COMP 170 L2 Independence between RVs Page 10

COMP 170 L2 Independence between RVs Page 11

COMP 170 L2 Independence between RVs Page 12 l No need to further check n P(X=0, Y=1) ?= P(X=0)P(Y=1) n P(X=1, Y=0) ? = P(X=1)P(Y=0) n P(X=1, Y=1) ?= P(X=1)P(Y=1)

COMP 170 L2 Independence between RVs l Draw two cards from a deck of 52 cards n X: number on first card n Y: number on second card n X and Y are independent when drawing with replacement n X and Y are not independent when drawing without replacement Page 13

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 14

COMP 170 L2 Independence and Expectation Page 15

COMP 170 L2 Independence and Expectation Page 16

COMP 170 L2 Page 17

COMP 170 L2 Page 18

COMP 170 L2 Illustrating Proof via Example Page 19

COMP 170 L2 Illustrating Proof via Example Page 20

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Sum of independent RVs n The Trend n Central limit theorem Page 21

COMP 170 L2 Variance of RVs l Probability starts with a process whose outcome is uncertain l Sample space: the set of all possible outcomes l A random variable (RV) is a function defined on the sample space l Different runs of the process might yield different outcomes l The RV might take different values in different runs n In other words, the value of RV varies across different runs l Some RVs vary more and some vary less n Number of heads in 1 coin flip n Number of heads in 10 coin flips n Number of heads in 100 coin flips Page 22

COMP 170 L2 Variance of RVs l Variance of an RV X n Measures how much it varies (across different runs of process) n Relative to the mean value Page 23

COMP 170 L2 Page 24

COMP 170 L2 Page 25

COMP 170 L2 Which RV vary the most? Flip fair coin l Number of heads in 1 flip l Number of heads in 10 flips l Number of heads in 100 flips Page 26 l Variance 1/4 10/4 100/4

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 27

COMP 170 L2 Calculating Variance Page 28

COMP 170 L2 Example 1: We have already seen l Number of heads in n flips n n=1 n n=10 (x1+X2+…+X10) n n=100 (X1+X2+…+X100) Page 29 l Variance 1/4 10/4 = 10 * 1/4 100/4 = 100 * 1/4 l Additivity is true here.

COMP 170 L2 Page 30 l Additivity is true here.

COMP 170 L2 Example 3 (Counter Example) Page 31

COMP 170 L2 Example 3 (Counter Example) Page 32

COMP 170 L2 Page 33

COMP 170 L2 Two Lemmas Page 34

COMP 170 L2 Page 35

COMP 170 L2 Page 36

COMP 170 L2 Page 37 An Application of Theorem 5.29

COMP 170 L2 A Corollary Page 38

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 39

COMP 170 L2 Standard Deviation l Standard deviation is another measure of how much a rv varies or how much a distribution spread out l It is the square root of variance. Page 40 l Next page l Shows several distributions, variances and standard deviations l Highlights the differences between variance and standard deviation

COMP 170 L2 Distributions, Variances, and Standard Deviations Page 41

COMP 170 L2 l The examples on the previous page show n Standard deviation is a natural measure of “spread” of distribution l Variance is easier to manipulate mathematically. Will see this in further study of probability theory and statistics. Page 42 Distributions, Variances, and Standard Deviations

COMP 170 L2 Outline l Independence of RVs n Distribution Functions of RVs n Independence between RVs n Expectation of product of independent RVs l Variance of RV n Definition and Examples n Additivity n Standard deviation l Central limit theorem Page 43

COMP 170 L2 A Pattern l If we flip of a coin a large number of times, n “The number of heads” has bell-shaped distribution. l This phenomenon is not unique to coin flipping Page 44

COMP 170 L2 Page 45 Test with.8 probability of getting correct answer

COMP 170 L2 Page 46

COMP 170 L2 Page 47

COMP 170 L2 Normal Distribution Page 48

COMP 170 L2 Page 49

COMP 170 L2 Page 50

COMP 170 L2 Recap:

COMP 170 L2 Recap: l E is independent of F If P(E|F) = P(E) n The information that “Event F occurred” does not change the probability of E

COMP 170 L2 Recap: Flip fair coin l Number of heads in 1 flip: Variance = 1/4 l Number of heads in 10 flips: Variance = 10/4 l Number of heads in 100 flips: Variance = 100/4

COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 1? n P(getting 1 at each throw) = 1/6 n Answer: 1/(1/6) = 6 Page 54

COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 2 and 3 in that order? n Expected number of throws to see 2: 6 n After that, expected number of throws to see 3: 6 n Answer: 6+6 = 12 Page 55

COMP 170 L2 Number of times until first success l Throw a fair die n How many times, on average, do you need to throw the die until you see a 2 and 3, where the order does not matter? n P( get 2 or 3 in one throw ) = 1/3 n Expected number throws until you see one of 2 or 3: 3 n After that, P( get the other number in each throw) = 1/6 n Expected number of throws until you see the other number: 6 n Answer: 3+6 = 9 Page 56

COMP 170 L2 Old Exam Question 1 Page 57 P( at least one copy of T1 in 20 weeks) = 1 – P( no copy of T1 in 20 weeks) = 1 – P ( no T1 in week1 AND no T1 in week 2 AND …) = 1- P(no T1 in week1) P(no T1 in week 2) … =

COMP 170 L2

l Can we do this? P(all 10 types of toys in 20 weeks) = P(copy of T1 in 20 weeks AND copy of T2 in 20 weeks AND …) = P(copy of T1 in 20 weeks) P(copy of T2 in 20 weeks)… NO, events not independent l Correct way Page 59 P(all 10 types of toys in 20 weeks) = 1 – P(A) Use inclusion-exclusion to calculate P(A)

COMP 170 L2

Page 61 Old Exam Question 2

COMP 170 L2