1. RANDOM VARIABLES, EXPECTATIONS, VARIANCES ETC. 1

2. Variable Recall: Variable: A characteristic of population or sample that is of interest for us. Random variable: A function defined on the sample space S that associates a real number with each outcome in S. 2

3. DISCRETE RANDOM VARIABLES If the set of all possible values of a r.v. X is a countable set, then X is called discrete r.v. The function f(x)=P(X=x) for x=x 1,x 2, … that assigns the probability to each value x is called probability density function (p.d.f.) or probability mass function (p.m.f.) 3

4. Example Discrete Uniform distribution: Example: throw a fair die. P(X=1)=…=P(X=6)=1/6 4

5. CONTINUOUS RANDOM VARIABLES When sample space is uncountable (continuous) Example: Continuous Uniform(a,b) 5

6. CUMULATIVE DENSITY FUNCTION (C.D.F.) CDF of a r.v. X is defined as F(x)=P(X≤x). Note that, P(a<X ≤b)=F(b)-F(a). A function F(x) is a CDF for some r.v. X iff it satisfies 6 F(x) is continuous from right F(x) is non-decreasing.

7. Example Consider tossing three fair coins. Let X=number of heads observed. S={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH} P(X=0)=P(X=3)=1/8; P(X=1)=P(X=2)=3/8 7 xF(x) (-∞,0)0 [0,1)1/8 [1,2)1/2 [2,3)7/8 [3, ∞)1

8. Example Let 8

9. JOINT DISTRIBUTIONS In many applications there are more than one random variables of interest, say X 1, X 2,…,X k. JOINT DISCRETE DISTRIBUTIONS The joint probability mass function (joint pmf) of the k -dimensional discrete rv X =( X 1, X 2,…,X k ) is 9

10. JOINT DISCRETE DISTRIBUTIONS A function f(x 1, x 2,…, x k ) is the joint pmf for some vector valued rv X =( X 1, X 2,…,X k ) iff the following properties are satisfied: f(x 1, x 2,…, x k )  0 for all (x 1, x 2,…, x k ) and 10

11. Example Tossing two fair dice  36 possible sample points Let X: sum of the two dice; Y: |difference of the two dice| For e.g.: – For (3,3), X=6 and Y=0. – For both (4,1) and (1,4), X=5, Y=3. 11

12. Example Joint pmf of (x,y) 12 x y 23456789101112 01/36 11/18 2 3 4 5 Empty cells are equal to 0. e.g. P(X=7,Y≤4)=f(7,0)+f(7,1)+f(7,2)+f(7,3)+f(7,4)=0+1/18+0+1/18+0=1/9

13. MARGINAL DISCRETE DISTRIBUTIONS If the pair (X 1,X 2 ) of discrete random variables has the joint pmf f(x 1,x 2 ), then the marginal pmfs of X 1 and X 2 are 13

14. Example In the previous example, – 14

15. JOINT DISCRETE DISTRIBUTIONS JOINT CDF: F(x 1,x 2 ) is a cdf iff 15

16. JOINT CONTINUOUS DISTRIBUTIONS A k -dimensional vector valued rv X =( X 1, X 2,…,X k ) is said to be continuous if there is a function f(x 1, x 2,…, x k ), called the joint probability density function (joint pdf), of X, such that the joint cdf can be given as 16

17. JOINT CONTINUOUS DISTRIBUTIONS A function f(x 1, x 2,…, x k ) is the joint pdf for some vector valued rv X =( X 1, X 2,…,X k ) iff the following properties are satisfied: f(x 1, x 2,…, x k )  0 for all (x 1, x 2,…, x k ) and 17

18. JOINT CONTINUOUS DISTRIBUTIONS If the pair (X 1,X 2 ) of discrete random variables has the joint pdf f(x 1,x 2 ), then the marginal pdfs of X 1 and X 2 are 18

19. JOINT DISTRIBUTIONS If X 1, X 2,…,X k are independent from each other, then the joint pdf can be given as And the joint cdf can be written as 19

20. CONDITIONAL DISTRIBUTIONS If X 1 and X 2 are discrete or continuous random variables with joint pdf f(x 1,x 2 ), then the conditional pdf of X 2 given X 1 =x 1 is defined by For independent rvs, 20

21. Example Statistical Analysis of Employment Discrimination Data (Example from Dudewicz & Mishra, 1988; data from Dawson, Hankey and Myers, 1982) 21 % promoted (number of employees) Pay gradeAffected classothers 5100 (6)84 (80) 788 (8)87 (195) 993 (29)88 (335) 107 (102)8 (695) 117 (15)11 (185) 1210 (10)7 (165) 130 (2)9 (81) 140 (1)7 (41) Affected class might be a minority group or e.g. women

22. Example, cont. Does this data indicate discrimination against the affected class in promotions in this company? Let X=(X 1,X 2,X 3 ) where X 1 is pay grade of an employee; X 2 is an indicator of whether the employee is in the affected class or not; X 3 is an indicator of whether the employee was promoted or not x1={5,7,9,10,11,12,13,14}; x2={0,1}; x3={0,1} 22

23. Example, cont. E.g., in pay grade 10 of this occupation (X 1 =10) there were 102 members of the affected class and 695 members of the other classes. Seven percent of the affected class in pay grade 10 had been promoted, that is (102)(0.07)=7 individuals out of 102 had been promoted. Out of 1950 employees, only 173 are in the affected class; this is not atypical in such studies. 23 Pay gradeAffected classothers 107 (102)8 (695)

24. Example, cont. E.g. probability of a randomly selected employee being in pay grade 10, being in the affected class, and promoted: P(X 1 =10,X 2 =1,X 3 =1)=7/1950=0.0036 (Probability function of a discrete 3 dimensional r.v.) E.g. probability of a randomly selected employee being in pay grade 10 and promoted: P(X 1 =10, X 3 =1)= (7+56)/1950=0.0323 (Note: 8% of 695 - > 56) (marginal probability function of X 1 and X 3 ) 24 Pay gradeAffected classothers 107 (102)8 (695)

25. Example, cont. E.g. probability that an employee is in the other class (X 2 =0) given that the employee is in pay grade 10 (X 1 =10) and was promoted (X 3 =1): P(X 2 =0| X 1 =10, X 3 =1)= P(X 1 =10,X 2 =0,X 3 =1)/P(X 1 =10, X 3 =1) =(56/1950)/(63/1950)=0.89 (conditional probability) probability that an employee is in the affected class (X 2 =1) given that the employee is in pay grade 10 (X 1 =10) and was promoted (X 3 =1): P(X 2 =1| X 1 =10, X 3 =1)=(7/1950)/(63/1950)=0.11 25

26. Production problem Two companies manufacture a certain type of sophisticated electronic equipment for the government; to avoid the lawsuits lets call them C and company D. In the pact, company C has had 5% good output, whereas D had 50% good output (i.e., 95% of C’s output and 50% of D’s output is not of acceptable quality). The government has just ordered 10,100 of these devices from company D and 11,000 from C (maybe political reasons, maybe company D does not have a large enough capacity for more orders). Before the production of these devices start, government scientists develop a new manufacturing method that they believe will almost double the % of good devices received. Companies C and D are given this info, but its use is optional: they must each use this new method for at least 100 of their devices, but its use beyond that point is left to their discretion.

27. Production problem, cont. When the devices are received and tested, the following table is observed: Officials blame scientists and companies for producing with the lousy new method which is clearly inferior. Scientists still claim that the new method has almost doubled the % of good items. Which one is right? Production method StandardNew ResultsBad59509005 Good5050 (46%)1095 (11%)

28. Production problem, cont. Answer: the scientists rule! The new method nearly doubled the % of good items for both companies. Company D knew their production under standard method is already good, so they used the new item for only minimum allowed. This is called Simpson’s paradox. Do not combine the results for 2 companies in such cases. Company CD StandardNewStandardNew ResultsBad950900050005 Good50 (5%)1000 (10%)5000 (50%)95 (95%)

29. 29 Describing the Population We’re interested in describing the population by computing various parameters. For instance, we calculate the population mean and population variance.

30. 30 EXPECTED VALUES Let X be a rv with pdf f X (x) and g(X) be a function of X. Then, the expected value (or the mean or the mathematical expectation) of g(X) providing the sum or the integral exists, i.e.,  <E[g(X)]< .

31. 31 EXPECTED VALUES E[g(X)] is finite if E[| g(X) |] is finite.

32. 32 Population Mean (Expected Value) Given a discrete random variable X with values x i, that occur with probabilities p(x i ), the population mean of X is

33. 33 – Let X be a discrete random variable with possible values x i that occur with probabilities p(x i ), and let E(x i ) =  The variance of X is defined by Population Variance Unit*Unit Unit

34. 34 EXPECTED VALUE The expected value or mean value of a continuous random variable X with pdf f(x) is The variance of a continuous random variable X with pdf f(x) is

35. 35 EXAMPLE The pmf for the number of defective items in a lot is as follows Find the expected number and the variance of defective items.

36. 36 EXAMPLE Let X be a random variable. Its pdf is f(x)=2(1-x), 0< x < 1 Find E(X) and Var(X).

37. 37 Laws of Expected Value Let X be a rv and a, b, and c be constants. Then, for any two functions g 1 (x) and g 2 (x) whose expectations exist,

38. 38 Laws of Expected Value  E(c) = c  E( X + c) = E( X ) + c  E(c X ) = cE( X ) Laws of Variance  V(c) = 0  V( X + c) = V( X )  V(c X ) = c 2 V( X ) Laws of Expected Value and Variance Let X be a rv and c be a constant.

39. EXPECTED VALUE 39 If X and Y are independent, The covariance of X and Y is defined as

40. EXPECTED VALUE 40 If X and Y are independent, The reverse is usually not correct! It is only correct under normal distribution. If (X,Y)~Normal, then X and Y are independent iff Cov(X,Y)=0

41. EXPECTED VALUE 41 If X 1 and X 2 are independent,

42. CONDITIONAL EXPECTATION AND VARIANCE 42

43. CONDITIONAL EXPECTATION AND VARIANCE 43 (EVVE rule) Proofs available in Casella & Berger (1990), pgs. 154 & 158

44. Example An insect lays a large number of eggs, each surviving with probability p. Consider a large number of mothers. X: number of survivors in a litter; Y: number of eggs laid Assume: Find: expected number of survivors, i.e. E(X) 44

45. Example - solution EX=E(E(X|Y)) =E(Yp) =p E(Y) =p E(E(Y|Λ)) =p E(Λ) =pβ 45

46. 46 SOME MATHEMATICAL EXPECTATIONS Population Mean:  = E( X ) Population Variance: (measure of the deviation from the population mean) Population Standard Deviation: Moments:

47. 47 SKEWNESS Measure of lack of symmetry in the pdf. If the distribution of X is symmetric around its mean ,  3 =0  Skewness=0

48. 48 KURTOSIS Measure of the peakedness of the pdf. Describes the shape of the distribution. Kurtosis=3  Normal Kurtosis >3  Leptokurtic (peaked and fat tails) Kurtosis<3  Platykurtic (less peaked and thinner tails)

49. KURTOSIS What is the range of kurtosis? Claim: Kurtosis ≥ 1. Why? Proof: 49

50. 50 Measures of Central Location Usually, we focus our attention on two types of measures when describing population characteristics: –Central location –Variability or spread

51. 51 With one data point clearly the central location is at the point itself. Measures of Central Location The measure of central location reflects the locations of all the data points. How? But if the third data point appears on the left hand-side of the midrange, it should “pull” the central location to the left. With two data points, the central location should fall in the middle between them (in order to reflect the location of both of them).

52. 52 Sum of the observations Number of observations Mean = This is the most popular measure of central location The Arithmetic Mean

53. 53 Sample meanPopulation mean Sample sizePopulation size The Arithmetic Mean

54. 54 Example The reported time on the Internet of 10 adults are 0, 7, 12, 5, 33, 14, 8, 0, 9, 22 hours. Find the mean time on the Internet. 0 0 7 7 22 11.0 The Arithmetic Mean

55. 55 The Arithmetic Mean Drawback of the mean: It can be influenced by unusual observations, because it uses all the information in the data set.

56. 56 Odd number of observations 0, 0, 5, 7, 8 9, 12, 14, 22 0, 0, 5, 7, 8, 9, 12, 14, 22, 33 Even number of observations Example Find the median of the time on the internet for the 10 adults of previous example The Median of a set of observations is the value that falls in the middle when the observations are arranged in order of magnitude. It divides the data in half. The Median Suppose only 9 adults were sampled (exclude, say, the longest time (33)) Comment 8.5, 8

57. The Median Depth of median = (n+1)/2 57

58. 58 The Mode of a set of observations is the value that occurs most frequently. Set of data may have one mode (or modal class), or two or more modes. The modal class The Mode

59. 59 Find the mode for the data in the Example. Here are the data again: 0, 7, 12, 5, 33, 14, 8, 0, 9, 22 Solution All observation except “0” occur once. There are two “0”s. Thus, the mode is zero. Is this a good measure of central location? The value “0” does not reside at the center of this set (compare with the mean = 11.0 and the median = 8.5). The Mode

60. 60 Relationship among Mean, Median, and Mode If a distribution is from a bell shaped symmetrical one, the mean, median and mode coincide If a distribution is asymmetrical, and skewed to the left or to the right, the three measures differ. A positively skewed distribution (“skewed to the right”) Mean Median Mode Mean = Median = Mode Mode < Median < Mean

61. 61 If a distribution is non symmetrical, and skewed to the left or to the right, the three measures differ. A positively skewed distribution (“skewed to the right”) Mean Median Mode Mean Median Mode A negatively skewed distribution (“skewed to the left”) Relationship among Mean, Median, and Mode Mean < Median < Mode

62. 62 Measures of variability Measures of central location fail to tell the whole story about the distribution. A question of interest still remains unanswered: How much are the observations spread out around the mean value?

63. 63 Measures of variability Observe two hypothetical data sets: The average value provides a good representation of the observations in the data set. Small variability This data set is now changing to...

64. 64 Measures of Variability Observe two hypothetical data sets: The average value provides a good representation of the observations in the data set. Small variability Larger variability The same average value does not provide as good representation of the observations in the data set as before.

65. 65 – The range of a set of observations is the difference between the largest and smallest observations. – Its major advantage is the ease with which it can be computed. – Its major shortcoming is its failure to provide information on the dispersion of the observations between the two end points. ? ? ? But, how do all the observations spread out? Smallest observation Largest observation The range cannot assist in answering this question Range The Range

66. 66 l This measure reflects the dispersion of all the observations l The variance of a population of size N x 1, x 2,…,x N whose mean is  is defined as l The variance of a sample of n observations x 1, x 2, …,x n whose mean is is defined as The Variance

67. 67 Why not use the sum of deviations? Consider two small populations: 10 98 74 1112 1316 8-10= -2 9-10= -1 11-10= +1 12-10= +2 4-10 = - 6 7-10 = -3 13-10 = +3 16-10 = +6 Sum = 0 The mean of both populations is 10... …but measurements in B are more dispersed than those in A. A measure of dispersion Should agrees with this observation. Can the sum of deviations Be a good measure of dispersion? A B The sum of deviations is zero for both populations, therefore, is not a good measure of dispersion.

68. 68 Let us calculate the variance of the two populations Why is the variance defined as the average squared deviation? Why not use the sum of squared deviations as a measure of variation instead? After all, the sum of squared deviations increases in magnitude when the variation of a data set increases!! The Variance

69. 69 Which data set has a larger dispersion? 131 32 5 AB Data set B is more dispersed around the mean Let us calculate the sum of squared deviations for both data sets The Variance

70. 70 13 1 3 2 5 AB Sum A = (1-2) 2 +…+(1-2) 2 +(3-2) 2 + … +(3-2) 2 = 10 Sum B = (1-3) 2 + (5-3) 2 = 8 Sum A > Sum B. This is inconsistent with the observation that set B is more dispersed. The Variance

71. 71 13 1 3 2 5 AB However, when calculated on “per observation” basis (variance), the data set dispersions are properly ranked.  A 2 = Sum A /N = 10/5 = 2  B 2 = Sum B /N = 8/2 = 4 The Variance

72. 72 Example – The following sample consists of the number of jobs six students applied for: 17, 15, 23, 7, 9, 13. Find its mean and variance Solution The Variance

73. 73 The Variance – Shortcut method

74. 74 The standard deviation of a set of observations is the square root of the variance. Standard Deviation

75. 75 Example – To examine the consistency of shots for a new innovative golf club, a golfer was asked to hit 150 shots, 75 with a currently used (7-iron) club, and 75 with the new club. – The distances were recorded. – Which club is better? Standard Deviation

76. 76 Example – solution Example Standard Deviation Excel printout, from the “Descriptive Statistics” sub- menu. The innovation club is more consistent, and because the means are close, is considered a better club

77. 77 Interpreting Standard Deviation The standard deviation can be used to – compare the variability of several distributions – make a statement about the general shape of a distribution. The empirical rule: If a sample of observations has a mound-shaped distribution, the interval

78. 78 Example A practitioner wants to describe the way returns on investment are distributed. – The mean return = 10% – The standard deviation of the return = 8% – The histogram is bell shaped. Interpreting Standard Deviation

79. 79 Example – solution The empirical rule can be applied (bell shaped histogram) Describing the return distribution – Approximately 68% of the returns lie between 2% and 18% [10 – 1(8), 10 + 1(8)] – Approximately 95% of the returns lie between -6% and 26% [10 – 2(8), 10 + 2(8)] – Approximately 99.7% of the returns lie between -14% and 34% [10 – 3(8), 10 + 3(8)] Interpreting Standard Deviation

80. 80 For any value of k  1, greater than 100(1-1/k 2 )% of the data lie within the interval from to. This theorem is valid for any set of measurements (sample, population) of any shape!! kIntervalChebyshev Empirical Rule 1at least 0% approximately 68% 2at least 75% approximately 95% 3at least 89% approximately 99.7% The Chebyshev’s Theorem (1-1/1 2 ) (1-1/2 2 ) (1-1/3 2 )

81. 81 Example – The annual salaries of the employees of a chain of computer stores produced a positively skewed histogram. The mean and standard deviation are $28,000 and $3,000,respectively. What can you say about the salaries at this chain? Solution At least 75% of the salaries lie between $22,000 and $34,000 28000 – 2(3000) 28000 + 2(3000) At least 88.9% of the salaries lie between $19,000 and $37,000 28000 – 3(3000) 28000 + 3(3000) The Chebyshev’s Theorem

82. 82 The coefficient of variation of a set of measurements is the standard deviation divided by the mean value. This coefficient provides a proportionate measure of variation. A standard deviation of 10 may be perceived large when the mean value is 100, but only moderately large when the mean value is 500 The Coefficient of Variation

83. Percentiles Example from http://www.ehow.com/how_2310404_calculate-percentiles.htmlhttp://www.ehow.com/how_2310404_calculate-percentiles.html Your test score, e.g. 70%, tells you how many questions you answered correctly. However, it doesn’t tell how well you did compared to the other people who took the same test. If the percentile of your score is 75, then you scored higher than 75% of other people who took the test. 83

84. 84 Sample Percentiles and Box Plots Percentile – The pth percentile of a set of measurements is the value for which p percent of the observations are less than that value 100(1-p) percent of all the observations are greater than that value.

85. 85 Sample Percentiles Find the 10 percentile of 6 8 3 6 2 8 1 Order the data: 1 2 3 6 6 8 8 7*(0.10) = 0.70; round up to 1 The first observation, 1, is the 10 percentile.

86. 86 Commonly used percentiles – First (lower) quartile, Q 1 = 25th percentile – Second (middle) quartile,Q 2 = 50th percentile – Third quartile, Q 3 = 75th percentile – Fourth quartile, Q 4 = 100th percentile – First (lower) decile= 10th percentile – Ninth (upper) decile = 90th percentile

87. 87 Quartiles and Variability Quartiles can provide an idea about the shape of a histogram Q 1 Q 2 Q 3 Positively skewed histogram Q 1 Q 2 Q 3 Negatively skewed histogram

88. 88 Large value indicates a large spread of the observations Interquartile range = Q 3 – Q 1 Interquartile Range

89. 89 1.5(Q 3 – Q 1 ) – This is a pictorial display that provides the main descriptive measures of the data set: L - the largest observation Q 3 - The upper quartile Q 2 - The median Q 1 - The lower quartile S - The smallest observation SQ1Q1 Q2Q2 Q3Q3 L Whisker Box Plot

90. 90 Box Plot – The following data give noise levels measured at 36 different times directly outside of Grand Central Station in Manhattan. 10775 75-1.5(IQR)=27 107+1.5(IQR) =155

91. 91 – Interpreting the box plot results The scores range from 60 to 125. About half the scores are smaller than 90, and about half are larger than 90. About half the scores lie between 75 and 107. About a quarter lies below 75 and a quarter above 107. Data is slightly positively skewed. Q 1 75 Q 2 90 Q 3 107 25%50% 25% 60125 Box Plot NOISE - continued

92. 92 Wendy’s service time appears to be the shortest on average and most consistent. Hardee’s service time variability is the largest Jack in the box is the slowest in service Box Plot Jack in the Box Hardee’s McDonalds Wendy’s Popeyes Example: A study was organized to compare the service time in 5 drive through restaurants.

93. 93 Popeyes Wendy’s Hardee’s Jack in the Box Wendy’s service time appears to be the shortest and most consistent. McDonalds Hardee’s service time variability is the largest Jack in the box is the slowest in service Box Plot Times are positively skewed Times are symmetric

94. 94 Paired Data Sets and the Sample Correlation Coefficient The covariance and the coefficient of correlation are used to measure the direction and strength of the linear relationship between two variables. – Covariance - is there any pattern to the way two variables move together? – Coefficient of correlation - how strong is the linear relationship between two variables

95. 95  x (  y ) is the population mean of the variable X (Y). N is the population size. Covariance x (y) is the sample mean of the variable X (Y). n is the sample size.

96. 96 If the two variables move in opposite directions, (one increases when the other one decreases), the covariance is a large negative number. If the two variables are unrelated, the covariance will be close to zero. If the two variables move in the same direction, (both increase or both decrease), the covariance is a large positive number. Covariance

97. 97 Compare the following three sets Covariance xixi yiyi (x – x)(y – y)(x – x)(y – y) 267267 13 20 27 -3 1 2 -7 0 7 21 0 14 x=5y =20Cov(x,y)=17.5 xixi yiyi (x – x)(y – y)(x – x)(y – y) 267267 27 20 13 -3 1 2 7 0 -7 -21 0 -14 x=5y =20Cov(x,y)=-17.5 xixi yiyi 267267 20 27 13 Cov(x,y) = -3.5 x=5y =20

98. 98 – This coefficient answers the question: How strong is the association between X and Y. The coefficient of correlation

99. 99 COV(X,Y)=0  or r = +1 0 Strong positive linear relationship No linear relationship Strong negative linear relationship or COV(X,Y)>0 COV(X,Y)<0 The coefficient of correlation

100. 100 If the two variables are very strongly positively related, the coefficient value is close to +1 (strong positive linear relationship). If the two variables are very strongly negatively related, the coefficient value is close to -1 (strong negative linear relationship). No straight line relationship is indicated by a coefficient close to zero. The Coefficient of Correlation

101. 101

102. Correlation and causation Recognize the difference between correlation and causation — just because two things occur together, that does not necessarily mean that one causes the other. For random processes, causation means that if A occurs, that causes a change in the probability that B occurs. 102

103. Correlation and causation Existence of a statistical relationship, no matter how strong it is, does not imply a cause-and-effect relationship between X and Y. for ex, let X be size of vocabulary, and Y be writing speed for a group of children. There most probably be a positive relationship but this does not imply that an increase in vocabulary causes an increase in the speed of writing. Other variables such as age, education etc will affect both X and Y. Even if there is a causal relationship between X and Y, it might be in the opposite direction, i.e. from Y to X. For eg, let X be thermometer reading and let Y be actual temperature. Here Y will affect X. 103

104. Example Dr. Leonard Eron, professor at the University of Illinois at Chicago, has conducted a longitudinal study of the long–term effects of violent television programming. In 1960, he asked 870 third grade children their favorite television shows. He found that children judged most violent by their peers also watched the most violent television. Dr. Eron noted, however, that it was not clear which came first — the child’s behavior or the influence of television. In follow-up interviews at ten–year intervals, Eron found that youngsters who at age eight were nonaggressive but were watching violent television were more aggressive than children who at age eight were aggressive and watched non–violent television. Eron claims that this establishes a cause–and–effect relationship between watching violent television and aggressive behavior. Can you think of any other possible causes? 104

105. Example - solution It could be that the difference in aggressive behavior is due to other familial influences. Perhaps children who are permitted to watch violent programming are more likely to come from violent or abusive families, which could also lead to more aggressive behavior. 105