Presentation is loading. Please wait.

Presentation is loading. Please wait.

Descriptive Statistics (Part 2) Standardized Data Standardized Data Percentiles and Quartiles Percentiles and Quartiles Box Plots Box Plots Grouped Data.

Similar presentations


Presentation on theme: "Descriptive Statistics (Part 2) Standardized Data Standardized Data Percentiles and Quartiles Percentiles and Quartiles Box Plots Box Plots Grouped Data."— Presentation transcript:

1

2 Descriptive Statistics (Part 2) Standardized Data Standardized Data Percentiles and Quartiles Percentiles and Quartiles Box Plots Box Plots Grouped Data Grouped Data Skewness and Kurtosis (optional) Skewness and Kurtosis (optional) Chapter 44

3 For any population with mean and standard deviation, the percentage of observations that lie within k standard deviations of the mean must be at least 100[1 – 1/k 2 ].For any population with mean and standard deviation, the percentage of observations that lie within k standard deviations of the mean must be at least 100[1 – 1/k 2 ]. Developed by mathematicians Jules Bienaymé ( ) and Pafnuty Chebyshev ( ).Developed by mathematicians Jules Bienaymé ( ) and Pafnuty Chebyshev ( ). Standardized Data Chebyshevs Theorem Chebyshevs Theorem

4 For k = 2 standard deviations, 100[1 – 1/2 2 ] = 75%For k = 2 standard deviations, 100[1 – 1/2 2 ] = 75% So, at least 75.0% will lie within + 2So, at least 75.0% will lie within + 2 For k = 3 standard deviations, 100[1 – 1/3 2 ] = 88.9%For k = 3 standard deviations, 100[1 – 1/3 2 ] = 88.9% So, at least 88.9% will lie within + 3So, at least 88.9% will lie within + 3 Although applicable to any data set, these limits tend to be too wide to be useful.Although applicable to any data set, these limits tend to be too wide to be useful. Standardized Data Chebyshevs Theorem Chebyshevs Theorem

5 The Empirical Rule states that for data from a normal distribution, we expect that forThe Empirical Rule states that for data from a normal distribution, we expect that for The normal or Gaussian distribution was named for Karl Gauss ( ).The normal or Gaussian distribution was named for Karl Gauss ( ). The normal distribution is symmetric and is also known as the bell-shaped curve.The normal distribution is symmetric and is also known as the bell-shaped curve. k = 1 about 68.26% will lie within + 1 k = 1 about 68.26% will lie within + 1 k = 2 about 95.44% will lie within + 2 k = 2 about 95.44% will lie within + 2 k = 3 about 99.73% will lie within + 3 k = 3 about 99.73% will lie within + 3 Standardized Data The Empirical Rule The Empirical Rule

6 Note: no upper bound is given. Data values outside + 3 are rare. Distance from the mean is measured in terms of the number of standard deviations.Distance from the mean is measured in terms of the number of standard deviations. Standardized Data The Empirical Rule The Empirical Rule

7 If 80 students take an exam, how many will score within 2 standard deviations of the mean?If 80 students take an exam, how many will score within 2 standard deviations of the mean? Assuming exam scores follow a normal distribution, the empirical rule statesAssuming exam scores follow a normal distribution, the empirical rule states about 95.44% will lie within + 2 about 95.44% will lie within + 2 so 95.44% x students will score + 2 from. How many students will score more than 2 standard deviations from the mean?How many students will score more than 2 standard deviations from the mean? Standardized Data Example: Exam Scores Example: Exam Scores

8 Unusual observations are those that lie beyond + 2.Unusual observations are those that lie beyond + 2. Outliers are observations that lie beyond + 3.Outliers are observations that lie beyond + 3. Standardized Data Unusual Observations Unusual Observations

9 For example, the P/E ratio data contains several large data values. Are they unusual or outliers?For example, the P/E ratio data contains several large data values. Are they unusual or outliers? Standardized Data Unusual Observations Unusual Observations

10 If the sample came from a normal distribution, then the Empirical rule statesIf the sample came from a normal distribution, then the Empirical rule states = ± 1(14.08) = ± 2(14.08) = ± 3(14.08) Standardized Data The Empirical Rule The Empirical Rule = (8.9, 38.8) = (-5.4, 50.9) = (-19.5, 65.0)

11 Standardized Data The Empirical Rule The Empirical Rule Outliers Outliers Unusual Unusual Are there any unusual values or outliers?Are there any unusual values or outliers?

12 A standardized variable (Z) redefines each observation in terms the number of standard deviations from the mean.A standardized variable (Z) redefines each observation in terms the number of standard deviations from the mean. Standardization formula for a population: Standardization formula for a sample: Standardized Data Defining a Standardized Variable Defining a Standardized Variable

13 z i tells how far away the observation is from the mean.z i tells how far away the observation is from the mean. = 7 – = Standardized Data Defining a Standardized Variable Defining a Standardized Variable For example, for the P/E data, the first value x 1 = 7. The associated z value isFor example, for the P/E data, the first value x 1 = 7. The associated z value is

14 = 91 – = 4.85 A negative z value means the observation is below the mean.A negative z value means the observation is below the mean. Standardized Data Defining a Standardized Variable Defining a Standardized Variable Positive z means the observation is above the mean. For x 68 = 91,Positive z means the observation is above the mean. For x 68 = 91,

15 Here are the standardized z values for the P/E data:Here are the standardized z values for the P/E data: Standardized Data Defining a Standardized Variable Defining a Standardized Variable What do you conclude for these four values?What do you conclude for these four values?

16 In Excel, use =STANDARDIZE(Array, Mean, STDev) to calculate a standardized z value.In Excel, use =STANDARDIZE(Array, Mean, STDev) to calculate a standardized z value. MegaStat calculates standardized values as well as checks for outliers.MegaStat calculates standardized values as well as checks for outliers. Standardized Data Defining a Standardized Variable Defining a Standardized Variable

17 What do we do with outliers in a data set?What do we do with outliers in a data set? If due to erroneous data, then discard.If due to erroneous data, then discard. An outrageous observation (one completely outside of an expected range) is certainly invalid.An outrageous observation (one completely outside of an expected range) is certainly invalid. Recognize unusual data points and outliers and their potential impact on your study.Recognize unusual data points and outliers and their potential impact on your study. Research books and articles on how to handle outliers.Research books and articles on how to handle outliers. Standardized Data Outliers Outliers

18 For a normal distribution, the range of values is 6 (from – 3 to + 3 ).For a normal distribution, the range of values is 6 (from – 3 to + 3 ). If you know the range R (high – low), you can estimate the standard deviation as = R/6.If you know the range R (high – low), you can estimate the standard deviation as = R/6. Useful for approximating the standard deviation when only R is known.Useful for approximating the standard deviation when only R is known. This estimate depends on the assumption of normality.This estimate depends on the assumption of normality. Standardized Data Estimating Sigma Estimating Sigma

19 Percentiles are data that have been divided into 100 groups.Percentiles are data that have been divided into 100 groups. For example, you score in the 83 rd percentile on a standardized test. That means that 83% of the test-takers scored below you.For example, you score in the 83 rd percentile on a standardized test. That means that 83% of the test-takers scored below you. Deciles are data that have been divided into 10 groups.Deciles are data that have been divided into 10 groups. Quintiles are data that have been divided into 5 groups.Quintiles are data that have been divided into 5 groups. Quartiles are data that have been divided into 4 groups.Quartiles are data that have been divided into 4 groups. Percentiles and Quartiles Percentiles Percentiles

20 Percentiles are used to establish benchmarks for comparison purposes (e.g., health care, manufacturing and banking industries use 5, 25, 50, 75 and 90 percentiles).Percentiles are used to establish benchmarks for comparison purposes (e.g., health care, manufacturing and banking industries use 5, 25, 50, 75 and 90 percentiles). Quartiles (25, 50, and 75 percent) are commonly used to assess financial performance and stock portfolios.Quartiles (25, 50, and 75 percent) are commonly used to assess financial performance and stock portfolios. Percentiles are used in employee merit evaluation and salary benchmarking.Percentiles are used in employee merit evaluation and salary benchmarking. Percentiles and Quartiles Percentiles Percentiles

21 Quartiles are scale points that divide the sorted data into four groups of approximately equal size.Quartiles are scale points that divide the sorted data into four groups of approximately equal size. The three values that separate the four groups are called Q 1, Q 2, and Q 3, respectively.The three values that separate the four groups are called Q 1, Q 2, and Q 3, respectively. Q1Q1 Q2Q2 Q3Q3 Lower 25% | Second 25% | Third 25% | Upper 25% Percentiles and Quartiles Quartiles Quartiles

22 The second quartile Q 2 is the median, an important indicator of central tendency.The second quartile Q 2 is the median, an important indicator of central tendency. Q 1 and Q 3 measure dispersion since the interquartile range Q 3 – Q 1 measures the degree of spread in the middle 50 percent of data values.Q 1 and Q 3 measure dispersion since the interquartile range Q 3 – Q 1 measures the degree of spread in the middle 50 percent of data values. Q2Q2Q2Q2 Lower 50% Lower 50% | Upper 50% Upper 50% Q1Q1Q1Q1 Q3Q3Q3Q3 Lower 25% Lower 25% | Middle 50% Middle 50% | Upper 25% Upper 25% Percentiles and Quartiles Quartiles Quartiles

23 The first quartile Q 1 is the median of the data values below Q 2, and the third quartile Q 3 is the median of the data values above Q 2.The first quartile Q 1 is the median of the data values below Q 2, and the third quartile Q 3 is the median of the data values above Q 2. Q1Q1Q1Q1 Q2Q2Q2Q2 Q3Q3Q3Q3 Lower 25% Lower 25% | Second 25% Second 25% | Third 25% Third 25% | Upper 25% Upper 25% For first half of data, 50% above, 50% below Q 1. For second half of data, 50% above, 50% below Q 3. Percentiles and Quartiles Quartiles Quartiles

24 Depending on n, the quartiles Q 1,Q 2, and Q 3 may be members of the data set or may lie between two of the sorted data values.Depending on n, the quartiles Q 1,Q 2, and Q 3 may be members of the data set or may lie between two of the sorted data values. Percentiles and Quartiles Quartiles Quartiles

25 For small data sets, find quartiles using method of medians:For small data sets, find quartiles using method of medians: Step 1. Sort the observations. Step 2. Find the median Q 2. Step 3. Find the median of the data values that lie below Q 2. Step 4. Find the median of the data values that lie above Q 2. Percentiles and Quartiles Method of Medians Method of Medians

26 Use Excel function =QUARTILE(Array, k) to return the kth quartile.Use Excel function =QUARTILE(Array, k) to return the kth quartile. =QUARTILE(Array, 3) =PERCENTILE(Array, 75) Excel treats quartiles as a special case of percentiles. For example, to calculate Q 3Excel treats quartiles as a special case of percentiles. For example, to calculate Q 3 Excel calculates the quartile positions as:Excel calculates the quartile positions as: Position of Q n Position of Q n Position of Q n Percentiles and Quartiles Excel Quartiles Excel Quartiles

27 Consider the following P/E ratios for 68 stocks in a portfolio.Consider the following P/E ratios for 68 stocks in a portfolio. Use quartiles to define benchmarks for stocks that are low-priced (bottom quartile) or high-priced (top quartile).Use quartiles to define benchmarks for stocks that are low-priced (bottom quartile) or high-priced (top quartile) Percentiles and Quartiles Example: P/E Ratios and Quartiles Example: P/E Ratios and Quartiles

28 Using Excels method of interpolation, the quartile positions are:Using Excels method of interpolation, the quartile positions are: Quartile Position Formula Interpolate Between Q1Q1Q1Q1 = 0.25(68) = X 17 + X 18 Percentiles and Quartiles Example: P/E Ratios and Quartiles Example: P/E Ratios and Quartiles Q2Q2Q2Q2 = 0.50(68) = X 34 + X 35 Q3Q3Q3Q3 = 0.75(68) = X 51 + X 52

29 The quartiles are:The quartiles are: QuartileFormula First (Q 1 ) Q 1 = X (X 18 -X 17 ) = (14-14) = 14 Percentiles and Quartiles Example: P/E Ratios and Quartiles Example: P/E Ratios and Quartiles Second (Q 2 ) Q 2 = X (X 35 -X 34 ) = (19-19) = 19 Third (Q 3 ) Q 3 = X (X 52 -X 51 ) = (26-26) = 26

30 So, to summarize:So, to summarize: These quartiles express central tendency and dispersion. What is the interquartile range?These quartiles express central tendency and dispersion. What is the interquartile range? Q1Q1Q1Q1 Q2Q2Q2Q2 Q3Q3Q3Q3 Lower 25% of P/E Ratios Lower 25% of P/E Ratios14 Second 25% of P/E Ratios Second 25% of P/E Ratios19 Third 25% of P/E Ratios Third 25% of P/E Ratios26 Upper 25% of P/E Ratios Upper 25% of P/E Ratios Because of clustering of identical data values, these quartiles do not provide clean cut points between groups of observations.Because of clustering of identical data values, these quartiles do not provide clean cut points between groups of observations. Percentiles and Quartiles Example: P/E Ratios and Quartiles Example: P/E Ratios and Quartiles

31 Whether you use the method of medians or Excel, your quartiles will be about the same. Small differences in calculation techniques typically do not lead to different conclusions in business applications. Percentiles and Quartiles Tip Tip

32 Quartiles generally resist outliers.Quartiles generally resist outliers. However, quartiles do not provide clean cut points in the sorted data, especially in small samples with repeating data values.However, quartiles do not provide clean cut points in the sorted data, especially in small samples with repeating data values. Data set A: 1, 2, 4, 4, 8, 8, 8, 8 Q 1 = 3, Q 2 = 6, Q 3 = 8 Data set B: 0, 3, 3, 6, 6, 6, 10, 15 Q 1 = 3, Q 2 = 6, Q 3 = 8 Although they have identical quartiles, these two data sets are not similar. The quartiles do not represent either data set well.Although they have identical quartiles, these two data sets are not similar. The quartiles do not represent either data set well. Percentiles and Quartiles Caution Caution

33 Some robust measures of central tendency and dispersion using quartiles are:Some robust measures of central tendency and dispersion using quartiles are: StatisticFormulaExcelProCon Midhinge =0.5*(QUARTILE (Data,1)+QUARTILE (Data,3)) Robust to presence of extreme data values. Less familiar to most people. Percentiles and Quartiles Dispersion Using Quartiles Dispersion Using Quartiles

34 StatisticFormulaExcelProCon Midspread Q 3 – Q 1 =QUARTILE(Data,3)- QUARTILE(Data,1) Stable when extreme data values exist. Ignores magnitude of extreme data values. Percentiles and Quartiles Dispersion Using Quartiles Dispersion Using Quartiles Coefficient of quartile variation (CQV) None Relative variation in percent so we can compare data sets. Less familiar to non- statisticians

35 The mean of the first and third quartiles.The mean of the first and third quartiles. For the 68 P/E ratios,For the 68 P/E ratios, Midhinge = A robust measure of central tendency since quartiles ignore extreme values.A robust measure of central tendency since quartiles ignore extreme values. Percentiles and Quartiles Midhinge Midhinge

36 A robust measure of dispersionA robust measure of dispersion For the 68 P/E ratios,For the 68 P/E ratios, Midspread = Q 3 – Q 1 Midspread = Q 3 – Q 1 = 26 – 14 = 12 Percentiles and Quartiles Midspread (Interquartile Range) Midspread (Interquartile Range)

37 Measures relative dispersion, expresses the midspread as a percent of the midhinge.Measures relative dispersion, expresses the midspread as a percent of the midhinge. For the 68 P/E ratios,For the 68 P/E ratios, Similar to the CV, CQV can be used to compare data sets measured in different units or with different means.Similar to the CV, CQV can be used to compare data sets measured in different units or with different means. Percentiles and Quartiles Coefficient of Quartile Variation (CQV) Coefficient of Quartile Variation (CQV)

38 A useful tool of exploratory data analysis (EDA).A useful tool of exploratory data analysis (EDA). Also called a box-and-whisker plot.Also called a box-and-whisker plot. Based on a five-number summary:Based on a five-number summary: X min, Q 1, Q 2, Q 3, X max Consider the five-number summary for the 68 P/E ratios:Consider the five-number summary for the 68 P/E ratios: X min, Q 1, Q 2, Q 3, X max Box Plots

39 Minimum Median (Q 2 ) Maximum Q1Q1Q1Q1 Q3Q3Q3Q3 Box Whiskers Right-skewed Center of Box is Midhinge Box Plots

40 Use quartiles to detect unusual data points.Use quartiles to detect unusual data points. These points are called fences and can be found using the following formulas:These points are called fences and can be found using the following formulas: Inner fences Outer fences: Lower fence Q 1 – 1.5 (Q 3 –Q 1 ) Q 1 – 3.0 (Q 3 –Q 1 ) Upper fence Q (Q 3 –Q 1 ) Q (Q 3 –Q 1 ) Values outside the inner fences are unusual while those outside the outer fences are outliers.Values outside the inner fences are unusual while those outside the outer fences are outliers. Box Plots Fences and Unusual Data Values Fences and Unusual Data Values

41 For example, consider the P/E ratio data:For example, consider the P/E ratio data: Ignore the lower fence since it is negative and P/E ratios are only positive.Ignore the lower fence since it is negative and P/E ratios are only positive. Inner fences Outer fences: Lower fence: 14 – 1.5 (26–14) = 4 14 – 3.0 (26–14) = 22 Upper fence: (26–14) = (26–14) = +62 Box Plots Fences and Unusual Data Values Fences and Unusual Data Values

42 Truncate the whisker at the fences and display unusual values and outliers as dots.Truncate the whisker at the fences and display unusual values and outliers as dots. Inner Fence Outer Fence UnusualOutliers Box Plots Fences and Unusual Data Values Fences and Unusual Data Values Based on these fences, there are three unusual P/E values and two outliers.Based on these fences, there are three unusual P/E values and two outliers.

43 Although some information is lost, grouped data are easier to display than raw data.Although some information is lost, grouped data are easier to display than raw data. When bin limits are given, the mean and standard deviation can be estimated.When bin limits are given, the mean and standard deviation can be estimated. Accuracy of grouped estimates depend on - the number of bins - distribution of data within bins - bin frequenciesAccuracy of grouped estimates depend on - the number of bins - distribution of data within bins - bin frequencies Grouped Data Nature of Grouped Data Nature of Grouped Data

44 Consider the frequency distribution for prices of Lipitor® for three cities:Consider the frequency distribution for prices of Lipitor® for three cities: Grouped Data Mean and Standard Deviation Mean and Standard Deviation Where m j = class midpoint f j = class frequency k = number of classes n = sample sizeWhere m j = class midpoint f j = class frequency k = number of classes n = sample size

45 Estimate the mean and standard deviation byEstimate the mean and standard deviation by Note: dont round off too soon.Note: dont round off too soon. Grouped Data Nature of Grouped Data Nature of Grouped Data

46 How accurate are grouped estimates compared to ungrouped estimates?How accurate are grouped estimates compared to ungrouped estimates? Now estimate the coefficient of variationNow estimate the coefficient of variation CV = 100 (s / ) = 100 ( / ) = 9.2% For the previous example, we can compare the grouped data statistics to the ungrouped data statistics.For the previous example, we can compare the grouped data statistics to the ungrouped data statistics. Grouped Data Nature of Grouped Data Nature of Grouped Data Accuracy Issues Accuracy Issues

47 For this example, very little information was lost due to grouping.For this example, very little information was lost due to grouping. However, accuracy could be lost due to the nature of the grouping (i.e., if the groups were not evenly spaced within bins).However, accuracy could be lost due to the nature of the grouping (i.e., if the groups were not evenly spaced within bins). Grouped Data Accuracy Issues Accuracy Issues

48 The dot plot shows a relatively even distribution within the bins.The dot plot shows a relatively even distribution within the bins. Effects of uneven distributions within bins tend to average out unless there is systematic skewness.Effects of uneven distributions within bins tend to average out unless there is systematic skewness. Grouped Data Accuracy Issues Accuracy Issues

49 Accuracy tends to improve as the number of bins increases.Accuracy tends to improve as the number of bins increases. If the first or last class is open-ended, there will be no class midpoint (no mean can be estimated).If the first or last class is open-ended, there will be no class midpoint (no mean can be estimated). Assume a lower limit of zero for the first class when the data are nonnegative.Assume a lower limit of zero for the first class when the data are nonnegative. You may be able to assume an upper limit for some variables (e.g., age).You may be able to assume an upper limit for some variables (e.g., age). Median and quartiles may be estimated even with open-ended classes.Median and quartiles may be estimated even with open-ended classes. Grouped Data Accuracy Issues Accuracy Issues

50 Generally, skewness may be indicated by looking at the sample histogram or by comparing the mean and median.Generally, skewness may be indicated by looking at the sample histogram or by comparing the mean and median. This visual indicator is imprecise and does not take into consideration sample size n.This visual indicator is imprecise and does not take into consideration sample size n. Skewness and Kurtosis Skewness Skewness

51 Skewness and Kurtosis Skewness Skewness Skewness is a unit-free statistic.Skewness is a unit-free statistic. The coefficient compares two samples measured in different units or one sample with a known reference distribution (e.g., symmetric normal distribution).The coefficient compares two samples measured in different units or one sample with a known reference distribution (e.g., symmetric normal distribution). Calculate the samples skewness coefficient as:Calculate the samples skewness coefficient as: Skewness =

52 In Excel, go to Tools | Data Analysis | Descriptive Statistics or use the function =SKEW(array)In Excel, go to Tools | Data Analysis | Descriptive Statistics or use the function =SKEW(array) Skewness and Kurtosis Skewness Skewness

53 Consider the following table showing the 90% range for the sample skewness coefficient.Consider the following table showing the 90% range for the sample skewness coefficient. Skewness and Kurtosis Skewness Skewness

54 Coefficients within the 90% range may be attributed to random variation.Coefficients within the 90% range may be attributed to random variation. Skewness and Kurtosis Skewness Skewness

55 Coefficients outside the range suggest the sample came from a nonnormal population.Coefficients outside the range suggest the sample came from a nonnormal population. Skewness and Kurtosis Skewness Skewness

56 As n increases, the range of chance variation narrows.As n increases, the range of chance variation narrows. Skewness and Kurtosis Skewness Skewness

57 Kurtosis is the relative length of the tails and the degree of concentration in the center.Kurtosis is the relative length of the tails and the degree of concentration in the center. Consider three kurtosis prototype shapes.Consider three kurtosis prototype shapes. Skewness and Kurtosis Kurtosis Kurtosis

58 A histogram is an unreliable guide to kurtosis since scale and axis proportions may differ.A histogram is an unreliable guide to kurtosis since scale and axis proportions may differ. Excel and MINITAB calculate kurtosis as:Excel and MINITAB calculate kurtosis as: Kurtosis = Skewness and Kurtosis Kurtosis Kurtosis

59 Consider the following table of expected 90% range for sample kurtosis coefficient.Consider the following table of expected 90% range for sample kurtosis coefficient. Skewness and Kurtosis Kurtosis Kurtosis

60 A sample coefficient within the ranges may be attributed to chance variation.A sample coefficient within the ranges may be attributed to chance variation. Skewness and Kurtosis Kurtosis Kurtosis

61 Coefficients outside the range would suggest the sample differs from a normal population.Coefficients outside the range would suggest the sample differs from a normal population. Skewness and Kurtosis Kurtosis Kurtosis

62 As sample size increases, the chance range narrows.As sample size increases, the chance range narrows. Inferences about kurtosis are risky for n < 50.Inferences about kurtosis are risky for n < 50. Skewness and Kurtosis Kurtosis Kurtosis

63 Applied Statistics in Business and Economics End of Chapter 4


Download ppt "Descriptive Statistics (Part 2) Standardized Data Standardized Data Percentiles and Quartiles Percentiles and Quartiles Box Plots Box Plots Grouped Data."

Similar presentations


Ads by Google