Summary statistics Using a single value to summarize some characteristic of a dataset. For example, the arithmetic mean (or average) is a summary statistic because it gives the average value of a dataset such as average blood pressure readings

4.1 Indices of Central Tendency (or location) (Arithmetic) Mean: average of a set of values Blood Pressure Readings X i 95X 1 98X 2 101X 3 87X 4 105X Sum Arithmetic Mean = = 486 / 5 = 97.2 mm Hg

4.2 Robust Measure of Location Mean is very sensitive (not robust) to extreme values Blood Pressure Readings X i 87X 1 95X 2 98X 3 101X X Mean = 97.2Decimal overlooked, Mean = 286.2

Robust measure of location The median ( the middle value of an ordered data set ) is less sensitive (robust) to extreme values in the data Blood Pressure Readings X i 87X 1 95X 2 98X 3 101X X 5 median value = is unchanged Trimmed mean (e.g. 10% trimmed mean is the average after deleting 10% of the data at both ends) is also less affected by extreme values

Intervals between failures of an air conditioner (in operating hours) 413, 14, 58, 37, 100, 65, 9, 169, 447, 184, 36, 201, 118, 34, 31, 18, 19, 67, 57, 62, 7, 22, 34, 90, 10 Mean = ?8% trimmed mean = ? Median = ?

Ordered values 7, 9, 10, 14, 18, 19, 22, 31, 34, 34, 36, 37, 57, 58, 62, 65, 67, 90, 100, 118, 169, 184, 201, 413, 447 Measures of location Sample size = 25 mean = 2302/25 = 92.1 hrs 8% of 25 = 2,leave out 2 obs at both ends 8% trimmed mean = 1426/21 = 67.9 hrs median = 13 th ordered value = 57 < 67.9 <92.1 hrs

Desirable properties of the median Not sensitive to extreme values in data More suitable for describing skewed distributions (e.g., median income vs average income) The relative positions of the data points are unchanged when log-transformed. As a result, the median of the log-transformed data is just the log of the median of the original data Not so for the mean, the mean of logX is not obtainable from the mean of X 87 < 95 < 98 < 101 < 105Med = 98

Relative positions of median and mean for skewed distributions Positively-skewed or skewed to the right (where the longer tail is) Mean > Median Negatively-skewed or skewed to the left (where the longer tail is) Mean < Median

When to use mean or median: Use both by all means. Mean performs best when we have a normal or symmetric distribution with thin tails. If skewed or when we want to limit the influence of outliers, use the median.

Indices of Dispersion or Spread Range: difference between the largest and the smallest value Problem: does not consider how values in between are scattered. In the following, for both sets of data, the numbers of observations, means, medians and ranges are all equal. Which one has more scatter? 10, 12, 13, 14, 15, 16, 17, 18, 20 10, 15, 15, 15, 15, 15, 15, 15, 20 datasets with same range but different scatter of values range

Indices of Dispersion A good index of dispersion should be one that summarises the dispersion of individual values from some central value like the mean X X X X X X mean

Indices of Dispersion Problem with averaging deviations of individual values from the mean is that it is always = = = = = where 97.2 is the mean of values 87, 95, 98, 101, 105 average of deviations of individual values from the mean

Indices of Dispersion Usual approach: consider square deviations from the mean and take their average sum of squares of deviations from the mean = = = = =

Variance calculation from a sample: customary to divide by n-1 (default option in most software) rather than by n = / 4 = 46.2 effective sample size - also called degrees of freedom

Variance of a sample Can be shown mathematically:

Why subtract 1 ? Results in a better estimator of the population variance Acknowledge the fact that the population mean is unknown and has to be estimated by the sample mean (effective sample size decreased by 1 for every parameter estimated) No need to subtract 1 if we calculate variance using deviations from the population mean

Variance of a sample Problem with variance is its awkward unit of measurement as values have been squared Problem overcome by taking square root of variance - revert back to original unit of measurement Square root of the variance gives the standard deviation

Sample Standard Deviation The Sample Standard Deviation (S or SD)

4.4 Robust Measure of Dispersion Variance is defined as the mean of the squared deviations and as such is even more nonrobust to extreme values than the mean (an extreme deviation becomes even more extreme after squaring) A robust measure of dispersion is IQR/1.35 whereIQR = 3 rd quartile – 1 st quartile = Inter-quartile range The reason for dividing IRQ by 1.35 is to make it compatible with the standard deviation when the underlying distribution is normal

Intervals between failures of an air conditioner (in operating hours) 413, 14, 58, 37, 100, 65, 9, 169, 447, 184, 36, 201, 118, 34, 31, 18, 19, 67, 57, 62, 7, 22, 34, 90, 10 Mean = ?8% trimmed mean = ? Median = ? SD=?IQR/1.35 = ?

Ordered values 7, 9, 10, 14, 18, 19, 22, 31, 34, 34, 36, 37, 57, 58, 62, 65, 67, 90, 100, 118, 169, 184, 201, 413, 447 Measures of location Sample size = 25 mean = 2302/25 = 92.1 hrs 8% of 25 = 2,leave out 2 obs at both ends 8% trimmed mean = 1426/21 = 67.9 hrs median = 13 th ordered value = 57 < 67.9 <92.1 hrs Measures of dispersionSD = hrs 1 st quartile = 7 th ordered value = 22 hrs 3 rd quartile = 19 th ordered value = 100 hrs IQR/1.35 = 78/1.35 = 57.8 hrs

5-Number Summary of a data set Min, 1 st quartile Median 3 rd quartile, Max Represent graphically by a box plot