Presentation on theme: "1 STAT131 Week 2 Lecture 1b Making Sense of Data Anne Porter."— Presentation transcript:
1 STAT131 Week 2 Lecture 1b Making Sense of Data Anne Porter
2 Review 1.Learning and Writing -what why how when 2.Statistics is a study of variation throughout a process
3 Review Statistical Process Process Ethics The nature of the question to be answered Expertise Design Sampling Measurement Description and Analysis (Making sense of data) Conclusions & Decision Making
4 Where we do what! In lectures the focus - What are we doing? Why are we doing it? When do we do it? In labs the focus - How do we do it? Check definitions, Do by hand (simple) and SPSS Making choices about what to use
5 Making sense of raw data A shoe seller sets up on campus & collects some data about what size shoes students wear. What do you see in this data?
6 Making sense of raw data What might we do to to make sense out of the shoe size data?
7 What might we do to make sense? Order the data Calculate the centre –Mean average score –Median middle score of ordered values –Mode most common score Find the spread –Range from minimum to maximum Look for outliers unusual values
8 Descriptive Statistics (mean, range) NMinimumMaximumMeanStd. Deviation SHOESIZE1504.000042.00009.8166673.2291752 What do these statistics tell us? Is this what the shoe seller needs to know?
9 Descriptive Statistics (mean, range) NMinimumMaximumMeanStd. Deviation SHOESIZE1504.000042.00009.8166673.2291752 Range=Maximum less minimum=42-4 =38 What do these statistics tell us? Is this what the shoe seller needs to know?No There is an error in the data! Minimum size 4, Maximum 42 Average is 9.81
10 Five number summary SHOESIZE NValid150 Percentiles258.000000 509.500000 7511.000000 Five number summary Minimum Maximum Lower quartile or 25th Percentile: shoe size with 25% of shoe sizes below it Median, 5oth percentile or middle shoe size Upper quartile 75th Percentile with 75% shoe sizes below it (ie 25% above it) The interquartile range shoe size 75th percentile-shoe size 25th percentile What is a percentile? How do you calculate quartiles? And is this what the shoe seller wants?
11 Five number summary SHOESIZE NValid150 Percentiles258.000000 509.500000 7511.000000 Five number summary Minimum 4 Maximum 42 Lower quartile, 25% of shoe sizes below = 8 Median, 50% of shoe sizes below it = 9.5 Upper quartile, 75% of shoe sizes below it =11 The interquartile range 75th percentile-shoe size 25th percentile 11-8 –(50% of sizes between 8 and 11) What is a percentile? Does the shoeseller have what is needed?
12 Percentiles - definition The k th percentile is a number that has k percent of the scores at or below it and (100-k)% above it The lower quartile has 25% of scores at or below that score
13 Quartiles Q 1 is value of the (n+3) / 4 th observation, and Q 3 is the value of the (3n+1) / 4 th observation. Interpolate if necessary. There are other approaches to calculating which may give different answers. If the answers are similar there is no problem The interquartile range= Q 3 - Q 1 If we have 17 heights what observation do we need to get the upper and lower quartile? What observation will give the median?
14 Quartiles The upper quartile is? The lower quartile is? The interquartile range is? 166 147 - 19
15 What other statistics or graphs might inform the shoe-seller? Centre - mean, median Spread –Maximum-Minimum = Range –Upper Quartile-Lower quartile = Interquartile range –75th percentile-25th percentile= Interquartile range Outliers
16 Ordering the data Shoe Size 4 5 6 : 42 Ordering is often useful but we can do better
17 Frequency or relative frequency table What is wrong with this display?
18 Frequency or relative frequency table What is wrong with this display? The data has been treated as if it were continuous. Some packages will do this but we want the data to be treated as discrete data
19 Frequency distribution (order plus count) We still have an error (42) But we have the frequency (count) of each shoe size. What might be better for the shoeseller?
20 Percentages of each size Why might this be useful rather than frequency?
21 Percentages of each size Why might this be useful rather than frequency? We only had a sample so this would suggest the percentage or even proportion of each size. Is this all the shoeseller needs?
22 There are better ways of looking at distributions What else might we do?
23 Stem-and leaf plot (with error) Some packages (SPSS) cut off the outliers and lists them as extremes. See if you can find a definition for an extreme as used in SPSS and an outlier from the text. Different packages, different procedures may use different definitions - check
25 What do we do with outliers? Know the context to see what values are possible Check the original data to see if it is a data entry error See if it is in different units and transform to the appropriate unit If an error and you do not know what it should be delete it and make a note If there is no reason to conclude it is an error leave it in Sometimes analyse with the point in and the point out of the the data set
26 Stem-and-leaf plot (42 removed) What does it reveal? Could it be better?
27 Stem-and-leaf plot (42 removed) What does it reveal? Could it be better? Change stems to focus on whole and half sizes. We should have transformed the 42. This is the difference between a lecture and data analysis, I deleted!
28 Stem-and-leaf with different stems What do we notice now? Do we have what the shoe seller needs?
29 Stem-and-leaf with different stems What do we notice now? There is a distribution within a distribution with fewer half sizes Do we have what the shoe seller needs? We need male and female data (Next lecture)
30 Graphical Excellence Convey the message about the data Axes, units, variable names, figure labels DO NOT Distort the data Use pie charts (there is always a better chart) More dimensions than necessary, 3D instead of 2D Unnecessary pattern, fill, ink, decoration
31 To reveal Centre Spread Outliers Distribution Patterns Anything unusual Comparisons (next lecture) And more But there are choices to be made
32 Centre Mean Median Mode Trimmed Mean Median, FIRST arrange the sample values from smallest to largest. N odd : Median of 8, 7, 9 is the middle of ordered scores 8 N even: Median of 4,7,8,9 =(7+8)/2=7.5 Mode is the most common score in the data set eg for 1,2,3,3,4,5,6 The mode is 3 Trimmed Mean Eg. Diving at the Olympics is the average of the judges scores after having tossed out the highest and the lowest scores
33 Question: mean vs median Data A: 60, 2, 3, 5 Data B: 6, 2, 3, 5 Mean A = 17.5 Mean B = 4 Median A = 4 Median B = 4 Which measure best typifies the data A? Why? Which measure best typifies the data set B? Why?
34 Question: mean vs median Data A: 60, 2, 3, 5 Data B: 6, 2, 3, 5 Mean A = 17.5 Mean B = 4 Median A = 4 Median B = 4 Which measure best typifies the data A? Why? Which measure best typifies the data set B? Why? For A the outlier 60 suggests the median (4) as the Mean (17.5) is dragged up by the outlier 60 For B both are the same. The median (4) used 2 points the mean (4) uses all the data
35 Question: mean vs median In what sense are the mean and median the same? In what sense are the mean and median different?
36 Question: mean vs median In what sense are the mean and median the same? In what sense are the mean and median different? They are both measures of the centre They may give different numerical values and for different data sets one may be better as a measure than the other or both may be required
37 Making Choices between mean & median The mean uses all the information in the sample, because each value is added in the sum. –mean subject to error if spurious values are entered. –median is less affected by wild values, we say it is robust. If the mean is similar to median –use the mean as it uses all data. –often easier to work with the mean If they are different because of non-symmetric distribution –Can be useful to report both The context of what the data are are used for may also determine what is an appropriate measure
38 Measures of Spread Range= maximum value - minimum value Interquartile range = Upper Quartile-Lower quartile =Q 1 - Q 3 Sums of Squares Variance (S 2 ) Standard Deviation
39 Use of standard deviation The mean and std deviation gives information about where most of the distribution of values is to be found. For many distributions, the range mean - 2 standard devs to mean + 2 standard devs (mean + 2SD) contains approx 95% of the distribution. (The very least that this spread can contain is 75% of the distribution.)
40 Criteria for a good measure of spread Whatever measure of variability (or spread) the measure should not be affected by adding a constant to each value so as to change the centre (or location) If there is spread in the data it should indicate this Should make sense in the context used Should be robust, not influenced by outliers or extreme points
41 Undesirable features of measures of spread Sensitive to outliers Does not use all data eg range based only on two scores Difficult to understand Eg sum of squares in this context as the answer is very big and gets bigger with every additional data point. But useful in other contexts
42 Revealing distributions Frequency Distribution Table Stem-and-Leaf Histograms Box-and-whiskers
43 Box-and-Whiskers plots Often just called box plots, they give a pictorial summary of the data for a single variable. They use the five-number summary: –minimum value, –Q 1, –median, –Q 3, –maximum value
44 Example: If minimum = 3, Q 1 = 6, median=10, Q 3 = 12, maximum = 16, the box plot would look like You must draw a scale for the box plot. 246141681210
45 In a horizontal box plot, a horizontal axis shows the scale. The boxs left and right boundaries are Q 1 and Q 3, and an inner line shows the median. Whiskers are drawn outwards from the box to the minimum and maximum values. Often the sample mean is also shown.
46 What values given rise to the box plot below: If minimum=, Q 1=, median=, Q 3 =, maximum=, the box plot would look like You must draw a scale for the box plot. 246141681210
47 What do you want to see in data? Information Meaning We must turn data into information in order to have meaning
48 What can we see in data? Location (centre) Spread Shape Outliers Unusual patterns Gaps, clusters How do batches differ
49 Tools for making meaning from data Ordering data Dot plots & jittered dot plots Stem-and-leaf plots Histograms, Boxplots, Bar charts Pie charts Frequency tables Numerical summaries
50 Selecting the tool depends on The question asked How the variable is measured The structure of the data Utility of the tool More in the next lecture and labs
51 Homework Textbook reading Utts & Heckard (2004) Chapter 2 Or Textbook reading Moore and McCabe pp38-55. Or Textbook reading, Griffiths, Stirling and Weldon, 1998, Chapters 1, 2, 6 (pp. ) Complete lab and preparation for next weeks lab.