1. Instructor: Dr. Amery Wu
Handout Thirteen: A Review of 482 and What Happens When the Data Violates the Assumptions of the Statistical Methods
EPSE 482
Introduction to Statistics for Research in Education
Instructor: Dr. Amery Wu

2. Lab Activity: Review the Statistical Methods Taught in this Course
Measurement of Data
Continuous
Categorical
Type of
the Inference
Descriptive
A1: summative/descriptive
A2: explanatory/predictive
B1: summative/descriptive
B2: explanatory/predictive
Inferential
C1: summative/descriptive
C2: explanatory/predictive
D1: summative/descriptive
D2: explanatory/predictive
Classify the following statistical methods into one of the four categories
McNemar test
Count/Percentage/Proportion
Two-tailed correlation test
Correlation
Chi-square test
Variance
Histogram
One/Two Way ANOVA
Sensitivity/specificity/PPV/NPV
One-sample t- test
Percentile
Two-sample t-test
Risk, ARD, NNT, OR
Simple regression
Mean
Skewness
Bar chart
Simple linear regression

3. Assumptions of General Linear Models
Normal distributions and independent observations are two of the major assumptions for statistical techniques subsumed under the general linear model (GLM), e.g., t-test, ANOVA, and ordinary least squares regression.
These statistical techniques work effectively and approprately ONLY when the observed data fit to the assumptions of GLM.

4. What next if you find that your data violate these assumptions?

5. When the Data is Non-Normal…
Transform the data, e.g., log-transformation or square root transformation. Personally, I do not recommend this method unless the meaning of the transformed scores is still conceivable.
Use non-parametric technique. Nonparametric techniques do not assume any forms of population distribution (e.g., normal, uniform, or binomial). It is recommended because it could be more effective than using a parametric test that is inappropriate.

6. When the Observations Are Dependent…
We have learned that dependence of observations could be dealt with by treating them as dependent samples (paired-samples t-test or repeated measures ANOVA). It is one appropriate technique for observation dependence.
Another useful method is to handle the data using random effects models (e.g., random effects ANOVA and random effects regression).

7. When the data is non-normal…

8. Brief Introduction to Parametric Tests
Parametric tests are inferential statistics relying on the assumption that data are drawn from the population of a given distribution.
Non-parametric tests are inferential statistics that do not make an assumption about the population distribution. Thus, they are also referred to as distribution-free tests.

9. Some Commonly Used Nonparametric Tests

10. Mann–Whitney U Test for Two Independent Samples
An example of Nonparametric Test
Mann–Whitney U Test for Two Independent Samples
The null hypothesis is that the two samples are drawn from a single population, and therefore their distributions are the same. The alternative hypothesis is that the distributions are different.
The basic idea for nonparametric techniques is that they work on the ranks of the data rather than the actual values of the data.
It requires the two samples to be independent, and the observations to be ordinal or continuous measurements (so that you can rank them).

11. An example of a Nonparametric Test
Mann–Whitney U test for Two Independent Samples
Gender
Kidrate
Rank 70 12 84 17 69 11 65 10 95 18 36 2 50 5 60 8 1 20 71 13 77 16 72 14 41 3
100 51 6
Using the kid’s self-report of pain by gender group of n=10.
Under the null the sum of ranks over boys should be the same as that over girls when the sample size for the two group are the same,

12. An example of a Nonparametric Test
Mann–Whitney U Test for Two Independent Samples
The test statistic for the Mann-Whitney test is U. This value is compared to the H table of critical values determined by the sample size of each group. If U is ≤ the critical value at some significance level (usually 0.05), it means that there is evidence to reject the null hypothesis that the population distributions are the same.
For sample sizes greater than 20, the z distributions can be used to approximate the significance level for the test. In this case, the calculated z is compared to the standard normal significance levels.

13. An example of a Nonparametric Test
Mann–Whitney U test for Two Independent Samples
The test statistic for the Mann–Whitney U test
The value of U is formulated to examine the sum of ranks taking into account sample size.
Notation: na is the sample size for group a, nb is the sample size for group b, Ra is the ranks of the cases in group a and Rb the ranks of the cases in group b.
The smaller value of Ua and Ub is considered the U statistic that will be used to conduct a hypothesis test.
Note that the value of U is formulated in such a way that a smaller value is more likely to be rejected, given that the null is false.

14. The H Table for the Mann–Whitney U
The degrees of freedom are na and nb (sample size for each group).
The critical values at 0.10 level are in light face and at 0.05 are in bold face. The value of U should be ≤ the critical values to be statistically significant.

15. Lab Activity : Mann–Whitney U Test Using SPSS
Note that you do not have to change the data values to ranks, SPSS will do it for you behind the scene.
Lab Activity: Compare the result of Mann-Whitney U test to that of independent samples t-test.

16. Using Parametric or Nonparametric?
The traditional thinking that non-parametric tests be used when data measurement is ordinal is confusing. The key criterion to choose between parametric and nonparametric is the population distribution requirement of the statistical technique imposed on a sample.
Many of the inferential statistics we have learned in this course assume the data are drawn from a normally distributed population.
If the sample data are pretty normal (at least approximately symmetrical), choose a parametric test, because, generally speaking, they are relatively more powerful if the null is false.
If your data are clearly non-normal, have limited response categories (ordered or not), or are ranks, choose a nonparametric test.
Note that one should also consider parametric techniques to model categorical data., e.g., the family of logistic regression.

17. When the observations are dependent…

18. When the observations are dependent…
Participant ID
Occasion
Diet
Pulse
Weight 1
Before Exercise
Vegetarian 95
140
During Exercise
134
165
After Exercise
186
154 2 66
185
109
150
144 3 69
110
119
177
161 4
Meat Eater 93
147
151
168
217
162 5 77
182
122
178
145 6 78
132
173
It is recommended that random effects models are used to model such data.

19. An Brief Introduction to Random Effects Models
The three occasions are regarded as a random sample of occasions from the population of occasions.
Because now the levels of the variable “occasion” are regarded as randomly sampled from the population, these types of design are referred to as “complex sampling designs,” and “random effects models” are used to analyze the data.
One can understand this type of data structure and analysis as sample within sample. For the example given on the last slide, a sample of 6 participants are within a sample of 3 occasions.
There is a hierarchy in the data structure; the participants are nested within the sample of occasions. Occasion is referred to as cluster.

20. An Brief Introduction to Random Effects Models
One-way Random Effects ANOVA
The factor is Diet (vegetarian=0 and meat eater=1). The occasions are before, after, and after exercise. The DV is pulse. For each occasion of measurement, a separate t-test/ANOVA is conducted. Within each occasion, the observations are independent for each ANOVA.
Participant ID
Occasion
Diet
Pulse 1 95 2 66 3 69 4 93 5 77 6 78
134
109
119
151
122
186
150
177
217
178
173
One ANOVA for Occasion 1
Another ANOVA for Occasion 2
The other ANOVA for Occasion 3

21. An Brief Introduction to Random Effects Models
One-way Random Effects Regression
The IV is weight. The DV is pulse. For each occasion of measurement, a separate liner regression line is fit. Within each occasion, the observations are independent for each regression analysis.
Participant ID
Occasion
Diet
Pulse
Weight 1 95
140 2 66
185 3 69
110 4 93
147 5 77
182 6 78
145
134
165
109
119
151
168
122
144
132
186
154
150
177
161
217
162
178
173
A regression line with one slope
A regression line with another slope
A regression line with the other slope

22. Final Reminders
Data =
Model +
Res. +
Res. =
Population
??????

23. Future The Next Courses
Repeated measures ANOVA (within-subjects ANOVA)
Mixed design ANOVA (within- and between-subjects ANOVA)
Analysis of covariance (ANCOVA)
Multiple regression: multiple IV one DV
Logistic regression: regression of categorical data on the IV(s)
Multivariate statistics: multiple DVs on the IV(s)
Current Developments in Statistical Methodology
Longitudinal design for studying growth and change
Structural equation modeling (SEM)
Multilevel (mixed) modeling for dependence (clustered) data
Mixed of qualitative and quantitative methods

24. Quantitative Methodology Network
Research
Question
Inferences
Research
Design
Good research relies on coherent planning and execution that integrate all the elements of this network.
Statistical
Analysis
Data

25. Strategies for Successful Outcomes for Learning Quantitative Research Methodology
Contextualizing through preview & examples
Building through mastery
Digesting by slicing
Consolidating by review and application
Internalizing through communication & critique
Recursive learning & experience