How Many Discoveries Have Been Lost by Ignoring Modern Statistical Methods? Rand R. Wilcox

The theme Despite what we learn, standard methods are NOT robust to violations of normality Despite what we learn, standard methods are NOT robust to violations of normality –Heteroscedasticity –Skewness –Outliers => Reduce chances of detecting true differences & obtaining accurate confidence intervals

Alternatives to the Mean: Need an estimator that performs as well as the mean under normal conditions AND is robust to departures from normality Need an estimator that performs as well as the mean under normal conditions AND is robust to departures from normality –4 options: 10%t trimmed mean 10%t trimmed mean 20% trimmed mean 20% trimmed mean Μ m – Mean estimator by some chap called Huber. Μ m – Mean estimator by some chap called Huber. Ө.5 – Median estimator by some chaps called Harrell & David Ө.5 – Median estimator by some chaps called Harrell & David

Dealing with Outliers: Sample mean & sample SD are inflated by outliers => masks them Sample mean & sample SD are inflated by outliers => masks them Trimming is not simply “throwing” data away and applying standard methods Trimming is not simply “throwing” data away and applying standard methods –This is a bad idea! If you take out extreme values and then continue => use of the wrong SE.

Skewness:

How much trimming & what to choose? Rule of thumb = 20% Rule of thumb = 20% Trimmed means tend to perform better that M estimators in more situations; M estimators are better with correlation & regression Trimmed means tend to perform better that M estimators in more situations; M estimators are better with correlation & regression

Why can’t we just test normality and then decide? Because conventional tests are insensitive..... Because conventional tests are insensitive..... Only way to determine if modern methods are useful is to use them Only way to determine if modern methods are useful is to use them Modern methods can be extended to more complex designs as well; including multivariate analyses Modern methods can be extended to more complex designs as well; including multivariate analyses

Correlation: Pearson’s r is not resistant to outliers; modern methods/alternatives can help e.g. Kendall’s Tau & Spearman’s rho Pearson’s r is not resistant to outliers; modern methods/alternatives can help e.g. Kendall’s Tau & Spearman’s rho Percentage Bend correlation: Percentage Bend correlation: –Population value of assoc is zero under independence (unusual apparently) –Good control over type I error in broad range of situations –Allows flexible choice re: how many outliers can be handled

Regression: OLS = poor choice for researchers; SE can be more than 100 times larger than some modern methods! OLS = poor choice for researchers; SE can be more than 100 times larger than some modern methods! Recommends a bootstrap method in conjunction with a robust estimator e.g. S- PLUS function regci Recommends a bootstrap method in conjunction with a robust estimator e.g. S- PLUS function regci Critics argue that robust regressoin estimators fail to check for curvature of the line – this can be fixed by using a “smoother” Critics argue that robust regressoin estimators fail to check for curvature of the line – this can be fixed by using a “smoother”

An example.....

An example (2):

Conclusions: Use of trimmed means and funky modern methods is recommended Use of trimmed means and funky modern methods is recommended Education in psychology should reflect modern advances in stats Education in psychology should reflect modern advances in stats Not all problems are solved, but you could be missing something really important due to the vulnerability of standard methods to minor departures from normality. Not all problems are solved, but you could be missing something really important due to the vulnerability of standard methods to minor departures from normality.