1. Nonparametrics.Zip (a compressed version of nonparamtrics) Tom Hettmansperger Department of Statistics, Penn State University
References:
Higgins (2004) Intro to Modern Nonpar Stat
Hollander and Wolfe (1999) Nonpar Stat Methods
Arnold Notes
Johnson, Morrell, and Schick (1992) Two-Sample Nonparametric Estimation and Confidence Intervals Under Truncation, Biometrics, 48,
Website:

2. Single Sample Methods

3. Inference: Confidence Intervals and Hypothesis Tests
Robust Data Summaries
Graphical Displays
Inference: Confidence Intervals and Hypothesis Tests
Location, Spread, Shape
CI-Boxplots (notched boxplots)
Histograms, dotplots, kernel density estimates.

4. Absolute Magnitude Planetary Nebulae Milky Way
Abs Mag (n = 81) …

7. But don’t be too quick to “accept” normality:

10. Null Hyp: Pop distribution, F(x) is normal
The Kolmogorov-Smirnov Statistic
The Anderson-Darling Statistic

12. Anatomy of a 95% CI-Boxplot
Box formed by quartiles and median
IQR (interquartile range) Q3 – Q1
Whiskers extend from the end of the box to the farthest point within 1.5xIQR.
For a normal benchmark distribution, IQR=1.348Stdev and 1.5xIQR=2Stdev.
Outliers beyond the whiskers are more than 2.7 stdevs from the median. For a normal distribution this should happen about .7% of the time.
Pseudo Stdev = .75xIQR

13. The confidence interval and hypothesis test

19. Additional Remarks:
The median is a robust measure of location. It is not affected by outliers.
It is efficient when the population has heavier tails than a normal population.
The sign test is also robust and insensitive to outliers. It is efficient when the tails are heavier than those of a normal population.
Similarly for the confidence interval.
In addition, the test and the confidence interval are distribution free and
do not depend on the shape of the underlying population to determine
critical values or confidence coefficients.
They are only 64% efficient relative to the mean and t-test when the
population is normal.
If the population is symmetric then the Wilcoxon Signed Rank statistic can be used, and it is robust against outliers and 95% efficient relative to the t-test.

20. Two-Sample Methods

21. Two-Sample Comparisons
85% CI-Boxplots
Mann-Whitney-Wilcoxon Rank Sum Statistic
Estimate of difference in locations
Test of difference in locations
Confidence Interval for difference in locations
Levene’s Rank Statistic for differences in scale
or variance.

28. Why 85% Confidence Intervals?
We have the following test of
Rule: reject the null hyp if the 85% confidence
intervals do not overlap.
The significance level is close to 5% provided
the ratio of sample sizes is less than 3.

29. Mann-Whitney-Wilcoxon Statistic: The sign statistic on the pairwise differences.
Unlike the sign test (64% efficiency for normal population, the MWW test
has 95.5% efficiency for a normal population. And it is robust against
outliers in either sample.

31. Mann-Whitney Test and CI: App Mag, Abs Mag
N Median
App Mag (M-31)
Abs Mag (MW)
Point estimate for d is
95.0 Percent CI for d is (24.530,25.256)
W =
Test of d=0 vs d not equal 0 is significant at
What is W?

33. What about spread or scale differences between the two populations?
Below we shift the MW observations to the right by 24.9 to line up with
M-31.
Variable StDev IQR PseudoStdev MW M

34. Levene’s Rank Test
Compute |Y – Med(Y)| and |X – Med(X)|, called absolute deviations.
Apply MWW to the absolute deviations. (Rank the absolute deviations)
The test rejects equal spreads in the two populations when difference
in average ranks of the absolute deviations is too large.
Idea: After we have centered the data, then if the null hypothesis
of no difference in spreads is true, all permutations of the combined data
are roughly equally likely. (Permutation Principle)
So randomly select a large set of the permutations say B permutations.
Assign the first n to the Y sample and the remaining m to the X sample
and compute MMW on the absolute deviations.
The approximate p-value is #MMW > original MMW divided by B.

35. Difference of rank mean abso devs 51.9793
So we easily reject the null hypothesis of no difference in spreads and
conclude that the two populations have significantly different spreads.

36. One Sample Methods
k-Sample Methods
Two Sample Methods

37. All one-sample and two-sample methods can be applied one at a time
Variable Mean StDev Median .75IQR Skew Kurtosis
Messier
Messier
NGC
NGC
NGC
All one-sample and two-sample methods can be applied one at a time
or two at a time. Plots, summaries, inferences.
We begin k-sample methods by asking if the location differences between
the NGC nebulae are statistically significant.
We will briefly discuss issues of truncation.

40. Kruskal-Wallis Test on NGC
sub N Median Ave Rank Z
Overall
KW = DF = 2 P = 0.000
This test can be followed by multiple comparisons.
For example, if we assign a family error rate
of .09, then we would conduct 3 MWW tests, each
at a level of .03. (Bonferroni)

42. What to do about truncation.
See a statistician
Read the Johnson, Morrell, and Schick reference. and then
see a statistician.
Here is the problem: Suppose we want to estimate the difference in locations
between two populations: F(x) and G(y) = F(y – d).
But (with right truncation at a) the observations come from
Suppose d > 0 and so we want to shift the X-sample to the right toward the
truncation point. As we shift the Xs, some will pass the truncation point and
will be eliminated from the data set. This changes the sample sizes and
requires adjustment when computing the corresponding MWW to see if
it is equal to its expectation. See the reference for details.

43. Comparison of NGC4382 and NGC 4494
Data multiplied by 100 and 2600 subtracted.
Truncation point taken as 120.
Point estimate for d is W =
m = 101 and n = 59
Computation of shift estimate with truncation d m n W
E(W)
25.3 88 59
5.10
4750.5
4366.0
28.3 84
3.60
4533.5
4248.0
30.3 83
2.10
4372.0
4218.5
32.3 81
0.80
4224.5
4159.5
33.3
-0.20
4144.5
33.1
-0.00
4161.5

44. What more can we do? Multiple regression
Analysis of designed experiments (AOV)
Analysis of covariance
Multivariate analysis
These analyses can be carried out using the website:

45. Professor Lundquist, in a seminar on compulsive thinkers, illustrates his brain
stapling technique.

46. The End