1. Topic 23: Diagnostics and Remedies

2. Outline Diagnostics –residual checks ANOVA remedial measures

3. Diagnostics Overview We will take the diagnostics and remedial measures that we learned for regression and adapt them to the ANOVA setting Many things are essentially the same Some things require modification

4. Residuals Predicted values are cell means, = Residuals are the differences between the observed values and the cell means Y ij -

5. Basic plots Plot the data vs the factor levels (the values of the explanatory variables) Plot the residuals vs the factor levels Construct a normal quantile plot and/or histogram of the residuals

6. KNNL Example KNNL p 777 Compare 4 brands of rust inhibitor (X has r=4 levels) Response variable is a measure of the effectiveness of the inhibitor There are 10 units per brand (n=10)

7. Plots Data versus the factor Residuals versus the factor Normal quantile plot of the residuals

8. Plots vs the factor symbol1 v=circle i=none; proc gplot data=a2; plot (eff resid)*abrand; run;

9. Data vs the factor Means look different …common spread in Y’s

10. Residuals vs the factor Odd dist of points

11. QQ-plot Due to odd (lack of and large)spread Can try nonparametric analysis – last slides

12. General Summary Look for –Outliers –Variance that depends on level –Non-normal errors Plot residuals vs time and other variables if available

13. Homogeneity tests Homogeneity of variance (homoscedasticity) H 0 : σ 1 2 = σ 2 2 = … = σ r 2 H 1 : not all σ i 2 are equal Several significance tests are available

14. Homogeneity tests Text discusses Hartley, modified Levene SAS has several including Bartlett’s (essentially the likelihood ratio test) and several versions of Levene

15. Homogeneity tests There is a problem with assumptions –ANOVA is robust with respect to moderate deviations from Normality –ANOVA results can be sensitive to the homogeneity of variance assumption Some homogeneity tests are sensitive to the Normality assumption

16. Levene’s Test Do ANOVA on the squared residuals from the original ANOVA Modified Levene’s test uses absolute values of the residuals Modified Levene’s test is recommended Another quick and dirty rule of thumb

17. KNNL Example KNNL p 785 Compare the strengths of 5 types of solder flux (X has r=5 levels) Response variable is the pull strength, force in pounds required to break the joint There are 8 solder joints per flux (n=8)

18. Scatterplot

19. Levene’s Test proc glm data=a1; class type; model strength=type; means type/ hovtest=levene(type=abs); run;

20. ANOVA Table SourceDF Sum of Squares Mean SquareF ValuePr > F Model4353.61208588.403021341.93<.0001 Error3573.79882502.1085379 Corrected Total39427.410910 Common variance estimated to be 2.11

21. Output Levene's Test ANOVA of Absolute Deviations Source DF F Value Pr > F type 4 3.07 0.0288 Error 35 We reject the null hypothesis and assume nonconstant variance

22. Means and SDs Level strength type N Mean Std Dev 1 8 15.42 1.23 2 8 18.52 1.25 3 8 15.00 2.48 4 8 9.74 0.81 5 8 12.34 0.76

23. Remedies Delete outliers – Is their removal important? Use weights (weighted regression) Transformations Nonparametric procedures

24. What to do here? Not really any obvious outliers Do not see pattern of increasing or decreasing variance or skewed dists Will consider –Weighted ANOVA –Mixed model ANOVA

25. Weighted least squares We used this with regression –Obtain model for how the sd depends on the explanatory variable (plotted absolute value of residual vs x) –Then used weights inversely proportional to the estimated variance

26. Weighted Least Squares Here we can compute the variance for each level Use these as weights in PROC GLM We will illustrate with the soldering example from KNNL

27. Obtain the variances and weights proc means data=a1; var strength; by type; output out=a2 var=s2; data a2; set a2; wt=1/s2; NOTE. Data set a2 has 5 cases

28. Proc Means Output Level of typeN strength MeanStd Dev 1815.42000001.23713956 2818.52750001.25297076 3815.00375002.48664397 489.74125000.81660337 5812.34000000.76941536

29. Merge and then use the weights in PROC GLM data a3; merge a1 a2; by type; proc glm data=a3; class type; model strength=type; weight wt; lsmeans type / cl; run;

30. Output SourceDF Sum of SquaresMean SquareF ValuePr > F Model4324.21309981.053274781.05<.0001 Error3535.00000001.0000000 Corrected Total39359.213099 Data have been standardized to have a variance of 1

31. LSMEANS Output type strength LSMEAN Standard ErrorPr > |t| 95% Confidence Limits 115.42000000.4373949<.000114.53204116.307959 218.52750000.4429921<.000117.62817819.426822 315.00375000.8791614<.000113.21895716.788543 49.74125000.2887129<.00019.15513210.327368 512.34000000.2720294<.000111.78775112.892249 Because of weights, standard errors simply based on sample variances of each level

32. Mixed Model ANOVA Relax the assumption of constant variance rather than including a “known” weight This involves moving to a mixed model procedure Topic will not be on exam but wanted you to be aware of these model capabilities

33. SAS Code proc glimmix data=a1; class type; model strength=type / ddfm=kr; random residual / group=type; run; This allows the variance to differ in each level and a degrees of freedom adjustment is used to account for this

34. GLIMMIX OUTPUT Fit Statistics -2 Res Log Likelihood122.11 AIC (smaller is better)132.11 AICC (smaller is better)134.18 BIC (smaller is better)139.88 CAIC (smaller is better)144.88 HQIC (smaller is better)134.79 Generalized Chi-Square35.00 Gener. Chi-Square / DF1.00 Covariance Parameter Estimates Cov ParmGroupEstimate Standard Error Residual (VC)type 11.53050.8181 Residual (VC)type 21.56990.8392 Residual (VC)type 36.18343.3052 Residual (VC)type 40.66680.3564 Residual (VC)type 50.59200.3164 Type III Tests of Fixed Effects Effect Num DF Den DFF ValuePr > F type414.8171.78<.0001 Really 3 groups of variances

35. SAS Code proc glimmix data=a1; class type; model strength=type / ddfm=kr; random residual / group=type1; run; Type1 was created to identify Type 1 and 2, Type 3, and Type 4 and 5 as 3 groups

36. GLIMMIX OUTPUT Fit Statistics -2 Res Log Likelihood122.13 AIC (smaller is better)128.13 AICC (smaller is better)128.91 BIC (smaller is better)132.80 CAIC (smaller is better)135.80 HQIC (smaller is better)129.74 Generalized Chi-Square35.00 Gener. Chi-Square / DF1.00 Covariance Parameter Estimates Cov ParmGroupEstimate Standard Error Residual (VC)Grp 11.55020.5859 Residual (VC)Grp 26.18343.3052 Residual (VC)Grp 30.62940.2379 Type III Tests of Fixed Effects Effect Num DF Den DFF ValuePr > F type419.877.68<.0001 Better BIC but same general type conclusion

37. Transformation Guides When σ i 2 is proportional to μ i, use When σ i is proportional to μ i, use log(y) When σ i is proportional to μ i 2, use 1/y For proportions, use arcsin( ) –arsin(sqrt(y)) in a SAS data step Box-Cox transformation

38. Example Consider study on KNNL pg 790 Y: time between computer failures X: three locations data a3; infile 'u:\.www\datasets512\CH18TA05.txt'; input time location interval; symbol1 v=circle; proc gplot; plot time*location; run;

39. Scatterplot Outlier or skewed distribution? Can consider transformation first

40. Box-Cox Transformation Can consider regression and 1-b 1 is the power to raise Y by Can try various “convenient” powers Can use SAS directly to calculate the power

41. E(logsig) = 0.90 +.79 logmu Power should be 1-.79 ≈ 0.20

42. Using SAS proc transreg data=a3; model boxcox(time / lambda=-2 to 2 by.2) = class(location); run;

43. Output Box-Cox Transformation Information for time LambdaR-SquareLog Like 0.24-73.040 -0.80.27-67.330 -0.60.31-62.316 -0.50.33-60.144 -0.40.35-58.239 -0.20.38-55.346* 0.0+0.39-53.830* 0.20.38-53.769< 0.40.36-55.118* 0.50.34-56.273 0.60.32-57.712 0.80.29-61.314 1.00.25-65.675 < - Best Lambda * - 95% Confidence Interval + - Convenient Lambda

44. Transforming data in SAS data a3; set a3; transtime = time**0.20; symbol1 v=circle i=none; proc gplot; plot transtime*location; run;

45. Much more constant spread in data!

46. Nonparametric approach Based on ranks See KNNL section 18.7, p 795 See the SAS procedure NPAR1WAY

47. Rust Inhibitor Analysis SourceDF Sum of SquaresMean SquareF ValuePr > F Model315953.46605317.82200866.12<.0001 Error36221.034006.13983 Corrected Total3916174.5000 Highly significant F test. Even if there is a violation of Normality, the evidence is overwhelming

48. Nonparametric Analysis Wilcoxon Scores (Rank Sums) for Variable eff Classified by Variable abrand abrandN Sum of Scores Expected Under H0 Std Dev Under H0 Mean Score 110128.0205.032.01411912.80 210355.0205.032.01411935.50 310255.0205.032.01411925.50 41082.0205.032.0141198.20 Average scores were used for ties. Kruskal-Wallis Test Chi-Square33.7041 DF3 Pr > Chi-Square<.0001

49. Last slide We’ve finished most of Chapters 17 and 18. We used program topic23.sas to generate the output.