1. Planned Contrasts and Data Management
Class 17

2. Topics Covered Today 1. Planned Contrasts
1. Planned Contrasts
2. Analysis of Residual Variance
3. Post-hoc tests
4. Data Management
a. Setting up data files
b. Cleaning data
5. Tuesday
a. Interactions in regression
b. Hand out ANOVA exercise

3. What we won't cover from Keppel Chapter 11:
What we won't cover from Keppel Chapter 11:
1. Orthogonal comparisons (pp )
2. Analysis of trend (pp. 211)
3. Multiple comparisons (pp )
4. Analysis of cell means (pp )

4. Planned Contrast: Function
1. Factorial ANOVA tests for orthogonal (perpendicular) interactions.
2. Some studies predict non-orthogonal interactions.
3. Planned contrast provides more predictive power to confirm non-orthogonal contrasts of any particular shape (“wedge”, “arrow” [like above] or other).

5. Planned Contrast: Execution (Conceptual)
1. Must predict ????? before gathering data.
Predict that Democratic women will be most opposed to gun instruction in school, compared to Democratic men, Republican men, and Republican women.

6. Planned Contrast: Execution (Conceptual)
1. Must predict pattern of interaction before gathering data.
Predict that Democratic women will be most opposed to gun instruction in school, compared to Democratic men, Republican men, and Republican women.

7. Convert Separate Factors into Single Factor
1. Two separate factors
Political Party Democrat, GOP
Gender Male, Female
Convert the two separate factors into a ????

8. Convert Separate Factors into Single Factor
1. Two separate factors
Political Party Democrat, GOP
Gender Male, Female
Convert the two separate factors into a single factor
genparty 1) Male Republican
2) Male Democrat
3) Female Republican
4) Female Democrat

9. Converting Multi-factors into Single Factor for Planned Contrast
Gender
Political Party
Male
Female
Republican
5.00
4.75
Democrat
4.50
2.75
Converted into single factor with four levels
GENPARTY
1 = Male/Republican
2 = Male/Democrat
3 = Female/Republican 4.75
4 = Female/Democrat

10. Planned Contrast: Execution (Conceptual)
3. Conduct one-way ANOVA, with new single variable as predictor.
4. Assign weights to the four levels, as follows:
1) Male Republican -1
2) Male Democrat ) Female Republican -1
4) Female Democrat 3
* Weights indicate which sub-groups are to be compared.
* Weights must add up to zero
5. Planned contrast then limits comparison to the indicated groups, but “counts” all subjects in terms of degrees of freedom and computation of error. This provides greater predictive power.

11. Graph of Gender X Political Party and Opposition to Gun Usage in School

12. Orthogonal Interaction
Univariate Analysis of Variance
[DataSet1]
Orthogonal Interaction

13. Planned Contrast, Page 1

14. Planned Contrast, Page 2 Contrast Tests Contrast -6.000 1.768 1.696
Assumes eq. var.
Doesn’t assume
eq. var.
value
-6.000
Std.
Error
1.768
1.696
-3.394
-3.539 12
5.501
.005
.014 t df
Sig (2 –tailed)

15. “Quality Control” for Planned Contrast
Issue: Planned contrast can be a very “liberal” test, confirming patterns that don’t closely fit with actual predictions.
Predicted this
Obtained this
Result of this –1, -1, -1, + 3 planned contrast is still significant
How to assess the “quality” of a significant planned contrast?

16. Analysis of Residual Variance
Logic of test:
Did (Between groups effect – ???) leave significant amount of ??? variance unexplained?
If so, then the contrast DID OR DID NOT do a good job?
However, if “what’s left over” (i.e., between – contrast) is not significant, then the contrast accounts for ???

17. Analysis of Residual Variance
Logic of test:
Did (Between groups effect – Contrast effect) leave significant amount of sytematic (non-random) variance unexplained?
If so, then the contrast did not do a good job It did not explain the outcome fully.
However, if “what’s left over” (i.e., between – contrast) is not significant, then the contrast accounts for most of the treatment.
In this case, the contrast did do a good job.

18. Steps in Analysis of Residual Variance Test
Get SPSS printout of planned contrast
Get t of contrast, square it to get contrast F (t = F )
Compute SS contrast (SSc): Multiply contrast F by mean sq. w/n (MSw) of oneway. This results in SS contrast (SSc).
Compute SS residuals (SSr): Get SS between (SSb) from oneway, and subtract SSc. (SSb – SSc) = SS residuals (SSr)
Compute MS contrast (MSc): Divide SSr by df, which is (oneway df – planned contrast df). This produces the MS contrast (MSc)
Compute F residuals: Divide MSc by MSw. MSc/MSw = F residuals
Compute df for F resid: numerator df = (df oneway – df contrast; see 5a, above), denominator df = df within (from oneway).
Check this F in F table from any stats book. If significant, contrast is not a good fit. If not significant, the contrast is a good fit.

19. Residuals Analysis Test
1. Get SPSS printout of planned contrast
2. F of contrast (Fcont) = t2; t = t2 = 11.49
3. SScontrast (SScont): F cont X MSw = X 1.04 = 11.95
4. SSresiduals (SSres): SSbetween (SSb) = 12.50
SSb – SScont = – = .55
5a. Contrast df = df oneway – df contrast = = 3
5b. MScontrast (MScont) = SSres / contrast df = .55/3 = 0.18
F residuals (Fresid): Divide MScont by MSw = 0.18/1.04 = .17
DF for Fresid = df contrast (see 5a, above), df within: (3, 12)
8. F table at (3, 12) df, for criterion p < .25; F = 1.56
9. Obtained Fresid < 1.56, therefore residual is not significant, therefore contrast result is a good fit for data.

20. Post Hoc Effects Do female democrats differ from other groups?
Do female democrats differ from other groups?
1 = Male/Republican
2 = Male/Democrat
3 = Female/Republican 4.75
4 = Female/Democrat 2.75
Conduct three t tests? Why or Why not?
Solution: XXX tests of multiple comparisons.
XXX tests consider the inflated likelihood of Type ? error
Kent's favorite—??? test of multiple comparisons,
which is the most generous.

21. Post Hoc Effects Do female democrats differ from other groups?
Do female democrats differ from other groups?
1 = Male/Republican
2 = Male/Democrat
3 = Female/Republican 4.75
4 = Female/Democrat 2.75
Conduct three t tests? NO. Why not?
Capitalizes on chance
Solution: Post hoc tests of multiple comparisons.
Post hoc tests consider the inflated likelihood of Type I error
Kent's favorite—Tukey test of multiple comparisons,
which is the most generous.
NOTE: Post hoc tests can be done on any multiple set of means, not only on planned contrasts.

22. Conducting Post Hoc Tests
1. Recode data from multiple factors into single factor, as per planned contrast.
2. Run oneway ANOVA statistic
3. Select "posthoc tests" option.
ONEWAY
gunctrl BY genparty
/CONTRAST=
/STATISTICS DESCRIPTIVES
/MISSING ANALYSIS
/POSTHOC = TUKEY
ALPHA(.05).

23. Post hoc Tests, Page 1

24. Post Hoc Tests, Page 2

25. Data Management Issues
Setting up data file
Checking accuracy of data
Disposition of data
Why obsess on these details? Murphy's Law
If something can go wrong, it will go wrong,
and at the worst possible time.

26. Creating a Coding Master
1. Get survey copy
2. Assign variable names
3. Assign variable values
4. Assign missing values
5. Proof master for accuracy
6. Make spare copy, keep in file drawer

27. Coding Master

28. Cleaning Data Set 1. Exercise in delay of gratification
1. Exercise in delay of gratification
2. Purpose: Reduce random error
3. Improve power of inferential stats.

29. Techniques for Cleaning Data
1. Print out data file and ???
2. Run “descriptives”: check ?????
3. Run “frequencies”
4. Run “cross tabs”: check if number ??? per condition looks OK
5. Correlations: Check if patterns of data move in expected direction
6. Multiple data entry: Locate and correct disparities in data entry.

30. Techniques for Cleaning Data
1. Print out data file and visually scan it.
2. Run “descriptives”: check means, min and max values, valid cases
3. Run “frequencies”
4. Run “cross tabs”: check if number of subs per condition looks OK
5. Correlations: Check if patterns of data move in expected direction
6. Multiple data entry: Locate and correct disparities in data entry.

31. Complete Data Set

34. Using Cross Tabs to Check for Missing or Erroneous Data Entry
Case A: Expect equal cell sizes
Gender
Oldest
Youngest
Only Child
Males 10 20
Females 5 15
TOTAL 25 40
Case B: Impossible outcome
Number of Siblings
Oldest
Youngest
Only Child
None 4 3 6
One
More than one 2
TOTAL 10 8

35. If No Errors Detected, chances are good that:
Check a Sub Sample
1. Determine acceptable error rate
Number of cases to randomly sample, by rate of acceptable
error rate:
Acceptable Error Rate
Number of Cases
to Randomly Review
If No Errors Detected, chances are good that:
.50 5
50% or fewer errors
.40 10
40% or fewer errors
.20 25
20% or fewer errors
.10 50
10% or fewer errors

36. Storing Data Raw Data Automated Data 2. File raw data by ID #
Raw Data
1. Hold raw data in secure place
2. File raw data by ID #
3. Hold raw date for at least X years post Y
Automated Data
1. One pristine source, one working file, one syntax file
2. Back up, Back up, Back up

37. Storing Data Raw Data Automated Data 2. File raw data by ID #
Raw Data
1. Hold raw data in secure place
2. File raw data by ID #
3. Hold raw date for at least 5 years post publication
Automated Data
1. One pristine source, one working file, one syntax file
2. Back up, Back up, Back up

38. Sort Raw Data Records
01-20
21-40
41-60
61-80
81-100

39. COMMENT SYNTAX FILE GUN CONTROL STUDY SPRING 2007
COMMENT DATA MANAGEMENT
IF (gender = 1 & party = 1) genparty = 1 .
EXECUTE .
IF (gender = 1 & party = 2) genparty = 2 .
IF (gender = 2 & party = 1) genparty = 3 .
IF (gender = 2 & party = 2) genparty = 4 .
COMMENT ANALYSES
UNIANOVA
gunctrl BY gender party
/METHOD = SSTYPE(3)
/INTERCEPT = INCLUDE
/PRINT = DESCRIPTIVE
/CRITERIA = ALPHA(.05)
/DESIGN = gender party gender*party .
ONEWAY
gunctrl BY genparty
/CONTRAST=
/STATISTICS DESCRIPTIVES
/MISSING ANALYSIS
/POSTHOC = TUKEY ALPHA(.05).

40. Cleaning Data Set 1. Exercise in delay of gratification
1. Exercise in delay of gratification
2. Purpose: Reduce random error
3. Improve power of inferential stats.