1. Research Methods and Data Analysis in Psychology Spring 2015 Kyle Stephenson

2. Overview – Day 13 Review Analyzing Complex Designs ▫Problem with multiple tests ▫ANOVA ▫Follow-up tests

3. Review What do you remember from last class?

4. Research Methods Basic Concepts Scientific Method Variance Effect Size Measurement Selecting Participants Research Design Descriptive Research Correlational Research Cross-sectional Designs Longitudinal Designs Experimental Studies Simple Experiments Advanced Experimental Design Statistics Descriptive Central Tendency Variation Distributions Outliers Graphing Inferential Correlation Pearson’s R Regression Means Differences T-tests ANOVA Interactions Presentation of Findings Scientific Writing Oral Presentation Ethics

5. Research Methods Basic Concepts Scientific Method Variance Effect Size Measurement Selecting Participants Research Design Descriptive Research Correlational Research Cross-sectional Designs Longitudinal Designs Experimental Studies Simple Experiments Advanced Experimental Design Statistics Descriptive Central Tendency Variation Distributions Outliers Graphing Inferential Correlation Pearson’s R Regression Means Differences T-tests ANOVA Interactions Presentation of Findings Scientific Writing Oral Presentation Ethics

6. Why We Need to Analyze Data Means of groups will vary (based on chance, i.e., error variance) whether or not the IV has any effect at all Need some way of telling whether this variation is larger than we would expect based on chance alone to know if our IV is having an effect

7. How Inferential Stats Work Step 1: Figure out how much means should differ based on chance (error variance) alone Step 2: Determine whether your means differ more than they should based on chance Step 3: Determine if this difference is big enough to conclude that you most likely have a “real” effect

8. Only Four Possible Outcomes of Any Analysis CorrectDecision Type II Error Type I ErrorCorrectDecision Reject null hypothesis Fail to reject null hypothesis Null hypothesis is false Null hypothesis is true

9. “Statistical Significance” Does NOT mean it’s important, or large Only means: We would be very unlikely to see a pattern of results like this based on chance alone Or: the difference between the means is so big that there is only a (5) out of (100) chance we would see it if the null hypothesis were true

10. Different t-tests One-sample ▫Does this group significantly differ from the population Independent-sample ▫Are these groups significantly different from each other? Paired-sample ▫Are the scores of this group different at these two times? (Most powerful)

11. How Big is The Difference? Effect size ▫How far apart the means of the groups are in units of standard error (similar to standard deviation) ▫UNLIKE statistical significance, does not depend on sample size (only depends on SIZE of the effect)

12. Take-Home Means of groups will differ no matter what One way to spot real effects is by determining whether the size of these differences is bigger than we would expect based on chance We use inferential stats to test how likely our results were to happen based on the null assumption If results are super unlikely, then we conclude that the result is “statistically significant”

13. Problem with Multiple Tests When an experiment has more than two conditions, it is no longer appropriate to use a t-test to analyze the differences between condition means. When one t-test is conducted, the probability of making a Type I error is only 5%. However, when several t-tests are conducted the overall probability of making a Type I is much higher.

14. What Does an ANOVA Do? Compares “within group variance” to “between group variance” If group doesn’t matter, these should be the same (group means should bounce around about as much as individual scores)

15. How Variance is Partitioned Within Groups SS (error variance) Between Groups SS (systematic variance) Deviation between each point and that group’s mean Sum squared deviation within each group Add these sums together Represents all variance that is not related to differences between groups**** Deviation between each group mean and the grand mean Square deviations and multiply by the size of the group Add these values together Represents variance variance that is related to differences between groups****

16. F-test F = MS bg / MS wg Compare this calculated value of F to the critical value of F based on the alpha-level and the degrees of freedom. If the calculated value of F exceeds the critical value of F, then we conclude that at least one of the means differs significantly from one or more of the others.

17. Follow-up to a Main Effect Post hoc tests or multiple comparisons are used to determine which means differ significantly Examples include: Tukey’s test, Scheffe’s test and Newman-Keuls test If the F-test is not significant, follow-up tests are not conducted because the independent variable has no effect.

18. ANOVA for Factorial Designs Sums of squares between-groups (SS bg ) can be broken down to test for different main effects and interactions. In a two-way factorial design (A X B), the total variance is composed of four parts 1.Main effect of A 2.Main effect of B 3.A x B interaction 4.Error variance

19. Follow-up to an Interaction If the F-test shows that an interaction is significant, tests of simple main effects are conducted. A simple main effect is an effect of one independent variable at a particular level of another independent variable. For a two-way interaction of A X B: 1.Simple main effect of A at B1 2.Simple main effect of A at B2 3.Simple main effect of B at A1 4.Simple main effect of B at A2

20. Take-Home ANOVA is more flexible than T-tets because it allows for >2 levels of your IV, and more than 1 IV. The rationale of ANOVA is very similar to T- tests: comparing systematic variance to error variance. If your IV affected your DV, systematic variance should be greater than error variance.