1. On-line resources http://wise.cgu.edu/powermod/index.asp http://wise.cgu.edu/regression_applet.asp http://wise.cgu.edu/hypomod/appinstruct.asp http://psych.hanover.edu/JavaTest/NeuroAnim/stats/StatDec. html http://psych.hanover.edu/JavaTest/NeuroAnim/stats/StatDec. html http://psych.hanover.edu/JavaTest/NeuroAnim/stats/t.html http://psych.hanover.edu/JavaTest/NeuroAnim/stats/CLT.html Note demo page

2. Effect sizes For a small effect size,.01, The change in success rate is from 46% to 54% For a medium effect size,.06, The change in success rate is from 38% to 62%. For a large effect size,.16, The change in success rate is from 30% to 70% R-squaredrCohen’s D Large.15.39.80 Medium.06.24.50 Small.01.10.20

3. But what does.10 really mean? PredictorOutcomeR2r Vietnam veteran status Alcohol abuse.00.03 TestostoneJuvenile delinquency.01.10 AZTDeath.05.33 PsychotherapyImprovement.10.32

4. Is psychotherapy effective? (after Shapiro & Shapiro, 1983) Therapy targetNumber of studies Cohen’s DrR2 Anxiety & depression30.67.319.6% Phobias76.88.5429% Physical and habit problems 106.85.5227% Social and sexual problems 76.75.4318% Performance anxieties 126.71.3714%

5. Calculating Cohen’s D Effect size = difference between predicted mean and mean of known population divided by population standard deviation (assumes that you know population and sample size) (imagine one population receives treatment, the other does not) d=  1  2 ) /   1 =mean of population 1 (hypothesized mean for the population that is subjected to the experimental manipulation)  2 =mean of population 2 (which is also the mean of the comparison distribution)  =standard deviation of population 2 (assumed to be the standard deviation of both populations

6. One other way to think about D D =.20, overlap 85%, 15 vs. 16 year old girls distribution of heights D=.50, overlap 67%, 14 vs. 18 year old girls distribution of heights D=.80, overlap 53%, 13 vs. 18 years old girls distribution of heights

7. Effect sizes are interchangeable

8. http://www.amstat.org/publications/jse/v10n3/aberson/power_applet.html

9. Statistical significance vs. effect size p <.05 r =.10 – For 100,000, p<.05 – For 10, p>.05 – Large sample, closer to population, less chance of sampling error

10. Brief digression Research hypotheses and statistical hypotheses Is psychoanalysis effective? – Null? – Alternate? – Handout Why test the null?

11. Statistical significance and decision levels. (Z scores, t values and F values) Sampling distributions for the null hypothesis: http://statsdirect.com/help/distributions/pf.htm

12. One way to think about it…

13. Two ways to guess wrong Truth for population Do not reject null hypothesis Reject null hypothesis Null is trueCorrect!Type 1 error Null is not trueType 2 errorCorrect! Type 1 error: think something is there and there is nothing Type 2 error: think nothing is there and there is

14. An example Null hypothesis is falseNull hypothesis is true Reject null hypothesisMerit pay works and we know it We decided merit pay worked, but it doesn’t. Do not reject null hypothesisWe decided merit pay does not work but it does. Merit pay does not work and we know it.

15. An example Imagine the following research looking at the effects of the drug, AZT, if any, on HIV positive patients. In others words, does a group of AIDs patients given AZT live longer than another group given a placebo. If we conduct the experiment correctly - everything is held constant (or randomly distributed) except for the independent measure and we do find a different between the two groups, there are only two reasonable explanations available to us: From Dave Schultz: Null hypothesis is false Null hypothesis is true Reject null hypothesis Do not reject null hypothesis

16. Power ->.10.20.30.40.50.60.70.80.90 Effect size |.01215383113144179219271354.0651014192430364457.1535681012141722 If you think that the effect is small (.01), medium, (.06) or large (.15), and you want to find a statistically significant difference defined as p<.05, this table shows you how many participants you need for different levels of “sensitivity” or power. Statistical power is how “sensitive” a study is detecting various associations (magnification metaphor)

17. Power ->.10.20.30.40.50.60.70.80.90 Effect size |.0170116156194232274323385478.06132026323845536277.156811131518202429 If you think that the effect is small (.01), medium, (.06) or large (.15), and you want to find a statistically significant difference defined as p<.01, this table shows you how many participants you need for different levels of “sensitivity” or power.

18. What determines power? 1.Number of subjects 2.Effect size 3.Alpha level Power = probability that your experiment will reveal whether your research hypothesis is true

19. How increase power? 1.Increase region of rejection to p<.10 2.Increase sample size 3.Increase treatment effects 4.Decrease within group variability

20. Study featurePractical way of raising power Disadvantages Predicted differenceIncrease intensity of experimental procedures May not be practical or distort study’s meaning Standard deviationUse a less diverse population May not be available, decreases generalizability Standard deviationUse standardized, controlled circumstances of testing or more precise measurement Not always practical Sample sizeUse a larger sample sizeNot practical, can be costly Significant levelUse a more lenient level of significance Raises alpha, the probability of type 1 error One tailed vs. two tailed test Use a one-tailed testMay not be appropriate to logic of study

21. What is adequate power?.50 (most current research).80 (recommended) How do you know how much power you have? Guess work Two ways to use power: 1. Post hoc to establish what you could find 2. Determine how many participants need

22. Outcome statistically significant Sample SizeConclusion YesSmallImportant results YesLargeMight or might not have practical importance NoSmallInconclusive NoLargeResearch H. probably false

23. Statistical power (for p <.05) r=.10 r=.30 r=.50 Two tailed One tailed Power: Power = 1 - type 2 error Power = 1 - beta