Presentation on theme: "Confidence Intervals, Effect Size and Power"— Presentation transcript:
1Confidence Intervals, Effect Size and Power Chapter 8
2Hyde, Fennema & Lamon (1990)Mean gender differences in mathematical reasoning were very small.When extreme tails of the distributions were removed, differences were smaller and reversed direction (favored girls)The gender differences depended on taskFemales were better with computation.Males were better with problem solving.
3Men, Women, and MathAn accurate understanding of gender differences may not be found in “significant” effects.Each distribution has variability.Each distribution has overlap.
4A Gender Difference in Mathematics Performance – amount of overlap as reported by Hyde (1990)
5Confidence IntervalsPoint estimate: summary statistic – one number as an estimate of the populatione.g., meanInterval estimate: based on our sample statistic, range of sample statistics we would expect if we repeatedly sampled from the same populatione.g., Confidence interval
6Confidence IntervalsInterval estimate that includes the mean we would expect for the sample statistic a certain percentage of the time were we to sample from the same population repeatedlyTypically set at 95%The range around the mean when we add and subtract a margin of errorConfirms findings of hypothesis testing and adds more detail
7Calculating Confidence Intervals Step 1: Draw a picture of a distribution that will include confidence intervalsStep 2: Indicate the bounds of the CI on the drawingStep 3: Determine the z statistics that fall at each line marking the middle 95%Step 4: Turn the z statistic back into raw meansStep 5: Check that the CIs make sense
8Confidence Interval for z-test Mlower = - z(σM) + MsampleMupper = z(σM) + MsampleThink about it: Why do we need two calculations?
12What are the advantages and disadvantages of each method? Think About ItWhich do you think more accurately represents the data: point estimate or interval estimate?Why?What are the advantages and disadvantages of each method?
13Effect Size: Just How Big Is the Difference? Significant = Big?Significant = Different?Increasing sample size will make us more likely to find a statistically significant effect
14What is effect size?Size of a difference that is unaffected by sample sizeStandardization across studies
15Effect Size & Mean Differences Imagine both represent significant effects: Which effect is bigger?
16Effect Size and Standard Deviation Note the spread in the two figures: Which effect size is bigger?
17Increasing Effect Size Decrease the amount of overlap between two distributions:1. Their means are farther apart2. The variation within each population issmaller
18Calculating Effect Size Cohen’s d Estimates effect sizeAssesses difference between means using standard deviation instead of standard error
20PrepProbability of replicating an effect given a particular population and sample sizeTo CalculateStep 1: Calculate the p value associated with our statistic (in Appendix B)Step 2: Use the following equation in Excel=NORMSDIST(NORMSINV(1-P)/(SQRT(2)))Use the actual p value instead of P in the equation
21Statistical PowerThe measure of our ability to reject the null hypothesis, given that the null is falseThe probability that we will reject the null when we shouldThe probability that we will avoid a Type II error
22Calculating PowerStep 1: Determine the information needed to calculate powerPopulation mean, population standard deviation, sample mean, samp0le size, standard error (based on sample size)Step 2: Determine a critical z value and raw mean, to calculate powerStep 3: Calculate power: the percentage of the distribution of the means for population 2 that falls above the critical value
29Factors Affecting Power Larger sample size increases powerAlpha levelHigher level increases power (e.g., from .05 to .10)One-tailed tests have more power than two-tailed testsDecrease standard deviationIncrease difference between the means
30Meta-Analysis Meta-analysis considers many studies simultaneously. Allows us to think of each individual study as just one data point in a larger study.
31The Logic of Meta-Analysis STEP 1: Select the topic of interestSTEP 2: Locate every study that hasbeen conducted and meets the criteriaSTEP 3: Calculate an effect size, often Cohen’s d, for every studySTEP 4: Calculate statistics and create appropriate graphs