Education 793 Class Notes Decisions, Error and Power Presentation 8

2 Review: Chain of reasoning Random selection can take several forms (such as simple, systematic, cluster, or stratified random sampling), and is intended to generate a sample that represents the population.

3 Four Steps 1. State the hypothesis H 0 vs. H Alternate 2. Identify your criterion for rejecting H 0 Directional ornon-directional test One-tailedortwo-tailed Set alpha level (Prob. incorrectly rejecting H 0 ) 3. Compute test statistic General form: Test = statistic – expected parameter standard error of statistic 4. Decide about H 0

4 Million Dollar Question To Reject or Not To Reject?

5 Review: Inferential error Type I: Alpha –Rejecting the null hypothesis with the null hypothesis is really true Type II: Beta –Failing to reject the null hypothesis when the null hypothesis is in fact false

6 Possible outcomes Decision In the population, H 0 is true In the population, H 0 is false Reject H 0 (H 0 false) Type I error  Correct decision Fail to reject H 0 (H 0 true) Correct decision Type II error 

7 Considerations Basic considerations –What is the statistic of interest? –Is a one-tailed versus two-tailed test appropriate? –What is the nature of the sample? Dependent versus independent How much power is available? –Power is 1 –  –Factors influencing power: Increasing the number of observations Reducing the error in the data

8 Statistical Power Power is the probability of correctly rejecting a false null hypothesis. As such, power is defined as: 1 - ß where ß is the Type II error probability Two kinds of power analyses: –A priori: Used to identify what the sample size needs to be to identify a specified effect –Post hoc: Used when the null hypothesis has not been rejected, and when you want to know the probability that you have committed a Type II error

9 What Power is all about Power analysis is about Type II errors, “missed effects” – failing to reject H 0 : when there really is an effect in the population “Power” is the antithesis of “risk of Type II error” –Risk of Type II error = 1 - power –Power = 1 - Risk of Type II error

10 Trade off Between  and Power Conundrum: As we decrease  (Type I Error),  Type II Error) increases. The two errors have an inverse relationship. In order to deal with the problem, researchers follow an established set of guidelines.  =.05 or.01 Which error is the most dangerous?

11 Power of a Test Example: Ho:  =72, H1:  <72  =.05, n=36 Review: Given the sample data, a decision is made to reject or not reject the null hypothesis.  =72 z crit =  =Type I Error This distribution is assumed to be true under the Null

12 Power of a Test If the null hypothesis is false, then the above distribution is NOT the one we sampled from. We must have sampled from one of the possible alternative distributions, of which there are an infinite number. Recall H1:  <72 z crit =  =Type I Error  =72

13 Power of a Test The steps in computing power require the researcher to set a predetermined difference from the two means (  1) that is meaningful (Effect Size). With three pieces of information,  N,  we can estimate the power of a test. We will let computer programs do this for us.

14 Factors Affecting Power Size of the difference between population means, what the book refers to as  1. –The greater the effect size (  1), the greater the power will be. Common sense, as the difference gets larger, it will be easier to detect. Significance level –As the power of a statistical test increases so does  This is the trade off between Type I and Type II Errors.

15 Factors Affecting Power Variance –All other factors being equal ( , N,  ), the smaller the standard deviation in the population, the greater the power. Sample size –By increasing N, we decrease the standard deviation in the sampling distribution. Hence the power is increased.

16 Power as a function of… the true difference in  significance level sample size

17 Next Week Chapter 12 p and Available through JSTOR at