1. Applied Epidemiologic Analysis - P8400 Fall 2002 Lab 3 Type I, II Error, Sample Size, and Power Henian Chen, M.D., Ph.D.

2. Applied Epidemiologic Analysis - P8400 Fall 2002 POPULATION N (5,1) Sample 1 n1=10 Sample 2 n2=10 Type I Error Believing that some zero population effect is nonzero on the basis of a test of statistical significance

3. Applied Epidemiologic Analysis - P8400 Fall 2002 SAS Program for Type 1 Error data type1error; do j=1 to 2; /* create two samples */ do i=1 to 10; /* the sample size is 10 */ x=rannor(0)+5; /* random sampling from a normal distribution population, N(5,1)*/ output; end; proc ttest; /* perform independent t-test */ class j; var x; run;

4. Applied Epidemiologic Analysis - P8400 Fall 2002 DATA: type1error Obs j i x 1 1 1 5.34655 2 1 2 4.38660 3 1 3 3.09380 4 1 4 5.24584 5 1 5 5.37742 6 1 6 3.25170 7 1 7 3.57868 8 1 8 4.54776 9 1 9 4.97216 10 1 10 5.03652 11 2 1 6.49084 12 2 2 3.94007 13 2 3 3.84986 14 2 4 4.56861 15 2 5 3.91149 16 2 6 5.25983 17 2 7 5.12068 18 2 8 6.70596 19 2 9 5.00503 20 2 10 5.30035

5. Applied Epidemiologic Analysis - P8400 Fall 2002 Output 1 of T-Test Statistics Variable j N Mean Std Dev Std Err x 1 10 4.4837 0.8788 0.2779 x 2 10 5.0153 1.0067 0.3183 x Diff (1-2) -0.532 0.9449 0.4226 T-Tests Variable Method Variances DF t Value Pr > |t| x Pooled Equal 18 -1.26 0.2245 x Satterthwaite Unequal 17.7 -1.26 0.2248 Equality of Variances Variable Method Num DF Den DF F Value Pr > F x Folded F 9 9 1.31 0.6923

6. Applied Epidemiologic Analysis - P8400 Fall 2002 Output 2 of T-Test Statistics Variable j N Mean Std Dev Std Err x 1 10 5.4088 0.7262 0.2296 x 2 10 4.3767 1.0543 0.3334 x Diff (1-2) -0.532 0.9052 0.4048 T-Tests Variable Method Variances DF t Value Pr > |t| x Pooled Equal 18 2.55 0.0201 x Satterthwaite Unequal 16 2.55 0.0214 Equality of Variances Variable Method Num DF Den DF F Value Pr > F x Folded F 9 9 2.11 0.2819

7. Applied Epidemiologic Analysis - P8400 Fall 2002 Type II Error Believing that some nonzero population effect is zero on the basis of a test of statistical significance POPULATION 1 N (6,1) Sample 1 n1=10 POPULATION 2 N (8,1) Sample 2 n2=10

8. Applied Epidemiologic Analysis - P8400 Fall 2002 SAS Program for Type 2 Error data type2error; do i=1 to 10; /* the sample size is 10 */ j=1; /* for sample 1 */ x=rannor(0)+6; /* random sampling from a normal distribution population, N (6,1)*/ output; end; do i=1 to 10; /* the sample size is 10 */ j=2; /* for sample 2 */ x=rannor(0)+8; /* random sampling from a normal distribution population, N (8,1)*/ output; end; proc ttest; /* perform independent t-test */ class j; var x; run;

9. Applied Epidemiologic Analysis - P8400 Fall 2002 Output 1 of T-Test Statistics Variable j N Mean Std Dev Std Err x 1 10 6.4906 0.8872 0.2806 x 2 10 7.7403 0.9913 0.3135 x Diff (1-2) -0.532 0.9407 0.4207 T-Tests Variable Method Variances DF t Value Pr > |t| x Pooled Equal 18 -2.97 0.0082 x Satterthwaite Unequal 17.8 -2.97 0.0083 Equality of Variances Variable Method Num DF Den DF F Value Pr > F x Folded F 9 9 1.25 0.7465

10. Applied Epidemiologic Analysis - P8400 Fall 2002 Output 2 of T-Test Statistics Variable j N Mean Std Dev Std Err x 1 10 6.5582 1.3303 0.4207 x 2 10 7.3892 1.0052 0.3179 x Diff (1-2) -0.8310 1.1790 0.5273 T-Tests Variable Method Variances DF t Value Pr > |t| x Pooled Equal 18 -1.58 0.1325 x Satterthwaite Unequal 16.8 -1.58 0.1337 Equality of Variances Variable Method Num DF Den DF F Value Pr > F x Folded F 9 9 1.75 0.4165

11. Applied Epidemiologic Analysis - P8400 Fall 2002 Power The probability of rejecting a null hypothesis that is false to a specified degree for a given sample size, Type I error, and effect size

12. Applied Epidemiologic Analysis - P8400 Fall 2002 Given Sample size Given Effect size Increase Alpha Increase Power Given Alpha Given Effect size Increase Sample Size Increase Power Given Sample size Given Alpha Increase Effect Size Increase Power Sample size Effect size Alpha Powe r

13. Applied Epidemiologic Analysis - P8400 Fall 2002 SAS Program for Power of Case-Control Study data power; c=3.89; /* case:control ratio=200/778 = 3.89 */ n=200; /* number of cases=200 */ do p0=0.05 to 0.50 by 0.05; /* exposure prevalence */ do or=1.5 to 3.0 by 0.5; /* odds ratios from 1.5 to 3.0 */ q0=1-p0; p1=(p0*or)/(1+p0*(or-1)); q1=1-p1; pbar=(p1+c*p0)/(1+c); qbar=1-pbar; zbeta=(sqrt(n*(p1-p0)**2)-1.96*sqrt((1+1/c)*pbar*qbar)) / sqrt((p1*q1)+p0*q0/c); /* zbeta is the standardized deviate*/ power=probnorm(zbeta); /* PROBNORM returns the probability from the standard normal distribution */ output; end; run; proc print data=power; var p0 or zbeta power; run;