1. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.1 Sample Size and Power Considerations

2. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.2 Sample Size Considerations  A scientist walks into my office and asks, “Do you have a minute? This will be easy for you. We’re setting up a study of the effect of combination therapies for controlling hypertension in middle- aged moderate hypertensives that have subnormal activity levels. How many people do I need?”

3. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.3 Sample Size Considerations  A pharmaceutical company calls me and says, “We believe we have found a cure for the common cold. How many patients do I need to study to get our drug approved by the FDA?”

4. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.4 Where do we begin? N = (Total Budget / Cost per patient)? Hopefully not!

5. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.5 Sample Size Calculations  First learn a little about the application and the problem. Learn the disease, the medicine, etc. Understand the question.  Visualize the final analysis. Choose a statistical method of analysis. Analysis determines sample size.  Example:  1-sample t-test or 2-group independent samples t- test? Something else?  1-sample vs. 2-sample problem?  1-sided vs. 2-sided test?  Independent or paired?

6. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.6 Sample Size Calculations  Formulate a PRIMARY question or hypothesis to test (or determine what you are estimating). Write down H 0 and H A.  Determine the endpoint. Choose an outcome measure. How do we “measure” or “quantify” the responses?

7. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.7 Sample Size Calculations  Choose simple designs and analyses if possible.  Stress the use of randomization, which can validate statistical testing in the analyses.  Example: 2-sample t-test (independent groups)  What is needed in order to determine the sample size?

8. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.8 What is Needed to Determine the Sample-Size?   Up to the investigator (often = 0.05)  How much type I error can you afford?  1-  (power)  Up to the investigator (often 80%-90%)  How much type II error can you afford?

9. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.9 What is Needed to Determine the Sample-Size?  Choosing  and  :  Weigh the cost of a Type I error versus a Type II error.  In early phase clinical trials, we often do not want to “miss” a significant result and thus often consider designing a study for higher power (perhaps 90%) and may consider relaxing the  error (perhaps 0.10).  In order to approve a new drug, the FDA requires significance in two Phase III trials strictly designed with  error no greater than 0.05 (Power = 1-  is often set to 80% but the FDA does not have a rule about power).

10. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.10 Sample Size Calculations  The idea is to choose a sample size such that both of the following conditions hold: 1.If the null hypothesis is true, then the probability if incorrectly rejecting is (no more than)  2.If the alternative hypothesis is true, then the probability of correctly rejecting is (at least) 1-  = power.

11. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.11 What is Needed to Determine the Sample-Size?  The “minimum difference (between groups) that is clinically relevant or meaningful”.  Requires clinical input  Equivalently, choose the mean for both groups (or the null and alternative differences)  Note for equivalence/noninferiority studies, we need the “maximum irrelevant or non-meaningful difference”.

12. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.12 What is Needed to Determine the Sample-Size?  Estimates of variability  Often obtained from prior studies  Consider a pilot study for this  SD for each group  Explore the literature and data from ongoing studies for estimates needed in calculations  Get SD for placebo group and treatment group  May need to validate this estimate later

13. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.13 Sample Size Calculations  Provide numbers for a range of scenarios and various combinations of parameters (e.g., for various values of the difference between groups, combinations of  and , etc.

14. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.14 Sample Size Calculations  Expect things to go wrong and plan for this BEFORE the study begins (and in sample-size calculations).  Think about the challenges of reality and the things that they don’t normally teach you about in the classroom.  Sample size estimates need to incorporate adjustments for real problems that occur but are not often a part of the standard formulas.

15. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.15 Sample Size Calculations  Each of these can affect sample size calculations:  Missing data (imputation?)  Noncompliance  Multiple testing  Unequal group sizes  Use of nonparametric testing (vs. parametric)  Noninferiority or equivalence trial  Inadvertent enrollment of ineligible subjects

16. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.16 Adjustment for Lost-to-Follow-up  Calculate sample size N.  Let x=proportion expected to be lost-to-follow- up.  N adj =N/(1-x)  One still needs to consider the potential bias of examining only subjects with non-missing data.  Note: no adjustment is necessary if you plan to impute missing values. If you use imputation, an adjustment for dilution effect may be warranted (for example when treating anyone with missing data as a “failure” or a “non-responder”).

17. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.17 Adjustment for Dilution Effect  Adjustment for dilution effect due to non-compliance or inclusion (perhaps inadvertently) of subjects that cannot respond:  Calculate sample size N.  Let x=proportion expected to be non- compliant.  N adj =N/(1-x) 2

18. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.18 Adjustment for Unequal Allocation  Adjustment for unequal allocation in two groups:  Let Q E and Q C be the sample fractions such that Q E +Q C =1.  Calculate sample size N bal for equal sample sizes (i.e., Q E =Q C =0.5)  N unbal =N bal ((Q E -1 +Q C -1 )/4)  Note power is optimized when Q E =Q C =0.5

19. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.19 Adjustment for Nonparametric Testing  Most sample-size calculations are performed using parametric methods  Adjustment for use of nonparametric test instead of a parametric test (Pitman Efficiency) in case parametric assumptions do not hold (Note: applicable for 1 and 2 sample t-tests):  Calculate sample size N par for 1 or 2 sample t- test.  N nonpar = N par /(0.864)

20. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.20 Adjustment for Equivalence/Noninferiority Studies  Calculate sample size for standard superiority trial but reverse the roles of  and .  Works for large sample binomial and the normal. Does not work for survival data.

21. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.21 More Adjustments?  Additional adjustments may be needed for stratification, blocking, and matching.

22. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.22 Example ACTG A5090  A Phase II, Placebo-Controlled, Double-Blind Study Of The Selegiline Transdermal System (STS) In The Treatment Of HIV-Associated Cognitive Impairment  Insert Example_A5090.

23. Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.23 More Examples  Insert More_examples