1. © Nancy E. Mayo 2004 Sample Size Estimations Demystifying Sample Size Calculations Graphics contributed by Dr. Gillian Bartlett

2. © Nancy E. Mayo 2004 Choosing the Study Population Question Background Reasonable Question Population

3. © Nancy E. Mayo 2004 Study Population ? Sample Size Exposure Outcome Confounding Analysis

4. © Nancy E. Mayo 2004 COMMON QUESTIONS 1.How many subjects (specimens) do I need? 2.How do I analyze my data? 3.What do I put in the data analysis section? COMMON ANSWERS 1.What is your question? 2.What is your outcome? 3.How is it measured? 4.How big an effect do you want to see? 5.Is the effect meaningful?

5. © Nancy E. Mayo 2004 Clinically Meaningful Change Meaningful to whom? Clinician - usually impairments Patient – function (disability), quality of life Society - health services utilization, cost Payer – disability, prescription medication

6. © Nancy E. Mayo 2004 Clinically Meaningful Change Norm referenced –refers to changes that would put someone within normal values or within a % of normal Criterion referenced –change anchored in future benefit –change is associated with increased probability of distant outcomes –relevant when impact is on pathology but benefit not reaped for years

7. © Nancy E. Mayo 2004 Clinically Meaningful Change Content referenced –for outcomes measured by scales –translates change into what would have had to have changed on the scale –e.g. 5 points on Barthel Index - changed 1 level on 1 item. Minimally detectable change –Subjects can detect improvement

8. © Nancy E. Mayo 2004 How BIG is BIG? Effect size Effect size: ratio of change to variability 0.2 - 0.3 – small 0.5 – moderate 0.8 - large

9. © Nancy E. Mayo 2004 signal is difficult to detect against excessive background noise Change greater than “noise”

10. © Nancy E. Mayo 2004 Raw vs. Cooked Data (order rare)

11. © Nancy E. Mayo 2004 Examples of the Pitfalls of Cooking Data

12. © Nancy E. Mayo 2004 Sample Size Formula = SD / delta Effect size = delta / SD Delta = difference DEMYSTIFIED

13. © Nancy E. Mayo 2004 Relationship between Effect Size and Sample Size 0 0.51 1.5 2 2.5 100 80 60 40 20 0 Sample Size per Group Effect Size (Two group design)

14. © Nancy E. Mayo 2004 Calculation of Sample Size for Comparing Two Independent Means ( z a – z b ) SD n = 2 ___________ x exp - x con Where: Z a = z value for the risk of a Type I error (significance level) 1.96 for 0.05 Z b = z value for the risk of a Type II error (power) 1.96 for 0.95 (two-tailed) -1.65 for 0.95 (one-tailed) SD = standard deviation of outcome in the general population x con = mean of control group x exp = mean of experimental group n = number of subjects per group 2

15. © Nancy E. Mayo 2004 Calculation of Sample Size for Comparing Two Independent Proportions n = z a √ 2 p con (1 - p con ) – z b √ p exp (1 – p exp ) + p con (1 - p con ) 2 ______________________________________________ p exp - p con Where: z a = z value for the risk of a Type I error (significance level) 1.96 for 0.05 z b = z value for the risk of a Type II error (power) 1.96 for 0.95 (two-tailed) -1.65 for 0.96 (one-tailed) p con = prevalence of outcome in control group p con = prevalence of outcome in experimental group n = number of subjects per group Colton (pg 168-169)

16. © Nancy E. Mayo 2004 Sample Size Required Per Group for Comparing Two Independent Means POWER.80.90.95 0.50 (2.0)578 1.0 (1.0)172327 1.25 (0.8)263442 1.50 (0.67)374960 2.0 (0.5)6086105 Ratio of SD to difference ∆ between means (∆/SD)

17. © Nancy E. Mayo 2004 Sample Size Required Per Group for Comparing Two Independent Proportions: 80% Power Prevalence of outcome in experimental group

18. © Nancy E. Mayo 2004 Prevalence of outcome in experimental group Sample Size Required Per Group for Comparing Two Independent Proportions: 95% Power

19. More complex data situations Convert each component to the simple 2- group comparison or correlation Estimate (calculate) sample size for the contrast that has the smallest effect and build up Remember if using correlation as the base, you are not testing it against 0 you are testing it against a correlation that you do not think is important © Nancy E. Mayo 2004

20. …More Consider the impact on power to maintain a given effect size if other variables are in the model © Nancy E. Mayo 2004

21. Regression Green indicates that adequate power (80%) can be achieved for moderate effect sizes with a sample size N > 50 + 8m, where m is the number of covariates to be modeled. © Nancy E. Mayo 2004

22. Adjustment only no parameters are estimated no hypotheses tested to maintain the same degree of power, only 1 additional subject is required per level (l) or per degree of freedom (df) inherent in the co- variate © Nancy E. Mayo 2004

23. Summary Variable under study N > 50 + 8m (moderate effect size, 80% power) Adjustment only Continuous = + 1 per covariate df Dichotomous = 5-9 events per covariate Sub-group analysis Sample size for main effect * 4 for interaction with group © Nancy E. Mayo 2004

24. Marking Scheme for Protocol Background 10 Question 5 Population 5 Design 5 Procedures 5 Measures 10 Analysis 5 Sample Size 5 Bonus points – above and beyond the call of duty