1. Core Research Competencies:
Sample Size, Why and How?
Presented by
Lawrence Mbuagbaw, MD, MPH, PhD
19th July 2016

2. Outline Introduction Why should we estimate a sample size?
What information is required to estimate a sample size for a randomized trial?
An example of sample size estimation for a randomized trial
How do we report a sample size estimation?
Other considerations

3. Introduction
How much data should I collect to answer the question of interest and have faith that the answers are correct?
Sample size calculation is an important part of the study design to ensure
Validity
Accuracy
Reliability
Scientific and ethical integrity of a study
Sample Size calculation should be performed prior to conducting the study

4. Introduction
We need samples because we can’t include every body in a study
Too cumbersome
Too expensive
May not be safe
Impossible
Yet it is possible to estimate a population parameter using statistics from a sample

5. Economic reasons Undersized Study: Oversized Study:
Unnecessary waste of resources for a study that would produce no answers
Oversized Study:
Unnecessary waste of resources a study that would produce significant results with no practical importance

6. Scientific reasons
Undersized Study:
If a study with negative results doesn’t have a sufficient sample size to detect a clinically important effect, then the negative results are interpretable
The treatment did not have an effect at least as large as the effect considered to be clinically relevant OR
The treatment had no effect
Oversized Study:
If the study has positive results, we may be detecting statistically significant effects with no clinical importance.

7. Ethical reasons Undersized Study: Oversized Study:
Expose subjects to unnecessary (sometimes potentially harmful or futile) treatments without the capability to advance knowledge
Oversized Study:
Expose an unnecessarily large number of subjects to potentially harmful or futile treatments

8. Sample Size requirements (RCT)
Study design
Study hypothesis
Nature of outcome
Statistical methods
Measure of variation
Number of tails
Important difference
Significance / power
Parallel two-arm individually randomised trial
Difference
Binary outcome
Chi-squared test
Mostly important for continuous outcomes and will be the standard deviation

9. Sample Size requirements (RCT)
Study design
Study hypothesis
Nature of outcome
Statistical methods
Measure of variation
Number of tails
Important difference
Significance / power
Two-sided Test
One-sided Test
*Recommend: two-sided test

10. Sample Size requirements (RCT)
Study design
Study hypothesis
Nature of outcome
Statistical methods
Measure of variation
Number of tails
Important difference
Significance / power
Comparative Trials
Minimal clinically important difference (MCID)
*MCID
Can be considered as the smallest change or difference in an outcome that is perceived as beneficial and would lead to a change in the patient’s management, assuming an absence of excessive side effects and costs

11. Sample Size requirements (RCT)
Study design
Study hypothesis
Nature of outcome
Statistical methods
Measure of variation
Number of tails
Important difference
Significance / power
Significance level: 0.05
Power: 80% or 90%
Conclusion from trial
Reality
Interventions
different
not different
Interventions different
(true positive)
Power=1-beta
(false positive)
Type 1 error
Alpha error
Significance level
p-value
(false negative)
Type 2 error
Beta error
(true negative)

12. Sample size requirements
If you have thought about all these things as they relate to your study, then you are ready to start estimating a sample size

13. Example:
Consider the research question:
Do invitation cards compared to usual care increase male partner attendance at antenatal care?

14. Sample size requirements
Study design: parallel group randomized trial
Study hypothesis: invitation cards are different from usual practice
Nature of outcome: binary (proportion of women who come with their male partners)
Statistical methods: Chi-square test
Measure of variation: not applicable (we can use a range of values for the important difference)
Number of tails: two
Important difference: 10% (12% in intervention and 2% in control)
Significance/power: alpha=0.05; power =90%

15. Step 1: Parameters Event rate in treatment group: 12% (p1)
Event rate in control group: 2% (p2)
Difference: 10% (p1-p2)
Relative Risk: 6.0 (p1/p2)
Level of significance: 0.05 (two-tailed)
Power: 90%

16. Step 2: Formula or software
Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr ;365(9467):

17. Step two: Formula Sample required in each arm
Constant derived from and 90%
Risk Ratio= P1/P2
Event rate in control group
Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr ;365(9467):

18. Step 3: Numerical application
Note that all the existing formulae do not produce identical results!
The differences are not great!
Total sample size= 132 x 2 =264

19. Step 4: What to report Sample size total and per group
Intervention/control group rate and difference
Measure of variance (for continuous outcomes or if you are reporting a range of values for binary outcomes)
Source of 2 and 3
Power, level of significance
Test used
Hypothesis
Tails
Attrition
Software or formula
Range of values for different estimates

20. Step 4: Reporting
Sample sizes of 132 per group (264 in total) are required to achieve an 90% power to detect a difference of 10% (derived from previous studies) between the intervention and control groups, assuming an event rate of 12% in the intervention arm and 2% (RR=6.0) in the control arm, at a level of significance set at These computations were done based on a chi-squared test of the null hypothesis that there is no difference between the intervention and control groups. After correcting for 15% attrition, a total sample size of (152 per group) is required.
The test statistic is two-sided meaning that results in either direction can be interpreted.
Computations were done using the formula by Schulz et al.

21. Step 4: More on reporting: attrition
Should be avoided
10-15% may be appropriate
Up to 20% usually frowned upon
Integrate in SS estimation
Method 1: N= (15/100 *n) +n, where N is the final sample size; and n is the initial sample size.
N= (15/100 *264) + 96 = 303.4= 304
Method 2: N= n/1-attrition =264/1-0.15= 310.5=311

22. Step 5: Consult a biostatistician
To verify your estimations
To learn more
Should always be involved in any RCT
Will need one for analysis
Your paper will be verified by one
Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. Apr ;365(9467):

23. Other considerations Costs
Consider a pilot study to obtain parameters for sample size estimation
Avoid retrospective planning
It is BAD science
Attrition or loss to follow-up can be avoided at the design stage
High attrition defeats the purpose of sample size estimation

24. Resources for sample size estimation
Online resources:
Free software:
Winpepi:
G*Power

25. Acknowledgments For CTN investigators
This work is supported by the CIHR Canadian HIV Trials Network