1. Statistical Considerations for Research: Power and Sample Size
Jennifer Massa, DSc
Ariel Mueller, MA
February 10, 2015

2. CARE Who Are We and How We Can Help?
CARE is a one-stop shop for department members who want help conducting research.
Our goal is to simplify and streamline research endeavors in the department, and help make you more successful in your research.

3. CARE Center: Resources
Development of a research idea
Patient screening & recruitment strategies
Study design
Data collection and source documentation
Mentorship facilitation
Audit readiness
Protocol writing
Research regulation education, team training
IRB communications
Database and Case Report Form building
Manuscript & grant writing
Sample processing / storage
Survey design
Data analysis

4. Statistical Considerations
Power and Sample Size
Sample size / power is an important aspect to consider throughout your trial
Study design
Data collection
Statistical analysis
Manuscript & grant preparations
Response to reviewers

5. What is Power and Sample Size?
Sample size is the number of observations in a sample
It is not always possible to look at the population as a whole
Our hope is that the sample is representative of the entire population, because we use the sample to make inferences about the population
The power or sensitivity of a statistical test is the probability that it correctly rejects the null hypothesis (H0) when it is false.
Power is the ability to correctly find a statistical difference in our sample when there is a true statistical difference in the population

6. Two hypotheses Ho: µ1 - µ2 = 0 HA: µ1 - µ2 ≠ 0
Truth
Conclusion
H0 is True
HA is True
Accept H0
No Statistically Significant Difference
True Negative
Type II Error, β
Reject H0
Statistically Significant Difference
Type I Error, α
True Positive
The Research Question
Two hypotheses
Ho: µ1 - µ2 = 0 HA: µ1 - µ2 ≠ 0

7. α = P (Type I Error) = P (Reject Ho | Ho is true)
False-positive
Only possible with a statistically significant difference
Type I error is controlled by choosing an appropriate alpha level (generally 0.05)

8. β = P (Type II Error) = P (Accept Ho | Ho is false)
Power = 1 - β
Type II error, β
False-negative
Only possible with null result, or no statistically significant difference
Type II error is controlled by choosing an appropriate power level (generally 0.80)
NIH Studies require 90% power

9. Relationship Between Type I and Type II Error
As α decreases, β increases and power decreases
As sample size increases, power increases

10. Power and Sample Size Approaches
Choose desired power and determine the number per group (N)
Used when designing your trial
Planning studies, sponsor trials, FDA studies, RCTs
Choose N and then assess the power of your study
Often used after completion of your study
QI or Pilot studies with a convenience sample (ex. survey all MICU-6 physicians and nurses)
Manuscript Preparations / Response to Reviewers

11. Importance in Clinical Trials
Why Does Sample Size Matter?
Sample Size is the building block of a good trial
Having the appropriate sample size / power allows us to confidently find associations in our sample
If our trial is underpowered we will not have the statistical power to find a statistically significant association, given that there is a true relationship

12. IRB Submission How to Survive a Sample Size Calculation
Section B3, Part B, Statistical Considerations
The information required for a sample size calculation depends on several aspects of the analysis plan:
What type of outcome
Binary, Continuous, Time to Event
How the outcome is measured
At one time or at multiple time points
Others (type of trial, resources, etc.)
Others may include the feasibility of a sample size calculation (for example, we would hope for 40 in each group but there are only 18 residents / year). This leads to a limitation of the study.

13. IRB Submission Required Information Binary Outcome Continuous Outcome
Proportion of ‘successes’ in each treatment group
Desired α and power levels
Continuous Outcome
Mean and standard deviation for each treatment group
Find this information from similar previous studies or pilot data
Others may include the feasibility of a sample size calculation (for example, we would hope for 40 in each group but there are only 18 residents / year). This leads to a limitation of the study.

14. An Example: Binary Outcome
The MPI Study
The purpose of was study is to evaluate the relationship between myocardial performance index (MPI) and mortality among patients with sepsis.
They hypothesized that patients with absolute worsening MPI from 0 to 24 hours would have increased mortality.

15. The MPI Study Binary Definition of MPI Predictor
MPI = (ICT + IRT) / ET
ICT: Isovolumetric contraction time
IRT: Isovolumetric relaxation time
ET: Ejection Time
MPI is continuous, however investigators expect lots of variability between patients
Because of this, they decided to use absolute worsening (MPI at 24 hours – MPI at Enrollment as a binary variable) as the predictor of mortality

16. The MPI Study Pilot Data
Investigators previously looked at longitudinal strain among patients with septic shock
During this study they also measured MPI for those patients
59 patients were enrolled in the pilot study
18 patients had worsening MPI
41 patients did not have worsening MPI
17 of the 59 patients died
8 had worsening MPI (44.44% mortality rate)
9 had preserved MPI (21.95% mortality rate)

17. The MPI Study Sample Size Calculation Power Proportion MPI Worsening
Proportion Without MPI Worsening
Sample Size Per Group
0.80
0.4444
0.2195 68
0.90 91
0.40
0.20 82
109
0.25
152
203

18. The MPI Study Sample Size Calculation Statistical Considerations
Power calculation revealed 68 patients were required in each group to obtain at least 80% power
Practical Considerations
We calculated the sample size using the exact proportions we observed
It is likely that the proportions we observe may not be identical to the pilot data
Based on these calculations and available resources, researchers aimed to enroll 100 subjects in each arm (200 total participants).

19. An Example: Continuous Outcome
The FiO2 Study
The purpose of was study is to evaluate the relationship between the level of administered oxygen and arterial partial oxygen pressure and postoperative neurocognition after cardiac surgery.
They hypothesized that cardiac surgical patients who undergo normoxic conditions throughout the intraoperative and early post-operative period will have better neurocognitive function than those with maintenance of hyperoxia.

20. The FiO2 Study Study Design
Patients were prospectively randomized to one of two experimental groups (normoxia or hyperoxia)
Study ventilator settings were applied immediately after induction of general anesthesia and continued throughout surgery and the subsequent ICU stay.
The primary outcome was MMSE score at the end of the study.
MMSE: Mini-Mental State Exam, a measure of neurocognition

21. The FiO2 Study Previous Data
Previous Paper in NEJM about post-op neurocognition among CABG patients
Showed a mean decline in MMSE in post-cardiac surgical patients from baseline of 1.5 points with a standard deviation of 3

22. The FiO2 Study Sample Size Calculation 80% Power Alpha = 0.05
Sample size calculation estimated mean differences in MMSE ranged from 0.5 to 2
SD of 3 in the normoxia group
SD of 3.5 in the hyperoxia group (taking into account possibility of increased variance in this group)

23. The FiO2 Study 1/8/15 Power % Effect Size Sample size in each arm
Average MMSE score in normoxia group
Average MMSE score in hyperoxia group 80 26 24 17 85 19 90 23
24.5 29 34 41 25 66 76 91
25.5
263
306
364

24. The FiO2 Study Sample Size Calculation Statistical Considerations
Power calculation revealed 29 patients were required in each group to obtain at least 80% power
Practical Considerations
To negate the possibility of patients lost to follow-up/dropouts, physicians opted to include a few additional patients in each group
Based on these calculations and considerations, 35 subjects in each arm (70 total participants) should be recruited to avoid type I and II error.

25. Understanding the Limitations
Theory vs. Practice
In all cases we strive to achieve a correctly powered trial
We recognize that this often isn’t possible!
No pilot data / previous studies
Limited resources
In this case, we do the best we can and recognize it as a limitation
Convenience samples
Feasibility trials
In the case of pilot studies, we treat feasibility as the primary goal, recognizing that future research is necessary to achieve the results with sufficient power.

26. Additional Statistics Resources
Enter your HUID or eCommons login and password
Type ‘biostatistics’ in the search bar
Additional Statistics Resources
Harvard Catalyst Video Education Library

27. Additional CARE Resources
To Apply
Balachundhar Subramaniam, MD MPH
CARE Director
Valerie Banner-Goodspeed
CARE Research Administrator
The CARE Application can be found at:

28. Questions and Contact Information
CARE Statisticians
Jennifer Massa, DSc
Epidemiologist
Ariel Mueller, MA
Biostatistician