1. The Importance of Adequately Powered Studies
William S. Weintraub, MD
Christiana Care Health System

2. I have no real or apparent conflicts of interest to report.
William S. Weintraub, MD
I have no real or apparent conflicts of interest to report.

3. Power and P Value
Power: The probability of finding a difference if there really is one
P Value: The probability of finding a difference is one does not really exist
Both are inferential statistics

4. Inferential Statistics?
Sample from a large unknown population
Sample
Underlying Population

5. What do P Value and Power Mean?
Power is the Sensitivity
Probability of finding a difference if there is one
P Value is the False Positive Rate
(1- Specificity)
The probability of finding a difference is one does not exist
What we really want in Predictive Value:
Given the results of the trail, the probability that there is a difference between arms

6. Frequentist vs Bayesian Statistics
Frequentist approaches offer statistical inference without knowledge of the underlying population
However the metrics are limited and often misinterpreted
Bayesian statistics offer meaningful metrics, but require assessment of prior knowledge of the underlying population

7. Types of Error
Finding a difference if there really isn’t one, α error, p less than specified limit (often 0.05).
Find no difference if there really is one, β error.
Both are related to small sample size.

8. How Do You Assure Adequate Power During Design
Power Calculation:
Determine power required (generally 0.80 to 0.95)
Determine p value required (generally 0.05 two tailed)
Determine clinically meaningful difference
Algorithm will find patients required to achieve power

9. Example: AIM HIGH
The Role of Niacin in Raising HDL-C to Reduce Cardiovascular Events in Patients with Atherosclerotic Cardiovascular Disease and Optimally Treated LDL-C
Randomized Trial of Niacin+Simvastatin vs Placebo+Simvastatin, composite outcome
With an estimated sample size of 3,300, the trial had 99% power to detect a 25% reduction in primary outcome in the niacin arm compared to placebo arm, assuming a hazard rate in the placebo arm and using one-sided P value of .025.

10. Study Design ER Niacin + 40-80 mg/day simvastatin
Placebo mg/day simvastatin R
Follow to end of study -2 -1 1 2 3 6 12
Months Relative to Randomization

11. Endpoints Primary Outcome Composite (Time to First Occurrence):
Coronary Heart Disease Death
Non-Fatal MI
Ischemic (Non-Hemorrhagic) Stroke
Hospitalization for ACS
Symptom-Driven Revascularization
Secondary Composite Endpoints:
CHD Death, Non-Fatal MI, Ischemic Stroke, or Hospitalization for High-Risk ACS
CHD Death, Non-Fatal MI or Ischemic Stroke
Cardiovascular Mortality

12. Statistical Analyses
Event-driven trial with projected 800 primary outcomes; year follow-up (mean 4.6 years)
85% power to detect a 25% reduction in the 5- component primary endpoint (one-sided test of significance; alpha level=0.025)
Pre-specified, conservative asymmetric boundaries for potential early stopping based on efficacy/lack of efficacy
Trial stopped on 5/25/11: lack of efficacy and concern of ischemic stroke imbalance with niacin after a 36-month average follow-up

13. After the Trail Use Confidence Interval to Assess Power
Original Power Calculation Offers No Additional Information

14. Primary Outcome 50 Combination Therapy 40 Monotherapy 30
HR 1.02, 95% CI 0.87, 1.21
Log-rank P value= 0.79
Cumulative % with Primary Outcome
16.4% 20
16.2% 10 1 2 3 4
N at risk
Time (years)
Monotherapy
1696
1581
1381
910
436
Combination Therapy
1718
1606
1366
903
428

15. Confidence Interval / Credible Interval
95% Confidence Interval: If the procedure for computing a 95% confidence interval is used over and over, 95% of the time the interval will contain the true parameter value.
95% Credible Interval: Given the assumptions of the prior distribution and results, there is a 95% probability that the true value lies within the interval

16. What About Additional Endpoints and Subgroups?
Watch out for multiple comparisons
Generally underpowered as power calculations are for the composite overall

17. Primary and Secondary Endpoints
All Cardiovascular Death
non-fatal MI or ischemic stroke
Composite of CHD Death,
hospitalization for high-risk ACS)
(CHD death, non-fatal MI, ischemic stroke,
Original Primary Endpoint
or Cerebral Revascularization
Symptom-Driven Coronary
Hospitalization for ACS
Ischemic Stroke
Non-fatal MI
CHD Death
Primary Endpoint
P=0.11
0.5 1
1.5 2
2.5 3
3.5
Niacin better
Niacin worse

18. Pre-Specified Subgroups
Overall
Age ≥ 65 years
Age < 65 years
Men
Women
Diabetes
No Diabetes
Metabolic Syndrome
No Metabolic Syndrome
Prior MI
No Prior MI
ON Statin at Entry
OFF Statin at Entry
0.5 1
1.5 2
Niacin better
Niacin worse

19. Conclusions
Underpowered studies are subject to β errors, failing to find a difference when there is one
Small studies are also subject to α error, finding a difference when there isn’t one – sometimes called the “tyranny of small numbers”
Power calculation is critical to design, but look at the data after the trial