Mark Pletcher 6/10/2011 Quantifying Treatment Effects

Rationale Any treatment involves tradeoffs Weigh benefits against risks/costs Benefit $$ Harm

Rationale Sometimes the decision is difficult! Benefit $$ Harm

Rationale Benefit $$ Harm How big is this box? And this one?

Rationale Tests can help us understand who is most likely to benefit from a treatment Benefit $$ Harm How big is this box? And this one?

Rationale Tests can help us understand who is most likely to benefit from a treatment Rapid strep to decide who will benefit from penicillin BNP to decide who will benefit from furosemide CRP to decide who will benefit from statins

Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself

Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself The topic for this lecture

Outline Is an intervention really beneficial? How beneficial is it? Pitfalls Examples

Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days

Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days *Statistics* are used to decide if should reject the null hypothesis and accept that the intervention is beneficial

Is the intervention beneficial? But statistics cannot help us interpret effect size

Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers?

Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers? Example: MI in 10% vs. 15% Q: What do we do with these two numbers?

Quantifying the Benefit Two simple possibilities: 10% / 15% = % - 10% = 5%

Quantifying the Benefit Two simple possibilities: 10% / 15% = % - 10% = 5% Relative Risk (RR) Absolute Risk Reduction (ARR)

Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small?

Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime

Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime Medium Big Small

Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime RR is NOT the best measure of effect size

Quantifying the Benefit Absolute risk reduction (ARR) is better ARR = Risk difference = Risk2 – Risk1

Quantifying the Benefit Absolute risk reduction (ARR) is better RRARR MI: 10% vs. 15% in 10 years.665% Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

Q: What does the 34% reduction mean?

Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11%

Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11% What is 34%?

Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11% Relative risk reduction (RRR) = 1 – RR = =.34 or 34%

Quantifying the Benefit RRR is no better than RR RRRRR MI: 10% vs. 15% in 10 years.6634% Death: 50% vs. 75% in 3 years.6634% Basal Cell CA: 2% vs. 3% in lifetime.6634%

Quantifying the Benefit RRR is ALWAYS bigger than ARR (unless untreated risk is 100%)

Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? We reduced risk by 34% Risk was 34% lower

Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? We reduced risk by 34% cant tell Risk was 34% lowercant tell Very hard to be unambiguous!

Quantifying the Benefit Another reason that ARR is better: Translate it into Number Needed to Treat NNT = 1/ARR

Why is NNT = 1/ARR? 67 no stroke anyway 22 strokes with Nimotop® 11 strokes prevented 22 strokes with with treatment 33 strokes with no treatment 100 SAH patients treated R2 R1

Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated1 treated9.1 treated 11 prevented.11 prevented1 prevented ==

Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated1 treated9.1 treated 11 prevented.11 prevented1 prevented == 1/ARR= NNT

Why is NNT 1/ARR? NNT best expressed in a sentence: Need to treat 9.1 persons with SAH using nimodipine to prevent 1 cerebral infarction

Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661%

Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661% 100

Quantifying the Benefit NNT expression practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661% 100 Statins Chemo Sunscreen every day

Quantifying the Benefit NNT expression practice Need to treat 20 patients with statins for 10 years to prevent 1 MI Need to treat 4 patients with chemo for 3 years to prevent 1 death Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell

Example 1 Randomized controlled trial of the effects of hip replacement vs. screws on re-operation in elderly patients with displaced hip fractures. Parker MH et al. Bone Joint Surg Br. 84(8):

Example 1 Re-operationNo Re-operation Hip Replacement Internal Fixation with Screws Parker MH et al. Bone Joint Surg Br. 84(8):

Example 1 Re-operationNo Re-operationRisk Hip Replacement /229 =5.2% Internal Fixation with Screws /226 =39.8%

Example 1 Re-operationNo Re-operationRisk Hip Replacement /229 =5.2% Internal Fixation with Screws /226 =39.8% RR= R1/R2= 5.2% / 39.8%=.13 RRR= 1-RR= 1-.13= 87% ARR= R2 – R1= 39.8% - 5.2%= 34.6% NNT= 1/ARR= 1/.346= 3 Need to treat 3 patients with hip replacement instead of screws to prevent 1 from needing a re-do operation

Example 2 JUPITER: Randomized controlled trial of high dose rosuvastatin in patients with LDL 2.0 Ridker et al. NEJM 2008; 359:

Example 2 Ridker et al. NEJM 2008; 359:

Example 2 Ridker et al. NEJM 2008; 359:

Example 2 HR= (R1/R2)(from regression)=.56 RRR= 1-HR= 1-.56= 44% ARR= R2 – R1= =.59 / 100py* =.0059 / py NNT= 1/ARR= 1/.0059 = 100/.59= 169 pys Need to treat 169 patients for a year to prevent 1 CVD event Or better: Need to treat 85 patients for 2 years to prevent 1 CVD event (average treatment duration in trial was 1.9 years) * py = person-years

Example 4 Warfarin vs. placebo for atrial fibrillation WarfarinPlacebo Risk of major bleed (/yr)1.2%0.7% Ann Intern Med 1999; 131:

Example 4 Warfarin vs. placebo for atrial fibrillation RR= R1/R2= 1.2% /.7%= 1.7 RR (flipped) = R2/R1=.7% / 1.2%=.59 RRR (flipped) = 1-RR= = 41% ARR= R2 – R1=.7% - 1.2%= -.5% ARI – Absolute risk increase = 0.5% NNT= 1/ARR= 1/-.5%= -200 NNH – Number needed to harm = -NNT = 1/ARI = 200 If you treat 200 Afib patients with warfarin, you will cause 1 major bleed

Circling back to test utility… Tests help determine: If the RR applies Treatment for a disease doesnt help if you dont have the disease! Interactions (RR is higher or lower than average) Statins more effective if CRP is high? Patients with gene XYZ more likely to have a side effect Baseline risk The higher the risk, the larger the ARR, the smaller the NNT

Key Concepts Test utility depends on how good the treatment is RR and p-values good for hypothesis testing/statistics ARR and NNT (and NNH) better for interpreting clinical importance ARR = risk difference NNT = 1/NNT Beware RRR and ambiguous language