1. Thrombolysis for Acute Ischemic Stroke?
Jerome R Hoffman, MA MD
Professor of Medicine Emeritus
UCLA School of Medicine

2. TPA for Stroke: Benefit vs Harm
Does it benefit any patients?
Does it harm any patients?
Does it benefit more patients than it harms?

3. Defining Outcomes
Efficacy: Outcomes obtained under ideal circumstances
Effectiveness: Outcomes obtained in “real world” circumstances

4. Threats to Efficacy- External Validity
Multiple studies:
1 (now 2) = “positive”
Most = neutral
A few = “negative”
How to explain … ?
NINDS:
if it’s 2 studies, 1st = data snooping
OR instead of RR
Should we throw out the other negative studies?
effect of chance p-multiple trials
is STK likely to be worse?
Attempts to dredge “+” outcomes from the other studies
“target population”
≥1 FP study
Vesnarinone
Magnesium
LMWH for stroke

5. The Original RCTs RCT Date # Time Mortality Fxn Other tPA ECASS I
Lancet 10/95
620
0-6h
18 / 13%
no difference
(except … “target population”)
NINDS
NEJM 12/95
624
0-3h
17 / 21%
13% better
ECASS II
Lancet 10/98
800
6.1 / 4.9%
90d
≤minimal difference (p=0.3)
no difference in 20% treated in <3h
ATLANTIS
JAMA 12/99
613
3-5h
11 / 7%
no difference (1/3 each)
PART A
Stroke 4/00
142
18 / 4%
>22 / 7%
stopped early
STK
MAST-I
Lancet 12/95
622
27 / 12%
MAST-E
NEJM 7/96
310
34 / 18%
80% bad outcomes
“killed off worst”
ASK
JAMA 9/96
340
0-4h
RR ~2
much worse
3 vs 4h

6. Threats to Efficacy- External Validity
Multiple examples of overly optimistic (single or grouped) small studies
Magnesium
Ancrod
What does it mean when a couple of studies are “+” … a bunch are neutral … & a few are “-”
Imagine flipping 1 penny … or 10 different pennies
NINDS:
if it’s 2 studies, 1st = data snooping
OR instead of RR
Should we throw out the other negative studies?
effect of chance p-multiple trials
is STK likely to be worse?
Attempts to dredge “+” outcomes from the other studies
“target population”
≥1 FP study
Vesnarinone
Magnesium
LMWH for stroke

7. Threats to Efficacy – Internal Validity Artificial recruitment times in NINDS
Marler, Neurology 12/00

8. Threats to Efficacy – Internal Validity Major differences in baseline severity
“TPA > placebo” in all NINDS patients at FINAL OUTCOME
placebo
tpa
In ’ subgroup, TPA >> placebo AT BASELINE
Least Severe Most severe
placebo
tpa
Marler, Neurology 12/00

9. Threats to Effectiveness
Stroke mimics
Common
Much more likely to “meet criteria” for tPA
Inaccurate CT reads
Missed hemorrhage
Unreliable estimates of “large” territory
Pressure to enroll patients

10. Effectiveness Studies: “Great!”
Tiny ones
“consistent with NINDS … consistent with anything”
Industry sponsored
Methodologic concerns
Publication bias
3 big ones: STARS, CASES, SITS-MOST
What they claim vs what they actually found
Point out 56/83 ATLANTIS centers in STARS.

11. Effectiveness Studies: “Awful!”
Non-selective, no publication bias, not ® sponsored
Large #s, very frequent protocol violations
Terrible outcomes (even when no protocol violation)
Katzan JAMA 3/00, Bravata Arch IM 9/02, Bateman Stroke 2/06,
Deng Neuro 2/06, Dubinsky Neuro 6/06
ｷ ICH in 1st 24h in only 1.7%;
ｷ 3mo mortality = 11.3%, vs 17.3% in NINDS + ECASS I & 2 + ATLANTIS;
ｷ 3mo mRS in 94.6% – for comparison, by d7 they were unable to measure NIHSS in 1/6) = 54 & 39% (vs 49 & 42% in the RCTs).

12. The AHA Brouhaha Money to the AHA Choice of experts
“Disclosure” of conflicts of interest
Suppression of dissent

13. NINDS revisited After 10 years hidden … let’s look at the raw data
Point out statistical shenanigans re ORs & re linear regression.

14. The cumulative distribution of change in NIHSS score from study onset to 90 days for all study subjects in each treatment group is depicted. Negative numbers indicate deterioration, positive values improvement. The graphs are similar; roughly 25% of subjects in each group deteriorate. For patients who deteriorated the curve for tPA sits just above the curve for placebo, reflecting the fact that there were slightly fewer patients who had extreme deterioration after having received tPA. Given the small numbers involved, and the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

15. Figure 2 repeats the layout of Figure 1 but adds stratification by time-to-treatment. All four curves are similar, but among patients who deteriorated the curves with earlier treatment lie below those with treatment at minutes, suggesting that those patients fared slightly worse. A small excess number of patients who received tPA between had dramatic improvement, as witnessed by the slight separation of this curve from the others at the extreme right of the graph. Given the small numbers involved, and the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

16. The 90 day change in NIHSS score for each study subject is graphed against the time from stroke onset to drug administration. Points are randomly jittered slightly in both dimensions to minimize superimposition. Patients signified with a “+” were dead at 90 days.  Lines represent lowess (locally weighted scatterplot smoothing) regression for each treatment.  The graph demonstrates that change in NIHSS score at 90 days was marginally better for the tPA group, particularly among the very few patients treated well before 90 minutes. There is otherwise no evidence of differential effect by time to treatment (which would require a curve for tPA that sloped downward, from left to right). Given there were only 8 placebo and 12 tPA patients treated by 75 minutes, when the curves appear most separate, as well as the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

18. Mean 90 day ∆ in NIHSS score*
Placebo tPA mean difference
≤90’ (-1.3, 4.8)
91-180’ (-1.9, 3.9)
Total (-0.7, 3.5)
*Positive values indicate improvement; ∆-NIHSS ranges from -42 to +42

19. % with ≥4 point improvement at 90d
Placebo tPA Difference (95% CI)
≤90’ (-9, 13)
91-180’ (-11, 10)
Total (-7, 9)

20. This figure repeats Figure 1 but includes only those patients who were alive at 90 days. In contrast to Figure 1, for patients who deteriorated the curve for placebo now sits above that for tPA, suggesting that any suggested benefit in Figure 1 is due to the 10 extra deaths in the placebo group (see text). Given the small numbers involved, and the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

21. Figure 6 repeats Figure 2 for patients alive at 90 days
Figure 6 repeats Figure 2 for patients alive at 90 days. The 4 curves for survivors are virtually indistinguishable, although the minute tPA group had a few more patients with marked deterioration and a few more patients with marked improvement than the other 3 groups. Given the small numbers involved, and the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

22. This figure repeats Figure 3 for those who survived till 90 days
This figure repeats Figure 3 for those who survived till 90 days. The graph reveals that almost all patients who don’t die improve, and that patients with similar baseline NIHSS scores have very similar outcomes regardless of treatment. Given the small numbers involved, and the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

23. This Figure repeats the format of Figure 4
This Figure repeats the format of Figure 4. It has the additional feature that patients who had an intracranial bleed but survived are depicted with a hollow square. In contrast to the small differences seen in Figure 4, Figure 8 demonstrates that those who lived to 90 days had virtually identical outcomes regardless of treatment.  This implies that any difference favoring tPA in Figure 4 is due to the excess deaths in the placebo group (which were almost certainly due to chance – see text). It remains possible to imagine a vague suggestion of marginal benefit with tPA given early, and marginal harm when it is given after 2.5 hours. Given there were only 6 placebo and 11 tPA patients treated by 75 minutes who were alive at 90 days, as well as the hazards of sub-group analysis, any observed differences should be viewed as hypothesis generating only.

24. Mean ∆ in NIHSS score in patients alive at 90d*
Placebo (n=248) tPA (n=258) mean difference
≤90’ (n=243) (-0.7, 2.8)
91-180’ (n=263) (-3.0, 0.7)
Total (-1.3, 1.2)
*Positive values indicate improvement; ∆-NIHSS ranges from -42 to +42

25. % with ≥4 point improvement among those alive at 90d
Placebo tPA Difference (95% CI)
≤90’ (-8, 12)
91-180’ (-16, 5)
Total (-9, 5)

26. Recent RCTs RCT Date # Time Mortality Fxn Other tPA ECASS III
NEJM 9/08
821
3-4.5h
7.7 / 8.4%
3-6% absolute benefit
DPA
Lancet Neuro 2/09
359
3-9h *
6 / 11 & 21%
slightly worse
* selected by MRI

27. IST-3 – The Largest RCT Ever
n>3K, up to 6h p-sx onset
276 patients treated /evaluated 2-blind
the other 90% done open-label, with “evaluation” by patient’s doctor … or phone call (Q-with whom) … or “letter gram” (Q-from whom) … all without any effort at validation

28. IST-3 – overall results
No statistical difference in any of multiple 6mo
However – a “positive trend” in neurologic function … using a measure they’d explicitly rejected, in print, prior to finding the results
What is the clinical meaning of a “trend” in a study of >3K subjects?
How about for a 2o outcome … done non-blind by a layman … who is also non-blind to “was there any tx” … using an unreliable measure the authors themselves rejected?

29. IST-3 – results in 2-bl subgroup
In the 276 patients treated /evaluated 2-blind …
no hint of any benefit (no hint of any trend)
obvious major harms
mortality = 11 vs 7% (NNTK=25)
sx-ICH in 7 vs 1%
fatal ICH in 4 vs <1%
increase in non–ICH swelling, deterioration not due to bleed or swelling (10 vs 8%), & need for ICU (24 vs 17%)

30. IST-3 Mortality at 6mo = 27% in both groups
suggesting that even though TPA killed a few early, it is the underlying stroke severity that dominates ultimate outcome

31. IST-3 – Authors Conclusions
“Offer tPA to all patients up to 6 hours!”
Laudatory editorial agrees!

32. Clot Removal

33. A Few Sources of Bias* Methodologic Unreliable outcome measures
mRS “by letter-gram” or phone
“stage shift”
Changing methods / changing outcomes
Selective rejection of methods … depending on results
*Bias in its standard English-language usage

34. A Few Sources of Bias* Methodologic Selective metaanalyses
Metaanalyses of outcomes not planned in the original studies
*Bias in its standard English-language usage

35. A Few Sources of Bias* Interpretive
Publication bias
Of studies
Of editorials
Of commentary within studies
Selective rejection of studies … if their results are negative (b/c they’re negative!)
*Bias in its standard English-language usage

36. A Few Sources of Bias* Interpretive
Conflating theory with evidence (DOOs with POOs)
MRI-based patient selection
Association vs cause
Ecological fallacy – the group does better … but not the patients of interest (those who got tPA) within the group!
Changing the theory when it doesn’t work
“Clot-busting” … but no effect in 1st 24h, so …
*Bias in its standard English-language usage

37. A Few Sources of Bias* Interpretive Illogical “proofs”
“every 15’ helps” (Saver, JAMA 6/13)
What they found
What one would expect with a completely worthless treatment
“the NNT is really 3 … or 2 … or less than 1!”
Everything is “positive” … regardless of results
*Bias in its standard English-language usage
Time to tPA & outcome in AIS. Saver (Fonarow). JAMA 6/13. Get c the Guidelines. 2/14
Reviewed >58K registry pts who got tPA, in med = 144’ (9% in <90’, ¾ in ’, 14% in 3-4.5h) by 15’ increments of T to tx Ù
9% in-H death, 5% ICH, 1/3 c “indep ambulation at H-d/c”, & 38% d/c’d home;
aOR for each of these = (ie improved) c each 15’ earlier T to tPA … though not in the 14% in the >3h group, who did a bit > those c intermediate T to tx).
[Forgetting ®-registry & unblinded adjusted eval of non-std outcome (they don’t report mRS …) – it would be bad enough if it took 58K pts to show this tiny benefit. But this actually shows no benefit c T … since # of TIAs âs incrementally over T, such that earlier tx should do at least that much incrementally better simply b/c of the extra TIA pts being included!]

38. A Few Sources of Bias* Ethical
Subtly changing the outcome measure … and pretending you didn’t
Excluding protocol violations in an effectiveness study
Including only some of the participants
Changing methods & outcomes mid-stream … once things look bad
“It’s unethical to do another RCT of <3h … but fine to keep trying at 6h
*Bias in its standard English-language usage
Time to tPA & outcome in AIS. Saver (Fonarow). JAMA 6/13. Get c the Guidelines. 2/14
Reviewed >58K registry pts who got tPA, in med = 144’ (9% in <90’, ¾ in ’, 14% in 3-4.5h) by 15’ increments of T to tx Ù
9% in-H death, 5% ICH, 1/3 c “indep ambulation at H-d/c”, & 38% d/c’d home;
aOR for each of these = (ie improved) c each 15’ earlier T to tPA … though not in the 14% in the >3h group, who did a bit > those c intermediate T to tx).
[Forgetting ®-registry & unblinded adjusted eval of non-std outcome (they don’t report mRS …) – it would be bad enough if it took 58K pts to show this tiny benefit. But this actually shows no benefit c T … since # of TIAs âs incrementally over T, such that earlier tx should do at least that much incrementally better simply b/c of the extra TIA pts being included!]

39. TPA for Stroke: Benefit vs Harm
Does it benefit any patients?
Does it harm any patients?
Does it benefit more patients than it harms?

40. TPA for Stroke: Benefit vs Harm
Does it benefit more patients than it harms?
Overall?
In very early patients?
With specific CT/MR findings?
Via IA route?
In expert hands?
DO WE KNOW?

41. TPA for Stroke Why are there no more 0-3h RCTs?
“We do not know how another trial would turn out, and if we do not come out ahead, we would have a terribly self-inflicted wound….[Another study] may be a good thing for America, but it wouldn’t be a good thing for us.”
Elliot Grosbard, Genentech scientist
Why are there more >3h RCTs?

42. When to Use an Unproven Therapy
Current outcomes are universally terrible
Little possible harm from proposed intervention
Potential for enormous benefit
Likelihood that effectiveness = efficacy
WHAT ABOUT 0 OUT OF 4?

45. Cleveland QI: “as good as NINDS”
At 9 Cleveland Clinic hospitals only
1 UH
47 patients total (~2.5%)
No chart review Methods
Protocol violations in “only” 19% (8-30%)
Nothing about outcomes!
Katzan, et al. Stroke Mar; 34:
Point out 56/83 ATLANTIS centers in STARS.

46. Threats to Efficacy: Other Studies
The STK studies
Are they relevant?
ECASS and “target population” data
Stands the scientific method on its head
Disproportionate “adjustment” in tPA group with bad outcomes
Not a criterion in NINDS
If it were important … impossible to ID on CT!

47. Effectiveness: STARS Large, with good results
NOT an efficacy study: all = ATLANTIS sites
58 of 83 sites - what about the other 25?
Point out 56/83 ATLANTIS centers in STARS.

48. Effectiveness: CASES
®-sponsored registry, with “investigators” paid for cases voluntarily submitted
“… estimate = 84%” of patients who got tPA
No 90d f/u on 13%, so outcome “imputed” from time of hospital discharge
No independent measure of actual outcomes
Point out 56/83 ATLANTIS centers in STARS.

49. Effectiveness: CASES Abstract: “excellent clinical outcome in 37%”
Methods: “excellent” defined as mRS = 0-1
Results: “… not significantly lower than NINDS (36.8% vs 39.9%), & similar to other series”
Interpretation: “36.8%”
Point out 56/83 ATLANTIS centers in STARS.

50. CASES: “Adjusted mRS” “…difference between baseline and 90d mRS …
for example, a patient with baseline and 90d mRS = 3 would be rated ‘excellent.’”
Abstract: Not mentioned
Methods: Not defined or mentioned
Results: Not mentioned (except in Figure)
Interpretation: Not mentioned
NINDS (tPA or placebo comparator): No such measure
Point out 56/83 ATLANTIS centers in STARS.

51. CASES: mRS 0-1 The actual “primary outcome” Achieved in 31.8%
not mentioned in abstract, results, or interpretation
most consistent with placebo in NINDS
< achieved in 40K placebo patients in CAST / IST
Add in selection bias, measurement bias, loss to f/u …<30%? … <25%? … <20%? … ???
Point out 56/83 ATLANTIS centers in STARS.

52. Effectiveness: CASES Abstract: “excellent clinical outcome in 37%”
Methods: “excellent” defined as mRS = 0-1
Results: “… not significantly lower than NINDS (36.8% vs 39.9%), & similar to other series”
Interpretation: “36.8%”
Point out 56/83 ATLANTIS centers in STARS.

54. Clot Removal - Multi-MERCI
164 (non-consecutive) pts
Cherry-picked
Excluded “couldn’t reach the clot” from Results
NIHSS = 19 (IQR 15-23)
≤8h p-sx onset
“successful recanalization” in 57% … 69% p-adjunctive IA tPA
Smith, Stroke 4/08

55. Multi-MERCI symptomatic ICH in “10%” [16/13=12%]
“clinically significant procedural complication” in “5.5%” [9/131=7%]
90d mRS = 0-2 in 36% [50/39% in NINDS]
90d mortality = 34%
Smith, Stroke 4/08

56. Recent “Clot Removal” studies*
3 RCTs vs various control groups
Just over 1K subjects
All in NEJM 3/13
None showed any benefit (with trends toward worse outcomes)

57. Clot Removal – Kidwell NEJM 3/13
N = 118, <8h p-sx (mean 5.5h) randomized to clot removal or “standard care” Ù
overall 90d mortality = 21%
symptomatic-ICH in 4%

58. Clot Removal – Kidwell NEJM 3/13
58% c “favorable pattern” on MRI: “small infarct core/ lots of salvageable tissue”
“successful recanalization” c embolectomy in 2/3, independent of y/n “favorable” MRI
“favorable” pattern pts did better (by some measures)
this was equally true in controls as in intervention group

59. Clot Removal – Kidwell NEJM 3/13
no difference in by tx group (50% each) or by y/n “favorable” pattern (ditto);
no difference by tx in 90d mean mRS (3.9 each), or in subgroups c or s “favorable” pattern (3.9 vs 3.4 & 4.0 vs 4.4) [so “favorable” pts did better without embolectomy!];
no difference in % c 90d mRS ≤2 (19% each), + no difference by y/n “favorable” (either between or within tx groups).

60. “Clot Removal” studies – Conclusions of Authors
“These first-generation devices aren’t good enough. Just wait until we use the newer ones!”