Podcast: Play in new window | Download

Subscribe: RSS

Date: April 30th, 2021

Guest Skeptic: Dr. Justin Morgenstern is an emergency physician and the creator of the excellent #FOAMed project called First10EM.com. He is also one of the SGEM Hot Off the Press Faculty.

Reference: Donaldson et al. Review article: Why is there still a debate regarding the safety and efficacy of intravenous thrombolysis in the management of presumed acute ischaemic stroke? A systematic review and meta-analysis. Emerg Med Australas 2016.

This SGEM Xtra is based on the new recommendation on TheNNT website for tPA in acute ischemic stroke.

This is the third time there has been a recommendation on this topic. The first review gave thrombolytics a “red color recommendation: no benefit.” The second review gave alteplase, a single agent, a “green color recommendation: benefit>harm.” Since no relevant trials were published between the two and both author groups examined essentially the same data and arrived at opposing conclusions, we wanted to understand and try to explain the conflicting interpretations.

Our interpretation of the available literature was to give it a “yellow colour recommendation: net benefits and harms unclear due to uncertainty in data”. This resulted in the summary statistic of the benefit NNT (not reported: Uncertain) and Harms in NNT (not reported: Uncertain). More details on the NNT Rating System are available.

It would be hubris to presume that our summary would arrive at the one true answer. But our goal wasn’t to provide an answer. Our goal was simply to explain the science as well as we could, so people could understand why there is a debate – and the uncertainty that underlies that debate.

The Donaldson et al SRMA included 10,431 patients in 26 randomized trials comparing intravenous thrombolysis with placebo or standard care in acute ischemic stroke [1]. Their efficacy endpoint was good functional outcome, defined as a modified Rankin Score (mRS) of 3 or less. This is defined as some residual disability requiring assistance but able to walk and care for personal needs independently. The harm endpoints were symptomatic intracranial hemorrhage (as defined by individual trials) and overall mortality

The authors report a 3.2% improvement in good neurologic outcome, a 5.4% increase in symptomatic intracranial hemorrhage, and a 2.5% increase in mortality. However, we question the certainty implied by these summary numbers.

Emberson and colleagues reported only on alteplase (a problem we will discuss further) and found a 5% improvement in neurologic outcomes, a 5.5% increase in intracranial hemorrhage, and a 1.4% increase in 90-day mortality that was not statistically significant [2].

A 2014 Cochrane review by Wardlaw et al and arrived at similar conclusions with significant improvement in neurologic outcomes, increased intracranial hemorrhage, and increased mortality [3]. Thus, our conclusions and discussion are unchanged by choice of review and reflect our belief that pooling data on this topic is overly simplistic and masks profound uncertainty.

We both really like TheNNT website, and the NNT as a concept. But there are problems with the NNT if used in isolation. One of the great conceptual difficulties of summary statistics like the number-needed-to-treat (NNT) is the implication of certainty. A major strength of the NNT is its simplicity, making complex research easier to understand. A weakness, however, is also its simplicity, because it can hide the complexity of research, ignore confidence intervals, and obscure biases. For most topics, these details are far more important than any individual number.

There is an SGEM Xtra on some of the limitations of the NNT/NNH summary statistics called the NNT – WET or DRI? It was based on an article published Dec 2019 in AEM by Reeves and Reynolds.

There are multiple sources or uncertainty around thrombolytics and stroke which we discussed in TheNNT recommendation.

Conflicting Individual Trial Results

The first source of uncertainty we highlighted was conflicting individual trial results. Among 26 trials in this systematic review by Donaldson et al, 24 research groups found no benefit in their selected primary outcome [1]. And the two that claim a benefit (NINDS part 2 and ECASS III) both had baseline imbalances that may explain the difference [4,5]. In fact, there are re-analyses that adjust for those imbalances in both trials, and the benefits disappear [6,7]. However, in some re-analyses of NINDS-2 the benefit is maintained, which adds to the uncertainty here [8,9].

We reviewed the NINDS trial with Dr. Swaminathan back on SGEM#70. More recently Prof Fatovich and I reviewed the reanalysis of ECASS-3 by Dr. Brian Alper on SGEM#297.

Clinical Heterogeneity of Individual Trials

Another source of uncertainty is the clinical heterogeneity of individual trials. The 26 trials are clinically heterogeneous, enrolling stroke patients of differing demographics, treatment times, stroke severities, anatomic territories, and thrombolytic agents. The author of the first NNT summary felt this was too much heterogeneity for appropriate pooling, a position supported by the major differences in conclusions drawn depending on which studies an author group chooses to include.

Selective Emphasis on Trials Claiming Benefit

There was also the selective emphasis on trials claiming benefit. It is circular and erroneous logic to claim efficacy for thrombolytics based on the trial characteristics of the two positive trials. First, there is legitimate debate about whether they were truly positive. Second, selectively highlighting positive results is a form of the “Texas sharpshooter fallacy”.

The Texas sharpshooter fallacy is committed when you cherry-picked a data cluster to suit your argument or found a pattern to fit a presumption. It comes from concept of a marksman shooting at the side of a barn. After firing multiple shots, they go up to the barn and draw the target around the spot where there are the most bullet holes.

For example, because both NINDS II and ECASS III used alteplase, some have suggested alteplase is a superior agent [4,5]. However, on close inspection, that logic falters: few trials have compared thrombolytic agents head to head, so there is no strong evidence to support that claim. There are nine additional trials of alteplase are negative. And systematic reviews consistently find no heterogeneity of effect between agents – in other worse, statistically speaking the different thrombolytics all look the same for efficacy[3,5,10].

Moreover, in evaluating drug efficacy, establishing a class effect is generally a prerequisite for debating or comparing individual agents [11]. Therefore, while it may increase complexity, we believe it is a mistake to exclusively examine data from the agent used in the two trials that claimed benefit. You can’t just retrospectively decide to throw out the trials you don’t agree with.

Likewise, while there are theoretical reasons to think early treatment is better, this has not been directly tested and is not strongly supported by data. Neither Donaldson et al. nor the Cochrane review find an interaction between time to treatment and effect.  IST-3, the largest placebo controlled randomized trial of thrombolytics for stroke, found better outcomes among those treated after 4.5 hours than in patients treated at 3-4.5 hours from onset of stroke symptoms [12]. Again, we feel it is best to consider this literature as a whole rather than using time windows selected based on outlying (i.e. positive) results.

Individual Trial Bias

Another source of uncertainty was individual trial bias. Bias is a major source of uncertainty in all scientific research. Importantly, using the GRADE tool [13], Donaldson et al. rate the risk of bias as “serious” for all outcomes.  One notable source is the outcome scales used, for instance the modified Rankin Scale (mRS) score. This score is known to have some subjectivity with poor inter-rater reliability and questionable validity. When trained neurologists examine the same patients there is substantial variability in mRS score assignments [14,15]. Compounding the problem, some trials assessed patients by phone or mail, a choice certain to increase variability and imprecision. For example in IST-3, which contributes nearly 40% of subjects in the Donaldson meta-analysis, results were obtained using telephone and mail follow-up, and non-blinded. This subjectivity is important, because removing IST-3 from the pooled analysis removes the statistical finding of benefit.

Stopping Early

Bias can be compounded in a SRMA when trials are stopped early. That is because larger trials are weighted more heavily in a meta-analysis. So early termination (which reduces trial size) can significantly affect results. Five thrombolytic trials were stopped early for harm or futility [16-20].  Together these would have enrolled more than 2,000 additional subjects who, had they been included, may have neutralized or even reversed findings from the two small trials claiming benefit, NINDS2 and ECASS III (combined n=1,445).

Furthermore, while over 10,000 subjects were enrolled in stroke trials, some individual trials for acute myocardial infarction enrolled far more, and in aggregate those trials included more than 60,000 [20,21].  The comparatively small number of participants in stroke trials means chance findings like baseline imbalances are both more likely and more influential, furthering uncertainty.

Harms

In contrast to the heterogeneous data on the potential benefits, the data on the potential harms are more certain. Exact numbers vary based on definitions and whether one focuses on fatal, symptomatic, or any hemorrhage, but an increase in intracranial hemorrhage is certain. More importantly, there is also an increase in mortality with thrombolytics [1,3]. Harms are also usually under-reported in studies [23,24].

Prior NNT summaries directly compared the number of patients with intracranial hemorrhage to the number with a good neurologic outcome, which can be misleading. Any harms due to intracranial hemorrhage are incorporated into a final neurologic outcome assessment, and therefore good functional outcome is far more important than intracranial hemorrhage.

There is, however, a difficult comparison to consider between the chance of neurologic benefit and the increased risk of death. Thrombolytics appear to increase mortality; [Donaldson et al and Wardlaw et al] however many patients may be willing to accept this risk for an increased chance of functional improvement. Shared decision making for this already very difficult decision is made even harder by the fact that a mortality harm is pretty  certain, the benefit is very uncertain.

Financial Conflicts of Interest

Financial conflicts of interest (fCOI) can also contribute to the uncertainty of the evidence. Unfortunately, fCOI are common in medical research. Although they do not always invalidate findings, they complicate interpretation because studies with financial conflicts are more likely to report positive findings [25-28]. Such conflicts are well documented in both the original thrombolysis research and the subsequent guidelines [29] and are known to affect the quality and conclusions of systematic reviews. This may be an unconscious contributor to the divergent conclusions seen for this topic [30].

Limitations

Like all such work, our summary has inherent limitations. Not included in this review, for instance, are newer trials using advanced imaging to select patients for thrombolysis [31-34], trials we felt should not be combined with studies using thrombolytics based largely on clinical criteria.

The Donaldson review also uses a different cut-off on the modified Rankin Scale score than others (≤3) to define a good outcome. The authors argue being able to walk and care for one’s self independently is the threshold most would consider a good outcome. While this cut-off may seem unconventional, other reviews find similar harms and benefits using different cutoffs, suggesting to us that cutoff points are not an important differentiator of conclusions [2,3].

The uncertainty in these data may help explain differing views of thrombolysis for stroke: trials are heterogenous with many sources of bias; rare positive findings are marred by baseline imbalances but continue to have outsize influence; outcomes were assessed using imprecise, subjective scales; early terminations tilt the data; outcomes overlap; and conflicts abound. There is no simple summary. We find the only appropriate conclusion is uncertainty.

Summaries of the literature cannot answer questions the literature cannot answer. Thrombolytics for stroke has been among the most debated topics in medicine for three decades, which tells you that the data is just not strong enough to be certain in either direction. The Donaldson et al. meta-analysis suggests a potential functional outcome benefit which must be balanced against increased mortality. However, the authors warn us that the findings are at serious risk of bias, and pooling results introduces more bias. Three NNT summaries have now arrived at three different conclusions. This question will not be answered by re-analyzing the same data.

In the meantime, it is difficult to offer clear guidance. Clinicians should be aware of the weaknesses in these data, and patients deserve to know as well. We believe the data are not compelling enough for specific recommendations. Shared decision making is essential, and physicians are left with the difficult task of guiding patients through a decision without a clear answer.

“Non-Expert EM Contrarians”

We need additional, carefully executed trials to provided us more information to guide our care. Unfortunately, there are some who feel that the “era of RCTs of IV tPA as a standalone therapy in acute ischemic stroke has now likely come to a close”. This statement is from a presentation at the recent International Stroke Conference. The presenter went on to say it would be “unethical to randomize to placebo therapy”.

Hopefully our review helps people understand why we feel that conclusion is incorrect. The vast majority of trials have been negative, and there are serious questions about the two positive trials. The data on harms is clearer: thrombolytics can increase mortality. With a drug that has this potential harm, you must be very certain that there is enough benefit to justify that risk. Replication is the core of science – and neither positive trial has ever been replicated. If anything, the bulk of the evidence seem to point towards harm. There is clearly a need for follow up RCTs. In fact, I would argue that it could be considered unethical to continue using this drug without getting follow up research that can confirm whether there is any benefit to outweigh the known harms.

This individual also called Prof Fatovich and myself “non-expert EM contrarians”. I guess it all depends on what definition of “expert” they are using. Neither of us are stroke neurologists but we are both EM physician. Dr. Fatovich has a PhD and is the Head of the Centre for Clinical Research in Emergency Medicine (CCREM) within Harry Perkins Institute of Medical Research while I have an MSc in physiology and been doing medical research for 37 years.

I did reach out to Dr. Jordon Bonomo an EM physician listed on the presentation. He was not aware of the “non-expert” comment, acknowledged it was unprofessional and apologized. I also spoke to Dr. Saver the senior author to inquire about the comment. He said it was his comment not Dr. Bonomo’s or his fellow. It was to indicate that we were not stroke neurologists but acknowledged we are experts at methodology. We had a very good conversation affirming our common ground on wanting stroke patients to get the best care.  We are thinking about making t-shirts with our new title and both Drs. Saver and Bonomo have requested a shirt.

The reference used in the background slide was an article Dr. Fatovich and I wrote for ACEP Now. It was on the reanalysis of ECASS-3 by Alper et al [6]. This resulted in a letter being sent from two Genentech physicians pointing out some concerns which was published with our response in ACEP Now.

Other Thoughts on Lytics for Stroke

  • Carpenter CR, Keim SM, Milne WK, Meurer WJ, Barsan WG; Best Evidence in Emergency Medicine Investigator Group. Thrombolytic therapy for acute ischemic stroke beyond three hours. J Emerg Med. 2011 
  • Milne WK, Lang E, Ting DK, Atkinson P. CJEM Debate Series: #TPA should be the initial treatment in eligible patients presenting with an acute ischemic stroke. CJEM. 2020 
  • Milne WK and Fatovich D. Will it take 50-100 years to get the right answer about tPA for stroke? Lown Institute July 2020
  • SGEM Xtra: No Retreat, No Surrender – Thrombolysis for Acute Ischemic Stroke.
  • SGEM Xtra: Walk of Life – Thrombolysis for Acute Ischemic Stroke
  • SGEM#29: Stroke Me, Stroke Me
  • Thrombolysis for Acute Stroke – Debate in Sweden March 2014
  • SGEM#290: Neurologist Led Stroke Teams – Working 9 to 5
  • SGEM#325: Thin Ice – Subgroup Analysis of the THAWS Trial

Dr. Justin Morgenstern

We think a lot of people out there find this topic very frustrating. It has been more than 25 years, and we are still repeating the same debate over and over again. There are two sides of this – the research side and the clinical side. This topic won’t be settled until we get follow up RCTs. We agree with Donaldson’s conclusions that further RCTs are urgently needed.

Clinically speaking, we think some people get so frustrated that they just tune out. But this discussion about how to clinically apply the evidence is really important. The reason we show up to work every day is our patients. Unfortunately, at this point in time there isn’t a clear answer. However, our patients deserve to know about the evidence for thrombolytics for acute ischemic strokes.

Earn CME Credits for this Episode

The SGEM will be back next episode doing a structured critical appraisal of a recent publication. Trying to cut the knowledge translation window down from over ten years to less than one year using the power of social media. So, patients get the best care, based on the best evidence. 

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

References:

  1. Donaldson et al. Review article: Why is there still a debate regarding the safety and efficacy of intravenous thrombolysis in the management of presumed acute ischaemic stroke? A systematic review and meta-analysis. Emerg Med Australas 2016.
  2. Emberson J, Lees KR, Lyden P, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet 2014
  3. Wardlaw JM, Murray V, Berge E, del Zoppo GJ. Thrombolysis for acute ischaemic stroke. Cochrane Database Syst Rev 2014
  4. National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995
  5. Hacke W, Kaste M, Bluhmki E, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med 2008
  6. Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3–4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ EBM. 2020
  7. Hoffman JR, Schriger DL. A graphic reanalysis of the NINDS Trial. Ann Emerg Med 2009
  8. Ingall TJ, O’Fallon WM, Asplund K, et al. Findings from the reanalysis of the NINDS tissue plasminogen activator for acute ischemic stroke treatment trial. Stroke 2004
  9. Kwiatkowski T, Libman R, Tilley BC, et al. The impact of imbalances in baseline stroke severity on outcome in the National Institute of Neurological Disorders and Stroke Recombinant Tissue Plasminogen Activator Stroke Study. Ann Emerg Med 2005
  10. Mann J. Truths about the NINDS study: setting the record straight. West J Med 2002
  11. McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996
  12. Sandercock P, Wardlaw JM, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial [published correction appears in Lancet. 2012
  13. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008
  14. Quinn TJ, Dawson J, Walters MR, Lees KR. Reliability of the modified Rankin Scale: a systematic review. Stroke 2009
  15. Zhao H, Collier JM, Quah DM, Purvis T, Bernhardt J. The modified Rankin Scale in acute stroke has good inter-rater-reliability but questionable validity. Cerebrovasc Dis 2010
  16. Albers GW, Clark WM, Madden KP, Hamilton SA. ATLANTIS trial: results for patients treated within 3 hours of stroke onset. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. Stroke 2002
  17. Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke 2000
  18. Donnan GA, Davis SM, Chambers BR, et al. Streptokinase for acute ischemic stroke with relationship to time of administration: Australian Streptokinase (ASK) Trial Study Group. JAMA 1996
  19. Multicenter Acute Stroke Trial-Europe Study Group, Hommel M, Cornu C, Boutitie F, Boissel JP. Thrombolytic therapy with streptokinase in acute ischemic stroke. N Engl J Med 1996
  20. Randomised controlled trial of streptokinase, aspirin, and combination of both in treatment of acute ischaemic stroke. Multicentre Acute Stroke Trial–Italy (MAST-I) Group. Lancet 1995
  21. Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico (GISSI). Lancet 1986
  22. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Lancet 1988
  23. Hodkinson A, Kirkham JJ, Tudur-Smith C, et al Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension BMJ Open 2013
  24. Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A et al. Quality of reporting in systematic reviews of adverse events: systematic review BMJ 2014
  25. Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003
  26. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003
  27. Perlis RH, Perlis CS, Wu Y, Hwang C, Joseph M, Nierenberg AA. Industry Sponsorship and Financial Conflict of Interest in the Reporting of Clinical Trials in Psychiatry. AJP 2005
  28. Bhandari M, Busse JW, Jackowski D, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 2004
  29. Lenzer J. Alteplase for stroke: money and optimistic claims buttress the “brain attack” campaign. BMJ 2002
  30. Hansen C, Lundh A, Rasmussen K, Hróbjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev 2019
  31. Davis SM, Donnan GA, Parsons MW, et al. Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial. Lancet Neurol 2008
  32. Ringleb P, Bendszus M, Bluhmki E, et al. Extending the time window for intravenous thrombolysis in acute ischemic stroke using magnetic resonance imaging-based patient selection. Int J Stroke 2019
  33. Ma H, Campbell BCV, Parsons MW, et al. Thrombolysis Guided by Perfusion Imaging up to 9 Hours after Onset of Stroke. N Engl J Med 2019
  34. Thomalla G, Simonsen CZ, Boutitie F, et al. MRI-Guided Thrombolysis for Stroke with Unknown Time of Onset. N Engl J Med 2018