SGEM Xtra: No Retreat, No Surrender – Thrombolysis for Acute Ischemic Stroke

Subscribe: RSS

Date: February 4th, 2019

Guest Skeptic: Dr. Jerome Hoffman is a Professor Emeritus of Medicine and Emergency Medicine, University of California, Los Angeles.

Dr. Hoffman has been a voice of reason for decades and taught a generation of physicians critical appraisal skills. This has been through various peer reviewed publications, doing Emergency Medicine Abstracts (EMA) audio program with Dr. Rick Bukata, teaching at the Emergency Medicine and Acute Care Course among other things.

There was a recent controversy on twitter and I thought Dr. Hoffman needed to join the conversation. Listen to the SGEM podcast to hear the many things we discussed.

The website TheNNT.com has been a good resource to find unbiased critical reviews of the literature. However, Dr. Hoffman did have some concerns about how the concept could be hijacked. It appears his concerns may have come true.

TheNNT updated their page on thrombolytics for acute ischemic stroke on January 11^th, 2019. They changed the recommendation from RED (No Benefit) to GREEN (Benefits > Harms). This was very disturbing on multiple levels.

Controversy about thrombolytics and stroke is not new and has existed since at least 1995 with the publication of the NINDS trial. The SGEM has summarized the dozen major RCTs looking at the issue of thrombolytics for acute ischemic strokes (SGEM Xtra). Six trials showed no benefit. Four trials were stopped early due to harm or futility and two trials reported benefit. I do not claim thrombolysis does not work because that would shift the burden of proof. Rather, I state that I am skeptical.

We did not re-hash all the trials in our conversation but instead focused on a couple key publications like NINDS and IST-3. The SGEM did a Classic Episode on NINDS with Anand Swaminathan (SGEM#70). Dr. Hoffman has published a graphic re-analysis of NINDS (Annals EM 2009) and Sayer et al responded in print (Annals EM 2010).

The other major trial that we discussed was the IST-3 study which was covered on SGEM#29. Hoffman and Cooper provided commentary on some of the limitations of IST-3 (EMA 2012).

“using a score that is highly unreliable even when calculated by a neurologist performing an in-person neurologic evaluation. He then acknowledges that this problem was enormously exacerbated in IST-3 because the scoring was done by a layperson; to make matters even worse, this was a layperson who, like the patient himself, was unblinded to treatment group.” Hoffman and Cooper

Another issue we discussed was the outcome measure of mRS or in the case of IST-3, the Oxford Handicap Score (OHS) which is very similar to the mRS. Previous research has shown the mRS to have poor inter-rater reliability (Wilson et al Stroke 2005) and issues with validity (Zhao et al Cerebrovasc Dis 2010).

The methodology was changed in IST-3 when they did not reach their original target of 6,000 patients. A second statistician persuaded the investigators to use an ordinal method that would only require 3,000 patients. Dr. Rory Spiegel (EM Nerd) has a good post on the problems with ordinal analyses that includes a number of references.

“The trial did not meet its original target of 6000 patients, and so was no longer adequately powered to detect a 3% absolute difference in the primary outcome (with 80% power and α=0·05). The statistical-analysis-plan writing committee, which did not have access to the accumulating data, was therefore expanded to include an independent statistician (Gordon Murray, University of Edinburgh, Edinburgh, UK) to advise on the correct approach. The writing group was persuaded by the recent empirical evidence that the ordinal method was both statistically more efficient (effectively reducing the sample size required in stroke trials) and robust against substantial deviations from the proportional assumption”. IST-3

Another major issue in IST-3 was the lack of blinding. The unblinded part is very problematic when interviewing the patients 90 days later using a subjective, self-reported OHS. Being aware of group allocation would bias participants toward the intervention group.

The conclusions of IST-3 also seemed to spin a negative study into a positive study.

“despite the early hazards, thrombolysis within 6 h improved functional outcome.” IST-3

By early hazards they were referring to the increased early death (NNH 25) and increased fatal and non-fatal bleeding (NNH 17). The improved functional outcome was on a post-hoc ordinal analysis with no benefit demonstrated in the primary outcome.

The authors of the updated TheNNT website reference the 2014 SRMA by Emberson et al in the Lancet. This study included the extremely flawed IST-3. That trial made up a large portion (45%) of the 6,700 patients included in the results. The NNT and NNH look impressive but should we really have confidence in the numbers given the limitations of the primary data?

One of the authors on the report was Dr. Eddy Lang. He is the Academic and Clinical Department Head and a Professor in the Department of Emergency Medicine, at the University of Calgary. He also holds the position of Senior Researcher with Alberta Health Services

Dr. Lang tweeted “the evidence speaks for itself, especially under 3 hours.” If it does speak it must be saying different things to different people because while he is hearing yes, I am hearing no. I would suggest we need to speak for the results by critically appraising the literature.

In the blog post they say that “Skepticism about thrombolytic treatment has arisen mainly in the emergency medicine literature. However, discussion of the strengths or limitations of the original trials is beyond the scope of this evidence-based summary”. Dr. Hoffman ask how can you possibly have an evidence-based summary without discussing the strengths or limitations of the literature?

Dr. Hoffman and I go through Dr. Lang and colleagues’ narrative and caveats supporting the change in recommendation from RED to GREEN on the podcast.

They start with some statistics about the devastating impact of strokes. We all agree that stroke has a terrible morbidity and mortality burden in society. However, this could be viewed as a logical fallacy (Appeal to Emotion). Does it matter how terrible the disease is if the treatment does not have a net benefit?

They correctly state the initial thrombolytic trials using streptokinase were uniformly negative and that streptokinase was shown to be harmful. However, Emberson excluded these from their review. A Cochrane review states that there is no difference between type of thrombolytics used (Wardlaw et al 2013). Dr. Hoffman explains why excluding the other thrombolytic trials in a SRMA is problematic.

TheNNT review goes on to promote the idea that time is brain. This has been challenged by Dr. Hoffman. In fact, even the IST-3 data used by tPA advocates does not support the claim that time is brain. This can be seen in the IST-3 forrest plot.

TheNNT authors claim that the outcome tool (mRS) is validated and reproducible. However, we have already referenced two articles that demonstrate the mRS does not have good inter-rater reliability or validity.

Dr. Jerome Hoffman

They also claim that the “rising proportion of patients with a good outcome in the control group in later time windows reflects that fact that stroke severity is lower at later time points; more severe stroke presents early.” Dr. Hoffman explains how this ignores the issue of TIAs. Patients with TIAs would be more prevalent in the early presenters and less prevalent over time. The higher proportion of TIAs early on could explain why those patients who get thrombolysis early do better than those who get it late.

We cannot only discuss the potential benefits but must also consider the potential harms of thrombolysis for acute ischemic stroke. TheNNT review does say that: “treatment has been attempted in the absence of a collaborative team structure, trial results have not been duplicated. Without the necessary structure, real-world treatment has been either neutral or frankly harmful”. However, if the RCTs do not demonstrate benefit why should we reorganize the whole stroke system?

Thrombolysis for acute ischemic stroke in the community has not had the same results compared to clinical trials. It usually has less benefit and more harm when used in the community setting (Khoury and Jackson).

The clearest signal in the thrombolytic for acute ischemic stroke literature is the increase in bleeding. TheNNT review seems to downplay this harm because it may not be symptomatic or responsible for the poor neurologic outcomes observed. They give an NNH for mortality of 71 and say it is a trade-off of improved neurologic function with a small risk of early death due to fatal ICH. But if the benefit is not true, then all you have is harm.

The issue of industry funding is mentioned. There are multiple references how funding can impact publication, results, interpretation and guidelines. TheNNT review mentions in their caveats the issue of industry funding saying: “Four of the trials were exclusively funded by industry; five were funded by public granting organizations.” Jeanne Lenzer wrote a good article in the BMJ that touches on some of these funding issues. There is also a Cochrane review on the influence of industry sponsorship on research outcomes (Lundh et al 2012).

“Sponsorship of drug and device studies by the manufacturing company leads to more favorable results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard ‘Risk of bias’ assessments.” Lundh 2012

We also touched upon how intellectual conflict of interest (COI) could impact research. Guyatt et al 2010 defined intellectual COI in clinical practice guidelines as:

“academic activities that create the potential for an attachment to a specific point of view that could unduly affect an individual’s judgment about a specific recommendation”. Guyatt 2010

These COIs have been shown to impact the interpretation of SRMA (Panagiotou and Ioannidis) and guidelines (Akl et al). The Institute of Medicine (IOM) has standards for SRMA which discusses how to manage bias and COI of the team conducting reviews. The IOM recommends excluding individuals with clear financial COI and “excluding individuals whose professional or intellectual bias diminish the credibility of the review in the eyes of the intended user” (IOM 2011).

The final thing in TheNNT post about changing the recommendation from RED to GREEN could also be considered logical fallacies (band wagon and appeal to authority). “Treatment of ischemic stroke with intravenous alteplase is strongly recommended (Level 1 or Grade A) by all major stroke guidelines.” Jeanne Lenzer has written another good article about why we can’t trust guidelines (BMJ 2013).

I am not sure if the claim that all major stroke guidelines strongly recommend IV tPA (Level 1 or Grade A) is true. In addition, who decides if a guideline is major? Certainly Australasian College for Emergency Medicine’s (ACEM) statement on IV thrombolysis for ischemic stroke does not strongly recommend alteplase. The largest organization of emergency physicians, the American College of Emergency Medicine (ACEP), only gives Level B and C recommendations not a Level A recommendation for tPA.

Level B Recommendations: Recommendations for patient care that may identify a particular strategy or range of strategies that reflect moderate clinical certainty (ie, based on evidence from 1 or more Class of Evidence II studies or strong consensus of Class of Evidence III studies).
Level C Recommendations: Recommendations for patient care that are based on evidence from Class of Evidence III studies or, in the absence of any adequate published literature, based on expert consensus. In instances where consensus recommendations are made,“consensus” is placed in parentheses at the end of the recommendation.

It is possible for different people to look at the same literature and come to different concussions. An example would be the SRMA by Donaldson, Fitzgerald, Fowler and Delaney (EMA 2016). Their key findings were:

“There is clear evidence of harm (increased incidence of symptomatic intracranial haemorrhage and early mortality) related to the use of thrombolysis for patients with presumed acute ischaemic stroke, but also some evidence that thrombolysis may be associated with improved functional outcomes.” Donaldson et al 2016

“The debate regarding the utility of thrombolysis for the treatment of presumed acute ischaemic stroke cannot be resolved given the current evidence and further high-quality randomised trials are required.” Donaldson et al 2016

We discussed some of the arguments thrombolytic advocates give like: that ship has sailed, get with the guidelines, it would be unethical to do a replication study and it would cost too much money. There is a crisis in science when it comes to replicating studies (Nature 2016). One very important reason to perform replication studies is because, according to Dr. John Ioannidis, most published research findings are false (PLoS 2005).

If it was unethical to do another study, why was it ethical to keep trying until they got the NINDS trial results? If it was unethical to do another study, why did researcher continue to do more studies after NINDS looking at longer time frames when data already existed demonstrating harm and no benefit? Dr. Hoffman give a great analogy involving flipping coins.

Dr. Daniel Fatovich published an editorial about IST-3 and the ethics of doing another trial in under three hours (EMA 2012).

“A larger trial that essentially reproduces the NINDS study must be performed. This would retest the single hypothesis regarding the possible benefit of tPA that is not clearly inconsistent with the available evidence – that treatment begun within 3 h of the onset of symptoms of acute ischaemic stroke might be beneficial. Although some might argue that it would be unethical to retest tPA within 3 h, most of the data indicate that it should not be given beyond 3 h, and yet this was retested in ECASS III. You cannot have it both ways!” Fatovich 2012

The burden of proof is on those making the positive claim. In this case, thrombolytics demonstrating a net benefit in those patients suffering acute ischemic stroke has not been met.

Dr. Hoffman ends with advising us not to retreat or surrender. We may have lost a few battles along the way but our goal remains the same, for patients to get the best care based on the best available evidence.

Additional FOAMed Resources: