SGEM Xtra: NNT – WET or DRI - The Skeptics Guide to Emergency Medicine

Subscribe: RSS

Date: December 17th, 2019

Reference: Reeves and Reynolds. The NNT-WET and NNT-DRI: (Mostly) Satirical New Metrics to Emphasize the Inherent Inefficiency of Clinical Practice. AEM Dec 2019.

Guest Skeptics: Dr. Mathew Reeves is a Professor and Interim Chair of the Department of Epidemiology and Biostatistics at the College of Human Medicine at MSU.

Dr. Joshua Reynolds is an Associate Professor of Emergency Medicine at the College of Human Medicine at MSU. Outside of his academic duties, he works clinically in the adult ED at Spectrum Health in Grand Rapids, Michigan, the tertiary care center for Western Michigan.

This is an SGEM Xtra and is a result of the December AEM publication suggesting new metrics to emphasize the inherent inefficiency of clinical practice. This (mostly) satirical article seems to be in the same theme of the annual BMJ holiday edition.

The BMJ has published some great studies in their holiday edition. We have covered two of them on the SGEM:

SGEM#6: Orthopedic Surgeons: Strong AND Smart!
SGEM#23: A Bump Up Ahead (Diagnosis of Appendicitis)

One of my other favourite BMJ holiday edition articles has been the classic parachute trial (Smith and Pell BMJ Dec 2003). Parachutes have been used for years to prevent orthopaedic, head and soft tissue injuries after a gravitational challenge (jumping out of planes). There was observational data that showed parachute use led to injury and there were case reports of people surviving falling/jumping out of a plane without a parachute or it opening properly. They could find no randomized controlled trials (RCT) to include in their systematic review and meta-analysis (SRMA).

The authors suggested taking evidence-based medicine (EBM) advocates up in a plane and have them randomized in a double-blinded fashion to a parachute or a sham (backpack). It would be a cross over trial. Those participants who survived the first jump would be randomized into the opposite group. Only then would there be definitive evidence for the efficacy of parachutes.

Since that SRMA published in 2003, there has been a randomized control trial conducted and published on the topic of parachutes. It was published last year in the 2018 BMJ holiday edition (Yeh et al BMJ Dec 2018). It will be covered as an SGEM Xtra in 2020.

NNT: Number Needed to Treat

Dr. Joshua Reynolds

The NNT stands for the Number Needed to Treat. It estimates the average number of patients who need to be treated to positively impact one person with therapeutic benefit. It was originally described in 1988 by Andreas Laupacis, an internist and clinical epidemiologist who was at McMaster University in Ontario at the time. (Laupacis A et al NEJM 1988).

How is the NNT Described Mathematically?

The “number needed to do anything” is the inverse of the absolute change in risk. So in this case, the number needed to treat is the inverse of the absolute risk reduction (ARR): NNT=1/ARR

An Example of Calculating the NNT

Let’s say that there is a new drug to treat a bad disease and it reduces mortality from 25% to 15%. The absolute risk reduction is 10%, so NNT is the inverse of 0.1 and the NNT would be 10. Likewise, if that same drug reduces mortality from 25% to 20%, then the absolute reduction is 5% and the NNT would be 20, or 1 divided by 0.05.

What is An Advantage To Using the NNT?

One advantage is that using a single number, NNT describes the absolute impact or effectiveness of a particular therapy. Interventions with lower NNT are considered more efficacious, since one must theoretically treat fewer patients to observe an effect.

Is the NNT Popular?

Yes, there is an entire Internet domain devoted to NNT (www.thennt.com) . This site extols the virtues of NNT to promote the most effective therapies while questioning those with insufficient benefit.

NNH: Number Needed to Harm

Dr.Mathew Reeves

When quantifying the harms associated with treatment, the corollary to NNT is “number needed to harm” (NNH), which is calculated as the inverse of absolute risk increase: NNH = 1/Absolute Risk Increase

The NNH estimates the average number of patients who need to be treated before one person is negatively impacted by a harmful side effect caused by the therapy.

Interventions with higher NNH are theoretically less risky, since more patients can be treated before an adverse treatment-related event occurs. When combined with NNT, these two numbers convey to patients, in a simple manner, the trade-offs between risks and benefits of treatment. Presumably, this is a simpler method to convey risks and benefits to patients than trying to describe relative or absolute changes in risk, or trying to describe odds ratios.

Are There Limitations to Using the NNT and NNH?

Yes, there are a number of limitations to using NNT and NNH estimates. One key issues is that you must know what time period these estimates are based upon. Every NNT and NNH has an explicit time period associated with the metric. Also the NNT and NNH do not capture clinical relevance or cost. The NNT may be very low for the primary outcome of a study, but if it is a disease-oriented outcome (DOO) with no patient-oriented outcome (POO) the NNT can be misleading. Another limit is cost. How much money does it cost for the intervention? If it costs pennies and has a very low NNT that would be great but would not be as good as a treatment with the same NNT that costs millions of dollars.

For more on this issue check out PEM Super Hero, Dr. Anthony Crocco (SGEM faculty member) who has a great white board video explaining the concept and application of the NNT and NNH (SketchyEBM).

NNT-WET: Number Needed to Waste Everybody’s Time

Lost in the populist enthusiasm for NNT is its inherent mathematical complement, which is a more realistic and clinically useful number for the practicing physician. Thus, we propose the “number needed to waste everybody’s time” (NNT-WET). The NNT-WET = NNT-1.

The NNT-WET estimates the average number of patients who need to be treated, but receive no therapeutic benefit, for someone else to benefit. NNT-WET is a direct measure of the inefficiency of clinical practice; it conveys the ineffectiveness of clinical interventions by measuring the effort required to help just one solitary patient.

In the postmodern era of limited medical resources and therapeutic nihilism, NNT-WET is the metric that provides the appropriate level of cynicism required by today’s practicing clinician.

Are There Any Advantages to Using the NNT-WET Over the NNT?

Yes, there are several. First, NNT fails to sufficiently emphasize that most patients do not benefit from treatments routinely used in clinical practice. Since the vast majority of NNT estimates exceed two, a given individual patient is unlikely to benefit from treatment (Figure 1A).

How Does the NNT-WET Change the Conversation?

The NNT-WET shifts the clinical conversation from the assumption that we must treat the patient (e.g., “This treatment is great—its NNT is only ten!”), to a state best described as therapeutic malaise (e.g., “The NNT-WET is nine . . . what’s the point?”). The NNT- WET helps illustrate that for most treatments, the costs, inconvenience, and risks are disproportionately applied to the many, so that only a single person (whom, most importantly, is not you!) can benefit.

But really this approach should be tested empirically. For example, one could present clinical scenarios to patients and/or clinicians detailing effect measures of proposed treatments with NNT or NNT-WET (not to mention absolute or relative risk reduction!). Our hypothesis is that scenarios based on NNT-WET in lieu of NNT would result in patient and clinicians selecting marginally effective treatments less frequently. We are awaiting review of our grant proposal from the nihilism study section of the NIH.

NNT-DRI: Number Needed To Divert Reckless Intervention

After laborious review of a thesaurus to make the acronyms work, we arrived at the “number needed to divert reckless intervention”. Using the same rationale as NNT-WET for NNT, we propose the NNT-DRI as a revised measure for NNH. The NNT-DRI = NNH -1.

The NNT-DRI estimates the average number of patients who need to be treated, and who escape the therapy’s adverse effects, in order for someone else to sustain the adverse event. It is a measure of the recklessness of clinical intervention; a small NNT-DRI indicates that only a few patients escape harm, whereas a large NNT-DRI is reassuring since regardless of whether any patient benefits, many patients are not harmed. A large NNT-DRI is a state of Hippocratic bliss; “primum non nocere”.

Are There Advantages to the NNT-DRI Over the NNH?

Yes, the NNH, which insufficiently acknowledges the patients that regularly escape therapeutic maleficence. Clinicians should rejoice in large NNT-DRI estimates that represent the multitudes of patients they have not harmed (Figure 1B).

The NNT-DRI helps illustrate that adverse effects of treatments are disproportionately applied to an unfortunate few, while the rest mange to escape them. NNT-DRI shifts the clinical conversation from a serious discussion of risk (e.g., “This treatment is dangerous, the NNH is only five.”), to a state of reassurance best described as willful ignorance (e.g. “Maybe so, but four of them will do just fine!”).

Can You Give A Practical Example of the NNT-WET and NNT-DRI?

Take thrombolysis for the treatment of acute ischemic stroke. Using risk estimates from the 2014 meta-analysis of individual patient data by Emberson et al in Lancet, we estimated the NNT to achieve excellent functional recovery 3 to 6 months after treatment ranged from 10 (0–3 hours after symptom onset) to 50 (4.5–6 hours after symptom onset).

These estimates translate to NNT-WET values ranging from 9 to 49, respectively. Thus, to impart therapeutic benefit, clinicians must labor to rapidly identify, evaluate, and treat between 9 and 49 patients who are exposed to the cost and risks of treatment without any of the corresponding benefits. Likewise, we estimated the NNH for 7-day fatal intracranial hemorrhage after treatment is a mere 40 regardless of the interval after symptom onset. Yet with a corresponding NNT-DRI value of 39, clinicians can find solace in knowing that 97% (39/40) of patients they treat with systemic thrombolytics will escape this particular peril.

Are There Any Limitations To the NNT-WET and DRI?

There are several limitations to consider in our work and NNT/NNH in general. Primarily, all “number needed to . . .” values are time-dependent, and the choice of the particular time interval is often arbitrary; this problem is exacerbated by incorrectly assuming that the risks and benefits of treatment are constant over time.

Other important limitations include the variable duration of clinical trials (primary source of NNT/NNH data), the danger of extrapolating across differing baseline risks (resulting in widely different NNT/NNH estimates), and the inability to predict which individual patient will be the one to sustain benefit or harm.

Finally, NNT and NNH foster the misconception that treatment decisions should only be quantified as a binomial probability for a given patient to receive benefit or harm. This “lottery” interpretation of treatment effects is only realistic in certain specific clinical scenarios determined by stochastic processes (e.g., true “accidents”). The lottery interpretation of NNT is not suitable for preventive interventions that delay adverse events rather than eliminate them.

SGEM Bottom Line: The NNT-WET and NNT-DRI represent a novel and (mostly) satirical tool for clinicians and patients to understand treatment options.

Should We Be Using the NNT-WET and DRI Clinically?

The NNT-WET and DRI represent true population-based measures that are uniquely patient-centered in that they apply to the vast majority of patents who are neither benefited nor are harmed. In other words, we are speaking about the proverbial “99%” instead of the “1%”. NNT-WET and NNT- DRI serve as reminders that clinical medicine excels in inefficiency; all of you clinician listeners out there can rest assured that most of your well-intentioned treatments have no effect whatsoever —for good or for ill.

Happy holidays to all the SGEMers. I hope you get some time off over the next couple of weeks to spend time with friends and families and wish you all the best in 2020.