SGEM Xtra: Presenting in a Northern Constellation

Subscribe: RSS

Date: June 27th, 2020

I had the pleasure of presenting at the Northern Constellation Faculty Development Conference 2020 on May 8th. This was the 9th annual conference put on by the Northern School of Medicine (NOSM).

Dr. Sarah McIsaac was kind enough to invite me to present at the Northern Constellation Conference. She is an anesthesiologist/intensivist at Health Sciences North, Assistant Professor and Medical Director of Faculty Development for NOSM.

This presentation is available to watch on the SGEM YouTube Channel or listened to on the SGEM podcast. All of the slides can also be downloaded from this link.

I was asked to give a presentation about evidence-based medicine (EBM), critical appraisal and relate it back to COVID. Certainly there has been a lot of information coming out on the topic and it can seem like you are drinking from a fire hose at times.

The presentation was broken down into three parts: Evidence-Based Medicine (EBM), critical appraisal and the Peltzman Effect (risk compensation):

Part I: Evidence-Based Medicine (EBM)

It is always good to define terms at the beginning of any discussion. I used the original definition of EBM given by Dr. David Sackett: “The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” (Sackett et al BMJ 1996)

There are three pillars to EBM than can be represented in a Venn diagram. People often make the mistake of thinking that EBM is just about the scientific literature. This is not true. The evidence informs and guides our care but it does not dictate our care. EBM also needs your clinical judgement based on your experience. We also need to engage with patients and ask them about their preferences and values. These three components make up EBM: The literature, our judgement and the patients values.

There is a hierarchy to the evidence and we want to use the best evidence so patients get the best care. The hierarch is usually described as a pyramid with the lowest form of evidence being expert opinion and the highest level being a systematic review. This is an over simplification of the levels of evidence. A good randomized control trial (RCT) can be more informative than a systematic review (SR) that only includes low quality study (GIGO – garbage in, garbage out).

There are arguments against EBM and it does have limitations. One that is often pointed out is that it would be unethical to do an RCT on harm. The 2003 Smith and Pell parachute trial is usually pointed to as an example (BMJ 2003). This could be considered a straw man argument because most medical practices are not parachutes (Hayes et al CMAJ 2018). In addition, a randomized control trial has been done assessing the efficacy of parachutes to prevent gravitationally related morbidity and mortality and was reviewed on SGEM#284.

Five alternatives to EBM were discussed (Adapted from Isaacs and Fitzgerald BMJ 1999) :

Eminence-Based Medicine (EmBM): The more senior the colleague, the less importance he or she placed on the need for anything as mundane as evidence. Experience, it seems, is worth any amount of evidence. These colleagues have a touching faith in clinical experience, which has been defined as ‘‘making the same mistakes with increasing confidence over an impressive number of years.” The eminent physician’s white hair and balding pate are called the “halo” effect.
Vehemence-Based Medicine (VBM): The substitution of volume for evidence is an effective technique for brow beating your more timorous colleagues and for convincing relatives of your ability.
Eloquence-Based Medicine (ElBM): The year round suntan, silk tie, Armani suit, and tongue should all be equally smooth. Sartorial elegance and verbal eloquence are powerful substitutes for evidence.
Nervousness-Based Medicine (NBM): Fear of litigation is a powerful stimulus to over investigation and over treatment. This may be a greater factor in the US while in Canada it might be more of being shamed or reported to the College. In an atmosphere of litigation/shame phobia, the only bad test is the test you didn’t think of ordering.
Confidence-Based Medicine (CBM): This is restricted to surgeons.

EBM is like democracy, it is the worst form of medicine except all the others that have been tried.

Part II: Five Steps of Critical Appraisal

Step 1: Use a PICO (population, intervention, comparison/control and outcome) to formulate the clinical question you are trying to answer
Step 2: Search for the best evidence (try the TRIP Database)
Step 3: Find the least bias evidence (bias is something that systematically moves us away from the “truth” not just noise in the data)
Step 4: Critically appraise the literature using a quality check list (SGEM Make it So)
Step 5: Decide if it is practice changing (consider the number needed to treat for benefit [NNTB] and the number needed to treat for harm [NNTH])

Then once you have done all that you can discuss it with the patient in front of you to ask about their preferences and what they value. There is an excellent video summarizing this process made by Dr. James McCormick and called Viva La Evidence.

Part III: The Peltzman Effect (Risk Compensation)

Sam Peltzman was a professor at the University of Chicago in the 1970’s. He wrote this classic paper about seatbelt regulations for cars. The lab data (tech studies) implied annual highway deaths would be 20% greater without mandating seatbelt in all cars. He argued that these benefits would be offset by more pedestrian deaths and more nonfatal accidents because of “driving intensity”. This was driving faster and more recklessly with the security of the safety belt.

Ultimately, Dr. Peltzman hypothesis about seat bests was shown to be wrong and they do have a net benefit. However, the idea of risk compensation where people adjust their behaviour in response to the perceived level of risk has been named The Peltzman Effect.

There are example of where an intervention did have a positive outcome (seat belts and bike helmets) but there are other examples where it did not (parachutes and electronic health records). A good article on risk compensation in medicine is by Prasad and Jena 2014.

COVID19 Management Strategies

The presentation ended with a discussion around COVID19. Clinicians and researchers have been throwing the kitchen sink at the disease. Medication include: Azithromycin, Steroids, Famotidine, IL-6 inhibitors, Chloroquine/Hydroxychloroquine, Remdesivir, Vitamin C, Zinc, etc.

Non-pharmacologic strategies to address this terrible global pandemic were also mentioned. this included the recommendation of universal mandatory mask wearing in public and the orders to shelter at home. Some of the unintended consequences of these strategies were raised.

The precautionary principle was also briefly mentioned as part of this section.A shout out to Dr. T. Greenhalgh who has been promoting this approach in the medical literature (Greenhalgh et al BMJ 2020). A rapid response to this article and the precautionary principle was published (Martin et al BMJ 2020). There has even been a rebuttal to the response (Greenhalgh J Clin Eval Prac 2020).

Will these social measures have a net positive impact on COVID19? I don’t know. The strength of any of our recommendations should at least be in part proportional to the evidence. Any intervention can have potential benefits and potential harms. It is a cognitive bias that we over estimate benefit and underestimate harm. Anyone who says they know what the net impact of these social policy is expressing some hubris.

My position is “I don’t know” and it was Dr. Richard Feynman who famously said “it’s ok to say…I don’t know”.

The audience was then encouraged not to forget our past mistakes from a previous global pandemic. Jeanned Lenzer and Shannon Brownlee wrote a wonderful article about how pandemic science is out of control. Here is just the first paragraph from their article:

“On September 14, 1918, in the midst of the worst pandemic in modern history, an article in the New York Times quoted Dr. Rupert Blue, then surgeon general of the US Public Health Service. Blue reported that doctors in many countries were treating their influenza patients with digitalis and the antimalaria drug quinine. There was no evidence that the two drugs were any more effective than folk remedies being used by patients, including cinnamon, goose grease poultices, and salt stuffed up the nose, but doctors were desperate and willing to try just about anything. They would eventually abandon quinine and digitalis as treatments for flu when studies showed they were not only ineffective but caused serious and sometimes deadly side effects.”

The New England Journal of Medicine also published an excellent article by Zagury-Orly and Schwartzstein called “COVID19 – A Reminder to Reason“. The last paragraph was very poignant.

“We are living through an unprecedented biopsychosocial crisis; physicians must be the voice of reason and lead by example. We must reason critically and reflect on the biases that may influence our thinking processes, critically appraise evidence in deciding how to treat patients, and use anecdotal observations only to generate hypotheses for trials that can be conducted with clinical equipoise. We must act swiftly but carefully, with caution and reason.”

The presentation ended with a reminder that it can feel like health care workers and essential service workers have the weight of the world on their shoulders. It is OK not to be OK. You do not need to be a super hero. Asking for help is a sign of strength and insight and not a weakness. Take good care of yourself so you can take good care of the patients.

The SGEM will be back next episode doing a structured critical appraisal of a recent publication. Trying to cut the knowledge translation window down from over ten years to less than one year using the power of social media. Ultimately we want patients to get the best care, based on the best evidence.