Date: August 16th, 2018

Reference: Hohl C et al. Prospective Validation of Clinical Criteria to Identify Emergency Department Patients at High Risk for Adverse Drug Events. AEM Aug 2018.

Guest Skeptic: Dr. Chris Bond is an emergency physician and clinical lecturer at the University of Calgary. He is currently the host of CAEP Casts, which highlights educational innovations from emergency medicine residency programs across Canada. Chris also has his own #FOAMed blog called Standing on the Corner Minding My Own Business (SOCMOB).

Case: A 54-year-old female presents to the emergency department (ED) with abdominal pain and profuse non-bloody diarrhea for the past 24 hours. Her vital signs are within normal limits and she is tolerating oral fluids. There have been no recent suspicious meals or water sources, camping, spelunkingsick contacts and she does not have any birds or reptiles at home. She takes no regular medications; however, she did start amoxicillin/clavulanate (Clavulin or Augmentin) for a sinus infection three days ago.

Background: Preventable adverse drug events (ADEs) are a common cause of ED visits, hospitalizations and death (Ref 1,2). ADEs cause or contribute to one in nine ED visits, and of deaths attributed to medical care, medications are the most common cause (Ref 3-6).

Unfortunately, 20-50% of ADEs are not recognized by emergency and inpatient unit physicians (Refs 7-10). When ADEs are not recognized and corrected, inappropriate medication use continues, resulting in excess morbidity and health care resource utilization.

Clinical pharmacists are trained to focus on medication management and are more likely to recognize medication-related ED presentations than physicians (Ref 11). However, clinical pharmacists are a scarce resource and not available in many centers.

In centers where they are available, routine medication review in all patients presenting to the ED is not feasible. Evidence based criteria to enhance the identification and treatment of ADEs are needed to ensure high-risk patients are evaluated by clinical pharmacists and to improve their outcomes.

Clinical Question: Can a clinical decision rule (tool) accurately identify patients presenting to the ED with adverse drug events?

Reference: Hohl C et al. Prospective Validation of Clinical Criteria to Identify Emergency Department Patients at High Risk for Adverse Drug Events. AEM Aug 2018.

  • Population: Patients age 19 years and older presenting to the ED who reported using at least one prescription or over the counter medication in the two weeks prior.
    • Exclusions: Non-English speaking and no translator with patient, violent behavior, intentional overdose, needlestick injury or sexual assault, previously enrolled, presented for a scheduled revisit, direct transfer to an admitting service, fast track patient, left against medical advice prior to seeing physician and pharmacist.
  • Intervention: Application of two different clinical decision rules (tools) to identify patients at risk for adverse drug events
  • Comparison: None
  • Outcome: A moderate or severe ADEs. This was defined as an “untoward and unintended event arising from the appropriate or inappropriate use of a prescription or over-the counter medication.”
    • ADEs Included: “adverse drug reactions, a response to a prescription or over-the-counter drug that is noxious and unintended, and occurs at doses normally used in man for the prophylaxis, diagnosis or therapy of disease, and events due to non-adherence or drug withdrawal, errors in prescribing, dispensing or medication administration, drug interactions, supra or subtherapeutic dosing, untreated indications and inappropriate drug use”.
    • Severe ADE: Death or required admission
    • Moderate ADE: Change in medical management

Dr. Corinne Hohl

We are finishing Season#6 with a SGEM Hot Off the Press episode from AEM. That means we have the lead author Dr. Corinne Hohl on the show. She is an Associate Professor in UBC’s Department of Emergency Medicine and a Scientist at the Centre for Clinical Epidemiology and Evaluation. She practices Emergency Medicine at Vancouver General Hospital. Her main research interests are in patient safety, health systems innovations and drug safety and effectiveness.

For Corinne’s work on ADEs, she was awarded a Best Paper Award from the American College of Emergency Physicians in 2011. In 2012, she was awarded with a New Investigator Award from the Canadian Institutes of Health Research. She was the recipient of a 2016 Canadian Institutes of Health Research Foundation grant to continue her work on ADE reporting, and is a member of the Canadian Drug Safety and Effectiveness Research Network.

The Two Rules (tools) used in this Study on ADEs:

Rule #1

** Medication changes included medication stops and starts, and changes to dose, frequency or route of administration.

Rule #2

Authors’ Conclusions: Our study validated clinical decision rules that can be applied by clinical pharmacists to limit the number of patients requiring medication review, while identifying the majority of patients presenting with clinically significant adverse drug events.

Quality Checklist for Clinical Decision Tools:

  1. The study population included or focused on those in the ED. Yes
  2. The patients were representative of those with the problem. Yes
  3. All importantpredictor variables and outcomes were explicitly specified. Yes
  4. This is a prospective, multicenter study including a broad spectrum of patients and clinicians (level II). Yes
  5. Clinicians interpret individual predictor variables and score the clinical decision rule reliably and accurately. Yes
  6. This is an impact analysis of a previously validated CDR (level I). No
  7. For Level I studies, impact on clinician behavior and patient-centric outcomes is reported? N/A
  8. The follow-up was sufficiently long and complete. Unsure
  9. The effect was large enough and precise enough to be clinically significant. Unsure

Key Results: There were 1,529 patients enrolled in the study with an average age of 59 years and 56% were female. The median number of prescribed medications was five. There were 184 patients (12%) diagnosed with 202 moderate or severe ADEs. There were no fatalities observed in the study attributed to ADEs.

Both rules (tools) identified more than 90% of adverse drug events

  • Primary Outcome: Identifying moderate or severe adverse drug events. Variables strongly associated with ADEs included: Age, use of opioids, antihypertensives and antibiotics, recent medication changes and number of prescription medications.

  • Some Secondary Outcomes:
    • Clinical pharmacist’s and treating physician’s outcome assessment was concordant in 90% of the cases.
    • In cases that were ultimately attributed to ADEs, emergency physicians did not attribute the presentation to a drug 35% of the time.
    • Emergency physicians were uncertain about whether an ADE had occurred in an additional 16%.

Listen to the podcast on iTunes to hear Corinne’s responses to our ten (two sets of five) nerdy questions.

  1. Selection Bias: These were not consecutive patients because it would apparently have impacted negatively on patient flow through the ED. You used some form of algorithm to select patients but that has not been validated. Can you explain the process and tell us about some of the unpublished data you have to support this method of patient selection?
  2. Night Shifts: Another issue is that you did not include night shifts because it was deemed inefficient and costly. Do you think this could have introduced some selection bias?
  3. Incentives: You used incentives to motivate the nurses to complete the forms.
    • What types of prizes did you have for the nurses?
    • Were some nurses more likely to complete the forms than others?
    • Did the nurses know the purpose of the study or were they blinded?
    • Once the incentives go away do you have any way to ensure that these forms continue to be filled out?
  4. Delirium: One of the presenting complaints likely to be most associated with ADEs is delirium in the elderly. Why not make this and other potential high yield complaints a focus for the pharmacists?
  5. Non-English-Speaking Patients: Non-English-speaking patients without a translator with them were excluded from the study due to ED flow reasons. This seems like a group that could be at very high risk for ADEs in the first place. For non-English speaking patients who did have a translator, what was the incidence of ADEs and how did this compare to the English-speaking population? Is there other previous literature in this area?
  6. Preventability: Using this rule, we can better identify ADEs. How many of these would be preventable though? As and example, if you don’t know someone has a sulfa allergy, or someone develops Stevens–Johnson syndrome, isn’t that just bad luck?
  7. Nursing Identification: There was a significant number of misclassifications made by the nursing staff. We don’t have pharmacists at all times of day, so how can we improve identification at triage or earlier in the ED visit?
  8. Rule #1 vs. Rule #2: Rule #1 had a slightly lower sensitivity than Rule 2 (91% vs. 96%) but higher specificity (38% vs. 23%), but Rule #1 seems much easier to use. Which would you recommend for other EDs and why?
  9. Electronic Medical Records (EMRs): I am glad you did not refer to EMRs as electronic HEALTH records (EHRs) because as described by ZdoggMD there is nothing “healthy” about putting a computer between the physician and patient. How do you think EMRs could be used to improve the identification of patients at risk for AEDs?
  10. Impact Analysis: This was not an impact analysis of these two tools. However, you did published an impact analysis before doing the validation study (Ref 12). Is this not putting the cart before the horse. Could you explain how this happened and what were the results of the impact analysis study?

Comment on Authors’ Conclusion Compared to SGEM Conclusion: We generally agree with the authors’ conclusion.

SGEM Bottom Line: Use of a clinical decision rule (tool) by clinical pharmacists and ED physicians can help to identify patients at risk for adverse drug events in the ED.

Case Resolution: Based on her recent initiation of antibiotics, you suspect this diarrhea may be due to an adverse drug event. You send stool cultures for C. difficile and tell the patient to stop the amoxicillin/clavulanate (Clavulin or Augmentin) as it is not needed for sinusitis anyway. The next day her stool assay is positive for C. difficile, and you call the patient to start her on appropriate therapy.

Dr. Chris Bond

Clinical Application: Clinical pharmacists can be a great resource to the emergency department. These two tools look promising, especially Rule #1, to micro allocate this important scare medical resources to those patients at higher risk of ADEs. It would be great to have another impact analysis of these tools before implementing them widely to ensure that they have a net patient-oriented benefit.

What Do I Tell My Patient? You have experienced an adverse drug reaction from the antibiotic used to treat your sinus infection. Antibiotics can disrupt the normal bacteria in your gut and this can sometimes result in the development of an infection called C. difficile. We will stop your other antibiotic and need to treat this infection with a different drug as it can become serious if left untreated.

 Keener Kontest: There was no winner to the last SGEM episode.

  • Question: In the movie Star Trek Generations Captain Kirk is trapped in the Nexus. He escapes with Captain Picard to save the Enterprise but is killed in the process. What were his final words?
  • Answer: It was fun…oh my

Listen to the SGEM podcast on iTunes to hear this the new keener question. If you know the answer send an email to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.

SGEMHOP: Now it is your turn SGEMers. What do you think of this episode on adverse drug events? Tweet your comments using #SGEMHOP. What questions do you have for Corrine and her team? Ask them on the SGEM blog. The best social media feedback will be published in AEM.

Also, don’t forget those of you who are subscribers to Academic Emergency Medicine can head over to the AEM home page to get CME credit for this podcast and article. Here is the process to obtaining your CME credits:

  • Go to the Wiley Health Learning website
  • Register and create a log in
  • Search for Academic Emergency Medicine – “August”
  • Complete the five questions and submit your answers
  • Please email Corey (coreyheitzmd@gmail.com) with any questions or difficulties.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’Guide to Emergency Medicine.


  1. Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal. 2004;170:1678-86. 2.
  2. IOM (Institute of Medicine). To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.
  3. Forster A, Asmis T, Clark H, et al. Ottawa Hospital Patient Safety Study: incidence and timing of adverse events in patients admitted to a Canadian teaching hospital. Canadian Medical Association Journal. 2004;170:1235-40. 4.
  4. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377-84. 5.
  5. Zed PJ, Abu-Laban RB, Balen RM, et al. Incidence, severity and preventability of medication- related visits to the emergency department: a prospective study. Canadian Medical Association Journal. 2008;178:1563-9. 6.
  6. Hohl CM, Brubacher J, Hunte G, et al. Clinical Decision Rules to Improve the Detection of Adverse Drug Events in Emergency Department Patients. Academic Emergency Medicine. 2012;19:640-9.
  7. Hohl CM, Robitaille C, Lord V, et al. Emergency physician recognition of adverse drug-related events in elder patients presenting to an emergency department. Acad Emerg Med. 2005;12:197-205. 8.
  8. Hohl CM, Zed PJ, Brubacher JR, Abu-Laban RB, Loewen PS, Purssell R. Do Emergency Physicians Attribute Drug-Related Emergency Department Visits to Medication-Related Problems? Annals of Emergency Medicine. 2010;55:493-502. 9.
  9. Classen D, Pestotnik S, Evans S, Lloyd J, Burke J. Adverse Drug Events in Hospitalized Patients. Journal of the American Medical Association. 1997;277:301-6. 10.
  10. Klopotowska JE, Wierenga PC, Smorenburg SM, et al. Recognition of adverse drug events in older hospitalized medical patients. Eur J Clin Pharmacol. 2013;69:75-85.
  11. Hohl CM, Zed PJ, Abu-Laban RB, Brubacher JR, Loewen PS. Comparative Performance of Emergency Physicians and Clinical Pharmacists in Evaluating Patients for Drug-Related ED Visits. Canadian Journal of Emergency Medicine. 2009;11:274.
  12. Hohl CM, Partovi N, Ghement I, et al. Impact of early in-hospital medication review by clinical pharmacists on health services utilization. PLoS ONE. 2017;12:e0170495.