SGEM#185: Every EHR Wants to “Rule” the World

Subscribe: RSS

[display_podcast]

Date: July 11th, 2017

Reference: Bookman K et al. Embedded Clinical Decision Support in Electronic Health Record Decreases Use of High-cost Imaging in the Emergency Department: EmbED study. AEM July 2017.

Guest Skeptic: Dr. Justin Morgenstern is an emergency physician and the Director of Simulation Education at Markham Stouffville Hospital in Ontario. He is the creator of the excellent #FOAMed project called First10EM.com

Case: You are working a very busy shift in the emergency department. Your first patient was involved in an motor vehicle collision (MVC), and you pulled up the app to review the Canadian CT Head Injury Rule with your resident. The second patient is an elderly woman presenting via EMS after a ground level fall in c-spine precautions. This time, you review the NEXUS Criteria for c-spine imaging with your resident. The third patient just got off a long flight and is short of breath. Once again, you find yourself on MDCalc, this time teaching the Well’s Criteria for pulmonary embolism and the PERC Rule.

Your resident is very grateful for all the teaching, but asks, “we seem to spend a lot of time looking up decision instruments. Wouldn’t it be easier if these tools were just part of the electronic health record (EHR), since we are already inputting all the patient data into the EHR anyway?” You think to yourself, “interesting, I wonder what impact integration of these rules into the EHR would have on CT ordering in my group?”

Background: Use of CT scanning has increased more than 20-fold since 1980[1],[2]. Although the CT is certainly a valuable tool, it is likely overused in emergency medicine.[3] CT has downstream harms, including radiation, incidental findings, over-diagnosis, financial costs, and negative impacts on emergency department throughput.[4][5]

Numerous clinical decision rules, such as the Canadian CT Head Rule[6], the NEXUS c-spine tool[7], the PERC Rule[8], and the Well’s Criteria[9], have been developed to help guide appropriate imagining.

However, uptake and appropriate use of these tools is not universal.[10] These authors question whether integration of a clinical decision support tool into the ordering system of the EHR would influence CT usage rates.

Clinical Question: Does deploying a novel, evidence-based, electronic health record (EHR) integrated clinical decision support (CDS) tool influence overall utilization of three specific high cost CT imaging studies: head, c-spine and PE?

Reference: Bookman K et al. Embedded Clinical Decision Support in Electronic Health Record Decreases Use of High-cost Imaging in the Emergency Department: EmbED study. AEM July 2017.

Population: 163 attending-level emergency physicians at five emergency departments (one academic and four community sites – two large urban sites, two smaller urban sites and one rural site).
- Excluded: Community sites in which both an attending and advanced practice provider both saw the patient
Intervention: A new clinical decision support (CDS) tool was integrated into the electronic medical record (EHR). The CDS tool integrated the Canadian CT Injury head rule, the NEXUS c-spine rule, the Well’s Criteria, and the PERC Rule.
Comparison: Baseline level of CT usage during a six month period before the CDS tool was integrated into the EHR.
Outcome: The impact in the overall ordering of non-contrast CT head, CT spine, and CT pulmonary angiogram (CTPA).

Dr. Kelly Bookman

Lead Author: Dr. Kelly Bookman is board certified in EM and medical informatics. She is the Medical Director and Associate Professor in the Department of Emergency Medicine at the University of Colorado. Kelly is also the Senior medical director of implementation and informatics at the UC Health emergency service Line.

Authors’ Conclusions: “Embedded clinical decision support is associated with decreased overall utilization of high cost imaging, especially among higher utilizers. It also affected low utilizers, increasing their usage consistent with improved adherence to guidelines, but this effect did not offset the overall decreased utilization for CT brain or CT c-spine. Thus, integrating CDS into the provider workflow promotes usage of validated tools across providers, which can standardize the delivery of care and improve compliance with evidence-based guidelines.”

Quality Checklist for Observational Trials:

Did the study address a clearly focused issue? Yes
Did the authors use an appropriate method to answer their question? Unsure. A controlled trial would be ideal.
Was the cohort recruited in an acceptable way? Yes
Was the exposure accurately measured to minimize bias? Yes. All providers were forced to go through the clinical decision tool before an order could be submitted, although they were allowed to ignore its advice.
Was the outcome accurately measured to minimize bias? Yes
Have the authors identified all important confounding factors? No
Was the follow up of subjects complete enough? Yes
How precise are the results? The confidence intervals are relatively small
Do you believe the results? Yes
Can the results be applied to the local population? Unsure. Even within their own results, we see different effects depending on how many CTs you were ordering before the intervention. The impact of integration of CDS tools into EHRs will probably depend on your baseline usage of those tools.
Do the results of this study fit with other available evidence? Unsure. Prior implementation studies of clinical decision support tool have sometimes shown decreased usage of imaging, but at other time have actually seen imagining increase. In fact, the first big implementation study of the Canadian CT head rule, by Dr Ian Stiell himself, demonstrated and increase in CT use by 13% after the rule was instituted.[11]

Key Results: There was a total of 235,858 patient visits during the study period. There was approximately a 6% decrease in targeted CTs (non-contrast head, c-spine, and CTPA) ordered during period after the intervention.

CT head and c-spine went down but CTPA was unchanged

Primary Outcomes:
- CT head corrected relative risk usage decreased by 10% (from 7.3% before to 6.6% after); 95% CI 7-13%, p<0.001,
- CT c-spine corrected relative risk usage decreased by 6% (from 2.1% before to 2.0% after); 95% CI 1-11%, p=0.03
- CTPA corrected relative risk usage was unchanged (1.5% in both time periods) (relative decrease of 2%; 95% CI -9% to +5%, p=0.42)
Secondary Outcomes: In a post-hoc subgroup analysis, change in CT usage as compared to baseline utilization:
- Baseline high users decreased CT use (18% decrease in CT brain, 14% decrease in CT c-spine, and 23% decrease in CT ).
- Baseline lowest third of users, there was no statistical difference noted in CT head, but both CT c-spine (29%) and CTPA (46%) studies were increased.

Listen to the SGEM Podcast on iTunes to hear Dr. Brookman’s answers to our nerdy questions.

1) Observational Study: This was a before and after observational study. Two out of three CT modalities had a statistically significant decrease over the study period. You correctly stated in the paper that this was an association only. Could there not be other confounding factors beside the embedded CDS tools responsible for the changes observed. In addition, the lack of control group to compare the intervention to makes this study more difficult to interpret.

2) Appropriateness of Scans: What is the right number of scans? Without clinical information, how can we know if the scans were appropriate or not? Was the increase in scans among low users a good thing or a bad thing? Were there more misses associated with the decreased CT rate?

3) Absolute versus Relative Numbers. Your results are presented as relative changes. The absolute changes are less than 1%. Why did you decide to use relative rather than absolute numbers? Do you think these changes regardless of whether they were absolute or relative translate into clinically important patient oriented outcomes?

4) Qualitative Methodology. As we were preparing for this podcast, my 18-year-old son Ethan shared a great article with us bemoaning the dominance of quantitative methodology at the expense of all other study types[12]. I noticed your study was originally mixed methods, with an incorporated qualitative analysis. Can you tell us about that?

5) Hypothesis Generating: You reported some post hoc subgroup analysis that showed different effects of the intervention depending on the baseline usage levels of the physicians. High users of CT seemed to lower their CT ordering, but lower users seem to order more CTs. Do you have plans to explore this finding further?

Comment on authors’ conclusion compared to SGEM Conclusion: We agree that CDS tools embedded into an EHR is associated with a decrease in CT utilization in this study, but the generalizability of these results remains to be seen.

SGEM Bottom Line: Embedding CDS tools into EHRs is associated with an impact on CT utilization but we need to know if it improves patient oriented outcomes.

Case Resolution: You discuss this fascinating study with your resident, and your department chief who happened to be walking by, and you agree that although EHR embedded CDS tools are probably not ready for generalized use, it would make a fantastic resident research project.

Clinical Application: This was an interesting study in an area we are sure to see a lot more research. EHRs are here to stay, and it would be great if we could harness their power to help make better decisions for our patients. However, for a variety of reasons, such tools are not ready for general use yet.

Dr. Justin Morgenstern

What do I tell my patient? There has been an interesting new study on integrating clinical decision support tools into the computer to help doctors make the best decisions about CT usage. Although we don’t have an electronic health record system like that here, let me pull up one of these tools on an app on my smart phone so that we can review your risk together and make a shared decision about the most appropriate care for you.

Keener Kontest: Last weeks’ winner was Magali Carlier from Belgium. She knew the famous physician was Virchow who first recognised that clots in the pulmonary artery originate in the venous system?

Listen to the podcast for this weeks’ question. If you know the answer then send it to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.

SGEMHOP: Now it is your turn SGEMers. What do you think of this episode? Tweet your comments using #SGEMHOP. What questions do you have for Dr. Brookman and her team? Ask them on the SGEM blog. The best social media feedback will be published in AEM.

CME Credits: Don’t forget those of you who are subscribers to Academic Emergency Medicine can head over to the AEM home page to get CME credit for this podcast and article. Here is the process:

Go to the Wiley Health Learning website
Register and create a log in
Search for “Academic Emergency Medicine – July”
Complete the five questions and submit your answers
Please email Corey (coreyheitzmd@gmail.com) with any questions or difficulties.

This episode marks the official end of Season#5 of the SGEM. Don’t panic, there will be some special content in July and August as part of a new SGEM Xtra series called: Legends of Emergency Medicine. The first episode has been posted with my EBM mentor Dr. Andrew Worster creator of Best Evidence in Emergency Medicine (BEEM).

Regular episodes of the SGEM will return this fall with its ongoing mission to cut the knowledge translation window down from over ten years to less than one year using the power of social media. The ultimate goal of the SGEM is for patients to get the best care based on the best evidence.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

References:

[1] Kocher KE, Meurer WJ, Fazel R, Scott PA, Krumholz HM, Nallamothu BK. National trends in use of computed tomography in the emergency department. Ann Emerg Med 2011;58:452–62.
[2] Raja AS, Ip IK, Sodickson AD, et al. Radiology utilization in the emergency department: trends of the last two dec- ades. AJR Am J Roentgenol 2014;203:355–60.
[3] Schuur JD, Carney DP, Lyn ET. A top-five list for emergency medicine: a pilot project to improve the value of emergency care. JAMA internal medicine. 2014; 174(4):509-15.
[4] Berrington de González A. Projected Cancer Risks From Computed Tomographic Scans Performed in the United States in 2007 Arch Intern Med. 2009; 169(22):2071-.
[5] Hoffman JR, Cooper RJ. Overdiagnosis of Disease Arch Intern Med. 2012; 172(15).
[6] Stiell IG, Clement CM, Rowe BH. Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA. 2005; 294(12):1511-8.
[7] Hoffman JR, Mower WR, Wolfson AB, Todd KH, Zucker MI. Validity of a set of clinical criteria to rule out injury to the cervical spine in patients with blunt trauma. National Emergency X-Radiography Utilization Study Group. The New England journal of medicine. 2000; 343(2):94-9.
[8] Kline JA, Courtney DM, Kabrhel C. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. Journal of thrombosis and haemostasis : JTH. 2008; 6(5):772-80.
[9] Wolf SJ, McCubbin TR, Feldhaus KM, Faragher JP, Adcock DM. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Annals of emergency medicine. 2004; 44(5):503-10.
[10] Brehaut JC. Clinical Decision Rules “in the Real World”: How a Widely Disseminated Rule Is Used in Everyday Practice Academic Emergency Medicine. 2005; 12(10):948-956.
[11] Stiell IG, Clement CM, Grimshaw JM. A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne. 2010; 182(14):1527-32.
[12] Holmes D, Murray SJ, Perron A, Rail G. Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. International journal of evidence-based healthcare. 2006; 4(3):180-6.