Pages Navigation Menu

Meet 'em, greet 'em, treat 'em and street 'em

SGEM#159: Computer Games – Computer Provider Order Entry (CPOE)

SGEM#159: Computer Games – Computer Provider Order Entry (CPOE)

Podcast Link: SGEM159

Date: July 5th, 2016

Guest Skeptic: Dr. Chris Bond. Chris is an emergency physician and clinical lecturer at the University of Calgary. He is currently the host of CAEP Casts, which highlights educational innovations from emergency medicine residency programs across Canada. Chris also has his own #FOAMed blog called Standing on the Corner Minding My Own Business (SOCMOB).

Background: Emergency department crowding is a growing issue across Canada. As more tertiary care EDs implement computerized provider order entry (CPOE), it is important to analyze emergency department metrics to see how CPOE may impact throughput.

Previous studies have shown that CPOE has no impact on mortality, and may in fact improve pain treatment and adherence to certain common presenting complaint medication protocols (such as stroke and renal colic) [1-4].

Some studies have shown there may be an impact on throughput in a number of possible areas such as decreased physician productivity, increased LOS for admitted patients, or increased time to order labs and imaging [5-7].

Other studies have shown that CPOE fixes some errors, creates new ones and frustrates physicians [8]. There is no consistent or comprehensive evidence in favor of CPOE [9].

A study looking at productivity in a community hospital emergency department showed the mean percentage of time spent on data entry was more than 40% and less than 30% spent on direct patient care. They calculated in a busy 10hr shift the number of mouse clicks was almost 4,000 [10].

To our knowledge there have been no studies to directly evaluate the impact of CPOE on emergency department wait times, a key variable in throughput and crowding.

Question: What impact will CPOE have on emergency department patient throughput?

Reference: Gray A et al. The impact of computerized provider order entry on emergency department flow. CJEM 2016.

  • Population: Emergency department patients 18 years and older presenting to two quaternary hospitals in July and August of 2013 and 2014.
    • Excluded: Patients with negative wait times or extreme outliers that exceeded 24 hours (presumed to represent an erroneously wrong day recorded). Also excluded any patients missing vital statistics (eg. Gender, age or CTAS score).
  • Intervention: Computerized provider order entry (CPOE)
  • Control: Non-computerized order entry
  • Outcome:
    • Primary Outcome: Emergency department throughput
      • Wait Time (WT): Time to first physician assessment after triage (minutes)
      • Length of Stay (LOS): Time to disposition after triage (minutes)
      • Left Without Being Seen (LWBS): Proportion of patients that LWBS/total patients for a given time period (%)
    • Secondary Outcome: Subgroup analysis
      • CTAS 1-5 (WT, LOS and LWBS) and admitted patients (WT and LOS)


Canadian Triage and Acuity Scale (CTAS): This was a national program started in (1999) to standardize emergency department triage in Canada.

Author’s Conclusions: CPOE implementation detrimentally impacted all patient flow throughput measures that we examined. The most striking clinically relevant result was the increase in LOS of 63 minutes for admitted patients. This raises the question as to whether the potential detrimental effects to patient safety of CPOE implementation outweigh its benefits.

checklistQuality Checklist for Observational Trials:

  1. Did the study address a clearly focused issue? Yes. The question was what impact did implementation of a new CPOE system has on wait times, length of stay and left without being seen.
  2. Did the authors use an appropriate method to answer their question? Yes. This was before and after CPOE implementation using administrative data.
  3. Was the cohort recruited in an acceptable way? Yes
  4. Was the exposure accurately measured to minimize bias? Yes
  5. Was the outcome accurately measured to minimize bias? Yes. The outcomes were objective (eg. WT, LOS, LWBS)
  6. Have the authors identified all-important confounding factors? No. This was only over a 2-month period before and after implementation of a new CPOE. A new CPOE is bound to have more difficulties in the initial start up phase as staff are learning to use it. Longer-term outcomes are needed.
  7. How precise are the results? Unsure. We will talk more about this in the Nerdy section.
  8. Do you believe the results? Yes
  9. Can the results be applied to the local population? Unsure. This depends on if you are using the same/similar CPOE and how much the CPOE does for you (eg. meds, diagnostic imagine, all orders, etc.). It also depends how long the CPOE has been in use in your particular setting, as more time will likely address some of the weaknesses of the CPOE and users will become better with it.
  10. Do the results of this study fit with other available evidence? Yes/No. There is conflicting evidence with respect to the effect of CPOE on LOS and other patient flow parameters.

Key Results:

  • Median WT increased by 5 minutes (78 vs. 83)
  • Median LOS increased by 10 minutes (254 vs. 264)
  • Proportion of LWBS increased by 0.9% (7.2% vs. 8.1%)
  • Median LOS for admitted patients increased by 63 minutes (713 vs. 776)
  • Proportion of LWBS increased significantly for CTAS 3, 4 and 5 patients (CTAS 5 patients 24% vs. 42%)

CPOE Table

Screen Shot 2015-04-25 at 3.11.12 PM

Dr. Andrew Gray

Dr. Andrew Gray

Listen to the podcast to hear Andy’s responses to our questions.

  1. Excluded Patients: You excluded 466 patients before CPOE and 1,235 after CPOE. Why did you have three times as many patients excluded after CPOE and do you think that impacted the results?
  2. Interquartile Range: You represented wait times and length of stay as medians with interquartile ranges. Why did you use these statistics to describe your data and do you think this gives you a precise estimate of the results?
  3. Statistical vs. Clinical Significance: You demonstrated statistically significant changes (a few minutes for WT and LOS, ~1% increases LWBS and ~1hr increase LOS for admitted patients who were waiting a median of 12hrs already) but do you think these represent clinically significant changes?
  4. Two Months of the Year: You only looked at two months (July and August) in 2013 before CPOE and 2014 after CPOE. These are summer months when you have new residents starting and lots of people taking holidays. Do you think these two months are representative of the whole year?
  5. Start Up Phase: The CPOE was introduced to the entire hospital system as part of a program called HUGO (Healthcare Undergoing Optimization) in April 2014. There is a learning curve with new systems. Perhaps more training or better training was needed. In other words, could the impact on emergency department flow be related to CPOE difficulties in the start-up phase of the HUGO project?
  6. External Validity: This study took place at the London Health Science Centre (LHSC) that included two quaternary care emergency departments in London, Ontario. Do you think your study has external validity to other emergency departments (Non-Teaching, Community, Rural, Non-Canadian)?
  7. Before/After: One of the problems with before and after studies is other changes over time could have been responsible for the differences observed. Do you think any other factors could have played a role besides CPOE?
  8. Patient Oriented Outcomes: You measured WT, LOS and LWBS but did you consider and measure other patient oriented outcomes like medication errors, adherence to evidence based medicine protocols, time to pain medications and overall patient satisfaction? These are other quality indicators that have been investigated in other CPOE studies.
  9. Lack of In-Patient Beds: Many Canadian hospitals, including yours, have occupancy above 80% and sometimes as high as 125%. This can lead to overcrowding in the emergency departments. What impact if any do you think this had on your study?
  10. Physician Satisfaction: A new study came out in the Mayo Clinic Proceedings showing that physicians’ satisfaction with electronic health records (EHRs) and CPOE was generally low and those using EHRs and CPOE were at higher risk for professional burnout (Shanafelt et al 2016). Did you see any issues with physician satisfaction due to the introduction of CPOE?

Comment on author’s conclusion compared to SGEM Conclusion: We agree that this implementation of CPOE at this quaternary hospital system had a detrimental impact on emergency department patient flow. It is unsure if these increased WT, LOS and LWBS rates are clinically important. We also question whether the potential benefits of CPOE outweigh the potential detrimental effects of CPOE on patient safety.

SGEM Bottom Line: Implementation of CPOE may initially be met with some difficulties, worsen emergency department patient flow and contribute to emergency department over-crowding. The long-term impact on patient oriented outcome and physician satisfaction remains to be seen.

 FOAMed Resources: Check out the video by ZdoggMD called EHR State of Mind

Keener Kontest: Last weeks’ winner was Patricia van den Berg from the Netherlands. She knew India was a country suffering from a cholera outbreak in the 1960’s that prompted the development of rehydration electrolyte solutions.

Listen to the podcast for this weeks’ keener question. Send your answer to The first correct answer will receive a cool skeptical prize.

SGEMHOP Social Media: Now it is your turn to have a say. What do you think about this SGEMHOP episode? What questions do you have for Dr. Gray about CPOE? Join the conversation on Twitter (#SGEMHOP), Facebook or the SGEM blog. The best social media feedback will be published in CJEM.


  1. Netherton et al. Computerized physician order entry and decision support improves emergency department analgesic ordering for renal colic. Am J Emerg Med 2014
  2. Yang et al. Implementation of a clinical pathway based on a computerized physician order entry system for ischemic stroke attenuates off-hour and weekend effects in the ED. Am J Emerg Med 2014
  3. Brunette et al. Implementation of computerized physician order entry for critical patients in an academic emergency department is not associated with a change in mortality rate. West J Emerg Med 2013
  4. Blankenship et al. Prospective evaluation of the treatment of pain in the ED using computerized physician order entry. Am J Emerg Med 2012
  5. Bastani et al. Computerized prescriber order entry decreases patient satisfaction and emergency physician productivity. Ann of Emerg Med 2010
  6. Spalding et al. Impact of computerized physician order entry on ED patient length of stay. Am J Emerg Med 2011
  7. Syed et al. Computer order entry systems in the emergency department significantly reduce the time to medication delivery for high acuity patients. Int J Emerg Med 2013
  8. Schiff GD et al. Computerized physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf April 2015
  9. Georgiou A et al. The effect of CPOE systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med 2013
  10. Hill RG Jr, Sears LM, Melanson SW. 4000 clicks: a productivity analysis of electronic medical records in a community hospital. Am J Emerg Med 2013

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

 FallsViewBEEM 2016







  • Pingback: SGEM#159: Computer Games – Computer Provider Order Entry (CPOE) | Online Medical Assistant Training Programs()

  • Kirsty Challen

    Interesting to see how these things designed to help us may not..?

    • TheSGem

      Thanks for summarizing another episode. This one was really interesting – how technology impacts our work flow.

      • Andy Gray

        Nice. I like the graphics!

  • Pingback: SGEM-HOP: Computerized Provider Order Entry (CPOE) - SOCMOB Blog()

  • Steve

    I was a part-time ED tech/secretary for about 10 years prior to starting my emergency medicine residency. If you would have told me then that I would be entering my own orders as a doctor I would have laughed out loud. Why would you make a doctor do a task that can be easily relegated to others? The ED doctor (and any other advanced care providers such as NPs and PAs) is the rate limiting step in the ED. The ED provider is the only one who can evaluate, diagnose, determine a treatment plan, and decide the patient’s final disposition. On the contrary, an ED secretary is more than capable of entering orders into a computer but they cannot do the things that a provider is trained to do. When CPOE is implemented, this adds another layer of tasks to the ED provider that slows down their workflow. When I was working as an ED secretary it was easy for the ED providers. They could check what tests they wanted, leave that checklist with me, and go on to see the next patient instead of having to sit down at a computer and type in the orders. This CPOE slow down is obvious from the results in this study and the increased length of stay and left without being seen rates are harmful to patients. The literature on whether CPOE decreases order entry errors is mixed at best. When I was an ED secretary I fully admit that I made mistakes with orders including an older patient who received a head CT by accident. However, the literature on CPOE also makes clear that ordering tests on the wrong patient is one of the most common CPOE errors, especially when you are charting and ordering within the same system.

    • Andy Gray

      It is very interesting hearing from someone who has worked both sides of the ED orders, as a clerk and a doctor.

    • Kirsty Challen

      As a Brit the concept of a clerk/scribe who does the ordering is just mind-blowing! We all do it all…..hence possibly less impact of CPOE as the doc simply has to type what they previously had to write?

  • Eddy Lang

    Great podcast and critique of this work. I worry that the message, at first glance, may bolster CPOE naysayers. My bottomline after listening and thinking of the state of the evidence on this as well as our own experience in Calgary would be: CPOE may have an association with increased LOS for admitted patients but without adjustment for boarding and EIPs there is no way to no if any relationship exists. It’s also hard to imagine why it would except if admitting services were spending alot more time writing admitting orders which delayed bed assignment which shouldn’t be the case. Unfortunately this is not reported as far as I can tell.

    I think CPOE has now been shown to improve patient outcomes and if this comes at a few extra minutes of waiting – bring it on.

    In our hands CPOE is looking to become an impressive KT tool and has allowed us to standardize prescribing for many conditions and eliminating low value tests simply be removing them from order sets. CPOE can offer impressive analytics and can promote communication and teamwork. I am far more efficient with it than without it if I simply consider not having to look for charts to place f/u orders.

    • Chris Bond


      Great points.

      I hope this doesn’t bolster the naysayers too much, as the literature surrounding CPOE is clearly a mixed bag, with the exception of reducing med errors and adverse events.

      I especially like your point regarding KT, which is one of the hidden advantages of CPOE. Having people responsible for the order sets allows them to be updated with the latest evidence and people will not order useless tests if they are not available as an option to order. Eg. Removing the INR from the chest pain order set is a huge savings for a test that will be normal in all the non anti-coagulated patients, and won’t provide any additional info for those on NOACs anyway.



    • Andy Gray

      My theory as to why admitted patients have an increased LOS at our institution was admitted patients are often more complex than the average patient, take longer work up and more meds/orders before disposition. This usually means more time at the computer. The actually admission process definitely takes longer with CPOE since much of the clerical tasks now fall on the doc. Rather than just ADDAVID, docs are now entering med reconciliations and creating admission encounters, looking up orders or creating sets for complicated treatments (like alternate days or delayed/timed tests). Everything runs smooth for the average/common patients, but when you try to personalize medicine with CPOE the work load increases dramatically.

  • Nadim Lalani

    Thanks for this post. I think it captures the reality and that is that there’s a mixed bag of literature pro and con CPOE.

    The current context is also that most urban centers are seeing older more complex patients that increasingly come from nursing homes in overcrowded, short-staffed EDs. When you add the fact that our skill set is expanding [i.e POCUS] … I feel that the amount of TIME it is taking to fully assess most patients is already increasing. Also different centres also have different cultures and resources e.g. Saskatoon ED’s there’s no ED porters and unit aids – so clerks do portering [and are often short-staffed!].

    My experience in Saskatoon – AFP funded model:
    Orders are still written orders … check boxes that are QUICK and EFFICIENT, but are entered in by clerks but have issues e.g. 1) cannot unbundle things – so male trauma victims often get serum BHCG levels 2) indications are filled out like (for Chest Xray) “r/o PE” – which are damaging to the brand of EM. 3) There are also different order entry systems across different sites [need to fax in a filled out xray req at one site vs check box at another] 4) Often orders get ordered using my name because clerk couldn’t read the internal resident’s writing – so I get serum alpha-fetoprotein levels come back to my mailbox that I am now [as per CMPA] ethically and legally responsible for

    CHARTING is on computer. I really struggled with this. The roll out was less than exemplary – local culture of battle-testing everything in the ER first regardless of surge| bed-block meant that it seemed hastily implemented with little flexibility. Thankfully we had great advocates for the ER team [Wes, Stacy, Bruce and Vern] who helped generate a more workable format, and helped get rid of impractical things on it [like neurology’s stroke evaluation at the top of the ER entry set!] and advocated for a more workable roll out. (and I should add a GREAT group of docs that seem to roll with all the punches) I felt that there is a real disconnect between the IT crowd and the ED. It appeared the IT folks didn’t get the clinical side and I feel that the ED concerns were sometimes simply ignored – and that the opportunities for physician input seemed token opportunities – but this is my opinion only – like others I am guilty of not going to all of the design meetings and then complaining after the fact.

    CPC definitely – BOGGED DOWN the ED. We had folks in our group that averaged less than 2 pts/hour before CPC and are down to around 1 patient per hour. We had folks staying 3-4h after shift to finish charting [and charging overtime] . My productivity declined by at least 2-3 patients per shift as I found that couldn’t type and take hx at the same [and needed to transcribe my written notes into the computer after each hx] – I felt this maintained my cognitive processes as I wasn’t constantly trying to backspace typos and missing bits of the hx. I felt I could sit and listen to the patient and maintain some sense of bedside manner. By the time I had left, I had improved alot [but still transcribed]. My work around was to ignore the check boxes and use SOAP note for everything together with templates for ATLS, PALS assessments.

    Benefits of CPC were that: 1) forced some colleagues to chart – who had illegible writing and | or never charted enough 2) bounce-backs had a clear picture of who saw them / when / what antibiotic was prescribed etc.- instead of trying to locate a chart 3) consultants and GP’s really valued this format. 4) there was a sense that it was a bit safer overall, but no data.

    Problems with CPC:
    1) WAY SLOWER 2) Still needed to be more user friendly 3) Trying to generate meaningful data etc was often met by “we don’t have that version of the software” “we cannot borrow other centers’ templates”.

    I felt in all that it harmed me more than helped, and led to increased waits and worsened the disparity between slow and fast docs – but I plodded on and did my best. I am grateful for the AFP funding model that allowed me to change my practise enough to “bend” to the system, but not so much that I did not still enjoy my job.

    My experience in Calgary fee-for-service model:
    CPOE. I really value my colleagues Tom, Shawn and others who have worked so hard to generate easily workable order sets that include things like “Well’s Score” for VTE. I like that if I type “ED meningitis” I get all the orders for this presentation [including CT and all four LP tubes pre-labeled] I like that if I go to any of the 4 hospitals I work at … it’s the exact same process. I get to tell the rads the exact reason I want that CT [which has lead to useful conversations where they call back and there’s dialogue]. It is a bit slower, but one gets up to speed as much as one can. I am not poky slow, but when I look at the clock – most (nursing home/chest/abdo pain/SOB) visits I see are 15-20 minutes including hx/pe/POCUS/answering Q’s/charting – this is the limit I am willing to tolerate in a FFS model.

    The only time I have really been frustrated with CPOE in Calgary is when I was forced to order all the meds for a nursing home patient going to a rehab bed – this took me >20 minutes. immediately after this on the same shift I had to manually enter orders for a homeless person to take home a list of ten medications – these two patients killed my productivity for about an hour. I felt that this was non-value added work that could have been done by another provider. Being fee-for-service I was acutely aware that I was generating zero income from that hour. I feel the FFS model is a huge disincentive for important-to-the-patient things like med reconciliation – I feel that there has to be a bit more work done on this. Thankfully at the South Campus there are amazing pharmacists who provide real value and help us with some of this stuff – definitely recommend this model to others.

    We DO do a little physician charting, but it’s at discharge where we are encouraged to write a 3 sentence summary and DC instructions – I have found this v useful with bounce-backs.

    Overall I am happy with the Calgary model. but i feel things like reconciling a patients medications and ordering all the home medications – needs to fall to other providers as I feel it’s non-value added for physicians to do it. In a FFS model I feel that computer physician CHARTING would make it impossible for me to keep my numbers (and income) up. I feel it would take an average 2.3-2.4 pph doc and make him more like a 2.0 pph doc. if you multiply this by an entire group over a year … it would mean many more waiting room patients being ignored in our already busy ED’s.

    You solicited my opinions – I have tried to give a balanced but honest response. I understand that I am a leader in my both departments and so apologize if my comments hurt the overall mission … this is not my intention – ultimately I always bow to the wisdom of the group and tow the line as best as I can. Best NL

    • Chris Bond

      Wow, thanks for the detailed response, Nadim.

      I didn’t realize you were charting electronically in Sask in addition to orders. That would be challenging to say the least. I would imagine the order entry is a lot of growing pains, something that you noted has been worked out amazingly well by some of our colleagues here in Calgary.

      I don’t know if I’d ever enjoy computerized charting in an ED setting, as in many cases, the patient is too sick to perform it in real time. In office/clinic settings or with lower acuity patients, it makes a lot more sense and can be more easily integrated into the initial patient encounter. I don’t have the reference, but there is some literature about patients appreciating that a physician is actually documenting details of their visit, and don’t find it nearly as intrusive to the patient-doctor relationship as we might think.

      While there are some major downsides for flow, one nice thing in a big department (space wise) is the ability to make follow-up orders from across the department when a lab result returns. This certainly does save some time, although this is more than offset when you have to enter admission orders on a patient because an admitting service isn’t around.

      This clearly is an issue with many pros/cons, and I’m not sure computerized order entry will ever get it “right”. That said, as comfort level with CPOE improves and it becomes fully integrated within our training, our efficiency will improve.

      Thanks again for your detailed comment.


      • Nadim Lalani

        Hi Chris [others] to clarify. To date Saskatoon physicians do computer CHARTING but not computer ORDERS [though i think this is being considered].
        I am overall happy with the Calgary CPOE – it took a lot of work from Tom, Shawn and others but it’s functional and as good as it can be I think. Thanks

    • Andy Gray

      The medication reconciliation used to fall on our physicians at LHSC as well, but they have now implemented a work force of pharmacy techs that do a med req on every patient in the ED prior to admission. This makes a HUGE difference to my productivity, and I’m sure would reflect in WT/LOS in a repeat study.

  • Chuck Wurster

    Great article and podcast. Thanks to everyone else for their comments as well. I really enjoyed the discussions.

    I would like to share some of my personal experiences with CPOE and how it has affected my practice and my current practice environment.

    I have been working at the Nanaimo Regional General Hospital emergency department in Nanaimo for the past 3 years after spending the first 12 years of my career in Nova Scotia using good old pen and paper. When I first started in Nanaimo they had already been using electronic charting for 6 months or so. We use Dragon Naturally Speaking software connected to portable computers on carts. After a bit of a learning curve, some training of the Dragon and some upgrades that our group forced IT to make that fixed slow servers, I was able to dictate efficient and accurate charts just as fast as I would have done in the past on paper. Bottomline for the charting aspect for me was that if you support the voice recognition software with the most up-to-date software version and provide good fast servers, then it works well. If you don’t, then it is slow, clunky and frustrating. As it stands now, the charting part of our computerized department works well.

    Then the health authority introduced iHealth this March and everything changed, and not for the better. Our entire hospital was changed to fully electronic charting at once. Before iHealth, the emergency department was the only place electronic documentation took place. Now, the whole hospital needed to document on the computer AND physicians needed to enter all orders with CPOE, including our department.

    It has been nothing short of a disaster. The user interface is cumbersome, non-intuitive and resembles something from the 1990s written in DOS. Our training was not adequate and we were given little input in how we would like the computer system to work for us. No one examined how using CPOE would affect our daily processes in the emergency department nor in the hospital as a whole.

    The biggest problem, however, was that the system, when we went live, was full of bugs. Many, many bugs. Things didn’t work the way we were trained and it become quite evident to most frontline workers that mistakes would happen if we weren’t extremely careful with how we clicked the mouse. Mistakes, including many medications errors did happen. We have documented many but I am certain that many have happened that we don’t know about.

    Nanaimo is a community hospital and, as such, emergency physicians like me end up writing most admissions orders because our consultants or hospitalists are not available. This was challenging before iHealth but become completely unfeasible after. Trying to complete medication reconciliation, for example, was impossible because our system was not set up properly to do it any timely or accurate manner.

    I won’t bore you with how things have gone since as you can probably get a lot of this from the media attention that has befallen our beleaguered hospital. Google away – you will get a lot of hits…..

    The emergency department and our ICU have now gone back to using pen and paper to write our orders. We still do admissions but these are all on paper until the hospitalist / consultants sees the patient and inputs orders into the CPOE. I don’t think we will be going back anytime soon to CPOE.

    How has CPOE affected me as emergency physician?

    When we were solely on CPOE, it completely crushed my efficiency. I went from being someone who could see 30 patients per shift into someone who was lucky to see 10. I am sure it affected many of my colleagues the same way. Our department went from being one of the most efficient in the country to a place where 8 hour waits became common.

    CPOE also has had a significant effect on my job satisfaction and many of my co-workers. We still have a great group and I love working with our docs, nurses and support staff, but our morale has been the pits for many months now. Things have improved a lot since we went back to paper orders (with the nurses getting a paper MAR that they can actually understand), but it is still tough to change a shift because no one wants to grab them up like they used to. Most of my colleagues have thought about working elsewhere, myself included.

    We are optimistic that our CPOE can be fixed. I do think that electronic charting is the way of future. I live in an iHome and everything in my house is connected by Wifi.

    But, if you design and implement CPOE you MUST make sure it is user-friendly, intuitive and error resistant. Our system was introduced completely opposite to this and it has destroyed my department’s efficiency and, more importantly, our morale. All we hear about is iHealth. We are bombarded by emails and meetings about it’s problems and the promises it will get better. We don’t talk about cool cases or have interesting rounds. We have a new simulation dummy that has been sitting in a sleep room in a bag for months and has never been used to practice so we can work better as a team in a resuscitation.

    I wish I could reply to specific comments made by Nadim and Eddy about metrics and other nuances of different CPOE systems. We are not even close to this in Nanaimo because we were given a broken system that was impossible to use.

    For all those of you who are close to getting CPOE (that would be most everyone eventually) then please ensure you have input into the vendor (we didn’t) and design of the system. Please ensure that it is tested, tested and tested again. And not just tested in a computer lab by non-clinicians, but by you personally in real-world simulations. Make sure your medication reconciliation system works if you have to do this. And please remember that the most important part of our work should focus on the patients. If you think that the computer is getting in the way of treating your patient then do something about it. Our group has done something and I am proud that we are being diligent that CPOE should improve how we care for our community. We aren’t there now and that’s why we are using paper orders. Hopefully we can re-introduce CPOE, it works well and we can go back to being the upbeat and efficient place we used to be. Heck, we might even start to doing some sims!!

    • Andy Gray

      You really highlight the fact that CPOE is very program dependent. Sounds like your software was an extreme example of bad software. I think the same message applies though as what were trying to show in this paper – “poor implementation” is the key. CPOE has good potential, but not properly tested/implementation can have a huge impact on flow. The flow issues can become patient safety and physician satisfaction problems if left unchecked.

  • Kevin Klauer

    The Data Cannot Justify the Expense

    This data is useful in supporting the immediate impact CPOE may have on ED operations. Although I do believe, anecdotally, and based on the available data, the negative operational impact has been lessened as we grow more familiar with the systems and their shortcomings. However. it is truly amazing that we, as physicians and other clinicians, and hospitals as those shouldering the capital expense, and the ultimate consumer, the patient, tolerate mere smoothing of the negative impact of this well-intentioned but ill-conceived “modern convenience.” Many years ago, the EMR vendors had a handful of installs and were not gaining traction. Why, the business case and return on investment simply could not be made for an EMR: A large capital expense with the potential for increased charge capture, increased patient safety and automation. The sales pitch was positive, but the test drive rarely resulted in proof of concept. The systems were designed without the input of the end-user, “Us.” This led to systems that were not ready for prime time. What prompted wide-spread adoption in the United States, was not a drastic improvement in the product, but were financial incentives from the CMS Meaningful Use program, incentivizing adoption by hospitals without incentivizing industry to improve their products. The data is fairly clear: 1. CPOE tends to reduce operational efficiency to a variable degree; 2. CPOE does not reduce, but creates new forms of, medication error/adverse drug events; 3. CPOE forces physicians to spend more time away from the bedside and more time in front of a computer; 4. CPOE may result in patient harm and have not been proved to improve quality, safety or experience of care.

    Having said that, properly implemented components of an EMR, not necessarily CPOE, can help with standardization, reduced practice variation and reduced cost, particularly when implemented in a large system. Opinions do vary and data is conflicting. However, credible data suggests these outcomes are possible.

    The overarching goal for Meaningful Use was to provide interoperability between systems. If judged on this metric/goal, it was a disastrous failure. “The program has been in place for 5 years and the great promise of Meaningful Use is just around the same corner it was back in 2011. The only measurable changes from the pre Meaningful Use era are the billions of dollars subtracted from our treasury and the minutes subtracted from our time with our doctors, balanced only by the expenses added to our medical bills and the misery added to physicians’ professional lives.”

    Additional suggested readings below:

    L. Poissant et al., “The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review,” Journal of the American Medical Informatics Association 12, no. 5 (2005): 505–516.

    Spellman Kennebeck S1, Timm N, Farrell MK, Spooner SA.

    Impact of electronic health record implementation on patient flow metrics in a pediatric emergency department. J Am Med Inform Assoc. 2012 May-Jun;19(3):443-7. doi: 10.1136/amiajnl-2011-000462. Epub 2011 Nov 3.

    Bukata WR. Electronic Health Records – Where’s the Beef? Emergency Medicine and Acute Care Essays. 2011;35(2):1–4.

    Park SY, et al. The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform. 2012 Mar;81(3):204-17.

    Hill RG Jr, et al. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med. 2013 Nov;31(11):1591-4.

    Ward MJ, Landman AB, Case K, Berthelot J, Pilgrim RL, Pines JM. The effect of electronic health record implementation on community emergency department operational measures of performance. Ann Emerg Med. 2014;63(6):723–730.

    Risko N, et al. The impact of electronic health record implementation on emergency physician efficiency and patient throughput. Healthc (Amst). 2014 Sep;2(3):201-4. doi: 10.1016/j.hjdsi.2014.06.003. Epub 2014 Aug 6.

  • Shawn Dowling

    Andy – Thank you for doing this study – as a Emerg Doc and academic who works on the frontline seeing patients AND has a funded role to optimize and maintain our CPOE order sets in Calgary – I value the need for assessing the impact of CPOE although I do have a visceral response when I hear “poor implementation” comment mentioned in the paper.

    1. First off, the details of the implementation are not outlined enough for us to be able to determine the implementation strategy (I suspect the journal limited your word count such that you could not describe the whole process). If we are to learn from collective CPOE implementations, the devil is in the details. CPOE is not a homogenous intervention. Who developed the order sets? How much Clinical Decision Support is built into the order sets? How often are physicians using the order sets (vs ordering “off the browse”)? How many alerts are used (alert fatigue is a well documented phenomenon)? The analogy I’ve used when discussing CPOE in the past – is that of a bike. I have the same bike that Ryder Hesjedal used in the Giro D’Italia last year. But by no means am I going to win any significant bike races. Ryder’s success has to do with more than just his Cervélo (his bike). CPOE is more than a software program. Its success and failure is dependent on the components (order sets), its fit (usability of the software), the engine (physician buy in/engagement) and its mechanics (the team responsible for maintaining and optimizing the CPOE environment). CPOE is most powerful when it’s part of an integrated Knowledge Translation strategy (including embedded Clinical Decision Support- not just forcing functions or alerts, patient safety measures, and linked to electronic health records.

    2. I find it very interesting that the greatest impact was seen on the least acute patients (CTAS 5), which is not-intuitive, given that CTAS 5 patients are those who I am least likely to order any tests on and when I do order tests, they generally are isolated tests (i.e. not requiring any significant amount of time). Unfortunately you were not able to adjust for other confounders such as access block or secular trends (one of the flaws of before-after designs as opposed to more rigorous methodological strategies like Interrupted Time Series Analysis).

    3. I think the key is that, anyone who has “lived” through a CPOE implementation (and I have and continue to!), one knows that the learning curve is huge. The few month run in period you allowed for was a good idea, but unfortunately not a long enough run-in period. I would actually make the argument, that if the wait times only increased on the magnitude of minutes 3 months post implementation (as your study showed) that this could be considered a success (or less of a failure – depending on your perspective). I can confidently say as users become more familiar with the system their efficiency will improve.

    Finally, I’ll end with reiterating the point I made initially – thanks for doing this study. There is a need to continue to continue to evaluate the impact of CPOE – both in terms of department flow, patient safety, resource usage (both appropriate and inappropriate) and ultimately, patient outcomes. The implementation, maintenance and optimization of CPOE is an iterative process and we need to continue to evaluate our own local data and share it with others so that collectively we can prioritize our efforts going forward.

    Shawn Dowling, MD, FRCP
    Emergency Physician – FMC and ACH, Clinical Assistant Professor, U of C
    Clinical Content Lead/Effectiveness Coordinator
    Medical Director, Physician Learning Program

  • Tom Rich

    I am always interested in reading articles about CPOE and its impact (both positive and negative on either direct patient care, or indirectly on patient care by changes to wait times.) Universally these studies have so many problems they are hard to interpret which is why I think we see so many conflicting views. But I do have a few comments. (Many of them already stated in previous posts)

    1. For anyone who does any research or analysis on wait time data (and trust me we have become experts in Calgary over the last years) it is almost impossible to look at one variable and control for other variables. So to say one factor contributed either negatively or positively over wait times is impossible to say. Just ask any ED who have seen their wait times significantly increase over the last few years that have not implemented CPOE.
    NB: In Calgary, when we looked at the one area that ED has full control over (medication delivery of ward stock medications in the ED, CPOE was found to significantly decrease the time of med order, to delivery). Interesting enough, the time to DI results over the same time significantly increased, yet the ED work flow (MD directly enters the DI order, and no longer writes, flips chart, waits for RN or unit clerk to enter the order to DI) obviously is decreased. Nothing like entering a DI order, and before you have even moved onto the next patient, a porter is there to take your patient to DI. This is a great point in showing that wait times impacted by controls outside of the ED can not necessarily be attributed to changes in ED business practice.

    2. Poorly designed CPOE is a bad thing. This can be in both negating any positive effects of an EHR such as drug to drug interaction alerts, allergy notification, drug health issue alerts as well as create an order entry nightmare (time consuming, confusing, and prone to ordering errors). A significant amount of time and effort must take place to look at business needs, work flow with IT, needs of physicians, order sets, properly designed Clinical Decision Support, knowledge transfer at the time of order entry to name just a few. This absolutely needs to be clinically driven and not IT driven. In Calgary, I suspect most Calgary ED MD’s would see CPOE as a huge advantage in your more complex patients. (So many comments that the complex patient is where CPOE falls apart) A lot of time, effort, and clinical decision support has gone into our complex order sets (such as DKA, ACS, GI bleed) with the most up to date literature supported evidence in practice. There is no way these orders could be created, or entered in as rapid a manner as we have with CPOE.

    3. Its not just about software and design. Something as simple as insufficient work stations, or poorly placed work stations is enough to negatively impact even the best designed CPOE system. I have toured enough departments and have actually seen MD’s line up behind work stations waiting their turn to use a computer. Even something seemingly insignificant and simple as time to log onto a system is a huge factor in the ED as it it done hundreds of times a shift.

    So interesting study but hard to conclude the impact of CPOE on wait times. Way too many other variable at play. But I will also say that there is no question a poorly designed CPOE system, with a poor training and implementation strategy, and lack of detail around business needs could have an significant negative impact on ED flow. The challenge around any CPOE study (and again why so many conflicting results) is the inability to standardize for CPOE design, training, implementation, and IT infrastructure: all of which would also impact the efficiency of a CPOE system in the ED.

    • Chris Bond


      Great points. I really do wonder about the differences in CPOE at various sites, as the quality of the CPOE likely makes a huge difference. As you say, the complex order sets can be extremely helpful in getting all of the diagnostics and treatment ordered simultaneously, and also helps prevent forgetting a test or two that you might otherwise.

      You’re right that the conflicting results are likely due to a number of factors, and controlling for a single variable is probably artificial.

      How would you propose that we actually do the studies to determine the effect of CPOE on flow/ED throughput?


      • Tom Rich

        Hi Chris
        This would be extremely challenging to do for so many reasons (as you can well imagine).
        Even from the very beginning, trying to isolate CPOE as a factor is challenging as CPOE is unlikely (and if implemented correctly) should never be implemented without a proper business assessment and analysis of current business process issues that can be improved with an electronic system (outside of just CPOE).
        Also, since workflow is such a key and important factor in ED business practice, very few departments would not being introducing other initiatives at the same time. (Or be impacted by issues outside of their control in a short period of time, as we have seen locally with a sudden increase in LOS data due to sudden increase in inpatient occupancy of ED treatment spaces). If CPOE had been introduced at the same time, would have been an easy mistake to link the changes in LOS data to CPOE.
        I think really the best (and still not perfect) analysis would be to take two sites, very similar in size, patient population, admission rates, and business practice) and have CPOE introduced at one site and not the other. These sites would have to be geographically close, and have similar management. They would also have to be static in changes for the study period (i.e. would not work if one site added a physician shift during the study period).
        We could have done this in Calgary when we went live with CPOE should we have chosen to keep a site from implementing. But as you can imagine not a great business decision for study reasons only.
        Outside of that, becomes a challenge to really study the impact of CPOE on flow/throughput alone.