Pages Navigation Menu

Meet 'em, greet 'em, treat 'em and street 'em

SGEM#170: Don’t Go Breaking My Heart – Ottawa Heart Failure Risk Scale

SGEM#170: Don’t Go Breaking My Heart – Ottawa Heart Failure Risk Scale

Podcast Link: SGEM170

Date: March 2nd, 2017

Reference: Stiell et al. Prospective and Explicit Clinical Validation of the Ottawa Heart Failure Risk Scale, With and Without Use of Quantitative NT-proBNP. AEM March 2017

Guest Skeptic: Dr. Justin Morgenstern is an emergency physician and the Director of Simulation Education at Markham Stouffville Hospital in Ontario. He is the author of the excellent #FOAMed blog called and is an associate editor of Emergency Medicine Cases.

Case: A 68-year-old woman with a history of congestive heart failure, hypertension, and hyperlipidemia presents to the emergency department with a three-day history of shortness of breath on exertion, orthopnea and mild leg edema, but with normal vital signs. Her renal function is normal and troponin is negative. Her ECG is normal. After a dose of intravenous furosemide, she says she feels a lot better and would like to go home.

Background: Heart failure is a serious condition that often presents to the emergency department. Guidelines exist to help physicians on the diagnosis and treatment of heart failure but none offer recommendations on who to admit (Yancy et al, Arnold et al, McMurray et al, and McKelvic et al).

Heart failure patients often have other co-morbid conditions increasing their rate of hospitalization (Blecker et al).  In addition, patients hospitalized for heart failure have a high risk of readmission to hospital after discharge (McAlister et al, Richter et al, and Yeung et al).

Patients in Canada are more often treated in the emergency department and discharged home compared to the U.S. (Stiell et al, Pang et al, Collins et al, and Schrader et al). This difference in admission rates is similar to what is seen in patients diagnosed with pulmonary embolism. We have discussed this issue on SGEM#51.

So who can go home and who needs to come in to hospital is a question faced regularly by emergency physicians. There are published risk-stratification tools to predict mortality in patients with acute heart failure but they are limited in their ability to inform the emergency physician in deciding disposition (admit or discharge home). See reference #19-28 in Dr. Stiell’s Hot Off the Press publication in AEM for these risk-stratification tools.

Clinical Question: Can the Ottawa Heart Failure Risk Scale (OHFRS) help make disposition decisions by accurately predicting the 30-day rate of serious adverse events in patients with acute heart failure?

Reference: Stiell et al. Prospective and Explicit Clinical Validation of the Ottawa Heart Failure Risk Scale, With and Without Use of Quantitative NT-proBNP. AEM March 2017

  • Population: Adult patients (over 49 years of age) presenting to the emergency department with shortness of breath (less than 7-days duration) due to acute heart failure (defined by the Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure of the European Society of Cardiology 2008 – Dickstein et al)
    • Excluded: Patients who did not fit definition of acute heart failure or were too sick to be considered for discharge after 2-12 hours of emergency department management (Resting oxygen saturation < 85% on room air or after being on home oxygen level for 20 minutes; 2. Heart rate greater than or equal to 120 bpm; 3. Systolic blood pressure < 85 mm Hg; 4. Confusion, disorientation, dementia; 5. Primary presentation is for ischemic chest pain requiring treatment or with acute ischemic ST-T changes on initial ECG; 6. ST-elevation MI on initial ECG; 7. Terminal status – death expected within weeks from chronic illness; 8. From nursing home or chronic care facility (not seniors residence); 9. Enrolled in previous 2 months; or, 10. On chronic hemodialysis.)
  • Intervention: The Ottawa Heart Failure Risk Scale (OHFRS).


  • Comparison: None
  • Outcome:
    • Primary Outcome: Serious Adverse Events (SAE)
      • Death from any cause by 30 days and any of the following within 14 days: Admission to a monitored unit, endotracheal intubation or non-invasive ventilation, myocardial infarction, major procedure, or relapse and hospital admission for those patients originally discharged home.
    • Secondary Outcomes:
      • Performance of the predictor variables, performance of the OHFRS with and without NT-proBNP and physician accuracy and acceptability 

Authors Conclusions: Prospective clinical validation found the OHFRS tool to be highly sensitive for SAEs in acute heart failure patients, albeit with an increase in admission rates. When available, NT-proBNP values further improve sensitivity. With adequate physician training, OHFRS should help improve and standardize admission practices, diminishing both unnecessary admissions for low-risk patients an unsafe discharge decisions for high-risk patients.

checklistQuality Checklist for Clinical Decision Tools:

  1. The study population included or focused on those in the ED. Yes
  2. The patients were representative of those with the problem. Unsure
  3. All important predictor variables and outcomes were explicitly specified. Yes
  4. This is a prospective, multicenter study including a broad spectrum of patients and clinicians (level II). Yes
  5. Clinicians interpret individual predictor variables and score the clinical decision rule reliably and accurately. No
  6. This is an impact analysis of a previously validated CDR (level I). No
  7. For Level I studies, impact on clinician behavior and patient-centric outcomes is reported. Not applicable
  8. The follow-up was sufficiently long and complete. Yes
  9. The effect was large enough and precise enough to be clinically significant. Unsure

Key Results: The study enrolled 1,100 patients with a mean age of 78 years, 53% being male and 44% arriving by ambulance. Of the study population, 43% were discharged home from the emergency department and 57% were admitted to hospital at the index emergency department encounter.

It is important to note that NT-proBNP, which is part of the OHFRS was measured in only 62.2% (684/1,1000) of the patients.

Primary Outcome: Serious adverse event rate was 15.5% (19.4% for patients admitted and 10.2% for those discharged from the emergency department).

Overall morality was 3.7%. There were numerous other serious adverse events (SAE) that we would consider important when considering a potential discharge, such as need for non-invasive ventilation, intubations, and myocardial infarctions. These can all be found in Table 3 of the paper.

  • Secondary Outcomes: When you look at each of the individual components of the score, most seem to perform well in predicting adverse events (see Table 3 ).
    • Performance of the OHFRS without NT-proBNP: A score of ≥ 1 was 91.8% sensitive for SAEs (as compared to emergency department physician decision which was 71.8% sensitive). However, this would have raised the admission rate from 57.6% to 77.6%.
    • Performance of the OHFRS with NT-proBNP: A score of ≥ 1 was 95.8% sensitive for SAEs (as compared to emergency department physician decision which was 69.8% sensitive). However, this would have raised the admission rate from 60.8% to 88.0%.
    • Physician Accuracy: Physicians were asked to place the patients into four different risk categories (low, medium, high or very high). This was compared to criterion interpretation (see table above) and resulted in agreement of 59%.
    • Physician Acceptability: Physicians rated on a five-point scale whether or not they were comfortable using the OHFRS (very comfortable to very uncomfortable). Physicians reported being uncomfortable or very uncomfortable in using OHFRS 12% of the time.

Because these are decision tools and not rules, we might be able to use different cut-offs to guide shared decision making. Using a score of ≥ 2 had a similar sensitivity for SAEs when compared to physician judgement (71.2% vs 71.8%), but would have decreased admission rates (57.2 vs 48.3%)

Screen Shot 2015-04-25 at 3.11.12 PM

Dr. Ian Stiell

Dr. Ian Stiell

Dr. Ian Stiell is Professor, Department of Emergency Medicine, University of Ottawa; Distinguished Professor and Clinical Research Chair, University of Ottawa; Senior Scientist, Ottawa Hospital Research Institute; and Emergency Physician, The Ottawa Hospital. He is internationally recognized for his research in emergency medicine with a focus on the development of clinical decision rules and the conduct of clinical trials involving acutely ill and injured patients. Dr. Stiell is best known for the development of the Ottawa Ankle Rules and Canadian C-Spine Rule, and as the Principal Investigator for the landmark OPALS Studies for prehospital care.

Listen to the SGEM Podcast on iTunes to hear Dr. Stiell’s answers to our nerdy questions.

  1. Eligibility: Many patients were screened but only 37% (1,869/4,999) ended up being eligible for the study. Let’s look at the top five of reasons why patients were not eligible:
    • Shortness of Breath for Greater than Seven Days (788) Many patients have chronic shortness of breath with heart failure who decompensate and present to the emergency department. Why did you exclude these patients?
    • No Clear Heart Failure on Chest X-Ray (394) – We know that chest X-ray lacks sensitivity for congestive heart failure so why not include patients that physicians felt were in acute heart failure regardless of the chest X-ray?
    • Off Study Hours (387). Patients were not enrolled consecutively. Only patients presenting between 8am and midnight were included. Congestive heart failure patients presenting at night might be different from those presenting during daytime hours and night time discharges could be higher risk than those during the day.
    • Patients from Nursing Home or Long Term Care Facility (376). Why did you specifically exclude nursing home or long-term care facility patients? These represent a significant proportion of heart failure patients we see and would like to be able to send back to their “home”. Do you think the OHFRS could be applied to these patients?
    • Patients with Confusion, Disorientation or Dementia (276). Did they use a validated score to assess confusion, disorientation or dementia or was it based on clinical gestalt?
  2. Enrolment: In addition, 41% (769/1,869) patients were not enrolled if clinical staff were too busy with other patient care responsibilities. Does this mean the OHFRS would be too time consuming to adopt into clinical practice a large part of the time?
  3. Scoring: Physicians were aware of the patient’s risk category on the OHFRS when they were making their disposition decisions. Although they were explicitly told not to base their decisions solely on this instrument, there is a risk of incorporation bias here.
    • Physicians were not very accurate when using this score. Furthermore, more than 10% of physicians indicated they were uncomfortable using this score. How will that affect its generalizability and impact going forward?
    • The primary outcome was based on the criterion interpretation of the individual score components rather than the treating physician’s interpretation. Given the inaccuracy of the treating physicians in using this score, this would result in an over-estimation of the accuracy of the score and limit generalizability.
  4. Primary Outcomes: There were two primary outcomes (death at 30 days and serious adverse events by 14 days). Why not just have one primary outcome and the rest secondary?
    • The other serious adverse events are not all equal (admission to a monitored unit, endotracheal intubation or non-invasive ventilation, myocardial infarction, major procedure, or relapse and hospital admission for those patients originally discharged home). Let’s look at relapse with admission to hospital specifically because we are not sure classifying that as a SAE makes sense. Unless something else accompanied that admission, like an intubation or and myocardial infarction, which would have been caught in the other SAEs, the admission by itself is not a bad thing, because the only other option was admitting the patients to hospital on the index visit. Having a few patients come back is just good medicine. Admitting 100 patients to prevent 20 of them having to come back for an admission without any other complications is crazy.
    • Mentioned in the discussion that admission of the patient could be a confounder because intensive treatment could prevent an adverse event. The opposite could also be true, with admission causing adverse events, such as clots, stress GI bleeds, unnecessary testing, or even unnecessary interventions (such as revascularization) just because the patient is in hospital.
    • The decision being considered here is: to admit or not to admit. We think it is reasonable to ask whether admission would have prevented the adverse events that were seen here. If admission cannot prevent the adverse events, it is unclear how this scale will help us. We will probably only be able to see those patient oriented outcomes if this scale is prospectively tested in a randomized, controlled trial. While the OHFRS may improve sensitivity, this study does not allow us to see whether increasing admissions would decrease SAEs, and in a system that is already full, it is unclear what benefit exists to admitting more people.
  5. Follow-Up: Serious adverse events within 30 days is exactly what we care about when making disposition decisions. However, most of the follow-up data was from health records and telephone follow up. They say there follow up was very complete, but I don’t see any numbers for how often they relied on hospital records versus phone follow up, and even though the Canadian system is pretty good, we know lots of patients move between hospitals and even provinces and can therefore get lost.

Comment on Authors Conclusion Compared to SGEM Conclusion: We agree that the OHFRS has potential utility to standardize admission practices.

SGEM Bottom Line: The Ottawa Heart Failure Risk Scale (OHFRS) can probably help make disposition decisions by accurately predicting the 30-day rate of serious adverse events in patients with acute heart failure. However, we would love to see this scale validated in a RCT so that we can see patient oriented outcomes before it is used widely.

Case Resolution: You perform a walk test, and your patient remains asymptomatic with a heart rate below 100 and an oxygen saturation over 90% for 3 minutes. You determine that she is low risk, both by your clinical assessment and using the components of the Ottawa Heart Failure Risk Scale. Based on this assessment, you engage in a shared decision making conversation, and she decides she would like to be treated as an outpatient, but will follow up as soon as possible with her primary care physician. 

Dr. Justin Morgenstern

Dr. Justin Morgenstern

Clinically Application: This information can help in making shared decisions with patients presenting to the ED with acute heart failure.

What do I tell my patient? There is a new scoring system that can help us decide which patients with acute heart failure should be admitted to hospital. There is moderate evidence that it can predict who is at risk for serious adverse events. Do you want to go through the scale together and decide whether you should be treated in hospital or at home?

Keener Kontest: Last episode’s winner was Jarek Gucwa a repeat winner from Poland. He knew 1977 was when glucagon was first proposed as a pharmacologic way to treat esophageal foreign bodies.

Listen to the podcast for this weeks’ question. If you know the answer then send it to with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.

Other FOAMed Resources:

  • EM Cases – Episode 56 The Stiell Sessions: Clinical Decision Rules and Risk Scales

SGEMHOPNow it is your turn SGEMers. What do you think of this episode? Tweet your comments using #SGEMHOP. What questions do you have for Dr. Stiell and his team? Ask them on the SGEM blog. The best social media feedback will be published in AEM.

Dr. Corey Heitz

Dr. Corey Heitz

CME Credits: Don’t forget those of you who are subscribers to Academic Emergency Medicine can head over to the AEM home page to get CME credit for this podcast and article. Here is the process:

  • Go to the Wiley Health Learning website
  • Register and create a log in
  • Search for “Academic Emergency Medicine – March”
  • Complete the five questions and submit your answers
  • Please email Corey ( with any questions or difficulties.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.



EMA 2017 courses

  • Kirsty Challen
    • TheSGem

      Thanks Kirsty for summarizing another paper and SGEM episode into an infographic #PaperinaPic

    • Ian Stiell

      Great pic! Thanks Kristy. Ian

  • TheSGem

    A press release from AEM on the Stiell et al paper covered on this episode of #SGEMHOP

  • TheSGem

    Twitter poll has been posted on the OHFRS.

    Question: Will you use the Ottawa Heart Failure Risk Score for the disposition of acute heart failure patients?

    Vote @TheSGEM

  • James Berry

    I think this tool is useful if recognized. Sadly, it it insurance companies, ie. medicare/medicaid that is deciding patient dispositions currently – not doctors! I would use it.

  • James Berry

    I think this tool is useful if recognized. Sadly, it it insurance companies, ie. medicare/medicaid that is deciding patient dispositions currently – not doctors! That being said, I would use it.

    • Ian Stiell

      I suppose we are luckier in Canada that the docs make the decision without oversight by any insurance company/HMO etc. That said, we suffer from a lack of hospital beds. Ian Stiell

    • Corey Heitz

      I would argue that physicians in the US do make these decision every day, not the insurance companies. Does whether or not the patient meets admission criteria (i.e., will the admission be reimbursed) factor into that decision? Sure. But it’s fairly cynical to say the insurance companies are “making” the decision. I’ve never once called someone else to decide for me if I’m going to admit a patient.

      • James Berry

        In Michigan we currently have accountable care organization representatives calling us before work ups are completed, attempting to influence dispositions. Sometimes inappropriately so. I have spent a fair amount of time over the phone discussing and validating what I intend to do on a single admission with multiple providers who have not even seen the patient in front of me. Like I said it does matter who else besides physicians recognizes these greatly appreciated clinical decision rules.

  • Andrew Bohn

    I like the concept, but only one point for “too ill to walk?” That one seems to me (without actually doing the research) as if it should be weighted more heavily. That last point is like the MACE composite outcome, which drives me nuts with the broad spectrum of things that qualify as major adverse events (revascularization, new MI or death). HR of 110 after 3 minute walk doesn’t seem too bad, exertional desaturation is a bit more concerning and “too ill to walk” is pretty darn worrisome to me. Patients meeting that criterion will likely have other points as well, but the “can you walk/do you desat on your baseline oxygen” question is usually the most important one to me in determining the disposition of a moderately well appearing heart failure or COPD patient

    • Chip with TOTAL EM

      I have to agree with these points. It reminds me of the HEART score. Most patients with a bump in troponin or concerning EKG changes are probably not going home even if the score is low.

    • Ian Stiell

      Great point Andrew. All the points came from the data. We have analyzed 1,700 cases and the points were calculated based on complex stats analysis. Unable to start or complete the walk test is only worth 1 point on the overall scale. I agree that the walk test is important – does everyone do it for their HF patients?

  • Chip with TOTAL EM

    I enjoyed this podcast and the paper. Although we probably need a system like this more in the United States given the way we currently manage this patient population, I believe our medicolegal system and barriers such as major insurance would prevent this.

    As a PA who works hard to give my patients the best care, I really believe this can be beneficial and we as clinicians should not give up such a potentially important tool. I know the authors may feel it is ready for prime time, but I feel that there will be a knowledge translation gap given the lack of other studies currently.

    Keep in mind that many of these patients will probably be admitted to rural hospitals versus some other types of patients who are often transferred. Unfortunately, non-academic centers are traditionally delayed in cutting down the KT window.

    None of this means we should abandon or ignore this tool. Even if some places do not widely accept other great evidence based practices by Ian Steel such as the Ottawa Ankle or Knee Rules (I’m sorry Ken, tools), that does not mean they do not work.

    It may take time for my facilities to incorporate the evidence. But if I do have an appropriate patient for this tool and consult with the emergency physician or hospitalist on them you bet that this discussion will start to happen. I encourage anyone else in emergency medicine to do the same not only for this tool but any other you think will help patients.


    Congratulations to the Ottawa team for further validating this new CHF rule (and to the SGEM-HOP team for their excellent critical appraisal, as usual). The insurance and malpractice barriers noted in the U.S. by other bloggers are real, but not insurmountable and quite frankly the U.S. cannot actually afford to admit (and re-admit and re-admit) acute decompensated CHF patients, particularly if we can identify lower risk subsets for short-term adverse outcomes. In addition to efforts to externally validate the Ottawa CHF “tool”, I envision two research priorities that I’ve not seen suggested in the article or by other bloggers. First, there seems to be a natural fit for this risk stratification instrument to identify ED patients most amenable and likely to benefit to community paramedicine interventions for CHF Second, the hierarchy of diagnostic evidence holds that further validation attempts demonstrate patient-centric benefits in applying the Ottawa instrument. In other words, do patients at higher risk for adverse outcomes actually benefit from admission or does admission just mean that the adverse outcomes occur in the hospital rather than at home? I’m also hoping to see a head-to-head prospective comparison of this new instrument with others like the new Lee et al. instrument that the authors discuss. Excellent work! Chris Carpenter

    • Corey Heitz

      100% agree. The ultimate question here is as you stated – does admission avoid adverse events?

      • Ryan Radecki

        I’m on the same ultimate page with Corey and Chris – the foundational presumption admission alters the adverse event window limits the effectiveness of a tool like this as a disposition aid. There are some generalizability issues, as well, and I am always apprehensive about distilling a complex medicosocial chronic disease into a simple tool.

        Some of these basic issues are discussed in my brief blog post:

      • Ian Stiell

        Thanks for the great feedback. Creating new decision tools is a multi-step process and we have only completed 2 steps, derivation and validation. Next step will be implementation to see if patients are really better off when the OHFRS is incorporated into practice. Finally there is the KT phase of spreading the word.
        Where we are now is a scale that has been validated but not yet tested. Clinicians are free to use it now because there are no other clinically validated tools out there for AHF in the ED.
        We know that U.S. practice is hugely different than that in Canada with the majority of AHF patients being admitted. In Canada we send many patients home but that decision is often based on gestalt. Now we have provided a bit evidence to help understand the medical risk. Other factors obviously include the opportunity to optimize outpatient therapy, early follow-up, and strong home support.
        Stay tuned for more AHF evidence from Canada.

    • Ian Stiell

      Really excellent points about the potential role of this scale. Bottom line, which we will be testing in a cluster randomized trial, is are the patients better off?

  • Ken Milne
  • Dr. Bob

    I would use it as a way of supporting my decision in the EMR, the same way I use the HEART score when I think a patient can be safely discharged: “The patient has low-risk chest pain, with a HEART score of 2, and is comfortable being discharged to follow up with primary care.”

    • Corey Heitz

      I think that’s a great way to use it. Support for decisions, evidence for decisions. Not “how I made the decision” or a “rule”, but one piece of the puzzle. “This patient’s score is 1, which means the likelihood of an adverse outcome is very low”

      • Ian Stiell

        I completely agree that this is just one piece of the puzzle for safe disposition decisions. Still, this is one more piece than ED docs previously had.

  • Gerben Keijzers

    The developed tool is fairly sensitive for SAE, but with a score of 1 the SAE rate is still >5%– much higher than we would
    accept for the workup of PE or ACS. Did the authors consider presenting a Receiver Operater Curve?

    As outlined by other comments: Predicting SAE rate is something else than intervening and avoiding/preventing one. The fact that SAE rates are relatively high in discharged patients suggest that this maybe part of the natural course of the disease. What is an avoidable SAE would be a more appropriate question but probably difficult or even impossible to answer.

    The SAEs are really a composite endpoint – this is understandable in
    terms of creating a predictor model/Clinical Decision Rule – however patient may care less about re-admission than about death, ventilation or PCI. A sensitivity analysis for just Death, ICU admission or revascularisation would provide insight in the robustness of the rule.

    One final comment relates to the exclusion criteria – Over one-third of HF
    patients have AF (or a history of AF) – excluding a ventricular rate >120
    suggests excluding a decent proportion of these patients. As a result the
    scoring in the scale for a HR over 110 really means that one could only score a point for a HR between 111-120 – this seems unusual.

    Overall – This is a well-written descriptive cohort study validating a previously derived Clinical Decision rule . The rationale is cleary described, the methodology in terms of validation is sound and the results are clearly described. The discussion provides food for thought and this body of work certainly adds to the exisiting literature and as shown in SGEM and other blogposts has opened a healthy and interesting discussion

    • Ian Stiell

      Great comments. very much appreciated!
      AHF patients are sick people, hence the baseline SAE rate of 5%.
      I like your idea of a sensitivity analysis for a more restricted definition of SAE. Stay tuned.

      We excluded the really ill patients because we don’t need a risk tool to know they need admission. They are not the ones we have trouble deciding about.

  • Ken Milne
  • Pingback: SGEM #170: Spanish – La Escala de Riesgo de Ottawa para Insuficiencia Cardiaca | The SGEM Global()