Podcast: Play in new window | Download
Subscribe: RSS
[display_podcast]
Date: December 19th, 2017
Reference: Perry J et al. Validation of the Ottawa Subarachnoid Hemorrhage Rule in patients with acute headache. CMAJ Nov 2017
Guest Skeptic: Dr. Chris Carpenter is from Washington University, Deputy Editor of Academic Emergency Medicine and faculty member of Emergency Medicine and Critical Care course.
Case: A 35-year-old female presents to your emergency department three-hours after the onset of a severe frontal headache. She describes the headache as throbbing, left retro-orbital, and associated with nausea but no vomiting. As a teenager, she had a history of frequent migraine headaches, but she cannot recall any migraine for at least ten years. The headache developed over an hour while at her desk job and was not associated with loss of consciousness, neck pain, fevers, or neurological deficits. Your physical exam is normal. Last week, your colleague missed a subarachnoid hemorrhage that was diagnosed as a ruptured cerebral aneurysm on the third emergency department visit, so you have a low cognitive threshold to initiate extensive diagnostic evaluations for any and all severe headache patients.
Background: Headaches represent about 2% of emergency department visits annually. The severe headache patient presenting with altered mental status, fever, or preceding head trauma usually generate sufficient concern to justify further diagnostic evaluations; Most other sudden onset headache cases ultimately prove to result from benign, non-life-threatening causes like migraine headache.
Causes of sudden onset headaches include cough, exertion, and post-coital, but can also include potentially life-threatening conditions like sinus thrombosis, vascular dissection, intracerebral hemorrhage, vasospasm, and aneurysmal subarachnoid hemorrhage. [Landtblom 2002, Delasobera 2012, de Bruijn 1996, Pascual 1996, Dodick 1999]
Migraine headaches are at least 50-times more common than SAH amongst emergency department headache patients, so SAH represents a needle in a haystack for a very common chief complaint. [Edlow 2003]
Missed SAH diagnosis occurs between 12%-53% of cases with emergency department providers estimated to miss 5% of them. [Edlow 2000, Vermeulen 2007] Unfortunately, one-fourth of aneurysmal SAH victims die within one-day and 50% of SAH survivors never return to work.
Identifying SAH early reduces these adverse outcomes if subsequent neurosurgical interventions (coiling or clipping) occur emergently. [Schievink 1997] Therefore, the possibility of aneurysmal SAH must be considered in emergency department patients presenting with severe headaches, but in an era of Choosing Wisely and Preventing Overdiagnosis avoiding advanced imaging while simultaneously reducing missed SAH is a tremendous challenge.
Computed tomography (CT) has become increasingly available for the evaluation of headache patients over the last 30 years ago. Early CTs were 4-slice and radiologists’ interpretative learning curves were steep.
These early CTs were imperfect (sensitivity ~90%) for identifying small amounts of blood in the subarachnoid space, so textbooks, guidelines, and several generations of emergency medicine trainees advised against a CT-only approach to rule-out aneurysmal SAH.
Instead, SAH could only be ruled out when a negative CT was followed immediately by a LP demonstrating cerebrospinal fluid (CSF) without either red blood cells or (at least 12 hours post-headache onset) xanthochromia.
However, in 2017 there are three problems with that approach:
- Contemporary CTs are far better at identifying blood in the subarachnoid space
- Lumbar pucture frequently identify blood that is not in the subarachnoid space (traumatic LPs)
- CSF xanthochromia is not an accurate diagnostic test for SAH
In fact, test-treatment studies based on conservative estimates of CT and liberal estimates of CSF diagnostic accuracy indicate that the vast majority of headache patients will not benefit from post-CT LP with a Number Needed to Tap (NNTap) up to 15,000 after an unremarkable CT.
Understanding the SAH diagnostic evidence available for bedside evaluation, advanced imaging, and lumbar puncture (LP) is therefore essential – and the landscape is shifting.
The topic of SAH has been covered three times on the SGEM including:
- SGEM#48: Thunderstruck – Subarachnoid Hemorrhage (with lead author Jeff Perry)
- Bottom Line: Ottawa SAH Tool is not ready for prime time to rule out low risk patients from investigations.
- SGEM#134: Listen, to what the British Doctors Say about LPs post CT for SAH
- Bottom Line: In this study, one patient would be diagnosed with SAH out of every 250 patients receiving a LP who presented to the emergency department with a headache that did not have their bleed identified on CT scan. So the NNTap 250.
- SGEM#140: CT Scans to Rule Out Subarachnoid Hemorrhages in A Non-Academic Setting
- Bottom Line: These community radiologists were just as good at reading CT heads as academic radiologists when looking for blood using a third-generation scanner. The NNTap was 760 but the one patient they did identify as a perimesencephalic non-aneurysmal SAH with a benign clinical course.
Clinical Question: Can the Ottawa SAH Rule reduce the number of acute headache patients that require further diagnostic evaluation for subarachnoid hemorrhage without increasing missed cases of SAH?
Reference: Perry J et al. Validation of the Ottawa Subarachnoid Hemorrhage Rule in patients with acute headache. CMAJ Nov 2017
- Population: Adults with acute or subacute headaches in the emergency department with clinical concern for SAH,
- Exclusions: Glasgow Coma Scale <15, head trauma within seven days, headache onset >14 days prior, similar headaches on ≥3 occasions over preceding six months, referral from another hospital with diagnosed SAH, papilledema, new focal neurological deficit, or previous cerebral aneurysm, SAH, brain neoplasm, ventricular shunt, or hydrocephalus.
- Intervention: Prospective, observational study with 6-month longitudinal follow-up (ie no intervention).
- Comparison: No comparator
- Outcome: SAH was defined by either (1) subarachnoid blood visible on unenhanced CT (based on final radiology report) or (2) visible inspection cerebrospinal fluid (CSF) xanthochromia or the presence of >1 x 106 red blood cells in the final tube of CSF with an aneurysm or arteriovenous malformation on cerebral angiography.
- This SAH definition was agreed upon by five emergency physicians and a neurosurgeon and was previously used in the derivation and validation of the Ottawa SAH Rule. Telephone follow-up for the subset of discharged patients without both a CT and a normal lumbar puncture occurred at 1- and 6-months after the initial emergency department visit by a surrogate method of identifying missed SAH that the authors previously used.
Ottawa SAH Rule
Dr. Jeff Perry
Dr. Jeff Perry is a Senior Scientist, Clinical Epidemiology, Ottawa Hospital Research Institute. Associate Professor, Department of Emergency Medicine, Faculty of Medicine, University of Ottawa.
Authors’ Conclusions: “We found that the Ottawa SAH Rule was sensitive for identifying subarachnoid hemorrhage in otherwise alert and neurologically intact patients. We believe that the Ottawa SAH Rule can be used to rule out this serious diagnosis, thereby decreasing the number of cases missed while constraining rates of neuroimaging.”
Quality Checklist for Clinical Decision Tools:
- The study population included or focused on those in the emergency department. Yes
- The patients were representative of those with the problem. Yes
- All important predictor variables and outcomes were explicitly specified. Yes
- This is a prospective, multicenter study including a broad spectrum of patients and clinicians (level II). Unsure
- Clinicians interpret individual predictor variables and score the clinical decision rule reliably and accurately. Yes
- This is an impact analysis of a previously validated CDR (level I). No
- For Level I studies, impact on clinician behavior and patient-centric outcomes is reported. No
- The follow-up was sufficiently long and complete. Yes/No
- The effect was large enough and precise enough to be clinically significant. Yes
Key Results: They enrolled 1,153 patients prospectively of whom 67 (5.8%) had confirmed SAH. An additional 590 potentially eligible patients were not enrolled among whom 33 (5.6%) had confirmed SAH.
The Ottawa SAH Rule was 100% sensitive (missed no SAH patients) and 13.6% specific.
- Primary Outcome:
- 100% sensitive (95% CI 94.6%-100%)
- 13.6% specific (95% CI 13.1%-15.8%)
- Positive likelihood 1.16 (95% CI 1.13-1.19)
- Negative likelihood ratio 0.0 (95% CI 0-0.5).
A pretest probability for SAH of 5.8% with a negative likelihood ratio of 0 would yield a post-test probability of 0%. However, the study was underpowered with an upper confidence interval limit of 0.5, which would bring the post-test probability to 3%. The true post-test probability for patients without any of the Ottawa SAH Rule features (i.e. low risk) is somewhere between 0% and 3% (and probably much closer to 0%).
Physicians were only uncomfortable or very uncomfortable using the rule in 9% of patients and demonstrated excellent agreement between the bedside physician and the coordinating center research nurse (κ = 0.82, 95% CI 0.76-0.87).
In 5.1% of cases, physicians interpreted the Ottawa SAH Rule as negative while the coordinating center labeled it as positive based upon the documentation.
In actuality, 88% of patients had CT imaging and another 1% had lumbar puncture without a CT. The authors hypothesize that if the rule had been used to guide CT or lumbar puncture decisions, only 84.3% (4.7% fewer patients) would have undergone additional testing (CT or LP).
Listen to the podcast on iTunes to hear Jeff’s responses to our five nerdy questions.
1. External Validity and Reproducibility: The six enrolling sites were the same hospitals (and likely most of the same physicians) engaged in the original Ottawa SAH Rule derivation studies. Whether physicians from different geographic regions and non-academic hospitals would accept, interpret, and use this decision aid on headache patients remains unproven. Emergency providers in some countries are less likely to accept clinical decision aids or apply them indiscriminately to populations in whom the rule was never validated.
2. Precision: The authors planned to enroll about 1,200 patients in order to identify approximately 75 SAH cases to achieve near 100% sensitivity with acceptably tight confidence intervals, but were only able to enroll 1,153 patients with 67 SAH cases. The study is therefore underpowered and the confidence intervals around the LR- of 0 extend to 0.5. Did the authors conduct a sensitivity analysis to test the impact on sensitivity and LR- of 1 or more false negative cases of SAH being identified if 75 or more patients had been enrolled?
3. Differential Verification Bias and Loss to Follow-Up: A certain skeptic and Legend of Emergency Medicine was critical of the initial derivation and validation studies based on two aspects of the study design.
- First, not every enrolled patient had both CT and post-CT LP (if the imaging was non-diagnostic for SAH). The IRBs would not permit this ideal-world approach due to the personal risks of these procedures in whom the physician did not feel they were medically indicated or the patient refused, as well as the costs to the healthcare system. When different criterion standards are applied to patients, differential verification bias (also known as double gold standard bias) is likely and observed estimates of sensitivity and specificity can be falsely lowered for disease processes (like SAH) that only become detectable during follow-up.
- Second, 8/1,153 (0.6%) were lost to follow-up. How would a best-case scenario (none of the 8 had a SAH – by far the most likely scenario) compare with a worst-case scenario (all 8 of the lost patients had a SAH and were somehow not found) affect the observed sensitivity of the Ottawa SAH Rule?
4. Shared Decision Making: Debate persists around when clinical decision aids have sufficient evidence to justify widespread uptake. Even if a decision aid demonstrates sufficient accuracy and reliability in disparate settings, there is no guarantee that providers will embrace the new clinical tools. One unmeasured obstacle to more efficient uptake of decision aids is the engagement of patients and families in the complex discussions around diagnostic evaluations. The science of developing patient decision aids is distinct from that of validating clinical decision rules and requires both a rigorous implementation strategy and sustainable funding environments. Is the Ottawa SAH Rule validated with sufficient confidence to use it as the basis for designing a patient decision aid?
5. Impact Analysis: Diagnostic and prognostic tests are usually disseminated on the assumption that accuracy equates to patient-centric benefit, but without actual studies to quantify (or even confirm) such benefit. In an ideal world, patients would present to the emergency department with “I have SAH” tattooed on their forehead, negating the need for a decision aid. In the real world, patients present with constellations of symptoms that clinicians need to identify, interpret, and transform into diagnoses that inform treatment options, urgency of further evaluation/consultation, and truly informed shared decision-making with patients.
- The process of transforming signs/symptoms into clinical action occur in a fishbowl of chaos in which providers are judged by various metrics including emergency department flow, patient satisfaction, and test-ordering rates relative to a standard imposed by someone else. Deciding whether or not to order diagnostic tests must be balanced against patient anxiety and preferences as well as each physician’s risk tolerance, while also considering transitions of care and the unknown risk tolerance of other emergency department providers to whom we may sign out or inpatient/outpatient providers.
- Therefore, evaluating whether the Ottawa SAH Rule is acceptable to providers (physician, nurse practitioners, physician assistants) in non-academic settings that are less aware and perhaps less accepting of decision aids is essential. So, we are going to ask you a series of five questions about impact analysis:
- Do you think these practitioners will believe this validation study or do you think they would need more validations studies before adopting this clinical decision instrument
- Will they apply the rule to headache patients similar to those enrolled in this study and remember all of the exclusion criteria?
- Do you think practitioners will explain the decision-aid and what it does/does not tell us to patients and families?
- Your study only showed a decrease of imaging by about 5%. If used and interpreted appropriately in disparate settings, do you think the Ottawa SAH Rule would increase or decrease downstream CT and LP testing?
- What implementation strategy does the Ottawa group have for this tool and how would they recommend implementation scientists around the world introduce and study decision aids like the Ottawa SAH Rule using StaRI reporting standards?
- In America they always need to think about payment. So, do you think payers will use the Ottawa SAH Rule as a penalty to deny payments for clinicians who deviate from the rule and order additional tests based on their gestalt?
Comment on Authors’ Conclusions: We generally agree with the authors’ conclusions except that we are not sure if the Ottawa SAH Rule will decrease the number of cases missed while constraining rates of neuroimaging if used in other clinical settings. Whether similar physician acceptability, reliability, and diagnostic accuracy will be observed in non-academic, non-Canadian emergency departments remains to be evaluated.
SGEM Bottom Line: The Ottawa SAH Rule needs external validation, a meaningful impact analysis performed and patient acceptability of incorporating this rule into a shared decision-making instrument before being widely adopted.
Case Resolution: The patient is non-high risk by the Ottawa SAH Rule. After her headache resolves with intravenous prochlorperazine, you explain the differential diagnosis of headache, including SAH. Your clinical gestalt is that the headache characteristics are atypical for SAH and more consistent with migraine. She expresses concern about cerebral aneurysms that she learned about from Dr. Google while awaiting headache resolution, so you explain the Ottawa SAH Rule and the fact that she has none of the high-risk criteria and her risk of SAH would be far less than 1%.
Understanding that your gestalt aligns with the new Ottawa CDR relieves the patient’s anxiety and she is discharged home with migraine abortive therapy in case the headache recurs. Two days later your nurses conduct routine follow-up calls and report that she has had no headache recurrence and has an appointment with her primary care provider later that day to evaluate the indications for further evaluation or specialty referrals.
Dr. Chris Carpenter
Clinical Application: It’s in the way that you use it! Decision aids are not meant to be used in cases where physicians are certain of the appropriate diagnosis or next steps. Instead, they’re meant to guide us in the gray zone cases where various factors lead to significant practice variability. If physicians use the rule indiscriminately on patients who they never would have obtained CT before, perhaps it will increase CT ordering. Additional research will tell, but I think clinicians just need to use the rule for those patients in whom they have lingering uncertainty.
What Do I Tell My Patient? The bad news it looks like you are having a migraine headache. The good news is we can treat it. There is other good news too. We considered other serious causes of your headache besides migraine. It looks like you do not have one of those other bad headaches. One thing we always worried about is could a bleed in the brain be causing the headache. This bleeding is called a subarachnoid hemorrhage. There is a check list six things to help us decide if a subarachnoid hemorrhage is likely. You have none of the six things. This means you have a 97-99% chance the headache is not a brain bleed. Now we could do a CT scan and lumbar puncture (spinal tap) to bring that number closer to 100%. However, these tests take time, cost money and have risks. The CT involves radiation that has a small chance of causing brain cancer. The spinal tap can cause infection, bleeding, damage the spinal cord or even make the headache come back. What are your thoughts about next steps?
Keener Kontest: Last weeks’ winner was Garreth Debiegun. Garreth is a Assistant Clinical Professor at Maine Medical Center & Tufts University School of Medicine. He knew the four humours in Latin were sanguis (blood), phlegma (phlegm), cholericus (yellow bile) and atra bilis (black bile).
Listen to the podcast on iTunes for this weeks’ keener question. If you think you know the answer then send an email to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.
Other FOAMed:
- Washington University Journal Club: January 2017 – Diagnosis of Subarachnoid Hemorrhage
- St. Emlyn’s: Journal Club – Ottawa SAH Rule
- Emergency Medicine Literature of Note: It’s SAH Silly Season Again!
You must be logged in to post a comment.