Podcast: Play in new window | Download

Subscribe: RSS

[display_podcast]

Date: December 9th, 2015

Guest Skeptics: Dr. Fareen Zaver is a Chief Resident in Emergency Medicine at the George Washington University, specifically in charge of the medical education resident curriculum and Grand Rounds schedule. She is the co-founder and lead editor of the ALiEM AIR-Pro series and is pursuing an academic career in Medical Education. She was born and raised in Calgary, Alberta and has constantly stood out with her Canadian accent and roots. She will always be a diehard Calgary Flames fan and is joining the University of Calgary Emergency Medicine department in August when she completes her residency.

Dr. Stefanie Gilbert is a Chief Resident in Emergency Medicine at The George Washington University in the heart of the nation’s capital. Her career interests include ED administration with a focus on operations and quality improvement. Prior to residency, she completed a Bachelor of Science degree in Kinesiology at The University of Michigan in Ann Arbor, MI followed by medical school at Wayne State University in downtown Detroit. She is proud of her Michigan roots and is a true wolverine at heart.

SGEM Journal Club Rules:SGEM Journal Club

  1. You must talk/tweet about SGEM-JC
  2. The answer to any SGEM-JC question can be –“it all depends”
  3. Don’t Panic – even your faculty is not sure of some of the answers
  4. It’s all about the methods!
  5. Be skeptical of anything you learn, even if you heard it on SGEM-JC

Case: Ms. Jones is a 45-year-old woman who presents to her local community hospital with the “worst headache” of her life that started suddenly four hours ago. She has a normal neurologic exam. She gets a non-contrast head CT that is read as normal by your local radiologist.

Background: We have talked about SAH a couple of times on the SGEM with the most recent time being the Hot Off the Press paper by Sayer et al in AEM. That retrospective UK study reported a NNTap (Number Needed to Tap) to diagnose one aneurysm not picked up on CT scan was 250.

There are a couple of other studies suggesting that a LP was not automatically needed to exclude the diagnosis of a SAH in patients presenting with an acute headache as the negative predictive value of a normal head CT performed at an academic, tertiary care hospital within 6 hours of headache onset is 100% (Perry et al BMJ 2011 and Backes et al Stroke 2012).

Ottawa SAH ruleDr. Jeff Perry’s group came up with the Ottawa SAH Rule that we covered on SGEM#48. The bottom line from that review was the “tool” was not ready for prime time because of the need for validation studies.

There has been at least one validation study done by Bellolio et al of the Ottawa SAH Tool showing 100% sensitivity and 7.6% specificity. They concluded the low specificity and its applicability to only a minority of emergency department patients limited the potential usefulness of the Ottawa SAH Rule. This study was done at an academic center. There have apparently been no studies done at non-academic sites until now.

Clinical Question: Can a SAH be ruled out in a patient presenting to a non-academic ED within 6hrs of headache onset by a head CT read by a community staff radiologist?

Reference: CT within 6 hours of headache onset to rule out subarachnoid hemorrhage in non-academic hospitals. Blok et. al. Neurology. Mar 2015.

  • Population: Adult patients presenting to a non-academic emergency department with spontaneous acute headache suspected for subarachnoid hemorrhage, negative CT and a lumbar puncture performed.
    • Inclusion: Normal level of consciousness without focal deficits, head CT within six hours after headache onset and reported negative and lumbar puncture performed greater than 12 hours after headache onset.
    • Excluded: Glasgow Coma Scale <15 at presentation, unknown time of ictus, age 16 years or younger and lumbar puncture performed earlier than 12 hours after headache onset.
  • Intervention: Lumbar puncture to achieve identification of CT-negative SAH patients.
  • Comparison: None
  • Outcome: Negative predictive value for detection of subarachnoid blood by staff radiologists working in a non-academic hospital

Author’s Conclusions: “Our results support a change of practice wherein a lumbar puncture can be withheld in patients with a head CT scan performed <6 hours after headache onset and reported negative for the presence of SAH by a staff radiologist in the described nonacademic setting.”

checklistQuality Checklist for A Diagnostic Study:

  1. The clinical problem is well defined. Yes. Headache represents 1-2% of all ED visits and SAH is one of the most devastating etiologies of headache, yet considerable controversy exists about whether an LP is essential to rule-out the diagnosis after a “negative” CT using contemporary high-resolution scanners.
  2. The study population represents the target population that would normally be tested for the condition (ie no spectrum bias). Yes. Patients with concerning headaches will present to non-academic emergency departments.
  3. The study population included or focused on those in the ED. Yes.
  4. The study patients were recruited consecutively (ie no selection bias). No. Retrospective chart review.
  5. The diagnostic evaluation was sufficiently comprehensive and applied equally to all patients (ie no evidence of verification bias). No. Although every patient had a CT and LP only the positive LP patients got the gold standard (CTA/MRA/DSA or 3-month follow-up).
  6. All diagnostic criteria were explicit, valid and reproducible (ie no incorporation bias). No.
  7. The reference standard was appropriate (ie no imperfect gold-standard bias). Unsure. They had a gold standard for CT interpretation. This was the academic physicians’ reading of the head CT for blood. However, there apparently is no internationally accepted gold standard for the interpretation of CSF spectrophotometry.
  8. All undiagnosed patients underwent sufficiently long and comprehensive follow-up (ie no double gold-standard bias). No. Differential verification bias or double gold standard occurs when the test results influence the choice of the reference standard. So a positive index test get an immediate/gold standard test. In this case if the LP was positive most patients went on to get further neuroimaging (although not all did). In contrast, the patients with a negative index test (LP negative) got clinical follow-up for disease. This can impact sensitivity and specificity. The authors admit the follow-up may have been incomplete and patients could have presented to another hospital with a SAH. For more information read Understanding the Direction of Bias in Studies of Diagnostic Test Accuracy (Kohn et al 2013).
  9. The likelihood ratio(s) of the test(s) in question is presented or can be calculated from the information provided. No. The authors do not report 2×2 tables and they cannot be reconstructed from the data presented since they only report CT- patients.  In order to reconstruct 2×2 tables, they would need to report CT+ and CT- patients.  Therefore, sensitivity, specificity, and LR’s cannot be computed.
  10. The precision of the measure of diagnostic performance is satisfactory. Yes. They provided 95% confidence intervals that were narrow.

Key Results: There were 760 consecutive patients who presented to one of eleven non-academic hospitals with acute headache suspected for SAH who had a head CT within six hours after headache onset reported as negative for the presence of blood by the non-academic staff radiologist and had a lumbar puncture greater than 12 hours after onset of acute headache.

The patient cohort had a median age of 45 years and 61% were women.

Negative Predictive Value 99.9% (95% CI 99.3%–100.0%)

Results community SAHLumbar punctures were positive for the presence of bilirubin in 52 patients. Independent review of the community radiologists by the academics found only one patient with a perimesencephalic nonaneurysmal SAH with a benign clinical course.

Of the 51 patients with negative CTs and positive LPs 28 went on to have CTA, MRA or DSA. Eight aneurysms were identified but felt that rupture was unlikely.

Twenty patients had no aneurysm identified on further imaging and twenty-three patients did not have any other imaging on clinical grounds. None were thought to have subarachnoid hemorrhage based on a median follow-up time of 53 months.


Screen Shot 2015-04-25 at 3.11.12 PM

  1. Methods:
    • pyramidRetrospective Chart Review: There is a hierarchy of evidence and a retrospective chart review is not a very high level of evidence. This does not make the conclusions wrong but weakens the strength of any conclusions that can be made from this type of study. Just because a retrospective chart review is a lower form of evidence does not mean it should not have strong methods. There are published quality checklists for retrospective chart reviews to assist researchers (Gilbert et al and Worster et al). The chart review methods were not described well in this study. There are many questions about who abstracted the data, were they blinding to the hypothesis, what was their training and how was quality of their abstraction assessed. This information would have been helpful.
    • STARD: This stands for Standards for Reporting Diagnostic accuracy studies. It is a checklist to help readers judge the potential for bias in a diagnostic study. The latest list contains 30 quality checks for the completeness and transparency of reporting diagnostic studies. There is no mention of following these guidelines. Lack of adherence to the STARD reporting standards makes it difficult to interpret the results.
  2. Relationship between disease prevalence and predictive value in a test with 95% sensitivity and 85% specificity. (From Mausner JS, Kramer S: Mausner and Bahn Epidemiology: An Introductory Text. Philadelphia, WB Saunders, 1985, p. 221.)

    Relationship between disease prevalence and predictive value in a test with 95% sensitivity and 85% specificity.
    (From Mausner JS, Kramer S: Mausner and Bahn Epidemiology: An Introductory Text. Philadelphia, WB Saunders, 1985, p. 221.)

    Negative Predictive Value (NPV): This is the proportion of people with a negative test who do not have the disease. The NPV is calculated by taking the true negatives and dividing them by the all negatives (true and false). This statistic depends on the prevalence of disease. Therefore the accuracy of a negative CT scan to rule out SAH cannot be interpreted without considering the per-test probability. This is getting into Bayesian thinking but is very important. A very sensitive test (even one which is very specific) will have a large number of false positives if the prevalence of disease is low. So if a CT scan is done in every headache patient (low pre-test probability for SAH) it will have a fantastic NPV. So if prevalence is low the number needed to scan will be very high. In fact, any test would look good to rule out disease if no one has the disease. All the positive tests (LP) would be false positives. The prevalence of SAH in this study was extremely low at 0.1%. This means the NPV will be close to 100% no matter what test is used.

  3. Lack of Gold Standard: Eight patients with intracranial aneurysms had negative CT heads and positive LPs. Seven were thought to be false positive based on further CSF testing. They do acknowledge the uncertainty of this interpretation because no internationally accepted gold standard exists for the interpretation of CSF spectrophotometry.
  4. Wrong Question: This does not answer the fundamental question we want answered. Do we need to do an LP to rule out SAH after a negative CT? This study demonstrates that non-academic radiologists are very good at identifying blood on a third generation CT scanner compared to academic radiologists. There was only one case out of 760 scans the academics felt did show blood that was not identified by the community radiologists. The blood was in the basal cistern and consistent with a non-aneurysmal perimescenphaic hemorrhage. The remaining 51 out 52 positive LPs were considered false positives.
  5. Harm: When considering a diagnostic test we also need to consider the harm. LPs are not benign procedures and can cause post-LP headaches, infections, nerve damage, and bleeding around your spinal cord. It is well recognized that harm is under reported in studies. The authors did not provide any information whether or not any of the 52 patients undergoing LP experienced an adverse events due to the diagnostic tests itself. However, they did mention in their discussion: “a lumbar puncture is associated with discomfort for the patient, costs, and may induce a potentially life-threatening complication such as subdural hematoma or cerebral venous sinus thrombosis in rare cases”. There are also the down stream consequences of false-positive CSF results. In this study 8 patients had aneurysms on subsequent vascular imaging. How will that knowledge affect these individuals and what impact will it have on their future healthcare?

Comment on author’s conclusion compared to SGEM Conclusion: We agree that an LP is not a useful test to diagnose SAH in a low prevalence population with a negative CT scan.

SGEM Bottom Line: These community radiologists were just as good at reading CT heads as academic radiologists when looking for blood using a third generation scanner

Case Resolution: You feel Ms. Jones has a low pre-test probability of having a SAH. Her head CT is negative and you discuss the risk and benefits of a lumbar puncture. A shared decision is made. She decides not to proceed with any further testing and will return if she develops focal neurologic symptoms, pain increases, loss of consciousness, seizure or is otherwise worried.

Clinically Application: A negative CT head scan read by a community radiologist using a third-generation scanner within 6 hours of headache onset is sufficient to exclude the diagnosis of SAH in a low prevalence population.

What do I tell my patient? Subarachnoid hemorrhages can present with sudden headaches. These types of bleeds in the brain can be devastating and even deadly. Our local radiologist looked at the CT scan of your brain and did not see any bleeding. They are just as good as radiologists at the big academic hospitals for reading these tests. CT scans are very good to rule out a bleed when done within six hours of headache onset but no test is 100%. A lumbar puncture could be done if we are really concerned. That involves sticking a needle in your back to get fluid from around your spinal column. This test can have complications. A common side effect of the test is a headache. In addition, often the test is falsely positive. That means doing even more testing that could potentially cause harm. There are studies suggesting hundreds or maybe even thousands of LPs would need to be done to find one of these serious/life-threatening bleeds not seen on CT scan. What do you want to do?

Keener Kontest: Last weeks’ winner was Greg Costello from the University of British Columbia. Greg knew three ways to reverse warfarin included Vitamin K, fresh frozen plasma, and prothrombin complex concentrate (PCC).

Listen to the podcast to hear Stas Haciski give this weeks’ question. Send your answer to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will win the cool skeptical prize.

If you want the SGEM to come to your program in the new year and do a live SGEM Journal Club then send me an email.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.