Podcast: Play in new window | Download

Subscribe: RSS

Date: November 30th, 2020

Reference: Ohle R et al. Can Emergency Physicians Accurately Rule Out a Central Cause of Vertigo Using the HINTS Examination? A Systematic Review and Meta-analysis. AEM 2020

Guest Skeptic: Dr. Mary McLean is an Assistant Program Director at St. John’s Riverside Hospital Emergency Medicine Residency in Yonkers, New York. She is the New York ACEP liaison for the Research and Education Committee and is a past ALL NYC EM Resident Education Fellow. 

Case: A 50-year-old female presents to your community emergency department in the middle of the night with new-onset constant but mild vertigo and nausea. She has nystagmus but no other physical exam findings. You try meclizine, ondansetron, valium, and fluids, and nothing helps. Her head CT is negative (taken 3 hours after symptom onset). You’re about to call in your MRI tech from home, but then you remember reading that the HINTS exam is more sensitive than early MRI for diagnosis of posterior stroke. You wonder, “Why can’t I just rule out stroke with the HINTS exam? How hard can it be?” You perform the HINTS exam and the results are reassuring, but the patient’s symptoms persist… 

Background: Up to 25% of patients presenting to the ED with acute vestibular syndrome (AVS) have a central cause of their vertigo – commonly posterior stroke. Posterior circulation strokes account for approximately up to 25% of all ischemic strokes [1]. MRI diffuse-weighted imagine (DWI) is only 77% sensitive for detecting posterior stroke when performed within 24h of symptom onset [2,3]. As an alternative diagnostic method, the HINTS exam was first established in 2009 to better differentiate central from peripheral causes of AVS [4].

But what is the HINTS exam? It’s a combination of three structured bedside assessments: the head impulse test of vestibulo-ocular reflex function, nystagmus characterization in various gaze positions, and the test of skew for ocular alignment. When used by neurologists and neuro-ophthalmologists with extensive training in these exam components, it has been found to be nearly 100% sensitive and over 90% specific for central causes of AVS [5-8].

Over the past decade, some emergency physicians have adopted this examination into their own bedside clinical assessment and documentation. We’ve used it to make decisions for our patients, particularly when MRI is not readily available. We’ve even used it to help decide whether or not to get a head CT.

But we’ve done this without the extensive training undergone by neurologists and neuro-ophthalmologists, and without any evidence that the HINTS exam is diagnostically accurate in the hands of emergency physicians.

Clinical Question: Can emergency physicians accurately rule out a central cause of vertigo using the HINTS examination?

Reference: Ohle R et al. Can Emergency Physicians Accurately Rule Out a Central Cause of Vertigo Using the HINTS Examination? A Systematic Review and Meta-analysis. AEM 2020

  • Population: Adult patients presenting to an ED with AVS
    • Exclusions: Non-peer-reviewed studies, unpublished data, retrospective studies, vertigo which stopped before or during workup, incomplete HINTS exam, or studies with data overlapping with another study used
  • Intervention: HINTS examination by emergency physician, neurologist, or neuro-ophthalmologist
  • Comparison: CT and/or MRI
  • Outcome: Diagnosis of HINTS examination for central cause for AVS (i.e., posterior stroke)

Authors’ Conclusions: “The HINTS examination, when used in isolation by emergency physicians, has not been shown to be sufficiently accurate to rule out a stroke in those presenting with AVS.”

Quality Checklist for Systematic Review Diagnostic Studies: 

  1. The diagnostic question is clinically relevant with an established criterion standard. Unsure
  2. The search for studies was detailed and exhaustive. Yes
  3. The methodological quality of primary studies were assessed for common forms of diagnostic research bias. Yes
  4. The assessment of studies were reproducible. Yes
  5. There was low heterogeneity for estimates of sensitivity or specificity. No
  6. The summary diagnostic accuracy is sufficiently precise to improve upon existing clinical decision-making models. Unsure

Key Results: They searched multiple electronic databases with no language or age restrictions and the gray literature. The authors identified 2,695 citations with five articles meeting inclusion criteria and a total of 617 patients.

There were no studies that included only emergency physicians performing the HINTS examination.

Essentially, the authors separated the studies into two cohorts according to the medical specialties of the HINTS examiners, and for each cohort they reported the sensitivity and specificity of the HINTS exam for diagnosis of posterior stroke.

The first cohort included neurologists and neuro-ophthalmologists. The sensitivity and specificity of the HINTS examination were 96.7% (95% CI; 93.1 to 98.5) and 94.8% (95% CI; 91 to 97.1).

In contrast, the second cohort (only one study) included emergency physicians and neurologists. The sensitivity and specificity were much lower at 83.3% (95% CI; 63.1 to 93.6) and 43.8% (95% CI; 36.7 to 51.2).

From these results, it was deduced that emergency physicians’ participation in the latter cohort resulted in the reduced diagnostic accuracy.

They did not combine the five studies into one summary result due to the heterogeneity of the included studies which was >40%.

 

1) Available Studies: Unfortunately, there were only five studies meeting the inclusion criteria, for a total of 617 patients. This is a known limitation of systematic reviews. Authors are limited by the available studies.

2) Biases: On the QUADAS-2 assessment, four of these studies had at least one component at high risk of bias, and three studies had unclear reports on at least one component, meaning that quality was low. Adherence to the STARD reporting guidelines was mediocre to poor overall because only two of the studies reported on most of the items in the guidelines. We will put the figure that represents the risk of bias of the included studies in the show notes.

The reference standard (index test) used in these studies for all patients recruited was CT or MRI. We know CT imaging has a low sensitivity for posterior. One of the studies allowed negative head CT alone as adequate imaging to rule out posterior stroke. With such low sensitivity of CT imaging for posterior strokes, this crucial diagnosis can be missed. Even with MRI-DWI, only has a reported sensitivity of 77%.

This problem in diagnostic testing studies is called the Imperfect Gold Standard Bias (Copper standard bias): It can happen if the “gold” standard is not that good of a test.

Another bias identified was partial verification bias (referral or workup bias). This happens when only a certain set of patients who were suspected of having the condition are verified by the reference standard (CT or MRI). So, the AVS patients with suspected strokes with a positive HINTS exam were more likely to get advanced neuroimaging than those with a negative HINTS exam. This would increase sensitivity but decrease specificity.

It is unknown if the original studies included consecutive patients or a convenience sample of patients. The later could introduce spectrum bias. Sensitivity can depend on the spectrum of disease, while specificity can depend on the spectrum of non-disease. Four out of the five studies had the ED physician identifying the patients for a referral. If patients with indeterminate or ambiguous presentations (rather than all patients presenting with AVS) were excluded this could falsely raise sensitivity.

Because there were few studies it made assessing publication bias difficult.

For those interested in understanding the direction of bias in studies of diagnostic test accuracy there is a fantastic article by Kohn et al AEM 2013. There is also a good book by Dr. Pines and colleagues on the topic 

3) Heterogeneity: The authors used the I2 statistic to represent heterogeneity. Our overall I2 values are 53% for sensitivity and 94% for specificity likely representing moderate and considerable heterogeneity, respectively. Notably, for neurologists and neuro-ophthalmologist cohort alone, the I2 was noted to be 0, representing low or negligible heterogeneity [9].

4) Precision and Reliability: There is poor precision overall – specifically for the cohort of emergency physicians with neurologists, the 95% confidence intervals were very wide for sensitivity (83%; 95% CI 63 to 94) and specificity (44%; 95% CI 37 to 51).

The HINTS exam cannot yet be relied upon by emergency physicians as a bedside tool to rule out stroke. We simply do not have the evidence to support its adequacy as a diagnostic tool in the hands of emergency physicians, and in fact we may now have a “hint” of evidence to the contrary. I get it. We all got so excited in 2009 when we read about the HINTS exam and how well it worked for neurologists and neuro-ophthalmologists. The idea of it was spellbinding and almost hypnotic – a bedside test that was quick and free and more sensitive than early MRI. We all looked up YouTube videos on how to perform the exam, and we had to triple check how to interpret it after we got back to our desks. We dove into this too fast and too deep, before receiving structured training on this difficult exam that we thought was simple, and before learning exactly which kinds of patients it was appropriate for. We need to take a step back and be methodical, and what we really need is a large multi-center RCT on the diagnostic accuracy of the HINTS exam in the hands of emergency physicians.

5) Generalizability and Validity of Conclusions: The authors did not restrict their search to any particular language, and thus they were able to find studies in four different countries (Argentina, Australia, Spain, and the United States). Four out of the five studies had good and detailed descriptions of the vertigo, nystagmus, and other symptoms of included patients. However, two out of the five studies were exclusively done on an in-patient cohort while the other three were ED, ED referred or ED/admitted patients with diagnosis of cerebellar stroke.

We mentioned that the authors grouped their studies into two cohorts of HINTS examiners: a) neurologists/neuro-ophthalmologists, and b) emergency physicians/neurologists. The former had much better numbers across the board. The latter was literally a big cluster for both accuracy and precision. The assumption is, because the cohort (a) neurologists performed well, the cohort (b) neurologists probably did too, so the sensitivity and specificity of the emergency physician HINTS exam was presumably even worse than 83 and 44%, respectively. But can we really make these indirect conclusions from the analyses done on these mixed-specialty cohorts?

Comment on Authors’ Conclusion Compared to SGEM Conclusion: Both the authors and SGEM acknowledge that there is still insufficient evidence on this topic. Both also note that the HINTS exam need not be completely eliminated from the ED assessment. The biggest difference in the two conclusions is that SGEM is cautioning listeners to gain expert-level skill in performance and interpretation of the HINTS examination before incorporating it into any aspect of their practice. Unstructured, uninformed, and incorrect use of the HINTS exam could do harm.

SGEM Bottom Line: The available evidence does not support the use of the HINTS examination alone by emergency physicians in patients with isolated vertigo or acute vestibular syndrome to rule out a posterior stroke.

Case Resolution: Despite your reassuring HINTS examination, you call in your tech for STAT brain MRI with diffusion-weighted imaging. This MRI 6 hours after symptom onset shows still no evidence of posterior stroke. However, you and your neurologist consultant agree the patient needs admission and further workup because early MRI is only 77% sensitive for posterior stroke.

Dr. Mary McLean

Clinical Application: If you are well-trained in the HINTS exam, feel free to do it for select patients. However, you must know when you can use it, how to perform it, how to interpret and communicate the findings, and how to document it in the patient’s record. Otherwise you could potentially do more harm than good by professing that we are experts in a skill that is proving more difficult than previously thought. At this time, emergency physicians should only perform or interpret the HINTS exam with extreme caution, particularly when using it to rule out central causes of AVS. A non-reassuring HINTS exam may appropriately raise your suspicion for central causes. Never perform the HINTS exam in patients with clear neurologic deficits concerning for stroke, or with any concern for trauma or cervical dissection. Also think really hard about whether you should perform it on patients at risk for carotid stenosis. The quick and jerking movements of a properly-performed head impulse test could theoretically propagate an existing dissection or dislodge an atherosclerotic plaque.

What Do I Tell the Patient? I’m concerned about a possible stroke in the back of the brain causing the dizziness. We will admit you to hospital for observation, and the neurologist will come by and do something called a HINTS examination. They have more skill in this area and they’ll use it to decide if you need another MRI, or any other intervention.

Keener Kontest: Last weeks’ winner was another win for Dr. Steven Stelts of NZ. He knew Dr. Johann Milkulicz was the Polish surgeon who started to wear mask in 1897.

Listen to the SGEM podcast to hear this weeks’ question. Send your answer to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.

Other FOAMed:

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

References:

    1. Merwick Áine, Werring David. Posterior circulation ischaemic stroke BMJ 2014; 348 :g3175
    2. Oppenheim C, Stanescu R, Dormont D, et al. False-negative diffusion-weighted MR findings in acute ischemic stroke. AJNR Am J Neuroradiol. 2000;21(8):1434-1440.
    3. Marx JJ, Thoemke F, Mika-Gruettner A, et al. Diffusionsgewichtetes MRT bei vertebrobasilären Ischämien. Anwendung, Sensitivität und prognostischer Wert [Diffusion-weighted MRT in vertebrobasilar ischemia. Application, sensitivity, and prognostic value]. Nervenarzt. 2004;75(4):341-346. doi:10.1007/s00115-003-1664-
    4. Kattah JC, Talkad AV, Wang DZ, Hsieh YH, Newman-Toker DE. HINTS to diagnose stroke in the acute vestibular syndrome: three-step bedside oculomotor examination more sensitive than early MRI diffusion-weighted imaging. Stroke. 2009;40(11):3504-3510. doi:10.1161/STROKEAHA.109.551234
    5. Carmona S, Martınez C, Zalazar G, et al. The diagnos- tic accuracy of truncal ataxia and HINTS as cardinal signs for acute vestibular syndrome. Front Neurol 2016;7:125.
    6. Chen L, Lee W, Chambers BR, Dewey HM. Diagnostic accuracy of acute vestibular syndrome at the bedside in a stroke unit. J Neurol 2011;258:855–61.
    7. Newman-Toker DE, Kerber KA, Hsieh YH, et al. HINTS outperforms ABCD2 to screen for stroke in acute continu- ous vertigo and dizziness. Acad Emerg Med 2013;20:986–19
    8. Kattah JC, Talkad AV, Wang DZ, Hsieh YH, Newman-Toker DE. HINTS to diagnose stroke in the acute vestibular syndrome: three-step bedside oculomotor examination more sensitive than early MRI diffusion-weighted imaging. Stroke 2009;40:3504–10
    9. Macaskill P et al. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Version 09 0. London: The Cochrane Collaboration, 2010.