Podcast: Play in new window | Download
Subscribe: RSS
Date: February 15th, 2019
Reference: Gottileb, Holladay and Peksa. Point-of-Care Ocular Ultrasound for the Diagnosis of Retinal Detachment: A Systematic Review and Meta-Analysis. AEM January 2019.
Guest Skeptic: Dr. Daniel Theodoro is an Assistant Professor of Emergency Medicine at Washington University School of Medicine in St. Louis and the Emergency Medicine Point of Care Ultrasound Section Chief.
Case: A 54-year-old diabetic female presents to your emergency department (ED) complaining of floaters of flashing lights and blurry vision. She has no pain and no history of trauma. She noticed that she couldn’t read her newspaper, like there was a wall of light between her left eye and the words on paper. She presents to the ED looking for answers.
The vision in her right eye is 20/40 and vision in the left eye is 20/50. The left eye field of vision is significant for floaters and decreased capacity to see medially. She has no afferent pupillary deficit and she has no obvious cranial nerve deficits. Her eye is not red, and her cornea is not hazy. The rest of her examination is unremarkable.
Background: Ocular complaints account for 3-4% of all ED visits but the cause for the vast majority of these are benign. One in five of patients with eye complaints, however, will require an ED work up and referral for vision preservation. This group of pathology includes diagnoses such as uveitis, macular degeneration, occipital lobe disorders (amaurosis fugax), and posterior chamber pathology such as vitreous hemorrhage, vitreous detachment and retinal detachment.
Retinal detachment is important because, in some cases, there is an intervention that will prevent and possibly restore vision. Since the preservation of vision and quality of life are closely related, cases with retinal detachment deserve proper follow up and referral to a retina specialist.
Traditionally posterior chamber pathologies are diagnosed with direct and indirect ophthalmoscopy. However, few doctors other than ophthalmologists are sufficiently expert enough to do this examination. So, in the majority of ocular cases in the ED the examination is skipped entirely.
In the FOTO-ED study, ED physicians only did fundoscopy in 14% of appropriate cases. In the study trained nurse practitioners took photos of patient’s funduscopic examination and the photos were reviewed by retina specialists in 24 hours. They enrolled 350 patients, but ED physicians only examined 33 patients whose findings were unknown and in whom fundoscopy may have had a role. In all 33 the diagnosis was missed. Granted that in two-thirds the findings were not in the posterior chamber (e.g. retinopathy and optic nerve pallor) but still, this observational study showed ED physicians haven’t developed or maintained fundoscopic skills.
Further complicating matters is that one study in California demonstrated that fewer than 50% of rural EDs and only 75% of urban EDs have ophthalmology coverage. There are currently some tele-ophthalmology services going up online. They require a photograph taken of the fundus by the practitioner that is remotely reviewed. These are known as 45-degree non-mydriatic ocular fundus photographs and one such company is known as Topcon.
To make matters worse direct ophthalmoscopy has poor test characteristics and even indirect ophthalmoscopy has limits until it’s in the hands of experienced and skilled ophthalmologists. In the hands of experienced operators, indirect ophthalmoscopy has an LR+ 44 and LR- of 0.23. Remember, you need a LR- of less than 0.01 to rule out a condition.
If you are repeatedly performing a skill or procedure, receiving feedback, and working to improve you are engaged in Ericsson’s “deliberate practice,” the key to becoming an expert. Most emergency physicians don’t get regular feedback when using an ophthalmoscope, so one has to ask, is there an easier way?
Physicians began to use ocular ultrasound for the diagnosis of posterior chamber (and anterior chamber) ocular pathology in the 1950s. It’s been about 17 years since the idea first entered the emergency medicine academic literature. Ocular ultrasound devices are becoming more widespread in EDs across North America. They are portable so they can be brought to the patient’s bedside and can be comparatively easier to perform than direct or indirect ophthalmoscopy.
The visual information is magnified using ultrasound and for physicians to look at the data at once, and the exam is easily repeated. That allows for repetition with immediate feedback.
The cons are that it remains a skill that we have to learn (albeit theoretically a more readily “learnable” skill than ophthalmoscopy). In 2015, a systematic review by Dr. Vrablik out of the University of Indiana found three studies quoting very high sensitivity and specificity. Since that review was published, larger trials have been reported in the literature.
Clinical Question: What are the testing characteristics of ocular point of care ultrasound when attempting to diagnose retinal detachment among a group of patients presenting with vision complaints?
Reference: Gottileb, Holladay and Peksa. Point-of-Care Ocular Ultrasound for the Diagnosis of Retinal Detachment: A Systematic Review and Meta-Analysis. AEM January 2019.
- Population: Prospective or RCTs of patients presenting to ED, Radiology Departments and other sites such as ophthalmology clinics being assessed for retinal detachment.
- Exclusions: Case reports, case series, retrospective studies, cadaver studies and conference abstracts.
- Intervention: Point of care ocular ultrasound (POCUS) performed by operators with backgrounds in Emergency Medicine and Radiology of varying experience levels without mention of a defined protocol other than the use of a “portable” ultrasound machine.
- Comparison: The “Gold Standard”or “Reference Test” had to include a confirmatory test defined as formal ophthalmologic exam, surgical findings, CT findings, MRI findings, or clinical follow up.
- Outcome: Diagnostic accuracy of POCUS (sensitivity, specificity, positive/negative likelihood ratio, positive/negative predictive value).
Authors’ Conclusions: “Point-of-care ocular ultrasound is sensitive and specific for the diagnosis of retinal detachment. Future studies should determine the ideal training protocol and the influence of color Doppler and contrast-enhanced ultrasound on diagnostic accuracy.”
Quality Checklist for Systematic Review Diagnostic Studies:
- The diagnostic question is clinically relevant with an established criterion standard. Yes
- The search for studies was detailed and exhaustive. Yes
- The methodological quality of primary studies were assessed for common forms of diagnostic research bias. Yes
- The assessment of studies were reproducible. Yes
- There was low heterogeneity for estimates of sensitivity or specificity. No
6. The summary diagnostic accuracy is sufficiently precise to improve upon existing clinical decision-making models. No
Key Results: Investigators found 2,620 articles in their literature search and 11 studies were included in the meta-analysis. The 11 studies were all observational studies with total of 844 patients. Emergency medicine clinicians performed the POCUS exam in six of the studies.
The overall diagnostic parameters of POCUS to diagnose retinal detachment was very good.
The area under the receiver operating characteristic (ROC) curve demonstrated high accuracy (0.988; 95% CI = 0.974 to 0.994) while the heterogeneity was moderate (I2 = 0.59). The funnel plot did not show evidence of publication bias.
1) Methods: They did an excellent job with their methods. They followed the PRISMA-DTA guidelines and registered their review with PROSPERO. The search was exhaustive, and they used the talents of a medical librarian. This found four studies that were not discussed in the prior systematic review and added another four studies of pretty high quality.
2) Confidence Intervals: The 95% confidence intervals around the point estimates were really wide. This makes us less confident in the results. Some of the range in the confidence interval could have been due to heterogeneity.
3) Heterogeneity: When putting together different studies into a systematic review there will be variability across studies. This variability can be clinical, methodological or both. Clinical variability can be due to the participants, interventions and outcomes studied. Methodological variability can be due to study design and risk of bias. The variability in studies is quantified using I2. Statistical heterogeneity then is the amount of difference in effect between studies being greater than expected by chance (randomness) alone. This study had an I2 of 0.59 suggesting moderate heterogeneity between the studies meta-analyzed.
The heterogeneity observed between the studies could have been due to a number of different biases. Biases are something that systematically moves us away from the truth.
- Partial Verification Bias: This isn’t mentioned in the papers but in studies with lower specificities and higher sensitivities than the pooled estimate it makes one wonder if it took place.
How does this type of bias occur? Only some people get a true gold standard (like a formal eye exam) and the ones that do are dependent on the result of the ocular POCUS because the protocol is not strictly enforced on all patients with visual complaints.
As and example, if the ocular POCUS has an obvious finding ophthalmology will likely see the patient and the patient will be entered into the study. Now imagine cases where the ocular POCUS is negative, the patient’s complaints are seemingly minor, and the patient has non-ophthalmologic follow up. The gold standard is not applied based on not only the negative ultrasound but the intensity of the complaints. Many of the studies used “convenience sampling” meaning tests were only done if all the pieces were in place and this encourages partial verification bias.
Statistically, when this happens the true negatives are decreased as are the false negatives. When this happens the false negatives are increased in greater proportion than the true negatives and this effect mathematically increases sensitivity and decreases specificity.
- Differential Verification Bias: In real estate it’s “location, location, location.” For differential verification bias (double gold standard bias), it can be “timing, timing, timing.” This is when different follow-ups apply to different patients, hence the “double” gold standard. This is important because the disease can change according to what type of follow up the patient ultimately receives. As an example, positive POCUS exams with large obvious retinal detachments are referred to a specialist in six hours but minor findings get follow up 72 hours later.
Another possible scenario is that the POCUS is negative for detachment but does identify a vitreous bleed. Follow up is assigned for 72 hours as opposed to six hours. If this vitreous bleed was really a mild tear missed by POCUS and it becomes worse in those 72 hours then POCUS looks bad because it categorized a mild tear as a false negative (decrease in sensitivity and specificity). Conversely, if the mild tear would re-adhere in 72 hours, the patient reports no vision trouble, the POCUS performance appears better than stated.
You need to dive into the studies that varied from the pooled estimates and see what they did. Sure enough in some there was one to six week follow up while in others there was immediate follow up. A lot can happen in six weeks, so readers have to be aware that this can happen and it could explain some of the heterogeneity when the results are way off the pooled estimate.
- Spectrum Bias: Did these studies include the sickest of the sick? How about the mild vision complaints? All diagnostic vision tests look great if you include only intense vision loss or if you exclude patients with vitreous detachments because they are similar.
One can loosely judge this by prevalence and convenience sampling. The prevalence of disease in some studies is near 50% but we know the majority of eye complaints are benign. So, what’s happening here? I don’t think everyone with “blurry vision” is being considered for inclusion in these studies.
However, I’m sure there are cases of potential stroke or other neurologic disease mimics that turn out to be posterior chamber pathologies. We just don’t know if these candidates are included in any of these studies. This is an area one could pick for future study in a large, multi-centered study with hard clinical endpoints and pre-scheduled, pre-defined follow ups.
Only one of the included studies followed the STARD criteria (Standards for Reporting of Diagnostic Accuracy Studies). The objective of the STARD initiative is “to improve the completeness and transparency of reporting of studies of diagnostic accuracy, to allow readers to assess the potential for bias in the study (internal validity) and to evaluate its generalisability (external validity)”. The lack of adherence to the STARD criteria suggests that the included studies were of lower quality. Seeing as the prevalence of disease seems to be high in all of the studies, the reader can assume this could have introduced some bias.
4) Index Test: Another thing we wanted to mention was the index test. In this case the ocular POCUS, as it pertains to training as well as the training issues surrounding the “criterion or gold standard.” Some of these studies had very experienced people that probably spent a lot of time learning how to perform these examinations. In some cases, the index test users got a 20-minute course. The study tries to addressed this with a series of post hoc analyses.
Likewise, some patients received follow up by retina specialists while others were followed by ophthalmology residents. In yet other cases highly trained POCUS providers were followed up by less trained ophthalmologists. This might set up an “imperfect” situation?
These relationships can go in many different ways and worse, they can arise on a case by case basis or “at the patient level,” a source of heterogeneity. The combinations are endless, and readers need to be aware of this possible source of bias and decide if it’s a real “apples to apples” comparison.
5) Mac On or Mac Off: The risk of missing a retinal detachment depends on the status of the macula in relation to the detachment. If the retinal detachment has taken the macula off then there’s not much to offer the patient. This is a “macula off” tear, meaning, the macula has been torn off. If the tear abuts but does not involve the macula then the macula is still on. A “macula on” tear is a true emergency because the patient can undergo interventions to tether the remaining retina and preserve vision.
Comment on Authors’ Conclusion Compared to SGEM Conclusion: We agree with the authors’ that ocular POCUS is sufficiently specific to diagnose retinal detachments. We disagree that the sensitivity is sufficiently high to rule out the diagnosis in cases of floaters and vision changes but think that this has more to do with training than the actual limitations of the technology. We need to lobby our training programs to spend more time teaching how to do the procedure to instil a high level of performance and confidence in our trainees.
SGEM Bottom Line: Early on in your POCUS training if you identify a retinal detachment make the call. Be wary if you don’t see any pathology and make sure the patient has immediate consultation or immediate follow-up. As you progress in your POCUS training you may be more confident with cases that you can rule out.
Case Resolution: You make the diagnosis of a retinal detachment, but you are concerned the macula remains intact (Mac On) because the tear does not reach the optic nerve. You consult ophthalmology who schedules the patient for a procedure to preserve vision.
Dr. Daniel Theodoro
Clinical Application: Ocular POCUS is highly specific meaning it’s excellent if the pathology is readily visible. However, more research is needed to determine if it’s unacceptable sensitivity is due to lack of training or limitations of ultrasound technology.
What Do I Tell the Patient? I typically use the “movie screen” analogy and say, imagine you’re at the theatre looking up at your favorite movie and part of the screen peels off. If it’s a large part of the screen then the movie is going to look all blurry and wavy. This is an emergency and I may have you see the eye surgeon right away. If I look into your eye and it seems like the screen is flat, then I’m going to ask you to see an eye doctor in the next two to three days because sometimes it appears like it’s glued tightly but a small corner of the screen has come undone. That small corner can get pretty big over time.
If you have no eye doctor or no good way of seeing an eye doctor in the next two to three days I may look with my ultrasound and, regardless of what I see, I will ask them to look into your eye so at least they start following your symptoms because preserving vision is an important piece of a happy life and would be worth the wait.
Keener Kontest: Last weeks’ winner was Brandon Snyder a first-year medical student from Ohio University Heritage College of Osteopathic Medicine. He knew the dose of magnesium for treating eclampsia is 4-6 grams via 500mL IV normal saline.
Listen to the SGEM podcast to hear this the new keener question. If you know the answer send an email to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.
Other FOAMed: There are several FOAM ED platforms with videos on how to perform the examination. Here are a few links on the topic.
- EM:RAP – Ultrasound of Retinal vs Vitreous Detachment
- Mike and Matt – Introduction to Bedside Ultrasound: Volume 2
- ACEP members can download a free iBook from the Apple store
You must be logged in to post a comment.