SGEM#127: Suspicious Minds vs. Clinical Prediction Rule in Children with Trauma

Subscribe: RSS

[display_podcast]

Date: September 8th, 2015

Guest Skeptic: Dr. Anthony Crocco is a Pediatric Emergency Physician and is the Medical Director & Division Head of the Division of Pediatric Emergency at McMaster’s Children’s Hospital. I often refer to him as a PedsEM Super Hero but the SGEMers probably know him for his RANThony’s on YouTube.

Season#3 finished with an SGEM#126 Hot Off the Press featuring Dr. Jeff Kline discussing outpatient management of venous thromboembolism. We are going to kick off Season#4 with another SGEM HOP from Academic Emergency Medicine.

SGEM Hot Off the Press Process:

We select a paper that has been accepted by the editors at Academic Emergency Medicine that is about to be published.
The SGEM then put a skeptical eye upon the manuscript using a modified BEEM critical appraisal tool.
One of the authors is then invited to discuss their work on the SGEM.
We podcast the review on iTunes and on the SGEM blog shortly after publication inviting the SGEMers to respond on the blog, via Twitter or Facebook.
A summary of the review including the top social media feedback is then published in a subsequent issue of AEM.

Case: A ten-year-old girl is a restrained rear-seat passenger in a motor vehicle collision. The driver of the other car was texting and ran a red light at low city speeds, “T-boning” her vehicle on her side. There was minimal intrusion into the vehicle space.

The girl presents to the emergency department via EMS with a c-spine collar and on a spine board. She is alert, appropriate and in no distress. Her exam is normal, including no evidence of abdominal wall trauma, tenderness or nausea/vomiting. The remainder of her exam is normal.

Given the mechanism of the injury you are concerned about blunt abdominal trauma. You wonder if you should do a CT abdomen, but are also concerned about the risk of radiation. You heard of a study in 2013 that developed a clinical decision instrument for this very question, but don’t know if you should use that, or trust your own ‘gut feeling‘.

Background: The leading cause of death in children is trauma. Blunt torso trauma makes up a significant portion of the mortality statistics. CT scanning has become a routine method for identifying children with intra-abdominal injuries. The amount of radiation being delivered to these vulnerable patients and the risks of developing radiation-induced malignancy has raised concern. There is evidence that clinical suspicion is not accurate in identifying children with intra-abdominal injuries. In addition, there is wide variation on the use of abdominal CT and over utilization in children with low pre-test probability of clinically important injuries.

Clinical Question: How well do clinicians identify children suffering from blunt torso trauma who have intra-abdominal injuries compared to an established clinical prediction rule?

Dr. Prashant Mahajan

Reference: Mahajan P et al. Comparison of clinical suspicion versus a clinical prediction rule in identifying children at risk for intra-abdominal injuries after blunt torso trauma. Sept 2015 AEM .

Population: Children <18 years old with blunt torso trauma presenting to an Emergency Department.
- Exclusion criteria: “injury occurring >24 hours prior to presentation, penetrating trauma, pre-existing neurological disorders preventing reliable examination, known pregnancy, or transfer from another hospital with prior abdominal CT scanning or diagnostic peritoneal lavage.” Patients were also excluded if the clinician did not document their level of clinical suspicion of intra-abdominal injury.
Intervention: Clinical suspicion of intra-abdominal injury as per ordinal scale (<1%, 1-5%, 6-10%, 11-50%, >50%)
Comparison: Intra-abdominal injury undergoing acute intervention.
Outcome: Descriptive statistics using clinician gauged risk <1% as cut-off.
- Sensitivity, specificity, negative predictive value, positive predictive value, positive and negative likelihood ratios were calculated. Results were compared to an un-validated clinical prediction rule metrics.

Authors’ Conclusions: “A clinical prediction rule had a significantly higher sensitivity than clinician suspicion for identifying intra-abdominal injury undergoing acute intervention, but a lower specificity. The higher specificity of clinician suspicion, however, did not translate into clinical practice, as clinicians frequently obtained abdominal computed tomography scans in patients they considered to be at very low risk. If validated, this clinical prediction rule can assist in clinical decision-making around computed tomography use after blunt abdominal trauma in children by limiting computed tomography scan use in low-risk patients.”

Quality Checklist for A Diagnostic Study:

The clinical problem is well defined. Yes. Whether or not to expose children to ionizing radiation is an ongoing concern and minimizing this exposure while not missing significant injuries is a concern for clinicians in the ED.
The study population represents the target population that would normally be tested for the condition (ie no spectrum bias) Yes
The study population included or focused on those in the ED. Yes
The study patients were recruited consecutively (ie no selection bias). Yes
The diagnostic evaluation was sufficiently comprehensive and applied equally to all patients (ie no evidence of verification bias). No
All diagnostic criteria were explicit, valid and reproducible (ie no incorporation bias). Yes
The reference standard was appropriate (id no imperfect gold-standard bias). Yes
All undiagnosed patients underwent sufficiently long and comprehensive follow-up (ie no double gold-standard bias). Yes
The likelihood ratio(s) of the test(s) in question is presented or can be calculated from the information provided. Yes
The precision of the measure of diagnostic performance is satisfactory. Yes

Key Results: Clinicians predicted 168/203 (83%) intra-abdominal injuries requiring intervention and predicted 9217/11716 (79%) negative patients.

Sensitivity= 82.8% (76.9 – 87.7) Specificity= 78.7% (77.9 – 79.4)

PPV= 6.3% (5.4 – 7.3) NPV= 99.6 % (99.5 – 99.7)

+LR= 3.9 (3.6 – 4.2) -LR= 0.22 (0.16 – 0.30)

Five questions for Dr. Mahajan about his research study. Listen to the podcast on iTunes for his responses.

There usually is some backstory about how a research project got started. So what was the inspiration for doing this project?
Your title does not specify that the standard which you are comparing the physicians performance metrics is an un-validated instrument which may mislead the reader into using this instrument prematurely. You mention the fact the instrument is unvalidated later in the text, but I worry that the message is lost.
The literature has many examples where the results of a derivation study were not echoed in the validation studies. Are there plans to take this clinical prediction instrument and attempt to validated it in a future study?
- We have to remember that before a CDI is ready for prime-time, it should go through three steps. The first is derivation – where the researchers try to see amongst a number of various variables, which ones appear to have predictive value. Unfortunately, just by chance alone, some variables may appear significant when they are not. This is why the second stage, validation, is so important. At this stage the researchers work to confirm the significance of the predictive variables from the derivation study. There are examples of seemingly awesome derived CDIs that, when it came time to validation, did very poorly. Recently the CATCH CT head rule was developed, and the initial results were promising. People were using the CDI and it apparently even showed up on a national licensing exam! The results from the validation studies for the CATCH rule, however, were not nearly as good and, as such, it should not be used. A final step, if a CDI passes validation, is impact analysis. This is where researchers see if the rule actually does what it’s supposed to in the real world.
- For a more detailed discussion of these three steps of developing a CDI, SGEMers should check out Anthony’s amazing website sketchyebm.com! He has these great five minute videos using whiteboards to explain CDIs and other EBM concepts. I also really love your tag line…Always draw your own conclusions.
The rates of CT scans based on physician behavior was 5318/11919 = 45%. The rates of CT using the un-validated decision instrument would have been 7004/12044 = 58%. The un-validated instrument appears to significantly increase CT rates.
We like to see patient oriented outcomes as the primary outcome. Although the study title describes the outcome being “children at risk for intra-abdominal injuries”, really the results are describing children with intra-abdominal injuries requiring intervention.

Comment on Authors’ Conclusion Compared to SGEM Conclusion: Until the decision instrument is validated, there is limited value in comparing clinicians’ judgment to it.

SGEM Bottom Line: Clinicians appear to do reasonably well at predicting which children with blunt torso trauma require investigations for intra-abdominal injuries requiring intervention. Nothing can be said at this time about the derived clinical decision instrument.

Case Resolution: Trauma labs and routine xrays were performed including chest and c-spine – all of which were normal. The c-spine collar was removed and the 10 year old was observed for a couple of hours, during which time she tolerated oral intake and was able to ambulate. She continued to be pain-free and is discharged home.

Clinically Application: No change for now.

What Do I Tell My Patient? I’m a bit worried that your child has a significant abdominal injury. We will need to do some more tests to find out if there are internal injuries. One the tests that can help us identify important abdominal injuries is a CT scan.

Keener Kontest – Listen to the podcast for this weeks’ Keener question. If you know the answer then send me an email TheSGEM@gmail.com with keener in the subject line. The first correct answer will receive a cool skeptical prize.

Don’t forget to give us feedback on Twitter, Facebook or the SGEM blog. The best comments will be published in Academic Emergency Medicine. Make sure you also check out SketchyEBM for a number of great EBM videos.