SGEM#412: I Can’t Choose…from all the Head Injury Prediction Rules

Subscribe: RSS

Reference: Easter JS et al. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Annals of Emergency Medicine 2014.

Date: July 10, 2023

Guest Skeptic: Dr. Joe Mullally is a paediatric trainee in the Welsh paediatric training program and interested in Paediatric Emergency Medicine. He is a student in the Paediatric Emergency Medicine Masters Program through Queen Mary University in London in collaboration with the Don’t Forget the Bubbles team.

Dr. Joe Mullally

Background: Children have big heads proportionally to their body compared to adults which makes them more at risk of traumatic brain injury (TBI). Computerized tomography (CT) is commonly used in the emergency department in the diagnosis of TBI. But we’re always trying to balance the potential harms and potential benefits in medicine. A CT scan does mean radiation to the pediatric brain which can increase the risk of leukemia or brain cancer later [1]. Thankfully, clinically important intracranial injuries are rare in children [2]. So, should we CT scan children with minor head injury?

The SGEM covered pediatric concussions and head imaging in SGEM #112 and the NEXUS II Pediatric Head CT Decision Instrument in SGEM #225. Today we’re talking about three other popular clinical decision rules (PECARN, CATCH, and CHALICE). But we also want to know, how do those rules compare to physician judgement?

Clinical Question: What is the diagnostic accuracy of clinical decision rules and physician judgment in identifying clinically important traumatic brain injuries (TBI) in children with minor head injury?

Reference: Easter JS et al. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Annals of Emergency Medicine 2014.

Population: Children less than 18 years of age presenting with head injury to a level 2 pediatric trauma center in the United States between 2012-2013. These children have to have 1) history of signs of blunt injury to the head 2) GCS scores ≥13, 3) injury within the previous 24 hours prior to presentation, 4) physician concern for potential TBI
- Excluded: Heightened TBI risk (GCS<13, brain tumors, ventricular shunts, on anticoagulants, or had bleeding disorders), or if they presented >24 hours after injury
Intervention: CT vs no CT
Comparison: Comparison of PECARN, CHALICE, CATCH, physician judgement, and physician practice
Outcomes:
- Primary Outcome: “Clinically important TBI” defined as death from TBI, need for neurosurgery, need for intubation >24hrs for TBI, or hospital admission >2 nights for TBI.
- Secondary Outcomes:
  TBI on scan
  TBI requiring neurosurgery (craniotomy, elevation of skull fracture, monitoring of intracranial pressure, or intubation for elevated intracranial pressure)
Type of Study: Single center prospective cohort study

Authors’ Conclusions: “Of the 5 modalities described (PECARN, CATCH, CHALICE, physician judgment and physician practice), only physician practice and PECARN identified all clinically important TBIs, with PECARN being slightly more specific. CHALICE was incompletely sensitive but the most specific of all rules. CATCH was incompletely sensitive and had the poorest specificity of all modalities.”

Quality Checklist for Observational Study:

Did the study address a clearly focused issue? Yes
Did the authors use an appropriate method to answer their question? Yes
Was the cohort recruited in an acceptable way? Yes
Was the exposure accurately measured to minimize bias? Unsure
Was the outcome accurately measured to minimize bias? Unsure
Have the authors identified all-important confounding factors? Yes
Was the follow up of subjects complete enough? Yes
How precise are the results? Fairly precise
Do you believe the results? Yes
Can the results be applied to the local population? Unsure
Do the results of this study fit with other available evidence? Yes
Funding of the Study. No financial conflicts of interest.

Results: During the study period, 1,526 children with head injury presented to the ED, and they enrolled 1,062 (70%).

The enrolled group had a median age of 6.1 years and 64% were male with 95% presenting with GCS of 15.

In comparison the non-enrolled group had a median age of 5 years, 58% male, and 99% had GCS of 15.

In summary, the enrolled group was a little bit older, with slightly higher proportion being male, and a bit fewer having GCS of 15.

They ended up excluded another 53 mostly because they presented >24 hours from time of injury ending up with a total of 1,009 children included in the study.

Key Results: Only PECARN clinical decision rule and physician practice had 100% sensitivity for detecting clinically important TBI.

Fifty-two (5%) had injuries seen on CT scan. The most common were skull fractures but there were a few subarachnoid and subdural hemorrhages as well.

Twenty-one (2%) had clinically important TBI.

Only four (0.4%) required neurosurgical intervention.

Selection Bias:

This study had some inclusion criteria that may have led to selection bias. Specifically, the physician concern for potential TBI is very subjective. They also excluded anyone who was not seen within 24 hours with the reasoning that risk of clinically important TBI decreases with time.

Although this may appear appropriate, we have probably encountered a child in the ED who presented greater than 24 hours from head injury. Maybe these families tried to wait it out after initial injury to avoid the ED visit, but the child continued to have persistent symptoms. What do you do if there may be physician concern for TBI in those circumstances? It’s unclear how these exclusion criteria impacted the study.

Verification Bias:

Verification bias occurs when only a proportion of the study participants receive confirmation of the diagnosis by the reference standard test (CT scan). Approximately half of patients were discharged after initial evaluation. A third of patients were observed for a median of three hours. The remainder underwent CT scan.

Practically, we’re not arguing that every single patient in the study should have received a CT scan. The authors resorted to other methods to follow up with patients.

It’s also important to realize that just because something may be radiologically significant, does not necessarily mean it is clinically significant. In this study, they noted 52 (5%) of patients who had injuries present on CT but only 21 (2%) had clinically significant TBI.

Follow up obtained for 87% of patients who did not undergo CT. 57% of those were evaluated by physician and 43% were evaluated by phone. The team also looked at trauma registries and quality improvement reports to see if any patients had died. This is imperfect. Maybe the patient sought care at an institution outside of the health system they originally presented to.

For more information on understanding the direction of bias in diagnostic studies we suggest you read the classic paper by Kohn et al 2013. There is also the new book by Dr. Jesse Pines on clinical decision rules and he was interviewed on an SGEM Xtra in June.

Inter-Rater Reliability:

The authors looked at agreement amongst physicians on the study variables. They considered a kappa value of >0.5 to be acceptable.

There are other sources that suggest a different interpretation: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [3].

Most of these variables and their associated kappa values are presented in Appendix 2. They looked at 180 observations total and most variables have good agreement. Worsening headache and intoxication have the lowest kappa’s of 0.49 and 0.43 respectively but there is no further discussion.

GCS only had a kappa of 0.65. Previous studies show that agreement among physicians on GCS scores is quite variable [4]. In children, there are modifications to the GCS scoring based on age which poses additional challenges. This also potentially impacts the patients who were included in this study. Were there patients inappropriately excluded or inappropriately included if there was variation in how GCS was determined amongst physicians?

Clinician Judgment:

The authors report that physicians were trained on the data collection instrument. After assessing the patient but before obtaining results of testing, they recorded the presence of predictor variables. Is it possible that completing the data collection instrument with all the predictor variables influenced their decision because it reminded or cued them to what the predictor variable were significant?

There was also discrepancy between physician estimate versus physician practice. Physicians’ estimate had sensitivity at 95% but physician practice had a sensitivity of 100% meaning they CT scanned more children compared to their estimate. We’re not sure why that is. It is also interesting to note that the specificity of physician estimate was better than physician practice.

Clinical Decision “Rules”:

I don’t really like the saying “clinical decision rule.” The “rule” part just feels so strict to me.

We know that evidence-based medicine has three pillars, clinical judgment, scientific literature, and patient/family value and preferences. There is a lot of room for shared decision making here.

How about we call these “tools” instead. They are there to help guide decision making but should not be taken as dogma.

Our friend Justin Morgenstern from First10EM takes a stronger position in his blog post called: Clinical decision rules are ruining medicine

Comment on Authors’ Conclusion Compared to SGEM Conclusion: We agree with the authors’ conclusion. However, it may be difficult to generalize the findings of this single institution study with low prevalence of clinically important TBI.

SGEM Bottom Line: Clinical decision tools should be used in conjunction with clinical judgment and parents/caregivers preferences to determine the need for CT scans in children with mild TBI.

Case Resolution: You tell the parents that it is likely the patient sustained a mild traumatic brain injury and discuss the option to undergo head CT versus close observation in the emergency department. The family decides that it would give them more peace of mind if a CT was performed. A head CT is performed and does not demonstrate any evidence of skull fracture or intracranial bleed. You counsel the family on signs and symptoms of concussion prior to discharging them from the emergency department.

Clinical Application: Joe works in a system that utilizes CHALICE. You may work in a healthcare system that uses a different prediction tool. It is important to realize that none of these tools are perfect and the studies all have their limitations. These tools should inform clinical decision-making, rather than dictate management.

What Do I Tell My Patient? Your child likely sustained a mild head injury, but I understand why you are concerned. I have not found any concerning findings on my physical exam. I think we can potentially do two things. We can observe her in the emergency department for a few hours. If she remains well, we can let you go home. If she becomes symptomatic, we can decide to obtain a head CT. Alternatively, we could perform a head CT now. Recognize that although a head CT can tell us whether there is a skull fracture, it does expose her to some radiation. We can discuss which option you would prefer.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

References:

Pearce MS, Salotti JA, Little MP, et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet. 2012;380(9840):499-505.
Kuppermann N, Holmes JF, Dayan PS, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374(9696):1160-1170.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282.
Gill MR, Reiley DG, Green SM. Interrater reliability of Glasgow Coma Scale scores in the emergency department. Ann Emerg Med. 2004;43(2):215-223.