Introduction

Mild traumatic brain injury (mTBI), or concussion, is an increasing public health concern not only due to its growing frequency of occurrence, but lack of guidelines and biomarkers that make diagnosis challenging [1]. This is particularly concerning in the pediatric population whose ongoing cognitive maturation makes them more vulnerable to head injury than the adult population [2]. The diagnostic utility of neuroimaging is limited, and current recommendations do not recommend routine imaging for mTBI unless otherwise clinically indicated [1]. Furthermore, literature evaluating sports-related mTBI has indicated that young athletes with mTBI may not recognize alarming symptoms due to a lack of awareness, understanding, or as a result of their cognitive impairment [3]. Thus, self-reported post-concussion symptoms are not reliable in this population. Additionally, while there are several concussion grading systems, there is little agreement between the systems on how they define and assess mTBI, and a collectively agreed gold-standard grading system does not exist [4]. Self-reporting concussion symptoms are often at odds with social pressures on a younger person to perform both educationally and athletically and place a strong bias making these measures more inaccurate [5].

Although neuropsychological tests were developed to more accurately measure cognitive deficits in mTBI, these tests have their limitations. Typically, they require a neuropsychologist or psychometrist to administer, score, and interpret the results of the battery of tests. More importantly, they lack adequate test sensitivity and specificity and are susceptible to practice effects that occur when tests are repeated more than once [6].

This study explores an alternative solution to address the barriers associated with mTBI assessment. The BrainCheck Sport Battery is a rapid, self-administered, and computerized neurocognitive test that assesses various areas of cognition, such as attention, processing speed, coordination, and executive functioning. Its diagnostic accuracy was previously validated as a testing method for traumatic brain injuries among adults, and it is classified as a diagnostic aid by the FDA [7]. However, the BrainCheck Sport Battery has not been explored for use in the detection of pediatric mTBI. Thus, the primary objective of this preliminary study is to assess the utility of BrainCheck Sport as a diagnostic tool for mTBI in the pediatric population.

Materials and methods

This study was approved by the Baylor College of Medicine institutional review board. Participant recruitment took place at the emergency department (ED) of Texas Children’s Hospital (TCH) in Houston, TX and Morton Ranch High School (MRHS) in Katy, TX. MRHS is a large local public high school with a diverse student population, multiple sports teams, and experienced staff assisting research. The recruitment period took place over one school semester at MRHS.

Results

Demographics

The overall sex distribution was unbalanced because most athletes participated in the study were males (Figure 1). Enrolled participant age ranged from 8 to 18 years, with a mean age of 14.7 and standard deviation of 1.6 (Figure 2). Nevertheless, there was no statistically significant difference in sex between the mTBI and control groups (χ² = 0.005, P-value = 0.94). Similarly, no significant difference in age was detected between the mTBI and control groups (W = 499.5, P-value = 0.48).

Display full size

Figure 1. 

Sex distribution of participants

Display full size

Figure 2. 

Age distribution of participants

Assessments scores on mTBI

The BrainCheck Sport assessments evaluate different cognitive domains. As shown in Figure 3 and Table 2, we found that mTBI and control participants showed significant differences on raw scores of Delayed Recognition (P < 0.001), Immediate Recognition (P < 0.001), the Stroop test (P < 0.01), and the Coordination test (P< 0.001). In contrast, no significant differences were observed for the Trail Making test, the Flanker test, and the Digit Symbol Substitution test (Table 2).

Display full size

Figure 3. 

Boxplots of BrainCheck Sport assessment metrics for the control group and the mTBI group

Table 2.

Mann-Whitney U test comparison between the mTBI and control groups on the BrainCheck Battery assessment metrics, with Benjamini-Hochberg adjusted P-values.

Assessment	Metrics	Adjusted P-value	Explanation
Coordination	balance_mean_distance_from_center	< 0.001*	Mean radial distance from center circle
	balance_percent_in_target_mean	< 0.001*	Mean of percentage time in circle
	balance_position	< 0.001*	Mean position from the circle center
	balance_total_duration_in_circle	< 0.001*	Total duration time in circle
	balance_total_duration_out_circle	< 0.001*	Total duration time out of circle
	balance_total_duration_unpressed	0.95	Total duration time not pressing the screen
	balance_total_number_exits	0.80	Total number of times existing the circle
Digit Symbol Substitution	digit_symbol_correct_per_second_mean	0.80	Mean of correct responses in a second
	digit_symbol_duration	0.10	Total time taken to complete assessment. Time starts after digit display and ends when correct answer is completed
Flanker	flanker_correct_mean	0.28	Mean of correct responses
	flanker_reaction_time_central_mean	0.87	Mean reaction time of all correct responses in central cue trials
	flanker_reaction_time_congruent_mean	0.51	Mean of all reaction times when arrows point in same direction
	flanker_reaction_time_correct_mean	0.87	Mean reaction time of all correct responses
	flanker_reaction_time_correct_median	0.93	Median reaction time of all correct responses
	flanker_reaction_time_incongruent_mean	0.95	Mean of all reaction times when arrows point in opposite directions
	flanker_reaction_time_incorrect_mean	0.28	Mean reaction time of all incorrect responses
	flanker_reaction_time_spatial_mean	0.83	Mean reaction time of all correct responses in spatial cue trials
Immediate and Delayed Recognition	delayed_recognition_correct	< 0.001*	Number of correct responses
	immediate_recognition_correct	< 0.001*	Number of correct responses
Stroop	stroop_basic_reaction_time_mean	0.91	Mean reaction time for neutral words
	stroop_basic_reaction_time_median	0.87	Median reaction time for neutral words
	stroop_effect_ms_mean	< 0.001*	Mean reaction time incongruent - congruent (in ms)
	stroop_effect_ratio	< 0.001*	Median reaction time of incongruent / median reaction time of congruent
	stroop_reaction_time_congruent_mean	0.006*	Mean reaction time for all correct answers in congruent trials
	stroop_reaction_time_incongruent_median	0.057	Mean reaction time for all correct answers in incongruent trials
Trail Making	trails_b_duration_median	0.98	Median of response times between each click for TrailMaking B
	trails_b_duration_mean	0.80	Mean of all response times between clicks for Trail Making B
	trails_a_duration_mean	0.87	Mean of all response times between clicks for Trail Making A

Display full size

*

Statistically significant P-values

These preliminary results demonstrate that participants who had an mTBI exhibited worse performance in BrainCheck Sport assessments of memory, executive, and coordination on the battery, which are consistent with results of golden standard neuropsychological tests [10–12]. In addition, they showed no significant deterioration in their cognitive performance and attention, which has also been observed in previous studies [13, 14].

Logistic regression

For the logistic regression model, 29 observations were deleted due to missing data in their Coordination assessment, therefore it utilized 86 of the participants in our control group and 10 of the participants in the mTBI group. We used “delayed_recognition_correct”, “stroop_effect_ratio”, “immediate_recognition_ correct”, “balance_percent_in_target_mean”, “digit_symbol_duration”, “flanker_correct_mean”, “trails_a_ duration_mean”, “trails_b_duration_median” as the metrics for our model. Based on the logistic regression model, the Stroop test, the Immediate Recognition test, the Coordination, and Trail-making tests were statistically significant at α = 0.05, while the rest were not statistically significant (Table 3). We performed a variance inflation factor (VIF) check, which showed no values greater than 2, rejecting the existence of collinearity in our model. Furthermore, the model residuals were checked and no influential outliers were present. We then performed ROC analysis methods to determine the optimal threshold for the logistic regression model that could maximize sensitivity and specificity in distinguishing between the control and mTBI groups. The ROC curve (Figure 4) had an area under the curve (AUC) of 0.884 (0.818–0.950, 95% CI). We found that the BrainCheck Sport Battery could achieve a sensitivity of 84% and a specificity of 81% when we chose 0.28 as the optimal decision threshold (Figure 4). In other words, if the model predicted probability for a participant is greater than 0.28, the participant is categorized to the mTBI group.

Display full size

Figure 4. 

ROC curve of logistic regression with reported AUC and 95% CI

Table 3.

Summary of logistic regression results

Assessment (metric)	β coefficient	β std. error	P-value
Coordination (balance_percent_in_target_mean)	–2.30	0.97	0.018*
Digit Symbol Substitution (digit_symbol_duration)	–1.93	0.89	0.03*
Immediate Recognition (immediate_recognition_correct)	–0.42	0.20	0.03*
Stroop (stroop_effect_ratio)	2.86	0.83	0.0006*
Trail Making B (trails_b_duration_median)	1.13	0.52	0.03*
Trail Making A (trails_a_duration_mean)	–0.75	1.04	0.83
Delayed Recognition (delayed_recognition_correct)	–0.19	0.14	0.18
Flanker (flanker_correct_mean)	0.015	0.07	0.83
Intercept	9.40	4.61	0.04*

Display full size

*

Statistically significant P-values

Discussion

The findings of this study demonstrated that BrainCheck Sport was able to discriminate mTBI patients from the control group on some battery assessments and achieved decent levels of sensitivity (84%) and specificity (81%) when classifying mTBI, despite the small sample size of the mTBI population. This is achieved by evaluating the cognitive function, memory, and coordination of the participants under tests. These preliminary results reflect that BrainCheck Sport has potential as a diagnostic aid for pediatric mTBI, with similar accuracy as in the adult population [7].

Our results demonstrated that after mTBI, patients experienced deficits on Immediate and Delayed Recognition tests; Coordination tests; and the Stroop color tests which represent deficits in cognitive domains consisting of short-term memory, coordination, and executive function, specifically inhibition and attention, respectively. These are typical symptoms observed after sustaining an mTBI. However, mTBI patients were not observed to have diminished performance on Trail Making, Digit Symbol Substitution, or the Flanker tests which represent cognitive domains consisting of executive function, specifically, cognitive flexibility, information processing, and speed, and visual attention, respectively. As suggested by previous studies, not all cognitive domains are affected by mTBI [13, 14]. However, De Beaumont et al. [15] found that athletes with a concussion history performed worse in the Flanker Test. This could be due to the repeated injuries these athletes sustained, significantly hindering their cognitive performance over time. In our study, we excluded participants with a history of mTBI, and our participants were relatively younger. This could explain the non-significant differences in Flanker test performance observed in our study, as older participants who have experienced repeated concussions over time could present different symptoms than younger participants with no prior mTBI history. On the other hand, the majority of our participants were only assessed within 96 h after injury leaving time for symptoms to possibly resolve. Further studies should control for time of administration after injury more stringently by restricting the timeframe to within 24 h or less.

It is important to keep in mind that our participants were from an atypical population. The control group was composed of athletes and was not fully representative of the pediatric population. Since the mTBI experimental group included both sports related and non-sports related mTBIs, a control group with similar demographics would have contributed to a more balanced design. In addition, both our experimental and control groups were composed of only a few females. Such sex differences in mTBI injury severity have been reported in previous literature [16]. To investigate whether there are sex differences in the neurocognitive measures examined in the BrainCheck Sport Battery, future research should utilize a more evenly distributed ratio of males to females in the participants pool. Additionally, despite being a statistically significant predictor, the coordination test occasionally suffered from software malfunctioning during testing and resulted in several cases of missing data in both the experimental and control groups. As a retrospective study, it is hard to determine if all cases of malfunctioning were properly documented and fully taken into consideration.

Another possible confounding factor in our sample was the combined grouping of pre-pubescent, pubescent, and post-pubescent aged participants. While we did not observe any significant difference in age between the control and patient groups, previous work has shown that performance on computerized testing batteries improves across adolescent development at even larger magnitudes within attention and working memory cognitive domains [17], which are demonstrated to be affected by mTBI. We also sacrificed statistical power with a smaller mTBI group. While a larger experimental group would likely have given more robust results, we proceeded with statistical analysis to look for preliminary results as diagnosing pediatric mTBI is challenging. The complexity in diagnosis is multifactorial, which includes: low reliability of subjective history of symptoms relative to the adult population and the subsequent increased importance on objective findings [3], current standard of practice to avoid neuroimaging unless there are obvious signs of serious intracranial injury [1], and the large range of symptoms after concussion with a wide spectrum of cognitive impairment [9]. Thus, providing these preliminary objective data may support clinicians with a diagnostic aid in this uniquely difficult population.

Currently, there are a variety of different test batteries and screeners used in clinical practice to aid in the diagnosis of mTBI. BrainCheck Sport has potential as another option among the computerized neurocognitive tests by providing a shorter, gamified test battery that attempts to comprehensively examine and assess cognitive functioning after brain injury. This preliminary study demonstrates that BrainCheck Sport performs well in detecting and classifying mTBI patients and may be useful as a diagnostic aid.