SC Connecticut News

Yale researchers have demonstrated that predictive models in neuroimaging can be effective across diverse datasets, a crucial step toward making these models clinically useful. The study highlights the importance of testing models on varied data to ensure they generalize well beyond their training environments.

Brendan Adkinson, lead author of the study published in Developmental Cognitive Neuroscience, noted, "It is common for predictive models to perform well when tested on data similar to what they were trained on. But when you test them in a dataset with different characteristics, they often fail, which makes them virtually useless for most real-world applications."

The research focused on overcoming differences across datasets, such as variations in age, sex, race and ethnicity, geography, and clinical symptom presentation. Adkinson emphasized that these differences should be seen as essential components rather than obstacles. "Predictive models will only be clinically valuable if they can predict effectively on top of these dataset-specific idiosyncrasies," he stated.

In their experiments, the team trained models to predict language abilities and executive function using three large but distinct datasets. Each model was then tested on the other two datasets. According to Adkinson, "We found that even though these datasets were markedly different from each other, the models still performed well by neuroimaging standards during testing."

Adkinson expressed interest in further exploring model generalizability concerning specific populations. He pointed out that current data collection efforts are concentrated in metropolitan areas and may not represent rural populations adequately. "If we get to a point where predictive models are robust enough to use in clinical assessment and treatment but they don’t generalize to specific populations like rural residents, then those populations won’t be served as well as others," said Adkinson.

The study underscores the potential for developing more inclusive predictive models that could eventually aid in personalized mental health and neurological treatments.