When Brain Imaging Goes Awry

Analysis questions the validity of many fMRI studies.

By Lacy Schley|Thursday, December 15, 2016
Brain imaging with fMRI reveals blood flow in the brain, serving as a proxy for activity, though some software accuracy was recently called into question. The images above, which weren’t part of that study, show the brain in a resting state.
Ann Choe et al./PLoS ONE/doi:10.1371/journal.pone.0140134/2015

Researchers in Sweden and the U.K. landed in the news over the summer, seeming to claim nearly 40,000 functional magnetic resonance imaging (fMRI) studies could be invalid. The problem? The most popular statistical analysis software packages could indicate significant brain activity where there wasn’t any, more than half the time.

FMRI is becoming a more common imaging technique in neuroscience largely because it maps brain activity over time. (Unlike MRI, which maps brain structure, fMRI reveals blood flow, a proxy for activity.) Once the MRI machine cranks out images, statistical analysis software pinpoints which brain areas were significantly more active and when.

The team claims the software packages they examined weren’t calibrated with real data, just simulated data. So they used fMRI scans from 499 healthy people in a resting state, and analyzed the data as they would in a traditional fMRI study. The researchers realized that one particular analysis variation was spitting out false positives at a rate of up to 70 percent.

While mainstream media jumped on the news, neuroscientists and other researchers have debated problems with fMRI statistical software and analysis methods for years. Plus, one of the authors, Thomas Nichols, issued a correction, dialing back the 40,000 figure: Updated language in the study now questions “the validity of a number of fMRI studies.”

“We have a collection of tools in the toolbox,” Nichols says. “What we showed is not that the whole toolbox is faulty . . . basically one wrench has a problem.”

Comment on this article