← Search

Psychological Science Vol. 31 No. 7 2020

What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis

Maxwell L. Elliott1; Annchen R. Knodt1; David Ireland2; Meriwether L. Morris1; Richie Poulton2; Sandhya Ramrakha2; Maria L. Sison1; Terrie E. Moffitt1,3,4,5; Annchen R. Knodt1,3,4,5; Ahmad R. Hariri1

1 Department of Psychology & Neuroscience, Duke University · 2 Dunedin Multidisciplinary Health and Development Research Unit, Department of Psychology, University of Otago · 3 Social, Genetic, & Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, & Neuroscience, King’s College London · 4 Department of Psychiatry & Behavioral Sciences, Duke University School of Medicine · 5 Center for Genomic and Computational Biology, Duke University

open access

Abstract

Identifying brain biomarkers of disease risk is a growing priority in neuroscience. The ability to identify meaningful biomarkers is limited by measurement reliability; unreliable measures are unsuitable for predicting clinical outcomes. Measuring brain activity using task functional MRI (fMRI) is a major focus of biomarker development; however, the reliability of task fMRI has not been systematically evaluated. We present converging evidence demonstrating poor reliability of task-fMRI measures. First, a meta-analysis of 90 experiments ( N = 1,008) revealed poor overall reliability—mean intraclass correlation coefficient (ICC) = .397. Second, the test-retest reliabilities of activity in a priori regions of interest across 11 common fMRI tasks collected by the Human Connectome Project ( N = 45) and the Dunedin Study ( N = 20) were poor (ICCs = .067–.485). Collectively, these findings demonstrate that common task-fMRI measures are not currently suitable for brain biomarker discovery or for individual-differences research. We review how this state of affairs came to be and highlight avenues for improving task-fMRI reliability.

DOI
10.1177/0956797620916786
Volume
31
Issue
7
Pages
792-806
Language
en
Sources
crossref openalex

Cite