Reproducibility in the absence of selective reporting: an illustration from large‐scale brain asymmetry research

Kong, Xiang-Zhen, ENIGMA Laterality Working Group, , and Franks, Clyde (2022) Reproducibility in the absence of selective reporting: an illustration from large‐scale brain asymmetry research. Human Brain Mapping, 43 (1). pp. 244-254.

[img]
Preview
PDF (Published version) - Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview
View at Publisher Website: https://doi.org/10.1002/hbm.25154
 
7
629


Abstract

The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p‐hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left–right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta‐analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an “ideal publishing environment,” that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically‐used sample sizes.

Item ID: 68004
Item Type: Article (Research - C1)
ISSN: 1065-9471
Copyright Information: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2020 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.
Funders: Max Planck Society (MPS)
Date Deposited: 11 May 2021 04:20
FoR Codes: 32 BIOMEDICAL AND CLINICAL SCIENCES > 3209 Neurosciences > 320903 Central nervous system @ 100%
SEO Codes: 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280103 Expanding knowledge in the biomedical and clinical sciences @ 100%
Downloads: Total: 629
Last 12 Months: 91
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page