Cross-national harmonization of neurocognitive assessment across five sites in a global study

Objective: Cross-national work on neurocognitive testing has been characterized by inconsistent findings, suggesting the need for improved harmonization. Here, we describe a prospective harmonization approach in an ongoing global collaborative study. Method: Visuospatial N-Back, Tower of London (ToL), Stop Signal task (SST), Risk Aversion (RA), and Intertemporal Choice (ITC) tasks were administered to 221 individuals from Brazil, India, the Netherlands, South Africa, and the USA. Prospective harmonization methods were employed to ensure procedural similarity of task implementation and processing of derived task measures across sites. Generalized linear models tested for between-site differences controlling for sex, age, education, and socioeconomic status (SES). Associations with these covariates were also examined and tested for differences by site with site-by-covariate interactions. Results: The Netherlands site performed more accurately on N-Back and ToL than the other sites, except for the USA site on the N-Back. The Netherlands and the USA sites performed faster than the other three sites during the go events in the SST. Finally, the Netherlands site also exhibited a higher tolerance for delay discounting than other sites on the ITC, and the India site showed more risk aversion than other sites on the RA task. However, effect size differences across sites on the five tasks were generally small (i.e., partial eta-squared < 0.05) after dropping the Netherlands (on ToL, N-Back, ITC, and SST tasks) and India (on the RA task). Across tasks, regardless of site, the N-Back (sex, age, education, and SES), ToL (sex, age, and SES), SST (age), and ITC (SES) showed associations with covariates. Conclusions: Four out of the five sites showed only small between-site differences for each task. Nevertheless, despite our extensive prospective harmonization steps, task score performance deviated from the other sites in the Netherlands site (on four tasks) and the India site (on one task). Because the procedural methods were standardized across sites, and our analyses were adjusted for covariates, the differences found in cognitive performance may indicate selection sampling bias due to unmeasured confounders. Future studies should follow similar cross-site prospective harmonization procedures when assessing neurocognition and consider measuring other possible confounding variables for additional statistical control.