The Mega Meta project: Substance use, anxiety and depression
The Center for Urban Mental Health at the AMC is cooperating with ASReview for an enormous, systematic review and meta-analysis (> 165.046 hits) to find out what factors and interaction of factors contribute to the onset, maintenance, and relapse of anxiety-, depressive-, and substance use disorders.
Some highlights:
- A team of 7 students and researchers screened 11.571 papers in a period of only 5 months!
- ASReview ran via a server so that everyone could screen at the same time.
- A part of the literature was also screened using a 17-layer CNN model developed by a former student of the master Applied Data Science, which is pretty cool!
- For post-processing the results hundreds of lines of code were written: DOI-retrieval, deduplication, merging different files – you name it.
- The next and ongoing step is data extraction to run some amazing meta-analysis.
All codebooks, search strings, scripts, and much more is published so that anyone can benefit from the work. More details below:
- Pre-registration
- Search protocol
- Screening protocol
- Server Setup
- Hyperparameter tuning
- Post-processing
- Data
- Pre-print article
This project is funded by a grant from the Centre for Urban Mental Health, University of Amsterdam, The Netherlands
Pre-registration
The following PROSPERO pre-registration was created for the MegaMeta project. The pre-registration describes the review question, the types of studies to be included, population, interventions, exposure(s) and much more.
Find the details in the PROSPERO pre-registration:
Brouwer, M., Wiers, R., van den Brand, S., Hofstee, L., van de Schoot, R., Bockting, C. (2021). Reviewing factors that contribute to substance use, anxiety and depressive disorders. PROSPERO: CRD42021266297
Search protocol
The search protocol repository contains the procedure for
- the search terms,
- the search query,
- selected key papers,
- de-duplication via Endnote.
Brouwer, M., Hofstee, L., de Boer, J., Weijdema, F., Lucassen, P., Sloot, P. M. A., Stronks, K., van Weert, J., Wiers, R., van de Schoot, R., & Bockting, C. (2021). Search Protocol for the Mega-Meta Study on Factors Contributing to Substance Use, Anxiety and Depressive Disorders. Open Science Framework. DOI: 10.17605/OSF.IO/M5UHY
Screening protocol
The screening protocol repository contains the procedure for screening the records identified in the search protocol.
Hofstee, L., Brouwer, M., Melnikov, V., & van de Schoot, R. (2021). Screening Protocol for the Mega-Meta Study on Factors Contributing to Substance Use, Anxiety and Depressive Disorders. Open Science Framework. DOI: 10.17605/OSF.IO/3ZNAR
Server Setup
To allow for continuation of the screening project between different screeners, a server was setup. Details on how this server was setup can be found in the GitHub repository, or zenodo publication
Melnikov, V. (2021). ASReview server setup for MegaMeta study. Zenodo. DOI:10.5281/zenodo.5768306
Hyperparameter tuning
This repository stores the scripts and plugins that were used for the creation of three final data files for screening titles and abstract in a second round. These final project files were created using a classifier based on a convolutional neural network, optimized using Optuna.
Teijema, J., & Van de Schoot, R. (2021). Hyperparameter-training for the Mega-Meta project. Zenodo. DOI: 10.5281/zenodo.5747050
Post processing
This repository stores the scripts to post-process the data after the title and abstract screening phase. Post-processing consists of the following steps.
- Merge the three output files after screening in ASReview;
- Obtain missing DOIs;
- Apply another round of de-duplication (the first round of de-duplication was applied before the screening started).
- Deal with noisy labels corrected in two rounds of quality checks;
van den Brand, S., Hofstee, L., Teijema, J., Melnikov, V., Kramer, B., Brouwer, M., & Van de Schoot, R. (2021). Scripts for Post-Processing Mega-Meta Screening Results. Zenodo. DOI: 10.5281/zenodo.5752358
Data
The data/output is available (under embargo) on the Dutch national centre of expertise and repository for research data:
- Intermediate Datasets containing the results of the search before and after deduplication, the ASReview project files of the three screening phases.
Brouwer, M. et al. (2022). Pre-processing data for the Mega Meta Project DANS. https://doi.org/10.17026/dans-29d-n6yg
2. Final dataset
-
- megameta_asreview_partly_labelled: A partly labeled database (n_relevant = 6,351; n_irrelevant= 4,411; n_unlabelled = 118,567) with the columns as described in Table 1. The data can be used for post-processing or can be imported into software to continue labeling using all previous labels as training data
- megameta_asreview_only_potentially_relevant: A dataset with only the relevant papers (n= 6,380). This database can be used to search for papers for a specific research question. We also created a dataset per domain (n_anxiety = 1522; n_depression = 2708; n_substance = 2306). Note that some papers are relevant for multiple domains.
Brouwer, M. et al. (2022). Final data for the mega meta project DANS. https://doi.org/10.17026/dans-z7w-9446
Pre-print
The entire process is described in much more detail in our pre-print:
Brouwer, M., Hofstee, L., van den Brand, S. A. G. E., Teijema, J., Ferdinands, G., de Boer, J., Weijdema, F., Kramer, B., Wiers, R., Bockting, C., de Bruin, J., van de Schoot, R. (2022, August 30). AI-aided Systematic Review to Create a Database with Potentially Relevant Papers on Depression, Anxiety, and Addiction. PsyArXiv. DOI:10.31234/osf.io/j6nqz
