We show that by using active learning, ASReview can lead to far more efficient reviewing than manual reviewing, while exhibiting adequate quality. Furthermore, the presented software is fully transparent and open source.
After manually screening 5050 studies, Smid et al included only 27 studies for their review. In the current study, both the Bayesian and logistic regression models found more than 80% of relevant publications after screening only 10% of all publications.
To determine the defaults we performed a simulation study and Naive Bayes + TF-IDF model performed the best.
The CORD-19 database is available in ASReview and can be used to search for relevant Corona-related publication using active learning.
The idea of combining human intelligence and machine learning can be found in machine learning techniques like active learning (AL) and Human-in-the-Loop (HITL) machine learning. In AL and HITL, the human classifies unlabeled items the model selects to learn from, e.g., items for which the model is uncertain. These techniques are proven to be effective for training models..
How A.I. may make sense of 50,000 coronavirus research papers BY JONATHAN VANIAN April 28, 2020 Some snapshots: A little over a month ago, the White House Office of Science and Technology Policy enlisted several big research groups and companies to collaborate on a major A.I. project intended to help experts unravel the…
The call ‘Corona: fast-track data’ is developed by the NWO to provide funding for projects that collect data that can only be conducted now, during the COVID-19 crisis, and specifically research into issues that arise in society during the crisis. The ASReview project was one of the awardees of these NWO fast grants with our…
