ASReview CNN with hyperparameter optimisation

This repository contains an extention for ASReview containing a convolutional neural network (CNN) model that may be utilised during a systematic review with ASReview. This extention includes a model combining Naive Bayes (NB) and CNN classifiers, starting with Naive Bayes for the first set amount of iterations, and switching to a CNN thereafter. The current switchpoint is set at 500 iterations, but can be adjusted by the user. This CNN makes use of hyperparamater optimisation, which is set to repeat every 300 iterations. The preferred feature extraction strategy for this model is the wide-doc2vec feature extractor.

To read more about the rationale behind utilising two models within one systematic review, please consult the simulation report.

Getting started

Install the new classifiers with:

pip install .

or

python -m pip install git+https://github.com/BartJanBoverhof/asreview-cnn-hpo.git

Usage

The nb-cnn switch model is defined in asreviewcontrib/models/classifiers/cnn_switch.py and can be used with --model cnn-switch.

Performance

A simulation study assessing the performance of this model can be found in the included report. In short, no direct evidence was found in favor of the current implementation of the cnn-switch with wide-doc2vec to outperform already implemented models such nb and lr, however, a differently optimised model may provide to show potential (see also: discussion section of the report).

License

Apache-2.0 License