(A.M.)

Background Brain-Computer Interface BCI is becoming more reliable thanks to the advantages of Artificial Intelligence AI Recently hybrid Deep Learning hDL which combines different DL algorithms has gained momentum over the past five years In this work we proposed a review on hDL-based BCI starting from the seminal studies in 2015 Objectives We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic Methods We have queried four scientific search engines Google Scholar PubMed IEEE Xplore and Elsevier Science Direct and different data items were extracted from each paper such as the database used kind of application online offline training tasks used for the BCI pre-processing methodology adopted type of normalization used which kind of features were extracted type of DL architecture used number of layers implemented and which optimization approach were used as well All these items were then investigated one by one to uncover trends Results Our investigation reveals that Electroencephalography EEG has been the most used technique Interestingly despite the lower Signal-to-Noise Ratio SNR of the EEG data that makes pre-processing of that data mandatory we have found that the pre-processing has only been used in 21 28 of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data Temporal-features seem to be the most effective with 93 94 accuracy while spatial-temporal features are the most used with 33 33 of the cases investigated The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47 of the cases Moreover half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency Significance To give useful information to the scientific community we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly We have indicated a list of open challenges emphasizing the need to use neuroimaging techniques other than EEG such as functional Near-Infrared Spectroscopy fNIRS deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained To implement new combinations of architectures such as RNN-based and Deep Belief Network DBN-based it is necessary to better explore the frequency and temporal-frequency features of the data at hand

About this institution

Publications Show all

Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review