EEG-fNIRS dataaugmentation based on the modified conditional-generative adversarial network
Objective Electroencephalogram-functional near-infrared spectroscopy (EEG-fNIRS) multimodal brain-computer interface based deep learning has a wide application prospect in rehabilitation engineering,but it faces the problem of insufficient data. In order to solve the problem of small amount of data when EEG-fNIRS multi-module brain-computer interface is combined with deep learning,this paper proposes an EEG-fNIRS multi-module signal data augmentation method based on modified conditional generative adventive network ( CGAN ) . Methods Firstly, EEG and fNIRS data were preprocessed by filtering, normalization and downsampling. Then,according to the non-stationary characteristics of EEG,self-attention mechanism was added in CGAN generator and discriminator to obtain the EEG data augmented model CGANE,which strengthened the ability to capture and learn time-varying critical information. At the same time,an up-sampling convolution layer was added to the CGAN generator and discriminator to obtain the fNIRS data augmented model CGANf to compensate for the low sampling rate and insufficient information of fNIRS. The conditional information of CGANE and CGANf was set as class label. Furthermore,CGANE and CGANf were used to amplify each channel EEG and fNIRS, respectively. Multi-channel EEG augmentaded data and multi-channel fNIRS [ including oxygenated hemoglobin concentration( HbO) and deoxygenated hemoglobin concentration( HbR) ] augmentaded data were sequentially fused to obtain EEG-fNIRS multimodal augmented data. Finally, data augmentation experiments were performed on the first six subjects of the public EEG-fNIRS multi-modal signal dataset TU-Berlin-A,and a one-dimensional convolutional neural network classifier was designed to evaluate the quality of augmented data. Results Experimental studies based on left and right hand motion imagery data from the first six subjects in the EEG-FNIRS multimodal signal open dataset TU-Berlin-A showed that when the data were enlarged by a factor of 5,the average classification accuracy of our method was 94. 81％. Conclusions CGANE and CGANf can generate EEG and fNIRS signals close to the real data distribution, which verifies the effectiveness of the improved CGAN and the EEG-fNIRS multimodal data augmentation method proposed in this paper.
万方数据
维普
