Enhancing Visual Representation Through Brain-machine Fusion:from Brain-in-the-loop Modeling to Brain-out-of-the-loop Application
Based on the deep neural network features and EEG responses at the visual cortex of rhesus monkeys,an adaptive"brain-in-the-loop"fusion model of image representation is proposed,it is demonstrated that the brain-machine fusion can provide complementary information and enhance the performance of deep neural network model.Two kinds of brain-machine fusion computation mode based on brain response reconstruction and sharing representation space are proposed to accomplish the"brain-in-the-loop modeling and brain-out-of-the-loop application",the application scenario of brain-machine fusion model is broadened.The effectiveness of sharing representation is proved by visualization method of feature saliency,a new thought is provided to make full use of neural response of brain visual representation.
NETL
NSTL
万方数据
