脑机融合增强视觉表征:从"脑在环路建模"到"脑不在环路应用"
Enhancing Visual Representation Through Brain-machine Fusion:from Brain-in-the-loop Modeling to Brain-out-of-the-loop Application
全诗兰 1闫建璞 1张子元 1董明皓 1梁继民1
作者信息
- 1. 西安电子科技大学电子工程学院,西安 710126
- 折叠
摘要
基于深度神经网络特征及恒河猴大脑视觉皮层脑电响应,提出了基于自适应信息融合方法的"脑在环路"图像表征模型,证明了脑机融合可以提供互补信息并提升深度神经网络模型性能;提出了基于大脑响应重建和共享表征空间的两种脑机融合计算模式,实现了"脑在环路建模,脑不在环路应用",拓宽了脑机融合模型的应用场景;通过特征显著性可视化方法证明了共享表征有效性,为充分利用大脑视觉表征的神经响应提供了新思路.
Abstract
Based on the deep neural network features and EEG responses at the visual cortex of rhesus monkeys,an adaptive"brain-in-the-loop"fusion model of image representation is proposed,it is demonstrated that the brain-machine fusion can provide complementary information and enhance the performance of deep neural network model.Two kinds of brain-machine fusion computation mode based on brain response reconstruction and sharing representation space are proposed to accomplish the"brain-in-the-loop modeling and brain-out-of-the-loop application",the application scenario of brain-machine fusion model is broadened.The effectiveness of sharing representation is proved by visualization method of feature saliency,a new thought is provided to make full use of neural response of brain visual representation.
关键词
脑机融合/视觉表征/脑在环路/脑不在环路/深度神经网络Key words
brain-machine fusion/visual representation/brain-in-the-loop/brain-out-of-the-loop/deep neural networks引用本文复制引用
基金项目
国家自然科学基金(U19B2030)
国家自然科学基金(61976167)
出版年
2024
NETL
NSTL
万方数据