深度神经网络架构轻量化方法综述
An Overview on Lightweight Methods for Deep Neural Network Architecture
林冲 1范加利 2闫文君 3陈姮 1杨颖1
作者信息
- 1. 中国人民解放军91206部队,山东青岛 266109
- 2. 海军航空大学青岛校区,山东青岛 266041
- 3. 海军航空大学信息融合研究所,山东烟台 264001
- 折叠
摘要
当前,深度神经网络作为学术界和工业界最受关注的研究方向之一,倍受广大科研人员青睐,但是存在架构很复杂、参数量巨大、计算成本、存储成本过高的缺点.因此,如何在保证神经网络性能可接受的情况下对其去冗余、实现轻量化设计成为热点问题.当前,各种轻量化方法如雨后春笋般涌现,为给希望利用轻量化神经网络解决具体问题的研究人员建立对网络轻量化方法的整体认识、快速选择合适的解决方案,文中对具有代表性的架构轻量化方法进行介绍:剪枝、架构搜索、知识蒸馏以及轻量化卷积核设计,并从不同角度对比分析各种方法优劣,最后在宏观层面展望神经网络轻量化的未来发展方向.
Abstract
At present,as one of the most concerned research directions in academia and industry,deep neural network is favored by most researchers,but it has the disadvantages of complex architecture,huge number of parameters,high computing cost and high storage cost.Therefore,how to reduce redundancy and realize lightweight design of neural network with acceptable performance becomes a hot issue.At present,various lightweight methods are springing up.In order to help researchers who want to use light-weight neural networks to solve specific problems to establish an overall understanding of network light-weight methods and quickly choose appropriate solutions,this paper introduces representative architecture lightweight methods:Pruning,architecture search,knowledge distillation and lightweight convolutional kernel design are compared and analyzed from different perspectives.Finally,the future development di-rection of neural network lightweight is prospected at the macro level.
关键词
深度神经网络/深度神经网络轻量化/神经网络架构轻量化/轻量化网络Key words
deep neural networks/deep neural networks lightweight/neural networks architecture light-weight/lightweight network引用本文复制引用
基金项目
国家自然科学基金面上项目(62371645)
山东省"泰山学者"建设工程专项(ts201511020)
山东省高等学校青年创新团队发展计划(2022kj084)
海军航空大学青年基金(H3202209003)
出版年
2024
NETL
NSTL
万方数据