首页|深度神经网络架构轻量化方法综述

深度神经网络架构轻量化方法综述

An Overview on Lightweight Methods for Deep Neural Network Architecture

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
当前,深度神经网络作为学术界和工业界最受关注的研究方向之一,倍受广大科研人员青睐,但是存在架构很复杂、参数量巨大、计算成本、存储成本过高的缺点.因此,如何在保证神经网络性能可接受的情况下对其去冗余、实现轻量化设计成为热点问题.当前,各种轻量化方法如雨后春笋般涌现,为给希望利用轻量化神经网络解决具体问题的研究人员建立对网络轻量化方法的整体认识、快速选择合适的解决方案,文中对具有代表性的架构轻量化方法进行介绍:剪枝、架构搜索、知识蒸馏以及轻量化卷积核设计,并从不同角度对比分析各种方法优劣,最后在宏观层面展望神经网络轻量化的未来发展方向.
At present,as one of the most concerned research directions in academia and industry,deep neural network is favored by most researchers,but it has the disadvantages of complex architecture,huge number of parameters,high computing cost and high storage cost.Therefore,how to reduce redundancy and realize lightweight design of neural network with acceptable performance becomes a hot issue.At present,various lightweight methods are springing up.In order to help researchers who want to use light-weight neural networks to solve specific problems to establish an overall understanding of network light-weight methods and quickly choose appropriate solutions,this paper introduces representative architecture lightweight methods:Pruning,architecture search,knowledge distillation and lightweight convolutional kernel design are compared and analyzed from different perspectives.Finally,the future development di-rection of neural network lightweight is prospected at the macro level.

deep neural networksdeep neural networks lightweightneural networks architecture light-weightlightweight network

林冲、范加利、闫文君、陈姮、杨颖

展开 >

中国人民解放军91206部队,山东青岛 266109

海军航空大学青岛校区,山东青岛 266041

海军航空大学信息融合研究所,山东烟台 264001

深度神经网络 深度神经网络轻量化 神经网络架构轻量化 轻量化网络

国家自然科学基金面上项目山东省"泰山学者"建设工程专项山东省高等学校青年创新团队发展计划海军航空大学青年基金

62371645ts2015110202022kj084H3202209003

2024

10.3969/j.issn.1673-5692.2024.02.010

中国电子科学研究院学报
中国电子科学研究院

中国电子科学研究院学报

影响因子：0.663
ISSN：1673-5692
年,卷(期)：2024.19(2)
  • 74