深度神经网络参数轻量化方法综述
An Overview on Lightweight Methods for Deep Neural Network Parameters
林冲 1闫文君 2纪纲 1于斌 1王莹1
作者信息
- 1. 中国人民解放军91206部队,山东青岛 266109
- 2. 海军航空大学信息融合研究所,山东烟台 264001
- 折叠
摘要
近年来,深度神经网络在各种具有挑战性的任务上取得了巨大的成功,不断刷新人们对人工智能的认识.但是,深度神经网络模型的参数量巨大、计算成本、存储成本过高,难以部署到资源受限的边缘计算设备中.因此,人们开始从网络的架构和参数量两个角度尝试对网络进行轻量化设计,同时保证神经网络性能可接受.本文从网络参数轻量化的角度出发,首先简要回顾深度神经网络发展历史和工作原理;其次,介绍当前主流的3类参数轻量化方法:参数量化、张量分解以及参数共享;然后,从思想描述、适用层级、训练方式等维度对比分析方法优劣;最后,对神经网络轻量化的未来发展方向进行展望.
Abstract
In recent years,deep neural networks have achieved great success on a variety of challenging tasks,constantly refreshing people's understanding of artificial intelligence.However,the deep neural network model is difficult to deploy to resource-limited edge computing devices because of its huge param-eters,high computing and storage costs.Therefore,people begin to try to lightweight network design from the perspective of network architecture and parameter number,while ensuring the performance of neural network is acceptable.In this paper,the development history and working principle of deep neural net-works are reviewed briefly from the angle of network parameter lightweight.Secondly,three kinds of cur-rent mainstream parameter lightweight methods are introduced:parameter quantization,tensor decomposi-tion and parameter sharing.Then the advantages and disadvantages of the method are compared and ana-lyzed from the dimensions of thought description,application level and training mode.Finally,the future development direction of neural network lightweight is prospected.
关键词
深度神经网络/人工智能/边缘计算设备/神经网络参数轻量化Key words
deep neural networks/artificial intelligence/edge computing devices/neural networks pa-rameters lightweight引用本文复制引用
基金项目
国家自然科学基金面上项目(62371645)
泰山学者工程专项基金(ts201511020)
山东省高等学校青年创新团队发展计划(2022kj084)
海军航空大学青年基金(H3202209003)
出版年
2024
NETL
NSTL
万方数据