首页|基于通道性能度量的神经网络结构搜索算法

基于通道性能度量的神经网络结构搜索算法

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
卷积神经网络的表征与预测能力往往依赖结构合理性,但其主流结构均由人工设计,存在设计难度高、算力要求强、时间开销大等问题.如何让神经网络自主搜索合理结构并节约计算资源是当前的研究重点.目前,基于部分通道连接的可微分结构搜索算法,以其高效的显存利用率在搜索速度和分类性能上表现良好.然而,其针对通道的随机采样策略易造成重要信息丢失,当通道连接不足时性能明显下降.为此,提出一种基于通道性能度量的神经网络结构搜索算法,利用注意力机制提取通道重要性系数,并以此对通道进行排序采样.此外,考虑到预热阶段导致搜索不充分,产生较大离散化误差,在结构权重连续化的过程中设计温度正则化系数,提升权重差异.实验表明,所提算法能够在节约计算资源的基础上搜索出更优的卷积神经网络结构.
Differentiable neural architecture search with channel performance measurement
The design of convolutional neural network architecture is critical for achieving high characterization and prediction performance.However,manual design can be time-consuming and computationally expensive.Automatic architecture search methods are thus becoming popular.Presently,partial channel connections for memory-efficient differentiable architecture search being efficient in terms of search speed,memory utilization,and classification performance.However,the random sampling strategy can cause the loss of important information.To address these issues,this paper proposes differentiable neural architecture search with channel performance measurement,which extracts importance coefficients of channels with attention mechanism and ranks the channels accordingly to select the most important ones.The proposed algorithm also includes a temperature regularization coefficient to improve the transformation from discrete to continuous architecture weight and reduce discretization errors.Experimental results show that the proposed algorithm achieves good performance while significantly reducing computational resources.

convolutional neural networkdifferentiable architecture searchchannel attentionchannel sortingdiscretization errortemperature regularization coefficient

潘杰、郑学驰、邹筱瑜

展开 >

中国矿业大学信息与控制工程学院,江苏徐州 221116

中国矿业大学机电工程学院,江苏徐州 221116

卷积神经网络 可微分结构搜索 通道注意力 通道排序 离散化误差 温度正则化系数

国家自然科学基金国家自然科学基金国家自然科学基金中央高校基本科研业务费专项江苏省高等学校优势学科建设工程项目

6217625862273349521741522021YCPY0111

2024

10.13195/j.kzyjc.2023.0257

控制与决策
东北大学

控制与决策

CSTPCD北大核心
影响因子：1.227
ISSN：1001-0920
年,卷(期)：2024.39(7)
  • 6