多模态高精度非线性激活函数协处理器设计
Coprocessor for Multi-Mode High Precision Nonlinear Activation Function
欧文辉 1王峥 2吴卓宇 3王伟伦 4甘志银1
作者信息
- 1. 华中科技大学武汉光电国家研究中心 武汉 430074
- 2. 中国科学院深圳先进技术研究院异构智能计算中心 深圳 518055;中科元物芯科技有限公司 深圳 518066
- 3. 中国科学院深圳先进技术研究院异构智能计算中心 深圳 518055;中科元物芯科技有限公司 深圳 518066;Department of Engineering,Durham University Durham DH1 3LE UK
- 4. 中国科学院深圳先进技术研究院异构智能计算中心 深圳 518055
- 折叠
摘要
针对片上部署非线性激活函数产生的精度损失以及硬件资源开销大的问题,提出一种基于三分法指数方法的多模态高精度非线性激活函数协处理器设计.首先分析激活函数在不同拟合参数下的近似误差以及运算量,为设计提供指导;然后设计一种模块化的硬件框架,通过复用指数、对数、sigmoid模块并结合浮点计算单元,能够以较低的面积开销部署多种激活函数.在Xilinx的Vertix系列FPGA上完成原型测试,实验结果表明,在仅增加32个查找表的情况下,所提设计tanh和sigmoid的近似误差仅为2项拆分指数方法的65.02％和69.00％,同时拟合范围扩大60％;与高精度分段线性逼近方法相比,该设计在仅用4％的查找表数量的情况下,将近似误差缩小82％.
Abstract
The deployment of nonlinear activation functions on a chip is suffering from accuracy loss and hardware resource overhead.To address these problems,a multi-mode high-precision coprocessor design framework for nonlinear activation functions is proposed and which is based on the three-split exponential method.In the first stage,the approximation error and the operation workload of nonlinear activation func-tions on different approximation parameters are analyzed to guide the design.In the second stage,a modular hardware framework is designed to deploy several nonlinear activation functions at a low cost by reusing exponential,logarithmic,and sigmoid modules and combining them with floating-point computation units.The prototype of the proposed framework has been implemented on Xilinx Vertix series FPGA.The experi-mental results show that with only 32 additional lookup table entries than the two-split exponential methods,the approximation error of tanh and sigmoid is 65.02％and 69.00％of that using the two-split method,and the fitting range is extended by 60％.Compared with the high-precision piecewise linear approximation method,the design has led to an 82％reduction in approximation error with only 4％of the number of lookup tables in use.
关键词
非线性激活函数/神经网络/数学拟合/FPGAKey words
nonlinear activation functions/neural network/mathematical fitting/FPGA引用本文复制引用
基金项目
广东省重点领域研发计划(2019B010155003)
广东省基础与应用基础研究基金(2020B1515120044)
出版年
2024
NETL
NSTL
万方数据