首页|基于FPGA的卷积神经网络和视觉Transformer通用加速器

基于FPGA的卷积神经网络和视觉Transformer通用加速器

扫码查看
针对计算机视觉领域中基于现场可编程逻辑门阵列(FPGA)的传统卷积神经网(CNN)络加速器不适配视觉Transformer网络的问题,该文提出一种面向卷积神经网络和Transformer的通用FPGA加速器.首先,根据卷积和注意力机制的计算特征,提出一种面向FPGA的通用计算映射方法;其次,提出一种非线性与归一化加速单元,为计算机视觉神经网络模型中的多种非线性和归一化操作提供加速支持;然后,在Xilinx XCVU37P FP-GA上实现了加速器设计.实验结果表明,所提出的非线性与归一化加速单元在提高吞吐量的同时仅造成很小的精度损失,ResNet-50和ViT-B/16在所提FPGA加速器上的性能分别达到了589.94 GOPS和564.76 GOPS.与GPU实现相比,能效比分别提高了5.19倍和7.17倍;与其他基于FPGA的大规模加速器设计相比,能效比有明显提高,同时计算效率较对比FPGA加速器提高了8.02%~177.53%.
FPGA-Based Unified Accelerator for Convolutional Neural Network and Vision Transformer
Considering the problem that traditional Field Programmable Gate Array(FPGA)-based Convolutional Neural Network(CNN)accelerators in computer vision are not adapted to Vision Transformer networks,a unified FPGA accelerator for convolutional neural networks and Transformer is proposed.First,a generalized computation mapping method for FPGA is proposed based on the characteristics of convolution and attention mechanisms.Second,a nonlinear and normalized acceleration unit is proposed to provide acceleration support for multiple nonlinear operations in computer vision networks.Then,we implemented the accelerator design on Xilinx XCVU37P FPGA.Experimental results show that the proposed nonlinear acceleration unit improves the throughput while causing only a small accuracy loss.ResNet-50 and ViT-B/16 achieved 589.94 GOPS and 564.76 GOPS performance on the proposed FPGA accelerator.Compared to the GPU implementation,energy efficiency is improved by a factor of 5.19 and 7.17,respectively.Compared with other large FPGA-based designs,the energy efficiency is significantly improved,and the computing efficiency is increased by 8.02%~177.53%compared to other FPGA accelerators.

Computer visionConvolutional Neural Network(CNN)TransformerField Programmable Gate Array(FPGA)Hardware accelerator

李天阳、张帆、王松、曹伟、陈立

展开 >

信息工程大学信息技术研究所 郑州 450002

空军95072部队 南宁 530000

复旦大学大数据研究院 上海 200433

计算机视觉 卷积神经网络 Transformer FPGA 硬件加速器

国家重点研发计划

2022YFB4500900

2024

电子与信息学报
中国科学院电子学研究所 国家自然科学基金委员会信息科学部

电子与信息学报

CSTPCD北大核心
影响因子:1.302
ISSN:1009-5896
年,卷(期):2024.46(6)