首页|FSS:algorithm and neural network accelerator for style transfer

FSS:algorithm and neural network accelerator for style transfer

扫码查看
原文链接
  • 万方数据
  • 维普
Neural networks(NNs),owing to their impressive performance,have gradually begun to dom-inate multimedia processing.For resource-constrained and energy-sensitive mobile devices,an efficient NN accelerator is necessary.Style transfer is an important multimedia application.However,existing arbitrary style transfer networks are complex and not well supported by current NN accelerators,limiting their appli-cation on mobile devices.Moreover,the quality of style transfer needs improvement.Thus,we design the FastStyle system(FSS),where a.novel algorithm andan NN accelerator are proposed for style transfer.In FSS,we first propose a novel arbitrary style transfer algorithm,FastStyle.We propose a light network that contributes to high quality and low computational complexity and a prior mechanism to avoid retraining when the style changes.Then,we redesign an NN accelerator for FastStyle by applying two improvements to the basic NVIDIA deep learning accelerator(NVDLA)architecture.First,a flexible dat FSM and wt FSM are redesigned to enable the original data path to perform other operations(including the GRAM operation)by software programming.Moreover,statistics and judgment logic are designed to utilize the continuity of a video stream and remove the data dependency in the instance normalization,which improves the accelerator performance by 18.6％.The experimental results demonstrate that the proposed FastStyle can achieve higher quality with a lower computational cost,making it more suitable for mobile devices.The proposed NN ac-celerator is implemented on the Xilinx VCU118 FPGA under a 180-MHz clock.Experimental results show that the accelerator can stylize 512x512-pixel video with 20 FPS,and the measured performance reaches up to 306.07 GOPS.The ASIC implementation in TSMC 28 nm achieves about 22 FPS in the case of a 720-p video.

neural network acceleratorstyle transferneural networkdeep learning

Yi LING、Yujie HUANG、Yujie CAI、Zhaojie LI、Mingyu WANG、Wenhong LI、Xiaoyang ZENG

展开 >

State Key Laboratory of ASIC & System,Fudan University,Shanghai 200120,China

Shanghai ExploreX Technology Co.,Ltd.,Shanghai 200120,China

National Natural Science Foundation of ChinaState Key Laboratory of ASIC and SystemZhuhai Fudan Innovation Institute,Key R&D program of Shandong Province

620740412021KF0092022CXGC010504

2024

10.1007/s11432-022-3676-2

中国科学:信息科学(英文版)
中国科学院

中国科学:信息科学(英文版)

CSTPCDEI
影响因子：0.715
ISSN：1674-733X
年,卷(期)：2024.67(2)
  • 50