Research on Pruning Technology for Vision Transformer Model
This article focuses on the pruning technology research of the Vision Transformer(ViT)model,exploring the pruning problem of QKV(Query,Key,Value)weights and Fully Connected(FC)weights in the multi head self attention mechanism.This article proposes three pruning schemes for the ViT model:QKV pruning only,FC pruning only,and simultaneous pruning of QKV and FC to explore the effects of different pruning strategies on the accuracy and parameter compression of the ViT model.The research conducted in this article provides important references for the compression and optimization of deep learning models,and has guiding significance for model simplification and performance optimization in practical applications.
NETL
NSTL
万方数据
