A Knowledge Distillation Method Based on Multi-layer Feature Alignment
Real-time target detection algorithms(e.g.,YOLO)are designed to efficiently perform object detection tasks on resource-limited edge devices.However,their detection performance is often limited.To address this challenge,a knowledge distillation method based on multi-layer feature alignment is proposed.In order to effectively retain the knowledge in the original data,a distillation metric that incorporates multiple intermediate layers of knowledge from the teacher and student models is introduced,and an alignment weighting factor is incorporated based on the differences between the intermediate layer features of the teacher model and the student model during the training process.Compared with existing knowledge distillation methods,this method enables the student model to learn more useful knowledge from the middle layer of the teacher model.The refined knowledge is used to incrementally train the existing model,avoiding the resource overhead of training multiple independent models.Through experimental comparisons under different scenarios and conditions,this method effectively improves the accuracy of target recognition while reducing the computational and storage costs of the models.The experimental analyses show that the proposed multilayer feature alignment distillation algorithm based on the YOLO model is validated by the COCO2017 dataset,the detection accuracy of the student model is improved from 33.3 to 40.7,the detection accuracy of the model is effectively improved.
国家科技期刊平台
NSTL
万方数据
