Convolutional Neural Network Model Compression Method Based on Cloud Edge Collaborative Subclass Distillation
In the current training and distribution process of convolutional neural network models,the cloud has sufficient compu-ting resources and datasets,but it is difficult to cope with the demand for fragmentation in edge scenes.The edge side can directly train and infer models,but it is difficult to directly use the convolutional neural network models trained in the cloud according to unified rules.To address the issue of low training and inference effectiveness of convolutional neural network algorithms for model compression in the context of limited resources on the edge side,a model distribution and training framework based on cloud edge collaboration is firstly proposed.This framework can combine the advantages of both cloud and edge sides for model retraining,meeting the edge's requirements for specified recognition targets,specified hardware resources,and specified accuracy.Secondly,based on the training approach of the cloud edge collaborative framework,new subclass knowledge distillation methods based on logits and channels(SLKD and SCKD)are proposed to improve knowledge distillation technology.The cloud server first provides a model with multi-target recognition,and then through the subclass knowledge distillation method,the model is re-trained on the edge side into a lightweight model that can be deployed in resource limited scenarios.Finally,the effectiveness of the joint training framework and the two subcategory distillation algorithm are validated on the CIFAR-10 dataset.The experi-mental results show that at a compression ratio of 50％,the inference accuracy is improved by 10％to 11％compared to models with full classification.Compared to the retraining of the model,the accuracy of the model trained through knowledge distillation method has also been greatly improved,and the higher the compression ratio,the more significant the improvement in model accu-racy.
万方数据
维普
