Abstract
By a News Reporter-Staff News Editor at Robotics & Machine Learning Daily News Daily News-New research on intelligent systems is the subject of a new report. According to news reporting from Anhui Agricultura l University by NewsRx journalists, research stated, "Knowledge distillation can transfer the knowledge from the pre-trained teacher model to the student model, thus effectively accomplishing model compression." Funders for this research include Anhui Provincial Key Research And Development Plan; Independent Project of Anhui Key Laboratory of Smart Agricultural Technolo gy And Equipment. The news editors obtained a quote from the research from Anhui Agricultural Univ ersity: "Previous studies have carefully crafted knowledge representation, targe ting loss function design, and distillation location selection, but there have b een few studies on the role of classifiers in distillation. Previous experiences have shown that the final classifier of the model has an essential role in maki ng inferences, so this paper attempts to narrow the gap in performance between m odels by having the student model directly use the classifier of the teacher mod el for the final inference, which requires an additional projector to help match features of the student encoder with the teacher's classifier. However, a singl e projector cannot fully align the features, and integrating multiple projectors may result in better performance. Considering the balance between projector siz e and performance, through experiments, we obtain the size of projectors for dif ferent network combinations and propose a simple method for projector integratio n. In this way, the student model undergoes feature projection and then uses the classifiers of the teacher model for inference, obtaining a similar performance to the teacher model."
NETL
NSTL