Multi-language Hate Speech Recognition Based on Multi-teacher Knowledge Distillation
Hate speech recognition on social media is a critical task in the field of open-source intelligence.To address the poor recognition performance of multilingual text models and the high computational resource require-ments of pre-trained models,we propose a multi-teacher knowledge distillation scheme.First,several large language models are used to obtain probability distribution matrices.Then,comprehensive soft labels are generated based on integrated general relevance weights and language-specific advantage weights to guide the student model training.Experimental results show that the student model distilled in this way can significantly reduce computation time and save computational resources while inheriting the language-specific advantages of each teacher model.
hate speech recognitionmultilingual textknowledge distillationlarge language models
NETL
NSTL
万方数据
