基于注意力机制和知识蒸馏的电影评分预测
MOVIE RATING PREDICTION BASED ON ATTENTION MECHANISM AND KNOWLEDGE DISTILLATION
刘彤 1于思洁 1倪维健1
作者信息
- 1. 山东科技大学计算机科学与工程学院 山东青岛 266590
- 折叠
摘要
针对简单的因子分解机模型(Factorization Machines,FM)对于高阶交互的时间复杂度高和神经网络解决复杂问题尺寸过大问题,以电影评分预测为例,提出一种注意力机制和知识蒸馏的深度网络预测模型(Knowledge Distillation Attention Deep Network,K-ADN).结合注意力网络区分交互特征的重要度而得到注意力值,利用深度神经网络(Deep Neural Networks,DNN)处理高阶特征组合,建立神经网络模型作为教师模型,从知识蒸馏技术出发,以教师模型确保精确度,以学生模型精简模型尺寸,以求获得更有效的评分预测结果.以豆瓣电影为数据来源进行的实验结果表明,该模型预测的精确度有所提高,通过知识蒸馏后参数量减少86％.
Abstract
In view of the high time complexity of high-order interaction and the large size of complex problem solved by neural network in the simple factorization machines(FM),a knowledge distillation attention deep network(K-ADN)model is proposed based on the movie score prediction as an example.Combined with the attention network to distinguish the importance of interactive features,the attention value was obtained.Deep neural networks(DNN)were used to deal with the combination of high-order features,and the neural network model was established as the teacher model.Starting from the knowledge distillation technology,the teacher model was used to ensure the accuracy,and the student model was used to simplify the model size,so as to obtain more effective scoring prediction results.The experimental results based on Douban movie show that the accuracy of the model is improved,and the parameters are reduced by 86％after knowledge distillation.
关键词
电影评分/深度神经网络/注意力网络/评分预测/知识蒸馏Key words
Movie rating/Deep neural network/Attention network/Scoring prediction/Knowledge distillation引用本文复制引用
基金项目
国家自然科学基金项目(71704096)
国家自然科学基金项目(61602278)
青岛社科规划项目(QDSKL2001117)
出版年
2024
国家科技期刊平台
NSTL
万方数据