RoBERTa-based Recommendation for Reviews and Ratings
Recommendation systems based on user reviews and ratings of items are a hot research area.However,the word embedding part of the original recommender system is expensive to train and difficult to generalize.We propose in a novel neural network based on the pre-trained language model via an atten-tion mechanism for the reviews and ratings,named based on a Pre-trained model an Attention Neural Net-work(PANN).The paper utilizes the RoBERTa as the pre-trained model,which uses a dynamic masking strategy and byte level vocabularies for training.It solves the problem of multiple meanings of words,and can extract features from the input sequence with high generalization performance.The model employs two networks,users and items as its two networks,to train parameters and extract the features respectively,and finally use the factorization machine to interact two features to predict the user's interest in the item,in other words,the possible user ratings upon items.The experimental results demonstrate the proposed model outperforms conventional models on most datasets.
万方数据
维普
