Reinforcement Recommendation System Based on Causal Mechanism Constraint
The application of historical data for training reinforcement learning recommendation systems is currently gaining attention from researchers.However,historical data leads to the incorrect estimation of state-actions in reinforcement learning models,resulting in data biases such as popularity and selection biases.The reason for this is that the distribution of historical data is inconsistent with the data collected by reinforcement learning strategies,and the historical data itself exhibits bias.To address this challenge,the use of causal mechanisms has proven effective in resolving data bias while constraining the distribution of data collected through policies.This paper proposes a reinforcement recommendation system based on causal mechanism constraint,comprising a causal mechanism constraint module and a comparison strategy module.The causal mechanism constraint module serves to limit the sample space that recommendation strategies can choose,thereby reducing errors in policy and data distributions.Notably,the causal mechanism constraint module considers the dynamic changes in the distribution of item popularity over time to alleviate popularity bias.Simultaneously,the comparison strategy module mitigates the impact of selection bias by balancing the importance of positive and negative samples.Experimental results on real datasets Ciao and Epinions show that,in comparison to Deep Q Network(DQN)-r,GAIL,SOFA,etc.,this algorithm exhibits superior accuracy and diversity.Moreover,the model with the causal constraint module improves the F-measure index by 2％and 3％,respectively,compared to the model without the causal constraint module,further verifying the effectiveness of the causal constraint module.
国家科技期刊平台
NSTL
万方数据
