工业应用中,动态多变的流式数据特性使强化学习算法在训练过程中很难在模型收敛性与知识遗忘之间实现很好的平衡.考虑工业现场内容请求与当前生产任务具有高度相关性,提出一种基于集成深度Q网络算法(Integrated Deep Q-Network,IDQN)的自适应缓存策略.算法在离线阶段利用不同历史任务数据,训练并保存多个历史任务模型.在线阶段每当检测到实时数据流的任务特征发生变化,则重新训练网络模型.如果实时数据流的特征隶属于历史任务,则向深度Q网络(Deep Q-Network,DQN)导入相应的历史任务模型进行网络训练.否则直接利用实时数据流训练并标记为新的任务模型.仿真实验结果表明,IDQN与参考算法相比,在内容请求流行度动态变化时能够有效减少模型收敛时间,提高缓存效率.
An integrated DQN-based adaptive cache algorithm for edge computing application
In industrial applications,reinforcement learning can hardly achieve a good trade-off between model convergence and knowledge forgetting during the training process,given the dynamic and variable characteristics of data streams.Since the content requests are highly correlated with current production tasks in industrial applications,an adaptive caching strategy based on integrated deep Q-network(IDQN)is proposed.It trains and saves multiple historical task models using different historical task data in the offline phase.In the online phase,the network model is retrained whenever the task features of the real-time data stream are changed.If the features of the real-time data stream are affiliated with any historical tasks,the corresponding historical task model is imported to the deep Q-network(DQN)for network training.Otherwise,the real-time data stream is directly used to train a new task model.The simulation results show that,compared with the reference algorithms,IDQN can effectively reduce the model convergence time and improve the caching efficiency even when the popularity of content requests changes dynamically.
国家科技期刊平台
NSTL
万方数据
