一种改进的平均奖赏强化学习方法在RoboCup训练中的应用
A research of an improved R-Learning used in the training of RoboCup
李瑾 1刘全 1杨旭东 1杨凯 1翁东良1
作者信息
- 1. 苏州大学计算机科学与技术学院,江苏苏州215006
- 折叠
摘要
强化学习在人工智能领域中是一种重要的解决学习控制问题的方法.在强化学习中,平均奖赏类型的强化学习方法适用于解决具有循环特性或者不具有终结状态的问题,然而平均奖赏强化学习存在收敛速度慢、对参数和环境敏感等问题.针对平均奖赏强化学习收敛速度缓慢这一问题,提出了一种改进的平均奖赏强化学习方法.同时,为了处理大状态空间、提高泛化能力,算法采用神经网络作为近似函数.算法在RoboCup中实验的训练表明该算法具有较快的收敛速度和较强的泛化能力.
Abstract
Reinforcement learning has become a central paradigm for solving learning-control problems in artificial intelligence. In reinforcement learning, it is more natural and computationally advantageous to formulate tasks so that the controller's objective is to maximize the average payoff received per time step in many problems, for example that the optimal behavior is a limit cycle. However, R-Learning has some problems, such as converge slowly and sensitive with parameter. To solve the problem of slow convergence, a improved algorithm R-Learning is proposed. The algorithm uses BP as the approximate function to generalize the state space. The experimental results of RoboCup show that the proposed algorithm converges faster and has the ability of generalization.
关键词
平均奖赏/强化学习/Keepaway/RoboCupKey words
average reward/reinforcement learning/Keepaway/RoboCup引用本文复制引用
基金项目
国家自然科学基金(61070223)
国家自然科学基金(61103045)
国家自然科学基金(60970015)
国家自然科学基金(61170020)
江苏省自然科学基金(BK2009116)
江苏省高校自然科学研究计划(09KJA520002)
江苏省高校自然科学研究计划(09KJB520012)
出版年
2012
NETL
NSTL
维普
万方数据