模式识别与人工智能2024,Vol.37Issue(10) :851-872.DOI:10.16451/j.cnki.issn1003-6059.202410001

多智能体强化学习理论及其应用综述

A Review of Multi-agent Reinforcement Learning Theory and Applications

陈卓然 刘泽阳 万里鹏 陈星宇 朱雅萌 王成泽 程翔 张亚 张森林 王晓辉 兰旭光
模式识别与人工智能2024,Vol.37Issue(10) :851-872.DOI:10.16451/j.cnki.issn1003-6059.202410001

多智能体强化学习理论及其应用综述

A Review of Multi-agent Reinforcement Learning Theory and Applications

陈卓然 1刘泽阳 1万里鹏 1陈星宇 1朱雅萌 2王成泽 2程翔 3张亚 4张森林 5王晓辉 6兰旭光1
扫码查看

作者信息

  • 1. 西安交通大学人工智能与机器人研究所 西安 710049
  • 2. 中国运载火箭技术研究院 北京 100076
  • 3. 北京大学电子学院 北京 100871
  • 4. 东南大学自动化学院 南京 210096
  • 5. 浙江大学电气工程学院 杭州 310027
  • 6. 中国电力科学研究院人工智能研究所 北京 100192
  • 折叠

摘要

强化学习是一种用于解决序列决策问题的常用机器学习方法,核心思想是让智能体与环境交互获得反馈,从而逐步学会最佳策略.随着实际应用对计算能力和数据规模的要求不断提高,单体智能转向群体智能逐渐成为人工智能未来发展的必然趋势,这为强化学习带来诸多新的机遇和挑战.文中首先从深度多智能体强化学习概念着手,针对目前的理论困境,如可拓展性较差、效用分配较难、探索-利用困境、环境非稳态、信息部分可观测等问题,进行提炼和分析.然后,详细阐述目前学者对于这些问题提出的多种解决方法及其优缺点.最后,介绍当前多智能体强化学习的典型训练学习环境和智慧城市建设、游戏、机器人控制、自动驾驶等复杂决策领域的实际应用,并总结协作多智能体强化学习面临的挑战和未来发展方向.

Abstract

Reinforcement learning(RL)is a widely utilized machine learning paradigm for addressing sequential decision-making problems.Its core principle involves enabling agents to learn optimal policies iteratively through feedback derived from interactions between an agent and the environment.As the demands for computational power and data scale of practical applications continue to escalate,the transition from single-agent intelligence to collective intelligence becomes an inevitable trend in the future development of artificial intelligence.Therefore,challenges and opportunities are abundant for RL.In this paper,grounded on the concept of deep multi-agent reinforcement learning(MARL),the current theoretical dilemmas are refined and analyzed,including limited scalability,credit assignment,exploration-exploitation dilemma,non-stationarity and partial observability of information.Various solutions and their advantages and disadvantages proposed by researchers are elaborated.Typical training and learning environment of MARL and its practical applications in complex decision-making fields,such as smart city construction,gaming,robotics control and autonomous driving,are introduced.The challenges and future development direction of collaborative multi-agent reinforcement learning are summarized.

关键词

深度强化学习/多智能体/效用分配/人类反馈/马尔科夫决策过程

Key words

Deep Reinforcement Learning/Multi-agent/Credit Assignment/Human Feedback/Mark-ov Decision Process

引用本文复制引用

出版年

2024
模式识别与人工智能
中国自动化学会,国家智能计算机研究开发中心,中国科学院合肥智能机械研究所

模式识别与人工智能

CSTPCDCSCD北大核心
影响因子:0.954
ISSN:1003-6059
段落导航相关论文