Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

扫码查看

原文链接

国家科技期刊平台
NETL
NSTL
万方数据

外文摘要：Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent rein-forcement learning(MARL).It is significantly more difficult for those tasks with latent variables that agents cannot directly observe.However,most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent.In this paper,we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders.It is called the multi-agent soft actor-critic with latent variable(MASAC-LV)algorithm,which uses varia-tional inference theory to infer the compact latent variables rep-resentation space from a large amount of offline experience.Besides,we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function.This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent.The proposed algorithm is evaluated on two collaboration tasks with confounders,and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

外文关键词：

Latent variable modelmaximum entropymulti-agent reinforcement learning(MARL)multi-agent system

作者：

Kun Jiang、Wenzhang Liu、Yuanda Wang、Lu Dong、Changyin Sun

展开 >

作者单位：

School of Automation,Southeast University,Nanjing 210096,China

School of Artificial Intelligence,Anhui University,Hefei 230601,China

School of Cyber Science and Engineering,Southeast University,Nanjing 211189,China

School of Automation,Southeast University,Nanjing 210096

Engineering Research Center of Autonomous Unmanned System Technology,Ministry of Education,Hefei 230601,China

展开 >

基金：

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of Chinathe"Zhishan"Scholars Programs of Southeast UniversityFundamental Research Funds for the Central Universities

项目编号：

62136008622360026192100462173251621031042242023K30034

出版年：

2024

DOI：

10.1109/JAS.2024.124281

自动化学报(英文版)

中国自动化学会,中国科学院自动化研究所,中国科技出版传媒股份有限公司

自动化学报(英文版)

CSTPCDEI

ISSN：2329-9266

年,卷(期)：2024.11(7)