首页|MADRL-based UAV swarm non-cooperative game under incomplete information

MADRL-based UAV swarm non-cooperative game under incomplete information

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Unmanned Aerial Vehicles(UAVs)play increasing important role in modern battlefield.In this paper,considering the incomplete observation information of individual UAV in complex combat environment,we put forward an UAV swarm non-cooperative game model based on Multi-Agent Deep Reinforcement Learning(MADRL),where the state space and action space are constructed to adapt the real features of UAV swarm air-to-air combat.The multi-agent particle environment is employed to generate an UAV combat scene with continuous observation space.Some recently popular MADRL methods are compared extensively in the UAV swarm non-cooperative game model,the results indicate that the performance of Multi-Agent Soft Actor-Critic(MASAC)is better than that of other MADRL methods by a large margin.UAV swarm employing MASAC can learn more effective policies,and obtain much higher hit rate and win rate.Simulations under different swarm sizes and UAV physical parameters are also performed,which implies that MASAC owns a well generalization effect.Furthermore,the practicability and conver-gence of MASAC are addressed by investigating the loss value of Q-value networks with respect to individual UAV,the results demonstrate that MASAC is of good practicability and the Nash equi-librium of the UAV swarm non-cooperative game under incomplete information can be reached.

UAV swarmReinforcement learningDeep learningMulti-agentNon-cooperative gameNash equilibrium

Ershen WANG、Fan LIU、Chen HONG、Jing GUO、Lin ZHAO、Jian XUE、Ning HE

展开 >

School of Electronic and Information Engineering,Shenyang Aerospace University,Shenyang 110136,China

College of Robotics,Beijing Union University,Beijing 100101,China

School of Engineering Science,University of Chinese Academy of Sciences,Beijing 100049,China

College of Smart City,Beijing Union University,Beijing 100101,China

展开 >

National Key R&D Program of ChinaNational Natural Science Foundation of ChinaAcademic Research Projects of Beijing Union University,ChinaAcademic Research Projects of Beijing Union University,ChinaAcademic Research Projects of Beijing Union University,ChinaAcademic Research Projects of Beijing Union University,ChinaSongShan Laboratory Foundation,ChinaApplied Basic Research Programs of Liaoning Province,ChinaApplied Basic Research Programs of Liaoning Province,ChinaSpecial Funds program of Civil Aircraft,ChinaSpecial Funds program of Shenyang Science and Technology,China

2018AAA010080462173237SK160202103ZK50201911ZK30202107ZK30202108YYJC0620220172022020502-JH2/10132022JH2/1013001500102022062706622-322-3-34

2024

10.1016/j.cja.2024.03.030

中国航空学报(英文版)
中国航空学会

中国航空学报(英文版)

CSTPCDEI
影响因子：0.847
ISSN：1000-9361
年,卷(期)：2024.37(6)
  • 5