首页|规则耦合下的多异构子网络MADDPG博弈对抗算法

规则耦合下的多异构子网络MADDPG博弈对抗算法

扫码查看
原文链接
  • 万方数据
  • 维普
针对多无人机博弈对抗过程中无人机数量动态衰减问题和传统深度强化学习算法中的稀疏奖励问题及无效经验抽取频率过高问题,本文以攻防能力及通信范围受限条件下的多无人机博弈对抗任务为研究背景,构建了红、蓝两方无人机群的博弈对抗模型,在多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient,MADDPG)算法的Actor-Critic框架下,根据博弈环境的特点对原始的MADDPG算法进行改进.为了进一步提升算法对有效经验的探索和利用,本文构建了规则耦合模块以在无人机的决策过程中对Actor网络进行辅助.仿真实验表明,本文设计的算法在收敛速度、学习效率和稳定性方面都取了一定的提升,异构子网络的引入使算法更适用于无人机数量动态衰减的博弈场景;奖励势函数和重要性权重耦合的优先经验回放方法提升了经验差异的细化程度及优势经验利用率;规则耦合模块的引入实现了无人机决策网络对先验知识的有效利用.
MADDPG game confrontation algorithm of polyisomer network based on rule coupling based on rule coupling
In order to overcome of dynamic attenuation of the number of UAVs in the process of multi-UAV game con-frontation,and solve the sparse reward problem in the traditional deep reinforcement learning algorithm and the high frequency of invalid experience extraction,a game model of red and blue UAV clusters is built in this paper based on the background of multi-unmanned aerial vehicles(Multi-UAVs)game with limited attack and defense capabilities and communication range.Under the Actor-Critic framework of multi-agent deep deterministic policy gradient(MADDPG)algorithm,the original MADDPG algorithm is improved according to the characteristics of the game scenario to solve the problem of the number attenuation,sparse rewards and high extraction frequency of invalid experience of UAVs in the original algorithm.On this basis,in order to improve the exploration and utilization of algorithm for effective experi-ences,a rule coupling module is built to assist UAV.The simulation experiment shows that the algorithm designed in this paper has improved the convergence speed,learning efficiency and stability.The use of polyisomer network makes the algorithm more suitable for the game scenario that the number of UAVs declines dynamically;the reward potential function and the priority experience playback method based on the importance weight coupling improve the degree of refinement of experience difference and the utilization rate of superior experience;the introduction of rule coupling module realizes the effective utilization of UAV decision network for priori knowledge.

deep reinforcement learningmulti-UAVsgame confrontationMADDPGActor-Criticrule couplingex-perience replaysparse rewards

张钰欣、赵恩娇、赵玉新

展开 >

哈尔滨工程大学 智能科学与工程学院, 黑龙江 哈尔滨 150001

深度强化学习 多无人机 博弈对抗 MADDPG Actor-Critic 规则耦合 经验回放 稀疏奖励

国家自然科学基金黑龙江省自然科学基金重庆市教委科学技术研究计划中国博士后科学基金面上项目黑龙江省博士后基金面上项目

61903099LH2020F025KJZD-K202004702021M690812LBH-Z21048

2024

10.11992/tis.202303037

智能系统学报
中国人工智能学会　哈尔滨工程大学

智能系统学报

CSTPCD北大核心
影响因子：0.672
ISSN：1673-4785
年,卷(期)：2024.19(1)
  • 35