首页|Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

扫码查看
原文链接
  • 万方数据
  • 维普
Unmanned aerial vehicles(UAVs)are recognized as effective means for delivering emergency communication services when terrestrial infrastructures are unavailable.This paper investigates a multi-UAV-assisted communication system,where we jointly optimize UAVs'trajectories,user association,and ground users(GUs)'transmit power to maximize a defined fairness-weighted throughput metric.Owing to the dynamic nature of UAVs,this problem has to be solved in real time.However,the problem's non-convex and combinatorial attributes pose challenges for conventional optimization-based algorithms,particularly in scenarios without central controllers.To address this issue,we propose a multi-agent deep reinforcement learning(MADRL)approach to provide distributed and online solutions.In contrast to previous MADRL-based methods considering only UAV agents,we model UAVs and GUs as heterogeneous agents sharing a common objective.Specifically,UAVs are tasked with optimizing their trajectories,while GUs are responsible for selecting a UAV for association and determining a transmit power level.To learn policies for these heterogeneous agents,we design a heterogeneous coordinated QMIX(HC-QMIX)algorithm to train local Q-networks in a centralized manner.With these well-trained local Q-networks,UAVs and GUs can make individual decisions based on their local observations.Extensive simulation results demonstrate that the proposed algorithm outperforms state-of-the-art benchmarks in terms of total throughput and system fairness.

unmanned aerial vehicle(UAV)trajectory designresource allocationmulti-agent deep rein-forcement learning(MADRL)heterogeneous agents

Xuanhan ZHOU、Jun XIONG、Haitao ZHAO、Xiaoran LIU、Baoquan REN、Xiaochen ZHANG、Jibo WEI、Hao YIN

展开 >

College of Electronic Science and Technology,National University of Defense Technology,Changsha 410073,China

Systems Engineering Institute,Academy of Military Sciences PLA,Beijing 100091,China

国家自然科学基金国家自然科学基金国家自然科学基金国家自然科学基金湖南省自然科学基金Science and Technology Innovation Program of Hunan Province

623714626193102062101569U19B20242022J J100682022RC1093

2024

10.1007/s11432-023-3906-3

中国科学:信息科学(英文版)
中国科学院

中国科学:信息科学(英文版)

CSTPCDEI
影响因子：0.715
ISSN：1674-733X
年,卷(期)：2024.67(3)
  • 55