基于Transformer的强化学习方法在智能决策领域的应用:综述

扫码查看

原文链接

NETL
NSTL
万方数据

中文摘要：强化学习已成为一种主导的决策范式,在许多现实应用中取得令人瞩目的成果.在大规模决策场景中,深度神经网络成为释放强化学习巨大潜力的关键所在.受自然语言和视觉领域中先进Transformer方法的启发,Transformer和强化学习的结合,突破了智能决策领域许多瓶颈.本文从基础模型、先进算法、代表性示例、典型应用和挑战分析等层面,归纳总结了基于Transformer的强化学习方法(TransRL),旨在深入分析当前强化学习方法的痛点,讨论TransRL如何突破强化学习范式的局限.据我们所知,本文是第一篇系统性回顾基于Transformer的强化学习方法在智能决策领域应用进展的综述,期望提供一个全面的TransRL讨论基础,推动强化学习在此领域的应用.为便于跟进TransRL的前沿进展,我们整理了最新相关论文及其开源项目,详见https://github.com/williamyuanv0/Transformer-in-Reinforcement-Learning-for-Decision-Making-A-Survey.

外文标题：Transformer in reinforcement learning for decision-making:a survey

外文摘要：Reinforcement learning(RL)has become a dominant decision-making paradigm and has achieved notable success in many real-world applications.Notably,deep neural networks play a crucial role in unlocking RL's potential in large-scale decision-making tasks.Inspired by current major success of Transformer in natural language processing and computer vision,numerous bottlenecks have been overcome by combining Transformer with RL for decision-making.This paper presents a multiangle systematic survey of various Transformer-based RL(TransRL)models applied in decision-making tasks,including basic models,advanced algorithms,representative implementation instances,typical applications,and known challenges.Our work aims to provide insights into problems that inherently arise with the current RL approaches,and examines how we can address them with better TransRL models.To our knowledge,we are the first to present a comprehensive review of the recent Transformer research developments in RL for decision-making.We hope that this survey provides a comprehensive review of TransRL models and inspires the RL community in its pursuit of future directions.To keep track of the rapid TransRL developments in the decision-making domains,we summarize the latest papers and their open-source implementations at https://github.com/williamyuanv0/Transformer-in-Reinforcement-Learning-for-Decision-Making-A-Survey.

外文关键词：

TransformerReinforcement learning(RL)Decision-making(DM)Deep neural network(DNN)Multi-agent reinforcement learning(MARL)Meta-reinforcement learning(Meta-RL)

作者：

袁唯淋、陈佳星、陈少飞、冯大为、胡振震、李鹏、赵卫伟

展开 >

作者单位：

国防科技大学信息通信学院,中国武汉市,430014

国防科技大学智能科学学院,中国长沙市,410072

国防科技大学并行与分布计算全国重点实验室,中国长沙市,410072

关键词：

Transformer 强化学习智能决策深度神经网络多智能体强化学习元强化学习

基金：

National Natural Science Foundation of China

项目编号：

62376280

出版年：

2024

DOI：

10.1631/FITEE.2300548

信息与电子工程前沿(英文)

浙江大学

信息与电子工程前沿(英文)

CSTPCD

影响因子：0.371

ISSN：2095-9184

年,卷(期)：2024.25(6)

参考文献量2