基于强化学习的两时间尺度系统最优跟踪控制
Optimal Tracking Control of Two-time-scale Systems Based on Reinforcement Learning
邓武丹 1李庆奎1
作者信息
摘要
针对两时间尺度系统的最优跟踪控制问题,提出了一种基于奇异摄动理论与强化学习技术的方法.首先,通过研究奇异摄动理论,将系统分解为快和慢 2 个子系统,解决了系统存在的奇异摄动参数问题.其次,将系统的跟踪问题分解为慢子系统的线性二次型跟踪(linear quadratic tracking,LQT)问题和快子系统的线性二次型调节(linear quadratic regulator,LQR)问题,进而利用策略Q-学习分别为2 个子系统设计控制器求解算法.仿真结果表明所提方法能实现系统的最优跟踪性能.
Abstract
Aiming at the optimal tracking control problem of two-time-scale systems,a method based on singular perturbation theory and reinforcement learning technique was proposed.Firstly,the system was decomposed into fast and slow subsystems based on singular perturbation theory,solving the singular perturbation parameter problem existing in the system.Secondly,the tracking problem of the two-time-scale system was decomposed into the linear quadratic tracking(LQT)problem for the slow sub-system and the linear quadratic regulator(LQR)problem for the fast subsystem.Furthermore,the policy Q-learning was used to design the controller solving algorithms for the two subsystems respectively.Finally,the results show that the proposed method can achieve the optimal tracking performance of the system.
关键词
两时间尺度系统/奇异摄动/Q-学习/最优跟踪控制Key words
two-time-scale system/singular perturbation/Q-learning/optimal tracking control引用本文复制引用
基金项目
国家重点研发计划项目(2020YFB1708200)
出版年
2024
NETL
NSTL
万方数据