Safe Q-Learning for Data-Driven Nonlinear Optimal Control With Asymmetric State Constraints

扫码查看

原文链接

国家科技期刊平台
NETL
NSTL
万方数据

外文标题：Safe Q-Learning for Data-Driven Nonlinear Optimal Control With Asymmetric State Constraints

外文摘要：This article develops a novel data-driven safe Q-learning method to design the safe optimal controller which can guarantee constrained states of nonlinear systems always stay in the safe region while providing an optimal performance.First,we design an augmented utility function consisting of an adjustable positive definite control obstacle function and a quadratic form of the next state to ensure the safety and optimality.Second,by exploiting a pre-designed admissible policy for initialization,an off-policy stabilizing value iteration Q-learning(SVIQL)algo-rithm is presented to seek the safe optimal policy by using offline data within the safe region rather than the mathematical model.Third,the monotonicity,safety,and optimality of the SVIQL algorithm are theoretically proven.To obtain the initial admissi-ble policy for SVIQL,an offline VIQL algorithm with zero ini-tialization is constructed and a new admissibility criterion is established for immature iterative policies.Moreover,the critic and action networks with precise approximation ability are established to promote the operation of VIQL and SVIQL algo-rithms.Finally,three simulation experiments are conducted to demonstrate the virtue and superiority of the developed safe Q-learning method.

外文关键词：

Adaptive critic controladaptive dynamic program-ming(ADP)control barrier functions(CBF)stabilizing value itera-tion Q-learning(SVIQL)state constraints

作者：

Mingming Zhao、Ding Wang、Shijie Song、Junfei Qiao

展开 >

作者单位：

School of Information Science and Technology,the Beijing Key Laboratory of Computational Intelligence and Intelligent System,the Beijing Laboratory of Smart Environmental Protection,and the Beijing Institute of Artificial Intelligence,Beijing University of Technology,Beijing 100124,China

School of Mechanical and Electrical Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China

关键词：

Adaptive critic control adaptive dynamic program-ming(ADP) control barrier functions(CBF) stabilizing value itera-tion Q-learning(SVIQL) state constraints

出版年：

2024

DOI：

10.1109/JAS.2024.124509

自动化学报(英文版)

中国自动化学会,中国科学院自动化研究所,中国科技出版传媒股份有限公司

自动化学报(英文版)

CSTPCDEI

ISSN：2329-9266

年,卷(期)：2024.11(12)