首页|Safe Q-Learning for Data-Driven Nonlinear Optimal Control with Asymmetric State Constraints

Safe Q-Learning for Data-Driven Nonlinear Optimal Control with Asymmetric State Constraints

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
This article develops a novel data-driven safe Q-learning method to design the safe optimal controller which can guarantee constrained states of nonlinear systems always stay in the safe region while providing an optimal performance。 First， we design an augmented utility function consisting of an adjustable positive definite control obstacle function and a quadratic form of the next state to ensure the safety and optimality。 Second， by exploiting a pre-designed admissible policy for initialization， an off-policy stabilizing value iteration Q-learning (SVIQL) algorithm is presented to seek the safe optimal policy by using offline data within the safe region rather than the mathematical model。 Third， the monotonicity， safety， and optimality of the SVIQL algorithm are theoretically proven。 To obtain the initial admissible policy for SVIQL， an offline VIQL algorithm with zero initialization is constructed and a new admissibility criterion is established for immature iterative policies。 Moreover， the critic and action networks with precise approximation ability are established to promote the operation of VIQL and SVIQL algorithms。 Finally， three simulation experiments are conducted to demonstrate the virtue and superiority of the developed safe Q-learning method。

SafetyIterative methodsOptimal controlQ-learningHeuristic algorithmsMathematical modelsLearning systems

Mingming Zhao、Ding Wang、Shijie Song、Junfei Qiao

展开 >

Beijing Key Laboratory of Computational Intelligence and Intelligent System, School of Information Science and Technology, Beijing, China|Beijing Laboratory of Smart Environmental Protection, Beijing Institute of Artificial Intelligence, Beijing University of Technology, Beijing, China

School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China

2024

10.1109/JAS.2024.124509

IEEE/CAA journal of automatica sinica

IEEE/CAA journal of automatica sinica

ISSN：
年,卷(期)：2024.11(12)
  • 40