A sequential quadratic hamiltonian algorithm for training explicit RK neural networks
Hofmann, S. 1Borzi, A.1
扫码查看
点击上方二维码区域，可以放大扫码查看
作者信息
1. Univ Wurzburg
折叠
Abstract
A sequential quadratic hamiltonian (SQH) algorithm for solving nonsmooth supervised learning problems (SLPs) in the framework of residual neural networks is presented. In this framework, a SLP is interpreted as an optimal control problem and the SQH algorithm determines a solution using the characterization of optimality given by a discrete version of the Pontryagin maximum principle. Convergence and stability of the proposed algorithm is investigated theoretically in the framework of residual neural networks with Runge-Kutta structure, and its computational performance is compared to that of the so-called extended method of successive approximations. Results of numerical experiments demonstrate the superior performance of the SQH algorithm in terms of efficiency and robustness of the training process. (c) 2021 Elsevier B.V. All rights reserved.
Key words
Residual neural networks/Runge-Kutta neural networks/Discrete Pontryagin maximum principle/Sequential quadratic hamiltonian algorithm/Method of successive approximations/Numerical optimization/CONVERGENCE
NSTL
Elsevier