首页|New Machine Learning Study Findings Reported from Chinese University of Hong Kon g (Gradient Descent Provably Escapes Saddle Points In the Training of Shallow Re lu Networks)
New Machine Learning Study Findings Reported from Chinese University of Hong Kon g (Gradient Descent Provably Escapes Saddle Points In the Training of Shallow Re lu Networks)
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
Researchers detail new data in Machine Learning. According to news reporting out of Shenzhen, People's Republic of Chi na, by NewsRx editors, research stated, "Dynamical systems theory has recently b een applied in optimization to prove that gradient descent algorithms bypass so- called strict saddle points of the loss function. However, in many modern machin e learning applications, the required regularity conditions are not satisfied." Funders for this research include Eric and Wendy Schmidt AI in Science Postdocto ral Fellowship, European Union (ERC, MONTECARLO), German Research Foundation (DF G). Our news journalists obtained a quote from the research from the Chinese Univers ity of Hong Kong, "In this paper, we prove a variant of the relevant dynamical s ystems result, a center-stable manifold theorem, in which we relax some of the r egularity requirements. We explore its relevance for various machine learning ta sks, with a particular focus on shallow rectified linear unit (ReLU) and leaky R eLU networks with scalar input. Building on a detailed examination of critical p oints of the square integral loss function for shallow ReLU and leaky ReLU netwo rks relative to an affine target function, we show that gradient descent circumv ents most saddle points."
ShenzhenPeople's Republic of ChinaAs iaCyborgsEmerging TechnologiesMachine LearningChinese University of Hong Kong
NETL
NSTL
