Optimization in Machine Learning:a Distribution-Space Approach

扫码查看

原文链接

NETL
NSTL
万方数据

外文摘要：We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space,but with a non-convex constraint set introduced by model parameterization.This observation allows us to repose such problems via a suitable relaxation as convex optimization problems in the space of distributions over the training parameters.We derive some simple relationships between the distribution-space problem and the original problem,e.g.,a distribution-space solution is at least as good as a solution in the original space.Moreover,we develop a numerical algorithm based on mixture distributions to perform approximate optimization directly in the distribution space.Consistency of this approximation is established and the numerical efficacy of the proposed algorithm is illustrated in simple examples.In both the-ory and practice,this formulation provides an alternative approach to large-scale optimiza-tion in machine learning.

外文关键词：

Machine learningConvex relaxationOptimizationDistribution space

作者：

Yongqiang Cai、Qianxiao Li、Zuowei Shen

展开 >

作者单位：

School of Mathematical Sciences,Laboratory of Mathematics and Complex Systems,MOE,Beijing Normal University,Beijing 100875,China

Department of Mathematics,National University of Singapore,21 Lower Kent Ridge Road,Singapore 119077,Singapore

基金：

National Natural Science Foundation of ChinaNational Research Foundation,Singapore,under the NRF fellowshipDistinguished Professorship of National University of Singapore

项目编号：

12201053NRF-NRFF13-2021-0005

出版年：

2024

DOI：

10.1007/s42967-023-00322-5

应用数学与计算数学学报

上海大学

应用数学与计算数学学报

影响因子：0.165

ISSN：1006-6330

年,卷(期)：2024.6(2)