首页|Findings on Machine Learning Detailed by Investigators at University of Technolo gy Sydney (On Taking Advantage of Opportunistic Meta-knowledge To Reduce Configu ration Spaces for Automated Machine Learning)
Findings on Machine Learning Detailed by Investigators at University of Technolo gy Sydney (On Taking Advantage of Opportunistic Meta-knowledge To Reduce Configu ration Spaces for Automated Machine Learning)
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
By a News Reporter-Staff News Editor at Robotics & Machine Learning Daily News Daily News-Investigators discuss new findings in Machine Learning. According to news reporting originating from Ultimo, Australia , by NewsRx correspondents, research stated, "The optimisation of a machine lear ning (ML) solution is a core research problem in the field of automated machine learning (AutoML). This process can require searching through complex configurat ion spaces of not only ML components and their hyperparameters but also ways of composing them together, i.e. forming ML pipelines." Financial support for this research came from CASLab, University of Technology S ydney (UTS). Our news editors obtained a quote from the research from the University of Techn ology Sydney, "Optimisation efficiency and the model accuracy attainable for a f ixed time budget suffer if this pipeline configuration space is excessively larg e. A key research question is whether it is both possible and practical to preem ptively avoid costly evaluations of poorly performing ML pipelines by leveraging their historical performance for various ML tasks, i.e. meta-knowledge. This pa per approaches the research question by first formulating the problem of configu ration space reduction in the context of AutoML. Given a pool of available ML co mponents, it then investigates whether previous experience can recommend the mos t promising subset to use as a configuration space when initiating a pipeline co mposition/optimisation process for a new ML problem, i.e. running AutoML on a ne w dataset. Specifically, we conduct experiments to explore (1) what size the red uced search space should be and (2) which strategy to use when recommending the most promising subset. The previous experience comes in the form of classifier/r egressor accuracy rankings derived from either (1) a substantial but non-exhaust ive number of pipeline evaluations made during historical AutoML runs, i.e. ‘opp ortunistic' meta-knowledge, or (2) comprehensive crossvalidated evaluations of classifiers/regressors with default hyperparameters, i.e. ‘systematic' meta-know ledge. Overall, numerous experiments with the AutoWeka4MCPS package, including o nes leveraging similarities between datasets via the relative landmarking method , suggest that (1) opportunistic/systematic meta-knowledge can improve ML outcom es, typically in line with how relevant that meta-knowledge is, and (2) configur ation-space culling is optimal when it is neither too conservative nor too radic al. However, the utility and impact of meta-knowledge depend critically on numer ous facets of its generation and exploitation, warranting extensive analysis; th ese are often overlooked/underappreciated within AutoML and meta-learning litera ture. In particular, we observe strong sensitivity to the ‘challenge' of a datas et, i.e. whether specificity in choosing a predictor leads to significantly bett er performance."
UltimoAustraliaAustralia and New Zea landCyborgsEmerging TechnologiesMachine LearningUniversity of Technology Sydney
NETL
NSTL
