首页|A Region-Based Analysis for the Feature Concatenation in Deep Forests
A Region-Based Analysis for the Feature Concatenation in Deep Forests
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
万方数据
Deep forest is a tree-based deep model made up of non-differentiable modules that are trained without backpropagation.Despite the fact that deep forests have achieved considerable success in a variety of tasks,feature concatenation,as the ingredient for forest representation learning,still lacks theoretical understand-ing.In this paper,we aim to understand the influence of feature concatenation on predictive performance.To en-able such theoretical studies,we present the first math-ematical formula of feature concatenation based on the two-stage structure,which regards the splits along new features and raw features as a region selector and a re-gion classifier respectively.Furthermore,we prove a re-gion-based generalization bound for feature concatenation,which reveals the trade-off between Rademacher com-plexities of the two-stage structure and the fraction of in-stances that are correctly classified in the selected region.As a consequence,we show that compared with the pre-diction-based feature concatenation(PFC),the advant-age of interaction-based feature concatenation(IFC)is that it obtains more abundant regions through distrib-uted representation and alleviates the overfitting risk in local regions.Experiments confirm the correctness of our theoretical results.
Deep forestOverfittingGeneraliza-tion boundRepresentation learning
LYU Shen-Huan、CHEN Yi-He、ZHOU Zhi-Hua
展开 >
National Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
NETL
NSTL
万方数据
