首页|Julius-Maximilians-University Wurzburg Reports Findings in Machine Learning (Pra ctical approaches in evaluating validation and biases of machine learning applie d to mobile health studies)
Julius-Maximilians-University Wurzburg Reports Findings in Machine Learning (Pra ctical approaches in evaluating validation and biases of machine learning applie d to mobile health studies)
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
By a News Reporter-Staff News Editor at Robotics & Machine Learning Daily News Daily News – New research on Machine Learning is th e subject of a report. According to news reporting out of Wurzburg, Germany, by NewsRx editors, research stated, “Machine learning (ML) models are evaluated in a test set to estimate model performance after deployment. The design of the tes t set is therefore of importance because if the data distribution after deployme nt differs too much, the model performance decreases.” Our news journalists obtained a quote from the research from Julius-Maximilians- University Wurzburg, “At the same time, the data often contains undetected group s. For example, multiple assessments from one user may constitute a group, which is usually the case in mHealth scenarios. In this work, we evaluate a model’s p erformance using several cross-validation train-test-split approaches, in some c ases deliberately ignoring the groups. By sorting the groups (in our case: Users ) by time, we additionally simulate a concept drift scenario for better external validity. For this evaluation, we use 7 longitudinal mHealth datasets, all cont aining Ecological Momentary Assessments (EMA). Further, we compared the model pe rformance with baseline heuristics, questioning the essential utility of a compl ex ML model. Hidden groups in the dataset leads to overestimation of ML performa nce after deployment. For prediction, a user’s last completed questionnaire is a reasonable heuristic for the next response, and potentially outperforms a compl ex ML model. Because we included 7 studies, low variance appears to be a more fu ndamental phenomenon of mHealth datasets. The way mHealth-based data are generat ed by EMA leads to questions of user and assessment level and appropriate valida tion of ML models. Our analysis shows that further research needs to follow to o btain robust ML models. In addition, simple heuristics can be considered as an a lternative for ML.”
WurzburgGermanyEuropeCyborgsEmer ging TechnologiesMachine Learning
NETL
NSTL
