首页|Evaluating the effectiveness of neuron coverage metrics: a metamorphic-testing approach

Evaluating the effectiveness of neuron coverage metrics: a metamorphic-testing approach

扫码查看
原文链接
  • NETL
  • NSTL
  • Springer Nature
Deep neural networks (DNNs) are now widely used in many sectors of our society. This phenomenon also means that if these DNNs contain faults, they will have profound adverse impacts on our daily lives. Thus, DNNs have to be comprehensively tested for "correctness" before they are released for use. Since such testing involves the use of a DNN test set, the comprehensiveness of this test set is of utmost importance. Until now, many researchers have proposed their own neuron-coverage (NC) metrics to measure the comprehensiveness of a DNN test set. However, their studies solely focused on those DNN testing scenarios with the presence of a test oracle. We observed that, in reality, there are many DNN testing scenarios where a test oracle does not exist and, therefore, the results of all previous studies may be inapplicable to these testing scenarios. Inspired by this observation, we have performed an empirical study to investigate the usefulness of some common and major NC metrics in terms of correlation analysis and invariability analysis. Our experiment results showed that, on the one hand, some NC metrics are useful measures of DNN test-set comprehensiveness (in terms of correlation analysis), but on the other hand, these metrics are not robust enough (in terms of invariability analysis).

Deep learning systemDeep neural networkMachine learningMetamorphic relationMetamorphic testingNeuron coverage

Zenghui Zhou、Pak-Lok Poon、Tsong Yueh Chen、Kun Qiu、Qinghua Zhao、Zheng Zheng

展开 >

School of Automation Science and Electrical Engineering, Beihang University, Beijing, China

School of Engineering and Technology, Central Queensland University, Melbourne, Australia

Department of Computer Science and Software Engineering, Swinburne University of Technology, Hawthorn, Australia

School of Electrical Engineering and Automation, Hefei University of Technology, Hefei, China

School of Computer Sciences, Beihang University, Beijing, China

展开 >

2025

10.1007/s11219-025-09716-5

Software quality journal

Software quality journal

ISSN：0963-9314
年,卷(期)：2025.33(2)
  • 48