Fuzz testing is an efficient method to find security critical bugs.In recent years,a plenty of works about fuzz testing have been proposed in both industry and academia.A variety of fuzz testing tools have been developed.These tools differ in techniques and performance so that the evaluation of fuzzers is demanded to understand these tools.But many existing evaluations have problems of bad interpretability,which leads to limited findings from the evaluation results.In this paper,we find that the evaluation results can be affected by plenty of factors,including fuzzing-hampering features contained in the target programs.However,existing evaluations pay little attention on fuzzing-hampering features,which leads to the inability to explain the reasons behind the evaluation results,even causing unclear or erroneous conclusions.In this regard,we propose a method to evaluate fuzzers based on fuzzing-hampering features.Our method treats fuzz-ing-hampering features as one of the controlled variables and performs fine-grained comparative testing to find out the relationships between evaluation results and fuzzing-testing features to identify the reason causing the different results,making the evaluation more interpretable.We also develop a method to construct benchmarks with which fuzz-ing-hampering features can be a controlled variable during the evaluation.To implement the idea and show its effective-ness,we summarized 5 fuzzing-testing features,quantitatively defined how to calculate the indicator of the capabilities of a fuzzer and constructed a bug benchmark named Bench4I,which included 118 synthetic programs with different fuzz-ing-hampering features.In the experiment,we evaluated 6 fuzzers.It shows that the tools'detailed capabilities can be in-ferred according to the indicators calculated from the evaluation results so that and the evaluation results become more interpretable.With the help of the evaluation,we also proposed several advices of using and improving these fuzzers.We put the improvement of QSYM into practice and gained a quite encouraging result.
维普
万方数据