Ablation based correspondence analysis of objects in deep learning interpretable heatmap
[Objective]Although deep learning has achieved remarkable success,criticisms in its stability,interpretability,and fairness remain.Prominently,it is well known as a correspondences driven machine learning method,and its trained models,even the large models,are somewhat involved with biases.According to the theory of stable learning,these biases,which are induced by false correspondences,prompt problems of the stability and the interpretability.Consequently,the correspondence analysis for the discrimination is considered as a promising solution.As the most widely used deep learning model,convolutional neural network(CNN)image classification model has managed to solve this problem on the agenda.[Methods]Differing from existing research that primarily focuses on the extraction of objects to give interpretable heatmap,we deem that the correspondence among objects in the heatmap should also be studied.Then,we present ablation correspondence analysis(Ablation-CA).The Ablation-CA firstly implements superpixel segmentation of an input image to obtain objects.Subsequently,the classification contributions of these objects are quantified with sensitivity analysis(SA)algorithm to figure out interpretable heatmap.Through synchronous ablations and correlation calculations,correlation values among objects in the heatmap are obtained successively.Finally,all the correspondent object groups are yielded into a sort list.[Results]By the testing on pre-trained models of CNN classification(Inception-v3)and standard image data(PASCAL VOC2012,CIFAR-10,and CSDN among others),it is proved that the Ablation-CA may output more semantics and better interpretable heatmap than main traditional methods may,including local interpretable model-agnostic explanations(LIME),randomized input sampling for explanation(RISE),class activation mapping(CAM),saliency,deep Taylor decomposition(DTD),layer-wise relevance propagation(LRP),XRAI(novel region-based attribution method),guided-backpropagation(GBP),and integrated gradients(IG).The superiority is mainly attributed to the superpixel segmentation and Monte Carlo method used in the Ablation-CA.Experimental results also show that Ablation-CA can effectively calculate objects correspondence of the CNN classification model.As a result,Ablation-CA heatmap may provide correspondence labels on the heatmap,which existing methods do not have.Objectively,some room for improvements remains.From experimental instances,the effect of Ablation-CA to the single image content functions properly,and the linear relationships among which can be analyzed rapidly.However,for some complex content images with nonlinear correlation,Ablation-CA does not perform sufficiently satisfactorily.Because the size of superpixel segmentation blocks is the most important hyperparameter that affects the effectiveness of Ablation-CA.we test the maximum correlation value of top 10 images in PASCAL VOC2012 which include linear correlation objects.It is found that the relationship between the correlation value and the number of segmentation blocks shows a fluctuating trend,namely first increasing and then decreasing.For the test dataset,the maximum value is achieved when the number of segmentation blocks lies within 30-50,and then the value gradually decreases with the increase of the number of segmentation blocks.Our analysis indicates that finer superpixel segmentation can remove some classification interference(relevant experiments show that the classification probability,obtained by ablation of interference superpixels,is even higher than the original image).However,overly fine segmentation damages the semantic information of image objects,resulting in the model misrecognition.Therefore,the segmentation block number must be specified within a rational range.[Conclusions]In this paper,we discuss a CA dimension,namely the correspondence among objects in the CNN image classification model samples.Clearly,our analysis differs from normal existing explainable methods for CNN.Preliminary experiments have demonstrated the feasibility and the effectiveness of Ablation-CA.The correspondence output by Ablation-CA may be used for many relevant applications,including false correspondence discrimination for stable learning,image semantic analysis,object-relation drawing for the automatic generation of knowledge graphs,and regularization for model evolution among others.Urgently,some aspects of Ablation-CA continue to be improved.For the purpose of discovering more and deeper correspondence from CNN,some complex correlation algorithms ought to be added into Ablation-CA.The function with respect to block number and correlation values needs to be explored so that a balance between semantics and the analysis is maintained.Moreover,faster algorithms are also required for the enormous computational complexity of large graphs.
deep learningcorrespondenceinterpretabilityablation analysisheatmap
万方数据
维普
