Multispectral image fusion method for surface defect detection of IC devices
邓耀华 1黄志海1
扫码查看
点击上方二维码区域，可以放大扫码查看
作者信息
1. 广东工业大学 机电工程学院,广东 广州 510006
折叠
摘要
针对单可见光或单红外条件下的IC器件表面缺陷对比度不足,缺陷检测精度低的问题,提出多光谱图像融合的IC器件表面缺陷检测方法.针对IC器件可见光与红外图像配准中存在尺度不一致和对比度反转问题,引入拉普拉斯金字塔和特征描述符重组策略改进ORB(Oriented FAST and Rotated BRIEF)图像配准算法.在图像配准的基础上,提出NSST_VP图像融合方法,以非下采样剪切波变换(Non-Subsample Shearlet Transform,NSST)得到红外图像和已配准可见光图像的低频和高频子带,对低频子带采用视觉显著图(Visual Significance Map,VSM)加权融合规则,高频子带则采用自适应脉冲耦合神经网络(PA-Pulse Coupled Neural Network,PA-PCNN)决策融合规则,进而通过NSST逆变换得到高质量多光谱融合图像.最后,将融合图像输入YOLOv8s模型进行检测.实验结果表明,改进ORB的图像配准平均精度为87.8%,比ORB图像配准精度提高了62%,NSST_VP图像融合算法在主观视觉效果和客观评价指标上均有所提高.在缺陷检测实验中,NSST_VP融合方法的均值平均精度(mean Average Precision,mAP)达到83.15%,比单可见光、单红外缺陷图像检测的mAP分别提高了22.97%,28.31%,比双树复小波变换融合、曲线变换融合、非下采样轮廓波变换融合方法的mAP分别提高了13.14%,15.01%,20.35%.
Abstract
To address the issue of low defect detection accuracy in IC devices due to insufficient contrast under either visible light or infrared conditions alone,this paper introduces a multi-spectral fusion ap-proach.Initially,to overcome scale inconsistency and contrast inversion challenges during IC device image registration,we enhance the ORB(Oriented FAST and Rotated BRIEF)algorithm with a Laplacian pyra-mid and feature descriptor recombination strategy.Following image registration,we propose the NSST_VP image fusion method,which processes the infrared and visible images'low and high frequency subbands through Non-Subsample Shearlet Transform(NSST).For fusion,the low frequency subband us-es a visual significance map(VSM)weighted rule,and the high frequency subband employs a PA-Pulse Coupled Neural Network(PA-PCNN)decision rule,with the final image produced by reversing the NSST.The fused image is then analyzed using the YOLOv8s model.Experimental findings reveal an 87.8%average accuracy with the improved ORB registration,marking a 62%enhancement over the stan-dard ORB.The NSST_VP fusion algorithm significantly boosts both subjective and objective metrics,achieving an mAP of 83.15%-surpassing single light mode detections by 22.97%and 28.31%,and outper-forming Dual-Tree Complex Wavelet,Non-Subsampled Contourlet,and Curvelet Transform fusion meth-ods by 13.14%,15.01%,and 20.35%,respectively.
维普
万方数据