Joint geometric and piecewise photometric line-scan image registration
Objective Image registration is a fundamental problem in computer vision and image processing.It aims to eliminate the geometric difference of an object in an image collected by different cameras at various times and poses.Image registration has been widely used in several visual applications,such as image tracking,image fusion,image analysis,and anomaly detection.Image registration methods can be classified into feature-based and direct registration methods.The for-mer calculates the parameters in a geometric transformation model by extracting and matching features,such as corners or edges,while the latter directly uses image intensity to infer the parameters.Evidently,choosing a reasonable geometric transformation model is the key to image alignment.The principles of line-scan and area-scan cameras are identical,and both cameras conform to the principle of pin-hole imaging.However,the imaging model of a line-scan camera is different from that of an area-scan camera due to the characteristics of its sensor.With the same change in camera pose,the loca-tions of the same 3D world points mapped to the two types of images are different.That is,the geometric transformation law of an object in the images caused by the pose change of the two types of cameras is different.When the image plane of a line-scan camera is nonparallel to the object plane,geometric transformation models commonly used for area-scan image registration,such as the rigid,affine,and projection transformation models,cannot conform to the geometric transforma-tion law of line-scan images.The direct registration method based on the geometric transformation model of an area-scan image cannot realize the geometric alignment of a line-scan image.Moreover,most existing direct image registration meth-ods for solving the image alignment problem is based on the brightness constancy assumption and only geometric transforma-tion is considered.In real-world applications,the variation of brightness is unavoidable and the brightness constancy assumption cannot address the problem of brightness attenuation when capturing images with a large-angle lens.Therefore,the line-scan image registration problem,which estimates geometric and photometric transformations between two images,is considered.Moreover,a direct registration method for line-scan images based on geometric and piecewise photometric transformations is proposed in this study.Method First,the optimization objective function of line-scan image registration is constructed by using the sum of squares difference of image intensity.In accordance with the geometric transformation model of line-scan images and the piecewise gain-bias photometric transformation model,the registration problem of a line-scan image is expressed as a nonlinear least squares problem.Second,the Gauss-Newton method is used to optimize the geometric and photometric transformation parameters in the registration problem.The nonlinear optimization objective func-tion is linearized by performing a first-order Taylor expansion.The Jacobian of the warp and photometric transformation is derived on the basis of the geometric transformation model of a line-scan image and the gain-bias model.Finally,to obtain the optimal geometric and photometric transformation parameters,the increments of the warp and photometric transforma-tion are repeatedly computed until they are below the threshold in accordance with the normal equation.As the initial value,the identity warp cannot be guaranteed near the optimal solution,and the iteration does not converge during registra-tion.This problem is solved by designing an initial value fast matching method that provides an initial solution closer to the optimal one.The process of the initial value fast matching method is as follows:fixed-size areas are selected from the four corners of the template image and then matched to the target image in the corresponding position.The minimum and maxi-mum coordinates of the optimal matching position in the horizontal and vertical directions are selected.Then,the scale and translation factors in the horizontal and vertical directions are solved,and the result is regarded as the initial value for the iteration.The initial value provided by the initial value fast matching method reduces geometric difference between the tem-plate and target images,and the success rate of the registration method is improved.Result To verify the proposed line-scan image registration method,a line-scan image acquisition system was built to obtain line-scan images of a planar object under different imaging poses and illumination variations.The experimental data also included electric multiple units(EMU)train line-scan images,which were collected by a line-scan camera in a natural environment.The images collected by the line-scan image acquisition system and the EMU train line-scan images were annotated separately,and the root-mean-square error(RMSE)of the annotated point coordinates was used as the evaluation index of the geometric error.The performance of the initial value fast matching method was verified on the line-scan image dataset collected in this study.The geometric error between the template image and the warped target image based on the initial value provided by the fast template block matching method was smaller than that based on the identity warp.This finding indicates that the initial value provided by the initial value fast matching method is closer to the optimal solution of the geometric transformation.Through the registration experiments on the collected dataset and the EMU train line-scan image,the results show that the RMSE of the annotated point coordinates is less than 1 pixel,and registration accuracy is excellent.Conclusion Our algo-rithm is more robust to lighting changes,and it improves the success rate of line-scan image registration.The joint geomet-ric and piecewise photometric line-scan image registration method proposed in this study can accurately align the images collected in practical application scenes.This condition is also a foundation for train anomaly detection based on line-scan images.Therefore,the direct registration method proposed in this study can accurately and robustly align line-scan images collected under nonparallel poses.
NETL
NSTL
万方数据
