Bipartite adversarial autoencoder network for unsupervised detection of changes in heterogeneous remote sensing images
Objective Heterogeneous remote sensing images from different sensors are quite different in imaging mecha-nism,radiation characteristics,and geometric characteristics.Thus,they reflect the physical properties of the ground tar-get at different levels.Therefore,no relationship exists between the observed values of the same object,which usually leads to"pseudo changes".As a result,the change detection task has more difficulty obtaining accurate change informa-tion of the observed ground objects.Efforts have been made toward unsupervised detection of changes in heterogeneous remote sensing images by designing various methods to obtain change information.However,traditional image difference operators based on the difference or ratio of radiation measurement is no longer applicable.Therefore,transferring the bitemporal images in a common space is a convenient way to calculate differences.Considering the excellent and flexible feature learning capability of deep neural networks,they have been widely applied in change detection tasks for heteroge-neous images to effectively alleviate the influence of"pseudo changes".Moreover,by fully utilizing the characteristics of deep neural networks,they can be designed to transform heterogeneous remote sensing images into the same feature domain.Then,the change information can be accurately represented.Inspired by the paradigm of image translation,het-erogeneous images are transformed into a common domain with consistent feature representations to enable direct compari-sons of data.The key point of this task is learning a suitable one-to-one mapping to build a relationship between distinct appearances of images and exclude the effect of interference factors.Therefore,this study proposes a bipartite adversarial autoencoder network with clustering(BAACL)to detect changes between heterogeneous remote sensing images.Method A bipartite adversarial autoencoder network is constructed to reconstruct the bitemporal images and achieve the transformation of heterogeneous images to common domain.Appropriate reconstruction loss regularization should be applied to obtain good data mapping from the constructed neural network optimization.Next,the bipartite convolutional autoencoders can be jointly trained to encode the input images and reconstruct them with high fidelity in output.Meanwhile,for the image-to-image translation task,additional structural consistency loss and adversarial loss terms are designed to constrain the net-work training process for converting heterogeneous images to the common data domain.The structural consistency loss term is designed to describe the internal structural relationships of images before and after translation.For this purpose,affinity matrixes are adopted to express the structural self-similarity within an image.The heterogeneous images can be conve-niently compared in a new affinity space.Notably,based on the paradigm of image translation,our model can be viewed as learning two"transformation functions",which are adopted to transform each of the heterogeneous images to the opposite data space.Furthermore,such a setup can regarded as a special case of an"adversarial mechanism",which is formulated by an adversarial loss to train autoencoders for matching the opposite image style.Considering the disadvantage of these changed pixels to the adversarial loss term in network optimization,the pseudo-difference image generated by two pairs of homogeneous images mapped in the common data domain is analyzed by clustering in an unsupervised manner.Thereafter,the obtained semantic information is adopted to further constrain the adversarial loss term.The overall performance of the BAACL network is illustrated on four sets of publicly available datasets of heterogeneous remote sensing images.Five popu-lar traditional and deep learning-based detection methods for changes in heterogeneous remote sensing images are selected and compared with this method to verify its effectiveness.Result Results obtained from the Italy,California,Tianhe,and Shuguang datasets can also illustrate the performance of the proposed BAACL,with the overall detection accuracies of up to 0.970 5,0.938 2,0.994 7,and 0.982 6,respectively.Meanwhile,the proposed method is superior to the five compared methods in terms of visual results of the final change map.A set of ablation experiments is designed to verify the influence of the improved adversarial loss term on network optimization performance.the performance of the BAACL network does not change dramatically with various proportions of sample changing regions due to the semantic regularization.It verifies the effectiveness of the proposed semantic information-based adversarial loss term for network optimization.Conclusion The semantic information-based adversarial loss term is designed to narrow the distance of the unchanged regions of the bitemporal images for image style consistency.The reason is that,for adversarial loss term without semantic regularization,a larger proportion of the change region of the current training sample will result in a worse trained network effect.There-fore,an accurate definition of the network constraint term will result in good network optimization and further affect the change detection performance.In view of the difficulty and high false alarm rate of detection methods for changes in hetero-geneous remote sensing images caused by factors such as seasons and data heterogeneity,the proposed bipartite adversarial autoencoder network for the detection of changes in heterogeneous remote sensing images not only can fully utilize network characteristics and semantic information to improve image style consistency but also can realize completely unsupervised change detection process.The change detection performance can be greatly improved as well.
NETL
NSTL
万方数据
