High Dynamic Range Imaging Based on Coherence-Aware Feature Aggregation
High Dynamic Range Imaging(HDRI)is a technology of fusing multiple Low Dynamic Range(LDR)images to extend image dynamic range,restore image contents and generate high dynamic range(HDR)images.It provides a practical solution to the problem of content loss in captured images due to the limited dynamic range of the camera sensors.With decades of studies,numerous promising approaches have been proposed and near-optimal performance has been achieved for the HDRI static scenes with no object motions and well-exposed contents.However,object motions or camera shifts are inevitable in practical scenarios.Directly using traditional HDRI methods would induce severe ghosting artifacts into the merged HDR image.This makes HDRI with simple merging process inapplicable in real-world applications,which poses a chal-lenge to the HDRI task.Thus,the study on HDRI of dynamic scenes has grown rapidly.Recent advances focus on exploring the power of deep convolutional neural networks(CNNs)to achieve a better performance.Among these CNN-based methods,the feature aggregation plays a crucial role in completing image contents and eliminating ghosting artifacts.Equipped with skip connections or attention modules,the features derived from multiple LDR images are first concatenated and then gradually focus on different local aspects via stacked convolutions.However,such aggrega-tion schemes generally neglect to utilize the rich contextual dependencies across LDR image sequence,the textural coherence among features have not been fully exploited.To address this issue,this paper proposes a novel Coherence-Aware Feature Aggregation(CAFA)scheme that samples grids with the same contextual information instead of the same position across input features during convolutional operations,so that contextual coherence can be explicitly incorpo-rated into feature aggregation.Based on CAFA,this paper further proposes Coherence-Aware HDR Network(CAHDRNet)for HDRI of dynamic scenes.To facilitate the incorporation of CAFA,the proposed CAHDRNet is constructed by designing three additional learnable modules.Firstly,a learnable feature extractor,which is built upon a VGG-19 pre-trained on ImageNet,is used to extract features from each LDR image.The parameters will be updated during end-to-end training.Such a design enables a joint feature learning of LDR images which creates a solid foun-dation for applying the coherence evaluation in CAFA.Then,the proposed CAFA module is applied to aggregate the features by sampling grids with the same contextual information in each image features.Next,a Multi-Scale Residual Hallucinating(MSRH)module is proposed to process the aggregated features,in which the features are learnt across different scales of dilated rates to achieve a more powerful feature representation and hallucinate plausible details in the missing regions.Also,a soft attention module is equipped to learn the importance of different image regions for obtaining the features that are complementary to the reference image during skip connection.Various experiments are conducted to validate the effectiveness of our proposed CAHDRNet,where it demonstrates superior performance over state-of-the-art(SOTA)methods.Specifically,the proposed CAHDRNet improves the HDR-VDP-2 and PSNR-L values on Kalantari's dataset over the second-best AHDRNet by 1.61 and 0.68,respectively.
high dynamic range imagingimage fusionfeature aggregationcontextual coherenceconvolutional sampling
NETL
NSTL
万方数据
