压缩超快成像(compressed ultrafast photography,CUP)是目前最快的被动式单次超快光学成像技术,它通过数据获取和图像重构两个步骤实现超快事件的捕捉,已发展为记录不可逆或难以重复超快事件的一种有力工具,且能够探测荧光动力学等自发光瞬态场景.然而,传统的迭代优化型算法在图像重构上的保真度较低,而端到端型深度学习算法则严重依赖训练数据,训练复杂度高、通用性不足,这限制了CUP对超快现象进行高空间分辨率的观测.为此,我们开发了一种新型的免训练自监督式神经网络算法,其通过即插即用框架(plug-and-play,PnP)与深度图像先验(deep image prior,DIP)的结合可实现CUP的低复杂度高保真图像重建,简称为PnP-DIP算法.PnP-DIP基于交替方向乘子法(alternating direction method of multipliers,ADMM),利用DIP和PnP去噪器解决图像恢复子问题,可以在防止数据过拟合和噪声累积的同时,显著提高图像重建的精度与收敛速度.通过数值模拟,我们理论上证明了PnP-DIP算法在重构原始动态信息方面相比传统ADMM算法具有更高的保真度.同时,我们分别利用PnP-DIP 对自主研制 CUP 系统观测的皮秒激光脉冲和 X 射线 闪烁体的时 空强度演化数据进行重构,实验上验证了其优越的图像重构性能.这一研究有望推动CUP在高时空分辨观测需求中的应用,并为超快动力学的实时探测作出重大贡献.
Realizing high-fidelity image reconstruction for compressed ultrafast photography with an untrained self-supervised neural network-based algorithm
Compressed ultrafast photography(CUP)is currently the fastest passive single-shot ultrafast optical imaging technology,serving as a potent tool for recording irreversible or difficult-to-repeat ultrafast events,as well as enabling the detection of self-luminescent transient scenes,such as fluorescence dynamics.CUP realizes the recording of ultrafast events through two steps,data acquisition and image reconstruction,and has achieved an ultrahigh sequence depth of over 300 frames and an ultrafast imaging speed of 10 million frames per second,significantly surpassing traditional imaging techniques.However,CUP suffers from low spatial resolution due to its high data compression ratio and undersampling characteristics.Furthermore,the image reconstruction process based on compressive sensing theory is complex and demands extensive computing resources.This limitation curtails CUP's ability to observe ultrafast phenomena with high spatial resolution.Recent efforts to enhance CUP performance have concentrated on enhancing both hardware and algorithmic components.As the hardware structure of CUP is relatively fixed,the advancement of sophisticated algorithms is particularly crucial in improving the quality of reconstructed images.Existing algorithms can be categorized into traditional iterative algorithms and deep learning algorithms.Pure deep learning algorithms face challenges related to the availability of training samples and model generality,hindering rapid transfer.Conversely,traditional iterative algorithms exhibit low computational accuracy and large errors.To address these challenges,we developed a new hybrid algorithm,which combines the plug-and-play(PnP)framework and deep image prior(DIP),named PnP-DIP,drawing upon the advantages of untrained neural networks and traditional iterative algorithms.The PnP-DIP algorithm is based on the alternating direction method of multipliers(ADMM)algorithm,which provides global convergence and parallel processing capabilities,making it highly suitable for tackling large-scale optimization problems.PnP-DIP employs self-supervised learning from DIP to provide a robust solution for image inverse problems and integrates the PnP framework for image denoising to serve as a regularizer,effectively preventing model overfitting.Notably,the proposed algorithm does not require pretraining and ensures both high fidelity and low complexity during the reconstruction process.To quantitatively evaluate the algorithm's performance,the proposed PnP-DIP algorithm is tested and analyzed using numerical simulations,and the reconstruction performance is compared with several commonly used algorithms.The simulation experimental results demonstrate that the proposed algorithm surpasses all competitors in terms of performance on all datasets,exhibiting exceptional robustness and scalability.Furthermore,the PnP-DIP algorithm was applied to reconstruct transient scenes recorded by a custom-built CUP system.This application enabled the measurement of the spatiotemporal evolution of a spatially modulated E-shaped picosecond laser pulse and the two-dimensional intensity evolution of an X-ray scintillator.The results revealed that the proposed method excelled in spatial resolution,continuity,and heterogeneity,accurately reflecting the inherent laws and features of spatial data,thus paving the way for practical applications of CUP.The flexibility of DIP allows this algorithm to be extended to multidimensional imaging models such as hyperspectral CUP and spectral-volumetric CUP,enabling the recovery of higher-dimensional data and expanding the application of CUP-based technology in capturing complex ultrafast physical events.This research is projected to promote the application of CUP in scenarios requiring high spatiotemporal resolution and make a significant contribution to the development of fundamental and applied sciences.
compressed ultrafast photographyimage reconstructionalternating direction method of multipliersplug-and-play frameworkdeep image prior
万方数据
维普
