A Multi-task Learning Method for 3D Reconstruction of Highly Reflective Surfaces
When applying binocular Phase-Shifting Profilometry(PSP)to highly reflective surfaces,the main difficulty lies in the lack of phase in the overexposed area,based on this consideration,multi-task learning was introduced into highly reflective surface phase profiling.However,in the early stage of multi-task learning,due to the influence of overexposed regions,the network tends to realize the correspondence retrieval of non-overexposed regions firstly,and the results obtained by the phase prediction module are not enough to constrain the stereo matching process,and the incorrect matching rate is prone to large oscillations,which reduces the network convergence speed.In order to solve this problem,considering that the fitting task of the network to the overexposed area and the non-overexposed area contains a natural priority relationship,and the traditional Pareto optimization of parallel multi-task learning is not suitable for scenarios with priority relationships,this paper proposes a multi-task learning method that dynamically updates the task priority,and controls the network to complete the learning according to above pattern.Firstly,the multi-task learning network is designed based on classic architecture UNET,which consists of phase prediction and correspondence retrieval module.In learning process,this two tasks are optimized simultaneously,thus,the correspondence retrieval process is guided by the result from phase predicting module.At the same time,a loss function consists of several parts is designed to be optimized,which concerns matching loss,prediction loss and consistency loss,corresponding to correspondence retrieval module,phase predicting module and the joint optimization respectively.Secondly,the application of Pareto optimization based on parallel multi-task learning in phase-shifting profilometry of highly reflective surfaces is derived.In this method,overexposed and non-overexposed areas are used as different data for network fitting.By introducing the multi-gradient descent method into multi-task weight search,this method can ensure that the network is optimized in a direction that can cause the loss function values of all tasks to decrease at all times,thus achieving relatively stable hyperparameter search in most cases.However,as mentioned earlier,setting two tasks as parallel priorities often leads to significant oscillations during network convergence,resulting in a decrease in network performance.To improve the reconstruction of high reflective surfaces based on parallel multi-task learning,a multi-task weight searching strategy with dynamic task priorities is proposed.On the basis of the above derivation,a key indicator is introduced,which is the proportion of pixels in non-overexposed areas that have completed stereo matching.In the early stage of network optimization,the task of fitting non overexposed area data is considered a high priority task.Once the indicator exceeds the specified threshold,it is considered that the network has basically completed the fitting of non-overexposed areas.At this time,the priority between tasks is adjusted,that is,the fitting task of the network to overexposed areas is considered a high priority task.To validate the proposed multi-task weight searching strategy,the results of manually selecting task weights and parallel multi-task weight searching strategy were compared with the results of the proposed multi-task weight searching strategy,which dynamically updates task priorities.In the comparative experiment,multiple test objects containing highly reflective areas were selected and reconstructed using three different methods.Set the incorrect matching rate as a qualitative evaluation indicator,the reconstruction results show that the multi-task weight searching strategy based on dynamic task priority proposed in this paper can achieve the most stable reconstruction performance.At the same time,compared with the parallel multi-task weight searching strategy,the network convergence speed of our method is faster and the network oscillation phenomenon is less obvious.
万方数据
维普
