In order to solve the problems that most of the current dense video description models use two-stage methods,which have low efficiency,ignore audio and semantic information,and have incomplete description results,a multi-modal and semantic information fusion dense video description method was proposed.An adaptive R(2+1)D network was proposed to extract visual features,a semantic detector was designed to generate semantic information,audio features were added to supplement it,a multi-scale deformable attention module was established,and a parallel prediction head was applied to accelerate the convergence rate and improve the accuracy of the model.The experimental results show that the model has good performance on the two benchmark datasets,and the evaluation index BLEU4 reaches 2.17.
维普
万方数据