In existing models for automatic medical image diagnosis report generation,only visual features of the input ima-ges are used to extract the corresponding semantic features,and there exist the weak correlations and the lack of contextual information between generated words.To address the above problems,a contrast-enhanced relational memory network model is proposed to improve the model's ability to distinguish different images through contrastive learning.An attention-en-hanced associative memory module is designed to continuously update based on the words generated at the previous time step to enhance the correlation between generated words in medical image reports,making the model capable of generating more accurate pathological information descriptions for medical images.Experimental results on the public IU X-Ray dataset and private Fetal Ultrasound dataset show that the proposed model significantly outperforms previous models in terms of Ci-der evaluation metric(compared with the classical AOANet model,the Cider metric is increased by 51.9％ on IU X-Ray and 3.0％ on the Fetal Ultrasound dataset).
关键词
医学图像报告生成/关联记忆网络/双层LSTM/上下文信息/对比学习
Key words
medical image report generation/associative memory network/double LSTM/contextual information/contras-tive learning
维普
万方数据