In this work,a visual language model is introduced in ophthalmic image disease recognition.And a multi-disease recognition algorithm based on a pre-trained contrasting language-images model is proposed.First,a multi-labeled fundus image dataset MDFCD8 containing 8 categories is constructed based on several publicly available fundus image datasets.Then,the generative artificial intelligence GPT-4(Generative Pre-trained Transformer 4)is utilized to generate expert knowledge describing the fine-grained pathological features of fundus images,which solves the problem of the lack of text labels in fundus image datasets.The paper calculates the average precision(AP),F1 score,and area under the receiver operating characteristic curve(AUC),and takes the mean value of the three as the final performance evaluation index.The experimental results showed that,the method proposed in this paper outperforms the traditional convolutional neural network and Transformer network by 4.8%and 3.2%,respectively.This study also conducted ablation experiments on each module to validate the effectiveness of the method,demonstrating the potential application of visual language modeling in the field of auxiliary diagnosis of ophthalmic diseases.
维普
万方数据