Vision-language models have achieved significant performance in various domains in recent years,due to their exceptional generalization capabilities.However,their generalization performance is challenged when it is applied to specialized domain data,such as ferrography data in lubricating oil.Adapting vision-language models to specific domains with limited data,especially for the adaptive learning of ferrography images,presents a new challenge.This study proposes a novel adaptive meta-learning approach that combines vision-language models with large language models,termed CLAML(CLIP-LLM Adaptation on Meta Learning).The CLAML method is based on the CLIP model,leveraging large language models(LLMs)to regenerate text descriptions.For different categories of ferrography data,LLMs generate text descriptions covering various aspects such as causes,morphology,size,and color.The multi-perspective ferrography information is then used to fine-tune the CLIP model,making it more suitable for specialized data like ferrography.The approach establishes a semantic bridge between images and text in specialized domains,enhancing zero-shot recognition capabilities.Additionally,by incorporating adaptive meta-learning methods in scenarios with limited samples,the model achieves rapid adaptation to ferrography images,and further improves their performance.Experimental results demonstrate the effectiveness of CLAML method in identifying wear types in ferrography images.
关键词
视觉语言模型/大语言模型/铁谱图像分类/零样本学习/自适应元学习
Key words
vision-language model/large language model/ferrography image classification/zero-shot learning/adaptive meta-learning
维普
万方数据