光学精密工程2024,Vol.32Issue(18) :2836-2845.DOI:10.37188/OPE.20243218.2836

基于自然图像模型微调的小鼠脑部电镜图像实例分割

Instance segmentation of mouse brain scanning electron microscopy images based on fine-tuning nature image model

承骜 赵国强 张若冰 王丽荣
光学精密工程2024,Vol.32Issue(18) :2836-2845.DOI:10.37188/OPE.20243218.2836

基于自然图像模型微调的小鼠脑部电镜图像实例分割

Instance segmentation of mouse brain scanning electron microscopy images based on fine-tuning nature image model

承骜 1赵国强 2张若冰 2王丽荣1
扫码查看

作者信息

  • 1. 苏州大学 电子信息学院,江苏 苏州 215000
  • 2. 中国科学院 苏州生物医学工程研究所,江苏 苏州 215000
  • 折叠

摘要

分割模型的准确性和鲁棒性是小鼠脑电镜图像处理中的主要考虑因素.针对电镜图像的技术特点,提出了高度稳健的二维分割算法,准确识别每张切片中各物体的形态结构.本文提出了基于大型自然图像模型的主干网络微调的大尺度二维电镜图像分割模型EM-SAM,用于脑部电镜图像中的实例分割.模型主干网络采用大型自然图像模型SAM中的已训练完成的图像编码器,在电镜图像处理任务中最大化模型提取图像特征的能力.此外,模型采用了U型的解码器设计,并通过小鼠脑电镜图像分割任务进行微调.实验结果表明:在公开数据集SNEMI3D中A-Rand可达到0.054;在公开数据集MitoEM中AP-50和AP-75分别可达到0.883,0.604.EM-SAM在电镜图像神经分割任务中准确性高、鲁棒性强,并且可针对不同任务进行微调.

Abstract

The accuracy and robustness of segmentation models are critical considerations in the process-ing of mouse brain electron microscopy images.We proposed a highly robust two-dimensional segmenta-tion algorithm tailored to the technical characteristics of electron microscopy images,aiming to accurately delineate the morphological structure of cells in each slice.Aiming at accurately delineate the morphologi-cal structure of cells in each section,a highly robust two-dimensional segmentation algorithm based on natu-ral image model tailored to the technical characteristics of electron microscopy images was proposed.EM-SAM was based on fine-tuning the backbone network of pre-trained large natural image model SAM for maximizing the capability of features extraction.The model employed the image encoder from the SAM architecture,augmented with a U-shaped decoder,and was fine-tuned specifically for the segmentation of mouse brain electron microscopy images.Experimental results demonstrate that A-Rand achieves 0.054 on public dataset SNEMI3D.Additionally,AP-50 and AP-75 reach 0.883 and 0.604,respectively,on public dataset MitoEM.EM-SAM exhibits high accuracy and robustness in neural segmentation tasks of electron microscopy images,and it can be fine-tuned for different tasks.

关键词

深度学习/分割/大模型/电镜图像/小鼠脑部

Key words

computer vision/segmentation/large-scale models/electron microscopy images/mouse brain

引用本文复制引用

基金项目

国家自然科学基金资助项目(32271430)

国家重点研发计划资助项目(2023YFF0715904)

出版年

2024
光学精密工程
中国科学院长春光学精密机械与物理研究所 中国仪器仪表学会

光学精密工程

CSTPCD北大核心
影响因子:2.059
ISSN:1004-924X
段落导航相关论文