查看更多>>摘要：Learning a generalizable medical image segmentation model is an important but challenging task since the unseen (testing) domains may have significant discrepancies from seen (training) domains due to different vendors and scanning protocols. Existing segmentation methods, typically built upon domain generalization (DG), aim to learn multi-source domain-invariant features through data or feature augmentation techniques, but the resulting models either fail to characterize global domains during training or cannot sense unseen domain information during testing. To tackle these challenges, we propose a domain Unifying and Adapting network (UniAda) for generalizable medical image segmentation, a novel “unifying while training, adapting while testing” paradigm that can learn a domain-aware base model during training and dynamically adapt it to unseen target domains during testing. First, we propose to unify the multi-source domains into a global inter-source domain via a novel feature statistics update mechanism, which can sample new features for the unseen domains, facilitating the training of a domain base model. Second, we leverage the uncertainty map to guide the adaptation of the trained model for each testing sample, considering the specific target domain may be outside the global inter-source domain. Extensive experimental results on two public cross-domain medical datasets and one in-house cross-domain dataset demonstrate the strong generalization capacity of the proposed UniAda over state-of-the-art DG methods. The source code of our UniAda is available at https://github.com/ZhouZhang233/UniAda.
Manuel TranSophia WagnerWilko WeichertChristian Matek...
2002-2015页
查看更多>>摘要：Building deep learning models that can rapidly segment whole slide images (WSIs) using only a handful of training samples remains an open challenge in computational pathology. The difficulty lies in the histological images themselves: many morphological structures within a slide are closely related and very similar in appearance, making it difficult to distinguish between them. However, a skilled pathologist can quickly identify the relevant phenotypes. Through years of training, they have learned to organize visual features into a hierarchical taxonomy (e.g., identifying carcinoma versus healthy tissue, or distinguishing regions within a tumor as cancer cells, the microenvironment, …). Thus, each region is associated with multiple labels representing different tissue types. Pathologists typically deal with this by analyzing the specimen at multiple scales and comparing visual features between different magnifications. Inspired by this multi-scale diagnostic workflow, we introduce the Navigator, a vision model that navigates through WSIs like a domain expert: it searches for the region of interest at a low scale, zooms in gradually, and localizes ever finer microanatomical classes. As a result, the Navigator can detect coarse-grained patterns at lower resolution and fine-grained features at higher resolution. In addition, to deal with sparsely annotated samples, we train the Navigator with a novel semi-supervised framework called S5CL v2. The proposed model improves the F1 score by up to 8% on various datasets including our challenging new TCGA-COAD-30CLS and Erlangen cohorts.
查看更多>>摘要：Accurate medical image segmentation is often hindered by noisy labels in training data, due to the challenges of annotating medical images. Prior research works addressing noisy labels tend to make class-dependent assumptions, overlooking the pixel-dependent nature of most noisy labels. Furthermore, existing methods typically apply fixed thresholds to filter out noisy labels, risking the removal of minority classes and consequently degrading segmentation performance. To bridge these gaps, our proposed framework, Collaborative Learning with Curriculum Selection (CLCS), addresses pixel-dependent noisy labels with class imbalance. CLCS advances the existing works by i) treating noisy labels as pixel-dependent and addressing them through a collaborative learning framework, and ii) employing a curriculum dynamic thresholding approach adapting to model learning progress to select clean data samples to mitigate the class imbalance issue, and iii) applying a noise balance loss to noisy data samples to improve data utilization instead of discarding them outright. Specifically, our CLCS contains two modules: Curriculum Noisy Label Sample Selection (CNS) and Noise Balance Loss (NBL). In the CNS module, we designed a two-branch network with discrepancy loss for collaborative learning so that different feature representations of the same instance could be extracted from distinct views and used to vote the class probabilities of pixels. Besides, a curriculum dynamic threshold is adopted to select clean-label samples through probability voting. In the NBL module, instead of directly dropping the suspiciously noisy labels, we further adopt a robust loss to leverage such instances to boost the performance. We verify our CLCS on two benchmarks with different types of segmentation noise. Our method can obtain new state-of-the-art performance in different settings, yielding more than 3% Dice and mIoU improvements. Our code is available at https://github.com/Erjian96/CLCS.git.
查看更多>>摘要：The joint use of multiple modalities for medical image processing has been widely studied in recent years. The fusion of information from different modalities has demonstrated the performance improvement for a lot of medical tasks. For nephropathy diagnosis, immunofluorescence (IF) is one of the most widely-used multi-modality medical images due to its ease of acquisition and the effectiveness for certain nephropathy. However, the existing methods mainly assume different modalities have the equal effect on the diagnosis task, failing to exploit multi-modality knowledge in details. To avoid this disadvantage, this paper proposes a novel customized multi-teacher knowledge distillation framework to transfer knowledge from the trained single-modality teacher networks to a multi-modality student network. Specifically, a new attention-based diffusion network is developed for IF based diagnosis, considering global, local, and modality attention. Besides, a teacher recruitment module and diffusion-aware distillation loss are developed to learn to select the effective teacher networks based on the medical priors of the input IF sequence. The experimental results in the test and external datasets show that the proposed method has a better nephropathy diagnosis performance and generalizability, in comparison with the state-of-the-art methods.
查看更多>>摘要：Chest X-ray (CXR) images have been widely adopted in clinical care and pathological diagnosis in recent years. Some advanced methods on CXR classification task achieve impressive performance by training the model statically. However, in the real clinical environment, the model needs to learn continually and this can be viewed as a domain incremental learning (DIL) problem. Due to large domain gaps, DIL is faced with catastrophic forgetting. Therefore, in this paper, we propose a Cross-domain invariant feature absorption and Domain-specific feature retention (CaD) framework. To be specific, we adopt a Cross-domain Invariant Feature Absorption (CIFA) module to learn the domain invariant knowledge and a Domain-Specific Feature Retention (DSFR) module to learn the domain-specific knowledge. The CIFA module contains the C(lass)-adapter and an absorbing strategy is used to fuse the common features among different domains. The DSFR module contains the D(omain)-adapter for each domain and it connects to the network in parallel independently to prevent forgetting. A multi-label contrastive loss (MLCL) is used in the training process and improves the class distinctiveness within each domain. We leverage publicly available large-scale datasets to simulate domain incremental learning scenarios, extensive experimental results substantiate the effectiveness of our proposed methods and it has reached state-of-the-art performance.
Christoforos GalazisSamuel ShepperdEmma J. P. BrouwerSandro Queirós...
2056-2067页
查看更多>>摘要：The functional analysis of the left atrium (LA) is important for evaluating cardiac health and understanding diseases like atrial fibrillation. Cine MRI is ideally placed for the detailed 3D characterization of LA motion and deformation but is lacking appropriate acquisition and analysis tools. Here, we propose tools for the Analysis of Left Atrial Displacements and DeformatIons using online learning neural Networks (Aladdin) and present a technical feasibility study on how Aladdin can characterize 3D LA function globally and regionally. Aladdin includes an online segmentation and image registration network, and a strain calculation pipeline tailored to the LA. We create maps of LA Displacement Vector Field (DVF) magnitude and LA principal strain values from images of 10 healthy volunteers and 8 patients with cardiovascular disease (CVD), of which 2 had large left ventricular ejection fraction (LVEF) impairment. We additionally create an atlas of these biomarkers using the data from the healthy volunteers. Results showed that Aladdin can accurately track the LA wall across the cardiac cycle and characterize its motion and deformation. Global LA function markers assessed with Aladdin agree well with estimates from 2D Cine MRI. A more marked active contraction phase was observed in the healthy cohort, while the CVD $\text {LVEF}_{\downarrow } $ group showed overall reduced LA function. Aladdin is uniquely able to identify LA regions with abnormal deformation metrics that may indicate focal pathology. We expect Aladdin to have important clinical applications as it can non-invasively characterize atrial pathophysiology. All source code and data are available at: https://github.com/cgalaz01/aladdin_cmr_la.
查看更多>>摘要：Photoacoustic tomography holds tremendous potential for neuroimaging due to its functional magnetic resonance imaging (fMRI)-like functional contrast and greater specificity, richer contrast, portability, open platform, faster imaging, magnet-free and quieter operation, and lower cost. However, accounting for the skull-induced acoustic distortion remains a long-standing challenge due to the problem size. This is aggravated in functional imaging, where high accuracy is needed to detect minuscule functional changes. Here, we develop an acoustic solver based on the boundary-element method (BEM) to model the skull and de-aberrate the images. BEM uses boundary meshes and compression for superior computational efficiency compared to volumetric discretization-based methods. We demonstrate BEM’s higher accuracy and favorable scalability relative to the widely used pseudo-spectral time-domain method (PSTD). In imaging through an ex-vivo adult human skull, BEM outperforms PSTD in several metrics. Our work establishes BEM as a valuable and naturally suited technique in photoacoustic tomography and lays the foundation for BEM-based de-aberration methods.
查看更多>>摘要：Federated learning (FL) methods for multi-organ segmentation in CT scans are gaining popularity, but generally require numerous rounds of parameter exchange between a central server and clients. This repetitive sharing of parameters between server and clients may not be practical due to the varying network infrastructures of clients and the large transmission of data. Further increasing repetitive sharing results from data heterogeneity among clients, i.e., clients may differ with respect to the type of data they share. For example, they might provide label maps of different organs (i.e. partial labels) as segmentations of all organs shown in the CT are not part of their clinical protocol. To this end, we propose an efficient communication approach for FL with partial labels. Specifically, parameters of local models are transmitted once to a central server and the global model is trained via knowledge distillation (KD) of the local models. While one can make use of unlabeled public data as inputs for KD, the model accuracy is often limited due to distribution shifts between local and public datasets. Herein, we propose to generate synthetic images from clients’ models as additional inputs to mitigate data shifts between public and local data. In addition, our proposed method offers flexibility for additional finetuning through several rounds of communication using existing FL algorithms, leading to enhanced performance. Extensive evaluation on public datasets in few communication FL scenario reveals that our approach substantially improves over state-of-the-art methods.
查看更多>>摘要：Denoising diffusion models have emerged as the go-to generative framework for solving inverse problems in imaging. A critical concern regarding these models is their performance on out-of-distribution tasks, which remains an under-explored challenge. Using a diffusion model on an out-of-distribution dataset, realistic reconstructions can be generated, but with hallucinating image features that are uniquely present in the training dataset. To address this discrepancy and improve reconstruction accuracy, we introduce a novel test-time adaptation sampling framework called Steerable Conditional Diffusion. Specifically, this framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. Utilising the proposed method, we achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities, advancing the robust deployment of denoising diffusion models in real-world applications.
NETL
NSTL
IEEE