首页|Logit prototype learning with active multimodal representation for robust open-set recognition

Logit prototype learning with active multimodal representation for robust open-set recognition

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Robust open-set recognition(OSR)performance has become a prerequisite for pattern recogni-tion systems in real-world applications.However,the existing OSR methods are primarily implemented on the basis of single-modal perception,and their performance is limited when single-modal data fail to provide sufficient descriptions of the objects.Although multimodal data can provide more comprehensive informa-tion than single-modal data,the learning of decision boundaries can be affected by the feature representation gap between different modalities.To effectively integrate multimodal data for robust OSR performance,we propose logit prototype learning(LPL)with active multimodal representation.In LPL,the input multimodal data are transformed into the logit space,enabling a direct exploration of intermodal correlations without the impact of scale inconsistency.Then,the fusion weights of each modality are determined using an entropy-based uncertainty estimation method.This approach realizes adaptive adjustment of the fusion strategy to provide comprehensive descriptions in the presence of external disturbances.Moreover,the single-modal and multimodal representations are jointly optimized interactively to learn discriminative decision boundaries.Finally,a stepwise recognition rule is employed to reduce the misclassification risk and facilitate the distinc-tion between known and unknown classes.Extensive experiments on three multimodal datasets have been done to demonstrate the effectiveness of the proposed method.

logit prototype learningmultimodal perceptionopen-set recognitionuncertainty estimation

Yimin FU、Zhunga LIU、Zicheng WANG

展开 >

School of Automation,Northwestern Polytechnical University,Xi'an 710072,China

National Natural Science Foundation of ChinaInnovation Foundation for Doctor Dissertation of Northwestern Polytechnical UniversityCultivation Foundation for Excellent Doctoral Dissertation of the School of Automation of Northwestern Polytechnical University

U20B2067CX2023015

2024

10.1007/s11432-023-3924-x

中国科学:信息科学(英文版)
中国科学院

中国科学:信息科学(英文版)

CSTPCDEI
影响因子：0.715
ISSN：1674-733X
年,卷(期)：2024.67(6)