Conventional relation extraction methods identify the relationships between pairs of entities from plain text,whereas multimodal relation extraction methods enhance relation extraction by leveraging information from multiple modalities.To address the issue of existing multimodal relation extraction models being easily disturbed by redundant information when processing image data,this study proposes a multimodal relation extraction model based on a bidirectional attention mechanism.First,Bidirectional Encoder Representations from Transformers(BERT)and a scene graph-generation model are used to extract textual and visual semantic features,respectively.Subsequently,a bidirectional attention mechanism is employed to establish bidirectional alignment between images and text,and from text to images,thus facilitating bidirectional information exchange.This mechanism assigns lower weights to redundant information in images,thereby reducing interference to the semantic representation of text and mitigating the adverse effect of redundant information on the result of relation extraction.Finally,the aligned textual and visual feature representations are concatenated to form integrated text and image features.A Multi-Layer Perceptron(MLP)is used to calculate the probability scores for all relation classifications and output the predicted relations.Experimental results on a Multimodal dataset for Neural Relation Extraction(MNRE)show that the model achieves precision,recall,and F1 scores of 65.53％,69.21％,and 67.32％,respectively,which are significantly higher than those of baseline models,thus demonstrating its effective improvement in relation extraction.
维普
万方数据