Study on Hypernymy Recognition Based on Combined Training of Attention Mechanism and Prompt Learning
The hypernymy between patent terms is an important semantic relationship.The identification of hypernymy between terms in patent text plays an important role in patent retrieval,query expansion,knowledge graph construction and other fields.However,due to the diversity of patent field and the complexity of language expression,the task of identifying the hypernymy be-tween terms still faces many challenges.This paper proposes a method to recognize the hypernymy of terms by integrating prompt learning and attention mechanism.This method is based on the distantly supervised framework,and uses the shortest de-pendent path between terms as an auxiliary feature to integrate into the prompt template.Graph neural network is used to in-tegrate the common information between terms into the joint training process of prompt learning and attention mechanism.Expe-rimental results on the patent text test dataset show that the AUC and f1 value of our method reache 94.94％and 89.33％,re-spectively,which are 3.82％and 3.17％higher than the PARE model.This method effectively removes the noise of the dataset annotated using distantly supervised methods,avoids the mismatch problem between the training target of the masked language model and downstream tasks,and fully utilizes the language knowledge information existing in the pre-trained language model.
Term relationship recognitionDistant supervisionPrompt learningAttention mechanismHypernymy
NETL
NSTL
万方数据
