首页|电力非结构化大文本特征提取研究

电力非结构化大文本特征提取研究

Research on feature extraction of unstructured large power texts

扫码查看
电力大文本中存在大量专业词汇缩写和别名等不规则表达,现有分词工具无法有效识别电气工程领域专业词汇,这对非结构化文本的分析和利用造成很大影响.首先,根据电气工程领域非结构化文本特点,提出一种电气工程领域词汇索引规则,基于该索引规则构建的索引集进行分词能够有效改善分词效果,为电力文本特征提取提供基础.其次,利用有效的长文本分割算法保留原始文本语义信息,将基于BERT模型提取的文本特征信息与Word2Vec提取的电力词汇特征信息进行联合嵌入,从而提取到准确的电力非结构化大文本特征.最后,通过实验证明了所提出的电力非结构化大文本特征提取方法的有效性.
Large power texts contain numerous abbreviations of technical terms,alternative names,and irregular ex-pressions.Existing word segmentation tools often fail to identify specialized vocabulary in the electrical engineering field,significantly hindering the analysis and utilization of unstructured texts.To address this challenge,this paper proposes a set of indexing rules tailored to the characteristics of unstructured texts in electrical engineering.Segmen-tation based on these rules can significantly enhance segmentation accuracy,laying a solid foundation for feature ex-traction of power texts.Furthermore,by employing effective long-text segmentation algorithms to preserve the seman-tic information of the original text,the paper integrates and embeds text feature information extracted by the BERT model with vocabulary feature information extracted by Word2Vec.This combined approach enables the extraction of precise features from large unstructured power texts.Finally,experimental results have demonstrated the effec-tiveness of the proposed method for extracting features from large unstructured power texts.

large power textfeature extractionBERTtext segmentationintegrate and embed

王家凯、黄佩卓、李勇乐、盛爽、刘洋、郑玲、魏振华

展开 >

国家电网有限公司大数据中心,北京 100052

华北电力大学,北京 100026

电力大文本 特征提取 BERT 文本分割 联合嵌入

国家自然科学基金国家电网公司大数据中心科技专项

62373150SGSJ0000YYJS2310054

2024

浙江电力
浙江省电力学会 浙江省电力试验研究院

浙江电力

CSTPCD
影响因子:0.438
ISSN:1007-1881
年,卷(期):2024.43(6)
  • 8