首页|情报学视角下的预训练语言模型研究进展

情报学视角下的预训练语言模型研究进展

扫码查看
原文链接
  • 万方数据
  • 维普
[目的/意义]对预训练语言模型在情报学与情报工作中的相关研究进行系统性的梳理与分析,为后续预训练模型与情报研究的融合提供借鉴.[方法/过程]首先,简述预训练模型的基本原理与发展历程,汇总情报研究中应用较为广泛的预训练模型.其次,宏观上分析预训练模型在国内外情报研究中的热点方向,微观上从情报组织、情报检索、情报挖掘等方面调研预训练模型相关研究成果,并细致分析归纳预训练模型的应用方式、改进策略与性能表现.最后,从预训练模型的语料、训练、评价、应用等方面总结当前预训练模型在情报学科中面临的机遇与挑战,展望未来发展.[结果/结论]当前BERT及其改型在情报处理中应用最广、表现最优.结合神经网络与微调的范式被用于各研究场景,尤其是领域信息抽取与文本分类任务.继续预训练、外部知识增强、架构优化等策略可进一步提升性能.如何平衡训练语料的规模与质量、提升模型易用性与安全性、高准度与多维度评价模型真实能力、加速学科知识挖掘工具落地应是未来考虑的关键问题.
Advances in Pre-trained Language Models From the Perspective of Information Science
[Purpose/Significance]This paper systematically reviews and analyzes the relevant research on pre-trained language models(PLM)in information science(IS)and intelligence work,and provides reference for the integra-tion of pre-trained models and IS research.[Method/Process]Firstly,it briefly described the basic principles and devel-opment of PLM,and summarized the widely used PLM in IS research.Secondly,it analyzed the research hotspots at the macro level and summarized the related achievements in information organization,information retrivel,and information mining at the micro level.And it explored the application methods,improvement strategies,and performance of PLM in detail.Finally,it discussed the opportunities and challenges of PLM in IS in corpus,training,evaluation,and application.[Result/Conclusion]Currently,BERT and its variants are the most widely used and perform best in IS.The paradigm combining neural networks and fine-tuning is used in various scenarios,especially in domain information extraction and text classification.Its performance can be improved by continuing pre-training,external knowledge enhancement,and architecture optimization.The key issues to be considered in the future are balancing the scale and quality of the training corpus,improving the usability and security of the model,evaluating the real ability of the model with high accuracy and multi-dimensionality,and accelerating the implementation of subject knowledge mining tools.

information scienceintelligence workpre-trained language modelsnatural language process-ingPLM

胡昊天、邓三鸿、王东波、沈思、沈健威

展开 >

南京大学信息管理学院 南京 210023

江苏省数据工程与知识服务重点实验室 南京 210023

南京农业大学信息管理学院 南京 210095

南京理工大学经济管理学院 南京 210094

江苏省质量和标准化研究院 南京 210029

展开 >

情报学 情报工作 预训练语言模型 自然语言处理 PLM

国家自然科学基金面上项目南京大学中央高校基本科研业务费专项Fundamental Research Funds for the Central Universities of Ministry of Education of China

71974094719740940108/14370317

2024

10.13266/j.issn.0252-3116.2024.03.012

图书情报工作
中国科学院文献情报中心

图书情报工作

CSTPCDCSSCICHSSCD北大核心
影响因子：2.203
ISSN：0252-3116
年,卷(期)：2024.68(3)
  • 119