首页|The Life Cycle of Knowledge in Big Language Models:A Survey

The Life Cycle of Knowledge in Big Language Models:A Survey

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Knowledge plays a critical role in artificial intelligence.Recently,the extensive success of pre-trained language models(PLMs)has raised significant attention about how knowledge can be acquired,maintained,updated and used by language models.Des-pite the enormous amount of related studies,there is still a lack of a unified view of how knowledge circulates within language models throughout the learning,tuning,and application processes,which may prevent us from further understanding the connections between current progress or realizing existing limitations.In this survey,we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods,and investigating how knowledge circulates when it is built,maintained and used.To this end,we systematically review existing studies of each period of the knowledge life cycle,summarize the main challenges and current limitations,and discuss future directions1.

Pre-trained language modelknowledge acquisitionknowledge representationknowledge probingknowledge editingknowledge application

Boxi Cao、Hongyu Lin、Xianpei Han、Le Sun

展开 >

Chinese Information Processing Laboratory,Beijing 100190,China

University of Chinese Academy of Sciences,Beijing 101408,China

State Key Laboratory of Computer Science,Institute of Software,Chinese Academy of Sciences,Beijing 100190,China

National Natural Science Foundation of ChinaCAS Project for Young Scientists in Basic Research,China

62 122 077YSBR-040

2024

10.1007/s11633-023-1416-x

机器智能研究(英文)
中国科学院自动化所

机器智能研究(英文)

CSTPCDEI
影响因子：0.49
ISSN：2731-538X
年,卷(期)：2024.21(2)
  • 176