首页|Assessing the risk of takeover catastrophe from large language models
Assessing the risk of takeover catastrophe from large language models
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
Wiley
This article presents a risk analysis of large language models (LLMs), a type of “generative”artificial intelligence (AI) system that produces text, commonly in responseto textual inputs from human users. The article is specifically focused on the risk ofLLMs causing an extreme catastrophe in which they do something akin to taking overthe world and killing everyone. The possibility of LLM takeover catastrophe has been amajor point of public discussion since the recent release of remarkably capable LLMssuch as ChatGPT and GPT-4. This arguably marks the first time when actual AI systems(and not hypothetical future systems) have sparked concern about takeover catastrophe.The article’s analysis compares (A) characteristics of AI systems that may be neededfor takeover, as identified in prior theoretical literature on AI takeover risk, with (B)characteristics observed in current LLMs. This comparison reveals that the capabilitiesof current LLMs appear to fall well short of what may be needed for takeover catastrophe.Future LLMs may be similarly incapable due to fundamental limitations of deeplearning algorithms. However, divided expert opinion on deep learning and surprisecapabilities found in current LLMs suggests some risk of takeover catastrophe fromfuture LLMs. LLM governance should monitor for changes in takeover characteristicsand be prepared to proceed more aggressively if warning signs emerge. Unless and untilsuch signs emerge, more aggressive governance measures may be unwarranted.
artificial intelligencecatastrophic risklarge language models
Seth D. Baum
展开 >
Global Catastrophic Risk Institute, Washington,District of Columbia, USA
NETL
NSTL
Wiley
