通用人工智能中的Long-Context技术分析
Analysis of Long-Context Technology in General Artificial Intelligence
赵立东 1郭敬明 1蒋燕1
作者信息
- 1. 上海燧原科技股份有限公司,上海 201306
- 折叠
摘要
阐述通用人工智能中的Long-Context技术原理,提出该技术的解决方案,包括位置编码外推、注意力机制计算优化、模型结构优化和检索增强生成.分析硬件系统的瓶颈,探讨Long-Context在通用人工智能时代的发展趋势.
Abstract
This paper expounds the principle of Long Context technology in general artificial intelligence and proposes a solution for this technology,including position encoding extrapolation,attention mechanism calculation optimization,model structure optimization,and retrieval enhancement generation.It analyzes the bottlenecks of hardware systems and explores the development trend of Long Context in the era of general artificial intelligence.
关键词
Long-Context/位置编码/注意力机制/检索增强生成Key words
Long Context/positional encoding/attention mechanism/retrieval enhanced generation引用本文复制引用
出版年
2024
NETL
NSTL
万方数据