首页|Enhancing Orthopedic Knowledge Assessments:The Performance of Specialized Generative Language Model Optimization

Enhancing Orthopedic Knowledge Assessments:The Performance of Specialized Generative Language Model Optimization

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Objective:This study aimed to evaluate and compare the effectiveness of knowledge base-optimized and unoptimized large language models(LLMs)in the field of orthopedics to explore optimization strategies for the application of LLMs in specific fields.Methods:This research constructed a specialized knowledge base using clinical guidelines from the American Academy of Orthopaedic Surgeons(AAOS)and authoritative orthopedic publications.A total of 30 orthopedic-related questions covering aspects such as anatomical knowledge,disease diagnosis,fracture classification,treatment options,and surgical techniques were input into both the knowledge base-optimized and unoptimized versions of the GPT-4,ChatGLM,and Spark LLM,with their generated responses recorded.The overall quality,accuracy,and comprehensiveness of these responses were evaluated by 3 experienced orthopedic surgeons.Results:Compared with their unoptimized LLMs,the optimized version of GPT-4 showed improvements of 15.3％in overall quality,12.5％in accuracy,and 12.8％in comprehensiveness;ChatGLM showed improvements of 24.8％,16.1％,and 19.6％,respectively;and Spark LLM showed improvements of 6.5％,14.5％,and 24.7％,respectively.Conclusion:The optimization of knowledge bases significantly enhances the quality,accuracy,and comprehensiveness of the responses provided by the 3 models in the orthopedic field.Therefore,knowledge base optimization is an effective method for improving the performance of LLMs in specific fields.

artificial intelligencelarge language modelsgenerative articial intelligenceorthopedics

Hong ZHOU、Hong-lin WANG、Yu-yu DUAN、Zi-neng YAN、Rui LUO、Xiang-xin LV、Yi XIE、Jia-yao ZHANG、Jia-ming YANG、Ming-di XUE、Ying FANG、Lin LU、Peng-ran LIU、Zhe-wei YE

展开 >

Department of Orthopedics Surgery,Union Hospital,Tongji Medical College,Huazhong University of Science and Technology,Wuhan 430022,China

Laboratory of Intelligent Medicine,Union Hospital,Tongji Medical College,Huazhong University of Science and Technology,Wuhan 430022,China

College of Chinese Medicine,Hubei University of Chinese Medicine,Wuhan 433065,China

Department of Orthopedics,Renmin Hospital of Wuhan University,Wuhan 433060,China

展开 >

2024

10.1007/s11596-024-2929-4

当代医学科学(英文)
华中科技大学同济医学院

当代医学科学(英文)

影响因子：0.748
ISSN：2096-5230
年,卷(期)：2024.44(5)