Evaluation of the Application Potential and Challenges of Large Language Models in the Field of Laboratory Medicine
Objective To evaluate the performance of ChatGPT-4.0 and ERNIE Bot-4.0 in the field of laboratory medicine,and ex-plore their application potential and challenges in this professional domain.Methods Using the national clinical medical laboratory technology(intermediate)examination questions as a benchmark,we compared the performance of the two models in mastering labora-tory medicine knowledge and answering consistency.We also and assessed the models'ability in interpreting test results and assisting diagnosis through 30 laboratory medicine cases.Results In the clinical medical examination technology test,both models passed the 60％qualification threshold.ChatGPT-4.0 was superior to ERNIE Bot-4.0 in terms of answering speed and consistency,but its answer-ing accuracy was significantly lower than that of ERNIE Bot-4.0(73.25％vs 80.75％).ERNIE Bot-4.0's accuracy rate was higher than the average accuracy rate of clinical aboratory personnel in this examination(78.03％).In the accuracy analysis of different question types,both performed worst in experimental technology questions(ERNIE Bot-4.0:66.32％,ChatGPT-4.0:60.53％)and best in bas-ic medical knowledge questions(both scoring 86.00％).In the case analysis test,ERNIE Bot-4.0 outperformed ChatGPT-4.0 in all cat-egories.Both models performed well in routine case analysis but made errors in complex case analysis.Conclusion In the field of la-boratory medicine,both large language models have shown certain application potential,especially in a Chinese context,where ERNIE Bot-4.0 significantly outperforms ChatGPT-4.0 in terms of answering accuracy and case analysis ability,indicating its relative advantage in domestic applications.However,both models still need improvement in experimental technical knowledge,complex case analysis ca-pabilities,and the accuracy and consistency of result output.At the current stage,there are still certain risks in directly applying such general large language models to clinical test result interpretation and assisted diagnosis,which provides a new research direction for the interpretation of test reports.
large language modelmedical laboratoryartificial intelligenceresult interpretationcase analysis
NETL
NSTL
万方数据
