Generative Artificial Intelligence Models in Public Education on Prevention of Cervical Cancer
[Objective]To evaluate the application of generative artificial intelligence model in public e-ducation on prevention of cervical cancer.[Methods]The available Chinese-text-generating models were chosen to create an interactive dialogue platform enabling written communication between the public and artificial intelligence models to generate public education texts about cervical cancer,The three large AI models was blindly set as models 1～3.The generated content was single-blind assessed by well-reputed public education experts in the specialization of cervical cancer by five dimensional scoring criteria(scientific accuracy,logical clarity,practical value,reference basis,stance and values).Statis-tical analyses were performed using SPSS 22.0.Paired samples t tests were used to analyze the differ-ences,and P＜0.05 was considered statistically significant.Special cases in the remarks were discussed separately.China National Knowledge Infrastructure(CNKI)was used to evaluate the repetition rate and clarify content sources.[Results]The five dimensional scores of generated content from the three models were as followed:Model 1:16.14±0.72,18.71±0.31,17.00±0.60,10.86±2.58,19.00±0.33,total score 81.71±3.85;Model 2:16.57±0.46,17.43±0.70,17.00±0.60,10.86±2.58,18.57±0.70,total score 80.43±3.00;Model 3:16.29±0.41,17.86±0.61,17.14±0.74,11.43±2.75,18.86±0.61,total score 81.57±3.92.There was no significant difference in pairwise comparison between models.The means scores of five dimensions in descending order were given as follows:stance and values(18.86±0.61),logical clarity(17.86±0.61),practical value(17.14±0.74),scientific accuracy(16.29±0.41),and reference basis(11.43±2.75).The concerns raised by experts were as follows:variables such as chang-ing questioning sentences,repeated questioning,or different questioning times may lead to differences in the generated text,some knowledge were not updated,no references were provided.Repetition rate test showed that the total copy ratio of generated content from the three models was 38.6％,44.9％,and 38.9％,respectively.The three texts were mainly generated from internet public data,and the pro-portion of content from professional journals was low.[Conclusion]The AI models generally provide sound responses to questions related to cervical cancer.No serious misleading or commercialized ten-dencies have been found,but the reference source is vague.More research is required to assess the practical value of the models.Medical experts need to pay more attention to and make endeavors to in-crease the content accuracy of the popularization of science texts on the internet.
cervical cancerscience popularizationartificial intelligencelarge model
NETL
NSTL
万方数据
