Automatic Identification Method of Policy Irony Comments Based on Large Language Models
Policy irony comments are extreme and sharp expressions whereby the public voices their opinions on public policies.Automatic and accurate identification is crucial for monitoring policy opinions.Given the scarcity of research on automatic identification methods for policy irony comments and the multiple difficulties involved,this paper proposes a method for automatically identifying policy irony comments based on large language model frameworks.Specifically,us-ing the ChpoBERT,LLaMA-2,GPT-2,and StructBERT frameworks,models for the automatic identification of policy iro-ny comments were constructed and compared.Based on a dataset of 111,628 valid policy comments collected from Sina Weibo,the first dataset of policy irony comments was manually annotated.Additionally,based on the presence or absence of topic labels,the data were further divided into two datasets—one with and one without topic labels—for model training and evaluation.We found that the model built on ChpoBERT achieved the best performance in terms of accuracy,recall,and F1 score,followed by the model built on LLaMA-2.After fine-tuning,the models demonstrated certain performance guarantees.The models constructed in this study establish clear and comparable baseline models for research on the accu-rate identification of policy irony comments,providing methodological support for policy sentiment monitoring.
policy irony commentslarge language modelsautomatic identificationpublic opinion on policiespolicy in-formatics
NETL
NSTL
万方数据
