Methods for Detecting Bias in Large-Scale Models Applied to Social Governance
With the rapid development of artificial intelligence,large language models(LLM)have been widely applied in many areas of social governance due to their analytical,generative,and reasoning capa-bilities.For instance,in public opinion monitoring on the Internet.However,as large models are pre-dominantly trained on open-source data,much of the textual data likely contains various biases which may lead to prejudices when deploying the models for social governance.This research thoroughly investigates political bias in large language LLMs like GPT-3.5 and LLama.Despite purported impartiality by training institutions,literature indicates LLMs frequently exhibit bias when responding to controversial topics,po-tentially causing issues in governance.We propose a novel empirical methodology requiring multiple LLMs to role-play supporting or opposing targets and design corresponding question-answering.Additional LLMs are then tested by comparing their answers against role-generated ones to infer model political bia-ses.To alleviate concerns over answer randomness,multiple responses per question were collected with randomized ordering.Experiments demonstrate significant partisan biases in different LLMs when han-dling contentious issues.These findings raise profound concerns that direct LLM application could amplify controversial information.Our research provides important implications for policymakers,media,and po-litical and academic stakeholders.
large language models(LLM)bias detectionsocial governance
NETL
NSTL
万方数据
