Probabilistic Graph Models for Peer Assessment Based on Student Grading Ability Under the Setting of Spot-checking
With the proliferation of many MOOC platforms,grading open-ended assignments submitted by many students presents a significant challenge in educational research.Peer assessment,which requires students to act as peer graders and evaluate their peers'submissions of assign-ments,is the mainstream solution to address this issue.Researchers have recently proposed various probabilistic graph models to evaluate peer graders'grading reliability and bias,effectively improving the estimated actual scores of assignments based on peer grades.However,the existing probabilistic graph models consider only the impact of students'scores on the current assignment regarding their grading reliability,failing to ac-count for their scoring deviation,which directly measures their reliability.This limitation affects the performance of these models.Therefore,this study proposes two novel probabilistic graph models,RPG6 and RPG7,which incorporate the peer graders'grading ability,quantified based on their score deviation within a small proportion of submissions being spot-checked by teachers.These models,constructed on the foundation of two existing probabilistic graph models,represent the grading reliability of peer graders as a variable dependent on their scoring deviation-aware grading ability rather than their scores for the current assignment.This approach enhances the estimation of the true scores of assignments.Real classroom experiments demonstrated that the proposed RPG6 and RPG7 models achieve greater accuracy in estimating the true scores of assign-ments in peer assessment activities.Specifically,the RMSE values of RPG6 and RPG7 are,on average,11.75%lower than those of the state-of-the-art method.
万方数据
