Opening the"Black Box"of Peer Review:An Analysis of Reviewer Behavior
Traditional peer review has been criticized for its lack of transparency and oversight,and open peer review provides an opportunity to open the black box of peer review,enhance supervision,and standardize the review behavior.Based on 2,534 manuscripts submitted to ICLR conferences from 2017 to 2019 and 7,602 related reviews from the OpenReview platform,this article conducts a quantitative and qualitative analysis of the behavioral characteristics of reviewers,focusing on expert confidence,review patterns,and review outcomes.The study shows that highly confident reviewers have a better understanding of the manuscripts and tend to provide more differentiated scores and specific comments;double-blind reviewers are more stringent in their scoring than their single-blind counterparts and less likely to mention language quality.In addition,comments on the core content play a crucial role in the final decision on manuscripts,those that receive more positive comments on experimental design and originality are more likely to receive better results.The article proposes some recommendations regarding the selection of reviewers,the types of peer review and the setting of review indexes,in order to improve the quality of peer review and promote peer review reform.
NETL
NSTL
万方数据
