Detection of Back Random Responding in Forced-Choice Questionnaires:The Change Point Analysis Method
Psychological tests mainly include cognitive tests and personality tests.In cognitive tests,participants can make a correct choice when they know the correct answer,regardless of the guessing factor.However,in personality tests,participants are free to improve their scores.Previous studies have shown that traditional personality tests are prone to acquiescence response,halo effect,impression management,and other abnormal responses.Many methods used to detect abnormal responses are only proposed for ability tests,but are lacking for personality tests.With the wide application of personality tests in the field of talent assessment,it becomes more urgent to detect whether there is an abnormal response in the test.The development of forced-choice questionnaires avoids the disadvantages of traditional personality tests to some extent.However,personality test is still affected by response style and random response,especially by back random response.Because the length of personality tests is long or the motivation of the participants is low,the participants are prone to random response in the latter part of the test(BRR).BRR is a common abnormal phenomenon in psychological tests.It can increase the error of potential trait estimation,which cannot reflect the real trait level of the participants.At the same time,it can seriously reduce the reliability and validity of the test.Change point analysis is a popular method for detecting abnormal responses in psychological tests.The advantage of CPA is that it can identify not only the abnormal response of a particular participant,but also detect the specific location of the change point(Shao,2016).Therefore,the CPA method can help researchers clean up the abnormal part of the data independently without deleting all the data of the participants during data analysis.In this way,the influence of abnormal response can be reduced,the valid data can be retained to the maximum extent and the accuracy of parameter estimation can be improved.On the basis of previous studies and in combination with the special nature of CPA and BRR,the study applied the existing methods of CPA to forced-choice questionnaires for the first time.Under the framework of MUPP-2PL,the existing methods Lmax,Rmax,and Wmax of CPA were compared and verified through simulation study.This was to provide an effective and reasonable method for detecting abnormal participants in forced-choice questionnaires.Monte Carlo simulation was used in this study.Firstly,under the framework of MUPP-2PL,the distribution characteristics of L,,,ax,R,,,ax,and Wmax in different test length and dimension correlation were discussed,and the 95th percentile of their respective experience distribution was obtained as the critical value(i.e.the criterion for judging whether BRR existed in the process of the test).Secondly,the detection effects of Lmax,Rmax,and Wmax on BRR were verified under different BRR prevalence,dimension correlation,BRR severity and test length.In all the experimental conditions,the type-Ⅰ errors of Lmax,Rmax,andWmax were close to the level of significance(α=.05).The power of is much higher than that of the other two methods.The absolute lag of is relatively the most accurate.The results of potential trait estimation showed that the accuracy of potential trait estimation was significantly improved after the CPA method was used to clean the abnormal response data.In general,the results of all CPA methods for BRR detection in forced-choice questionnaires were satisfactory and the results of the three methods were highly consistent(κ＞.61,p＜.001).In addition,the results of empirical data have also reached similar conclusions.The labeling overlap rate of Wlmax and Lmax for abnormal subjects reached 46.7％,but the labeling overlap rate of Rmax,Wmax,and Lmax were lower respectively.
forced-choice questionnaireschange point analysisback random responsemulti-unidimensional pairwise-preference two-parameter logistic model
万方数据
维普
