The Effect of the Conscientious Responders Scale on Random Responding Rates in Psychological Questionnaires Noor Shubear, Zdravko Marjanovic, Lisa Bajkov, & Tsz Yin Fung Thompson Rivers University The validity of questionnaire data is based on the expectation that responders are conscientious responders (CR)—they answer items as honestly and accurately as they can. The problem is, for a variety of reasons such as disinterest and fatigue, some responders are random responders (RR)—they answer items carelessly or indiscriminately, without regard for what items are asking them to consider. The CRS is a 5-item validity scale (see below) that differentiates between CR and RR, which has been shown to be an effective tool in detecting RR with 93% accuracy rate. (CRS; Marjanovic, Holden, et al., Marjanovic, Struthers, et al., 2014). However, a question remains to be answered is whether the presence of the CRS scale in psychological questionnaires affects the rates of random responding. Does the presence of the CRS items in a questionnaire deter responders from answering randomly by alerting them that the validity of their data can be assessed? The purpose of this study was to test whether the presence of CRS scale in a psychological questionnaire would (1) affect the rates of random responding as compared to a questionnaire without the CRS scale, and (2) result in any significant differences in the means and standard deviations. Table 1 Random Responding Differences between CRS-Embedded and Non-Embedded Questionnaires Random NEQ CEQ Responding Mean Meant-test p-value NEU_ISD EXT_ISD OPE_ISD AGR_ISD CON_ISD __________________________________________________________________ Note. NEU=neuroticism, EXT=extraversion, OPE=openness to experience, AGR= agreeableness, CON=conscientiousness. Table 2 Mean-Scores Differences between CRS-Embedded and Non-Embedded Questionnaires NEQ CEQ MeasuresMeanMeant-test p-value NEU EXT OPE AGR CON __________________________________________________________________ Note. NEU=neuroticism, EXT=extraversion, OPE=openness to experience, AGR= agreeableness, CON=conscientiousness. Introduction The CRS utilizes instructional item content. Each CRS item, randomly embedded in a questionnaire, instructs responders exactly how to respond to that particular item. CRS 1. To answer this question, please choose number three, neither agree nor disagree. 2. Choose the first option—strongly disagree—in answering this question. 3. To respond to this question, please choose number five, strongly agree. 4. Please answer this question by choosing number two, disagree. 5. In response to this question, please choose number four, agree. Compliant responses are scored as 1s and incompliant responses are scored as 0s. Items are summed to make a score that ranges from 0 (all incompliant responses) – 5 (all compliant responses). Because, on a 5-point response scale, it is so statistically unlikely that RR can answer several CRS items compliantly by chance alone, high CRS sum scores (3 – 5) must reflect conscientious responding whereas low CRS sum scores (0– 2) probably reflect random responding. Design and ResultsResults and Discussion Hypotheses 1.The CRS-embedded questionnaire will produce smaller ISD scores compared to the non-embedded questionnaire. 2. Means and standard deviation will be about equal across both questionnaires. Method Participants and Procedure A 109 undergraduate students were randomly divided into two groups. The first group- composed of 55 students- completed a 208-item paper-and-pencil questionnaire that did not contain the CRS. The other group- composed of 54 student- completed a 218-item paper-and-pencil questionnaire which contained a double size CRS. For the purpose of this study, we only analyzed data from the 60-item five-factor inventory (NEO-FFI) to determine the effect of CRS in psychological questionnaires. To test the effect of the CRS on random responding rates, we used a statistical approach that does not alter the look and/or the content of the questionnaire as CRS does. The inter-item standard deviation (ISD; Marjanovic, Holden, et al., 2015) is statistical technique that reflects how the responses are closely clustered around the mean score. CR produces a score that is closely clustered around the mean ( i.e., small ISD score), while RR produces the opposite effect.. Measures All items were answered on a 5-point Likert scale Conscientious Responders Scale (CRS; Marjanovic, Struthers, et al., 2014) Inter-Item Standard Deviation (ISD; Marjanovic, Holden, et al., 2015) NEO Five Factor Inventory (NEO-FFI; McCrae, Robert R.; Costa, Paul T., Jr., 1978; 2010), Its 60 items measure five factors of personality; neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness. Results All studied measures produced high estimates of reliability (Cronbach alpha >.70). Hypothesis 1 statistics are presented in table 1. The results do not support the hypothesis. The presence of the CRS in a psychological questionnaires seems to have no influence on reducing or deterring rates of random responding as compared to questionnaires that do not contain CRS. Hypothesis 2 statistics are presented in table 2. The results do support the hypothesis. The presence of the CRS in psychological questionnaires seems to have no effect on means and standard deviations of psychological measures. Questionnaires with and without the CRS in it produced similar descriptive statistics. Conclusion 1. The presence of CRS items in a questionnaire had no effect of reducing or deterring rates of participant random responding. 2. A positive finding: the CRS had no effect on means and standard deviations across the CRS-embedded and non-embedded questionnaires.