1. Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion Jill E. Ellingson, Eric D. Heggestad, and Erin E. Coyne October 13 – 14, 2006 ETS Technical Advisory Group Meeting

2. Current Retesting Policy Applicants allowed to voluntarily retake assessment after period of time if displeased with outcome –Applicant elects to retake the assessment Common in organizations which use assessment tools for hiring Most often used in conjunction with cognitively- loaded assessments

3. Personality Assessment Retesting Organization directs certain applicants whose responses are likely distorted to retake the personality assessment Responses deemed distorted on basis of embedded intentional distortion scale –Flags extreme response profiles Applicants informed that responses were flagged as suspect Hiring decisions made using retested scores

4. Key Questions Does retesting flagged applicants lower previously inflated personality scale scores? What psychological mechanism operates within applicants to help explain why they would adjust their responses?

5. Hypothesis 1: Retesting flagged individuals will result in decreased personality scale scores in the second assessment relative to the first assessment. Scale Score Changes Flagged applicants have positively biased score profiles Retest effect evident in degree to which second assessment scores are lower Preliminary research suggests that scores may be lowered up to 0.7 standard deviation units (Ellingson & Heggestad, 2003)

6. Role of Guilt: Appraisal Theory Event Evaluation Factors Relevance? Congruence? Associated values? Accountability? Coping potential? Behavior Emotion

7. Role of Guilt: Applicant Appraisal Told to retest Evaluation Factors Personally relevant Incongruent Violates personal standards Personally accountable Coping potential? Guilt Behavior Hypothesis 2: Retesting flagged individuals will result in increased feelings of guilt in the second assessment relative to the first assessment.

8. Role of Guilt: Applicant Appraisal Told to retest Evaluation Factors Personally relevant Incongruent Violates personal standards Personally accountable Make reparation Guilt Hypothesis 3: The level of guilt reported by flagged individuals in the second assessment will moderate the degree to which personality scale scores change. Respond honestly

9. Sample and Measures 288 undergraduate students Measures: –NEO-Five Factor Inventory (NEO-FFI) –Balanced Inventory of Desirable Responding- Impression Management scale (BIDR-IM) –Personal Feelings Questionnaire-2 Guilt scale (PFQ2-G) –Positive and Negative Affect Schedule-Expanded (PANAS-X) –Test of Self-Conscious Affect-3 (TOSCA3)

10. Procedure All participants: 1.Completed the TOSCA3 2.Completed NEO-FFI and BIDR-IM under motivating instructions 3.Completed the PFQ2-G and PANAS-X regarding feelings had while taking the personality measure Sorted participants into 3 groups based on BIDR-IM score Low Control Group Low BIDR-IM score High Control Group High BIDR-IM score Flagged Retest Group High BIDR-IM score Retested for neutral reason Told responses were suspect and unusable Asked to retest Time 1 Time 2

11. Effect Sizes NEO-FFILow ControlHigh ControlFlagged Retest Extraversion 0.360.290.45 Conscientiousness 0.040.220.59 Agreeableness 0.120.260.56 Openness 0.040.100.28 Emotional Stability-0.100.050.27 Average d 0.090.180.43 Impression Management-0.03-0.130.54 State Guilt 0.20 0.330.07 Positive values indicate that Time 1 score was larger than Time 2 score.

12. Repeated-measures MANCOVA: Personality Scales SourcePillai’s TraceFdfpη2η2 Within-subjects effects Time 0.0181.045 (5, 280)0.3920.018 Time x Trait Guilt 0.010.577 (5, 280)0.7180.010 Time x Condition.1705.232(10, 562)0.0000.085 Between-subjects effects Trait Guilt 0.0935.739 (5, 280)0.0000.093 Condition 0.2558.229(10, 562)0.0000.128 Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt

13. Repeated-measures ANCOVA: Personality Scales Within-subjects effectsBetween-subjects effects TimeTime x GuiltTime x ConditionTrait GuiltCondition NEO-FFIFη2η2 Fη2η2 Fη2η2 Fη2η2 Fη2η2 Extraversion2.320.0080.060.0002.200.0151.100.0040.750.005 Conscientiousness0.950.0030.010.00016.78*0.106 4.02*0.01417.25*0.108 Agreeableness1.300.0050.010.000 7.84*0.0521.240.00433.07*0.189 Openness2.620.0091.010.004 6.61*0.044 8.21*0.0280.380.003 Emotional Stability 2.450.0091.550.00512.59*0.081 6.26*0.022 7.94*0.053 Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt * p <.05

14. Repeated-measures ANCOVA: Agreeableness Interaction

15. Repeated-measures ANCOVA: State Guilt SourcePillai’s TraceFdfpη2η2 Within-subjects effects Time 0.0041.274(1, 283)0.2600.004 Time x Trait Guilt 0.002.466(1, 283)0.4950.002 Time x Condition 0.004.537(2, 283)0.5850.004 Between-subjects effects Trait Guilt 81.3534.500(1, 283)0.0350.016 Condition 244.1066.752(2, 283)0.0010.046 Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt

16. Moderated Regressions: Understanding Score Change Full Model Predicting Time 2 Personality Scale Scores Standardized Beta Coefficients and Variance Explained PredictorsExtraversionConscientiousnessAgreeablenessOpenness Emotional Stability Step 1 Time 1 Score.838*.888*.829*.910*.850* Step 2 Time 2 State Guilt-.083-.195*-.291* -.138*.141* Low Control Condition.097.176*.064.106* -.163* High Control Condition.087.143*.101*.085* -.114* Step 3 Guilt x Low Control.074.104*.225*.135*-.072 Guilt x High Control.077.123*.098*.058†-.077† R-squared.688.730.726.826.722 Δ R-squared.004.010*.018*.007*.004 * p <.05 † p <.10

17. Moderated Regressions: Conscientiousness Interaction

18. Conclusion Retesting flagged applicants will result in a set of personality scale scores that are less positively inflated The appraisal profile of guilt helps explain this effect –Flagged applicants who feel guilty as a result of being retested decrease their scores in response.