Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion Jill E. Ellingson, Eric D. Heggestad, and Erin E. Coyne October 13 – 14, 2006 ETS Technical Advisory Group Meeting

Current Retesting Policy Applicants allowed to voluntarily retake assessment after period of time if displeased with outcome –Applicant elects to retake the assessment Common in organizations which use assessment tools for hiring Most often used in conjunction with cognitively- loaded assessments

Personality Assessment Retesting Organization directs certain applicants whose responses are likely distorted to retake the personality assessment Responses deemed distorted on basis of embedded intentional distortion scale –Flags extreme response profiles Applicants informed that responses were flagged as suspect Hiring decisions made using retested scores

Key Questions Does retesting flagged applicants lower previously inflated personality scale scores? What psychological mechanism operates within applicants to help explain why they would adjust their responses?

Hypothesis 1: Retesting flagged individuals will result in decreased personality scale scores in the second assessment relative to the first assessment. Scale Score Changes Flagged applicants have positively biased score profiles Retest effect evident in degree to which second assessment scores are lower Preliminary research suggests that scores may be lowered up to 0.7 standard deviation units (Ellingson & Heggestad, 2003)

Role of Guilt: Appraisal Theory Event Evaluation Factors Relevance? Congruence? Associated values? Accountability? Coping potential? Behavior Emotion

Role of Guilt: Applicant Appraisal Told to retest Evaluation Factors Personally relevant Incongruent Violates personal standards Personally accountable Coping potential? Guilt Behavior Hypothesis 2: Retesting flagged individuals will result in increased feelings of guilt in the second assessment relative to the first assessment.

Role of Guilt: Applicant Appraisal Told to retest Evaluation Factors Personally relevant Incongruent Violates personal standards Personally accountable Make reparation Guilt Hypothesis 3: The level of guilt reported by flagged individuals in the second assessment will moderate the degree to which personality scale scores change. Respond honestly

Sample and Measures 288 undergraduate students Measures: –NEO-Five Factor Inventory (NEO-FFI) –Balanced Inventory of Desirable Responding- Impression Management scale (BIDR-IM) –Personal Feelings Questionnaire-2 Guilt scale (PFQ2-G) –Positive and Negative Affect Schedule-Expanded (PANAS-X) –Test of Self-Conscious Affect-3 (TOSCA3)

Procedure All participants: 1.Completed the TOSCA3 2.Completed NEO-FFI and BIDR-IM under motivating instructions 3.Completed the PFQ2-G and PANAS-X regarding feelings had while taking the personality measure Sorted participants into 3 groups based on BIDR-IM score Low Control Group Low BIDR-IM score High Control Group High BIDR-IM score Flagged Retest Group High BIDR-IM score Retested for neutral reason Told responses were suspect and unusable Asked to retest Time 1 Time 2

Effect Sizes NEO-FFILow ControlHigh ControlFlagged Retest Extraversion Conscientiousness Agreeableness Openness Emotional Stability Average d Impression Management State Guilt Positive values indicate that Time 1 score was larger than Time 2 score.

Repeated-measures MANCOVA: Personality Scales SourcePillai’s TraceFdfpη2η2 Within-subjects effects Time (5, 280) Time x Trait Guilt (5, 280) Time x Condition (10, 562) Between-subjects effects Trait Guilt (5, 280) Condition (10, 562) Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt

Repeated-measures ANCOVA: Personality Scales Within-subjects effectsBetween-subjects effects TimeTime x GuiltTime x ConditionTrait GuiltCondition NEO-FFIFη2η2 Fη2η2 Fη2η2 Fη2η2 Fη2η2 Extraversion Conscientiousness * * *0.108 Agreeableness * *0.189 Openness * * Emotional Stability * * *0.053 Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt * p <.05

Repeated-measures ANCOVA: Agreeableness Interaction

Repeated-measures ANCOVA: State Guilt SourcePillai’s TraceFdfpη2η2 Within-subjects effects Time (1, 283) Time x Trait Guilt (1, 283) Time x Condition (2, 283) Between-subjects effects Trait Guilt (1, 283) Condition (2, 283) Within-subjects factor (Time): Time 1, Time 2 Between-subjects factor (Condition): Low Control, High Control, Flagged Retest Covariate: Trait Guilt

Moderated Regressions: Understanding Score Change Full Model Predicting Time 2 Personality Scale Scores Standardized Beta Coefficients and Variance Explained PredictorsExtraversionConscientiousnessAgreeablenessOpenness Emotional Stability Step 1 Time 1 Score.838*.888*.829*.910*.850* Step 2 Time 2 State Guilt *-.291* -.138*.141* Low Control Condition * * -.163* High Control Condition *.101*.085* -.114* Step 3 Guilt x Low Control *.225*.135*-.072 Guilt x High Control *.098*.058†-.077† R-squared Δ R-squared *.018*.007*.004 * p <.05 † p <.10

Moderated Regressions: Conscientiousness Interaction

Conclusion Retesting flagged applicants will result in a set of personality scale scores that are less positively inflated The appraisal profile of guilt helps explain this effect –Flagged applicants who feel guilty as a result of being retested decrease their scores in response.