1. Development of a System for Measuring Personality Traits Using Sentence Completions
Stephen P. Joy, Ph.D. Albi Beshi, B.A. Heather Mills, B.A.
Albertus Magnus College
Abstract
Results
We introduce a system for measuring major personality traits using sentence completions. The Rotter Incomplete Sentences Blank (RISB) and Eysenck Personality Questionnaire (EPQ-R) were administered to three samples (N = 173). The rating system was developed using members of the first sample who were high or low on each (self-reported) trait and cross-validated on the second sample, then modified and applied to the third sample. Inter-rater reliability was satisfactory for all three scales. Correlations with self-reported traits were generally robust, though we continue to struggle to differentiate Extraversion from Neuroticism.
The following tables present the principal results. In general, each RISB personality scale correlated very strongly with its EPQ-R analogue and much less strongly with the other self-report scales. There was, however, a disturbingly strong correlation between the RISB Extraversion and Neuroticism scales. The RISB Neuroticism scale appeared to be about as strong a correlate of self-reported low Extraversion as it was of high Neuroticism. When we modified these two rating systems to render them more distinctive (sample #3), the RISB Neuroticism scale improved markedly, but the RISB Extraversion scale diminished sadly in validity.
Correlations between RISB and Self-Reported P-E-N Traits, Samples #1 and #2
RISB Trait Rating
EPQ-R Trait P1 P2 E1 E2 N1 N2
Psychoticism .72*** .51*** *
Extraversion *** .51***.42** -.48**
Neuroticism ** *** .45**
Lie -.33*
Correlations among RISB-Rated Traits, Samples #1 and #2
P E N
Psychoticism *
Extraversion -.56** ***
Neuroticism .58** -.70*** -
Note: Sample 1 is above the diagonal, sample 2, below.
Correlations between RISB-Rated and Self-Reported P-E-N Traits, Sample #3
RISB Trait
EPQ-R Trait P E N
Psychoticism .53*** -.25* .40
Extraversion
Neuroticism ***
Lie -
* p<.05; ** p<.01; *** p<.001
Correlations among RISB-Rated Traits, Sample #3
Psychoticism ** .55***
Extraversion **
Neuroticism
Introduction
Sentence completion tests are widely used by clinicians. Among published tests of this kind, the Rotter Incomplete Sentences Blank (RISB) is most popular, because it enjoys a reliable, well-validated and clinically relevant scoring system. However, it yields only a global measure of Adjustment. Therefore, when the RISB is used for anything other than initial screening, interpretation is purely qualitative. This limits its value, though clients may reveal important truths about their concerns.
It seems likely that 40 sentences written to “express your true feelings” will include measurable information that goes beyond the single dimension of Adjustment. For example, clinicians may be concerned with clients’ personality traits. It would be convenient to use the RISB to derive nomothetic trait scores as well as idiographic revelations. We set out to develop scoring systems for this purpose.
Eysenck’s three-trait model (Psychoticism, Extraversion, and Neuroticism: P-E-N) was used. Like other trait models, it was based on the factor analysis of self-report measures. The Eysenck Personality Questionnaire (EPQ-R) was used in the present study.
Method
Three samples of undergraduates were tested. The first sample (N = 45) included 35 females; the second (N = 44), 31 females; the third (N = 84), 61 females. All participants completed the RISB and the Eysenck Personality Questionnaire (EPQ-R; Eysenck & Eysenck, 1985). The EPQ-R is a 100-item self-report inventory that measures the “Really Big Three” traits of Psychoticism, Extraversion, and Neuroticism.
The EPQ-R was scored for the first sample. The first author studied the RISB protocols for the 14 participants with the highest self-reported scores and for the 14 with the lowest self-reported scores on each trait. Response types that occurred more frequently in one group or the other were noted, then examined to identify common themes. These were used to derive the original RISB personality scales.
A trained student rater scored all 45 RISB protocols using the system developed for each trait; these scores were then correlated with EPQ-R scores. These students then scored the RISB protocols from the second sample. This was done to cross-validate the original results. Additional students have been trained as the systems were developed; their results were used to determine inter-rater reliability.
The second and third authors of the present poster were the first individuals trained to employ all three systems. In addition to rescoring the protocols from the first two samples, they scored a new set (sample #3). The rating system was modified before this new cross-validation in response to issues that arose in the earlier studies. In particular, a few scoring categories appeared to be invalid in the first cross-validation sample. More problematic, we found it difficult to identify indicators of high Neuroticism that were not also indicators of low Extraversion.
Reliability Analysis: Intraclass Correlation Coefficients
Psychoticism Extraversion Neuroticism
sing pool sing pool sing pool
Sample #
Sample #
Sample #
Note: The number of raters varies by sample and scale. Only two raters (AB and HM) scored sample #3.
STICIM
Min
Max M SD
Interdependence
2.29
6.00
4.57
0.93
Sexual Intimacy
3.40
4.93
0.95
Comfort Intimacy
1.43
5.86
4.16
1.23
Trust
3.25
4.53
0.92
Warmth Motivation
2.50
1.00
Sexual Desire
3.20
5.21
0.91
Trust Motivation
3.75
5.18
0.64
IC Overall
3.08
5.84
4.55
0.77
IM Overall
3.70
4.99
0.70
Graphic Indicators M SD
Colors
4.70
2.34
Warm Colors
2.45
1.59
DYR Overall
19.33
8.49
Discussion
These findings clearly indicate that major personality traits are expressed through one’s verbal productions, including sentence completions, and can be judged reliably using a fairly simple rating system. More work needs to be done to develop the scoring manual to the point where independent judges could be counted on to obtain similar scores, but there is good reason to hope that this can be achieved. The main task at present is to find ways of measuring Extraversion and Neuroticism that do not overlap unduly. (It should be noted that self-report measures suffered from the same problem for many years, and that even in our samples using the EPQ-R the two scales correlated as strongly as -.38.) As the system evolves, it will open up a whole new range of usefulness for the RISB in personality assessment.
References
Thanks to Natassia Parlato, Michelle Battista, and Michael Lennon for their valuable work as raters.
Contact
For further information contact: