Amber S. Emanuel 1, James A. Shepperd 2, Virginia J. Dodd 1 & Henrietta Logan 1 1 Department of Behavioral Sciences & Community Dentistry, University of Florida 2 Department of Psychology, University of Florida Introduction Method Results Literature Cited Conclusion Using Psychological Instruments in Community Samples Many psychological instruments are designed by researchers at universities who carefully develop, test, and subject items to rigorous psychometric analyses to ensure that the end measures are internally consistent, reliable across time, and demonstrate strong convergent, discriminant, and predictive validity. Most of these instruments are validated and normed on college students or other educated samples, raising questions about whether they are generalizable to other groups. As part of ongoing research on oral cancer screenings among rural residents of North central Florida, we examined the reliabilities of the following commonly used psychological instruments: Behavioral Activation/Inhibition Scale (BIS/BAS; Carver & White, 1994); Regulatory Focus Questionnaire (RFQ; Higgins et al., 2001); and the Need for Cognition Scale (NCS; Cacioppo, Petty, & Kao, 1984). These instruments were examined because they are widely used in health research. Our sample was generally low in education and literacy skills. We predicted that these scales would show low reliability coefficients in our sample, particularly among the least educated participants. Participants were 332 rural North Florida residents who were taking part in ongoing oral cancer screening research. Participants completed all measures individually via paper and pencil in no specific order. Behavioral Inhibition System/Behavioral Approach System Scale. The BIS/BAS has 4 four subscales: 7 items assess BIS, 5 items asses BAS- Reward, 4 items assess BAS-Drive, and 4 items assess BAS-Fun (Carver & White, 1994). Items were assessed on a 4-point Likert scale from 1 (very true of me) to 4 (very false of me). The scale authors report that the BIS/BAS is reasonably reliable (Behavioral Inhibition Scale α =.74; Behavioral Activation Scales αs =.66 to.73; Carver & White, 1994). Regulatory Focus Questionnaire. The RFQ has 11 questions and uses a 5-point Likert scale ranging from 1 (never or seldom true) to 5 (very often true). Six items comprise the promotion subscale and five items comprise the prevention subscale. The scale authors report that the RFQ is highly reliable (Prevention Scale, α =.80; Promotion Scale, α =.73; Higgins et al., 2001). Need for Cognition Scale. The NCS has 18-item and participants respond items such as, “I prefer my life to be filled with puzzles that I must solve” and “I prefer complex to simple problems” using a 9-point Likert scale ranging from 4(very strong agreement) to -4(strong agreement). Higher scores on the scale represent a stronger motivation to enjoy cognitive activities. The scale authors report that the NCS is highly reliable (α =.90; Cacioppo, Petty, Feinstein, & Jarvis, 1996). Due to errors in data collection, only a subsample of participants (n=65) fully completed the NCS. Researchers must be sensitive to the psychometric limitations of instruments and take seriously the admonition that an instrument validated in one demographic group may not be valid in another. Our findings are a reminder that generalizations of both research findings and instruments should be applied cautiously in heterogeneous cultures like the U.S. Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306–307. Cacioppo, J. T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119(2), 197–253. Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales. Journal of Personality and Social Psychology, 67(2), 319–333. Higgins, E. T., Friedman, R. S., Harlow, R. E., Idson, L. C., Ayduk, O. N., & Taylor, A. (2001). Achievement orientations from subjective histories of success: Promotion pride versus prevention pride. European Journal of Social Psychology, 31(1), 3–23. Reliabilities and Reading Level for the Scales Cronbach’s AlphaReading Level Instrument (Number of items) Entire Sample Low Education High Educatio n Flesch- Kincaid Grade Level Flesch Reading Ease NFC (18) * RFQ – Promotion (6) * RFQ – Prevention (5) * BIS (7) * BAS – Fun (4) BAS – Reward (5) BAS – Drive (4) Note: * indicates that Cronbach’s α for the two education levels differ at p <.05. The majority of participants were male (70%), single (58%), and African American (95%). In addition, 53% of participants were age 40 or above, 68% were classified as having “low education”( high school diploma or GED). Consistent with our hypothesis, several instruments demonstrating reliability in well educated, highly literate norming samples, demonstrated less than desirable reliability in our less educated, less literate sample. Overall scale reliabilities were low among these less educated participants. The scales demonstrating the lowest reliability and mean inter-item correlation values were also the most difficult to read, namely the NCS and RFQ-Promotion scale.