1. Department of Political Science
Survey research techniques: Matching the method with the research question
Robert M. Stein
Department of Political Science
Rice University
March 2013

2. Popular survey techniques
Face-to-face interviews
Mail surveys
Self-administered surveys
Telephone interviews
Live interviewer
Interactive voice recorded
Web based surveys (with/without telephone recruitment)
Focus groups

3. Major research issues with surveys
Sampling
Reactivity
Psychology of the survey response
The ‘don’t know response’
Response time
Social desirability
Question wording and placement
Meaning of survey responses

4. Survey methods for studying who votes
Telephone, face-to-face surveys
Strengths: Sample size, limited to registered voters
Weaknesses : Social desirability, reactivity, recall problems, sample selection bias (unlisted phone number
Exit poll
Strengths: Validated voters, limited number of questions
Weaknesses : Reactivity, unable to study non-voter
IVR poll
Strengths: Large sample, less invasive, lower cost
Weaknesses: Low response rate, limited number of questions, skewed sample
‘Surveyless’ Survey (e.g., annotated voter histories)
Strengths: Unambiguous, valid and reliable information
Weaknesses: Limited and bias time series

5. Overreporting voting
Robert Bernstein, Anita Chadha and Robert Montjoy, “Overreporting Voting: Why it Happens and Why it Matters.” Public Opinion Quarterly 65(2002):22-44
Allyson L. Holbrook and Jon A. Krosnick, “Measuring voter turnout by using the randomized response technique: Evidence calling into question the method’s validity,” Public Opinion Quarterly 74(2010):
Brian D. Silver, Paul R. Abramson and Barbara A. Anderson, “The presence of others and overreporting of voting in American National Elections,” Public Opinion Quarterly 50(1986):
Robert F. Belli, Sean E. Moore and John VanHoewyk, “An experimental comparison of question forms used to reduced vote overreporting,” Electoral Studies 25(2006):

6. Social expectation and over reporting of voting
Treatment #1
In talking to people about elections, we often find that a lot of people were not able to vote because they weren't registered, they were sick, or they just didn't have time. How about you--did you vote in the elections this November?
Treatment #2
In talking to people about elections, we often find that a lot of people were not able to vote because they weren't registered, they were sick, or they just didn't have time. Which of the following statements best describes you: One, I did not vote (in the election this November); Two, I thought about voting this time - but didn't; Three, I usually vote, but didn't this time; or Four, I am sure I voted?

7. Live telephone and IVR surveys of 2009 Houston mayoral election voters
Live telephone interview
500 registered voters interviewed 11/15/09 -11/18/09
Error rate +/- 4.5%
Interview length: minutes (95%)
Response rate: 32%
IVR interview
586 registered voters interviewed 11/15/09 -11/18/09
Error rate +/- 3.3%
Interview length: minutes (95%)
Response rate: 8.5%

10. Response time (latency) measures of contextual effects on voting behavior *
Problem: Studies of contextual processes have always involved the possibility that if individuals’ aggregation into geographic units is not exogenous to their values on the dependent variable, then what appear to be “contextual processes” may be due solely to selection effects.
Solution: measure contextual variables separately from the individual-level variables, and later to connect them to the survey data by means of linkage variables. Latency measures of response time to survey generated questions.
Methodology: Cati (computer assisted telephone interviewing) technology used to generate latency measures to query to respondent’s assessment of the partisanship of their neighbors
“M. Johnson, W. Phillips Shively and R.M. Stein, “Contextual data and the study of elections and voting behavior: connecting individuals to environments.” Electoral Studies 21(2002):

11. Hypotheses
When perception of partisanship of one’s neighbor’s (i.e., (generally speaking, do you usually think of your neighborhood as Republican, Democratic or Independent?) is latent (i.e., faster response time) context sould have a significant and positive effect on vote choice.
Republicans (Democrats) residing in neighborhoods that perceive to be Democratic (Republican) will be more likely to vote for Democrats (Republicans) when their perception of the partisan make-up of their neighborhood is readily accessible.
The accessibility of one’s context will be greater when that context is congruent with a respondent’s personal preference.

12. Research design
Live telephone interview with 750 registered voters in Harris County, Texas, conducted September 23-29, 1999
Retrospective vote choice in the 1996 Presidential election, partisanship and perception of partisanship of neighborhood
Response times collected on Cati system similar to Bassli and Fletcher (1991).

16. An Answer: Risk perceptions and evacuations from hurricanes.
Analyzing surveys
When are answers to survey questions revealing of individual perceptions, preferences and behavior?
How might multiple survey responses reveal more information than single responses?
An Answer: Risk perceptions and evacuations from hurricanes.