1. What is an Opinion Survey or Poll?
GV917

2. Opinion Surveys and Polls
An Opinion Survey is a conversation between people made possible by a relationship of trust between them
This relationship differs in different survey modes, but it is always a ‘conversation’ and so subject to the complexities of human interaction
There are different types of opinion surveys and they all have advantages and disadvantages

3. Factors influencing responses to surveys
Relationships between interviewer and interviewee (interviewer effects)
The type of survey being done (mode effects)
The type of questions asked: factual, attitudinal – question wording and balance are important
The subject matter of the survey – some things are easy to ask about, others much less so
The order of the questions in a survey
The type of responses sought – closed or open questions

4. An illustration of Complexities of Survey Questions: A Simple Factual Question from the European Social Survey 2002
F30 CARD 56 Using this card, if you add up the income from all sources, which
letter describes your household's total net income? If you don't know the
exact figure, please give an estimate. Use the part of the card that you
know best: weekly, monthly or annual income.
J 01
R 02
C 03
M 04
F 05
S 06
K 07
P 08
D 09
H 10
U 11
N 12
(Refused) 77
(Don’t know) 88

5. A Simple Factual Question from the European Social Survey 2002 - (look at Non-Response)

6. A Simple Attitudinal Question from the European Social Survey How interested are you in Politics? are you (READ OUT)

7. The Interview ‘Conversation’
Public opinion is dynamic and varied
Some people are well informed and give comprehensive answers to questions; Others are not very well informed and give very sketchy answers
Some people really don’t have an opinion about some issues and so will say that they don’t know or alternatively answer randomly
We have to know what polls can do and what they can’t do

8. What can Opinion Surveys Do?
Opinion surveys can measure facts about individuals quite well– their social characteristics, backgrounds and experiences
They can measure people’s attitudes quite well in relation to straightforward issues concerning politics and society and their own behaviour
They can give us a representative picture of what the nation is thinking about key political issues like the future of European politics
If they are panels they can give us trend data about the way people think or behave over time

9. What Can’t Opinion Surveys Do?
Opinion surveys are not good at getting in-depth narratives from people such as their life histories or detailed accounts of their time use – these should use qualitative in-depth studies
They are not good at probing really complicated issues in which people need to think about things a lot before answering
They have a limited ability to predict what people will do or say in the future – some things are predictable but other things not

10. Question Wording Effects – Rules of Thumb for drafting questions
Keep questions simple not complicated
“Are you interested in politics?”
NOT
“Are you interested in Parliamentary politics as opposed to wider issues like Global Warming or the Debt Crisis?”
Avoid ambiguity
“Have you travelled by air in the last year?”
NOT:
‘Do you ever travel by air and by car?”

11. Question Wording Effects – Rules of Thumb for drafting questions
Be specific not vague
‘Do you think the Conservative Government is doing a good job or a bad job?’
NOT
‘What do you think of the political situation right now?’
Avoid leading questions
‘How likely is that you will vote at the next election?’
‘What party are you going to vote for at the next election?’

12. A Question Wording Experiment- the Case of Party Identification
The traditional wording of the party identification question in election studies was inherited from the American National Election Study of the early 1950s, and used by Butler and Stokes in the first British Election Study in It appears in all subsequent surveys
‘Generally speaking, do you think of yourself as Labour, Conservative, Liberal Democrat (Scottish Nationalist/Plaid Cymru) or what?
If respondents say ‘None’ it is followed by a supplementary question:
‘Do you generally think of yourself as a little closer to one of the parties than the others?’
This is then followed up with:
Would you call yourself very strong {Party}, fairly strong, or not very strong?

13. Criticisms of the Traditional Partisanship Question
It is too leading – it assumes people are attached to a political party, when many may not be. This may have been OK in 1963 but not any more when fewer people are attached to political parties.
For analysis purposes responses to the supplementary question are often added to the responses to the first question – making the bias in analysis even bigger.

14. Revised Party Identification Sequence
‘Some people think of themselves as usually being a supporter of one political party rather than another. Do you usually think of yourself as being a supporter of one particular party or not?’
IF YES
‘Which party is that?’
Note this is much less leading – giving respondents ‘permission’ to say that they don’t support a party

15. What Difference Does it Make?
Response Category
Traditional wording
Revised Wording
None
17.1
47.8
Labour
35.9
26.9
Conservatives
26.1
18.3
Liberal Democrats
14.6
5.1
Scottish Nationalists
1.4
0.5
Plaid Cymru
0.4
0.3
Greens
0.6
UKIP
1.0
BNP
0.2
0.1
Others
1.9

16. Question Wording Revisited
Question wording is really important. If one wants to bias answers, then it is easily done by phrasing the question in a given way.
However, a biased question can still be useful. The traditional question has been used in every election study since 1963 and it will be included in the next study.
Why? Because it provides a time series.

17. The Decline of Partisanship in Britain, 1964 to 2005

18. Mode Effects
These are associated with different ways of administering a survey
Face-to-face or ‘in-person’ surveys
Telephone surveys
Internet surveys
Postal Surveys
There are advantages and disadvantages of different modes

19. Face-to-Face Surveys - Advantages
We can get national probability samples. This is very important for inferring the characteristics of populations from sample
The interviewer can persuade individuals to participate and thus increase the response rate
The interviewer can respond to individuals who don’t understand a question and clarify or explain things
The interviewer can report on how the interview went – eg.was the respondent attentive or not?

20. Face-to-Face surveys - Disadvantages
Face-to-Face surveys are very Expensive (e.g., $5500 per voter in the US National Election Study in 2004)
Because they are expensive the N’s are often small – 792 voters in 2004 ANES
Slooooow Moving – they take an average of 6 weeks to complete
Limited Ability for Panels – that is interviewing the same people on more than one occasion
One Study Only! Data Mining, Pre-Test Biases, replication is very difficult
In the case of election studies there is limited or No Inter-Election surveys

21. MODE EFFECTS 1 Reported Party Choice in Face-to-Face and Internet Post-Election Surveys and Actual Vote in Britain, 2005 General Election

22. Mode Effect II - Turnout

23. Advantages of Internet Surveys
They are cheap – in the election study the face-to-face survey costs up to ten time more than an internet survey
They are fast – one can get 5,000 respondents within 48 hours on the internet while this would take 6 weeks face-to-face
It is possible to do experiments (eg. Feedback to voters experiment)
BUT – at the moment they are not probability samples

24. Conclusions
Opinion Surveys are the core methodology for empirical social science and are essential for studying individuals
A lot of aggregate analysis – such as modelling the relationship between the economy and behaviour is based on national data such as GDP per capita and spending by consumers which is collected by surveys
But users of surveys have to know their limitations