1 Social Research Methods Questionnaire Design

2 References Oppenheim, A.N Questionnaire Design and Attitude Measurement. London: Heinemann de Vaus, D. A Surveys in Social Research, 4th Ed. London: UCL Press (Chapters 6 & 15) Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires, Cambridge: Cambridge University Press. Plus Hoinville, G., Jowell, R and associates (1985) Survey Research Practice. London: Gower Moser, C. A. and Kalton, G (1971) Survey Methods in Social Investigation. Aldershot: Gower

3 When is a questionnaire appropriate? Have good idea of what to expect Literate respondents or Time to administer if not literate Interested in counts, frequencies etc. Want to make generalisations to larger population

4 Procedure Working out the aims of the questionnaire Selecting the question styles Designing the questions Structuring the questionnaire Piloting the questionnaire Revising the questionnaire Administering the questionnaire.

5 Information which can be obtained from questionnaires Facts and Knowledge (biography) Actions Opinions/beliefs Attitudes Motives Past Behaviour Likely Future Behaviour

6 Choosing the questions What type of information do you need from this question? How will the answers serve the aims of the questionnaire? How will you present the findings from answers to this question? Do you need numerical scores from the answers?

7 Open questions 1. What do you think the Government should be doing about global warming?

8 Closed questions 1. What do you think is the most important factor to consider when buying food? Tick just ONE box a) Value for money 1 b) Hygienic processing 2 c) No chemical additives 3 d) Good flavour when cooked 4 e) Easy to prepare 5

9 Open or closed questions? Open questionsClosed Questions More representative of respondent’s true opinions Very easy to score and analyse Less open to researcher’s bias Give predictable outcomes Useful to indicate points of interest or weaknesses Useful for statistical illustrations or summaries

10 Questionnaire structure Title Introductory paragraph Content questions in sections use filter questions put more difficult questions towards the end do not give away answers to later questions End and thanks

11

12 Questionnaire layout Allow space Instructions (general, section, question, filter question) Numbers for computer coding One side of page only (space for comments)

13 Telephone Questionnaires Retention problem = too many categories in answer to remember 1. use numbered scale 2. get direction of feeling then intensity 3. include in question then repeat list Provide guides for interviewer on questionnaire – distinguish from questions Make clear what is to be read to respondent and what is not (e.g. instructions for prompt)

14 Checklists and ratings

15 Semantic Differentials How would you rate your mother on these scales? WarmheartedI I I I I Coldhearted PatientI I I I I Impatient Self-assuredI I I I I Always worried UnselfishI I I I I Selfish

16 Why pilot a questionnaire? 1. To expose the questionnaire to new forms of thinking and understanding. 2. To evaluate the quality of the answers provided by the questionnaire. 3. To assess and revise the phrasing and content of the questions. 4. To explore issues of questionnaire structure and question sequence.

17 Use Piloting to check for: Variation in type of answer Meaning Redundancy Scalability Non-response (too much) Acquiescent response (too much agreement with agree/disagree Q) Flow Filtering Timing

18 Scaling Needs to be Unidimensional Linear (equal intervals) Reliable Valid (esp. for standard scales)

19 Major types of scales Thurstone Judges rank agree/disagree items Lickert Judges score large list of items Guttman/Scalogram Items eliminated to form a ranked scale Paired comparisons Pairwise comparing of items

20 Lickert Scales

21

22

23 Some problems of question-writing:- Leading questions Are you against giving too much power to trades unions? Why don’t you go more often to the supermarket?

24 Ambiguous questions Is your work made more difficult because you are expecting a baby? Also words like ‘Books’, ‘Dinner’ Multiple-content questions Have you suffered from headaches or sickness lately?

25 Over-complex vocabulary Avoid big words and technical terms e.g. “Personality”, “Social mobility”, “Anomie”, “Patriarchy”. Simple alternatives acquaintinform or tell assisthelp considerthink initiatebegin; start majorimportant; chief; main; purchasebuy requirewant; need residelive statesay sufficientenough terminateend

26 Over-elaborate phrasing and too complex question Has it happened to you that over a long period of time, when you neither practised abstinence, nor used birth control, you did not conceive? Yes No vague (what is a ‘long period’ too factual (‘has it happened to you that …’) complex- Long sentence and double negative

27 Implicit questions (based on assumptions) How many children do you have? (Asked of all people) How old is your car?

28 How often? Use either: In the last week how often.…? On average …? When did you last.…?

29 Embarrassing questions e.g. sexual mores, alcohol consumption, law breaking, taboo subjects (e.g. death, menstruation) Create permissive atmosphere by range of possible answers. e.g. not “Does your child steal?” but, “Which of the following best describes your child? 1. He or she never steals 2. He or she sometimes steals (including from home) 3. He or she has stolen on several occasions.

30 Vague terms e.g. a long time recently near you kind of fairly often on the whole

31 Prestige bias “Doctors say that …” “Research has shown that …” “Is the government right to.…”

32 Patronising tone Avoid this. “Answer these questions to the best of your ability” “‘Dwelling’ means the place where you regularly live”. Use the Sun/Mirror style