1. 1 Questionnaire Design P McKeown EPIET Introductory Course 2003 VERYIER DU LAC, FRANCE

2. 2 So far……… Introduction Outbreak or Incident Salmonella in Northern Ireland Steps in investigation Surveillance  Questionnaire Design Disease Detectives  Looking for clues  Building a case

3. 3 Objectives To understand: Why we use questionnaires The objectives of a questionnaire The advantages and disadvantages of questionnaires The different types of questions that can be used and the circumstances in which they would be used The way in which questionnaires can be designed and how to avoid some common problems and pitfalls

4. 4 What is a questionnaire? An instrument (form) to collect answers to questions Collects factual data – gathers information or measures A series of written questions / items in a fixed, rational order Every item is answered (unless instructed otherwise) Who completes it? The respondent – public/professional Someone on behalf of the respondent The interviewer

5. 5 Types of Questionnaire Prospective or retrospective Completed by participant or researcher Advantages of self-administered include: Cheap and easy to administer Preserve confidentiality Can be completed at respondent's convenience Can be administered in a standard manner

6. 6 Types of Questionnaire Advantages of interview-administered include: participation by illiterate people clarification of ambiguity They can be administered face to face, via the telephone, by post or email

7. 7 What makes an effective questionnaire? An effective questionnaire is highly structured to allow the same types of information to be simply collected from a large number of people in the same way so that the data can be analysed quantitatively and systematically

8. 8 And why do we want one? A well designed questionnaire: Will give you appropriate data which should allow you to answer your research question Will minimise potential sources of bias, thus increasing the validity of the questionnaire People are much more likely to complete a well designed questionnaire

9. 9 Assumptions The use of questionnaires is based on certain assumptions: That individuals will have had common understanding of particular issues and will share general beliefs (went to a wedding/ that we are trying to find out more about meningitis because this disease is a threat to our children and young people) That this understanding and these beliefs are shared and can be clearly expressed and quantified

10. 10 Before starting….. Questionnaire design should be an integral part of the study, not an “add-on” Therefore: Decide on the goals of the study (for yourself and for others) Know the subject – literature, experts Know the respondents - (SEG, occupation, special sensitivities, educational, ethnic) Know how much – cost, time, sample size, response rate

11. 11 Fundamental Principle An epidemiological investigation is a scientific loop that tests a hypothesis: ProblemHypothesis Study Questionnaire Question Results Answer

12. 12 Objectives of a questionnaire To maximise the proportion of subjects answering the questionnaire –as high a response rate as possible To ensure that the information obtained is as accurate and as relevant to our study as possible = as simple and as focussed as possible

13. 13 Advantages of questionnaires Can reach a large number of people relatively easily and economically (especially postal questionnaires) Provide quantifiable answers Relatively easy to analyse

14. 14 Disadvantages of questionnaires May provide only limited insight into the problem: Limited response allowed by questions You may not have asked the right questions eg Andorra, norovirus and ice-cubes Varying response they may not understand your question and may intrepret it differently Postal questionnaires  low response rate Hard to chase after missing data – need to get it right first time

15. 15 Stages in designing a questionnaire Decide on aim of the study and the purpose of using a questionnaire Decide on content of the questions Decide on the format of the questions Decide on presentation and layout Formulate coding schedule (if appropriate) Pilot and refine questionnaire

16. 16 Basic Rules KISS  keep it short and simple Appearance is crucial and affects: Response rate Ease of data summarisation and analysis Length of questionnaire: shorter   response rates Question order is important: Easy  difficult General  particular Factual  abstract Start with closed format questions. Start with questions relevant to the main subject. Try to avoid starting with demographic and personal questions

17. 17 Basic Rules Number all items and pages Put an identifying mark on all pages Put the return address on the questionnaire Put the study title in bold on the first page Print directions in bold Remember the Unique Identifier

18. 18 Basic Rules Group questions by topic or response options Don’t put the most important item last Open with relevant non-threatening questions It is most likely to be completed if easy on the eye, relevant, logical and as short as possible.

19. 19 Content of Questions Keep a clear focus on your research question Literature search; what do we need to know in order to ask the most relevant and probing questions? Demographic information Contact information (if non-anonymised)

20. 20 Format of Questions Ask only for the information you need Start with simple questions – get them on your side Appropriate for responding audience – eg middle class wedding guests vs prisoners Keep sentences simple and short

21. 21 Format of Questions Ask for one piece of information at a time: If you had not passed water for 6 hours but were watching a James Bond movie, and your wife brought you a sandwich would you get up to go to the toilet in the next 20 minutes? Y/N Remember “don’t know”

22. 22 Format of Questions Be precise – Do you often have to get up at night to pass water? Y/N Be appropriate/sensitive – bowel habits Aim always to minimise bias

23. 23 Bias in Questions Information Bias = systematic differences in the measurement of a response Recall bias – eg cases being more likely to remember than controls Observer bias – inter and intra – a questionnaire is an important way to reduce this Non-response bias Those who respond are different from those who do not Reduce by ensuring high response rate or Correcting for during analysis (eg age, sex, SEG)

24. 24 Format of questions Two main question formats Closed format  give a ‘fixed’ response Open format  allow people to express their views in their own words: What is your most distressing symptom? Please describe: _________________________________________ _________________________________________ ______________

25. 25 Open or Closed? Closed – forced choice Simple and quick Reduces discrimination against those who are less literate Easy to code, record, and analyse results quantitatively Easy to report results Open format – free text Allows exploration of issues Can be used even if a comprehensive range of alternative choices cannot be compiled

26. 26 Closed Questions A number of types: 1.Straightforward response 1. What is your age in years? _____ yrs How long have you worked for Company X? _____ What is your sex (gender)? M F Did you stay in Hotel X on 23/6/02? Y N Did you eat dinner in the company canteen on 23/6/02? Y N

27. 27 2. Checklist Which of the following did you eat for dinner on 23/6/02? Chicken Beef Ham Salad Egg mayonnaise Closed Questions

28. 28 Closed Questions 3. Adjectival/adverbial responses Are you limited by hip pain during any of the following activities? Not at all Slightly Moderately Severely Walking Sitting Lying in bed

29. 29 Closed Questions 4. Numerical rating scale How useful would you think a community methadone clinic in your area would be? (please circle) 1 2 3 4 5 6 7 Not at all useful Very useful

30. 30 Closed Questions 5. Visual analogue scale At rest, how bad is your hip pain? (please mark where on the line below) No pain Worst pain imaginable

31. 31 Closed Questions 6. Ranking scales Please rank these activities in order of the amount of pain they cause you. (1 would be most pain and 4 would be least) Sitting Running Lying in bed Walking

32. 32 Closed Questions 7. Scales for measuring attitude eg Lickert A local methadone clinic would be useful for my community No, I strongly disagree No, I disagree quite a lot No, I disagree just a little I’m not sure about this Yes, I agree just a little Yes, I agree quite a lot Yes, I strongly agree

33. 33 Open Questions Used to provide fuller information Open, free-text form Often used in qualitative research, focus groups, at the preliminary and piloting stage of questionnaires What do you think is the reason you became ill? ________________________________________________ Do you think service X would make your life easier? If so, in what way? If not, why not? _________________________________________________ _______________________________________________

34. 34 Problems and Pitfalls Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering: Have you ever had neck pain and diarrhoea? Ambiguity..... e.g. Do you got to the toilet a lot?

35. 35 Problems and Pitfalls Avoid jargon How often do you get up at night to PU? (pass urine) Should IVDUs be treated in the community? Avoid options that are not mutually exclusive when they need to be What age are you? 16-20 20-25 25-30 35-40

36. 36 Problems and Pitfalls Avoid leading questions Do you think that the food in the hotel made you sick? Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to exhaustion? Avoid making the questionnaire too long Typographical / spelling errors Questionaire Diahurria Vommit Eppidemiology

37. 37 Ensuring a High Response Rate Make the questionnaire relevant Ensure respondents see its potential and usefulness (we are trying to find out why you and your family/people in became ill while holidaying in this resort)

38. 38 Ensuring a High Response Rate Include a brief covering letter Who you are Who you work for Why you are investigating/researching Where you obtained the respondent’s name How and where you can be contacted Absolute guarantee of confidentiality Self-addressed envelope!!

39. 39 Questionnaire Validation Many existing questionnaires can be used or adapted eg SF-36 Validated Others e.g. for an outbreak have not been validated and will need to be tested Piloting is a common way of doing this

40. 40 Piloting and Evaluation Pilot with a similar group of people to your intended subjects Piloting a essential Highlights problems before starting Often concentrates initially on effects of alternative wording Then will concentrate on respondents and interviewers overall impressions With a final polishing after several amendments to ensure response rate

41. 41 Presentation and layout Decide on the order of the items / questions Where to place sensitive questions? Make the layout clear, consistent and easy on the eye Try to get on overall simple “feel” Ensure there is adequate space to answer Choose a large enough font size Avoid flippancy Avoid fancy logos/printed on recycled paper/is an equal opportunity employer etc Do not slip to another page

42. 42 Presentation and layout Using colour or printing questionnaire on coloured paper may help Use filter questions, if necessary Give clear instructions about how to answer the questions

43. 43 Coding Schedule Questionnaire can be pre-coded Quicker and easier to hands responses prior to data entry eh Epi-Info or SPSS Examples: M=1 F=2 DK=3 Ill=1 Not ill=2 DK=3 Single=1 Married=2 Separated=3 Divorced=4 DK=5 No medical intervention=1 Saw GP=2 Went OPD=3 Admitted Hospital=4 Admitted ICU=5 Died=6 DK=7

44. 44 Summary A well designed questionnaire: Will give you appropriate data which should allow you to answer your research question Will minimise potential sources of bias, thus increasing the validity of the questionnaire People are much more likely to complete a well designed questionnaire

45. 45 FINALLY, keep your questionnaire short and the questions simple, focussed and appropriate Question 764 …“If you are an attractive, blonde woman aged 25-50, can I take you to dinner?”