Questionnaire Design EPIET Introductory Course Lazareto, Menorca 6 October 2011 K Alpers C Campese, P McKeown, V Bremer, V Prikazsky

What is a questionnaire? A tool for data collection A series of written questions in a fixed, rational order There are other forms for data collection: Observation Unstructured interviews

A well designed questionnaire Good appearance (easy for the eye) Short and simple Relevant and logical ⇒ High response ⇒ Easier to collect to summarize to analyse Minimises potential sources of bias

Bias Systematic difference in the response measurement Recall bias Cases more likely to remember than controls Observer bias Different interviewers – different interpretations Different interpretation of similar questions Non-response bias telephone interviews: more females, elderly

How to reduce bias Structured questionnaire Ensure high response rate Random choice of interview partners Training of interviewers Random choice: eg. Last birthday

Advantages of questionnaires Can reach a large number of people Relatively easy and economic Relate directly to study question Provide quantifiable answers Relatively easy to analyse

Disadvantages of questionnaires Provide only limited insight into a problem the range of possible responses is limited the question maybe misleading Varying response Unclear question can lead to misunderstanding misinterpretation Do not allow for mistakes must be right from the beginning missing data hard to chase

Types of Questionnaires Interviewer-administered face to face

Types of Questionnaires Interviewer-administered face to face telephone

Types of Questionnaires Interviewer-administered face to face telephone Self-administered by post

Types of Questionnaires Interviewer-administered face to face telephone Self-administered by post Email Internet Social Networks Social Networks

Self-administered questionnaire Advantages cheap and easy to administer preserves confidentiality completed at respondent's convenience not influenced by interviewer

Self-administered questionnaire Advantages cheap and easy to administer preserves confidentiality completed at respondent's convenience not influenced by interviewer Disadvantages low response questions can be misunderstood no control by interviewer only literate persons time delay (post) e-mail/internet: need computer access/software

Interviewer-administered questionnaire Advantages participation of illiterate people clarification of ambiguities quick answers

Interviewer-administered questionnaire Advantages participation of illiterate people clarification of ambiguities quick answers Disadvantages interviewer bias needs more staff resources only short questionnaires possible especially on telephone difficult for sensitive issues

Before starting to design a questionnaire Write a study protocol! define objectives define study questions Risk factors for being stung by jellyfish at Lazareto (Menorca), October 2011 Study protocol Prepared by cohort 17 Menorca, 6 October 2011

Study protocol and questionnaire Definition of objectives Design questions questions must relate to objectives collect appropriate information for analyses Control and reading List of variables to collect Pilot test Analysis plan

Use existing questionnaires As an inspiration Don´t need to re-invent the wheel Have been tested BUT: adapt to present situation!

Questionnaires for outbreak investigations Exploratory Generate hypotheses Detailed Only some cases Based on existing questionnaires Open questions Analytical study Testing hypothesis Focus on possible vehicles Questionnaire needs to be adapted to the type of study

Questionnaire needs to be adapted to study population Know the respondents language education occupation ethnic group sensitive issues

Structure of a questionnaire Identification Interview introduction Instructions on how to answer Questions Conclusion

1 - Identification On first page On all pages Return address Study title On all pages Identifyer Page numbers Data protection: Identifiers kept separately from names

2 - Introduction Covering letter/ interview introduction Who are you / you work for Why are you investigating Where did you obtain the respondent’s name How and where can you be contacted Guarantee of confidentiality Length of interview (be honest) ⇒ Usefulness of study should be clear to all respondents

Example of introduction Good morning , My name is ..... , I work for …….. You may have been already informed that a survey on risk factors for being stung by a jellyfish will be done this week in Mahon. This study has been approved by the national ethical committee. Only anonymized data will be analysed. You have been randomly selected to participate in this study. Your participation is voluntary. The interview will take about 10 minutes. I will start with the first question .....

3 - Instructions Minimise potential sources of bias Guide for Interviewers Guide for Respondents in self-administered questionnaires Which questions can be skipped => where to jump to Selection of multiple answers possible Use different fonts (e.g. bold or italics) Example: I will ask you a few questions about your contacts with jellyfish. You will answer by yes or no If no, go to question 27

4 - Questions Content Order Format Coding Pitfalls

4.1 - Content of questions Clear focus on research question avoid sidetracking avoid unnecessary information

4.2 - Question order Decide on the order of items/questions easy  difficult general  particular factual  abstract Group questions by topic Be aware of ordering effects Don’t put the most important items last Where to place sensitive questions? Grouping: Eg in an outbreak investigation: Demographic information Symptoms Lab diagnosis Predisposing factors exposures

Starting questions “Door-opener” Simple Closed format Relevant to main subject Non-offending Neither demographic nor personal questions

4.3 - Format of questions Adjust to responding audience professionals vs. public middle class vs. prisoners Keep sentences simple and short Define key words (“fully vaccinated”) Remember options “don’t know” “don’t want to answer”

Ask one thing at a time 7 - Do you like to go swimming and do you mind to be stung by jellyfish?  Yes  No Ask for one information at a time Answer options Mutually exclusive and exhaustive Vertical order of answer options Otherwise you won´t know which part people are answering

Be accurate 8 – What is the jellyfish situation?  Good  Bad Versus 8 - How often did you see jellyfish during the last week?  Once  Twice  Three times or more  Never Don´t know

Be objective 10 - Do you agree that jellyfish are annoying?  Yes  No versus 10 - According to you, jellyfish in Lazareto are…  not at all a public health problem  not a major public health problem  a potential public health problem  an important public health problem  no opinion/don´t know

Be simple 11- Did you see more than an average of 33 jellyfish/m2 salt water surface on more than 3 occasions that you went swimming in the morning last week?  Yes  No versus 11- Have you seen jellyfish on more than 3 mornings last week?  Don´t know

Two main question formats Closed format  forced choice Yes  Always  No  Sometimes  Don’t know  Never  Open format  free text 12 – What did you do to avoid being stung by jellyfish? Please describe : __________________________________________________________________________________

Advantages of open questions not directive detailed and unexpected answers possible > allow exploration of issues to generate hypotheses useful for exploring knowledge and attitudes qualitative research focus groups trawling questionnaires Vorteile: Nicht direktiv: Befragte werden nicht in eine Richtung gedrängt Zur Hypothesen-Generierung: wenn mehrere Menschen ähnliche (unerwartete) Antworten haben, allgemeine Tendenz? Abfrage von Wissen oder Einstellungen Detaillierte, auch unerwartete Antworten möglich Erforscht, wie Menschen über ein Problem sprechen: sprachlich Nachteile: Antwort Interviewer-abhängig: Interviewer schreibt nicht alles genau so hin sondern interpretiert auch hinein Zeit- und arbeitsintensiv: Lange Antworten, Redefluss, aber auch Auswertung Kodierprobleme Schwierig zu analysieren: welche Einteilungen wählen? Vergleiche zwischen Gruppen problematisch

Disadvantages of open questions interviewer bias time-consuming coding problems difficult to analyse difficult to compare groups

Advantages of closed questions Straightforward response Simple Quick Less discrimination against less verbally expressive people Easy to code, record, analyse Easy to compare Easy to report

Disadvantages of closed questions restricted number of possible answers Possible loss of additional information Compromise insert field “others, if yes specify : __________”

Examples for closed questions 2 - What is your age in years? ___ years 3 - How long have you stayed at Lazareto? ___ days 4 - What is your sex (gender)?  Male  Female 5 - Did you go swimming on 4 October 2010?  Yes  No  Don’t know

Closed questions: Checklist 14 - Which of the following beaches have you visited during your stay in Menorca?  Lazareto beach Cala’n Porter Rafalet Macarella Sa Mesquida

Checklist 14 - Which of the following beaches have you visited during your stay in Menorca? Lazareto beach  Yes  No  Don´t know Cala’n Porter  Yes  No  Don´t know Rafalet  Yes  No  Don´t know Macarella  Yes  No  Don´t know Sa Mesquida  Yes  No  Don´t know

Rating scale 15 – How often did you see jellyfish during the past weeks? Always Sometimes Seldom Never Mornings     Lunchtime     Evenings    

Rating scale Numerical 16 - How severe was your pain after you were stung? (please circle) 1 2 3 4 5 6 7 Not painful at all Very painful Analogue 17- How severe is your pain (put the tick on the line) 0 10

Likert Scale Rensis Likert, 1903-1981 Psychometric scale Five (or more) ordered response levels 18 – Jellyfish also have the right to swim in the Mediterranean sea  I strongly disagree  I disagree  I neither agree or disagree  I agree  I strongly agree

4.4 - Coding Answers can be pre-coded Quicker and easier data entry Examples: Male  1 Ill  1 Female  0 Not ill  0 Don’t know  9 Don’t know  9 Lazareto beach  1 Harbour  2 Es Castel  3 Mahón  4

4.5 - Avoid pitfalls jargon/abbreviations/slang 21- Should jellyfish victims receive a PEP? (post exposure prophylaxis) not mutually exclusive options 22 - What is your age ?  16-20  20-25  25-30  35-40

Avoid…. Leading questions 23 - Do you think we should forbid EPIET-fellows to go swimming in the mornings? Making the questionnaire too long Typographical / spelling errors

5 - Conclusion Don’t forget to thank the interviewed persons Tell them when the results will be available and where

Example of conclusion This is the end of the interview. Thanks for answering this questionnaire. The result of this study will be available on the ECDC website in six weeks. Do you have any others questions you wish to ask ? In some outbreak investigations you might want to add: We are still in a very early phase of our investigation. Would you agree to be contacted again in case further questions arise?

Presentation and layout Clear and consistent adequate space to answer large font size appropriate page breaks avoid messy layouts too many and fancy logos printing questionnaire on coloured paper may help

Pilot your questionnaire! Pilot with a group of people -> similar to your target subjects Highlight problems before starting Misunderstandings Look for alternative wording Final polishing

Summary A well designed questionnaire: collects appropriate data -> to answer your research question minimises potential sources of bias -> increases the validity of the replies Will much more likely be completed It is worth to spend time in the design of your questionnaire!