1. ‘Hints for Designing Effective Questionnaires ’
Author: Robert B. Frary
Presntor: DeCarlo Bridges

2. Objectives To understand: Why we use questionnaires
Objectives of a questionnaire
Advantages and disadvantages of questionnaires
Design of questionnaires
Different types of questions used
How to avoid common problems and pitfalls

3. What is a questionnaire?
An instrument to
collect answers to questions
collect factual data – gathers or measures information
A series of written questions / items in a fixed, rational order

4. Effective Questionnaire
A well designed questionnaire:
Gives appropriate data which answers your research question
Minimises potential sources of bias
Will more likely be completed

5. Objectives of a questionnaire
To maximise response rate
To ensure that obtained information is accurate and relevant to our study
= as simple and as focused as possible

6. Advantages of questionnaires
Can reach a large number of people relatively easily and economically (especially postal/ questionnaires)
Provide quantifiable answers
Relatively easy to analyse

7. Disadvantages of questionnaires
Provides only limited insight into problem
Limited response allowed by questions
Maybe not the right questions are asked
Varying response
Misunderstanding/misinterpretation
Need to get it right first time
Hard to chase after missing data

8. Types of Questionnaire
Self-administrated
By post
/Internet
Interviewer-administrated
Face to face
Telephone

9. Self-administered questionnaire
Advantages:
Cheap and easy to administer
Preserves confidentiality
Completed at respondent's convenience
Administered in a standard manner
No influence by interviewer

10. Self-administered questionnaire
Disadvantages:
Low response rate
Questions can be misunderstood
No control by interviewer
Time loss

11. Interview-administered questionnaire
Advantages:
Participation by illiterate people
Clarification of ambiguity
Quick answers

12. Interview-administered questionnaire
Disadvantages:
Interviewer bias
Needs more resources
Only short questionnaires possible
Difficult for sensible issues

13. What makes a well designed questionnaire?
Highly structured
Collects the same types of information
Standardized
Allows quantitative and systematic analysis

14. Stages in designing a questionnaire (1)
Planning the study:
Decide on goals of study
Know the subject
literature, experts
Formulate a hypothesis
Define information needed to test the hypothesis
Was sind die Grundsätze des Fragebogendesigns?
Man fängt nicht erst an, die Fragen zu formulieren (auch wenn das das Natürlichste erscheint), sondern formuliert erst eine Hypothese. Beispiel: Rauchen erhöht das Risiko, an Lungenkrebs zu erkranken.
Anschließend muss man definieren, welche Informationen man benötigt, um die Hypothese zu prüfen.
Trotzdem, und das ist eine wahre Herausforderung, muss man sich kurz fassen. Einen 80-seitigen Fragebogen beantwortet niemand.
Man sollte natürlich nur nach Informationen fragen, die notwendig sind und die wir noch nicht haben.
Und zu guter Letzt sollten wir einfachen Fragen stellen und einfache Antwortenmöglichkeit vorgeben, damit alle das verstehen, was man wollte.

15. Stages in designing a questionnaire (2)
Determine study population:
Know the respondents
Occupation
Special sensitivities
Education
Ethnic
Language

16. Stages in designing a questionnaire (3)
Design questions:
Content of the questions
Format of the questions
Presentation and layout
Coding schedule (if appropriate)
Pilot and refine questionnaire

17. Stages in designing a questionnaire (4)
Complete study protocol:
Determine
Cost
Time
Sample size
Response rate

18. Basic Rules (1) KISS  keep it short and simple Appearance affects
Length of questionnaire: shorter  response rates
Appearance affects
Response rate
Data summarization and analysis
 easy on the eye

19. Basic Rules (2) Number all items and pages
Put an identifying mark on all pages
Put return address on questionnaire
Put study title in bold on first page
Print directions in bold
Self-addressed envelope!!
Remember the Unique Identifier

20. Basic Rules (3) Question order Easy  difficult General  particular
Factual  abstract
Starting questions
Simple
With closed format
Relevant to main subject
Non-threatening
Neither demographic nor personal questions
Be aware of ordering effects!

21. Basic rules (4) Group questions by Don’t put most important item last
Topic/ response options
Don’t put most important item last
Questionnaire likely to be completed if
relevant
logical

22. Introduction of questionnaire
Covering letter/introduction for interview
Who you are
Who you work for
Why you are investigating/researching
Where you obtained the respondent’s name
How and where you can be contacted
Absolute guarantee of confidentiality
Length of interview (be honest)

23. Ensuring a High Response Rate
Ensure respondents see its potential and usefulness

24. Content of Questions Clear focus on research question
Avoid sidetracking
Avoid unnecessary information
Demographic information
Contact information (if non-anonymous)

25. Format of Questions Adjust to responding audience
Professionals vs. public
Middle class vs. prisoners
Keep sentences simple and short
Define key words

26. Format of questions Two main question formats
Closed format  give a ‘fixed’ response
Yes, No, Don’t know
Always, sometimes, never
Open format  allow people to express their views in their own words:
What is your most distressing symptom? Please describe: ____________________________________________________________________________________

27. Open or Closed? Closed – forced choice Advantages: Disadvantages:
Simple and quick
Reduces discrimination against less literate
Easy to code, record, analyze
Easy to compare
Easy to report results
Disadvantages:
Restricted number of possible answers
Loss of information
Possible compromise:
Insert field

28. Open or Closed? Open format – free text Advantages: Disadvantages:
Not directive
Allows exploration of issues - generate hypothesis
Used even if no comprehensive range of alternative choices
Good for asking about knowledge and attitudes
Detailed and unexpected answers possible
Disadvantages:
Answer depends on interviewer
Time-consuming
Coding problems
Difficult to analyze!
Difficult to compare groups
Vorteile:
Nicht direktiv: Befragte werden nicht in eine Richtung gedrängt
Zur Hypothesen-Generierung: wenn mehrere Menschen ähnliche (unerwartete) Antworten haben, allgemeine Tendenz?
Abfrage von Wissen oder Einstellungen
Detaillierte, auch unerwartete Antworten möglich
Erforscht, wie Menschen über ein Problem sprechen: sprachlich
Nachteile:
Antwort Interviewer-abhängig: Interviewer schreibt nicht alles genau so hin sondern interpretiert auch hinein
Zeit- und arbeitsintensiv: Lange Antworten, Redefluss, aber auch Auswertung
Kodierprobleme
Schwierig zu analysieren: welche Einteilungen wählen?
Vergleiche zwischen Gruppen problematisch

29. Problems and Pitfalls Avoid leading questions
Avoid making questionnaire too long
Typographical / spelling errors

30. Presentation and layout
Using color or printing questionnaire on colored paper may help
Use filter questions, if necessary
Give clear instructions about how to answer the questions

31. Summary A well designed questionnaire:
Will give appropriate data which allow to answer your research question
Will minimise potential sources of bias, thus increasing the validity of the questionnaire
Will much more likely be completed

32. FINALLY, keep your questionnaire short and the questions simple, focused and appropriate
Question 764 …“ CLASS …. . ..HAVE
……ANY ……………Questions?…….. …
…….. …… ……… …
………. ……….. ……..
………… ….. …. …….