1. QUESTIONNAIRE DESIGN: GENERAL PRINCIPLES
Lu Ann Aday, Ph.D.
The University of Texas
School of Public Health

2. MEASUREMENT: Definition of Variables
TYPE
DESCRIPTION
CONCEPTUAL
DEFINITION
Major concept or issue of interest, e.g., obesity
OPERATIONAL
Specific questions asked on the survey to obtain information on the concept
VARIABLE
Variable(s) constructed from the questions to actually be used in the analyses, e.g., Body Mass Index

3. MEASUREMENT: Levels of Measurement
PROPERTY
Nominal
Ordinal
Interval
Ratio
Classify X
Rank
Distance between points
Distance from zero

4. MEASUREMENT: Example Concept: Obesity
LEVEL/
DEFINITION
Nominal
Ordinal
Interval
OPERATIONAL
Do you consider yourself overweight, underweight, or just about right?
Would you say you are very overweight, somewhat overweight, or only a little overweight?
About how tall are you without shoes?
About how much do you weigh without shoes?
VARIABLE
1=overweight
2=underweight
3=about right
1=very overweight
2=somewhat overweight
3=only a little overweight
Body Mass Index (BMI)=weight divided by height, squared

5. MEASUREMENT: Example Concept: Family Income
LEVEL/
DEFINITION
Nominal
Ordinal
Ratio
OPERATIONAL
Did anyone in your family have income from wages or salaries in the past 12 months?
Which of the following best describes your family’s total income from wages & salaries in the past 12 months?
What was your family’s total income from wages & salaries during the past 12 months?
VARIABLE
1=Yes
2=No
1=<$35,000
2=$35,000-$49,999
3=$50,000+
$ __,____/year

6. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000)
1. Choose simple over specialized words.
2. Choose as few words as possible to pose the question.
3. Use complete sentences to ask questions.
4. Avoid vague quantifiers when more precise estimates can be obtained.

7. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000)
5. Avoid specificity that exceeds the respondent’s potential for having an accurate, ready-made answer.
6. Use equal numbers of positive and negative categories for scalar questions.
7. Distinguish undecided from neutral by placement at the end of the scale.

8. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000)
8. Avoid bias from unequal comparisons.
9. State both sides of attitude scales in the question stems.
10. Eliminate check-all-that-apply question formats to reduce primacy effects.
11. Develop response categories that are mutually exclusive.

9. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000)
12. Use cognitive design techniques to improve recall.
13. Provide appropriate time referents.
14. Be sure each question is technically accurate.
15. Choose question wordings that allow essential comparisons to be made with previously collected data.

10. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000)
16. Avoid asking respondents to say yes when they mean no.
17. Avoid double-barreled questions.
18. Soften the impact of potentially objectionable questions.
19. Avoid asking respondents to make unnecessary calculations.

11. SCALES: Definition
Scales are measurement instruments that are collections of items [questions/statements] intended to reveal theoretical variables [abstract concepts] not readily observable by direct means, e.g., patient satisfaction, locus of control.

12. SCALES: Underlying Structure
ABSTRACT CONCEPT, e.g., Unsatisfactory Patient Encounters
Specific Items, e.g., series of statements regarding the nature of Patient Encounters
Uniform Response Categories, e.g., scale of 1 to 5, where 1 is very typical and 5 is not at all typical of patient encounters

13. (Circle one response for each item.)
SCALES: Example— Unsatisfactory Patient Encounters (Aday & Cornelius, 2006, NDMS, Q. 3)
Very
Typical 1
(Circle one response for each item.)
Not at all 5
1. The patient was trying to manipulate me. 2 3 4
11. I felt that the patient didn’t believe me.
18. The patient and I seemed to come from different worlds.

14. SCALES: Methodological Issues
ABSTRACT CONCEPT
Internal consistency reliability:
Verify that the specific items in the scale relate to the CONCEPT.
Construct validity:
Verify that the specific items in the scale define and distinguish the CONCEPT from other concepts.

15. SCALES: Methodological Issues
Specific Items
Take reading level of respondents into account in phrasing items.
Avoid jargon and colloquialisms.
Express only one idea in each item.
Use both positively & negatively worded items, but avoid use of negatives (e.g., “not”) in negatively wording an item.

16. SCALES: Methodological Issues
Uniform Response Categories
Decide on a reasonable number of response categories, e.g., 3-10, taking concept and respondents’ ability to discriminate into account.
Select meaningful labels/terms for categories, e.g., strongly agree, agree, disagree, strongly disagree; excellent, good, fair, poor.

17. MEASUREMENT MATRIX: National Dental Malpractice Survey (Aday & Cornelius, 2006, Table 15.1)
QUESTION
CONCEPT
LEVEL
OBJECTIVE 3
Doctor-patient communication: characteristics of unsatisfactory patient encounters (Likert scale)
ordinal
(interval) 10
Practice characteristics: avg. office waiting time for patient 28
Practice finances: % patients with insurance coverage
interval 35
Malpractice insurance: no. of complaints
1, 2, 3 54
Demographics: gender
nominal
2, 3

18. SURVEY ERRORS: General Principles for Formulating Questions
Systematic Errors
Variable Errors
Questionnaire
effects: under- or over-reporting
Respondent effects: yeah-saying
Mode effects: systematic
Questionnaire effects: order & context
Interviewer effects: interviewer variability
Mode effects: variable
Solutions to errors
See Chapters Ten and Twelve.
See Chapters Ten, Eleven, and Twelve.
See Chapters Five and Ten.
See Chapter Thirteen.
See Chapter Five.

19. REFERENCES
DeVellis, Robert F. (2003). Scale Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage.
Dillman, Don A. (2000). Mail and Internet Surveys: The Tailored Design Method. Second Edition. New York: John Wiley & Sons, Inc.