1. Structured Interviews and Instrument Design Part I
Lecture 5
Info 271B

2. “Continuum” of Interview Situations
Unstructured
Structured
Open-ended interviews
Interview Guides, still open-ended
Identical Instrument (questions, etc) given to all participants

3. What is a Survey? To survey: Research Surveys
“act of looking or seeing, observing”
Research Surveys
Qualitative interviews
Focus groups
Specific, systematic, quantitative data-collection instruments

5. Example Surveys
US Census
General Social Survey
Online Surveys

6. Structured Interview Formats
Face-to-face, self-administered ( , mail, etc), phone surveys
Typically either cross-sectional or longitudinal
Often a balance between
(1) standardization
(2) ability to reach sample
(3) available resources
(4) type of questions you are asking

7. Standardization of Wording and Format in an Interview Instrument
Must have clear wording
Must have clear formatting
Must take into account the sample population that will actually take/use the survey instrument
Culture, language, interpretive ambiguity
Must have a logic for each question
Must have a logic for the question responses (if provided)
Must have a logic for the sequencing of questions

8. Types of Questions (review)
Dichotomous
Categorical; only 2 primary responses are possible (True/False, Yes/No)
Nominal
Categorical; numbers only used as placeholders for responses.
Ordinal
Categorical; numbers imply an underlying ordered structure
Interval
Semantic differential
Likert-style questions

9. Question Logic: Purposeful Questions
Avoid asking unnecessary questions
Tendency to put in the ‘kitchen sink’ but other concerns (Considering participant fatigue, comprehension, time and other costs are all part of the survey design process)
Example: Survey on computer usage
Socio-demographic questions (age, gender)?
Risk behavior questions (drug use, etc)?
Does the response type allow you to answer the intended question?
Examples:
Closed or Open-ended questions?
Meaningful and interpretable responses?

10. Question Logic: Closed versus Open Questions
(Q1a) What do you spend most of your time doing on the Internet?
(Q1b) Which of the following things do you do most of the time on the Internet?
(a)
(b) Browse the web
(c) Play games
(d) Other

11. Question Logic: Redundancy
Usually we try to avoid redundant questions
Exception 1: Sometimes it is a good idea to ask two questions that tap same concept (first as a scale, then as a categorical decision)
Example:
First ask respondent to rate 1-10 how much they agree with several statements.
Next, ask respondents to rank the statements by how much they agree with them (i.e., 5 statements, rank them 1-5).
Exception 2: For complex concepts, we may have to ask several similar questions to ‘get’ at the underlying concepts.
“Most people are generally trustworthy” (agree/disagree scale)
“In general, I can trust others” (agree/disagree scale)

12. Question Response Logic: Scales versus Categories
As a basic rule, try to get as much information as you can without missing the intended concept.
Example: Age
Are you 18 or older (yes/no)?
How old are you (in years)? _________
With general scales and Likert-scales, consider having no “middle” category (neutral, no opinion).
(More on Scales and Scaling on Thursday)

13. Question Response Logic: Categorical Responses
Responses must be mutually exclusive
Examples (bad): What is the best college basketball team?
1) UNC-Chapel Hill
2) Duke
3) Any team that is not Duke.
Example (bad): Where do you live?
Berkeley, San Fran, Bay Area, Other
Responses must be exhaustive
Example (bad): What operating system (OS) do you use?
1) Windows
2) OSX
Use ‘Don’t Know’, ‘Other’, ‘Not Applicable’ when absolutely necessary.
Consider whether you really want respondents to indicate that they don’t know, or if you want to force a response if possible.

14. Mutual Exclusivity and Exhaustive Responses
How do you perceive communication between your department and other departments in the university? [Check one] (1) There is much communication (2) There is sufficient communication (3) There is little communication (4) There is no communication (5) No basis for perception
‘little’ ‘much’ and ‘sufficient’ are completely different issues.

15. Question Sequence
Static order for questions needs to have some rationale/logic
Grouping similar items together
coherent instrument or to keep topics clear in survey
Scattering similar items throughout survey
perhaps to hide an underlying concept that you want to measure without priming the respondent
Personal demographic questions often work best at end of survey (response rate and completion)
Randomization for all respondents if order effect is a concern.

16. Clear Wording / Leading Questions
Examples:
“What ISP do you use?
“What company or service provider do you use to get access to the Internet, if applicable?”
Leading Questions
“Don’t you think we should support our troops in Iraq?”
“How strongly do you agree or disagree with the following question: ‘We should support our troops in Iraq’”
Key issues:
Unambiguous
Clear vocabulary that isnt condescending
Respondents must know enough to be able to respoond

17. Clear Formatting, Logic and Branching
Not all questions apply to everyone
Example:
“How much do you spend on gas heat each month?”
Branching is a possible solution
“Do you have gas heat? If yes, go to next question. If not, skip to question #3.
Condense when possible to avoid unnecessary branching.
“How much do you spend on gas heat each month? (write a value, or check ‘not applicable’)
Purpose for all questions!
Be especially careful with contingent questions and filters

18. ‘Unpacked’ Questions
Some people believe that the internet reduces face-to-face interaction and may cause ‘internet addiction’. Do you agree or disagree with these people? (1) Agree (2) Disagree (3) Do Not Know (4) Refuse to Answer