1. Questionnaires
September 22, 2008

2. Human subjects research
Observations
Case studies
Recordings
Experiments
Questionnaires
Interviews
Before starting
IRB (Institutional Review Board)

3. Human subjects research
For each type of HSR, we will examine
a. What it is/when to use it (with what kind of data)
b. How to use it (example studies using these methods)
c. Advantages and disadvantages
observer’s paradox
data collection
data analysis
variable control

4. 4. Experiments b. How to use them What is typically tested:
Response times
Responses (Accuracy)
Eye movements

5. Experiment 3: Reading
Measuring eye movements while reading

6. Experiment 3: Reading Eye movement terms . . . 1. Saccade:
Movements of eyes during reading
2. Regression:
Saccades that move backward
3. Fixation:
Time we spend looking at a certain location 6

7. Eye trackers 7

8. 8

9. The more proficient we become in reading,
the less regressions and fixations we have.
Fixations are 14 characters long
5-6 characters before and 7-8 after fixation 9

10. 4. Experiments c. Advantages:
Provide precise data for/against a given hypothesis (e.g. dichotic)
With enough detail when published, can be replicated
Disadvantages:
Can results be generalized outside of experiment (e.g. language learning)
Often, quite hard to set up experiment correctly, and to account for all variables
Some people don't make good research subjects (e.g. dichotic for children)

11. 4. Experiments
Stanley Milgram experiments in (
Question of how Nazis could get people to torture other people. Shock experiment with lab coated scientist standing by.

12. 5. Questionnaires a. What it is/when to use them
Types of Questionnaires
group/individual open/closed
Face-to-face (Utah project)
Telephone (Telsur)
Postal (old dialect studies)
On-line (Dialect survey,

13. 5. Questionnaires
b. How to use it (example studies using these methods)
Different types of questions
a. Completion: (A person who helps put out fires is called a ____): are all of the answers relevant?
b. Picture: but are all of the answers relevant: tennis shoes, Nikes
c. Multiple choice: (sneakers, tennis shoes, gym shoes): but have you listed them all?
d. Likert-type scales: (This speaker sounds friendly/Jewish/low class): always positive/negative on one side?
e. True/false (Do you say who or whom): overly simplistic response
f. Open-ended questions ("What do you think is different about Utah dialect"): potentially leading / biasing ("What is is about Utah dialect that you find particularly strange?)

14. The Utah English Project Team
Diane, Wendy, Jared and Danny

15. Some of the Questions from the Survey
In what building does a farmer keep his animals?
Name the days of the week
What is the term for a sweetened carbonated beverage?
If you had to say there’s a difference, what’s the difference between a bag and a sack?
What do you say when people say thank you to you?

16. 5. Questionnaires Questionnaires and surveys Examples:
Please pronounce the following words: mail, still/steel, pull/pool, full/fool
Does the u in student sound like the oo in too or the u in use
After Bill had _____ (bought) the computer, he realized he'd made a mistake
Which is better: Who am I talking to? / To whom am I talking?
What do you call the thing you rent from the video store (movie, show, video)
What do you call the paper container you take things home from the store in? (sack, bag)
Count to ten.
What are the days of the week
What do people here say funny?

17. Some challenges in creating and presenting questionnaires
Achievement of a random sample
Confusion over some of the questions
Worry about keeping anonymity
Worry about saying the wrong thing
Anger of some of the groups (non-LDS participants)
Belief that they did not have a Utah “accent”
Suspicions about our work/danger to us
Overexcitement about helping us out
People’s feelings about language (Utah English)
Coaching by husbands/wives

18. Pragmatic surveys: 1. role-playing
You need to borrow money from a person that you have known for about 2 months (a friend, but not a best friend). Ask the person if you can borrow $20.
2. fill in the blank questionnaires
A: An older gentlemen sits next to you at the bus stop and says, “You look very nice today.”
B: _____________________________________
A: A young man your age sits next to you at the bus stop and says, “You look very nice today.”
3. multiple choice questionnaires
If an older gentleman approached me and said, “You look very nice today,” I would
turn and look the other way
say “thank you” and keep walking
blush and smile
stop and talk to him
e. other: ______________________________

19. 5. Questionnaires General issues in terms of questions:
Usable data (numbers) vs patronizing (treating them as data-producers)
Number of questions
Ordering of questions
"Cross-referencing"
Running a pilot study first (for large projects)

20. 5. Questionnaires c. Advantages and Disadvantages Advantages:
Specific items relate to specific hypothesis -- easy to elicit what you want
Every subject gets the same questions (compared to trying to hear same pronunciation or construction in all subjects in spontaneous speech)
Disadvantages:
Observer’s paradox
People’s beliefs about what they do often don’t coincide with their behavior. People think they say X more or less often than they really do. Stigmatized regionalisms/ethnolect
People may tell you what you want to hear (MTC motivation study)
People may sometimes lie and researchers are sometimes biased (political surveys, customer satisfaction surveys)
Mismatch between researcher and subjects (e.g. African-American children and white, male, middle-aged researcher)