Questionnaires September 22, 2008

Human subjects research Observations Case studies Recordings Experiments Questionnaires Interviews Before starting . . . IRB (Institutional Review Board) http://orca.byu.edu/IRB/

Human subjects research For each type of HSR, we will examine a. What it is/when to use it (with what kind of data) b. How to use it (example studies using these methods) c. Advantages and disadvantages observer’s paradox data collection data analysis variable control

4. Experiments b. How to use them What is typically tested: Response times Responses (Accuracy) Eye movements

Experiment 3: Reading Measuring eye movements while reading

Experiment 3: Reading Eye movement terms . . . 1. Saccade: Movements of eyes during reading 2. Regression: Saccades that move backward 3. Fixation: Time we spend looking at a certain location 6

Eye trackers . . . 7

8

The more proficient we become in reading, the less regressions and fixations we have. Fixations are 14 characters long 5-6 characters before and 7-8 after fixation 9

4. Experiments c. Advantages: Provide precise data for/against a given hypothesis (e.g. dichotic) With enough detail when published, can be replicated Disadvantages: Can results be generalized outside of experiment (e.g. language learning) Often, quite hard to set up experiment correctly, and to account for all variables Some people don't make good research subjects (e.g. dichotic for children)

4. Experiments Stanley Milgram experiments in 1961-1962 (http://www.new-life.net/milgram.htm) Question of how Nazis could get people to torture other people. Shock experiment with lab coated scientist standing by.

5. Questionnaires a. What it is/when to use them Types of Questionnaires group/individual open/closed Face-to-face (Utah project) Telephone (Telsur) Postal (old dialect studies) On-line (Dialect survey, http://www.uwm.edu/~vaux/)

5. Questionnaires b. How to use it (example studies using these methods) Different types of questions a. Completion: (A person who helps put out fires is called a ____): are all of the answers relevant? b. Picture: but are all of the answers relevant: tennis shoes, Nikes c. Multiple choice: (sneakers, tennis shoes, gym shoes): but have you listed them all? d. Likert-type scales: (This speaker sounds friendly/Jewish/low class): always positive/negative on one side? e. True/false (Do you say who or whom): overly simplistic response f. Open-ended questions ("What do you think is different about Utah dialect"): potentially leading / biasing ("What is is about Utah dialect that you find particularly strange?)

The Utah English Project Team Diane, Wendy, Jared and Danny

Some of the Questions from the Survey In what building does a farmer keep his animals? Name the days of the week What is the term for a sweetened carbonated beverage? If you had to say there’s a difference, what’s the difference between a bag and a sack? What do you say when people say thank you to you?

5. Questionnaires Questionnaires and surveys Examples: Please pronounce the following words: mail, still/steel, pull/pool, full/fool Does the u in student sound like the oo in too or the u in use After Bill had _____ (bought) the computer, he realized he'd made a mistake Which is better: Who am I talking to? / To whom am I talking? What do you call the thing you rent from the video store (movie, show, video) What do you call the paper container you take things home from the store in? (sack, bag) Count to ten. What are the days of the week What do people here say funny?

Some challenges in creating and presenting questionnaires Achievement of a random sample Confusion over some of the questions Worry about keeping anonymity Worry about saying the wrong thing Anger of some of the groups (non-LDS participants) Belief that they did not have a Utah “accent” Suspicions about our work/danger to us Overexcitement about helping us out People’s feelings about language (Utah English) Coaching by husbands/wives

Pragmatic surveys: 1. role-playing You need to borrow money from a person that you have known for about 2 months (a friend, but not a best friend). Ask the person if you can borrow $20. 2. fill in the blank questionnaires A: An older gentlemen sits next to you at the bus stop and says, “You look very nice today.” B: _____________________________________ A: A young man your age sits next to you at the bus stop and says, “You look very nice today.” 3. multiple choice questionnaires If an older gentleman approached me and said, “You look very nice today,” I would turn and look the other way say “thank you” and keep walking blush and smile stop and talk to him e. other: ______________________________

5. Questionnaires General issues in terms of questions: Usable data (numbers) vs patronizing (treating them as data-producers) Number of questions Ordering of questions "Cross-referencing" Running a pilot study first (for large projects)

5. Questionnaires c. Advantages and Disadvantages Advantages: Specific items relate to specific hypothesis -- easy to elicit what you want Every subject gets the same questions (compared to trying to hear same pronunciation or construction in all subjects in spontaneous speech) Disadvantages: Observer’s paradox People’s beliefs about what they do often don’t coincide with their behavior. People think they say X more or less often than they really do. Stigmatized regionalisms/ethnolect People may tell you what you want to hear (MTC motivation study) People may sometimes lie and researchers are sometimes biased (political surveys, customer satisfaction surveys) Mismatch between researcher and subjects (e.g. African-American children and white, male, middle-aged researcher)