1. Data collection methods Questionnaires Interviews Focus groups Observation –Incl. automatic data collection User journals –Arbitron –Random alarm mechanisms Mixed methods

2. Interviewing Goals Selection of respondents Interview logistics Content Question formulation –“Attitude” –Question types, questioning strategies Dealing with the data

3. Goals What do you need to know? From whom? How can you get at this information? Validity and reliability issues –Recollection and self-reporting

4. Interviews vs. surveys Interviews: –Open-ended –Can establish relationship with respondent –Can follow up on responses –Flexible –Time consuming Questionnaires –Self-administered –Can survey many people –Easy to analyze –Easy for them to say no –Limited ability to ask for clarification

5. Uses of Interviews in Usability User and task analysis –About current activity –Ask about preferences, perceived needs –Evaluation of available tools, services… –Including: contextual interviews Watch them do their work, activity and ask about it Usability assessment at various stages of development –Ask them about usability issues –Interview them before and after they use a prototype –Ask them about a product that they are using –And so forth

6. Goals - Content Respondents’ characteristics – demographics, experience, training… Respondents’ opinions, preferences, understandings, reasoning, opinions, feelings… (Why did you x?) History Recollection of behavior Hypotheticals Responses (“Try this and tell me …”)

7. Goals: selection of respondents From whom do you need info? From whom can you get info? What info can you get from people whom you can reach? How much time will it take? What resources do you have? Value? In general: –Range of different kinds of people –Several of each kind –Look for people who are good information providers: articulate, observant, cooperative –Rule of thumb: keep going until you don’t learn anything more

8. Locating, identifying respondents Ask people for suggestions Pre-qualify via phone or email Look at org charts and the like Know that some interviews will be throwaways Be open to the unexpected Look for opposing views Be cautious: –Managers vs. front-line employees –Those whom they think you need to talk with –People with axes to grind

9. Logistics You are asking them to do you a favor and give you their valuable time Contact them ahead of time Purpose of study, why them, what you want to know Scheduling – be flexible; be respectful of their time; be clear how long you need Convenient for them Pairs, groups? Sometimes. In their location if you can NOT too many in one day Do your homework

10. Logistics, more 2 interviewers if possible –Questioner; note-taker Tape record if possible –Ask permission, explain reasons –Turn off if needed –Make sure everything works, have back-ups Detailed notes of your own ASAP –Tape recorder, laptop –Include impressions, physical setting, chance events…

11. The interview Explain goals of project, interviews – how you will use data Promise and explain confidentiality Set them at ease, establish relationship Show that you have done your homework Know your goals going into interview –Have a list of questions and/or topics –Be willing to follow digressions if potentially useful Ask questions respectfully; follow up as needed End on time Ask them: Anything else I should have asked? Thank them. Ask how to contact if you have later questions. The “going out the door” comments

12. Gaining Trust Be honest Be interested Be sympathetic – but not artificially so Depersonalize conflicts, problems LISTEN; ask questions for clarification, but also know when to keep silent Be willing to show your ignorance when necessary How you talk about others to them is how they assume you will talk about them to others

13. Interview structures Demographics, easy questions first Specific to general –Specific instance > principles General to specific –“What do you generally shop for online?” –Evaluation of THIS site End with something easy and congenial

14. Question formulation Match respondents’ language Match respondents’ behavior, models of their work, their world

15. Asking good questions What do you need to know? Open-ended, neutral Non-defensive, non-offensive Some possibilities for difficult questions: –Devil’s advocate –Paint a scenario –Point to others (but don’t break confidentiality) –Point to events (“at the last meeting, there was a debate about…” Reflective listening/paraphrasing –Incl describing what they did (when observing them) Non-directive probes –“tell me more about…” Be (somewhat) willing to go where interviewee wants to go

16. Question types

17. What to do with the data Label tapes well Detailed records of who was interviewed, when, where, by whom, their contact info, where your files are… Transcriptions and notes Key points, themes, findings Be obsessive about data – hard to know what you will need later

18. Interviewing in Field Studies Ask about goals, don’t just focus on tasks, listen for goals for the benefit of others Probe goals, tasks presented as goals Neutral vs. leading or blaming questions Don’t be shy, ask for more information, provide active feedback that you are listening Ask user if your interpretation is correct, listen for “no” in pauses, maybes