Presentation on theme: "Focus Groups Ciaran B. Trace MAC 2010 Symposium: Archival User Studies"— Presentation transcript:
1Focus Groups Ciaran B. Trace MAC 2010 Symposium: Archival User Studies Dayton, Ohio, October 21-23, 2010Focus Groups
2Outline of the presentation Introduction to focus groupsBefore you beginCarrying out the focus groupData analysisSum upQuestions?
3Resources used in this presentation Richard A. Krueger and Mary Anne Casey, Focus Groups: A Practical Guide for Applied Research, Sage: Los Angeles, 2008.David W. Stewart, Prem N. Shamdasani, Dennis W. Rook, Focus Groups: Theory and Practice, Sage: Thousand Oaks, 2007.Denise Troll Covey, Usage and Usability Assessment: Library Practices and Concerns Washington, D.C.: Digital Library Federation Council on Library and Information Resources, January Available atDaniel L. Morgan, Focus Groups as Qualitative Research, Sage: Thousand Oaks, 1997.
4What are focus groups?“A focus group is an exploratory, guided interview or interactive conversation among seven to ten participants with common interests or characteristics. The purpose of a focus group is to test hypotheses; reveal what beliefs the group holds about a particular product, service, or opportunity and why; or to uncover detailed information about complex issues or behaviors from the group’s perspective. Focus group studies entail several such group conversations to identify trends and patterns in perception across groups. Careful analysis of the discussions reveals insights into how each group perceives the topic of discussion.” [Covey 2002]
5Key elementsNotion of an interactive conversation among a small number of participantsNot focused on the experiences and opinions of any one participant [would need to do interviews for this]Evidence of the differences and similarities in the groupMore than one focus group held in order to properly explore trends and patterns in perception across groups (each focus group will last anywhere between one and two hours)Need participants and also a moderator and an observerNeed to record data and subsequently analyze that data
6What can focus groups be used for? Determining patron satisfaction with resources and servicesHelping to shape the future direction of an archive as part of the strategic planning processDeveloping and evaluating web and catalog interfacesDeveloping and evaluating user education or reference services
7Things to note…A form of qualitative research method (with all that that entails)Moderator has a distinctive roleInteraction occurs in a group settingThere isn’t a very high degree of naturalismGenerally, you are recording not what people do, prefer etc….. But what they say they do, say they prefer, etc… Data is verbal and self-reportedSmall number of participants, selection criteria, and frequent use of convenience sampling mean that you cannot generalize the resultsThe results are generally intelligible and accessible to all audiences
8Things to note…Can be used as a self-contained method, as an initial/ supplementary source of data, or as part of a multi-method approach (paired with surveys, individual interviews, participant observation etc)Paired with surveys – focus groups as a means to construct questionnaires (domains, dimensions, wording)Paired with individual interviews – focus groups as a means for devising the interview schedule for unfamiliar topics and vice versaPaired with participant observation – focus groups as a tool to help you chose among potential field sitesAs you can see…. focus groups can be used as a means to initially identify appropriate language, questions, or tasks, and as a follow-up tool to get clarification
9Before you begin… Articulate the purpose of the focus group Why is there a need for a study such as this?What is the problem that the study seeks to address?What kinds of information do you want to get from the study?What wants the information?How will the information be used?
10Before you begin…Figure out what types of people can give you the information that you wantWho has the greatest amount of insight on this issue?What particular characteristics should these people have?
11Before you begin…Figure you how many focus groups you need to hold and how they should be sequencedUsually hold 3 or 4 focus groups – allows you to be sure that you have reached saturation point and allows for data to be compared across groupsIf you want to compare and contrast how certain types of people talk about an issue separate them into different groupsIf you anticipate that participants will have conflicting opinions ,hold focus groups in two stages (separate the groups initially and then bring them together)
12Before you begin… Determine how you will capture the focus group data At minimum, will have an observer taking notesCan also audio tape the focus groups (transcribe partially or in toto?)Can also video tape the focus groups (transcribe partially or in toto?)Video taping is rarely done – it is considered too intrusiveCan take a combined approach – notes and audio, for example
13Before you begin…Think about the ethical issues and determine if you need to obtain human subjects clearance…Nature of the group setting brings up privacy concerns – limits the kind of topics the researcher can pursueWill need to get explicit permission from participants to tape (audio and video)Decide up front who will be able to listen to the audio tapes (should limit access to research staff)Make decisions about how the data will be used so that this information can be conveyed to the participants
14Recruit and schedule participants Using the criteria you have already identified, recruit and schedule participants.Not looking for generalizability but to minimize biasNot random sampling but theoretically motivated sampling or segmentationRemember that focus groups are composed of people who are similar to each other in a way that matters to the researcher – homogeneity is important but you need enough diversity to allow for contrasting opinionsInvolves scheduling six to twelve participants in designated demographic groupsUsually avoid mixing people with different levels of power or expertise“Ideally” participants should not know each otherMay want to “pre-screen” to make sure the participants have the requisite characteristics
15Prepare interview questions Think about the types of questions you need to ask ….and how you are going to sequence the questionsAlso think about what makes for good questions…..Normally ask about a dozen questions for a two hour focus groupRemember that the interview guide will have a direct bearing on the content of the group discussion
16Good questions… Good questions are… Open ended and one dimensional Sequenced (e.g. general before specific, positive before negative etc)Evoke conversation and allow for depth (ask participants to think back, avoid asking them “why”)Engage participants (can go beyond oral questions and engage people in doing things – listing, rating, choosing among alternatives)
17Good questions… Good questions… Use words that participants would use when talking about the issue, are easy to say and easy to understand, are short, are clear, include well thought out directions…..
18Types of questions used in focus groups 1. Opening question – icebreaker, for all participants, question should be designed to be easily and quickly answered (30 seconds or less), shouldn’t be used to highlight differences among the group (age, occupation, educational level), generally not included in the data analysis
19Types of questions used in focus groups 2. Introductory questions – open ended questions that introduce the topic of discussion, get people thinking about their connection to the topicShould be questions that interest all participantsMethod - ask people to remember when they first encountered the organization, service etc and to describe the experience. Or “what is the first thing that comes to mind when you hear the phrase….”
20Types of questions used in focus groups 3. Key questions - focus on the key areas of concerns. Most of the questions fall into this category.4. Ending question(s) - E.g. is there anything we should have talked about but didn’t.
21Role of moderatorRecruit, schedule, and train a moderator and observer(s) for each focus groupSkilled moderator is key - develop rapport with the participant, remain impartial, and keep the discussion moving and focused on the research objectivesModerator must have knowledge of the background topicModerators must be able to handle the human element…Moderators and observers must avoid doing anything that may bias the results of the studyGoal is to encourage conversation among the participants – NOT between the moderator and the participants
22Role of observerMain task is to observe the session and to take extensive notesNot only capturing what is said but also documenting nonverbal signs and body language – tells you a lot about how the group feels about the topic as well as giving some indication of how many people hold the same ideaNotes should therefore include meta-comments, e.g. “Jim seemed surprised at Angela’s response…”
23Handling logistical issues Handle all logistical issues associated with running focus groupsSecuring compensation?, scheduling rooms, ordering food/drink, having copies of focus group questions for moderator and observer, testing the audio- or videotape equipment, creating name placards for all those taking part in the focus groupOn the day - getting the room ready (light and temperature), setting up and testing equipment, bringing supplies (handouts, name placards etc)
24On the day…introduction, welcome, background The actual focus group itself…role of the moderatorWelcomes the participants and thanks them for participating, introduces and explains their role as moderator and the role of the observer, reiterates the purpose of the focus group, confirms human subject requirements (anonymity, taping, how data will be used etc), briefly describes the ground rules and how the focus group will be conducted, emphasizes that the goal of the focus group is not to reach consensus - disagreements and discussion are important
25NCSU sample focus group script (introduction, welcome, background) North Carolina State University LibraryThanks for taking time to join us to talk about some library tools, services, and resources that can help students with their class assignments. We here at the library are hoping to develop a suite of web tools to assist students with their learning experience. We really want to make these web pages as useful for students as possible. That's why we've asked you to come talk with us. You've probably noticed our microphone. We're going to tape record this session because we don't want to miss any of your comments. People often say helpful things in these sessions that we can't write down fast enough plus we don't want you to feel that you need to talk at a pace we can write. Please feel free to talk casually and naturally. (Co-moderator) and I will take what we learn from your tonight and share your ideas with our colleagues working on these web pages. Again, this is meant to be a relaxed discussion so we will be on a first name basis for the duration of this session. Each of you as you can see has a name placard in front of you. Hopefully the names on them are correct. We assure you that your names will not be linked in any way to any written materials produced from this focus group.
26NCSU sample focus group script (introduction, welcome, background) We're going to ask you look at some content and tools that we are thinking about including on these web pages. First we will provide a static version and would like you to rank them by your first impression of their function, and we will discuss as a group which of the pieces of content seem most interesting and helpful from your perspective and how you might envision yourself using each piece. You can even suggest new pieces of content that we might have not have thought of. After this initial exercise and discussion we will have each of you revisit each piece of content and explore their functionality as currently designed using the laptops in front of you. Try and note mentally how using these tools reinforces or dilutes the power of the tool you had initially assessed. We would like you to once again rank your top fives content pieces, singling out one as the most valuable. We will then discuss each other's rankings and how things may have changed. We feel that these activities and discussions will help us dig into what you think about tools, services, and resources that are helpful to you or that you can imagine being helpful to other students. There are no wrong answers for what we're going to discuss. We know you might have differing points of view and we value everyone's ideas so we really want to hear them.
27First and subsequent questions…. Sometimes the first question is asked round-robin and is a question that each participant can answer. In this way each participant responds and gets comfortable talking.Subsequent questions are answered less formally, and more conversationally.Moderator clarifies anything that participants do not understand.The moderator may invite further comments to better understand what the participants are saying, For example, “Would you explain that further?” or “Please give me an example.”The moderator uses verbal and body language to invite comments from shy participants and to discourage others who may be trying to dominate the conversation.
28Subsequent questions…. The probe should also be paired with another important tool…. the pause. Pause used as a way of eliciting a response from the participants.Moderator must be comfortable with these silences.
30Data analysis and reporting You might or might not transcribe the focus group tapes - cost of transcribing may exceed the benefits of having a full transcriptionAt minimum, can use the tapes to annotate the observer’s notesIdeally, at least two people analyze the data—the moderator and observer—high inter-rater reliabilityGoal is to examine, categorize and make sense of the data.
31Data analysis and reporting Review – doing a thorough read through of the data – notes, transcripts, etcConduct content analysis on the dataInitial coding – involves reading the data and assigning labels or codes to each piece of relevant informationCan be done by hand, using note cards, or using a computer.Choice of strategies - note all mentions of the code, note whether each individual mentioned a given code, note whether each groups discussion mentioned a particular codeReduce this information to manageable aggregates by creating memos for key themes that emerge from this initial codingTest the themes by going back to look at the data
32Data analysis and reporting Which topics to emphasize in the reportHow many groups mentioned the topicHow many people within each group mentioned the topicDegree of enthusiasm displayed for the topicNotion of “group-to-group” validationBlend of summary of discussions and direct reporting of comments
33Advantages of focus groups Advantages of focus groups To sum up…Advantages of focus groupsAdvantages of focus groupsA way to gather insight into behavior and experience from the participants’ perspectiveAllows for the gathering of rich data, in the participants own wordsProduce a concentrated amount of data on a precise topic quickly and at relatively low costGroup discussion creates synergy and facilitates recall and insightDirect interaction with participants allows for clarification of responses, probing of responses, and follow-up questionsResults of focus group research are user friendly and easy to understand
34Disadvantages of focus groups Disadvantages of focus groups To sum up…Disadvantages of focus groupsDisadvantages of focus groupsQuality of the responses depends on clarity of the questions, the moderator’s skills, and the participants’ understanding of the goals of the study and what is expected of themRely heavily on the observational skills of the moderator and observerQuality of the information is impacted if participants are unduly influenced by others, stray off topic, or seek consensusHawthorne effect - cannot observe something without changing itFindings have to be interpreted at the group level and are not generalizable