2Group Data Collection and Data Generating Techniques The Delphi Technique (by Kofi)Focus Groups (by Jenny)Brainstorming (by Cristian)Nominal GroupTechnique (by Sara)
3Attributes of small groups Group-based techniques involve the simultaneous use of multiple participants in the data generation/collection processThey vary greatly in terms of structure and application”They have a role to play in the exploratory stages of a study where the researcher seeks to involve potential research participants in shaping the direction and scope of an investigation”(Carney et al. 1996: 1024)They are also used if the purpose of the researcher is to increase the depth and scope of discussions on ideas and issues (More 1987)There is a better chance of getting to the “truth” if you combine the judgment of many, rather than one.Allows you to understand social phenomenon from the viewpoints of the actors.Complex problems often can only be addressed by pooled intelligence (Moore 1987, as cited by Clayton 1997).
4What is Delphi Technique? “It is a method for the systematic collection and aggregation of informed judgments from a group of experts on specific questions or issues. Repeat rounds of this process can be carried out until full consensus is reached”(Reid 1988: as in Williams & Christine 1994: 180)It is used to establish facts, forecast, generate ideas, make decisions, and obtain information when adequate information is not readily available.Three main groups are involved:Decision makerResearcherParticipants“obtain the most reliable consensus of opinion of a group of experts…by a series of intensive questionnaires interspersed with controlled opinion feedback” (Dalkey & Helmer 1963)
5The Delphi Technique Key Investigators The name Delphi ”originates from the oracle at Delphi, where the ancient Greeks were said to be able to forecast future events”(Williams and Christine 1994: 181)First conceptualized in 1953 in the USA by Air Force sponsored Rand Corporation for defense researchNow widely used as a tool for solving problems, planning, and forecasting.
6Underlying LogicHas roots in the Hegelian principle of achieving Oneness of Mind through thesis, antithesis, and synthesis.Anonymity and confidentiality of group members enhances accuracy of responsesheterogeneity within the group increases trustworthiness and authenticity.Consensus serves as a test of validityThere are problems and questions that are best answered by people’s judgments and opinions.
7Features of Delphi Technique Highly structuredNon-interactiveAnonymity-questionnaireIteration-members look at all the responses of others (sometimes in the form of stats)Controlled feedback-members provide feedback to other’s responsesStatistical aggregation of group responses
8Delphi Technique Group Member-characteristics Reflect current knowledgeAre recognized and credible in the fieldRepresent diverse perspectives
9Delphi Technique Process Researcher sets question(s) on an in issue of interestResearcher identifies and selects expertsResearcher sends out initial open-ended question (s) to the experts usually by post.Experts respond to question (s) and return them to researcher.Researcher aggregates and analyzes responses.Researcher sends out categorized data back to group members to rank and add comments.More rounds of the same until a high level of consensus is achieved.
10Disciplines that use this technique MilitaryEducationMedicineBusinessEconomicsSocial and Biological SciencesUrban and regional planning
11Strengths of the Delphi Technique It’s flexibility allows for considerable diversity in applicationConsensus on expert opinion provides evidence for concurrent validityMinimizes the influences of dominant people on small groups (Write, 2006)Panel members have the opportunity to retract, alter, or add new ideas in subsequent “rounds.”The anonymity and confidentiality of panel members ensure honest responsesEfficient means of gathering information and wisdom from geographically separated experts at relatively low costResults can be viewed as highly trustworthy if “quality” members were chosen.Personal and political conflict among group members can be avoided.Good for situations where purely quantitative methods are not appropriate; where human judgment and input is necessary.It allows for the participation of larger groups of people than other group techniques
12Weaknesses of the Delphi Technique Large time and energy investment required of membersLack of accountability with the anonymous nature of decision makingAbsence of a concrete criteria for identifying expertsLow response rates in final “rounds” result in response biasesTrustworthiness and reliability depends on the selection of, the willingness of, and ability of group members.Results are highly dependent on how the researcher frames the topic and questions, and interprets the results.Difficulty of determining the point at which consensus is reachedThere is no consensus on the size of panel which often ranges between members (Mead 1992)There is no evidence of replicability of findings (Williams and Christine 1994)
13Focus Group History1930’s – social scientists searching for an alternative to the individual interview – seeking in-depth information, and a less intrusive role for the researcher.Stuart A. Rice one of the first social scientists to lead the search for a new method.Shift to nondirective interviews with open ended questions, continuing work in the social sciences by Roethlisberger, Dickson, and Rogers in areas such as employee motivation and psychotherapy.
14More History on Focus Groups World War II interview techniques used in groups, and the focus group was born. Pioneered by Robert K. Merton in his studies of morale in the U.S. military. A classic work is The Focused Interview (1956) by Merton, Marjorie Fiske, and Patricia L. Kendall.Began being used by business sector for marketing in 1950’s.Not widely accepted until the 1980’s by academics.
15NowadaysMarket research: Learn how to sell more of a product or service. Short turnaround in processing results, so no in-depth analysis of results. Confidentiality is key; methods and results kept private to reduce competition.Academic/Scientific: More concerned with reliability, rigor, peer review, etc. Process had to be “defensible, systematic, and verifiable” (Kreuger & Casey, p. 161). Goal is to contribute to theory or a body or research.Non-profit/Public: Pragmatic; goals are to make decisions, improve programs/services, and be responsive. A mix of need for speedy results and in-depth content analysis.Participatory Research: In early 90’s - began using volunteers as researchers, from the community.
16Characteristics of Focus Groups Involve people: 6-8 or 10-12, Small enough for everyone to contribute but large enough to provide diversity of interaction.People have certain characteristics: chosen because they have something in common. Recruited based on the desired similarity. Traditionally were people who didn’t know each other, but not always the case now.Qualitative data: a range of opinions across several groups (Kreuger & Casey, p. 11). (Naturalistic approach).Focused discussions: questions are determined ahead of time, sequenced, and open ended. Purpose is NOT to come to consensus.Skilled moderatorOpen, permissive environmentUsually conducted until reach saturation. Typically 3-4 groups for each type of participant.
17Strengths of Focus Groups Ability to explore a topic in-depth, and identify key ideas or concepts.Allows researcher to understand attitudes and perceptions.Ability to interpret non-verbal responses.Group interaction canfacilitate new ideas andways of thinking.
18Weaknesses of Focus Groups Dominant individuals may jeopardize individual objectivity (Uhl 1983 as cited in Clayton, 2006).Pressure to conform may be present.Results from a small, purposive sample can be hard to defend to a quantitatively oriented audience.Can be difficult to facilitate and requires a skilled moderator.Can be costly - incentives are usually necessary.
19To use or not to use?….that is the question Uses of Focus Groups:Decision makingProduct or program developmentCustomer satisfactionPlanning and goal settingNeeds assessmentQuality MovementsUnderstanding employee concernsPolicy making and testingPrimary or secondary research tool.Do not use focus groups when:You want consensusYou want to educateYou are seeking sensitive informationEmotionally charged or potential for conflict
20Nominal Group Technique (NGT) First Developed by A.L. Delbecq and A.H.VandeVen in 1972 (Sample, 1984)It is a structured variation of a small group discussion method.Used to form a consensus.Used when there iscontroversy or conflict
21Nominal Group Technique-what is involved? Generating Ideas: Each individual in the group silently generates ideas and writes them down.Recording Ideas: Group members engage in a round-robin feedback session to concisely record each idea.Discussing Ideas: Each recorded idea is then discussed to obtain clarification and evaluation.Voting on Ideas: Individuals vote privately on the priority of ideas, and the group decision is made based on these ratings.
22Here is how NGT worksDivide the people present into small groups of 5-6State an open-ended question.Give individuals time to brainstorm by themselves and write down their ideas.Have group members share their ideas one at a time around the table and have someone record on flip chartHave individuals look at the compiled list and pick the best ideas and rate privately.Have individuals share their choices with each other and then present to the larger group, which ideas got the most points.
23Strengths Balances participation across members. Balances influence of individuals.Can generate more creative ideas than interacting groups.Can produce a greater number of ideas than do traditional interacting groups.Reduces the conforming influence common to most face-to-face group meetings.Encourages participants to confront difficult issues on a problem-solving basis rather than on a personal assault basis.Leads to greater sense of closure and accomplishment.
24WeaknessesRequires advance preparation, which means that it cannot be a spontaneous technique.Tends to be limited to a single-purpose, single-topic meeting-narrow focus.Needs agreement from all participants to use the same structured method, whichsome people might resist.Can feel mechanical.
26Key Investigators Alex F. Osborn (1953) Applied Imagination Described the BSTPresented example of how BST could be usedFirst Time Tested(1958) by Taylor, Berry and BlockClaiming that groups using BST could generate more ideas than the same number of individuals working separately.
27Disciplines that led the way BusinessPsychologyMarketingEducationIn all these cases the groups are seen as the essential part of an organization.
28Has the purpose for which the methods were developed changed? Has not changed dramatically.BST still mainly focuses on generating ideas in the business area, but also in new areas such as education, psychology.However, those ideas are more focused on solving a specific problem or issue.BST has adapted to the growing wave of technology (electronic brainstorming).
29Original and Modern Techniques Central Theme: “groups generate more ideas if their members concentrate on producing whatever ideas occur to them” (Gallupe et al., 1992, p. 351)Developed to look for new ideas that would help to sell products.An experimentation process.Find things that helpIdentify and avoid things that were inhibitiveIs both a group technique and a problem-solving technique.Doesn’t focus on a specific problem or issue.Up to 12 participants, which is confirmed in other studies
30Underlying Logic of BST GENERAL RULESSuppression of critical attitudes.Is to postpone criticism during brainstormingDon’t try to drive with your brakes onFree wheeling is welcomeRelease the imaginationBe wild in your thinkingQuantity of ideas is desiredAs many as possibleThe law of probability goes in favorCombination and improvement of ideasOthers can reinforce or supplement otherideasEveryone’s experience is different
32The Brainstorming Technique NOWADAYSGeneration of ideas in response to a given question or issue. (More focused)Maintain the same general rules, but add some operative rulesSomeone must be asked to manage the sessionA good environment is essential (Facilitator responsibility)Several techniques can be usedFace-to-face brainstormingFreewheeling, Carousel, Slip MethodAnonymous brainstormingElectronic brainstorm
33Strengths of Brainstorming Listening exercise that allows creative thinking for new ideasEncourages full participation because all ideas equally recordedDraws on group's knowledge and experienceSpirit of congeniality is created - one idea can spark off other ideas
34Weaknesses of Brainstorming Low productivitydue to the facilitatorcan be unfocusedpeople may have difficulty getting away from known realitycriticism and evaluation may occurWhat the facilitator has to doFollow the original rule, but also:Don’t tell storiesDon’t explain ideasKeep people talking
35Weaknesses of Brainstorming Low productivitydue to Social factorsProductive blocking- not all can speak at the same time.Forget ideas/or decide not to share themDifficult to process ideasOther ideas distract/interfere with thinkingSocial comparison process - matching performanceTo the least productive memberInhibitive the generation of rich ideasEvaluation Apprehension- fear of critiqueNot to present original ideasFree - riding and Social loaf- When group members perceived their own contribution to beUnidentifiableDispensable
36Brainstorm Members Should... Enter in the process with an open mindCome from different disciplinesNot critique other ideas during the sessionBe willing to work on other ideas (improve them)Respect and follow the instructions of the facilitator
37NOMINAL (Activity) BRAINSTORM What are ways the UI could be more environmentally sustainable?Brainstorm GroupGo in hallwayNominal GroupStay in this room
38Nominal group Basic Steps Individual brainstorming (5 min.). Sharing your ideas round robin (moderator will write them down).Discuss list, if necessary.Ranking the ideas individually by voting on the top 5.Share your top five (moderator will tally votes).