1. PS430 Program Design and Evaluation SEMINAR #7 Dr. Edward Cumella

2. AGENDA Focus Groups

3. Program Development & Evaluation3 A Focus Group Is... What A carefully planned discussion To obtain perceptions of a defined interest area

4. Program Development & Evaluation4 A Focus Group Is... Where In a permissive, non-threatening environment

5. Program Development & Evaluation5 A Focus Group Is... Who Approximately seven to ten people With common characteristics relating to discussion topic

6. Program Development & Evaluation6 A Focus Group Is... How Conducted by a trained interviewer (moderator, facilitator). Three focus groups are the minimum for a study

7. 7 Why Do Focus Groups? To collect qualitative data To determine feelings, perceptions, and manner of thinking of participants regarding products, services, programs, or opportunities Attitudes and perceptions are developed in part by interaction with other people To promote self-disclosure among participants It's dangerous to take "customers" for granted

8. 8 When to Conduct Focus Groups Before a program begins, during a program, or after a program ends Focus groups are effective when o People have something to share (motivations) o The goal is to understand human behavior Focus groups are not effective when o People are divided or angry o The goal is to gather factual data o Organization is trying to improve its image

9. 9 Selecting Participants Set exact specifications for who you want on the focus group Maintain control of the selection process Use the resources of the sponsoring organization in recruiting Beware of biased groups Develop a pool of eligible participants and then randomly select final participants So how would we apply these rules at GlobalEd?

10. Program Development & Evaluation10 Selection Strategies List On location Nominations Random phone screening Ads in newspapers and bulletin boards

11. Program Development & Evaluation11 Incentives for Participants Money ($20-$50) Food Gifts Positive, upbeat invitation Which would we use at GlobalEd?

12. Program Development & Evaluation12 Systematic Notification Procedures 1. Set meeting times 2. Contact potential participants by phone or in person (1-2 weeks before meeting time) 3. Send a personalized invitation 4. Phone (or contact) each person the day before the focus group

13. 13 Moderator Skills Is mentally prepared Selects appropriate location Records the discussion Uses purposeful small talk Has a smooth & snappy introduction Uses pauses and probes Uses subtle group control Controls reactions to participants Uses an assistant moderator Uses appropriate conclusion

14. Program Development & Evaluation14 Beginning the Focus Group Discussion The first few moments in focus group discussion are critical. Create a thoughtful, permissive atmosphere Provide the ground rules Set the tone Recommended introduction pattern Welcome Overview and topic Ground rules First question

15. Program Development & Evaluation15 Beginning the Focus Group Discussion Welcome Overview and topic Ground rules

16. Program Development & Evaluation16 Asking Questions That Yield Powerful Information Carefully prepare focus questions Use standardized questions Use open-ended questions Avoid dichotomous questions "Why?" is rarely asked Use "think back" questions

17. Program Development & Evaluation17 Ending Questions Summary question "Is this an adequate summary?" All things considered question Ask participants to reflect on the entire discussion and then offer their positions or opinions Final question "Have we missed anything?

18. Program Development & Evaluation18 Systematic Analysis Start while still in the group Continue immediately after the focus group Later--within days--analyze the series of focus groups Finally, prepare the report with your findings

19. Program Development & Evaluation19 Focus Group Analysis Tips When analyzing focus group data, consider… Individual words Context (who is the speaker?) Consistency across members Frequency of comments Intensity of comments Responses that are very specific

20. 20 Focus Group: EXAMPLE Moderator: How do you feel about the way our students are getting along with each other at GlobalEd?

21. 21 Focus Group: EXAMPLE Parent 1: My son has been at GlobalEd for 3 years. He used to love coming to school. Now I can’t get him out of bed in the morning. He is sick more than he used to be; I think he is looking for excuses not to go to school. His grades are still good, thank God, but he’s not really happy. A few weeks ago it got so bad that I was pretty close to pulling him out of GlobalEd. He came home from school and was really withdrawn. I couldn’t get him to talk. He was angry at me, I could tell, because I made him go to school that day when he said he had a headache in the morning. So I called the school that afternoon and found out that one of Mike’s friends had been rushed to the hospital in an ambulance due to a fight he was in. It sounded like several local boys from here in DC were picking on this kid, Mike’s friend, who is from Malaysia. Mike’s friend is a thin boy, very smart, shy, not too good at sports. The other kids beat him up in the boy’s bathroom and the school nurse wanted to be sure he did not have a head injury so she called the ambulance. Turns out the young man is OK. But Mike is scared and he is angry at what is happening at the school. We know that all schools have some violence in them today, but this really hit close to home. We love Mike’s teachers and we know he is getting a good education, but the climate at the school is just not OK anymore. Something has to change and FAST! We have to protect our kids!

22. 22 Focus Group: EXAMPLE Parent 2: My daughter is a freshman. She loves GlobalEd. She’s on the girls’ basketball team and has a lot of friends. She sometimes tells us about bullying that she sees going on at GlobalEd, but it hasn’t happened to her or anyone on the team. I guess the girls on the team hang together so they are immune to what is happening with the rest of the students. We go to all her games. The basketball team is mostly White, but there is one really tall girl from Nigeria on the team, and everybody adores her. So GlobalEd is working for out really well for our daughter.

23. 23 Focus Group: EXAMPLE Parent 3: Our son is a senior. He can’t wait to graduate and get out of GlobalEd. I have had to send him to counseling for the past year to deal with all the ugly things some of the other students have said to him. If he wasn’t on scholarship, and we could afford a better school, we would have moved him already to a different school. But we’re stuck. The local public school is terrible; at least at GlobalEd our son is getting a good education and is prepared for college. No one has hurt him physically, or we would have taken him out of the school. But he goes to Karate, and sometimes we think that’s the only reason he has been safe at GlobalEd. But the pounding on his self-esteem has been severe. He’s on an anti-depressant, sees his counselor every week, and is doing a little better, but he has been picked on a lot in the past year. Mostly stupid stuff, teenage stuff—like calling him gay because he is a Christian and does not want to have sex until he is married. But some of the taunting has been racist. He’s biracial—my husband is Black and I’m obviously White. Some of the kids from Romania and the Czech Republic—they seem to be a clique in the school—have been calling him a “half-breed” and other things I don’t want to say in public. We’ve always done a good job helping our son to understand and be proud of his identity, so he’s strong, but the constant humiliation at school has taken a real toll on him.

24. Program Development & Evaluation24 Focus Group: EXAMPLE When analyzing focus group data, consider… Individual words Context (who is the speaker?) Consistency across members Frequency of comments Intensity of comments Responses that are very specific

25. Program Development & Evaluation25 Reporting Focus Group Results Written Communications Strategy Strive for enlightenment, not criticism Make points memorable Use narrative and bulleted formats Give thought to how you will present the report orally

26. END OF SEMINAR FOR TONIGHT! Thank you for being here!! Thank you for contributing!