1. Educational Action Research Todd Twyman Summer 2011 Week 2

2. Qualitative Research Goal: to understand deeply Goal: to understand deeply Intense and/or prolonged contact with life situation Intense and/or prolonged contact with life situation Approach: Interpret as you go; keep questions open so you don’t miss new insights that lead to discovery, be open to emerging themes as you go. Approach: Interpret as you go; keep questions open so you don’t miss new insights that lead to discovery, be open to emerging themes as you go.

3. Qualitative Data In-depth view of the phenomena. In-depth view of the phenomena. Must be situated in context. Must be situated in context. Multiple data sources are important. Multiple data sources are important.

4. How will you gather your data? Observation Observation Interview / focus groups Interview / focus groups Extant data sources / document analysis Extant data sources / document analysis Survey with open-ended questions Survey with open-ended questions

5. Field Observation Take clear notes. Take clear notes. Include both observations of what you see AND reactions / reflections as you go. Include both observations of what you see AND reactions / reflections as you go. Conduct “member checks” to be sure your interpretation makes sense. Conduct “member checks” to be sure your interpretation makes sense. Try to remain outside the situation; write down what you observe NOT what you interpret! Try to remain outside the situation; write down what you observe NOT what you interpret!

6. Observation vs Interpretation Observation: Captures what you see, avoiding interpreting. Observation: Captures what you see, avoiding interpreting. I see a woman with grey hair walking slowly across the quad. I see a woman with grey hair walking slowly across the quad. Interpretation: Imbues what you see with additional meaning/judgments. Interpretation: Imbues what you see with additional meaning/judgments. I see an old, tired woman limping across the quad. I see an old, tired woman limping across the quad.

7. Field Observation Observations Reflections Time What I saw What I think about this…

8. 3 Sources of Data Dependent on Appropriate Questions Interviews Interviews Focus Groups Focus Groups Open-Ended Survey Questions Open-Ended Survey Questions

9. Writing Good Questions Avoid questions that begin with the word “why”. Avoid questions that begin with the word “why”. Avoid questions that can be answered with a “yes” or “no” response. Avoid questions that can be answered with a “yes” or “no” response. Avoid ‘tipping your hand’ through the wording of your questions -- try to remain neutral in how you ask them. (both semantically and through tone). Avoid ‘tipping your hand’ through the wording of your questions -- try to remain neutral in how you ask them. (both semantically and through tone).

10. Interviews Be prepared (recording device / paper with pencils; list of questions; on time). Be prepared (recording device / paper with pencils; list of questions; on time). Concentrate on helping the person you are interviewing feel at ease. Concentrate on helping the person you are interviewing feel at ease. Ask open-ended questions that allow the person to provide you with rich insight. Ask open-ended questions that allow the person to provide you with rich insight. Ask follow-up questions to probe more deeply into topics as they arise. Ask follow-up questions to probe more deeply into topics as they arise.

11. Interviews Include information to establish context (setting, atmosphere, emotions -- if any). Include information to establish context (setting, atmosphere, emotions -- if any). Check to be sure you captured their meaning (member checking). Check to be sure you captured their meaning (member checking). Consider what is not said as well as what is said, and how it is said. Consider what is not said as well as what is said, and how it is said. Have your interview protocol ready in advance. Have your interview protocol ready in advance.

12. Example Interview (Todd) 1. Describe the ESL population at your school.  How many?  At what grade levels?  New Comer/Developing/Transitional?  Language backgrounds?   Cultural/SES backgrounds? 3. Can you tell me about the goals for the ESL Program?  Timeframe for transitioning?  NC/D/T?  L1 use?  Family interaction?

13. Focus Groups Make sure you have introduced participants. Make sure you have introduced participants. Have a facilitator AND a note-taker. Have a facilitator AND a note-taker. Solicit opinions from all participants. Solicit opinions from all participants. Ensure all participants are treated respectfully. Ensure all participants are treated respectfully. Consider your groupings carefully. Consider your groupings carefully.

14. Focus Groups Like interviews, have your focus group protocols ready to go ahead of time. Like interviews, have your focus group protocols ready to go ahead of time. Provide participants with an abbreviated list of questions; retain the full protocol to guide you in case you get nervous or distracted. Provide participants with an abbreviated list of questions; retain the full protocol to guide you in case you get nervous or distracted. Send ‘Thank You’ cards (more personal than email) to all participants. Send ‘Thank You’ cards (more personal than email) to all participants.

15. Extant Data Sources / Document Analysis Read with an open mind. Read with an open mind. Look for recurring themes as well as those that occur rarely. Look for recurring themes as well as those that occur rarely. Be creative in searching out appropriate documentation. Be creative in searching out appropriate documentation. Student work. Student work. School newsletters. School newsletters. Minutes from School Board meetings. Minutes from School Board meetings.

16. Survey (open-ended questions) Ask questions that will prompt sufficiently detailed responses. Ask questions that will prompt sufficiently detailed responses. Focus on what is said, how it is said, and what is not said. Focus on what is said, how it is said, and what is not said. Consider following up especially intriguing responses with an interview. Consider following up especially intriguing responses with an interview.

17. How will you make sure your data are reliable? What sort of training will observers/ scorers/ raters receive? What sort of training will observers/ scorers/ raters receive? How will you determine if your observers/ scorers/ raters are performing reliably? How will you determine if your observers/ scorers/ raters are performing reliably? How will you check for how your own biases / life experiences might affect your results? How will you check for how your own biases / life experiences might affect your results?

18. If you’re going to be interviewing Have you: Have you: Practiced with your recording devices? Practiced with your recording devices? Thought out the types of follow-up questions you might want to ask? Thought out the types of follow-up questions you might want to ask? Had someone check to be sure you are not ‘leading’ the people you are interviewing? Had someone check to be sure you are not ‘leading’ the people you are interviewing?

19. Validity in Qualitative Studies Use multiple sources of data (triangulation) Use multiple sources of data (triangulation) Have analyses / interpretations checked by participants (member checking) Have analyses / interpretations checked by participants (member checking) Have more than one person conduct analyses; check for common interpretations between those engaged in analysis Have more than one person conduct analyses; check for common interpretations between those engaged in analysis Use ‘double blind’ techniques for all analyses Use ‘double blind’ techniques for all analyses

20. Practicing Field Observations Observation: Go someplace with people where you can do a 10 minute observation. Try to be an ‘impartial observer’. Observation: Go someplace with people where you can do a 10 minute observation. Try to be an ‘impartial observer’. Take field notes (you’ll be sharing these). Take field notes (you’ll be sharing these). Pay particular attention to the flow of human traffic. Who is there? How are they interacting? Try to capture the experience with rich description. Pay particular attention to the flow of human traffic. Who is there? How are they interacting? Try to capture the experience with rich description.

21. Practice Interviewing Focus on drawing out your partner’s thoughts / feelings on what he/she observed outside. Focus on drawing out your partner’s thoughts / feelings on what he/she observed outside. Remember to: Remember to: Use open-ended questions. Use open-ended questions. Use your best active listening techniques to encourage your partner to explore his/her thoughts deeply. Use your best active listening techniques to encourage your partner to explore his/her thoughts deeply. Go beyond the surface, using follow-up questions to get at your partner’s underlying beliefs of which he/she may not even be aware. Can you identify certain perspectives? Go beyond the surface, using follow-up questions to get at your partner’s underlying beliefs of which he/she may not even be aware. Can you identify certain perspectives?

22. Practice Interviewing Step 1 Write your Interview Protocol Write your Interview Protocol Step 2 Interview Interview Step 3 Debrief Debrief

23. Debrief Interviewing Describe how it felt to be interviewed. Were there things the person interviewing you did / could have done to make the interaction more positive? Describe how it felt to be interviewed. Were there things the person interviewing you did / could have done to make the interaction more positive? Short discussion of interviewing experience. Short discussion of interviewing experience.

24. Writing Open-Ended Survey Questions Working with a partner, brainstorm some possible open-ended survey questions you could use to gather information about your topic. Working with a partner, brainstorm some possible open-ended survey questions you could use to gather information about your topic. Meet in groups of 4, discuss and critique each others’ questions. Meet in groups of 4, discuss and critique each others’ questions. What ‘rules of the road’ should folks keep in mind as they write such questions? What ‘rules of the road’ should folks keep in mind as they write such questions?