1. What Can Qualitative Software Do for My Research? November 5, 2013 APHA Conference

2. Abt Associates | pg 2 Outline of Session  Overall Goal: Understand how qualitative data analysis software can improve the rigor of your public health research  Short introduction to qualitative analysis and computer-assisted qualitative data analysis software (CAQDAS)  Introduction to coding and coding exercise  NVivo demonstration and exploration

3. Abt Associates | pg 3 A Brief History of CAQDAS  1981: Lyn and Tom Richards develop NUD*IST, the precursor to NVivo  1994: Miles and Huberman discuss the use of software in qualitative analysis in their widely cited text  2007: National Science Foundation publishes guidelines for the use of software in qualitative data analysis  2013: NVivo, AtlasTi, EZ-Text, ANSWR, MaxQDA, HyperResearch and Dedoose are among the most commonly used tools today  Today at APHA 2013: Over 30 presentations mention using NVivo in their abstracts (see handout)

4. Abt Associates | pg 4 How can Software Help Improve the Rigor of Qualitative Analysis? GATHER Existing Research Current Information Data Sources ORGANIZE Data Sources Coding Structure Demographics Literature Review ANALYZE Coding Queries Memos Visuals VISUALIZE RESULTS Reports Journal Articles Dissertations Study Presentations Transparency Saturation Methodology

5. Abt Associates | pg 5 Considerations in Choosing to Use Software  Sample size and multiplicity of data sources  Emphasis on replicability, rigor and transparency  Likelihood that there will be future opportunities to perform secondary analyses on the same dataset  Desire to publish in peer-reviewed journals  Interest in merging close-ended attributes into the qualitative dataset  Building capacity of analysis team including training time and costs  Budgetary parameters and software investment

6. Abt Associates | pg 6 Promoting Reliability and Validity in Analysis  Document the process of analysis including what is the statement of a respondent and what is interpretation by a researcher  Involve multiple analysts to check biases  Document in detail the process by which analytical themes or codes are developed  Train coders or analysts on coding structure and create well- defined themes. Refine again and again.  In analysis, check inter-rater reliability  Develop conventions for transcribing data so that transcripts are comparable across data source  Develop saturation guidelines

7. Abt Associates | pg 7 Future Directions in Qualitative Analysis  Mixed methods tools, such as the capacity to work with datasets containing both fixed response and open-ended material  Web-based data, including social media and online discussion boards  Capacity for larger samples, especially large quantities of text (qualitative studies are no longer small)  More tools for comparing coding by researcher, theme, and participant group

8. Abt Associates | pg 8 A Future Look at Mixed Methods with NVivo Descriptive Statistics Inferential Statistics Meta-Analysis Coding of Text Audio, Video and Image Data Open-Ended Survey and Interview Responses Transcribing tools for Audio and Video Import/Export from Excel, text and database files Open-Ended and Fixed Response Questions Within and Between Group Analysis of Coding Text Analysis Kappa Coefficient Cluster Analysis of Word and Coding Similarity Qualitative Quantitative NVivo

9. Abt Associates | pg 9 Coding Exercise

10. Abt Associates | pg 10 Brief Coding Exercise: Purpose  We will code a short transcript manually –Given the short timeframe we will not do this thoroughly  We will talk about what we found and how we might begin to refine our codes  We will discuss how we would use qualitative software to accomplish the same steps

11. Abt Associates | pg 11 Coding Any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding. (Anselm L. Strauss, Qualitative Analysis for Social Scientists, 1987, p. 27)

12. Abt Associates | pg 12 What is Coding?  Codes are short words or phrases that symbolize the essence of a piece of text, visual image, or other qualitative data.  Codes reduce a large quantity of data into more manageable “themes.”  Interpret qualitative data into meaningful themes (meaningful depends on the lens of the analyst)

13. Abt Associates | pg 13 EXAMPLE: Focus Group on Treating Chronic Fatigue Patients Physician Participant “There are also trends over times. When I was in training, everybody who we now consider chronic fatigue or even chronic fibromyalgia was largely looked into a group that they called the hypochondriacal patients. Now you hardly ever hear the diagnosis hypochondriasis anymore.” Code  Trends in Diagnoses OR  Physician training OR  Chronic fatigue syndrome familiarity

14. Abt Associates | pg 14 Coding process  Initial codes will be defined, redefined, collapsed as more and more data are coded  Analysts will develop definitions, inclusion and exclusion criteria for each code

15. Abt Associates | pg 15 Analysis of Codes/Themes  Codes/themes are analyzed for patterns, e.g. –frequency, –similarity and differences across respondent types, –meaning, –sequence, –associations with other codes, –causation, etc.

16. Abt Associates | pg 16 Brief Coding Exercise - Instructions  Read the interview with Thomas – 10 minutes  Remember the purpose of the research. (Description in packet.)  Look for themes. Put parentheses around the text and write the word/phrase that summarizes the text next to it.  Note questions or ideas that occur to you as you read the interview.  Report out and discussion – 10 minutes

17. Abt Associates | pg 17 Brief Coding Exercise – Themes What themes did you come up with?

18. Abt Associates | pg 18 Brief Coding Exercise – Discussion  What codes are similar to each other?  How will we capture the changing definitions over time?  How will we merge codes?  How will we assess inter-rater reliability?  What questions or comments occurred to you as you read the interview?

19. Abt Associates | pg 19 Brief Coding Exercise – Summary  What works manually with a small amount of qualitative research becomes more complicated as the number of qualitative sources increase  Defining, redefining, merging, and separating codes is easier to do and easier to keep track of with qualitative software  Documenting the analysis process systematically is a benefit of the software; replicability is possible

20. Abt Associates | pg 20 NVivo Demonstration and Exploration

21. Abt Associates | pg 21 Let’s explore NVivo in practice How does NVivo store data? Sources may be text, audio, video, pictures, categorical or social media data How do I code in NVivo?Select data and assign to one or more Nodes How can I see my coding in NVivo?Turn on Coding Stripes, and open Nodes Isn’t there more the software can do? Text Search and Word Frequency Queries can help you code. Matrix Coding Queries can reveal patterns within and across themes or groups of participants.

22. Abt Associates | pg 22 Systematic, Rigorous, Quick  Increase accessibility of data and transparency of analysis o Node content, memos, annotations, coding stripes, event log  Identify and test ideas about emerging patterns and themes o Text Search Query, Matrix Coding Query  Utilize open-ended text and non-text data o Transcripts, audio, video, pictures, survey and other categorical data, social media  Generate output for reporting o Query results, node exports, visualizations  Support team-based analysis o View team members’ coding, calculate Kappa coefficient

23. Abt Associates | pg 23 Utilizing Non-text Data Classification (First Cell) PersonGenderAge RangeYears in Down East Primary Residence BarbaraFemale40-4940Yes CharlesMale60-6972Yes DorthyFemale20-3940Yes HelenMale70-79N/AYes Attributes (First Row) Nodes (First Column)

24. Abt Associates | pg 24 Questions and Comments?

25. Abt Associates | pg 25 Contact Information  Lisa LeRoy, MBA, PhD Lisa_leroy@abtassoc.com 617-349-2723  Ilana Ron, MSc Ilana_ron@abtassoc.com 301-347-5339  Cynthia Jacobs, EdD C.Jacobs@qsrinternational.com 617-491-1850