Analyzing & evaluating qualitative data Kim McDonough Northern Arizona University

Epistemology in qualitative research Theory of knowledge ◦ Constructionist—Meaning/knowledge vary across people and over time ◦ Holistic—complex topic is viewed from multiple perspectives ◦ Subjectivist–Knowledge/meaning is personal ◦ Context-based—Knowledge is negotiated socially and historically

Search for knowledge ◦ Truth is relative & context specific ◦ Goal is interpret how people construct meaning ◦ Start with experiences and build theory inductively ◦ Data provides information that must be interpreted ◦ Interpretation is shaped by the researcher’s own experiences

“Good” qualitative research Maintains a single focus/idea/problem Uses rigorous data collection ◦ Sufficient time in the field ◦ Many sources from multiple perspectives ◦ Detailed summary of each source Employs rigorous data analysis ◦ Multiple levels of abstraction ◦ Verification of the accuracy of the findings Engaging report ◦ Clear, detailed writing ◦ Accurately reflexes complexity of the context

Appropriate topics for qualitative research The research question asks how or what ◦ Focus on understanding The topic needs to be explored ◦ Theory doesn’t exist yet, is insufficient, or hasn’t been tested in a particular context ◦ The small pieces that make up the big picture haven’t been identified yet ◦ Time & resources are available ◦ A receptive audience exists

The natural setting is the focus of inquiry ◦ Obtain information from participants in their natural environment ◦ Reveal contextualized patterns/truths The researcher’s interests ◦ Take a personal role in the project ◦ Serve as a voice for the participants

Basic tenets of qualitative analysis The search for patterns in data and ideas that help explain the existence of those patterns The goal ◦ Reduce huge amounts of text to manageable units for further analysis ◦ Interpret the contribution of those manageable units to existing knowledge or practice

Steps in data analysis Preliminary steps: Organize the data ◦ Organize the data ◦ Label/identify source of the data ◦ Convert to appropriate text units if necessary ◦ Enter any numeric information into spreadsheet ◦ Determine which data sources are available for each participant ◦ Make decisions about inclusion or exclusion criteria based on completeness of data

Form general impressions ◦ Read all the data multiple times  Do not analyze each data source separately  Do not prioritize one source of data over another ◦ Get a sense of the whole before trying to break it into parts ◦ Make notes—short phrases, ideas, or key concepts

Analysis: Form initial categories ◦ Search for categories, themes, or dimensions ◦ Identify & name the major themes ◦ Describe each of the themes ◦ Check descriptions to refine overlapping categories

Classify data segments by themes ◦ Read the entire data set again ◦ Identify segments in all sources that belong to each theme ◦ Keep track of segments that don’t fit with a theme

Evaluation: Assess themes & segment assignment ◦ Evaluate whether the themes are appropriate in light of all the segments ◦ Decide if all the segments fit with an existing theme ◦ Rename/combine/separate themes if necessary ◦ Create new themes if necessary

Interpretation ◦ Make sense of the patterns in the data ◦ Step back from the summarizing the data and find links to larger meaning  Based on insights/intuition/hunches  Based on an existing construct, idea, theory, practice  Based on similarity/divergence from previous research findings

Verifying accuracy in qualitative data analysis Prolonged engagement/persistent observation/time on site ◦ Building trust with participants, learning the culture, checking misinformation and distortions, time on site Triangulation/Diversity of method ◦ Use multiple sources, methods, investigators ◦ Elicit multiple perspectives ◦ Identify corroborating evidence from multiple sources

Clarifying researcher bias ◦ State past experiences, biases, prejudices & orientations that may have shaped the inquiry & interpretation Peer review/debriefing ◦ An external check of the research process ◦ “The devil’s advocate”

Member checks ◦ Solicit the informants’ views about the credibility of the findings External audits ◦ Someone with no connection to the study ◦ Allow an external auditor to examine the process and product of analysis ◦ Determines whether the findings, interpretations, conclusion are supported by the data

Rich, thick description ◦ Describe in detail and participants and the setting ◦ Allows readers to make decisions about transferability Negative case analysis ◦ Refines working hypotheses as data analysis/interpretation unfolds ◦ Revise hypotheses until all cases fit ◦ Account for outliers and exceptions

Deciding which techniques to use ◦ Use at least two (Creswell) ◦ Easiest, cost-efficient, most popular procedures  Triangulation  Rich, thick description  Member check

Discussion Procedures you have used for verifying evaluation ◦ Successful ones? ◦ Challenges or concerns? Current qualitative projects? Suggestions for data sources & perspectives?