1. ANALYZING AND REPORTING QUALITATIVE RESEARCH
Chapter 17

2. Analyzing Qualitative Data
Complex and time consuming process of reducing, organizing, and synthesizing qualitative data to determine what is important

3. Three Stages of Analysis
Familiarizing and Organizing
Coding and Reducing
Interpreting and Representing

4. Familiarizing and Organizing

5. Familiarizing
Transcribe all data – transcribe words directly, include all utterances, hesitations, pauses, notes on nonverbals and volume
Immersing self in the data – become familiar with data by reading and re-reading notes and transcripts, viewing and re-viewing videotapes, and listening repeatedly to audio

6. Organizing
Create reflective log – notes or memos of researcher thoughts on key ideas
Create data log – list of all data sources
Organize files – by interview, question, people, places, or some other structure
Create back up copies or original data

7. Criteria for Selecting Technology Tools for Analysis
Ease of use/operating system compatibility
Type of data (text, images, multimedia)
Ability to read, search and review text
Memo-writing functions
Ability to develop, apply, display, revise codes
Analysis features
Ability to import/export data
Ability to support multiple researchers and merge data

8. Coding and Reducing

9. Coding
Step 1 called axial, open, preliminary, or provisional coding – read data and look for units of meaning (words, sentences, phrases, patterns, events that appear regularly and appear important)
Create names for codes from actual words of respondents (in-vivo codes) or create names that reflect underlying concept or use a priori codes from literature

10. Coding, Continued
Identify each unit (word, sentence, paragraph, etc.) with a code
Codes can represent what you expected to find, what you did not expect, or interesting or unusual information
Goal of coding is to break apart the data
Lean coding – strategy used by some starting with a fewer number of codes

11. Strategies in Developing Codes
Ask questions about data
Make comparisons
Consider different meanings of words
Flip-flop technique (look at opposite of term)
Draw on personal experience
Wave red flag (words like never, always)
Look at language (how people use terms)

12. Strategies in Developing Codes, Continued
8. Look at expressed emotion
9. Look for words that indicate time
10. Think in metaphors and similes
11. Look for negative cases
12. Consider narrative structure
13. Dismantle dichotomies
14. Examine silences
15. Attend to disruptions

13. Reducing: Categories Group units together that have the same codes
Use codes to develop tentative categories
Categories are one level of abstraction above initial codes
Some items may belong in more than one category

14. Reducing: Categories, Continued
Inductive coding – simultaneous comparison of units of meaning
Refine categories to reduce large number of categories
Categorizing is re-sorting coded data into larger units

15. Types of Categories
Organizational – may be anticipated and established prior to data collection
Substantive – primarily descriptive and not related to abstract theory
Emic – based on participant words
Etic – based on researcher interpretation
Theoretical – more abstract, may be derived from theory, generally etic

16. Reducing: Themes Link categories to create themes
Themes are level of abstraction above categories
Look for connections and relationships among categories
Most difficult process of analysis

17. Data Pyramid

18. Constant Comparative Method
Combines inductive category coding with simultaneous comparison of all units of meaning
Examine each unit of meaning to determine distinctive characteristics
Compare categories and combine with similar categories
If no similar units, create new category
Process of continuous refinement

19. Negative Case or Discrepant Data Analysis
Look for data that is negative or discrepant from the main body of data
Negative data contradict the main categories
Discrepant cases provide a different perspective on a category
Reduces tendency to rely on first impressions
Researcher may revise based on these

20. Interpreting and Representing

21. Interpreting
Reflecting on the words and actions of participants and abstracting important understandings
Inductive process of making generalizations based on connections and common aspects among categories and themes
Bringing out meaning, telling a story, developing a plausible explanation

22. Interpreting No set of rules for interpreting
Quality depends on background, perspective, knowledge, and theoretical orientation of researcher and intellectual skills brought to the task
Must be supported by data
Affected and influenced by qualitative approach used (e.g. narrative, ethnography, case, etc.)

23. Representation How the data are presented
Most frequently presented by themes, topics, or cases
Typically detailed text augmented by graphs, pictures, diagrams, figures, or frameworks
New technologies allow more visual representations
Varies by qualitative approach used

24. Class Activity
In small groups, choose three different qualitative approaches and discuss how interpreting and representing findings might vary among them

25. Reporting Qualitative Research

26. Elements of Qualitative Reports
Abstract
Introduction
Research design (including steps to ensure credibility and dependability)
Methods (site and participant selection, data collection methods, data analysis procedures)
Findings
Interpretations and implications
References
Appendices
Findin

27. Notes on Findings Findings may be reported in a variety of ways
A challenge is deciding what not to include from vast amount of data
Use of direct quotes, audio and video excerpts etc. to help reader vicariously experience
Unlike quantitative research, may incorporate interpretations and links to literature

28. Writing in Qualitative Reports
Qualitative research generally heavily narrative style
Thick rich descriptions – places readers in the setting
Tell the story the data tell
Generally little technical language
First person acceptable
May not follow conventional format
Frequently incorporate visual representations to help reader “see”

29. Various Approaches to Writing
Journalistic style
Translator style
Realist tales
Historical report
Confessional tales
Techniques found in drama and other artistic modes

30. Class Activity
Discuss how different qualitative approaches might lead to different styles of reporting

31. Technology and Qualitative Research

32. Uses of Technology in Qualitative Research
As a source of data
As a data collection tool
As a data organizing tool
As an analysis tool
As a reporting tool

33. Technology Use in Analysis
Data storage and analysis
Data searching and retrieval
Coding
Developing and testing theories
Visual representation
Writing reports

34. Technology cannot replace the human mind in interpreting data
Technology Use
Technology cannot replace the human mind in interpreting data

35. Class Activity
Discuss the advantages and disadvantages of using technology in the various stages of qualitative research

36. Rigor in Qualitative Research

37. Standards for Rigor

38. Credibility

39. Credibility Similar to internal validity in quantitative research
Addresses the issue of truth value – the truthfulness of the findings
Are the findings believable?
How confident can you be in the findings?

40. 5 Approaches to Enhancing Credibility
Structural corroboration or coherence
Consensus
Referential or interpretive adequacy
Theoretical adequacy
Control of bias

41. Structural Corroboration/Coherence
Multiple types of data related to each other support or contradict the interpretation
Data triangulation – data collected using one procedure or instrument confirm data collected using a different procedure or instrument (e.g., interview and observation)
Methods triangulation – different methods used to investigate the problem (e.g., ethnography and document analysis)

42. Consensus Agreement among competent others
Peer debriefing/Peer review – colleagues or peers given the raw data and the interpretation and consider whether interpretation reasonable given the evidence
Investigator triangulation – multiple researcher collect data independently and compare results

43. Referential/Interpretive Adequacy
Accurate portrayal of meaning
Member checks – soliciting feedback from participants about the accuracy of the findings
Low-inference descriptors – using verbatim quotes and detailed descriptions to help the reader “see” the setting and “hear” the participants

44. Theoretical Adequacy/Plausibility
Extent to which theoretical explanation fits the data
Extended fieldwork – did researcher spend enough time to ensure range of activities observed and trust of participants developed
Theory/interdisciplinary triangulation – did researcher consider other theories or disciplines
Pattern matching – did data match earlier predictions made on theory

45. Control of Bias
Steps to ensure personal beliefs, attitudes, preferences, and feelings do not interfere with interpretation
Reflexivity – use of reflection to actively seek out and identify own biases, usually though journaling
Negative case sampling – intentionally seeking examples of the opposite expected; looking for contradictory data

46. Transferability

47. Transferability
Degree to which findings can be applied or generalized to other contexts or groups
Similar to external validity in quantitative research
Not a goal of the qualitative researcher to transfer, but an obligation to provide sufficient information for readers and users of the information

48. 3 Approaches to Enhancing Transferability
Descriptive adequacy
Similarity
Limiting reactivity

49. Descriptive Adequacy
Detailed description – accurate and complete descriptions of the context and participants is provided
Comparison – comparing demographic data about participants to larger population

50. Similarity
Cross-case comparisons – investigate more than one case to determine similarities
Literature comparison – Compare to other cases found in the literature
Describing limitations – let the reader know potential for:
Selection effects – construct is unique to single group
Setting effects – results the effect of a specific context
History effects – unique historical experiences of the participants

51. Limiting Reactivity Limiting the effect of the research itself
Reflective statements – help the reader understand through describing own biases
Detailed methods descriptions – detailed descriptions of factors that might have influenced study

52. Dependability

53. Dependability
Consistency or extent to which variation can be tracked and explained
Similar to reliability in quantitative research
Also known as trustworthiness in qualitative studies

54. 4 Approaches to Enhancing Dependability
Documentation
Consistent findings
Coding agreement
Corroboration

55. Documentation Audit Trail
Documents the who, what, when, and why of study
Includes raw data, researcher’s decisions about who to interview or what to observe and why, how hypotheses were develop and refined and tested, findings of the study, etc.
Allows third party to attest to dependability of procedures and whether findings are logically derived from data

56. Consistent Findings
Show consistent findings across multiple settings or multiple researchers
Replication logic – conducting study in multiple locations or with multiple subjects
Stepwise replication – two researchers divide the data, independently analyze it and compare results

57. Coding Agreement
Intra-rater agreement (also known as code-recode) – researcher codes the data, leaves it alone for awhile, and then codes it again and compares the two sets of coded data
Inter-rater agreement (also inter-observer agreement) – ask a peer to code a transcript or code a video and compare results to researcher’s

58. Corroboration
Triangulation – use of multiple data sources or multiple methods and comparison of findings

59. Confirmability

60. Confirmability
The extent to which the research is free of bias in its procedures and interpretations
Similar to objectivity in quantitative research
Also known as neutrality
Focuses on confirmability of the data and interpretations

61. 3 Strategies for Enhancing Confirmability (all described earlier)
Documentation
Corroboration
Control of Bias

62. Evaluating Qualitative Research

63. Evaluation Elements Clear research question and conceptual framework
Relationship between study and literature shown
How and why site and participants selected explained
Data collection methods and role of researcher explained
Data analysis procedures explained
Strategies for credibility, transferability, dependability, and confirmability described

64. Evaluation Elements, Continued
Abundant raw data and descriptive data separate from interpretation
Evidence of ethical standards and control of bias
Answers research question
Significance made explicit
Generalization qualified
Reported in accessible way

65. Class Activity
Discuss how the basic criteria for evaluating qualitative research may differ based on the approach used