1. Incorporating NVivo into a Large Qualitative Project
Using NVivo software as part of a multi team analysis of in depth interview data in a mixed methods evaluation
Karla Eisen, MSW, MPH
Izabella Zandberg, PhD
Susan Berkowitz, PhD

2. Overview Project overview
Approach and structure of qualitative analysis
Incorporating NVivo
Different approaches of analytic teams
Lessons learned
2/18/2019

3. Project Overview
IECRN – a large scale mixed method study of clinical research networks (CRNs) in the United States and internationally.
Multiple Components
Descriptive Survey Component- detailed description of characteristics of a purposive sample of clinical research networks (CRNs); modules for different practice domains.
Original sample of 133/response rates varied by module.
2/18/2019

4. Project Overview : Descriptive Study
Qualitative Component: in-depth, semi-structured interviews to discern barriers and facilitators of CRNs’ efficiency and effectiveness in 6 domains of network functioning.
Management and Governance
Training and Professional Development
Recruitment and Retention
Network Operations
Information Technology
Data Management
Quantitative Component
2/18/2019

5. Qualitative Data Collection
6 surveys with qualitative component
Range of respondents: Principal Investigator, Administrative Manager, Data Manager, etc.
Interview teams grouped by domain clusters
Varying level of experience in qualitative interviewing
Quantitative data retrieval mixed with some qualitative interviews
Interviews recorded and transcribed (transcripts)
Interviews summarized (summary files).
2/18/2019

6. Approach to Qualitative Analysis
One qualitative analysis group of nine analysts
Separate analytic teams clustered by domain area
Bi weekly meetings to discuss emerging themes, cross domain analysis, etc.
Analysts from multiple sites, e-room used
Separate NVivo databases
Separate report sections by analytic teams merged into one report
NVivo training.
2/18/2019

7. NVivo Training First training Second training
Far in advance of data collection period
Incorporated day long overview on qualitative research (to prepare for qualitative interviewing and emphasize theoretical approach).
Second training
Very close to start of analysis
Used actual project data collected
Basic coding tree already developed
Ongoing support via consultation with trainer available throughout analysis period.
2/18/2019

8. Structure of Analysis
Analysis unique to domains, but similar in structure
Question Areas (unique)
Barriers and facilitators (shared)
Major concepts and categories (partially shared)
Basic coding tree developed and refined through discussion within analyst group
Further code refinement anticipated within each analytic team
Tension between “pre set” coding categories and unique content.
2/18/2019

9. Different Approaches to Qualitative Analysis with NVivo: Approach 1
Proximity to data
Two out of three members of the team conducted the interviews they were analyzing.
One team member developed  interview instruments and trained  interviewers on their use. 
Good data quality (all transcripts)
Coding structure developed (starting pre-NVivo) and revised iteratively
Regular meetings to discuss the analytic process, emerging themes, code refinement, etc.
Different levels of involvement with the software
Team members highly motivated to learn NVivo
NVivo tools used to guide analysis (Merge, Node Reports, and Matrix Intersections).
2/18/2019

10. Different Approaches to Qualitative Analysis with NVivo: Approach 2
Proximity to data
Analysts not directly involved in interviewing
Some involvement in developing interview instruments.
Data quality inconsistent
Wide range of analytic skills and experience within the team
Used basic coding structure
Different levels of involvement with the software
Used NVivo for rough data categorization (node reports)
Approached NVivo more quantitatively (focus on inter coder reliability, quality checks, etc.).
2/18/2019

11. Different Approaches to Qualitative Analysis with NVivo: Approach 3
Proximity to data
Analysts mixed involvement in interviewing
Data quality inconsistent
Analysts worked primarily alone
Used basic coding structure
Used NVivo to code and organize text, but did not take advantage of other software functions
Additional analysis done traditionally “by hand.”
2/18/2019

12. Different Approaches to Qualitative Analysis with NVivo: Approach 4
Proximity to data
Analysts not directly involved in interviewing
One team member helped revise and train on instruments. 
Data quality marginal
Wide range of analytic skills and experience within the team (Senior analyst and content expert)
Team dissolution
Limited use of coding structure
NVivo not used
Analysis conducted solely using traditional “pencil and paper” methods.
2/18/2019

13. Lessons Learned Challenge of Diverse Analytic Teams
Experience with instrument design and conducting qualitative interviews
Qualitative analysis experience/abilities
Quality of qualitative data to analyze
Prior exposure to NVivo or other qualitative analysis software packages
Motivation to learn software and use in analysis process
Staff from multiple companies, multiple divisions within companies (expectations?)
2/18/2019

14. Lessons learned (continued)
Proclivity to use NVivo related to experience and abilities of analysts, team make up and support, and comfort with software as an analytic tool
Learning curve (NVivo)
Start up time, qualitative data management issues
Time–pressured, deadline-driven non-academic environment
Full potential of the software unexplored
Unable to successfully integrate attributes from quantitative data.
2/18/2019

15. Next Time Around…………
More review of coding consistency within teams and across teams
More practice running reports (matrix intersection “cross tabs” and attributes)
More use of NVivo features (attributes, memos, etc.)
More practice moving from NVivo reports to meaningful finished text
More discussion of emerging themes across groups
More time for writing, editing, and rewriting report.
2/18/2019

16. Contact: 
Karla Eisen
WESTAT 1650 Research Blvd. Rockville, MD (301)
2/18/2019