Presentation on theme: "Lynne H Johnston Sheffield Hallam University ESRC Methods Festival: Can Software Enhance the Quality of Qualitative Research? 18th July 2006 Using QSR."— Presentation transcript:
Lynne H Johnston Sheffield Hallam University ESRC Methods Festival: Can Software Enhance the Quality of Qualitative Research? 18th July 2006 Using QSR NVivo within Doctoral Research: Reflections on Processes Duncan Branley Goldsmiths College, University of London
2 Presentation Outline Why might you want to use software in your research? How can software improve the quality of qualitative research? –Reflecting on the use of software within doctoral research from the perspective of students, supervisors, examiners, and software trainers. –The impact of the ESRC Training Guidelines upon working practices. What are the common challenges associated with the use of software? What are the possible solutions? Some reflective questions to finish.Some reflective questions to finish.
3 Why might you want to use software in your research? Learning to use software appropriately is mandated by the ESRC via its Training Guidelines (2005). Not a pointless hoop – rather it can help you: –Organise your research project at a simple logistical level. –Explore your data far more quickly than a manual method could, enabling a potentially more nuanced analysis. These broad factors can influence the quality of the experience of doing the research and of the research output – your thesis.
5 Who is writing about software and method? Software developers (e.g. Richards 1998, 1999a, 1999b, 2000, 2002, Bazeley & Richards 2000, Morse and Richards 2002) Software trainers and consultants (e.g. Bazeley 2002, 2003a, di Gregorio 2000, 2003a, 2003b, Jackson 2003; Johnston, in press) Reflective reports about the impact of computing on analysis processes is still largely confined to specialist conferences (e.g. QSR Strategies Conference Series (http://www.qual-strategies.org/) –In the UK, funding for attendance at conferences is usually on the condition that research students are presenting their work. Thus, many UK based research students are not funded to attend such specialist conferences until it is too late in the doctoral process. Textbooks – compartmentalise into separate chapters – but with cross- references. Procedural rather than methodologically-reflective integration?
6 What Do the ESRC Training Guidelines Require? 2001/5, ESRC funded students should be trained in QDAS – not that they have to use them in their research. –Psychology: use of relevant software packages for qualitative data analysis –Anthropology: Students should understand the epistemological implications of their choice of methods… use of appropriate software for handling qualitative data –Sociology: skills in the use of at least one quantitative and one qualitative software package.
7 Are the ESRC Training Guidelines Working? Training models for QDA software – integration or separation? –Bought in by graduate schools for MRes and PhD students. Requests (and time) to integrate technical and methodological training is rare –Trained by technical staff with some academic and methodological awareness –Integrated as a core part of the training programme with assessed work required. Impact on roundedness of research training, rather than necessarily on the research product (thesis). Timing: –Is it taught early enough in the research career? –Is it supported throughout?
8 Supervisory Support and Skills? ESRC has no requirement (or support?) for PhD supervisors/ examiners to be trained in software use: –Mid career and late career qualitative researchers are not being supported to update their skills in QDAS (many have less knowledge about QDAS than their students). –They could opt for a training bursary for a specific course, but that is voluntarist. Students are often left isolated with no direct supervisory (or departmental) support to use software appropriately. –A two day training workshop is not enough. What level of support would be productive – and sustainable?
9 Common challenges associated with the use of software within doctoral research Summary: Technical and methodological learning curves Students start using software too late in the PhD process. They become caught up in technical learning of software functionality Research methods training continues to (falsely) separate the software (tool) from analysis processes (methodology) The methods literature has failed to integrate software and method Supervisors are not trained how to supervise students who are using software
10 Quality Challenge: Gaining Analytical Distance Avoiding the Coding Trap Too much coding, not enough reflecting: description not analysis. Why? Increased the popularity of qualitative research to those from traditionally positivistic backgrounds and thus lack qualitative methods training? The transparency that comes with QDA software may have merely highlighted a problem that has always existed (i.e. historically in QR processes were invisible/alchemical) The free tutorials, which are distributed with the software, have influenced the way in which people used the software because they use them to learn QDA! – In built tools for gaining analytic distance (e.g. project reports, coding reports, coding stripes, memoing, linking, modelling) have not been written about extensively or practically within the methods literature. Exception is Handling Qualitative Data (Richards, 2005).
11 QDA software programmes have increased the popularity of qualitative research to those from traditionally positivistic backgrounds Not a problem if people are explicit about the approach used (e.g. Pattern analysis not GT) Some researchers engaging in mixed methods research are failing to address simple conceptual issues such as what does frequency mean in QR? Do the search tools in NVivo help researchers to focus too much attention of what is said rather than what is not said? Or is this simply a reflection of a positivistic bias in the user (and some of the free tutorials.....)? What is the driver for the recent attention in the UK to assess quality in qualitative research. What are the implications in terms of rigour and transparency in QDA?
12 Has the transparency that comes with QDAS merely highlighted a problem that has always existed? Unprecedented levels of transparency are now available in doctoral research. No guidelines exist regarding the balance between product and process. What exactly should examiners be assessing? For instance, the following questions remain largely ignored within the literature: 1.How do students demonstrate the processes they have gone through in the final written document? 2.Should they be asked to include their NVivo project as a CD at the back of their thesis, and what counts as data? 3.What will the examiner expect them to discuss regarding the impact of QDA software on the way in which they conducted the research?
13 The impact of demonstration tutorials on the coding trap For some new users the demonstration tutorials may be their first real practical exposure to qualitative research. Each tutorial comes with its own implicit epistemological position and methodological approach. Most users get to a certain point in a tutorial and then stop because: –they feel they know enough, they are bored, they want to working on their own data For some this will involve stopping after they have learned how to code and retrieve. Thus many users fail to fully appreciate the purpose and importance of the 'search tool' Users need to understand the search tool BEFORE they transcribe and code their textual material
14 What do you need to know? New software users are unlikely to know what they need to know to get the most from the software: As teachers, we must remind ourselves that the vast majority of Microsoft Word (2002) users successfully use the software, yet remain unfamiliar with many of the features. This same insight should be applied to NVivo users. (Kaczynski 2003 p. 115). But…..the work done with NVivo is more complex than with word
15 Potential solutions? Get students to use QDA software for the management of their project? –Increases familiarity with the tool and also can inculcate reflexivity from early on. –Avoids the analysis stage fragmentation. Expand supervisory teams? –include someone with experience of using QDA software in research or at least someone with technical and methodological awareness. Develop some reflective literature and conference presentations? –From researchers rather than technical enthusiasts
16 Some Summary Questions… To what extent have QDAS packages changed the way we do QR? Should technical and methodological aspects of the software be integrated in training for doctoral students? What are the implications for supervisors and examiners and what role should the ESRC play in this respect? If demonstrations tutorials are used as an aid to teaching and learning, what are the potential pitfalls ?
17 References Bazeley, P. (2003) Computerized data analysis for mixed methods research. In A. Tashakkori and C. Teddlie (eds) Handbook of mixed methods in social and behavioral research (London: Sage), pp Bazeley, P. (2002) The evolution of a project involving an integrated analysis of structured qualitative and quantitative data: from N3 to NVivo. International Journal of Social Research Methodology, 5(3), Bazeley, P. and Richards, L. (2000) The NVivo Qualitative Project Book (London: Sage). di Gregorio, S. (2000) Using NVivo for your literature review. Strategies in Qualitative Research: Issues and results from analysis using QSR NVivo and NUD*IST, 29-30th September, London, UK. di Gregorio, S. (2003a) Analysis as cycling: Shifting between coding and memoing in using qualitative analysis software. Strategies in Qualitative Research: Methodological Issues and Practices Using QSR NVivo and NUD*IST, London, England 8-9th May. di Gregorio, S. (2003b) Teaching grounded theory with QSR NVivo. Qualitative Research Journal, 3 (Special Issue), (http://www.latrobe.edu.au/aqr/journal/special_AQR2003.pdf) Economic and Social Research Council (2001; 2005) Post graduate training guidelines, third /fourth editions (Swindon, UK: Economic and Social Research Council). Jackson, K. (2003) Blending technology and methodology: a shift towards creative instruction of qualitative methods with NVivo. Qualitative Research Journal, 3 (Special Issue), , (http://www.latrobe.edu.au/aqr/journal/special_AQR2003.pdf).
18 Johnston, L. H. (in press). Technical and methodological learning curves: Reflections on the use of QSR NVivo in doctoral research. International Journal of Social Research Methodology. Kaczynski, D. (2003) Curriculum development strategies using qualitative data analysis strategies. Qualitative Research Journal, 3 (Special Issue), , (http://www.latrobe.edu.au/aqr/journal/special_AQR2003.pdf). Morse, J.M. and Richards, L. (2002) Readme First: for a users guide to qualitative methods (London: Sage). Richards, L (1998) Closeness to data: the changing goals of qualitative data handling. Qualitative Health Research, 8, Richards, L (1999a) Data alive! The thinking behind NVivo, Qualitative health Research, 9 (3), Richards, L (1999b) Using NVivo in Qualitative Research (London: Sage). Richards, L. (2000) Pattern analysis and why it isnt grounded theory. Strategies in Qualitative Research: Issues and results from analysis using QSR NVivo and NUD*IST, 29-30th September, London, UK. Richards, L. (2002) Qualitative computing: a methods revolution? International Journal of Social Research Methodology, 5 (3) Richards, L. (2005). Handling Qualitative Data: A Practical Guide. (London: Sage). References