1. Introducing Framework © : increasing quality and transparency in QDA Kandy Woodfield Deputy Director Qualitative Research Unit ESRC Methods Festival: Can Software Enhance the Quality of Qualitative Research? 18th July 2006

2. Introducing Framework © July 18th 2006 - ESRC Methods Festival Session coverage The Framework © approach to QDA key features of the approach developments over time Our new software package being launched in 2007 how different to existing CAQDAS packages? increasing transparency? enhancing quality?

3. Introducing Framework © July 18th 2006 - ESRC Methods Festival Framework ® approach to QDA Framework © is an approach to qualitative data management developed at the QRU, NatCen in mid 1980s it is widely used by applied policy researchers uses an analytical framework consisting of a series of thematic matrices (grids) has applications for all types of QR which involve interpretative analysis –exploration of the world of research participants by identifying and describing substantive themes –searching for patterns and explanations

4. Introducing Framework © July 18th 2006 - ESRC Methods Festival Key features of Framework © The analytical framework is developed inductively to fit the data it consists of a series of thematically structured matrices it can be used with all forms of textual qualitative data Data is synthesised into a series of matrices, the analytical framework synthesised data is displayed in a grid which facilitates cross-case and within- case analysis synthesis retains links back to the original data Provides a systematic and consistent approach to data handling Facilitates creative thinking: the act of creating a synthesis activates the analytical brain

5. Introducing Framework © July 18th 2006 - ESRC Methods Festival Example of Framework matrix Data is summarised into cells Matrix display facilitates between and within case analysis of case data

6. Introducing Framework © July 18th 2006 - ESRC Methods Festival Developments over time Originally paper based, hand written summaries difficult to archive hard to team work no flexibility in layout or ordering of cases Moved to spreadsheet host in 1990s easy to set up and edit input facilitates team work and audit checking provides easily archivable dataset limited functionality for sorting and ordering BUT proved to be a clunky host (especially in relation to outputs) Next step - to search for a new host for Framework

7. Introducing Framework © July 18th 2006 - ESRC Methods Festival Framework © - a new tool for managing, ordering & interrogating qualitative data Key features Windows based single user or networked package import text (V.1) documents into Framework © to create project each project comprises series of cases –cases may be individuals or a series of linked data units for longitudinal & other approaches using multiple data sources data managed into an analytical framework drag and drop facility –allows analytical frameworks to be modified –data re-assigned automatically

8. Introducing Framework © July 18th 2006 - ESRC Methods Festival The new Framework © software Data units can be stored in variety of formats as imported text with case labels (socio-demographic/classificatory) which can be applied at various points during the process (facilitating pattern searching and sub-group analysis) as indexed text (coded) by theme/sub theme – free indexing: iterative process of deriving new analytical framework (good for grounded theory) – dynamic indexing: application of index generated manually – combination of free and dynamic as data summarised into themes/sub themes/lower levels

9. Introducing Framework © July 18th 2006 - ESRC Methods Festival Case data is imported into Framework

10. Introducing Framework © July 18th 2006 - ESRC Methods Festival Analyst applies data unit labels to allow for later sorting and identification of group based patterns and themes

11. Introducing Framework © July 18th 2006 - ESRC Methods Festival You design your analytical framework

12. Introducing Framework © July 18th 2006 - ESRC Methods Festival Your framework can include different levels of themes and sub themes

13. Introducing Framework © July 18th 2006 - ESRC Methods Festival Analyst indexes the data unit

14. Introducing Framework © July 18th 2006 - ESRC Methods Festival Analyst summarises the data unit into the analytical framework

15. Introducing Framework © July 18th 2006 - ESRC Methods Festival The new Framework © software Analysis facilitated by the process of creating analytical queries asking the software to produce indexed text extracts, matrix extracts or notebook extracts by theme or sub theme –can produce universal output (all cases, all themes) or filter by themes and participants/sub groups thematic pattern searching –allows you to identify where themes have been applied to cases and produces a graphical display to allow you to easily detect where patterns lie in the case/dataset/sub group simple text searching –for word occurrence/word associations and positional location within the dataset customised outputs to screen/paper, free text or matrices

16. Introducing Framework © July 18th 2006 - ESRC Methods Festival You can add notes as you go along and retrieve, search or archive these later

17. Introducing Framework © July 18th 2006 - ESRC Methods Festival When you are ready to explore the dataset you can create an analytical query

18. Introducing Framework © July 18th 2006 - ESRC Methods Festival You can choose what type of data & data units you want Framework to include

19. Introducing Framework © July 18th 2006 - ESRC Methods Festival Then you specify which themes or sub themes you are interested in

20. Introducing Framework © July 18th 2006 - ESRC Methods Festival Framework stores all queries and you can return to these at any time

21. Introducing Framework © July 18th 2006 - ESRC Methods Festival You leave an audit trail

22. Introducing Framework © July 18th 2006 - ESRC Methods Festival Framework will also look for thematic patterns across the dataset

23. Introducing Framework © July 18th 2006 - ESRC Methods Festival How does Framework © differ from other CAQDAS packages? Framework © will: enable data synthesis make it less laborious free up time for interpretation allow the analyst to customise outputs including matrices of summarised data a new era of code and retrieve?

24. Introducing Framework © July 18th 2006 - ESRC Methods Festival How does Framework © differ from other CAQDAS packages? The networked version allows multiple team members to work simultaneously on a shared dataset –collectively analyse a dataset –share thoughts/analytical queries in real time The dynamic audit trail facility provides systematic documentation of changes and developments –when and by who Comprehensive project administration system complies with new expectations relating to data protection

25. Introducing Framework © July 18th 2006 - ESRC Methods Festival Adding quality? Spencer et als (2003) Quality Framework for the Cabinet Office identified criteria for assessing quality in qualitative studies. In relation to analysis: Evidence of the use of data management methods that preserve context and are suited to the analytical task Portrayal of detail, depth and complexity Detection of implicit and explicit explanations & underlying factors Identification and discussion of patterns of association Portrayal of links between the data, interpretation and conclusions -Evidence of analytical building blocks -Discussion of how explanations/interpretations/conclusions were derived Framework will support all of these

26. Introducing Framework © July 18th 2006 - ESRC Methods Festival Adding quality to data management Quality enhanced by: In-built system checks enforce consistent, systematic treatment of data. For example, direct linking of data to original script/document Closeness to data original data always on display original data directly linked to summaries and matrices Dynamic audit trail visible account of changes to the analytical pathway including date/ownership of change and rationale for change, can log: - development of analytical framework - application of that framework to data units - summarisation of data into conceptual categories

27. Introducing Framework © July 18th 2006 - ESRC Methods Festival Adding quality to the analytical process Framework has a high degree of operational flexibility, this facilitates: analytical pluralism allowing users to customise the approach they take depending on their needs the development of frameworks/indexes and analytical queries can be driven by the data rather than shaped by the package the ability to work with data in multiple formats facilitates creative thinking the documentation of analytical pathways in notebook encourages greater reflexivity the ability to tailoring analytical queries & customise outputs allows users to be innovative in their approach to the dataset

28. Introducing Framework © July 18th 2006 - ESRC Methods Festival Adding quality to the analytical process The package enhances the analytical process by allowing users to: ask analytical queries of the whole dataset or subgroups within it - facilitating investigation of patterns and associations produce matrix outputs and/or text outputs from all or part of the dataset - allowing the user to select an output format which suits them and the data - allowing the user to work with data at different levels apply typologies and classifications at any point - enhances the analytical power providing another means of exploring patterns and explanations perform simple and complex text searches depending on the analytical task identify positional relationships between themes/groups/cases search and explore the notebook for their own and others conceptual thinking

29. Introducing Framework © July 18th 2006 - ESRC Methods Festival Increasing transparency Transparency in the analytical process is enhanced by the: dynamic audit trail function which provides: -a systematic, visible and chronological account of changes to the analytical pathway - including ownership of changes notes function which provides a means of sharing and documenting analytical developments, allowing the user to record and display: - the building blocks of emergent findings - the development of classifications or interpretative theories and how cases/data have been assigned to these - development of concepts, theories and interpretations - how and why particular interpretations have been chosen

30. Introducing Framework © July 18th 2006 - ESRC Methods Festival The new Framework © software: launch schedule Currently in final stages of development and in-house testing Autumn 2006 beta testing Summer 2007 launch of full single user and networked version coupled with programme of training sessions For further information William OConnor : w.oconnor@natcen.ac.uk Kandy Woodfield : k.woodfield@natcen.ac.uk Or see: www.natcen.ac.uk