Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head of Information and Research Disability Rights Commission

Presentation from the perspective of research commissioner and of research provider Three key developments in UK: –evidence-based policy and practice –utilitarian view of research –effective dissemination

Implications: –‘marketised’ research relationships –increasing heterogeneity of ‘providers’ and ‘clients’ –different skills sets required –‘quality guarantee’ in doubt or not primary concern? –different ‘normative worlds’ in collision

Example of consultancies: –cross pollinators reduce ‘silos’, enhance transferability –match makers more effective partnership working –translators and processors information usable and relevant –multiple dissemination routes, formative techniques wider audience, timely

Company X: –SME research and consultancy company –Works solely with public sector clients (i.e. national, regional, local government, public bodies) –Six employees use QDAS

Prior experience: –4 had general undergrad social research training –1 did qualitative postgrad research –1 no background in qualitative research at all –None had used any QDAS before

Training (not mutually exclusive): –1 had formal external training by specialist –4 had ‘on the job’ training –3 had formal internal training by colleague - implication? –1 asked colleague –1 read a manual

Type of research QDAS used on: –All were large-scale mixed-method national policy evaluations –Mostly semi-structured interviews, one structured focus group –Volume of data - from around 30 to more than 100 documents –All individuals used QDAS on actual projects immediately after training

Perceived adequacy of training: –All felt training was adequate, irrespective of: background in qualitative research/data experience in using QDAS mode of training timing of training

Functions used: –All used QDAS for preparing and uploading documents; code; perform matrix node searches –Fewer used it to design coding structure; define codes; generate reports; create memos –2 used Merge function

Confidence and weakness: –All confident in functions with regular use –Less confident in functions with sporadic use or never used –Awareness of more ‘sophisticated functions’ that they had never used but no indications of knowledge of what these functions actually are

Project management: –All trained in specific project teams –Division of labour - data management, data analysis –‘Need to know’ and consistency

Data analysis: –‘Core’ analysis team –Structured coding design –Descriptive or topic codes –Largely descriptive analysis, lack of theorising

Discussion: –Need to engage. Pragmatic rather than idealistic response. Can’t ignore or shun as ‘wrong’ or ‘unorthodox’ –QDAS can offer some tools to help mitigate against the worst of ‘bad practise’, depending on: type of research type of team management type of outputs and hence analysis required

Discussion: –Allows things that can be systematised to be systematised –Easy checking –Not overwhelm individuals, e.g. ‘need to know everything’

What to look for: –Good guidance exist, but tend to target people with some understanding of research –What to look for and what to ask for when it’s not there. Inability to articulate causes frustrations on both sides, fuel continued misunderstanding –QDAS not the only way, but can help. Some risks (e.g. ‘wow’ factor).

What to look for: –Samples of documents –Numbers of documents, all ‘analysed’ –Codes –Use of codes –…and, dare we hope, a theoretical ‘model’?

Thank you for your attention and enjoy the rest of the conference! Dr Chih Hoong Sin