Dr Ian Abrahams Combining randomised control trials with qualitative research approaches: The best of both worlds York

. Whilst randomised control trials (RCTs) are designed to find differences between different groups they do not adequately explain the ‘how’ and/or ‘why’ those differences occur. To do this we need to use RCTs in conjunction with other qualitative methods, e.g. observations, focus groups, interviews. 2

A combined approach In the evaluation of the Wellcome funded CPD (Continuing Professional Development) primary science specialist programme, RCTs have been used in conjunction with: case-study visits to schools that involved observations, interviews and focus groups. The qualitative component is evaluated in terms of Guskey’s (2002) five level model of impact. 3

Guskey’s five levels of CPD Guskey’s (2002) five levels of CPD: 1. Participants’ reflection 2. Participants’ learning 3. Organisational change 4. Participants’ use of new learning 5. Impact on students 4

. This is the first large-scale use of an RCT in conjunction with a qualitative research approach certainly in science education and possibly beyond. From a research perspective there is curiosity as to the extent to which the finding will be mutually supportive. 5

Guskey’s five levels of CPD In particular if the RCT finds a statistically significant difference in the subject knowledge and/or confidence of teachers in one group compared to another how is this manifesting itself in school in terms of organisational change (Guskey level 3) and the use of the new learning (Guskey level 4). 6

Guskey’s five levels of CPD The advantage of using a ‘mixed methods’ approach is that it combines the analytical strength of a numerical RCT with the explanative depth of a qualitative study. 7

. The qualitative case-study visits also provide an opportunity to investigate what teachers think not only about the role of research in education but of evidence-based practice and RCTs in particular. 8

. The main problem with organising the qualitative component of the study has been gaining access for what is essentially a whole day visit. Our current solution is to be as flexible to the needs of the school as possible whilst ensuring that all visits are completed within the time frame allowed by the project. 9

Response -Getting Practical What has emerged is that very few teachers are yet familiar with RCTs but that there is wide- spread support for evidence informed practice. 10

Response -Getting Practical References: Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching: Theory and Practice, 8(3/4), 381–