1. Ian Abrahams and Rachael Sharpe, University of York Michael Reiss, IoE University of London The effectiveness of the ‘Getting Practical’ Continuing Professional Development (CPD) programme NSLC: York 2012 1

2. A problem A problem with practical work, at least in terms of using it to develop conceptual understanding, is its name. ‘Practical work’ suggests – erroneously – (and not only to students) that it’s essentially about doing things rather than thinking about things. 2

3. ‘Hands-on’ vs ‘minds-on’ For students and teachers alike practical work is often seen predominantly as a ‘hands-on’ rather than a ‘hands-on’ and ‘minds-on’ activity. Whilst such an approach can be effective in enabling students to produce, and see, phenomena – often by adhering to recipe style tasks – it’s less effective in getting students to think about and understand their observations using scientific ideas and terminology. 3

4. Response -Getting Practical To improve the effectiveness of practical work the government funded (£900k) the Getting practical: Improving practical work in science project as a national CPD programme aimed and both primary and secondary teachers of science 4

5. Training schedule Training options: 1 x 6 hours – one day 2 x 3 hours – two half days 3 x 2 hours – three twilight sessions Groups comprised either primary, secondary or a mixture of both 5

6. Sample Evaluation: Sample 20 secondary and 10 primary from across England Multi-site, condensed, case study approach Pre and post-training visits 18 months 6

7. School type by location Type of schoolPrimarySecondary Rural38 Urban712 7

8. Guskey’s five levels of CPD Guskey’s (2002) five levels of CPD (Continuing Professional Development): 1. Participants’ reflection 2. Participants’ learning 3. Organisational change 4. Participants’ use of new learning 5. Impact on students 8

9. Pre-training: Primary Hands-on and minds-on ‘Carpet time’ – whole-class time devoted to developing the use/understanding of scientific words or the scaffolding of ideas Non-subject specialists appeared better able to empathise with the difficulties faced by their students in science lessons In many cases Levels 1:i and 2:i merged 9

10. Pre-training: Secondary Pre-training observations support previous findings (Abrahams & Millar, 2008) ‘Hands-on’ and ‘minds-off’ Focus on the production of phenomena Heavy recipe style orientation Little whole-class time devoted to developing the use/understanding of scientific words or the scaffolding of ideas 10

11. Post-training secondary observation Generally speaking no change despite many positive reviews of IPWiS. Notable exception was a school in which IPWiS was pushed by a strong charismatic Head of Science with full support (time & funding) from the Senior Management Team (SMT). 11

12. Post-training observation No discernable difference (this is not a criticism as many of the primary teacher observed were already doing a lot of what IPWiS set out to achieve). 12

13. Intended outcomes in the domain of observable objects (Domain o) in the domain of ideas (Domain i) at level 1 (what the students do) Students operate equipment in a way that generates the phenomenon that the teacher intended. Students talk about the task and phenomenon using scientific ideas and terminology that the teacher intended. at level 2 (what the students learn) Students state what they have learnt about setting up and using equipment and what they observed. Students use intended ideas and terminology to link their observations with the correct scientific theory. 2x2 Effectiveness Matrix 13

14. Effectiveness of CPD Employed a Cascade model Train the trainers of the trainers Too many trainers distort the message (the issue of Chinese whispers) Short duration of training nominally 6 hours Advantage of Cascade model being its relatively low cost 14

15. Guskey’s levels 1 & 2  Participants’ reflection  Participants’ learning Levels 1 and 2 were achieved in all cases in so far as teachers were able to reflect on their CPD and had a clear idea of, amongst other things, the need for a more equitable balance between ‘hands-on’ and ‘minds-on’ in their practical lessons. 15

16. Guskey’s level 3  Organisational change Impact was found to be dependent on who undertook the training, e.g. whether they were a head of department or an Newly Qualified Teacher (NQT), and the extent of SMT support. 16

17. In only one case was there observed to be change at level 3  Involved a very experience Head of Science  Full support of the school’s Senior Management Team (both financially and in terms of time)  Familiar with research on practical work from articles they had read in SSR  Chose to undertake the CPD (rather than being sent) 17

18. Guskey’s level 4  Participants’ use of new learning Whilst teachers appeared to be able to reflect on the Getting Practical CPD and were able to discuss the message of message there was no evidence of any significant change in practice 18

19. Average percentage (%) of whole- class lesson time spent by teacher on discussing and/or demonstrating Average percentage (%) of lesson time spent by students What to do with objects or materials Ideas and models to be used Manipulating objects and materials Primary: pre-CPD132529 Primary: Post-CPD132223 Secondary: pre- CPD 171142 Secondary: post- CPD 191443 19

20. Analysis of Secondary results Using a paired t-test for related data showed that: What to do with objects or materials: no statistically significant (p = 0.16) change Ideas and models to be used: no statistically significant (p = 0.31) change Manipulating objects and materials: no statistically significant (p = 0.87) change 20

21. Analysis of Primary results Using a paired t-test for related data showed that: What to do with objects or materials: no statistically significant (p = 0.63) change Ideas and models to be used: no statistically significant (p = 0.38) change Manipulating objects and materials: no statistically significant (p = 0.46) change 21

22. Guskey’s level 5  Impact on students Given the nature of the study it was not possible to evaluate any impact of the CPD on the students 22

23. Summary of findings This study found that the Getting Practical CPD was, generally speaking, effective in terms of Guskey’s first two levels. It was only effective at level 3 in one case where conditions might be seen as being optimal in terms of the teacher’s enthusiasm and the support of the schools Senior management Team (SMT) 23

24. Although the CPD was designed to encourage teachers to think about how and why they were using practical work comments from the teachers indicated that more specific examples of effective practical work were needed rather than generic guidance 24

25. Whilst there was no statistically significant difference between pre and post-CPD time allocations when considered across both primary and secondary groups there was one clear case (the same teacher mentioned above) whose style of teaching – post CPD - did reflect a much more equitable balance between ‘hands-on’ and ‘minds-on’. 25

26. Implications Although a relatively short six hour training programme was relatively effective in raising teachers’ awareness of a ‘message’ it would appear that. For many teachers, lasting change requires sustained training. The effectiveness of CPD, in terms of Guskey’s levels 3 and 4, might be enhanced if the training is undertaken by an interested senior member of a department who has the active support of the school’s SMT. 26

27. References Abrahams, I., & Millar, R. (2008). Does practical work really work? A study of the effectiveness of practical work as a teaching and learning method in school science. International Journal of Science Education, 30(14), 1945–1969. Guskey, T.R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45-51. 27