Evaluating the national Action Research for Physics programme - Follow up of impact on post-16 physics uptake Willeke Rietdijk Caro Garrett Marcus Grace Science Education Research Conference, York3 July 2012

The Action Research for Physics Programme (ARPP) Research commissioned by the National Network of Science Learning Centres and the Department for Education (DfE). Organised and managed by the nine Regional Science Learning Centres between September 2009 and February The ARP programme followed on from the findings and recommendations of the Girls in the Physics Classroom (Hollins et al., 2006) and Girls into Physics (Daly et al., 2009) projects.

The course A model of professional development incorporating action research, with, as its aims: –to try out new approaches to teaching physics... –...which lead to an increase in young peoples engagement with the subject... –...and pursuit of physics post professional development days Main interventions in –Careers –Teaching and Learning

Main aims of the evaluation: i) to examine the effectiveness of the programme in changing students attitudes to physics, and their aspirations for further studies ii) to document and categorise the nature of effective practice across the action research case studies.

Reflections at the end of main study (2011) Significant increase in participating students stated likelihood of taking physics post-16, but monitoring of this proved problematic (and too early at that stage) Impact of the CPD programme on teachers practice came through very strongly – more confidence, more enthusiasm, improved teaching techniques Teachers felt students were more interested, motivated and engaged; contradictory results from student questionnaires; focus group findings were mixed

Comparisons between Final student Questionnaire and the Control group (N=656) Questionnaire item Intervention Group Mean (SD) Control Group Mean (SD) T-value How interested in physics are you?2.67 (.812)2.52 (.868)-2.27* How difficult do you find physics?2.48 (.664)2.29 (.636)-3.85** How difficult do you find the words/terms used in physics lessons? 2.71 (.651)2.52 (.690)-3.63** Has anyone talked to you about careers in physics? 1.69 (.465)1.44 (.504)6.67*** How likely are you to study physics after your GCSEs? 2.67 (1.10)2.45 (1.31)-2.40 *** How much time is there in physics lessons to think things through properly? 1.68 (5.15)1.60 (.560)-1.87*** * p <.05; **p <.01; ***P<.001

Follow-up of the ARPP evaluation Part 1: development of 20 case studies. Part 2: research questions: 1.What were students reasons for choosing physics post-16? 2.What aspects of the classes and the teaching had contributed to this decision? (Focus groups) 3.What was the 2011 physics AS uptake compared to 2009 and 2010? 4.How did students feel about Physics AS so far? (particularly related to expectations and GCSE teaching experiences)

Research Methods Part 1, case studies booklet: Interviews with the teachers of the top 20 action research projects, to explore the details and successes of their intervention – part of the report and will return to briefly later. Part 2: Quantitative: Student questionnaire for AS Physics students (whole of AS cohort including previous Year 11 ARPP participants) –N=147, from 6 colleges Qualitative: Revisiting a sample of Year 11 ARPP participants now in Year 12 –4 student focus group meetings across 3 SLC regions; 1 phone interview All teachers participating in ARPP with a Year 11 group were asked about their physics post-16 uptake numbers in 2011 as compared to 2009 and 2010.

ARPP teacher interventions used in the 6 sample schools Introducing electrical concepts before the maths. Questioning strategies: 1.no-hands up and mini whiteboard strategies; 2.using Blooms taxonomy, letting students come up with and answer each others questions. Increasing the relevance of physics by bringing in anecdotes about real-life physics. Increasing the amount of small group discussions in lessons. A variety of techniques to increase engagement, in and outside of lessons.

FINDINGS Student questionnaire sample (N=147) All year 12, AS physics students 35% girls and 64% boys 31% had been part of an ARPP intervention during Year 10/11 80% Triple science students of which 80% A or A* grade 20% Additional science students: 65 % A or A* grade November 2011: 11% still deciding about continuing AS Physics choice in Year 12! Not all students remembered correctly whether they had been part of an intervention Focus groups: None of the students in the focus groups could remember the teachers specific intervention

1. Reasons for choosing Physics AS Why did you choose Physics AS? % whole group % boys% girls Gender difference significant to I am interested in the subject as a whole p=.031* I am interested in particular parts of the subject n/a I think it is a good challenge n/a I am good at it p=.06 I need it for further university studies n/a I particularly enjoyed physics last year p=.043* I have always enjoyed my physics lessons n/a It looks good on my CV p=.004** I need it for future jobs n/a I had an inspiring teacher n/a I find it easy p=.026* I need it for a work experience placement n/a It goes well with mathematics (reason given in open space with other reasons) 4.1 *=p<.05; **=p<.01; ***=p<.001

Reasons for choosing physics Most important reasons for choosing physics were: an inspirational teacher, interest/enjoyment of the subject, and needing it for future studies/careers GCSE topics that motivated students to pursue physics at AS level: astrophysics and nuclear physics Girls more often indicated they had chosen physics because they were interested in parts of the subject (as opposed to the whole subject), and because they thought the subject provides a good challenge (but differences here not significant)

2. Focus group findings Reasons for choosing physics the same as in questionnaire responses Difference teacher GCSE/AS mentioned quite often – not as enthused by current teacher compared to ARPP intervention teacher, less engaged, more difficult explanations, more book work Teachers pivotal in decision according to quite a few students: passion and enthusiasm rubbing off; instilling confidence, making subject more enjoyable Simple explanations, varied ways of teaching/presenting materials, looking for ways to help digest big chunks of information were mentioned as important teaching strategies to help students understand physics and engage them.

3. School data on post-16 uptake after ARPP School Number opting for Physics School A No data givennone 6 boys out of 15 (all boys school). School B 49/266 = 18.4% 47/273 = 17.2%55/272 = 20.2% School C usually 120 students meet the entry requirement for AS physics 9/90 students in lower 6 th = 10% 19 /100 students in lower 6 th = 19% 34/100 in lower 6 th = 34% School DNo data Of the 56 pupils in last year's cohort, 11 are now doing AS level physics School E2 boys5 boys9 boys and one girl School F School G18 out of 207 AS level students = 8% 36 out of 262 AS level students = 14% 30 out of 221 AS level students = 14%

4. Feelings about AS Physics so far (open question) A good challenge (11 respondents) Being able to go into GCSE topics in more detail and further complexity (11) Topics enjoyed: Electricity/circuits (16), particle physics (12), practicals (11), quantum physics (7), the mathematics involved (5), SUVAT (4), and mechanics (4) Very interesting - news ideas coming out, understand more about how things work. Working in smaller classes where everyone wants to participate, unlike GCSE You get to do the in-depth side of GCSE. You learn more complex things than at GCSE. Less interesting about AS physics so far: circuits and electricity (31), calculations and equations (14); materials (5); motion (5); fluids (5) [I enjoy] Not a lot - it is so difficult! The concepts of motion are interesting, just too hard! [I enjoy] little at the moment. But it will get better, I have read the AS specifications/year outline. Old content. Repetitive, feels unimportant Same as GCSE just more work. Focus groups: corresponding findings.

Impact of ARPP teacher intervention Statistical comparisons made: –Answers from students who were part of ARPP compared to those not part of ARPP When they chose physics Reasons for choosing physics (Topics? Teacher? Intervention-related (indirect)? What contributed most to their decision to choose physics AS What they liked about physics GCSE (trying to gauge if intervention-related) No statistical differences in any questionnaire item between ARPP and non- ARPP students – so statistically no clear impact of the programme on reasons for choosing physics post-16 No specific impact of the type of interventions could be found in terms of statistical differences in motivations to choose physics, or answers to the open questions. HOWEVER: students did not seem to remember very well whether they had been part of an intervention, and did not seem to remember at all what this intervention was if they did

Conclusions Some schools report a distinct increase in Physics AS numbers in the year after the ARP programme Difficult to find impact of ARPP intervention on students reasons for choosing physics from students responses; BUT general feedback from focus groups shows that teachers are very important in students choices for AS physics (passion, enthusiasm, engaging teaching techniques, instilling confidence)

Indirect impact? ARPP teacher participation greater teacher confidence & enthusiasm about physics / better teaching strategies students more enthused/motivated/confident? more students opt for Physics AS?

Limitations Small sample (147 students from only 6 schools/ intervention studies) –Findings difficult to generalise –Not easy to find statistical impact from a small sample Self-selection of teachers

Beyond the programme and evaluation Bearing in mind the effect of teacher effects on student decisions, as in the conclusions: Impact of the programme on teachers CPD Impact of the teachers CPD on their colleagues Impact on the teachers practice and sustainability Potential for effective teacher development via collaboration between Teaching Schools and Science Learning Centres……… how long is a piece of string?

Beyond the programme and evaluation Bearing in mind the effect of teachers effects on student decisions, as in the conclusions: Impact of the programme on teachers CPD Impact of the teachers CPD on their colleagues Impact on the teachers practice and sustainability

Beyond the programme and evaluation Bearing in mind the effect of teachers effects on student decisions, as in the conclusions: Impact of the programme on teachers CPD Impact of the teachers CPD on their colleagues Impact on the teachers practice and sustainability

Finally… A case studies booklet is currently being compiled for publication – 20 most successful action research studies Final (main) ARPP report is available from the NSLC website: reports reports

References Daly, A., Grant, L. and Bultitude, K. (2009) Girls into Physics: Action Research: Evaluation Report, London: Department of Children, Schools and Families, UK Hollins, M., Murphy, P., Ponchaud, B. and Whitelegg, E. (2006) Girls in the Physics Classroom: A Teachers Guide for Action. London, Institute of Physics Murphy, P. and Whitelegg, E. (2006). Institute of Physics Report: Girls in the Physics Classroom: A Review of the Research on the Participation of Girls in Physics. London: Institute of Physics All accessible from the IoP website