1. ATSE conferenceAugust 2011 Celebrating University /School Partnerships ATSE conference 23 August2011 ASE headquarters, Hatfield A tale of two parts…

2. Engaging students with physics: an evaluation of the national “Action Research for Physics” programme Caro Garrett (c.garrett@soton.ac.uk) Willeke Rietdijk Marcus Grace ATSE conference 23 August2011 ASE headquarters, Hatfield ARPP slides: Willeke Rietdijk

3. The Action Research for Physics Programme (ARPP) Research commissioned by the National Network of Science Learning Centres and DCSF. Organised and managed by the nine Regional Science Learning Centres between September 2009 and February 2011. The ARP programme followed on from the findings and recommendations of the Girls in the Physics Classroom (Hollins et al., 2006) and Girls into Physics (Daly et al., 2009) projects.

4. The course A model of professional development incorporating action research, with, as its aims: –to try out new approaches to teaching physics... –...which lead to an increase in young people’s engagement with the subject... –...and pursuit of physics beyond GCSE level.

5. Programme content 3 separate CPD sessions of one day, focusing on: o action research theory and explanation; o physics teaching strategies; o feedback from individual teacher’s action research findings 2 rounds of action research in between the 3 sessions o Intervention in one of the 6 strands from the Girls into Physics project: Careers, Teaching & Learning; School Culture; Progression; Classroom Management; Workforce o Intervention to be developed over 2 rounds of action research to incorporate feedback from first round in second round

6. Main aims of the evaluation i) to examine the effectiveness of the programme in changing pupils ’ attitudes to physics, and their aspirations for further studies (where possible, to follow pupils and ascertain actual numbers continuing beyond year 11 with physics studies) ii) to document and categorise the nature of effective practice across the action research case studies.

7. Teacher participant profiles 110 completed baseline questionnaire: 61 female, 49 male Subject backgrounds: 60 physics, 14 biology, 10 chemistry, 16 other Physics related qualifications: 6 PhD, 15 Masters, 49 Bachelors, 12 A level, 10 GCSE Teaching experience: 2 10 years. 64 completed final reports

8. Examples of teacher projects Interventions in the area of Teaching and Learning covered the following main areas: –Questioning techniques –Collaborative work / experimenting with single-gender/mixed-gender groups; grouping according to level (single or mixed) –Bringing in the context and applications at the beginning of a topic –Generally bringing in more real-life context into the lesson –More practical work –Working with concept maps –Using video peer assessment strategies –Use of more/new/creative/different/visual materials –Bringing in more cutting edge/wow! physics and addressing “the big questions” –Including more discussion –Reducing textbook work –Going outside the classroom more –Bringing in cross-curricular activities/lesson plans Careers and Guidance interventions often consisted of the following activities: –Getting outside speakers in; setting up physics clubs –Letting students research careers and/or do presentations about what physicists do –Posters/displays up in corridor/classrooms and regular updating of these

9. Research Methods of the Evaluation Research Quantitative: 3 Pupil Questionnaires 2 Teacher questionnaires ‘Year above’ Control group pupil questionnaire Teachers’ Senior Managers Questionnaires Qualitative: 2 rounds of Pupil Focus group meetings Teacher focus group meetings at each SLC at the end of the programme to assess impact of taking part in course 1:1 Course tutor interviews about their views on the success of the programme and any recommendations for future running of the course

10. Timeline of research activities Autumn 09 First professional development session Teacher questionnaire 1 (pre-intervention) Pupil questionnaire 1 (pre-intervention) Pupil focus groups (pre-intervention) Spring 10 Pupil questionnaire for Year group (control group) a year above the intervention groups Summer 10 - Autumn 10 Pupil questionnaire 2 (mid-intervention) Second professional development session Autumn 10- Winter 11 Third professional development session Autumn 10 Teacher questionnaire 2 (post-intervention) Teacher focus groups (post-intervention) Winter 11 Pupil questionnaire 3 (post-intervention) Questionnaires for senior managers (post-intervention) Dec 10 – Feb 11 Pupil focus groups 2 (post-intervention) Winter 11 CPD tutor interviews (post-intervention)

11. Pupil survey questions Interest in physics Experienced level of difficulty of physics (and words/terms used) Views on relevance of physics (open) Careers talks; career plans Intention of choosing physics post-16 Physics lessons aimed more at boys/girls/both? How often links made in physics classes with: –Other subjects –Everyday life –Worldwide issues How much time reported in classes for: –Discussion –Reflection (“Time to think things through properly”) Pupil Questionnaire 2+3: –More/less interested/difficult than before? –Most interested in recently? –Any other comments ab0ut your physics lessons, particularly over the last 6 months?

12. Teacher survey questions Demographics Open questions about: –Main concerns about teaching physics –What needs changing in your classes to encourage post-16 take-up of physics Teaching strategies used in physics lessons (“How often do you...” ) –11 items; repeated in final questionnaire for statistical comparison of answers pre-post course; i.e. Encouraging dialogue Differentiation for genders/abilities Links with careers, physics topics, other subjects, everyday lives of pupils, global/social challenges Teacher Questionnaire 2: –Action research intervention –Views on success and impact of course (on pupils, classes, and teacher) –Open questions about changes in practice, changes to SOW, remaining concerns Impact on colleagues/Dept and wider school level

13. Senior Managers Questionnaire questions Reasons for authorising teacher participation in ARPP Nature and impact of teachers’ action research intervention (finding out how much they are aware of this) 9 impact of course items (pupil, teacher, department, school) Usefulness of action research Usefulness of teachers’ participation in ARPP Intentions of using teachers’ findings in future staff CPD, sending other teachers on the course, recommending course to other CPD managers

14. Statistical relationships Pupils: Interest in physics Experienced difficulty level of physics Physics careers talks reported Post-16 take-up of physics intention Lessons aimed at which gender Amount of links reported with Other subjects Everyday life Worldwide issues Gender differences in: –Interest –Difficulty level experienced –Intention to take up physics post-16 –Reported level of discussion time/reflection time –Lessons aimed at which gender Correlations (between two continuous variables, i.e. How often pupils feel physics is linked with everyday life and how likely they are to study physics post-16) Chi-square (categorical data – differences between groups on variables, i.e. Differences between boys and girls in interest in physics) T-tests (continuous data – differences between groups on variables, i.e. Comparing pupil interest in physics before and after action research intervention)

15. Between Pupils and Teachers 1.Matching what teachers say they do in their physics classes and what pupils report happens in physics lessons: –Links made in classes with everyday life, other subjects, and worldwide issues; physics careers talks, amount of dialogue/discussion reported 2. Statistically comparing –Pupil interest; –Pupil experience of difficulty level of physics; –Intention of post-GCSE take-up of physics; –Gender of teacher –Teacher strategies (i.e. Differentiation, dialogue, links with everyday life, other subjects, and worldwide issues, physics careers talks, amount of dialogue/discussion reported) –Gender of teacher (Sometimes splitting the pupil file into girls and boys and then do the same comparisons) With:

16. For final Teacher and Pupil Questionnaires also: Statistical comparison of –Teacher Action Research intervention areas (6 strands) –Teacher indications of level of impact of interventions on Pupil Teacher With –Pupil interest (+ increase) –Pupil experience of difficulty level (+increase) –Pupil intention of physics post-16 take-up –Reported amount of discussion/reflection time in classes

17. Results

18. Pupil Questionnaire 1 (2050 completed; 58% female; 45% Yr 10s, 42% yr 9s) Only 14.4% are ‘very likely’ or ‘definitely’ going on to study physics after their GCSEs, girls only even lower percentages: 10.8% Only 25% of pupils claim to have had talks about careers in physics, mostly with their science teacher and/or a parent. 30% of pupils feel that there is not enough time in physics lessons for discussion 39% report that there isn’t enough time to think things through properly (girls 45%) Girls find physics significantly less interesting and significantly more difficult than boys. Girls find the words used in physics significantly more difficult than boys. There is a significant positive correlation between whether pupils say they have had careers talks and how interested they are in physics. Pupils are significantly more interested in physics when physics is linked to their daily life, worldwide issues, or to other subjects. Pupils are most interested in physics when lessons are aimed at their own gender.

19. Teacher Questionnaire 1 (110 completed; 56% female; 55% with a physics subject background; 65% with a physics-related degree) 89% said they often encourage dialogue between teacher and pupil 59% often encourage dialogue between pupil and pupil. 90% never differentiate for boys and girls Over 80% use gender-neutral examples and illustrations, use non- technical language and analogies, make links with everyday life, other subjects and global and social issues 74% make links with careers (but only 25% of students report this!) Most frequent concern mentioned by teachers is ability to engage and enthuse pupils and making physics relevant (by 28% of them) Similarly, the most frequent aspect mentioned that needs changing in their classes by physics teachers is increasing the relevance of physics (by 38% of them).

20. Links between pupil and teacher variables The more often teachers link physics with everyday life, the more interested pupils are in physics. (Not for links made with other topics, global/social issues, careers in lessons!) A significant negative relationship between how often teachers encourage dialogue between pupils and themselves and how interested pupils are in physics, and how easy they find physics! Very strong positive correlation between how often teachers indicate that they link physics with everyday life and with careers, and how often pupils feel that this is happening - pupils are clearly aware of their teachers’ efforts in this respect. Pupils find physics significantly easier when teachers indicate that they link physics with global and social issues more often Girls find physics significantly less difficult, and consider the post-16 take up of physics more, when their teacher is female – no gender relationship for boys Pupils did not find physics easier or more interesting when their teacher had a physics degree! (Biology-easier; engineering-more interesting (boys)

21. Pupil Questionnaire 2 (after teachers’ 1 st rounds of action research) (921 (37 classes) completed; 60% female) Almost a third find physics more interesting, and less difficult than before The percentage saying physics is ‘quite difficult’ has dropped from 52% to 44% between the two measurement points, a significant effect. Pupils report they have had far more talks about careers in physics in their lessons (49% from 27%) - a significant change. Significantly more pupils are now saying that physics is often linked to their everyday life and worldwide issues in classes. Significantly more pupils now feel that the amount of thinking time is ‘about right’.

22. Pupil Questionnaire 3 (after teachers’ 2nd rounds of action research) (958 pupils (38 groups) completed; 58% female) More than a third find physics more interesting, and 25% find it less difficult than before (20% find it more difficult!) Girls keep finding physics significantly less interesting and significantly more difficult than boys (words/terms used in physics as well) Girls now seem more decided about choosing physics post-16 (positively or negatively; boys show more intention of choosing it; % girls ‘very likely’ or ‘definitely’ choosing physics post-16 has also slightly increased) Pupils are now most interested in their physics lessons when they feel these are aimed equally at both genders and least when they are aimed at the opposite gender Pupils are now significantly more interested in physics when teachers make more links with global and social issues more (before this link was only with everyday life)

23. Comparisons between PQ1, 2 and 3 ^ ^ Significant difference in answers to question between baseline and interim measurement * Significant difference in answers to question between baseline and final measurement ** Significant difference in answers to question between interim and final measurement Pupil Questionnaire 1 Pupil Questionnaire 2 Pupil Questionnaire 3 Finding physics very or quite interesting 60.3%56.9% ^ 61.2% ** Finding physics very or quite difficult 52.6%46.6%56.3% Finding words/terms used in physics very or quite difficult 42.1%41.6%42.4% Finding physics more interesting than before n/a28.9%34.6% Finding physics less difficult than before n/a28.2%25.2% Definitely or very likely choosing physics post-16 14.4%16.9% ^ 20.9% */** Reporting careers talks 24.9%49.0% ^ 51.3% Physics often linked to other subjects in classes 12.2%15.3%21.7% Physics often linked to everyday life in classes 39.3%42.0%40.9% Physics often linked to worldwide issues in classes 32.3%35.1% ^ 36.0% Reporting ‘not enough’ time for discussion 29.8%26.6% ^ 28.6% Reporting ‘not enough’ time for reflection 39.1%36.7% ^ 35.5% *

24. Final Teacher Questionnaire 64 completed this out of 110 teachers starting the Programme 56.3% were ‘very positive’ about taking part in the ARP Programme, 42.2% ‘positive’ 95% felt it had been effective in increasing their pupils’ engagement with physics (25% ‘very effective’; 70% ‘quite effective’) 64% felt an increased motivation/enthusiasm to teach physics since starting the programme; 50% said their own interest in physics had increased 86% have made changes to their classes as a result of the programme as a whole; 50% have made changes to their schemes of work 72% increased discussion time; 58% increased reflection time 91% feel they have learnt quite a lot, or a lot about action research 100% find action research useful for improving classroom practice Many teachers (39%) say their learning is being applied by their colleagues, and also on a wider school level in 31% of cases, and in other schools in 11% of cases These teachers significantly more often than before the Programme: –Link physics with everyday life in lessons –Use gender-neutral examples

25. Increase in teacher confidence levels Large percentages of these 64 teachers are now feeling more confident than before the Programme in: –Getting pupils/girls engaged in physics (78%/56% feel more confident) –Making physics relevant (78%) –Boosting pupils’/girls’ confidence in physics (63%/59%) –Increasing pupils’ awareness of careers/futures in physics (70%) –Making abstract physics more ‘visible’ for pupils (70%) –Making the most of resources (62%)

26. Links between teacher effects of ARPP and student interest [statistcally sig links] [Student] How interested in physics are you? [Teacher] How has your confidence in teaching physics changed for you since you started the programme? * (t<.05) [Teacher] What is your general feeling about taking part in the Action Research for Physics Programme? ** (t<.01) [Teacher] How effective do you feel your participation in the Action Research for Physics Programme has been in increasing your pupils' engagement with physics? ** (t<.01)

27. Comparisons between Final Pupil Questionnaire and the Control (‘Year Above’) group 656 control group respondents Compared only between schools of which Final Pupil Questionnaire received Significant differences of Final Pupil Questionnaire with Control Group: –Reduced experienced level of difficulty of physics, and of words/terms used –Increase in pupil interest in physics –Increase in likelihood of post-16 physics uptake –Increase in reported reflection time –Higher number of reported careers talks, esp. by science teachers

28. Limitations and issues encountered during the evaluation When did teachers actually give questionnaires to their pupils? In some schools, the information gained applied to physics lessons of another teacher as the ARPP teacher had not started teaching this group physics Some teachers reported pupil ‘questionnaire fatigue’ Not all teachers actually did a 1 st round of action research, or had a different topic during the 1 st round, but still sent baseline and interim questionnaires Sometimes it seemed in the pupil focus groups that the teachers had already long finished their interventions and the pupils couldn’t really think what might have changed – might apply to final questionnaires as well In the case of career interventions, these were often ‘bolted on’ outside of classes so pupils might not have found changes in their classes Spurious factors! 68 teachers carried out unique projects in their unique circumstances – the more generalised measurement of the success of the course/interventions as a whole is therefore subject to many factors and we cannot strictly refer to the evaluation of ‘one intervention’. There was some variability in the delivery across the SLCs

29. Summary of Focus Group Meetings Prior to interventions “Physics deals with unanswered questions; it is not common knowledge, we don’t know everything about it yet – which makes it fascinating” “Boring” aspects often mentioned are: circuits, friction, and a feeling of lots of repetition. Maths is most often mentioned as a difficult aspect. Frustrating aspects often mentioned are: the lack of deeper explanations about how things work, and that in physics it is harder to ‘see’ things happening (especially by girls). Physics practicals are generally very much enjoyed, and found helpful. Most groups say careers are hardly mentioned in class and that they don’t have much of an idea what you can do with physics. Girls more often feel that physics is a male-oriented profession and are not eager to enter into if there are few women there. In many groups it is felt there is quite a lot of time for discussion in class. Many feel that they should be taken out of the classroom more – more activities around the school, but also physics trips are much desired.

30. Teacher focus group meetings 9 held at end of CPD day 3 at all Science Learning Centres (live or VC) Teachers very positive about the benefits for their practice of discussing classroom strategies and ideas with other teachers on the course Many teachers feel formal engagement with action research has had many benefits –realise they do it all the time, –but the course ensured greater focus, –and the cyclical nature of action research impacted more on their practice –some say they will try to do this more often Many teachers feel pupils are more engaged now Discussion in class is found to be extremely useful Many teachers not sure of impact on post-16 take-up (hard to tell – and even if increased take-up, what are the reasons?)

31. Part 2 Celebrating University /School Partnerships Revisit an earlier slide…..

32. The Action Research for Physics Programme (ARPP) Research commissioned by the National Network of Science Learning Centres and DCSF. Organised and managed by the nine Regional Science Learning Centres between September 2009 and February 2011. The ARP programme followed on from the findings and recommendations of the Girls in the Physics Classroom (Hollins et al., 2006) and Girls into Physics (Daly et al., 2009) projects.

35. School/university partnership? CPD Are you a teacher or a researcher? 2/3 strand contracts. Input on PGCE? Engagement with school mentors? Link with SLCs? Action Research in the schools we work with? Teaching Schools??????????????????????????