1. 1 Gender and PISA Mathematics: Irish Results in Context SEAN CLOSE and GERRY SHIEL Educational Research Centre, St. Patrick’s College, Dublin, Ireland Presentation to the National Symposium on the Results of PISA 2006

2. 2 Overview Background to the Study Aims of the Study Overall Results by Gender Overarching Ideas by Gender Competency Clusters by Gender Proficiency Levels by Gender Item Format by Gender Attitudes to Mathematics by Gender Conclusions

3. 3 Background to the study Strong focus on gender differences in mathematics from c. 1970 to 1990, but less so in recent years as differences became less pronounced Males still outperform females generally at the extremes of the mathematics performance distribution, particularly in certain content areas Males continue to obtain higher self efficacy scores in maths than females, while females show higher levels of anxiety about maths than males

4. 4 Aims of the study One aim of the study was to examine gender differences in mathematics of the Irish PISA survey samples with respect to performance on (i) the overall PISA mathematics scale and related proficiency levels; (ii) the four PISA overarching ideas and contexts; (iii) the three PISA competency clusters; (iv) the different item formats; and (v) the self-efficacy and anxiety scales.

5. 5 Aims of the study A second aim was to relate these findings to the performance of the Irish survey samples on the Junior Certificate mathematics examination

6. 6 Overall results by gender (1) Irish overall performance was about average (c. 500) across the three PISA mathematics surveys Significant gender difference in favour of Irish males in each PISA survey, as was also the case with most OECD countries Contrasts with reading literacy, where Ireland’s mean score was significantly above the OECD average in each PISA survey, and females had a mean score that was significantly higher than males.

7. 7 Mean scale scores of students on PISA mathematics, by gender and year – Ireland and OECD average Year/Gender 2000*20032006 Mean Scale Score IrelandOECDIrelandOECDIrelandOECD All 503500503500501498 Male 510506510506507503 Female 497495 494496492 Male-Female** 13111511 **Significant differences are in bold *Only two of the four overarching ideas (Shape and Space and Change and Relationships) were assessed in 2000

8. 8 Overall results by gender (2) Also contrasts with results of the same students on their Junior Cert mathematics examination where female students outperform males by a small margin It is noteworthy that no significant gender differences in performance in mathematics were found in two national surveys in 1999 and 2004 at 4th grade in primary school. Also, in TIMSS 1995, there were no significant gender differences in mathematics found among Irish students at the 4th or 8th grades

9. 9 Mean Junior Certificate Performance Scores * in Mathematics by Gender and Year – PISA Sample and All Candidates 200020032006 PISA sample All candidates PISA sample All candidates PISA sample All candidates Mean Junior Cert Performance Score Mean Junior Cert performance score Male7.68.18.57.98.68.2 Female8.08.28.68.38.88.6 Total7.88.18.68.18.78.4 Significant differences are in bold. Significance is not assessed in the case of ‘All Candidates’ as these consist of the population of students taking Junior Certificate Mathematics. *The Junior Certificate Performance Scale (JCPS) is used to place the performance of all students who sat the Junior Certificate mathematics examination results by assigning scores to students’ grades on a scale from 1 (Foundation level, Grade F) to 12 (Higher level, Grade A).

10. 10 Results for overarching ideas by gender(1) When the performance of Irish students on each of the four PISA mathematical content subscales in 2003 was analysed by gender, it emerged that males significantly outperformed females on all four subscales This was also the case with most of the other participating countries, with the exception of the Quantity subscale, where less than half the countries showed significant differences in favour of males

11. 11 Results for overarching ideas by gender(2) The difference for Irish students on the Space and Shape subscale (25 scale points) was much larger than it was for the other three content areas – Change and Relationships (13), Quantity (9) and Uncertainty (15). the difference in Space in Shape contributes more to the low overall Irish mathematics score than any of the other overarching ideas

12. 12 Scale scores of Irish students in PISA 2003 by gender and overarching idea Overarching Idea/Gender No of Items MaleFemale Male- Female SED Scale score Space and Shape20489463254.3 Change and Relationships22512500134.4 Quantity2350649794.3 Uncertainty19525509154.6 Significant differences in bold. SED: Standard Error of the Difference

13. 13 Results for overarching ideas by gender(3) On average across the three JC examination levels, a little over a quarter of the JCE items mapped on to the PISA Space and Shape category. PISA Space and Shape does not cover synthetic geometry which is the main focus of geometry in the JCE curriculum (along with transformational geometry)

14. 14 Results for overarching ideas by gender(4) None of the PISA Space and Shape items fitted into the Junior Certificate syllabus category of Geometry, but rather many of them were seen as better fitting the Applied Arithmetic and Measure content area Only 5 of 85 PISA items were classified as Algebra. In TIMSS females scored significantly higher than males in many countries One third of JC items set in practical context compared with 80% of PISA items

15. 15 Percent of Items in the 2003 Junior Cert Mathematics Exam Papers Corresponding to the 4 PISA Overarching Ideas JC Examination Level / PISA category PISA 2003 Maths Test (85 Items) 2003 JCE Higher Maths (77 Items) 2003 JCE Ordinary Maths (81 Items) 2003 JCE Foundation Maths (32 item) Percent of Items Quantity 27.116.930.953.1 Space and Shape 23.533.827.212.5 Change and Relationships 25.937.732.121.9 Uncertainty 23.511.79.912.5

16. 16 Percentages of PISA items and 2003 JCE questions mapping onto the Junior Certificate syllabus content areas Junior Cycle Content AreaApprox. Percent of PISA items in Content Area Approx Percent of 2003 JCE questions in Content Area 1. Sets 06 2. Number Systems 101 3, Applied Arithm/Measure 33 4. Functions and Graphs 410 5. Algebra 516.6 6. Geometry 016.6 7. Trigonometry 06.6 8. Statistics 2010 Not on either syllabus 30- Based on data from Cosgrove et al, (2005); and the Chief Examiner’s report on the JCE mathematics for 2003 (www.examinations.ie)

17. 17 Results for Competency Clusters by Gender Although differences in mean percent scores between male and female students on each of the Reproduction, Connections or Reflection clusters did not reach significance, they were somewhat in favour of males on each cluster In the PISA test, there was a reasonable spread of items across the three competency clusters. In the JCE mathematics papers (2003), most of the items fell into the Reproduction cluster, with very few in the Connections cluster and none in the Reflection cluster Given the stronger performance of females on JCE mathematics, one might have expected females to have done better at least on PISA reproduction tasks.

18. 18 Mean percentage mathematics scores of Irish Students in PISA 2003, by gender and competency cluster Competency Cluster No of Items FemalesMales Males-Females (Difference) Percent Correct MeanSEMeanSEMeanSED Reproduction 25 62.01.9265.8 1.843.8 2.66 Connections 40 45.0 1.9948.8 1.873.7 2.73 Reflection 19 33.7 1.9137.7 1.804.0 2.62

19. 19 Percentage of items testing the three competency clusters, distributed across the PISA 2003 test and the 2003 JC exam

20. 20 Results for Proficiency Levels by Gender Though there were no significant gender differences among Irish students at any of the six proficiency levels on the combined mathematics scale, there were significant differences at Level 6 (2.5% male vs 1.1% female) and below Level 1 (8.6% male vs 13% female) on the Space and Shape subscale, and at Level 6 (5.2% male vs 2.7% female) on the Uncertainty scale The greater number of males at Level 6 is in keeping with findings in large scale surveys in general that males are more dominant at the high end of distributions of mathematics achievements, an effect which increases with age

21. 21 Percentages of Irish students at each combined mathematics proficiency level in PISA 2003 and percentage differences, by gender Level FemalesMales Males-Females Difference % students SE % Students SE% DiffSED < Level 15.20.744.20.79 1.01.08 Level 113.51.2810.81.13 2.71.71 Level 224.71.3722.51.44 2.21.99 Level 328.21.3627.81.46 0.42.00 Level 419.41.2121.01.63 -1.62.03 Level 57.40.8310.81.09 -3.41.37 Level 61.60.362.90.50 -1.30.62

22. 22 Results for Item Format by Gender Two categories of items were examined: multiple choice (both simple and complex) versus constructed response formats (i.e., short, open extended and closed extended) A difference in favour of males was significant for the multiple choice format but not for the constructed response format This could be related to females’ lesser self confidence and greater anxiety in mathematics, leading to less of the risk- taking behaviour which may be involved in multiple choice items

23. 23 Mean percentage mathematics scores of Irish students on PISA 2003, by gender and item format Format Number of Items FemalesMales Males- Females Difference Percent MeanSEDMean SEDMean SED Multiple Choice 28 47.82.0853.51.905.72.8 Construct Response 56 47.41.8950.21.812.82.62 Significant differences in bold

24. 24 Attitudes to mathematics by gender (1) Strongly associated with PISA mathematics performance was self-efficacy, which explained a remarkable OECD average of 23 percent of total variance in student performance In general, however, males had significantly higher self-efficacy scores than females within countries, including Ireland Although it was not as strong a variable as self- efficacy, anxiety about maths accounted for about 13 percent of variance in student performance

25. 25 Attitudes to mathematics by gender (2) In all countries bar two (Poland and Serbia), male students reported significantly less anxiety about mathematics than females The male advantage on these attitudinal variables may be the result of the broader socio-cultural context or educational policies and practices

26. 26 Attitudes to mathematics by gender (3) In Ireland, about half of all females and one third of all male students attend higher performing single sex schools. Traditionally, attitudes to mathematics and the physical sciences have been quite different with these subjects being less favoured in all-female schools compared with male schools. Although this effect has been reducing over time, it may still have some influence on females’ attitudes to mathematics if not their achievement.

27. 27 Conclusions (1) The poor performance of Irish students generally, and females in particular, on the Space and Shape items suggest the need for more emphasis on this area in the Junior Cycle curriculum (grades 7-9). A similar recommendation has been made in the report on the 2004 National Assessment of Mathematics Achievement at fourth grade where performance on Shape and Space and Measure were of some concern

28. 28 Conclusions (2) Another finding which showed that more males than females achieve at the highest levels of proficiency on the mathematics combined scale, is common in many countries and may be associated with lower female participation in higher level mathematics courses and in science, engineering and technology courses at senior secondary school and college

29. 29 Conclusions (3) The significantly higher performance of males on multiple choice items warrants some attention. Multiple choice items are used widely on standardised tests (but not on the Junior Certificate mathematics exam). The finding that Irish males are higher than females in self- efficacy in mathematics is in keeping with international trends on these attributes, which have held firm over many years. This advantage on the part of males may also have been associated with greater risk-taking on their part in the face of the unusual nature of the PISA items and consequently partly contributed to the gender differences in performance.

30. 30 Conclusions (4) Motivation may also play a part as female students in single-sex schools are strongly focused on ‘high-stakes’ examinations such as the Junior Certificate examination – more so that male students, and hence may approach PISA in a different way. A range of factors contribute to gender differences in PISA mathematics.

31. 31