1. Assessing Gender Differences in Students' Understanding of Magnetism using Magnetism Conceptual Survey Jing Li Department of Physics and Astronomy University of Pittsburgh Co-Author: Chandralekha Singh

2. Several prior studies on gender effects in college introductory physics – K. Heller, APS April meeting Talk, Denver, CO, (2009). – M. Lorenzo, C. Crouch and E. Mazur, Am. J. Phys. 74(2), 118, (2006). – S. Pollock, N. Finkelstein and L. Kost, Phys. Rev. ST PER 3, 010107 (2007). – L. Kost et al., Phys. Rev. ST PER 5, 010101 (2009). – P. Kohl, H. Kuo, PERC Proceedings. 1179, 173 (2009). – L. Kost et al., PERC Proceedings, 1179, 177 (2009). – L. Kost et al., PhysRev: ST Phys Ed. Rsrch 6 (2), 020112 (2010). Generally observed that in traditionally taught courses – Males outperform females on FCI/BEMA – Even if in some courses the final exam grades on traditional exams comparable (not consistent across various studies) Especially designed courses (e.g., at Harvard course developed by the Mazur group) can reduce the gender difference Values affirmation and Self-efficacy issues and accumulation of bias of various kinds over time may play a role in gender differences – Miyake et al. Science 330, 1234 (2010). Background

3. Difficult to pinpoint one reason for gender differences between males and females since issue is very complex Also, remedies suggested by various groups are different – For example, Z. Hazari, G. Sonnert & P. Sadler (AAPT meeing, Ann Arbor, 2009) claim that presence of women role models is unlikely to bridge the gap between male and female performance because female students are unlikely to identify with the role models. – Also see, Z. Hazari, R. Tai and P. Sadler, Science Education, 847, 2007. – Hazari and G. Potvin, E. Journal of Science Education, 10(1), 33, 1-33, 2005. – Hazari, P. sadler and R. Tai, Phys Teach, 46, 423, 2008. – They suggest that female students ‘ identity as a scientist and self-efficacy issues are at the heart of gender difference in science/math Others emphasize the importance of role models – For example, S. Yennello (AAPT Meeting, Ann Arbor, 2009) Background

4. Here we investigate gender effects in traditionally taught introductory algebra-based and calculus-based introductory physics on magnetism concepts using Magnetism Conceptual Survey (MCS) we developed by administering it as a pre-test and post-test to large classes Many questions in magnetism require the ability to visualize in three dimensions (force on moving charges or current carrying wires, magnetic field produced by current carrying wires etc.) Some research suggests that females generally have a better verbal ability but worse spatial ability than males which can restrict their reasoning in 3D – often there is a correlation between students’ spatial ability and their self- confidence – D. Law, J. Pellegrino and E. Hunt, Psychological Science, 4(1), 35 (1993). – 12. M. Beth Casey, R. Nuttall and E. Pezaris, Journal for Research in Mathematics Education, 32(1), 28 (2001). Background

5. Advantage of multiple-choice test – Objective, easy to grade, results for different groups can be compared Disadvantage is that thought process is not revealed by test answers alone – However, research-based multiple-choice tests in which distracter choices are designed with input from students can be very effective After discussion of the development of Magnetism Conceptual Survey – Gender effects – before and after instruction – Different types of classes such as algebra-based, calc-based, honors students Development of the MCS

6. we developed a 30 item multiple-choice test on magnetism (up to Faraday’s law) to explore the difficulties students have in – interpreting these concepts – Correctly identifying & applying concepts in different situations To design good distractor choices, during test development students were – given free-response questions asking them for reasoning for their responses – interviewed individually The magnetism test was developed by consulting with faculty members who teach introductory physics What students should definitely be able to do – Three faculty members went over several versions of the test. – Several introductory students were asked to answer the test questions individually in interviews. Development of the MCS

7. The test was administered as both a pre- and post-test to a large number of students at University of Pittsburgh. – three algebra-based classes and eight calculus-based classes. – Students were taking a traditional algebra-based or calculus-based E&M course. Pre-tests were administered in the first lecture or recitation at the beginning of each semester. Post-tests were administered in the recitations after instructors finished all concepts on MCS – Students were asked to work on the test in one class period (40-50 minutes). We keep only the students who took both the pre- and post-test except those in one algebra class in which most students didn’t provide their names. – 267 algebra-based students took the pre-test, and 273 students took the post-test. – 575 calculus-based students took both the pre- and post-tests. Administration

8. Reliability of MCS Reliability is a measure of the self consistency of a whole test. It is high if students are not guessing KR-20 for MCS 0.83 for introductory physics – Good from standards of test design MCS administered to 42 graduate student to benchmark performance – KR-20 is 0.87 – Average score is 83%

9. A measure of consistency of a single test item with the whole test. The desired value should be greater than 0.2. Point biserial coefficient for MCS is 0.42. Point Biserial Coefficient

10. A measure of the difficulty of a single test question. It is calculated by taking the ratio of the number correct responses on the question to the total number of students who attempted the question. The desired value is over 0.3. Average for MCS is 0.46. Item Difficulty

11. A measure of the discriminatory power of each item in a test. Can be calculated as The desired value should be greater than 0.3. Average item discrimination for MCS is 0.33. Item Discrimination Index

12. For algebra-based course, there is no significant difference between men and women on the pre-test and post-test. Assessing Gender Difference in the Performance in Algebra-based Courses Algebra based Pre-test GroupNumberMeanS.D.Sig.(2-tailed) Men917.32.510.940 Women1067.12.36 Algebra based Post-test GroupNumberMeanS.D.Sig.(2-tailed) Men11013.25.200.360 Women12112.35.57

13. For calculus-based course, men and women don’t have significant different on the pre-test while men outperformed women on the post-test. Assessing Gender Difference in the Performance in Calculus-based Courses Calculus based Pre-test GroupNumberMeanS.D.Sig.(2-tailed) Men4038.53.400.490 Women1687.83.05 Calculus based Post-test GroupNumberMeanS.D.Sig.(2-tailed) Men40315.36.200.019 Women16813.05.38

14. Assessing Gender Difference in the Performance of the Honors Students For honour students, women do not perform as well on the post-test. Honored students GroupNumberMeanS.D.Sig.(2-tailed) Men7517.45.890.030 Women2014.16.21

15. Gender Performance on individual Questions In algebra-based classes, males outperformed females on 20 questions 4 questions have difference more than 10%. Females outperformed males more than 10% only on 1 question. In calculus-based classes, males outperformed females on 28 questions 9 questions have difference more than 10%. Female s only performed slightly better on the other two questions.

16. Possible Explanations on Gender Difference Other research using FCI/BEMA have shown gender difference in traditional introductory physics In addition, to societal factors and accumulated bias discussed in earlier studies In magnetism, most questions require ability to visualize in 3D – figure out the directions of magnetic field Lorentz force. Some prior research suggests that females generally worse spatial ability than males – restrict their reasoning in 3D – there is a correlation between students’ spatial ability and their self-confidence

17. Pre-test were given at the beginning of the course – There is a statistical difference Post-test were given at the end of the course – Statistically significant difference Calculus-based courses do better than algebra-based courses Algebra-based vs. Calculus-based course GroupNumberMeanS.D.Sig. (2- tailed) algebra2677.22.480.000 calculus5758.33.31 GroupNumberMeanS.D.Sig. (2-tailed) algebra27312.25.340.010 calculus57514.66.05

18. Why should there be a difference between algebra-based and calc-based students when conceptual questions? Possible explanations Calc-based students have better Math skills and Scientific reasoning skills Calc-based students are less likely to have cognitive overload while learning Calc-based students are more likely to build a robust knowledge structure Algebra-based vs. Calculus-based course groupNum ber MeanS.D.Sig. (2- tailed) Alge1087.432.360.042 Calc2848.243.89 groupNumb er MeanS.D.Sig. (2- tailed) Alge21313.455.340.000 Calc24115.876.31