1. California Educational Research Association Annual Meeting CERA Session 3 Day 1 – Preparation and Professional Development Rancho Mirage, CA – December 4 th, 2008 Rebecca E. Buschang Validating Measures of Math Teacher Knowledge

2. 2 / 31 Plan Background information Study methods and measures Results Conclusions Future directions

3. 3 / 31 Background Math teacher knowledge is complex Content knowledge Pedagogical content knowledge Knowledge of students and groups of students Knowledge of curriculum and resources Currently, no widely accepted measures of math teacher knowledge Shulman, 1986, Hill et al, 2004, 2005, 2008

4. 4 / 31 Need for valid measures Assessments of different forms of math teacher knowledge… Give us information about level of knowledge and types of knowledge teachers have Can be helpful to professional development providers

5. 5 / 31 Purpose and research questions Purpose: to gather validity evidence about how different tasks measure math teacher knowledge Research Questions Do these measures distinguish between expertise? Do these measures relate to each other?  Pedagogical content knowledge  Content knowledge

6. 6 / 31 Methods 86 participants math experts (n = 13) teacher experts (n = 10) novice teachers (n = 17) experienced teachers, non- NBC teachers (n = 46) 4 tasks + background survey

7. 7 / 31 Overview of measures Content Knowledge Content knowledge mapping task PCK Knowledge Pedagogical content knowledge mapping task Multiple choice measure of teaching knowledge Student work analysis task

8. 8 / 31 Content knowledge map

9. 9 / 31 Pedagogical content knowledge map Solve for the unknown 5x + 3(x + 4) = 28 2 1 1

10. 10 / 31 Multiple choice measure of teaching knowledge (example #1)

11. 11 / 31 Multiple choice measure of teaching knowledge (example #2)

12. 12 / 31 Multiple choice measure of teaching knowledge (example #3)

13. 13 / 31 Student work analysis task

14. 14 / 31 Analysis of Data Group differences (ANOVAs) Correlations between tasks (Spearman) Whole group Split by participant group

15. 15 / 31 Group Differences

16. 16 / 31 Performance on concept knowledge map

17. 17 / 31 Performance on PCK knowledge map

18. 18 / 31 Performance on multiple choice measure of math teaching knowledge

19. 19 / 31 Performance on multiple choice measure of math teaching knowledge

20. 20 / 31 Performance on student work analysis task 3 4

21. 21 / 31 Summary of findings (group differences) Performance differed by group Two tasks sensitive to expertise

22. 22 / 31 Relation between tasks

23. 23 / 31 Correlations Between Tasks MC measure Content KM PCK KMStudent work analysis MC measure 1.46**.53**.04 Content KM 1.49**.05 PCK KM 1.15 Student work analysis 1

24. 24 / 31 Correlations Between PCK Tasks PCK KMStudent work analysis MC measure Math experts.02-.09 Teacher experts.66.64* Experienced teachers.64**.22 Novice teachers.08-.16 PCK knowledge map Math experts----.45** Teacher experts---.92** Experienced teachers---.37 Novice teachers.32

25. 25 / 31 Correlations between content and PCK tasks MC measure PCK KMStudent work analysis Content Knowledge map CK experts.07.43-.45 PCK experts.29.07.13 ET.42**.58**.02 NT.47.36

26. 26 / 31 Summary of findings (relation of tasks) Evidence that different types of knowledge are being measured Evidence that there are differences by expertise Evidence that math teacher knowledge is complex

27. 27 / 31 Overall findings and conclusions Evidence for validity Two tasks sensitive to expertise (group differences) Tasks measuring different types of knowledge (correlations) Using different expert groups generated key results Teacher knowledge is complex to assess Limitations Small sample size in several groups

28. 28 / 31 Future Directions Create measures more sensitive to expertise Create measures that are sensitive to pedagogical content knowledge but not masked by content knowledge Parse apart different aspects of pedagogical content knowledge

29. 29 / 31 Acknowledgements Dr. Eva L. Baker Dr. Gregory K.W.K Chung Girlie C. Delacruz Joanne Michiuye

30. Rebecca E. Buschang buschang@cse.ucla.edu