1. Reconceptualizing Policy as Designs for Supporting Learning Paul CobbKara Jackson Vanderbilt University

2. Purpose Describe and illustrate a learning design perspective for analyzing policies Justification - usefulness – Anticipate limitations of specific policies – Understand why specific policies play out in particular ways in particular situations Feeds back to inform the revision of the policy

3. Overview Develop the learning design perspective on policies Illustrate the usefulness of this perspective – Efforts of a US school district to improve the quality of mathematics instruction Policies for improvement in mathematics How they were implemented Develop entailments of this perspective on policies

4. Policy and Change A policy specifies either: Changes in a group of people’s practices – Principals will act as instructional leaders by observing classroom instruction and giving feedback Changes in results/outcomes that require the members of one or more groups to change in practices – Schools will increase students test scores in mathematics

5. Policy and Change A policy is an attempt by members of one group to influence the practices of members of another group (Coburn & Stein, 2006) – Policies necessarily involve relations of power

6. Dominant Views of Policy Researchers in educational policy typically begin by analyzing national or state policies Study extent to which the targets of policy implement the policy as intended – Change their practices as intended by policymakers (Stein, 2004)

7. Dominant Views of Policy Dissemination of information about the intent of the policy – Know what changes they should make in their practices Incentives and accountability – Motivate to make intended changes

8. Policy and Learning Any serious policy - any policy that does not simply endorse current practice and call for more of it - requires learning on the part of those who implement it (Cohen & Barnes, 1993)

9. Policy and Learning Ambitious policies require practitioners to develop new capabilities and to unlearn present capabilities Implementation of a policy is a species of learning, and policy is a sort of instruction (Cohen et al, 2007, italics added)

10. Policies as Designs for Supporting Learning Three components that correspond to the what, how, and why of policy What: A vision for the practices of members of the target group – Principals will act as instructional leaders by observing classroom instruction and giving teachers feedback

11. Policies as Designs for Supporting Learning How: The designed supports for learning for members of the target group – Professional development for principals as instructional leaders

12. Policies as Designs for Supporting Learning Why: An (often implicit) rationale that explains how the supports will bring about the intended improvements in practice by scaffolding the learning of members of the target group

13. The How of Policy: Supports for Learning New positions or changes in responsibilities for existing positions – School-based mathematics coaches Support principals as instructional leaders in mathematics Tools – Curriculum maps

14. The How of Policy: Supports for Learning Tools can be conceptual as well as material (e.g., principles for organizing mathematical ideas implicit in the content maps) – Must be reified by the members of the target group – What is constituted as tool is an empirical question

15. The How of Policy: Supports for Learning Intentional learning events – Group of people work together on an ongoing basis with the explicit goal of improving their own practices Either ongoing or discrete

16. The How of Policy: Supports for Learning Ongoing intentional learning events – Regularly scheduled meetings that build on one another; group is relatively small so it can become a genuine community of practice Discrete intentional learning events – One-off professional development sessions (e.g., on using the content maps) – Regularly scheduled meetings that do not build on each other (e.g., monthly principal meetings)

17. The How of Policy: Supports for Learning Incidental learning events – At least two people working together to a function of the school Weekly meetings between principal and coach to discuss quality of math teaching and consider how to support teachers’ learning – Improving their own practices is not an explicit goal – Nonetheless, learning opportunities can arise in the course of the joint work incidentally Either ongoing or discrete

18. The How of Policy: Supports for Learning New organizational routines Organizational routine: A repetitive, recognizable pattern of interdependent actions, carried out by multiple actors (Feldman & Pentland, 2003) – Learning Walks with the math coach Assess quality of mathematics instruction in the school and thus identify teachers’ needs

19. Summary: Policies as Learning Designs What: Envisioned practices How: Supports for learning – New positions – Tools – Intentional learning events – Incidental learning events – New organizational routines Why: Justification or rationale

20. Background: US Educational System Decentralized education system – Local control of schooling Each US state divided into a number of independent school districts – Rural districts with less than 1,000 students – Urban districts with 100,000 students or more

21. Background: US Educational Policy No Child Left Behind Policy (NCLB) – Standards for mathematics learning 50-80 standards per grade common – Assessments at the end of each school year to test whether students are achieving these standards Primarily procedural skill at expense of conceptual understanding – Yearly student achievement goals in mathematics for each school

22. Background: US Educational Policy Instruments used to influence practice are typically disconnected from the learning that teachers and school leaders have to do to develop more effective practices— long on pressure and short on support (Knapp & Shields, 1995) Policy rarely attends to what school leaders and teachers would have to learn to carry it out (Elmore, 2000; Spillane & Thompson, 1999)

23. Background: Research Project Central question: What does it take to improve the quality of mathematics instruction on a large scale? Four urban districts – High proportion of students from traditionally underserved groups – Limited financial resources – High teacher turn over

24. Background: Research Project Most schools and districts clueless about how to respond productively to high-stakes accountability – A small minority have reasonably worked out strategies (Elmore, 2000) Investigating the four districts’ instructional improvement efforts in middle-school mathematics (12-14 years old students) – District B as an illustrative case

25. Background: US Educational Policy National Council of Teachers of Mathematics’ (NCTM) Principles and Standards for School Mathematics – Build on students’ current reasoning to achieve a mathematical agenda that focuses on central mathematical ideas Consistent with research in mathematics education and related fields

26. Background: US Educational Policy Teacher adjusts instruction to the students – Ongoing assessment of students’ reasoning Teaching becomes non-routine – A complex and demanding activity

27. Background: US Educational Policy Deep understanding of mathematics – Mathematical knowledge for teaching Knowledge of how students’ reasoning develops in particular mathematical domains – Anticipate range of solutions Know-in-action how to achieve a mathematical agenda by building on students’ (diverse) solutions

28. Background: US Policy Context NCLB - specifies a result that requires unspecified changes in instructional practice – Increased student mathematics achievement NCTM - specifies an envisioned form of instructional practice

29. Background: US Policy Context The how of both national policies: District and school leaders will formulate concrete local policies for improvement – Potentially competing national policies are key aspects of the contexts in which district leaders make policies for mathematics teaching and learning

30. National Policies as Discourses NCLB and NCTM constitute alternative policy Discourses – Discourse of high-stakes accountability Increase student performance in mathematics – Discourse of instructional reform Improve quality of mathematics teachers’ instructional practices (Confrey et al., 2000)

31. National Policies as Discourses Discourses are sociohistorical coordinations of people, objects (props), ways of talking, acting, interacting, thinking, valuing, and (sometimes) writing and reading that allow for the display and recognition of socially significant identities (Gee, 1997)

32. Background: District B 80,000 students – 56.9% Hispanic 27% of all students Limited English Proficient (LEP) – 26.3% African American – 15% White

33. Background: District B Eighth grade state mathematics standards – 38% of African American students – 55% of Hispanic students 27% of LEP students – 76% of White students

34. Background: District B District leaders situated primarily in the discourse of instructional reform – Betting that test scores will increase as the quality of mathematics instruction improves Coherent set of strategies for supporting teachers’ and school leaders’ learning

35. Background: District B A resource for formulating and implementing district policies: – Guide How problems are framed – Account – Legitimate (Feldman & Pentland, 2003)

36. Background: District B Leaders in most urban districts situated in the discourse of high-stakes accountability – Teach directly to the test – Game the accountability system

37. Background: District B Third year of collaborating with District B – Data for this analysis is from Year 2 October: Interview district leaders to document current strategies for improving middle-school mathematics – Each strategy is a policy Specifies the What, How, and sometimes the Why The set of policies constitutes District B’s Theory-of- Action for instructional improvement in middle- school mathematics

38. Visions for Role Groups’ Practices ROLE GROUPENVISIONED PRACTICES TeachersHigh-quality instructional practices that support all students’ learning of significant mathematical ideas Mathematics Coaches Support both teachers’ development of high- quality instructional practices and principals’ development of content-specific instructional leadership practices PrincipalsSupport and hold teachers accountable for developing high-quality instructional practices

39. Visions for Role Groups’ Practices ROLE GROUPENVISIONED PRACTICES District Math Specialists Support mathematics coaches’ development of effective coaching practices District Leadership Specialists Support and hold principals accountable for developing effective instructional leadership practices

40. Situated Account of District B Leaders’ Policy-Making National Discourses Expertise Outside District Expertise Within District High-stakes accountability Math education reform Prior involvement in instructional improvement efforts Professional network (e.g., professional development contractors) Central office staff, including Mathematics Department

41. District B as an Illustrative Case

42. Vision for Principals’ Practices Support and hold teachers accountable for developing high-quality instructional practices – Develop a vision of high-quality mathematics instruction – Conduct “learning walks” (sometimes with coaches) to assess building needs and determine the nature of assistance needed by teachers – Observe classroom instruction and give feedback – Work with the coach to ensure coach provides appropriate professional development

43. The “How” and “Why” of District Policy for Principals New Positions, or New Responsibilities for Existing Positions Tools Intentional Learning Events Ongoing Discrete Incidental Learning Events Ongoing Discrete Organizational Routines

44. Means of Support for Principals’ Learning MEANS OF SUPPORTDISTRICT B DESIGNED POLICY New Positions or New Responsibilities for Positions Created Mathematics Coach position, supports principals’ instructional leadership ToolsCurriculum Maps Student work for analysis in monthly principal meetings

45. Means of Support for Principals’ Learning MEANS OF SUPPORT DISTRICT B DESIGNED POLICY Intentional Learning Events Ongoing Discrete Monthly principal meetings (the meetings do not necessarily building on one another). Incidental Learning Events Ongoing Weekly meeting between principals and coaches in which coaches share observations about the quality of math instruction and determine how to support struggling teachers. Discrete

46. Means of Support for Principals’ Learning MEANS OF SUPPORTDISTRICT B DESIGNED POLICY Organizational Routines (Feldman & Pentland, 2003) Learning Walks with Mathematics Coaches

47. Potential Limitations of Policy Limited opportunities to work with a more knowledgeable other on their own practices Limited intentional learning events that would sufficiently support principals’ development as instructional leaders – Discrete learning events not likely to support principals’ development of a vision of high-quality mathematics instruction or how to support teachers’ development of ambitious instructional practices

48. Potential Limitations of Policy In incidental learning events, principals are not explicitly working on their own practice – Weekly principal/coach meetings Principals are not supported to learn how to use tools (e.g., content maps) in their own practice

49. Documenting Principals’ Actual Practices Interviews in January with Principal, Mathematics Coaches, and Teachers – Determine how the District’s Theory-of-Action is playing out in schools and classrooms Analysis involves triangulating Principals’, Coaches’, and Teachers’ accounts of the Principals’ practices

50. Principals’ Envisioned vs. Actual Instructional Leadership Practices ENVISONED PRACTICESACTUAL PRACTICES Developing a vision of high-quality mathematics instruction Most Principals have developed vision of high-quality instruction that is compatible with what the district is aiming for; however visions are not developed; principals tend to focus on the form rather than function of high- quality instruction (Saxe, Spillane) Observing instructionMost Principals spend considerable amounts of time observing instruction but they focus on the forms of instruction and do not communicate appropriate expectations for instructional improvement

51. Principals’ Envisioned vs. Actual Instructional Leadership Practices ENVISONED PRACTICESACTUAL PRACTICES Conducting “learning walks” (sometimes with coaches) to assess building needs and determine the nature of assistance needed by teachers Principals only occasionally take “learning walks,” and only one principal reported doing “learning walks” with a coach. Coach’s schedules make it difficult to schedule learning walks. Working with the coach to ensure coach provides appropriate professional development to the staff Principals and coaches meet regularly. In half of the schools, meetings focus on issues such as the pacing of instruction, while in the other schools, they focus on teachers’ classroom practices.

52. Designed and Implemented Policy Identify differences between envisioned and actual practices Account for these differences by – Situating principals’ learning in the school and district settings in which they work – Analyzing the supports for principals’ learning

53. Situating Principals’ Learning Additional aspects of the institutional setting that proved relevant included: – Accountability relations with Leadership Specialists – Means of supporting principals’ learning – Expertise of coach in school – Expertise of teachers in school

54. Situating Principals’ Enacted Practices in the Institutional Setting Accountability relations with Leadership Specialists – Although the policy specified that district leadership specialists’ were to hold the principals accountable for supporting the improvement of teachers’ instructional practices, the principals reported that they were held accountable for: 1) Raising test scores primarily 2) Focusing on improvement of instruction secondarily – Implications: Principals do not communicate clear instructional expectations to teachers (e.g., nature of the feedback they provide)

55. Situating Principals’ Enacted Practices in the Institutional Setting Means of Support – Principals received inadequate support for developing instructional leadership practices Limited opportunities to work with a more knowledgeable other on their practices – Very few Learning Walks with Coach

56. Situating Principals’ Enacted Practices in the Institutional Setting Means of Support – No ongoing intentional learning events Discrete intentional learning events (e.g., principal monthly meetings) without ongoing learning are insufficient to support the development of either a vision of high-quality instruction or effective instructional leadership practices

57. Situating Principals’ Enacted Practices in the Institutional Setting Means of Support – What the district intended as tools to support principals’ learning (e.g., content maps) did not became tools for them because principals weren’t supported to learn how to use the tools

58. Research Team’s Feedback Based on our analysis, each spring we – Provide a written report to the District Leaders – Meet with the District Leaders to discuss the report Report/discussion includes – Detailed feedback regarding how the District’s Theory-of-Action is playing out – Actionable recommendations

59. Accountability Relations MIST RECOMMENDATION MAY 2008REVISION TO POLICY FALL 2009 Address the tension that principals experience between improving the quality of instruction and raising test scores. Increased emphasis on how District Leadership Specialists will communicate expectations to principals and how they will support principals’ development of instructional leadership practices Leadership specialists and principals conduct Learning Walks together Weekly Instructional Leadership meeting at schools (District leadership specialists attends) Clarify what principals should expect instructionally of teachers, and how principals can communicate those expectation; requires coordinated efforts of Curriculum & Instruction and Leadership

60. Supports for Principals’ Learning MIST RECOMMENDATION MAY 2008 REVISION TO POLICY FALL 2009 Provide professional development that focuses on recognizing high-quality math instruction and giving feedback to teachers Principal meetings will focus more deeply on supporting principals to develop forms of instructional leadership practices (e.g., sustained learning events specific to the curriculum and recognizing high- quality instruction) No emphasis on providing feedback on instruction

61. Policies as Designs for Supporting Learning Usefulness: Explanatory and predictive power – Allows us to anticipate limitations in policies – Allows us to understand why policies play out in particular ways in specific situations Explain why members of role groups develop particular practices and not others in the institutional settings in which they work

62. Policies as Designs for Supporting Learning Institutional settings in which principals work are the immediate contexts of their learning The supports as they are actually enacted are key aspects of these (evolving) institutional settings – Math coach Weekly meetings (Learning Walks with coach) – Monthly principal meetings – (Tools – curriculum map)

63. Policies as Designs for Supporting Learning Resulting situated analysis of policy implementation relates: – The practices that principals developed – The institutional setting of their learning How of policy as implemented

64. Policies as Designs for Supporting Learning Develop specific actionable recommendations that might make the policy more effective – Propose adjustments to the district’s improvement strategies – the how of policy – Testable conjectures about: Envisioned improvements in principal’s practices The means of supporting that learning Design experiment at the level of a large school district

65. Generality of the Learning Design Perspective on Policy District B: – District leaders conceptualized instructional improvement in terms of supporting (and motivating) others’ learning – Explicit vision of high-quality mathematics instruction – Coherent set of improvement strategies

66. Generality of the Learning Design Perspective on Policy Claim: Perspective is also useful when: – Policymakers conceptualize instructional improvement in terms of disseminating information about intended practices – Policy specifies only intended results/outcomes Increase in student mathematics achievement (test scores)

67. Generality of the Learning Design Perspective on Policy Teachers’ initial instructional practices + initial student test scores Consequences of the policy – Changes in the institutional setting of teaching Supports, incentives, accountability Changes in teachers’ instructional practices + resulting student test scores Explain why teachers changed their practices in the ways documented

68. Policymaking at Multiple Levels Principals were the targets’ of district policy Principals made policies that targeted teachers (and math coaches)

69. Policymaking at Multiple Levels What: Vision for teachers’ instructional practices – Form rather than function view How: Means of achieving vision – Observed classroom and communicated expectations – Met with math coach regularly In some schools, focused on teachers’ instructional practices

70. Policymaking at Multiple Levels Teachers’ made policy that targeted students – What: Vision for students’ mathematical practices Instructional goals – How: Means of achieving the vision Instructional practices

71. Policymaking at Multiple Levels Network of policy makers - each makes policy in a setting shaped by others’ policy making efforts – District leaders National policy Discourses – Principals National policy Discourses District leaders’, leadership specialists’, and mathematics specialists’ policymaking – Teachers National policy Discourses Principals, mathematics coaches’, and mathematics specialists’ policy making efforts

72. Process of Policy Implementation Dominant perspective – A single policy travels down through the education system Distortion, resistance Learning design perspective – Policymaking at multiple levels of the education system Develop policy vision and attempt to achieve it Situated reorganization of practices

73. Usefulness Revisited The learning design perspective an an analytical lens Initial indications that it might be useful in the practice (of policymaking)

74. Usefulness Revisited District leaders’ view instructional improvement as a process of: – Supporting others’ learning – Disseminating information about desired practices and pressing for compliance Extent to which mathematics specialists viewed as a valued resource