The NEA KEYS 2.0: A Strategy for Union-Led Transformation KEYS Workshop Washington, DC August 14-15, 2013 Jacques Nacson, Ph.D. jnacson@nea.org

Our Objectives Examine how KEYS promotes improvement in teaching and learning conditions and student outcomes in schools and districts Consider the power of KEYS to advance NEA organizational goals Identify NEA resources that support KEYS and the strategies to initiate KEYS and begin the continuous school improvement process

Beliefs and Assertions Underpinning the Development of KEYS Conditions of teaching and learning define school quality and influence student achievement Assessment of conditions is an essential first step for improvement and turnaround planning The system may be failing; not the people Capacity building is critical for successful school improvement and KEYS helps to build capacity Internal capacity and system accountability are pre-requisites for meeting test-based accountability requirements

KEYS Theoretical Roots Organizational quality (organizational conditions) : W. Edward Deming Effective Schools Research Curriculum & instruction: Newman and Wehlage (University of Wisconsin) APA’s Learner’s Centered Principles Charlotte Danielson’s framework for teaching NBPTS standards Professional development, learning communities and trust relationships: Milbrey Mclaughlin, Judith Warren Little, Ann Lieberman, Michael Fullan, Anthony Bryk and the Learning Forward professional development standards

KEYS 2.0 As a School Assessment Tool KEYS staff survey KEYS family survey KEYS community survey KEYS student survey KEYS school facilitator implementation survey

KEYS Website Access to Surveys http://www.keysonline.org/about.html

What the KEYS Staff Survey Measures (250 Questions – 42 Indicators) Shared understanding and commitment to high goals (5 indicators) Open communication and collaborative problem-solving (9 indicators) Continuous assessment for teaching & learning (5 indicators) Personal and professional learning (11 indicators) Resources to support teaching & learning (5 indicators) Curriculum and instruction (7 indicators)

The Empirical Validation: (Research-Based) A Three Phase Process: --The Advisory Groups --The Development of the Instrument --The Analyses of the Data

NEA KEYS 2.0 Survey Development The Advisory Groups NEA KEYS Advisory Group: Michael Fullan, Willis Hawley, Ann Lieberman, Joe Murphy, Jomills Braddock, Susan Moore Johnson KEYS 2.0 Research Advisory Group: Jomills Braddock, Mark Smylie, Floraline Stevens, Helen Marks, Sylvia Rosenfield

NEA KEYS 2.0 Survey Development of Instrument Identified critical concepts and generated the blueprint or specifications for the survey Developed sets of questions and assembled into questionnaire format Focus groups feedback and revisions ready for pilot testing

NEA KEYS 2.0 Survey Development Pilot Testing New Instrument School is primary unit of analysis National random sample of schools: NCES All NEA Regions represented 38 schools in sample (68%) 28 NEA, 10 AFT 13 elementary (5 urban, 5 suburban, 3 rural) 13 middle (4 urban, 3 suburban, 6 rural) 12 high (4 urban, 5 suburban, 3 rural)

NEA KEYS 2.0 Survey Development Pilot Testing New Instrument Total number of respondents = 1491 (52%) Number of teachers responding = 1061 (71%) Education Support Professionals (378) and administrators (52) responding = 430 (29%)

NEA KEYS 2.0 Survey Development Data Analyses Factor analyses to identify factors (indicators) based on clustering of items Regression analyses to examine the relationships of the factors (indicators) to objective and subjective measures of student achievement To date more than 2,500 schools and 340 districts have participated in KEYS and our data base is larger than any school assessment and improvement system in the market

What the KEYS Family Survey Measures (8 Themes 52 Questions) Shared responsibility for student learning Shared understanding of and commitment to high goals for student learning Frequency of communication between school and parent Parent-school trust Sense of welcome, comfort at school Frequency and type of parent involvement in school Parent support for student learning at home School outreach to parents The KEYS Parent Survey is much shorter than the staff survey. Focused on issues which research has shown are of greatest concern to parents under eight themes.

What the KEYS Community Survey Measures (6 Themes 12 Questions) Shared responsibility for student learning Shared understanding and commitment to high goals for student learning Guardian school-trust relationship Sense of welcome and comfort at school Frequency and type of guardian involvement in school School outreach to guardians

What the KEYS Student Survey Measures (20 Themes 150 Questions) Clarity of shared goals and expectations Teachers taking responsibility for student success Student-centered curriculum All students can learn Welcoming school environment Home support for learning Safe environment Learning how to learn Engaging instructional activities In-depth learning Personal instruction and feedback Academic achievement Academic efficacy Assessment matters Cultural capital Social supports Student Engagement in School Student voice Student-teacher trust Support services are adequate

KEYS 2.0 As a Process for Continuing School Improvement Differentiating among inputs, processes and outcomes in education Commitment to 3DM (data driven decision making) Application of the KEYS continuous school improvement cycle (action research, inquiry cycle, PDSA, etc…) KEYS 2.0 online resources and strategies for analyzing KEYS data and planning for continuous improvement

Differentiating Among Inputs, Outputs and Process Measures and Objectives in Education --What are some examples of inputs in education? What would be an input objective or goal? --What are some examples of outputs or outcomes in education? What would be an outcome objective or goal? --What are some examples of processes in education? What would be a process objective or goal?

Reliance on evidence as the basis for moving forward Connecting KEYS to Student Outcomes: The Continuing School Improvement Cycle: Reliance on evidence as the basis for moving forward A collaborative culture to support the continuous improvement process The process is iterative with feedback loops at every level and at end of the cycle

KEYS 2.0 and the Continuous School Improvement Process (Cycle) Identify gap in current student outcome goals Conduct KEYS 2.0 Surveys Celebrate successes & revisit challenges Product evaluation of (process goals & student outcome goals) Initial & in-depth KEYS and other Data Analysis Adjustments based on process evaluation of implementation data Action Planning: possible interventions, best option, specific steps Implementation with monitoring of treatment fidelity

KEYS Online Resources and Strategies for Analyzing the KEYS Data NEA KEYS data analyses and interpretation trainings of school and district facilitators KEYS data reports integrating strategies and resources for continuous school improvement, including action planning and implementation Commitments of the district to support and provide assistance and sustain the schools’ improvement efforts

KEYS-Informed Continuous Improvement Report & Repeat Orientation Initial Scan of 42 Indicators Organizing Cluster Selection (gap analysis) Cluster 1 Overview Deeper Analysis Learn More Action Plan Overview Deeper Analysis Learn More Action Plan Inquiry Cluster 2 Overview Deeper Analysis Learn More Action Plan Cluster 3 Evaluate Monitor & Adjust SIP Implemen-tation Coherent SIP Impact

Initial Scan of 42 Indicators Purpose: build awareness of school profile and check for understanding about interpreting KEYS data.

Six Keys and 12 Clusters

Cluster Selection (gap analysis)

Inquiry (Overview)

Inquiry (Deeper Analysis)

Inquiry (Learn More)

Inquiry (Action Planning)

KEYS 2.0 and School Capacity Building Why is capacity important? How do we define capacity? What is meant by organizational capacity? How does KEYS serve to increase or build school (organizational) capacity?

Defining School Quality, Capacity Building and Continuous Improvement School quality defined as the internal capacity of the school to assess accurately and objectively its strengths and weaknesses, to determine root causes and to take appropriate actions to implement improvements and evaluate the effects of such interventions as part of a continuous improvement cycle

Expanding the Definition of School Capacity (Resource-Based) Definition + KEYS 2.0’ Definition Effective distributed leadership Mutual trust and respect Collegial, professional culture and community Collaborative problem solving Program coherence Shared responsibility and accountability Strong parental support Knowledge-based decisions Effective professional development Teacher quality School infrastructure Educational resources Technology Class size Pre-K and all day k Special programs Proven educational programs and practices This slide is intended to demonstrate how KEYS expands the definition of school capacity beyond the needs for additional resources. There is no doubt that adequate financial based resources are necessary but they are NOT sufficient. The KEYS information provides the additional “social” capacity needed to ensure success.

KEYS 2.0 As a Measure of Internal School Capacity and Accountability INPUTS OUTCOMES PROCESSES

KEYS 2.0 as a Measure of Internal School Capacity and Accountability INPUTS Successful Citizenship Post Sec. Enrollments Satisfying Careers Etc. Increased Satisfaction-students, staff, parents Improved Attendance (Students/Staff) Higher Graduation Rates (Lower Drop Out Rates) Improved Student Learning/ Achievement OUTCOMES PROCESSES $Adequate/Equitable Quantity and Quality of Teaching Pool Readiness of Students to Learn National, State, District Policies External Supports Partnerships-Community Parent/Business

KEYS 2.0 as a Measure of Internal School Capacity and Accountability INPUTS Successful Citizenship Post Sec. Enrollments Satisfying Careers Etc. Increased Satisfaction-students, staff, parents Improved Attendance (Students/Staff) Higher Graduation Rates (Lower Drop Out Rates) Improved Student Learning/ Achievement OUTCOMES PROCESSES $Adequate/Equitable Quantity and Quality of Teaching Pool Readiness of Students to Learn National, State, District Policies External Supports Partnerships-Community Parent/Business

KEYS and Internal Capacity Building and Process Accountability PROCESSES KEY 1 Shared Understanding and Commitment to High Goals

Shared Understanding and Commitment to High Goals: The Big Picture KEY 1 Shared Understanding and Commitment to High Goals: The Big Picture Effective leadership Clear focus, purpose, and directions related to academic emphasis and instructional issues Shared commitments, collective responsibility and trust

KEYS and Internal Capacity Building and Process Accountability PROCESSES KEY 1 Open Communication and Collaborative Problem Solving KEY 2 Shared Understanding and Commitment to High Goals

Open Communication and Collaborative Problem Solving: The Big Picture KEY 2 Open Communication and Collaborative Problem Solving: The Big Picture Culture and shared decision making structures Professional learning community with major focus on academics Collaborative and supportive environment Strong sense of trust and collegiality Cooperative labor-management relations Supportive parent-community-school relations

KEYS and internal Capacity Building and Process Accountability PROCESSES KEY 1 KEY 2 Continuous Assessment for Teaching and Learning KEY 3 Shared Understanding and Commitment to High Goals Open Communication and Collaborative Problem Solving

Continuous Assessment for Teaching and Learning: The Big Picture KEY 3 Continuous Assessment for Teaching and Learning: The Big Picture Culture of data-based decision-making Action research (school and classroom levels) Process and product evaluation of instructional programs Use of multiple data sources to make decisions that affect student learning Use of feedback from stakeholders to shape and modify curricular offerings

KEYS and Internal Capacity Building and Process Accountability PROCESSES KEY 1 KEY 2 KEY 3 Shared Understanding and Commitment to High Goals Open Communication and Collaborative Problem Solving Continuous Assessment for Teaching and Learning KEY 4 Personal & Professional Learning 43

Personal and Professional Learning : The Big Picture KEY 4 Personal and Professional Learning : The Big Picture Quality teaching/ collective efficacy (content and pedagogy and general knowledge) Teacher and staff selection and assignments Professional development including professional community, mentoring, peer assistance Teacher and staff evaluation Compensation, rewards, recognitions

KEYS and Internal Capacity Building and Process Accountability PROCESSES KEY 1 KEY 2 KEY 3 Shared Understanding and Commitment to High Goals Open Communication and Collaborative Problem Solving Continuous Assessment for Teaching and Learning Appropriate Use of Resources for Teaching and Learning KEY 5 Personal & Professional Learning KEY 4

Resources for Teaching and Learning: The Big Picture KEY 5 Resources for Teaching and Learning: The Big Picture Appropriate use of resources with decisions based on quality not price Availability of a broad range of resources: -- Safe and healthy learning environment --Curriculum materials (books, library resources and computer services) --Social, emotional and psychological services --Health and recreational services --Art, music and physical education

KEYS and Internal Capacity Building and Process Accountability PROCESSES KEY 1 KEY 2 KEY 3 Shared Understanding and Commitment to High Goals Open Communication and Collaborative Problem Solving Continuous Assessment for Teaching and Learning KEY 6 Curriculum and Instruction Appropriate Use of Resources for Teaching and Learning Personal & Professional Learning KEY 4 KEY 5

Curriculum and Instruction: The Big Picture KEY 6 Curriculum and Instruction: The Big Picture Challenging content standards Curriculum aligned to the challenging standards Wide range of assessments appropriate to the curriculum Flexible and varied pedagogy to address individual differences among students Appropriate curricular programs responsive to the varying needs of students

Redefining School accountability: A systems Approach KEYS 2.0 as a Measure of Internal and Accountability School Capacity INPUTS OUTCOMES PROCESSES $ Adequate/Equitable Improved Student Learning/ Achievement KEY 1 KEY 2 KEY 3 Shared Understanding and Commitment to High Goals Open Communication and Collaborative Problem Solving Continuous Assessment for Teaching and Learning Higher Graduation Rates(Lower Drop Out Rates) Quantity and Quality of Teaching Pool Readiness of Students to Learn Improved Attendance (Students/Staff) National, State, District Policies Appropriate Use of Resources for Teaching and Learning Increased Satisfaction (Student/Staff/Parent) Personal & Professional Learning Curriculum and Instruction External Supports Partnerships-Community Parent/Business Successful Citizenship Post Sec. Enrollments Satisfying Careers Etc. KEY 4 KEY 5 KEY 6

In Summary Building capacity is essential if a school is to take steps to improve and meet performance based accountability requirements Capacity building must be defined as both resource-based measures and KEYS measures School capacity is also a measure of internal school accountability Schools that have strong internal accountability are more likely to be successful in meeting external requirements KEYS is an effective tool to measure internal capacity and accountability and take steps to improve these constructs

The “Elevator” Speech What is an elevator speech? Purpose? The KEYS take on the “elevator” speech Strategies for developing a KEYS-related elevator speech Practice – practice - practice

Evaluative Criteria for a KEYS “Elevator” Speech The Hook to grab attention Provide clear information Explain what/how KEYS Explain benefits – impact Be passionate, enthusiastic

KEYS Results Reports Staff Report: displays aggregate of school respondents' scores for each of 42 KEYS Indicators and points to resources Family/Parent Report: displays aggregate responses for each question, clustered under 8 themes with comparison to staff Community Report: assesses and displays a subset of the questions from the parent survey, clustered under 6 themes Student Report: displays aggregate student responses to a set of questions clustered under 20 different categories Special “Theme” Reports: display KEYS results composed of indicators from more than one Key around alternative themes: Collaborative Culture School as a Learning Organization Standards-Based Curriculum, Instruction and Assessment Data-Based Decision Making Closing the Achievement Gap District Report: displays aggregate of school responses and distribution of school responses for each of 42 indicators in the staff survey

KEYS Website Report Page http://www.keysonline.org/results/index.html?school_id=826

KEYS Results Reports Staff Report: All Indicators

KEYS Results Reports Staff Report: One Key at a Time

KEYS Results Reports Staff Report: One Indicator at a Time

KEYS Results Reports Staff Report: Questions from One Indicator

KEYS Results Reports Notice the statistics and PDF report.

KEYS Results Reports Staff Report: Multiple Runs

KEYS District Report: Aggregate Scores for Key 1—Multiple School Runs

KEYS District Results for a Single Indicator

KEYS District Results Aggregate Scores for Key 6

KEYS District Results Distribution of School Scores for Indicator 6 KEYS District Results Distribution of School Scores for Indicator 6.5 (Interventions)

KEYS District Results Aggregate Scores for Key 5 (Resources)

KEYS District Results Distribution of School Scores for Indicator 5 KEYS District Results Distribution of School Scores for Indicator 5.4 (Safe & Healthy Learning Environment)

District-Wide KEYS: A Partnership between the District and the Local Association Promotes local association, district, parent and community partnerships and establishes the local as an equal partner to address mutual concerns through the creation of KEYS steering committees Links KEYS to ongoing district school improvement efforts Ensures the commitment and support of all stakeholders through partnership agreements, including appropriate resources to individual schools Establishes procedures, including training and technical assistance that apply to schools across the district KEYS district reports provide valuable information for allocation of technical assistance and resources to schools

Benefits of a KEYS District-Association Partnership A more rational basis for allocating resources and technical assistance to schools. The ability to track patterns of school and district performance over time More effective planning for district-wide improvement strategies The ability to create online professional communities among schools across the district Shared accountability and commitments and improved district-association relations A stronger and more effective local association with greater membership engagement and commitment

How Can KEYS Support Our Priority Schools Provides a framework to establish priorities and supports continuous school improvement Encourages shared understanding and clear directions regarding areas needing improvement Highlights “bright spots” to celebrate and replicate Provides motivation, inspiration, and buy-in for staff & stakeholders to commit to and implement turnaround efforts Supports the internal capacity to make improvement and accountability possible KEYS provides the opportunity to the staff and parents and community members in priority schools to have a voice in how to go about to bring change and improvement to their school rather than having the interventions imposed top down. KEYS helps to establish: Priorities with respect to what needs must be addressed first Shared understanding regarding goals and objectives as well as implementation and monitoring and evaluation plans Opportunities to learn together and celebrate successes Build trust and develop capacity to bring about the needed changes for school improvement