Pennsylvania’s Educator Effectiveness System: PVAAS Teacher Reporting SAS Institute Hershey, PA - December 3, 2012 Welcome and Introductions of the team: Dr. Kristen Lewald, PVAAS Statewide Project Director for PDE Joan Perekupka, PVAAS Statewide Team for PDE Jennifer Ross, PVAAS Statewide Team for PDE Stefani Barbero, Manager, Educator Support, SAS EVAAS® for K – 12 Dr. Kristen Lewald, PVAAS Statewide Project Director for PDE Joan Perekupka, PVAAS Statewide Team for PDE Jennifer Ross, PVAAS Statewide Team for PDE Stefani Barbero, Manager, Educator Support, SAS EVAAS® for K - 12

Session Focus and Plan 75 minute session: The Big Picture and Key Concepts Purpose and Benefits of PVAAS Teacher Reporting PDE’s Plan for Guidance for LEAs PVAAS in the Teacher Effectiveness System Concept of PVAAS Teacher Reporting Which Teachers, Subjects/Courses, Students? Other Variables Being Considered? The Process and Timeline Your Input in the Process Planning and Support Questions/Feedback The intent of the session is to give you the big picture and the key concepts about PVAAS as part of Pennsylvania’s Educator Effectiveness System. Since we have 1 hour and 15 minutes together today, we would like to share the presentation, then take questions - that way we ensure everyone gets to hear all of the content we are planning to present. We will then take questions, some for which we may not have answers as of yet. There are index cards for everyone to use to write down your questions as we work through the session. Our topics for today include: 75 minute session: The Big Picture and Key Concepts Purpose and Benefits of PVAAS Teacher Reporting PDE’s Plan for Guidance for LEAs PVAAS in the Teacher Effectiveness System Concept of PVAAS Teacher Reporting Which Teachers, Subjects/Courses, Students? Other Variables Being Considered? The Process and Timeline Your Input in the Process Planning and Support Questions/Feedback

Your Questions and Comments Index Cards To keep track of your questions during the session We will take questions at the end of the session We may not have answers to all questions If we do not get to your question(s) during the session and you want a reply, include your email address on the index card Give to facilitator We will reply to your question(s) via email Index Cards To keep track of your questions during the session We will take questions at the end of the session We may not have answers to all questions If we do not get to your question during the session and you want a reply, include your email address on the index card Give index cards to facilitator We will reply to your question(s) via email

Benefits of PVAAS Teacher Reporting Teachers yielding high growth can: Mentor beginning teachers Serve as Instructional Coaches Become lead teachers Serve as members of a school-wide planning committee Participate in curricular planning Serve on professional development committees Promote differentiated instruction Tutor students in need of extra support For teachers needing extra support in improving student progress , the district can then: Provide mentors for these teachers Provide support from Instructional Coaches and lead teachers Provide support for planning and preparation Align professional development needs to data Model differentiated instruction Offer opportunities for observing teachers yielding high growth with students Teachers yielding high growth can: Mentor beginning teachers Serve as Instructional Coaches Become lead teachers Serve as members of a school-wide planning committee Participate in curricular planning Serve on professional development committees Promote differentiated instruction Tutor students in need of extra support For teachers needing extra support in improving student progress , the district can then: Provide mentors for these teachers Provide support from Instructional Coaches and lead teachers Provide support for planning and preparation Align professional development needs to data Model differentiated instruction Offer opportunities for observing teachers yielding high growth with students

PDE Guidance PDE is developing statewide policy and guidance for PVAAS Teacher Reporting that fairly represents the unique schedules, instructional programs, and PA certified educator contributions to student growth in the tested subjects/courses where PVAAS reporting is applicable. PDE will provide flexibility and guidance for LEAs to locally address, within the state policy and guidelines, teacher attribution to student growth via PVAAS Teacher Reporting. PDE Guidance: PDE is developing statewide policy and guidance for PVAAS teacher reporting that fairly represents the unique schedules, instructional programs, and PA certified educator contributions to student growth in the tested subjects/courses where PVAAS reporting is applicable. PDE will provide flexibility and guidance for LEAs to locally address, within the state policy and guidelines, teacher attribution to student growth based on PVAAS Teacher Reporting.

PDE Guidance PDE’s policy and guidelines will provide a framework for LEAs to make sure the right teachers are linked to the right students for the right proportion of time to yield PVAAS Teacher Reporting. Teacher PDE’s policy and guidelines will provide a framework for LEAs to make sure the right teachers are linked to the right students for the right proportion of time to yield PVAAS Teacher Reporting. Student Proportion of Time

PDE Guidance The guidelines will address areas such as: Which PA certified educators receive PVAAS Teacher Reporting? Proportion of Time? Student Teacher Which courses/subjects? The guidelines will address areas such as: Which PA certified educators receive PVAAS Teacher Reporting? Proportion of Time? Student Teacher Which courses/subjects? We will discuss these areas today in GENERAL, but MORE specific guidance will be provided by PDE at a later time. Hence all of your questions about these areas will not answered today, but we will discuss what PDE is considering.

PDE’s Plan for Implementing PVAAS Teacher Reporting “Start Small, Get It Right” Pilot the Process in Year 1 All LEAs Details TBD Monitor and Adjust based on Year 1 Pilot Start Small, Get It Right Pilot the Process in Year 1 All LEAs Details to be determined Monitor and Adjust based on Year 1 Pilot Pennsylvania has been monitoring and adjusting as the Danielson framework in the Educator Effectiveness system has been implemented; PA is following the same concept with PVAAS Teacher Reporting.

PVAAS: PDE’s System for Growth PVAAS uses the EVAAS methodology from SAS EVAAS for K-12 SAS EVAAS has been providing teacher level reporting since 1996 EVAAS methodology has been published, research-based, and peer-reviewed SAS EVAAS has 4 statewide implementations of Teacher Reporting (PA, OH, TN, NC) + 24 regional & district implementations across the US PA has been learning from other states and districts in the US PVAAS uses the EVAAS methodology from SAS EVAAS for K-12 SAS EVAAS has been providing teacher level reporting since 1996 EVAAS methodology has been published , research-based, and peer-reviewed SAS EVAAS has 4 statewide implementations of teacher level reporting (PA, OH, TN, NC) + 24 regional/district implementations across the US PA has been learning from other states, districts in the US

Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 Two Resources (Handouts): Summary of EVAAS Research on Teacher Effectiveness Share! District and Building Administrators Curriculum Directors Special Ed Directors HR Staff Content Leaders/Specialists Teachers School Board These are 2 documents- one a shorter version- to share with various audiences- regarding the research and policy implications on value-added analyses and teacher effectiveness - as it relates to student academic progress. Share! District and Building Administrators Curriculum Directors Special Ed Directors HR Staff Content Leaders/Specialists Teachers School Board Questions? Email pdepvaas@iu13.org

PVAAS as Part of the System This session will specifically focus on PVAAS- the 15% portion of the EE system for teachers.

PVAAS Teacher Reporting: When Is it Included in the Educator Effectiveness System? It is included when a teacher has a 3 year average. The 3 year average is a 3 year rolling average Year 1 VA Report Year 2 VA Report Year 3 VA Report Year 4 VA Report This data will be included in the Educator Effectiveness System when a teacher has a 3 year average. The 3 year average is 3 year rolling average [Explain the concept of a 3 year rolling average] 3 Year Avg. VA Report 3 Year Avg. VA Report

What is the PVAAS Teacher Reporting Based Upon? A 3 year average will be based upon 3 Value-Added reports in a tested course/subject A measure of the teacher’s impact on the academic progress of students over time 3 years of performance 3 groups of students A 3 year average will be based upon 3 Value-Added reports in a tested course/subject A measure of the teacher’s impact on the academic progress of students over time 3 years of performance 3 groups of students

PVAAS Teacher Reporting: When Will Reporting Be Available? PVAAS Teacher Reporting will be provided (“curtain opened”) before it will be included IN a teacher’s summative evaluation Single year reporting will be provided before the 3 year average is yielded Single year reporting will start in fall 2013 Who sees it before it “counts”? The Teacher The Teacher’s Administrator(s) Value-added data will “follow” a teacher as he/she moves from school to school, from LEA to LEA, in Pennsylvania (pending regulatory review) PVAAS Teacher Reporting will be provided (“curtain opened”) before it will be included IN a teacher’s summative evaluation Single year reporting will be provided before the 3 year average is yielded Single year reporting will start in fall 2013 Who sees it before it “counts”? The Teacher The Teacher’s Administrator(s) Value-added data will “follow” a teacher as he/she moves from school to school, from LEA to LEA, in Pennsylvania (pending regulatory review)

Concept of Growth for PVAAS teacher Reporting

Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 What is PVAAS? PVAAS Analyses PSSA Writing PSSA Science PSSA Reading PSSA Math LOOKING AHEAD Planning for Students’ Needs: Student Projections to Future Tests LOOKING BACK Evaluating Schooling Effectiveness: Value Added & Diagnostic Reports PVAAS: 2 Types of Info Looking Back (Teacher Reporting) Looking Forward PVAAS is a reporting system based on Value-Added analysis, which is a way of quantifying the progress students make, or the “value” that educators add to student achievement. PVAAS provides a wide range of reports, but they fall into two broad categories: Reports that help educators look back (Value-Added Reports and Performance & Quintile Diagnostic Reports), and reports that help educators look ahead (Student Projections and Projection Summaries). PVAAS Teacher reporting in data for “looking back” and evaluating how effective a teacher was with regard to the academic progress of students. Keystones Questions? Email pdepvaas@iu13.org

Achievement vs. Progress Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 Achievement vs. Progress Student Achievement: Where are we? Student Progress: How far have we come? By concentrating on the progress students make, PVAAS puts the emphasis on what educators can influence! Measuring achievement is important, however only tells us where students are at a specific point in time. It’s a snapshot, one moment in time. Progress tells us how far we’ve come with students. It takes into account where students were academically when they were last assessed. In other words, it gives us a measure of academic progress for students by taking into account BOTH their endpoint and their entering achievement level. Progress depends on what actually happens during the school year. Progress depends on the effectiveness of the instructional program …how well we as educators meet students’ needs. Students arrive at school on day 1 of the school year at different levels of achievement. By concentrating on progress, PVAAS puts the emphasis on what educators CAN influence. You can change the instructional delivery, you can change what happens in the classroom, you can think about what courses students take, what interventions they receive, all of that. You can change the direction for a student; you can change the amount of progress for a group of students. Questions? Email pdepvaas@iu13.org

Regardless of the entering achievement level of the group of students… Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 Regardless of the entering achievement level of the group of students… Students should not lose ground academically A reasonable goal would be to at least maintain the achievement level of the group To make a connection from what you may understand about the current PVAAS district/school value-added reporting to PVAAS teacher value-added reporting-- how many of you have participated in PVAAS session where we talked about the “concept of growth”? – where we described the PA Standard for Academic Growth as maintaining the achievement level for a group of students? We are talking about the concept of growth in the same way with teacher reporting– again, meeting the standard for PA Academic Growth means we have maintained the achievement level for a group of students. AND, Regardless of the entering achievement level of the group of students… Students should not lose ground academically A reasonable goal would be to at least maintain the achievement level of the group All groups of students can make progress or growth All groups of students can make progress or growth

PVAAS Teacher Reporting: Types of Information Teacher Reporting provided by Tested Course/Subject Overall Teacher Value-Added/Growth Report Single Y ear Value-Added 3 Year Avg. Value-Added when available Diagnostic Report for Tested Course/Subject Subgroup Diagnostic Reports for Tested Course/Subject *Italics: Information beyond summative evaluation Teacher Reporting provided by Tested Course/Subject Overall Teacher Value-Added/Growth Report Single Y ear Value-Added 3 Year Avg. Value-Added when available Diagnostic Report for Tested Course/Subject Subgroup Diagnostic Reports for Tested Course/Subject *Italics: Information beyond summative evaluation

Which Courses, Subjects, Teachers, Students? PVAAS Teacher Reporting Which Courses, Subjects, Teachers, Students?

Which Teachers in the Tested Subjects/Courses? A PA certified educator with full or partial responsibility for content specific instruction of the assessed eligible content as measured by a PA state assessment **Additional guidance will follow** A PA certified educator with full or partial responsibility for content specific instruction of the assessed eligible content as measured by a state assessment **Additional guidance will follow**

PVAAS Teacher Reporting: Which Tested Courses/Subjects? PSSA: With/Without Accommodations Grades 4-8: Reading and Math Grades 4 & 8: Science Grades 5 & 8: Writing Future: Transition into ELA Assessment Keystone Exams: With/Without Accommodations Algebra I Literature Biology With and without accommodations PSSA: Grades 4-8: Reading and Math Grade 4, 8: Science Grade 5, 8: Writing Future: Transition into ELA Assessment Keystones: Algebra I Literature Biology PASA is not included in PVAAS analyses, this is due to various factors.

PVAAS Teacher Reporting: Keystone Exams FAQs Which Students Taking the Keystone Exams? Students enrolled in a Keystone-related course when they take the Keystone exam Which Keystone Exams? Retest? After Supplemental Instruction? Only if the student is enrolled in a Keystone-related course when they retake the Keystone exam If a student receives supplemental services/remediation, they are only included in PVAAS Value-Added Reporting IF they are retaking the Keystone-related course, followed by retaking the Keystone exam [Draw attention to Handout/FAQs] Which Students Taking the Keystone Exams? Students enrolled in a Keystone-related course when they take the Keystone exam Which Keystone Exams? Retest? After Supplemental Instruction? Only if the student is enrolled in a Keystone-related course when they take the Keystone exam If a student receives supplemental services/remediation, they are only included in PVAAS Value-Added Reporting IF they are retaking the Keystone-related course, followed by retaking the Keystone exam

Other Variables under Consideration? PVAAS Teacher Reporting Other Variables under Consideration?

What Variables are Being Considered? PDE is taking into consideration these facts: All students do not arrive on the first day of school or the first day of a course All students do not stay until the last day of school or the last day of a course All teachers do not start on the first day of school or the first day of a course All teachers do not stay until the last day of school or the last day of a course **Proportion of Time: Additional guidance will follow** PDE is taking into consideration these facts: All students do not arrive on the first day of school or the first day of a course All students do not stay until the last day of school or the last day of a course All teachers do not start on the first day of school or the first day of a course All teachers do not stay until the last day of school or the last day of a course **Additional guidance will follow**

What Variables are Being Considered? PDE is taking into consideration: Proportion of Instructional Responsibility Full or partial responsibility Examples: Co-teaching Team Teaching General Education Teacher + Special Education Teacher Etc. **% Instructional Responsibility: Additional guidance will follow** PDE is taking into consideration: Proportion of Instructional Responsibility: Full or partial responsibility Examples: Co-teaching Team Teaching General Education Teacher + Special Education Teacher **Additional guidance will follow**

Other Variables Under Consideration Demographic Variables Economically Disadvantaged English Language Learners Student with IEPs SAS EVAAS Analyses for PDE To determine if there is a need to make any adjustments based on demographic variables Using actual PA teacher level data To identify and address any unintended consequences Demographic Variables Economically Disadvantaged English Language Learners Student with IEPs SAS EVAAS Analyses for PDE To determine if there is a need to make any adjustments based on demographic variables Using actual PA teacher level data To identify and address any unintended consequences

PVAAS Teacher Reporting The process

Year 1 Pilot: 2012-2013 Timeline Winter 2012-2013 Spring 2013 Develop PDE Guidance Documents Customize PVAAS Web-based System for Teacher Reporting PIMS Pre-population of PVAAS Roster Verification System Spring 2013 Roster Verification Window for LEAs via Web-based System Teachers Administrators – Building and District Fall 2013 Release of First PVAAS Teacher Reporting Winter 2012-2013 Develop PDE Guidance Documents Customize PVAAS Web-based System for Teacher Reporting PIMS Pre-population of PVAAS Roster Verification System Spring 2013 Roster Verification Window for LEAs via Web-based System Teachers Administrators – Building and District Fall 2013 Release of First PVAAS Teacher Reporting

Professional Development & Support Roster Verification Process Statewide Webinars Virtual Learning Modules Guidance Documents FAQs PVAAS Teacher Reporting Live sessions across Pennsylvania Help Menus Professional Development and Support: Roster Verification Process Statewide Webinars Virtual Learning Modules Guidance Documents FAQs PVAAS Teacher Reporting Live sessions across Pennsylvania Help Menus

New Term: Roster Verification Goal: To provide PVAAS Teacher Reporting based on accurate data Roster Verification: A process for teachers and administrators to make sure the right teachers are linked to the right students for the right proportion of time  This is a local process! A secure, web-based system will be provided to LEAs for this process Double level login security to ensure security of Teacher Reporting Let’s address this new term we are using – Roster Verification The goal is to provide PVAAS Teacher Reporting based on accurate data. Roster Verification is a local process! It is a process for teachers and administrators to make sure the right teachers are linked to the right students for the right proportion of time. A secure, web-based system will be provided to LEAs for this process. Double level login security to ensure security of Teacher Reporting

Your Role Is Foundational Input Into Roster Verification Input from the teachers and the administrators in the roster verification process to ensure accurate data for PVAAS Teacher Reporting Engagement in the roster verification process Locally determined, based on PDE Guidance Release of PVAAS Teacher Reporting Drill-down to see which students and the proportion of time Transparency Student used in reporting Proportion of time %Instructional Responsibility Input Into Roster Verification Input from the teachers and the administrators in the roster verification process to ensure accurate data for PVAAS Teacher Reporting Engagement in the roster verification process Locally determined, based on PDE Guidance Release of PVAAS Teacher Reporting Drill-down to see which students and the proportion of time Transparency Student used in reporting Proportion of time %Instructional Responsibility

PVAAS Teacher Reporting Preparing Your Staff

How Can You Prepare Your Staff? Teachers and Administrators Include teachers in tested AND non-tested courses/subjects Are they aware of the PA Educator Effectiveness System? Are they aware of the role of PVAAS in the Educator Effectiveness System? Information from this session Teachers and Administrators: Including teachers in tested AND non-tested courses/subjects Are they aware of PA Educator Effectiveness System? Are they aware of the role of PVAAS in the Educator Effectiveness System? (In other words, the information from this session today)

How Can You Prepare Your Staff? Do they understand the current PVAAS reporting? Concept of Measuring Growth PVAAS School Value-Added and Diagnostic Reporting Resources Archived webinars Virtual Learning Modules IU PVAAS Contact(s) PVAAS Statewide Team for PDE Do they understand the current PVAAS reporting? Concept of Measuring Growth PVAAS School Value-Added and Diagnostic Reporting Resources Archived webinars Virtual Learning Modules IU PVAAS Contact(s) PVAAS Statewide Team for PDE

Understanding PVAAS District/School Reporting to Teacher Reporting Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 Understanding PVAAS District/School Reporting to Teacher Reporting District/School Value-Added  Teacher Value-Added District/School Performance Diagnostic Report  Diagnostic Report If your teachers understand the District/School Value-Added Reports, they will be much better prepared to understand their own Teacher Value-Added Reports. If your teachers understand the District/School Diagnostic Reports, they will be much better prepared to understand their own Teacher Diagnostic Reports. Questions? Email pdepvaas@iu13.org

Teachers: Access, Understanding, and Use of Current PVAAS Reporting Grade Level PVAAS Reports for Team Planning School Level PVAAS Reports for School-level Planning District Level PVAAS Reports for District-level Planning PVAAS Student Projection Reports for Student-level Planning Do your teachers NOW have access to these existing PVAAS reports? Do they understand them? Know how to use them? Grade Level PVAAS for Team Planning School Level PVAAS for School-level Planning District Level PVAAS for District-level Planning PVAAS Student Projections for Student-level Planning

Feedback? Questions?

Protocols for Principals & Teachers to Put PVAAS to Use Fall 2012 Questions? PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com Questions related to the statewide implementation of PVAAS or PVAAS materials can be directed to pdepvaas@iu13.org, or 717-606-1911. Remember to go to https://pvaas.sas.com to access your current PVAAS reports. Questions? Email pdepvaas@iu13.org