1. Pennsylvania’s Educator Effectiveness System: PVAAS Teacher Reporting
SAS Institute
Hershey, PA - December 3, 2012
Welcome and Introductions of the team:
Dr. Kristen Lewald, PVAAS Statewide Project Director for PDE
Joan Perekupka, PVAAS Statewide Team for PDE
Jennifer Ross, PVAAS Statewide Team for PDE
Stefani Barbero, Manager, Educator Support, SAS EVAAS® for K – 12
Dr. Kristen Lewald, PVAAS Statewide Project Director for PDE
Joan Perekupka, PVAAS Statewide Team for PDE
Jennifer Ross, PVAAS Statewide Team for PDE
Stefani Barbero, Manager, Educator Support, SAS EVAAS® for K - 12

2. Session Focus and Plan
75 minute session: The Big Picture and Key Concepts
Purpose and Benefits of PVAAS Teacher Reporting
PDE’s Plan for Guidance for LEAs
PVAAS in the Teacher Effectiveness System
Concept of PVAAS Teacher Reporting
Which Teachers, Subjects/Courses, Students?
Other Variables Being Considered?
The Process and Timeline
Your Input in the Process
Planning and Support
Questions/Feedback
The intent of the session is to give you the big picture and the key concepts about PVAAS as part of Pennsylvania’s Educator Effectiveness System.
Since we have 1 hour and 15 minutes together today, we would like to share the presentation, then take questions - that way we ensure everyone gets to hear all of the content we are planning to present. We will then take questions, some for which we may not have answers as of yet. There are index cards for everyone to use to write down your questions as we work through the session.
Our topics for today include:
75 minute session: The Big Picture and Key Concepts
Purpose and Benefits of PVAAS Teacher Reporting
PDE’s Plan for Guidance for LEAs
PVAAS in the Teacher Effectiveness System
Concept of PVAAS Teacher Reporting
Which Teachers, Subjects/Courses, Students?
Other Variables Being Considered?
The Process and Timeline
Your Input in the Process
Planning and Support
Questions/Feedback

3. Your Questions and Comments
Index Cards
To keep track of your questions during the session
We will take questions at the end of the session
We may not have answers to all questions
If we do not get to your question(s) during the session and you want a reply, include your address on the index card
Give to facilitator
We will reply to your question(s) via
Index Cards
To keep track of your questions during the session
We will take questions at the end of the session
We may not have answers to all questions
If we do not get to your question during the session and you want a reply, include your address on the index card
Give index cards to facilitator
We will reply to your question(s) via

4. Benefits of PVAAS Teacher Reporting
Teachers yielding high growth can:
Mentor beginning teachers
Serve as Instructional Coaches
Become lead teachers
Serve as members of a school-wide planning committee
Participate in curricular planning
Serve on professional development committees
Promote differentiated instruction
Tutor students in need of extra support
For teachers needing extra support in improving student progress , the district can then:
Provide mentors for these teachers
Provide support from Instructional Coaches and lead teachers
Provide support for planning and preparation
Align professional development needs to data
Model differentiated instruction
Offer opportunities for observing teachers yielding high growth with students
Teachers yielding high growth can:
Mentor beginning teachers
Serve as Instructional Coaches
Become lead teachers
Serve as members of a school-wide planning committee
Participate in curricular planning
Serve on professional development committees
Promote differentiated instruction
Tutor students in need of extra support
For teachers needing extra support in improving student progress , the district can then:
Provide mentors for these teachers
Provide support from Instructional Coaches and lead teachers
Provide support for planning and preparation
Align professional development needs to data
Model differentiated instruction
Offer opportunities for observing teachers yielding high growth with students

5. PDE Guidance
PDE is developing statewide policy and guidance for PVAAS Teacher Reporting that fairly represents the unique schedules, instructional programs, and PA certified educator contributions to student growth in the tested subjects/courses where PVAAS reporting is applicable.
PDE will provide flexibility and guidance for LEAs to locally address, within the state policy and guidelines, teacher attribution to student growth via PVAAS Teacher Reporting.
PDE Guidance:
PDE is developing statewide policy and guidance for PVAAS teacher reporting that fairly represents the unique schedules, instructional programs, and PA certified educator contributions to student growth in the tested subjects/courses where PVAAS reporting is applicable.
PDE will provide flexibility and guidance for LEAs to locally address, within the state policy and guidelines, teacher attribution to student growth based on PVAAS Teacher Reporting.

6. PDE Guidance
PDE’s policy and guidelines will provide a framework for LEAs to make sure the right teachers are linked to the right students for the right proportion of time to yield PVAAS Teacher Reporting.
Teacher
PDE’s policy and guidelines will provide a framework for LEAs to make sure the right teachers are linked to the right students for the right proportion of time to yield PVAAS Teacher Reporting.
Student
Proportion
of Time

7. PDE Guidance The guidelines will address areas such as:
Which PA certified educators receive PVAAS Teacher Reporting?
Proportion of Time?
Student
Teacher
Which courses/subjects?
The guidelines will address areas such as:
Which PA certified educators receive PVAAS Teacher Reporting?
Proportion of Time?
Student
Teacher
Which courses/subjects?
We will discuss these areas today in GENERAL, but MORE specific guidance will be provided by PDE at a later time.
Hence all of your questions about these areas will not answered today, but we will discuss what PDE is considering.

8. PDE’s Plan for Implementing PVAAS Teacher Reporting
“Start Small, Get It Right”
Pilot the Process in Year 1
All LEAs
Details TBD
Monitor and Adjust based on Year 1 Pilot
Start Small, Get It Right
Pilot the Process in Year 1
All LEAs
Details to be determined
Monitor and Adjust based on Year 1 Pilot
Pennsylvania has been monitoring and adjusting as the Danielson framework in the Educator Effectiveness system has been implemented; PA is following the same concept with PVAAS Teacher Reporting.

9. PVAAS: PDE’s System for Growth
PVAAS uses the EVAAS methodology from SAS EVAAS for K-12
SAS EVAAS has been providing teacher level reporting since 1996
EVAAS methodology has been published, research-based, and peer-reviewed
SAS EVAAS has 4 statewide implementations of Teacher Reporting (PA, OH, TN, NC) + 24 regional & district implementations across the US
PA has been learning from other states and districts in the US
PVAAS uses the EVAAS methodology from SAS EVAAS for K-12
SAS EVAAS has been providing teacher level reporting since 1996
EVAAS methodology has been published , research-based, and peer-reviewed
SAS EVAAS has 4 statewide implementations of teacher level reporting (PA, OH, TN, NC) + 24 regional/district implementations across the US
PA has been learning from other states, districts in the US

10. Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
Two Resources (Handouts): Summary of EVAAS Research on Teacher Effectiveness
Share!
District and Building Administrators
Curriculum Directors
Special Ed Directors
HR Staff
Content Leaders/Specialists
Teachers
School Board
These are 2 documents- one a shorter version- to share with various audiences- regarding the research and policy implications on value-added analyses and teacher effectiveness - as it relates to student academic progress.
Share!
District and Building Administrators
Curriculum Directors
Special Ed Directors
HR Staff
Content Leaders/Specialists
Teachers
School Board
Questions?

11. PVAAS as Part of the System
This session will specifically focus on PVAAS- the 15% portion of the EE system for teachers.

12. PVAAS Teacher Reporting: When Is it Included in the Educator Effectiveness System?
It is included when a teacher has a 3 year average.
The 3 year average is a 3 year rolling average
Year 1
VA Report
Year 2
VA Report
Year 3
VA Report
Year 4
VA Report
This data will be included in the Educator Effectiveness System when a teacher has a 3 year average.
The 3 year average is 3 year rolling average
[Explain the concept of a 3 year rolling average]
3 Year Avg.
VA Report
3 Year Avg.
VA Report

13. What is the PVAAS Teacher Reporting Based Upon?
A 3 year average will be based upon 3 Value-Added reports in a tested course/subject
A measure of the teacher’s impact on the academic progress of students over time
3 years of performance
3 groups of students
A 3 year average will be based upon 3 Value-Added reports in a tested course/subject
A measure of the teacher’s impact on the academic progress of students over time
3 years of performance
3 groups of students

14. PVAAS Teacher Reporting: When Will Reporting Be Available?
PVAAS Teacher Reporting will be provided (“curtain opened”) before it will be included IN a teacher’s summative evaluation
Single year reporting will be provided before the 3 year average is yielded
Single year reporting will start in fall 2013
Who sees it before it “counts”?
The Teacher
The Teacher’s Administrator(s)
Value-added data will “follow” a teacher as he/she moves from school to school, from LEA to LEA, in Pennsylvania (pending regulatory review)
PVAAS Teacher Reporting will be provided (“curtain opened”) before it will be included IN a teacher’s summative evaluation
Single year reporting will be provided before the 3 year average is yielded
Single year reporting will start in fall 2013
Who sees it before it “counts”?
The Teacher
The Teacher’s Administrator(s)
Value-added data will “follow” a teacher as he/she moves from school to school, from LEA to LEA, in Pennsylvania (pending regulatory review)

15. Concept of Growth for PVAAS teacher Reporting

16. Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
What is PVAAS?
PVAAS Analyses
PSSA Writing
PSSA Science
PSSA Reading
PSSA Math
LOOKING AHEAD
Planning for Students’ Needs:
Student Projections to Future Tests
LOOKING BACK
Evaluating Schooling Effectiveness:
Value Added & Diagnostic Reports
PVAAS: 2 Types of Info
Looking Back (Teacher Reporting)
Looking Forward
PVAAS is a reporting system based on Value-Added analysis, which is a way of quantifying the progress students make, or the “value” that educators add to student achievement. PVAAS provides a wide range of reports, but they fall into two broad categories: Reports that help educators look back (Value-Added Reports and Performance & Quintile Diagnostic Reports), and reports that help educators look ahead (Student Projections and Projection Summaries).
PVAAS Teacher reporting in data for “looking back” and evaluating how effective a teacher was with regard to the academic progress of students.
Keystones
Questions?

17. Achievement vs. Progress
Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
Achievement vs. Progress
Student Achievement: Where are we? Student Progress: How far have we come? By concentrating on the progress students make, PVAAS puts the emphasis on what educators can influence!
Measuring achievement is important, however only tells us where students are at a specific point in time. It’s a snapshot, one moment in time.
Progress tells us how far we’ve come with students. It takes into account where students were academically when they were last assessed. In other words, it gives us a measure of academic progress for students by taking into account BOTH their endpoint and their entering achievement level.
Progress depends on what actually happens during the school year. Progress depends on the effectiveness of the instructional program …how well we as educators meet students’ needs.
Students arrive at school on day 1 of the school year at different levels of achievement. By concentrating on progress, PVAAS puts the emphasis on what educators CAN influence. You can change the instructional delivery, you can change what happens in the classroom, you can think about what courses students take, what interventions they receive, all of that. You can change the direction for a student; you can change the amount of progress for a group of students.
Questions?

18. Regardless of the entering achievement level of the group of students…
Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
Regardless of the entering achievement level of the group of students…
Students should not lose ground academically
A reasonable goal would be to at least maintain the achievement level of the group
To make a connection from what you may understand about the current PVAAS district/school value-added reporting to PVAAS teacher value-added reporting-- how many of you have participated in PVAAS session where we talked about the “concept of growth”?
– where we described the PA Standard for Academic Growth as maintaining the achievement level for a group of students?
We are talking about the concept of growth in the same way with teacher reporting– again, meeting the standard for PA Academic Growth means we have maintained the achievement level for a group of students.
AND, Regardless of the entering achievement level of the group of students…
Students should not lose ground academically
A reasonable goal would be to at least maintain the achievement level of the group
All groups of students can make progress or growth
All groups of students can make progress or growth

19. PVAAS Teacher Reporting: Types of Information
Teacher Reporting provided by Tested Course/Subject
Overall Teacher Value-Added/Growth Report
Single Y ear Value-Added
3 Year Avg. Value-Added when available
Diagnostic Report for Tested Course/Subject
Subgroup Diagnostic Reports for Tested Course/Subject
*Italics: Information beyond summative evaluation
Teacher Reporting provided by Tested Course/Subject
Overall Teacher Value-Added/Growth Report
Single Y ear Value-Added
3 Year Avg. Value-Added when available
Diagnostic Report for Tested Course/Subject
Subgroup Diagnostic Reports for Tested Course/Subject
*Italics: Information beyond summative evaluation

20. Which Courses, Subjects, Teachers, Students?
PVAAS Teacher Reporting
Which Courses, Subjects, Teachers, Students?

21. Which Teachers in the Tested Subjects/Courses?
A PA certified educator with full or partial responsibility for content specific instruction of the assessed eligible content as measured by a PA state assessment
**Additional guidance will follow**
A PA certified educator with full or partial responsibility for content specific instruction of the assessed eligible content as measured by a state assessment
**Additional guidance will follow**

22. PVAAS Teacher Reporting: Which Tested Courses/Subjects?
PSSA: With/Without Accommodations
Grades 4-8: Reading and Math
Grades 4 & 8: Science
Grades 5 & 8: Writing
Future: Transition into ELA Assessment
Keystone Exams: With/Without Accommodations
Algebra I
Literature
Biology
With and without accommodations
PSSA:
Grades 4-8: Reading and Math
Grade 4, 8: Science
Grade 5, 8: Writing
Future: Transition into ELA Assessment
Keystones:
Algebra I
Literature
Biology
PASA is not included in PVAAS analyses, this is due to various factors.

23. PVAAS Teacher Reporting: Keystone Exams
FAQs
Which Students Taking the Keystone Exams?
Students enrolled in a Keystone-related course when they take the Keystone exam
Which Keystone Exams?
Retest? After Supplemental Instruction?
Only if the student is enrolled in a Keystone-related course when they retake the Keystone exam
If a student receives supplemental services/remediation, they are only included in PVAAS Value-Added Reporting IF they are retaking the Keystone-related course, followed by retaking the Keystone exam
[Draw attention to Handout/FAQs]
Which Students Taking the Keystone Exams?
Students enrolled in a Keystone-related course when they take the Keystone exam
Which Keystone Exams?
Retest? After Supplemental Instruction?
Only if the student is enrolled in a Keystone-related course when they take the Keystone exam
If a student receives supplemental services/remediation, they are only included in PVAAS Value-Added Reporting IF they are retaking the Keystone-related course, followed by retaking the Keystone exam

24. Other Variables under Consideration?
PVAAS Teacher Reporting
Other Variables under Consideration?

25. What Variables are Being Considered?
PDE is taking into consideration these facts:
All students do not arrive on the first day of school or the first day of a course
All students do not stay until the last day of school or the last day of a course
All teachers do not start on the first day of school or the first day of a course
All teachers do not stay until the last day of school or the last day of a course
**Proportion of Time: Additional guidance will follow**
PDE is taking into consideration these facts:
All students do not arrive on the first day of school or the first day of a course
All students do not stay until the last day of school or the last day of a course
All teachers do not start on the first day of school or the first day of a course
All teachers do not stay until the last day of school or the last day of a course
**Additional guidance will follow**

26. What Variables are Being Considered?
PDE is taking into consideration:
Proportion of Instructional Responsibility
Full or partial responsibility
Examples:
Co-teaching
Team Teaching
General Education Teacher + Special Education Teacher
Etc.
**% Instructional Responsibility:
Additional guidance will follow**
PDE is taking into consideration:
Proportion of Instructional Responsibility:
Full or partial responsibility
Examples:
Co-teaching
Team Teaching
General Education Teacher + Special Education Teacher
**Additional guidance will follow**

27. Other Variables Under Consideration
Demographic Variables
Economically Disadvantaged
English Language Learners
Student with IEPs
SAS EVAAS Analyses for PDE
To determine if there is a need to make any adjustments based on demographic variables
Using actual PA teacher level data
To identify and address any unintended consequences
Demographic Variables
Economically Disadvantaged
English Language Learners
Student with IEPs
SAS EVAAS Analyses for PDE
To determine if there is a need to make any adjustments based on demographic variables
Using actual PA teacher level data
To identify and address any unintended consequences

28. PVAAS Teacher Reporting
The process

29. Year 1 Pilot: 2012-2013 Timeline Winter 2012-2013 Spring 2013
Develop PDE Guidance Documents
Customize PVAAS Web-based System for Teacher Reporting
PIMS Pre-population of PVAAS Roster Verification System
Spring 2013
Roster Verification Window for LEAs via Web-based System
Teachers
Administrators – Building and District
Fall 2013
Release of First PVAAS Teacher Reporting
Winter
Develop PDE Guidance Documents
Customize PVAAS Web-based System for Teacher Reporting
PIMS Pre-population of PVAAS Roster Verification System
Spring 2013
Roster Verification Window for LEAs via Web-based System
Teachers
Administrators – Building and District
Fall 2013
Release of First PVAAS Teacher Reporting

30. Professional Development & Support
Roster Verification Process
Statewide Webinars
Virtual Learning Modules
Guidance Documents
FAQs
PVAAS Teacher Reporting
Live sessions across Pennsylvania
Help Menus
Professional Development and Support:
Roster Verification Process
Statewide Webinars
Virtual Learning Modules
Guidance Documents
FAQs
PVAAS Teacher Reporting
Live sessions across Pennsylvania
Help Menus

31. New Term: Roster Verification
Goal: To provide PVAAS Teacher Reporting based on accurate data
Roster Verification: A process for teachers and administrators to make sure the right teachers are linked to the right students for the right proportion of time  This is a local process!
A secure, web-based system will be provided to LEAs for this process
Double level login security to ensure security of Teacher Reporting
Let’s address this new term we are using – Roster Verification
The goal is to provide PVAAS Teacher Reporting based on accurate data.
Roster Verification is a local process! It is a process for teachers and administrators to make sure the right teachers are linked to the right students for the right proportion of time.
A secure, web-based system will be provided to LEAs for this process.
Double level login security to ensure security of Teacher Reporting

32. Your Role Is Foundational
Input Into Roster Verification
Input from the teachers and the administrators in the roster verification process to ensure accurate data for PVAAS Teacher Reporting
Engagement in the roster verification process
Locally determined, based on PDE Guidance
Release of PVAAS Teacher Reporting
Drill-down to see which students and the proportion of time
Transparency
Student used in reporting
Proportion of time
%Instructional Responsibility
Input Into Roster Verification
Input from the teachers and the administrators in the roster verification process to ensure accurate data for PVAAS Teacher Reporting
Engagement in the roster verification process
Locally determined, based on PDE Guidance
Release of PVAAS Teacher Reporting
Drill-down to see which students and the proportion of time
Transparency
Student used in reporting
Proportion of time
%Instructional Responsibility

33. PVAAS Teacher Reporting
Preparing Your Staff

34. How Can You Prepare Your Staff?
Teachers and Administrators
Include teachers in tested AND non-tested courses/subjects
Are they aware of the PA Educator Effectiveness System?
Are they aware of the role of PVAAS in the Educator Effectiveness System?
Information from this session
Teachers and Administrators:
Including teachers in tested AND non-tested courses/subjects
Are they aware of PA Educator Effectiveness System?
Are they aware of the role of PVAAS in the Educator Effectiveness System? (In other words, the information from this session today)

35. How Can You Prepare Your Staff?
Do they understand the current PVAAS reporting?
Concept of Measuring Growth
PVAAS School Value-Added and Diagnostic Reporting
Resources
Archived webinars
Virtual Learning Modules
IU PVAAS Contact(s)
PVAAS Statewide Team for PDE
Do they understand the current PVAAS reporting?
Concept of Measuring Growth
PVAAS School Value-Added and Diagnostic Reporting
Resources
Archived webinars
Virtual Learning Modules
IU PVAAS Contact(s)
PVAAS Statewide Team for PDE

36. Understanding PVAAS District/School Reporting to Teacher Reporting
Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
Understanding PVAAS District/School Reporting to Teacher Reporting
District/School Value-Added  Teacher Value-Added
District/School Performance Diagnostic Report  Diagnostic Report
If your teachers understand the District/School Value-Added Reports, they will be much better prepared to understand their own Teacher Value-Added Reports.
If your teachers understand the District/School Diagnostic Reports, they will be much better prepared to understand their own Teacher Diagnostic Reports.
Questions?

37. Teachers: Access, Understanding, and Use of Current PVAAS Reporting
Grade Level PVAAS Reports for Team Planning
School Level PVAAS Reports for School-level Planning
District Level PVAAS Reports for District-level Planning
PVAAS Student Projection Reports for Student-level Planning
Do your teachers NOW have access to these existing PVAAS reports?
Do they understand them?
Know how to use them?
Grade Level PVAAS for Team Planning
School Level PVAAS for School-level Planning
District Level PVAAS for District-level Planning
PVAAS Student Projections for Student-level Planning

38. Feedback? Questions?

39. Protocols for Principals & Teachers to Put PVAAS to Use
Fall 2012
Questions? PVAAS Materials or Statewide Implementation PVAAS Report Web Site
Questions related to the statewide implementation of PVAAS or PVAAS materials can be directed to or
Remember to go to to access your current PVAAS reports.
Questions?