Pennsylvania Department of Education PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System

Pennsylvania Department of Education Session Overview History of PVAAS Pennsylvania’s Intent for PVAAS PVAAS Methodology and Reports Timelines for PVAAS Implementation Data Needed for PVAAS Analyses Costs District Preparation for PVAAS Supports

Pennsylvania Department of Education A Focus on Standards

Pennsylvania Department of Education A Standards-based System Goal of high achievement, college readiness Standards aligned Assessments aligned Safety nets aligned Curriculum aligned Instruction aligned Instructional materials aligned

Pennsylvania Department of Education Data Driven Decision Making Tools Provided to Districts Leading for Learning! Getting Ready for the PSSA Getting Results! Data Tools Data Reports PA GROW Context Tools Quality Reviews Tutoring Evaluations PVAAS

Student Planning Process Focus: Classroom of Students Who: Teacher Periodic Grade Level Planning Process Focus: Groups of Students Who: Teacher Teams How: Regular 1-2 hour meetings Annual Building-wide Planning Process Focus: All Students Who: School-wide Team How: Data Retreat, School Planning Process District-Level Support (Budgetary Support, Professional Development, Resources and Time) Student Learning Data School Structures for Data-Informed Decision Making School Level PSSA & PVAAS Standardized Assessments District End-of-Year Tests Final Benchmark Test Classroom Level Initial: PSSA/PVAAS/final tests – student level Cyclical: Benchmark Data – Student Level Continuous Individual Classroom Assessments Progress Monitoring Grade/Course Level Initial: PSSA/PVAAS/final tests – class/subgroup levels Cyclical: Benchmark Data - grade level District quarterly assessments Common Classroom Data Classroom Summaries Demographic/Perceptual/Process Data School Level School Demographics Discipline Data Attendance Data Mobility Rate Grade/Course Level Class Demographics Class Engagement Data Satisfaction Data Attendance Data Classroom Level Qualitative Data Student Historical Information Student Medical Information Student Learning Information © Lancaster-Lebanon IU13

Pennsylvania Department of Education History of PVAAS 2002: Recommended by Pennsylvania League of Urban Schools (PLUS) to State Board of Education 2002: State Board of Education passed resolution for “value-added approach across the Commonwealth” 2002 (SY04-05): 3 Pilot Phases 100 Districts, including the School District of Philadelphia and Pittsburgh City School District

Pennsylvania Department of Education History of PVAAS Fall ‘04 PDE Statewide Workgroup Led by PA Secretary of Education, Dr. Zahorchak 50+ Members Districts Governor’s Office Higher Education Statewide Associations including, but not limited to: PSEA, PFT, PASA, PSBA, PAIU, PAESSP, PASCD Legislators Business and Industry SY05-06 Statewide Implementation Districts 1st Reporting in Fall 06

Pennsylvania Department of Education PA’s Intent for PVAAS Another Tool for Continuous Improvement Measure of Growth to Add to Achievement/Status Measures Not to be Used in Isolation No Teacher Level Analyses

Pennsylvania Department of Education PA’s Intent for PVAAS Integration of PVAAS into existing PDE initiatives: School Improvement Planning Distinguished Educator Training Governor’s Institutes EAP 4Sight Progress Monitoring Content Initiatives: Literacy and Mathematics Response to Intervention PIL PIMS (PA Information Management System)

Pennsylvania Department of Education What is PVAAS?

Pennsylvania Department of Education Student Learning Measures Status Measures PSSA % proficient or not proficient Other local assessment data Improvement Measures Safe Harbor Other local assessment data NEW MEASURE: Progress/Growth Measure PVAAS

Pennsylvania Department of Education Value Added Analysis Provides a measure of what “effect” schooling has in a single year Formally described: Mixed model multivariate longitudinal analyses –The statistical methodology has been reviewed by nationally renowned statisticians since the 1980’s –The reporting has been used by educators for more than 10 years

Pennsylvania Department of Education PVAAS IS NOT : Another test Being used for teacher level accountability in Pennsylvania

Pennsylvania Department of Education PVAAS IS : A statistical analysis of existing assessment data A precise measurement of student progress/growth over time Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans

Pennsylvania Department of Education How is PVAAS analysis done? Year 1 Year 2 Year 3 PSSA PSSA PSSA (7th) 5th 6th Predicted Actual 1.We collect all of the individual student data that is available for a minimum of two years. 2.All of these data are used to develop individual student profiles. Reading Math ?? 1320??

Student A Testing Scores Pool of Testing Histories Computer Prediction Model 1 Prediction Model 2 Prediction Model 6 Prediction Model 3 Prediction Model 4 Prediction Model 5 The PVAAS Methodology – Part 1

The PVAAS Methodology – Part 2 Student A Testing Scores Prediction Model 1 Prediction Model 2 Prediction Model 6 Prediction Model 3 Prediction Model 4 Prediction Model 5 Student D Predicted Score Student E Predicted Score Student A Predicted Score Student B Predicted Score Student F Predicted Score Student C Predicted Score Mean of Student Scores for a particular school Mean of Predicted Scores for a particular school School Effect  Mean of Student Scores – Mean of Predicted Scores

PVAAS Ratings of School Effect Remember: School Effect  Mean of Student Scores – Mean of Predicted Scores So, if the students perform as expected, School Effect  0. The ratings are in terms of the expected value of SE’s 0 +2 SE’s Not Detectably Different AboveBelow Standard Error (SE) is a measure of variation in the prediction

Pennsylvania Department of Education Achievement and Progress

Pennsylvania Department of Education Types of PVAAS Reporting District Level By School School Level By Grade Reading and Math Student Reports Longitudinal Reports “Projections” on Future PSSAs Diagnostic Reports By Quintiles By Performance Levels Diagnostic Subgroup Reports Custom Reporting/Querying

District Summary Report

School Report Above Expected Progress Below Average Achievement

Pennsylvania Department of Education Student Projections

Search for students by subgroup or achievement

Identified students meeting search requirements

Pennsylvania Department of Education How Can PVAAS Help Me?

Pennsylvania Department of Education Key Learning from PVAAS Value-added information will tell you: How much progress students have made in each subject area and grade level How much progress students at different prior achievement levels have made. How students’ progress in one curricular area or program compares to their progress in others

Pennsylvania Department of Education PSSA and PVAAS – The Power of 2! Without data all we have is an opinion! PSSA tells us where a student is. PVAAS tells us where that student is headed. Location and direction are critical partners in the quest for having all students achieve proficiency The data from both achievement and progress measures gives schools valuable information to make data informed decisions.

Pennsylvania Department of Education PA Proposal to USDOE PA submitted proposal to USDOE to use “projections” as means to recalculate proficiency/AYP; did not include value- added reporting as part of proposal PA is waiting to hear from USDOE

Pennsylvania Department of Education Timelines for Implementation Timeline Released in 2002 SY01-02 and SY02-03: Pilot Phase I SY03-04: Pilot Phase II SY04-05: Pilot Phase III SY05-06: Statewide Implementation  All 501 Districts Receive PVAAS Reporting in Fall ‘06  Pilot districts get comprehensive PVAAS reporting  401 Districts Receive Limited Reporting  Grades 4 & 6 SY06-07: Comprehensive Reporting for All Districts

Pennsylvania Department of Education Data Needed for PVAAS PSSA reading and math will be the only data needed for statewide implementation Only common assessment for all 501 districts No data preparation is needed No purchase of any data is needed All PSSA data is sent direct from DRC to SAS, Inc.

Pennsylvania Department of Education Costs Covered by PDE Costs for PSSA Data Submission Costs for PVAAS Analyses Costs for PVAAS Web-Based Reporting Costs for Communication Materials Costs for Professional Development Materials Costs for Professional Development Sessions Available through SY06-07 Regional sessions Videoconferences Webinars

Pennsylvania Department of Education How Can Districts Prepare? Plan to Use PVAAS Data as part of Continuous Improvement Planning District Level Building Level Grade Level Content level Subgroup Level Student Level

Pennsylvania Department of Education How Can Districts Prepare? Professional Development on PVAAS District Administrative Team Building Level Administrators Teacher Leaders Classroom Teachers Other Key Constituents Communications to Board and Community

Pennsylvania Department of Education Supports Communications Kits Posted on PDE Website Professional Development Materials Professional Development Supports Staff Trained: PVAAS Statewide Core Team/PATTAN PDE Staff IU Staff Distinguished Educators PVAAS Statewide Work Group Members

Pennsylvania Department of Education For Information, Resources or Training PVAAS Core Team:

Pennsylvania Department of Education