1. Pennsylvania Value-Added Assessment System Overview: PVAAS
Hatboro-Horsham School District
February 15, 2013

2. Session Essential Questions:
During this session, you will begin to form answers to these questions:
What is PVAAS? What is a growth model?
How is PVAAS data different from PSSA or Keystone data?
How will this impact my yearly evaluation?
What resources are available if I’d like to learn more?

3. Measuring Growth, or Progress
Students don’t all start the year at the same place academically
If we are going to measure the academic progress that students make, we have to consider more than proficiency level. In addition, it’s critical to recognize that students don’t all start the year at the same place academically.
PVAAS + Other Data Consultation 2012
PVAAS Statewide Team for PDE June 2012

4. Measuring Growth or Progress
4/15/2017
Measuring Growth or Progress
Standard for PA Academic Growth
Regardless of their entering achievement level, students should not lose ground academically
Is it not a reasonable goal to maintain the achievement level of a group of students?
Not necessarily increase their achievement level, but maintain their achievement level.
In Pennsylvania, growth is assessed against the Standard for PA Academic Growth. It is based on the philosophy that regardless of the entering achievement level of a group of students, they should not lose ground academically.
So we ask, is it not a reasonable goal to maintain the achievement level of a group of students?
Is that not a reasonable expectation?
In other words, if students are fairly high achieving, is it not reasonable to expect that we, as educators, keep them fairly high achieving?
And if students are low achieving, is it not reasonable to expect that we would at least keep those students at that level of achievement (and not have them slip further behind)?
We would hope that everyone would agree that this is a reasonable expectation.
PVAAS + Other Data Consultation 2012

5. PVAAS Growth Measure & Color Indicator
4/15/2017
PVAAS Growth Measure & Color Indicator
PVAAS yields a Growth Measure AND a Growth Color Indicator.
The Growth Measure represents the amount of growth a group of students has made.
The Growth Color Indicator represents the level of evidence we have in the data to say whether or not the group of students increased, decreased, or maintained their achievement levels on average.
PVAAS Value Added reports yield a Growth Measure and a Growth Color Indicator.
[click] The Growth Measure represents the amount of growth a group of students has made.
[click] The Growth Color Indicator represents the level of evidence we have in the data to say whether or not the group of students increased, decreased, or maintained their achievement levels on average. In other words, the color tells us how sure we are that the group of students increased, decreased, or maintained their achievement level on average.
PVAAS + Other Data Consultation 2012

6. Concept of Measuring Growth in PVAAS – Grades 4-8 Math & Reading
4/15/2017
Concept of Measuring Growth in PVAAS – Grades 4-8 Math & Reading
Prior Achievement for Group
Most Recent School Year for Group
Higher Achievement
When the PSSA is administered in consecutive grade levels as it if in reading and math for grades 3-8, PVAAS uses a growth standard method to measure the growth or progress of a group of students.
Let’s consider this group of students. Each year, these students are tested with the PSSA. Each dot represents a student in the group. Those students are going to score along the achievement spectrum from low to high achievement. To estimate the achievement level of this group of students,
[click] PVAAS uses all of the student level data from all years, from all grades, in both Reading and Math from each year this group of students was tested:
[click] to estimate the average achievement level of the group.
[click] After the next round of testing, PVAAS uses all the prior data for the group
[click] and adds the data from the most recent round of testing
[click] to get a new estimate of the average achievement level of the group. Now we can compare the prior achievement level of the group to the new achievement level of the group
[CLICK If the evidence indicates the achievement is roughly at the same point within the distribution of scores as the previous year, this would indicate that the group of students maintained their achievement level which in PVAAS would be indicated as Green – evidence that the student group met the standard for PA Academic Growth.
[click] If there is moderate evidence that the average achievement level of this group of students was at a higher point within the distribution, this would indicate that the group of students increased their average achievement level indicated with a Light Blue in PVAAS – moderate evidence that the student group exceeded the standard for PA Academic Growth.
[click] Similarly if there is significant evidence that the average achievement level of this group of students was at a higher point within the distribution, this would indicate that the group of students increased their average achievement level indicated with a Dark Blue in PVAAS – significant evidence that the student group exceeded the standard for PA Academic Growth.
[click] Now let’s consider now what it would look like if the there was moderate evidence that the average achievement level of this group of students was at a lower point within the distribution, this would indicate that the average achievement level of the group of students slipped indicated with a Yellow – moderate evidence that the student group did NOT meet the standard for PA Academic Growth.
[click] Similarly if there was significant evidence that the average achievement level of this group of students was at a lower point within the distribution, this would indicate that the average achievement level of the group of students slipped indicated with a Red – significant evidence that the student group did NOT meet the standard for PA Academic Growth.
Higher Achievement
Lower Achievement
Lower Achievement
- Represents the mean achievement level of the group of students
PVAAS + Other Data Consultation 2012

7. PVAAS + Other Data Consultation 2012
4/15/2017
Concept of Measuring Growth in PVAAS Science, Writing, Grades 9-11 Math & Reading
Prior Achievement
When the PSSA is not administered in consecutive grade levels as in science, writing, or grades 9-11 math and reading, PVAAS uses a predictive method to measure the growth or progress of a group of students.
Let’s consider this group of students who were last tested in 8th grade. Each dot represents a student in the group. Those students are going to score along the achievement spectrum from low to high achievement.
[click] PVAAS uses all of the student’s prior data to predict the scaled score or achievement level for a student on the 11th grade PSSA.
[click] Then the predicted scaled scores for all students are used
[click] to calculate the mean predicted achievement level for the group.
[click] Next, these students were tested again with the PSSA in grade 11 again the students will score along the achievement spectrum from low to high achievement.
[click] The mean scaled score or achievement level is calculated for the group.
[CLICK If the evidence indicates the achievement is roughly at the level that what was predicted, this would indicate that the group of students maintained their achievement level which in PVAAS would be indicated as Green – evidence that the student group met the standard for PA Academic Growth.
[click] If there is moderate evidence that the average achievement level of this group of students was higher than predicted, this would indicate that the group of students increased their average achievement level indicated with a Light Blue in PVAAS – moderate evidence that the student group exceeded the standard for PA Academic Growth.
[click] Similarly if there is significant evidence that the average achievement level of this group of students was higher than predicted, this would indicate that the group of students increased their average achievement level indicated with a Dark Blue in PVAAS – significant evidence that the student group exceeded the standard for PA Academic Growth.
[click] Now let’s consider now what it would look like if the there was moderate evidence that the average achievement level of this group of students was lower than predicted, this would indicate that the average achievement level of the group of students slipped indicated with a Yellow – moderate evidence that the student group did NOT meet the standard for PA Academic Growth.
[click] Similarly if there was significant evidence that the average achievement level of this group of students was lower than predicted, this would indicate that the average achievement level of the group of students slipped indicated with a Red – significant evidence that the student group did NOT meet the standard for PA Academic Growth.
Higher Achievement
Mean Predicted Achievement
Predicted Achievement
Lower Achievement
- Represents the mean predicted achievement level of the group of students
- Represents the mean achievement level of the group of students
PVAAS + Other Data Consultation 2012

8. Schools with Lower Achievement
4/15/2017
Schools with Lower Achievement
Is a “green” in PVAAS sufficient for moving students to higher levels of achievement?
Prior Achievement for Group
Most Recent School Year for Group
Higher Achievement
Higher Achievement
Let’s now consider how this concept applies for a school where the students are lower achieving.
[click] Remember, to estimate the achievement level of this group of students, PVAAS uses all of the student level data from all years, from all grades, in both Reading and Math from each year this group of students was tested:
[click] to estimate the average achievement level of the group.
[click] After the next round of testing, PVAAS uses all the prior data for the group
[click] and adds the data from the most recent round of testing for the group
[click] to get a new estimate of the average achievement level of the group. Now we can compare the prior achievement level of the group to the new achievement level of the group
[CLICK If the achievement is roughly at the same point within the distribution of scores as the previous year, this would indicate that there is evidence that the group of students maintained their achievement level which in PVAAS would be indicated as Green.
[CLICK] So we ask……is a “green” in PVAAS sufficient for moving students to higher levels of achievement? The good news is that the average achievement level of the group did not slip lower and they met the standard for PA Academic Growth.
However, our goal for this group would be to raise their achievement to a higher level. To do that
[click] we would want see a Light Blue or Dark Blue reported in the Value Added Report indicating there is moderate or significant evidence that the average achievement level of this group of students was at a higher point within the distribution or that the group of students increased their average achievement level.
Again, let’s connect this example to a Value Added Report.
Lower Achievement
Lower Achievement
- Represents the mean achievement level of the group of students
PVAAS + Other Data Consultation 2012

9. Schools with High Achievement
4/15/2017
Schools with High Achievement
Can this group of high achieving students still meet the standard for PA Academic Growth?
Prior Achievement for Group
Most Recent School Year for Group
(100% Advanced)
(100% Advanced)
Higher Achievement
Higher Achievement
Let’s consider how the concept applies for a school where the students are very high achieving.
[click] Perhaps 100% of their students have scored in the Advanced range each year that students have taken the PSSA.
To estimate the achievement level for this group of students,
[click] PVAAS uses all of the student level data from all years, from all grades, in both Reading and Math from each year this group of students was tested:
[click] to estimate the average achievement level of the group.
[click] The question is…Can this group of high achieving students still meet the standard for PA Academic Growth?
[click] After the next round of testing, PVAAS uses all the prior data for the group
[click] and the new testing data
[click] to get a new estimate of the average achievement level of the group. Now we can compare the prior achievement level of the group to the new achievement level of the group
[CLICK If the achievement is roughly at the same point within the distribution of scores as the previous year, this would indicate that there is evidence that the group of students maintained their achievement level which in PVAAS would be indicated as Green.
Would you be satisfied that a group of students who were high achieving, on average maintained that high achievement level? Some people might say “yes”.
[click] However, if the achievement goal for the school or this group would be to raise their achievement to a higher level, then
[click] we would want see a Light Blue or Dark Blue reported in the Value Added Report indicating there is moderate or significant evidence that the average achievement level of this group of students was at a higher point within the distribution or that the group of students increased their average achievement level.
Let’s go back to the growth chart.
Lower Achievement
Lower Achievement
- Represents the mean achievement level of the group of students
PVAAS + Other Data Consultation 2012

10. Measuring Growth at a Deeper Level
4/15/2017
Measuring Growth at a Deeper Level
PVAAS does NOT use the percentages of students at various PSSA performance levels to measure growth.
Instead, estimated growth measures for groups of students in PVAAS are in terms of NCE units (grades 4-8 Math & Reading) or scaled score points (Science, Writing, & grades 9-11 Math & Reading).
So hopefully through this conceptual model, it is clear that:
[click] PVAAS does NOT use the percentages of students at various PSSA performance levels to measure growth.
[click] Instead, estimated growth measures for groups of students in PVAAS are in terms of NCE units (grades 4-8 Math & Reading) or scaled score points (Science, Writing, & grades 9-11 Math & Reading).
PVAAS + Other Data Consultation 2012

11. Regardless of the entering achievement level of the group…
4/15/2017
Regardless of the entering achievement level of the group…
Students should not lose ground academically
A reasonable goal would be to at least maintain the achievement level of the group
So….regardless of the entering achievement level of the group. Whether that achievement level is:
[click] Higher
[click] Middle
[click] or Lower
[click] Students should not lose ground academically or receive a yellow or red color indicator.
[click] A reasonable goal for all groups of students would be to at least maintain the achievement level of the group or receive a green color indicator
[click] And…all groups of students, even higher achieving students can make progress or growth or receive a light blue or dark blue color indicator.
All groups of students can make progress or growth
PVAAS + Other Data Consultation 2012

12. PVAAS + Other Data Consultation 2012
May 2012
C. G. – pg. 2
Practice!
Turn to a partner
Explain the concept of growth
Previous School Year
Current School Year
Higher Achievement
Higher Achievement
[Note to facilitator: Have folks turn to a partner and practice explaining the concept of growth]
Lower Achievement
Lower Achievement
PVAAS + Other Data Consultation 2012
PVAAS Statewide Team for PDE June 2012

13. How is PVAAS data different from the PSSA data I am used to seeing?
Achievement (PSSA and Keystones) vs. Growth (PVAAS)
Achievement – student is measured against a static level of proficiency based on their grade level and rated advanced, proficient, etc. For many students, this may not reflect changes year to year (ex – many students rate “advanced” every year).
Growth – measures student performance relative to that student’s previous performances. Did a student move from below basic to basic? From advanced to proficient? From “low advanced” to “high advanced?” Growth models are more sensitive to individual student progress over time, and compare that progress to an expected standard for growth.
DMW

14. How will this impact my yearly evaluation?

15. Additional Resources Overview of today’s session handouts
The best way to learn more about PVAAS is through exploring the PVAAS site:
There are a number of video learning modules available for viewing. After logging in to the PVAAS site, select the e-learning tab in the top right of the screen.
Other materials – common district document location