1. Effectively Communicate PVAAS with your Board and Community Pennsylvania Value-Added Assessment System (PVAAS): Effectively Communicate PVAAS with your Board and Community PVAAS Statewide Core Team Fall, 2011

2. Session Description This 60 minute session will provide strategies and approaches for discussing PVAAS with your local school board and community, as well as communicating PVAAS results. Approaches that have been used by actual districts in Pennsylvania will be discussed. Materials and resources to use as tools for communication about PVAAS to your local school board and community will be reviewed. Opportunities for questions and discussion will be provided. There are trainer notes included for this presentation.

3. Agenda Documents & Suggested Handouts to use with School Board Members Considerations for Communicating PVAAS to School Boards and Your Community PVAAS Public Reporting Site Examining the School Board PowerPoint template Questions

4. Suggested Approach for Initial Session with School Board Keep it simple…keep it conceptual Discuss what PVAAS is and is not Start with a focus on Scatter Plots: Achievement vs. Growth Summarize important reports! Suggest starting with Reading and Math only District Value-Added Report Value-Added Summary Report (building level results) District (Single Grade) Projection Summary Report Performance Diagnostic Summary Report (if needed) Go For The BIG IDEAS!

5. Suggested Approach for Discussion with Community Members Keep it simple…keep it conceptual Discuss what PVAAS is and is not Start with a focus on Scatter Plots: Achievement vs. Growth (Across Grades Only!) Summarize important information! District & School Value-Added Across Grades Only School Search Average Growth Index Across Grades Only Go For The BIG IDEAS!

6. Supports and Resources to Prepare & Present at PVAAS to School Board Members and/or Community

7. Show: DVD for School Boards PVAAS for School Board Members by School Board Members Tom Gentzel, PSBA 2 Superintendents School Board Members 12 minutes Email pdepvaas@iu13.org for a copy of the DVD

8. Suggested Resources & Documents for Board Members Copy of PPT Presentation Guides to PVAAS Reporting 2011 Guide to PVAAS Reporting for LEAS 2011 Guide to PVAAS Public Reporting 2011

9. Suggested Resources & Documents for Board Members Copies of Key District Reports District Value-Added Summary Reports District Value-Added Reports District Projection Summary Reports District Scatter Plots Web-based Scatter Plots (demonstrate on public reporting site) Grade level or Across Grades

10. Example

11. PDE PVAAS Webpage Help Menus, PPT Presentations with Trainer Notes, Resource Materials, Video Clips, & District Case Studies

12. SAS PVAAS Website New & Archived Webinars, Podcasts, Guide to PVAAS Reporting for LEAs, Guide to PVAAS Public Reporting, & Intermediate Unit Supports https://pvaas.sas.com

13. Key Concepts for EVERY PVAAS Session

14. Key Concepts: ANY PVAAS Session AchievementProgress Measures students’ performance at a single point in time Compares student performance to a standard Critical to students’ post- secondary opportunities Highly correlated with demographic factors, such as socioeconomic status Measures students’ progress across time/years Compares student performance to his/her own prior performance Critical to ensuring students’ future academic success Not correlated with demographic factors

15. Key Concept: ANY PVAAS Session  PVAAS reporting reflects the effectiveness of your district’s or school’s Standards-Aligned System  PVAAS reporting reflects the district’s or school’s system regarding curriculum, assessment, and instruction

16. Key Concepts: ANY PVAAS Session There is NO relationship between demographic variables such as socioeconomic status and growth. There are high achieving schools making high growth; There are high achieving schools making low growth; There are low achieving schools making high growth; There are low achieving schools making low growth. Growth reporting reflects what WE do with students in terms of academic growth in schools/districts. -- VAAS can remove the effects of factors not under the control of the school. (McCaffrey, Lockwood, Koretz & Hamilton, 2003; Ross, Wang, Sanders, Wright & Stringfield, 1999a; Wright, Horn & Sanders, 1997).

17. Key Concept: ANY PVAAS Session  PVAAS District and School Value-Added Reports do NOT provide a measure of teacher effectiveness!  Often people ask about the meaning of these reports, especially in situations where there is only one classroom teacher in a grade level. This is NOT a report on the effectiveness of the classroom teacher, even in that situation!  Teacher effectiveness reporting would be done using a different statistical analyses than the one used for reporting on district or school effectiveness. Additional data would be collected as well.

18. DECISION POINTS Who? - What? -When? - How?

19. Decision Point: Audience What are the backgrounds of school board members? Do the board members have experience with growth data/PVAAS? Do the board members have any background knowledge/foundation in Statistics?

20. Decision Point: Intended Goal for Session (s) PVAAS Overview? Review of Actual District/School Data? Part of Planning Process? Comprehensive Planning School Improvement Planning

21. Decision Point: Focus of Data District? District? School? School? Grade? Grade? Subgroups? Subgroups? Which subjects? Which subjects? Math, Reading, Science, WritingMath, Reading, Science, Writing Remember, less is more! Remember, less is more!

22. Decision Point: Reports Needed District-wide Focus Scatter Plots District Value-Added Think ahead if there is only one building per grade Value-Added Summary/Individual schools District (Single Grade) Projection Summary Reports (Optional) Performance Diagnostic Summary If your audience is interested in growth of students at different achievement levels

23. Examples of Reports

24. Talking About Your Results Think about your message before sharing your results ! What do the results mean? Do you have a plan of action regarding the areas needing improvement (Yellow and Red)? What is your plan of action for attending to the Green, Light Blue, and Dark Blue areas?

25. Talking About Your Results Think about your message before sharing your results! What do the results mean? What do the results mean? Do you have a plan of action regarding the areas needing improvement (Yellow and Red)? Do you have a plan of action regarding the areas needing improvement (Yellow and Red)? What is your plan of action for attending to the Green, Light Blue, and Dark Blue areas? What is your plan of action for attending to the Green, Light Blue, and Dark Blue areas?

26. Decision Point: Session Facilitators(s) Principal Team of Principals District Administrator(s) District Administrator(s) + Principal(s) Support from Intermediate Unit PVAAS Contacts PVAAS Statewide Team for PDE Regional sessions for district administrators and School Board members

27. OTHER CONSIDERATIONS

28. PVAAS Projections are an Approved Way for Schools to Make AYP (Growth Model)

29. PVAAS Public Reporting Site Value-Added (Growth) Reports at District and School Levels Math, Reading, Science, and Writing Grades 4 through 8, and 9-11 School Search Capability Allows users to find and view the achievement and progress of groups of students in public schools across the Commonwealth of Pennsylvania. Can search for similar schools based on grade levels tested and various demographics. Web-based scatter plots

30. Preparing for Parent Requests Parents may ask about data or students. What is available at the student level? Projections! Percent likelihood of a student reaching Basic, Proficient and/or Advanced levels on future PSSA Some PA districts have shared PVAAS student projections with parents. Plan in advance for how to respond such requests

31. Questions? Comments? Email pdepvaas@iu13.org for a PowerPoint version of this session. pdepvaas@iu13.org

32. Presenting PVAAS to an Individual School Board Power Point Template with Trainer Notes

33. Pennsylvania Value-Added Assessment System (PVAAS): A Compass for Improvement Presented at [District] School Board Meeting [Date]

34. DVD for School Boards PVAAS for School Board Members by School Board Members Tom Gentzel, PSBA Tom Gentzel, PSBA 2 Superintendents 2 Superintendents School Board Members School Board Members 12 minutes 12 minutes

35. TWO Types of PVAAS Information Today Looking Back/Evaluation… Value-added Growth Reports For Groups of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Groups of Students

36. PVAAS IS : Analysis of existing assessment data Longitudinal PSSA scores in available content areas Math, Reading, Science, Writing Provides reliable estimates A measure of a school’s effectiveness Critical Relationship: Growth Data vs. Achievement Data

37. PVAAS: Is not another test Critical concept Is not a comparison of single test scores

38. Review of Literature Is there a relationship between… Demographics vs. Achievement? YES Demographics vs. Growth? NO Demographics are inherently controlled for in PVAAS due to the use of all longitudinal data on students!

39. High Achievement Average Achievement Low Achievement Current Data Collection Approaches Achievement is based on each student’s ending point.

40. PVAAS Adds Another Variable To Achievement…GROWTH! Below Growth Standard At or Above Growth Standard Growth is based on each student’s starting point!!

41. Viewing Achievement and Progress Together PVAAS Web-based Scatter Plots

42. Scatter Plot Quadrants and Their Meanings Quad 2 - Slipping Higher Achievement Lower Growth Quad 1 - Excelling Higher Achievement Higher Growth Quad 4 - Improving Lower Achievement Higher Growth Quad 3 - Underperforming Lower Achievement Lower Growth

43. Example: Regional Scatter Plot

44. Example: Regional Scatter Plot: Same Schools, Different Subject

45. Web-based Scatter Plots on PVAAS Public Site

46. Example

47. District Scatter Plots Math and Reading

48. District Scatter Plot – Grade 4 Math and Reading

49. District Scatter Plot – Grade 5 Math and Reading

50. District Scatter Plot – Grade 6 Math and Reading

51. District Scatter Plot – Grade 7 Math and Reading

52. District Scatter Plot – Grade 8 Math and Reading

53. District Scatter Plot – Grade 11 Math and Reading

54. Overview of PVAAS Reporting

55. PVAAS Reporting District Level Reporting District Level Reporting School Level Reporting School Level Reporting Grade Level Reporting Grade Level Reporting Reading (Grades 4-8, 11)Reading (Grades 4-8, 11) Math (Grades 4-8, 11)Math (Grades 4-8, 11) Science (Grades 4, 8, & 11)Science (Grades 4, 8, & 11) Writing (Grades 5, 8, & 11)Writing (Grades 5, 8, & 11) Subgroup Reporting (ex. IEP, ED, ELL) Subgroup Reporting (ex. IEP, ED, ELL) Student Level Reporting Student Level Reporting Projections to Future PSSA PerformanceProjections to Future PSSA Performance Reading, Math, Science, WritingReading, Math, Science, Writing Grades vary based on subject areaGrades vary based on subject area

56. Have all of our groups of students met the standard for PA Academic Growth? Grades 4-8? Grades 9-11? Value-Added Growth Reports

57. PVAAS Value-Added Growth Descriptors PVAAS Value-Added Growth Descriptors DB Significant evidence that the district/school exceeded the standard for PA Academic Growth LB Moderate evidence that the district/school exceeded the standard for PA Academic Growth G Evidence that the district/school met the standard for PA Academic Growth Y Moderate evidence that the district/school did not meet the standard for PA Academic Growth R Significant evidence that the district/school did not meet the standard for PA Academic Growth

58. District/School Value-added Report Grades 4 through 8: Math & Reading Question for Educators: Are all grade levels meeting the standard for PA Academic Growth? NOT on Public Site

59. District/School Value-added Report Grades 9-11 Math & Reading; Science; & Writing Question for Educators: Are all grade levels meeting the standard for PA Academic Growth? NOT on Public Site

60. District Value-added Summary Report Question for Educators: Which schools at which grade levels are exceeding the standard for PA Academic Growth? NOT on Public Site

61. How well are we supporting our high achieving students or low achieving students? Performance Diagnostic Summary Reports

62. Performance Diagnostic Growth Descriptors, Grade 4-8, Math/Reading Exceeded the standard for PA Academic Growth Met the standard for PA Academic Growth Did not meet the standard for PA Academic Growth

63. Performance Diagnostic Summary Report Questions for Educators: How effective are schools across the district in making academic progress with groups of students at different predicted performance levels? Are there difference in growth patterns across schools? NOT on Public Site

64. What is the likelihood that students will be reach a Proficient level or higher on a future PSSA? What is their likelihood of reaching an Advanced level? PVAAS Projection Summary Reports PVAAS Student Projection Reports

65. Student Projection Methodology Is this the same methodology as the one used to estimate growth? No, the projection methodology is a separate modeling process that focuses on individual students. What data are used in this methodology? ALL available longitudinal data in both Reading and Math are used in projection calculations for all four subjects. What students have projections? In what subjects? Projections are available for all students who have a minimum of 2 years of historical data. The next slide details the grade levels and subjects in which projections are reported.

66. PVAAS Fall 2011 Reporting: Projections to Basic, Proficient, Advanced Math and Reading:  3  4 or 5  4  5 or 6  5  6 or 7  6  7 or 8  7  8  8  11 Writing:  3  5  4  5  5  8  6  8  7  8  8  11 Science:  3  4  4  NA  5  8  6  8  7  8  8  11 Grade PSSA Last Taken Grade PSSA Last Taken Grade PSSA Last Taken

67. NOT on Public Site District (Single Grade) Projection Summary Report

68. PVAAS Student Projection Report NOT on Public Site

69. PVAAS Help Menus

70. Questions: PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com

71. www.pde.state.pa.us 333 Market Street Harrisburg, PA 17126

72. Questions from this webinar session? pdepvaas@iu13.org 717-606-1911