1. Selecting Demonstrable Improvement Indicators September 18, 2015 Ira Schwartz Assistant Commissioner of Accountability Shibu Joseph Associate, Education Research

2. Goals for Today’s Webinar Provide overview of Demonstrable Improvement Process. Explain how to use the Demonstrable Improvement Spreadsheet to Select Indicators. Clarify Next Steps and How to Request Assistance. Answer Questions, which you can send to receivership@nysed.gov. receivership@nysed.gov 2

3. Demonstrable Improvement Recap For more details, please read the PowerPoint presentation “Making Demonstrable Improvement: Request for Feedback” posted online at: http://p12.nysed.gov/accountability/de/SchoolReceivership.html http://p12.nysed.gov/accountability/de/SchoolReceivership.html

4. Requirements for Demonstrable Improvement Persistently Struggling Schools must annually make Demonstrable Improvement or they will be placed in Independent Receivership. Struggling Schools must annually make Demonstrable Improvement or after a two-year period they will be placed in Independent Receivership. Commissioner shall consider:  Performance on Metrics (Indicators)  Years of Identification  Superintendent’s Successful use of Receivership Powers to Implement the School’s Plan 4

5. Guiding Principles One or more indicator(s) shall be established for each Metric specified in legislation. For each indicator, a school can make progress by achieving either a “universal goal” or a school specific progress target. The indicator goals and the school specific targets generally increase over the three-year period. Most indicators will be based on student performance; some indicators will be based on implementation of programs and/or processes. The State Education Department will select some of the school indicators, and the School Receiver in consultation with the Community Engagement Team shall select some. Selected indicators will be based primarily on where the school’s performance needs the most improvement. School Receiver may seek to have local measures approved by the Commissioner. The result of the process shall be a judgment that a) the school made Demonstrable Improvement, b) the school did not make Demonstrable Improvement unless there are shown to be extenuating or extraordinary circumstances, or c) the Commissioner shall review the totality of the school's record to make a determination as to whether or not the school made Demonstrable Improvement. 5

6. How Demonstrable Improvement is Determined Some indicators will be considered Level 1; some indicators will be considered Level 2; some indicators may be either Level 1 or 2, depending on circumstances. A minimum of five Level 1 and five Level 2 indicators will be selected for a school; a school with elementary-middle and secondary grades will have to select seven Level 1 and seven Level 2 indicators, indicators from both grade levels must be included. Both Level 1 and Level 2 indicators shall be weighted 50% in computing the Demonstrable Improvement Index. Each indicator within Level 1 and Level 2 shall be weighted equally. The Demonstrable Improvement Index shall range from 0% to 100%. If a school achieves an index of 67% or higher, the school has made demonstrable improvement. If a school achieves below 40%, it has not, unless the school can demonstrate it would have achieved 67% of its goals absent extenuating or extraordinary circumstances. The Commissioner shall review the entirety of the record and after consulting with district and Community Engagement Team determine whether a school with an index of 40% or higher, but less than 67% shall be considered to have made Demonstrable Improvement. 6

7. Level 1 Indicators There are seven Level 1 indicators for elementary and middle schools and seven for high schools. A school with both elementary-middle and high school grades will have twelve indicators. If a school’s performance is below a Level 1 goal for an indicator, that indicator will be assigned to a school. If there are five or more indicators for which the school is below the 2015-16 goal then all of these will be used. If there are fewer than five (seven for schools with EM & HS Grades), then the associated Level 2 indicators will be used as Level 1 indicators. If the combined Level 1 indicators and associated Level 2 indicators are still fewer than five (seven for schools with EM & HS Grades), then the Level 1 indicators for which the school is above the 2015-16 goal can be used. An associated Level 2 indicator is a Level 1 indicator used for a specific accountability group (e.g., the percentage of students with disabilities in Grades 3-8 math performing at or above Level 2 is an associated Level 2 indicator for the Level 1 Grades 3-8 math indicator.) After year 1, if a school’s Level 1 indicator falls below the 2016-17 goal, then that indicator will be assigned as an additional Level 1 indicator to the school for 2016-17 and 2017-18. 7

8. The Level 1 Indicators Elementary and Middle: Making Priority School Progress Percent of Students at or above Level 2 in ELA Percent of Students at or above Level 2 in math Mean Student Growth Percentile in ELA Mean Student Growth Percentile in math Percent of Students at or Above Level 3 in Science Serious Incidents (VADIR) High School: Making Priority School Progress 4-year High School Graduation Rate 5-year High School Graduation Percent of Students Graduating with Regents Diploma with Advanced Designation Percent of 10 th graders passing Math Regents Percent of 11 th graders passing ELA Regents Serious Incidents (VADIR) 8

9. Level 2 Indicators Level 2 Indicators Include: Indicators for students subgroups (i.e., English language learners, low- income students, racial/ethnic groups and students with disabilities). Implementing a Community School Model, expanded learning time and other key system initiatives. School climate (e.g., attendance, suspensions). Gaps between a student group and students who are not members of the group (e.g., between students with disabilities and students without disabilities). Students passing courses. High School Student Promotion Rates (promoted from grades 9,10 & 11). College- and Career- Readiness. Developmentally Appropriate Child Assessments: Pre-K to Third Grades. Teachers Teaching out of Certification Area. Teacher Turnover. Post-graduation plans for students. Local measures approved by the Commissioner. 9

10. Computing the Demonstrable Improvement Index: Example 10 IndicatorLevelPerformanceGoalIndicator MadeWeighting Made Priority School Progress Level 1Did Not Make ProgressMake ProgressNo0% Grades 3-8 math percent at or above Level 2 Level 128%26%Yes10% Grades 3-8 ELA all students SGP Level 148%46%Yes10% Grades 4 and 8 Science percent at or above Level 3 Level 135%39%No0% Grades 3-8 Math SGPLevel 145%47%No0% Implement Community School Model Level 2First Year Implementation Yes7.14% Expanded Learning TimeLevel 2Implement ProgramProgram ImplementedYes7.14% DTSDE Teacher Practices and Decision Making Level 2Developing Yes7.14% Percent of Newly Hired Teachers with State Provided Growth Ratings of Effective or Above Level 280%90%No0% Grades 3-8 ELA low-income SGP Level 252%51%Yes7.14% Chronic Absenteeism Level 2 (Local Indicator) Fewer than 29% of students chronically absent 30% Chronically Absent (For Example) No0% ELL students gaining one Level on NYSESLAT Level 2 (Local Indicator) 41%50%Yes7.14% Index Result 55.70%

11. Determining Demonstrable Improvement In this example, because the school’s Demonstrable Improvement Index is above 40%, but below 67%, the Commissioner reviews the entire performance of the school. After the review, the Commissioner determines the school has made Demonstrable Improvement. Note: Goals and progress targets become more rigorous in Years 2 and 3. 11

12. Additional Information About the Index Once selected, indicators remain in place for three years. If a determination cannot be made in the current year about an indicator, the weights for that year will be adjusted. If a determination cannot be made in a future year about an indicator, another indicator may be selected. After year 1, if a school’s Level 1 indicator falls below the 2016-17 goal, then that indicator will be assigned as an additional Level 1 indicator to the school for 2016-17 and 2017- 18. After year 1, the Department will be reviewing the goals and progress targets for 2016-17 and 2017-18. If any are determined to be insufficiently or overly rigorous, the Department reserves the right to adjust them. If a school meets the criteria for removal from Priority Status, the district may petition for removal, even if the school does not make Demonstrable Improvement. 12

13. Selecting Demonstrable Improvement Indicators

14. Opening the Excel Spreadsheet File NYSED has e-mailed to each Superintendent/EPO a Microsoft Excel spreadsheet file with the list of Demonstrable Improvement indicators available for selection for each of the Persistently Struggling and Struggling Schools in the district. Please follow these steps to select the indicators: Open the Microsoft Excel file with the Demonstrable Improvement indicators data. It is possible that the file will open with a notification “Security Warning: Macros have been disabled. Enable Content.” If so, please click on “Enable Content” for the file to work properly. A macro is a set of instructions that are programmed into the spreadsheet to automate tasks. 14

15. Opening the Excel Spreadsheet File It is also possible that the file will open with a notification “Protected View Office has detected a problem with this file. Editing it may harm your computer. Click for more details.” If so, click on the message and select the option “Edit Anyway.” It will now display the message “Security Warning: Macros have been disabled. Enable Content.” Please click on “Enable Content” for the file to work properly. 15 Step 1 Step 2 Step 3

16. Demonstrable Indicators File The file has two tabs; school_worksheet and district_worksheet. Please review the data on the district_worksheet Tab first. 16

17. Demonstrable Indicators File: district_worksheet Tab Even though the file will open to the “school_worksheet” Tab, please review the “district_worksheet” first to get a better understanding of which indicators the school(s) are below the 2015-16 goals. The tab “district_worksheet” is a database with indicator information for all the Persistently Struggling and Struggling Schools in the district. This tab also has the baseline for the school(s), goals and progress targets for 2015-16, 2016-17 and 2017-18. The “district_worksheet” tab has filters enabled for each of the columns; that means you can make data visible or hide data for the filters you choose. This spreadsheet will have all the indicators listed, even if the school did not have baseline data for the specific indicator. Therefore, use the filter to hide unwanted rows. 17

18. district_worksheet Tab Here is a partial view of the district_worksheet Tab filtered by Status column to hide indicators with “NA/No Baseline Available” status. 18

19. Demonstrable Indicators File: school_worksheet Tab 19 1.The Excel file will open with the “school_worksheet” Tab visible. This is the tab from where indicators have to be selected for the respective schools. 2.The “school_worksheet” will open with no data for any school visible. 3.When the school is selected from the drop down menu the data gets populated for that school. Pre-selected Level 1 Indicators and Selecting Level 2 as Level 1 Indicators 4.An elementary-middle (EM) or a high school (HS) must have a minimum of five Level 1 indicators; a school with both EM and HS grades must have a minimum of seven Level 1 indicators. 5.Level 1 indicators for which the school’s baseline is below the 2015-16 goal will be highlighted in yellow. These indicators have been pre-selected by the Department and cannot be changed. 6.An indicator counter is provided in cell C6 to display how many Level 1 and Level 2 as Level 1 indicators are selected. 7.If the school does not have the minimum required number of Level 1 indicators pre-selected, a message will be displayed stating “A school must have a minimum of 5 (7, if the school has EM and HS grades) Level 1 or Level 2 as Level 1 indicators.”

20. Demonstrable Indicators File: school_worksheet Tab 20 8.If this message is displayed, the school must select Level 2 as Level 1 indicators to meet the minimum requirement of five or seven indicators. Most of the Level 2 indicators will have a drop down in the “Level” column that will allow the indicator to be selected as a Level 2 as Level 1 indicator. 9.Once the minimum count of Level 1 or Level 2 as Level 1 indicators are reached, the message “A school must have a minimum of 5(7) Level 1 or Level 2 as Level 1 indicators” will disappear. 10.If the file does not display the message “A school must have a minimum of 5(7) Level 1 or Level 2 as Level 1 indicators,” then the school is not required to select any Level 2 as Level 1 indicators. 11.A Level 2 as Level 1 indicator will be weighted the same as a Level 1 indicator for calculating the Demonstrable Improvement index.

21. Demonstrable Indicators File: school_worksheet Tab 21 Selecting Level 2 Indicators 12.A school with EM or HS grades must have a minimum of five Level 2 indicators; a school with both EM and HS grades must have a minimum of seven Level 2 indicators. 13.Level 2 indicators for which the school’s baseline is below the 2015-16 goals will be available for selection. 14.Schools must select the minimum five (seven, if the school has EM and HS grades) Level 2 indicators. 15.If the school does not have the minimum required count of Level 2 indicators selected, a message will be displayed stating “A school must have a minimum of 5 (7, if the school has EM and HS grades) Level 2 indicators.” 16.Once the minimum count of five (seven) Level 2 indicators is reached, the message “A school must select a minimum of 5 (7) Level 2 indicators” will disappear. 17.Schools can select more than the minimum five (seven) Level 2 indicators. 18.An indicator counter is provided in cell C7 to display how many Level 2 indicators are selected. 19.Schools can select only one DTSDE Tenet Indicator. If more than one DTSDE Tenet Indicator is selected, a message will be displayed “Warning: Only one DTSDE Tenet Indicator can be selected.” Please deselect the additional DTSDE Tenet Indicator(s).

22. Demonstrable Indicators File: school_worksheet Tab 22 Schools with Insufficient Level 1, Level 2 as Level 1 and Level 2 Indicators Available for Selection 1.There are a few schools statewide with either: a.Insufficient indicators available for selection. b.Limited choice of indicators available for selection (e.g., majority of indicators available for selection are gap related or student growth related). 2.Districts must seek permission from the Department to add indicators for which their baselines are already above the 2015-16 goals. 3.The “above the goal” indicators selected must be spread across various performance areas.

23. Selecting the Level 1 Indicators: Example – School B In the school_worksheet Tab, click on the school drop down menu and select a school. Let us select School B. 23

24. Selecting the Level 1 Indicators: Example – School B 1.The spreadsheet will be populated with Level 1 indicators pre-selected by the Department and Level 2 indicators available for selection. 2.The counter on cell C6 indicates only 4 Level 1 Indicators were pre-selected by the Department. The school needs to select one additional “Level 2 as Level 1” indicator to reach the minimum of 5 for this school. 3.Please note the message on cell D6. 24

25. Selecting the Level 1 Indicators: Example – School B 4.The school decided to select 3-8 ELA Hispanic Students Level 2 and above as the Level 2 as Level 1 indicator. 5.For the indicator, click on the Level 2 cell and select from the drop down Level 2 as Level 1. This will automatically highlight the cell and update the Indicator Selector to Yes. The counter on cell C6 has incremented to 5. 6.Please note that the message on cell D6 has disappeared. The message on cell D7 will remain till Level 2 indicators are selected. 25

26. Selecting the Level 2 Indicators: Example – School B 1.The school has to select 5 Level 2 indicators. 2.The school selects the 5 Level 2 indicators by selecting “Yes” from the Indicator Selected drop down menu. The counter on cell C7 has incremented to 5. 3.Please note that the message on cell D7 has disappeared. 4.Since the school has selected Chronic Absenteeism as one of the indicators for which goals/progress targets are not available, a message is displayed on cell I5. 5.After the minimum Level 1 and Level 2 indicators are selected, click on the Indicator Selected column, drop down the filter menu and uncheck the “No” filter. This will hide the unselected indicators. 26

27. Printing and Saving the File 1.For printing, click on File menu, select Print and click on Show Print Preview. 2.The selected Indicators will print on legal size paper. 3.If you see unselected indicators during Print Preview, repeat Step 5 on Slide 26. 4.The file may be saved with the respective school’s name. 27

28. Schools with Insufficient Indicators: Example – School C 1.School C has only one Level 1 indicator pre-selected by the Department and a limited choice of available indicators for selection. 2.With the permission of the Department, the school may select indicators with the status “Meets Goal/ Not Selected by SED” or “Meets Goal/Not Available for Selection.” 3.The Status column has a drop down filter menu to select indicators with the aforementioned status. 4.School C selected Indicator numbers 11,12, 15 and 33 for which the school is already above the 2015-16 goal. To make Demonstrable Improvement the school’s 2015-16 performance still needs to be at or above the 2015-16 goal or progress target. 28

29. TBD Indicator: Teacher Attendance Teacher Attendance along with Chronic Absenteeism are “to be determined” Level 2 indicators the Department has not defined or set baselines, goals or progress targets. We use the Federal definition for Teacher Attendance given below. During the regular school year, the average percentage of days that teachers are present when they would otherwise be expected to be teaching students in an assigned class. Teachers should not be considered present for days taken for sick leave and/or personal leave. Personal leave includes voluntary absences for reasons other than sick leave. If a school selects Teacher Attendance as one of its indicators for Demonstrable Improvement, the district must be able to submit data using the above definition so that the Department can set the 2013-14 school year baseline. Goals and progress targets will then be set for the 2015-16, 2016-17 and 2017-18 school years. 29

30. TBD Indicators: Chronic Absenteeism Schools in Receivership that elect to include tracking and monitoring chronic absenteeism as an Indicator can incorporate a three-year phase in plan as specified below: Year 1: by December 1, 2015, the district will develop a comprehensive chronic absenteeism policy for each school using the indicator and an implementation plan aimed at reducing chronic absenteeism. Such policies and plans will include tracking individual student absences on a regular basis defined as preferably weekly, but no less than monthly; a process of identifying students who are missing 5% or more of the days that the student has been enrolled and school has been in session; prevention and intervention strategies, where appropriate, for students missing 5% or more of the days the student has been enrolled and school is in session; prevention and intervention strategies for students missing 10% or more of the days the student has been enrolled and school is in session; and a way to track overall chronic absenteeism rates. By no later than February 1, 2016, each school will begin to implement its chronic absenteeism plan. Year 2: Each school will review the results of the chronic absenteeism policy and implementation plan and will create a base year of chronic absenteeism data as well as adapt and amend the policy and plan to meet the individual needs of the school. This year should also be used to engage with and involve the community by sharing the chronic absenteeism process results from Year 1 and incorporating feedback into the plan for Year 2. Year 3: Each school will reduce its rate of chronic absenteeism. Based on the data from Year 2, each school will reduce the number of students who are chronically absent by utilizing the policies and procedures. The school will also continue to engage with and involve the community in an effort to further reduce chronic absenteeism into future years. Districts that already have the ability to compute a chronic absenteeism rate may instead beginning in Year 1 or 2 submit an indicator based on reduction of chronic absenteeism. 30

31. Local Indicator Templates The Department has shared two templates (performance data and climate survey) with examples that districts can use as guides to submit Local Indicator proposal(s) for approval. 31

32. Local Indicator Templates 32

33. Next Steps Please discuss the selected Demonstrable Indicators with the school’s Community Engagement Team. Submit the file with the selected indicators to receivership@nysed.gov by Friday, October 2, 2015. receivership@nysed.gov If you need technical assistance with the Excel file please call (718) 722-4553 or send an e-mail to receivership@nysed.gov. receivership@nysed.gov Local Indicator proposal(s) must be submitted by Friday, October 2, 2015. We will make a recording of this webinar available at: http://p12.nysed.gov/accountability/de/SchoolReceivership.html http://p12.nysed.gov/accountability/de/SchoolReceivership.html 33