1. School Performance Framework Sponsored by The Colorado Department of Education Summer 2010 Version 1.3

2. Materials

3. Outcomes: School Performance Framework Understand the purpose of the school and district performance frameworks. Define and appropriately use terminology included in the school and district performance framework reports. Interpret the metrics (by performance indicator) included in the SPF and explain why each of the metrics were included. Describe the “cut-points” for each metric and how an overall plan type assignment is determined. Interpret the performance of one local school as presented by the School Performance Framework Report. Identify next steps in using the school performance framework reports.

4. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

5. School Performance Framework Purposes Focus attention on what matters most Provide a body of evidence related to state-identified performance indicators to support school and district performance management Support school and district efforts to evaluate their performance Establish a common framework for the state to use to hold schools accountable for performance Identify schools that need additional support (priority improvement and turnaround)

6. Focus Attention on What Matters All students will exit Colorado’s K-12 education system ready for postsecondary education and workforce success. --Colorado’s Achievement Plan for Kids (SB 08-212)

7. On what does the framework focus? Four key performance indicators: –Academic Achievement –Academic Growth –Academic Growth Gaps –Postsecondary and Workforce Readiness

8. How does the framework focus on these indicators? Assigns a rating to each of the performance indicators so schools/districts know the extent to which they meet state expectations. –Exceeds, Meets, Approaching, Does not meet The ratings roll up to an overall evaluation of the school/district’s performance. That evaluation informs the plan type assigned to the school. –Performance, Improvement, Priority Improvement, Turnaround

10. Mean vs. Median

11. Percentage vs. Percentile

12. Percentiles Range from 1 - 99 Indicate the relative standing of a student’s score to the norm group. (i.e. how a particular compares with all others) Growth Percentiles Indicate a student’s standing relative to their academic peers, or students with a similar score history (how his/her recent change in scores compares to others’ change in scores).

13. Experiencing Student Growth Percentile Using the Student Growth Card, form a group with all of the “students” with the same 3 rd grade scale score – academic peers. Within your group get in order by 4 th grade scale score. Turn your cards over and compare your growth percentiles.

14. Experiencing Median Growth Percentile Using your Student Growth card, identify your school (A, B, C, D, E). Form a group with others from the same school. Put yourselves in order (in a line) by growth percentile. Identify the person in the middle (median). What is the median student growth percentile for your school?

15. Adequate Growth What is adequate growth? Based on catch-up and keep-up growth.

16. Catching Up To be considered to be Catching Up: The student scores below proficient (unsatisfactory or partially proficient) in the previous year The student demonstrates growth adequate to reach proficiency within the next three years or by tenth grade, whichever comes first.

17. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up 95 Proficient Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 55

18. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 85 Proficient

19. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 80 Proficient

20. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 76 Proficient

21. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up 95 Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 85 80 76 76 is the minimum-this student’s adequate growth value Proficient

22. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 76 Proficient

23. Calculating Adequate Growth for Students Scoring Below Proficient: Catching Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 76 55 55 th percentile growth will not be enough for this student to catch up – her current growth is not adequate. Proficient

24. Keeping Up To be considered to be Keeping Up: The student scores at the proficient or advanced level in the previous year. The student demonstrates growth adequate to maintain proficiency for the next three years or until tenth grade, whichever comes first.

25. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up 12 Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 79 Proficient

26. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 25 Proficient

27. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 38 Proficient

28. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 50 Proficient

29. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up 12 Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 25 38 50 50 is the maximum - this student’s adequate growth value Proficient

30. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 50 Proficient

31. Calculating Adequate Growth for Students Scoring Above Proficient: Keeping Up Not Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 50 79 Proficient 79 th percentile growth will be enough for this student to keep up – his current growth is adequate.

32. Median Adequate Growth AGPSorted AGPsMedian AGP 45 78 99 32 11 91 55 67 43 10 77 Median Adequate Growth for this school is 55 Search for the middle value… Adequate growth percentiles for all catch-up and keep- up students

33. SPF Terminology Consider the SPF Terminology (Appendix of the SPF Technical Guide). Use the following legend to mark each term: –  = “I got it” – ? = Could use further clarification –* = New term or new definition for a familiar term With your table group –Discuss ? –Identify any unanswered questions for full group

34. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

35. 1 year vs. 3 year data CDE has provided two different versions of the School Performance Framework Reports. –The most recent year of data (2009) –The most recent three years of data (2007- 2009) Read 1-year vs. 3-year Data (technical guide) –Why are there two options? –When would the 3-year version be used to assign the school plan type?

36. School Performance Framework Overview Page 1: Summary of school performance, including plan type assignment and ratings for each performance indicator Page 2: Details of school performance by indicator and sub-indicator Page 3: Scoring guides including cut scores

37. School Performance Framework, Page 1 Four possible plan type assignments Plan Type Assignment Indicator Ratings Four possible indicator ratings

38. Performance Indicators Read the definitions of each of the following performance indicators on the first page of the sample SPF –Academic Achievement –Academic Growth –Academic Growth Gaps –Postsecondary and Workforce Readiness Discussion: Why does the School Performance Framework focus attention on these areas of performance?

39. SPF: Performance Indicator Summary, Page 1

40. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

41. Performance Indicators and Data Look at Page 2, first column

42. Page 2: Sub-Indicators Sub-indicators for each performance indicator: Points (earned and eligible) Percent of points Ratings School Performance Data

43. Defining State Expectations 1.Performance data 2.Basis for comparison 3.Definition of what level of performance meets state expectations –Scoring Guide on page 3 of the SPF –SPF Technical Guide includes reference data

44. Academic Achievement (status) Performance Data: the percent of students in the school scoring at the proficient or advanced achievement level Comparison: Compared to the distribution of percent proficient and advanced scoring students in all other schools* in the state. Scoring Guide: sets the percentile cut-points for each rating. * Does not included schools with N of less than 16 or Alternative Education Campuses

45. Math ElemMiddleHigh N of Schools1021482380 15th percentile47.6929.165.69 50th percentile68.9752.6626.32 90th percentile87.1875.7949.94 Example What would result in a rating of “Meets” for academic achievement for high schools in mathematics? (p. 3) Below the 90 th percentile and at or above the 50 th percentile of all* schools. (scoring guide, page 3 of SPF) For 2009 greater than or equal to 26.32% and less than 49.94% proficient and advanced Academic Achievement The school's percentage of students scoring proficient or advanced was: at or above the 90th percentile of all schools.Exceeds below the 90th percentile but was at or above the 50th percentile of all schools.Meets at or above the 15th percentile of all schools.Approaching below the 15th percentile of all schools.Does Not Meet

46. Practice What percent proficient and advanced would result in a rating of “Approaching” for academic achievement for high schools in reading?

47. Practice What percent proficient and advanced would result in a rating of “Approaching” for academic achievement for high schools in reading? –At or above the 15 th percentile and below the 50 th percentile of all* schools. (scoring guide, page 3 of SPF) –For 2009 greater than or equal to 41.51% and below 69.57% proficient and advanced (technical guide)

48. Academic Growth Performance data: Median Growth Percentile and Median Adequate Growth Comparison: Two steps 1.Did my school meet adequate growth? (yes or no) 2.How did my school’s growth compare to state expectations for growth? Scoring guide: sets the cut-points for median growth percentile using two tables that depend on whether or not the school met adequate growth

49. Adequate Growth 1.What was my school’s median growth percentile? 2.What was my school’s median adequate growth percentile? 3.Did my school meet adequate growth?

50. Scoring Guide for Academic Growth

51. High School Writing Example What was the median growth percentile in writing? –58.0 What was median adequate growth in writing? –82.0 Did this school meet adequate growth? –No

52. Cut-Points for Academic Growth Did we meet median adequate growth? NO, so use the NO scoring guide. 58.0

53. Practice: High School Reading What was the median growth percentile for high school reading? What was median adequate growth for high school reading? Did this school meet adequate growth?

54. Practice What was the median growth for high school reading? –52.0 What was median adequate growth for high school reading? –35.0 Did this school meet adequate growth? –YES

55. Cut-Points for Academic Growth Did we meet median adequate growth? YES, so use the YES scoring guide. 52.0

56. Growth Gaps Performance data: Median Student Growth Percentile for disaggregated student groups. Comparison: Two Steps 1.Did the growth of disaggregated groups in my school meet adequate growth? (yes or no) 2.How did the growth of disaggregated groups in my school compare to state expectations for growth? Scoring guide: sets the cut-points for median growth percentile using two tables that depend on whether or not the disaggregated group met adequate growth

57. Practice What was the median growth percentile for students who were eligible for free/reduced lunch in reading? What was the median adequate growth percentile for students who were eligible for free/reduced lunch in reading? Did the students who were eligible for free/reduced lunch meet adequate growth? What this school’s Growth Gaps rating for free/reduced lunch students?

58. Practice What was the median growth percentile for students who were eligible for free/reduced lunch in reading? –42.0 What was the median adequate growth percentile for students who were eligible for free/reduced lunch in reading? –41.5 Did the students who were eligible for free/reduced lunch meet adequate growth? –Yes What this school’s Growth Gaps rating for free/reduced lunch students? –Approaching

59. State Expectations for Postsecondary and Workforce Readiness Performance DataComparison Graduation RateAbove 80% Drop-Out RateAt or below state average Colorado ACT CompositeAt or above the state average Technical Guide: How graduation and dropout rates are calculated. The state average dropout rate. The state average ACT Composite Score. Note: State averages are different for 1-year vs. 3- year metrics

60. High School Practice What was the graduation rate? –88.6% What rating did that earn? –Meets (above 80% but below 90%) What was the dropout rate? –2.2% What rating did that earn? –Meets (below state average of 5.1, but above 1%) What was the average Colorado ACT composite score? –16.4 What rating did that earn? –Does not meet (At or below 19)

61. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

62. Indicator Points Points earned on a scale of 0-4 Sum sub-indicator points for total indicator points. Convert to % (total indicator points/100) Points EarnedRating 4Exceeds 3Meets 2Approaching 1Does not meet 0Not eligible for points

63. Example Consider the Academic Growth Gaps indicator (high school example) How many points was this high school eligible to earn? –48 How many points were earned? –29 What percentage of points were earned for this indicator? –60.4 In which sub-indicator content area did this school earn the most points? –Writing

64. Indicator Ratings Based on the % of points earned of the points eligible for the indicator. Cut-points on page 3 of SPF Cut-Points for each performance indicator Cut-Point: The school earned … of the points eligible on this indicator. Achievement; Growth; Gaps; Postsecondary and Workforce Readiness at or above 87.5%Exceeds at or above 62.5% - below 87.5%Meets at or above 37.5% - below 62.5%Approaching below 37.5%Does Not Meet What rating would the example high school example earn with 60.4% of points for Academic Growth Gaps?

65. High School Practice How many points did this school earn out of eligible points on the Academic Growth indicator? –3 in reading + 2 in math + 3 in writing = 8 points out of 12 eligible (4 per subject) What percent of points did this school receive on the Academic Growth indicator? –66.7% What rating did that earn on Academic Growth? –Meets

66. Framework Points Each indicator is weighted differently in the overall plan type assignment (page 1 SPF). Multiply the percent of points earned by the weight for each indicator (Eligible points). These are your framework points for each indicator. IndicatorES/MS WeightHS Weight Academic Achievement2515 Academic Growth5035 Academic Growth Gaps2515 Postsecondary and Workforce Readiness 035

67. High School Example What percent of points did this high school earn on the Academic Growth indicator? –66.7% What framework points did this high school earn on the Academic Growth indicator? –66.7% x 35 framework points = 23.3 framework points

68. Arriving at a Plan Type Assignment Sum framework points for each indicator. Convert this sum to a percentage (of 100). Use % of framework points to determine the plan type assignment. See “Cut-points for plan type assignment” on page 3 to determine plan type assignment. Cut-Points for plan type assignment Cut-Point: The school earned … of the total framework points eligible. Total Framework Points at or above 60%Performance at or above 45% - below 60%Improvement at or above 30% - below 45%Priority Improvement below 30%Turnaround

69. High School Example What framework points did this school earn on each indicator? –5.6 for Achievement, 23.3 for Growth, 9.1 for Growth Gaps, 20.4 for Postsecondary & Workforce Readiness What total framework points did this school earn? –5.6 + 23.3 + 9.1 + 20.4 = 58.4 What percent of framework points did this school earn? –58.4 out of 100 eligible = 58.4% What plan assignment does this school receive? –Improvement

70. Distribution of Preliminary School Performance Framework Plan Assignments: All School Levels See Technical Guide pages 26-27 for distribution by school level.

71. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

72. Resources to Support SPF Interpretation School Performance Framework Technical Guide Guiding Questions for Interpreting the School Performance Framework (page 18 of the technical guide)

73. Interpreting your School Performance Framework Report Take out a School Performance Framework report for a school in your district (or use the elementary sample report in the technical guide). Use the guiding questions for interpreting the School Performance Framework Report. Do you have any questions about what you see on your SPF report?

74. Agenda: School Performance Framework Purpose and Terminology Overview Indicator Points and Plan Type Assignments Interpreting the School Performance Framework Using the School Performance Framework Performance Data by Indicator

75. Next steps with the SPF Reports Identify for which indicators and sub-indicators school performance –does not meet state expectations, or –does not meet school or district expectations. Drill down... into additional data related to priority sub-indicators. Use this data to focus improvement planning efforts. Use plan type assignments to target support for schools.

76. Additional State Data Sources School Growth Summary, District Growth Summary CSAP score reporting Colorado Growth Model (both public and private) Student-level CSAP files (from CTB) Student-level flat files (growth, CSAPA, PSWR) from CEDAR

77. Additional Local Data Sources What additional local data sources could you analyze to drill-down in areas where the SPF indicates performance has not met expectations? Consider: –Interim/benchmark assessment results –Progress monitoring assessment results –End of unit assessment results –Disaggregated by various student groups

78. More School and District Reports coming soon... July: Revised (based on your feedback) School Performance Framework Reports with 2007- 2009 data. July: District Performance Framework Reports with 2007-2009 data August: School and District Performance Framework Reports (2008-2010 data)

79. Provide Feedback!!! Parking Lot –+ the aspects of this session that you liked or worked for you. –  The things you will change in your work or would change about this session. –? Questions that you have – Light bulb: ideas, a-has, innovations On-line Feedback: www.surveymonkey.com/spffeedback www.surveymonkey.com/spffeedback