1. accountability 2013 Interpreting Your 2013 Accountability Report It’s Like Learning To Read All Over Again Ervin Knezek ervin@lead4ward.com John Fessenden john@lead4ward.com TEPSA Webinar | September 5, 2013

3. goals 2013 accountability  Construction  Clarity  Of calculation... and application  Connection  Of indexes  What measures/data are included?  To instructional processes and systems TEPSA Webinars (May 10 and 17) Today Ongoing

4. the vibe

5. Your Questions 5

6. the BIG picture

7. p. 21p. 22

8. Accountability | Overall Design Four Performance Indexes Designed to meet 4 statutory goals 1.Improving student achievement at all levels in core subjects 2.Ensuring progress of all students toward Advanced Academic Performance (STAAR Level III) 3.Closing Performance Gaps among groups in Advanced Academic Performance (STAAR Level III) 4.Closing gaps among groups in % of students graduating under Recommended or Advanced high school programs Index 1 Index 2 Index 3 Index 4 p. 22

9. 2013 Accountability Achievement  All Students Only  Single percentage combined over ALL Subject Areas  ≥ Level II (Phase 1) STAAR  ≥ Met Std Grade 11 TAKS Progress 10 student groups – All Students – 7 Race/Ethnicity – SpEd – ELL Minimum size – 10 for All Students – 25 for other groups By Subject Area – Rdg & Math in 2013* Based on PROGRESS – 1 point for % Met – 2 points for % Exceeded Closing Gaps 3 student groups – Eco Dis and – 2 lowest Race/Ethnicity groups from prior year By Subject Area (all subjects) 1 point for % Level II (Phase 1) Level III to be added (2014) Postsecondary 4-year or 5-year Graduation Rate (or Annual Dropout Rate) – 10 student groups % RHSP/DAP – 8 student groups (not SpEd or ELL ) STAAR Postsecondary Readiness – Level II Final (2014) Index 1 Student Achievement Index 2 Student Progress Index 3 Closing Performance Gaps Index 4 Postsecondary Readiness Target = 50Target = ≈ 5 th %ile Target = 55Target = 75 Target = 30291721 ESMSHSDist p. 21

10. Interpreting our Reports It’s like learning to read all over again!

11. 2013 accountability... PartSection# Pages 1Accountability Summary1 2Index Calculations and Data Tables12 3Distinction Designations (Campuses Only)4 (8/29/13) 4System Safeguards3 (8/29/13) Total Pages20 (8/29/13)... a report in 4 parts p. 1 p. 2-13 p. 14-17 p. 18-20

12. Index 1

13. Student Achievement Index 1 Student Achievement p. 2 p. 3-4

14. Index 1: Student Achievement Index 1 Student Achievement p. 22 10 Additional Student Groups reported  7 Race/ Ethnicity  SpEd  Eco Dis  ELL Used for System Safeguards

15. p. 2 Calculate Overall Pass Rate For All Students 1.Add # Passed (≥ Phase-in 1 Level II) each subject area (R + M + W + S + SS) 277 + 284 + 65 + 60 + 34 = 720 2.Add # Tests taken each subject area (R + M + W + S + SS) 485 + 473 + 174 + 127 + 127 = 1,386 3.Divide # Passed by # Tests to determine % Passed 720 ÷ 1,386 = 51.9%, which ROUNDS to 52% 4.Assign 1 point for each % Passed = Index 1 Score 52% = 52 POINTS Overall pass rate IS NOT average pass rate – Average pass rate would equal: (57% + 60% + 37% + 47% + 27%) ÷ 5 = 47% Index 1 Score 57%60% 37%47%27%

16. p. 3-4 p. 2 Where did these numbers come from? Pages 3-4! % at Final Level II (13%) % at Level III (3%) [Instructional Planning]

17. Disaggregated Data  There are a LOT of numbers on pages 3 and 4!  “I thought Index 1 ONLY evaluated the All Students group!”  True!  BUT... the data are included for System Safeguards

18. p. 3 System Safeguard – Low Performance in a single cell for a particular student group does NOT automatically cause the district/campus to miss Index 1 – However, ANY student group meeting MSC with a pass rate below 50% MUST be addressed in the district/campus improvement plan MSC = 0 for All Students | 25 for all other Student Groups p. 13

19. So how do we “find” System Safeguards? 1.In each subject area, determine which student groups meet MSC MSC = 0 for ALL Students | 25 for all other student groups 2.Then determine if % Passing < 50% (at Phase-in Level II or above) for any group meeting MSC 3.If % Passing < 50% AND Student Group meets MSC, then Student Group’s Performance MUST be addressed in improvement plan p. 13 p. 3

20. p. 13 Others? # Groups Missing Safeguard 8 # Groups Meeting Safeguard 9 p. 3 # Groups Meeting MSC Reading, Math, Writing 17

21. p. 4 Science? Social Studies? # Groups Meeting MSC Reading, Math, Writing 17 # Groups Missing Safeguard 8 # Groups Meeting Safeguard 9 Science, Social Studies 88 0 TOTAL 25 169

22. p. 3 v p. 4 System Safeguards 25 student groups met MSC (across 5 subject areas) ONLY 9 Met Safeguard target of 50% 16 Missed Safeguard target (including EVERY group in Writing, Science, Social Studies) BUT campus MET Index 1 Target! All 16 Student Groups that Missed Safeguard target of 50% must be addressed in CIP

23. System Safeguards Are summarized with Y/N indicators in Part 4 of the Campus’ 2013 Accountability Report (p. 18) p. 18p. 3

24. p. 19 System Safeguards Performance Rate data from p. 3-4 are duplicated on p. 19 (slightly reorganized)

25. p. 19 System Safeguards Performance Rate data from p. 3-4 are duplicated on p. 19 (slightly reorganized) Participation Rate data are also provided

26. Index 1 is summarized in the campus’ 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1

27. 27 Your Questions Index 1 Student Groups? MSC? Safeguards?

28. Index 2

29. Student Progress Index 2 Student Progress p. 5 p. 6-7

30. Index 2: Student Progress Index 2 Student Progress p. 22

31. Index 2: Student Progress Based on actual student progress (2012 to 2013) Index 2 Student Progress STAAR Level III focuses on progress toward...

32. Each student is assigned a progress category based on his/her change in scale score in relation to progress expectations – Did Not Meet – Met – Exceeded Index 2: Student Progress Index 2 Student Progress

33. Index 2: Student Progress Subject areas evaluated for 2013 – Reading Grade 4 through English II – Math Grade 4 through Algebra I – Writing English I to English II Index 2 Student Progress

34.  Performance Standard  Met Progress  Exceeded Progress  Index 2 is based on a fraction where  Numerator = # of points based on % of students who Met or Exceeded Progress  1 point for % Met  2 points for % Exceeded  Denominator = Maximum Points Possible based on number of student groups meeting minimum size criteria (MSC)  Maximum Points Possible = 200 points for each student group meeting MSC Index 2: Student Progress Index 2 Student Progress

35. What is “progress”? Index 2: Student Progress Index 2 Student Progress With exceptions for high-performing (“almost perfect”) and low-performing (“lower than guessing”) students, the general rule is Met Progress = Level II Final (Higher Grade) – Level II Final (Lower Grade) Exceeded Progress = Level III (Higher Grade) – Level II Final (Lower Grade)

36. p. 23 Subject and Grade Level/Course Items Points Level II Standard (Final Recom- mended) Level III Standard Grade 4 (Eng. & Span.)4815991677 Grade 5 (Eng. & Span.)5016271710 Met Progress? Level II 5 th Grade – Level II 4 th Grade 1627 – 1599 = 28

37. p. 23

38. Exceeded Progress? Level III 5 th Grade – Level II 4 th Grade 1710 – 1599 = 111 p. 23 Subject and Grade Level/Course Items Points Level II Standard (Final Recom- mended) Level III Standard Grade 4 (Eng. & Span.)4815991677 Grade 5 (Eng. & Span.)5016271710

39. p. 23

40. 5 th Grade STAAR Math Progress (2013) 5 th Grade Math 2013 4 th Grade Math 2012 Scale Score Differential Progress Category Performance Level Raw Score Scale Score Performance Level Scale Score Alex III 48 (out of 50) 1847III 2016 (Perfect Score) -169 EXCEEDED PROGRESS Vinnie I261481I1320161 EXCEEDED PROGRESS Arthur II391627II160720 DID NOT MEET PROGRESS

41. Index 2 Simple Example All Students # Tests 500 # Met 300 # Exceeded 100 % Met 60% % Exceeded 20% 1 point for % Met OR Exceeded 80 1 point for % Exceeded 20 Weighted Progress Points 100 Maximum Possible Points 200 Index 2 Score (Weighted Progress Points ÷ Maximum Possible Points) 50 1 point for % Met 60 2 points for % Exceeded 40 Weighted Progress Points 100

42. Index 2 Data Reports  Build “back to front”  Start with page 7  Then page 6  Then page 5

43. p. 7  For each student group meeting MSC in each subject, determine:  % Met or Exceeded Progress  % Exceeded Progress From Index 1 (p. 2)  REMINDERS (Why Index 2 numbers will not match Index 1):  Only students with 2 years of “matched” data are included  Progress starts at 4 th grade (no 3 rd graders receive a Progress Measure)  ELL students are “mostly excluded” from Index 2 in 2013  STAAR Modified and STAAR Alternate are not included in Index 2 in 2013  Writing is ONLY evaluated for English I to English II  MSC = 10 tests for All Students | 25 tests for other Student Groups

44. p. 6  Data from page 7 are reproduced on page 6  In each subject area, for each Student Group meeting MSC  Assign 1 point for each % Met OR Exceeded  Assign another 1 point for each % Exceeded  Total = Weighted Progress Rate for each Student Group meeting MSC  Effectively 1 point for each % Met and 2 points for each % Exceeded  Add Weighted Progress Rate in each subject for each student group meeting MSC  Maximum Points = # Student Groups meeting MSC x 200 + + ++

45. p. 5  Data from page 6 are summarized on page 5 (remember: read back to front!)  Weighted Progress Rate for each Student Group meeting MSC in each subject area is reported and summed  Reading: 75 + 64 + 77 = 216 points  Math: 70 + 67 + 72 = 209 points  Sum the Total Points in Reading and the Total Points in Math  216 + 209 = 425  Sum the Maximum Points in Reading and the Maximum Points in Math  600 + 600 = 1200  425 ÷ 1200 = 35.4, which rounds to 35... SO Index 2 Score = 35 points Index 2 Score + + + +++

46. Index 2 is summarized in the campus’ 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1

47. 47 Your Questions Index 2 Progress Categories Points Data

48. Index 3

49. Closing Performance Gaps p. 8 p. 9-11

50. Index 3: Closing Performance Gaps Index 3 Closing Performance Gaps  Impacts every student group of which the ELL student is a member p. 24

51. Index 3: Closing Performance Gaps  Index 3 emphasizes Level III (Advanced Academic Performance) for “Performance Gap Groups”  BUT... STAAR Level III cannot be included in accountability until 2014, so for 2013...  Measure = % of students in Performance Gap Groups “passing” state assessment  Evaluated in EACH subject area (Reading, Math, Writing, Science, Social Studies)  Then cumulated to a single score Index 3 Closing Performance Gaps

52. Index 3: Closing Performance Gaps  Index 3: Closing Performance Gaps? Index 3 Closing Performance Gaps All Eco Dis % Achieving Level III

53. Index 3: Closing Performance Gaps  Student Groups Identified for Evaluation in 2013  Economically Disadvantaged (no MSC)  2 lowest performing Race/Ethnicity groups meeting MSC (≥ 25 tests) across all subjects from Index 1 in 2012  The 2 Race/Ethnicity Groups are determined based on combined performance across all subjects (Index 1)  So the Race/Ethnicity Groups analyzed in Index 3 are the same in all subjects  For Index 3 evaluation, MSC for each Race/Ethnicity Group in each subject area = 25 tests taken by students in that Race/Ethnicity Group in that subject area  If a Race/Ethnicity Group does not meet MSC in a subject area, then it is NOT evaluated in that subject area Index 3 Closing Performance Gaps

54. p. 8 Summary Data – page 8 How were the 2 Race/Ethnicity groups selected? From 2012 performance data

55. p. 10 Summary Chart of 2012 Performance for the 7 Race/Ethnicity Groups Which Race/Ethnicity groups met MSC in 2012 (≥ 25 tests taken by students in that Student Group across all subject areas)? – African American, Hispanic, White If ≥ 3 Race/Ethnicity groups meet MSC, then 2 lowest performing are used for Index 3 – African American (44%) and Hispanic (51%) If only 2 Race/Ethnicity groups meet MSC, then only 1 Race/Ethnicity group is used for Index 3 (the lower performing group) If only 1 (or 0) Race/Ethnicity groups meet MSC, then Index 3 is based solely on Economically Disadvantaged

56. Index 3 Data Reports  Build “back to front”  Start with pages 10 and 11  Then pages 8 and 9  Then top of page 8

57. p. 10  For each subject and each student group, build from the bottom row up  Total Tests  # at Level II Phase 1 or above  % at Level II Phase 1 or above

58. p. 11  Repeat for Science and Social Studies  Total Tests  # at Level II Phase 1 or above  % at Level II Phase 1 or above  Notice that there are fewer than 25 tests taken by African American students in Science and Social Studies  A Student Group is NOT EVALUATED in a subject area UNLESS it meets MSC IN THE SUBJECT AREA (≥ 25 tests taken by students in that student group in that subject area)

59. p. 8 These data come from pages 10-11 p. 10

60. p. 8 In each subject area, for each Student Group meeting MSC:  Assign 1 point for each % at Phase-in 1 Level II or above  Then total Weighted Points  Maximum Points = 100 x Number of Groups meeting MSC ++ ++

61. p. 9  Determine Weighted Performance Rate for Writing, Science, Social Studies  <25 tests taken by Af. Am. students in Sci and SS  Af. Am. student group is NOT evaluated for Index 3 (Max. Pts. = 200) ++ ++ ++

62. p. 8 Index 3 Summarized Data – top of page 8 Total Weighted Performance for each Student Group meeting MSC in each subject area (from bottom of page 8 and page 9) Total Points per subject and Maximum Points per subject Summed across all subjects Total points (750) ÷ Maximum Points (1300) = 57.6, which rounds to 58 Index 3 Score = 58 points Index 3 Score

63. p. 2  What will be the Race/Ethnicity groups in Index 3 for this campus in 2014?  Look at Index 1 data table for 2013  3 Race/Ethnicity groups meet MSC (≥ 25 tests taken by students in that group)  2 lowest performing are: African American and Hispanic  In 2014, Index 3 will be weighted:  1 Point for each % at Level II  2 points for each % at Level III

64. Index 3 is summarized in the campus’ 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1

65. 65 Your Questions Index 3 Closing Gaps Future – % at Level III

66. Index 4

67. Postsecondary Readiness Index 4 Postsecondary Readiness p. 12 p. 13

68. Index 4 Postsecondary Readiness Index 4: Postsecondary Readiness  For 2013 Accountability, Index 4 is based on  Graduation Rate (Cohort Measure)  4-year rate for Class of 2012 Cohort (entered 9 th grade in 2008-09), OR  5-year rate for Class of 2011 Cohort (entered 9 th grade in 2007-08)  RHSP/DAP Diploma Rate (Annual Measure)  %age of students who were awarded diplomas in 2011-12 school year REGARDLESS of when they entered 9 th grade  For high schools without a Graduation Rate, the Grade 9-12 Annual Dropout Rate is used

69. Coming Soon to Index 4...  Postsecondary Readiness in 2014...  Scheduled to include for all campuses  % of students passing one or more STAAR assessments at Level II – Final

70. Index 4 is summarized in the campus’ 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1

71. 71 Your Questions Index 4 Graduation Rate RHSP/DAP Rate

72. 2013 Accountability Achievement  All Students Only  Single percentage combined over ALL Subject Areas  ≥ Level II (Phase 1) STAAR  ≥ Met Std Grade 11 TAKS Progress 10 student groups – All Students – 7 Race/Ethnicity – SpEd – ELL Minimum size – 10 for All Students – 25 for other groups By Subject Area – Rdg & Math in 2013* Based on PROGRESS – 1 point for % Met – 2 points for % Exceeded Closing Gaps 3 student groups – Eco Dis and – 2 lowest Race/Ethnicity groups from prior year By Subject Area (all subjects) 1 point for % Level II (Phase 1) Level III to be added (2014) Postsecondary 4-year or 5-year Graduation Rate (or Annual Dropout Rate) – 10 student groups % RHSP/DAP – 8 student groups (not SpEd or ELL ) STAAR Postsecondary Readiness – Level II Final (2014) Index 1 Student Achievement Index 2 Student Progress Index 3 Closing Performance Gaps Index 4 Postsecondary Readiness Target = 50 Target = 55Target = 75 Target = 30291721 ESMSHSDist

73. 2013 Accountability Achievement 52 points Progress 35 points Closing Gaps 58 points Postsecondary N/A Index 1 Student Achievement Index 2 Student Progress Index 3 Closing Performance Gaps Index 4 Postsecondary Readiness Target = 50 Target = 55Target = 75 Our Sample Campus Target = 30291721 ESMSHSDist Met Standard

74. Campus Distinction Designations

75. Distinction Designations  Campuses that earn Met Standard are eligible for 3 campus distinction designations: p. 14-17 1.Top 25% Student Progress 2.Academic Achievement Distinction Designation (AADD) in Reading/ELA 3.AADD in Math

76. Campus Distinction Designation p. 16  Campus must rank in top quartile (top 10 scores) of Campus Comparison Group Top 25% Student Progress

77. Determining Academic Achievement Distinction Designations (AADDs)

78. Academic Achievement Distinction Designations (AADDs)  AADDs recognize outstanding campus achievement in ELA and Math  Each campus is compared to its campus comparison group on a number of indicators in ELA and in Math that are specific to its campus type  To earn AADD in a subject area, a campus must be in the top quartile of its campus comparison group on the following percentages of the indicators applicable to the campus group type in that subject area:  Elementary and Middle Schools: ≥ 50% of the indicators  High Schools and K‐12 campuses: ≥ 33% of the indicators

79. AADDs p. 24  Reading/ELA Indicators 1.Attendance Rate 2.Exceeded Progress Reading 3.Grade 7 Writing (Level III) 4.Grade 8 Reading (Level III)  Math Indicators 1.Attendance Rate 2.Exceeded Progress Math 3.Grade 5 Math (Level III) 4.Algebra I by Grade 8 Participation 5.Algebra I by Grade 8 Performance (Level III)

80. p. 14

81. p. 15

82. Distinction Designations But how far off are we?

83. p. 14 Score Needed To Get Into Q1 97.1% 17% 2% 16%

84. p. 15 Score Needed To Get Into Q1 97.1% 11% 13% 50%

85. Distinction Designations How do we know this?

86. p. 19 NEW! Released 8/29/2013 My Score Where I WANT To Be

87. Distinction Designations are summarized in the campus’ 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1

88. AADDs (for Elementary Campuses) p. 37  Reading/ELA Indicators 1.Attendance Rate 2.Exceeded Progress Reading 3.Grade 3 Reading (Level III) 4.Grade 4 Writing (Level III)  Math Indicators 1.Attendance Rate 2.Exceeded Progress Math 3.Grade 5 Math (Level III)

90. 2013 Accountability Achievement How did we do? Progress Are our interventions working? Closing Gaps Are we meeting the needs of under- performing groups? Postsecondary Are students ready for college or career? Index 1 Student Achievement Index 2 Student Progress Index 3 Closing Performance Gaps Index 4 Postsecondary Readiness Are we meeting the needs of under-performing groups? Are our interventions working? Are students ready for college or career? How did we do?

91. Appealing Ratings Deadline: September 9, 2013 Appeals are only accepted for a change in rating Appeals are not accepted for: – System Safeguard Measures – Distinction Designations Appeals are not an opportunity to correct data submissions

93. accountability connect  Year-long support for district leaders  Webinars, tools, resources, templates  Telephone/email support  District and campus- level support through district contact  Interpretation of data  Appeals  Planning for 2014

94. Thank You! John Fessenden john@lead4ward.com