1. New Hampshire’s 2007 AYP Status Reports & Follow The Child Growth Reports August 20, 2007

2. NH Department of Education 2 Table of Contents What is an accountability system ? 2007 AYP Status Results  p.4 release schedule  p.5 2006/2007 Comparison  p.6 Basic Calculation  P.7 Gr. 3-8 NECAP Achievement Levels  P.8 Taskforce Recommendations  P.9 Minimum n definition  P.10-11 Index Definition and example  P.12-13 NECAP and NH-Alt conversions  P.14-15 Starting Points and AMOs  P16-18 Confidence Intervals and chart  P.19 Safe Harbor and example  P.20-22 Sample Report Shells  P.23 Special Notes on Data  P.24 Results and Resources  P.25 Future work NH Follow The Child Growth Reports  P.26 History of the NH Growth Model  P.27 Why did NH submit a proposal for a growth model?  P.28 What is the NH Follow The Child Growth Model?  P.29 NH Follow The Child Growth Expectations  P.30 Growth Groups by NECAP Scaled Score  P.31 Reading Growth Targets  P.32-33 Sample Report Shells  P.34 What has changed now that the Growth Model is New Hampshire’s?  P. 35 Why Use Two Models?  P.36 So What Does All This Mean?  P.37 Who created NH Model?  P.38 Timeline for AYP and Growth Release  P.39 Contact Information

3. August 20, 2007 NH Department of Education 3 What is an Accountability System? State Standards - Curriculum Frameworks - Standards for School Approval Valid & Reliable Assessment System - state assessment - local formative, benchmark & competency based assessments Data Analysis Tools - Performance Pathways Accountability Reporting - Assessment Reports - NCLB AYP Status Reports - NH Growth Reports Statewide System of Support - School Improvement Coaches (content, data, special education, leadership, NH-Alt) - Leadership Institute - Literacy Action Plan - Numeracy Plan - Ongoing PD - DINI support - High School Vision Statement - High School Redesign - PD Master Plans - Special Education Focused Monitoring - Root Cause Analysis program

4. August 20, 2007 NH Department of Education 4 2007 AYP Status Results – release schedule (impacts school year 2007-2008) AYP DefinitionGradesTesting Data with dates Report Issued School District State Index based on NECAP and NH-Alt (since 2006) Elementary and Middle Grades Gr. 3-8 NECAP (Oct 2006) Gr. 2-7 NH-Alt (2005-2006) Aug. 28 2007 10:00 AM District% Basic or above based on NHEIAP and NH-Alt (no information included In this ppt) High SchoolGr. 10 NHEIAP (May 2006) Gr. 10 NH-Alt (2005-2006) Aug. 28 2007 10:00 AM School State High SchoolGraduation Rate Only Aug. 28 2007 10:00 AM

5. August 20, 2007 NH Department of Education 5 2007 AYP Status Results – 2006/2007 Comparison What’s the Same as in 2006 ? Factors included (participation, other indicator, performance) Index values (20, 40, 60, 80, 100) AMOs (Annual Measurable Objective) Report shell (mostly) What’s Different? Calculating Safe Harbor Cleaning up the demographic data Report Shell (2% calculation & overall determination)

6. August 20, 2007 NH Department of Education 6 Calculate “Other Indicator” for the school (attendance rate at 3-8; graduation rate at high school) Calculate Participation rate for the school and each subgroup (based on testing year) Calculate Performance for the school and each subgroup (based on teaching year) –Calculate Index (Compare to AMO target) –If not OK, check confidence interval (99%) –If still not OK, check safe harbor –Check 2% in case of Special Education subgroup 2007 AYP Status Results – Basic Calculation

7. August 20, 2007 NH Department of Education 7 Grades 3-8 NECAP Achievement Levels NECAP Cut points Commissioners from VT, RI, and NH adopted cut points on January 20, 2006 Proficiency at the student level includes performance at achievement levels 3 and 4. Achievement Levels: Level 1: Substantially Below Proficient Level 2: Partially Proficient Level 3: Proficient Level 4: Proficient with Distinction Scaled Scores will be reported as a 3-digit number where the first digit is the grade level and the other part will be a score 00 to 80 (reported as whole numbers at the student level). X40 is the reported cut point for Proficient. Grade 3: 300-380 Grade 4: 400-480 Grade 5: 500-580 Grade 6: 600-680 Grade 7: 700-780 Grade 8: 800-880

8. August 20, 2007 NH Department of Education 8 2007 AYP Status Results – Taskforce Recommendations An External AYP Task Force met to make recommendations to the Department. After review by the Internal AYP Task Force, we have arrived at the following decisions:  Racial categories (no change) Defined by Beginning of Year ( BOY ) submissions  Other Indicator (no change) Attendance Rate (90% or improvement over previous year) Calculated from End of Year (EOY) submissions  Participation assigned to testing school  Performance assigned to teaching school  Full Academic Year (FAY) for performance calculations Continuous enrollment from October 1 to the end of the teaching year

9. August 20, 2007 NH Department of Education 9 2007 AYP Status Results – Minimum “n” definition (Minimum number of students in a group required in order to perform calculation) 95% Participation rate: 40 for each group within testing grades in school or district 75% Graduation rate: 40 within school or district 90% Attendance rate: 40 within school or district  Includes all grades 1-8 in the school or district Performance targets:11 for each group within testing grades in school or district

10. August 20, 2007 NH Department of Education 10 2007 AYP Status Results - Index Definition Index System provides partial credit for scores below Proficient. A school’s index score will be the average of all student index points assigned to the school. Proficiency LevelIndex Points Level 1: Substantially below Proficient X000 1a20 1b40 Level 2: Partially Proficient 2a60 2b80 Level 3: Proficient Level 4: Proficient with Distinction 100

11. August 20, 2007 NH Department of Education 11 An Index Example: Two schools Two schools, both with 180 students tested and 61.1% of their students scoring proficient or better in reading. READINGMaple Street CSPine Street MS Levelpoints# of students total# of students total X00030050 Level 1a20 4005100 Level 1b401040010400 Level 2a605300201200 Level 2b805400302400 Level 3100606000606000 Level 4100505000505000 Sum1801250018015100 Index69.483.9

12. August 20, 2007 NH Department of Education 12 Scaled Scores for each Index NECAP Reading 01a1b2a2b3 and 4 300301 to 321322 to 330331 to 335336 to 339340 to 380 400401 to 421422 to 430431 to 435436 to 439440 to 480 500501 to 519520 to 529530 to 534535 to 539540 to 580 600601 to 617618 to 628629 to 634635 to 639640 to 680 700701 to 717718 to 728729 to 734735 to 739740 to 780 800801 to 815816 to 827828 to 833834 to 839840 to 880 Mathematics 01a1b2a2b3 and 4 300301 to 323324 to 331332 to 335336 to 339340 to 380 400401 to 421422 to 430431 to 435436 to 439440 to 480 500501 to 525526 to 532533 to 536537 to 539540 to 580 600601 to 625626 to 632633 to 636637 to 639640 to 680 700701 to 727728 to 733734 to 736737 to 739740 to 780 800801 to 827828 to 833834 to 836837 to 839840 to 880

13. August 20, 2007 NH Department of Education 13 Raw Score to Performance Index Conversion table for NH-Alt * The lowest score any scoreable portfolio can earn is 13 raw score points. ** A raw score of 0 is only possible if the portfolio submitted was judged to be unscoreable by two trained and independent scorers. Portfolio Raw Score Points Earned Proficiency LevelPerformance Index Level Performance Index Points Assigned 47-52Proficient with Distinction4100 38-46Proficient3100 34-37Partially Proficient2b80 29-33Partially Proficient2a60 21-28Substantially Below Proficient1b40 13*-20Substantially Below Proficient1a20 0**Un-scoreable (SBP)00

14. August 20, 2007 NH Department of Education 14 AYP Status Def: Starting Points Starting Points: For each content area separately, a baseline was created via the 20% method outlined in NCLB:  Determine index for each school  Rank schools by each index  Identify at “20 th percentile” school “20 th percentile” school: the school where 20% of the students in the whole list attend that school or a school with a lower index.  That school’s index is the starting point Starting points: Reading 82, Mathematics 76

15. August 20, 2007 NH Department of Education 15 AYP Status Def: AMOs Annual Measurable Objectives 2 year intervals, equally spaced to 100 Grades 3-8Index ReadingMathematics Starting Point (2005-2006)8276 2006 – 20078276 2007 – 20088682 2008 – 20098682 2009 – 20109188 2010 – 20119188 2011 – 20129594 2012 – 20139594 2013 – 2014100 So, Pine St. MS met the AMO since their index (83.9) is greater than or equal to 82 in reading, but Maple St. CS did not.

16. August 20, 2007 NH Department of Education 16 AYP Status Def: Confidence Intervals 1. Calculate the within- school variance for each school. 2. Calculate the average within-school variance for the state. 3. Calculate the standard error for each group 4. Calculate the 99% confidence interval for each group Where: index student = index score for the student mean_index school = average index score for the school to which the student belongs n index student = the number of student index scores for the school of interest n schools = the number of schools, and n group = number of students in the group For each subject separately: For NH Reading:603.38615 Math:800.72625 The smallest index allowed with the CI is: AMO – CI group

17. August 20, 2007 NH Department of Education 17 Same as last year since AMOs are the same.

18. August 20, 2007 NH Department of Education 18 Maple St. CS still did not meet AMO with the confidence interval since their index (69.4) is not greater than or equal to 77.7

19. August 20, 2007 NH Department of Education 19 AYP Status Def: Safe Harbor (with an example) Even though a group does not make performance expectations, it may have improved enough to be okay. To make Safe Harbor a group must meet the 10% rule and the additional indicator. The 10% rule requires that the “complement” of the group’s Index (100-Index) be at least 10% lower than the previous year. In addition, the percent of students scoring proficient or better must increase. October 2005 NECAP & 2004-05 NH-Alt October 2006 NECAP & 2005-06 NH-Alt ReductionSH Goal (’05 100-Index) x (.10) Safe Harbor? Is reduction > or = (SH Goal)? Index100-IndexIndex100-Index Maple Street CS 613969.430.639-30.6 = 8.4 39 x.10 = 3.9 Is 8.4>3.9 Yes Gould School 6139633739-37 = 239 x.10 = 3.9 Is 2>3.9 No Maple St. CS, however, improved enough to meet performance expectations by meeting safe harbor requirements.

20. August 20, 2007 NH Department of Education 20

21. August 20, 2007 NH Department of Education 21

22. August 20, 2007 NH Department of Education 22

23. August 20, 2007 NH Department of Education 23 2007AYP Status Results - Special Notes on Data Reports are based on student demographic and program participation data reported by districts  EOY (End of Year) files  BOY (Beginning of Year) files  SPEDIS/NHSEIS, ESL, F&R systems Assessment reports were released in January 2007 and districts/schools had about a month to review and report and correct discrepancies in student demographic data.

24. August 20, 2007 NH Department of Education 24 2007 AYP Status Results and Resources www.ed.state.nh.us/education/ayp  AYP Status Reports at grades 3-8  AYP Status support material  CSV file of all results  New Hampshire’s AYP Status Reports & Follow The Child Growth Reports (ppt)  Separate HS District Reports (old AYP def)  Separate HS School Reports (graduation rate only)

25. August 20, 2007 NH Department of Education 25 AYP Status – Future work Out of District Students in Private Placements  How to include students placed in out-of-district private placements? Writing and Science  Neither Writing nor Science is scheduled to be a part of AYP at this time

26. August 20, 2007 NH Department of Education 26 History of the NH Growth Model Growth Model being considered by Accountability Task Force Lyonel Tracy becomes NH Commissioner Follow The Child Initiative Begins Spellings invitation to states to submit growth model proposals NH submits Growth proposal in February 2006 USED defers NH proposal until two years of assessment data NH revises and resubmits in November 2006 USED asks for clarifications in December 2006 NH submits revisions in January 2007 Peer Review #1 in March 2007 Peers ask for compilation of all revisions and clarifications (Task Force reviews and refines proposal) Peer Review #2 in May 2007 May 22 – conference call informing us that the peers voted to not approve our model – but, we could come to DC in June to work toward approval June 4, 2007 – AYP Task Force advises Commissioner to run the NH Growth Model as our own pilot this year

27. August 20, 2007 NH Department of Education 27 Why did NH submit a proposal for a growth model? The Accountability Task Force had been considering growth for a while and the idea reflected the goals of Follow The Child To build on the NH philosophy of continuous improvement and longitudinal student growth To allow us to determine individual growth targets for students that accelerate progress & close the achievement gap To include individual targets as part of a larger system that includes interim testing, personalization, and remediation, if necessary

28. August 20, 2007 NH Department of Education 28 What is the NH Follow The Child Growth Model? ( www.ed.state.nh.us/education/FTC/growth_model.pdf) www.ed.state.nh.us/education/FTC/growth_model.pdf We calculate growth targets for EVERY student, and tally the number of students meeting or exceeding their target The targets are based on the previous year’s NECAP score – and if no NECAP score is available, the target is proficiency Targets are based on the distance to proficiency – closing the gap (as measured by the number of standard deviations below proficiency) This ensures comparability between grades If students are already proficient, the target is designed to encourage a level that exceeds proficiency

29. August 20, 2007 NH Department of Education 29 NH Follow The Child Growth Expectations >1 SD below ½ to 1 SD below 0 to ½ SD below 0 to ¼ SD above ¼ to 1¼ SD above > 1 ¼ SD above Narrow gap to prof. by one third of the # of SD below Narrow gap to prof. by one half of the # of SD below Proficiency Drop by no more than ¼ SD Stay at least 1 SD above prof. Previous NECAP scaled score. Targets for next testing cycle.

30. August 20, 2007 NH Department of Education 30 Growth Groups by NECAP Scaled Score Group 12345 Math >1SD below 1SD – ½SD Below Proficient-½SD to Proficient + ¼SD ¼SD – 1¼SD above >1¼SD above Grade 3300-329330-334335-342343-353354-380 Grade 4400-428429-434435-442443-454455-480 Grade 5500-528529-534535-542543-554555-580 Grade 6600-628629-634635-642643-654655-680 Grade 7700-729730-734735-742743-753754-780

31. August 20, 2007 NH Department of Education 31 Reading Growth Targets Grades 3 to 4Grades 4 to 5Grades 5 to 6Grades 6 to 7Grades 7 to 8 ScoreTargetScoreTargetScoreTargetScoreTargetScoreTarget 300416400513500612600713700814 301416401514501612601714701814 302417402514502613602714702815 303417403515503614603715703816 304418404516504614604716704816 305419405516505615605716705817 306419406517506616606717706818 307420407518507617607718707818 308420408518508617608718708819 309421409519509618609719709820 310422410520510619610720710820 Complete reading and mathematics growth targets charts available at: www.ed.state.nh.us\education\FTC\growth\reading_growth_targets.pdf www.ed.state.nh.us\education\FTC\growth\math_growth_targets.pdf

32. August 20, 2007 NH Department of Education 32 Confidential list of students with scores from 2005 and 2006, and targets for 2006 and 2007.

33. August 20, 2007 NH Department of Education 33 Public report. Growth targets have been set at the school level (listed next to each content area heading). The school and each group may meet the school growth targets directly or with the use of confidence interval.

34. August 20, 2007 NH Department of Education 34 What has changed now that the Growth Model is New Hampshire’s? Confidence Interval based on growth targets and group size (99%) to identify groups not reaching growth targets with more confidence AMO Targets for future years to be determined after reviewing pilot results Can we/do we include growth reports for high school?

35. August 20, 2007 NH Department of Education 35 Why Use Two Models? The FTC growth model encourages schools to focus on all students, not just the students that scored just below the performance level cuts. In contrast, the status/index model rewards schools for improvement that crosses achievement levels, regardless of the amount of growth. New Hampshire feels that a valid accountability system should incorporate both status and growth and public reporting of other assessments throughout the year.

36. August 20, 2007 NH Department of Education 36 So What Does All This Mean? NCLB Accountability is based on the Index The FTC Growth Report is a NH Pilot The school growth reports are for informational purposes to inform teachers, administrators, and the public The student roster information is for teacher instructional use and student goal setting Growth targets give us “one more picture” of student performance

37. August 20, 2007 NH Department of Education 37 Who created NH Model? The New Hampshire Accountability Task Force made up of: District and school personnel NHDOE staff Representatives from university system, school boards, parents Technical advisors from National Center for the Improvement of Educational Assessment (NCIEA) and Measured Progress

38. August 20, 2007 NH Department of Education 38 Timeline for AYP and Growth Release Tuesday, August 28 at 10:00 AM – public release of index reports and school, district, & state growth reports 2007-08 school year: review & work with districts on growth targets and growth reports Use growth results to determine AMOs and future use in NH Accountability System

39. August 20, 2007 NH Department of Education 39 Contact Information Deb Wiswell (Accountability) dwiswell@ed.state.nh.us 603-271-3828 Tim Kurtz (Curriculum and Assessment) tkurtz@ed.state.nh.us 603-271-3846 Gary Guzouskas (School Improvement and Appeals) gguzouskas@ed.state.nh.us 603-271-5873