1. Presenter: Dr. Charity Fleming Smith Assistant Commissioner Office Public School Academic Accountability Arkansas Department of Education Building a Professional Learning Community: Three Accountability Issues

2. Agenda: Three Accountability Issues 1. Statewide Student Participation in the Central High School 50th Anniversary Celebration – September 25, 2007 2.Test Security Issues 3.Act 35 Gains Model

3. 1. Statewide Student Participation in the Central High School 50th Anniversary Commemoration September 25, 2007 “ It is our mission to encourage young people to take executive responsibility for their education, to step forward boldly and seize any available opportunity to expend awareness and understanding. Little Rock Nine Foundation

4. Theme: Seizing Educational Opportunities: Character and Courage Art and Literacy Scholarship Contest This contest is open to all 12th grade students. Students will: Meet distinguished public officials and visit historical, national monuments and museums in the state’s capitol, including the Clinton Library and the Emancipation Proclamation Exhibit Attend an awards luncheon for all state finalists to receive recognition for their efforts, and the announcement of winners Participating by satellite in the events at Central High from historic Philander Smith College Six Thousand dollars in scholarships will be awarded to student winners, and All schools are encouraged to participate Note: This is made possible by financial support from the Arkansas General Assembly (Cost of transportation, and meals will be covered).

5. What is Correct English Desktop Editor Writing Enhancement Software?  Software that…  Identifies core grammar mistakes,  Provides task-specific revision tips, and  Supports proper Word Choice & Enrichment …to make writing more clear, polished, and effective.  Software that targets all types of writers, purposes and applications, and  Improves writing as a form of communication.

6. What is Correct English Desktop Editor Writing Enhancement Software?  Software that…  Identifies core grammar mistakes,  Provides task-specific revision tips, and  Supports proper Word Choice & Enrichment …to make writing more clear, polished, and effective.  Software that targets all types of writers, purposes and applications, and  Improves writing as a form of communication. Student writers will have a 30-day opportunity to enhance writing skills, and edit essays for the scholarship contest Sponsored by Vantage Learning

7. 2. Test Security The Arkansas Department of Education (ADE), through the Arkansas State Board of Education (SBE), is authorized pursuant to specific federal and state statutes and regulations to investigate alleged violations by a public education agency of requirements associated with its provision of educational programs. Specifically: Arkansas Code Ann. Sections 6-11-105 (Supp.1989) and 6-15-401et seq., as amended.

8. Potential Disciplinary Action The Attorney’s office shall make a preliminary report based on the team’s recommendations and submit the report to the ADE’s Commissioner. After a review by the Commissioner, a course of action shall be taken. The Commissioner may recommend possible action taken against an individual’s Teaching License.

9. 3. Act 35 Annual Improvement Gain Model 3. Act 35 Annual Improvement Gain Model Intention of Act 35 The intention of Act 35 is to promote student learning at all levels so that “all students have an opportunity to demonstrate increased learning” and “meet the expected academic standards.”

10. Intention of Act 35 Cont. The Act 35 Annual Rating System was designed for the purpose of improving student achievement through school accountability and recognition. Act 35, §6-15-1901 (c), requires that each school be classified into “two (2) category levels” as follows: –“Category One” for the “school’s improvement gains” tracked longitudinally using value-added calculation known as the annual improvement category level and –“Category Two” based on “performance from the prior year” referred to as the annual performance category (or “status”).

11. Annual Improvement Category The annual improvement category for rating schools will report each school’s improvement gains by tracking students’ longitudinal achievement gains on the state’s criterion-referenced tests. The base year for the growth model is 06-07. The first reports will be available in December of 2008. A school’s annual improvement gain is based upon the changes in student achievement from one year to the next. No value will be added if instruction does not move a student’s achievement from a given performance category to a higher performance category. If a student’s achievement moves to a lower performance category, then value is lost.

12. Initial Design Considerations 1. As required by Act 35, growth will be based on the tests included in the Arkansas Comprehension Testing, Assessment and Accountability Program (ACTAAP). 2. In subsequent years, if there are changes in any of the tests, the new test versions will be linked to current performance standards. 3. This annual improvement gain model is applicable to any type of student achievement test and is not dependent upon a particular scale score.

13. Initial Design Considerations 4. Any test used for school accountability shall be aligned to the Arkansas Content Standards. 5. An annual improvement gain model will be based on an aggregation of student changes in literacy and mathematics for grades 3-8 and for literacy in grade 11. 6. The annual improvement gain model shall be designed with the expectation that (a) students who are proficient or higher will either maintain or improve their performance classification, and (b) students who are basic or below basic will reach proficiency. 7. The annual improvement gain model shall be transparent, replicable, and easily understood by Arkansas stakeholders.

14. Calculations for Annual Student Growth Student growth is based upon changes in student performance levels across two adjacent years. To assess annual changes more precisely each student performance level (Below Basic, Basic, Proficient, and Advanced) is split into two sub- categories. The following table lists the points assigned to the sub-categories.

15. Table 1: Student Performance Sub-categories Student Performance Sub-categoriesPoints Assigned Below Basic 11 Below Basic 21.5 Basic 12 Basic 22.5 Proficient 13 Proficient 23.5 Advanced 14 Advanced 24.5

16. L i t e r a c y M a t h e m a t i c s Arkansas Benchmark (Grade 6) Arkansas Benchmark (Grade 6) Combined Population Longitudinal Gains

17. L i t e r a c y M a t h e m a t i c s Arkansas Benchmark (Grade 8) Arkansas Benchmark (Grade 8) Combined Population Longitudinal Gains

18. Mid - Year (January) Spring (April) Arkansas Algebra I Arkansas Algebra I Combined Population Longitudinal Gains

19. Mid - Year (January) Spring (April) Arkansas Geometry Arkansas Geometry Combined Population Longitudinal Gains

20. Scale Scores That Define the Sub-categories The scale scores that define the sub-categories are listed in the Technical Memorandum from Drs. Huynh Huynh, Robert Kennedy, and Eugene Kennedy, dated April 19, 2007. The memorandum was reviewed and approved by the Technical Advisory Committee on Accountability.

21. Value-added Points Table 2 shows the value-added points for a student based upon whether or not he/she improves in achievement, stays the same in achievement, or regresses in achievement. For example, if a student in third grade earns a achievement rating of Proficient 2 and the following year the same student in fourth grade receives an achievement rating of advanced 1, he/she would be given a value-added score of +0.5. Similar value-added scores would be calculated for each child in each school.

22. Table 2: Value-added Points for Changes in Student Achievement Previous Year Current Year 11.522.533.544.5 10.511.522.533.5 1.5-.50.511.522.53 2-.50.511.522.5 -1.5-.50.511.52 3-2-1.5-.50.511.5 3.5-2.5-2-1.5-.50.51 4-3-2.5-2-1.5-.50.5 4.5-3.5-3-2.5-2-1.5-.50

23. Computation of Annual Improvement Gain Index For each school, the Annual Improvement Gain Index is the average of all value-added points across subject areas and grades. Potentially, the range of the Annual Improvement Gain Index is from -3.5 to +3.5. The value of zero indicates no growth, positive values indicate improvement, and negative values indicate a decline in achievement.

24. Multiply the number of students (5) by the points assigned to the category. For example multiply (5) times (4) to get the points produced by the students in the Advanced category 1. Add up the points for each category. Divide the total points for the school (285) by the number of students (100) to get a rating score (2.85), which is meeting standards. Number of StudentsCategoriesPoints Assigned to CategoriesTotal 5Below Basic 115 5Below Basic 21.57.5 10Basic 1220 Basic 22.550 30Proficient 1390 20Proficient 23.570 5Advanced 1420 5Advanced 24.522.5 Total Points for the school for all categories285 How School Ratings Are Calculated

25. Cut ScorePerformance CategoryRating 3.23 - 4.0Schools of excellence5 2.79 - 3.22Schools exceeding the standards4 2.21 – 2.78Schools meeting standards3 1.719 – 2.20Schools approaching the standards (alert) 2 1.0 - 1.718Schools in need of immediate improvement 1 Example of Standards Setting

26. Procedures for Setting Cut Scores for Annual Improvement Gain Levels According to Act 35, schools will be classified into one of five Annual Improvement Gain Levels. Therefore, four (4) cut scores will need to be set. The general steps previously used in setting cut scores for Category Two, Annual Performance Category Levels (Status Index), will be implemented. The steps are: (1) preliminary work by the Technical Advisory Committee on Accountability, (2) advice by Arkansas stakeholders at a meeting, and (3) adoption by the Arkansas State Board of Education.

27. Preliminary Work by the Technical Advisory Committee on Accountability The distribution of the school Annual Improvement Gain Index will be compiled based on the matched Spring 2006 and Spring 2007 data. The above distribution will be divided into ten equal parts. A representative sample of schools in each part will be selected and relevant data about those schools will be provided.

28. Preliminary Work by the Technical Advisory Committee on Accountability Those data shall include, but not be limited to, the following variables: –School size –School grade level configuration –Demographic data, such as proportion of each ethnic group, mobility, free and reduced lunch, ELL, Special Education students, etc. –Proportion of students who meet the AYP growth trajectory as defined in the Arkansas AYP Growth Model –Ratings on status models in the 2005, 2006, 2007 –Other characteristics as defined by field review

29. Advice by Arkansas Stakeholders A meeting of Arkansas stakeholders will be convened to recommend Annual Improvement Gain cut scores to Arkansas Department of Education. The same process used in the status standard setting meeting will be implemented. Additionally for continuity, some of the same panelists will be invited to participate. The new panel will be supplemented with new persons.

30. Presentation Made at State Board of Education Meeting The recommended cut scores will be presented to the State Board along with relevant data for their deliberation and final action.

31. Timeline November 2008 to Rate Schools on Growth 07-08 Testing (Using data from 06-07 and 07-08) July 2007Present Initial Plan to State Board for information September 20-21, 2007 (TAC Meeting) Run computation of baseline year school year 2006-2007. Review growth analysis across the two years October 2007School Awareness Training November 15-16, 2007 (TAC Meeting) Standard Setting for school gain index December 2007Initial report on school gain index to State Board January 2008State Board action on criteria for annual improvement gain category levels February 2008School awareness with stakeholders (second round)

32. Timeline March 13-14, 2008 (TAC Meeting) First draft of Technical Report-Annual Improvement Gains Category April 2008Students are assessed July 2008NORMES produces initial matched data set August 2008Initial matched data sent to ADE for review September 1-15, 2008Schools review matched data set and report any necessary revisions to the ADE and NORMES (online) October-November 2008NORMES and ADE validate preliminary gain scores December 2008Preliminary release of Annual Improvement Gains Report on ADE website

33. Annual School Performance Report Timeline September 30, 2009 Release of achievement data from NORMES October 15, 2009 Deadline for requesting the Annual Performance Report (Rating Schools) November 2009 Annual Performance Reports are available for those who request it March 15, 2010 Annual Improvement Gains Report in the Annual School Performance Report (Report Card) March 15, 2011 Official release of both reports: Annual Improvement Gains Report and the Annual School Performance Report reflecting the Spring 2010 achievement data in the Annual School Performance Report (Report Card)