1 Utah Performance Assessment System for Students U-PASS Accountability Plan Judy W. Park Assessment & Accountability Director Utah State Office of Education October 7, 2005

2 U-PASS Product of Utah Code 53A-1-605, establishing an accountability system for Utah public schools Uses multiple measures to determine acceptable levels of school performance Focus is on individual student progress Includes total school and subgroup accountability Electronic Report Assessments & Indicators Disaggregated information Additional Information Previous U-PASS Report Elements

3 U-PASS Legislation 53A (1) …develop a plan to analyze the results of the U-PASS scores for all grade levels and courses required under Section 53A and the student behavior indicators referred to in Subsection 53A (3) (e).

4 U-PASS Legislation 53A (b) …identify schools not achieving state established acceptable levels of student performance in order to assist those schools in raising their student performance levels.

5 U-PASS Accountability Plan 95% Student Participation on Assessments 95% Student Participation on Assessments Status Scores Status Scores Percentage of students proficient on the collection of assessments and other indicators Percentage of students proficient on the collection of assessments and other indicators Progress Scores Progress Scores Longitudinal measure based on comparing the achievement levels of the same student from one year to the next Longitudinal measure based on comparing the achievement levels of the same student from one year to the next

6 Schools will be identified as: Achieved State Level of Performance Achieved State Level of Performance Participation is 95% and Participation is 95% and Total School Status is acceptable or progress is medium or high Total School Status is acceptable or progress is medium or highAND Subgroup Status is acceptable or progress is medium or high Subgroup Status is acceptable or progress is medium or high Needs assistance Needs assistance Participation is less than 95% or Participation is less than 95% or Total School Status is not acceptable and progress is low Total School Status is not acceptable and progress is lowOR Subgroup Status is not acceptable and progress is low Subgroup Status is not acceptable and progress is low

7 U-PASS Accountability Plan Multiple Assessments & Indicators Multiple Assessments & Indicators English language arts CRT English language arts CRT Math CRT Math CRT Science CRT Science CRT Direct Writing Assessment Direct Writing Assessment Utah Basic Skills Reading Assessment Utah Basic Skills Reading Assessment Utah Basic Skills Writing Assessment Utah Basic Skills Writing Assessment Utah Basic Skills Math Assessment Utah Basic Skills Math Assessment UALPA UALPA UAA UAA Attendance Attendance Graduation Rate Graduation Rate

8 Elementary/Middle Status and Progress Score Language Arts (35%) Language Arts (35%) ELA CRT (35%) ELA CRT (35%)Or ELA CRT (30%) ELA CRT (30%) & DWA 5% Science (20%) Science (20%) Science CRT Science CRT Math (35%) Math (35%) Math CRT Math CRT Attendance (10%) Attendance (10%)

9 High School Status and Progress Score Language Arts (30%) Language Arts (30%) ELA CRT (60%) ELA CRT (60%) Or CRT 50% & DWA 10% Or CRT 50% & DWA 10% UBSCT reading (25%) UBSCT reading (25%) UBSCT write (15%) UBSCT write (15%) Science (25%) Science (25%) All Science CRT All Science CRT Mathematics (25%) Mathematics (25%) Math CRT & Courses Math CRT & Courses(50%) UBSCT math (50%) UBSCT math (50%) Attendance (10%) Attendance (10%) Graduation rate (10%) Graduation rate (10%)

10 U-PASS Accountability Plan English Language Learners English Language Learners Students focus on learning academic English skills first (up to 3 years) Students focus on learning academic English skills first (up to 3 years) After three years, or when students achieve language proficiency, they become accountable for proficiency on U-PASS assessments After three years, or when students achieve language proficiency, they become accountable for proficiency on U-PASS assessments

11 ELL Students 2005 ELL students (level A and B) in country less than 3 years, exempt from U-PASS Accountability Plan 2006 ELL students (level A or B) in country less than 3 years, their proficiency on CRT and DWA will be substituted for proficiency on UALPA

12 U-PASS Accountability Plan Students with Disabilities Students with Disabilities Five participation options Five participation options Standard administration Standard administration Accommodations Accommodations Modifications Modifications Utah Alternate Assessment – 1% Utah Alternate Assessment – 1% Additional flexibility – 2% ? Additional flexibility – 2% ?

13 U-PASS Accountability Plan Economically disadvantaged Economically disadvantaged English Language Learners English Language Learners Ethnicity Ethnicity African American African American American Indian American Indian Asian Asian Caucasian Caucasian Hispanic Hispanic Pacific Islander Pacific Islander Students with Disabilities Students with Disabilities Total Group Total Group ELL proficiency levels ELL proficiency levels A,B,C,D,E A,B,C,D,E Gender Gender Migrant Migrant Mobility Mobility Students without Disabilities Students without Disabilities Extensive subgroup reporting

14 U-PASS Accountability Plan Aggregate Subgroup Accountability Every student who belongs to a subgroup other than white Individual student proficiencies are added together and divided by the number of students to determine the subgroup proficiency level Any individual subgroup not meeting proficiency will appear on the front page of the report

15 U-PASS Accountability Plan Subgroup Status Accountability Example Total School 100 White 82 Hispanic 12 American Indian 2 Pacific Islander 2 Low Income 29 SWD 12 ELL 8 Calculation: = 65 student tests included in the subgroup calculation

16 U-PASS Accountability Plan Subgroup Status Accountability Allows greater accountability Students in a subgroup with less than 10 are now included in the aggregate calculation Students are more fairly represented Each student only counts once, regardless of the number of subgroups in which the student qualifies. Increased reliability We are evaluating a larger group More students and more schools are represented Increased validity We do not overweight small populations. This lifts the burden of one subgroup being the sole determinant of unacceptable status of the school.

17 Progress Value Table Draft (to be finalized later this month) Year 1 Year 2 Level 1a1b2a2b34 1a b a b

18 U-PASS Progress Example Cherry Middle School Total Population 800 Students # of Students Progress Score 50x100 =5, x125= x150= x200= x250= x300= x350=35000 Total800 (divide into) = 225 Progress Score - 225

19 U-PASS Timeline CRT established baseline 2005 CRT begins progress score 2005 Status & Progress for all but High School and ELL UBSCT in effect for 2006 UALPA implemented Status & Progress for all but HS and ELL progress UBSCT implemented 2006, progress 2007 UALPA implemented 2006, progress Status & Progress for all

20 Previous U-PASS Report Elements School summary information Student summary information NRT results by State, District and School CRTs for the past two years by District and School

21 Previous U-PASS Report Elements

22 Previous U-PASS Report Elements

23 Previous U-PASS Report Elements

24 The 2005 U-PASS Report Web-based report format An increased amount of information available on each school Allows for “drill-down” information

25

26 1

27 2

28 3

29 Must achieve both criteria

30 Final Result

31 Subgroups Needing Assistance

32 Status “Drill Down”

33 Second Level “Drill Down”

34 Third Level “Drill Down”

35 Progress “Drill Down”

36 Second Level “Drill Down”

37 Progress Table “Draft” Third Level “Drill Down”

38 Additional Reports

39 AYP Reports

40 U-PASS Additional Information: NRT ACT/SAT AP Concurrent Enrollment School Summary Information Student Summary Information Percent of student reading on grade level (reported for grades 1-10) Dropout Rate (disaggregated by ’08) Disciplinary Action Course Taking Patterns and Trends

41 Timeline for U-PASS  December ’05  Electronic Report available for the following elements: Summary report Drill down to Level 1 of Participation Drill down to Level 1 of Status for LA CRT’s – 2 yrs. Drill down to Level 1 of Status for Math CRT’s – 2 yrs. (applied math 1 and 2 available ’06) Drill down to Level 1 of Status for Science CRT’s – 2 yrs.

42 Timeline for U-PASS  December ‘05  Attendance  Graduation Rate  (Disaggregated report available in ’08)  Other reports:  School summary information  Student summary information  NRT, AP, ACT, SAT, Concurrent Enrollment, Percent of students reading on grade level (grades 1-10)

43 Timeline for U-PASS Results will be published beginning December 2005 and continuing through May 2006 as information becomes available and the capacity to collect, report, and disaggregate the information is completed. Results for U-PASS 2006 will be available by September 2006

44 U-PASS Final Decisions October 18 Set the bar for Status What percentage is acceptable? Set the bars for Progress What is low What is medium What is high

45 Questions?