Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Measure Up! Benchmark Assessment Quality Assurance Process RCAN September 10, 2010.

Similar presentations


Presentation on theme: "1 Measure Up! Benchmark Assessment Quality Assurance Process RCAN September 10, 2010."— Presentation transcript:

1 1 Measure Up! Benchmark Assessment Quality Assurance Process RCAN September 10, 2010

2 2 Measure Up! Objective To monitor and improve Benchmark Assessments in order to attain the most accurate possible measurement of student achievement with respect to California Content Standards Tests.

3 Measure Up! Components Content—Structure & Course Guides Predictability – Correlation of Benchmark Exams to CST scores – Association of Benchmark Exams with CST Performance Levels Item Analysis – Difficulty – Discrimination – Representative of CST items Evolving

4 4 PSUSD Benchmark Structure CST ~153 ID 60 to 75 Items Blueprint Aligned Benchmark #1 ~45 ID ~20-35 Items 1 st 45 ID Paced Standards Partial Match to CST Benchmark #2 ~90 ID ~20-35 Items 2 nd 45 ID Paced Standards Partial Match to CST Benchmark #3 ~135 ID ~20-35 Items 3 rd 45 ID Paced Standards Partial Match to CST

5 Algebra I (8 th Grade) Aggregation of 3 Benchmarks Algebra I (8 th Grade) CST Predictability

6 6 Prof = Algebra I BM (8 th Grade) Algebra I CST 2009 (8 th Grade)

7 7 N = 23 N = 119N = 125 N = 51 N = 5

8 8

9 ELA 10 th Grade Aggregation of 3 Benchmarks ELA 10 th Grade CST Predictability

10 10 ELA 10 th Gd BM ELA 10 th Gd CST 2009 Prof = 53 53

11 11 N = 26 N = 151 N = 326 N = 249 N = 36

12 12

13 US History 11 th Grade Aggregation of 3 Benchmarks US History 11 th Grade CST Predictability

14 14 r =.46 US His 11 th Gd BM US His 11 th Gd CST 2009 Prof = 40 63

15 15 N = 68 N = 116 N = 142 N = 72 N = 7

16 16

17 Science 8 th Grade Aggregation of 3 Benchmarks Science 8 th Grade CST Predictability

18 18 r =.77 Science 8 th Gd BM Science 8 th Gd CST 2009 Prof = 39 44

19 19 N = 84 N = 307 N = 494 N = 275 N = 24

20 20

21 Math 6 th Grade Aggregation of 3 Benchmarks Math 6 th Grade CST Predictability

22 22 r =.84 Math 6 th Gd BM Math 6 th Gd CST 2009 Prof = 43 48

23 23 N = 164 N = 393 N = 460 N = 224 N = 31

24 24

25 ELA 4 th Grade Aggregation of 3 Benchmarks ELA 4 th Grade CST Predictability

26 26 r =.80 Prof =

27 27 N = 88 N = 176 N = 334 N = 388 N = 93

28 28

29 29 Item Level Analysis

30 30 Item Difficulty The p value for any item – percentage of correct answers – usually in decimal form Ideally p value range is.30 to.80 for most items For example – p value of.28 = 28% of the test takers got the item right – p value of.75 = 75% of the test takers got the item right – P value of.95 = 95% of the test takers got the item right

31 31 Item Difficulty Monitoring Item NumberN StudentsStandardp Value 1853NS NS NS MG MG MG PS PS PS PS3.3.95

32 32 Item Discrimination Is item “discriminating” appropriately between higher & lower scoring students Discrimination Index (DI) = difference between how upper half and lower half of students score on an item DI ranges between -1 and +1 We want items to discriminate positively

33 33 Item Discrimination Monitoring Item Number N Students Standardp ValueDIMax DI 1853NS NS NS MG MG MG PS PS PS PS

34 th Grade Math 3 rd Benchmark th Grade Math 3 rd Benchmark Representative of CST Items and our continual Revision Process

35 35 7NS Item p =.33 DI =.34

36 36 RTQ for 7NS1.6 CST = 1 RTQ = 1

37 37 7NS Item p =.42 DI =.31

38 38 Additionally… Item #3 replaced (as #3) with item modeled after RTQ # 16 (7NS1.7*)—CST = 5, RTQ = 7 Item #15 replaced (as #13) with item modeled after RTQ #47 (7AF1.5) Item #s 23 & 24 replaced (as #s 21 & 22) with items modeled after RTQ #s 89 & 88 (7MG3.4*)

39 39 Measure Up! Next Steps Benchmark Exam Structure Institutionalize System CAHSEE Writing Prompt Discrimination Distractor Shaping

40 40 Questions?


Download ppt "1 Measure Up! Benchmark Assessment Quality Assurance Process RCAN September 10, 2010."

Similar presentations


Ads by Google