Download presentation

Published byAstrid Cripps Modified over 2 years ago

1
**Using Test Item Analysis to Improve Students’ Assessment**

Institutional Assessment starts with Classroom Assessment

2
**Learning Objectives of This Session**

1. Explain difficulty index and discrimination index 2. Calculate difficulty index and discrimination index 3. Identify ineffective distracters 4. Evaluate multiple-choice test items based on analysis results 5. Apply table of specifications to improve content validity

3
**Purpose of Item Analysis**

1. Ensure accurate measurement of knowledge or skill 2. Enhance student learning 3. Increase student engagement 4. Avoid demoralizing students 5. Increase confidence in drawing conclusions Outcome achievement Level of knowledge or skill mastery Teaching effectiveness

4
**Components of a multiple-choice item**

Test items used to measure the lowest level of cognitive taxonomy are (stem) Analysis (distracter) Application (distracter) Knowledge (key) Comprehension (distracter) The correct answer usually numbered as 1 The wrong answers usually numbered as 0

5
**Two important indexes for Item Analysis**

Item Difficulty Index To tell how hard the item is Item Discrimination Index To tell how well the item to distinguish between high ability and low ability students

6
Item Difficulty Index Is defined as the percentage or proportion of test takers who correctly answer the item. For example, in a class of 30 students, if 20 students get the item correct and 10 are incorrect, the item difficulty index is 20/30 =0.67 Range from 0 to 1

7
**ITEM DIFFICULTY = NO. CORRECT / TOTAL**

Students Item1 Item2 Item3 Item4 Item5 Robert Millie Dean Shenan Cuny Corky Randy Jeanne Iliana Lindsey Item p =

8
**Optimal P Values for Items with Varying Number of Options**

Optimal Mean p Value 2 0.85 3 0.77 4 0.74 5 0.69

9
**Special Assessment Situations and Item Difficulty**

Previously discussed item difficulty is most applicable to norm-referenced tests For criterion-referenced tests or classroom tests, it is normal to have average p values as high as 0.9 because we expect most students to be successful If a test were developed to select the upper 25%, it would be desirable to have items with p values that average 0.25 In summary, although a mean p of 0.5 is optimal, item difficulty levels vary with purpose of a test.

10
**Item Discrimination Index**

Is defined as the difference of item difficulty between those who succeeded (called upper group or high- achievement group) and those who failed the test (called lower group or low-achievement group) D = discrimination index (range from -1 to 1) PU = difficulty index in the upper group PL = difficulty index in the lower group For example, Pu=0.8, PL=0.3, D= =0.5

11
**Guidelines for Evaluating D Values**

Discrimination Index 0.4 and larger excellent good fair (it is OK for classroom tests) poor Negative miskeyed or major flaw

12
Student ID 1 2 3 4 5 6 7 8 9 10 Total R Q G I B F E T S C K M O A D N H L J P Discrmination-index

13
Distracter Analysis It allows you to examine how many students in the upper group and the lower group selected each option on a multiple-choice item We expect distracters to be selected by more students in the lower group than students in the upper group. An effective distracter must be selected by some students.

14
**Distracter Analysis Options Item 1 A* B C D Number in the upper group**

22 3 2 Number in the lower group 9 7 8 6

15
**Distracter Analysis Options Item 2 A* B C D Number in the upper group**

17 9 4 Number in the lower group 13 6 11

16
**Building a Table of Specifications**

1. Selecting content areas 2. Selecting learning outcomes to be tested 3. Determining the levels of objectives 4. Determining the question type 5. Determining the points for each question 6. Building a table

17
**A Sample of Table of Specifications**

Content Area Learning Objectives Level of objective Item Type number point Item Analysis Explaining item difficulty and discrimination index Calculate P and D Identify ineffective distracters Comprehension Application Multiple-choice Constructed 2 1 4 Preparing a classroom test Apply table of specifications Project 5

Similar presentations

OK

Dan Thompson Oklahoma State University Center for Health Science Evaluating Assessments: Utilizing ExamSoft’s item-analysis to better understand student.

Dan Thompson Oklahoma State University Center for Health Science Evaluating Assessments: Utilizing ExamSoft’s item-analysis to better understand student.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on job evaluation questionnaire Maths ppt on real numbers class 10 Nokia acquired by microsoft ppt online Ppt on history of indian cricket Download ppt on community planning for disaster management Ppt on media revolution Ppt on power system harmonics analysis Download ppt on global warming for free Ppt on heritage and culture of rajasthan india Cornea anatomy and physiology ppt on cells