Principles of Instructional Design Assessment

Recap Checkpoint Questions Prerequisite Skill Analysis

Why do we need better assessments? Students given instruction aimed at conceptual understanding do better on skills tests than students drilled on the skills directly. The more we focus on raising test scores, the more instruction is distorted and the less credible are the scores themselves. Assessment tasks should be redesigned to closely resemble real learning tasks.

Assessing Learner Outcomes

“How will we know when we’re there?” TESTING!

Reasons to Test diagnosis student difficulty placement check student progress report what skills the student has mastered provides information about the type of revision required Standards

Alignment CONTENT OBJECTIVE TEST ITEM

Choosing An Assessment Method Choose a methods that –matches the target of the instruction –aligns with the method of instruction

Assessment Methods Selected Response –True/False –Multiple Choice –Matching Performance Based - Constructed Response –Short Answer –Fill in the Blank –Show your work

Assessment Methods Performance Based - Motor Skills –Plan and carry out performance (athletic skills, dance, speeches, readings, demonstrations Product Based –Plan, produce and use a product (art exhibits, dioramas, research papers, poems, videotapes)

Assessment Methods Process –use a process to examine individual steps/tasks Portfolio –purposeful collections of student work –contains combinations of other assessments Interview/Observation

Categories of Assessment Criterion-referenced (objective- referenced) Norm-referenced

Criterion-Referenced /Objective-Referenced Tests and evaluates student progress Provides information about the effectiveness of the instruction Identifies gaps in learning Measures performance of all students compared to the number of subordinate skills or objectives passed Measured competence

Norm-Referenced Tests Compare the performance of students with each other.

Activity On page 51 of the workbook, determine if each item is criterion-referenced or norm-referenced.

Testing Types for Instructional Design Entry skills test –below the entry behaviors line Pretests –Includes all items appearing above the entry behaviors line. Posttests –Does not include entry behaviors, includes all subskills. Embedded Tests –Part of the instructional strategy. May appear every few pages, or after a major sequence of instruction.

Activity Practice, page 95, Smith & Ragan

What is Mastery? How do instructional designers respond to the concept of the normal curve? How would you set mastery for a course? What are critical skills?

Test Construction Issues related to validity of items Number of items to write Item types by learning domain

Validity Do the test items actually measure what they are intended to measure? Are the test items aligned with the objectives? Are items representative of a range of possible items? Are objectives adequately sampled?

Number & Type of Item Intellectual skills Verbal Information Psychomotor Skills Cognitive Strategy Attitude

Activity Are these items valid? Look closely at the objective and the possible test item on page 52 of the workbook. Are they aligned? Can a valid assessment be obtained?

Reliability Do the items consistently measure what it claims and is there a high degree of confidence in the scores produced. EXAMPLE: If the same assessment was given on another day to same learners would their scores be basically the same?

Threats to Reliability Using too few items to measure objectives Using the wrong type of instrumentation to measure specific domains of knowledge Incomplete directions Unfamiliar vocabulary Unfamiliar assessment formates Uncomfortable room conditions

Practicality trade-offs/compromises

Scoring Assessments -- Rubrics When? What are they? Levels Characteristics

Developing Rubrics Develop your own rubric –You are assessing the performance of students who are cooking a pizza. –Create a 3-level rubric

Assessment Instrument Blueprint Objectives Forms of Items Number of Items Proportionality of Items Directions for Administration Scoring Methods Weighting Mastery

Next Week Read Chapter 7, Smith & Ragin Workbook, Chapter 9 Project Assignment 3 Due